I’m working to further simplify my frequency counter design. This one is simpler than my previous design both in hardware and software! Here’s a video to demonstrate the device in its current state:
I utilize the ATMega48’s hardware counter which is synchronous with the system clock, so it can only measure frequency less than half of its clock speed. I solve this issue by dividing the input frequency by 8 and clocking the chip at 12mhz. This allows me to measure frequencies up to about 48MHz, but can be easily adapted to measure over 700MHz (really?) by dividing the input by 128. Division occurs by a 74HC590 8-bit counter (not a 74HC595 as I accidentally said in the video, which is actually a common shift register), allowing easy selection of input divided by 1, 2, 4, 8, 16, 32, 64, or 128. The following image shows the o-scope showing the original signal (bottom) and the divided-by-8 result (top)
The device outputs graphically to a LCD simply enough. That LCD is from eBay and is only $3.88 shipped! I’m considering buying a big box of them and implementing them in many more of my projects. They’re convenient and sure do look nice!
The signal I test with comes from an oscillator I built several months ago. It’s actually a SA612 style receiver whose oscillator is tapped, amplified, and output through a wire. It’s tunable over almost all of 40m with a varactor diode configuration. It was the start of a transceiver, but I got so much good use out of it as a function generator that I decided to leave it like it is!
THIS IS HOW THE PROGRAM WORKS: I don’t supply a schematic because it’s simple as could be. Divide the input frequency to something relatively slow, <1MHz at least. Configure the 16-bit counter to accept an external pin as the counter source (not a prescaled clock, as I often use in other applications). Then set the timer value to 0, _delay_ms() a certainly amount of time (1/10th second), and read the counter value. Multiply it by 10 to account for the 1/10th second, then multiply it by 8 to account for the divider, and it’s done! It will update 10 times a second, with a resolution down to 10*8 = 80 Hz. It’s well within the range of amateur radio uses! If you’re considering replicating this, read up on how to use hardware counters with ATMEL AVR microcontrollers. That should be enough to get you started! Here’s the code I used…
While trying to attack the problem described in the previous entry, it became clear that a logic analyzer would be necessary. I thought I’d try to build one, and my first attempt was so close to being successful, but not quite there. It records 19 channels (the maximum pins available on the ATMega48 not being occupied by the status LED or USB connection pins) at a rate just under 1,000 samples per second. The USB connection to the PC is obvious, and it utilizes the V-USB project to bit-bang the USB protocol. I’m posting this in part because some of the comments to my entry two posts ago were disheartening, discouraging, and even down-right viscous! I made a simple way to send numbers to a PC through the sound card, so what? Don’t be nasty about it! Meh, internet people. Anyway, here’s a marginally more proper way to send data to a PC with USB and an AVR (logging and interface designed in python), but I’ll probably still get yelled at for it.
As you can see from the video, it’s good but not good enough. If I could get samples at 2,000 per second I’d probably be OK, but it’s just not quite fast enough with it’s current, ultra-simplistic method of sample recording. I’ll figure out a fancier way to build a spectrum analyzer – it’s obvious the platform is there, it just needs some refinement.
A few stills:
UPDATE! The more I think about it, the more I think this might be just good enough to work! Look at the stagger in those peaks near the top – that’s probably the lines telling which character to display. Data between the peaks indicates the value to be provided, and I should have enough time to accurately measure that… Maybe this is good enough after all? I’ll have to run some more tests tomorrow…
Where’s the code? It kills me to do this, but I need to withhold the chip side code. I’m working on an idiot’s guide to USB connectivity with AVR microcontrollers, and I’d rather post the simplest-case code first, then share complicated stuff like this. I’ll post the python scripts:
# LOGIC.PY - this script will log (or print) raw data from the USB device
from optparse import OptionParser
import time
import usb.core
import usb.util
import os
while True:
print "\nTrying to communicate with the Gator Keyer ...",
dev = usb.core.find(find_all=True, idVendor=0x16c0, idProduct=0x5dc)
if len(dev)==0: print "FAIL"
dev=dev[0]
dev.set_configuration()
print "there it is!"
break
def readVals():
x=dev.ctrl_transfer(0xC0, 2, 2, 3, 4).tolist()
return x
def toBinary(desc):
bits=[]
for i in range(7,-1,-1):
if (2**i>desc):
bits.append('0')
else:
bits.append('1')
desc=desc-2**i
return bits
def toStr(lists):
raw=[]
for port in lists: raw+=toBinary(port)
return ''.join(raw)
### PROGRAM START ##################
live=False
#live=True
start=time.time()
if live==True:
while True:
a,b,c,d=readVals()
if not a==123: continue #bad data
elapsed=time.time()-start
print "%.010f,%s"%(elapsed,toStr([b,c,d]))
else:
times=0
data=''
f=open("out.txt",'a')
while True:
a,b,c,d=readVals()
if not a==123: continue #bad data
elapsed=time.time()-start
data+="%.010f,%s\n"%(elapsed,toStr([b,c,d]))
times+=1
if times%1000==999:
print "%d readings / %.02f = %.02f /sec"%(times,elapsed,times/elapsed)
f.write(data)
data=""
#logicGraph.py - this will show the data in a pretty way
import matplotlib.pyplot as plt
import numpy
c={
0:"",
1:"",
2:"blk sol",
3:"yel str",
4:"yel sol",
5:"pur sol",
6:"pur str",
7:"",
8:"",
9:"",
10:"blu sol",
11:"blu str",
12:"orn sol",
13:"orn str",
14:"pnk sol",
15:"pnk str",
16:"",
17:"",
18:"",
19:"",
20:"",
21:"",
22:"",
23:"",
24:"",
}
print "loading"
f=open("out.txt")
raw=f.readlines()
f.close()
print "crunching"
times=numpy.array([])
data=numpy.array([])
for line in raw:
if len(line)<10: continue
line=line.replace("\n",'').split(',')
times=numpy.append(times,float(line[0]))
bits = []
for bit in line[1]:
if bit=="1":bits.append(1)
else:bits.append(0)
data=numpy.append(data,bits)
columns=24
rows=len(data)/columns
data=numpy.reshape(data,[rows,columns])
print "DONE processing",len(data),"lines\n\n"
print "plotting..."
plt.figure()
plt.grid()
for i in range(len(c.keys())):
if c[i]=="": continue
plt.plot(times,data[:,i]+i*1.1,'-',label=c[i])
plt.legend()
plt.show()
This page describes a method of sending data from a microchip to a PC using pulses of data. It’s an alternative to more traditional serial or USB methods of connectivity. It’s not intended as a solution for consumer products, but rather an easy hack for hobbyists to employ if they don’t have the equipment for other methods. This method doesn’t require *any* circuitry, just a sound card. The one built in your computer is fine, but I’m using a $1.30 USB sound card for simplicity. It boils down to just a single microcontroller pin connected to a PC sound card microphone jack!
This is the finished product ready to send data to a PC:
MY PROBLEM: I want to send data from a simple microcontroller to a PC. While USART and a serial port is the common solution like I’ve done before, it’s not convenient because it requires a level converter (like a MAX232, about $4), crystal (specific values based on bit and error rate, if you’re lucky you might have a right value in your junk box), and an archaic PC which actually has a serial port. A usb serial port adapter sounds clever, but many aren’t supported on Linux, Windows Vista, or Windows 7. Also, many small chips (most of the ATTiny series) don’t have built in serial capabilities, so it has to be bit-banged in software! Yuk! The second choice would be USB. This requires a crystal too, zener diodes, and bit-banging the USB protocol with something like V-USB since most of the AVR series don’t have built in USB (do they even make breadbordable DIP chips with USB?). Even so, it requires drivers, custom software, cross-platform frustrations, etc. I know PIC has some 18f series chips with USB, but I don’t feel like switching architectures just to send a few bytes of data to a PC. FDTI has a FT232R chip which is a USB serial port adapter, but it’s expensive (about $5) and doesn’t come in dip, so no breadboarding! Sure there are adapter boards, but that just adds the cost. I’m not excited about a $5 solution for a $1 microcontroller. I even did a bit of trolling on AVR Freaks to see if anyone could help me out – just more of the same!
MY SOLUTION: Send data through the sound card! USB sound cards are $1.30 (shipped) on eBay! It couldn’t be simpler. Send pulses, measure distance between pulses. Short pulses are a zero, longer ones are a 1, and very long pulses are number separators. A Python solution with PyAudio allows 1 script which will work on Mac, Linux, Windows, etc, and because it calibrates itself, this will work on any chip at any clock rate. Data is initiated with calibration pulses so timing is not critical – the PC figures out how fast the data is coming in. Check it out! (scroll way down for a bidirectional communication solution)
Here is a sound card I used for bidirectional communication:
Output graph (python and excel) of temperature when I put a soldering iron near the sensor:
~ UNIDIRECTIONAL SOLUTION ~
The following code is designed to have a chip send data to your PC automatically. This can be run on any micro-controller (PIC or AVR I guess, the concept is the same) at any clock rate. Just make sure the sound card is recording fast enough to differentiate pulses. (keep scrolling down for a bidirectional solution)
A NOTE ABOUT MY CODE: This is just the code I used for my demonstration. It might be more logical for you to write your own since the concept is so simple. I’m a dental student, not a programmer, so I’m sure it’s not coded very elegantly. I didn’t work hard to make this code easy to read or easy to share. With that being said, help yourself!
/*The following code is written in AVR-GCC for an ATTiny44a.
It reads ADC values on 3 pins and reports it each second along
with a number which increments each time data is sent.
It's designed as a starting point, allowing anyone to
customize it from here!*/
#include <avr/io.h>
#include <avr/delay.h>
#include <avr/interrupt.h>
// bytes we want to send to the PC
volatile int data1=0;
volatile int data2=0;
volatile int data3=0;
volatile int data4=0;
void solid(){ // dont touch
_delay_ms(1);
pulse(1);pulse(1);pulse(1);pulse(3);pulse(3);
pulse(3);pulse(5);pulse(5);// CALIBRATION PULSES
}
void pulse(char size){ // dont touch
PORTA|=_BV(PA3);
_delay_us(100);
PORTA&=~_BV(PA3);
while (size){size--;_delay_us(100);}
}
void sendVal(unsigned long tosend){ // dont touch
pulse(5); // send a space
while (tosend){
if (tosend&1){pulse(3);} // send ONE
else {pulse(1);} // send ZERO
tosend=tosend>>1;
}
}
int readADC(char adcNum){
_delay_ms(1);
ADMUX=adcNum; // select which ADC to read, VCC as ref.
ADCSRA=0b11000111; // enable, start, 128 prescale
while (ADCSRA&( 1<<ADSC)) {}; // wait for measurement
return ADC;
}
void takeReadings(){
data1=readADC(0); // ADC0
data2=readADC(1); // ADC1
data3=readADC(2); // ADC2
data4++; // incriment just because we want to
}
void sendStuff(){ // EDIT to send what you want
solid(); //required
sendVal(12345); //required
sendVal(12345); //required
sendVal(54321); //required
sendVal(data1);
sendVal(data2);
sendVal(data3);
sendVal(data4);
pulse(1); //required
}
int main(){
DDRA|=_BV(PA2)|_BV(PA3);
for (;;){
_delay_ms(1000);
takeReadings();
sendStuff();
}
return 0;
}
"""
file name: listenOnly.py
This is the PC code to listen to the microphone and display
and log the data. It probably does NOT need adjustment!
Make sure the correct sound card is selected (in the code)
and make sure microphone input is turned up in volume control.
This code is what was used on my PC for the demonstration
video. This is the listenOnly.py file which will turn any audio
detected from a sound card into data, optionally logging it
(if the last few lines are uncommented). This also works to
capture data for the bidirectional communication method,
described below on this website.
If this is running but no data is coming through, make sure the
microphone is selected as a recording device, the correct sound
card is selected, and the microphone volume is turned to high.
REQUIRED: To run this, you need to have the following installed:
-- Python 2.6
-- numpy for python 2.6
-- matplotlib for python 2.6
-- pyaudio for python 2.6
(other versions may work, but this is what I'm using)
"""
import numpy
import pyaudio
import matplotlib.pyplot as plt
import wave
import time
def listCards(dontAsk=True):
p=pyaudio.PyAudio()
print "SOUND CARDS:"
for i in range(p.get_default_host_api_info()["deviceCount"]):
if p.get_device_info_by_index(i)["maxInputChannels"]>0:
cardName = p.get_device_info_by_index(i)["name"]
cardIndex = p.get_device_info_by_index(i)["index"]
print "[%d] %s"%(cardIndex,cardName)
if dontAsk: return
return int(raw_input("CARD NUMBER TO USE:"))
cardID=1
listCards()
print "USING CARD:",cardID
rate=44100.0
sampleSize=1024
def data2vals(data):
vals=numpy.array([])
lastPeak=0
for i in range(1,len(data)):
if data[i]==True and data[i-1]==False:
if lastPeak>0: vals=numpy.append(vals,i-lastPeak)
lastPeak=i
return vals
def binary2dec(binary):
binary=binary[:-1]
dec=0
s=""
for i in range(len(binary)):
dec=dec*2
dec+=binary[i]
s="%d"%binary[i]+s
#print s,"=",dec #11111100101100000 = 3391
return dec
def readVals(vals):
if len(vals)<7: return False
vals2=[]
aLow = min(vals[0:3])
aMed = min(vals[3:6])
aHigh = vals[6]
thresh1=sum([aLow,aMed])/2+2
thresh2=sum([aMed,aHigh])/2+2
#print "tresholds:",thresh1,thresh2
#print vals
vals=vals[8:]
binary=[]
for i in range(len(vals)):
if vals[i]>thresh2:
vals2.append(binary2dec(binary))
binary=[]
if vals[i]>thresh1:binary=[1]+binary
else:binary=[0]+binary
vals2.append(binary2dec(binary))
for i in range(len(vals2)):
if vals2[i]==54321: return vals2[i+1:]
return False
def playFile():
chunk = 1024
wf = wave.open("short_onenum.wav", 'rb')
p = pyaudio.PyAudio()
stream = p.open(format =
p.get_format_from_width(wf.getsampwidth()),
channels = wf.getnchannels(),
rate = wf.getframerate(),
output = True)
data = wf.readframes(chunk)
while data != '':
stream.write(data)
data = wf.readframes(chunk)
stream.close()
def captureData():
pyaud = pyaudio.PyAudio()
stream = pyaud.open(format=pyaudio.paInt16,channels=1,
rate = 44100,input_device_index=cardID,input=True,output=True)
sample=numpy.array([])
while True:
sampleNew=numpy.fromstring(stream.read(sampleSize),dtype=numpy.int16)
sampleNew=(sampleNew<-25000)*1
if True in sampleNew: sample=numpy.append(sample,sampleNew)
else:
if len(sample):
stream.close()
return sample
stream.close()
tone_quiet=0
def buildNumber(num=123):
if num>255: print "NUMBER TOO HIGH!!!"
#print num,'=',
num+=1
for i in [7,6,5,4,3,2,1,0]:
if num>2**i:one();num=num-2**i;#print"1",
else: zero();#print"0",
#print
space()
def pulse():
global data
data+=[-30000]*10
def space():
global data
data+=[tone_quiet]*900
pulse()
def one():
global data
data+=[tone_quiet]*600
pulse()
def zero():
global data
data+=[tone_quiet]*300
pulse()
def silence(msec=1000):
global data
data+=[tone_quiet]*int(41.1*msec)
data=[]
def sendAudio(numbers=[11,66,77]):
global data
data=[]
silence(100)
buildNumber(250)
print "SENDING",
for numba in numbers:
buildNumber(numba)
print numba,
buildNumber(250)
silence(100)
data=numpy.array(data)
data=-data
data=data.tostring()
print
p = pyaudio.PyAudio()
stream = p.open(rate=44100, channels=1, format=pyaudio.paInt16,
input_device_index=cardID, output=True)
stream.write(data)
stream.close()
p.terminate()
i=0
while True:
i+=1
val=readVals(data2vals(captureData()))
if val == False: continue
line=""
for item in val: line+=str(item)+","
print i,line
#f=open('log.csv','a')
#f.write("%s\n"%line)
#f.close()
~ BIDIRECTIONAL SOLUTION ~
What if we want to send data TO the microcontroller? The solution is a little more complex, but quite doable. Just add an extra wire to the sound card’s speaker output and attach it to PCINT0 (the highest level internal interrupt). This is intended for advanced users, and if you’re doing this you probably are better off with USB or serial anyway! … but heck, why not do it as a proof of concept!
Note that the USB sound card speaker output was not powerful enough to trigger the digital input pin of the AVR, so an inverting buffer was made from a single NPN transistor (2n3904). The hardware interrupt was attacked to the collector, and the collector was attached through +5V through a 220 ohm resistor. The emitter was grounded. The base was attached directly to the sound card output. I also tried running the sound card output through a small series capacitor (0.1uF) and biasing the base to ground through a 1Mohm resistor and it worked the same. Hardware, simple. Chip-side software… a little more complex.
### VIDEO ###
"""
This code is what was used on my PC for the
demonstration video. The listenonly.py file
(above on site) was also used without modification.
"""
import pyaudio
from struct import pack
from math import sin, pi
import wave
import random
import numpy
import time
RATE=44100
maxVol=2**15-1.0 #maximum amplitude
p = pyaudio.PyAudio()
stream = p.open(rate=44100, channels=1, format=pyaudio.paInt16,
input_device_index=1, output=True)
def pulseZero():
global wvData
wvData+=pack('h', 0)*30
wvData+=pack('h', maxVol)
def pulseOne():
global wvData
wvData+=pack('h', 0)*40
wvData+=pack('h', maxVol)
def pulseSpace():
global wvData
wvData+=pack('h', 0)*50
wvData+=pack('h', maxVol)
def buildNumber(num=123):
if num>255: print "NUMBER TOO HIGH!!!"
num+=1
for i in [7,6,5,4,3,2,1,0]:
if num>2**i:
pulseOne()
num=num-2**i
else:
pulseZero()
wvData=""
wvData+=pack('h', 0)*2000
pulseOne() #required before sending data
buildNumber(55)
buildNumber(66)
buildNumber(77)
buildNumber(123)
wvData+=pack('h', 0)*2000
while True:
print "SENDING",
stream.write(wvData)
raw_input()
/*
This code is what was used on my AVR
microcontroller for the demonstration video
*/
#include <avr/io.h>
#include <avr/delay.h>
#include <avr/interrupt.h>
volatile long commandIncoming=0;
volatile char command1=0;
volatile char command2=0;
volatile char command3=0;
volatile char command4=0;
volatile char bitsGotten=0;
// timing thresholds are critical! Send pulses to the chip
// and have it report the time between them. Use this to
// determine the best threshold value for your application.
// The ones here must be changed if you run at a speed other
// than 1mhz or if you use different timings in PC software
#define thresh_low 100 // between this and the next
#define thresh_high 130 // is the range for a logical 'one'
// ######## OUTGOING AUDIO DATA #########
void solid(){
_delay_ms(1); //LONG LOW
pulse(1);pulse(1);pulse(1);pulse(3);pulse(3);
pulse(3);pulse(5);pulse(5);// CALIBRATION PULSES
}
void pulse(char size){
PORTA|=_BV(PA3);
_delay_us(100);
PORTA&=~_BV(PA3);
while (size){size--;_delay_us(100);}
}
void sendVal(unsigned long tosend){
pulse(5); // send a space
while (tosend){
if (tosend&1){pulse(3);} // send ONE
else {pulse(1);} // send ZERO
tosend=tosend>>1;
}
}
// ######## INCOMING AUDIO DATA #########
// NOTE THAT INPUTS ARE NORMALLY *HIGH* AND DROP *LOW* FOR SIGNAL
SIGNAL (PCINT0_vect) { // audio input trigger
TIMSK0|=(1<<TOIE1); //Overflow Interrupt Enable
if (TCNT0<10){return;} // seem too fast? ignore it!
// Enable the following line to test custom timings
//command1=command2;command2=command3;
//command3=command4;command4=TCNT0;
bitsGotten++;
commandIncoming=commandIncoming*2; // shift left
if (TCNT0>thresh_low){commandIncoming++;} // make 1
TCNT0=0;
}
ISR(TIM0_OVF_vect){ // TIMER OVERFLOW
if (bitsGotten){sendStuff();}
}
void fillCommands(){
command1=(char*)(commandIncoming>>24);
command2=(char*)(commandIncoming>>16);
command3=(char*)(commandIncoming>>8);
command4=(char*)(commandIncoming);
}
void sendStuff(){
TIMSK0=0; //Overflow Interrupt
cli(); // disable interrupts!
fillCommands();
solid(); // start data transmissions with this
sendVal(12345);
sendVal(12345);
sendVal(54321);
sendVal(command1);
sendVal(command2);
sendVal(command3);
sendVal(command4);
sendVal(1234567890);
pulse(1);
bitsGotten=0;
sei(); // enable interrupts again!
TIMSK0|=(1<<TOIE1); //Overflow Interrupt
}
// ######## MAIN PROGRAM #########
int main(){
DDRA|=_BV(PA2)|_BV(PA3);
// SET UP FOR SOUND CARD INTERRUPT
MCUCR = 0b00000010; // trigger interrupt on falling edge
GIMSK = 0b00010000; // pin change interrupt enable 0
GIFR = 0b00010000; // flag register, same as above
PCMSK0 = (1<<PCINT0); // Set Pin to use (PCINT0)
sei(); // enable global interrupts
// SET UP 8-bit COUNTER
TCCR0B|=0b00000010;
//TCCR1B|=(1<<CS12)|(1<<CS10); // prescaler 1024
TIMSK0|=(1<<TOIE1); //Enable Overflow Interrupt Enable
TCNT0=0;//Initialize our varriable (set for 1/15th second?)
// MAIN PROGRAM
for (;;){}
return 0;
}
In closing, I’m tickled this works so well. It’s funny to me that no one’s really done this before in the hobby field. I’m sure I’m not the only one who wished there were an easy way to do this. I’m sure the process could be greatly improved, but this is a fun start. Wow, it’s late, I should get to bed. I have to treat patients tomorrow morning!
PS: If you replicate this concept, let me know about it! I’d love to see your project!
My current project involves needing to create stereo audio in real time with Python. I’m using PyAudio to send the audio data to the sound card, but in this simple example I demonstrate how to create mono and stereo sounds with Python. I’m disappointed there aren’t good simple case examples on the internet, so I’m sharing my own. It doesn’t get much easier than this!
from struct import pack
from math import sin, pi
import wave
import random
RATE=44100
## GENERATE MONO FILE ##
wv = wave.open('test_mono.wav', 'w')
wv.setparams((1, 2, RATE, 0, 'NONE', 'not compressed'))
maxVol=2**15-1.0 #maximum amplitude
wvData=""
for i in range(0, RATE*3):
wvData+=pack('h', maxVol*sin(i*500.0/RATE)) #500Hz
wv.writeframes(wvData)
wv.close()
## GENERATE STERIO FILE ##
wv = wave.open('test_stereo.wav', 'w')
wv.setparams((2, 2, RATE, 0, 'NONE', 'not compressed'))
maxVol=2**15-1.0 #maximum amplitude
wvData=""
for i in range(0, RATE*3):
wvData+=pack('h', maxVol*sin(i*500.0/RATE)) #500Hz left
wvData+=pack('h', maxVol*sin(i*200.0/RATE)) #200Hz right
wv.writeframes(wvData)
wv.close()
The output is two sound files which look like this:
I’m sitting in class frustrated as could be. The Internet in this room (D3-3 in the dental tower of Shands Hospital at UF) is unbelievably annoying. For some reason, everything runs fine, then functionality drops to unusable levels. Downloading files (i.e., PDFs of lectures) occurs at about 0.5kb/s (wow), and Internet browsing is hopeless. At most, I can connect to IRC and enjoy myself in #electronics, #python, and #linux. I decided to channel my frustration into productivity, and wrote a quick Python script to let me visualize the problem.
Notice the massive lag spikes around the time class begins. I think it’s caused by the retarded behavior of windows update and anti-virus software updates being downloaded on a gazillion computers all at the same time which are required to connect to the network on Windows machines. Class start times were 8:30am, 9:35am, and 10:40am. Let’s view it on a logarithmic scale:
Finally, the code. It’s two scripts. One pings a website (kernel.org) every few seconds and records the ping time to “pings.txt”, and the other graphs the data. Here are the two scripts:
import socket, time, os, sys, re
def getping():
pingaling = os.popen("ping -q -c2 kernel.org")
sys.stdout.flush()
while 1:
line = pingaling.readline()
if not line: break
line=line.split("\n")
for part in line:
if "rtt" in part:
part=part.split(" = ")[1]
part=part.split('/')[1]
print part+"ms"
return part
def add2log(stuff):
f=open("pings.txt",'a')
f.write(stuff+",")
f.close()
while 1:
print "pinging...",
stuff="[%s,%s]"%(time.time(),getping())
print stuff
add2log(stuff)
time.sleep(1)
import pylab, time, datetime, numpy
def smoothTriangle(data,degree,dropVals=False):
triangle=numpy.array(range(degree)+[degree]+range(degree)[::-1])+1
smoothed=[]
for i in range(degree,len(data)-degree*2):
point=data[i:i+len(triangle)]*triangle
smoothed.append(sum(point)/sum(triangle))
if dropVals:
print "smoothlen:",len(smoothed)
return smoothed
#smoothed=[smoothed[0]]*(degree+degree/2)+smoothed
#while len(smoothed)<len(data):smoothed.append(smoothed[-1])
while len(smoothed)<len(data):smoothed=[None]+smoothed+[None]
if len(smoothed)>len(data):smoothed.pop(-1)
return smoothed
print "reading"
f=open("pings.txt")
raw=eval("[%s]"%f.read())
f.close()
xs,ys,big=[],[],[]
for item in raw:
t=datetime.datetime.fromtimestamp(item[0])
maxping=20000
if item[1]>maxping or item[1]==None:
item[1]=maxping
big.append(t)
ys.append(float(item[1]))
xs.append(t)
#print xs
#raw_input("WAIT")
print "plotting"
fig=pylab.figure(figsize=(10,7))
pylab.plot(xs,ys,'k.',alpha=.1)
pylab.plot(xs,ys,'k-',alpha=.1)
pylab.plot(xs,smoothTriangle(ys,15),'b-')
pylab.grid(alpha=.3)
pylab.axis([None,None,None,2000])
#pylab.semilogy()
#pylab.xlabel("time")
pylab.ylabel("latency (ping kernel.org, ms)")
pylab.title("D3-3 Network Responsiveness")
fig.autofmt_xdate()
#pylab.show()
pylab.savefig('out.png')
pylab.semilogy()
pylab.savefig('out2.png')
fig.autofmt_xdate()
print "done"
The VD Labs webpage has been published! I hope that the new VD Labs page will be a single location where I can link to descriptions and downloads of useful radio, audio analysis, and QRSS-related software. It will eventually be the home of the next (recoded-from-scratch) version of QRSS VD, but let’s not get too far ahead of ourselves!
Since I ran out of steam from working so much on QRSS VD, I didn’t think I’d be publishing mush more “useful” software, but this one blind-sighted me. People on the Knights QRSS mailing list were talking about dividing QRSS transmissions into images which line up with the period of the transmitters repeated messages and projecting the images together in an attempt to average-out the noise, and boost the signal. It’s a simple idea, and it’s the basis behind how a lot of poor imaging devices can improve their output clarity by software (MRI anyone?). I was overwhelmed by dental school obligations the last few weeks, and it pained me so much to read what people were doing (or at least trying to do) and having to sit it out. Now that I have a free day (yay for weekends!) I sat down and wrote some code. I introduce VD Labs QRSS Stitcher and QRSS Stacker!
My expression is completely flat right now. I simply cannot believe I’m about to say what I’m preparing to say. I spent nearly a year cracking large prime numbers. In short, I took-on a project I called The Flowering N’th Prime Project, where I used my SheevaPlug to generate a list of every [every millionth] prime number. The current “golden standard” is this page where one can look-up the N’th prime up to 1 trillion. My goal was to reach over 1 trillion, which I did just this morning! I was planning on being the only source on the web to allow lookups of prime numbers greater than 1 trillion.
However, when I went to look at the logs, I realized that the software had a small, fatal bug in it. Apparently every time the program restarted (which happened a few times over the months), although it resumed at its most recent prime number, it erased the previous entries. As a result, I have no logs below N=95 billion. In other words, although I reached my target this morning, it’s completely irrelevant since I don’t have all the previous data to prove it. I’m completely beside myself, and have no idea what I’m going to do. I can start from the beginning again, but that would take another YEAR. [sigh]
So here’s the screw-up. Apparently I coded everything correctly on paper, but due to my lack of experience I overlooked the potential for multiple appends to occur simultaneously. I can only assume that’s what screwed it up, but I cannot be confident. Honestly, I still don’t know specifically what the problem is. All in all, it looks good to me. Here is the relevant Python code.
My goal is to create a QRPP (extremely low power) transmitter and modulation method to send QRSS (extremely slow, frequency shifting data) efficiently, able to be decoded visually or with automated image analysis software. This evolving post will document the thought process and development behind AJ4VD’s Frequency Shift Keying method, vdFSK.
Briefly, this is what my idea is. Rather than standard 2-frequencies (low for space, high for tone) QRSS3 (3 seconds per dot), I eliminate the need for pauses between dots by using 3 frequencies (low for a space between letters, medium for dot, high for dash). The following images compare my call sign (AJ4VD) being sent with the old method, and the vdFSK method.
Traditional QRSS:
Again, both of these images say the same thing: AJ4VD, (.- .— ….- …- -..). However, note that the above image has greater than a 3 second dot, so it’s unfairly long if you look at the time scale. Until I get a more fairly representative image, just appreciate it graphically. It’s obviously faster to send 3 frequencies rather than two. In my case, it’s over 200% faster.
vdFSK method:
This is the code to generate audio files converting a string of text into vdFSK audio, saving the output as a WAV file. Spectrographs can be created from these WAV files.
### generate_audio.py ###
# converts a string into vdFSK audio saved as a WAV file
import numpy
import wave
from morse import *
def makeTone(freq,duration=1,samplerate=5000,shape=True):
signal = numpy.arange(duration*samplerate)/float(samplerate)*float(freq)*3.14*2
signal = numpy.sin(signal)*16384
if shape==True: #soften edges
for i in range(100):
signal[i]=signal[i]*(i/100.0)
signal[-i]=signal[-i]*(i/100.0)
ssignal=''
for i in range(len(signal)): #make it binary
ssignal += wave.struct.pack('h',signal[i])
return ssignal
def text2tone(msg,base=800,sep=5):
audio=''
mult=3 #secs per beep
msg=" "+msg+" "
for char in msg.lower():
morse=lookup[char]
print char, morse
audio+=makeTone(base,mult)
for step in lookup[char]:
if step[0]==".":
audio+=makeTone(base+sep,int(step[1])*mult)
if step[0]=="-":
audio+=makeTone(base+sep*2,int(step[1])*mult)
if step[0]=="|":
audio+=makeTone(base,3*mult)
return audio
msg="aj4vd"
file=wave.open('test.wav', 'wb')
file.setparams((1, 2, 5000, 5000*4, 'NONE', 'noncompressed'))
file.writeframes(text2tone(msg))
file.close()
print 'file written'
And the other file needed…
### morse.py ###
# library for converting between text and Morse code
raw_lookup="""
a.- b-... c-.-. d-.. e. f..-. g--. h.... i.. j.--- k-- l.-.. m--
n-. o--- p.--. q--.- r.-. s... t- u.- v...- w.-- x-..- y-.-- z--..
0----- 1.---- 2..--- 3...-- 4....- 5..... 6-.... 7--... 8---.. 9----.
..-.-.- =-...- :---... ,--..-- /-..-. --....-
""".replace("\n","").split(" ")
lookup={}
lookup[" "]=["|1"]
for char in raw_lookup:
"""This is a silly way to do it, but it works."""
char,code=char[0],char[1:]
code=code.replace("-----","x15 ")
code=code.replace("----","x14 ")
code=code.replace("---","x13 ")
code=code.replace("--","x12 ")
code=code.replace("-","x11 ")
code=code.replace(".....","x05 ")
code=code.replace("....","x04 ")
code=code.replace("...","x03 ")
code=code.replace("..","x02 ")
code=code.replace(".","x01 ")
code=code.replace("x0",'.')
code=code.replace("x1",'-')
code=code.split(" ")[:-1]
#print char,code
lookup[char]=code
Automated decoding is trivial. The image above was analyzed, turned into the image below, and the string (AJ4VD) was extracted:
The code to do this:
### decode.py ###
# given an image, it finds peaks and pulls data out
from PIL import Image
from PIL import ImageDraw
import pylab
import numpy
pixelSeek=10
pixelShift=15
def findPeak(data):
maxVal=0
maxX=0
for x in range(len(data)):
if data[x]>maxVal:
maxVal,maxX=data[x],x
return maxX
def peaks2morse(peaks):
baseFreq=peaks[0]
lastSignal=peaks[0]
lastChange=0
directions=[]
for i in range(len(peaks)):
if abs(peaks[i]-baseFreq)<pixelSeek:
baseFreq=peaks[i]
if abs(peaks[i]-lastSignal)<pixelSeek and i<len(peaks)-1:
lastChange+=1
else:
if abs(baseFreq-lastSignal)<pixelSeek:c=" "
if abs(baseFreq-lastSignal)<pixelSeek:c=" "
if abs(baseFreq-lastSignal)<pixelSeek:c=" "
directions.append([lastSignal,lastChange,baseFreq,baseFreq-lastSignal])
lastChange=0
lastSignal=peaks[i]
return directions
def morse2image(directions):
im=Image.new("L",(300,100),0)
draw = ImageDraw.Draw(im)
lastx=0
for d in directions:
print d
draw.line((lastx,d[0],lastx+d[1],d[0]), width=5,fill=255)
lastx=lastx+d[1]
im.show()
im=Image.open('raw.png')
pix=im.load()
data=numpy.zeros(im.size)
for x in range(im.size[0]):
for y in range(im.size[1]):
data[x][y]=pix[x,y]
peaks=[]
for i in range(im.size[0]):
peaks.append(findPeak(data[i]))
morse=peaks2morse(peaks)
morse2image(morse)
print morse
Wow, that’s a mouthfull. This is a total hack, but it works — and barely I might add! I spent all night jumping through hoops to get this thing to run on Windows. The problem is that I designed my previous UI in a version of GLADE which is newer than that supported by Windows. It looks like it’s not backward-compatible, so I have to re-design the GUI from scratch using an earlier version of GLADE. I’ll probably stick to GTK version 2.12 and Python version 2.6 because they play nicely on Windows. It’s a quick and dirty script, but I was able to make the following run on Windows as a single EXE file!
While continuing to investigate my options for the new version of QRSS VD, I re-visited Glade, the GTK GUI designer. In short, it lets you draw widgets (combo boxes, scrollbars, labels, images, buttons, etc) onto windows and then makes it easy to add code to the GUI. I *hated* the old QRSS VD development because of the ridiculously large amount of time I had to spend coding the UI. Hopefully by migrating from TKinter to GTK – while it opens a whole new can of worms – will let me add functionality rapidly without hesitation.
Here’s a quick screenshot of my running this new version of the software with a GUI I made in less than an hour. The bars for brightness and contrast can be adjusted which modify the spectrograph in real time. The audio is whatever is playing in Pandora. I like the “fantastic plastic machine” radio station!
