Archive for the 'Python' Category

The VD Labs webpage has been published! I hope that the new VD Labs page will be a single location where I can link to descriptions and downloads of useful radio, audio analysis, and QRSS-related software. It will eventually be the home of the next (recoded-from-scratch) version of QRSS VD, but let’s not get too far ahead of ourselves!

>>> VD Labs Webpage

Since I ran out of steam from working so much on QRSS VD, I didn’t think I’d be publishing mush more “useful” software, but this one blind-sighted me. People on the Knights QRSS mailing list were talking about dividing QRSS transmissions into images which line up with the period of the transmitters repeated messages and projecting the images together in an attempt to average-out the noise, and boost the signal. It’s a simple idea, and it’s the basis behind how a lot of poor imaging devices can improve their output clarity by software (MRI anyone?). I was overwhelmed by dental school obligations the last few weeks, and it pained me so much to read what people were doing (or at least trying to do) and having to sit it out. Now that I have a free day (yay for weekends!) I sat down and wrote some code. I introduce VD Labs QRSS Stitcher and QRSS Stacker!

Converting Argo captures into continuous images:

example output:

Doing the same thing, with ultra-narrow images:

File produced:

Using QRSS Stacker to project images:

Another example output:

Screenshots:

DOWNLOAD:

very soon…

My expression is completely flat right now. I simply cannot believe I’m about to say what I’m preparing to say. I spent nearly a year cracking large prime numbers. In short, I took-on a project I called The Flowering N’th Prime Project, where I used my SheevaPlug to generate a list of every [every millionth] prime number. The current “golden standard” is this page where one can look-up the N’th prime up to 1 trillion. My goal was to reach over 1 trillion, which I did just this morning! I was planning on being the only source on the web to allow lookups of prime numbers greater than 1 trillion.

However, when I went to look at the logs, I realized that the software had a small, fatal bug in it. Apparently every time the program restarted (which happened a few times over the months), although it resumed at its most recent prime number, it erased the previous entries. As a result, I have no logs below N=95 billion. In other words, although I reached my target this morning, it’s completely irrelevant since I don’t have all the previous data to prove it. I’m completely beside myself, and have no idea what I’m going to do. I can start from the beginning again, but that would take another YEAR. [sigh]

So here’s the screw-up. Apparently I coded everything correctly on paper, but due to my lack of experience I overlooked the potential for multiple appends to occur simultaneously. I can only assume that’s what screwed it up, but I cannot be confident. Honestly, I still don’t know specifically what the problem is. All in all, it looks good to me. Here is the relevant Python code.

def add2log(c,v):
	f=open(logfile,'a')
	f.write("%d,%d\n"%(c,v))
	f.close()

def resumeFromLog():
	f=open('log.txt')
	raw=f.readlines()[-1]
	f.close()
	return eval("["+raw+"]")

For what it’s worth, this is what remains of the log file:

953238,28546251136703
953239,28546282140203
953240,28546313129849
...
1000772,30020181524029
1000773,30020212566353
1000774,30020243594723

Briefly, this is what my idea is. Rather than standard 2-frequencies (low for space, high for tone) QRSS3 (3 seconds per dot), I eliminate the need for pauses between dots by using 3 frequencies (low for a space between letters, medium for dot, high for dash). The following images compare my call sign (AJ4VD) being sent with the old method, and the vdFSK method.

Traditional QRSS:

Again, both of these images say the same thing: AJ4VD, (.- .— ….- …- -..). However, note that the above image has greater than a 3 second dot, so it’s unfairly long if you look at the time scale. Until I get a more fairly representative image, just appreciate it graphically. It’s obviously faster to send 3 frequencies rather than two. In my case, it’s over 200% faster.

vdFSK method:

This is the code to generate audio files converting a string of text into vdFSK audio, saving the output as a WAV file. Spectrographs can be created from these WAV files.

### generate_audio.py ###
# converts a string into vdFSK audio saved as a WAV file

import numpy
import wave
from morse import *

def makeTone(freq,duration=1,samplerate=5000,shape=True):
    signal = numpy.arange(duration*samplerate)/float(samplerate)*float(freq)*3.14*2
    signal = numpy.sin(signal)*16384
    if shape==True: #soften edges
        for i in range(100):
            signal[i]=signal[i]*(i/100.0)
            signal[-i]=signal[-i]*(i/100.0)
    ssignal=''
    for i in range(len(signal)): #make it binary
        ssignal += wave.struct.pack('h',signal[i])
    return ssignal

def text2tone(msg,base=800,sep=5):
    audio=''
    mult=3 #secs per beep
    msg=" "+msg+" "
    for char in msg.lower():
        morse=lookup[char]
        print char, morse
        audio+=makeTone(base,mult)
        for step in lookup[char]:
            if step[0]==".":
                audio+=makeTone(base+sep,int(step[1])*mult)
            if step[0]=="-":
                audio+=makeTone(base+sep*2,int(step[1])*mult)
            if step[0]=="|":
                audio+=makeTone(base,3*mult)
    return audio

msg="aj4vd"
file=wave.open('test.wav', 'wb')
file.setparams((1, 2, 5000, 5000*4, 'NONE', 'noncompressed'))
file.writeframes(text2tone(msg))
file.close()

print 'file written'

And the other file needed…

### morse.py ###
# library for converting between text and Morse code
raw_lookup="""
a.- b-... c-.-. d-.. e. f..-. g--. h.... i.. j.--- k-- l.-.. m--
n-. o--- p.--. q--.- r.-. s... t- u.- v...- w.-- x-..- y-.-- z--..
0----- 1.---- 2..--- 3...-- 4....- 5..... 6-.... 7--... 8---.. 9----.
..-.-.- =-...- :---... ,--..-- /-..-. --....-
""".replace("\n","").split(" ")

lookup={}
lookup[" "]=["|1"]
for char in raw_lookup:
    """This is a silly way to do it, but it works."""
    char,code=char[0],char[1:]
    code=code.replace("-----","x15 ")
    code=code.replace("----","x14 ")
    code=code.replace("---","x13 ")
    code=code.replace("--","x12 ")
    code=code.replace("-","x11 ")
    code=code.replace(".....","x05 ")
    code=code.replace("....","x04 ")
    code=code.replace("...","x03 ")
    code=code.replace("..","x02 ")
    code=code.replace(".","x01 ")
    code=code.replace("x0",'.')
    code=code.replace("x1",'-')
    code=code.split(" ")[:-1]
    #print char,code
    lookup[char]=code

Automated decoding is trivial. The image above was analyzed, turned into the image below, and the string (AJ4VD) was extracted:

The code to do this:

### decode.py ###
# given an image, it finds peaks and pulls data out
from PIL import Image
from PIL import ImageDraw
import pylab
import numpy

pixelSeek=10
pixelShift=15

def findPeak(data):
	maxVal=0
	maxX=0
	for x in range(len(data)):
		if data[x]>maxVal:
			maxVal,maxX=data[x],x
	return maxX

def peaks2morse(peaks):
	baseFreq=peaks[0]
	lastSignal=peaks[0]
	lastChange=0
	directions=[]
	for i in range(len(peaks)):
		if abs(peaks[i]-baseFreq)<pixelSeek:
			baseFreq=peaks[i]
		if abs(peaks[i]-lastSignal)<pixelSeek and i<len(peaks)-1:
			lastChange+=1
		else:
			if abs(baseFreq-lastSignal)<pixelSeek:c=" "
			if abs(baseFreq-lastSignal)<pixelSeek:c=" "
			if abs(baseFreq-lastSignal)<pixelSeek:c=" "
			directions.append([lastSignal,lastChange,baseFreq,baseFreq-lastSignal])
			lastChange=0
		lastSignal=peaks[i]
	return directions

def morse2image(directions):
	im=Image.new("L",(300,100),0)
	draw = ImageDraw.Draw(im)
	lastx=0
	for d in directions:
		print d
		draw.line((lastx,d[0],lastx+d[1],d[0]), width=5,fill=255)
		lastx=lastx+d[1]
	im.show()

im=Image.open('raw.png')
pix=im.load()
data=numpy.zeros(im.size)
for x in range(im.size[0]):
	for y in range(im.size[1]):
		data[x][y]=pix[x,y]

peaks=[]
for i in range(im.size[0]):
	peaks.append(findPeak(data[i]))

morse=peaks2morse(peaks)
morse2image(morse)
print morse

Wow, that’s a mouthfull. This is a total hack, but it works — and barely I might add! I spent all night jumping through hoops to get this thing to run on Windows. The problem is that I designed my previous UI in a version of GLADE which is newer than that supported by Windows. It looks like it’s not backward-compatible, so I have to re-design the GUI from scratch using an earlier version of GLADE. I’ll probably stick to GTK version 2.12 and Python version 2.6 because they play nicely on Windows. It’s a quick and dirty script, but I was able to make the following run on Windows as a single EXE file!

WHAT A NIGHTMARE

While continuing to investigate my options for the new version of QRSS VD, I re-visited Glade, the GTK GUI designer. In short, it lets you draw widgets (combo boxes, scrollbars, labels, images, buttons, etc) onto windows and then makes it easy to add code to the GUI. I *hated* the old QRSS VD development because of the ridiculously large amount of time I had to spend coding the UI. Hopefully by migrating from TKinter to GTK – while it opens a whole new can of worms – will let me add functionality rapidly without hesitation.

Here’s a quick screenshot of my running this new version of the software with a GUI I made in less than an hour. The bars for brightness and contrast can be adjusted which modify the spectrograph in real time. The audio is whatever is playing in Pandora. I like the “fantastic plastic machine” radio station!