Archive for March, 2010

While working on the spectrograph software I’m so psychotic about completing this spring break, I happened to capture a cool signal from Italy. IW4DXW was sending some cool signals that I captured around 10.140 MHz. See how his callsign is written “visually” on the spectrogram? I thought I’d post it because it’s an encouraging sign that my software is going in the right direction. Also note the numerous QRSS FSK signals around it! So cool.

UPDATE! The guy emailed me his station information.

Holy… 200mW? That’s so awesome. Maybe one day I’ll build something equally as cool and epically useless =oD I will now return to my psychotic programming project what has been eating-up my spring break. [boom] I just snapped a screenshot for the future memory of this break. Yes, I’m on a XP machine. I’m at the W4DFU radio club station (they have nicer computers than I do!)

Several days straight of coding all day every day has started to mess with my mind. I’ve got to incorporate some relaxation time! Something good ‘ol youtube can assist with. I share with you the following videos today instead of Python code (there’s always time for that later).

So here’s the problem. I’m working on a software project and I’d love the startup screen to be flashy, but not tacky. I want the program to be ENTIRELY scripted, so pretty bitmap images are out. I have to generate it from the script, but how can I reliably print text on an image with fonts vary across operating systems? (i.e., many OS’s don’t have “arial.ttf”, or no truetype fonts at all!) The solution was a creative process which will unfold before you.

First, I created a 2D binary array to represent the alphabet using pixel fonts such as this as a reference. The word I’m trying to create is “QRSS VD” (the name of my program). I store the data in strings, as seen below. I use 1’s to mark pixels, and spaces to mark empty spaces.

data="""
1111 1111 1111 1111   1  1 111
1  1 1  1 1    1      1  1 1  1
1  1 1111 1111 1111   1  1 1  1
1 11 1 1     1    1   1 1  1  1
1111 1 11 1111 1111    1   111
"""

Once I think it looks nice, I replace the spaces with zeros to make it take up a lot of visual space…

data2="""
1111011110111101111000100101110
1001010010100001000000100101001
1001011110111101111000100101001
1011010100000100001000101001001
1111010110111101111000010001110
"""

Then I further obscure it by replacing linebreaks with a different symbol, such as the number 2, then break the lines so they’re not lined up…

b="1111011110111101111000100101110210010100101000010000001001010012"
b+="1001011110111101111000100101001210110101000001000010001010010012"
b+="1111010110111101111000010001110"

The result is a pretty cool way to obscure the text. I don’t know why you’d want to, but if you want to make sure that no one goes in and changes the letters around (at least without making them think pretty hard about it) you could look at ways to further encrypt this data stream. From here, I create an image using the Python Imaging Library, setting pixel values to 255*b[x,y] (so 0 stays 0 and 1 becomes 255, perfect for an 8-bit image). After enlarging, here’s the result:

That’s cool huh? Now let’s make it a little bit less pixelated. It’s not a cure-all method, but blurring it up a little with a bilinear filter helps a lot…

Then I apply the code below which applies a cool colormap to the pixel values. I’ll provide cleaner code for this later (I have a really cool way of generating colormaps and saving them as arrays of RGB tuples). Then I go through and plot some random sin wavs on top of it. Sweet! Here are 6 images generated from the program run 6 times. Notice the randomness of the sine wavs!

Vwa la! A different image is generated every time the script runs, and it requires no external files (bitmaps or fonts) and should work well on all operating systems. Take the idea and run with it!

from PIL import Image
from PIL import ImageOps
from PIL import ImageFilter
from random import randint
import scipy

def genLogo():
	colormap=[(0, 0, 129), (0, 0, 134), (0, 0, 139), (0, 0, 143), (0, 0, 148), (0, 0, 152), (0, 0, 157), (0, 0, 161), (0, 0, 166), (0, 0, 170), (0, 0, 175), (0, 0, 180), (0, 0, 184), (0, 0, 189), (0, 0, 193), (0, 0, 198), (0, 0, 202), (0, 0, 207), (0, 0, 211), (0, 0, 216), (0, 0, 220), (0, 0, 225), (0, 0, 230), (0, 0, 234), (0, 0, 239), (0, 0, 243), (0, 0, 248), (0, 0, 252), (0, 0, 255), (0, 0, 255), (0, 0, 255), (0, 0, 255), (0, 2, 255), (0, 7, 255), (0, 11, 255), (0, 14, 255), (0, 18, 255), (0, 23, 255), (0, 27, 255), (0, 31, 255), (0, 34, 255), (0, 39, 255), (0, 43, 255), (0, 47, 255), (0, 51, 255), (0, 54, 255), (0, 59, 255), (0, 63, 255), (0, 67, 255), (0, 71, 255), (0, 75, 255), (0, 79, 255), (0, 83, 255), (0, 87, 255), (0, 91, 255), (0, 95, 255), (0, 99, 255), (0, 103, 255), (0, 107, 255), (0, 111, 255), (0, 115, 255), (0, 119, 255), (0, 123, 255), (0, 127, 255), (0, 131, 255), (0, 135, 255), (0, 139, 255), (0, 143, 255), (0, 147, 255), (0, 151, 255), (0, 155, 255), (0, 159, 255), (0, 163, 255), (0, 167, 255), (0, 171, 255), (0, 175, 255), (0, 179, 255), (0, 183, 255), (0, 187, 255), (0, 191, 255), (0, 195, 255), (0, 199, 255), (0, 203, 255), (0, 207, 255), (0, 211, 255), (0, 215, 255), (0, 219, 254), (0, 223, 251), (0, 227, 248), (2, 231, 245), (5, 235, 241), (7, 239, 238), (11, 243, 235), (14, 247, 232), (18, 251, 228), (21, 255, 225), (23, 255, 222), (27, 255, 219), (31, 255, 215), (34, 255, 212), (37, 255, 208), (40, 255, 205), (44, 255, 203), (47, 255, 199), (50, 255, 195), (54, 255, 192), (57, 255, 189), (60, 255, 186), (63, 255, 183), (66, 255, 179), (70, 255, 176), (73, 255, 173), (76, 255, 170), (79, 255, 166), (83, 255, 163), (86, 255, 160), (89, 255, 157), (92, 255, 154), (95, 255, 150), (99, 255, 147), (102, 255, 144), (105, 255, 141), (108, 255, 137), (112, 255, 134), (115, 255, 131), (118, 255, 128), (121, 255, 125), (124, 255, 121), (128, 255, 118), (131, 255, 115), (134, 255, 112), (137, 255, 108), (141, 255, 105), (144, 255, 102), (147, 255, 99), (150, 255, 95), (154, 255, 92), (157, 255, 89), (160, 255, 86), (163, 255, 83), (166, 255, 79), (170, 255, 76), (173, 255, 73), (176, 255, 70), (179, 255, 66), (183, 255, 63), (186, 255, 60), (189, 255, 57), (192, 255, 54), (195, 255, 50), (199, 255, 47), (202, 255, 44), (205, 255, 41), (208, 255, 37), (212, 255, 34), (215, 255, 31), (218, 255, 28), (221, 255, 24), (224, 255, 21), (228, 255, 18), (231, 255, 15), (234, 255, 12), (238, 255, 8), (241, 252, 5), (244, 248, 2), (247, 244, 0), (250, 240, 0), (254, 236, 0), (255, 233, 0), (255, 229, 0), (255, 226, 0), (255, 221, 0), (255, 218, 0), (255, 215, 0), (255, 211, 0), (255, 207, 0), (255, 203, 0), (255, 199, 0), (255, 196, 0), (255, 192, 0), (255, 188, 0), (255, 184, 0), (255, 180, 0), (255, 177, 0), (255, 173, 0), (255, 169, 0), (255, 165, 0), (255, 162, 0), (255, 159, 0), (255, 155, 0), (255, 151, 0), (255, 147, 0), (255, 143, 0), (255, 140, 0), (255, 136, 0), (255, 132, 0), (255, 128, 0), (255, 125, 0), (255, 121, 0), (255, 117, 0), (255, 114, 0), (255, 110, 0), (255, 106, 0), (255, 102, 0), (255, 99, 0), (255, 95, 0), (255, 91, 0), (255, 88, 0), (255, 84, 0), (255, 80, 0), (255, 76, 0), (255, 73, 0), (255, 69, 0), (255, 65, 0), (255, 62, 0), (255, 58, 0), (255, 54, 0), (255, 51, 0), (255, 47, 0), (255, 43, 0), (255, 39, 0), (255, 36, 0), (255, 32, 0), (255, 28, 0), (255, 25, 0), (255, 21, 0), (253, 17, 0), (248, 14, 0), (244, 10, 0), (240, 6, 0), (235, 2, 0), (230, 0, 0), (225, 0, 0), (221, 0, 0), (217, 0, 0), (212, 0, 0), (207, 0, 0), (203, 0, 0), (198, 0, 0), (194, 0, 0), (189, 0, 0), (185, 0, 0), (180, 0, 0), (175, 0, 0), (171, 0, 0), (166, 0, 0), (162, 0, 0), (157, 0, 0), (152, 0, 0), (148, 0, 0), (144, 0, 0), (139, 0, 0), (134, 0, 0), (130, 0, 0), (134, 0, 0), (130, 0, 0)]
	def red(val):
		return colormap[val][0]
	def green(val):
		return colormap[val][1]
	def blue(val):
		return colormap[val][2]
	def colorize(im):
		r=Image.eval(im,red)
		g=Image.eval(im,green)
		b=Image.eval(im,blue)
		im=Image.merge("RGB",(r,g,b))
		return im
	b="1111011110111101111000100101110210010100101000010000001001010012"
	b+="1001011110111101111000100101001210110101000001000010001010010012"
	b+="1111010110111101111000010001110"
	b=b.split("2")
	im=Image.new("L",(33+15,7+13))
	data=im.load()
	for y in range(len(b)):
		for x in range(len(b[y])):
			data[x+6,y+6]=int(b[y][x])*255
	scale=15
	im=im.resize((im.size[0]*scale,im.size[1]*scale))
	data=im.load()
	def drawSin(width,height,vertoffset,horizoffset,thickness,darkness):
		for x in range(im.size[0]):
			y=scipy.sin((x-horizoffset)/float(width))*height+vertoffset
			for i in range(thickness):
				if 0<=y+i<im.size[1] and 0<=x<im.size[0]:
					#print x,im.size[0],y+i,im.size[1]
					data[x,y+i]=data[x,y+i]+darkness

	for i in range(5):
		print "line",i
		drawSin(randint(5,75),randint(-100,200),randint(0,im.size[1]),\
				randint(0,im.size[0]),randint(3,15),70)
	for i in range(10):
		im=im.filter(ImageFilter.SMOOTH_MORE)
	im=colorize(im)
	return im

im=genLogo()
im.save('logo.png',"PNG")

Wow, I can’t believe I took-on such a massive challenge this week! Some irony lies in the fact that I’ve worked harder and learned more in the last 4 days of all-day work (researching, reading, skimming other peoples’ code, and writing my own) than of the last 9 months of dental school. This is an incredible feeling of accomplishment. My program, *MINE*, which I coded 100% from scratch (using Python’s scripting platform as a strong base coupled with the Python Imaging Library (PIL), Tk bindings (Tkinter)). It polls the soundcard continuously and makes incredibly large spectrographs. I’ll explain more about it and its rationale later. It’s not finished, but it’s working… and working pretty darn well. I’m floored!

Here’s what it looks like when it’s running…

Pretty nice ‘eh? Yeah, that’s a GUI, but it can be run entirely from a headless server through a console as well. (see where I’m going with this?) Here’s some more of the output, cropped to emphasize QRSS signals. Keep in mind that the image below was cropped to less than 3,000 pixels high, whereas the original is over 8,000 pixels high!!!

My project is coming along nicely. This isn’t an incredibly robust spectrograph program, but it sure gets the job done quickly and easily. The code below will produce a realtime scrolling spectrograph entirely with Python! It polls the microphone (or default recording device), should work on any OS, and can be adjusted for vertical resolution / FFT frequency discretion resolution. It has some simple functions for filtering (check out the detrend filter!) and might serve as a good start to a spectrograph / frequency analysis project. It took my a long time to reach this point! I’ve worked with Python before, and dabbled with the Python Imaging Library (PIL), but this is my first experience with realtime linear data analysis and high-demand multi-threading. I hope it helps you. Below are screenshots of the program (two running at the same time) listening to the same radio signals (mostly Morse code) with standard output and with the “detrending filter” activated.

And the code…

import pyaudio
import scipy
import struct
import scipy.fftpack

from Tkinter import *
import threading
import time, datetime
import wckgraph
import math

import Image, ImageTk
from PIL import ImageOps
from PIL import ImageChops
import time
import random
import threading
import scipy

#ADJUST RESOLUTION OF VERTICAL FFT
bufferSize=2**11
#bufferSize=2**8

#ADJUSTS AVERAGING SPEED NOT VERTICAL RESOLUTION
#REDUCE HERE IF YOUR PC CANT KEEP UP
sampleRate=24000
#sampleRate=64000

p = pyaudio.PyAudio()
chunks=[]
ffts=[]
def stream():
        global chunks, inStream, bufferSize
        while True:
                chunks.append(inStream.read(bufferSize))

def record():
        global w, inStream, p, bufferSize
        inStream = p.open(format=pyaudio.paInt16,channels=1,\
                rate=sampleRate,input=True,frames_per_buffer=bufferSize)
        threading.Thread(target=stream).start()
        #stream()

def downSample(fftx,ffty,degree=10):
        x,y=[],[]
        for i in range(len(ffty)/degree-1):
                x.append(fftx[i*degree+degree/2])
                y.append(sum(ffty[i*degree:(i+1)*degree])/degree)
        return [x,y]

def smoothWindow(fftx,ffty,degree=10):
        lx,ly=fftx[degree:-degree],[]
        for i in range(degree,len(ffty)-degree):
                ly.append(sum(ffty[i-degree:i+degree]))
        return [lx,ly]

def smoothMemory(ffty,degree=3):
        global ffts
        ffts = ffts+[ffty]
        if len(ffts)< =degree: return ffty
        ffts=ffts[1:]
        return scipy.average(scipy.array(ffts),0)

def detrend(fftx,ffty,degree=10):
        lx,ly=fftx[degree:-degree],[]
        for i in range(degree,len(ffty)-degree):
                ly.append((ffty[i]-sum(ffty[i-degree:i+degree])/(degree*2))\
                          *2+128)
                #ly.append(fft[i]-(ffty[i-degree]+ffty[i+degree])/2)
        return [lx,ly]

def graph():
        global chunks, bufferSize, fftx,ffty, w
        if len(chunks)>0:
                data = chunks.pop(0)
                data=scipy.array(struct.unpack("%dB"%(bufferSize*2),data))
                #print "RECORDED",len(data)/float(sampleRate),"SEC"
                ffty=scipy.fftpack.fft(data)
                fftx=scipy.fftpack.rfftfreq(bufferSize*2, 1.0/sampleRate)
                fftx=fftx[0:len(fftx)/4]
                ffty=abs(ffty[0:len(ffty)/2])/1000
                ffty1=ffty[:len(ffty)/2]
                ffty2=ffty[len(ffty)/2::]+2
                ffty2=ffty2[::-1]
                ffty=ffty1+ffty2
                ffty=(scipy.log(ffty)-1)*120
                fftx,ffty=downSample(fftx,ffty,2)
                #fftx,ffty=detrend(fftx,ffty,30)
                #fftx,ffty=smoothWindow(fftx,ffty,10)
                #ffty=smoothMemory(ffty,3)
                #fftx,ffty=detrend(fftx,ffty,3)
                #print len(ffty)
                #print min(ffty),max(ffty)
                updatePic(fftx,ffty)
                reloadPic()
                #w.clear()
                #w.add(wckgraph.Axes(extent=(0, -1, 6000, 3)))
                #w.add(wckgraph.LineGraph([fftx,ffty]))
                #w.update()

        if len(chunks)>20:
                print "falling behind...",len(chunks)

def go(x=None):
        global w,fftx,ffty
        print "STARTING!"
        threading.Thread(target=record).start()
        while True:
                #record()
                graph()

def updatePic(datax,data):
     global im, iwidth, iheight
     strip=Image.new("L",(1,iheight))
     if len(data)>iheight:
             data=data[:iheight-1]
     #print "MAX FREQ:",datax[-1]
     strip.putdata(data)
     #print "%03d, %03d" % (max(data[-100:]), min(data[-100:]))
     im.paste(strip,(iwidth-1,0))
     im=im.offset(-1,0)
     root.update()

def reloadPic():
     global im, lab
     lab.image = ImageTk.PhotoImage(im)
     lab.config(image=lab.image)

root = Tk()
im=Image.open('./ramp.tif')
im=im.convert("L")
iwidth,iheight=im.size
im=im.crop((0,0,500,480))
#im=Image.new("L",(100,1024))
iwidth,iheight=im.size
root.geometry('%dx%d' % (iwidth,iheight))
lab=Label(root)
lab.place(x=0,y=0,width=iwidth,height=iheight)
go()

UPDATE! I’m not going to post the code for this yet (it’s very messy) but I got this thing to display a spectrograph on a canvas. What’s the advantage of that? Huge, massive spectrographs (thousands of pixels in all directions) can now be browsed in real time using scrollbars, and when you scroll it doesn’t stop recording, and you don’t lose any data! Super cool.