Archive for the 'General' Category

I don’t have long, but the picture speaks for itself. Sent from Florida to Canada. I used VE1VDM’s “Big Ears Grabber” in Canada.

I’ve been exchanging emails with Lubnaan Shaikhto for a little while regarding a project involving a realtime spectrograph controlled by a microcontroller (ATTiny2313). He had a few questions, I had a few recommendations, and several weeks went by and I heard nothing… until today when I saw this awesome youtube video of his incredible success! The guy built his own USB AVR programmer just for this project too. Double score! Way to go Lubnaan!

I’m proud to announce that my first public version of QRSS VD has been released!

>>>Join the QRSS VD User Group

>>>Visit the QRSS VD Webpage

After priding myself on my ingenuity a few weeks ago for documenting my homemade stealth indoor apartment antenna for 40m and 20m, it seems that the green movement has contrived a plan to cripple my successes. So far I’ve made a few dozen contacts in Morse code with my humble little setup (~20 watts of power, direct conversion receiver, indoor homemade antenna). The photo shows some QSL cards I’ve gotten. Anyhow, my apartment manager decided that my apartment needed to have solar panels added to it. It’s too early to tell for sure, but spinning the dial a few times and hearing *nothing* makes me think that it dramatically impacted my reception (and likely transmission) in a dramatic way.

A few days ago…

yesterday…

today

R.I.P. AJ4VD station… [sigh]

While working on the spectrograph software I’m so psychotic about completing this spring break, I happened to capture a cool signal from Italy. IW4DXW was sending some cool signals that I captured around 10.140 MHz. See how his callsign is written “visually” on the spectrogram? I thought I’d post it because it’s an encouraging sign that my software is going in the right direction. Also note the numerous QRSS FSK signals around it! So cool.

UPDATE! The guy emailed me his station information.

Holy… 200mW? That’s so awesome. Maybe one day I’ll build something equally as cool and epically useless =oD I will now return to my psychotic programming project what has been eating-up my spring break. [boom] I just snapped a screenshot for the future memory of this break. Yes, I’m on a XP machine. I’m at the W4DFU radio club station (they have nicer computers than I do!)

Several days straight of coding all day every day has started to mess with my mind. I’ve got to incorporate some relaxation time! Something good ‘ol youtube can assist with. I share with you the following videos today instead of Python code (there’s always time for that later).

So here’s the problem. I’m working on a software project and I’d love the startup screen to be flashy, but not tacky. I want the program to be ENTIRELY scripted, so pretty bitmap images are out. I have to generate it from the script, but how can I reliably print text on an image with fonts vary across operating systems? (i.e., many OS’s don’t have “arial.ttf”, or no truetype fonts at all!) The solution was a creative process which will unfold before you.

First, I created a 2D binary array to represent the alphabet using pixel fonts such as this as a reference. The word I’m trying to create is “QRSS VD” (the name of my program). I store the data in strings, as seen below. I use 1’s to mark pixels, and spaces to mark empty spaces.

data="""
1111 1111 1111 1111   1  1 111
1  1 1  1 1    1      1  1 1  1
1  1 1111 1111 1111   1  1 1  1
1 11 1 1     1    1   1 1  1  1
1111 1 11 1111 1111    1   111
"""

Once I think it looks nice, I replace the spaces with zeros to make it take up a lot of visual space…

data2="""
1111011110111101111000100101110
1001010010100001000000100101001
1001011110111101111000100101001
1011010100000100001000101001001
1111010110111101111000010001110
"""

Then I further obscure it by replacing linebreaks with a different symbol, such as the number 2, then break the lines so they’re not lined up…

b="1111011110111101111000100101110210010100101000010000001001010012"
b+="1001011110111101111000100101001210110101000001000010001010010012"
b+="1111010110111101111000010001110"

The result is a pretty cool way to obscure the text. I don’t know why you’d want to, but if you want to make sure that no one goes in and changes the letters around (at least without making them think pretty hard about it) you could look at ways to further encrypt this data stream. From here, I create an image using the Python Imaging Library, setting pixel values to 255*b[x,y] (so 0 stays 0 and 1 becomes 255, perfect for an 8-bit image). After enlarging, here’s the result:

That’s cool huh? Now let’s make it a little bit less pixelated. It’s not a cure-all method, but blurring it up a little with a bilinear filter helps a lot…

Then I apply the code below which applies a cool colormap to the pixel values. I’ll provide cleaner code for this later (I have a really cool way of generating colormaps and saving them as arrays of RGB tuples). Then I go through and plot some random sin wavs on top of it. Sweet! Here are 6 images generated from the program run 6 times. Notice the randomness of the sine wavs!

Vwa la! A different image is generated every time the script runs, and it requires no external files (bitmaps or fonts) and should work well on all operating systems. Take the idea and run with it!

from PIL import Image
from PIL import ImageOps
from PIL import ImageFilter
from random import randint
import scipy

def genLogo():
	colormap=[(0, 0, 129), (0, 0, 134), (0, 0, 139), (0, 0, 143), (0, 0, 148), (0, 0, 152), (0, 0, 157), (0, 0, 161), (0, 0, 166), (0, 0, 170), (0, 0, 175), (0, 0, 180), (0, 0, 184), (0, 0, 189), (0, 0, 193), (0, 0, 198), (0, 0, 202), (0, 0, 207), (0, 0, 211), (0, 0, 216), (0, 0, 220), (0, 0, 225), (0, 0, 230), (0, 0, 234), (0, 0, 239), (0, 0, 243), (0, 0, 248), (0, 0, 252), (0, 0, 255), (0, 0, 255), (0, 0, 255), (0, 0, 255), (0, 2, 255), (0, 7, 255), (0, 11, 255), (0, 14, 255), (0, 18, 255), (0, 23, 255), (0, 27, 255), (0, 31, 255), (0, 34, 255), (0, 39, 255), (0, 43, 255), (0, 47, 255), (0, 51, 255), (0, 54, 255), (0, 59, 255), (0, 63, 255), (0, 67, 255), (0, 71, 255), (0, 75, 255), (0, 79, 255), (0, 83, 255), (0, 87, 255), (0, 91, 255), (0, 95, 255), (0, 99, 255), (0, 103, 255), (0, 107, 255), (0, 111, 255), (0, 115, 255), (0, 119, 255), (0, 123, 255), (0, 127, 255), (0, 131, 255), (0, 135, 255), (0, 139, 255), (0, 143, 255), (0, 147, 255), (0, 151, 255), (0, 155, 255), (0, 159, 255), (0, 163, 255), (0, 167, 255), (0, 171, 255), (0, 175, 255), (0, 179, 255), (0, 183, 255), (0, 187, 255), (0, 191, 255), (0, 195, 255), (0, 199, 255), (0, 203, 255), (0, 207, 255), (0, 211, 255), (0, 215, 255), (0, 219, 254), (0, 223, 251), (0, 227, 248), (2, 231, 245), (5, 235, 241), (7, 239, 238), (11, 243, 235), (14, 247, 232), (18, 251, 228), (21, 255, 225), (23, 255, 222), (27, 255, 219), (31, 255, 215), (34, 255, 212), (37, 255, 208), (40, 255, 205), (44, 255, 203), (47, 255, 199), (50, 255, 195), (54, 255, 192), (57, 255, 189), (60, 255, 186), (63, 255, 183), (66, 255, 179), (70, 255, 176), (73, 255, 173), (76, 255, 170), (79, 255, 166), (83, 255, 163), (86, 255, 160), (89, 255, 157), (92, 255, 154), (95, 255, 150), (99, 255, 147), (102, 255, 144), (105, 255, 141), (108, 255, 137), (112, 255, 134), (115, 255, 131), (118, 255, 128), (121, 255, 125), (124, 255, 121), (128, 255, 118), (131, 255, 115), (134, 255, 112), (137, 255, 108), (141, 255, 105), (144, 255, 102), (147, 255, 99), (150, 255, 95), (154, 255, 92), (157, 255, 89), (160, 255, 86), (163, 255, 83), (166, 255, 79), (170, 255, 76), (173, 255, 73), (176, 255, 70), (179, 255, 66), (183, 255, 63), (186, 255, 60), (189, 255, 57), (192, 255, 54), (195, 255, 50), (199, 255, 47), (202, 255, 44), (205, 255, 41), (208, 255, 37), (212, 255, 34), (215, 255, 31), (218, 255, 28), (221, 255, 24), (224, 255, 21), (228, 255, 18), (231, 255, 15), (234, 255, 12), (238, 255, 8), (241, 252, 5), (244, 248, 2), (247, 244, 0), (250, 240, 0), (254, 236, 0), (255, 233, 0), (255, 229, 0), (255, 226, 0), (255, 221, 0), (255, 218, 0), (255, 215, 0), (255, 211, 0), (255, 207, 0), (255, 203, 0), (255, 199, 0), (255, 196, 0), (255, 192, 0), (255, 188, 0), (255, 184, 0), (255, 180, 0), (255, 177, 0), (255, 173, 0), (255, 169, 0), (255, 165, 0), (255, 162, 0), (255, 159, 0), (255, 155, 0), (255, 151, 0), (255, 147, 0), (255, 143, 0), (255, 140, 0), (255, 136, 0), (255, 132, 0), (255, 128, 0), (255, 125, 0), (255, 121, 0), (255, 117, 0), (255, 114, 0), (255, 110, 0), (255, 106, 0), (255, 102, 0), (255, 99, 0), (255, 95, 0), (255, 91, 0), (255, 88, 0), (255, 84, 0), (255, 80, 0), (255, 76, 0), (255, 73, 0), (255, 69, 0), (255, 65, 0), (255, 62, 0), (255, 58, 0), (255, 54, 0), (255, 51, 0), (255, 47, 0), (255, 43, 0), (255, 39, 0), (255, 36, 0), (255, 32, 0), (255, 28, 0), (255, 25, 0), (255, 21, 0), (253, 17, 0), (248, 14, 0), (244, 10, 0), (240, 6, 0), (235, 2, 0), (230, 0, 0), (225, 0, 0), (221, 0, 0), (217, 0, 0), (212, 0, 0), (207, 0, 0), (203, 0, 0), (198, 0, 0), (194, 0, 0), (189, 0, 0), (185, 0, 0), (180, 0, 0), (175, 0, 0), (171, 0, 0), (166, 0, 0), (162, 0, 0), (157, 0, 0), (152, 0, 0), (148, 0, 0), (144, 0, 0), (139, 0, 0), (134, 0, 0), (130, 0, 0), (134, 0, 0), (130, 0, 0)]
	def red(val):
		return colormap[val][0]
	def green(val):
		return colormap[val][1]
	def blue(val):
		return colormap[val][2]
	def colorize(im):
		r=Image.eval(im,red)
		g=Image.eval(im,green)
		b=Image.eval(im,blue)
		im=Image.merge("RGB",(r,g,b))
		return im
	b="1111011110111101111000100101110210010100101000010000001001010012"
	b+="1001011110111101111000100101001210110101000001000010001010010012"
	b+="1111010110111101111000010001110"
	b=b.split("2")
	im=Image.new("L",(33+15,7+13))
	data=im.load()
	for y in range(len(b)):
		for x in range(len(b[y])):
			data[x+6,y+6]=int(b[y][x])*255
	scale=15
	im=im.resize((im.size[0]*scale,im.size[1]*scale))
	data=im.load()
	def drawSin(width,height,vertoffset,horizoffset,thickness,darkness):
		for x in range(im.size[0]):
			y=scipy.sin((x-horizoffset)/float(width))*height+vertoffset
			for i in range(thickness):
				if 0<=y+i<im.size[1] and 0<=x<im.size[0]:
					#print x,im.size[0],y+i,im.size[1]
					data[x,y+i]=data[x,y+i]+darkness

	for i in range(5):
		print "line",i
		drawSin(randint(5,75),randint(-100,200),randint(0,im.size[1]),\
				randint(0,im.size[0]),randint(3,15),70)
	for i in range(10):
		im=im.filter(ImageFilter.SMOOTH_MORE)
	im=colorize(im)
	return im

im=genLogo()
im.save('logo.png',"PNG")

Wow, I can’t believe I took-on such a massive challenge this week! Some irony lies in the fact that I’ve worked harder and learned more in the last 4 days of all-day work (researching, reading, skimming other peoples’ code, and writing my own) than of the last 9 months of dental school. This is an incredible feeling of accomplishment. My program, *MINE*, which I coded 100% from scratch (using Python’s scripting platform as a strong base coupled with the Python Imaging Library (PIL), Tk bindings (Tkinter)). It polls the soundcard continuously and makes incredibly large spectrographs. I’ll explain more about it and its rationale later. It’s not finished, but it’s working… and working pretty darn well. I’m floored!

Here’s what it looks like when it’s running…

Pretty nice ‘eh? Yeah, that’s a GUI, but it can be run entirely from a headless server through a console as well. (see where I’m going with this?) Here’s some more of the output, cropped to emphasize QRSS signals. Keep in mind that the image below was cropped to less than 3,000 pixels high, whereas the original is over 8,000 pixels high!!!

My project is coming along nicely. This isn’t an incredibly robust spectrograph program, but it sure gets the job done quickly and easily. The code below will produce a realtime scrolling spectrograph entirely with Python! It polls the microphone (or default recording device), should work on any OS, and can be adjusted for vertical resolution / FFT frequency discretion resolution. It has some simple functions for filtering (check out the detrend filter!) and might serve as a good start to a spectrograph / frequency analysis project. It took my a long time to reach this point! I’ve worked with Python before, and dabbled with the Python Imaging Library (PIL), but this is my first experience with realtime linear data analysis and high-demand multi-threading. I hope it helps you. Below are screenshots of the program (two running at the same time) listening to the same radio signals (mostly Morse code) with standard output and with the “detrending filter” activated.

And the code…

import pyaudio
import scipy
import struct
import scipy.fftpack

from Tkinter import *
import threading
import time, datetime
import wckgraph
import math

import Image, ImageTk
from PIL import ImageOps
from PIL import ImageChops
import time
import random
import threading
import scipy

#ADJUST RESOLUTION OF VERTICAL FFT
bufferSize=2**11
#bufferSize=2**8

#ADJUSTS AVERAGING SPEED NOT VERTICAL RESOLUTION
#REDUCE HERE IF YOUR PC CANT KEEP UP
sampleRate=24000
#sampleRate=64000

p = pyaudio.PyAudio()
chunks=[]
ffts=[]
def stream():
        global chunks, inStream, bufferSize
        while True:
                chunks.append(inStream.read(bufferSize))

def record():
        global w, inStream, p, bufferSize
        inStream = p.open(format=pyaudio.paInt16,channels=1,\
                rate=sampleRate,input=True,frames_per_buffer=bufferSize)
        threading.Thread(target=stream).start()
        #stream()

def downSample(fftx,ffty,degree=10):
        x,y=[],[]
        for i in range(len(ffty)/degree-1):
                x.append(fftx[i*degree+degree/2])
                y.append(sum(ffty[i*degree:(i+1)*degree])/degree)
        return [x,y]

def smoothWindow(fftx,ffty,degree=10):
        lx,ly=fftx[degree:-degree],[]
        for i in range(degree,len(ffty)-degree):
                ly.append(sum(ffty[i-degree:i+degree]))
        return [lx,ly]

def smoothMemory(ffty,degree=3):
        global ffts
        ffts = ffts+[ffty]
        if len(ffts)< =degree: return ffty
        ffts=ffts[1:]
        return scipy.average(scipy.array(ffts),0)

def detrend(fftx,ffty,degree=10):
        lx,ly=fftx[degree:-degree],[]
        for i in range(degree,len(ffty)-degree):
                ly.append((ffty[i]-sum(ffty[i-degree:i+degree])/(degree*2))\
                          *2+128)
                #ly.append(fft[i]-(ffty[i-degree]+ffty[i+degree])/2)
        return [lx,ly]

def graph():
        global chunks, bufferSize, fftx,ffty, w
        if len(chunks)>0:
                data = chunks.pop(0)
                data=scipy.array(struct.unpack("%dB"%(bufferSize*2),data))
                #print "RECORDED",len(data)/float(sampleRate),"SEC"
                ffty=scipy.fftpack.fft(data)
                fftx=scipy.fftpack.rfftfreq(bufferSize*2, 1.0/sampleRate)
                fftx=fftx[0:len(fftx)/4]
                ffty=abs(ffty[0:len(ffty)/2])/1000
                ffty1=ffty[:len(ffty)/2]
                ffty2=ffty[len(ffty)/2::]+2
                ffty2=ffty2[::-1]
                ffty=ffty1+ffty2
                ffty=(scipy.log(ffty)-1)*120
                fftx,ffty=downSample(fftx,ffty,2)
                #fftx,ffty=detrend(fftx,ffty,30)
                #fftx,ffty=smoothWindow(fftx,ffty,10)
                #ffty=smoothMemory(ffty,3)
                #fftx,ffty=detrend(fftx,ffty,3)
                #print len(ffty)
                #print min(ffty),max(ffty)
                updatePic(fftx,ffty)
                reloadPic()
                #w.clear()
                #w.add(wckgraph.Axes(extent=(0, -1, 6000, 3)))
                #w.add(wckgraph.LineGraph([fftx,ffty]))
                #w.update()

        if len(chunks)>20:
                print "falling behind...",len(chunks)

def go(x=None):
        global w,fftx,ffty
        print "STARTING!"
        threading.Thread(target=record).start()
        while True:
                #record()
                graph()

def updatePic(datax,data):
     global im, iwidth, iheight
     strip=Image.new("L",(1,iheight))
     if len(data)>iheight:
             data=data[:iheight-1]
     #print "MAX FREQ:",datax[-1]
     strip.putdata(data)
     #print "%03d, %03d" % (max(data[-100:]), min(data[-100:]))
     im.paste(strip,(iwidth-1,0))
     im=im.offset(-1,0)
     root.update()

def reloadPic():
     global im, lab
     lab.image = ImageTk.PhotoImage(im)
     lab.config(image=lab.image)

root = Tk()
im=Image.open('./ramp.tif')
im=im.convert("L")
iwidth,iheight=im.size
im=im.crop((0,0,500,480))
#im=Image.new("L",(100,1024))
iwidth,iheight=im.size
root.geometry('%dx%d' % (iwidth,iheight))
lab=Label(root)
lab.place(x=0,y=0,width=iwidth,height=iheight)
go()

UPDATE! I’m not going to post the code for this yet (it’s very messy) but I got this thing to display a spectrograph on a canvas. What’s the advantage of that? Huge, massive spectrographs (thousands of pixels in all directions) can now be browsed in real time using scrollbars, and when you scroll it doesn’t stop recording, and you don’t lose any data! Super cool.

I’m stretching the limits of what these software platforms were designed to to, but I’m impressed such a haphazard hacked-together code as this produces fast, functional results. The code below is the simplest case code I could create which would graph the audio spectrum of the microphone input (or a WAV file or other sound as it’s being played). There’s some smoothing involved (moving window down-sampling along the frequency axis and sequence averaging along the time axis) but the darn thing seems to keep up with realtime audio input at a good 30+ FPS on my modest maching. It should work on Windows and Linux. I chose not to go with matplotlib because I didn’t think it was fast enough for my needs in this one case (although I love it in every other way). Here’s what the code below looks like running:

NOTE that this program was designed with the intent of recording the FFTs, therefore if the program “falls behind” the realtime input, it will buffer the sound on its own and try to catch up (accomplished by two layers of threading). In this way, *EVERY MOMENT* of audio is interpreted. If you’re just trying to create a spectrograph for simple purposes, have it only sample the audio when it needs to, rather than having it sample audio continuously.

import pyaudio
import scipy
import struct
import scipy.fftpack

from Tkinter import *
import threading
import time, datetime
import wckgraph
import math

#ADJUST THIS TO CHANGE SPEED/SIZE OF FFT
bufferSize=2**11
#bufferSize=2**8

# ADJUST THIS TO CHANGE SPEED/SIZE OF FFT
sampleRate=48100
#sampleRate=64000

p = pyaudio.PyAudio()
chunks=[]
ffts=[]
def stream():
        global chunks, inStream, bufferSize
        while True:
                chunks.append(inStream.read(bufferSize))

def record():
        global w, inStream, p, bufferSize
        inStream = p.open(format=pyaudio.paInt16,channels=1,\
                rate=sampleRate,input=True,frames_per_buffer=bufferSize)
        threading.Thread(target=stream).start()

def downSample(fftx,ffty,degree=10):
        x,y=[],[]
        for i in range(len(ffty)/degree-1):
                x.append(fftx[i*degree+degree/2])
                y.append(sum(ffty[i*degree:(i+1)*degree])/degree)
        return [x,y]

def smoothWindow(fftx,ffty,degree=10):
        lx,ly=fftx[degree:-degree],[]
        for i in range(degree,len(ffty)-degree):
                ly.append(sum(ffty[i-degree:i+degree]))
        return [lx,ly]

def smoothMemory(ffty,degree=3):
        global ffts
        ffts = ffts+[ffty]
        if len(ffts)< =degree: return ffty
        ffts=ffts[1:]
        return scipy.average(scipy.array(ffts),0)

def detrend(fftx,ffty,degree=10):
        lx,ly=fftx[degree:-degree],[]
        for i in range(degree,len(ffty)-degree):
                ly.append(ffty[i]-sum(ffty[i-degree:i+degree])/(degree*2))
                #ly.append(fft[i]-(ffty[i-degree]+ffty[i+degree])/2)
        return [lx,ly]

def graph():
        global chunks, bufferSize, fftx,ffty, w
        if len(chunks)>0:
                data = chunks.pop(0)
                data=scipy.array(struct.unpack("%dB"%(bufferSize*2),data))
                #print "RECORDED",len(data)/float(sampleRate),"SEC"
                ffty=scipy.fftpack.fft(data)
                fftx=scipy.fftpack.rfftfreq(bufferSize*2, 1.0/sampleRate)
                fftx=fftx[0:len(fftx)/4]
                ffty=abs(ffty[0:len(ffty)/2])/1000
                ffty1=ffty[:len(ffty)/2]
                ffty2=ffty[len(ffty)/2::]+2
                ffty2=ffty2[::-1]
                ffty=ffty1+ffty2
                ffty=scipy.log(ffty)-2
                #fftx,ffty=downSample(fftx,ffty,5)
                #fftx,ffty=detrend(fftx,ffty,30)
                #fftx,ffty=smoothWindow(fftx,ffty,10)
                ffty=smoothMemory(ffty,3)
                #fftx,ffty=detrend(fftx,ffty,10)
                w.clear()
                #w.add(wckgraph.Axes(extent=(0, -1, fftx[-1], 3)))
                w.add(wckgraph.Axes(extent=(0, -1, 6000, 3)))
                w.add(wckgraph.LineGraph([fftx,ffty]))
                w.update()
        if len(chunks)>20:
                print "falling behind...",len(chunks)

def go(x=None):
        global w,fftx,ffty
        print "STARTING!"
        threading.Thread(target=record).start()
        while True:
                graph()

root = Tk()
root.title("SPECTRUM ANALYZER")
root.geometry('500x200')
w = wckgraph.GraphWidget(root)
w.pack(fill=BOTH, expand=1)
go()
mainloop()