<?xml version="1.0" encoding="windows-1252"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>The Blogging Protagonist &#187; Linux</title>
	<atom:link href="http://www.SWHarden.com/blog/category/linux/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.SWHarden.com/blog</link>
	<description>A collection of thoughts in technological degradation</description>
	<lastBuildDate>Tue, 27 Jul 2010 21:45:07 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.6</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Converting ASCII Text to CW Morse Code with Linux</title>
		<link>http://www.SWHarden.com/blog/2010-02-02-converting-ascii-text-to-cw-morse-code-with-linux/</link>
		<comments>http://www.SWHarden.com/blog/2010-02-02-converting-ascii-text-to-cw-morse-code-with-linux/#comments</comments>
		<pubDate>Tue, 02 Feb 2010 15:58:54 +0000</pubDate>
		<dc:creator>Scott</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Radio]]></category>

		<guid isPermaLink="false">http://www.SWHarden.com/blog/?p=1683</guid>
		<description><![CDATA[I wanted a way to have a bunch of Morse code mp3s on my mp3 player (with a WPM/speed that I decide and I found an easy way to do it with Linux.  Rather than downloading existing mp3s of boring text, I wanted to be able to turn ANY text into Morse code, so [...]]]></description>
			<content:encoded><![CDATA[<p><strong>I wanted a way to have a bunch of Morse code mp3s on my mp3 player (with a WPM/speed that I decide</strong> and I found an easy way to do it with Linux.  Rather than downloading existing mp3s of boring text, I wanted to be able to turn ANY text into Morse code, so I could copy something interesting (perhaps the news?  hackaday? bash.org?).  It&#8217;s a little devious, but my plan is to practice copying Morse code during class when lectures become monotonous.  [The guy who teaches infectious diseases is the most boring person I ever met, I learn nothing from class, and on top of that he doesn't allow laptops to be out!]  So, here&#8217;s what I did in case it helps anyone else out there&#8230;</p>
<p><strong>Step 0: GET THE REQUIRED PROGRAMS!</strong> Yes, there&#8217;s a step zero.  Make sure you have installed <a href="http://www.Python.org" onclick="javascript:urchinTracker ('/outbound/article/www.Python.org');">Python</a>, <a href="http://cwtext.sourceforge.net/" onclick="javascript:urchinTracker ('/outbound/article/cwtext.sourceforge.net');">cwtext</a>, and <a href="http://lame.sourceforge.net/" onclick="javascript:urchinTracker ('/outbound/article/lame.sourceforge.net');">lame</a>. Now you&#8217;re ready to roll!</p>
<p><strong>Step 1: PREPARE SOME TEXT!</strong> I went to Wikipedia and copy/pasted an ENTIRE article into a text file called <strong>in.txt</strong>.  Don&#8217;t worry about special characters (such as &#8221; and * and #), we&#8217;ll fix them with the following python script.</p>
<pre class="prettyprint python">
import time
f=open("out.txt")
raw=f.read()
f.close()

cmd  = """echo "TEST" | cwpcm -w 7 | """
cmd += """lame -r -m m -b 8 --resample 8 -q9 - - > text.mp3"""

import os
i=0
for chunk in raw.split("\n")[5:]:
        if chunk.count(" ")>50:
                i+=1
                print "\n\nfile",i, chunk.count(" "), "words\n"
		do = cmd.replace("TEST",chunk).replace("text","%02d"%i)
		print "running:",do,
		time.sleep(1)
		print "\n\nSTART ...",
                os.system(do)
		print "DONE"
</pre>
<p><strong>Step 2: MAKE MP3s OF THE TEXT!</strong> There should be a new file, <strong>out.txt</strong>, which is cleaned-up nicely.  Run the following script to turn every paragraph of text with more than 50 words into an mp3 file&#8230;</p>
<pre class="prettyprint python">f=open("out.txt")
raw=f.read()
f.close()
cmd = """echo "TEST" | cwpcm -w 13 | sox -r 44k -u -b 8 -t raw - text.wav"""
cmd+="""; lame --preset phone text.wav text.mp3; rm text.wav"""
import os
i=0
for chunk in raw.split("\n")[5:]:
	if chunk.count(" ")&gt;50:
		i+=1
		print i, chunk.count(" "), "words"
		os.system(cmd.replace("TEST",chunk).replace("text","%02d"%i))</pre>
<p>Now you should have a directory filled with mp3 files which you can skip through (or shuffle!) using your handy dandy mp3 player.  Note that &#8220;-w 13&#8243; means 13 WPM (words per minute).  Simply change that number to change the speed.</p>
<p>Good luck with your CW practice!<br />
&#8211;Scott (AJ4VD)</p>
]]></content:encoded>
			<wfw:commentRss>http://www.SWHarden.com/blog/2010-02-02-converting-ascii-text-to-cw-morse-code-with-linux/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>pySquelch &#8211; Frequency Activity Reports via Python</title>
		<link>http://www.SWHarden.com/blog/2009-06-18-pysquelch-frequency-activity-reports-via-python/</link>
		<comments>http://www.SWHarden.com/blog/2009-06-18-pysquelch-frequency-activity-reports-via-python/#comments</comments>
		<pubDate>Fri, 19 Jun 2009 03:59:01 +0000</pubDate>
		<dc:creator>Scott</dc:creator>
				<category><![CDATA[Circuitry]]></category>
		<category><![CDATA[General]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Radio]]></category>

		<guid isPermaLink="false">http://www.SWHarden.com/blog/?p=1267</guid>
		<description><![CDATA[I&#8217;ve been working on the pySquelch project which is basically a method to graph frequency usage with respect to time.  The code I&#8217;m sharing below listens to the microphone jack on the sound card (hooked up to a radio) and determines when transmissions begin and end.  First, I&#8217;ll entice you by showing some [...]]]></description>
			<content:encoded><![CDATA[<p><b>I&#8217;ve been working on the pySquelch project</b> which is basically a method to graph frequency usage with respect to time.  The code I&#8217;m sharing below listens to the microphone jack on the sound card (hooked up to a radio) and determines when transmissions begin and end.  First, I&#8217;ll entice you by showing some nice graphs of the output!  I ran the code below for 24 hours and this is the result&#8230;<br />
<a href="http://www.SWHarden.com/blog/images/1png.png" onclick="javascript:urchinTracker ('/outbound/article/www.SWHarden.com');"><img src="http://www.SWHarden.com/blog/images/1png-525x253.png" alt="1png" title="1png" width="525" height="253" class="alignnone size-medium wp-image-1268" /></a><br />
<b>Pretty good &#8216;eh?</b> This graph represents traces of the frequency activity with respect to time.  The semi-transparent gray line represents the raw frequency usage in fractional minutes the frequency was tied-up by transmissions.  The solid blue line represents the same data but smoothed by 10 minutes (in both directions) by the Gaussian smoothing method modified slightly from my <a href="http://www.swharden.com/blog/2008-11-17-linear-data-smoothing-in-python/" >linear data smoothing with Python page</a>.<br />
<a href="http://www.SWHarden.com/blog/images/2png.png" onclick="javascript:urchinTracker ('/outbound/article/www.SWHarden.com');"><img src="http://www.SWHarden.com/blog/images/2png-525x253.png" alt="2png" title="2png" width="525" height="253" class="alignnone size-medium wp-image-1269" /></a></p>
<p><b>I used the code below to generate the log, and the code further below to create the graph from the log file.</b> Assuming your microphone is enabled and everything else is working, this software will require you to determine your own threshold for talking vs. no talking.  Read the code and you&#8217;ll figure out how test your sound card settings.</p>
<p><b>If you want to try this yourself</b> you need a Linux system (a Windows system version could be created simply by replacing <i>getVolEach()</i> with a Windows-based audio level detection system) with Python and the alsaaudio, numpy, and matplotlib libraries. Try running the code on your own, and if it doesn&#8217;t recognize a library &#8220;aptitude search&#8221; for it.  Everything you need can be installed from packages in the common repository.</p>
<pre class="prettyprint python">
#pySquelchLogger.py
import time, random, alsaaudio, audioop
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE,alsaaudio.PCM_NONBLOCK)
inp.setchannels(2)
inp.setrate(1000)
inp.setformat(alsaaudio.PCM_FORMAT_S8)
inp.setperiodsize(100)
addToLog=""
lastLogTime=0

testLevel=False ### SET THIS TO 'True' TO TEST YOUR SOUNDCARD

def getVolEach():
        # this is a quick way to detect activity.
        # modify this function to use alternate methods of detection.
	while True:
		l,data = inp.read() # poll the audio device
		if l>0: break
	vol = audioop.max(data,1) # get the maximum amplitude
	if testLevel: print vol
	if vol>10: return True ### SET THIS NUMBER TO SUIT YOUR NEEDS ###
	return False

def getVol():
        # reliably detect activity by getting 3 consistant readings.
	a,b,c=True,False,False
	while True:
		a=getVolEach()
		b=getVolEach()
		c=getVolEach()
		if a==b==c:
			if testLevel: print "RESULT:",a
			break
	if a==True: time.sleep(1)
	return a

def updateLog():
        # open the log file, append the new data, and save it again.
	global addToLog,lastLogTime
	#print "UPDATING LOG"
	if len(addToLog)>0:
        	f=open('log.txt','a')
        	f.write(addToLog)
        	f.close()
        	addToLog=""
	lastLogTime=time.mktime(time.localtime())

def findSquelch():
        # this will record a single transmission and store its data.
	global addToLog
	while True: # loop until we hear talking
		time.sleep(.5)
		if getVol()==True:
			start=time.mktime(time.localtime())
			print start,
			break
	while True: # loop until talking stops
		time.sleep(.1)
		if getVol()==False:
			length=time.mktime(time.localtime())-start
			print length
			break
	newLine="%d,%d "%(start,length)
	addToLog+=newLine
	if start-lastLogTime>30: updateLog() # update the log

while True:
	findSquelch()
</pre>
<p><b>The logging code (above) produces a log file like this (below).</b> The values represent the start time of each transmission (in <a href="http://en.wikipedia.org/wiki/Unix_time" onclick="javascript:urchinTracker ('/outbound/article/en.wikipedia.org');">seconds since epoch</a>) followed by the duration of the transmission.</p>
<pre class="prettyprint python">
#log.txt
1245300044,5 1245300057,4 1245300063,16 1245300094,13 1245300113,4 1245300120,14 1245300195,4 1245300295,4 1245300348,4 1245300697,7 1245300924,3 1245301157,4 1245301207,12 1245301563,4 1245302104,6 1245302114,6 1245302192,3 1245302349,4 1245302820,4 1245304812,13 1245308364,10 1245308413,14 1245312008,14 1245313953,11 1245314008,6 1245314584,4 1245314641,3 1245315212,5 1245315504,6 1245315604,13 1245315852,3 1245316255,6 1245316480,5 1245316803,3 1245316839,6 1245316848,11 1245316867,5 1245316875,12 1245316893,13 1245316912,59 1245316974,12 1245316988,21 1245317011,17 1245317044,10 1245317060,6 1245317071,7 1245317098,33 1245317140,96 1245317241,15 1245317259,14 1245317277,8 1245317298,18 1245317322,103 1245317435,40 1245317488,18 1245317508,34 1245317560,92 1245317658,29 1245317697,55 1245317755,33 1245317812,5 1245317818,7 1245317841,9 1245317865,25 1245317892,79 1245317972,30 1245318007,8 1245318021,60 1245318083,28 1245318114,23 1245318140,25 1245318167,341 1245318512,154 1245318670,160 1245318834,22 1245318859,9 1245318870,162 1245319042,57 1245319102,19 1245319123,30 1245319154,18 1245319206,5 1245319214,13 1245319229,6 1245319238,6 1245319331,9 1245319341,50 1245319397,71 1245319470,25 1245319497,40 1245319540,8 1245319551,77 1245319629,4 1245319638,36 1245319677,158 1245319837,25 1245319865,40 1245319907,33 1245319948,92 1245320043,26 1245320100,9 1245320111,34 1245320146,8 1245320159,6 1245320167,8 1245320181,12 1245320195,15 1245320212,14 1245320238,18 1245320263,46 1245320310,9 1245320326,22 1245320352,27 1245320381,15 1245320398,24 1245320425,57 1245320483,16 1245320501,40 1245320543,43 1245320589,65 1245320657,63 1245320722,129 1245320853,33 1245320889,50 1245320940,1485 1245322801,7 1245322809,103 1245322923,5 1245322929,66 1245323553,4 1245324203,15 1245324383,5 1245324570,7 1245324835,4 1245325200,8 1245325463,5 1245326414,12 1245327340,12 1245327836,4 1245327973,4 1245330006,12 1245331244,11 1245331938,11 1245332180,5 1245332187,81 1245332573,5 1245333609,12 1245334447,10 1245334924,9 1245334945,4 1245334971,4 1245335031,9 1245335076,11 1245335948,16 1245335965,27 1245335993,113 1245336107,79 1245336187,64 1245336253,37 1245336431,4 1245336588,5 1245336759,7 1245337048,3 1245337206,13 1245337228,4 1245337309,4 1245337486,6 1245337536,8 1245337565,38 1245337608,100 1245337713,25 1245337755,169 1245337930,8 1245337941,20 1245337967,6 1245337978,7 1245337996,20 1245338019,38 1245338060,127 1245338192,30 1245338227,22 1245338250,15 1245338272,15 1245338310,3 1245338508,4 1245338990,5 1245339136,5 1245339489,8 1245339765,4 1245340220,5 1245340233,6 1245340266,10 1245340278,22 1245340307,7 1245340315,28 1245340359,32 1245340395,4 1245340403,41 1245340446,46 1245340494,58 1245340554,17 1245340573,21 1245340599,3 1245340604,5 1245340611,46 1245340661,26 1245340747,4 1245340814,14 1245341043,4 1245341104,4 1245341672,4 1245341896,5 1245341906,3 1245342301,3 1245342649,6 1245342884,5 1245342929,4 1245343314,6 1245343324,10 1245343335,16 1245343353,39 1245343394,43 1245343439,62 1245343561,3 1245343790,4 1245344115,3 1245344189,5 1245344233,4 1245344241,6 1245344408,12 1245344829,3 1245345090,5 1245345457,5 1245345689,4 1245346086,3 1245347112,12 1245348006,14 1245348261,10 1245348873,4 1245348892,3 1245350303,11 1245350355,4 1245350766,5 1245350931,3 1245351605,14 1245351673,55 1245351729,23 1245351754,5 1245352123,37 1245352163,21 1245352186,18 1245352209,40 1245352251,49 1245352305,8 1245352315,5 1245352321,6 1245352329,22 1245352353,48 1245352404,77 1245352483,58 1245352543,17 1245352570,19 1245352635,5 1245352879,3 1245352899,5 1245352954,4 1245352962,6 1245352970,58 1245353031,21 1245353055,14 1245353071,52 1245353131,37 1245353170,201 1245353373,56 1245353431,18 1245353454,47 1245353502,13 1245353519,106 1245353627,10 1245353647,12 1245353660,30 1245353699,42 1245353746,28 1245353776,29 1245353806,9 1245353818,21 1245353841,10 1245353853,6 1245353862,224 1245354226,4 1245354964,63 1245355029,4 1245355036,142 1245355180,148 1245355330,7 1245355338,23 1245355363,9 1245355374,60 1245355437,142 1245355581,27 1245355609,5 1245355615,2 1245355630,64 1245355700,7 1245355709,73 1245355785,45 1245355834,85 1245355925,9 1245356234,5 1245356620,6 1245356629,12 1245356643,29 1245356676,120 1245356798,126 1245356937,62 1245357001,195 1245357210,17 1245357237,15 1245357258,24 1245357284,53 1245357339,2 1245357345,27 1245357374,76 1245357452,28 1245357482,42 1245357529,14 1245357545,35 1245357582,74 1245357661,30 1245357693,19 1245357714,38 1245357758,11 1245357777,37 1245357817,49 1245357868,19 1245357891,31 1245357931,48 1245357990,49 1245358043,24 1245358082,22 1245358108,17 1245358148,18 1245358168,7 1245358179,6 1245358186,19 1245358209,17 1245358229,5 1245358240,9 1245358252,10 1245358263,6 1245358272,9 1245358296,26 1245358328,49 1245358381,6 1245358389,38 1245358453,19 1245358476,24 1245358504,21 1245358533,76 1245358628,24 1245358653,10 1245358669,105 1245358781,20 1245358808,14 1245358836,6 1245358871,61 1245358933,0 1245358936,44 1245358982,11 1245358996,25 1245359023,15 1245359040,32 1245359076,19 1245359099,13 1245359117,16 1245359138,12 1245359161,33 1245359215,32 1245359249,14 1245359272,7 1245359314,10 1245359333,36 1245359371,21 1245359424,10 1245359447,61 1245359514,32 1245359560,42 1245359604,87 1245359700,60 1245359762,23 1245359786,4 1245359791,8 1245359803,6 1245359813,107 1245359922,29 1245359953,22 1245359978,86 1245360069,75 1245360147,22 1245360170,0 1245360184,41 1245360239,15 1245360256,34 1245360301,37 1245360339,1 1245360342,28 1245360372,20 1245360394,32 1245360440,24 1245360526,3 1245360728,3 1245361011,4 1245361026,35 1245361064,137 1245361359,5 1245362172,11 1245362225,21 1245362248,51 1245362302,20 1245362334,42 1245362418,12 1245362468,7 1245362557,9 1245362817,3 1245363175,4 1245363271,4 1245363446,3 1245363539,4 1245363573,4 1245363635,1 1245363637,3 1245363740,5 1245363875,3 1245364075,4 1245364354,14 1245364370,19 1245364391,49 1245364442,34 1245364478,23 1245364502,80 1245364633,15 1245364650,8 1245364673,16 1245364691,47 1245364739,53 1245364795,39 1245364836,25 1245365353,4 1245365640,11 1245365665,5 1245365726,8 1245365778,7 1245365982,4 1245366017,13 1245366042,6 1245366487,4 1245366493,4 1245366500,4 1245366507,3 1245366622,5 1245366690,5 1245366946,4 1245366953,16 1245366975,8 1245366996,7 1245367005,7 1245367031,6 1245367040,9 1245367051,7 1245367059,23 1245367084,76 1245367166,158 1245367740,4 1245367804,3 1245367847,4 1245367887,9 1245369300,10 1245369611,12 1245370038,10 1245370374,8 1245370668,5 1245370883,5 1245370927,7 1245370945,9 1245370961,16 1245370978,414 1245371398,135 1245371535,252 1245371791,238 1245372034,199 1245372621,4 1245372890,5 1245373043,7 1245373060,9 1245373073,6 1245373081,68 1245373151,10 1245373162,49 1245373212,79 1245373300,12 1245373313,38 1245373353,20 1245373374,59 1245373435,28 1245373465,94 1245373560,11 1245373574,53 1245373629,22 1245373654,6 1245373662,334 1245373998,169 1245374176,41 1245374219,26 1245374246,51 1245374299,31 1245374332,57 1245374391,55 1245374535,4 1245374759,7 1245374769,200 1245374971,215 1245375188,181 1245375371,81 1245375455,59 1245375516,33 1245375552,19 1245375572,56 1245375629,220 1245375850,32 1245375884,26 1245375948,7 1245375964,114 1245376473,4 1245376810,13 1245378296,10 1245378950,12 1245379004,3 1245379569,4 1245379582,4 1245379615,6 1245380030,3 1245380211,4 1245380412,14 1245380727,4 1245380850,4 </pre>
<p><b>This log file</b> is only 7.3 KB.  At this rate, a years&#8217; worth of log data can be stored in less than 3MB of plain text files.  Awesome! The data presented here can be graphed (producing the image at the top of the page) using the following code:</p>
<pre class="prettyprint python">
#pySquelchGrapher.py
print "loading libraries...",
import pylab, datetime, numpy
print "complete"

def loadData(fname="log.txt"):
	print "loading data...",
	# load signal/duration from log file
	f=open(fname)
	raw=f.read()
	f.close()
	raw=raw.replace('\n',' ')
	raw=raw.split(" ")
	signals=[]
	for line in raw:
		if len(line)&lt;3: continue
		line=line.split(',')
		sec=datetime.datetime.fromtimestamp(int(line[0]))
		dur=int(line[1])
		signals.append([sec,dur])
	print "complete"
	return signals

def findDays(signals):
	# determine which days are in the log file
	print "finding days...",
	days=[]
	for signal in signals:
		day = signal[0].date()
		if not day in days:
			days.append(day)
	print "complete"
	return days

def genMins(day):
	# generate an array for every minute in a certain day
	print "generating bins...",
	mins=[]
	startTime=datetime.datetime(day.year,day.month,day.day)
	minute=datetime.timedelta(minutes=1)
	for i in xrange(60*60):
		mins.append(startTime+minute*i)
	print "complete"
	return mins

def fillMins(mins,signals):
	print "filling bins...",
	vals=[0]*len(mins)
	dayToDo=signals[0][0].date()
	for signal in signals:
		if not signal[0].date() == dayToDo: continue
		sec=signal[0]
		dur=signal[1]
		prebuf = sec.second
		minOfDay=sec.hour*60+sec.minute
		if dur+prebuf&lt;60: # simple case, no rollover seconds
			vals[minOfDay]=dur
		else: # if duration exceeds the minute the signal started in
			vals[minOfDay]=60-prebuf
			dur=dur+prebuf
			while (dur>0): # add rollover seconds to subsequent minutes
				minOfDay+=1
				dur=dur-60
				if dur< =0: break
				if dur>=60: vals[minOfDay]=60
				else: vals[minOfDay]=dur
	print "complete"
	return vals

def normalize(vals):
	print "normalizing data...",
	divBy=float(max(vals))
	for i in xrange(len(vals)):
		vals[i]=vals[i]/divBy
	print "complete"
	return vals

def smoothListGaussian(list,degree=10):
	print "smoothing...",
	window=degree*2-1
	weight=numpy.array([1.0]*window)
	weightGauss=[]
	for i in range(window):
		i=i-degree+1
		frac=i/float(window)
		gauss=1/(numpy.exp((4*(frac))**2))
		weightGauss.append(gauss)
	weight=numpy.array(weightGauss)*weight
	smoothed=[0.0]*(len(list)-window)
	for i in range(len(smoothed)):
	  smoothed[i]=sum(numpy.array(list[i:i+window])*weight)/sum(weight)
	while len(list)>len(smoothed)+int(window/2):
		smoothed.insert(0,smoothed[0])
	while len(list)>len(smoothed):
		smoothed.append(smoothed[0])
	print "complete"
	return smoothed

signals=loadData()
days=findDays(signals)
for day in days:
	mins=genMins(day)
	vals=normalize(fillMins(mins,signals))
	fig=pylab.figure()
	pylab.grid(alpha=.2)
	pylab.plot(mins,vals,'k',alpha=.1)
	pylab.plot(mins,smoothListGaussian(vals),'b',lw=1)
	pylab.axis([day,day+datetime.timedelta(days=1),None,None])
	fig.autofmt_xdate()
	pylab.title("147.120 MHz Usage for "+str(day))
	pylab.xlabel("time of day")
	pylab.ylabel("fractional usage")
	pylab.show()
</pre>
<p><b>If you have any questions,</b> Google first, then feel free to contact me if you still can&#8217;t get it.  Good luck!!</p>
]]></content:encoded>
			<wfw:commentRss>http://www.SWHarden.com/blog/2009-06-18-pysquelch-frequency-activity-reports-via-python/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Python-Powered Frequency Activity Logger</title>
		<link>http://www.SWHarden.com/blog/2009-06-12-python-powered-frequency-activity-logger/</link>
		<comments>http://www.SWHarden.com/blog/2009-06-12-python-powered-frequency-activity-logger/#comments</comments>
		<pubDate>Fri, 12 Jun 2009 22:42:32 +0000</pubDate>
		<dc:creator>Scott</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Radio]]></category>

		<guid isPermaLink="false">http://www.SWHarden.com/blog/?p=1245</guid>
		<description><![CDATA[As anyone who visits this website can tell you, I&#8217;m inexplicably drawn toward inventing new ways of using Python (often in conjunction with MatPlotLib) to graphically visualize data in new ways.  When I found out a fellow ham in Orlando was using his computer to stream (and serve recorded streams of) a popular local [...]]]></description>
			<content:encoded><![CDATA[<p><b>As anyone who visits this website can tell you,</b> I&#8217;m inexplicably drawn toward inventing new ways of using <a href="http://www.Python.org" onclick="javascript:urchinTracker ('/outbound/article/www.Python.org');">Python</a> (often in conjunction with <a href="http://matplotlib.sourceforge.net" onclick="javascript:urchinTracker ('/outbound/article/matplotlib.sourceforge.net');">MatPlotLib</a>) to graphically visualize data in new ways.  When I found out a fellow ham in Orlando was using his computer to stream (and serve recorded streams of) a popular local repeater frequency over the internet. I got excited because of the potential for generating [quasi-useful, and at least interesting] data from the existing setup.  Since this guy already has his radio connected to his PC&#8217;s microphone jack, I figured I could write a Python app. to check the microphone input to determine if anyone is using the frequency.  By recording when people start and stop talking, I can create a log of frequency activity.  Later I can write software to visualize this data.  I&#8217;ll talk about that in a later post.  For now, here&#8217;s how I used Python and a Linux box (Ubuntu, with the python-alsaaudio package installed) to generate such logs.</p>
<p><object width="500" height="405"><param name="movie" value="http://www.youtube-nocookie.com/v/wnqsv03hu3U&#038;hl=en&#038;fs=1&#038;rel=0&#038;color1=0x006699&#038;color2=0x54abd6&#038;border=1"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube-nocookie.com/v/wnqsv03hu3U&#038;hl=en&#038;fs=1&#038;rel=0&#038;color1=0x006699&#038;color2=0x54abd6&#038;border=1" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="500" height="405"></embed></object></p>
<p><b>We can visualize this data</b> using some more simple Python code.  Long term it would be useful to visualize frequency activity similarly to <a href="http://www.swharden.com/blog/2009-05-20-graphing-computer-usage/" >how I graphed computer usage at work over the last year</a> but for now since I don&#8217;t have any large amount of data to work with. I&#8217;ll just write cote to visualize a QSO (conversation) with respect to time.  It should be self-explanatory.  This data came from data points displayed in the video (provided at the end of this post too).<br />
<a href="http://www.SWHarden.com/blog/images/qsographpng.png" onclick="javascript:urchinTracker ('/outbound/article/www.SWHarden.com');"><img src="http://www.SWHarden.com/blog/images/qsographpng-525x141.png" alt="qsographpng" title="qsographpng" width="525" height="141" class="alignnone size-medium wp-image-1252" /></a></p>
<p><b>And, of course, the code I used to generate the log files (seen running in video above):</b>  Briefly, this program checks the microphone many times every second to determine if its state has changed (talking/no talking) and records this data in a text file (which it updates every 10 seconds).  Matplotlib can EASILY be used to graph data from such a text file.</p>
<pre class="prettyprint python">
##################################
##### UPDATED CODE IS IN A NEWER ENTRY #####
### CHECK THE PYTHON AND RADIO CATEGORIES ###
##################################
import alsaaudio, time, audioop, datetime
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE,alsaaudio.PCM_NONBLOCK)
inp.setchannels(1)
inp.setrate(4000)
inp.setformat(alsaaudio.PCM_FORMAT_S16_LE)
inp.setperiodsize(1)

squelch = False
lastLog = 0
dataToLog = ""

def logIt(nowSquelch):
	global dataToLog, lastLog
	timeNow = datetime.datetime.now()
	epoch = time.mktime(timeNow.timetuple())
	if nowSquelch==True: nowSquelch=1
	else: nowSquelch=0
	logLine="%s %d\n"%(timeNow, nowSquelch)
	print timeNow, nowSquelch
	dataToLog+=logLine
	if epoch-lastLog>10:
		#print "LOGGING..."
		f=open('squelch.txt','a')
		f.write(dataToLog)
		f.close()
		lastLog = epoch
		dataToLog=""

while True:
	l,data = inp.read()
	if l:
		vol = audioop.max(data,2)
		#print vol #USED FOR CALIBRATION
		if vol>800: nowSquelch = True
		else: nowSquelch = False
		if not nowSquelch == squelch:
			logIt(nowSquelch)
			squelch = nowSquelch
	time.sleep(.01)
</pre>
<p><b>To use this code</b> make sure that you&#8217;ve properly calibrated it.  See the &#8220;vol>800&#8243; line?  That means that if the volume in the microphone is at least 800, it&#8217;s counted as talking, and less than it&#8217;s silence.  Hopefully you can find a value that counts as silence when the squelch is active, but as talking when the squelch is broken (even if there&#8217;s silence).  This is probably best achieved with the radio outputting at maximum volume.  You&#8217;ll have to run the program live with that line un-commented to view the data values live.  Find which values occur for squelch on/off, and pick your threshold accordingly.</p>
<p><b>After you run the code, it should output data like this:</b></p>
<pre class="prettyprint pythin">
##################################
##### UPDATED CODE IS IN A NEWER ENTRY #####
### CHECK THE PYTHON AND RADIO CATEGORIES ###
##################################
2009-06-12 17:13:28.347551 1
2009-06-12 17:13:29.306556 0
2009-06-12 17:13:30.176076 1
2009-06-12 17:13:41.533238 0
2009-06-12 17:13:45.435846 1
2009-06-12 17:14:21.890423 0
2009-06-12 17:14:25.605424 1
2009-06-12 17:14:41.904548 0
2009-06-12 17:14:43.903870 1
2009-06-12 17:14:55.981045 0
2009-06-12 17:14:59.377337 1
2009-06-12 17:15:12.365393 0
2009-06-12 17:15:15.773681 1
2009-06-12 17:15:33.165695 0
2009-06-12 17:15:36.155376 1
2009-06-12 17:16:04.181281 0
2009-06-12 17:16:07.423343 1
2009-06-12 17:16:15.724043 0
2009-06-12 17:16:18.411122 1
2009-06-12 17:16:35.304314 0
2009-06-12 17:16:36.371054 1
2009-06-12 17:16:50.818980 0
2009-06-12 17:16:53.995472 1
2009-06-12 17:17:26.675227 0
2009-06-12 17:17:29.969474 1
2009-06-12 17:17:47.907444 0
2009-06-12 17:17:50.361119 1
2009-06-12 17:18:04.630950 0
2009-06-12 17:18:05.565729 1
2009-06-12 17:18:11.225585 0
2009-06-12 17:18:14.787122 1
2009-06-12 17:18:41.261046 0
2009-06-12 17:18:43.819297 1
2009-06-12 17:18:52.164121 0
2009-06-12 17:18:53.100013 1
2009-06-12 17:19:05.348166 0
2009-06-12 17:19:09.316964 1
2009-06-12 17:19:19.107860 0
2009-06-12 17:19:19.941314 1
2009-06-12 17:19:26.254173 0
2009-06-12 17:19:33.844900 1
2009-06-12 17:19:46.501087 0
2009-06-12 17:19:49.360008 1
2009-06-12 17:19:59.917436 0
2009-06-12 17:20:04.335205 1
2009-06-12 17:20:17.622417 0
2009-06-12 17:20:18.501208 1
2009-06-12 17:20:26.866216 0
2009-06-12 17:20:28.694763 1
2009-06-12 17:20:36.097119 0
</pre>
<p><b>After that you can visualize</b> the data with the following code.  Note that this is SEVERELY LIMITED and is only useful when graphing a few minutes of data.  I don&#8217;t have hours/days of data to work with right now, so I won&#8217;t bother writing code to graph it.  This code produced the graph seen earlier in this page. Make sure matplotlib is installed on your box.</p>
<pre class="prettyprint">
##################################
##### UPDATED CODE IS IN A NEWER ENTRY #####
### CHECK THE PYTHON AND RADIO CATEGORIES ###
##################################
import pylab

def loadData():
	#returns Xs
	import time, datetime, pylab
	f=open('good.txt')
	raw=f.readlines()
	f.close()
	onTimes=[]
	timeStart=None
	lastOn=False
	for line in raw:
		if len(line)<10: continue
		line = line.strip('\n').split(" ")
		t=line[0]+" "+line[1]
		t=t.split('.')
		thisDay=time.strptime(t[0], "%Y-%m-%d %H:%M:%S")
		e=time.mktime(thisDay)+float("."+t[1])
		if timeStart==None: timeStart=e
		if line[-1]==1: stat=True
		else: stat=False
		if not lastOn and line[-1]=="1":
			lastOn=e
		else:
			onTimes.append([(lastOn-timeStart)/60.0,\
							(e-timeStart)/60.0])
			lastOn=False
	return onTimes

times = loadData()
pylab.figure(figsize=(8,3))
for t in times:
	pylab.fill([t[0],t[0],t[1],t[1]],[0,1,1,0],'k',lw=0,alpha=.5)
pylab.axis([None,None,-3,4])
pylab.title("A little QSO")
pylab.xlabel("Time (minutes)")
pylab.show()
</pre>
]]></content:encoded>
			<wfw:commentRss>http://www.SWHarden.com/blog/2009-06-12-python-powered-frequency-activity-logger/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Fixing Slow Internet in Ubuntu</title>
		<link>http://www.SWHarden.com/blog/2008-11-27-fixing-slow-internet-in-ubuntu/</link>
		<comments>http://www.SWHarden.com/blog/2008-11-27-fixing-slow-internet-in-ubuntu/#comments</comments>
		<pubDate>Thu, 27 Nov 2008 05:22:18 +0000</pubDate>
		<dc:creator>Scott</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Linux]]></category>

		<guid isPermaLink="false">http://www.SWHarden.com/blog/?p=520</guid>
		<description><![CDATA[ I recently swapped my two main PCs in my house.  The &#8220;headless&#8221; (no monitor) media PC (whose job consists of downloading, storing, and playing movies) connected directly to my TV, and our standard desktop PC which my wife uses most of the time.  I decided to do the swap because the media [...]]]></description>
			<content:encoded><![CDATA[<p> <b>I recently swapped my two main PCs in my house.</b>  The &#8220;headless&#8221; (no monitor) media PC (whose job consists of downloading, storing, and playing movies) connected directly to my TV, and our standard desktop PC which my wife uses most of the time.  I decided to do the swap because the media PC was way nicer than our desktop PC, and since the media PC is just playing movies and downloading torrents, I figured the extra processing power / ram / video acceleration could be put to better use.  Anyhow, I decided (in both cases) to completely start fresh by wiping hard drives clean and reinstalling Ubuntu linux (I&#8217;m using 8.10 currently).  However, after the installation I noticed a peculiar problem.  I&#8217;ll quote it to emphasize it&#8230;  </p>
<p> <table><tr><td style="text-indent: 25px; background-color: #E5E5E5; padding: 10px; border-top-width: 1px; border-bottom-width: 1px; border-left-width: 7px;border-top-style: solid; border-right-style: solid;border-bottom-style: solid;border-left-style: solid;border-top-color: #B5B5B5; border-right-color: #B5B5B5;border-bottom-color: #B5B5B5; border-left-color: #B5B5B5;border-right-width: 1px;background-image: url(http://www.swharden.com/graphics/layout_2006_08_12/quotes.jpg); background-position: left top; background-repeat: no-repeat;">Browsing the internet was very slow.  When I&#8217;d click a link on a website, it would take several seconds before it seemed to even try to go to the next page.  The same thing would happen if I manually typed-in a new website.  I tried disabling IPv6 in firefox&#8217;s about:config and in the /etc/init.d/aliases file, but it didn&#8217;t help!</td></tr></table>  </p>
<p> <b>The solution for me was simple, and since</b> I spent a lot of time searching forums I know I&#8217;m not the only one with this problem.  Disabling IPv6 was suggested in 99% of similar posts.  My solution took a while to uncover, so I figured I&#8217;d write it here.  The basic problem is that my DHCP (auto-configured IP address) settings were screwed up, and my manually setting them I fixed the problem.  Here&#8217;s what I did&#8230;  </p>
<p>
 <img src="http://www.SWHarden.com/blog/images/dnsfix1.png" alt="" title="dnsfix1" width="244" height="197" class="aligncenter size-full" />  </p>
<p> Start by right-clicking your network icon (wireless in my case) and selecting <b>connection information</b>  </p>
<p>
 <img src="http://www.SWHarden.com/blog/images/dnsfix4.png" alt="" title="dnsfix4" width="445" height="484" class="aligncenter size-full wp-image-522" />  </p>
<p> Check out your current configuration.  <b>Is a local address (192.168.*.*) set for the primary DNS server?</b>  If so, that&#8217;s your problem!  Note your secondary server.  We&#8217;ll set it as your primary&#8230;  </p>
<p>
 <img src="http://www.SWHarden.com/blog/images/dnsfix1.png" alt="" title="dnsfix1" width="244" height="197" class="aligncenter size-full" />  </p>
<p> Continue by right-clicking your network icon (wireless in my case) and selecting <b>edit connections*.  Open the tab corresponding to your internet connection (wired or wireless &#8211; wireless in my case), select your connection, and click </b><b>Edit</b>  </p>
<p>
 <img src="http://www.SWHarden.com/blog/images/dnsfix3.png" alt="" title="dnsfix3" width="468" height="623" class="aligncenter size-full wp-image-523" />  </p>
<p> Use this screen to <b>manually enter the information from the information screen you saw earlier, but making sure not to list any local IP addresses as the DNS servers.</b>  Save your settings, close the windows, and the problem should be immediately corrected.  Leave &#8220;search domains&#8221; blank, that&#8217;s important too. Good luck!!!</p>
]]></content:encoded>
			<wfw:commentRss>http://www.SWHarden.com/blog/2008-11-27-fixing-slow-internet-in-ubuntu/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Run Ubuntu Live CD From a USB Drive</title>
		<link>http://www.SWHarden.com/blog/2008-11-22-run-ubuntu-live-cd-from-a-usb-drive/</link>
		<comments>http://www.SWHarden.com/blog/2008-11-22-run-ubuntu-live-cd-from-a-usb-drive/#comments</comments>
		<pubDate>Sat, 22 Nov 2008 20:36:16 +0000</pubDate>
		<dc:creator>Scott</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Linux]]></category>

		<guid isPermaLink="false">http://www.SWHarden.com/blog/?p=506</guid>
		<description><![CDATA[ I accidentally nuked my laptop&#8217;s 80G hard drive this morning (D&#8217;OH!) while shuffling around partitions.  Supposedly there&#8217;s a valid windows (XP) installation on there still that&#8217;s about 20G.  I&#8217;d love to repair it so I can use it today while I&#8217;m in the confocal room, but I don&#8217;t have an Ubuntu CD, [...]]]></description>
			<content:encoded><![CDATA[<p> <b>I accidentally nuked my laptop&#8217;s 80G hard drive this morning</b> (D&#8217;OH!) while shuffling around partitions.  Supposedly there&#8217;s a valid windows (XP) installation on there still that&#8217;s about 20G.  I&#8217;d love to repair it so I can use it today while I&#8217;m in the confocal room, but I don&#8217;t have an Ubuntu CD, Windows CD, or any CD for that matter!  I looked around, but I guess blank CD-Rs aren&#8217;t something that&#8217;s standard in molecular biology laboratories.  Anyhow, I wanted to install the new Ubuntu 8.10 Linux distribution, and I&#8217;ve downloaded the ISO, but since I can&#8217;t find a CD to burn it to I decided to try booting from a USB drive (something I&#8217;ve never done before).  I found an <i>AWESOME</i> program which specialized in putting ISO files onto bootable USB drives.  It&#8217;s called <a href="http://unetbootin.sourceforge.net/" onclick="javascript:urchinTracker ('/outbound/article/unetbootin.sourceforge.net');">UNetBootin</a> and it&#8217;s free (of course), runs on Linux or Windows,  and has some built-in options for various linux distributions.  I can repair my PC now!  Yay!  <br />
 <img src="http://www.SWHarden.com/blog/images/unetbootin.jpg" alt="" title="unetbootin"  align="center" /></p>
]]></content:encoded>
			<wfw:commentRss>http://www.SWHarden.com/blog/2008-11-22-run-ubuntu-live-cd-from-a-usb-drive/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
