# Realtime FFT Audio Visualization with Python

I’m no stranger to visualizing linear data in the frequency-domain. Between the high definition spectrograph suite I wrote in my first year of dental school (QRSS-VD, which differentiates tones to sub-Hz resolution), to the various scripts over the years (which go into FFT imaginary number theory, linear data signal filtering with python, and real time audio graphing with wckgraph), I’ve tried dozens of combinations of techniques to capture data, analyze it, and display it with Python. Because I’m now branching into making microcontroller devices which measure and transfer analog data to a computer, I need a way to rapidly visualize data obtained in Python. Since my microcontroller device isn’t up and running yet, linear data from a PC microphone will have to do.  Here’s a quick and dirty start-to-finish project anyone can tease apart to figure out how to do some of these not-so-intuitive processes in Python. To my knowledge, this is a cross-platform solution too. For the sound card interaction, it relies on the cross-platform sound card interface library PyAudio. My python distro is 2.7 (python xy), but pythonxy doesn’t [yet] supply PyAudio.

The code behind it is a little jumbled, but it works. For recording, I wrote a class “SwhRecorder” which uses threading to continuously record audio and save it as a numpy array. When the class is loaded and started, your GUI can wait until it sees newAudio become True, then it can grab audio directly, or use fft() to pull the spectral component (which is what I do in the video). Note that my fft() relies on numpy.fft.fft(). The return is a nearly-symmetrical mirror image of the frequency components, which (get ready to cringe mathematicians) I simply split into two arrays, reverse one of them, and add together. To turn this absolute value into dB, I’d take the log10(fft) and multiply it by 20. You know, if you’re into that kind of thing, you should really check out a post I made about FFT theory and analyzing audio data in python.

Here’s the meat of the code. To run it, you should really grab the zip file at the bottom of the page. I’ll start with the recorder class:

import matplotlib
matplotlib.use('TkAgg') # THIS MAKES IT FAST!
import numpy
import scipy
import struct
import pyaudio
import pylab
import struct

class SwhRecorder:
"""Simple, cross-platform class to record from the microphone."""

def __init__(self):
"""minimal garb is executed when class is loaded."""
self.RATE=48100
self.BUFFERSIZE=2**12 #1024 is a good buffer size
self.secToRecord=.1
self.newAudio=False

def setup(self):
"""initialize sound card."""
#TODO - windows detection vs. alsa or something for linux
#TODO - try/except for sound card selection/initiation

self.buffersToRecord=int(self.RATE*self.secToRecord/self.BUFFERSIZE)
if self.buffersToRecord==0: self.buffersToRecord=1
self.samplesToRecord=int(self.BUFFERSIZE*self.buffersToRecord)
self.chunksToRecord=int(self.samplesToRecord/self.BUFFERSIZE)
self.secPerPoint=1.0/self.RATE

self.p = pyaudio.PyAudio()
self.inStream = self.p.open(format=pyaudio.paInt16,channels=1,
rate=self.RATE,input=True,frames_per_buffer=self.BUFFERSIZE)
self.xsBuffer=numpy.arange(self.BUFFERSIZE)*self.secPerPoint
self.xs=numpy.arange(self.chunksToRecord*self.BUFFERSIZE)*self.secPerPoint
self.audio=numpy.empty((self.chunksToRecord*self.BUFFERSIZE),dtype=numpy.int16)

def close(self):
"""cleanly back out and release sound card."""
self.p.close(self.inStream)

### RECORDING AUDIO ###

def getAudio(self):
"""get a single buffer size worth of audio."""
return numpy.fromstring(audioString,dtype=numpy.int16)

def record(self,forever=True):
"""record secToRecord seconds of audio."""
while True:
for i in range(self.chunksToRecord):
self.audio[i*self.BUFFERSIZE:(i+1)*self.BUFFERSIZE]=self.getAudio()
self.newAudio=True
if forever==False: break

def continuousStart(self):
"""CALL THIS to start running forever."""
self.t.start()

def continuousEnd(self):
"""shut down continuous recording."""

### MATH ###

def downsample(self,data,mult):
"""Given 1D data, return the binned average."""
overhang=len(data)%mult
if overhang: data=data[:-overhang]
data=numpy.reshape(data,(len(data)/mult,mult))
data=numpy.average(data,1)
return data

def fft(self,data=None,trimBy=10,logScale=False,divBy=100):
if data==None:
data=self.audio.flatten()
left,right=numpy.split(numpy.abs(numpy.fft.fft(data)),2)
if logScale:
ys=numpy.multiply(20,numpy.log10(ys))
xs=numpy.arange(self.BUFFERSIZE/2,dtype=float)
if trimBy:
i=int((self.BUFFERSIZE/2)/trimBy)
ys=ys[:i]
xs=xs[:i]*self.RATE/self.BUFFERSIZE
if divBy:
ys=ys/float(divBy)
return xs,ys

### VISUALIZATION ###

def plotAudio(self):
"""open a matplotlib popup window showing audio data."""
pylab.plot(self.audio.flatten())
pylab.show()

And now here’s the GUI launcher:

import ui_plot
import sys
import numpy
from PyQt4 import QtCore, QtGui
import PyQt4.Qwt5 as Qwt
from recorder import *

def plotSomething():
if SR.newAudio==False:
return
xs,ys=SR.fft()
c.setData(xs,ys)
uiplot.qwtPlot.replot()
SR.newAudio=False

if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)

win_plot = ui_plot.QtGui.QMainWindow()
uiplot = ui_plot.Ui_win_plot()
uiplot.setupUi(win_plot)
uiplot.btnA.clicked.connect(plotSomething)
#uiplot.btnB.clicked.connect(lambda: uiplot.timer.setInterval(100.0))
#uiplot.btnC.clicked.connect(lambda: uiplot.timer.setInterval(10.0))
#uiplot.btnD.clicked.connect(lambda: uiplot.timer.setInterval(1.0))
c=Qwt.QwtPlotCurve()
c.attach(uiplot.qwtPlot)

uiplot.qwtPlot.setAxisScale(uiplot.qwtPlot.yLeft, 0, 1000)

uiplot.timer = QtCore.QTimer()
uiplot.timer.start(1.0)

win_plot.connect(uiplot.timer, QtCore.SIGNAL('timeout()'), plotSomething)

SR=SwhRecorder()
SR.setup()
SR.continuousStart()

### DISPLAY WINDOWS
win_plot.show()
code=app.exec_()
SR.close()
sys.exit(code)

Note that by commenting-out the FFT line and using “c.setData(SR.xs,SR.audio)” you can plot linear PCM data to visualize sound waves like this:

Finally, here’s the zip file. It contains everything you need to run the program on your own computer (including the UI scripts which are not written on this page)

If you make a cool project based on this one, I’d love to hear about it. Good luck!

Scott W Harden

Scott Harden has had a lifelong passion for computer programming and electrical engineering, and recently has become interested in its relationship with biomolecular sciences. He has run a personal website since he was 15, which has changed names from HardenTechnologies.com, to KnightHacker.com, to ScottIsHot.com, to its current SWHarden.com. Scott has been in college for 10 years, with 3 more years to go. He has an AA in Biology (Valencia College), BS in Cell Biology (Union University), MS in Molecular Biology and Microbiology (University of Central Florida), and is currently in a combined DMD (doctor of dental medicine) / PhD (neuroscience) program through the collaboration of the College of Dentistry and College of Medicine (Interdisciplinary Program in Biomedical Science, IDP) at the University of Florida in Gainesville, Florida. In his spare time Scott builds small electrical devices (with an emphasis on radio frequency) and enjoys writing cross-platform open-source software.

1. ##### Mike Seese

Cheers Scott! I was about to start programming something in C++ or Java, did a Google search, and you were on top! Curious as to what your big picture is here. I’m using it for some theories behind voice recognition.

2. ##### r3c4ll

hi Scott, i try to run the realTimeAudio.py but obtain an error:

Traceback (most recent call last):
File “realTimeAudio.py”, line 1, in
import ui_plot
File “/home/r3c4ll/source/python/SWHRecorder/ui_plot.py”, line 56, in
from PyQt4 import Qwt5
ImportError: cannot import name Qwt5

1. ##### Anonymous

Try apt-get install python-qwt5-qt4

1. ##### Anonymous

For the folks running into this same error on a Mac: try brew install pyqwt

1. ##### Moy

Hi, I tried to run the same file in windows 7 and I got the same error
File “realTimeAudio.py”, line 1, in
import ui_plot
File “/home/r3c4ll/source/python/SWHRecorder/ui_plot.py”, line 56, in
from PyQt4 import Qwt5
ImportError: cannot import name Qwt5

3. ##### Hmm

“I simply split into two arrays, reverse one of them, and add together.”

Wait, what? If the halves are identical, this does nothing but double your numbers. Why do that?

If your signal is real, use numpy.fft.rfft, which is faster, too.

4. ##### Diego

Hi Scott, your work is amazing!! Do you know hao to do the same but with an external sound card pluged it to a USB port??
Thank you!

5. ##### Jake

Hello Scott,

I want to truly thank you for this post — It’s almost exactly what I’ve wanted to achieve (and read about) for a long time now, but my mathematical abilities are not up to par enough to tackle it myself. This code and post have been very informative, and will be very useful probably to many, definitely to myself, so thanks again.

6. ##### Richard

hear is a tip use numpy.fft.rfft to compute the fft on real component data. It will give you n/2+1 complex variables for an even sized real array, though the two end components will be completely real as well.

7. ##### Anonymous

I think this is better:

ys = numpy.abs(numpy.fft.rfft(data))

8. ##### Chunwei

hi friends:
your this essay is very helpful,but i have one question about it, i want use your recorder file display by Tkinter, try two days alredy.
i am quite new about python, can you just sent me some example code about that?
display the curve by tkinter :)))
think you very much

9. ##### dktak

Hi Scott,

It looks very interesting. But as Diego’s question, how would you use a USB external sound-card ?? I think it is somewhere around the “TODO” area, you would set some connection-informations.

10. ##### reinhard fuchs

Hi Scott,
Do you know hao to do the same but with an external sound card pluged it to a USB port??
I have this same question as DIEGO.
If ,could i get the answer ?
Mit freundlichem Gruss Reinhard

11. ##### dktak

Got it to work with an external sound card. Have played around with AlsaMixer, and after figuring out, that I had to launch realTimeAudio.py it worked.

1. ##### born2freedom

dktak

Can you give some details of how you got it working with the USB soundcard?

I am trying to modify the code to use on RaspberryPi and Beaglebone black.using Debian wheezey

1. ##### matt

Hi born2freedom,

have you firgured out how to run the program on a Raspberry Pi?
I’m sitting here trying the same…

Thanks
matt

12. ##### Wael

well done, Scott

13. ##### Matt

Hello,

Great program Scott. Just wondering if anybody as managed to get the program to work with a USB soundcard.
If so how??

14. ##### hystrix

I would love to get this working with a Raspberry Pi and the Wolfson Pi Audio Card. Does anyone know how I would do that?

I get the following, plus other errors ending in Invalid Sample Rate….

ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:957:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
Cannot connect to server socket err = No such file or directory

Yes….you’ve guessed it – I don’t really know what I’m doing :-)

15. ##### Sheung Lu

I got this to work on a raspi, I’m interested in using the gpu to do the FFTs as it should be faster (http://www.raspberrypi.org/accelerating-fourier-transforms-using-the-gpu/). Any idea on how to get this to work with your python code?

16. ##### erick

Hi Scott, nice code.
When a execute the script (python realTimeAudio.py), I get numbers on my terminal (ubuntu) that respond to noise levels, but until now i can’t get the plot.
How should I call the script from terminal?

I’m trying to use this code to made a heart recording (using a modified mic) for teaching purposes.

Sorry for my english and thanks.

17. ##### Anonymous

Hi Scott,

I made your code work in a mac running 10.9 , and I find it a very nice source of getting to know for learning advanced python skill. I tested it with the phone idle-tone, it peaks at around 400-450 Hz, which is fits nicely to Germany’s 425Hz idle tone frequency. It would be nice to adjust the y-axis dimension, couldn’t really find anything hard-coded so that I could play around with it, however I found a work around by decreasing the mic sensitivity…

cheers

18. ##### Anonymous

ok I found it :)
line 31: uiplot.qwtPlot.setAxisScale(uiplot.qwtPlot.yLeft, 0, 1000)

19. ##### Austin

Hey Scott,

Just wanted to post an interesting project I did based on your project. It is a basic music visualizer that warps an image in a spiral.

In my program I use the data from the fft of the audio signal to calculate the strength of the spiral effect.

My repo can be found here.

Thanks for posting this code and explanation!

- Austin