Arduino Projects: Digital Voice Recorder Part 2

Last month, we introduced our Digital Audio Recorder by looking at the basics of timer interrupts and how the Arduino Uno’s ATMEGA328P controller can be programmed to create precise timing intervals ideal for audio sampling. We also looked at the ‘double buffering’ technique that allows us to simultaneously record samples and save data to the flash card every 45microseconds.

Well, a month is a long time in tech and we’ve made some major changes and improvements – including boosting the sample rate maximum from 22.05kHz to as high as 48kHz (reliant on the microSD card). We’ll explain how, but first, we’ll look at the FAT filesystem and how we implemented Microsoft’s WAV file structure to enable the recorded file to play almost anywhere.

FAT filesystem

Like many audio file formats, WAV has a structure designed so that when the file is loaded into any suitable media player, it knows how to play it. Otherwise, the file is just random data. The key is the initial 44-byte data block known as the ‘WAV file header’, which identifies the file and holds important information on the sample size, bit depth, number of channels and so on. This is what our recorder must create as it begins recording.

Last month, we looked at the microSD card interface and basic flash card storage alignment, but before we get to creating the WAV header, we first have to understand how data is arranged on and written to the card.

wavheaderEvery storage device needs a file system, a way of knowing what’s stored where and arguably the most common is Microsoft’s File Allocation Table filesystem or FAT. Arduino supports file storage with the SD library using FAT16 or FAT32 structures, but to improve performance, we’re using the new-and-improved sdfatlib library.

There are two standard ways to write data to storage – you can write it as ASCII text, where each character requires one byte of storage; or you can write it in more efficient binary form. To write a WAV file, we need this second form, but before we can get started, there are other limitations we need to get around.

If we look back at the WAV file header, it contains small memory blocks or ‘chunks’, with each one defining a parameter of the audio inside, whether it’s the sample rate, file size, bit depth or so on. Some of those blocks are two-bytes wide, most are four-bytes. The problem we have with the sdfatlib library is that while it supports both ASCII and binary writing, it can only write in binary format one byte per command, so before we can write these WAVE blocks as datatypes, we have to convert them to bytes and write them to the card one byte at a time.

WAV file format

So let’s go back to the WAV file format and that 44-byte header. The wave header is written to the card in the writeWavHeader() method with the basic structure you can see in the block diagram. Here it is in detail, starting with the file ID:

  • Offset 0 (4 bytes length) – ChunkID. This contains the four ASCII characters ‘RIFF’ (Resource Interchange File Format) and identifies the file.
  • Offset 4 (4) – FileSize. The overall size of the file, the four bytes (32-bits) gives us a 4GB maximum filesize (written after the recording is completed).
  • Offset 8 (4) – Format. The four ASCII characters ‘WAVE’ indicate standard WAV format.

The next 24-bytes tell us specific parameters of the audio:

  • Offset 12 (4) – SubID. This contains the characters ‘fmt ‘ (including the space).
  • Offset 16 (4) – SubSize. Sets the size of the data chunk to 16-bytes for PCM wave format.
  • Offset 20 (2) – AudioFormat. Although we use four bytes here, this is set to 0x0001 indicating non-compressed PCM/WAV audio follows.
  • Offset 22 (2) – Channels. The number of audio channels (in our case, one).
  • Offset 24 (4) – SampleRate. The audio sample rate, written as binary.
  • Offset 28 (4) – DataRate. Sets how fast data needs to be streamed and is calculated by the equation:
    • DataRate = samplerate x channels x (bitspersample / 8)
  • Offset 32 (2) – BlockAlign. Indicates how the audio data is aligned, so that the player knows which bytes belong to which channel.
  • Offset 34 (2) – SampleBits. The number of bits in each sample (we set this to ‘8’).

The last 8-bytes set up the rest of the file:

  • Offset 36 (4) – SubID2. This contains the ASCII characters ‘data’ indicating the audio data begins here.
  • Offset 40 (4) – Sub2Size. The size of the audio data following, set by the equation: Size = no. of samples x no. of channels x no. of bytes per sample.

Again, this is written after recording is completed.

Having to write these datatypes a byte at a time means the order of each byte stored (called the ‘byte order’) is critical; otherwise, a datatype of ‘1’ may end up becoming ‘16777216’ if we’re not careful. Another thing we must do is keep track of the number of samples we capture, so this info can be written back to the file header after the capture is completed.

cardThere’s a lot to do so here’s how it’s done – first, the only header info we don’t know is the filesize and data chunk size, so we start the recording process by opening up the file ‘REC00000.WAV’, write the file header info we do know and dummy-fill the spots we don’t. Once that’s done, we start capturing the audio and storing it away. Since each sample is one byte (8 bits), the number of samples we capture is also the number of bytes we store. We keep track of this number in a long-integer variable called ‘byteWriteCount’, which is incremented every sample.

When we stop capturing, we use the sdfatlib’s ‘seekSet’ command to go back to the file header, write in the filesize and datasize information and finally, close the file.

After all that is done, the file on the card is now a genuine WAV file playable on any compatible device, including phones, tablets and Windows Media Player on your PC or notebook.

Controlling the recorder

To keep things really simple, our digital audio recorder has just two buttons – Record and Stop. The Arduino continually monitors two of its digital input lines (D5 and D6) and when the Record button (D6) is pressed, the recording function begins. When the Stop button (D5) is pressed, we deactivate the timer interrupt that launches the interrupt service routine (ISR) capturing the samples, write out the information to the WAV file header and close the file.

To give you some basic indication of what the recorder is doing, two LEDs light up, one for ‘recording’ and the other for ‘stopped’. Any problem with the card on boot results in the LEDs flashing.

Circuit design

We’ve used the Arduino DIY prototyping shield to house the microSD card module, buttons, LEDs and audio input circuitry. It’s a tight squeeze but keeps things compact.

Again to simplify things, the audio input circuit just contains a resistor divider to bias the ADC input to half the supply rail – this ensures that we accurately capture the positive (samples 128 to 255) and negative (samples 128 to 0) halves of the audio waveform. You’ll need to feed it with a high-level audio input from your PC’s line output level or phone/tablet’s headphone socket.

We’ve used a 3.5mm panel socket here and a 3.5mm male-to-male cable will connect those devices to the recorder.

Modifying the buffer

loggerThe microSD flash card reader module is a new release you’ll find on eBay for around $5 – it supports any SDHC card up to 32GB and contains built-in level translation to convert the 5V signals from the Arduino to 3.3V level for the flash card.

From last month, you’ll remember that we’re forced to use the microSD card’s one-bit SPI (serial peripheral interface) bus rather than the faster four-bit parallel mode, simply because that’s all the Arduino Uno supports. We implemented this alongside a double 512-byte buffering system that records samples into one buffer while the other buffer is written to the card.

However, we found this month that we were temporarily losing samples after 40 seconds or so of recording. The reason wasn’t the recording mechanism itself, but the microSD card’s SPI bus speed and how data is written to the flash storage.

With a 22.05kHz sample rate, the two 512-byte buffer blocks each provide 512/22050 or 23.2milliseconds of audio storage. We benchmarked the write time for 512-byte blocks to the card and on average, it took just three milliseconds (3ms), so there was no problem. Or so we thought.

However, on occasions, we discovered it was taking the microSD card reader as long as 65milliseconds to write the same-sized block – three times as long as the buffer itself can handle and that’s where the samples were being lost.

But we also found that if we dropped the buffer size down to 128 bytes, the average write time dropped to just 0.5milliseconds, with some as fast as 88microseconds. Unfortunately, the occasional write still took up to 65milliseconds to complete, but there were far fewer of these.

So the solution was to redesign the two 512-byte buffer blocks into an eight-block ring buffer that still gave us the maximum buffer size but with improved write speeds.

Setting the sample rate

circuitNot only that, the faster performance means we’ve been able to incorporate an adjustable sample rate, up from the original locked 22.05kHz to as high as you can get away with depending on your microSD card (48kHz maximum). You set it in the source code (it’s pretty obvious where) before you flash your Arduino Uno board with it.

Because of variations in microSD cards, the maximum sample rate you can achieve depends more on the card than on the ATMEGA328P controller. We’ve tested it and captured sample-stable recordings up to 48kHz.

But here’s the key – if the card’s SPI bus write speed is too slow, you’ll hear ‘skips’ in your recording where samples haven’t been written to the card in time. Unfortunately, we can’t tell you which cards are faster since it depends on the card’s controller and flash write speed. You have to ‘suck it and see’.

However, we’ve found in general,  that smaller-capacity cards formatted with a larger ‘sector’ or allocation unit size (AUS) seem to offer faster performance in SPI mode. We tested this with 4GB and 8GB cards formatted with 1KB and 32KB AUSs – the 32KB AUS-formatted option delivered faster write speeds all-round. Formatting the card just before use can also make a difference.

recorderWhat’s it sound like?

Given how few resources we have, and now with higher sample rates, the audio quality is surprisingly good! With only 8-bit sample depth, the signal-to-noise ratio (SNR) is roughly around 40dB (cassette tape), so there’s a bit of background hiss, but if you record audio that has serious dynamic range compression, you don’t notice it (AC/DC’s Back in Black, for example, sounds great).

Sure, it’s not CD-quality, but the techniques here are just the same – and that still makes this a seriously good way to learn how digital audio works, from analog sampling through to file storage.

Limitations

bufferReally, the only reason we run into any issues at all is simply the ATMEGA328P’s lack of RAM. The chip is fast enough – it just doesn’t have enough RAM to compensate for the occasional few slow card block-writes, but there are ways to solve this.

One simple fix is to reduce the sample rate, albeit at the cost of audio quality. A more complex lossless option is adding external RAM, such as a 23K640 64KB SPI RAM chip. We could also swap the Arduino Uno for the up-rated Arduino Mega2560 with its 8KB of RAM.

Other ideas

Obviously we are right on the edge of the ATMEGA328P’s capabilities here, but slow that sample rate right down and there’s an application that’s right in the Arduino’s wheelhouse – data logging. That data can be anything from weather measurements to a digital voltmeter or even a security system. The ATMEGA328P’s 10-bit ADC range means you can capture analog data down to 0.1% of full scale, or you can just record changes on any of the digital inputs.

And that’s what makes Arduino so cool – it has so many toys to play with, there’s always a new idea just around the corner.

As usual, you’ll find the source code for our Digital Audio Recorder at our Arduino webpage. While it has been tested, the code comes ‘as is’ with no warranty of any kind.

  • Sai Yamanoor

    You mentioned providing inputs from line out of a PC. Can I plug in a microphone off the shelf for recording?

    • No, the signal output from a typical microphone is around 1mV (0.001 volts) – nowhere near enough to register on the analog-to-digital converter. You will need some form of preamplifier to get the microphone output up to a suitable level. (The line output from a PC is already around 1Vrms, which works nicely.)

      • Sai Yamanoor

        I used a Sparkfun Electret microphone board as well. It was nosiy. Should I rather put an electret microphone in a pre-amplifier circuit and use it? I am just trying to record some decent audio

        • Record it from what? The Sparkfun microphone uses an OPA344 opamp, which is supposed to deliver very decent SNR (according to the specs, THD+SNR < 0.0006%, which is very good).
          Bear in mind, too, that the ADC here in the Arduino at this sampling speed is only 8-bit (Yep, it's 10-bit normally, but only at low sample rates), so it should be similar to cassette tape in terms of noise levels. If it's noisier than that, you may have other issues to deal with. If you can, get your sound from a line-level output and you won't need the microphone or preamp.
          Cheers,
          Darren.

          • Sai Yamanoor

            Thanks. I was just using a breadboard connection between the sparkfun and the breakout. I was using a piece of wire between the pin A5 and the microphone output. I know this piece of wire can introduce some noise, could this render it in audible? I was not able to hear anything when I did this small test. Lets say I decided to design a PCB based on this concept. Are there considerations like where I have to place the microphone? E.g could lengthy PCB tracks induce noise?

          • Well, before you go that far, I’d just confirm that you can get it to record audio from the headphone socket of your PC/laptop/phone/tablet/whatever. I haven’t used a microphone module with it, so first, stick to the original project and make sure that’s working. If it’s not, there are other problems to fix before you give that mic module another go.

          • Sai Yamanoor

            Cool project. Never thought it would be possible to build a recorder on the arduino.. given the 2K RAM

          • Yeah, there are a few tricks involved in making it work, but the fact the ATMEGA328P has all the bits we need ready to go is really what makes this project possible (that and a good fast MicroSD card… 🙂

  • ogdento

    Hey Darren, just wanted to let you know the source code link for part 2 points to something for the attacknid

    • Okay… I’ll see what’s the trouble.
      Cheers,
      Darren.

      • A. P.

        The first version was very interesting, this one even more. However, the code is still badly linked, any chance on getting that fixed? I would love to check out the ring buffer implementation.

  • olexij

    Hey, the source code for part 2 is still missing, points to attacknid. Would you please provide it? Thanks!

  • ogdento

    bummer… figured i’d check back but the link is still broken. i’m on the edge of my seat!!

    • Click on my name here and go to my website, you can get the source code from there.
      Cheers,
      Darren.

  • ABC123

    Hi Darren, love what you’ve done here. Looks great! Having tried the link for the code on here and failed I followed your advice to visit your own personal website however that isn’t working either. Is there any other way of getting to see your code for V2? Many thanks in advance

    • Hey abc123,
      I’ve asked the APC team to have a look at the link. In the meantime, my website is back up (not sure why it went down), so you can try again there too.
      Cheers,
      Darren.

    • Give it another go now. I’ve been told the file on the APC site has now been updated.
      Cheers,
      Darren.

  • Young Sub Lee

    Hello, I am interested about your post. Because it is similar to my project!
    Some reasons, I have to use Arduino Mega. Do you think that your code and library will be worked well in Mega 2560?

    Young Sub Lee

    • Honestly, no idea. But can’t see why not. May need slight tweaking to cater for different pinouts, but the ATMEGA2560 is just a bigger brother to the ATMEGA328P, so it should.
      Cheers,
      Darren.

      • Young Sub Lee

        As you assume, when I used you code on Arduino MEGA2560, it worked pretty well. However, I combined another code which uses three headers (WiFi.h, SPI.h, SD.h). I think some parts of your code and another coed crushed at SD card parts. Do you have some ideas? I need your help. I can send another code as email. Thank you.

  • Bruce Lee

    Hello, Darren. I was really impressed what you did here. I have a question about your project. I found that your SdFat header is different from the original SdFat in SD header. Nowadays, I am trying to merge two arduino codes. But i guess your adapted SdFat header and the original SD header conflict each other. So, do you happen to know how to coexist your SdFat header with SD header? Thank you!

    • Hi Bruce,
      Yeah, there are different versions of the library floating around, but I think there is also some undocumented (or should I say ‘less-documented’) methods in this library. One of the best things I can suggest is to look at some of the examples for Sdfat you’ll find in the Arduino IDE (File / Examples / SD) and look at how the header is used. Try to understand what the code is doing and replicate it in your own code.
      Cheers,
      Darren.

  • Mateo San

    hi Darren,,i just used ur code on arduino mega,,,it does everything even creates the wav file in sd card but it doesnt save anythin in it!3 days in a row im changing pins n still nothing!do i have to define the spi for mega or what?can u help me out plz?

    • Hi Mateo,
      Yeah, the Mega is a different beast to the Uno, so I suspect the code, particularly around the ring buffer, will need some tweaking. I have a Mega and it’s on my to-do list, as I think it will actually work better than the Uno. The Mega has 8K of RAM instead of the Uno’s 2KB – that should give four times the ring buffer time and overcome the occasional slow block write time of the SD card.
      But first, I have to read the Mega datasheet from Atmel to figure out which way’s up! 🙂
      Cheers,
      Darren.

      • Mateo San

        i dont mind abt the buffer or the ram i just need to make it save somthing into the .wav file…my mind got numb so sorry if im askin somthin silly…im trying to figure out whts goin on here without effort..can u plz guide me on wht can i keep n wht not to?wht am i doin wrong? (note: im using an elecfreak 3.2″ tft shield and a 3.2″ tft module with sd card,,,i need only the sd card part so i dont think thts the problem)

        #include

        #include

        #include

        #include

        #define MEGA_SOFT_SPI 1

        #define USE_SOFTWARE_SPI 1

        #define MOSI 51

        #define MISO 50

        #define SCK 52

        //uint8_t const SOFT_SPI_MOSI_PIN = 51;

        //uint8_t const SOFT_SPI_MISO_PIN = 50;

        //uint8_t const SOFT_SPI_SCK_PIN = 52;

        //uint8_t const SOFT_SPI_CS_PIN = 53;

      • Prit Zaveri

        Hey,
        I’m using UNO but I’m facing the same problem. .wav file is created but nothing recorded on it.
        What possibly could be wrong ?
        Any suggestion ?
        Thanks

  • eleck

    hi Darren,
    what change should I made in the code to keep the recording audio for only two minutes loop…
    thanks in advance

    • Changes needed are too big to include here. If you’re wanting a continuous keep-the-last-two-minutes kind of thing, it’ll need something of a rewrite.
      No promises, but I’ll see what I can do for a future Arduino Masterclass story.
      Cheers,
      Darren.

      • jch

        If it helps, I’m quite interested in this “continuous keep-the-last-xx-minutes” functionality too. Would be great if it could be adjusted up to some feasible maximum time frame. I now see your reply is 2 months old, any progress to date?

  • leblanc

    Hi Darren,

    I’ve a question, can we make a sort of audio prothesis ? so with 1 arduino can we record a noise, filter it and send it to a speaker in real time ?

    thx for your job !

    Dorian

    • That’s a good question. I’m not going to say no, but it’s going to be tough. You’d have to drop the sample rate to have enough clock cycles left over to do any sort of digital filtering (I’d suggest reading up on Fast Hartley Transforms). With two Arduinos, you could do it – one is going to be tough. You have to remember, you’re dealing with only 8-bit audio here as well – that’s fine for rock music, but for anything with considerable dynamic range (> 40dB), you’re not going to win here.
      Of course, if you’re thinking an Arduino Due, your prospects of success improve considerably.
      Cheers,
      Darren.

  • eleck

    my stored audio file’s date modified date is always 2000-01-01 0:00

    how to bring current time in wave file?..please help me

  • Jason Ruthermore

    Thoughtful analysis . Coincidentally , if anyone has been searching for a a form , We used a sample document here http://goo.gl/3o1qVq

  • Cristhian Macoh Musada

    Hi Darren

    Very good project, and implementing audio recording on arduino without using any other component aside from the microphone and its amplifier.

    I also wanted to test it out, but it seems that the code only reaches up to StartRec function because when I would view the file on my PC I can only see 44 bytes of file size.

    Is there something I’m doing wrong? Thank You

    • Cristhian Macoh Musada

      Any got it working, on my connections I already used an pull up resistor, but forgot to change the pinMode on the code to only INPUT

      So Im able to record now, but Im hearing a unwanted noise of something switching, I can here my voice or sound but the noise is just to load to make the recording clear. Could this be a ground loop if yes how can I try remove this. Thank You