Large format vinyl cutter at BuildBrighton Foison C24

Last night was BuildBrighton night (in fact, almost every night is a BuildBrighton night these days, but Thursdays are the "traditional" come-along-and-hang-out night) and as is usual, come the witching hour, cool things start to happen.

This video was taken at around ten past midnight, after we spent a couple of hours restoring the large format vinyl cutter that we donated a few months back. It's a Foison C24 and has been used about three times in it's lifetime. Hopefully, now it's in it's new home and working, and more people have access to it, it'll get used quite a bit more.

As you can see, it's quite a whizzy-fast machine. Because it's designed to use a drag knife, it takes corners quite slowly, but really flying down the straights!

For testing, we replaced the drag knife with a fat-tipped Sharpie and drew on some copper board. Of course, it's only a matter of time before we're drawing PCBs directly onto the copper board, by-passing the whole press-n-peel stage of etching ;-)

DipTrace for home etching, gerber export, 3d views and milling outlines

The more time we spend with DipTrace, the more we like it, here at Nerd Towers.
We first came across it while designing PCBs for our miniature guitars last year - changing over from ExpressPCB because we needed to generate gerber files to have some circuit boards manufactured online.

It's easy to use, has a massive library of components, but uses an approach similar to Eagle; when placing components in the schematic, you have to choose the "correct" component with the appropriate footprint while you're drawing the schematic. We much prefer the ExpressPCB method: draw a schematic then link it to the PCB designer - so long as the number of pads in the PCB layout software matches the number of pins in the schematic, you can use pretty much any component footprint to represent any component in the schematic.

As a result, we still turn to ExpressPCB whenever we need to "knock out" a few boards for home etching.
But since we've had to design some more boards for manufacture (for a recent alarm clocks project) we've returned back to DipTrace to generate more gerbers.

And we've found that DipTrace has some pretty cool features - even for the homebrew enthusiast. Firstly, you can create homebrew boards easily and - using the ExpressPCB method - print to a PDF file using CutePDF and flip in Inkscape, reading for printing onto some press-n-peel.

Of course, DipTrace can export to Gerber (so the file(s) can be emailed for manufacture by a professional PCB fabricators). But it also has a few other handy export options.

We've already discovered the brilliant 3D view:

The great thing about this 3d generator is that you can place components so that they "hang off" the board. Not many tools let you do this. In the example above, we've created our audio board but cut the dimensions down, so that the SD card holder is fixed at one end. There's no need to waste copper board under the rest of the sd card holder - not all PCB layout tools allow you to create a board outline smaller than the components on it!

As well as 3D, and creating toner-tranfer images, DipTrace supports exporting to DXF.
Why is this so impressive? Well, if you've a CNC router, you can get it to draw around the outside of each of your pads and traces:

Wow! Now that's cool. Anyone with any experience of using Mach3 or similar CNC software knows that it's relatively easy to get from a DXF drawing to some g-code which we can use to drive the CNC router and - rather than all that nasty etching with stinky chemicals - we could rout out the PCB board from a sheet of copper-clad board.

All these output options from the same PCB layout. DipTrace is looking like a really useful bit of kit. And perhaps something we should persevere with for a little while. After all, up to 500 pins, it's FREE as well.

Etching heater

Etching with Ferric Chloride is a messy business. So the quicker you can get it over with the better - and for a few reasons. One way to decrease the time taken to etch is agitation: basically stir the mixture or dunk the board in and out of the etchant.

The problem with this, of course, is more opportunity for spillage. And with ferric chloride being such a nasty staining solution, you don't want it splashing around the place.

Another way to speed up etching is using heat. Ferric Chloride works best at about 50 degrees. But sitting the solution in a bath of hot water only increases it's temperature by a few degrees at best. So we tried one of these things - it's called an etching heater, but really it's just a fancy aquarium heater!

It's like a super-duper tropical aquarium heater in a sealed glass case- complete with temperature control on the top, so you can set the thermostat to anywhere between 40 and 60 degrees: perfect for ferric chloride etching! Just search ebay for "etching heater". This one cost less than £15 including delivery to the UK and took about 8 days to arrive.

We put about a litre of ferric chloride solution (made up quite strong, with a full 500g packet of crystal dissolved into about 1L of warm water) into a sweetie jar. This should allow us to etch rather larger boards if we need to. Just stand the heater in the solution, switch on and back away. After about five minutes, the entire thing is up to perfect etching temperature.

(heating the solution is easy - plop in the heater and switch on!)

A relatively small board can now be etched in about two-to-three minutes, rather than the 15-20 minutes it has been taking recently, when etching "cold" (as the winter nights draw in, it does get colder in the nerd cupboard, so ambient temperature is about 16-20 degrees).

The great thing about etching quickly is that because the board is in the solution for a lot less time, the chance of under-cutting is much less (under-cutting is where the etchant eats away at the copper under the toner-transfer mask because the board is left for too long in the solution). By etching hot and quickly, we get really sharp, clear etching, even with thin traces. 

In the photo above, we've managed to get traces down to 0.2mm and they've etched perfectly. When etching "cold" we would rarely go below 0.38mm trace size to avoid over-etching the tiny thin traces.

Here's the board cleaned up, with a micro sd card connector soldered in place, ready for populating. In short, 

DipTrace and the amazing 3D render

I recently had to get some PCBs made up for a little project, so had to leave my favoured ExpressPCB behind and use DipTrace again, in order to generate some gerber files for manufacturing (normally we'd just etch our own boards, but time is tight with this one, so they're on a three-day turnaround from a UK-based manufacturer).

I was looking for a gerber viewer when I came across Jason's post about a web-based 3d renderer: https://plus.google.com/u/0/105009608886388132613/posts/1oST39owVAZ

Now that looked exciting. I uploaded the gerber files generated from DipTrace and eagerly waiting for the image to appear..... nothing. Whether it was the gerber file, my lack of 3d graphics hardware in my little notebook or something else, I don't know. I do know that other people commented that they couldn't get it to work either. So no 3D renderings for us.....

Then I spotted this little icon on the toolbar:

So what would that do....?

Ta-da! A perfectly rendered (and populated) PCB. I'm not quite sure how to get the components to sit down in the board properly (seriously, who solders their components so they stand up like that?) but as a quick overview of how your final board could look (especially now we're moving away from through-hole and trying to use SMT wherever we can) it gives great results!

We might just give DIPTrace another go.
Familiarity and laziness have kept us using ExpressPCB. But these 3D renderings do add a fresh dimension to documenting projects (-no- pun intended) so maybe it's time to give it a whirl. The freeware version is quite restrictive with only 300 pins allowed. But for simple, little projects like this one, that's more than enough!

DC blocking capacitor

We've had a few questions about the in-line capacitor on our audio player board. This is the capacitor between the PIC pwm output pin and the gate pin on the transistor amplifier. At first glance, it seems a bit counter-intuitive: we're "blocking" the output from the PIC to the speaker. What's it there for?

Well, here's the thing - the circuit will actually work without the capacitor in place.
Go ahead, try it. You can connect the output from the PIC directly to the gate on the transistor, and you'll still hear sound. It might not be quite as loud, but it'll still be an amplified signal. So what does the "blocking capacitor" do?

Consider what's happening - we're using PWM to drive a speaker.
It's not the most efficient use of the speaker, since we're not generating negative voltages, so we're only every driving the speaker "forwards" using a positive voltage. But let's ignore that for a moment!

Our PWM is creating an "average" voltage over a particular time period, to move the speaker by a certain amount. Now most digitally recreated sound waves go from minus the maximum amplitude (let's call it -5V for simplicity) and plus maximum amplitude (e.g. +5V). When there's no sound/signal, there's no voltage 0V.

But because we don't have negative values in our PWM , we're having to use a value 0-255 to represent the full range from -5 (in this example) to +5.

So -5 (the lowest possible value in the wav file) becomes PWM value zero (the lowest possible value in our PWM output). And +5 (the maximum value in the wav file) becomes PWM value 255 (the highest possible value in our PWM output).

So where the audio file is normally zero (half-way between max and min) our output PWM value is actually 128 (half-way between 0-255). When there is no audio signal, our PWM output is 128, or half-duty cycle, or (on average) 2.5V output.

Now driving 2.5v continuously through the coil in our speaker is going to cause it to heat up and over time, eventually burn out the speaker. So what we do is put a capacitor between the output pin and the gate (which in turn controls the amount of current going into the speaker).

So now, while our audio is silent, there's nothing actually coming out of the other side of the capacitor.

• When the wav signal falls, the capacitor discharges  and creates a high output.
• When the wav signal rises, the capacitor starts charging up and the "output" side falls low.
• The output from the capacitor lags the input by a single sample width.

This means that while the signal is fluctuating - ie. we're using an AC signal - the capacitor allows the signal to "pass-through" albeit one sample width behind the input. When the AC signal becomes DC (a period of sustained silence, for example) the output from the capacitor drops to zero and no voltage is passed into the speaker. The "blocking capacitor" acts like a filter - stopping DC current but allowing AC signals through.

This is a very crude method of driving the speaker, but one that works.
The downside is that where a very quiet audio signal is being played, with large gaps of silence, or very quiet background noise, the speaker is hardly active at all, and you can hear all the other "noise" in the line (the PWM carrier signal, for example).

We can hear this at the start of the song being played in the earlier video.
For the first ten seconds or so, the background sound of rain just sounds like a digital noise and you can hear the hum of the PWM signal. This is because parts of the audio signal are being treated as "rippling DC current" rather than a true AC signal. However, once the audio signal amplitude increases, a large AC signal is generated and the blocking capacitor allows all of the signal to pass into the speaker and the audio clarity improves.

Different speakers sound different

It's almost so obvious as be incredible!
Here's our PIC-based wav audio player with a cheap 15p speaker from Farnell. The difference between this and the speaker on the alarm clock is amazing! We've tried all kinds of different ways of amplifying the output but the best/loudest to date is this 16 ohm, 700hz, 90dB piezeo speaker with a humble BS170 FET transistor.

The audio in this example is much clearer than with the earlier speaker (and louder too).
Perhaps this is because rather than put the speaker in a box, we're holding it to create a cavity behind the speaker cone. You can definitely hear the speaker gain more definition between hanging loosely and being "encased in flesh"!

Interestingly, the best sound came when we created a tube effect rather than a closed box shape with the hand holding the speaker. Whether this only applies to hands, or to enclosures in general, we've yet to see.....

FAT/FAT16 tables - finding the data

Having successfully found the root directory on our sd card, we now need to start actually reading the data back from it. This is the last little link in a chain of

• initialise the sd card
• find the lba (logical block address) by reading sector zero
• read the mbr (master boot record)
• calculate where the root directory starts
• find the file we're interested in
• read the file

The easiest way to find a file is to load a filename into an 11-byte buffer. The easiest format to use is the name, padded with spaces plus the extension, minus the full-stop; so wav001.raw becomes wav001[space][space]raw. Or, perhaps more accurately, WAV001[space][space]RAW (since FAT16 likes to store filenames in uppercase).

A root directory entry looks like this:

• The first eight bytes are the file name - FAT16/MSDOS only supported up to eight characters in a filename.
• The next three bytes are the file extension
• Then there's an attributes byte
• Followed by time stamps for the file creation/modified date.
• The next two bytes are the starting cluster for the data (note, cluster not sector)
• The last four bytes are the total file length (in bytes)
• Every root directory entry uses exactly 32 bytes

It's the first bunch of bytes we're interested in - the file name, starting cluster and total file length:

Now we just read the root directory, comparing the first 11 bytes of each 32-byte entry in the root directory to the bytes in our filename buffer and if they match, we've found the file we're interested in.

Once we've got our file, we need to work out the file-length (in bytes) and the starting sector for the first cluster of data.

Byte 26 of 32 is the least-significant byte of a two-byte word  (e.g. 0x02)

Byte 27 of 32 is the most-significant byte of a two-byte word (e.g. 0x00)

The starting cluster for the file in this example is therefore 0x0002

Bytes 28-31 represent the file length, starting with the least significant byte of a 4-byte value.

In this example:

Byte 28 = 0x43

Byte 29 = 0x5e

Byte 30 = 0x09

Byte 31 = 0x00

The total file length is 0x00095e43 which in decimal works out as 613,955

Looking at the file wav003.raw in Windows Explorer confirms that the file is, indeed, 613,955 bytes in size

Now previously, we worked out a whole heap of important things from our MBR (master boot record) including where the actual data started, and the number of sectors per cluster (in a FAT16 formatted disk, this is usually 32 sectors per cluster, making each cluster 16kb)

If we know which sector the actual data begins from, and which cluster to begin reading from, we can calculate at which sector the cluster begins.

unsigned long clusterToSector(unsigned long cluster){
     // a cluster is a complicated thing:
     // first there's an ofset to the master boot record
     // then some reserved sectors
     // then a fat table (maybe two)
     // then the root directory (fixed length)
     // THEN there's the first data block,
     // which is where we start reading our clusters from
     // BUT there's a catch - clusters start from zero not 2
     // so whichever cluster number we've been given, subtract 2
     
     unsigned il;
     il=cluster-2;
     il=il*sectorsPerCluster;
     il=il+dataStart;
     return(il);     
}

Great stuff!
Convert the cluster number (in this case, cluster 0x0002) into a sector (in this case, because clusters start from 2 - it's an anomaly of the FAT16 format - our first cluster is also the first sector where the data starts. We've already calculated this value)

"The actual data starts at sector 615 + 32 = 647"

If we jump to sector 647 and start reading back the data, we find that there is, indeed, data in there!
But with a file that's 613,955 bytes long, it's not all going to fit into a single cluster (one cluster is 32 sectors and each sector is 512 bytes, so that's only 32x512 = 16,384 bytes - 16kb)

So where's the rest of the data?
That's where the FAT table comes in!

Firstly, take the cluster number and double it. That's the same as bit-shifting the entire value one place to the left. In our example, our starting cluster was 0x02 so we double this and end up with 0x04
This tells us where in our FAT table to find the next (two-byte) cluster number.

Since the FAT tables themselves are written across a number of sectors, we need to convert this cluster_doubled value into a sector and byte offset, to read the actual "next cluster value" back.

Divide the cluster_doubled value by 512 to get the sector number
The remainder is the byte offset.
In our example, this gives us sectors zero, byte offset 4
So we want the first sector (sector zero) in our FAT table, fourth and fifth byte

Since the FAT table begins at sector 143, we add zero to this, open the sector (using our earlier functions) and read back all 512 bytes. When we get byte four, this makes up the least significant byte of a two-byte (16-bit) value. Byte five is the most significant byte.

In fact, when we open our sd card, read back bytes four and five from sector 143, we get
Byte 4 = 0x03
Byte 5 = 0x00

This tells us that the file continues at cluster 0x0003.
Using the same technique, we open and read the data from cluster 0x0003 (sector 647 + (3-2)*sectorsPerCluster = 647 + 32 = 679)

We continue reading data from the sector(s) and calculating the next cluster where the file continues until we've either read back the entire file (total bytes read > file size in bytes) or the FAT table returns 0xFF as the next cluster (this is a special response to say "no more clusters for this file").

This is summarised in the following function (remove comments around UART calls to see what the microcontroller is actually doing when calculating next FAT clusters).

unsigned char openNextBlock(){

     unsigned short iFATSector;

     unsigned short iFATBytes;

     unsigned short iy;

     unsigned short ix;

     

     //UARTPrintLn("opening next block of data");

     

     // the cluster_number_doubled is the location in the FAT table

     // where we can find the next block of data. If this entry is 0xFF

     // it means that there's no more blocks of data, otherwise the entry

     // is the next cluster where the file continues from

     

     iFATBytes=nextFatClusterDoubled & 511;

     iFATSector=nextFatClusterDoubled>>9;

     iFATSector=iFATSector+FATStart;

     

     //UARTPrintLn("look up next cluster in FAT ");

     //UARTPrint("sector ");

     //UARTPrintNumber(iFATSector);

     //UARTPrint(" byte offset ");

     //UARTPrintNumber(iFATBytes);

     //UARTPrintLn(" ");

     

     // check the FAT tables for the next cluster for the current file

     r=sdReadBlock(iFATSector);

     for(iy=0; iy < 512; iy++){

          r=readByte();

          if(iy==iFATBytes){ix=r; nextFatCluster=ix;}

          if(iy==(iFATBytes+1)){ix=r; ix=ix<<8; nextFatCluster=nextFatCluster+ix;}

     } 

          
     // close the currently open sector
     sdSecReadStop();

     nextFatClusterDoubled=nextFatCluster<<1; // this is the same as multiplying by two!


      
     //UARTPrint("next FAT cluster ");
     //UARTPrintNumber(nextFatCluster);
     //UARTPrintLn(" ");          
     
     if(nextFatCluster==0xFFFF){          
          // if we're at the end of the block, send the end of block marker
          //UARTPrintLn("no futher fat clusters to follow");
          return(0xFF);
     }else{
          
          // open the next sector
          iSector=clusterToSector(nextFatCluster);
          sectorCount=0;
          //UARTPrint("file continues at sector ");
          //UARTPrintNumber(iSector);
          //UARTPrintLn(" ");
          
          return(0);
     }
}

Another video interruption - serial audio with a PIC 16F1825

While writing up how SD cards work, and reading files using FAT16, we've also been developing our little audio board a bit further. We got as far as playing audio files from our SD card and fitting them to the alarm clocks for a tester project the other day.

We've now taken the project one stage further, and introduced serial/uart support.
So rather than play a file immediately, you can send serial commands to the audio board.

• # - send back a list of all files found in the root directory
• | - pause playback
• > - resume playback 
• \ - half current volume (max four times)
• / - double current volume (to max)

At any time you can send in a filename over serial.
If the filename is in the format xxxxx.yyy (where xxx is any string up to 8 characters and .yyy is a full stop followed by a three-character extension) the chip will search the root directory for a file with the same filename and extension.

If a file is found, it begins playing immediately.
If the chip is in the middle of playing a wav, the current wav file is stopped, the current sector is closed, and the chip plays the new sound from the start.

Note - our video is blurry (again) thanks to wonderful camera-phone technology! And trying to hold the speaker in one hand, the phone in the other, and typing at the same time ;-)

The PWM carrier signal generates a hum which is much more noticeable in the video than in real life, but in a future revision, we'll look to remove this using a low-pass filter or capacitor across the speaker terminals.

At present we're playing raw (headerless) PCM wav files.

The frequency is hard-coded to 22khz and all sounds are played in mono at 8-bit. This requires the sounds to be prepared before copying them to the SD card. A future revision will be able to play different types of wav files, but for now, we've a single-chip, low-cost, low component-count audio player board which works with all SD cards we've tried it with!

Understanding FAT tables

While it's possible to simply read and write data to and from our SD card serially (treating the card as one big eeprom chip) the most useful way to use the card is to format it using FAT16.
This will allow the card to be removed from the microcontroller and the data either read back or written to using a PC. This is far more useful that having to use some dedicated hardware to stream read/write data to/from the card and report it's finding back over UART/serial.

FAT is quite a complicated structure and needs careful understanding; it's very easy to get lost and end up reading chunks of data from the wrong sector and mangling files - especially if files on the card are frequently added or deleted and the files end up fragmented (i.e. written over a number of non-consecutive sectors).

To get started, download a hex editor (we used HxD) and format your sd card to FAT or FAT16 (not FAT32) format.

Here's your first gotcha:
When you mount an sd card into a hex editor to view the entire file contents, the hex editor usually starts from the LBA (logical block address). It will often call this sector zero.
But on the SD card, this may not actually be physical sector zero. If you use the previous card reading functions and write out the contents of sector zero over serial, you'll see it doesn't look like a boot record - it's mostly zeros. Sector zero usually just contains information about where you'll find the actual master boot record (and where the hex editor will call sector zero).

We found the easiest thing to do was use the UART tool on the PicKit2 software to view the serial data as it was read back from the sd card. We used the "save to disk" option and then read through the file after it had been saved as a text file. You can use whichever serial-based tools you're happy with for debugging at this point:

The last two bytes of the entire sector were 0x55 and 0xAA
This is the FAT signature to say that we're reading a FAT-formatted drive. We just double-checked that our code was working and actually reading back sensible data by looking for this signature at the end of the sector-ful of data.

Most of our sector zero data was zero - 0x00.
Having read up a little about FAT tables and file layouts, we were expecting a bit more than this - we were told to expect things like the OS name/type, volume label and so on, but nothing here looked anything like it. But that's because we weren't looking at "logical sector zero" - the first sector in the file structure - we were looking at physical sector zero.

So our first job was to find the logical block address LBA.
In amongst all the zeros, there was a little bit of data, starting at byte 447.
This is the data describing the actual drive itself - sectors, clusters, heads and cylinders (used in older drives). The data we're after is the two bytes at 454 (0x1C6) and 455 (0x1C7).
This is the sector number containing our master boot record MBR.

Another gotcha coming up - where data is written using two or more bytes, the order of bytes is back-to-front to what you'd expect: the least significant byte comes first. So in our example, we've got 0x87 followed by 0x00. But this actually translates to 0x0087.

Using the same routines as we did for reading sector zero, we get the PIC to read back the contents of sector 0x87 (135 decimal).

Straight away we can see that this has some useful information. Already, we can see the name MSDOS5.0 being spelled out. So what about all the other junk in there? Here's how the boot record is laid out:

• Jump instruction (3 bytes)
• Operating system (8 bytes)
• Bytes per sector (2 bytes)
• Sectors per cluster (1 byte)
• Number of reserved sectors (2 bytes)
• Number of file allocation tables (1 byte)
• Max number of possible root entries (2 bytes)
• Small sectors (2 bytes)
• Media type (1 byte)
• Sectors per file allocation table (2 bytes)
• More stuff we're not particularly interested in

The bits of information we're interested in are the bytes per sector (11 bytes into the record), sectors per cluster (13 bytes in) number of reserved sectors (14 bytes in) number of FAT tables (16 bytes in) the maximum possible number of entries in the root directory (17 bytes in) and the number of sectors used by each FAT table (22 bytes into the record).

Looking at our log files, we find entries 11 and 12 are 0x00 and 0x02 respectively.
But we have to remember that data is stored back-to-front, so the bytes per sector value is actually 0x0200 which is 512 in decimal. This looks like a sensible value so we can assume we're correctly reading the data back.

Sectors per cluster (a single byte) returns 0x20 which in decimal is 32. Again, a sensible-looking value and one we'd expect (32 sectors of 512 bytes = 16kb per sector, which is the FAT16 standard).

Number of reserved sectors is important for us to work out where the FAT tables and data actually reside on the card. Reading bytes 14 and 15 we get the (don't forget to flip them around) value of 0x0008 - i.e. there are eight reserved sectors following this boot record.

Usually FAT tables are duplicated to help provide a means of data recovery should one table become corrupted. Although very few people ever use the second table (and in fact, should either of the tables become corrupt, the host machine should stop reading/writing to the sd card and report a FAT mismatch error) it still takes up space and needs to be accounted for when working out where the data starts on the disk. Byte 16 is 0x02 - ie the FAT table is duplicated once; there are two working copies of the FAT system on our card - again, the expected value.

Maximum possible number of entries in the root directory is important - we can work out where the root directory is, and the data is stored immediately after it, so we need to know how many sectors the root directory takes up. Bytes 17 and 18 tell us that the max number of records in our root directory is 0x0200 or 512 in decimal.

The number of sectors used by each FAT table is in bytes 22 and 23.
In our case, we have the value 0x00ec or 236 in decimal.

From all this information we can deduce:

• Our boot record is at sector 135 (in your hex editor, this is sector zero)
• After 8 reserved sectors, the FAT tables begin at physical sector 143 (in your hex editor, these would start at sector 8 - we're going to stick with physical sectors to avoid confusion, but be aware that your sectors will be offset in the hex editor)
• Each FAT table is 236 sectors in size
• There are two FAT tables
• Our root directory starts at sector 135 + 8 + (2*236) = 615
• The root directory can hold up to 512 entries. Since this is FAT16 (each entry uses 16 bytes) this means our root directory takes up 512/16 = 32 sectors
• The actual data starts at sector 615 + 32 = 647

Our SD card already contains a few files.
Since we're going to (eventually) be playing audio, we put some RAW audio data on there (files imaginatively called wav001.raw, wav002.raw etc.) and a short text file called FAT16.txt

From our deductions, we run our "read-sector" routines on sector 615 and see what we get:
(your card may have different offset values for the boot record, reserved sectors, FAT table sizes etc, so use the equations above, but substitute with your card's values to get the actual sector to read from - your card may not have the root directory starting at sector 615. In fact, we're not sure that ours does yet.....)

Holy cow Batman, it looks like we've found our root directory........

Reading and writing to a single sector on an SD card with a PIC microcontroller

Having got our sd card to respond to the SPI initialisation routines and "boot up" correctly, it's time to actually start reading and writing data to the card.
The first thing to understand is that an SD card is made up from a series of sectors. Each sector is 512 bytes in size (almost always for SD cards) and the SD card likes to read and write entire sectors at a time.

not to scale ;-)

Reading data is a case of sending the appropriate "command packet" - 6 bytes which are:

• the command byte (CMD17)
• 4 parameter bytes (the address of the sector to read)
• two CRC bytes (an accurate crc value is not actually necessary in SPI mode, but two bytes still need to be sent)

It is possible to read multiple sectors of data, one after the other, and use the sd card like one massive serial eeprom, but for now, we're going to just work one sector at a time (this approach is much better suited to working with FAT16-based files, which we'll discuss later).

The command value to read a single sector of data is CMD17. After sending the command, we have to check for an "ok" response from the sd card then wait for the "not-busy" response (any non-0xFF value).

Boolean sdReadBlock(UInt32 sec){
     UInt8 v;
     UARTPrint("Starting to read block ");
     UARTPrintNumber(sec);
     UARTPrintLn(" ");
     
     v =  sdCommandAndResponse (17, ((UInt32)sec) << 9);
     if(v & FLAG_TIMEOUT) {
          UARTPrintLn("start read block command 17 timeout");
          fatal(1);
          return false;
     }else{          
          do{
               v = sdSpiByte(0xFF);
          }while(v == 0xFF);
          if(v != 0xFE) {
               UARTLog("Read block response",v);
               fatal(2);
               return false;
          }     
          
          curPos=0;
          return true;
     }
}

This function accepts a single 4-byte parameter, the sector address to open, and returns true or false to indicate whether the card responded correctly.
It also resets a counter curPos, which will keep track of the number of bytes we've read from the current sector on the disk. When we've had exactly 512 bytes back from the disk, we'll have to "close" the current sector by reading back the two bytes that make up the crc for the previous 512-byte data stream.

void sdSecReadStop(){     
     
     if(curPos<512){
          UARTPrint("add stuff bytes to close sector - ");
          UARTPrintNumber(curPos);
          UARTPrintLn(" ");
     }
     
     while(curPos < 512){
          sdSpiByte(0xFF);
          curPos++;
     }
     
     // read back the two CRC bytes
     UARTPrint("CRC ");
     r=readByte();
     UARTByte(r);
     UARTPrint(" ");
     r=readByte();
     UARTByte(r);
     UARTPrintLn(" ");
     UARTPrintLn("End read sector");
     UARTPrintLn(" ");
     
}

Now we can open a sector and close it again, the last piece of the puzzle is to stream the data back from the sd card and do something with it! When we come to make our audio player, we'll use the data immediately - sending it to the speaker - but for testing, or for more general purpose use, we'll put the data into a series of buffers and use it later.

Now - here's a thing.
So far, everything discussed can be ported to another platform (AVR/Ardunio for example) but when it comes to storing the data in our internal buffers, there's a peculiarity with the PIC microcontroller - or at least, with most compilers that support using arrays. It may be the same with Arduino and/or other compilers, but we've always found that large arrays are often difficult to use.
This is because of the way PICs store data in RAM - it uses "banks" to keep the data in memory, and one array can't usually span more than one bank of data. In practice, this means a limit of about 90 elements in an array.

To keep things simple, we're going to store an entire sector's worth of data (512 bytes) in 8 eight arrays of 64 elements.

unsigned char wavData1[64];
unsigned char wavData2[64];
unsigned char wavData3[64];
unsigned char wavData4[64];
unsigned char wavData5[64];
unsigned char wavData6[64];
unsigned char wavData7[64];
unsigned char wavData8[64];


unsigned char readSector(){
     unsigned short cy;
     unsigned short cv;
        
     r=sdReadBlock(iSector);               
     for(cy=0; cy < 512; cy++){
         r=readByte();               
         UARTByte(r);    //write to serial for debugging
                  
         if(cy< 64){
            cv=cy;
            wavData1[cv]=r;
         }else if(cy<128){
            cv=cy- 64;
            wavData2[cv]=r;
         }else if(cy<192){
            cv=cy-128;   
            wavData3[cv]=r;
         }else if(cy<256){
            cv=cy-192;   
            wavData4[cv]=r;
         }else if(cy<320){
            cv=cy-256; 
            wavData5[cv]=r;
         }else if(cy<384){
            cv=cy-320;   
            wavData6[cv]=r;
         }else if(cy<448){
            cv=cy-384;   
            wavData7[cv]=r;
         }else if(cy<512){
            cv=cy-448;   
            wavData8[cv]=r;
         }
     }           
                                        
     sdSecReadStop();     
     UARTPrintLn(" ");
     return(1);
}

Now we can read a sector-ful of data, it's time to write some data back.
There are any number of ways you can get data into the PIC/microcontroller in order to write them to the sd card. We'll leave that side of things for another post - for now, here are some functions to write data to a specific sector on the card.

The command to write a single sector-ful of data to the card is CMD24
(you can write an entire stream across multiple sectors if you're using the card as a large serial eeprom chip, but this makes things difficult if we're going to work with FAT16 formatted cards in future).

When writing data, the 6-byte data packet consists of:

• single command byte
• four parameter bytes (sector to write to)
• single crc byte
• a single byte representing a token confirming the command request

Boolean sdWriteBlockStart(UInt32 sec){
     UInt8 v;
     v =  sdCommandAndResponse(24, ((UInt32)sec) << 9);
     if(v) {
          UARTLog("write block start response",v);
          fatal(1);
          return false;
     }else{          
          UARTPrintLn("write block started");     
          // keep track of how many bytes we've written in this sector
          // (when we hit 512 we should expect some extra bytes in the packet data)     
          bytesWritten=0;     
          
          // send the correct token for CMD17/18/24 (0xFE)
          // REMEMBER the token for CMD25 is 0xFC               
          r=sdSpiByte(0xFE);
          return true;
     }
}

Once the card has responded with an "ok" after starting to write a sector, the next bytes streamed to the card are written to the selected sector. We need to keep track of the number of bytes written - when we hit 512, it's time to close the current sector, read back the crc bytes, then open another - you don't have to write to the sequentially next sector every time, you can write to any old location if you like!

Boolean sdWriteByteToSector(UInt8 b){     
     UInt8 ix;
     r=sdSpiByte(b);          
     bytesWritten++;     
}


Boolean sdWriteToSectorClose(){          
     // finish closing the sector
     UARTPrintLn("writing stuff bytes to close sector");
     while(bytesWritten<512){
          sdSpiByte(0xFF);
          bytesWritten++;               
     }
          
     // send two CRC bytes
     sdSpiByte(0x00);
     sdSpiByte(0x00);
          
     // response should immediately follow
     // (for writing, we're looking for 0bXXX00101 data accepted)
     r=sdSpiByte(0x00);     
     UARTLog("write finish response",r);
     
     // now wait for the sd card to come out of busy
     while(r!=0x00){
          r=sdSpiByte(0x00);
     }
          
     UARTPrintLn("write finished");          
}



// write some data to sector numbered five:
void writeSomeData(){
    UARTPrintLn("Writing one block");
    ret=sdWriteBlockStart(0x05);          
    if(!ret){
         fatal(3);
    }else{

         for(iy=0; iy<=255; iy++){
              sdWriteByteToSector((255-iy));
         }                    
     
         sdWriteByteToSector(0x10);
         sdWriteByteToSector(0x11);
         sdWriteByteToSector(0x12);
         sdWriteByteToSector(0x13);
         sdWriteByteToSector(0x14);                    
                    
         sdWriteToSectorClose();                         
    }
}

That's pretty much it.
We can now read and write to an individual sector on the sd card.
There are commands for reading and writing to/from multiple blocks at a time, but we'll leave those for again; for FAT formatted cards, we need to be able to read data out-of-sequence and possibly even write files into different (non-consecutive) sectors.

How SPI works with an SD card

SD cards have two main operating modes.
Their default mode is high-speed through 4-bit wide port but we're going to be working with the "legacy" SPI (two-wire) mode.

In SPI mode, the master device (our microcontroller) talks to the slave device (the sd card) using a data and a clock line. Every time the clock line goes from low-to-high (or, if you prefer, from high-to-low - you can change this to suit the application needs) the receiving device looks at the data line. If it's high, it receives the single-bit value 1, if it's low, zero.

in this example, when the clock line goes from low-to-high (sometimes called a rising edge trigger) as denoted by the red vertical lines in the clock timing diagram, the state of the data line is converted into a value

The great thing about SPI is that it's not time dependent. Because the master device sends the clock line along with the data, it can be speeded up and slowed down (this is not possible using methods such as UART/serial, which has a fixed data rate; ie. the data has to be moved within a specific time period).

Using this clock-and-data method, we can send commands to the sd card, to tell it to do specific things. So we can send a specific value (the sd format sets out specific values to send for specific commands) to get it to reset, for example.

When an SD card has received and understood a command, it can remain in a busy state for quite some time. It is important to wait until the card has finished doing whatever you asked it to do, before blasting more data or commands at it. To do this, we poll the card (continuously ask it for data) until it gives us a "ready" token.

SPI is actually a data exchange mechanism. There's no difference between reading and writing bytes between the devices. As one device sends a byte of data, so the other transmits one. After sending a single byte of data from an SPI buffer another (possibly different) byte may appear in it's place - this is the byte that has been received.

So to read a byte from the sd card, we have to send it a byte too. It's normal to send either all zeros (0x00) or all ones (0xFF in hex is 255 in decimal, which is 11111111 in binary) for "don't care" bytes - so if you're just reading data from the other device, and it's not important what data you send, it's common to either send 0x00 or 0xFF.

Whenever a command is sent to an SD card, it follows a specific format:
There is a single command byte
There are four "parameter" bytes - data which tells the recipient how to perform the command requested
There is a single CRC (checksum) byte to prove that the previous bytes have been transmitted correctly.

To send data to our SD card, we need a couple of functions:

UInt8 sdSpiByte(UInt8 data){
     ssp1buf = data;
     while(!(ssp1stat & (1<<BF)));
     return ssp1buf;
} 


static inline void  sdSendCommand(UInt8 cmd, UInt32 param){     
     UInt8 send[6];
     
     send[0] = cmd | 0x40;
     send[1] = param >> 24;
     send[2] = param >> 16;
     send[3] = param >> 8;
     send[4] = param;
     send[5] = (sdCrc7(send, 5, 0) << 1) | 1;
     
     for(cmd = 0; cmd < sizeof(send); cmd++){
          sdSpiByte(send[cmd]);
     }
}

This provides us with a simple method of sending commands (and their parameters) to the SD card.

The first function simply puts a value into the hardware SPI buffer then waits for the SPI busy register value to go "not-busy" before returning the value it finds in the buffer (which is now the value received from the other device - data exchange remember!)

The second function actually sends commands to the sd card. Every command byte sent to an SD must have bit 6 set (so the device can recognise it as a command and not some data).
Bit 6 in binary is 01000000 which is 64 in decimal or 0x40 in hex.
So to make sure that every command byte has bit 6 set, we always OR the command byte with 0x40 (so when we send command zero, for example, it's actually transmitted as 0x40, command one is sent as 0x41 and so on).

But every time we send a command (and it's parameters), we have to wait for a not-busy response from the SD card. While the SD card is busy, it holds its "output pin" high - the data clocked out of it is always 11111111 (or 0xFF) So we build these little functions:

static inline UInt8  sdReadResp(void){
     UInt8 v, i = 0;     
     do{                
          v = sdSpiByte(0xFF);
     }while(i++ < 128 && (v == 0xFF));     
     return v;
}

static UInt8 sdCommandAndResponse(UInt8 cmd, UInt32 param){   
     UInt8 ret;          
     sdSpiByte(0xFF);
     sdSendCommand(cmd, param);
     ret = sdReadResp();     
     return ret;
}

This sdCommandAndResponse function sends a "dummy byte" to the sd card.
It then sends the command byte, followed by the 4-byte parameter value(s).
Next it calls the read-response function, which continuously sends the dummy byte 0xFF to the sd card, until the response back is not busy. When the response goes not busy, the response value is returned to the sdCommandAndResponse function. The response could either be "all ok" or it may be some kind of error code to explain why the command given could not be completed.

One last function to mention is the CRC generating function.
Strictly speaking, once we've told our card to work in SPI legacy mode, we don't actually need to generate the CRC values, but it's included here for completeness.

Note: We didn't actually create this function, we ported it from another sd card library for another platform:

static UInt8 sdCrc7(UInt8* chr,UInt8 cnt,UInt8 crc){
     UInt8 i, a;
     UInt8 Data;

     for(a = 0; a < cnt; a++){          
          Data = chr[a];          
          for(i = 0; i < 8; i++){               
               crc <<= 1;
               if( (Data & 0x80) ^ (crc & 0x80) ) {crc ^= 0x09;}               
               Data <<= 1;
          }
     }     
     return crc & 0x7F;
}

Before we can actually start sending data over SPI, we need to set up the PIC to use the hardware SPI peripheral. This means writing some values to particular registers in the chip. The names of these registers should be similar across different PIC models, but may not be exactly the same if you're using a different chip:

static void sdSpiInit(void){
     ssp1add          = 21;              //slow clock down to < 400khz
     ssp1con1     = 0b00101010;          //spi master, clk=timer2
     ssp1stat     = 0b11000000;          //CPHA = 0
}

The important registers here are the SSP1ADD (multiplier value) ad SSP1CON1 register.
When the last four bits of SSP1CON1 are 1010 this slows down the SPI clock speed - by how much depends on the value in the SSP1ADD register (and the actual speed, as in time taken to send each clock pulse, is dependent on the overall processor clock speed so will change with each chip model). When the last four bits are set to 0000, the SPI clock changes on every instruction cycle. When it's set to 1010, it changes on every x clock cycles, where x is the value held in the SSP1ADD register.

The SD card initialisation routine after powering up is:

• Set the clock speed to less than 400khz
• Hold the chip select line on the card low and send in about 80 clock pulses
• Pull the chip select line high to tell the card we're talking to it
• Send in the "soft reset" command (CMD0)
• Wait for the sd card to respond "ok" with the value 0x01
• Send in the "initialise card" command (CMD1)
• Repeat sending CMD1 until the card responds with an ok value of 0x00
• Set the sector size using CMD16 with a parameter 512 (each sector is 512 bytes)
• Turn of the CRC requirement by sending CMD59
• (all future transmissions do not require a valid crc value)
• The next time the card responds with an "ok" value, it has been initialised and we can ramp the clock up to full speed.

All of the initialisation routines have to be carried out at the relatively slow clock speed of not more than 400khz. So we need a couple of extra functions to enable us to set the clock speed and enable the chip select line:

static void sdClockSpeed(Boolean fast){
     if(fast){
          ssp1con1 = 0b11110000
     }else{
          ssp1con1 = 0b11111010
     }
     
}

void sdChipSelect(bool active){
     portc.3 = !active;
}

For debugging, we've written a simple "fatal error" function to let us know which part of the initialisation failed (if there are any problems). When a fatal error is hit, this function reports it, then puts the microcontroller into "sleep mode" so that the program flow is permanently interrupted.

void fatal(UInt8 val){          //fatal error: flash LED then go to sleep
     UInt8 i, j, k;

     for(j = 0; j < 5; j++){     
          for(k = 0; k < val; k++)
          {
               delay_ms(100);
               P_LED = 1;
               delay_ms(100);
               P_LED = 0;
          }
          
          UARTLog("Error",val);
          
          delay_ms(250);
          delay_ms(250);
     }
     
     while(1){
          asm sleep
     }
}

By using all these functions together, we can now write our sdInit routine:

Boolean sdInit(){     
     UInt8 v, tries = 0;
     Boolean SD;
     
     SD = false;
     
     sdSpiInit(); // initialise SPI
     sdClockSpeed(false); // slow clock     
     sdChipSelect(false); // CS inactive          
     
     for(v = 0; v < 20; v++) {
          sdSpiByte(0xFF);     //lots of clocks to give card time to init
     }

     sdChipSelect(true); // CS active
     
     v = sdCommandAndResponse(0, 0, true);
     if(v != 1){fatal(2);}

     v = sdCommandAndResponse(1, 0, true);     
     for(int i=0;v != 0 && i<50;++i){
          delay_ms(50);
          v = sdCommandAndResponse(1, 0, true);
     }
     if(v){fatal(3);}
     
     v = sdCommandAndResponse(16, 512, true);          //sec sector size
     if(v){fatal(5);}
     v = sdCommandAndResponse(59, 0, true);          //crc off
     if(v){fatal(6);}
     
     // now set the sd card up for full speed
     sdClockSpeed(true);                    
     return true;
}

If this function successfully returns true, the SD card has been successfully initialised and the SPI lines set to run at maximum speed for the fastest (spi-based) data transfer.

Talking to an SD card from a PIC 16F1825 microcontroller

Following on from our earlier post, we're continuing with our create-an-audio-module project which reads wav file data from an SD card and plays it through a speaker.

So far, we've written the basic framework, using Sourceboost C so now it's time to start fleshing it out.
Before we actually start making any sound, we're going to make extensive use of the UART/serial comms to report back at every stage of the data transfer, so we can pinpoint any problem areas. We're aiming to read an entire file off the sd card and stream it's contents back to the PC over serial.
Once we've done this, we'll modify the code to send the data to a PWM module to actually play the sound.

First up, with any Sourceboost project, we need our main() function.

void main(){
     unsigned char response;
     Boolean ret;     

     init();
     delay_ms(50);     
     ret = sdInit();
     if(!ret){
          UARTPrintLn("SD init failed");
          fatal(1);
     }else{
     
          setupFAT();
          response=openFile(1);
          if(response){
                         
               readCluster();
                              
               while(response!=0xFF){
                    response=openNextBlock();
                    if(response!=0xFF){
                         readCluster();
                    }else{
                         UARTPrintLn("end of file clusters");
                    }
               }
          }
          
          //just to prove we got here!
          UARTPrintLn("Done");
               
     }

     while(1){}
}

The main function pretty much speaks for itself, since it's calling a whole heap of other (hopefully appropriately named) functions:

Firstly, we initialise the microcontroller.
This involves setting the registers to define how all our different multi-function pins are set up. One of the great things about the PIC range of microcontrollers is their versatility - each pin can have multiple functions and be used for a variety of different things. Unfortunately, this can also mean hours and hours of headaches, as it can make debugging quite difficult.
So in the main, we should always call an initialise routine after powering up, to put the microcontroller into a "known state". In Oshonsoft, this includes commands such as

• CONFIG PORTA=OUTPUT 
• ALLDIGITAL 

etc.
These BASIC commands are simply macros to put specific values into specific registers on the chip. In this program, we're going to to that manually, following the datasheet.

void init(){
     osccon                    = 0b11110000;     //32 MHz clock     
     intcon                    = 0b10000000;     //ints enabled, all sources masked
     apfcon0                    = 0b01000000;    //MOSI on RA4
     trisa                    = 0b11001100;      //out on 0, 1, 4, 5, in on
     trisc                    = 0b00000010;      //portc.1 is our spi input
     ansela                    = 0;              //no analog inputs
     anselc                    = 0;              //no analog inputs
     option_reg               = 0b10000000;      //no weak pull-ups, timer0=osc/8
     wdtcon &= ~(1<          
     UARTInit();
     curPos=0;     
}

The first register value OSCON sets the PIC to internal oscillator, 32Mhz.
The INTCON register is where we set up interrupts (bit 7 is the global interrupts bit)
APFCON0 is the alternative pin configuration register - we've put the SPI data out (MOSI = master out, slave in) onto pin RA.4 - on this particular chip the SPI out pin can go on different pins if required.
TRISA is the register where we set which pins are inputs and which are outputs. A 1 makes the corresponding bit an input, zero makes it an output (so bit 7 of the TRISA register maps to pin 7 on PORTA).
ANSELA and ANSELC are the analogue registers for ports A and C - setting these both to zero is the same as the Oshonsoft command ALLDIGITAL.

Once we've got the chip up and running, and in a known state, we allow a few milliseconds just for all the internal voltages to settle and allow any multi-function pins to take the state requested.

Next, we call the initialise SD card routine.
This function will return a value to say whether or not the SD card started up correctly. If it didn't we report an error back and stop the code executing there (there's no point trying to read data back if the SD card isn't responding as we expect it to!)

If the SD card does respond to say it's set up correctly and ready to receive commands, we then call a function to read the FAT16 tables on the card. This involves finding the boot sector on the disk, finding the location of the actual FAT tables themselves, finding the location of the root directory on the disk and calculating where the actual data starts.
This is a long and complicated process and will be explained fully over a number of blog posts! It's hard going, but worth sticking with - being able to read and write data in a PC readable format is very very useful.

The setupFAT function returns a value to say whether or not the PIC was able to make sense of the contents of the card, and if it did, we open a file and start reading clusters of data from the card.
As long the file consists of more data, the function will keep calling and reading data back.

But already things are getting a bit confusing - clusters, sectors, boot sectors..... it's time to look into how FAT works

A short interruption for a demo

Here's a quick movie showing our final audio player actually being used in a final product.
The PIC has been programmed to read a value from eeprom, increase it by one, rollover back to zero once a certain value is reached, and use this value to decide which sound to play.
In doing this, we can get our alarm clock to play a different sound each morning!

The original alarm clock mechanism has been kept the same - when the alarm time is reached, the clock mechanism pulls a line low. We use this as the ground for our board, so that it is switched on an off by the alarm clock mechanism (the mechanism keeps the line low for about 15 minutes after passing the alarm time). By doing this, we're conserving the batteries, since the entire circuit is only every powered for about 30 minutes each day (once for 15 mins in the morning, once in the evening, unless the alarm on/off switch has been moved to disconnect the ground rail from the circuit).