MIDI magic - reading midi files via a byte array

It's been a while since we did any MIDI firmware.
MIDI is one of those things that we didn't really care much for a few years back - old 80s synths and analogue oscillators to make a few bleepy bloopy farty noises didn't really hold much interest. But after discovering that you can send MIDI messages over serial/uart at a specific baud rate (31250bps) we actually had a bit of fun creating MIDI controllers - not so much to create the actual noise (that's what the synth part is for) but to create unusual interfaces to trigger the synth sounds.

To date, we've only really played with "live" MIDI messages - parsing incoming data as it arrives or creating "outgoing" MIDI messages to play sounds on a synth. With our light-up guitar neck, we'd like to be able to display notes from a MIDI sequence, not just in real time, but also to be able to scroll backwards and forwards along the "timeline".

This is to allow the player to find where to put their fingers, then wind the song forwards to find where the next note is, then back again, to work out the best way of fretting the notes. Whichever way we look at it, we're going to have to go beyond just "live" MIDI messaging and get our hands dirty with the old .mid file system!

MIDI files use a pretty basic protocol. We're parsing them by loading the file as a single byte array, then walking through it.

The first few bytes in an MIDI file should be the midi file header, 77,84,104,100 (these values relate to the ascii characters MThd).

With MIDI, getting data from one device to another as quickly as possible was always key, so there's very little overhead in the data. After each "block marker" comes a "length of data" value. How this works is key to how to read MIDI files:

For any variable length data, it's packed as a seven-bit value. The leading bit (MSB) is either set to indicate that there's another byte to come, or clear, if that's the end of the variable length data. So if you had a "chunk" of data with a length of 100 bytes, the variable data would be a single byte with the (decimal) value 100. Easy huh?

But if the length value was greater than 127, it would be transmitted as two (or more) bytes. For example, to store the value 180 as variable length data, it would be stored as 129, 32. Let's look at how that works:

Our value is split over two bytes. The first byte is sent, with the leading bit (MSB) set to indicate that the multi-part value is not yet complete. Just the same as when we receive two-byte values into word or integer data types, we're going to have to do a bit of bit-shifting to create the actual final value. When we receive the next byte, we bit-shift the current value then OR the following byte value (minus the leading bit) to the running total. If the next byte has it's MSB set, we continue doing this, until we receive a bit with the leading/MSB bit cleared (indicating the end of a multi-part value).

Unlike when we receive a 16-bit value as two 8-bit bytes, we're not simply bit-shifting eight places; because our values are only seven bits long, we have to remember to bit shift only seven places to the left. It's also important to remember to mask out the leading/MSB bit (as this is a flag or marker, and does not make up part of the value we're trying to read!)

It took us quite a while to understand this concept, and we struggled for a while, reading things like strings and copyright notices, and other variable-length values. But once we got over this little stumbling block, the rest of the parsing routine was very simple.

For no real reason, we built our midi parser using PHP.
The file is loaded and the data pushed into a byte array:

$uid="ABC";
if(isset($_GET['uid'])){ $uid=trim($_GET['uid']);}

$p=0;
$file_name=$uid.".mid";
$path=dirname(__FILE__).'\\midi\\'.$file_name;
$handle = fopen($path, "rb");
$fsize = filesize($path);
$midi_string = fread($handle, $fsize);
$midi = unpack("C*",$midi_string);
$midi_file_length = count($midi);
fclose($handle);

We're using the variable $midi to store the byte array and the variable $p to indicate where in the MIDI file we're reading data from. Here's a generic function which reads a multi-byte value, and updates the "pointer" position once the "length value" has been read out of the byte array.

function getVariableLengthValue(){
      $midi=$GLOBALS['midi'];
      $p=$GLOBALS['p'];

      $pk=$midi[$p] & 127;
      while($midi[$p] & 128){
            $p++;
            $pk=$pk<<7;
            $pk+=$midi[$p] & 127;
      }

      $GLOBALS['p']=$p+1;
      return($pk);
}

Using these functions and the MIDI messaging tables from here http://www.midi.org/techspecs/midimessages.php to read back the right number of bytes in each "chunk" we managed to get a quick-and-dirty midi parser up and running quite quickly:

Now we've successfully parsed our midi file, we can shove the note data into a couple of arrays and create an interface to convert the note on/note off messages into LED on/LED off signals for our light-up guitar neck.

Laser cut guitar fingerboard

With our sanding block ready, electronics designed, firmware written and guitar parts drying, it was time to focus, once again, on making a mould for our resin fingerboard.

This time, we're going to cast the resin fingerboard as one piece, but with slots to add the fret wire afterwards. Using the excellent StewMac Fretboard Calculator we drew up our fingerboard in Inkscrape and split the fingerboard into two pieces, at around the 9th fret

Fretboard 25_5 Scale

After placing the fret lines at the correct intervals, we drew a large rectangle around the lines and tapered it to match our existing guitar neck. On our neck, the nut is just 42mm wide, and the fingerboard is 56mm wide at the 21st fret. This gives the fingerboard a much more tapered appearance than on our earlier (Gear4Music) guitar neck.

The individual pieces were then cut from some 3mm mdf sheet.

And the entire pattern repeated, only this time with the fret wires "cut" at 200mm/sec, so as to create only the lightest of engraved patterns, onto 2mm mdf

Next was simply a matter of applying PVA glue to the base and positioning each piece (perfectly) onto the fingerboard blank

The score marks on the base piece made it easier to make sure each "block" was not only positioned in the right place on the board, but also to ensure a consistent gap between the fret slots. Where the new fingerboard is made from two sections (at the 9th fret) we will assemble them in such a way as to leave a small gap between the two pieces - just enough to fit the fret wire between the two  sections.

The fingerboard blank after assembly. Note how it has more of a taper than the earlier fingerboard (which came from a different guitar). Also of note is that the spacing of the frets is the same, despite the fingerboards being different lengths. This is because they are both from guitars which have a 25.5" scale length (i.e. the distance from the nut to the bridge). Although this fingerboard is shorter than the first, the total length of each string on the guitar(s) is the same - so the fret spacing should also be the same - it's just that our new fingerboard finishes sooner than the first!

Now we've got our fingerboard blank (with slots) and our radius sanding block, we just need to leave our PVA glue plenty of time to dry fully before sanding it to shape.

our mdf guitar fingerboard blank, after sanding with a 13.5" radius sanding block

Stratocaster guitar kit

While we've been busy making resin fingerboards, RGB LED circuit boards, sanding blocks to make fingerboard blanks, writing firmware and building a web-based pattern editor, we've also had to find time to build the rest of our guitar-based project.

While it's very tempting to have a go at building the body with a large-scale CNC (possibly even outsourcing the cutting of the base wood) and hand-hacking the electronics, a far more sensible answer presented itself while browsing the Streetwise Guitars website - a DIY guitar kit.

The kit arrived in just a day, and it didn't take long for Steve to make the case for actually doing this thing properly - with multiple layers of primer, lots of sanding inbetween coats and proper polishing, to get a really pro finish.

The guitar body arrived in an "unfinished" state - the idea being that the recipient can apply their own finish. As we're going to paint our guitar body (rather than simply sand and apply lacquer for a "natural wood" effect) we left the wood with a rough surface before applying the first coat of primer.

Much to Steve and Other Steve's disgust, I insisted on a metallic purple finish for the guitar body (acrylic car paint, 1994 Ford Fiesta Purple). So we sprayed on a couple of coats of Army Painter shining silver primer.

The primer didn't keep its shiny lustre for long - as Steve insisted on properly sanding the primer between coats, which dulls the finish (and roughs it up slightly to act as a key for the next layer of paint).

It took three coats of primer, with some 400 and 800 grit wet-and-dry between coats to get a surface that was completely smooth and entirely covered all of the wood (it's very easy to over-sand on the edges and expose the wood again!). The different layers were applied over a number of days, to allow plenty of curing/drying time between layers.

The first layer of purple went on easily. But, depsite numerous warnings against applying the paint too thickly, it wasn't long before there was evidence of paint runs on our lovely shiny metallic purple surface!

Steve's really not going to like this one! Let's hope a bit of sanding once it's dry can get that dribble mark out!

On the second coat of purple paint (following, of course, a rub down with yet more wet-n-dry) the paint can spluttered to a halt and the last few drips of paint sneezed out onto the guitar body. So that's it for a few days, until the car-paint shop opens again and we can buy yet more paint to finish the job!

Creating a guitar fingerboard sanding block

After deciding that drilling holes into a wooden fingerboard was a non-starter of an idea, and that casting the frets as part of a resin neck (either with fret-wire added to the mould or casting the frets directly from resin) was no good, there seemed to be only one option left:

To cast the fingerboard and then add the fret wires back in afterwards.

To try this idea out, we need a fingerboard blank without the fret wires embedded in it. Unfortunately, fingerboard blanks are either completely blank (and rectangular cut) pieces of hardwood (into which you cut the fret slots and sand/shape the sides) or entire fingerboards are provided with the fret wires already embedded. Try as we might, we can't find a fingerboard cut to size, with slots, but no fret wire.

Quite often you can buy a fingerboard blank, or a guitar neck with fitted fingerboard. Very rarely can you get hold of a pre-cut and fitted fingerboard. But finding a fingerboard without frets fitted is pretty much impossible.

There are plenty of places to buy entire guitar necks - but removing the fingerboards can be tricky. We got lucky with our Gear4Music guitar neck - a bit of heat and it came straight off (although not quite as straight-forward nor injury-free as it sounds). We tried removing a fingerboard from an old Marlin guitar and it refused to budge. Even the one that came with our kit couldn't be removed easily.

Even after copious amounds of heat, our knife blade wouldn't slip between the fingerboard and the neck, and it looks more like we're actually cutting into the wood of the fingerboard. That's some tough glue they use on such a cheap, chinese guitar clone!

Even when we've managed to completely remove a fingerboard without destroying it (or warping it beyond recognition during the removal process) removing the fret wires from an existing fingerboard has always proved more difficult than we expected - the only way we've so far managed to remove frets from a fingerboard is by bending the wood back to encourage the wire to pop out (though sometimes - and especially if the fret wire has been epoxy-glued into place - the fingerboard can splinter and crack before the wire comes free.)

So we've decided that working with existing fingerboard is just too fiddly - we'll have to have a go at making a blank fingerboard ourselves. Getting the fret distances right should be relatively easy, thanks to the laser cutter and StewMac's fretboard calculator. But before we cut any wood, we're going to have to shape it into a rounded shape.

Most electric guitar fingerboards have a "radius".

What we need is a concave sanding block - something to run along the fingerboard blank to create a slightly domed surface. There are numerous articles online about creating a sanding block with a specific radius, but almost all involve marking blocks of wood, planes, spoke-shaves, rasp files and/or heavy duty belt sanders.

Of course, with access to a laser cutter, it only made sense to use that to create our curved block.

Our first idea, using 6mm dowelling rods and laser-cut mdf was a bit tricky to implement. So Steve suggested slotted struts. A far better idea!

Using the same princple as the "living hinge" we part-cut some slots into another piece of mdf, to create a curved surface to accept the sandpaper. Even with the cuts in the curved section, it required a lot of force to hold it in place against the curve of the sanding block

After hunting around for something with exactly a 13.5" radius curved surface to strap the block onto (to act as a clamp while the glue dries) Jake came up with the ideal solution - a "negative" of the sanding block.

Now we've something to clamp against the underside of our concave sanding block, it's just a case of applying plenty of PVA wood glue and clamping the whole lot together.

The bench vice was perfect for clamping the whole assembly for about 40 minutes while the PVA wood glue dried.

This creates one continuous strip of a curved surface, with a 13.5" radius along the entire length of the block, secured every centimetre or so, to ensure a continuous, consistently curved surface - just the thing for making a curved, sanded surface!

It was only after completing our first sanding block and trying it along our guitar neck, we realised our first mistake:

We settled on a 13.5" radius because I have a Yamaha Pacifica which I think plays particularly well. Since no-one else had any other preference, we went with that. While it's quite likely that the radius on the fingerboard of our existing guitar neck is not exactly 13.5" we seem to be quite a way off the mark here.

It was only after double-checking the Inkscape design files, we realised the pattern had been drawn with a circle with a 13.5" diameter - making the radius half of what we thought it was, 6.75"

In the 50s it was common to find guitars with rounder, smaller-radiused necks. The rounder neck makes them easier to play chords with (and, consequently, more difficult to play leads and single-note licks). Early electric guitars were mainly accompanying instruments, rather than instruments in their own right, so a lot of players preferred the rounder necks, to make chord playing easier.

It was only really when lead guitar playing became popular in the early 60s onwards that players started to look for flatter, larger radius guitar necks. Even the most "classic" of electric guitars rarely had a radius much smaller than about 7". It turns out we'd created a perfectly curved sanding block that was no use for modern guitar playing!

Luckily, having perfected the technique of making the sanding block, relatively quickly and easily, it was just a matter of modifying the design files and laser-cutting some more mdf. The second version looked like a much closer fit:

With our sanding block built, the next job is going to be making the fingrboard blank.
Files for creating a 13.5" radius sanding block:

Radius Block 13-5in With Struts and Negative

Testing a resin cast guitar neck

Already we've discovered that our one-piece cast-resin neck isn't going to be hardwearing enough to use as a playable guitar (even just scraping a taught guitar string across the frets causes tiny little shavings to peel off).

But the smokey-black finish does create a very impressive finish! At BuildBrighton this week, we took our RGB LED lights and our black resin-finish fingerboard and quite literally stuck the two together.

The dark resin completely obscured the underlying PCB but the LEDs were plenty bright enough when illuminated, giving a really impressive finish. The "twinkle" sequence, with fading LEDs, was particularly nice!

Resin cast guitar neck part IV

Undeterred by the setbacks of the previous casting, we were determined that casting a guitar neck has got to be far easier than building one from scratch. Since the whole point of creating a custom guitar neck is to put our "guitar tutor" fret lights behind it, a solid wood neck is out of the question (though it wasn't until we'd already bought a new controller board for our desktop CNC and half-built a milling rig to try it anyway).

Sure, we could probably route some channels in the back, and add some holes in the front and let the LEDs shine through. We could probably even fill the holes with coloured resin and  sand it all smooth - but ultimately it would still look like what it was; a wooden neck with a load of holes drilled into it.

What we were after was a quick and easy way of creating a repeatable fingerboard. Which led us to ask "how hard is cured resin?"

Since the major problem with our last resin casting attempt was the fret wires (the bubbles were also a problem, but that was down to poor application when filling the mould, not because of the actual idea) we started to ponder - what if we cast the entire fingerboard, with the frets as  part of a single-piece of resin?

There seemed only one thing to do; we already had a mould with indentations for frets - instead of putting fret wire in and pouring resin on top, why not just fill the entire mould with resin, and see how it turns out?

Bubbles are a concern, but with a bit of care, we should be able to eliminate them. We dashed off to the local art suppliers and paid through the nose for some "craft casting" epoxy resin and some compatible glass-based ink. We added the tiniest bit of black dye to the two-part resin while mixing it up, then poured it - extra slowly - into the mould.

The whole thing looked like a large inky puddle. But as the resin poured from the mixing tub, we could already see it taking on the slightly translucent, grey-black appearance. If we could shine our LEDs through that, it would look great!

After 48 hours we tried to turn out the mould.

Maybe it was the dye additive (although the craft shop was insistent it was compatible with the - overly expensive - resin kit we'd also bought). Or maybe this particular resin needed more than 48 hours to cure properly. Maybe it's because we didn't add any fibreglass this time. But it was quite alarming when the entire fingerboard bent along with the rest of the mould - similar to how a stick of rock bends after you've sucked one end, forgotten about it, and left it in the window for about eight weeks.

This one was going to take a little longer to cure than our first attempt!

After another two days to cure fully,  the black fingerboard turned out relatively solid. Even after all this time, the "back" (or top, as it was when cast in the  mould) feels a little bit tacky if left in one place for any length of time!

The level of detail captured, however, is incredible.
On our original fingerboard, we'd placed some pencil crosses, as we had originally planned on drilling and CNC milling the actual wood. Not only has the silicone mould captured the grain of the wood on the surface (we're still split as to whether this is Really Cool or A Bit Shite) it even captured the pencil markings (the pencil marks are not gouges in the wood, just very faint lines - almost invisible now of the wooden original).

We're going to have to leave this for another few days to go off fully, before fixing it to a guitar neck and seeing if cast-resin frets are actually playable (or whether they just wear off after a couple of two-step string bends!)

Full colour guitar LED neck with WS2812B RGBs

While we've been casting and waiting for resin to go off, we've also been looking at our guitar LED system. It works quite nicely, but it somehow seems to be lacking that "wow" factor. Or maybe it's just because until now, we've not really come up with anything "new" - just recreated existing guitar light systems, like the FretLight guitar.

It was while pondering how to really make our effort stand apart from the crowd, that someone suggested a full-colour RGB system.

The good old WS2182B RGB LED. To this day, I can't bring myself to call them "neopixels". I think the idea of "neopixels" along with the Arduino plug-n-play style approach to coding is pretty limiting in a lot of respects. Neopixels are what WS2812B LEDs become when sold through the Adafruit store. But I can't help but make the distinction - in much the same way an AVR ATMega328P microcontroller is not an Arduino (even though it is used in most Arduino Uno boards).

Anyway, with all that out of the way, we finally settled on using an AVR (and, yes, the pre-built FastLED libraries for Arduino) to drive an array of RGB LEDs. But that's only because when I shared my PIC-based code to do the same thing on a 32Mhz 16F1825 a few of the nerds said "if you built it in Arduino, we could share code" (although "share code" doesn't mean "work together on this project" as I often expect it to, and more often means "give me what you've done, once you've written and debugged it, and I'll make demands on what else I want it to do in the future" - but that's a whole other rant about why I personally don't usually bother much with building projects on the Arduino platform!)

So having agreed to "share code" and use the Arduino platform, we set about building a RGB LED matrix to replace our shift-register based, single-colour LED guitar board(s).

The great thing about the WS2812B LEDs is that they are easily daisy-chained, making laying out the circuit board quite easy - even for etch-at-home, single-sided boards.

As the LEDs could end up drawing a bit of current (each one up to 60mA at full 100% white brightness) a single, snaking power line is unlikely to be able to handle the current requirements. So we redesigned the board, added some through-holes and ran some big wide copper strips up the back!

Big, wide, copper strips are connected to the front of the circuit board....

...by using some 1mm PCB rivets and plenty of solder!

A bit of Arduino-style copy-and-paste coding, and the LEDs were soon lit up

(so far we've only soldered up six out of a possible nine "frets" or columns of LEDs, but enough to prove that the concept works - the circuit design is then repeated across a number of sections to create one, large, fingerboard-length circuit.)

We added an LCD screen and rotary encoder, and built a simple menu system to allow the user to select from a number of options, including setting the brightness, the default LED colour, serial baud rate and so on.

We also added a serial/I2C eeprom chip, so that we could store multiple "patterns" to display on the LEDs. A simple protocol allows us to define a 16-colour palette, and then display any pattern (or image if you want to think of them like that) on the LEDs. Here's an early development version being tested:

Current functionality includes:

• change overall brightness
• set the key and scale type (major/minor)
• change between midi and serial data mode
• set serial baud rate (most common baud rates supported)
• load and display patterns from external eeprom
• play "twinkles" animation

also included (but not shown in the video) are:

• allow two or more patterns to be displayed on top of each other
• change between full-colour and monochrome (single, default colour, mode)

Colour patterns and images are currently stored on an external eeprom chip and are displayed by selecting them from the menu. Image/pattern data can also be sent to the device via serial at a number of different baud rates - this would allow an animation to be controlled by an external device (PC or other microcontroller) and sent to the guitar fretboard, allowing interactive, animated patterns to be displayed along the neck during a performance.