Simplified serial string stuffing with PIC and Oshonsoft

If there's one thing we love more than alliteration at Nerd Towers, it's simplifying repetitive tasks. One thing that the Arduino platform, and it's library-based approach to coding has in it's favour, is that it makes prototyping common/simple projects quite easy.

Take serial communications, for example.
Serial.begin(9600) opens the hardware serial port at 9600 baud.
With PIC and Oshonsoft we have HSeropen 9600.
Different syntax, but the same result.

In PIC-land, it's the same as setting the appropriate BAUDCON and SPBRG registers, to get the right baud rate for the current crystal/clock speed. The Arduino IDE is probably doing something similar.

Getting data is pretty easy on both platforms, but where Arduino really shines, is it's string-handing ability. And in particular, the relative ease with which you can build up a string, from incoming serial data.

String s="";
While Serial.bytesAvailable(){
  char b=Serial.readChar();
  s+=b;
}

With PIC and Oshonsoft, we've always just dumped the data into a large buffer, and processed the incoming data. The downside to this approach is that you effectively need to stop all serial communications which you work on the received data. And this isn't something you want to do each time. There's no equivalent of Serial.bytesAvailable. And that's a shame, because, for prototyping stuff quickly, that's a really useful function.

And after messing about with those wifi modules the other night, what we really need is a quick and easy string handling routine (that can handle data coming in at no less than 115,200 bps). So we came up with an idea which, after looking at the source code for the hardware serial library in Arduino, is very similar to the Arduino implementation.

Basically, we're going to keep a "circular buffer".
This means we've got an array of 80 bytes as our buffer. We keep track of which element of the buffer we're going to write any incoming data to (called the "head" of the buffer) and which element we're going to read data from (called the "tail").

The idea is we can use an RX-data-in interrupt to write incoming data, one byte at a time, into the circular buffer, without affecting where we're reading the data back from. Whenever the RX interrupt fires, we advance the head pointer one element along the array, and store the incoming byte into that location

It's called a circular buffer because when either of the pointers need to advance beyond element 79 (the last element in an 80 byte buffer) they reset back to zero and continue filling the array.

When we want to see what data we've received over serial, we read the data out of the buffer, starting from the tail. While we're reading this data back, it's quite possible for more data to be coming in over serial, which in turn gets written to the buffer, advancing the head pointer.

Because of this, it's quite possible that the head pointer can be ahead of the tail (as is the case for the first few bytes received). But after time, it's also possible that the tail has "moved along" a few bytes, and the head pointer has "wrapped around" back to zero, and is "behind" the tail pointer in the array. This is normal.

The only time we have a problem is if the head pointer tries to "cross over" the tail pointer.
When this occurs, we've had a buffer overrun. i.e. we've received more data into our buffer than we've managed to read back out in the time it's taken to fill up (i.e. we've received 80 bytes more than we've read out of the buffer). There are two ways this could be handled - simply stop advancing the head pointer, and any further data received is lost; or advance the tail pointer just ahead of the head pointer, retaining the newest data, and overwriting the older data.

Whichever approach is used will result in lost data, and it depends on the application as to which data has a higher priority and should be kept. For simplicity, we're using the first approach - if we've received a full 80 characters into our buffer and not managed to read them out quickly enough, we keep reading the serial data from the UART module (thus avoiding a true, hardware buffer overrun which would disable the serial peripheral on the chip) but we don't store it anywhere. It's just as easy to amend the code to retain the newer data by overwriting the old data and advancing the tail pointer.

The advantage to this approach, rather than dumping incoming bytes into a "regular"  linear array, is that it allows us to monitor the incoming data for a particular character or combination of characters. If we're looking for the word "HELLO" in our data, we'd have to traverse the array looking for

array(i+0)="H" AND array(i+1)="E" AND array(i+2)="L" AND array(i+3)="L" AND array(i+4)="O"

which can get quite cumbersome. Far easier to read back the contents of the array, until a specific character (such as a line break or carriage return) is hit, and process the contents as a string of data. While processing each individual line of (ascii) data received, the interrupt-driven RX routine is  still populating the rest of the buffer - something we couldn't do if we had to process our array of data immediately after receiving the character or byte acting as a delimiter in our data.

Taking all these ideas and putting them together, we've written a simple serial buffer routine that stores data coming in and sets a flag when a particular character (in the example, it's a carriage return) is received into the UART receiver.

In our example, we demonstrate how processing the data can continue outside of receiving characters on the serial UART peripheral; our example code can detect if a line in the serial data contains the word "OK" (and flashes an LED if it does) or sends the data back out to the serial TX (for debugging) after a delay of about a second. Of course, this doesn't do much on it's own, but it's a great start, for parsing incoming ASCII messages from the wifi module; especially because responses are often written as AT type commands, one on each line, with either OK or ERROR written on it's own line to indicate the success (or otherwise) of each command sent to the wifi module.

It's not quite as elegant as the Arduino's while Serial.bytesAvailable() function - but it's far easier than either handling characters as soon as they arrive at the serial RX port, or traversing a byte array of characters, to see which words they make up!

Define CONFIG1 = 0x0984
Define CONFIG2 = 0x1cff
Define CLOCK_FREQUENCY = 32
Define STRING_MAX_LENGTH = 79
AllDigital

symbols:
     Symbol led_pin = PORTC.2
     Symbol tx = PORTA.0
     Symbol rx = PORTA.1
   
declarations:
     Dim serial_ready As Bit
     Dim serial_buffer(80) As Byte
     Dim serial_head As Byte
     Dim serial_tail As Byte
     Dim serial_rx As Byte
     Dim serial_string As String
     Dim serial_delimiter As Byte
     Dim serial_ignore As Byte
     Dim i As Byte
   
configpins:
     ConfigPin led_pin = Output
     ConfigPin rx = Input
     ConfigPin tx = Output
   
initialise:
     'set the oscillator to 32mhz internal
     OSCCON = 11110000b 'oscillator = 32mhz
     WaitMs 50 'wait for internal voltages to settle

     'put the tx/rx lines onto the programming pins for easy development!
     APFCON0.TXCKSEL = 1 'tx onto RA.0
     APFCON0.RXDTSEL = 1 'rx onto RA.1
   
     'create an interrupt on the serial receive.
     'as data comes in, add to the circular buffer
     PIE1.RCIE = 1
     INTCON.PEIE = 1
     INTCON.GIE = 1
   
     'blank the buffers
     For i = 0 To 79
           serial_buffer(i) = 0
     Next i
     serial_string = ""
     serial_head = 0
     serial_tail = 0
   
     'open the hardware serial port
     Gosub serial_open
   
     'testing:
     'flash the led to show we're alive
     For i = 1 To 4
           High led_pin
           WaitMs 500
           Low led_pin
           WaitMs 500
     Next i
   
     serial_string = "Ready" + CrLf
     Gosub serial_send
   
     'if we're receiving ascii data over serial, each line ends with CrLf
     serial_delimiter = 0x0d
     serial_ignore = 0x0a
   
     'prepare the serial buffer
     serial_ready = 0
   
loop:
     If serial_ready = 1 Then
   
           serial_ready = 0
   
           'testing:
           led_pin = Not led_pin
         
           Gosub get_serial_string
         
           If serial_string = "OK" Then
                 For i = 0 To 4
                       High led_pin
                       WaitMs 100
                       Low led_pin
                       WaitMs 100
                 Next i
           Else
                 'testing:
                 WaitMs 1000
                 Gosub serial_send
           Endif
     Endif
Goto loop
End

On Interrupt
     Save System
   
     If PIR1.RCIF = 1 Then
   
           'get the byte from the rx buffer to clear the RCIF flag
           serial_rx = RCREG
           If serial_rx = serial_delimiter Then
                 'this is a carriage return (or a zero-byte) character
                 serial_ready = 1
           Else
                 If serial_rx = serial_ignore Then
                       'this is a linefeed character so ignore
                 Else
                       'store the received byte in the serial buffer
                       serial_head = serial_head + 1
                       If serial_head > 79 Then serial_head = 0
                       If serial_head = serial_tail Then
                             'crash!
                             'don't put any more data into the buffer until you've read some back
                             '(or maybe empty the older data first?)
                       Else
                             serial_buffer(serial_head) = serial_rx
                       Endif
                 Endif
           Endif
     Endif
Resume

get_serial_string:
     Dim buffer_start As Byte
     Dim buffer_end As Byte
     Dim buffer_end2 As Byte
     Dim k As Byte
   
     serial_string = ""
     buffer_start = serial_tail + 1
     If buffer_start > 79 Then buffer_start = 0
   
     If serial_head > serial_tail Then
           'this is easy
           buffer_end = serial_head
           buffer_end2 = 0xff
     Else
           'the head has reached 80 and wrapped around to zero
           buffer_end = 79
           buffer_end2 = serial_head
     Endif

     For i = buffer_start To buffer_end
           k = serial_buffer(i)
           serial_string = serial_string + Chr(k)
     Next i
     If buffer_end2 <> 0xff Then
           For i = 0 To buffer_end2
                 k = serial_buffer(i)
                 serial_string = serial_string + Chr(k)
           Next i
     Endif
   
     'remember where to start reading from next time
     serial_tail = serial_head

Return

serial_open:
     BAUDCON.4 = 0 'SCKP synchronous bit polarity
     BAUDCON.3 = 0 'BRG16 enable 16 bit brg
     BAUDCON.1 = 0 'WUE wake up enable off
     BAUDCON.0 = 0 'ABDEN auto baud detect

     TXSTA.6 = 0 'TX9 8 bit transmission
     TXSTA.5 = 1 'TXEN transmit enable
     TXSTA.4 = 0 'SYNC async mode
     TXSTA.3 = 0 'sednb break character
     TXSTA.2 = 0 'BRGH high baudrate
     TXSTA.0 = 0 'TX9D bit 9

     RCSTA.7 = 1 'SPEN serial port enable
     RCSTA.6 = 0 'RX9 8 bit operation
     RCSTA.5 = 1 'SREN enable receiver
     RCSTA.4 = 1 'CREN continuous receive enable

     SPBRGH = 0 'brg high byte
     SPBRG = 51 'brg low byte(see datasheet For 9600@32mhz)

Return

serial_send:
     Dim k As Byte
     Dim j As Byte
   
     k = Len(serial_string)
     k = k - 1
     For i = 0 To k
           j = serial_string(i)
           TXREG = j
           While TXSTA.TRMT = 0
                 'wait
           Wend
     Next i
   
     'send a carriage return/line feed combo at the end of the line
     TXREG = 0x0d
     While TXSTA.TRMT = 0
           'nothing
     Wend
     TXREG = 0x0a
     While TXSTA.TRMT = 0
           'nothing
     Wend
   
     serial_string = ""
   
Return