Creating a Low-Level Serial Protocol

When developing systems of any complexity we’re tasked with either making multiple modules inside a system talk to each other, or the system itself talks to the outside world. It’s at this point that we need to develop a protocol to be able to send any amount of coherent data back and forth. Whether you’re using SPI or UART for this specific application there’s a couple common questions you need to ask yourself.

• How do I make it flexible?
• How do I make it robust?
• How do I make it fast?
• What’s the simpliest way to do this?

A proposed protocol

The basic algorithm for using this would be:

1. Spool incoming serial data in interrupt until the START DELIMITER is found.
2. Read in the next two values to determine our length, set this as a local variable.
3. Read more data into the buffer until our length is matched.
4. Determine if the END DELIMITER is accurate.
   • If it is, memcopy into buffer and raise a flag to process in main loop.
   • If not, zero counters and return to the beginning

In a separate function, called from the main loop we can then parse our packet:

1. Extract payload from the buffer.
2. Calculate the CRC and compare with the CRC in the packet
   • If it matches, proceed
   • If they do not match, abandon, the packet is malformed
3. Process Primary switch statement that branches out the code based on input command.
4. Assemble return packet and reply

NOTE: It’s important to do this in a separate function and to use multiple buffers. This way the system can be reading in the next packet while we’re processing the previous. Using multiple buffers also prevents writing over data from a new packet while you’re still completing the previous command.

Does it meet our goals?

Flexibility

The flexibility of the protocol lands on the use of the length bytes at the beginning. As long as this is specified we can create up to 256 commands that retrieve or process any amount of data you want. If all of the buffer sizes and indexes follow the read-in values then you can freely add commands as you see fit.

Robustness

Going back to the length commands, we’ve added robustness by being able to specify any command or length value. If the checksums match then we simply look for the command in our supported list and if the firmware does not have support for that command we can reply with an error that the command is unrecognized. This adds protection from valid commands that are not yet supported

As for data integrity, we have to pass through multiple gates in order for the packet to be considered VALID. First we have to have read-in the appropriate amount of bytes, then we’ve validated a FRAME by detecting the START and END delimiters. After this we calculate the location of the CRC and compare it. After this we still haven’t parsed the individual command yet. The data can then still be rejected if it’s out of bounds or formatted wrong.

Speed

Speed is gained by processing all of this in bytes instead of ASCII. This allows us to minimize the amount of data transmitted. We also gain some speed advantage by copying our possible packet into a separate buffer as soon as we’ve read it in. We’re able to read-in the next while processing the current packet.

Simplicity

I find this method to be remarkably simple. There’s very little tinkering with the base protocol format after it’s in place and you can add or remove supported commands by only changing the code in the main processing function.

Conclusion

This may not be the end-all-be-all of serial protocols, but it’s a proven true, simple, and flexible approach.