Friday, May 25, 2012

A smattering of MyForth: CRC16 checksums

I talk a lot of about using MyForth in this blog.  MyForth was written by Charlie Shattuck as a minimalist 8-bit Forth for the 8051.  As an 8-bit Forth it doesn't have a lot of math capabilities (how much math can you really do in just 8 bits?).  However, I've added a few primitives to help manipulate 16 bits (which represents as two 8 bit numbers on the stack, with the MSB topmost).

So, I needed to compute a 16 bit checksum.  The Silab C8051F930 has a built in CRC engine, but I wasn't using that particular MCU.  However, I wanted to maintain some compatibility across Silab chips, so I looked at the C8051F930 datasheet and found a C implementation of the built in CRC checksum.

Here is the basic C function (abridged):
#define POLY 0x1021
unsigned short crc (unsigned short CRC_acc, unsigned char CRC_input)
  unsigned char i;
  CRC_acc = CRC_acc ^ (CRC_input << 8);
  for (i = 0; i < 8; i++) {
    if ((CRC_acc & 0x8000) == 0x8000) {
      CRC_acc = (CRC_acc << 1) ^ POLY;
    } else {
      // if not, just shift the CRC value
      CRC_acc = CRC_acc << 1;
  return CRC_acc;
And here is my MyForth translation:
$1021 constant POLY 
: crc-xor-poly ( accum16 -- accum16 )
    -if d2* POLY ## dxor ; then d2* ;

: >crc ( accum16 c -- accum16 )
    0 # swap dxor
    8 # 2 #for crc-xor-poly 2 #next ;
I added a couple of helper functions to MyForth to deal with the 16 bit numbers:
: dxor rot xor push xor pop ;
: d2* swap 2* push 2*' pop swap ;
I don't expect you to understand the code (and I will not attempt a detailed explanation here), but I thought it would be interesting to show what minimalist Forth code can look like.  

It may be worth pointing out a few things:

That the stack is 8 bit wide and 16 bit numbers are pushed as two 8 bit values (MSB topmost).  

The "-if" is a quick way to check if the 7th bit is set on the MSB (hence negative). 
"If" condition values are not consumed off of the stack, so that allows me to test and use the top value without a "dup".

The weird looking "for" loop (" 2 #for") indicates that I am using Register 2 to hold the loop value. Yes, I have to do book keeping on registers (remember the days of old?), but it isn't bad as it seems.  Deeply nested loops are frowned upon in Forth.

There is certainly a brevity and density to the MyForth source. ;-)

Wednesday, May 23, 2012

Beaglebone and RabbitMQ and Erlang/OTP...really?

So, what good is a 24/7  home sensor base station if it loses messages?

I originally planned to run the beagle board based station as an AMQP client talking to a cloud server where data would be correlated and presented to users.  Unfortunately, from a design perspective, this doesn't address temporary "internet outages". What does the station do with a bunch of events when it can't reach the cloud server?

Well, how about running RabbitMQ on a Beaglebone?  This local instance of RabbitMQ would act as a cache and shovel events/messages to the cloud server RabbitMQ when it is available.  By using a local "work" queue, I can utilize a different delivery path in case the internet connection has gone south (power outage?) for a pre-defined duration. This backup path could be a cellular modem.

 I am using the nerve erlang distribution and after building a bunch of supplemental Erlang packages, I was able to get the latest RabbitMQ (2.8.2) running on the Beaglebone.  I haven't put it through its paces, but it does seem to run the management plug-in and I can navigate it with a web browser.

Resource-wise, the "freshly" launched RAM footprint is around 30MB.  Performance may be an issue, but it looks like I have some room for at least a few sensor event messages.  And, I don't need spectacular performance, I need reliability.

I will put it through some stress tests, but I like the overall architectural approach: Erlang/OTP and RabbitMQ in the home base station.

Thursday, May 10, 2012

Bluegiga BLE112 + Beaglebone + Erlang = ?

So, I got a Beaglebone yesterday.  I am considering it as the potential base host for my home monitoring system.  After verifying that it would boot, I downloaded a Buildroot based Erlang image from .   So far, so good.

I plugged in the Bluegiga BLE112 bluetooth USB dongle and it was correctly recognized as a serial port (/dev/ttyACM0).  So far, so good.

So, would it take my initial base software (just a BLE112 dongle test)?  I am using Feuerlab's serial port library (as recommend to me by Ulf Wiger in a comment to my previous BLE112 blog entry).

So, I downloaded and built the Buildroot environment for nerves (mainly to get the ARM cross compiler installed with the uclibc library).  So far, so good.

The serial port library compiled without a hitch (after setting TARGET_SYS to my ARM compiler suite). But, it couldn't be that simple, could it?

Copied my sources (including the compiled serial library) to the microSD, booted Beaglebone and gave it a try. It worked.

Today was a good day.

Now, here is the bigger task at hand: Determine if a 256MB RAM ARM is sufficient to do some serious Erlang work (maybe with some OTP too?).  I've seen the neat tricks (running Erlang on Raspberry Pi), but how about something more than blinking lights?

Stay tuned.

Saturday, May 05, 2012

Ignoring the wheel: Taking non-mainstream approaches to embedded design

As I look at the documentation page for the BLE112 (Bluegiga's Bluetooth LE module), I am reminded of how complex computing has become. There are videos, spec sheets and software guides (well over a dozen documents not including slick sheets and qualification documents).  All of this for a small embedded device that uses and 8051. This is all "high level" documentation.  At the end of all this you can develop your BLE112 comms  using an API or their own scripting language.

I understand the complexities involved and why there is so much documentation.  But that isn't all: Add to that all of the Erlang OTP stuff I am planning on doing on the server side.  There is lots of documentation; lots of "other people's stuff" I need to master.  

Sometimes I "ignore the wheel".  This is like "re-inventing the wheel" but tends to avoid the actual construction of a wheel itself.  "Ignoring the wheel" is about coming up with your own means of transportation.

Rather than use a mainstream approach, I roll my own solutions.  This usually means ignoring an already written body of software (libraries), but when I start from scratch, I intimately understand everything I am working with.  I *have* to intimately understand everything I am working with.

Now, one trade-off that must be made for "ignoring the wheel" is that you have to get creative with your resources.

Let me propose an example:  Building a (pro quality) Data Logger the size of your thumb.  Now, this data logger must read some arbitrary sensor every 10 seconds and log it to persistent storage until it is later pulled off of the device and analyzed later on a PC.  It must be able to capture tens-of-thousands of  short  (10 bytes?) log events. The whole thing (case and battery included) should be about the size of your thumb.

How would you approach designing it?  (Remember, this should be a "pro quality" logger, not a toy -- it must work flawlessly.)

Linux on an ARM?  Small, but probably not small enough and I doubt you could run it off a small (coin cell size) battery -- power consumption is an important factor here.

Okay, maybe something Arduino-like.  Or maybe a PIC, or even a Silabs 8051. You want something tiny that uses very little power.  The tiny MCUs tend to have less RAM and Flash program space, but this is just a simple data logger, right?

Now, what about the log storage medium?  A microSD card is small. Plus, you can take it out and plug it into your computer to dump the data.  This sounds great.

So, you go with a microSD.  Now, of course you'll have to format it as FAT16 or FAT32 to make it readable by the PC (besides there are plenty of FAT libs for MCUs out there, right?).

Now you have a problem: FAT is simple, but still requires choosing a library and getting it to compile. Plus, you'll need (at least) 512 bytes of RAM to hold sectors/buffers. You did pick an MCU with more than 512 bytes of RAM right?

Is the FAT implementation reliable? Is it rock solid? Can you trust your important logs to it?
Now, how do you arrange the logging? Will you exceed the maximum FAT file size? How do you name the file?

Remember the original goal: You are building a data logger, not a database.

Okay, you get the picture. For hobbyist needs, a simple logger (like OpenLog) fits in a pinch. But things start to get complicated when you consider reliability and longevity.

How can we simplify this design?

First, do you really need FAT?  Can you develop a custom "log system" that writes to the "raw" microSD and develop a reader on the PC side?  Do you really need the flexibility of a "file system". Consider this: Figure out the max size of a log entry (typically a logger works with structured sensor data: GPS, environmentals, etc). On a 2GB microSD you can fit around 10 million 20 byte logs. Is that adequate?

Second,  do you really need a microSD? What if you used a serial Flash storage chip?  Maybe one that doesn't require 512 byte RAM buffers for sector writes. Atmel makes a family of serial flash chips that have "on board" pages that you can randomly access before committing to sectors.
Maybe you can pull the data off serially with a cable, or maybe via wireless?

Here is an interesting observation I've made about "Ignoring the wheel" in my own designs:  It reinforces the XP tenent: "do the simplest thing that could possibly work". Because I deal a lot with anemic microcontrollers (8051) and minimalist languages (Forth), the ordeal of supporting FAT on a microSD makes me question whether or not it is "needed" to log a bunch of data as quickly and reliably as possible.

If the customer says I need to use a microSD, okay. But what exactly are they expecting for the FAT support?  Could something like this ( work?  It only supports 1 fixed size file at a time, but it is very simple. Essentially, all I would need for a data logger is "write" capability to the filesystem. Why have a full FAT implementation on a tiny (write only) logger?

Can I ignore the wheel and simply do something simple?