Sunday, April 22, 2012

Why Forth still matters in this ARM/Linux/Android world

My sensors (for my home monitoring system) need to run off of coin cell batteries (the sensors need to be tiny). They also need to run for at least 1 year (under normal circumstances).

The sensor transceivers consume around 15mA when transmitting. This is a lot for a coin cell battery.  The general sensor development strategy is to use communication sparingly: If you don't need to broadcast data, don't.

With a battery that can only source 200mA for just one hour, you really need to start thinking about low power design. An ARM/Cortex capable of running Linux consumes a lot of current. You are going to use a low power 8 or 16 bit processor (e.g. Silabs 8051, MSP430, etc).

Now add C and some libraries to the mix. The more generic/portable stuff you do, the longer the processor is going to stay awake. While it is awake, it is consuming power. The longer it "sleeps", the longer your battery will last.

So, I am using a very low level Forth (MyForth) on a Silabs 8051 low power processor. MyForth is essentially a high level macro assembler -- there is very little overhead to support Forth.

There are no libraries that come with MyForth, so I had to roll my own code.  This keeps me true to the Forth philosophy (or at least Chuck Moore's philosophy) of only doing what you need to do to meet the task at hand. I am not writing "generic" libraries. I have to fully understand the devices I am interfacing with.  I doubt that I could write tighter/faster code for my transceiver -- I am engaging it at a primitive level. No abstractions but the ones that MyForth provides at a (mostly) macro level.  Forth, like Tcl, is more idiom reuse oriented -- you can do a lot with just a little code.

If I need to work harder to make sure that my sensor works 24/7 for at least a year (or two!) off of a coin cell battery, then I will.

My stove monitor (temperature + motion sensor) runs on a Silabs C8051F912. That MCU has 768 bytes of RAM and 16KB of Flash. I am talking to an RFM12B transceiver over a SPI interface and controlling a low power IR motion sensor module.  Currently, I am broadcasting motion and temperature over the air using less than 128 bytes of RAM (including Forth stack) and less than 4KB of Flash. I spend only a few seconds awake, a most of the time asleep consuming less than .001mA of battery current.
You can do this in C (of course!), but I don't think you could reach the code density (especially as you start using third party libs).  I can also hand tune my code directly in assembler where needed.

Yep, Forth still matters to me.

Does the Roku need Erlang/OTP?

My Roku streaming player locks up every once in a while (every couple of months).  It seems to do this in two different ways:  Sometimes it seems to get confused about the wi-fi connection and sometimes it just freezes in the middle of a movie.  We are heavy Roku users in my household, and it works flawlessly most of time, so it is hard to get a good idea why this is happening (there is no consistent scenario).

So, here is an embedded device that is suppose to work 24/7 and my only recourse when it locks up is to unplug the power supply and plug it back in.  There are no buttons on the unit itself to "reboot" it.

This kind of device is ripe for the "Erlang" approach.  Now, of course, if you've been doing embedded firmware for any decent length of time, you'd realize that you don't "need" Erlang to fix this.  Watchdog timers, process monitors, a soft-reboot button, etc are the first things that come to mind.

But, a lot of the software that run on these set-top boxes seem to come the "other side" (non-embedded developers). I have no proof of this, but the lock ups smell of that kind of development mentality. When I use to build highly available internet servers systems, I always made sure that I had terminal access so that I could log in and kill/restart stuff to fix problems.  I knew that my stuff needed to run standalone, but I also knew that I could log into a running system, look around at what the problem was, restart stuff and take the fix back to development for the next release.

You don't get any of that with "appliance" devices.  So, the more I did embedded work, the more I developed a "no login; no logs" mindset.  Stuff needs to run, damn the logs.

This year I had to do some Cloud apps. I used Erlang/OTP.  I thought I caught all of failure conditions but some third party code would fail every once in a while.  The system would run for a week or two, but then mysteriously crash. Thank goodness for the logs. I logged in and reviewed about 1MB of logging to find the problem.  I fixed it, uploaded new code and restarted the servers.

This doesn't work for a Roku.  Once it ships, there is no developer login.  The device must never lock up.  The user must never lose control.  Even in the need of a full reset, the user should be able to do this from the couch.  All processes (and devices) must be monitored.

My home monitoring system base station is currently using Erlang/OTP -- not because Erlang solves these problems, but because Erlang/OTP was designed to solve these problems.

Saturday, April 21, 2012

Where is my jetpack?

I'm getting old, and I am getting impatient with technology.  Things are getting smaller, sleeker and sexier -- but they aren't getting any smarter.

Where are the big ideas?  Why are we still writing web application frameworks? Why are we porting Linux to anything that has a microcontroller and going "look! Isn't that cool? It's running on my watch."

Where is my jetpack?
Or, more relevant to computer technology:

Where is my Dynabook ? (No, an ipad isn't a Dynabook, and the XO is too cumbersome.
Where is my Hitchiker's Guide to the Galaxy? (The WikiReader is close, but it needs graphics and a voice reader)
Where is my "House of the Future"?  (It costs way damn too much and has no coherent operating "vision")

Yes, I know, I am bitching and moaning. Why don't I do something about it?
Well, that is why I am working on the "House of the Future" (my home monitoring project).
I'm thinking of forking this blog to document my progress.

Saturday, April 14, 2012

Counter Point (Bluegiga BLE112: A Game Changer?)

After writing the last blog entry, a thought kept re-occurring: "Mission Accomplished."  You may have immediately caught on to the reference: President Bush's 2003 Mission Accomplished speech.

I certainly don't want to trivialize the aftermath of that speech (are we really done yet?), but  it does remind me of the faith we (the software community) place in abstractions and generalizations.

A scripting language onboard the BLE112 and so it will only take a few lines of code to do my sensors?  Sounds great.  But, there are no reports from the ground yet.  Is the scripting language stable? Are there bugs hidden in the implementation?  Are there subtle things that it does automatically that will bite me in the ass?

The same questions can be asked about Embedded Erlang. It sounds fantastic, and of course the first major Erlang deployment was indeed embedded (the AXD301 switch). But a lot has been added to Erlang and OTP since then. Is all of it good? Is all of it stable and proven?

This is not a criticism of BLE112 scripting or Erlang, but a reminder of why I went "low level" 6 years ago in the first place: The abstractions and libraries were killing me.

A thought: My Roku locks up every couple of months.  I have to hard reboot it.  I am guessing that it isn't a hardware problem nor a problem with the OS (linux?). Somewhere an app is failing.  My home sensor system can't do this. It must work 24/7 for as long as it is powered.

While Erlang has a great approach to fault tolerance, it is not the complete answer. You must intelligently use it's recovery features and you must avoid software faults to begin with.

"Not invented here" is preached against, but if you can't trust the provided tools, what do you do?

I'll have to see if Bluegiga's scripting is solid. But, what do I do if the sensors start silently failing weeks (or months) after deployment. How do I debug it?

In order to get my Erlang base station up and running I will need my Erlang code to interface with a serial port. There is no support in Erlang to do this.  I was going to write one (or use "netcat") to create a TCP bridge for the serial port and let Erlang talk to it. Will this eventually introduce subtle problems? How do I manage the bridge?  The Erlang approach says to expect that the bridge will fail and simply restart it (and resync the rest of the processes with it).  This is a smart approach, but with the "bridge" I have left the Erlang eco-system.

All of these issues nag me, and they should.  On the one hand, I think that the embedded world (mostly hardware types writing assembly, C and maybe C++)  is way behind the "high level programming" world in providing flexibility.  But with that flexibility, software folks have brought with them all of the little devils that come with high level programming.

Food for thought.

Bluegiga BLE112: A Game Changer?

Bluegiga has introduced a Bluetooth LE (Low Energy) module: BLE112
Their approach: Add battery, SPI|UART|analog  based sensor, write a script and you've got a running sensor node.

This is great. It could save me a lot of work: Most of the sensor-side work is done for me.  They have introduced a scripting language that runs on the Bluetooth module (A TI CC2450 -- 8051 core).  It is a very low power design with great idle/sleep consumption savings.

Bluetooth LE wants to supplant ANT and Zigbee. It wants to do what ARM (Cortex) has done to the portables market.  This Bluegiga module is so far the most impressive implementation I've seen so far (on paper).

If this module does what I want for my home sensor network, then I can focus on the base station.  This is where things get interesting anyway.

I'm think more about using Erlang there. I think the time has come to introduce robust server technology to the embedded world.  This presentation agrees.  Exciting times ahead :-)