Enzo

Thursday, October 30, 2014

Sensor Networks with Java (Part II)

Hi there,

Getting back from GeeCON Prague and EclipseCon Europe I thought it might be time for part 2 of my Sensor Networks post.
In the last post I've explained how to setup the network and how to use Java to read out the data from the Coordinator Node.

For a session at Java One this year I've created a little demo project that also shows another example on how to connect to a XBee Series 2 using Java. You can find the code for that little demo here.

But now back to my Sensor Network, now that I've setup all the hardware I needed a way to store and visualize the data.

Storing data
To store the sensor data you have different options

  • local
  • cloud

Because a cloud solution was a bit overkill for the kind of stuff that I would like to do and I also like to keep as much data in-house as possible I've decided to go with solution one...local storage.
Well like I've mentioned in the last post I would like to use as many embedded devices as possible (at least because they are small, cheap and I can setup them up in my office easily). The easiest way would be installing a little database on a Raspberry Pi, BeagleBoneBlack or similar hardware and store the Sensor Network data in this database.
But because these devices are using SD-Cards as their hard-drive this would mean I would have to install the database on the SD-Card. In principle no problem BUT SD-Cards are not made for continuous file access which means your SD-Card will get corrupted very soon. One solution would be using an USB memory stick on these devices and store the data on the stick. The disadvantage of this approach might be the speed of the memory stick and to be honest I don't really like the idea :)
So I was looking for an embedded board that has a SATA port where I could attach a real hard-drive. Long story short I've decided to go with the CubieBoard 2 which you could see on the following picture



The CubieBoard 2 is a really nice board with following specs

  • DualCore ARM A7 1 GHz CPU
  • Mali-400 MP GPU
  • 1 GB RAM
  • 1 Micro SD slot
  • 1 SATA 2.0 connector
  • RJ45 Ethernet
  • I2C, SPI, LVDS and more

So with the ability to connect a SATA hard-drive I'll be able to put the database to the hard-drive which will give me more speed. Even if a SSD drive is also not the best solution for a database drive I decided to use one because it's faster and if you take a small size SSD it will be cheap (I'm using a 64GB SATA 2 drive from Sandisk which costs around 40 €).
As database I decided to go with mongoDB but you can use whatever you like. I used mongoDB because there was an ARM port available (which is not really up to date but hey...it works). 
The problem with mongoDB was that you have to build the version on your own, means there is no out of the box binary that you can use. You will find some blogposts on the web that describe how to do that (e.g. here).
After that was done my CubieBoard based database-server was ready to go.
So let's take a look at the current setup, we have the Raspberry Pi that acts as Coordinator Node in my Sensor Network and that get events everytime a XBee is sending data. Then I have the CubieBoard 2 which is running mongoDB and could store the data in that database.
The question is how should the data get from the Coordinator Node into the database?
Well the simplest solution would be using the mongoDB Java driver to store the data directly from the Raspberry Pi over my internal network into the mongoDB on the CubieBoard. But I used a different approach...

Communication
Because all the embedded devices are running on my local network at home I needed to find a way to access my internal network from the outside. I could have open ports for each device and access it from outside but I decided to use some kind of a central communication server.
Therefor I decided to go with MQTT (Message Queue Telemetry Transport). When using MQTT you need a so called MQTT broker that receives all incoming messages and forwards them to the subscribed clients. Lucky me there is a very small free MQTT broker available called Mosquitto. Because this broker is small and doesn't need huge hardware to run it was the perfect fit for my project. My broker is running on an Odroid-U3 from hardkernel which looks like this.



This board has more than enough power to run Mosquitto as you will find out when taking a look at the specs.
So with the MQTT broker in place I had a central communication server where all devices can talk to. 
The Coordinator Node now publishes a MQTT message every time it receives data from a XBee Sensor Node in the Sensor Network.
Because of the Publish-Subscribe model now everybody that is interested in that data can subscribe to the topic and can use it.
Instead of directly writing the data from the Coordinator Node into the mongoDB instance on the CubieBoard 2 I've wrote another small Java program that is running on the CubieBoard 2 and that is listening to data from the Coordinator Node.
With this approach I could also place the mongoDB server where ever I like. But the main reason for the additional Java program was the reason that I can now also do some data preprocessing on the CubieBoard 2 before storing the data to the database which is nice.
So now we have the Raspberry Pi Coordinator Node that receives the XBee data and publishes it via MQTT. Then there is the CubieBoard 2 that is running the mongoDB instance and is listening to MQTT messages from the Coordinator to store the preprocessed data in the mongoDB.

Visualization
Ok now that I can measure the and store the data it was time to create something to visualize the data and because I love Java I decided to create a JavaFX based desktop client to visualize the measured data.
Instead of only monitoring the Sensor Network data I've added more features like getting the current weather conditions from the closest airport, showing weather information from my own weather station outside (because the airport is around 30km away) and more.
But here let's only focus on the Sensor Network Data...
Now that we have all the data available in the database we can use different kind of visualizations, first of all we can visualize the live Sensor Node data which I did like follows



As you can see I use a gauge to visualize the last measured temperature of each Sensor Node (the three gauges without a value are ran out of battery...waiting for new batteries at the moment :)). The color of the gauge will change with the temperature and in addition also the current supply voltage will be visualized with a little battery icon. The color of the battery also indicates the state of the battery.
Well it's nice to see the current situation but I'm interested in the temperature of the last 7 days...so I simply used a line chart to visualize the data from the database like this...




Here you can see the temperature variation of each Sensor Node over the last 7 days which is useful to indicate weather trends etc.
Because we have the supply voltage also available in the database why not visualizing the voltage of each Sensor Node...here you go...



As you can see on this chart I can easily identify Sensor Nodes that will need a new battery soon (e.g. Anton in the chart above). This is very useful because the batteries will last for a long time but when they are going down the drain it will just take a fews days before they will die. So with this chart I can identify weak batteries by looking at the chart once a week.

Because the temperature do not only vary from day to day but also within one day I was interested in how the temperature in each room varies over the day. Therefor I've created a little RadarChart that shows the temperature for each room, here is a screenshot...



As you can see I can select each Sensor Node on the left side and one of the last 7 days on the bottom. With this you can monitor the temperature variation from each Sensor Node on each of the last 7 days which is really interesting some times.

All the controls that you see on the screen shots above are taking from the Enzo JavaFX controls library that you can find here.
Please keep in mind that all controls in that library are made for my personal demos and are not production ready (but might be useful as a start) :)

Because this post is long enough I won't show you all the mobile clients that I've also created like an iOS and Android client and also a SmartWatch client. But in principle they all look like the desktop version and thanx to Gradle I build all these versions for all these platforms in one Multiproject Gradle build which means more than 90% of the code is the same for all platforms...remember the credo of Java...write once run anywhere :)

That's it for today...keep coding...

Tuesday, October 21, 2014

Sensor Networks with Java (Part I)

Hi there,

End of last year I've started to build a little Sensor Network that I've placed in my house. I never thought that many people are interested in that but I was wrong, nearly every time I did a presentation about that topic many people asked me for a blog post about the project...and here it is :)

First of all this is not really about a Smart Home or something similar, this is simply a project to monitor data (temperature in my case). The original idea was monitoring multiple cold stores which seems to be something that is really useful :)
But because I didn't had the cold stores I've decided to measure the temperature in nearly every room of our house.
The things I was interested in have been the behavior of the room temperature compared to the temperature outside. Does the temperature in the rooms follow the temperature outside of the house and how fast does it follow. To be able to compare those values I had to measure the conditions outside and in inside of the house.
Well measuring is one thing but you also have to analyze the data right? This means I needed to store the data somewhere and also needed to visualize the data somehow. Because I'm working from home I don't have a big server room where I can put a computer that acts as a server so one other requirement was to use as many embedded devices as possible in the whole setup (means for communication, storage, etc.).

Wife factor
One very important (if not most important) factor in the whole project was what I call the "Wife factor". When I've started with the project I've used hardware like Raspberry Pi, BeagleBoneBlack and Arduino Yun to measure the temperature in different rooms in our house. It worked very nice BUT most of these devices have some kind of status led or I/O led which is blinking all the time. In addition all of those devices needed a power supply which means you have to place them close to a power plug. When my wife saw all these lights blinking and wires lying around she was not very amused.
I figured out that as long as she does not see the so called Sensor Nodes everything is fine.

Size matters
That said one thing was pretty clear, the Sensor Nodes have to be small and they should run on batteries...for months!!! I've tried different hardware like Raspberry Pi, Arduino Yun, Arduino + XBee but all of these approaches did not really work out well.
The biggest problem was the need for a power supply which limited the location I could place the Sensor Nodes. In the end I've took a closer look to the XBee itself which looks like follows...



XBee's are modules that provide a cost-effective wireless connectivity to devices in ZigBee mesh networks. They are produced by Digi and there are different modules available.
For my Sensor Network I'm using the XBee S2 or XBee ZB. If you take closer look to the XBee you will figure out that it does not only give you a cheap reliable wireless connection but also has pins for PWM, AD, RX/TX etc. 
So in principle this is a little micro controller with a wireless connection feature which made it the perfect fit for my Sensor Nodes.

Network Topology
For my approach I've used the simplest network setup that is supported, the Star Network. With this network topology I just need one XBee that acts as a so called Coordinator for the network and to that all Sensor Nodes connect to. Another advantage of the Star Network is the low power consumption because I don't have any Router Nodes (that have to be always on) between the Sensor Nodes and the Coordinator. To make it work every Sensor Node has to know the uniqe Serial Number of the Coordinator which has the advantage that the Sensor Nodes don't need to broadcast their data on the network but can directly "talk" to the Coordinator which again saves some energy.

Programming XBees
To let the Sensor Nodes know the Serial Number of the Coordinator Node you have to program them somehow. Lucky me that Digi created a nice tool that makes it very easy to setup the XBees. The tool is called XCTU and could be found here (btw it's a Java application :) ).
Now that we have the software to setup a XBee we somehow need to connect the XBee to our computer. Therefor the easiest solution is a little piece of hardware called XBee Explorer and can be find at SparkFun. It looks like follows...



You could also find other versions from other vendors. As you can see it has a socket for the XBee and a micro USB port that you can connect to your computer.
If you then start XTCU you should see something similar to the following...


If you follow the instructions you should be able to discover the XBee that is connected to your computer. 
The next step is the setup of the XBee, therefor you first should update the firmware of the XBee to the correct version. 

Attention:
The Coordinator Node needs a different firmware than the Sensor Nodes!!!

If you also use a XBee S2 you should see something similar to this...


For the product family XB24-SB I've selected the firmware with the version 21A7 for the Coordinator Node as you can see on the following screenshot



As you can see on the screenshot there are firmwares for the so called API mode and the so called AT mode. The API (Application Programming Interface) mode is a frame-based method for sending and receiving data to and from a radio's serial UART. The API mode gives you more flexibility because you can send and receive packets to and from the XBee. In AT or transparent mode the XBee simply relays all serial data to the receiving XBee.
So I've choosen the API mode for my project.
After you've updated the firmware of your Coordinator XBee you have to program it using the XCTU software. Don't worry, programming in this case means simply set the right parameters in the Radio Configuration part of the XCTU window.

Setup of Coordinator Node:
For the Coordinator Node there are only a few things you have to set:

Variable    Name                Value                            
ID          PAN ID              A unique id that is the same for                                     all Nodes in your Sensor Network
                                (e.g. 312)

NI          Node Identifier     A name for this XBee
                                (e.g. coordinator)

Attention:
Write down the variables SH and SL which is the Serial Number of the Coordinator Node because you will need it for the setup of the Sensor Nodes!!!

Don't forget to save the Coordinator Node profile from within the XCTU software. So you can re-use it later if needed.


Setup of Sensor Nodes:
For the Sensor Nodes you have to set up a few more things:

Variable    Name                Value                            
ID          PAN ID              The same PAN ID as on the 
                                Coordinator (e.g. 312)

NJ          Node Join Time      1 x 1 sec

DH          Destination High    High Address of Coordinator Node
            Address             e.g. 13A200

DL          Destination Low     Low Address of Coordinator Node
            Address             e.g. 40C05061

NI          Node Identifier     Some name (e.g. sensor-node0)

SM          Sleep Mode          Cyclic Sleep [4]

SN          Number of Cyclic    1E
            Sleep periods

SO          Sleep Options       4

AP          API Mode            2

SP          Cyclic Sleep Period 3E8 x 10 ms

ST          Time before Sleep   3E8 x 1 ms

D3          AD3/DIO3 Config.    ADC [2]

IR          IO Sampling Rate    400 x 1 ms

V+          Supply Voltage      0C00
            High Threshold


These are a lot of values that seems to be a bit cryptic but if you know what how it works it's logical. Let me try to explain the important parts.
First of all you have to know my setup to understand the settings above.
To measure the temperature I simply use a TMP36 temperature sensor and the circuit looks like this



You can see that this is a really simple circuit which could be very small. To get better results one could add an additional capacitor between +3.6V and the signal pin of the TMP36 but it also works without it. The signal pin of the TMP36 is connected to the AD3 pin of the XBee.
Because when measuring the temperature of a room you don't expect the temperature to change very fast it makes sense to measure the temperature only every couple of minutes and put the XBee to sleep between the measurements.
So the procedure that I use in my Sensor Network measures the temperature every 5 minutes (which still is to often :) ).

So now we know all parameters to understand the setup of the XBee, let's have a look:

We need to configure pin AD3 as an analog input pin which is done by the parameter:

D3          AD3/DIO3 Config.    ADC [2]


Enabling the cyclic sleep mode is done by the parameter:

SM          Sleep Mode          Cyclic Sleep [4]


Now we need to define the sleep cycle which is done by the following parameters:

SN          Number of Cyclic    1E
            Sleep periods

SO          Sleep Options       4
            Extended cyclic
            sleep (SN x SP)

SP          Cyclic Sleep Period 3E8 x 10 ms

ST          Time before Sleep   3E8 x 1 ms
                        1000ms wake time

IR          IO Sampling rate    400 x 1 ms
            Sample ever 1024ms
            (Should be a bit
            higher than ST to
            send 1 sample)

Attention:
All numeric parameters in the XCTU software are in hexadecimal!!!

Means we define one sleep cycle as long as 

(SP) 0x3E8 => 10000 ms and the number of cycles to sleep should be (SN) 0x1E -> 30 which means the XBee should sleep 10000 x 30 ms = 300000 ms = 5 Minutes.

The sample time is set to (IR) 0x400 => 1024 ms and the time before the XBee goes to sleep again is set to (ST) 0x3E8 => 1000ms which means it will wake up after 5 Minutes, will sample data on pin AD3 and before it can sample data again it will fall a sleep again.

Now you should save this profile in XCTU.

You have to repeat that procedure for each Sensor Node and because you saved the profile the only thing you have to change is the NI parameter which is the name of the Sensor Node. This name will later be part of the message and makes it easier to identify the Sensor Node.


Connecting the Coordinator:
Because I would like to use only embedded devices I've decided to use a Raspberry Pi as my "Coordinator Node Server". Means the Coordinator Node will be hooked up to the Raspberry Pi. The easiest way I found out was to use a little add-on board named "Slice of Pi" which looks like this



This board has a socket for the XBee which makes it really easy to connect the XBee to the Pi.

Talking to the XBees
Now that we have the hardware in place the Sensor Network is working already but how could we get the data?
Well there is a really nice Java project on GoogleCode named xbee-api. With this Java library it's really easy to connect to the XBee that acts as a Coordinator Node attached to the Raspberry Pi. To be able to connect to the XBee that is connected to the Slice of Pi board you need to make sure that the following settings on the Pi are set:

1. Remove all references to ttyAMA0 from /boot/cmdline.txt

2. Comment the line "T0:23respawn:/sbin/getty -L ttyAMA0 115200 vt100" in /etc/inittab

3. Sudo apt-get install librxtx-java

The nice thing about the xbee-api library is that you can attach a listener to the XBee Coordinator and you will get notified everytime a XBee Sensor Node sends it's data.

If you would like to play around with it you will find examples on how to use the API with a Receiver and Sender here.


This should be enough for today...so keep coding... :)

Friday, September 12, 2014

Friday Fun Component XVIIII

Aloha,

Yesterday night I needed some time to relax and when I looked on my Galaxy S4 I saw a little widget installed called Minimal Clock which is a nice example of a simple flat ui clock.
So I've decided to clone that widget in JavaFX and here is the result:


As you can see it looks very similar to the original version but it doesn't have all the features. My version only shows the time and date but one could define the colors for all the elements.
Like I said this control was only created to relax but hey...if you like it...fork it on bitbucket

BitBucket Repository

That's it for today and don't forget...keep coding :)

Thursday, July 17, 2014

Off to new horizons...

It's now more than 2 years ago when I've started working for Canoo Engineering and I have to say that I really enjoyed that time. It was always great to discuss topics with a bunch of really smart people.

But sometimes you get the chance to head to new horizons...

For me it was the question:
"Would you like to work together with Angela Caicedo, James Weaver, Stephen Chin and Simon Ritter in the evangelism team?"

And as you might guess already there was only one answer for me...YES OF COURSE!!!

So I'm happy to announce that as of 8/15/2014 I will join Oracle in Simon Ritter's team as Java Technology Evangelist.
As you might already know from my colleagues my job will be about blogging, speaking at conferences and Java User Groups and spreading the word about all things Java (especially Java in the IoT space).

So I'm really looking forward to spend all my energy in the technology that I like sooooo much...Java.

Keep coding...

Tuesday, June 24, 2014

OpenJFX for ARM v6 (hard and soft float)

Hi there,

This post is for all of you that play around with JavaFX on embedded devices. Not everyone is able or willing to build it's own version of OpenJFX and so I decided to build it regularly and provide a download link to it.
So here we go, I've set up the build in a Linux virtual machine on my mac and built the OpenJFX v6 hard and soft float version today.
You can find both versions here:

JDK8 OpenJFX ARMv6 hard float

JDK8 OpenJFX ARMv6 soft float

To use this you first need to install Java SE 8 embedded on your embedded system which you can find here (hard and soft float are available):

Java SE 8 Embedded and Java SE 8 Embedded HardFloat 

After you have installed Java8 on your embedded device you need to tell the Java runtime that you will use another jfxrt.jar instead of the default one. 
If you have installed JDK8 in /opt/jdk1.8.0 and the OpenJFX version in ~/openjfx on your embedded device you have to call your jar like follows:

/opt/jdk1.8.0/bin/java -Djava.ext.dirs=~/openjfx/armv6hf-sdk/rt/lib/ext -jar YOUR_APP.jar

Attention:
The soft float version won't run on the Raspberry Pi (but you should be able to use the hard float version).

I hope this is useful for one or the other of you...

Keep coding...

Monday, May 5, 2014

Java 8 on Intel Galileo

Aloha,

Last week my Intel Galileo board arrived and last weekend I've started to play around with it. The target was to put Java on the Galileo (what else) and see what is possible. So the first thing I did was downloading and installing a Debian Wheezy Linux on a micro sd card. You can find different preconfigured distributions that have ssh server etc. already setup (I used this one). 
So after the Galileo booted in Debian I connected via ssh and set up the Linux in the same way I used to setup the Raspberry Pi. 
So usually I set up a new user, add him to the sudoers group, install mingetty to enable autologin and setup avahi daemon to be able to connect to the board from the Finder in OS X.
After that was done I've simply downloaded the appropriate JDK8 version for the Galileo platform (jdk-8u5-linux-i586.tar.gz).
The installation of JDK8 is the same as on the Pi, so simply copy the tar file to the Galileo and execute the following commands on the command line:

mkdir -p /opt
sudo tar zxvf jdk-8u5-linux-i586.tar.gz -C /opt
rm jdk-8u5-linux-i586.tar.gz

With this commands you install JDK8 in /opt/jdk1.8.0_05.
As next step I've setup JAVA_HOME as follows

sudo nano /etc/bash.bashrc

add the following two lines to the end of the file

export JAVA_HOME=/opt/jdk1.8.0_05
export PATH=$JAVA_HOME/bin:$PATH

Save the file with CTRL+O
The next time you login on the Galileo you should be able to type

java -version

on the command line and you should see something like this

java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05_b13)
Java HotSpot(TM) Client VM (build 25.5-b02, mixed mode)

Now that I had Java 8 on the Galileo the question was how to make use of the GPIO and Analog ports of the Galileo. Because there is no direct way in Java to access the IO-ports of the Galileo board (because there is nothing like Pi4J for the Galileo) you have to use a little workaround to address the IO-ports.
Fortunately there is a nice website that explains how to address the IO-ports of the Galileo board from the Linux console. The approach is to use the file system to address the IO-ports. Well addressing the file system is very easy to do in Java and so I wrote a little library that enables me to address the ports on the Intel Galileo from Java.
At the moment this is just something to play with and because it's based on the file system approach it's not really something you would like to use on the regular basis but he...it works :)
My idea was to use the IO-ports of the Galileo with the names of the Arduino shield connector. Means we have 6 analog ports (A0 - A5) and 13 digital ports (D0 - D13) that can be used. I just made two little tests with D13 and A0 and both worked but I can't guarantee that everything works as it should...so be careful.
At the moment you can set and read the digital pins D0..D13 and you can read the analog pins A0...A5. There is no support for pulse-width modulation (pwm) at the moment but I'll maybe add it in the near future.

Because I had a spare TMP36 analog temperature sensor lying around I've connected it to the 5V and GND pins of the Intel Galileo and the data pin I've connected to A0. With my GalileoIO library the following code is needed to read out the value at A0 and print it on the console.


// Read analog value from A0 with TMP36 connected
public class Main {
  public Main() {
    // Instantiate the GalileoIO library
    final GalileoIO galileoIO = new GalileoIO();            
        
    // Read out the voltage at A0 in millivolt
    double voltage = galileoIO.getAnalog(Analog.A0);
        
    // Calculate the temperature from the voltage of the TMP36
    double temp = (voltage - 500d) / 10d;        
        
    // Print the temperature on the console
    PrintStream out = new PrintStream(System.out, true, "UTF-8");
    out.println("Temperature: "
                temp + 
                "\u00B0C ("
                voltage + 
                " [mV])");                
    System.exit(0);
  }
   
  public static void main(String[] args) {
    new Main();
  }

}

Another example would be to set a digital pin to high and the code you need to do this is as follows

// Set D13 to high
public class Main {
  public Main() {
    // Instantiate the GalileoIO library
    final GalileoIO galileoIO = new GalileoIO();            
        
    // Set D13 to high
    galileoIO.setDigital(Digital.D13, Pin.Value.HIGH);
        
    // Wait for 3 seconds
    try {
      Thread.sleep(3000);
    } catch (InterruptedException exception) {}    
    
    // Set D13 to low
    galileoIO.setDigital(Digital.D13, Pin.Value.LOW);

    System.exit(0);
  }
   
  public static void main(String[] args) {
    new Main();
  }

}

To make sure the IO-ports will be unexported again I've added a shutdown hook to the GalileoIO library that will unexport all used ports when the JVM shuts down.

If you are interested in the code feel free to fork it on BitBucket.

That's it for today...so keep coding...

Friday, May 2, 2014

Enzo is on Maven Central

Just a short announcement...if you like you could find the JavaFX 8 controls library "Enzo" now on Maven Central.

That was really short...so keep coding...