Who let the dogs out...

Hi there,
long time no post...but today I have some interesting news for you about JavaFX on embedded devices.
At Java One 2012 Oracle showed a lot of JavaFX demos running on embedded hardware like e.g. the BeagleBoard xM. Because I'm a long time SunSPOT user I was totally fascinated by the new possibilities.
During the bus ride to the Oracle appreciation event I had an veeeeerrrrrryyyyy interesting discussion with Todd Costella (blog, twitter) and Carl Dea (twitter) about using embedded hardware together with Java/JavaFX. 
Long story short, I ordered a BeagleBoard xM and a 10" LCD touchscreen to test JavaFX on it.
So in this post I will try to give you an introduction on how to setup the BeagleBoard xM with JavaFX. I hope that will make it easier for those of you that are willing to play around with JavaFX on embedded.


Hardware Requirements:

• BeagleBoard xM (available at DigiKey, Watterot etc.)
• 5 Volt power adapter for the BeagleBoard xM
• Ethernet cable
• USB Mouse
• USB Keyboard
• Output device (one or the other)
• 10" LCD (1024x600) touchscreen (available at Chalkboard Electronics, Watterot etc.)
• HDMI to DVI adapter (when using a standard monitor)
• USB to Serial Adapter (Recomended: with FTDI chipset)
• A class 10 Micro SD card (4GB or more)

Software Requirements:

• JDK 7 (1.7.10 Developer Preview available here)
• Angstrom image
• based on 2011.03 if you don't need touch input
• based on 2012.05 if you need touch screen support

You could in principle use every Linux distribution that supports rendering to framebuffer using EGL and OpenGL ES 2.0

You will also find ready to go Ubuntu distributions on the web but Ubuntu runs a bit slower on the BeagleBoard because it could not run the processor at it's full speed of 1GHz but only at 800 MHz.

Preparation:

There is a really good guide on how to setup the whole system already at the Oracle page. If you follow this guide you might be able to create the image with the online image builder but for this took maaaaaannnnnnyyyyy attempts before I finally got a working image. For that reason you could download the images from my DropBox:

• Angstrom image based on 2011.03 kernel
• Angstrom image based on 2012.05 kernel please download these files
• Angstrom image link and Angstrom image file
• MLO link and MLO file
• Modules

With having this images available the whole procedure get's a lot easier. Especially if you need touchscreen support, because I was not able to create this image with the online builder. Instead I had to setup a virtual machine running Ubuntu to build the kernel by myself. 

At this point I have to say THANK YOU VERY MUCH to Daniel Blaukopf (Oracle) who sent me detailed instructions on how to build the 2012.05 kernel. Without his help I would not have been able to create the kernel.

For everyone that is willing to build the kernel here is a pdf with Daniels instructions. All others please download the 2012.05 image files above and follow this instructions

Prepare the SD card and write the file systems on your host
You have to replace the /dev/sdX with correct device for you Micro SD card that is connected to your host pc. For example /dev/sdb or /dev/mmcblk0.
Copy all the files from the 2012.05 image to a folder like /deploy/images/beagleboard on your host pc. Now you have to open a terminal session and go to this folder and execute the following commands

cd /deploy/images/beagleboard



sudo sh ./makecard.txt /dev/sdX 



mkdir -p /tmp/boot /tmp/rootfs 



sudo -s



mount /dev/sdX1 /tmp/boot



mount /dev/sdX2 /tmp/rootfs



tar jxf systemd-GNOME-image-beagleboard.tar.bz2 -C /tmp/rootfs



tar zxf modules-3.2.28-r122a-beagleboard.tgz -C /tmp/rootfs



cp MLO /tmp/boot



wget http://www.beagleboard.org/angstrom-mirror/www.angstrom-distribution.org/demo/beagleboard/u-boot.img \

-0 /tmp/boot/u-boot.img



cat > /tmp/boot/uEnv.txt << EOF

vram=24M

dvimode="1024x600MR-32@60 mem=99M@0x80000000 mem=384M@0x88000000 omapfb.vram=0:12M,1:8M,2:4M"

optargs="consoleblank0"

console="console=ttyO2,115200n8"

mmcroot="/dev/mmcblk0p2"

EOF



sync

umount /tmp/boot

umount /tmp/rootfs

In this case we use the uboot.img from the Angstrom website instead of the one we built to make sure USB devices will work properly.

Boot the BeagleBoard xM with the new micro sd card
The first boot will take a long time, so please be patient. If you have a serial console attached, you will see it pause for up to 90 minutes.
When the device has finished doing its initial setup, log in as root with an empty password and execute:

depmod -a

reboot

I hope this will work for you, otherwise drop me a mail or leave a comment what went wrong and I will try to help you get it running.

Tips and Tricks:
I came across some problems and most of them related to the fact that I'm not a terminal hacker/linux kernel master etc.
So I had to learn a lot of things that I didn't know before and to make life easier for you I will share some tips and tricks with you that will make it hopefully easier for you to run JavaFX on your BeagleBoard xM. I have to say that I'm using a Mac which is a bonus over Windows because OS X is also based on Unix which means I have a terminal available and many things are very similar between Linux and OS X. If you are on windows you should install something like Cygwin which will give you the same possibilities.

1. No sudo on Angstrom
On the Angstrom Linux distribution there is NO sudo command. So everytime you read something like sudo COMMAND, just type the command and you will be fine.


2. Create a new user
If you see the login screen for the first time it's time to create a new user. Therefor you simple have to press CTRL+ALT+F1 which will close the GNOME login screen and you will see a terminal. Please type the following commands to add a new user

adduser YOUR-USER-NAME

passwd YOUR-USER-NAME 

After this is done you simply have to press CTRL+ALT+F2 to get back to the GNOME login screen.


3. Update and Upgrade
Before start working with the BeagleBoard xM you should update the packages on your Angstrom installation by executing the following commands in a terminal session

opkg update

opkg upgrade

4. Install additional useful software
The Angstrom distribution uses the opkg package system you could install additional programs using the opkg install command. Here is a list of programs that I installed because I found them usefull

Bash shell: 

opkg install bash

chsh /bin/bash 

Gedit text editor:

opkg install gedit

Midori web browser:

opkg install midori

5. Adjusting the screen resolution
If you use an external monitor to display the Linux GNOME desktop you could configure the screen resolution in the file uEnv.txt that you will find on the boot partition of your micro sd card. The line you have to modify looks like this:

dvimode="1024x600MR-32@60 mem=99M@0x80000000 mem=384M@0x88000000 omapfb.vram=0:12M,1:8M,2:4M"

Here is a short list of modes that might be useful

• 1280x1024MR-16@60
• 1280x1024MR-24@60
• 1280x1024MR-32@60
• 1024x768MR-16@60
• 1024x768MR-24@60
• 1024x768MR-32@60
• 800x600MR-16@60
• 800x600MR-24@60
• 800x600MR-32@60 
• 1024x600MR-32@60 (for 10" LCD multitouch screen)

But keep in mind that you are working on an embedded device with reduced capabilities. So when running on an external monitor I used 800x600MR-32@60 because this is a resolution most of the monitors are able to handle and it uses 600 lines which means it has the same vertical resolution as the 10" LCD panel.


6. Problems with OpenGL (demo or when running JavaFX)
The OpenGL demos could be found in the folder /usr/bin/SGX/demos/Raw on the ext2 parition of your micro sd card. One of the demos is the Water demo which shows a sailing boat on some kind of a lake in full screen. To start it just execute the following command in a terminal session

/usr/bin/SGX/demos/Raw/OGLES2Water

you should check the following file

/usr/bin/cpuType

on the ext2 partition of your micro sd card. Please make sure that the mentioned file contains the following parameter in red

...

sed -e 's/0x00005C00/OMAP3503/' 

-e 's/0x00001C00/OMAP3515/' 

-e 's/0x00004C00/OMAP3525/' 

-e 's/0x00000C00/OMAP3530/' 

-e 's/0x00000E00/ OMAP3530/' 
...

If you downloaded the 2011.03 kernel above you HAVE TO add the red line to the file because it's not in there !!!
After adding this line the OpenGL demo should run. And keep in mind that you have to reset the board to stop the running OpenGL demo.


7. Setup JAVA_HOME
It's a good idea to set the JAVA_HOME environment variable on the beagleboard to avoid typing always the complete path to the java folder. First of all you should have installed the JavaFX Developer Preview for ARM that you will find here. To do so, simply download the file on to your beagleboard and execute the following commands

mkdir -p /opt
tar zxvf jdk-7u10-ea-fx-8_0_0-embedded-linux-arm-sfp.tar.gz -C /opt

This will create a folder /opt on your ext2 partition and extract the jdk1.7.0_10 in this folder.
To set the JAVA_HOME environment variable you should now add these lines

export JAVA_HOME=/opt/jdk1.7.0_10

export PATH=$JAVA_HOME/bin:$PATH 

to one of the following files:

/etc/profile.d/jdk.sh

/etc/profile 

/etc/bash.bashrc 

~/.bash_profile 

If you don't find this files on your ext2 partition in the given folders, you simply have to create them using for example gedit.


8. Copy files from your host computer to the BeagleBoard xM
You should connect your BeagleBoard xM to your local network to make your life easier. If you would like to copy for example some Java jar files from your host computer to the BeagleBoard xM you could either shutdown the BeagleBoard xM, remove the micro sd card, put the card into your host computer, copy the files to the micro sd card (if your host computer could handle ext2 filesystems), plug the micro sd card back into your BeagleBoard xM and start it. Or you could simply use the scp command in a terminal session to copy files via the network directly from your host computer to the BeagleBoard xM.

Copy files using scp, go to a terminal window and type

scp /path/to/local/file USERNAME@HOST:/path/to/remote/file



scp ./test.jar root@beagleboard:/home/root/test.jar

Copy folders using scp, go to a terminal window and type

scp -r /local/folder/* USERNAME@HOST:/remote/folder



scp -r ./test/* root@beagleboard:/home/root/test

9. Run JavaFX files on your BeagleBoard xM
Because you have to type a lot of stuff to run a JavaFX file on your BeagleBoard xM it makes sense to create a little shell script that contains this stuff. Therefor you simply have to open gedit and create something like this

#!/bin/bash

java -Djavafx.platform=eglfb -cp /opt/jdk1.7.0_10/jre/lib/jfxrt.jar:/PATH/TO/JAR/YOUR.JAR com.javafx.main.Main 

Save this file with the ending .sh
To call your java code you now only have to execute the following command in a terminal window

./YOUR_SCRIPT_NAME.sh

And that's it.


10. JavaFX controls on BeagleBoard xM
I tried to use some of the controls from the JFXtras project on the BeagleBoard xM but to my surprise they did not show up on the BeagleBoard xM. Instead I only saw a black screen that has the right size but it's just black. The JavaFX code is running but nothing is rendered. So I took a look at the Oracle demos that are available here to see how they did it. I saw that instead of creating a control containing Control, Skin and Behavior they simply created one big class and put everything in there. So I did the same and took one of the JFXtras controls and put the code from Control, Skin and Behavior into one class and suddenly it shows up.
I also made some tests with some standard controls like a Button and it also did not show up on my BeagleBoard xM. I filled a bugreport in the JavaFX jira bugtracker and hope it will be fixed soon (or maybe I did something wrong).
So if you come across some strange behaviors, please file a bugreport in the JavaFX jira to give the JavaFX team the chance to fix it.


11. The BeagleBoard doesn't respond at all
It might happen that the BeagleBoard xM does not respond at all and that only the diode D5 (close to the micro sd card slot) is blinking. If this happens, most of the times there's something missing on the boot partition or maybe you just forgot to add the following line to the uEnv.txt file on the boot partition

mmcroot="/dev/mmcblk0p2"

12. The driver for the 10" lcd multitouch display
If you would like to use the multitouch support of the 10" lcd screen you have to make sure that the kernel-module-hid-multitouch is installed. Just type the following into a terminal session to make sure it is installed

opkg install kernel-module-hid-multitouch

Please keep in mind that the multitouch support only for an image based on 2012.05 kernel is possible (as far as I know).

13. What to do when the touchscreen doesn't work anymore
I had the problem that the 10" LCD touchscreen worked for a while but suddenly was not recognized as a touch input device anymore. Well after digging around on the web I got the hint in the right direction again from Daniel Blaukopf (can't say enough thank you to you). Because the touch events are handled via a usb connection the solution was really easy. Just pull the usb plug that comes from the LCD screen out of the BeagleBoard xM and reboot the BeagleBoard xM. After you logged in again, plug the usb plug in again. If you type the following commands in a terminal directly after you plugged in the usb plug you should see the multitouch device appear in your list. Here is the command

dmesg | tail

The result should end with something similar to this

hid-multitouch XXXX:XXXX:XXXX:XXXX: input, hidraw3: USB HID v1.11 Device [Cando Corporation Cando 10.1 Multi Touch Panel with Controller] on usb-ehci-omap.0-2.4/input0

14. You get no touch events in you JavaFX app
If you start your JavaFX app from your GNOME desktop it might happen that the touch events won't be received by your app. You could get around this by starting your JavaFX app from the console only, without GNOME. To do that you just have to press CTRL + ALT + F1 to get to the console. Here you have to simply login with your credentials and now start your JavaFX app from the prompt. Now the touch events will reach you JavaFX app.



That's all I have to say for today and with these tips you should be able to get JavaFX up and running on your BeagleBoard xM. The next BlogPost will be about using the BeagleBoard xM with Java to measure some data, so stay tuned and...

keep coding...