Intelligent watering system Part II – Using Raspberry Pi Zero W’s as additional “Antennas” to extend Jeedom’s Bluetooth range

Posted on by Arnauld Dravet

Spring is back so it is the perfect moment to check if my watering system passed the winter, and reactivate it. At the same time i’ll explain in detail the system I set up with external bluetooth antennas. We will indeed improve our Jeedom test installation for automatic watering, by adding (and actually cloning) some Bluetooth “repeaters” (called the antennas) in order to increase the range inside which we can receive moisture detectors data. Those repeaters are very cheap Raspberry Pi Zero W (the W is important, it adds the Bluetooth functionality to the Pi Zero).

The goal is to deploy them outdoor, where moisture detectors are located. They will use their Bluetooth capacities to gather data from moisture sensors which are out of range from the Jeedom Bluetooth controller (located indoor), and they will use my domestic wifi network (reachable outdoor) to relay moisture data to Jeedom. Once Raspbian got installed on Raspberry’s ZeroW, the software part mainly consists in using Jeedom’s BLEA plugin embedded functionalities to automagically deploy on the ZeroW’s what’s needed to make them become BLEA Antenna, and we will also improve a little bit the system so that we check if wifi is ok to eventually relaunch it in case of a loss of connection.

At the end, the system will be composed by a main Jeedom Controller, with wifi but without Bluetooth, located indoor, and 4 jeedom antennas, with bluetooth (to reach the moisture sensors) and wifi (to feed Jeedom’s controller with sensors data). I present below the previous diagram I created in the tutorial Part I, the added parts are in red.

For this tutorial I will use 4 Raspberry Zero W, to demonstrate the ability to relay information from a place where the central Jeedom controller can not reach a moisture sensor, and to demonstrate the connexions between the antennas.

Note that, as we will clone a firstly configured “master SD card” on other SD cards, I strongly suggest to use the exact same SD Cards in your raspberrys to avoid troubles due to small differences in cards sizes OR to create your source raspbian image on a smaller SD card than the one you will really use for your antennas !! If your master SD card is 8Gb and you clone it on 16Gb SD cards, after cloning you will still be able to extand the Linux partition size on each cloned raspberry by using rasp-config tools, as we will show later in this tutorial.

Configure a dedicated user on each ZeroW

I assume here:

  • You got Raspian installed and working on one of your Raspberry ZeroW (we will clone this Raspberry later to the 3 other ones). FYI I re-checked my previous tutorial about installing Raspbian on a Pi Zero W, with the latest Raspian version at the moment (2021-01-11-raspios-buster-armhf-lite.img), and this tutorial is all based on this version on the 4 Pi Zero W.
  • You are root on the Raspberry Pi Zero, or you know how to use sudo as we configured it in the installation tutorial.

Now we add the pluginblea dedicated user, it will be used by the Jeedom controller to connect through ssh/scp to the antenna.

adduser pluginblea
visudo

Then add at the end of the file:

pluginblea ALL=(ALL) NOPASSWD: ALL

Then type CTRL-X then CTRL-Y (or CTRL-k then X with joe if you changed the default editor as I did in my Raspbian installation tutorial).

Make the blea daemon start automatically

(This part is greatly inspired by this post and my suggestion and this update).

Warning: I create the script now, so that it will be onboarded in the SD Card clone we will perform later to easily deploy a new antenna. But it will not work yet as we didn’t yet create the antenna from Jeedom.

First we need to know your Jeedom’s controller IP, and the BLEA API Key. The BLEA API Key can be found in Jeedom Settings>System>Setup>API tab>API key Bluetooth Advertisement. Don’t forget to check on the right that it’s enabled.

Now insert those lines in /etc/init.d/blearpistart, and dont forget to edit line 23 to insert your jeedom controller’s IP, and BLEA API key.

joe /etc/init.d/blearpistart
#!/bin/sh
#/etc/init.d/blearpistart

### BEGIN INIT INFO
# Provides: Jeedom BLEA Plugin
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Simple script to start a program at boot
# Description: A simple script similar to one from www.stuffaboutcode.com which will start / stop a program a boot / shutdown.
### END INIT INFO

# If you want a command to always run, put it here
touch /tmp/blea && chmod 666 /tmp/blea

# Carry out specific functions when asked to by the system
case $1 in
start)
echo "Starting BLEA"
# run application you want to start
/usr/bin/python /home/pluginblea/blead/resources/blead/blead.py --loglevel error --device hci0 --socketport 55008 --sockethost "" --callback http://:PORT/plugins/blea/core/php/jeeBlea.php --apikey 
stop)
echo "Stopping BLEA"\n
# kill application you want to stop
sudo kill `ps -ef | grep blea | grep -v grep | awk '{print $2}'`
;;
*)
echo "Usage: /etc/init.d/blearpistart {start|stop}"
exit 1
;;
esac

exit 0

Note that:

  • You need to enable Jeedom to be reached by HTTP
  • You need to enable the API to be reachable by anywhere

Now make this script executable:

chmod 755 /etc/init.d/blearpistart

Now we will create the service file used by SYSTEMCTL.

joe /etc/systemd/system/blearpistart.service

Insert those lines:

[Unit]
Description=BlEA service
After=hciuart.service dhcpcd.service bluetooth.service
[Service]
Type=oneshot
ExecStart=/etc/init.d/blearpistart start
[Install]
WantedBy=multi-user.target

Now we activate the service:

systemctl enable blearpistart.service

The output should be as following if no error:

Synchronizing state of blearpistart.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable blearpistart
Created symlink /etc/systemd/system/multi-user.target.wants/blearpistart.service → /etc/systemd/system/blearpistart.service.

Note that depending on your Raspbian version, you may instead need to use:

update-rc.d blearpistart defaults

Checking Connection to Jeedom

(I documented this on this post in Jeedom’s forum).

On the Raspberry Zero, and even if we disabled the power management on the wlan0 interface by using “wireless power off” in the network interfaces file, you may still experiment some disconnections, especially if your raspberrys are not well receiving the wifi signal. So I added a small script which will be used to regularly stop the blea daemon, ping our main server, and if we detect it is not reachable, we reboot the Pi, or else we relaunch the blea daemon. This script is based on another one found on Internet, you need to change “IP_FOR_TEST” by the IP of the Jeedoms controller you want to ping.

Yes it can probably be optimized as stoping the daemon and eventually rebooting it is a little bit overkill !! But it works, my four antennas are stable over time.

mkdir /opt/check_lan
joe /opt/check_lan.sh

Now add those lines in /opt/check_lan.sh and don’t forget to change line 5 to put your Jeedom’s controller’s IP.

#!/bin/sh

# cron script for checking wlan connectivity
# change  to whatever IP you want to check.
IP_FOR_TEST="IP_TO_TEST"
PING_COUNT=1

PING="/bin/ping"
IFUP="/sbin/ifup"
IFDOWN="/sbin/ifdown --force"

INTERFACE="wlan0"

FFLAG="/opt/check_lan/stuck.fflg"
logger "Stopping BLEA antenna"
systemctl stop blearpistart.service
logger "Testing if $INTERFACE can ping $IP_FOR_TEST"
# ping test
$PING -c $PING_COUNT $IP_FOR_TEST > /dev/null 2> /dev/null
if [ $? -ge 1 ]
then
logger "$INTERFACE seems to be down, trying to bring it up..."
if [ -e $FFLAG ]
then
logger "$INTERFACE is still down, REBOOT to recover ..."
rm -f $FFLAG 2>/dev/null
sudo reboot
else
touch $FFLAG
logger $(sudo $IFDOWN $INTERFACE)
sleep 10
logger $(sudo $IFUP $INTERFACE)
logger "Starting BLEA antenna"
systemctl start blearpistart.service &
fi
else
logger "$INTERFACE is up"
rm -f $FFLAG 2>/dev/null
logger "Starting BLEA antenna"
systemctl start blearpistart.service &
fi

Now you will want to make sure this script will run every 15mn (you may change that) by using crontab:

crontab -e

add this line:

*/15 * * * * root /opt/check_lan.sh \u003e/dev/null

FYI, the logger command used in this script will log the output in /var/log/syslog. You may want to monitor it sometimes by using tail -f /var/log/syslog

Backup & Clone your Raspberry SD Card as a template !

Backup

Now is the time you will probably want to dump your Raspberry SD Card into a file, as we have now something like a “standard antenna installation”, that you can easily duplicate on other Raspberrys, before we add it in Jeedom.

First, you should change the name of your raspberry by using ‘raspi-config’, then “System Options”, “Hostname’, and name it as you want. Personally, I named them relatively to their location. (eg. EAST1, EAST2, WEST1, WEST2).

Also, properly stop your raspberry with “shutdown now” instead of removing the USB power cable.

sudo su -
/sbin/shutdown now

Now we will use the “HDD Raw Copy Tool” freeware tool, on a Windows computer. Insert the raspberry’s SD Card in your computer (again, it will probably complain about a drive which should be formatted, cancel it). Launch HDD Raw Copy Tool, then choose you card reader as SOURCE and click CONTINUE.

On the next screen, choose a location to store the image file which will be created, and click CONTINUE.

On the next screen, double check the settings, and click START … Go get a coffee …

When you see “Task complete” at the end of the log window, you can remove your source SD Card from your computer, and now you have a perfect clone of your pre-configured raspberry, ready to be copied on other PI Zero W and then to be configured as Jeedom’s BLEA antennas.

Clone

Those operations are to be done on each SD Card you want to use on your production raspberrys.

Insert a target SD Card in your computer, and use the same HDD Raw Copy tool we used before to backup our master Raspberry in an image file. This time we will write the image on other SD cards, by selecting as SOURCE the image file, then as destination the SD Card, and writing it. This operation will be quite long, depending on your image and target sizes.

Once image writing operation is done, we will test it in another Raspberry. Once again we will have to find it’s IP from our router or DHCP server. It should boot correctly, connect on our wifi network. We will only change its hostname so that it is different from our Master raspberry, by using raspi-config. This is also the moment you may want to extand the partition size, still by using raspi-config, if your master image is smaller than the target SD Card.

I suggest you note each raspberry’s MAC address & IP address. In my case i even wrote their MAC Address on their case.

Create the antennas in Jeedom

Starting by now, all those operations will have to be done on EACH raspberry Pi Zero W which was cloned from your master image.

First, in Jeedom, we will create our Antennas, inside the BLEA plugin.

On the screen which will appear, we will have to enter the antenna’s name, IP, Port, ssh’s login & password (we use the user we created dedicated to jeedom earlier), and the device associated to the bluetooth interface of the antenna (hci0 in our case).

Then I recommend to save right now the antenna, and next we will use the “Send files” button to automagically have our Jeedom’s controller send the required files on our antenna. Once this is done we will use the “Launch dependencies” button to automagically have our antenna compile the required files locally.

Despite the information message explaining the files were successfully sent, you can also verify this by connecting to this antenna with SSH and validate that there is a newly created ‘blead’ directory in the pluginblea home directory.

Now we compile the dependencies.

This operation will actually take a long time. You can manually check the log file by using the green button dedicated to that in Jeedom’s UI, but you can also monitor /tmp/blea_dependencies on your antenna.

Go have a few coffees … It will require ~30mn for dependencies to be installed. But, you can also parallelize tasks, and create your other antennas right now, push the files, and launch the dependencies; it won’t stop the one already building. This is what I did with my 4 raspberrys, and you can notice on the left that we can see the 4 antennas, but they have a red status while the dependencies are compiling, as the blea daemon is not launched.

When the dependencies are successfully compiled (which is my case with this raspbian version), you can turn on automatic daemon management in your antennas, and lunch them by using the green “Run” button.

Note on the previous screenshot that I already launched an antenna, and it is now appearing with a green status in the list. After I launched the 4 antennas, they are all seen by Jeedom as running.

You can now visualize your network, and the devices detected and linked to your antennas.

Note that we see the main controller, which is still equipped with an external bluetooth usb stick, and which is called “Local”, and the 4 antennas we deployed. For now we only have Flowercare detected, as this is the one we used in previous tutorial about a simple BLEA controller. A cool feature here is that JEedom will try to approximately guess the position of each sensor, depending on the place you moved your antennas in this view, and the RSSI signal (check below).

Another interesting view, is the “Health” view of the BLEA plugin.

It wil ldisplay for each of your bluetooth device, its Mac address, type, status and batterie, but also the RSSI per antenna or controller. The less the RSSI is, the better the signal is. Also note, the Antenna transmission & reception columns, in this case the Flowercare we have detected is “stick” to the main controller, but we will change that later.

Adding all our Flowercares

This is very simple to have our other devices detected. We will use the “Scan” function of the BLEA plugin, and tell Jeedom we are only looking for Miflora’s equipments.

When Jeedom will find a device of the selected type it doesn’t already know, a screen will pop, asking you information about this device. I strongly suggest here to give explicit names to your Flowercares !! Like the plant name, the place it will be located, etc. Later in your automation scripts, you will need those explicit names to check the values and perform the good actions, without ambiguity.

So we assign this first one a name (and the number I wrote on each sensor), its parent object so it will be displayed in Jeedom’s dashboard, a function category, and then we will go to check its settings.

In the settings we can see it is stick to a specific antenna, but we can change it.

It is not a problem to change the reception and transmission setting to something larger: every antenna will be able to receive and transmit data to this equipment. IT may be useful if you move your sensors often, or during your final outdoor setup. For now we dont change it, as we will check later in the Health & network views, if it changes something to the links between the components.

Now we have to do the same detection/naming stuff until all our sensors are detected. In my case since those sensors are now used for 3 years and were totally inactive during winter, i had to change a few batteries before they all get detected, and I found one is now unable to be detected, had to use my test sensor as a spare.

The final organization I used in Jeedom for this tutorial is as following. I created some objects in my “Home” root object, to distinguish Indoor from Outdoor equipments. Then, inside Outdoor, I defined one object per terace I have, one East, one West. Each equipment has been assigned to its target destination.

Real gains of the antennas

We will test if our antennas are really giving us an extended range. First, i will show the Health & Network views from the BLEA Plugin, with all the equipments and antennas still on my desk. It means they are all really close from each others !

We can still see on which antenna every sensor is bound. Now, i will shutdown all antennas.

As we can see in the Health view, all the sensors are now only detected by the Local antenna, which is the bluetooth usb stick plugged on my Jeedom controller. Only the RSSI from this antenna is displayed for each equipment. Also, in the Network view, we can see all antennas are down/red, and the links between sensors and antennas are now only pointing to the local controller. In the BLEA documentation and this excellent Sarakha’s article (french) we can read that whatever the antenna configured for reception or transmission in the sensor’s configuration, as soon a sensor is seen by any bluetooth antenna, the device is considered as present and usable.

Now I will move my sensors in various places of my terraces, where they could really be located, still with only my main jeedom controller active, no antennas.

It becomes interesting: clearly all sensors moved on the East terrace are out of range. The main controller can still see the West sensor though, as my desk is closed from this terrace. Now, because the Intel NUC on which I run my production Jeedom won’t have this external bluetooth dongle i’m using on my testbed, i’ll tell BLEA to NOT use any local bluetooth controller. Therefore, we will ONLY rely on external antennas, which are still powered off right now.

I checked the option “No local”, and saved the configuration. To truely test this configuration, i will even shutdown the Jeedom Controller, and remove the bluetooth usb stick. After booting it again without bluetooth, here are the Health & Network views.

Now, clearly none sensor is detected, which is perfectly normal ! I will then move my antennas in each “corner” of my appartement, switch them on. Then on the Network view, i’ll move them approximately to where they physically are from each others. I just wait a few minutes after each antenna is back online in Jeedom, for the network to “stabilize” itself.

So now, clearly all the antennas are up, and are acting as real relays to the Jeedom controller. We can see in the Health view that we even have some kind of redundancy if one of the antenna on a terrace had to go down. The network view is almost accurate, the sensors are close to perfectly displayed on the map.

Conclusion

Watering !

This is our main goal … so we actually just have to expand a little bit what we already did with Jeedom scenarios in the tutorial part I, as now we have much more sensors and water valves to control (though I only have one for this tuto but the principle remains exactly the same with several Fibaro FGS222 and several water valves).

So now, all our sensors are active, and we can see on Jeedom’s dashboard that they are actively feeding data.

Monitoring

We already setup some kind of monitoring script on each antenna, so that they will reboot in case they can not ping the Jeedom controller. But this is technical monitoring, not fnctional to make sure there is no water leak somewhere with a valve staying opened, or a part of your garden not being watered because a sensor is not working well.

So, inside Jeedom, some good practices would be:

  • To monitor and send an alert if any Antenna is down for too long;
  • To monitor and alert if a sensor has not provided new data for too long;
  • To monitor and alert sensors batteries;
  • To monitor and alert if a water valve is opened for too long (avoid flooding and water consumption).

This requires a little bit of coding inside scenarios, and it will be the perfect target for a next turotial about monitoring plugins or scenarios ! Incoming.

Alerting

A nice thing is also to add some notifications to be sent by email or pushover or other external services, to inform you by various ways, that the system decided to NOT floor because of the weather forecast, or decided to floor because the humidity check was triggered. It will also be a perfect target for an incoming tutorial about the notification system I developed inside Jeedom by using both scenarios and embedded PHP. This system can notify you or other people by email, pushover, voice, etc.

Troubleshooting

If you have problems with dependencies after cloning your antenna template, try to remove/reinstall all the python PIP & bluepy stuff:

sudo pip3 uninstall bluepy
sudo apt remove python-pip python3-pip
sudo apt autoremove
sudo apt-get reinstall build-essential libssl-dev libffi-dev python-dev
sudo apt-get reinstall python-pip
sudo apt-get reinstall python3-pip
sudo pip3 install bluepy
sudo setcap cap_net_raw+e /usr/local/lib/python3.7/dist-packages/bluepy/bluepy-helper\nsudo setcap cap_net_admin+eip /usr/local/lib/python3.7/dist-packages/bluepy/bluepy-helper\nsudo /sbin/reboot

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.