Intelligent watering system Part II – Using Raspberry Pi Zero W’s as additional “Antennas” to extend Jeedom’s Bluetooth range

Spring is back so it is the perfect moment to check if my watering system passed the winter, and reactivate it. At the same time i’ll explain in detail the system I set up with external bluetooth antennas. We will indeed improve our Jeedom test installation for automatic watering, by adding (and actually cloning) some Bluetooth “repeaters” (called the antennas) in order to increase the range inside which we can receive moisture detectors data. Those repeaters are very cheap Raspberry Pi Zero W (the W is important, it adds the Bluetooth functionality to the Pi Zero).

The goal is to deploy them outdoor, where moisture detectors are located. They will use their Bluetooth capacities to gather data from moisture sensors which are out of range from the Jeedom Bluetooth controller (located indoor), and they will use my domestic wifi network (reachable outdoor) to relay moisture data to Jeedom. Once Raspbian got installed on Raspberry’s ZeroW, the software part mainly consists in using Jeedom’s BLEA plugin embedded functionalities to automagically deploy on the ZeroW’s what’s needed to make them become BLEA Antenna, and we will also improve a little bit the system so that we check if wifi is ok to eventually relaunch it in case of a loss of connection.

At the end, the system will be composed by a main Jeedom Controller, with wifi but without Bluetooth, located indoor, and 4 jeedom antennas, with bluetooth (to reach the moisture sensors) and wifi (to feed Jeedom’s controller with sensors data). I present below the previous diagram I created in the tutorial Part I, the added parts are in red.

For this tutorial I will use 4 Raspberry Zero W, to demonstrate the ability to relay information from a place where the central Jeedom controller can not reach a moisture sensor, and to demonstrate the connexions between the antennas.

Note that, as we will clone a firstly configured “master SD card” on other SD cards, I strongly suggest to use the exact same SD Cards in your raspberrys to avoid troubles due to small differences in cards sizes OR to create your source raspbian image on a smaller SD card than the one you will really use for your antennas !! If your master SD card is 8Gb and you clone it on 16Gb SD cards, after cloning you will still be able to extand the Linux partition size on each cloned raspberry by using rasp-config tools, as we will show later in this tutorial.

Configure a dedicated user on each ZeroW

I assume here:

  • You got Raspian installed and working on one of your Raspberry ZeroW (we will clone this Raspberry later to the 3 other ones). FYI I re-checked my previous tutorial about installing Raspbian on a Pi Zero W, with the latest Raspian version at the moment (2021-01-11-raspios-buster-armhf-lite.img), and this tutorial is all based on this version on the 4 Pi Zero W.
  • You are root on the Raspberry Pi Zero, or you know how to use sudo as we configured it in the installation tutorial.

Now we add the pluginblea dedicated user, it will be used by the Jeedom controller to connect through ssh/scp to the antenna.

adduser pluginblea
visudo

Then add at the end of the file:

pluginblea ALL=(ALL) NOPASSWD: ALL

Then type CTRL-X then CTRL-Y (or CTRL-k then X with joe if you changed the default editor as I did in my Raspbian installation tutorial).

Make the blea daemon start automatically

(This part is greatly inspired by this post and my suggestion and this update).

Warning: I create the script now, so that it will be onboarded in the SD Card clone we will perform later to easily deploy a new antenna. But it will not work yet as we didn’t yet create the antenna from Jeedom.

First we need to know your Jeedom’s controller IP, and the BLEA API Key. The BLEA API Key can be found in Jeedom Settings>System>Setup>API tab>API key Bluetooth Advertisement. Don’t forget to check on the right that it’s enabled.

Now insert those lines in /etc/init.d/blearpistart, and dont forget to edit line 23 to insert your jeedom controller’s IP, and BLEA API key.

joe /etc/init.d/blearpistart
#!/bin/sh
#/etc/init.d/blearpistart

### BEGIN INIT INFO
# Provides: Jeedom BLEA Plugin
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Simple script to start a program at boot
# Description: A simple script similar to one from www.stuffaboutcode.com which will start / stop a program a boot / shutdown.
### END INIT INFO

# If you want a command to always run, put it here
touch /tmp/blea && chmod 666 /tmp/blea

# Carry out specific functions when asked to by the system
case $1 in
start)
echo "Starting BLEA"
# run application you want to start
/usr/bin/python /home/pluginblea/blead/resources/blead/blead.py --loglevel error --device hci0 --socketport 55008 --sockethost "" --callback http://:PORT/plugins/blea/core/php/jeeBlea.php --apikey 
stop)
echo "Stopping BLEA"\n
# kill application you want to stop
sudo kill `ps -ef | grep blea | grep -v grep | awk '{print $2}'`
;;
*)
echo "Usage: /etc/init.d/blearpistart {start|stop}"
exit 1
;;
esac

exit 0

Note that:

  • You need to enable Jeedom to be reached by HTTP
  • You need to enable the API to be reachable by anywhere

Now make this script executable:

chmod 755 /etc/init.d/blearpistart

Now we will create the service file used by SYSTEMCTL.

joe /etc/systemd/system/blearpistart.service

Insert those lines:

[Unit]
Description=BlEA service
After=hciuart.service dhcpcd.service bluetooth.service
[Service]
Type=oneshot
ExecStart=/etc/init.d/blearpistart start
[Install]
WantedBy=multi-user.target

Now we activate the service:

systemctl enable blearpistart.service

The output should be as following if no error:

Synchronizing state of blearpistart.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable blearpistart
Created symlink /etc/systemd/system/multi-user.target.wants/blearpistart.service → /etc/systemd/system/blearpistart.service.

Note that depending on your Raspbian version, you may instead need to use:

update-rc.d blearpistart defaults

Checking Connection to Jeedom

(I documented this on this post in Jeedom’s forum).

On the Raspberry Zero, and even if we disabled the power management on the wlan0 interface by using “wireless power off” in the network interfaces file, you may still experiment some disconnections, especially if your raspberrys are not well receiving the wifi signal. So I added a small script which will be used to regularly stop the blea daemon, ping our main server, and if we detect it is not reachable, we reboot the Pi, or else we relaunch the blea daemon. This script is based on another one found on Internet, you need to change “IP_FOR_TEST” by the IP of the Jeedoms controller you want to ping.

Yes it can probably be optimized as stoping the daemon and eventually rebooting it is a little bit overkill !! But it works, my four antennas are stable over time.

mkdir /opt/check_lan
joe /opt/check_lan.sh

Now add those lines in /opt/check_lan.sh and don’t forget to change line 5 to put your Jeedom’s controller’s IP.

#!/bin/sh

# cron script for checking wlan connectivity
# change  to whatever IP you want to check.
IP_FOR_TEST="IP_TO_TEST"
PING_COUNT=1

PING="/bin/ping"
IFUP="/sbin/ifup"
IFDOWN="/sbin/ifdown --force"

INTERFACE="wlan0"

FFLAG="/opt/check_lan/stuck.fflg"
logger "Stopping BLEA antenna"
systemctl stop blearpistart.service
logger "Testing if $INTERFACE can ping $IP_FOR_TEST"
# ping test
$PING -c $PING_COUNT $IP_FOR_TEST > /dev/null 2> /dev/null
if [ $? -ge 1 ]
then
logger "$INTERFACE seems to be down, trying to bring it up..."
if [ -e $FFLAG ]
then
logger "$INTERFACE is still down, REBOOT to recover ..."
rm -f $FFLAG 2>/dev/null
sudo reboot
else
touch $FFLAG
logger $(sudo $IFDOWN $INTERFACE)
sleep 10
logger $(sudo $IFUP $INTERFACE)
logger "Starting BLEA antenna"
systemctl start blearpistart.service &
fi
else
logger "$INTERFACE is up"
rm -f $FFLAG 2>/dev/null
logger "Starting BLEA antenna"
systemctl start blearpistart.service &
fi

Now you will want to make sure this script will run every 15mn (you may change that) by using crontab:

crontab -e

add this line:

*/15 * * * * root /opt/check_lan.sh \u003e/dev/null

FYI, the logger command used in this script will log the output in /var/log/syslog. You may want to monitor it sometimes by using tail -f /var/log/syslog

Backup & Clone your Raspberry SD Card as a template !

Backup

Now is the time you will probably want to dump your Raspberry SD Card into a file, as we have now something like a “standard antenna installation”, that you can easily duplicate on other Raspberrys, before we add it in Jeedom.

First, you should change the name of your raspberry by using ‘raspi-config’, then “System Options”, “Hostname’, and name it as you want. Personally, I named them relatively to their location. (eg. EAST1, EAST2, WEST1, WEST2).

Also, properly stop your raspberry with “shutdown now” instead of removing the USB power cable.

sudo su -
/sbin/shutdown now

Now we will use the “HDD Raw Copy Tool” freeware tool, on a Windows computer. Insert the raspberry’s SD Card in your computer (again, it will probably complain about a drive which should be formatted, cancel it). Launch HDD Raw Copy Tool, then choose you card reader as SOURCE and click CONTINUE.

On the next screen, choose a location to store the image file which will be created, and click CONTINUE.

On the next screen, double check the settings, and click START … Go get a coffee …

When you see “Task complete” at the end of the log window, you can remove your source SD Card from your computer, and now you have a perfect clone of your pre-configured raspberry, ready to be copied on other PI Zero W and then to be configured as Jeedom’s BLEA antennas.

Clone

Those operations are to be done on each SD Card you want to use on your production raspberrys.

Insert a target SD Card in your computer, and use the same HDD Raw Copy tool we used before to backup our master Raspberry in an image file. This time we will write the image on other SD cards, by selecting as SOURCE the image file, then as destination the SD Card, and writing it. This operation will be quite long, depending on your image and target sizes.

Once image writing operation is done, we will test it in another Raspberry. Once again we will have to find it’s IP from our router or DHCP server. It should boot correctly, connect on our wifi network. We will only change its hostname so that it is different from our Master raspberry, by using raspi-config. This is also the moment you may want to extand the partition size, still by using raspi-config, if your master image is smaller than the target SD Card.

I suggest you note each raspberry’s MAC address & IP address. In my case i even wrote their MAC Address on their case.

Create the antennas in Jeedom

Starting by now, all those operations will have to be done on EACH raspberry Pi Zero W which was cloned from your master image.

First, in Jeedom, we will create our Antennas, inside the BLEA plugin.

On the screen which will appear, we will have to enter the antenna’s name, IP, Port, ssh’s login & password (we use the user we created dedicated to jeedom earlier), and the device associated to the bluetooth interface of the antenna (hci0 in our case).

Then I recommend to save right now the antenna, and next we will use the “Send files” button to automagically have our Jeedom’s controller send the required files on our antenna. Once this is done we will use the “Launch dependencies” button to automagically have our antenna compile the required files locally.

Despite the information message explaining the files were successfully sent, you can also verify this by connecting to this antenna with SSH and validate that there is a newly created ‘blead’ directory in the pluginblea home directory.

Now we compile the dependencies.

This operation will actually take a long time. You can manually check the log file by using the green button dedicated to that in Jeedom’s UI, but you can also monitor /tmp/blea_dependencies on your antenna.

Go have a few coffees … It will require ~30mn for dependencies to be installed. But, you can also parallelize tasks, and create your other antennas right now, push the files, and launch the dependencies; it won’t stop the one already building. This is what I did with my 4 raspberrys, and you can notice on the left that we can see the 4 antennas, but they have a red status while the dependencies are compiling, as the blea daemon is not launched.

When the dependencies are successfully compiled (which is my case with this raspbian version), you can turn on automatic daemon management in your antennas, and lunch them by using the green “Run” button.

Note on the previous screenshot that I already launched an antenna, and it is now appearing with a green status in the list. After I launched the 4 antennas, they are all seen by Jeedom as running.

You can now visualize your network, and the devices detected and linked to your antennas.

Note that we see the main controller, which is still equipped with an external bluetooth usb stick, and which is called “Local”, and the 4 antennas we deployed. For now we only have Flowercare detected, as this is the one we used in previous tutorial about a simple BLEA controller. A cool feature here is that JEedom will try to approximately guess the position of each sensor, depending on the place you moved your antennas in this view, and the RSSI signal (check below).

Another interesting view, is the “Health” view of the BLEA plugin.

It wil ldisplay for each of your bluetooth device, its Mac address, type, status and batterie, but also the RSSI per antenna or controller. The less the RSSI is, the better the signal is. Also note, the Antenna transmission & reception columns, in this case the Flowercare we have detected is “stick” to the main controller, but we will change that later.

Adding all our Flowercares

This is very simple to have our other devices detected. We will use the “Scan” function of the BLEA plugin, and tell Jeedom we are only looking for Miflora’s equipments.

When Jeedom will find a device of the selected type it doesn’t already know, a screen will pop, asking you information about this device. I strongly suggest here to give explicit names to your Flowercares !! Like the plant name, the place it will be located, etc. Later in your automation scripts, you will need those explicit names to check the values and perform the good actions, without ambiguity.

So we assign this first one a name (and the number I wrote on each sensor), its parent object so it will be displayed in Jeedom’s dashboard, a function category, and then we will go to check its settings.

In the settings we can see it is stick to a specific antenna, but we can change it.

It is not a problem to change the reception and transmission setting to something larger: every antenna will be able to receive and transmit data to this equipment. IT may be useful if you move your sensors often, or during your final outdoor setup. For now we dont change it, as we will check later in the Health & network views, if it changes something to the links between the components.

Now we have to do the same detection/naming stuff until all our sensors are detected. In my case since those sensors are now used for 3 years and were totally inactive during winter, i had to change a few batteries before they all get detected, and I found one is now unable to be detected, had to use my test sensor as a spare.

The final organization I used in Jeedom for this tutorial is as following. I created some objects in my “Home” root object, to distinguish Indoor from Outdoor equipments. Then, inside Outdoor, I defined one object per terace I have, one East, one West. Each equipment has been assigned to its target destination.

Real gains of the antennas

We will test if our antennas are really giving us an extended range. First, i will show the Health & Network views from the BLEA Plugin, with all the equipments and antennas still on my desk. It means they are all really close from each others !

We can still see on which antenna every sensor is bound. Now, i will shutdown all antennas.

As we can see in the Health view, all the sensors are now only detected by the Local antenna, which is the bluetooth usb stick plugged on my Jeedom controller. Only the RSSI from this antenna is displayed for each equipment. Also, in the Network view, we can see all antennas are down/red, and the links between sensors and antennas are now only pointing to the local controller. In the BLEA documentation and this excellent Sarakha’s article (french) we can read that whatever the antenna configured for reception or transmission in the sensor’s configuration, as soon a sensor is seen by any bluetooth antenna, the device is considered as present and usable.

Now I will move my sensors in various places of my terraces, where they could really be located, still with only my main jeedom controller active, no antennas.

It becomes interesting: clearly all sensors moved on the East terrace are out of range. The main controller can still see the West sensor though, as my desk is closed from this terrace. Now, because the Intel NUC on which I run my production Jeedom won’t have this external bluetooth dongle i’m using on my testbed, i’ll tell BLEA to NOT use any local bluetooth controller. Therefore, we will ONLY rely on external antennas, which are still powered off right now.

I checked the option “No local”, and saved the configuration. To truely test this configuration, i will even shutdown the Jeedom Controller, and remove the bluetooth usb stick. After booting it again without bluetooth, here are the Health & Network views.

Now, clearly none sensor is detected, which is perfectly normal ! I will then move my antennas in each “corner” of my appartement, switch them on. Then on the Network view, i’ll move them approximately to where they physically are from each others. I just wait a few minutes after each antenna is back online in Jeedom, for the network to “stabilize” itself.

So now, clearly all the antennas are up, and are acting as real relays to the Jeedom controller. We can see in the Health view that we even have some kind of redundancy if one of the antenna on a terrace had to go down. The network view is almost accurate, the sensors are close to perfectly displayed on the map.

Conclusion

Watering !

This is our main goal … so we actually just have to expand a little bit what we already did with Jeedom scenarios in the tutorial part I, as now we have much more sensors and water valves to control (though I only have one for this tuto but the principle remains exactly the same with several Fibaro FGS222 and several water valves).

So now, all our sensors are active, and we can see on Jeedom’s dashboard that they are actively feeding data.

Monitoring

We already setup some kind of monitoring script on each antenna, so that they will reboot in case they can not ping the Jeedom controller. But this is technical monitoring, not fnctional to make sure there is no water leak somewhere with a valve staying opened, or a part of your garden not being watered because a sensor is not working well.

So, inside Jeedom, some good practices would be:

  • To monitor and send an alert if any Antenna is down for too long;
  • To monitor and alert if a sensor has not provided new data for too long;
  • To monitor and alert sensors batteries;
  • To monitor and alert if a water valve is opened for too long (avoid flooding and water consumption).

This requires a little bit of coding inside scenarios, and it will be the perfect target for a next turotial about monitoring plugins or scenarios ! Incoming.

Alerting

A nice thing is also to add some notifications to be sent by email or pushover or other external services, to inform you by various ways, that the system decided to NOT floor because of the weather forecast, or decided to floor because the humidity check was triggered. It will also be a perfect target for an incoming tutorial about the notification system I developed inside Jeedom by using both scenarios and embedded PHP. This system can notify you or other people by email, pushover, voice, etc.

Troubleshooting

If you have problems with dependencies after cloning your antenna template, try to remove/reinstall all the python PIP & bluepy stuff:

sudo pip3 uninstall bluepy
sudo apt remove python-pip python3-pip
sudo apt autoremove
sudo apt-get reinstall build-essential libssl-dev libffi-dev python-dev
sudo apt-get reinstall python-pip
sudo apt-get reinstall python3-pip
sudo pip3 install bluepy
sudo setcap cap_net_raw+e /usr/local/lib/python3.7/dist-packages/bluepy/bluepy-helper\nsudo setcap cap_net_admin+eip /usr/local/lib/python3.7/dist-packages/bluepy/bluepy-helper\nsudo /sbin/reboot

Quick tutorial: installing Debian raspbian on a Raspberry Pi (Zero)

I’ll explain here the basic configuration I use on my Debian raspberrys when I use them with Jeedom automation system. We just need a raspberry pi zero, an SD Card (I use 16Gb ones, you can use a smaller one), a 1.5A USB power supply, and a Windows computer in order to download & burn the Raspbian image file on our SD Card. Installing Jeedom is out of scope of this article, but we already discussed it in a previous article (not on a Raspberry though but on a VM).

Get the raspbian image

We need to download & install Win32diskimager and then download the raspbian lite image (lighter, no graphical environment). Unzip the image somewhere you will easily find it.

Next, open Win32diskimager, tell it to use your raspbian unpacked image file with the button” Image file” and in “Device” select your SD card. Please take care to choose the right device, as it will overwrite everything ! Then click on “Write” and have a coffee.

This image has an empty alt attribute; its file name is image-22.png

When the operation will be done, a Windows popup asking you to format a drive will appear, this is normal, one of the two filesystems wrote on the SDCard is not recognized by Windows, as it is a Linux EXT partition. You should also see a new drive in your Explorer, called “boot”, this is also normal.

Configure the Wifi before we boot on the SD Card

We could configure the Wifi on the raspberry after booting on it, but it would require you to plug it on an external display to configure it. Fortunately, we can set it up before, just by puting a file on the “boot” partition, and this file will be used by wpa_supplicant, the software used for Wifi connectivity.

Create a text file containing the code below, and save it as ‘wpa_supplicant.conf’ at the root of your “boot” drive, on the SD Card. You may need to adjust your country code on the first line.

country=FR
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
        ssid="My SSID"
        psk=My un-encrypted passphrase
        scan_ssid=1

With the latest Buster Raspbian release, you must ensure that the file contains the three first lines at the top.

Also, if you are using a hidden network, the extra option scan_ssid, may help connection.

About the passphrase: the password can be configured either as the ASCII representation, by using quotes as per the example above, or as a pre-encrypted 32 byte hexadecimal number. You can use the wpa_passphrase utility (with another Linux box) to generate an encrypted PSK. It takes the SSID and the passphrase as as inputs, and generates the encrypted (hashed actually) PSK. With the example from above, you can generate the encrypted PSK with the following command:

wpa_passphrase your_SSID your_un-encrypted_SSID_passphrase

network={
        ssid="your_SSID"
        #psk="your_un-encrypted_SSID_passphrase"
        psk=28d0eca5ddde6c3f53331833547dfa68eb732030f5a48b8b499ab2600015d4be

Configure the SSH server before we boot on the SD Card

Same technic will be applied here: simply create an empty file called ‘ssh’ at the root of yout “boot” drive, on the SD Card. No extension needed.

You can now remove the SD Card from your Windows computer and insert it in your Raspberry.

Boot the Raspberry and connect to it

Now, insert the SD Card in your Raspberry, and plug the USB power. The green light should light a few times until it stabilizes to full color. If your Wifi file is OK (network name & passphrase) you should see it connecting on your network. To check his IP, you have to connect on your Wifi router and refresh a few times the list of connected devices to visually detect the new device.

On your Wifi router, or whatever the component on your network acting as a DHCP server, I strongly recommand to assign a static IP to your raspberry so that it will be much more easier to administrate it later, and configure it in Jeedom.

Then use your favorite SSH client to connect to the raspberry’s IP. The default login and password after a fresh installation is ‘pi’ / ‘raspberry’.

First step is to change the default passwd:

passwd –> enter ‘raspberry’ (default passwd after a fresh installation –> enter your new password

Tweak the configuration

Change to the root user by using:

sudo su -

First, but this is a personal taste, i’ll install the joe text editor.

apt-get install joe

I like to make sure it will be the default editor:

update-alternatives --config editor

Then I install the locate package, to easily find files on the filesystem (by using first ‘updatedb’ to index the FS).

apt-get install locate
updatedb

Then I suggest to change the Timezone and extend the root partition to the available size on the SD Card:

raspi-config

Choose “4 Localisation Options”

Choose “I2 Change Timezone”

And select your country/town.

Then choose “7 Advanced Options”

Choose “A1 Expand FileSystem”

Choose YES when it will ask to reboot.

Then we will tweak a little bit the wifi interface. Reconnect by SSH, and edit the file /etc/network/interfaces and make sure it contains those lines (if you want to use DHCP; otherwise change the 8th line from ‘dhcp’ to ‘static’ if you want to fix the IP and ensure you got DNS servers properly configured in /etc/resolv.conf):

sudo su -
joe /etc/network/interfaces
source-directory /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto wlan0
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
wireless-power off

Now we will make sure the default Python version used is the “old good” 2.7, while the v3 is installed ans useable, system-wide:

update-alternatives install /usr/bin/python python /usr/bin/python3.7 1
update-alternatives install /usr/bin/python python /usr/bin/python2.7 2

As the highest number at the end of each line is the priority to be used, Python 2.7 will be used by default. Now we update a few things.

apt-get install python-pip python3-pip libglib2.0-dev
python2 -m pip install upgrade force pip
python3 -m pip install upgrade force pip
pip install bluepy
pip3 install bluepy
setcap cap_net_raw+e /usr/local/lib/python3.7/dist-packages/bluepy/bluepy-helper\nsetcap cap_net_admin+eip /usr/local/lib/python3.7/dist-packages/bluepy/bluepy-helper

Note that the line 5 above should return that it is already deployed:

Requirement already satisfied: bluepy in /usr/local/lib/python3.7/dist-packages (1.3.0)

I suggest you reboot with /sbin/reboot to check it’s reconnecting well – In my case it is !

Update the RPI & Raspian

sudo su -
apt-get update
apt-get full-upgrade

Then we will free some space on our root directory …

apt-get autoclean
apt-get autoremove
apt-get clean

Now I get a fully functional raspberry pi zero runing a specific debian, connected to my wifi network, and able to be used through SSH. This will be one of my bases for later articles.

%d bloggers like this: