Restoring Jeedom from a Debian 8.5 to a Debian 10.3 Virtual Machine

We will here install the excellent Domotic/Home Automation system called Jeedom from an old Debian 8.5 on a fresh new Debian 10.3 Virtual Machine. Actually in this post I will not only install it, but i’ll also restore a backup of my fully functional Jeedom V3 installation that use to run on a Debian 8.5. I decided to upgrade by installing a full new system as I ran into more & more problems with Python dependencies each time I had to upgrade the packages, or some Jeedom plugins.

VM Configuration

For my Virtual machines, I run vSphere 6.5 on an Intel NUC, this is perfectly supported (except the sound card but we don’t need it for this use).

Here is the configuration for my actual Debian8.5:

This image has an empty alt attribute; its file name is Pasted.png

First i’ll deploy the netinstall version of Debian 10.3 on my vsphere datastore. Then i create a new VM with the same capacity of the old one, it runned smoothly.

This image has an empty alt attribute; its file name is Pasted-3.png

Note that i’m not running the latest vSphere update … Debian 10 is not known in the dropdown list. It’s not a problem.

This image has an empty alt attribute; its file name is Pasted-4.png

I’ll use my “datastore 1” datastore, it is an SSD drive.

This image has an empty alt attribute; its file name is Pasted-5.png

Same settings as my Debian 8.5 VM. Note that you need to change the dropdownlist selection inside the CD/DVD Drive, to use your Debian iso file downloaded & uploaded on your vSphere datastore earlier.

This image has an empty alt attribute; its file name is Pasted-6.png

Once created, here is the list of all my VMs. I made some cleanup before. Notice I’m also upgrading my gekko trading bot VM from a Debian 9.3 to a new 10.3 VM. Now let’s start our Jeedom 10.3 VM.

This image has an empty alt attribute; its file name is Pasted-7.png

Debian Installation

I dont need any graphical install … Let’s go for text install. I won’t show every screen, but basic options I choosed:

  • French language, timezone, keyboard (nobody’s perfect),
  • DEBIAN10-3-JEEDOM as hostname
  • Domain: home
  • After setting the root password, I created a new ‘jeedom’ user
  • I choosed LVM to partition the disk. I did not do it on my previous 8.5 install (don’t remember if the choice existed), but it will be easier later if we need to add some disk space or change existing one.
  • I choosed to have /home, /var and /tmp in separated partitions. It is much better to prevent applications in your various ~home dirs to fill all your disk space, or logs in /var/log, or various stuff in /tmp
  • To be honest, after two attempts to restore Jeedom, I needed to modify the LVM volumes inside the install to delete the default home and var logical volumes, in order to recreate them and allocate 5Gb to var, as my old Jeedom database is huge and the first time it completely filled the /var directory
  • Then I Next, next, next …
  • The install will download & install the required packages
  • In my case I didnt need a proxy server, no desktop environment
  • BUT i DO want a web server and an ssh server
This image has an empty alt attribute; its file name is Pasted-9.png

The install process will again download & install the required packages. I’ll choose to install GRUB on /dev/sda as it will be the only OS on this VM.

Now we reboot.

This image has an empty alt attribute; its file name is Pasted-10.png
This image has an empty alt attribute; its file name is Pasted-11.png

Then i’ll connect on the VM with the VMWare console, and update  the VM with:

Shell

Installing the ‘net-tools’ package will allow us to use ‘ifconfig’ to get our IP address.

This image has an empty alt attribute; its file name is Pasted-13.png

Hint: this is the moment you want to define this IP address as a static one on your router so it won’t change in the future. This is a must for jeedom as much equipments or routines will refer to your box by using its IP address.

We also want to install the VMWare tools. I won’t install the ones from my vSphere installation since it is not up to date. I’ll install the Open VM Tools.

Shell

Notice the difference on your vSphere UI (maybe after the reboot we will do later):

This image has an empty alt attribute; its file name is Pasted-16.png

We now see the IP address, and vSphere does see the VMWare tools installed. Perfect.

When prompted, press y to continue the installation, it will install a few dependencies, and when it’s finished, as we installed a buch of things, I suggest to reboot the VM by using:

Shell

Now you should be able to SSH to your VM. I’m using Putty from a W10 laptop.

This image has an empty alt attribute; its file name is Pasted-14.png
This image has an empty alt attribute; its file name is Pasted-15.png

Jeedom installation, Mysql tweaking, Apache2 configuration

What is following is extracted from various parts of the Debian installation process from Jeedom documentation (§1 & §4 + some links from it). I won’t follow everything as the Jeedom documentation (as far as I remember) will install jeedom in the default Debian /var/www directory, and I changed that on my previous installation: the whole Jeedom installation website will in fact be located in /home/jeedom (root of my jeedom user). Yes it does require a few changes in the default httpd (Apache) conf file. 

Shell

The lines above will install sudo (then add our jeedom user created during the VM installation to the sudo group) and fail2ban (to limit the rate of connexions on opened ports from the exterior) and ffmpeg as it will be needed later by Jeedom (replacement of libav-tools). Now as I dont want my sudoers to enter the root passwd, i’ll edit the /etc/sudoers file and change (or comment) this line:

Plain Text

With this one:

Plain Text

Now with the user jeedom you should be able to perform root actions by using (for example) ‘sudo ls -al /root’.

Then we will install Jeedom once before we will restore our backup.

Shell

Note #1: in case you are installing jeedom on an already existing system with a running version of mysql, you can pass arguments to the script (eg. ./install.sh -w /home/jeedom/html -m Jeedom) as I did to directly deploy it in /home/jeedom/html and avoid a few operations to move it later.

Note #2: in case you want to reset the previous Jeedom installation steps and rerun it with a clean database & target html dir, perform the following steps:

  1. Remove the Mysql jeedom database & user
  2. Clean the /var/www/html directory OR /home/jeedom/html in case you modified WEBSERVER_HOME in the Jeedom install.sh file
Shell

Running the install script will install a bunch of packages: mysql, php, python dependencies, etc. At the end it should display a successfull message, and you need to note the root mysql password displayed, in case you didn’t define it by passing it as seen above as an argument to the install script.

We will now change the root passwd for mysql, to restore the one used on our Debian8.5 installation (hint: you can see it from your Debian 8.5 Jeedom UI, in the configuration menu, then OS/DB tab), and give more privileges to the jeedom user (this is needed to restore some stuff during the old Jeedom database restoring).

Shell

Here you need to use the root password for the Mysql Database you defined in your install script, or noted after the Jeedom installation.

Plain Text

Next, I’ll want to move the default mysql database files from the default dir /var/lib/mysql to my /home/jeedom HOMEDIR, where there is more space.

First we make sure about the default mysql datadir used for storing data.

Shell

We stop mysql and check it is well stopped (look for a line “Status: “MariaDB server is down”” in the systemctl status output).

Shell

We’ll copy the existing database directory to the new location with rsync. Using the -a flag preserves the permissions and other directory properties, while-v provides verbose output so you can follow the progress. And we rename the old database with an extension, so that we we ensure mysql can not use this one anymore. We will remove it after.

Shell

Now I’ll edit the /etc/mysql/mariadb.conf.d/50-server.cnf file (Debian specific) to change the datadir directive and point it on my new location, and edit /etc/apparmor.d/tunables/alias to create a pointer between mysql old dir and new dir.

Plain Text
/etc/mysql/mariadb.conf.d/50-server.cnf
Plain Text
/etc/apparmor.d/tunables/alias

And we restart AppArmor.

Shell

Before restarting mysql we will need to create a basic directory structure in the old files directory as it checked by the default mysql startup scripts, and we will need to authorize Mysql/MariaDB to be run from the /home directories by addind an authorization.

Shell
Plain Text

Then we need to reload systemctl configuration files.

Shell

And we restart Mysql, and check it is running well.

Shell
Plain Text

OK there are some complains about the mysql update, but it’s launched and working. Now a quick sanity check inside mysql to verify it is using the new datadir.

Plain Text

Perfect. We can now remove our old database directory.

Shell

Now we will connect to our fresh Jeedom UI (by using http://<our vm ip>), and go for the graphical installation (sorry my screenshots are in French). Default Jeedom login/pass is admin/admin.

This image has an empty alt attribute; its file name is image-13.png

Next we change the default password. I’ll reuse the one i’m using on my other Jeedom installation. And in next screen i’ll tell Jeedom what is my login/pass for Jeedom’s market. If you don’t have one yet there is a link to create one.

After validating the account, Jeedom will display its start page, very empty but that’s normal.

This image has an empty alt attribute; its file name is image-14.png

So, at this stage, we have a working standard Jeedom V4 installation, listening on port 80 of our VM. Fine but not enough.

Now, i’ll tweak the Apache2 configuration to reach /home/jeedom/html as default directory, enable HTTPS and use personal TLS certificates bought on Gandi.

Note that if we want to get more info in Apache2 log files, we will need to edit the /etc/apache2/apache2.conf “LogLevel warn” line to “LogLevel debug”.

In /etc/apache2/apache2.conf, I need to add the following lines:

Plain Text

I check /etc/apache2/ports.conf to verify the default HTTPS listener port.

Plain Text

I’ll modify the ErrorLog directive in /etc/apache2/conf-available/security.conf so that it will log in /var/log/apache2, not in /var/www/html/logs.

Plain Text

Now I modify /etc/apache2/sites-available/000-default.conf to use /home/jeedom/html as root dir and modify the log files.

Plain Text

I modify /etc/apache2/sites-available/default-ssl.conf to configure our HTTPS listener with personal TLS certificates bought on Gandi for my fqdn.

Plain Text

In this file notice the use of a few specific files for my installation:

Plain Text

Those are the files i’ll later need to backup on my Debian8.5 apache2 dir and restore on my Debian10.3 apache2 dir.

We’ll also enable SSL and make the default-ssl.conf file loadable by Apache2 by linking it from sites-available to sites-enabled:

Shell

I backup the SSL certificates used on my previous Jeedom installation, copy it on my new VM, and restore it. On the old Debian 8.5 VM:

Shell

On the new Debian10.3 VM:

Shell

Then I check Apache configuration, and I restart Apache.

Shell

Now i’ll check if my ports 80 and 443 are opened:

Shell

The output is good:

Plain Text

Now let’s try to reach Jeedom UI (by using http://192.168.1.82):

This image has an empty alt attribute; its file name is image.png

And now by HTTPS (by using https://192.168.1.82):

This image has an empty alt attribute; its file name is image-1.png

This warning is normal as i’m reaching the UI by using the VM’s IP, not the FQDN defined in the TLS server certificate I told Apache2 to use in my default-ssl.conf file. When I validate the warning I can access Jeedom’s UI this is perfect.

This image has an empty alt attribute; its file name is image-2.png

Now i’ll want to backup & restore the little script used to update my (unfortunately) dynamic public IP address on Internet so that I can permanently reach Jedom from the outside, and other small stuff.

On the Debian 8.5 box:

Shell

On the Debian 10.3 box, i’ll copy it with scp, untar it, and install a required python package to make it work.

Shell

Then I need to add a crontab job to check my ipaddress & eventually update it:

Shell

Then add this line:

Plain Text

Zwave stick & Jeedom Backup

Now I will backup my Aeotec Zwave USB stick, and my Debian 8.5 Jeedom installation.

About the Aeotec ZWave stick (thanks to Nechry an active contributor for Jeedom about his article):

  • Windows 10 will already have the driver or it will be downloadable automatically, for other OS or to get the inf files manually check here
  • you’ll need the Network Key for the stick, it will probably be the default key, but in doubt in my old Jeedom installation i’ll check it in resources/openzwaved/ozwave/manager_utils.py, it will be a suite of 16 hexa values « 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10 » which will need to be converted as « 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 » in the backup utility

On my W10 laptop, on which I never plugged this USB stick, Windows automatically configured the device, and then I launch the zwave_500.exe utility downloaded from Aeotec’s website.

This image has an empty alt attribute; its file name is Pasted-21.png

I can check the stick is well detected because it displays a COM port in the title bar, and a “Load Zstick Success” in the status bar at the bottom of the window.

Then I need to check « Enable Security » and enter my network/security key and click “set”.

This image has an empty alt attribute; its file name is Pasted-22.png

Then I click on “Read Eeprom” button and choose where to save the backup file. And get a coffee …

This image has an empty alt attribute; its file name is Pasted-23.png

Jeedom backup

Next I need to save my whole functional Jeedom, through its integrated backup functionality.

This image has an empty alt attribute; its file name is Pasted-24.png
This image has an empty alt attribute; its file name is Pasted-25.png
This image has an empty alt attribute; its file name is AUTkcZOOeQbGAAAAAElFTkSuQmCC
This image has an empty alt attribute; its file name is A+iXRq2WLGgfAAAAAElFTkSuQmCC

Jeedom restauration

First, i’ll go back in vSphere administration UI to reallocate my Aeotec Zstick, and bluetooth module to the new VM. This is really simple, just unallocate them from the old DEbian8.5 VM, and reallocate them as a new USB device to the new VM.

This image has an empty alt attribute; its file name is image-17.png

I copy the backup on my Debian 10.3 Box, in the standard backup directory of Jeedom.

Shell

Then i’ll restore it through my new Jeedom’s UI (sorry this is a Jeedom v3 configuration > backups UI screenshot, forgot to take the one from the fresh V4 … But it’s almost the same).

This image has an empty alt attribute; its file name is image-15.png

It will be a long task, and the UI may not update well as files will be deleted and restored. You should be able to follow the restauration process through internal jeedom log file:

This image has an empty alt attribute; its file name is Pasted-30.png
This image has an empty alt attribute; its file name is +AQJ5MjGIABGIABGPgtBq64pvsfJwlLHkCSGFkAAAAASUVORK5CYII=

OK so obvisouly there was at least 1 error, but the result is still OK … Dont’t know if this is a big problem or not. First let’s try to reboot the VM and access Jeedom’s UI. First i’ll shut off the old Jeedom on my Debian 8.5.

After a sanity reboot, I reconnect on Jeedom’s UI and It is running, I can see my dashboard.

This image has an empty alt attribute; its file name is image-16.png

The problems I quickly found are:

  1. Check Jeedom > Health menu, it will probably complain about the external network configuration. This is normal, my backup restored the network config made to be reached from the outside, but it’s still pointing on my old VM. I’ll need to assign the IP address of my old VM to the new one on my DHCP server (in my case my internet box) OR modify the external & Internal IP in Jeedom.
  2. In Jeedom > Update Center, it complains that I have too much Jeedom’s declared on the Jeedom market. This is normal i’m using the “free” version, and the same login to access the market. I just connect on the Market website, check my declared boxes, and remove the old one.
  3. Some modules were disabled. Not important ones (forecast.io plugin to display weather, etc.) but they are disabled and can’t manage to get them back.
  4. You need to reinstall the dependancies for all your modules that need some in their configuration, this is VERY important: Bluetooth Advertisement, ZWave, KRoomba, etc. By using this Debnian 10.3 installation, I had absolutely NO problem with dependencies building, the modules were working well after.

Other than that, all my scenarios, seetings, were perfectly restored … I spent a few days on making this tutorial and finding the good VM parameters & restauration process, but it was definitely needed for the health of my home automation.

Hope it can help some people.

1 thought on “Restoring Jeedom from a Debian 8.5 to a Debian 10.3 Virtual Machine

Leave a Reply