The Raspberry Pi version of Weather Display (consolewd) allows you to transfer files to your web server using FTP. That works for many people, but FTP is a rather old protocol and cyber-security experts consider it to be insecure, mostly because it sends your password as clear text. Anyone who is able to monitor your network or your server’s network (or any point between) could find out your password when you use FTP to transfer files.
consolewd doesn’t support SFTP - a more secure version of FTP (Secure-FTP), so I worked out how to implement a more secure transfer process. I created the instructions below to help set up your Raspberry Weather Pi to synchronise consolewd data files to your web server. It uses a secure transfer mechanism based on cryptographic keys rather than passwords. This process should work on other Linux instances but you might need to tweak a few commands/locations to get it working.
The consolewd created files transferred are:
- All files beginning with ‘clientraw’
- customclientraw.txt
- extravpdata.txt
You can modify the config file created to send different files if you wish.
The following steps are all done on your Raspberry Pi…
0) Introduction
These instructions use a number of placeholders that you should change to suite your own environment. The instructions assume you’re trying to copy files, e.g. clientraw.txt, from your Raspberry Pi running consolewd to your web server. The placeholders are:
- - a user with SSH access permissions on your web server
- <hostname.tld> - the hostname of your web server, e.g. mywebserver.myweatherdomain.com
- <src_dir> - the directory on your Raspberry Pi where consolewd is installed, e.g. /home/pi/consolewdfiles/
- <target_dir> - the directory you want to upload your files into on your web server. Note that the directory is relative to the directory that logs into on the server. So if when logs in you’re in ‘/home/myweathersite’ and you wanted to upload the files to /home/myweathersite/public_html/data/’ <target_dir> would be ‘public_html/data/’. Note also, that the trailing ‘/’ is essential.
1) Create SSH (cryptographic) keys
This assumes that there are no existing RSA key pairs existing on your Raspberry Pi (I think this is the default condition). Run ssh-keygen and press at all the prompts…
$ ssh-keygen
Generating public/private rsa key pair
Enter file in which to save the key (/home/pi/.ssh/id_rsa): <RETURN>
Enter passphrase (empty for no passphrase): <RETURN>
Enter same passphrase again: <RETURN>
Now copy the generated public key file to your web server using ssh-copy-id…
$ ssh-copy-id -i /home/pi/.ssh/id_rsa.pub <username>@<hostname.tld>
When prompted enter the password for ‘[email protected]’.
IMPORTANT: Don’t give the key files to anyone else because it will allow them to log into your server as if they were you.
Test access to your server using the keys by:
$ ssh '<username>@<hostname.tld>'
If the connection works, log out using exit. You may be prompted to confirm the server fingerprint is OK. Answer ‘yes’ if you’re happy that you’re talking to your own server.
2) Install lsyncd (including dependencies)
$ sudo apt install lsyncd
If prompted about dependencies, confirm that it’s OK to install them.
3) Create an lsyncd config file to copy the files to your server
$ cd <src_dir>
$ nano lsyncd-wd.conf.lua
Cut and paste the config below into the file and then modify it, replacing placeholders with your own values.
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status",
nodaemon=true,
}
sync {
default.rsync,
source = "<srd_dir>",
target = "<hostname.tld>:<target_dir>",
delay = 5,
filter = {
'+ /clientraw**',
'+ /customclientraw.txt',
'+ /extravpdata.txt',
'- /',
'- /**',
'- /*',
},
rsync = {
compress = true,
rsh = "/usr/bin/ssh -l <username> -i /home/pi/.ssh/id_rsa",
},
}
4) Run lsyncd for the first time
Before running it you’ll need to create the log directory.
$ sudo mkdir /var/log/lsyncd
$ sudo lsyncd <src_dir>/lsyncd-wd.conf.lua
If prompted to confirm the fingerprint of your server, confirm it if you’re sure you’re talking to your own server.
If the screen scrolls showing normal responses log onto your server to check the the files you wanted to transfer are being synchronised.
5) Final setup
When all is working well, terminate lsyncd using Ctrl+C then edit the ‘lsyncd-wd.conf.lua’ file and remove the ‘nodaemon=true’ line. This will allow lsyncd to operate in the background. Restart it using:
$ sudo lsyncd <src_dir>/lsyncd-wd.conf.lua
This time there should be a single confirmation like:
12:00:47 Normal: — Startup, daemonizing —
lsyncd will now sit in the background copying the files you’ve selected to your web server each time a file changes. If a file hasn’t changed it won’t be transferred until it next changes.
