Added MQTT support

Ian

If this is an option for you, I would probably invest in a Raspberry Pi 3B+ and deploy Mosquitto there. Later on, you may also want to install there MySQL/MariaDB etc.

Personally I like to “leave the NAS” alone.

My 2c

Renato

Thanks Renato - I have mariaDb and mysql working successfully on my Synology - it gathers data from my Blitzortung lightning detector:

http://www.theshackbythebeach.com:443/mybo/index.php?bo_page=map

I’ll investigate teh options tonight - thansk for the suggestions - cheers

Hi Renato - It’s taken me a few days to do testing etc

Partial success - long story short - for unknown reasons running Node-red on my Synology won’t allow me to connect to the Mqtt broker thats also running on the Synology. WD is able to publish to the broker fine and I can subscribe to the broker using a test mqtt.fx app. I can connect from Node-red on the synology to another mqtt device on a different IP and it works fine, I even created some new gauges to display the other mqtt data. I then tried installing Node-red on my windows laptop and I could connect to the Synology broker and display the WD gauges.

So something is broken on the Synology that doesn’t allow Node-red to connect - I tried localhost and 127.0.0.1 as well as the IP of the Synology without luck. I even tried a different Node-red docker container and it had the same problem.

I’ll keep checking … thanks anyway
Cheers

Brian/Ian

Here is an example of the “art of possible” I came across last night.

https://pysselilivet.blogspot.com/2018/09/weatherlink-weather-template_26.html

To implement this, all is needed is to replace the WeatherLink source (i.e. the http request object) with the message coming from WD. Also, some of the Function objects would have to be modified as well to account for available datapoints.

Next

  1. Currently working on a NR script that would persist the message in a MySQL database. Trying to decide if storing the message as a single JSON datatype or individual columns…
  2. Persist the data in a timeseries database (i.e. InfluxDB) for later rendering on a dashboard (i.e. Grafana)

Next we could look at replacing the FTP process all together by having a script run on the web server that would subscribe to the message and convert it into a text file to b e stored locally (i.e. pull vs push).

Will keep you posted on the progress

Renato

And there you have it.

Data is being persisted in an InfluxDB TimeSeries database and the data is rendered on the Grafana dashboard.

For this to be possible, I had to:

  1. Strip the “Client ID” from the message, since its presence would result in an incorrect JSON structure

  2. All the values in the customomclientraw file are entered in double quotes. This means that every datatype would be created as strings. I went thru the entire structure of the file and removed the double quotes on those fields that would generate float values (i.e temp, tempTL, tempTH, intemp etc)

  3. Brian, I noticed that the 2 values for presstrendval and temptrend should be of type float but they are actually being generated as strings (since values greater than 0 are created with a leading “+” sign). Is this by design?


re your Q 2, is this for the customclientraw, or?

also have you come across the MQTT skin
Belchertown

Brian

Yes, I was referring to the customclientraw.

Also, do you think you could make the Client ID optional? Or better yet, make it part of the perhaps part of the customclientraw file. Shirt term, I am going to hard code the value “ClientID”:“Southampton” at the front of teh file… As I mentioned, I had to remove it from the message since it is creating an invalid JSON array.

As far as the MQTT skin Belchertown, yes I have. Will look into that next. As for its ability to “auto refresh” the Belchertown page, it relies on the mqtt websocket service. To test that, I will need to modify my mosquitto/mqtt deployment since at present I am not exposing the service.

Thanks
Renato

what if you set the clientid as blank?

tried that, it populated a system generated value.

Renato

ah
that is by design, sorry
but you could set it as a space ?

Brian

Tried that, It is still a problem, since now the text preceding the JSON record is a space followed by an exclamation point. For the data to be recognized as a valid JSON array, I still need to strip everything to the left of the “{”

Renato

try
http://www.weather-display.com/downloadfiles/cronwdmqtt.zip
and with it set to nothing

I did, I have cleared the Client ID field on the form but I am still getting this…

w1nbPUexvuStwFFlP4TQ9aa!{…

Renato

then its not doable
as the component when it sees the clientid set to blank, will then set an auto generated one

Brian

I have added the ability to store the message into a MYSQL database.

To optimize the design, the flow inserts into 2 tables:

MQTT holds the entire message array
MQTT_Summary gets populated by the “UPSERT” block of the flow. AT any given time, this table will only contain 1 record, the most current one. This will improve the performance when trying to display “t0” data.

Lastly, on the database side, I have created an event that will purge records on the MQTT table as records become older than X days.

Let me know what would be the best way to share the scripts with those users who may be interested in replicating the approach.

Renato


Hi Brian,

I ended up setting up a mqtt broker via CloudMqtt and WD is successfully able to update it.

YEsterday though the CloudMqtt server I connect to had a harware failure and was offline for a few hours - the WD mqtt connection program obviously disconnected. When the server came back up, I had to manually re-connect via the Connect button in the WD mqtt window.

Is it possible for an auto connection retry to occur, say once every 10 mins or so?

Thanks
ian

I guess a auto re connect it a good idea for when you have set for updates fast where the program does not restart (and so re connect)

Thanks Brian - remind me what the frequency threshold is above which the the process doesn’t terminate?

I set in the component to stay alive/auto reconnect
seems to work
(i.e if I disconnect my internet it will re connect when I reconnect that)
if the update rate is less than 60 seconds it will stay running

try this new update
http://www.weather-display.com/downloadfiles/cronwdmqtt.zip
vers 1.8

Thanks - new version works fine with my local testing - I’ll monitor with the connection to CloudMqtt - Regards Ian