Added MQTT support

i have added that option now to the latest .zip update (normal download)

Brian

tried the new version…

  1. The option to send customclientraw does not appear to stick. Had to try different combinations between checking and un-checking clientraw and customclientraw to finally have customclientraw and clientraw both stick.

  2. The published message still looks like this:
    Southampton01!12345 4.4 2.6 296 7.2 59 1013.0 2.3 24.6 267.7 0.00 0.00 19.8 51 100.0 1 0.0 0 0 0.3 -54.4 255.0 -65.0 -67.8 -100.0 -100.0 -100 -100 -100 23 43 38 N3KJE-11:43:38_PM 0 0 15 4 0.00 0.00 100 100 100 100 100 5.7 5.0 21.1 7.2 1 Night_time/Dry/Clear_skies 0.7 3 3 3 3 3 6 6 4 3 4 4 3 2 2 3 7 4 3 3 3 17.4 -0.3 3085.0 4/15/2019 27.7 5.0 21.1 4.4 0.0 3 4 4 3 5 1 3 3 1 3 7.8 7.7 7.7 7.7 7.6 7.4 7.4 7.4 7.4 7.2 2.3 2.3 2.3 2.3 2.3 2.3 2.3 2.3 2.3 2.3 21.1 7.2 7.2 11.0 0 — — 297 0 0 -100.0 -100.0 0 -100 -100 -100 -100 0.0 23.8 19.7 4.5 1013.0 992.5 11 11:24_PM 2:59_PM 23.5 3.1 18.9 -0.5 9 2019 0.0 -1 1 -1 297 252 287 290 278 286 300 288 19 296 0.0 255.0 2.4 4.3 40.18528 75.02556 1.3 92 52 2.3 11:35_PM 0.0 0.0 0.0 0.0 0.0 0.0 109.8 2:00_AM 11:42_PM 299 !!C10.37S94!!

i.e. without the field names as opposed to the one WD if FTP’ing which looks like this:

{“date”:“11:48 PM”,“dateFormat”:“m/d/y”,“temp”:“45.0”,“tempTL”:“45.0”,“tempTH”:“70.0”,“intemp”:“67.7”,“dew”:“31.5”,“dewpointTL”:“31.1”,“dewpointTH”:“66.0”,“apptemp”:“41.2”,“apptempTL”:“37.4”,“apptempTH”:“74.3”,“wchill”:“43.9”,“wchillTL”:“39.7”,“heatindex”:“45.0”,“heatindexTH”:“70.0”,“humidex”:“5.0”,“wlatest”:“3.0”,“wspeed”:“3.1”,“wgust”:“10.3”,“wgustTM”:“20.0”,“bearing”:“299”,“avgbearing”:“303”,“press”:“29.916”,“pressTL”:“29.308”,“pressTH”:“29.916”,“pressL”:“28.756”,“pressH”:“30.838”,“rfall”:“0.09”,“rrate”:“0.000”,“rrateTM”:“0.000”,“hum”:“59”,“humTL”:“52”,“humTH”:“92”,“inhum”:“52”,“SensorContactLost”:“0”,“forecast”:“partly cloudy and cooler.”,“tempunit”:“F”,“windunit”:“mph”,“pressunit”:“inHg”,“rainunit”:“in”,“temptrend”:“-1.1”,“TtempTL”:“11:42 PM”,“TtempTH”:“2:00 AM”,“TdewpointTL”:“11:44 PM”,“TdewpointTH”:“3:29 AM”,“TapptempTL”:“11:44 PM”,“TapptempTH”:“2:48 AM”,“TwchillTL”:“2:00 AM”,“TheatindexTH”:“2:00 AM”,“TrrateTM”:“2:30 PM”,“ThourlyrainTH”:“”,“LastRainTipISO”:“4/15/2019 10:34 PM”,“hourlyrainTH”:“0.000”,“ThumTL”:“2:30 PM”,“ThumTH”:“5:08 AM”,“TpressTL”:“3:31 AM”,“TpressTH”:“11:46 PM”,“presstrendval”:"+0.095 ",“Tbeaufort”:“5”,“TwgustTM”:“2:59 PM”,“windTM”:“12.7”,“bearingTM”:“298”,“timeUTC”:“2019,04,16,03,48,26”,“BearingRangeFrom10”:“274”,“BearingRangeTo10”:“19”,“UV”:“0.0”,“UVTH”:“5.1”,“SolarRad”:“0”,“CurrentSolarMax”:“0”,“SolarTM”:“984”,“domwinddir”:“West northwesterly”,“WindRoseData”:[7.0,8.0,3.0,0.0,0.0,0.0,0.0,0.0,0.0,2.0,6.0,9.0,97.0,955.0,181.0,38.0],“windrun”:“68.58”,“version”:“10.37S”,“build”:“94”,“ver”:“10”}

Renato

I’m trying to get a similar dash to Renato’s - I’ve now got Node-RED running fine on my Synology server and I’ve started adding the elements per his dashboard - however I don’t know what to actually enter in each of the split and route boxes etc. Maybe I need to do some reading (a lot) but if Renato could do a quick-start post it would be very much appreciated … Thanks in advance

VK2HIM

Salutations from N3KJE

  1. Go to the ND page. It should be accessible from http://IP_ADDRESS:1880

  2. On the “Hamburger icon” (4 horizontal lines on the top right of the screen) click the option Import → Clipboard

  3. In the “Paste nodes here” block, paste the following code:

[{"id":"3184c35f.9c6f6c","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"bea7e640.b05dd8","type":"mqtt in","z":"3184c35f.9c6f6c","name":"","topic":"Southampton/WD/","qos":"2","broker":"7a840a3e.085984","x":268,"y":109,"wires":[["f7b827d2.c06358"]]},{"id":"f7b827d2.c06358","type":"split","z":"3184c35f.9c6f6c","name":"","splt":" ","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":494,"y":131,"wires":[["de3d28a6.46d508"]]},{"id":"de3d28a6.46d508","type":"switch","z":"3184c35f.9c6f6c","name":"route","property":"parts.index","propertyType":"msg","rules":[{"t":"eq","v":"1","vt":"num"},{"t":"eq","v":"2","vt":"num"},{"t":"eq","v":"3","vt":"num"},{"t":"eq","v":"4","vt":"num"},{"t":"eq","v":"5","vt":"num"}],"checkall":"true","repair":false,"outputs":5,"x":689,"y":539,"wires":[["ac7b2495.072268"],["f67efdc1.922c9"],["3edabef.cfd1642"],["dc4caa53.e2c0b8","1d404512.0be63b"],["e959a54f.7adff8"]]},{"id":"ac7b2495.072268","type":"ui_gauge","z":"3184c35f.9c6f6c","name":"","group":"d702a2bb.6772d","order":1,"width":"0","height":"0","gtype":"gage","title":"Average Wind Speed","label":"Knots","format":"{{value}}","min":0,"max":"50","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":1132.5,"y":133,"wires":[]},{"id":"f67efdc1.922c9","type":"ui_gauge","z":"3184c35f.9c6f6c","name":"","group":"d702a2bb.6772d","order":2,"width":"0","height":"0","gtype":"gage","title":"Wind Gust","label":"Knots","format":"{{value}}","min":0,"max":"50","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":1105,"y":232,"wires":[]},{"id":"3edabef.cfd1642","type":"ui_gauge","z":"3184c35f.9c6f6c","name":"Degrees","group":"d702a2bb.6772d","order":3,"width":"0","height":"0","gtype":"compass","title":"Wind Direction","label":"Degrees","format":"{{value}}

my bad
try again a new .zip update
the new setting was only being saved if you clicked on test send

Brian

I am still not seeing the field names as part of the message. Just the values (and not a key/value combo).

Renato

my bad again
use this version
http://www.weather-display.com/downloadfiles/cronwdmqtt.zip
vers 1.7

Thanks Renato - good to see a fellow ham!!

I’ll follow these instructions when I’m home from work and report back - cheers!

Hi Renato,

I’m having trouble configuring WD for mqtt - I’m not sure if I’ve configured it correctly or I’ve got Node-Red wrong?

Server - I’m using the IP address of the PC that’s running WD - is that correct? 192.168.1.75
Port 1883
Client ID I left blank but I tried 99 with no luck
QoS - At most once
Topic is Gerringong/WD/
blank username password
update rate 60 or 10 made no difference

When cron WD MQTT ver 1.6 starts - opening teh windo shows blank in Received and Status and Message to send says:

This action requires a connected state (C:\Users\Brian\Documents\tmssoftware\TMS MQTT\TMS.MQTT.Client.pas, line 518)

I configured Node-Red using your script and matched the same settings for Topic - QOS 0 - Server 192.168.1.75 and port 1883 - blamk Client ID and user/password blank.

After deploying and restarting flows it says connecting with nothing else.

Any clues? Thanks

Brian

Yes, now it works.

Thanks again
Renato

vk2him

First I would download mqtt-spy. This utility allows you to connect to the mqtt server and validate that mosquitto is properly configured and up and running.

You can find it here https://github.com/eclipse/paho.mqtt-spy/wiki/Downloads

Once started go to Connections → Manage connections to create a new one for your environment. With a new connection created, click on “Control Panel”. You should now see a button for your server appear in the middle section. Click on it. If it turns green then it means that your mqtt server is up and running.

Next check your WD setting:

I am attaching a screenshot of my setup. Please noe:

  1. I am using a non-standard port (i.e. 30925 rather than the default1883)
  2. make sure that the "send customclientraw is NOT checked since the version of teh dashboard I shared with you was created using clientraw.

Let me know if this works for you.


Thanks for the help Renato - I think I see my problem. I’m assuming your ip address 192.168.1.50 you have setup in WD is your mqtt server?

I didn’t realise I needed a mqtt server - I have a Solar/Power meter that directly outputs mqtt and I’ve been able to connect and receive data using Node-Red running on my Synology directly connected to the solar meter without needing a mqtt server. I (wrongly) assumed WD was also able to also directly connect without a mqtt server.

So now I’ll investigate mqtt servers - it may be better to run the mqqt server on my Synology than on the Windows10 laptop that WD is currently running.
I’ll try again tonight
Regards
Ian

Might be worth to check out CloudMQTT.com, they have a free option.

Ian

If this is an option for you, I would probably invest in a Raspberry Pi 3B+ and deploy Mosquitto there. Later on, you may also want to install there MySQL/MariaDB etc.

Personally I like to “leave the NAS” alone.

My 2c

Renato

Thanks Renato - I have mariaDb and mysql working successfully on my Synology - it gathers data from my Blitzortung lightning detector:

http://www.theshackbythebeach.com:443/mybo/index.php?bo_page=map

I’ll investigate teh options tonight - thansk for the suggestions - cheers

Hi Renato - It’s taken me a few days to do testing etc

Partial success - long story short - for unknown reasons running Node-red on my Synology won’t allow me to connect to the Mqtt broker thats also running on the Synology. WD is able to publish to the broker fine and I can subscribe to the broker using a test mqtt.fx app. I can connect from Node-red on the synology to another mqtt device on a different IP and it works fine, I even created some new gauges to display the other mqtt data. I then tried installing Node-red on my windows laptop and I could connect to the Synology broker and display the WD gauges.

So something is broken on the Synology that doesn’t allow Node-red to connect - I tried localhost and 127.0.0.1 as well as the IP of the Synology without luck. I even tried a different Node-red docker container and it had the same problem.

I’ll keep checking … thanks anyway
Cheers

Brian/Ian

Here is an example of the “art of possible” I came across last night.

https://pysselilivet.blogspot.com/2018/09/weatherlink-weather-template_26.html

To implement this, all is needed is to replace the WeatherLink source (i.e. the http request object) with the message coming from WD. Also, some of the Function objects would have to be modified as well to account for available datapoints.

Next

  1. Currently working on a NR script that would persist the message in a MySQL database. Trying to decide if storing the message as a single JSON datatype or individual columns…
  2. Persist the data in a timeseries database (i.e. InfluxDB) for later rendering on a dashboard (i.e. Grafana)

Next we could look at replacing the FTP process all together by having a script run on the web server that would subscribe to the message and convert it into a text file to b e stored locally (i.e. pull vs push).

Will keep you posted on the progress

Renato

And there you have it.

Data is being persisted in an InfluxDB TimeSeries database and the data is rendered on the Grafana dashboard.

For this to be possible, I had to:

  1. Strip the “Client ID” from the message, since its presence would result in an incorrect JSON structure

  2. All the values in the customomclientraw file are entered in double quotes. This means that every datatype would be created as strings. I went thru the entire structure of the file and removed the double quotes on those fields that would generate float values (i.e temp, tempTL, tempTH, intemp etc)

  3. Brian, I noticed that the 2 values for presstrendval and temptrend should be of type float but they are actually being generated as strings (since values greater than 0 are created with a leading “+” sign). Is this by design?


re your Q 2, is this for the customclientraw, or?

also have you come across the MQTT skin
Belchertown

Brian

Yes, I was referring to the customclientraw.

Also, do you think you could make the Client ID optional? Or better yet, make it part of the perhaps part of the customclientraw file. Shirt term, I am going to hard code the value “ClientID”:“Southampton” at the front of teh file… As I mentioned, I had to remove it from the message since it is creating an invalid JSON array.

As far as the MQTT skin Belchertown, yes I have. Will look into that next. As for its ability to “auto refresh” the Belchertown page, it relies on the mqtt websocket service. To test that, I will need to modify my mosquitto/mqtt deployment since at present I am not exposing the service.

Thanks
Renato