Forecast vs Actual

Hi Tom,
Hope all well, just thought I’d share some new displays (incomplete) of rolling Forecast vs Actual.
http://www.australiawx.net/forum/index.php?topic=63.msg134#msg134
Kind regards,
Tony

Great work, Tony! :D/ =D> :D/

I especially like the Highcharts version. Any chance these scripts will be made available sometime?

Great charts, well done :thumbright:

Thanks Niko and Jerry,
I still have a long way to go. Going back and tidying code, still building more and refining.
Will take a while till I’m happy and although will never be to the guru’s standards I would be happy to share jerry when complete, although it is based on Cumulus mysql for actuals, though I’m sure you would be able to adapt that.

Just did my Cloud Cover (Sky %) chart. :lol:
Stand alone : https://southfranklinweather.com/weather/realtime_graphs_wxsim_cloudSA.php
and on my main Saratoga index page (may play with colors over time) : https://southfranklinweather.com/weather/

kind regards,

Greetings to all

Congratulations to Tony for his new creation of the sky cloud coverage chart.

It is a magnificent adaptation of the original (non-interactive) graphic by Henkka (nordicweather.net).

I think I have managed to adapt it to my WXSIM data ::

stand alone: http://kocher.es/weatherdisplay/wxsimgraph-solonubes-sql.php

Integrated in my WXSIM forecast page: http://kocher.es/previsiones_nuevo.php

Thank you very much Tony :smiley:

Saludos desde San sebastian

Javier

Hi Javier,

1: Thank you for the kind words and glad you were able to adapt it.
2: I wonder if the moderators would like to place this thread in the ChildBoard - Wxsim Display Utilities (as I don’t feel the track this is taking now, is core wxsim related).
3: Javier, I noticed on my version yesterday that I had not covered the issue of TimeZones/DST well, so have made the relevant mods to mine, also notice because you must have grabbed my files prior. That yours is displaying incorrectly in my timezone. You should see the very first entry as the RunTime of WxSim, then you should see the current red (time line). See image below of my version.

I tested my mods on a couple of other PC’s I have in different timezones and they now display correctly. ( php/java/mysql/highcharts ) lovely combinations working together but timezone’s are a pain between their interface.

Anyway, what I did was :

A: in the core php file adjusted to :


<?php
date_default_timezone_set('Australia/Hobart'); // change accordingly
$dst = date('I');
$utc_offset = (10 + $dst); // change 10 to (corrected) (1) I think for you

etc..

B: in the Highcharts Script adjusted to : (amended : 2018-05-08 16:58 AEST)


var currenttime = new Date();
var d = new Date(currenttime);
var yr = d.getUTCFullYear();
var mth = d.getUTCMonth();
var dy = d.getUTCDate();
var hr = d.getUTCHours();
var mn = d.getUTCMinutes();

etc....

C: in the Highcharts Script adjusted to :


    xAxis: {
        plotLines: [{
                dashStyle: 'ShortDot', // 'Solid','ShortDash','ShortDot','ShortDashDot','ShortDashDotDot','Dot','Dash','LongDash','DashDot','LongDashDot','LongDashDotDot' whatever ?
                color: 'tomato',
                width: 2,
                value:  Date.UTC(yr,mth,dy,hr,mn,0),
		zIndex:5
             }]	          
        },

etc.....

D: in the Highcharts Script adjusted to :


// Apply the theme
Highcharts.setOptions(Highcharts.theme);
Highcharts.setOptions({
	time: {timezone: 'Australia/Hobart', // change accordingly
	timezoneOffset: - <?php echo $utc_offset ; ?> * 60}
 });

Highcharts.chart('container', {
     time: {
         timezone: 'Australia/Hobart', // change accordingly
	timezoneOffset: - <?php echo $utc_offset ; ?> * 60
     },	
    chart:{

etc....

Hope that all makes sense matey, knowing you = you will figure it out. :lol:

Also adjusted my rolling 7 day F/Cast vs Actual / Forecast with above mods. ; https://southfranklinweather.com/weather/wxsimcompare.php Still have some validations to resolve.

Kind regards,


wxsimcloud1.png

Thank you Tony

Then go to apply your tips in the script :smiley:

Javier

Saludos desde San Sebastian, Spain

Thanks Niko,
Javier, sorry I think your offset is 1 not 2 ( https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
Tony

Hi just a quick update before bed,

Tidied up my Cloud Cover % so it fit better from a theme color on my main index page, and added some gradient to the cloud :

Main : https://southfranklinweather.com/weather/
Stand Alone : https://southfranklinweather.com/weather/realtime_graphs_wxsim_cloudSA.php

Added a couple of new charts : Temp and Dew Point High and Low (Differential/Delta) though in Rolling 7 Day (30 min interval to real time), thought it would be good to visualise (I’m happy with it) though will when time permits add 24 hr (Daily High/Low) Delta as well:

Stand Alone : https://southfranklinweather.com/weather/realtime_graphs_wxsim_Act_WX_Delta.php
New Comparison page : https://southfranklinweather.com/weather/wxsimcompare.php

Tidied up some scripts and layouts on the Forecast v Actual (which is included in the above, hopefully from my testing I have covered the Timezone / DST functionality to display correctly, regardless of viewers timezone.

Stand Alone : https://southfranklinweather.com/weather/realtime_graphs_wxsim_Act_WX.php

Heaps more to do, maybe next weekend get back to it, bad weather on way may get washed away :lol:

Kind regards,

Hi, Tony

I really like all your creations.
I feel a healthy envy :lol:

I’m looking forward to knowing your system to create the comparisons

Javier

I agree, its looking awesome :smiley:

Hi Javier,
I spent tooo much time on this revamping for ease of use by others I have not had time to enhance my own as I wanted to.
any way tooo tired to add detail, files should speak for themselves.
Hopefully it works.
All must realise this is for use with Cumulus and not WD. should be easy to modify.
Cheers matey,

reply on forum as I will be away for a while.

regards,
tony


wxsimcsvmysql.zip (104 KB)

Hi,
Additional options for display : https://southfranklinweather.com/weather/wxsim_forecast_charts.php
And a table view of current forecast detail : https://southfranklinweather.com/weather/wxsim_forecast_table.php
Regards,

Regards Tony

I thank you very much for the possibility you offer us to share your code.

Then I start to study it.

Many thanks :slight_smile:

Saludos desde San Sebastian

Javier

Hi Javier,
Thanks for that, I have had a few hours sleep now and felt bad not supplying a process.

1: Edit : " 1wxsim_settings.php " with your info.
1A: Upload entire Folder : " wxsimcsvmysql " with all it’s files, to your server.

2: In your browser run : " your website address /wxsimcsvmysql/2wxsim_create_tables_views_run_once_only.php "
2A: Check in Cpanel/MySql (or whatever you use) to ensure new Tables : “wxsim_raw_csv” , “wxsim_historic” and View : “wxsim_current_csv_vw” have been created, they will be empty but structure there.

3: In your browser run : " your website address /wxsimcsvmysql/3wxsim_create_views_views_run_once_only.php "
3A: Check in Cpanel/MySql (or whatever you use) to ensure new Views : “RealtimeComparevw” , “WxsimComparevw” and “WxSimActTempLowHighDelta” have all been created, they will be empty but structure there.

4: In your browser run : " your website address /wxsimcsvmysql/4wxsim_cron_csv_import.php "
4A: This will Import “latest.csv” file contents into “wxsim_raw_csv” and also use the “wxsim_current_csv_vw” concat the Raw DateTimes field and Add the output of that view into the “wxsim_historic”.
4B: Check in Cpanel/MySql (or whatever you use) to ensure All Tables and Views now contain data. (you can now run one of the Graph Files which should display data (though historic data will only be from current run). That will build over time.

5: Finally, You need to set the " 4wxsim_cron_csv_import.php " file to run via Cron after your WxSim Run and after “latest.csv” has uploaded to your server.
( eg. every 6 hours or however often you upload ).

6: You can then delete :"2wxsim_create_tables_views_run_once_only.php " and "3wxsim_create_views_views_run_once_only.php " from your server.

Hopefully that all works.
I have no idea what other country “latest.csv” files are like so possability it may fail. If so look at :
$fieldseparator = “,”; // CSV Field Seperator
$lineseparator = “\n”; // CSV Line Seperator

Other than that I can not help, you will need to get assistance from gurus / modify to suite.
Just realised I did not make the Units Variable, and won’t so maybe someone else can, you’ll need to change those in the Graph files if you need to eg. C to F etc. note: no conversions occur as it is wxsim data.

Kind regards,

Regards Tony

I feel that you have slept little (I suspect that we have the same profession) :D.

I have installed your script on my server; the tables and the views have been created perfectly.

The only view that has been populated with data is “RealtimeComparevw”.

http://kocher.es/wxsimcsvmysql/4wxsim_cron_csv_import.php

http://kocher.es/wxsimcsvmysql/latestwxsimsqlsave.txt

All other tables and views remain empty.

Maybe it’s a problem with the decimal sign (,)

Thank you very much for your interest and expertise :smiley:

I made the following change to the file “4wxsim_cron_csv_import.php”:

Change decimal sign (,) to (.)

foreach(explode($lineseparator,$csvcontent) as $line) {

    $lines++;
    
    $line = str_replace(",",".",$line); // replace the decimal sign (,) by decimal point (.)

    $line = trim($line," \t");

    $line = str_replace("\r","",$line);

    /************************************
    This line escapes the special character. remove it if entries are already escaped in the csv file
    ************************************/
    $line = str_replace("'","\'",$line);
    
    $line = str_replace(",",".",$line); // cambia las comas decimales(,) por puntos (.))

And, now it seems that the decimal sign is correct:

http://kocher.es/wxsimcsvmysql/latestwxsimsqlsave.txt

However, the tables and views are not populating #-o

       [url=http://view-source:http://kocher.es/wxsimcsvmysql/latestwxsimsqlsave.txt[/url]

Hi Javier,

mm I had not thought of people having wxsim in hour intervals (mine is in 30 min intervals).

You will need to manually (Drop/Delete) the 3 views :“RealtimeComparevw” , “WxsimComparevw” , “WxSimActTempLowHighDelta”.

Then run the attached file “3wxsim_create_views_views_run_once_only2.php”

Script changed from (30 * 60) to (60 * 60)

oh and if that works, you will need to manually change all graph files Subtitle text from “Rolling 7 Days + 30 min Realtime” to "Rolling 7 Days + 1 hour Realtime

I’m at work now, so no more from me for a while.

Regards


3wxsim_create_views_views_run_once_only2.zip (1.28 KB)

Hi Javier,

The issue with yours seems to be that your latest.csv only contains around 43 fields where as mine has 76 (I thought all csv files were same - obviously not).
I cannot rewrite for you nor assist until this weekend.

But what you can do is manually delete the “wxsim_raw_csv” table.
Then in “2wxsim_create_tables_views_run_once_only.php” you will need to remove lines from below section of create table to match/ leaving only the ones in your csv.
Then check all other files do not have a field that is not present in your csv.


// sql to create "wxsim_raw_csv" table
$sql = "CREATE TABLE IF NOT EXISTS $wxsimrawcsvtable (
  `Year` int(4) DEFAULT NULL,
  `Month` int(2) DEFAULT NULL,
  `Day` int(2) DEFAULT NULL,
  `Time` decimal(10,1) DEFAULT NULL,
  `WXType1` varchar(20) DEFAULT NULL,
  `WXType2` varchar(20) DEFAULT NULL,
  `Temperature` decimal(4,1) DEFAULT NULL,
  `HiTemp` decimal(4,1) DEFAULT NULL,
  `LowTemp` decimal(4,1) DEFAULT NULL,
  `RelHum` decimal(4,1) DEFAULT NULL,
  `DewPt` decimal(4,1) DEFAULT NULL,
  `WetBulb` decimal(4,1) DEFAULT NULL,
  `WindSpd` decimal(4,1) DEFAULT NULL,
  `WindDir` decimal(4,1) DEFAULT NULL,
  `TotPrcp` decimal(4,2) DEFAULT NULL,
  `SnowDpth` decimal(4,1) DEFAULT NULL,
  `SLP` decimal(6,1) DEFAULT NULL,
  `StnPres` decimal(6,1) DEFAULT NULL,
  `WindChl` decimal(4,1) DEFAULT NULL,
  `HeatInd` decimal(4,1) DEFAULT NULL,
  `VlyTmp` decimal(4,1) DEFAULT NULL,
  `HillTmp` decimal(4,1) DEFAULT NULL,
  `TLvl1` decimal(4,1) DEFAULT NULL,
  `T850mb` decimal(4,1) DEFAULT NULL,
  `10p5Thk` decimal(6,1) DEFAULT NULL,
  `LI` decimal(4,1) DEFAULT NULL,
  `SkyCov` decimal(4,1) DEFAULT NULL,
  `SCL1` decimal(4,1) DEFAULT NULL,
  `SCL2` decimal(4,1) DEFAULT NULL,
  `SCL3` decimal(4,1) DEFAULT NULL,
  `SCL4` decimal(4,1) DEFAULT NULL,
  `SCL5` decimal(4,1) DEFAULT NULL,
  `VisTrans` decimal(4,1) DEFAULT NULL,
  `VIS` decimal(4,1) DEFAULT NULL,
  `SunAlt` decimal(4,1) DEFAULT NULL,
  `SolarRad` decimal(4,1) DEFAULT NULL,
  `UVIndex` decimal(3,1) DEFAULT NULL,
  `FreezingLevelhighest` decimal(6,1) DEFAULT NULL,
  `SnowLevel` decimal(6,1) DEFAULT NULL,
  `Convectionindex` decimal(4,2) DEFAULT NULL,
  `Severeindex` decimal(4,2) DEFAULT NULL,
  `1minGust` decimal(4,1) DEFAULT NULL,
  `10minGust` decimal(4,1) DEFAULT NULL,
  `1hrGust` decimal(4,1) DEFAULT NULL,
  `6hrGust` decimal(4,1) DEFAULT NULL,
  `15m50ftTemperature` decimal(4,1) DEFAULT NULL,
  `GrassTemperature` decimal(4,1) DEFAULT NULL,
  `SoilSurfaceTemperature` decimal(4,1) DEFAULT NULL,
  `SoilTemperatureDepth1` decimal(4,1) DEFAULT NULL,
  `SoilTemperatureDepth2` decimal(4,1) DEFAULT NULL,
  `SoilTemperatureDepth3` decimal(4,1) DEFAULT NULL,
  `SoilTemperatureDepth4` decimal(4,1) DEFAULT NULL,
  `SoilTemperatureDepth5` decimal(4,1) DEFAULT NULL,
  `SoilMoistureDepth1` decimal(4,1) DEFAULT NULL,
  `SoilMoistureDepth2` decimal(4,1) DEFAULT NULL,
  `SoilMoistureDepth3` decimal(4,1) DEFAULT NULL,
  `SoilMoistureDepth4` decimal(4,1) DEFAULT NULL,
  `SoilMoistureDepth5` decimal(4,1) DEFAULT NULL,
  `ET0ShortReferenceCrop` decimal(6,4) DEFAULT NULL,
  `ET0LongReferenceCrop` decimal(6,4) DEFAULT NULL,
  `ActualEvapotransipration` decimal(6,4) DEFAULT NULL,
  `TotalET0ShortReference` decimal(5,4) DEFAULT NULL,
  `TotalET0LongReference` decimal(5,4) DEFAULT NULL,
  `TotalActualEvapotranspiration` decimal(5,4) DEFAULT NULL,
  `IRDown` decimal(4,1) DEFAULT NULL,
  `IRUp` decimal(4,1) DEFAULT NULL,
  `SensibleHeat` decimal(4,1) DEFAULT NULL,
  `LatentHeat` decimal(4,1) DEFAULT NULL,
  `NetRadiation` decimal(6,1) DEFAULT NULL,
  `HeatConductedUptoSurface` decimal(4,1) DEFAULT NULL,
  `SoilTensionDepth1` int(4) DEFAULT NULL,
  `SoilTensionDepth2` int(4) DEFAULT NULL,
  `SoilTensionDepth3` int(4) DEFAULT NULL,
  `SoilTensionDepth4` int(4) DEFAULT NULL,
  `SoilTensionDepth5` int(4) DEFAULT NULL,
  `TotalIrrigation` int(4) DEFAULT NULL
)";

I also see many others have downloaded the files, though not given any feedback.
l also attached copy of my latest.csv for you to compare.

regards,


latest (5).zip (44.4 KB)

Thank you very much Tony

I appreciate your help and the effort you make for it.

I’m not in any hurry and I do not want to create problems in your work.

Thank you very much :wink:

Javier