Problem with Saratoga WU-forecast.php

Hi Ken,

I’ve noticed over the last few months the WU-forecast.php program won’t always retrieve the latest forecast from Weather Underground and has failed with various php errors.

I haven’t had a chance to look into it until yesterday when I found I was using the older version 3.03 so I updated to v3.04

With v3.04 I’m still getting an occasional error but this time the errors are different - In my dashboard I sometimes now get the following at the top and the forecast is blank :

Warning: Invalid argument supplied for foreach() in /Applications/XAMPP/xamppfiles/htdocs/wx/WU-forecast.php on line 550

Warning: Invalid argument supplied for foreach() in /Applications/XAMPP/xamppfiles/htdocs/wx/WU-forecast.php on line 564

Warning: Division by zero in /Applications/XAMPP/xamppfiles/htdocs/wx/WU-forecast.php on line 632

Note that I’ve inserted one extra line into my WU-forecast.php at 174 to hard-code the $SITE[‘fcsturlWU’] compared to yours so I can run WU-forecast.php standalone for testing, so the above line numbers will be one higher than your standard code.

If I run WU-forecast.php standalone when those errors appear, the forecast is blank, including the town name which is a period “.”, if I then click to force update it generally works.

If it helps, $SITE[‘fcsturlWU’] is set to

$SITE['fcsturlWU']  = 'http://www.wunderground.com/cgi-bin/findweather/getForecast?query=Gerringong%2C+Australia'; // Gerringong

Thanks
Ian

Hi Ian,

This is likely caused by a transient connection issue from your webserver to wunderground.com resulting in a ‘null’ forecast being seen.

I ran your wxforecast.php page with ?force=1 and in the view-source of the page it showed

which is (now) a fairly normal response from the access to wunderground.

Thanks for the tips Ken,

When I get the error this is in the page source:



<b>Warning</b>:  Invalid argument supplied for foreach() in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>713</b>



<b>Warning</b>:  Invalid argument supplied for foreach() in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>727</b>



<b>Notice</b>:  Undefined variable: WUforecasticons in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>795</b>



<b>Warning</b>:  Division by zero in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>795</b>



<b>Notice</b>:  Undefined variable: WUforecasticon in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>796</b>



<b>Notice</b>:  Undefined variable: WUforecasticons in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>801</b>



<b>Notice</b>:  Undefined variable: WUforecasticons in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>801</b>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>WeatherUnderground 0-day Forecast - </title>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
</head>
<body style="font-family:Verdana, Arial, Helvetica, sans-serif; font-size:12px; background-color:#FFFFFF">

<!-- WU-forecast.php (ML) Version 3.04 - 27-Feb-2018 on PHP 5.6.32 -->
<!-- temps in C -->
<!-- WU_API Raw URL='http://www.wunderground.com/cgi-bin/findweather/getForecast?query=Gerringong%2C+Australia' -->
<!-- WU API New URL='http://api.wunderground.com/api/42cac6f726e5ac53/forecast10day/geolookup/lang:EN/q/Australia/Gerringong.json' -->
<!-- loading from ./WU-forecast-json-0-en.txt (0 bytes) -->
<!-- using charsetInput='UTF-8' charsetOutput='ISO-8859-1' doIconv='1' doRTL='' -->
<!-- processing JSON entries for forecast -->
<!-- rawJSON size is 0 bytes -->
<!-- WU_prepareJSON: Success. Valid UTF-8. -->
<!-- json_decode returns  - No errors -->


<b>Notice</b>:  Undefined variable: WUforecasttext in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>873</b>



<b>Notice</b>:  Undefined variable: PHP_SELF in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>875</b>



Forecast blank? <a href="?force=1">Force Update</a>

  <table width="640px" style="border: none;" class="WUforecast">
          <tr align="center" style="background-color: #FFFFFF;">
      <td><b>WeatherUnderground 0-day Forecast: </b><span style="color: green;">
	   </span>
      </td>
    </tr>
	    <tr>
      <td align="center">
	    <table width="100%" border="0" cellpadding="0" cellspacing="0">  
	<!-- lengths title=0 cond=0 temps=0 -->
        </table><!-- end icon table -->
     </td>
   </tr><!-- end print icons -->
   	</table>
  <p>&nbsp;</p>

<table style="border: 0" width="640px" class="WUforecast">
	

<b>Notice</b>:  Undefined variable: WUforecasttitles in <b>/Applications/XAMPP/xamppfiles/htdocs/wx/wxforecast.php</b> on line <b>1019</b>

   </table>
<p>&nbsp;</p>
<p>Forecast from <a href="http://www.wunderground.com/cgi-bin/findweather/getForecast?query=Gerringong%2C+Australia">WeatherUnderground</a> 
for .
</p>
</body>
</html>

It would be “nice” if I could program it to retry a few times automatically on failure, particularly as my dashboard will show php error messages when it fails.

Thanks
Ian

One thing to try is to change the network timeout in the WU-forecast.php script

   $numberOfSeconds=4;   

to

   $numberOfSeconds=8;   

to give your webserver up to 8 seconds to successfully contact weatherunderground.com API.

A second thing to try is to avoid writing missing data to the cache (and reuse the cached version instead).

Change

      $fp = fopen($cacheName, "w"); 
	  if (!$fp) { 
	    $Status .= "<!-- unable to open $cacheName for writing. -->\n"; 
	  } else {
        $write = fputs($fp, $html); 
        fclose($fp);  
		$Status .= "<!-- saved cache to $cacheName (". strlen($html) . " bytes) -->\n";
	  } 

to

    if(strlen($html) > 20000) {
      $fp = fopen($cacheName, "w"); 
      if (!$fp) { 
        $Status .= "<!-- unable to open $cacheName for writing. -->\n"; 
      } else {
        $write = fputs($fp, $html); 
        fclose($fp);  
        $Status .= "<!-- saved cache to $cacheName (". strlen($html) . " bytes) -->\n";
      }
    } elseif (file_exists($cacheName)) {
      $html = implode('', file($cacheName));
      $Status .= "<!-- problemm loading from api .. reusing existing cache file -->\n"; 
      $Status .= "<!-- loading from $cacheName (" . strlen($html) . " bytes) -->\n"; 
    }

and that will only save the cache if >20K bytes are available, and if a cache file exists, load it instead.

Together, they may avoid the issue with slow/timeout/bad API returns by ‘recycling’ your last good forecast from them.

Excellent suggestions Ken - thanks for that - I’ll make the changes and will monitor. Cheers

Hi Ken,

For future reference in case others have this issue:- I had to slightly modify one of your suggestions to make it work properly. It seems that the string being > 20k wasn’t quite large enough as I got the same errors again. I looked at the json size when it failed and founded to needed to be >22k.

So the modified code is now

if(strlen($html) > 22000) {
      $fp = fopen($cacheName, "w"); 
      if (!$fp) { 
        $Status .= "<!-- unable to open $cacheName for writing. -->\n"; 
      } else {
        $write = fputs($fp, $html); 
        fclose($fp);  
        $Status .= "<!-- saved cache to $cacheName (". strlen($html) . " bytes) -->\n";
      }
    } elseif (file_exists($cacheName)) {
      $html = implode('', file($cacheName));
      $Status .= "<!-- problemm loading from api .. reusing existing cache file -->\n"; 
      $Status .= "<!-- loading from $cacheName (" . strlen($html) . " bytes) -->\n"; 
    }

Has been quite stable now for days where previously I was getting the error a few times per day.
Cheers!!!

Rather than size, I decided to look for an actual forecast in the contents of the JSON using

    if(preg_match('|fcttext|Uis',$html)) {
      $fp = fopen($cacheName, "w"); 
      if (!$fp) { 
        $Status .= "<!-- unable to open $cacheName for writing. -->\n"; 
      } else {
        $write = fputs($fp, $html); 
        fclose($fp);  
        $Status .= "<!-- saved cache to $cacheName (". strlen($html) . " bytes) -->\n";
      }
    } elseif (file_exists($cacheName)) {
      $html = implode('', file($cacheName));
      $Status .= "<!-- WU API return has no forecast .. reusing existing cache file -->\n"; 
      $Status .= "<!-- loading from $cacheName (" . strlen($html) . " bytes) -->\n"; 
    }

… that’s the one I’ll distribute shortly. I found a few forecast sites with <20000 characters in the return, but were still good forecasts. This test allows all the JSON with forecast content and only saves the cache then… If it’s time to reload the cache, the API result is checked for forecast content and saved if present, otherwise the existing cache file is reused with the slightly older forecast.

All distributed.

Standalone from https://saratoga-weather.org/scripts-WUforecast.php

Template users from https://saratoga-weather.org/wxtemplates/updates.php with a query of 13-Jun-2018, Base-World, Plugin-*

Thanks again Ken - I’ve now implemented your distribution,
Cheers
ian