Jump to content

METAR Service for Scripts


Recommended Posts

Good Day All...

 

I'm looking at the logs, and seeing lots of different creative ways to retrieve the METAR data from VATSIM..

 

If you are using the old www.vatsim.net/data/search_metar.php?id=ICAO feed, please move to metar.vatsim.net/ICAO immediately.. the old search_metar.php will be going away very shortly...

 

Additionally, if you need ALL the metars, or a large number of them, please retrieve www.vatsim.net/data/metar.txt

 

This file contains all metars currently active in the network.

 

Please ONLY retrieve this file once every 15 minutes.. if you retrieve any sooner than that, your site WILL be banned..

 

Is there any other functionality in this area that would help you?

Mike Evans

Link to post
Share on other sites

Hey Mike,

 

Just one thing, I have a WX page that's called from a link on our home page. There is no cron, however I can't throttle users selecting this page as it's a link on our homepage. I hate to think that I need to create a table and cron to store this data locally, but I don't want our site banned because 10 users all wanted to see the metars in a short time.

 

Other ideas perhaps?

Gerry Hattendorf

ZLA Webmaster

VATSIM Supervisor

Link to post
Share on other sites
Hey Mike,

 

Just one thing, I have a WX page that's called from a link on our home page. There is no cron, however I can't throttle users selecting this page as it's a link on our homepage. I hate to think that I need to create a table and cron to store this data locally, but I don't want our site banned because 10 users all wanted to see the metars in a short time.

 

Other ideas perhaps?

 

Download the result and store it in a file locally? Requires very little change to your code that way - other than the check to see if it's x minutes out of date

 

Regards,

A...

0
Link to post
Share on other sites

That's exactly how I process VATSIM data as well as weax. Download file and save it locally every 15 min. Every time a person makes a data request on the site, we pull and process from the local copy instead of downloading a new one. Provides much faster response as well since the file is on the server...just ensure you point to its local address and not an http: address!

 

Dave

Link to post
Share on other sites
Thanks guys, I'm working with some VERY OLD code, and the page we have is just a stupid anchor to the vatsim site. I guess I'll busy for the next day to update my database, and my scripts.

 

Nobody said you have to update the database So if you're wanting to cut corners you can do as suggested and save the file locally

 

So we would need to setup a cron job to take down the file locally and present it on the local server every 15 minutes, correct? I think I understand.

 

Nope - if you're existing method is to request the metar when a user views your site, all you need to do is modify your script to download the file and store it locally. Then, read the metar from the file. If the file is older than 15 minutes, it should then re-download it.

 

This way, you don't waste resources when nobody is looking at the metars (for example, late at night)

 

Regards,

A...

0
Link to post
Share on other sites
If you are using the old http://www.vatsim.net/data/search_metar.php?id=ICAO feed, please move to metar.vatsim.net/ICAO immediately.. the old search_metar.php will be going away very shortly...

Will http://metar.vatsim.net/data/metar.php still remain? This is what VatsimPHP uses, and it gets that from the status.txt file itself.

 

Please ONLY retrieve this file once every 15 minutes.. if you retrieve any sooner than that, your site WILL be banned..

As a note, VatsimPHP retrieves this date every THIRTY minutes, for those of you worried if you are using VatsimPHP for your information.

Brian Beach

VatsimPHP Developer: http://www.bbflights.com/VatsimPHP/

AFA Detroit Hub Director: http://www.flyafa.com/

userbar603862kx9.gif

Link to post
Share on other sites

Nope - if you're existing method is to request the metar when a user views your site, all you need to do is modify your script to download the file and store it locally. Then, read the metar from the file. If the file is older than 15 minutes, it should then re-download it.

 

If you have only few requests a day it would be better remaining with the old system.. or even better, fetch the data you need and store only this with a timestamp, if somebody else requests them within 15 minutes you already have them if not download them.

 

It doesn't make sense to download the compete WX list every 15 minutes if you have 20 METAR requests a day.

 

Luca

Luca Benelli - C3 - P2

Link to post
Share on other sites

Would it not be better to only get the data when required? By this I mean if the data is requested a check should be made locally. If no current data exists (ie is older than 15 mins) then it should download and keep this data in the even somebody else requires the info.

 

If no request is made no data should be needed rather than a cron pulling data every 15 mins even if its not being used.

 

Thoughts?

Norman

sig_FSLBetaTester.jpg

Link to post
Share on other sites
  • Board of Governors
Would it not be better to only get the data when required? By this I mean if the data is requested a check should be made locally. If no current data exists (ie is older than 15 mins) then it should download and keep this data in the even somebody else requires the info.

 

If no request is made no data should be needed rather than a cron pulling data every 15 mins even if its not being used.

 

Thoughts?

 

Makes sense, Ideally what I think I will due is not display the METAR on the homepage but have an area for a person to check, and only download the data when they need it.

Matthew Cianfarani
Vice President - Network Infrastructure
VATSIM Board of Governors

Link to post
Share on other sites

I've finished updating my script, and due to file permission errors and "racing" with local text files, I decided to create a table in my database. A simple code snippet;

 

function GetMetar($icao)
{
  // Sanitize the user input to prevent SQL Injection
  $icao = mysql_real_escape_string($icao);
  $icao = strtoupper($icao);
  if (strlen ($icao) == 0)
     return "";

  $string = "";
  $curtime = time();

  // Lets look in the database to see if we have a METAR for this airport and it's within 20 minutes current use it.
  $query = mysql_query("SELECT * FROM currentIcoMetar where airportNameID = '$icao'");
  if ($metar = mysql_fetch_object($query))
  {
     $string = $metar->theMetarStuff;
     if ($metar->lastUpdateTime + 1200 < $curtime)
     {
        if (!$string = file_get_contents("http://metar.vatsim.net/$icao"))
           $string = "CURRRENT METAR UNAVAILABLE";
        else
           mysql_query("update currentIcoMetar set lastUpdateTime = $curtime, theMetarStuff = '$string' where airportNameID = '$icao'");
     }
  }
  else
  {
     if (!$string = file_get_contents("http://metar.vatsim.net/$icao"))
        $string = "CURRRENT METAR UNAVAILABLE";
     else
        mysql_query("insert into currentIcoMetar airportNameID , unixtime, theMetarStuff values ('$icao', $curtime, '$string')");
  }
  $rowString = "<tr><td cl[Mod - Happy Thoughts]=\"metarleft\">$icao</td><td nowrap cl[Mod - Happy Thoughts]=\"metarright\">$string<td></tr>";
  return $rowString;
}

 

 

As I'm caching METAR data for 20 minutes, this would save bandwidth, but I'm still plagued with a user wanting to check METARS several airports we don't normally keep.

 

So my question is, if we have a flood of pilots checking ICAO codes we don't store, is my site going to be banned because of unexpected visitors exceeding the 15 minute limit?

 

Cheers!

Gerry Hattendorf

ZLA Webmaster

VATSIM Supervisor

Link to post
Share on other sites

Ok, maybe I wasn't clear. If I put this snippet:

 

$OHAREMETARLOC = "http://metar.vatsim.net/kord";

.

.

.

$ordMetar = file_get_contents($OHAREMETARLOC);

echo $ordMetar ;

 

onto my homepage so anyone who comes to the site can see the current Ohare VATSIM metar , it will be hit EVERY time someone hits the page or refreshes the page.

 

I don't see the need to store the file off anywhere and just hit the service in real time.

Jeff Thomas

VP-IT

https://joinava.org

Link to post
Share on other sites

If your page should ever get thousand hits a minute, for whatever reason, that would be a thousand hits on the metar page.

 

For quite some time i used something like this (in php) :

 

$metartime=filemtime("mymetarfile.txt"); 
$now=time(); 
$diff=$now-$metartime; 
if ($diff > 900){

  download the KORD metar to mymetarfile.txt 

  }

echo the content of mymetarfile.txt

 

In this way you don't have to donwload the complete metar file, you donwload it at most once every 15 minutes and you don't download it if it is not needed.

 

This is good also for those who need more than one metar... if you get lots of requests you'd better donwload the complete file with a cronjob but for low traffic sites this should be enough.

Luca Benelli - C3 - P2

Link to post
Share on other sites

Ok, thanks Luca. If I ever get 1000 hits in a minute, I'll have a whole lot of bigger problems to deal with but no sense in p[Mod - Happy Thoughts]ing on the problem.

 

Here's what I came up with (pretty much the same as Gerry's above but with files instead of a database):

 

public static function getMetar($airportCode) {
	$delay=900; //15 minutes
	$metarFile=$airportCode."_metar.txt";

	//* Update metar file if needed
	if(!file_exists($metarFile) || time()-filemtime($metarFile) > $delay) {
		copy("http://metar.vatsim.net/".$airportCode, $metarFile);
	}

	return $file_get_contents($metarFile);

}

 

I only call it for Ohare right now, but this gives me the ability to call it with other codes if I ever want to expand. Of course, if I expand to too many airports, I'll switch and download the entire file and p[Mod - Happy Thoughts] it.

Jeff Thomas

VP-IT

https://joinava.org

Link to post
Share on other sites

Jeff.. what's the IP address of your web server, I can tell you pretty quickly if you'll need to download the whole file or not based on our logs.. If you have any doubts, download the whole file.. a .txt file is far less "expensive" for our server versus a php script, no matter how small it is..

Mike Evans

Link to post
Share on other sites
  • 2 weeks later...

Just check the publishing times in the METAR, you will find out, that just in a very rare numbers the metar will be published more often then every 30 minutes. Most stations publish them hourly, others less frequently.

Flo

Florian Harms

VATSIM Europe Division / DCRM

Supervisor

Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
×
×
  • Create New...