Jump to content

You're browsing the 2004-2023 VATSIM Forums archive. All content is preserved in a read-only fashion.
For the latest forum posts, please visit https://forum.vatsim.net.

Need to find something? Use the Google search below.
PLEASE READ - Webmaster Support Forum
This forum will be retired in the near future. Please direct all queries to our dedicated GitHub support page https://github.com/vatsimnetwork/developer-info/discussions 
Here you can find documentation on our services and we are continuing to migrate pertinent information into the Wiki pages https://github.com/vatsimnetwork/developer-info/wiki

Stats server

Miguel Frias

By Miguel Frias,
in VATSIM Webmasters

 Share

Recommended Posts

Miguel Frias

Miguel Frias

Posted
Posted

The stats web server has been producing quite a delay in getting data. I'm getting delays such as:

 

Rendered in 19121ms.

 

This used to be much quicker. Is there an index missing in a database/table? Is there a DNS issue due to the recent VATSIM server moves?

 

Miguel

Miguel Frias

Senior Instructor (I3) & Certified Pilot (P4), ZLA I-11 graduate

Portugal vACC Training Director (ACCPT2), VATEUD Operations Director (VATEUD8)

Portugal vACC, VATEUD, VATSIM

1107190.png1107190.png

Link to comment
Share on other sites
Abdelrahman M. Elfeky 1044

Abdelrahman M. Elfeky 1044

Posted
Posted
The stats web server has been producing quite a delay in getting data. I'm getting delays such as:

 

Rendered in 19121ms.

 

This used to be much quicker. Is there an index missing in a database/table? Is there a DNS issue due to the recent VATSIM server moves?

 

Miguel

 

Same over here ... it takes sometime for me as well

 

Regards,,

s5_logo.png
Link to comment
Share on other sites
Christoph Neukirch

Christoph Neukirch

Posted
Posted

There's no DNS issue, it's connecting on localhost. Simply there're some expensive, complex and locking queries which have to be tweaked in the future.

 

kind regards

Christoph

Link to comment
Share on other sites
Miguel Frias

Miguel Frias

Posted
  • Author
Posted

Christoph, it has been working quite well until the latest server upgrades (or at least that timeframe).

 

If it is a SQL thing, then could it be an index that got corrupted or dropped?

 

Miguel

Miguel Frias

Senior Instructor (I3) & Certified Pilot (P4), ZLA I-11 graduate

Portugal vACC Training Director (ACCPT2), VATEUD Operations Director (VATEUD8)

Portugal vACC, VATEUD, VATSIM

1107190.png1107190.png

Link to comment
Share on other sites
Christoph Neukirch

Christoph Neukirch

Posted
Posted

Miguel,

 

the stats database was exported on the old box and imported on the new one via CSV into exactly the same structure so i guess it's not a problem with the database itself. Any ways the tables are getting bigger and bigger over time and the queries are more expensive from time to time depending on the query. I've done some tuning already to some strange things, other queries need rework in the application to get rid of very expensive JOINs.

 

kind regards

Christoph

Link to comment
Share on other sites
Miguel Frias

Miguel Frias

Posted
  • Author
Posted

Hmmm. I would expect a "mysqldump" instead of a CSV. Otherwise the keys will probably need regenerating.

 

Miguel

Miguel Frias

Senior Instructor (I3) & Certified Pilot (P4), ZLA I-11 graduate

Portugal vACC Training Director (ACCPT2), VATEUD Operations Director (VATEUD8)

Portugal vACC, VATEUD, VATSIM

1107190.png1107190.png

Link to comment
Share on other sites
Christoph Neukirch

Christoph Neukirch

Posted
Posted

CSV dumps are much more efficient for import than normal SQL dumps. The key will always be rebuilt, no matter it's SQL or CSV dump. The key will never be in any dump format, this wouldn't make any sense.

 

kind regards

Christoph

Link to comment
Share on other sites
Miguel Frias

Miguel Frias

Posted
  • Author
Posted

Copy. Even then, just the simple login after the captcha takes 24 seconds rendering. Only a month ago it was almost immediate. Don't think that in just one month things would get to this point.

 

Miguel

Miguel Frias

Senior Instructor (I3) & Certified Pilot (P4), ZLA I-11 graduate

Portugal vACC Training Director (ACCPT2), VATEUD Operations Director (VATEUD8)

Portugal vACC, VATEUD, VATSIM

1107190.png1107190.png

Link to comment
Share on other sites
Christoph Neukirch

Christoph Neukirch

Posted
Posted

It's normal behavior like start swapping to the hard disk. When a query is starting to produce bigger temporary tables which are to big for in memory, this tables will be created to the disk. You get the point?

 

kind regards

Christoph

Link to comment
Share on other sites
Miguel Frias

Miguel Frias

Posted
  • Author
Posted

Christoph, I do. What I don't understand is how in a month the latency is huge compared to none which is what existed.

 

Miguel

Miguel Frias

Senior Instructor (I3) & Certified Pilot (P4), ZLA I-11 graduate

Portugal vACC Training Director (ACCPT2), VATEUD Operations Director (VATEUD8)

Portugal vACC, VATEUD, VATSIM

1107190.png1107190.png

Link to comment
Share on other sites
Rahul Parkar

Rahul Parkar

Posted
Posted

I have to agree with Miguel here.

 

Something isn't right, I know VATSIM is big, but the time it takes to process even the simple stats center login has jumped from near instant to 24 seconds as Miguel said... A jump like this doesn't just "happen".

 

Are we sure that everything was copied over and run exactly as it was supposed to be run? Something isn't running at full capacity, I'd be willing to guess it's something to do with the SQL, most probably the keys as Miguel said.

 

Cheers!

Rahul

Rahul Parkar

"On second thoughts Nappa, catch it, catch it with your teeth" -- Vegeta

Professional Nerd. (Professionally not professional)

Link to comment
Share on other sites
Tim Krajcar

Tim Krajcar

Posted
Posted
CSV dumps are much more efficient for import than normal SQL dumps.

 

I wrote the web portion of the current stats server, and it indeed does some complex queries against a medium-size dataset, but if the indexes are built properly and it's on good hardware, it should be relatively zippy. Certainly not 19 seconds.

 

I would guess one or both of those is not the case.

Tim Krajcar

Link to comment
Share on other sites
Jack Richardson 1125776

Jack Richardson 1125776

Posted
Posted

Hello,

 

Something else. As well as the server being very slow (my rendering time shows as 42138ms) to search by CID and name, the callsign search facility appears to be, at least presently defunct. When attempting these searches I the following shows up:

Internal Server Error

 

The server encountered an internal error or misconfiguration and was unable to complete your request.

 

Please contact the server administrator, [email protected] and inform them of the time the error occurred, and anything you might have done that may have caused the error.

 

More information about this error may be available in the server error log.

It's not really a great problem, but the Stats pages used to be very handy for quick searches, and now it's either very slow or doesn't work at all.

 

Thanks,

Jack

1125776.jpg
Link to comment
Share on other sites
  • 2 weeks later...
Miguel Frias

Miguel Frias

Posted
  • Author
Posted

Any news on this? Regular searches seem to be fine. To me it just takes 19 seconds or more after logging in.

 

Miguel

Miguel Frias

Senior Instructor (I3) & Certified Pilot (P4), ZLA I-11 graduate

Portugal vACC Training Director (ACCPT2), VATEUD Operations Director (VATEUD8)

Portugal vACC, VATEUD, VATSIM

1107190.png1107190.png

Link to comment
Share on other sites
Daniel Hawton

Daniel Hawton

Posted
Posted

Search by callsign doesn't work for me, at least with ATC callsigns... and it takes 30+ seconds before the standard HTTP 500 code page is displayed.

Link to comment
Share on other sites
 Share
Go to topic listing