Jump to content

New VATSIM Dataserver

Aidan Stevens

By Aidan Stevens,
in General

 Share

Recommended Posts

Aidan Stevens

Aidan Stevens

Posted
Posted

We've heard your requests, and we understand that the current, colon-delimited datafile has been a pain to deal with and acts as a barrier to entrance for many new developers. As such, we have rewritten the dataserver which comes with a brand new JSON file that includes a 1:1 field mapping with the colon-delimited file. Documentation for this new file can be found here. This is officially v1.0 and is now considered in production.

We are also doing our best to embrace open-source and community involvement with our technology, so you can find the code for the new dataserver on our GitHub, here. Please feel free to create issues and open pull requests.

Please note that this does not mean we are deprecating the old file! We will continue to maintain that file for years to come to ensure that all of the current VATSIM tools you know and love continue to function properly.
Please see the latest tech blog post for an update on the legacy data feed.

Thanks for reading, and I hope you enjoy!

Regards,

  • Like 7
  • Thanks 4
Link to comment
Share on other sites
Luke Kolin

Luke Kolin

Posted
Posted (edited)

Is there any metadata in the feed as to when it was generated, or is it real-time?

Cheers!

Luke

PS: Do you support TLS on the data server?

Edited by Luke Kolin

... I spawn hundreds of children a day. They are daemons because they are easier to kill. The first four remain stubbornly alive despite my (and their) best efforts.

... Normal in my household makes you a member of a visible minority.

Link to comment
Share on other sites
Nick Harasym

Nick Harasym

Posted
Posted
4 hours ago, Mateusz Zymla said:

Would love to know it as well, if it's live, or at least close-to-real (as previous data txt distributed is basically too old for some real-time managing apps), it's a game-changer.

Maybe Kafka will be a thing, more info shortly???

  • Like 1
Link to comment
Share on other sites
Mateusz Zymla

Mateusz Zymla

Posted
Posted (edited)

Yeah, we are thinking about introducing an A-CDM plugin/software/website (not decided yet), and for now on there are no publicly available any complete soft for that, so we would be forced to extract data from EuroScope. We don't know how we'd like it to look like, but extracting data directly from VATSIM data servers would make our life easier for sure 😉

Edited by Mateusz Zymla

Mateusz Zymla - 1131338

VATSIMer since 2009, IRL pilot rated.

Link to comment
Share on other sites
Luke Kolin

Luke Kolin

Posted
Posted
3 hours ago, Nick Harasym said:

The above example works however it was written for our "closed beta" of the Kafka feed. Building up the consumer will be the same however the data will change slightly. 

Is it too late to get access to this?

Cheers!

Luke

... I spawn hundreds of children a day. They are daemons because they are easier to kill. The first four remain stubbornly alive despite my (and their) best efforts.

... Normal in my household makes you a member of a visible minority.

Link to comment
Share on other sites
Luke Kolin

Luke Kolin

Posted
Posted

Thanks. Any chance we can get an 'effective date' field in the data server JSON feed?

Cheers!

... I spawn hundreds of children a day. They are daemons because they are easier to kill. The first four remain stubbornly alive despite my (and their) best efforts.

... Normal in my household makes you a member of a visible minority.

Link to comment
Share on other sites
Jason Cochran

Jason Cochran

Posted
Posted

+1 for Kafka. Very nice, guys.

Though we have struggled with what seems to be higher than normal CPU consumption of the brokers, we are currently using Kafka at scale in a very large enterprise... and it's super dependable and mostly runs itself.

Link to comment
Share on other sites
  • 3 weeks later...
  • Board of Governors
Matt Bozwood-Davies

Matt Bozwood-Davies

Posted
  • Board of Governors
Posted
On 4/19/2020 at 12:04 AM, Luke Kolin said:

Any chance we can get an 'effective date' field in the data server JSON feed?

This was pushed up in a new version the other day with a non-breaking change. There is a new top-level bject "general" which contains the following properties

  • version - incremental changes
  • reload - not sure what this was used for in the txt file but we left it there for similarity
  • update - same format as the txt timestamp
  • update_timestamp - UTC timestamp iso format
  • connected_clients
  • unique_users 
"general": {
  "version": 1,
  "reload": 1,
  "update": "20200524002549",
  "update_timestamp": "2020-05-24T00:25:49.9636563Z",
  "connected_clients": 1168,
  "unique_users": 1105
}

---

On 4/26/2020 at 8:14 PM, Mario Trunz said:

Is there a way we can get the last 10-15 flights based on the VATSIM User ID?

Please see the API for this: 

 

Matt Bozwood-Davies
 Vice President Technical Development    
## [email protected]
## www.vatsim.net
## ##
Vatsim-color-tagline.png.7b888e6f6d0ad80d1d50939ae0a60bcc.png
Link to comment
Share on other sites
Luke Kolin

Luke Kolin

Posted
Posted

Fantastic - thank you!

Cheers

Luke

... I spawn hundreds of children a day. They are daemons because they are easier to kill. The first four remain stubbornly alive despite my (and their) best efforts.

... Normal in my household makes you a member of a visible minority.

Link to comment
Share on other sites
  • 2 weeks later...
  • 1 month later...
Joshua Channin

Joshua Channin

Posted
Posted (edited)

Is it intended that the resource is blocked by CORS for client side javascript requests as an example? "Access to fetch at 'http://cluster.data.vatsim.net/vatsim-data.json' from origin 'http://localhost:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled."

@Aidan Stevens

Edited by Joshua Channin
Link to comment
Share on other sites
  • 1 month later...
Joshua Channin

Joshua Channin

Posted
Posted (edited)

That makes sense Nestor. Mostly wanted to confirm the behavior was intended. I do think from a developer advocacy level, this restricts working with VATSIM to those developers able and willing to support their frontend client with an application server caching the data rather than a frontend client only and the caching occurs at the VATSIM level. But all things considered with VATSIM's free-to-use services as you mentioned, I concur with your thoughts.

Edited by Joshua Channin
Link to comment
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share
Go to topic listing
×
×
  • Create New...