New VATSIM Dataserver

[Aidan Stevens 12]

We've heard your requests, and we understand that the current, colon-delimited datafile has been a pain to deal with and acts as a barrier to entrance for many new developers. As such, we have rewritten the dataserver which comes with a brand new JSON file that includes a 1:1 field mapping with the colon-delimited file. Documentation for this new file can be found here. This is officially v1.0 and is now considered in production.

We are also doing our best to embrace open-source and community involvement with our technology, so you can find the code for the new dataserver on our GitHub, here. Please feel free to create issues and open pull requests.

Please note that this does not mean we are deprecating the old file! We will continue to maintain that file for years to come to ensure that all of the current VATSIM tools you know and love continue to function properly.

Thanks for reading, and I hope you enjoy!

Regards,

[Gunnar Lindahl 14]

Great work team - thank you!

[Luke Kolin 0]

Is there any metadata in the feed as to when it was generated, or is it real-time?

Cheers!

Luke

PS: Do you support TLS on the data server?

Edited April 13 by Luke Kolin

[Mateusz Zymla 1]

Would love to know it as well, if it's live, or at least close-to-real (as previous data txt distributed is basically too old for some real-time managing apps), it's a game-changer.

[Nick Harasym 5]

4 hours ago, Mateusz Zymla said:

Would love to know it as well, if it's live, or at least close-to-real (as previous data txt distributed is basically too old for some real-time managing apps), it's a game-changer.

Maybe Kafka will be a thing, more info shortly???

[Mateusz Zymla 1]

9 hours ago, Nick Harasym said:

Maybe Kafka will be a thing, more info shortly???

any further reading for now 😉 ?

[Nick Harasym 5]

15 minutes ago, Mateusz Zymla said:

any further reading for now 😉 ?

Basically, write a consumer client for a Kafka server and you will be able to consume a real time feed of data shortly. 

Most languages already have support for Kafka https://cwiki.apache.org/confluence/display/KAFKA/Clients

[Andrew Ogden 13]

Here's an open source one I've purpose written if you're interested for now: https://github.com/andrewogden1678/VATMap_Kafka

Edited April 14 by Andrew Ogden

[Nick Harasym 5]

The above example works however it was written for our "closed beta" of the Kafka feed. Building up the consumer will be the same however the data will change slightly. 

[Mateusz Zymla 1]

Yeah, we are thinking about introducing an A-CDM plugin/software/website (not decided yet), and for now on there are no publicly available any complete soft for that, so we would be forced to extract data from EuroScope. We don't know how we'd like it to look like, but extracting data directly from VATSIM data servers would make our life easier for sure 😉

Edited April 14 by Mateusz Zymla

[Luke Kolin 0]

3 hours ago, Nick Harasym said:

The above example works however it was written for our "closed beta" of the Kafka feed. Building up the consumer will be the same however the data will change slightly. 

Is it too late to get access to this?

Cheers!

Luke

[Luis Henrique 1]

Thank you so much!

[Nick Harasym 5]

  

22 hours ago, Luke Kolin said:

Is it too late to get access to this?

Cheers!

Luke

 

For the current Kafka offer that is floating out around there, yes we won't be giving credentials for it till we have the production one ready.

[Luke Kolin 0]

Thanks. Any chance we can get an 'effective date' field in the data server JSON feed?

Cheers!

[Mario Trunz 0]

Is there a way we can get the last 10-15 flights based on the VATSIM User ID?

[Jason Cochran 2]

+1 for Kafka. Very nice, guys.

Though we have struggled with what seems to be higher than normal CPU consumption of the brokers, we are currently using Kafka at scale in a very large enterprise... and it's super dependable and mostly runs itself.

[Martijn Rammeloo 2]

Is there something going wrong with the Charset?

Image

[Matt Bozwood-Davies 0]

On 4/19/2020 at 12:04 AM, Luke Kolin said:

Any chance we can get an 'effective date' field in the data server JSON feed?

This was pushed up in a new version the other day with a non-breaking change. There is a new top-level bject "general" which contains the following properties

• version - incremental changes
• reload - not sure what this was used for in the txt file but we left it there for similarity
• update - same format as the txt timestamp
• update_timestamp - UTC timestamp iso format
• connected_clients
• unique_users

"general": {
  "version": 1,
  "reload": 1,
  "update": "20200524002549",
  "update_timestamp": "2020-05-24T00:25:49.9636563Z",
  "connected_clients": 1168,
  "unique_users": 1105
}

---

On 4/26/2020 at 8:14 PM, Mario Trunz said:

Is there a way we can get the last 10-15 flights based on the VATSIM User ID?

Please see the API for this: 

 

[Luke Kolin 0]

Fantastic - thank you!

Cheers

Luke

[Martijn Rammeloo 2]

@Aidan StevensThe datafeed (JSON) seems to be serving outdated data.

Screenshot was taken seconds ago, 1833Z

Edited Wednesday at 06:34 PM by Martijn Rammeloo