Frequency inaccuracies in VATSIM API

[Damian McCracken]

The API which supplies frequency data to 3rd party tool sets like aivlasoft EFB appears to have an issue with the third figure behind the decimal separator. 

I have performed several flights and found that on frequencies where the 3rd digit after the decimal is a 5, it is incorrectly represented as a 0. Talking to the supplier of the EFB software, they dug into the data and they are showing in their software the exact frequency the VATSIM API is giving them. 

Screenshots below detail differences between vPilot and EFB and finally the JSON data supplied by the API with the EFB is using. I understand aivlasoft recently migrated over to the latest JSON format / API supplied from VATSIM. 

Can someone review the data / how the frequency data is constructed in the API ?

(My orginal thread https://forum.aivlasoft.com/topic/5113-frequency-differences-on-vatsim/)

Edited April 5, 2021 at 10:39 AM by Damian McCracken
spelling

[Frank Kopp]

Same at EDDB today - VatSpy also seems to use the faulty json data:

vPilot shows and requires these frequencies:

I'm using MSFS2020 with the Working title CJ4 and I had to tune up to the 3rd digit manually to hear the controller.

I assume if the controllers use frequencies with 3 digits the API should provide all 3 digits, right?

 

[Damian McCracken]

I should make clear from my original post, in MSFS I could not communicate with the controller until I manually tuned by adjusting to .xx5 from .xx0 on all occasions. 

[Frank Kopp]

Also raised this in the vPilot forum:

Frequencies which are given with 2 digits but actually need 3 (eg. .220/.225) not working - vPilot - VATSIM Community

[Ross Carlson]

The API, the data feed, and apps like VAT-Spy are just showing whatever frequency the controller reports as their frequency. Many controllers still change the third digit to a zero ... this is a holdover from many years ago when some older sims and/or pilot clients could not tune a frequency that had a 5 in the third decimal place.

The reason you see the correct frequency in vPilot is because vPilot adjusts the frequency to show the correct value. So, for example, if the controller reports 129.420 as their frequency, vPilot shows it as 129.425 since 129.425 is the correct real-world frequency, and 129.425 is what pilots can actually tune in their sim. (At least the sims that vPilot supports.)

[Frank Kopp]

Thanks for closing the other post, Ross, Wasn't sure if that would have been another team when it comes to vPilot. 

According to you answer the root cause is the Controllers then - they should put in 3 digits, right?

I just learned about the .025-steps - so this should only be an issue with .020 and .070 then, correct?

But if a user doesn't now this and use tools like VatSpy to note down frequencies (without double checking in vPilot) it doesn't work. 

Maybe the simplest solution would be to have controllers input all 3 digits then? 🙂

Another solution would be to make sure vPilot understands this issue and still connects the channel when the 3rd digit is missing (or a 0).

WDYT - what would be the best solution?

Thanks 

Frank

[Harry Sugden]

The short answer as to why we use 129.420 instead of 129.425 is that we have been made to believe that 129.425 would cause issues for pilots with older sims, mainly for text communications.

11 hours ago, Ross Carlson said:

Many controllers still change the third digit to a zero ... this is a holdover from many years ago when some older sims and/or pilot clients could not tune a frequency that had a 5 in the third decimal place.

This gives the impression that controllers are able to use frequencies with 5s on the end no bother? But I thought that this would still cause an issue with FS2004 default planes/FSX default planes/planes that only have 5 digit radios (which even includes the PMDG!)?

Edited April 7, 2021 at 12:34 PM by Harry Sugden

[Luke Brown]

If vpilot corrects for this, alongside all of the other remaining approved clients, we could look to switch to using the correct frequencies.

Any idea @Matt Bozwood-Davies?

Edited April 7, 2021 at 12:29 PM by Luke Brown

[Ross Carlson]

 

Quote

According to you answer the root cause is the Controllers then - they should put in 3 digits, right?

I think so, yes, but there are others that think we should stay with the current system. I'm not really sure why.

Quote

I just learned about the .025-steps - so this should only be an issue with .020 and .070 then, correct?

I think you mean .x20 and .x70, but yes.

Quote

But if a user doesn't now this and use tools like VatSpy to note down frequencies (without double checking in vPilot) it doesn't work. 

It sounds like you're using an aircraft that lets you enter the frequency manually and allows you to tune invalid frequencies. To my knowledge, even with 833 KHz frequency spacing, any frequency ending in .x20 or .x70 is not valid and should not be able to be tuned.

Note that this is not an issue for the vast majority of vPilot users. Most aircraft will not let you tune invalid frequencies.

I could definitely make a change to vPilot to auto-correct invalid frequencies, but really, the aircraft should not allow them to be tuned in the first place. (Unless I'm wrong, and .x20 and .x70 are actually valid, in which case, someone please explain to me how they are valid, and I will consider adding this capability to vPilot.)

Edited April 7, 2021 at 04:11 PM by Ross Carlson

[Ross Carlson]

3 hours ago, Harry Sugden said:

This gives the impression that controllers are able to use frequencies with 5s on the end no bother? But I thought that this would still cause an issue with FS2004 default planes/FSX default planes/planes that only have 5 digit radios (which even includes the PMDG!)?

That may have been the case with older pilot clients that didn't handle the missing third decimal place correctly. Currently, swift is the only pilot client that supports FS2004, and it properly adds the third decimal place so it's not an issue.

And this has never been an issue with FSX because it properly gives all three decimal places when clients like vPilot read the frequency from the sim, even if the panel only shows two decimal places.

Edited April 7, 2021 at 04:12 PM by Ross Carlson

[David Stone]

As to the 'Why?' controllers still use the old system (.xx0) is because the AFV crew has not given the go ahead to use the half-frequencies yet and we were all scolded for changing previously. And when we do get the go ahead, everyone will have to update their POF files and the AFV VCCS before that can take place. Does anyone out there have further info from the AFV team as to when that go ahead will be given would be my question.

[Frank Kopp]

9 minutes ago, Ross Carlson said:

 

I think so, yes, but there are others that think we should stay with the current system. I'm not really sure why.

I think you mean .x20 and .x70, but yes.

It sounds like you're using an aircraft that lets you enter the frequency manually and allows you to tune invalid frequencies. To my knowledge, even with 833 KHz frequency spacing, any frequency ending in .x20 or .x70 is not valid and should not be able to be tuned.

Note that this is not an issue for the vast majority of vPilot users. Most aircraft will not let you tune invalid frequencies.

I could definitely make a change to vPilot to auto-correct invalid frequencies, but really, the aircraft should not allow them to be tuned in the first place. (Unless I'm wrong, and .x20 and .x70 are actually valid, in which case, someone please explain to me how they are valid, and I will consider adding this capability to vPilot.)

 

I use MSFS2020 and tested this with the FlyByWire A320 and the Working Title CJ4 - both allow the input of .x20/.x70.

Also I use the EFB2 from Aivlasoft which conveniently list all frequencies along a route. It also allows setting of frequencies into the sim via buttons. As EFB2 reads the published frequencies it can't set the correct frequency if the controller and the API do not provide the 3rd digit. 

See this ticket: Frequency differences on VATSIM - EFB v2 - Support forum - AivlaSoft Support Forum

From my perspective as a user not knowing the history of older sims or clients I would say that the frequencies published via the VATSIM API should be identical to the frequencies of vPilot. Otherwise there will always be some inconsistency, right?

Thanks

[Ross Carlson]

13 minutes ago, Frank Kopp said:

From my perspective as a user not knowing the history of older sims or clients I would say that the frequencies published via the VATSIM API should be identical to the frequencies of vPilot. Otherwise there will always be some inconsistency, right?

Seems to me, that would just be moving the inconsistency to a different location. If the feed was updated to correct the frequency, then the feed would not match the controller's actual primary frequency. That could have implications elsewhere.

In my opinion, we should fix the actual problem, which is controllers using invalid frequencies.

[Andre Bohni]

Ehm i dont think you can blame the controller, because they use the frequency that are set be the ATC-TD or Ops-Department of there VACC (or how ever its called in your part of the World). And then i think they are told by the Division what Frequencies are to be used for what Position.

Correct me if i am Wrong here.

[Ross Carlson]

2 minutes ago, Andre Bohni said:

Ehm i dont think you can blame the controller, because they use the frequency that are set be the ATC-TD or Ops-Department of there VACC (or how ever its called in your part of the World). And then i think they are told by the Division what Frequencies are to be used for what Position.

Correct me if i am Wrong here.

I'm not saying anyone should be "blamed" ... I'm saying that the actual problem is that controllers are using invalid frequencies. The actual person that is responsible for that decision is irrelevant, and in fact it goes back to many years ago when these invalid frequencies were first used. Personally, I think that it was a bad decision way back then, and the original client developers should have added the proper third digit, and we never would have been in this mess to begin with. But, as they say, hindsight is 20-20, and that's water under the bridge.

[Tobias Dammers]

I think at this point it's a good idea to clarify the difference between FREQUENCIES and CHANNELS, and why 8.33 spacing makes it all such a terrible mess.

In the old 25 kHz days, things were simple. You had channels spaced 25 kHz apart, and you would refer to them by their exact frequency. 118.00, 118.025, 118.050, 118.075. Easy peasy.

But with the frequency space getting ever more crowded, and the precision of radio equipment improving, a new standard was devised that packs more channels into the same frequency space: "8.33 kHz channel spacing". The idea is simple: just put 3 times as many frequencies into the same space, such that there are two extra channels between each pair of 25 kHz channels. The distance between those new channels is now 8⅓ kHz, rounded to 8.33. So we getfrequencies like these: 118.000, 118.008333333...., 118.0166666666666..., 118.025, 118.03333333333...., 118.04166666666..., 118.050, etc. That's kind of awkward though, for two reasons: first, some of these have infinitely long decimal expansions; and second, those that don't look like 25 kHz channels. The latter bit matters because radio equipment configured for 25kHz channels will transmit and receive on a wider spectrum, so while the frequency may be the same, we still want to make sure that the radio equipment is configured for the narrower 8.33 frequency bands when we're transmitting or listening on an 8.33 channel. In a nutshell, if you tune a 25 kHz radio to 118.025, you will hear transmissions made from 25 kHz radios on 118.025, but you will also hear transmissions from 8.33 radio equipment on channel 118.030 (which uses the same 118.025 frequency, but over a narrower range), as well as 8.33 transmissions on the adjacent channels, 118.015 (actual frequency 118.0166666...) and 118.035 (actual frequency 118.0333333...), because your 25 kHz radio listens to the entire frequency range roughly from 118.0125 to 118.0375.

These problems are solved by rounding the awkward infinite expansions to the nearest 5 kHz, and adding 5 kHz to each frequency that aligns with a 25 kHz channel to distinguish the spacings. So in 8.33 spacing, the frequency 118.000 is reported as 118.005, 118.0083333333... becomes 118.010, 118.016666666... becomes 118.015, 118.025 becomes 118.030, and so on. And now if you dial 118.025, the radio equipment recognizes this as a 25 kHz channel, and listens on 118.025 +/- 0.0125; but if you dial 118.030, it listens on 118.025 +/- 0.004166666...

In other news, the number you dial into an 8.33-capable radio is no longer the actual frequency; it is a made-up number that references a 25 kHz or 8.33 kHz channel, and the radio equipment automatically maps that to the right frequency.

Note further some interesting consequences of this:

• Some possible inputs are not used at all. For example, 118.020 is not a thing, the nearest channels are 118.015 (118.016666... @8.33), 118.025 (118.025 @25), and 118.030 (118.025 @8.33)
• Some frequencies are covered by more than one channel: 118.025 and 118.030 both map to 118.025 MHz, but with different spacings.

See this page for some more in-depth explanations and a handy conversion table.

Unfortunately, the distinction between channel (what you enter into your radio, and what is displayed) and frequency (what you're actually transmitting on) isn't always made clear, and to add insult to injury, some VATSIM clients / sims are still incapable of using 8.33 frequencies. The combination of these two leads to the terrible mess we're seeing.

For practical day-to-day usage, the workaround is this:

• Limit frequencies / channels used on the network to the frequencies used for 25 kHz channels (e.g. 118.025)
• In airspaces that use 8.33 spacing IRL, report the corresponding 8.33 channel, not the frequency (e.g. 118.030)
• Remember that things are borked, and accept the fact that this may cause a deviation by 0.005 either way on either end. So when you're told to contact 118.030, and that doesn't work, try 118.025 instead.

Ultimately, what we should have is a proper implementation of how things work - I would suggest using actual frequencies on the network, ignoring the bandwidth difference (which is not an issue since we're not dealing with actual radio frequencies, and we don't have to separate them out on the receiving end), and leaving the conversion to and from channels to the pilot clients and/or sims.

Edited April 8, 2021 at 07:37 AM by Tobias Dammers

[Martin Loxbo]

That's probably the best explanation of 8.33 channels and frequency spacing that I've ever read. I can think of a few RW colleagues who could benefit from reading it. 😄 (The number of times you hear people surprised that they can hear the ATIS on 121.975 when the official channel is 121.980...!)

To add to the confusion on VATSIM: It used to be that only 4 digits are transmitted over the network for each frequency (I assume this must have been changed since the bottleneck now is that some flight sims don't support 8.33 spacing?). The leading 1 was dropped and only the next 4 digits were actually sent between the FSD servers. So 123.450 would be transmitted as 2345, and 118.020 and would be transmitted as 1802. I think there was a rounding "error" here (in some clients only perhaps?) that meant that 118.025 would be rounded up and be transmitted as 1803. So this meant that if a controller selected 118.025 this would be impossible to tune correctly in (some) aircraft radios / flight sims due to the frequency being matched as 1803 instead of 1802.

Maybe someone with a deeper technical knowledge of the inner workings of VATSIM could confirm the current status and whether we still need to use the "last digit has to be zero" workaround.

[Koen Meier]

Simconnect for anything but p3d v5 when it comes to fsx related sims is not able to handle the spacing. Msfs not sure, xplane not sure either. We could tune everything in the pilot client and the problem would be solved but that lacks immersion. Cause we do not want to drop support for older sims.

[Christoph Reule]

55 minutes ago, Koen Meier said:

Simconnect for anything but p3d v5 when it comes to fsx related sims is not able to handle the spacing. Msfs not sure, xplane not sure either. We could tune everything in the pilot client and the problem would be solved but that lacks immersion. Cause we do not want to drop support for older sims.

MSFS2020, from my knowledge (might be incomplete, please correct if necessary), isn't able to handle the spacing, either (still "old" FSX based SimConnect). XPlane 11, however, can do.

That's a "crucial" point when it comes to 8.33 kHz spacing on the network.

Wondering how they manage that on the "other" network...

Edited April 8, 2021 at 10:37 AM by Christoph Reule
typo

[Ross Carlson]

8 hours ago, Martin Loxbo said:

To add to the confusion on VATSIM: It used to be that only 4 digits are transmitted over the network for each frequency (I assume this must have been changed since the bottleneck now is that some flight sims don't support 8.33 spacing?).

Are you sure about this? AFAIK, frequencies have always been sent over the network with 5 digits. (Just the leading 1 dropped.) To my knowledge, the reason we are in this mess is because older sims would only report 2 decimal places to the pilot client, using a format known as BCD16, which only supports 4 digits. Because of this, 123.025 would be reported to the pilot client as 2302, and the pilot client would always blindly add a zero at the end and send it over the network as 23020. (When, in my opinion, it should have added a 5 at the end instead of a zero, when the 4 digit value coming from the sim ended in a 2 or a 7.) My assumption is that, at the time, they didn't foresee that the third decimal place would actually become significant due to 833 spacing, so they figured there was no reason to add a 5 instead of a zero, when the second decimal place contained a 2 or a 7. (In all other cases, adding a zero in the third decimal place is correct.)

Note that my development experience on VATSIM only goes back to 2004, so I may be wrong, and they did actually send only 4 digits over the network prior to that. I just find it unlikely because going from 4 to 5 digits would break all clients unless some intelligent translation was done on the server.

8 hours ago, Martin Loxbo said:

Maybe someone with a deeper technical knowledge of the inner workings of VATSIM could confirm the current status and whether we still need to use the "last digit has to be zero" workaround.

With the retirement of legacy clients, there's no longer any need to maintain that workaround that I'm aware of.

[Ross Carlson]

7 hours ago, Christoph Reule said:

MSFS2020, from my knowledge (might be incomplete, please correct if necessary), isn't able to handle the spacing, either (still "old" FSX based SimConnect). XPlane 11, however, can do.

MSFS does use SimConnect, but it's not limited to the set of variables and data formats that were available in FSX. They could add support for 833 just like P3D has.

A key thing to understand when it comes to SimConnect is that it's not the reading of the frequency that is limited to 2 decimal places. It's actually the writing of the value. In other words, SimConnect needs to provide a way to write a frequency with all three digits, in order to support 833. SimConnect has always provided all three decimal places when reading the frequency, going back to the first version of FSX.

P3D has added a way to write frequencies with all three digits. (In fact, you write the frequency in Hz, so it would even support frequencies that differ by less than 1 KHz.) Asobo just needs to do the same thing for MSFS, if they haven't already. (They may have already done this, I'm not sure.) This would allow aircraft builders (and pilot client authors) to set 833 frequencies via SimConnect.

Of course, this is all academic until we no longer support FS9 and FSX on the network, since those sims will never be able to tune an 833 frequency, or we force pilots of those older sims to use the pilot client (with something like a dot command or dropdown) to tune 833 frequencies.

Edited April 8, 2021 at 06:04 PM by Ross Carlson

[Jonas Kuster]

And there is no limitation in using a 5 for the 3rd decimal from the FSD protocol?

[Martin Loxbo]

1 hour ago, Ross Carlson said:

Are you sure about this? AFAIK, frequencies have always been sent over the network with 5 digits. (Just the leading 1 dropped.) To my knowledge, the reason we are in this mess is because older sims would only report 2 decimal places to the pilot client, using a format known as BCD16, which only supports 4 digits. Because of this, 123.025 would be reported to the pilot client as 2302, and the pilot client would always blindly add a zero at the end and send it over the network as 23020.

Looks like I got it sort of backwards - your explanation makes more sense.

1 hour ago, Ross Carlson said:

With the retirement of legacy clients, there's no longer any need to maintain that workaround that I'm aware of.

Which means as of now there is no need for the workaround?

[Ross Carlson]

3 hours ago, Jonas Kuster said:

And there is no limitation in using a 5 for the 3rd decimal from the FSD protocol?

To my knowledge, neither the network protocol nor the FSD server itself imposes any limitation on use of the third decimal place. AFAIK, this is just a client-side issue with sims and pilot clients. (I don't know of any ATC clients that enforce the invalid .x20 and .x70 frequencies, either.)

2 hours ago, Martin Loxbo said:

Which means as of now there is no need for the workaround?

AFAIK, that's correct.

[Damian McCracken]

Hi All, 

Appreciate the engaging debate of the issue I reported. Apologies if missed it, but what is the final course of action here to bring everything in line?  

Regards,