VATSIM.net

Downloads Available on mitrecaasd.org

The following two documents are now available for download from mitrecaasd.org

1] Comparison of real-world (RW) and VATSIM FMS. MITRE flew several MSFS simulators through an RNAV procedure under consideration in MIA and compared the results with real world (RW) flights of the same procedure. These are preliminary results; more details are in preparation for an aviation journal.

2] Quick-look at the MITRE/VATSIM KATL Event of Sept 27, 2008. Unzip this file and load it into GoogleEarth (GE) to view the traffic "as-flown" in the event and the RNAV procedures in use around vKATL.

• raw data was acquired by the MITRE data collector 14:16 - 22:56 EDT, Sept 27 2008.
• event boundaries were 20:00 - 23:00 EDT, Sept 27 2008.
• data collector limited to a 200 nmi range around KATL
• objects in the .kml file are grouped by: RNAV procedure, Registered flights (subgrouped by CLEAN, FLCON3, and Crossing flights), and Other flights (any other flights captured in the data collection, and not seen in the list of final registrants)
• RNAV-1 conformance bounds are not included. If you want to see lateral conformance, use the ruler tool in GE.
• only Registered flights will be analyzed. As such, all registered fight IDs have been "sanitized". Other flight IDs have not been sanitized (they will not be analyzed).
• this file is preliminary. No analyses are supplied or are even available right now (October 22, 2008). A thorough understanding of the traffic requires, among other things, voice transcripts, and pilot/controller debriefs. We're almost finished with the voice transcripts, but making sense of everything is not trivial. As results become available, they will either be integrated into the .kml file and re-released or made available in other ways.
• questions/comments/observations are welcome. Please post those here. We think this kind of medium helps tell a story and could help suggest important, unasked questions. Your observations are important.

Thanks. We look forward to your comments.

Is the FMS data being shared with the developers of the add-on aircraft in question? They'd surely be interested in seeing how their FMS simulations compare to the real thing.

If you can tell me how to get in touch with them, we'd be delighted to share whatever data we're allowed to share.

The data from our flight simulators probably isn't a problem, but the data from the field might be. If they're interested, I'll check.

Hmm … add-on developers are notoriously difficult to contact, and they don't even seem to have e-mail addresses, so I'm not sure how to get in touch with them.

Really? I have never had any difficulty at all in making contact with the developers of most of the addon planes used during the event.

There are many add-on developers along with their staffs, partners and beta-testers (of all the FMCs tested in the study) that are active members of VATSIM and who read these forums. I'm sure most are by now aware of the availability of this data. I also agree with Norman...the add-on developers we are talking about here are, if anything, notoriously EASY to contact.

Again, we're happy to share whatever we're allowed to share. If you're one of these developers and would like more info, please contact me via PM on this board. We can proceed from there.

1.Very interesting article....but why for us, for our community it is not full version?
2.Could You share more about the experiment as well as ideas and own practical algoritm?
3.Why did you check just navigation performance from FMC's point of view ( where is ARIANA B-737 or PIC Wilko B-737,btw)...for example, why not to check the navigation performance of PROTU TU-154B2, within typical analogue system?
4. How I can see, the performance of FMC simulated systems follow the lows of flying dymamic, and that is expalainable, ....may be just some little, but significant approximations should have done observed deviatons from the real results.

Good question. Once a document is released to the internet, it is not publishable in a peer-reviewed or refereed journal or conference proceedings. We could write up the full paper and release it to the VATSIM community/internet, but then we could not submit it for publication - we'd prefer to actually have our results published somewhere.

We recognize that not everyone has access to these journals and have used additional resources that we wouldn't ordinarily use, to prepare this extra document for the community. We hope you find it interesting.

I assume you mean the KATL CDA experiment. The analysis is incomplete right now. Right now I do not plan on publishing the results in a journal so I think we can release whatever we get, whenever we get it (like a .kml file).

As far as the details of the experiment, I'll make sure the briefing materials are made available at mitrecaasd.org/vatsim. Right now i think you have to log in, and since the logins have been disabled (the event is over), you may not be able to access them. I'll make them available, and make an announcement here. Keep an eye out.

As far as our "algorithm", can you elaborate?

In the KLAS event we observed some things that FMC performance might explain. We decided to look closer at some of the FMCs. So we picked the most capable, most popular FMC-equipped flightsims. Details of the KLAS event are forthcoming.

The report is not intended to be an exhaustive review of all FMCs.

Just for future reference, if we need a broader international fleet mix, would you be willing to fly your TU-154B2 or ARIANA B-737 for us?

Not all simulated (or real-world for that matter) FMCs have the same level of capability. They may all have "VNAV capability", but they do not all work the same way. Some have rudimentary VNAV abilities, while others are quite sophisticated. We were trying to see if they behaved differently.

Also, real-world FMCs have capabilities not currently found in the simulated FMCs.

One Tu-154M maniac here from Australia - not sure the timezone will suit though...

Also into INS powered Tinmouse737-200

Hmmmm.... let's try something, if you're willing... Can you marshall some Tu-154 maniacs to fly the CLEAN at on off-peak time for ZTL? Do you know other Tu-154 pilots? Would you be able to get a few together at one time? One is fine, 10 is great, 100 is fantastic. Perhaps Sergey would like to join you?

Remember, the CLEAN arrival is really an exercise for the FMCs. If your flightsim does not have an FMC that can handle multiple simultaneous restrictions, then you'll have to hand-fly your plane and the results will reflect your.... adequacies.... not those of the FMC.

Since your timezone is so far off, this might work. Lets limit it now to the TU-154 maniacs only....

• all you TU-154 maniacs work together to pick one time period to fly the CLEAN arrival when ZTL is unstaffed. The time can be several hours long if you wish.
• tell me 24 hrs in advance when you plan to file/fly (this part is of some importance... I need to start the data collector). I'll also need to know your flight id, VATSIM userid, and FMC capability.
• fly the CLEAN arrival, making sure you begin your descent at or after BEBAD
• let me know via PM or this topic, if you think the data is good (meaning you weren't vectored or given clearances along the CLEAN arrival that would affect conformance with the restrictions)

I'll take a quick look at the data and let you know what I find.

Let me know what you decide. Thanks for your interest.

Hi! Wonderful to hear that someone is interested from this old russian mashine... I'll come...I'll tell you the day tomorrow...
Cheers
Vassil Dimitrov
Sofia,Bulgaria

Please coordinate with Trent and the other Tu pilots. Give everyone a few days to decide on a single time period.

All documents (including the event description you seek) are now available for download at http://mitrecaasd.org/vatsim/download

I suspect the PMDG FMC "cost IDX" is mainly involved here in case of the soft restraints. Or more precisely it isn't implemented how the real one is operating ( http://www.boeing.com/commercial/aeroma ... ticle5.pdf ).

A friend of mine (B738 type rated and flying) argues also that the PMDG is hugely underperforming related to the real one. His argue is, that PMDG used the "rated performance data" instead of statistical average operational data.