VATUSA TESTING

[Graeme Florance 906409]

My fellow controllers have you ever missed a question on a VATUSA test? Well i did recently in fact I missed six, and as you will probably guess that meant I failed the exam. I felt that i had answered four of those questions correctly, and so deceided to challenge. My appeal in some respects was upheld in that two of the four questions have now been reworded for furture tests. That in itself is a success. Unfortunately, for reasons i cant understand my test score stands. This will probably be my last post on the matter as I am sure my controlling days will be over once this post goes live. The crusade was never about my test score but the many others that have been wronged along the way and didnt stand up for themselve or did and were treated with the same level of contempt. Below are three communications between myself and VATUSA1 Gary Milsaps who had the final say in the matter despite the advice of some of his senior staff. I will leave you to make your own determinations on the matter. For those i have worked with and flown with along the way you have my deepest respect

 

Letter 1 From Gary

Holiday cheer to all,

 

No loop exists to be out of. Jason did indeed email me with his information. I agree with his [Mod - Happy Thoughts]essment that the answers to both questions were poorly conceived and offered less than the best of answers. As Alex has affirmed, they are now changed.

 

Now as for Graeme’s appeal…my decision stands. Though it seems hard-nosed and mean-spirited, it is not. As Jason pointed out and as supported by the examination record, Graeme’s examination had 6 wrong answers…clearly if he had answered the four non-challenged questions correctly, he would have easily p[Mod - Happy Thoughts]ed.

 

Sometimes it is all too easy to lose sight of the overall value of purpose of these exams. While a simple score is the determinant factor, it is getting the student to look at the reference information – not repeat it through some rote memorization; else the exams would not be open-book. When someone fails an open-book exam, it speaks of not taking one’s time to review the information, thinking through the question and applying what one reads to selecting the best answer. Further, if this is done, it is not inconceivable that an astute student would challenge the question before submitting a completed exam thus precluding the possibility of a failing submittal. These types of actions are even more expectant from a student working on their C1 exam versus an OBS or S1 - one elevates further through their training, the questions and information being reviewed should be elevated as well.

 

Lastly, except in one case of a complete failure of the VATUSA examination system; to date, I have not reversed any failure nor reduced the waiting period appreciably. If I once do so outside of this strict observance, where do I then draw the line in light of a continued sense of fair and equitable application of the established policies?

 

Hope this clarifies this a bit…

 

Gary Millsaps

Director, VATSIM - United States Division

 

My Response

 

Gary,

 

What is the true definition of change? In my humble opinion, “changeâ€

[Andrew Pongress 947901]

Maybe someone can explain this to me, but if two questions were changed for future tests that leads me to believe that Graeme answered them correctly hence why they were changed.

That means that he would have only missed 4 questions, which means his score would have been 84%.

 

Isn't that a p[Mod - Happy Thoughts]ing grade?

Why isn't an exception being made here?

 

I'm afraid to take my S3 test now.

Edited December 23, 2008 at 05:14 AM by Guest

[Matt Hartley 949013]

any further action on my or anyone else’s part in changing your examination grade would only be a response that benefits you personally

 

And the harm here would be what, exactly? Apparently, two questions were worded badly enough that they will be changed on future exams, but a person who was harmed by said questions and played some role in having them changed has to take it with a smile?

 

That's messed up.

 

Matt

[Micheal Kosiba 1086633]

Sometimes it is all too easy to lose sight of the overall value of purpose of these exams. While a simple score is the determinant factor, it is getting the student to look at the reference information

 

 

ok the right anwsers have to be in the text for you to change them so y can't you just give MR. Florance a p[Mod - Happy Thoughts]ing grade i mean he did anwser the questions right so you should let him retake the test right now! i think he is in a good spot and next time i take a test i am looking up all questions to make sure they are correct in real life and the stuff you have here!!!!!!!!!!!!!

Edited December 23, 2008 at 05:34 AM by Guest

[Dan Leavitt 1081600]

Maybe someone can explain this to me, but if two questions were changed for future tests that leads me to believe that Graeme answered them correctly hence why they were changed.

That means that he would have only missed 4 questions, which means his score would have been 84%.

 

Isn't that a p[Mod - Happy Thoughts]ing grade?

Why isn't an exception being made here?

 

If the questions that were wrong, were bad enough to have caused change on all future tests related to that question set, then why is the person who brought about this change the one being held back. He is the one who brought about the change, so he then should be given the chance to show his knowledge of the subject, by answering the new questions.

 

If this ruling is allowed to stand, then what is VATSIM about??? I thought the main points of VATSIM were to have fun enjoying our hobby of flying/controlling and learning while doing that. Who says the learning is restricted to the students. Management is also subject to learning. If there is anything to be learned here, it is that no matter who you are and what rank you possess, mistakes are made, and something needs to be learned from the mistakes. I understand that management has already accepted that they made a mistake, and changed the tests in the future. But to punish the person who brought about these changes, by not allowing him to retake the questions that were worded wrong, there is something wrong there.

 

I understand the point that Gary makes about Graeme missing 4 other questions on the open book test, but if the 2 poorly worded questions were changed for Graeme's test, there is the possibility that Graeme gets those 2 questions right, and p[Mod - Happy Thoughts]es his test, and none of situation becomes known.

 

Knowing that an issue exists during a very important phase of training and certification, should it not be the initiative of our current administration to correct these issues and confront our community with the most direct and truthful responses? Rather than turning a blind eye, we should offer restitution to our constituents by creating a new test that will more readily meet the needs/demands of our current network. Something that is justly able to gauge the student's abilities and knowledge of the network and its policies. It can no longer be acceptable to [Mod - Happy Thoughts]ume that students will understand the vague/improper context of which the questions are worded, the grossly outdated format from which they are administered, and the, at times, questionable validity/quality of the questions administered. We have some of the brightest minds the internet/the industry has to offer at our expense, and I believe, if given the opportunity, our members will create some remarkable advances within our training department that can pull us away from our current disposition.

 

Also, isn't Graeme part of VATSIM'S constituents, just because he took the test before these changes were brought about doesn't mean that he isn't deserving of the chances that are now being afforded to people taking this same test in the future.

 

I believe that there needs to be change brought about to this situation, and if nothing happens, I and many others will have lost much hope in the workings of VATUSA, and its management.

 

 

Dan Leavitt (1081600)

[Ernesto Alvarez 818262]

i can understand both sides. but can anyone here really say 100% even if the 2 questions that were changed wouldve been worded correctly the poster wouldve gotten them right the first time? no "if's"

 

been there, ive failed tests, ive questioned why they were wrong, only to look back and understand them why they were wrong even tho i was 100% sure i answered correctly. next time around ill get it right (sometimes )

 

i dont think anyone has said the poster cant retake the tests. he's just got to be patient and wait like the rest of us

 

sit back, take the time to get prepared for the next time. failing a test isnt the end of the world. you can certainly retake the tests.

 

C1 exam isnt easy thats for sure and frustrating, least when i took it several years ago.

[Graeme Florance 906409]

Ernesto,

 

Thanks for the response and not once have i ever said i wanted my test score amended, i do want the failure removed from my record because i dont feel its valid. I also feel i should be afforded the oppertunity to retake the COMPLETE test at the earliest oppertunity.

[Dan Leavitt 1081600]

i can understand both sides. but can anyone here really say 100% even if the 2 questions that were changed wouldve been worded correctly the poster wouldve gotten them right the first time? no "if's"

 

The problem at heart is not a matter of IF he wouldn't have gotten them right, but the fact that they were admittedly wrong by SENIOR VATUSA members/management, and since they are openly acknowledged as wrong, Graeme should be afforded the chance to take 2 questions that are worded correctly. That is the main problem, the willingness to let a student, which could be any of us, this opportunity to redeem ourselves for a problem on the test that is openly made wrong.

 

Also, Graeme wouldn't have had to get both questions right. He got 6 wrong, and you can p[Mod - Happy Thoughts] with 5 questions incorrect. Who is to say that Graeme shouldn't be afforded the chance to take 2 questions, and get 50%(1 question) correct.

 

Dan Leavitt (1081600)

Edited December 23, 2008 at 05:59 AM by Guest

[Micheal Kosiba 1086633]

You need to see the people that make up this server and communitythe few head people that make all the decisons we the people thank work under these atc areasmake it up to so if you want really a fair decison you need to ask the rest of vatusa there stand point on the issue and go from there. And we are still not asking to just give him a p[Mod - Happy Thoughts] let him sit down and take the two questions again and if he gets 1 or both of the right let him have his p[Mod - Happy Thoughts] and have a good day.

[Tim Krajcar]

I do not know if it's the case in this instance, but just because VATUSA elected to reword a question because its common interpretation was not as intended by its author does not mean that the answers that the old wording of the question elicited were correct.

 

It's also worth noting that I think I've sent feedback on question wording, varying in scale from minor to glaring, on every ATC test I've ever taken; wording test questions is a notoriously difficult task even for full-time experts in the field, to say nothing of our legion of overworked and 100% underpaid volunteers

[Robert Reynoso 997462]

I have had this happen to me as well. Many of the questions are awkwardly worded and I have found myself disadvantaged as a student in the VATUSA training environment. As an addition, I do not believe that addressing the issues that arise within your community are a waste of time.

[Graeme Florance 906409]

to say nothing of our legion of overworked and 100% underpaid volunteers

 

Of which i am one and i desreve the same respect and understanding that they do.

[Tim Krajcar]

Of which i am one and i desreve the same respect and understanding that they do.

 

You have a short waiting period before you can retake your C1 written. There is nothing to stop you from working traffic at almost any level anywhere in the world, spending time studying the C-1 materials or practicing working CTR-level traffic in Sweatbox, or even performing CTR duties when plugged in with an instructor.

 

The analogy does not apply.

[Alex Bailey 969331]

I was going to not even acknowledge this thread, but I'd like to mention a couple things now that Tim and some others brought it to my mind.

 

First issue - if a question is wrong or you have an issue with it, you should report it as being wrong and not submit the exam.

 

Second - the questions you missed were NOT incorrect questions, they were valid. One question regarding wake turbulence was 100% valid, however, my kindness got the best of me and I decided to build an additional caveat into the question to make it even more straight forward (read: copy and paste the question into Google and you have your answer). The other question you are disputing had "All of the above" rather than "Any of the above" as the answer choice. Now come on... Any time you take an exam you know that all of the above means that each individual choice above is correct. Again, I made a revision to make it much more clear.

 

You failed the exam, and given your past history on previous exams, Gary has made his decision which will remain final. Any further concerns you may have can be emailed to me directly as it would be to your benefit to speak with me rather than trying to drag VATUSA through the mud in the forums.

 

See you at the next train wreck, folks.

Edited December 23, 2008 at 06:29 AM by Guest

[Andrew Pongress 947901]

Of which i am one and i desreve the same respect and understanding that they do.

 

You have a short waiting period before you can retake your C1 written. There is nothing to stop you from working traffic at almost any level anywhere in the world, spending time studying the C-1 materials or practicing working CTR-level traffic in Sweatbox, or even performing CTR duties when plugged in with an instructor.

 

The analogy does not apply.

 

Tim, everything you said has absolutely no relevance to the issue at hand.

The issue is that Graeme was not given any opportunity to re-take questions that were approved for change by a VATUSA Director based on Graeme's original test.

 

I see nothing in any of Graeme's posts or posted emails that indicates the purpose of this is to p[Mod - Happy Thoughts] his test. From what I can see, his purpose is to raise awareness that fairness isn't being displayed at the highest levels of the VATSIM community.

Get your points and facts straight, sir.

Edited December 23, 2008 at 06:30 AM by Guest

[Dan Leavitt 1081600]

Of which i am one and i desreve the same respect and understanding that they do.

 

You have a short waiting period before you can retake your C1 written. There is nothing to stop you from working traffic at almost any level anywhere in the world, spending time studying the C-1 materials or practicing working CTR-level traffic in Sweatbox, or even performing CTR duties when plugged in with an instructor.

 

The analogy does not apply.

 

Tim,

 

You have strayed from this topic and taken it to the extremes. The point is not that Graeme wants to be given a free p[Mod - Happy Thoughts], or that he thinks that he is above any training/studying for his upcoming exam. The problem at hand is that a student, who has used his proper outlets, to have changes brought about, and because of VATUAS'S unwillingness to think about the broad spectrum, this has become a great disservice to all students of VATSIM, because, if they are not willing to look at what has hapened to Graeme, what is to make any student think that this won't happen to them, and make them push to advance further up in their training in VATSIM.

 

What is so hard about having VATUSA management open their eyes to this outcry, and listen to to their staff(which is mostly comprised of us, the students that are getting screwed by this decision) and give Graeme, and any other student that has failed due to these poorly worded questions, the chance to redeem themselves and retake different questions amounting to the same amount of questions that have now been discarded due to validity.

 

Dan Leavitt (1081600)

[Graeme Florance 906409]

Alex Well if thats not the sound of ranks closing i dont know what is

Edited December 23, 2008 at 06:36 AM by Guest

[Graeme Florance 906409]

Alex,

 

Should i post your email that recomends my score be adjusted. You were my first point of contact in this and you didnt even respond to my email with as much as a read reciept.

 

Here is the hipocracy in its most obvious example

 

From: Alex Bailey

To: J. Jason Vodnansky

Sent: Monday, December 15, 2008 4:33 PM

Subject: Re: Graeme Florance, CID 906409, Controller Exam questions

 

As far the Small behind Heavy, I will remove large from the question. I know the intent of the question, and anyone taking the exam should look at the 7110.65 and understand the difference between Section E and F. F is for landing aircraft, which is what the question asked.

 

The next one should say Any of the above, not all. I'll fix that one as well, and see what I can do about giving Graeme a p[Mod - Happy Thoughts]ing score. Is he ready for his C-1 promotion?

 

Alex

Edited December 23, 2008 at 06:41 AM by Guest

[Alex Bailey 969331]

Graeme,

 

Jason responded to you for me, and pointed out that the chain of command exists for a reason. I saw no need to respond, since he was handling the situation. My initial impression was to ask Gary to amend your score, but it highlights a quality in myself that tends to get the best of me. I'll be a horrible parent here in a few years, because I'm sure I will hand my kids candy to keep them from whining.

 

My email is in my signature, and I'd be happy to continue this discussion through that avenue. Taking your private matters to an online forum to gain support of the m[Mod - Happy Thoughts]es is not the proper way to handle the situation. If you disagree, I'm sure Norman or RJ would love to hear from you.

 

Happy Holidays

Edited December 23, 2008 at 06:41 AM by Guest

[Dan Leavitt 1081600]

The other question you are disputing had "All of the above" rather than "Any of the above" as the answer choice. Now come on... Any time you take an exam you know that all of the above means that each individual choice above is correct. Again, I made a revision to make it much more clear

 

Alex,

 

You acknowledge that the question had to be reworded, which means that there was in fact an issue with the question. With that being said, how can you say that all of the above, and any of the above are the same thing. All of the above means that all of the answers are correct at the same time, any of the above means that all of the choices are correct but not simultaneously.

 

According to Webster's Dictionary:

 

All refers to: 1. the whole of (used in referring to quantity, extent, or duration)

2. the whole number of (used in referring to individuals or particulars, taken collectively): ALL students.

 

Any refers to: 1. One: a, an, or some; one or more without specification or identification. EX: If you have ANY witnesses, produce them. Pick out ANY six you like.

 

 

I don't think anything more needs to be said about the differences of ANY or ALL.

 

Dan Leavitt (1081600)

[Tim Krajcar]

There are two posts previous that are both quoting the same statement of mine as if it was my sole contribution to the thread. Please read up and read my first posting, which prompted Graeme's response, which then in turn prompted the reply I made which was directed solely at him in his circomestance, not at the larger issue.

 

I would personally have a much greater sympathy for the "plight of the students", as it seems to be getting rung up as, if any of the laundry Graeme chose to publicly air was the slightest bit dirty, instead of direct but professional responses.

 

Gary chose not to make an unprecedented decision and I support his right to do so. I encourage those who disagree to apply for a VATUSA staff position next time it becomes open and then you will perhaps understand what it is like to spend hours per week making decisions for people who, when they disagree, choose to drag you into the public court of opinion and second-guess you to death. Even better, maybe YOU will get to be VATUSA1 and make all the decisions and see how you like it.

 

A final word before I hit the hay for the night on the subject of "fairness". I love when people try to complain that VATSIM is unfair because I don't think I've ever encountered a more egalitarian and fair organization in my life. Aside from a few cases that I am sure will be immediately be brought up in rebuttal, VATSIM is on the whole a very fair and open society. Graeme's self-indulgent statement in his original post stating that "I am sure my controlling days will be over once this post goes live." couldn't be farther from the truth; all he is damaging here is his own reputation, not his right to control, unless he chooses to push the issue and breaks the CoC/CoR.

 

Why is it 'fair' that students who take a test and fail it should be given a p[Mod - Happy Thoughts] if the questions are changed after they take the test?

Isn't that UNfair to all of the students who encountered this situation previously and were not given a p[Mod - Happy Thoughts]?

 

I thought about avoiding this topic as it didn't take a forum genius to figure out the drama llama would show up in it sooner or later, and it seems I should've done so, according to several of the posters here. Fear not; I've unsubscribed the topic and have no intention of returning.

[Andrew Pongress 947901]

Graeme,

 

Jason responded to you for me, and pointed out that the chain of command exists for a reason. I saw no need to respond, since he was handling the situation. My initial impression was to ask Gary to amend your score, but it highlights a quality in myself that tends to get the best of me. I'll be a horrible parent here in a few years, because I'm sure I will hand my kids candy to keep them from whining.

 

My email is in my signature, and I'd be happy to continue this discussion through that avenue. Taking your private matters to an online forum to gain support of the m[Mod - Happy Thoughts]es is not the proper way to handle the situation. If you disagree, I'm sure Norman or RJ would love to hear from you.

 

Happy Holidays

 

Graeme did try to properly handle the situation by going through private channels with legitimate concerns, and he was spat in the face.

Now you're comparing him to children whining for candy?

 

I've been with VATSIM for 3 years and I've never seen such a blatant disregard for decency or respect from a VATSIM Director towards a member with such scorn.

I can't believe you typed what you did.

[Micheal Kosiba 1086633]

if it comes to comparing him to kids and giving them candy to stop from whining.Graeme should get his candy but not for whining but for doing the right thing and his candy should be get to redo the few questions he missed. and if he chosses the right way to eat the candy he should get anther peaice wich should be his promotion!

[Wade Williams 877539]

NOTAM: This thread is 3 years old.

 

I suspect it's been resolved by now.

[Tod Allison 996532]

Gentlemen.

 

I worked in the Nuclear Industry for 20 years and was involved in writing test questions at one point, as well as evaluating test questions and answers through test analysis. The nuclear industry is very strict about how test questions are written, and if they are poorly worded, or incorrect, then they are thrown out. If a student takes the test with those poor questions, and gets them wrong, prior to the test analysis, then he has the opportunity to challenge the questions. If his challenge is successful (and from what I've read, the 2 questions Graeme had concerns about was successfully challenged due to poor wording or whatever) then the student's final grade is given without considering the poor questions. For instance, if it was a 100-question exam, and 1 question was deemed to have been a poor question or even wrong, then that question gets thrown out and the student is graded on a 99-question exam. Maybe this might be taken into consideration either for Graeme right now or for future students when exam questions are founded to be poor or bad questions.