WASHINGTON, D.C. — Pilots and future pilots might find new and different approaches to studying and testing for certificates and ratings if recommendations from a panel are adopted by the FAA.
Over the past several years the aviation training community has raised concerns that the FAA has not kept up with training methods and technology. The training community has faulted the FAA for what they called piecemeal changes and often unilateral efforts to revise standards, training materials and testing methodologies.
Responding to this criticism, the FAA chartered the Airman Testing Standards and Training Aviation Rulemaking Committee (ARC) in 2011. Comprised of a wide representation of the aviation community, ARC members began its work in early October, met four times, conducted numerous conference calls, convened in sub groups, and six months later, in April of 2012, presented a draft report to the FAA with nine recommendations. This gives a six-month period for follow-up questions and to consider any additional tasks the FAA might assign before the term of the ARC expires.
Here is a brief summary of the nine recommendations:
1. Establish by Sept. 30, 2012, a stakeholder body or coordinated bodies of subject matter experts and relevant FAA policy offices to:
- Assist with the development of knowledge questions;
- Provide continuous review of standards;
- Undertake the review and development of handbook content; and
- Assist with the review of the current bank of FAA knowledge test questions for validity.
2: Revise the quality management system process through which key policy offices make recommendations to the FAA Airman Testing Standards Branch about needed changes to training and testing documents.
3: Transition to a single testing standards document, the airman certification standards (an update to the PTS), to include knowledge, skills, and risk management.
4: Concerning the philosophy of question development:
- The FAA should maintain discretion to write questions that reference multiple documents so that applicants must correlate data;
- Although questions must refer to specific FAA guidance documents, the FAA should not use exact quotes from specific passages unless testing specific required rote knowledge;
- Test questions should be written to be pertinent to safe operations and necessary for sound airmanship;
- Tests should incorporate scenario-based questions that assess the applicant’s ability to manage the many risks of flying;
- Test questions should not only be relevant to the way pilots operate in the real world, utilizing current technologies both in and outside the cockpit, but also test how those technologies can be used to facilitate proper risk management skills; and
- The FAA should adopt a continuous review process to ensure test questions are relevant to the current technology used in aviation, with priority given to removing obsolete information from the tests.
5: Return the knowledge test item question bank to the public domain by Dec. 31, 2012, in a way that maintains the integrity of questions requiring calculations or interpolations in accordance with the guidance below:
- Remove numbers from questions that require calculations or interpolations;
- For scenario-based questions testing risk management skills, remove any facts and numbers that determine the appropriate course of action, such as wind direction; and
- For questions that appropriately test rote knowledge, provide a sufficient number and variety of questions to ensure broad knowledge (such as airspace requirements, regulations, and airport signage and markings).
After three to five years, the FAA should determine whether it is appropriate to make the question bank — completely or in part — nonpublic, provided the following conditions have been met:
- The advisory group identified in recommendation 1 has been operating for at least three years and will continue to operate for knowledge tests for every certificate or rating;
- The advisory group has reviewed all test questions in use; and
- Correlation between knowledge tests and practical tests indicates that the new testing system has not been effective in creating airmen who demonstrate improved knowledge and risk management skills.
6: Urgently allocate additional resources for an improved computer system (including both hardware and software) for development, maintenance, and delivery of knowledge tests that can:
- Randomly generate tests that include all required knowledge areas (instead of manually created form tests);
- Display onscreen images with regularly updated figures in place of FAA computer testing supplements;
- Improve data management; and
- Be updated and maintained as technology improves.
7: Improve the feedback mechanism by June 30, 2013, by:
- Providing the applicant and instructor the specific missed questions to identify the deficient knowledge by review of the Airman Knowledge Test Report;
- Publishing the aggregate results of knowledge testing failure areas to provide a mechanism through which training organizations, providers, and publishers can improve and better target their instruction; and
- Reviewing the benefit of integrating the results of aggregate knowledge testing into the Aviation Safety Information Analysis and Sharing (ASIAS) system.
8: Establish and continuously communicate a schedule for publishing standards, handbooks, and knowledge test questions by June 30, 2013.
To communicate new important safety information while adhering to the publication schedule, the ARC recommends the FAA establish a process through which high priority topics are identified and communicated to stakeholders by use of “hot sheets” that provide time-sensitive information critical to flight safety between scheduled publication dates.
9: Continue to administer a single knowledge test for each certificate or rating and not transition to testing and scoring individual required subject areas.
The FAA may follow the recommendations all or in part, or reject them all or in part.
Members of the ARC included representatives from the various aviation alphabet groups, universities, subject matter experts, and the FAA.
Charles Spence is GAN’s Washington, D.C., correspondent.
People who read this article also read articles on airparks, airshow, airshows, avgas, aviation fuel, aviation news, aircraft owner, avionics, buy a plane, FAA, fly-in, flying, general aviation, learn to fly, pilots, Light-Sport Aircraft, LSA, and Sport Pilot.