The future of flight training

A team of students got to work this summer developing augmented-reality (AR) holograms that will eventually play a part in training the next generation of aviators at Embry-Riddle Aeronautical University.

Simulation Science, Games and Animation (SSGA) students at the university’s campus in Prescott, Arizona, created virtual CRJ-700 models to be used in classrooms, using HoloLens 2 goggles.

“The holograms they have made will be a wonderful tool to start using AR for teaching, practicing of skills and, ultimately, testing,” said Dr. Michelle Hight, assistant professor of Aeronautical Science and Fixed Wing program chair.

The holograms work like this: A student wearing the headset can step through “slides” inside interactive virtual training modules, giving them access to view schematics and information about the CRJ-700 aircraft in 3D.

Another type of hologram model in development could also let them look around the flight deck, manipulating the buttons, switches and controls as they would in real life — the HoloLens 2 can even “see” their hands, university officials noted.

Some of the other holograms under development depict the landing gear operation and the external fuel panel.

“We encouraged the team to consider the reasons for using augmented reality rather than virtual reality on the project,” associate professor and SSGA Program Chair Derek Fisher explained. “They determined that while virtual reality might provide a more immersive alternate experience, AR provides opportunities to focus on interacting with the computer-generated jet while collaborating in the familiar environment of the classroom, with real-life colleagues.”

The holograms have informational labels and can be rotated and resized by the user, giving students the opportunity to do a unique sort of virtual hands-on study, becoming familiar with where controls and important parts of the aircraft are located. When it’s time for quizzing, the parts’ labels can be removed, and students will be asked to identify either location of parts or schematics to test their knowledge, university officials explained.

“This is the first step down a very good road for us as a university, potentially toward an entire class in AR,” Hight added. “Even if students had to attend class from home, this brings the jet to life. Every student who has a HoloLens 2 could join in 3D, standing or sitting in a virtual classroom, interacting with hologram learning tools rather than a flat screen. This is a timely pedagogical development for a world evolving and responding to a global pandemic.”

Virtual Tools, Real Training

SSGA students learn to design and develop interactive and immersive experiences on a daily basis, according to Fisher, which made them the obvious choice to be added to the hologram-development team.

“The goal on this particular project was to leverage the emerging technology of the HoloLens 2 and connect the technology and information back to the human experience,” Fisher said.

Over the summer, a team of five students worked on the AR prototype over a five-week period. Next, the team will improve the software so that multiple users can interact with the same jet simulation simultaneously. After testing the new prototype, the group will seek funding to expand the program to a full-class experience.

Some advanced forms of the software have even already been used in flight deck procedures training, allowing students to participate in virtual tutorials.

“We are on the leading edge of this technology at Embry-Riddle — both for the pilot students, who are the end users, and the Games & Sims students, the developers,” Hight added. “This is really the future of pilot training. Imagine what this could do for distance learning.”