Improving Human Performance,
Cost (Operational) Efficiency,
Safety and Resilience

Virtual/Augmented Reality applications for tower

Solution part PJ.05-W2-97.1 aims to support the air traffic controllers by means of applications for the virtual or augmented reality in ATC tower environment. The technology involves the use of tracking labels, air gestures and attention control. It takes into account the work already performed during SESAR Wave 1 continuing to provide significant improvements thanks to advanced interaction methods with HMI.

Virtual and augmented reality applications (e.g. Head Mounted See Through Devices) in tower environments, including tracking labels, allow a reduction of the ATCOs head-down time and an increase of situational awareness. As a result, operations are eased and the safety can be improved under any meteorological conditions, consequently increasing airport resilience.

The use of in-air gestures for user interaction can speed up and make simpler human-system interaction.

The attention guidance for virtual and augmented reality applications enables to monitor and guide the attention of the controller, by measuring and comparing controller reaction times, attention distribution (through an eye-tracker) and decision-making effectiveness for the situation to be solved.

The solution applies to very large, medium and small airports.

Intended Benefits

  • Improve Human Performance and Safety through increased situational awareness resulting from improved and innovative working environment and HMI.
  • The improved usability and reduction of mental workload, will also improve Operational Efficiency and increase Resilience to LVC.


Automatic Speech Recognition at the TWR CWP
supported by AI and Machine Learning

In solution part PJ.05-W2-97.2 automatic speech recognition (ASR) enables the recognition and translation of spoken language (e.g. ATCO commands) into the system, thus reducing human error and workload, while improving HMI usability and efficiency in performing interactions.

Artificial Intelligence (AI) and Machine Learning algorithms support the speech recognition engine by deriving command hypotheses from the contextual knowledge.

The solution applies to medium and small airports.

Intended Benefits

  • Improve Human Performance, Operational Efficiency and Safety through improved and innovative working environment and HMI, easing the usability and reducing mental workload, while increasing ATCO productivity.


Interacting with tower CWP
by means of touch screen (multi touch input)

Solution part PJ.05-W2-97.3 aims to support the Air Traffic Controllers by means of innovative multi-touch interactions for tower controller working positions (CWP).

Multi-touch input devices consist of a surface (e.g., a touchscreen) that allows to recognize the occurrence of a (hand) contact with the surface itself for selection or input improving performance of interactions. The Multi-Touch Interaction (MTI) system is expected to support human system interaction and improve ATCO operational efficiency through intuitive input device and gesture set design, thus overcoming mental bottlenecks.

Thanks to the information and functionalities provided by supporting technical system, safety can be increased, while human performance can benefit of a workload and human error reduction.

The solution applies to medium airport.

Intended Benefits

  • Improve human performance and safety through improved and innovative working environment and HMI, easing the usability and reducing mental workload, while increasing ATCO productivity.

Information material