AI Assistance for Spatial Disorientation

Background

Spatial disorientation — the inability to correctly perceive one’s position, motion, or attitude relative to the Earth — is a leading cause of fatal aircraft accidents. When pilots lose their natural sense of balance, they often make corrections that worsen the situation, leading to unrecoverable crashes. This project investigates how real-time AI assistance can help humans recover from spatial disorientation in a virtual inverted pendulum (VIP) task: a controlled, 1-DOF simulation of the balance challenges pilots face under disorienting conditions.

Approach

Participants performed the VIP task using a joystick while experiencing simulated spatial disorientation. AI assistants — including reinforcement learning agents (SAC, DDPG, AIRL) and supervised learning models (MLP, RNN, LSTM, GRU) — were trained to provide real-time corrective cues rendered as directional arrows on screen.

Before involving human subjects, 21 candidate assistants were screened in a high-throughput digital twin setting, where simulated “Good”, “Medium”, and “Bad” pilot models (trained on MARS apparatus data) were paired with each assistant. The top-performing candidates were then brought into a two-session human subject study.

A key innovation is the Human-in-the-Loop (HITL) dyadic training paradigm: after the human is assisted by the AI, the human then corrects the AI by guiding it through the task. Disagreement episodes are logged and the AI is fine-tuned on human demonstrations, creating bidirectional adaptation where both the human and the AI improve over time.

Key Findings

• RL assistants (SAC, DDPG) reduced crashes more effectively than SL assistants, but generated ~3–4× more disagreement episodes — reflecting strategies that contradicted human physical intuition
• SAC-AIRL, which recovers implicit reward functions from human data, emerged as the most effective disorientation countermeasure in both digital-twin and human-subject evaluations
• Human subjects followed AI suggestions ~44% of the time (σ=14%), substantially lower than the ~80% seen in other assisted decision-making domains
• 70% of participants preferred HITL fine-tuned assistants over the original versions, despite following them 7% less — fine-tuned models were rated as more trusted and preferable
• Post-HITL, SL models showed measurable behavioral convergence with human subjects in velocity-position phase portraits, and reduced oscillation and RMS velocity more effectively than pre-HITL versions
• Experts perceived AI as more impactful on their performance, but novices were better calibrated — a reversal of the typical trust pattern seen in high-stakes domains

Publications

• Bidirectional Human-AI Learning in Real-Time Disoriented Balancing — AAAI Demo Program 2025
• Combating Spatial Disorientation in a Dynamic Self-Stabilization Task Using AI Assistants — HAI '24 ⭐ Best Paper Nomination
• Embodying Human-Like Modes of Balance Control Through Human-in-the-Loop Dyadic Learning — AAAI Spring Symposium 2024

Code

The full system implementation is available on GitHub.