AI Assistance for Spatial Disorientation

Background

Spatial disorientation — the inability to correctly perceive one’s position, motion, or attitude relative to the Earth — is a leading cause of fatal aircraft accidents. When pilots lose their natural sense of balance, they often make corrections that worsen the situation, leading to unrecoverable losses of control.

This project investigates how AI assistants can help humans maintain balance and recover from disorientation in real time, using a virtual inverted pendulum task that simulates the postural control challenges pilots face under spatial disorientation conditions.

Approach

We use the Multi-Axis Rotation System (MARS) to induce spatial disorientation in human subjects while they perform a balance task. A range of AI assistants — including reinforcement learning agents (SAC, DDPG, AIRL) and deep learning models (LSTM, MLP, GRU) — are trained to provide real-time corrective cues.

A key innovation is our Human-in-the-Loop (HITL) dyadic training paradigm: humans and AI mutually correct each other’s actions during interaction. The AI is fine-tuned on moments of human disagreement, making it progressively more aligned with human behavior and preferences. This bidirectional adaptation results in both parties improving over repeated interactions.

To scale evaluation before human studies, we developed digital twin models representing pilots of different proficiency levels (Good, Medium, Bad), enabling high-throughput screening of 21 candidate assistants before human testing.

Key Findings

• RL assistants objectively reduced crashes and oscillations, but were rated as less trusted and less preferred than deep learning assistants
• Fine-tuned (HITL-trained) models improved human performance more than non-fine-tuned versions (55% vs. 50% perceived improvement), and 70% of participants preferred them despite slightly lower adherence rates
• AI models trained on human data showed behavioral convergence with human subjects, visible in phase portrait analysis of velocity vs. angular position
• The All-Proficiency model — trained on diverse human behaviors — showed the most substantial improvement after HITL training

Publications

This work has resulted in three publications:

• Bidirectional Human-AI Learning in Real-Time Disoriented Balancing — AAAI Demo Program 2025
• Combating Spatial Disorientation in a Dynamic Self-Stabilization Task Using AI Assistants — HAI '24 ⭐ Best Paper Nomination
• Embodying Human-Like Modes of Balance Control Through Human-in-the-Loop Dyadic Learning — AAAI Spring Symposium 2024

Code

The full system implementation is available on GitHub.