Augmentations for Robust and Efficient Imitation Learning in Streamed Video Games

The paper addresses the challenges of imitation learning in modern streamed video games, where collecting human demonstrations is costly and network conditions introduce visual artifacts. The core innovation lies in "streaming augmentations" designed to mimic common low-bandwidth streaming issues: pixelated blocks, scrubs, global blur, and ghosting. These augmentations are applied during the training of predictive inverse dynamics models (PIDM), which learn to map visual observations to actions.

Enhancing Training Efficiency and Robustness

The study demonstrates that agents trained with these spatiotemporal augmentations achieve substantially higher evaluation performance, up to 41% better, compared to agents trained without them, even with the same data budget. This indicates a significant improvement in training efficiency, as less expensive human demonstration data is needed to achieve superior results. The augmentations effectively broaden the training data distribution, making the learned policies more generalizable to varied visual inputs.

This enhanced efficiency is critical for scaling AI development in complex 3D environments, where manual data collection is a major bottleneck. By making better use of existing data and reducing the need for pristine, artifact-free demonstrations, the proposed method offers a practical solution to a long-standing problem in imitation learning.

Mitigating Network Lag Impact

A critical finding is the enhanced robustness of augmented agents against network lag. When network delays are introduced, agents trained with streaming augmentations experienced only a 7.45% degradation in performance, a stark contrast to the 49.82% drop observed in agents trained solely on original data. This resilience is vital for real-world applications where stable network conditions cannot be guaranteed, ensuring that game-playing AI can maintain performance even under adverse streaming conditions.

This significant reduction in performance degradation highlights the practical utility of the augmentations. In a world where cloud gaming and streamed entertainment are becoming increasingly common, AI agents must be able to operate reliably despite fluctuating network quality. The ability to withstand substantial lag without a catastrophic performance drop is a major step forward for deployable game AI.

Implications for Game AI Development

The research highlights a simple yet powerful method for developing more reliable and efficient game-playing AI. By systematically introducing simulated streaming artifacts during training, the agents learn to generalize better across different visual qualities and network latencies. This approach not only reduces the dependency on vast amounts of pristine human demonstration data but also paves the way for more practical and deployable AI agents in the rapidly growing domain of streamed interactive entertainment.

Ultimately, this work contributes to making imitation learning a more viable and robust technique for creating sophisticated AI behaviors in dynamic and visually complex environments. It suggests that focusing on realistic data corruption during training can lead to more resilient and adaptable AI systems, moving closer to agents that can perform effectively in real-world, imperfect conditions.