AI Learns to Make Decisions with Less Data

In many real-world scenarios, from medical diagnostics to robotics, artificial intelligence systems often face a critical challenge: gathering all the data they need can be expensive, risky, or impractical. For instance, in healthcare, running extensive tests on patients increases costs and potential health risks, yet skipping them might lead to poor treatment decisions. This paper introduces a framework that enables AI to learn how to make optimal decisions while actively selecting which data to collect, reducing unnecessary information acquisition without compromising performance. The researchers developed a model-based approach using a variational auto-encoder to create high-quality representations of partially observed states, which guide the AI in efficiently acquiring features and maximizing rewards. This method was tested in two domains: a bouncing ball control task with high-dimensional pixel inputs and a medical simulator for sepsis treatment based on real-world data. In the bouncing ball task, the AI had to navigate a ball to a target location by selecting which quadrants of an image to observe, while in the sepsis simulator, it chose which patient measurements to take. Results showed that the proposed framework outperformed conventional baselines, achieving similar or better outcomes with significantly fewer observations. For example, in the bouncing ball task, it required only about 8 observations per episode on average to reach the target, compared to nearly 18 for standard methods, and in the sepsis simulator, it improved patient discharge rates by over 5% while reducing measurement costs. The key innovation lies in the sequential representation learning, which uses past observations and actions to infer missing data, allowing the AI to 'fill in the gaps' and make informed decisions even when information is incomplete. This approach not only enhances sample efficiency but also opens doors for applications in cost-sensitive environments like healthcare and education, where minimizing data acquisition is crucial. However, the study notes limitations, such as the assumption that all features have equal acquisition costs and the potential need for human supervision in high-risk domains to address biases. Future work may explore hierarchical representations to handle larger state spaces and adapt the method to more diverse real-world settings.