MIT Researchers Introduce ADEV, an Automated Tool for 'Probabilistic Models' in AI

A new tool brings the benefits of AI programming to a much broader class of problems. Deep learning, the driving force behind significant advancements in artificial intelligence (AI) over the past decade, owes much of its success to programming languages that automate complex mathematical calculations. However, when faced with scenarios involving fundamental uncertainty, such as climate modeling and financial planning, conventional deep learning approaches fall short.

Recognizing this limitation, a team of researchers from MIT has developed ADEV, a groundbreaking tool that extends automatic differentiation to handle models that incorporate random choices and uncertainty. Automatic differentiation, the technique employed by deep learning platforms to adjust parameters and optimize models, revolutionized the field by automating mathematical operations. Yet, deterministic functions alone cannot adequately address problems reliant on probability theory, where randomness and uncertainty play a significant role. Implementing deep learning platforms in such cases can lead to incorrect results. ADEV, created by MIT researchers, offers a solution by enabling automatic differentiation in models that make random choices, allowing for a broader range of applications and facilitating rapid experimentation with models that reason in uncertain scenarios.

Lead author Alex Lew, a PhD student in electrical engineering and computer science at MIT, expressed his hope that ADEV would diminish apprehension surrounding probabilistic models. The perception that probabilistic models are more challenging to work with arises from the need to derive low-variance, unbiased gradient estimators manually. Lew believes that ADEV's framework for building these estimators automatically will make probabilistic models more enticing, leading to novel discoveries and advancements in AI and beyond. Sasa Misailovic, an associate professor at the University of Illinois at Urbana-Champaign, lauded ADEV for establishing a solid mathematical foundation for efficient software implementations in the emerging field of probabilistic programming. By providing automated mathematical tools and scalable inference algorithms, ADEV extends the benefits of probabilistic programming to a wider range of problems where decision-making and action-taking are essential, not just probabilistic inference. In addition to climate modeling and financial planning, ADEV finds applications in operations research, such as simulating customer queues in call centers to minimize wait times and optimizing algorithms for physical object manipulation by robots. Co-author Mathieu Huot expressed enthusiasm about ADEV's potential as a design space for novel low-variance estimators, addressing a crucial challenge in probabilistic computations.

The research conducted by MIT, in collaboration with Oxford University, was honored with the prestigious SIGPLAN Distinguished Paper award at POPL 2023. Co-authored by Vikash Mansinghka, leader of MIT's Probabilistic Computing Project, and Mathieu Huot and Sam Staton from Oxford University, the study was supported by the National Science Foundation, the DARPA Machine Common Sense program, and the Siegel Family Foundation. Mansinghka emphasized the significance of ADEV in tackling decision-making under uncertainty, as many crucial choices, ranging from climate policy to tax codes, hinge on this aspect. ADEV simplifies the exploration of new problem-solving methods by automating the most challenging mathematical processes.

Whether applied to probabilistic programming scenarios or other problems amenable to probabilistic models, ADEV empowers researchers to tune parameters automatically, steering outcomes towards desired results while avoiding undesired consequences. The advent of ADEV marks a major milestone in the field of AI, as it equips researchers and practitioners with a unified framework to reason gradient estimation without bias.

Its clean, elegant and compositional approach promises to revolutionize probabilistic computations, allowing for more efficient and reliable AI systems that can make informed decisions in the face of uncertainty.