As a data-driven approach to optimal control, reinforcement learning (RL) has tremendous potential to optimise a wide variety of real-world systems that were previously unamenable to mathematical optimisation due to the lack of explicit models of dynamics. Among the key challenges of real-world RL, I am interested in sample efficient learning and offline learning.
For applied work, I am interested in precision agriculture, a form of agriculture that exploits advanced farming technologies for increased productivity. Modern sensor devices and actuators provide high spatiotemporal granularity of management units. To fully exploit the technologies and achieve right management at the right place at the right time, it is necessary to discover good policies that process high-dimensional sensor feedback and prescribe right management for each parcel of a field anytime farmers make decisions. I tackle this challenging spatiotemporal control problem using RL and Bayesian optimisation.
An optimization technique with two appealing features: sample efficiency and flexibility for complex objective functions. It has a lot of potential for applications to agricultural production, which usually takes time due to the seasonal production cycle. In other words, evaluating an agricultural production function or input response function is expensive, and therefore Bayesian optimizaton is a cost-efficient technique to identify good input combinations for profitability.
Agent-based modeling has become one of the standard approaches to studying complex systems and emergent behavior. In agent-based models, an observed macroscopic phenomenon emerges as a result of interaction among heterogeneous agents in a dynamically evolving environment. Agents typically follow simple decision rules and influence each other either directly or indirectly through the environment, which itself evolves according to its own rules and agent actions. Because the processes being explicitly modeled are complex, researchers use computer simulations to examine outcomes over a wide range of parameter values. In short, agent-based models are laboratory experiments conducted in silico. The idea is nicely captured by the following video (though we usually don’t have physical robot agents).
- The case for fully Bayesian optimisation in small-sample trials
- Deep reinforcement learning for irrigation scheduling using high-dimensional sensor feedback
- with Allan Peake and Karine Chenu
- Paper, Code
- An agent-based model of insect resistance management and mitigation for Bt maize: A social science perspective
- with Paul Mitchell and Terrance Hurley
- Published at Pest Management Science
- Machine learning for optimizing complex site-specific management
- with Vivak Patel and Paul Mitchell
- Published at Computers and Electronics in Agriculture
- Adaptive experimental design using Bayesian optimization to improve the cost efficiency of field trials
- with Vivak Patel, Shawn Conley, and Paul Mitchell
- Presented at 2019 ASA-CSSA-SSSA International Annual Meeting
- A bandit algorithm for efficient on-farm research
- with Paul Mitchell
- Presented at 2018 AAEA Annual Meeting