Research

As a data-driven approach to optimal control, reinforcement learning (RL) has tremendous potential to optimise a wide variety of real-world systems that were previously unamenable to mathematical optimisation due to the lack of explicit models of dynamics. Among the key challenges of real-world RL, I am interested in sample efficient learning and offline learning.

For applied work, I am interested in precision agriculture, a form of agriculture that exploits advanced farming technologies for increased productivity. Modern sensor devices and actuators provide high spatiotemporal granularity of management units. To fully exploit the technologies and achieve right management at the right place at the right time, it is necessary to discover good policies that process high-dimensional sensor feedback and prescribe right management for each parcel of a field anytime farmers make decisions. I tackle this challenging spatiotemporal control problem using RL and Bayesian optimisation.


Bayesian optimisation

An optimization technique with two appealing features: sample efficiency and flexibility for complex objective functions. It has a lot of potential for applications to agricultural production, which usually takes time due to the seasonal production cycle. In other words, evaluating an agricultural production function or input response function is expensive, and therefore Bayesian optimizaton is a cost-efficient technique to identify good input combinations for profitability.


Agent-based modeling

Agent-based modeling has become one of the standard approaches to studying complex systems and emergent behavior. In agent-based models, an observed macroscopic phenomenon emerges as a result of interaction among heterogeneous agents in a dynamically evolving environment. Agents typically follow simple decision rules and influence each other either directly or indirectly through the environment, which itself evolves according to its own rules and agent actions. Because the processes being explicitly modeled are complex, researchers use computer simulations to examine outcomes over a wide range of parameter values. In short, agent-based models are laboratory experiments conducted in silico. The idea is nicely captured by the following video (though we usually don’t have physical robot agents).


Papers