Optimal & Data-Driven Control
Introduction
Let's get some perspective and think about the world in terms of dynamical systems, whose state can be
mathematically described through systems of ordinary differential equations (ODE's). This has been a
very effective way of modeling real-world phenomena such as the fluid flow over a wing, the population dynamics in
a city or the spread of disease, the stock market climate or the movement of planets around a solar system.
Treating all these as dynamical systems with data from the real world allows us to build predictive models.
Often we want to go beyond just describing the system of interest, and we want to actively manipulate the system of interest by changing its behavior. This could be done by imposing a certain preplanned control logic aiming to manipulate it or actually using sensors to measure the system and make decisions based on how the system is responding.
What is optimal control?
Optimization constrained by dynamics. This is the 1st part of the lecture.
What is data-driven (a.k.a Machine Learning)?
A set of non-linear optimization algorithms based on data that help us:
I) learn dynamical models purely from data or in other words apply machine learning for system identification to design controllers
II) learn directly control laws purely from data
III) learn sensor & placement accuracy for effective controller design
Optimal Control
Optimal & Modern control theory includes how to write down a system description of a control system with inputs and outputs. This system description encompasses a system of linear differential equations enabling you to design controllers to manipulate the behavior of that system or design estimators like the Kalman filter to reconstruct various aspects of that system behavior if you had limited sensors.
Thus, the overarching view of control theory can be summarized in the following bullet points:
- * You have a dynamical system fo interest that we want to make more stable
- * Write down the system fo equations
- * Design some control policy that changes the system behavior to be more desirable
Types of Control
open loop: you think about your system - you pre-plan a trajectory and you just enact that control law Why feedback? 1. Uncertainty (makes the pre-planned trajectory sub-optimal - but if we measure we can adjust the control law even if I don't have a perfect model of my system) 2.
Model Predictive Control
Model Predictive Control (MPC) is a powerful optimization strategy for feedback control. The principle behind MPC is the following: if you have a model of your system of interest, you run forward in time set of forecasts of this model for different actuation strategies you optimize over the control input \(u\) over a short time period and essentially you determine your immediate next control action based on that optimization. Once you've applied that immediate next control action, then you re-initialize your optimization over, you move your window over, you re-optimized to find your next control inputs and this thing essentially keeps marching that window forward and forward in time.
MPC @ APMonitor https://apmonitor.com/pdc/index.php/Main/ModelPredictiveControl
Reinforcement Learning
…