what is optimal control? -凯发k8网页登录

design and implement control techniques to satisfy system objectives

optimal control is a condition of dynamic systems that satisfy design objectives. optimal control is achieved with control laws that execute following defined optimality criteria. some widely used optimal control techniques are:

linear quadratic regulator (lqr)/linear quadratic gaussian (lqg) control

the linear quadratic regulator (lqr) is a full state feedback optimal control law, $u= -kx$, that minimizes a quadratic cost function to regulate the control system.

fig 1. schematic of linear quadratic regulator controller.

this cost function depends on system states $(x)$ and control inputs $(u)$, as shown below.

$$j(u)=\int_{0}^{\infty} (x^t qx u^t ru 2x^t nu)dt$$

based on the performance specifications, weighting factors q, r, and n are set for this optimal control law to define the appropriate balance between system state regulation and the cost of control actuation.

not all state measurements will be accessible in many optimal control problems. in these cases, states must be estimated using an observer. this is typically done using an observer such as a kalman filter. a kalman filter combined with an lqr controller constitutes a linear quadratic gaussian (lqg) controller.

fig 2. schematic of linear quadratic gaussian controller.

to learn more, check out this matlab tech talk (17:23) on lqr control.

model predictive control

the model predictive control (mpc) is used to minimize a cost function in multi-input multi-output (mimo) systems that are subject to input and output constraints. this optimal control technique uses a system model to predict future plant outputs. using the predicted plant outputs, the controller solves an online optimization problem, namely a quadratic program, to determine the optimal adjustments to a manipulable variable that drives the predicted output to the reference. mpc variants include adaptive, gain-scheduled, and nonlinear mpc controllers. the type of mpc controller used depends on the prediction model (linear/nonlinear), constraints (linear/nonlinear), cost function (quadratic/nonquadratic), throughput, and sample time. to learn more, check out about mpc variants.

fig 3. schematic of model predictive control.

advances in microprocessor technology and efficient algorithms have increased the adoption of this optimal control method in applications such as automated driving, optimal terrain tracking in aerospace applications, etc.

to learn more, check out this matlab tech talk series on model predictive control.

reinforcement learning

reinforcement learning is a machine learning technique in which a computer agent learns optimal behavior through repeated trial-and-error interactions with a dynamic environment. the agent uses observations from the environment to execute a series of actions, with the aim of maximizing the agent’s cumulative reward metric for the task. this learning occurs without human intervention and without explicit programming.

fig 4. schematic of reinforcement learning.

this optimal control method can be used for decision-making problems and as a nonlinear control alternative for applications that use conventional control methods such as automated driving, robotics, scheduling problems, and dynamic calibration of systems.

to learn more, check out this matlab tech talk series about reinforcement learning.

extremum seeking control

extremum seeking is an optimal control technique that automatically adapts control system parameters to maximize an objective function using model-free real-time optimization. this method does not require a system model and can be used for systems where parameters and disturbances slowly change over time. this optimal control technique is suitable for stable systems that can tolerate noise in control and where only a small number of control system parameters need to be adapted.

applications of extremum seeking control include adaptive cruise control, maximum power point tracking (mppt) for solar arrays, and anti-lock braking systems (abs).

fig 5. schematic of extremum seeking control.

h-infinity synthesis

h-infinity synthesis is an optimal control tool/technique for designing single-input single-output (siso) or mimo feedback controllers to achieve robust performance and stability. compared to classical control techniques such as loop shaping with bode or pid tuning, h-infinity is better suited for multivariable control systems that require cross-coupling between channels.

with h-infinity, the control objectives are formulated in terms of the normalized closed-loop gain. h-infinity synthesis automatically computes a controller that optimizes performance by minimizing this gain. this is useful because many control objectives can be expressed in terms of minimizing gains. this includes objectives such as disturbance rejection, sensitivity to noise, tracking, loop shaping, loop decoupling, and robust stability. variations of h-infinity synthesis can be used to handle both fixed structure or full-order controllers.

to learn more, check out this on h-infinity synthesis.

the table below compares the optimal control methods described above:

optimal control method	is optimization carried out at runtime? (yes/no)	how does the optimization process work for this optimal control process?	can it handle hard constraints?* (yes/no)	does it use model-based technique? (yes/no)	what is the throughput? (high/low)
lqr/lqg	no	uses closed-form solution that works with known linear time-invariant systems	no	yes	high
	implicit mpc (yes)	using a prediction model, solves an online optimization problem to compute the optimal control actions	yes	yes	low (nonlinear mpc), high (linear mpc)
	explicit mpc (no)	solution to the optimization problem for computing optimal control actions is calculated offline	yes	yes	high
reinforcement learning	yes**	learns optimal behavior for a task to maximize a reward metric	no***	depends on training algorithm	low (with training), medium-high (during inference)
extremum seeking control	yes	perturbs and adapts control parameters to maximize an objective function	no	no	high
h-infinity synthesis	no	automatically computes a controller that minimizes normalized closed-loop gain	no	yes	high