# What is Optimal Control?

## Optimal Advertising

Imagine that you’re the head of PR at a fledging startup, and you are assigned to do some advertising in order to draw in more customers. How would you go about it in order to maximize the company’s revenue stream?

Believe it or not, this is a problem that is actually studied by math, in the field that is known as optimal control theory.

**Optimal control theory** aims to find the best way to do something according to some performance measure and under some constraints. In the case of our advertising example, we can specify three elements:

- the
**advertising goodwill**, which summarizes the effects of advertising expenditures on product demand, - the
**advertising rate**, which determines how aggressive (in terms of cash) you advertise, and - the
**revenue stream**, which you obviously want to maximize.

The higher the amount that you allocate for advertising, the more people will be exposed to your product, leading to higher product demand. However, you can’t escape the law of diminishing returns, and so incremental product demand will be lower at higher advertising rates. This property can be modeled as a differential equation for the advertising goodwill.

Optimal advertising has a lot of literature behind it. Check out a review of the field by Feichtinger.

## Ingredients of an Optimal Control Problem

An optimal control problem, peeling away all the layers, basically involves three elements – the same three in our advertising problem. You have a **state process**, which represents a quantity which you are able to observe and control; a **control process**, which you can tweak to steer the state as you wish; and an **objective**, which is to maximize or minimize a certain quantity called the **objective functional. **We want to find the control process which satisfies our objective in the best way possible, that is, gives us the largest or smallest value of the objective functional, depending on the problem.

In our advertising problem, advertising goodwill is the state process. Increasing or decreasing the advertising rate has a corresponding effect on the advertising goodwill. Since advertising rate is the item that we can manipulate, we take it as our control process. Lastly, the revenue stream, which takes into consideration both the advertising goodwill and rate, is our objective functional.

Optimal control has far-reaching applications in almost every field, probably because every field involves something that can be optimized. Two fields that have found great applications of optimal control are biology and finance.

## Human Body’s Response to Exercise

One of the great successes of optimal control is its use in simulating the human body. We can actually use optimal control theory to determine how the human body reacts to changes in its equilibrium state and how it is able to make the **transition from one equilibrium state to another**. Specifically, optimal control can be used to see how the body, specifically the heart rate and breathing rate, adjust when one goes from rest to exercise.

In this scenario, the state process now consists of a number of quantities, accounting for the fact that the human body is a complex system. Usually, we look at cardiovascular and respiratory variables when looking at response to exercise since these are most relevant. Some quantities included in the state are the blood pressures and gas concentrations in the blood vessels and gas pressures in the lungs. A set of differential equations, called Grodins’ system, can be used to represent the dynamics of the cardiovascular and respiratory systems.

The controls that the body uses in the transition are the heart rate and the alveolar ventilation, a respiratory quantity. In this case, we aim to find the controls which minimize the deviation of arterial pressure to the equilibrium exercise value and the carbon-dioxide level to 40 mmHg, which is found to be more or less constant regardless of the level of exercise.

Not only that, we can use optimal control one more time to estimate certain unmeasurable parameters of the body. We do this by changing our objective to finding the parameters which minimize the difference of the model output with patient data. This application is known as **parameter estimation**.

## The Portfolio Problem

Suppose that aside from being head of PR, you’re also moonlighting as a hedge-fund manager. Can you really use optimal control to maximize your earnings?

Well, theoretically, you can. As an application of optimal control to finance, we have what is known as **Merton’s portfolio problem**. Here, we try to manage a portfolio, which we consider to be our state. Our portfolio consists of two assets: a risky asset such as a stock, and a risk-free asset like a bank or money market account. Our goal is to find how much we should allocate in each asset so as to maximize our *satisfaction *or our **utility**.

There are lots of variations of the portfolio problem. The most common formulation assumes that the investor derives satisfaction from two things: consumption and terminal wealth.

Merton’s model assumes that the investor consumes money in order to be satisfied. You can think of this as spending money on food or buying clothes. Our assumption is, the more the investor spends, the happier he is. Not only that, the investor also derives satisfaction from terminal wealth, which is basically the money left at the end of the investment period.

We can see that the investor must find the best trade-off between consuming money and saving money since those two act against each other and both contribute to his satisfaction. In that case, we have two controls here: the consumption rate and the money invested in the stock. You may ask why the money invested in the risk-free asset is not included. To get that, you just subtract the money invested in the stock from the total value of the investor’s portfolio.