Controls Fundamentals Notes

I’m learning Continuous Time Stochastic Modeling (CTSM) for a research project. This relies on the Extended Kalman Filter (EKF) to work. I’m in the process of writing notes to solidify my intuition of control theory fundamentals. This also serves as a reference for my future self.

Autonomous Systems

A function is a mapping from an input to an output.

If we want to describe an object, we often need more than one variable. These can all be stored in $x$, making it a vector. This is known as the state. The state of a pendulum for example is:

What if we want a function that describes the state of an object over time? This would take the form $x(t)$. However, functions like these are often incredibly difficult to derive. It’s often much easier to write a function that describes what’s happening to a state at any point in time.

This is known as the dynamics of an object. An object with dynamics is known as a system. The dynamics of the pendulum, for example, is just one function:

This means that the velocity of the pendulum changes as a function of the position. To define the state of the pendulum, we only include variables we care about. Looking at the dynamics, we need the angle $\theta$. However, we also need the $\dot{\theta}$, since this controls how position evolves. You can see this if you try to integrate $\ddot{\theta}$: the constant $\dot{\theta}(0)$ appears.

Why not include angular acceleration? A state can include any number of variables like acceleration, jerk, snap, the earth’s velocity, magnetic flux, etc. However, we typically only include the variables that govern how the object evolves. In the case of a pendulum, we don’t need to know angular acceleration in order to determine what happens next.

A state space is the collection of all possible states a system might have. In the case of the pendulum, the state space is:

Linear Systems

Earlier, we said that it’s helpful to describe our system with one or more differential equations:

However, these equations are unwieldy when you have a lot of state vectors. When your system is linear (every variable evolves as a scalar times a combination of other variables), you can describe your dynamics with a matrix, A:

A is known as the state matrix. It must be an $n \times n$ matrix because it’s transforming $x$ of size $n$ into the change in $x$, also of size $n$.

Controllability

Up until now, we’ve been describing an autonomous system. If the system has an external influence, it’s known as a controlled system. The ways a system is influenced can be described in an input (or control) vector $u$. The input vector usually contains different variables than the state vector. It also changes over time, making it a function of $t$, just like $x$. Unlike the state vector, it’s described without a differential equation, since it’s not determined by the system. The input vector on a controlled pendulum might be the torque on the pivot:

$u(t) = [\tau(t)]$

Our equation now needs to account for this input vector.

If our system is linear we can write this as:

B is known as the input matrix, how the state changes as a result of the input. It must be an $n \times m$ matrix because it’s transforming $u$ of size $m$ into $x$ of size $n$.

So what does controllability mean? It turns out there’s a very precise definition: A system is controllable if you can get to any state, from any state, given a finite time. An uncontrollable system means there are some states you can’t get to no matter how long you have. For example, in a system of two pendulums with one torque driving both identically, you can never reach a state where the two pendulums have different angles or different angular velocities.

How do you prove a system is controllable? Consider the input matrix $B$. By choosing your input carefully, you can get to the direction $Bu$. Since $u$ is any input, we can consider the set of achievable directions to just be the span of $B$‘s columns. However, in most cases, this isn’t enough to get to every state in your state space. In our earlier example, you need to arrive at any target angle and any target angular velocity given a single torque input. This means you want to control two state variables using one input. This is only possible due to the way the system evolves over time.

If you consider how the dynamics act on $B$ over time, you get $AB$, the first order term of how the input’s reach develops. If you consider how the dynamics act on this new matrix, you get $A^2B$, the second order term of how the input’s reach develops. This process can be continued for any number of $A$s.
Multiplying by A either adds a new direction or the direction is contained in the current span. As soon as the direction is contained in the current span, the span is closed under $A$, so further multiplying $A$ can’t increase the dimension. Since the state space is $n$-dimensional, the closure happens within $n$ blocks, so $A^{n-1}B$ is the last block that can increase the rank. Therefore, once you reach $A^nB$, this can be written as a combination of lower order terms. This is an application of the Cayley-Hamilton theorem, which states that $A^n$ is a combination of lower order powers. As a result, any matrix from $A^{n}B$ and beyond doesn’t tell you anything about whether your system is controllable.

$B$, $AB$ … $A^{n-1}B$ are all $n \times m$ matrices and all tell us something. We care about whether the span of all these possible directions let us reach our full state space. To do this, we can put them into a single matrix, known as the controllability matrix. This is $n \times nm$.

The columns of $\mathcal{C}$ are possible directions achievable by the system. The total space of achievable directions is simply any combination of these directions, with a dimension of $\text{rank}(\mathcal{C})$ This space only covers the full state space when $\text{rank}(\mathcal{C}) = n$ In other words: If our $\mathcal{C}$ is full rank, the system is controllable.

Observability

How do you know what your system is actually doing? One way to answer this question is to take readings of your system using sensors like encoders. These readings can be stored in an output vector $y$. $y$ usually has different variables than $x$ or $u$. The output $y$ is a function of the state $x$, which means as the state changes, the output also changes.

$y$ can also be expressed as a linear function:

C is known as the output matrix. It must be a $k \times n$ matrix because it’s transforming the $x$ of size $n$ into $y$ of size $k$.

So what does observability mean? A system is observable if you can construct the full state based on the output $y$ over a finite time. A system is unobservable if you can’t construct a full state, no matter how long you have. An unobservable system might be two identical pendulums with a sensor that measures the sum of their angles. In this case, if the pendulums are decoupled, it’s impossible to determine what their individual angles are.

How to prove a system is observable? There’s an interesting duality between observability and controllability. Consider the output matrix $C$. To provide you with the output for a given $x$, the state is multiplied by $C$ to produce the direction $Cx$. In many cases, this vector isn’t enough information to determine your original state. Like $u$, $y$ often has fewer variables than $x$, making $y$ underdetermined. To continue our pendulum example, the hinge might have a single gyroscope providing you with angular velocity. As a result, you need to construct both angle and angular velocity from a single angular velocity variable.

If you consider how $C$ acts on the dynamics, you get $CA$, the map from $x$ to $\dot{y}$. If you consider how this new map acts on the dynamics you get $CA^2$, the map from $x$ to $\ddot{y}$. Just like when determining controllability, this process can be continued for any number of $A$s. Similarly, once you reach $CA^n$, this can be written as a combination of the maps. In another application of Cayley-Hamilton, any matrix from $CA^{n}$ and beyond doesn’t tell you anything about whether your system is observable.

$C$, $CA$ … $CA^{n-1}$ are all $k \times n$ matrices and all tell us something. We care about whether the span of the rows can jointly distinguish every state. To do this, we can put them into a single matrix, known as the observability matrix $\mathcal{O}$. This is $nk \times n$. $\mathcal{O}$ stacks rows rather than columns.

Like $\mathcal{C}$, the rows of $\mathcal{O}$ are directions in state-space measurable by the system. The span of these rows is simply any combination of these directions, with a dimension of $\text{rank}(\mathcal{O})$ This space only covers the full state space when $\text{rank}(\mathcal{O}) = n$. In other words: If our $\mathcal{O}$ is full rank, the system is observable. The symmetry between $\mathcal{C}$ and $\mathcal{O}$ is undeniable!