TUD.Math. - Control of Discrete-Time Stochastic Systems - 2025 Spring
Course Control of Discrete-Time Stochastic Systems
/College Regeling van Discrete-Tijd Stochastische Systemen
2025 Spring Semester
Technische Universiteit Delft - Afdeling Wiskunde
/Delft University of Technology - Department of Mathematics
Course guide
Welcome to this page!
Contents of this page
Dates and times in 2025:
Quarter 3: February 13, 20, 27, March 6, 13, 20, and 27;
Quarter 4: April 24, May 1, 8, 15, 22, (29 May, TUD closed for a national holiday),
June 5, 12;
Thursdays 08:45 - 10:30 hours in the morning,
Lecture room: Building 36, Room HB 02.230.
Jan H. van Schuppen (Lecturer)
Office Room HB 04.160 of Building 36.
He is present at the TUD only on Thursdays.
He can be reached by email or phone on other days.
Afdeling Wiskunde, Faculteit Elektrotechniek, Wiskunde en Informatica,
Technische Universiteit Delft, Delft.
Department of Mathematics, Faculty of Electrical Engineering,
Mathematics, and Computer Science,
Delft University of Technology, Delft.
Tel. +31 20 695 0177 (Mo. - Wd., Fr.)
Tel. +31 6 5240 2576 (during workdays)
Email J.H.vanSchuppen AT tudelft DOT nl and vanschuppenjanh AT freedom DOT nl
The aim of the course is to provide an introduction to the
basic concepts and results of control of discrete-time stochastic systems.
Control and signal processing problems arise in engineering,
economics, and biology.
Solution of such problems requires mathematical
models in the form of dynamic systems.
In distinction with other areas of mathematics,
a dynamic system describes the relation between an input and
an output signal in terms of the state.
The course is addressed at mathematics students
of the master level,
those in their second year of the master level.
Students from other areas of engineering,
like electrical, mechanical, and aerospace engineering,
are also welcome because the topic of the course is used in those areas.
It is possible that the course will be offered in the hybrid mode
with a live lecture and with online access to invited participants.
Contact the lecturer if you are interested in online participation.
-
LO1.
To analyse
the conditional expectation
of a Gaussian random variable conditioned on another such variable,
of a finite-valued random variable conditioned on another such variable; and
the concept of conditional independence.
-
LO2.
To analyse elementary properties of stochastic processes.
-
LO3.
To evaluate properties of a Gaussian stochastic system,
of a finite stochastic system, and of a general stochastic system,
in particular the forward and the backward system representations
of such systems and their relations; and
the concepts of stochastic observability and
of stochastic co-observability.
-
LO4.
To analyse the results of a stochastic realization of
a stationary Gaussian process,
in particular the equivalence conditions of minimality
of such a realization and
the formulation of the set of equivalent minimal realizations;
the results of a realization of a deterministic linear system,
and the concept of dissipativity of such a system.
-
LO5.
To analyse a stochastic control system,
in particular the stochastic controllability of such a system.
-
LO6.
To evaluate the dynamic programming procedure,
the value function, and the optimal control law,
for optimal control of a completely-observable stochastic system,
both on a finite horizon and on an infinite horizon.
-
LO7.
To evaluate
the Kalman filter of a Gaussian stochastic system,
the Kalman predictor of a Gaussian stochastic system, and
the filter of a finite stochastic system.
-
LO8.
To evaluate the dynamic programming procedure,
the value function, and the optimal control law,
for control of a partially-observable stochastic system
both on a finite horizon and on an infinite horizon.
-
LO9.
To understand several of the research issues of control of
a distributed or a decentralized stochastic control system.
-
To register for the course follow the procedures stated
in the Course Catalogue for master level students in mathematics.
-
For questions about the course you may contact the lecturer
at the email address listed above.
-
Prior knowledge:
-
Required:
Mathematical analysis as provided at the batchelor level; and
linear algebra of the batchelor level;
-
Advised: (1) knowledge of elementary probabily theory;
(2) control and system theory of deterministic linear systems.
-
There will be a written or oral exam,
to be decided based on the number of course participants.
-
Home work sets with exercises.
Weekly homework sets will be published on the web and
at the first lecture be handed out in class.
Solutions to homeworksets are due at the lecturer one week after
they have been published.
-
Assessment:
The final grade of the course consists of the following components:
-
Weekly homework sets;
-
An oral exam to be carried out after the last lecture of the course.
-
Final grade calculation:
(0.5 * the grade of the oral exam plus
0.5 * the average grade of the solutions of the weekly homework sets,
if solutions of all sets were sent to the lecturer
(proportionally if fewer solutions were provided)
).
Last update 23 January 2025.
This page is maintained by
Jan H. van Schuppen.