Stochastic Processes

Learning Outcomes

Learning Outcomes

Having successfully completed this module you will be able to:

• Derive the Kolmogorov equations for a Markov process with time independent and time/age dependent transition intensities
• Demonstrate how a Markov jump process can be simulated
• Describe a time-inhomogeneous Markov chain and its simple applications
• Determine the stationary and equilibrium distributions of a Markov chain
• Solve the Kolmogorov equations in simple cases
• Understand the definition of a stochastic process and in particular a Markov process, a counting process and a random walk
• Recall the definition and derive some basic properties of a Poisson process
• Understand survival, sickness and marriage models in terms of Markov processes
• Classify a stochastic process according to whether it operates in continuous or discrete time and whether it has a continuous or a discrete state space, and give examples of each type of process
• Classify the states of a Markov chain as transient, null, recurrent, positive recurrent, periodic, aperiodic and Ergodic
• Calculate the distribution of a Markov chain at a given time
• Describe a Markov chain and its transition matrix
• Understand, in general terms, the principles of stochastic modelling
• State the Kolmogorov equations for a Markov process where the transition intensities depend not only on age/time, but also on the duration of stay in one or more states
• Demonstrate how a Markov chain can be simulated
• Define and explain the basic properties of Brownian motion, demonstrate an understanding of stochastic differential equations and then to integrate, and apply the Ito formula

Markov Chain

Definition and basic properties

Classification of states and decomposition of state space

The long term probability distribution of a Markov chain

Modelling using Markov chains

Time-homogeneous Markov jump process

Poisson process and its basic properties

Birth and death processes

Kolmogorov differential equations

Structure of a Markov jump process

Time-inhomogeneous Markov jump process

Definition and basics

A survival model

A sickness and death model

A marriage model

Sickness and death with duration dependence

Basic principles of stochastic modelling

Classification of stochastic modelling

Postulating, estimating and validating a model

Simulation of a stochastic model and its applications

Brownian motion: Definition and basic properties. Stochastic differential equations, the Ito integral and Ito formula. Diffusion and mean testing processes. Solution of the stochastic differential equation for the geometric Brownian motion and Ohrnstein-Uhlenbeck process.

Lectures, problem classes, coursework, surgeries and private study

Summative

This is how we’ll formally assess what you have learned in this module.

Referral

This is how we’ll assess you if you don’t meet the criteria to pass this module.

Repeat

An internal repeat is where you take all of your modules again, including any you passed. An external repeat is where you only re-take the modules you failed.