Baum-Welch Algorithm next up previous
Next: Gradient based method Up: Maximum Likelihood (ML) criterion Previous: Maximum Likelihood (ML) criterion

Baum-Welch Algorithm

This method can be derived using simple ``occurrence counting'' arguments or using calculus to maximize the auxiliary quantity


over tex2html_wrap_inline2812 [],[, p 344-346,]. A special feature of the algorithm is the guaranteed convergence .
To describe the Baum-Welch algorithm, ( also known as Forward-Backward algorithm), we need to define two more auxiliary variables, in addition to the forward and backward variables defined in a previous section. These variables can however be expressed in terms of the forward and backward variables.

First one of those variables is defined as the probability of being in state i at t=t and in state j at t=t+1. Formally,


This is the same as,


Using forward and backward variables this can be expressed as,


The second variable is the a posteriori probability,


that is the probability of being in state i at t=t, given the observation sequence and the model. In forward and backward variables this can be expressed by,


One can see that the relationship between tex2html_wrap_inline2838 and tex2html_wrap_inline2840 is given by,


Now it is possible to describe the Baum-Welch learning process, where parameters of the HMM is updated in such a way to maximize the quantity, tex2html_wrap_inline2684 . Assuming a starting model tex2html_wrap_inline2698 ,we calculate the ' tex2html_wrap_inline2846 's and ' tex2html_wrap_inline2848 's using the recursions 1.5 and 1.2, and then ' tex2html_wrap_inline2850 's and ' tex2html_wrap_inline2852 's using 1.12 and 1.15. Next step is to update the HMM parameters according to eqns 1.16 to 1.18, known as re-estimation formulas.




These reestimation formulas can easily be modified to deal with the continuous density case too.

Narada Warakagoda
Fri May 10 20:35:10 MET DST 1996

Home Page