Learning partially observable Markov models from first passage times

We propose a novel approach to learn the structure of partially observable Markov models (POMMs) and to estimate jointly their parameters. POMMs are graphical models equivalent to hidden Markov models (HMMs). The model structure is built to support the first passage times (FPT) dynamics observed in the training sample. We argue that the FPT in POMMs are closely related to the model structure. Starting from a standard Markov chain, states are iteratively added to the model. A novel algorithm POMMPHit is proposed to estimate the POMM transition probabilities to fit the sample FPT dynamics. The transitions with the lowest expected passage times are trimmed off from the model. Practical evaluations on artificially generated data and on DNA sequence modeling show the benefits over Bayesian model induction or EM estimation of ergodic models with transition trimming.