What is the difference between a hidden Markov model and a hidden semi-Markov model?

In a hidden Markov model the chance of leaving a state is the same every day, however long the state has lasted, so durations are geometric and the most likely stay is a single step. A hidden semi-Markov model draws how long each visit lasts from a distribution chosen for that state, such as a negative binomial, and moves on only when the visit ends, so its transition matrix typically has zeros on the diagonal. Bulla and Bulla (2006) found this reproduces the slow decay in the autocorrelation of squared daily returns better than a plain HMM.

Can a hidden Markov model detect a regime change in real time?

Only through its filtered probabilities, which use data up to the present. The smoothed probabilities of the forward-backward algorithm, and the Viterbi path, also use later observations, so a regime history built from them looks sharper than anything an investor could have known at the time; parameters fitted on the full sample leak in the same way. Even filtered, a change is recognised only once enough evidence has arrived, so a regime model confirms a shift with a lag rather than foreseeing it.

How many states should a regime model have?

There is no fixed answer. Studies of equity returns commonly use two to four states, such as calm and turbulent, because each state needs enough episodes to be estimated. Adding states almost always improves the fit to the past, so the number is usually chosen with an information criterion such as the BIC, with out-of-sample likelihood, and by whether each state has a clear economic meaning.

Does the Regime Radar use a hidden Markov model?

Yes, for the composite. A six-state hidden semi-Markov model fuses the nine axis readings into one state, from strong risk-on to crisis. The nine axis states come from percentile classifiers; a hidden Markov model is trained for each axis only as a cross-check and does not set the published states. The Radar's chance of a change within five sessions starts from the composite's duration distributions and how long the current state has lasted; a gradient-boosted model combines that estimate with market-stress, volatility, credit-spread, yield-curve and early-warning inputs, and the result is calibrated by isotonic regression followed by Platt scaling.

FAQ · Hidden Markov model

What is a hidden Markov model (HMM) in finance?

Reviewed against the platform's code on Sep 26, 2026

A hidden Markov model (HMM) is a statistical model of a system that moves between states that cannot be observed directly, following fixed transition probabilities, while each state produces data with its own distribution. In markets, the hidden states are regimes, such as calm or turbulent, and the model infers from returns or indicators how likely each is today.

Why it matters

Markets do not behave the same way all the time. Volatility, correlations and the reward for risk shift between calm and turbulent periods, and a model fitted to the average of both is wrong in each. An HMM gives those periods a formal structure: how each regime behaves, how likely each is to follow another and, as every new observation arrives, the probability of being in each regime now. A probability, rather than a hard label, lets a regime scale position sizing and risk limits by how sure the model is. Hamilton (1989) popularised the idea in economics with a two-state model of expansions and recessions in US output growth.

How it works

The model has hidden states, a transition matrix whose entry in row i, column j is the probability of moving from state i to state j at the next step, and an emission distribution for each state, often a Gaussian with its own mean and variance. The Baum-Welch algorithm estimates them from data, and the forward algorithm updates each state's probability as observations arrive (Rabiner, 1989). In a plain HMM the time spent in a state is geometric: with a 98% daily chance of staying, a regime lasts 50 days on average, yet its single most likely length is one day. A hidden semi-Markov model instead gives each state an explicit duration distribution (Yu, 2010).

How Opulence Alpha applies it

The composite on Opulence Alpha's Regime Radar, from strong risk-on through neutral to crisis, comes from a six-state hidden semi-Markov model that fuses the nine axis readings. Each axis's state comes from a percentile classifier, with thresholds set for that axis. The model updates by forward filtering, never smoothing with later data. Its transition matrix has a zero diagonal, so duration distributions govern how long states last. A switch needs a new state's probability to beat the current one's by over 10 percentage points (5 to enter crisis, 15 to leave) and, after a five-session warm-up, the current state to have lasted its minimum run of at least three sessions. Stress rules on volatility and credit can override it.

Today's six-state composite on the Regime Radar →

Related concepts

Market regime detectionMarket regime detection classifies the current market state — volatility, credit, liquidity, policy, the business cycle — so strategies and risk can adapt. How nine axes and a hidden semi-Markov model do it.Regime-aware portfolio constructionRegime-aware portfolio construction lets the market regime shape how a portfolio is built and how much risk it takes — tightening in stress, never loosening automatically.Look-ahead biasLook-ahead bias: a backtest using information not yet available on the decision date. Where it comes from, how to prevent it, and how Opulence Alpha avoids it.Probabilistic forecastingProbabilistic forecasting publishes a distribution of possible returns, with calibrated intervals, instead of one expected number — so a position can be sized from its uncertainty.

All concepts →Sector behaviour in every market regime →

Questions

What is the difference between a hidden Markov model and a hidden semi-Markov model?: In a hidden Markov model the chance of leaving a state is the same every day, however long the state has lasted, so durations are geometric and the most likely stay is a single step. A hidden semi-Markov model draws how long each visit lasts from a distribution chosen for that state, such as a negative binomial, and moves on only when the visit ends, so its transition matrix typically has zeros on the diagonal. Bulla and Bulla (2006) found this reproduces the slow decay in the autocorrelation of squared daily returns better than a plain HMM.
Can a hidden Markov model detect a regime change in real time?: Only through its filtered probabilities, which use data up to the present. The smoothed probabilities of the forward-backward algorithm, and the Viterbi path, also use later observations, so a regime history built from them looks sharper than anything an investor could have known at the time; parameters fitted on the full sample leak in the same way. Even filtered, a change is recognised only once enough evidence has arrived, so a regime model confirms a shift with a lag rather than foreseeing it.
How many states should a regime model have?: There is no fixed answer. Studies of equity returns commonly use two to four states, such as calm and turbulent, because each state needs enough episodes to be estimated. Adding states almost always improves the fit to the past, so the number is usually chosen with an information criterion such as the BIC, with out-of-sample likelihood, and by whether each state has a clear economic meaning.
Does the Regime Radar use a hidden Markov model?: Yes, for the composite. A six-state hidden semi-Markov model fuses the nine axis readings into one state, from strong risk-on to crisis. The nine axis states come from percentile classifiers; a hidden Markov model is trained for each axis only as a cross-check and does not set the published states. The Radar's chance of a change within five sessions starts from the composite's duration distributions and how long the current state has lasted; a gradient-boosted model combines that estimate with market-stress, volatility, credit-spread, yield-curve and early-warning inputs, and the result is calibrated by isotonic regression followed by Platt scaling.

References

Hamilton, J. D. (1989). A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle. Econometrica 57(2).
Rabiner, L. R. (1989). A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2).
Bulla, J. & Bulla, I. (2006). Stylized facts of financial time series and hidden semi-Markov models. Computational Statistics & Data Analysis 51(4).
Yu, S.-Z. (2010). Hidden semi-Markov models. Artificial Intelligence 174(2).

Educational content about research methods. Not investment advice.