Skip to content

Commit

Permalink
Fix citations
Browse files Browse the repository at this point in the history
  • Loading branch information
gdalle committed Feb 22, 2024
1 parent 4646570 commit f28be9b
Show file tree
Hide file tree
Showing 2 changed files with 48 additions and 7 deletions.
40 changes: 40 additions & 0 deletions paper/HMM.bib
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
@software{antonelloHMMGradientsJlEnables2021,
title = {{{HMMGradients}}.Jl: {{Enables}} Computing the Gradient of the Parameters of {{Hidden Markov Models}} ({{HMMs}})},
shorttitle = {Idiap/{{HMMGradients}}.Jl},
author = {Antonello, Niccolò},
date = {2021-06-07},
doi = {10.5281/zenodo.4454565},
url = {https://doi.org/10.5281/zenodo.4454565},
urldate = {2023-09-12},
organization = {{Zenodo}},
keywords = {hmm},
file = {/home/gdalle/snap/zotero-snap/common/Zotero/storage/PEFYSLF7/4906644.html}
}

@inproceedings{bengioInputOutputHMM1994,
title = {An {{Input Output HMM Architecture}}},
booktitle = {Advances in {{Neural Information Processing Systems}}},
Expand Down Expand Up @@ -84,6 +97,17 @@ @thesis{dalleMachineLearningCombinatorial2022
file = {/home/gdalle/snap/zotero-snap/common/Zotero/storage/CEVJMUP4/Dalle - Machine learning and combinatorial optimization al.pdf}
}

@software{hmmlearnHmmlearnHiddenMarkov2023,
title = {Hmmlearn: {{Hidden Markov Models}} in {{Python}}, with Scikit-Learn like {{API}}},
author = {{hmmlearn}},
date = {2023},
url = {https://github.com/hmmlearn/hmmlearn},
urldate = {2023-09-12},
abstract = {Hidden Markov Models in Python, with scikit-learn like API},
organization = {{hmmlearn}},
keywords = {hmm}
}

@software{mouchetHMMBaseJlHidden2023,
title = {{{HMMBase}}.Jl: {{Hidden Markov Models}} for {{Julia}}},
author = {Mouchet, Maxime},
Expand Down Expand Up @@ -171,6 +195,22 @@ @software{rowleyLogarithmicNumbersJlLogarithmic2023
keywords = {hmm}
}

@article{schreiberPomegranateFastFlexible2018,
title = {Pomegranate: {{Fast}} and {{Flexible Probabilistic Modeling}} in {{Python}}},
shorttitle = {Pomegranate},
author = {Schreiber, Jacob},
date = {2018},
journaltitle = {Journal of Machine Learning Research},
volume = {18},
number = {164},
pages = {1--6},
issn = {1533-7928},
url = {http://jmlr.org/papers/v18/17-636.html},
urldate = {2019-05-16},
keywords = {hmm},
file = {/home/gdalle/snap/zotero-snap/common/Zotero/storage/QKMY8X8M/Schreiber_2018_pomegranate.pdf;/home/gdalle/snap/zotero-snap/common/Zotero/storage/9GWGT5RK/17-636.html}
}

@software{whiteJuliaDiffChainRulesJl2022,
title = {{{JuliaDiff}}/{{ChainRules}}{{.jl}}: V1.44.7},
shorttitle = {{{JuliaDiff}}/{{ChainRules}}.Jl},
Expand Down
15 changes: 8 additions & 7 deletions paper/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@ Given a sequence of observations and a parametric family of HMMs $\mathbb{P}_\th
In generic graphical models, these problems are often intractable, but HMMs have a tree-like structure that yields exact solution procedures with polynomial complexity.
The package `HiddenMarkovModels.jl` leverages the Julia language [@bezansonJuliaFreshApproach2017] to implement those algorithms in a _generic_, _fast_ and _reliable_ way.

| Problem | Formula | Algorithm |
| ------------------------------- | ----------------------------------------------------------------------------- | ---------------- |
| Best state sequence | $\mathrm{argmax}_{X_{1:T}}~\mathbb{P}_\theta(X_{1:T} \vert Y_{1:T}, U_{1:T})$ | Viterbi |
| Observation sequence likelihood | $\mathbb{P}_\theta(Y_{1:T} \vert U_{1:T})$ | Forward |
| State marginals | $\mathbb{P}_\theta(X_t \vert Y_{1:T}, U_{1:T})$ | Forward-backward |
| Maximum likelihood parameter | $\mathrm{argmax}_\theta~\mathbb{P}_\theta(Y_{1:T} \vert U_{1:T})$ | Baum-Welch |
| Inference problem | Algorithm |
| -------------------------------------------------------------------------------------------------------------- | ---------------- |
| Best state sequence $\mathrm{argmax}_{X_{1:T}}~\mathbb{P}_\theta(X_{1:T} \vert Y_{1:T}, U_{1:T})$ | Viterbi |
| Observation sequence likelihood $\mathbb{P}_\theta(Y_{1:T} \vert U_{1:T})$ | Forward |
| State marginals $\mathbb{P}_\theta(X_t \vert Y_{1:T}, U_{1:T})$ | Forward-backward |
| Maximum likelihood parameter $\mathrm{argmax}_\theta~\mathbb{P}_\theta(Y_{1:T} \vert U_{1:T})$ | Baum-Welch |

# Statement of need

Expand Down Expand Up @@ -77,6 +77,7 @@ Model use is just as intuitive:
state_seq, obs_seq = rand(hmm, 100)
logdensityof(hmm, obs_seq)
forward_backward(hmm, obs_seq)
baum_welch(hmm, obs_seq)
```

The extendable `AbstractHMM` interface allows incorporating features such as priors or structured transition matrices, as well as control variables, simply by defining three methods:
Expand All @@ -99,7 +100,7 @@ Special care was given to code quality, type stability, and compatibility checks

However, it is also _limited in scope_.
It centers around CPU efficiency, and remains untested on GPU.
Furthermore, it does not perform probability computations in the logarithmic domain, but instead uses the scaling trick from @rabinerTutorialHiddenMarkov1989 with a variation borrowed from `HMMBase.jl`.
Furthermore, it does not perform probability computations in the logarithmic domain, but instead uses the scaling trick [@rabinerTutorialHiddenMarkov1989] with a variation borrowed from `HMMBase.jl`.
Thus, its numerical stability might be worse than that of Python counterparts on challenging instances.
Luckily, thanks to unrestricted number types, users are free to bring in third-party packages like `LogarithmicNumbers.jl` [@rowleyLogarithmicNumbersJlLogarithmic2023] to recover additional precision.

Expand Down

0 comments on commit f28be9b

Please sign in to comment.