Test-of-time is a small framework to benchmark time-series forecasting models via backtesting. It offers a simple template for benchmarking as well as a variety of ready-to-use datasets and models.
Our vision is to become the central hub for benchmarking time series forecasts. As a user, you need to either (1) find a model that is best suited for your use case, or (2) evaluate the performance of your model for different use cases and derive its improvement potential.
We enable you to do this by integrating commonly used datasets and available models from open source or existing frameworks. We bundle these resources in one place and make them easily usable through a lean benchmarking template. This enables fast benchmarking as a standard. In addition, we provide you with flexibility though customizable features, such as customizing the individual experiments in a benchmark, adding your own models or dataset.
You can now install test-of-time directly with pip:
pip install test-of-timeIf you would like the most up to date version, you can instead install direclty from github:
git clone <copied link from github>
cd test-of-time
pip install .##Features
- offers a simple benchmarking template to compare multiple models on multiple datasets via backtesting
- provides various error metrics
- provides a selection of ready-to-use models and datasets
- offers to add your own model
For a list of past changes, please refer to the releases page.
You can find a first tutorial on how to use the framework in the tutorial section.
We compiled a Contributing to Test-of-Time page with practical instructions and further resources to help you become part of the family.
If you have any question or suggestion, you can participate with our community right here on Github (coming soon)
We also have an active Slack community for our main framework NeuralProphet. Come and join the conversation!
Test-of-time is a sub-framework of the open-source community project NeuralProphet. It is supported by awesome people like you. If you are interested in joining the project, please feel free to reach out to Oskar - email on the NeuralProphet Paper.