MiniRocket .fit() OOM issue with large datasets

[Arderiu]

I have one question regarding MiniRocket: I have a training dataset with size (13604, 1, 3000), and when I call the mrf.fit(X_train) I get a OOM error. A solution I found is to just fit with a subset of the training set, e.g. mrf.fit(X_train13000:,:,]). I assume, this will negatively impact the performance of the feature computation... Is there any way to fit the model "batch by batch" to avoid OOM issues? Thanks!

[oguiza]

No, there isn't @Arderiu.
Actually, the authors of the paper recommend using MiniRocketClassifier or MiniRocketRegressor when the dataset is < 10k samples.
However, there's a Minirocket version written in Pytorch that can be used with any dataset size.
You can find more information in this tutorial.

[RoyiAvital]

Is there a way to wrap a PyTorch data loader?
In all examples it uses X, y which are NumPy.

[oguiza]

I don't understand what you mean by "wrap a Pytorch data loader".

[RoyiAvital]

Use a PyTorch data loader instead of a NumPy array.

[oguiza]

Yes, you can use MiniRocket or MiniRocketPlus (https://timeseriesai.github.io/tsai/models.minirocketplus_pytorch.html) with a regular Pytorch dataloader like any other model in tsai.