Periodic Saving

[JulianDay-Cooney]

Hello, I'm still relatively new to Bonsai and I'm not sure if this is even possible: I'm attempting long duration recordings (6-14 days) and I would like to be able to save the current data (effectively stop and start bonsai programmatically) in order to save the data as I go along. My data streams are video (saved with videowriter), extracted information from videos (motion, ROIs, etc--all saved with csvwriter), and photometry data (collected with neurophotometrics and saved with photometrywriter). There was a conversation in the google group called "periodic video saving using ffmpeg" that made it seem possible, but there were no workflows attached and the conversation was not very informative.

Has anyone been able to figure this out?

Thanks,

Julian

[bruno-f-cruz]

Hi,

The general solution is probably to instantiate your data pipeline and simply record whenever you want to. You can achieve this using Observables. In the attached workflow, I am recording 2min chunks every 5min. The Timer at the highest level of the workflow will tick every 5min and start an observable that will subscribe to the Data stream. Inside, in each "saving iteration" (i.e. CreateObservable) the data is sinked into a csv and VideoWriter. Once a second Timer (which starts at the same time as the subcription to Data) emits a value, the observable and the corresponding data chunks will close.

Hope it helps,
B
PeriodicRecording.zip

[JulianDay-Cooney]

Thank Bruno! I think that works. So, to be clear, if I set both the timer period and the observables Due time to both be 5 minutes, then I will get a recording saved for the previous 5 minutes? My ultimate goal is for the recording to be continuous so that I can concatenate the data and not have any breaks in the data.

[bruno-f-cruz]

If you want to close and start another logging immediately, I would look at #869.
´Switch´ will probably work in your case. Just make sure you are not actually losing samples at the transitions (I dont think this will be a problem in your case, but check it by logging the sample number).

[Brandon-Fricker]

Hi Bruno,

I noticed if I manually turn off a script, it doesn't save all of the video. So, I created a timer similar to the second timer here where the video capture device node goes to a takeUntil node connected to the timer. However, the recorded video still continues to be shorter than expected (If I make the dueTime 2:07 minutes, I get a 1:07 minute video, for example). Do you know what might be causing this issue? Thank you!

[bruno-f-cruz]

Can you include the workflow?

[Brandon-Fricker]

Hi Bruno,
I have attached two workflows here, the dual camera one for fiber photometry does not have a timer yet, but is the one I'm actually using where a 15 minute recording resulted in a 12 minute video (The FP data and video should be time locked, I think....). I made this simplier timer version to test if it would solve my issue. Could it be that the video being created is just at a faster FPS? Looking at the pipeline, it should be 30 and 30fps respectively. I appreciate your time and help!

Bonsai_multicamera_and_timer.zip