Sharktopoda 2 integration

[kevinsbarnard]

Related to #2. Sharktopoda 2 (MBARI's new video player) supports rendering and editing of localizations via a remote protocol. Gridview should be integrated with Sharktopoda 2 to facilitate this.

[kevinsbarnard]

From VARS meeting 2/8/2023:

Requirements

• Open video(s) for selected
• Show localizations in Sharktopoda
• Propagate edits & deletions back to gridview → VARS
• Also delete the annotation (pending user approval)

[kevinsbarnard]

Additional requirement (see #6):

• Color boxes according to concept label (consistent with GridView colors)

[kevinsbarnard]

Opening video does not seek to the correct frame on Mac. This is likely due to a failed connection because of the forced IPv6 UDP socket -- need to investigate more.

[kevinsbarnard]

sharktopoda-client v0.4.3 introduces handling of IPv4 and IPv6 ephemeral sockets. Need to test if this resolves the issue above.

[lonnylundsten]

Hoping to have completed by 11/30/2023; Appears to be mostly complete but KRW will be testing this week to see if there are any outstanding issues.

[kevinsbarnard]

From KRW:

I spent a little time this morning troubleshooting the shark-2 issue with Gridview and VARS Anno and figured out when it is not working correctly. If I open Gridview and choose open file on an image, it seeks to the correct elapsed time in video on shark-2. Then I open VARS Annotation, open a video file, close VARS annotation, and then go back to Gridview (has been open the whole time) and try to Open Video on an image, it opens the video in shark-2 from Gridview but seeks to the beginning of the video file, elapsed time 0, not to the correct elapsed time within video. So it seems like VARS annotation is changing something with shark-2 after it has been opened and then Gridview doesn't work correctly even after closing VARS annotation.

It sounds like the following is happening:

1. S2 is opened and not connected to any application
2. VARS Gridview is opened and connects to S2; S2 is now connected to VARS Gridview
3. VARS Gridview is able to send commands to S2, and S2 replies since it's connected to VARS Gridview; videos can be opened, seeked, and have localizations sent over
4. VARS Annotation is opened and connects to S2; S2 is now connected to VARS Annotation and drops the connection to VARS Gridview
5. VARS Annotation is able to send commands, and S2 sends replies to VARS Annotation since it's connected. VARS Gridview is able to send commands, but does not receive replies as VARS Annotation has broken its connection, and S2 only maintains a single connection; videos can be opened, but the "open done" reply is never returned, so VARS Gridview does not send a seek/add localization command.

So, it sounds like gridview needs the ability to re-connect without restarting the application, since VARS Annotation will interrupt the connection. In that vein, a nice-to-have could be an S2 connection status indicator (e.g. red X vs. green check).

[kevinsbarnard]

From v0.4.0, there is now a "Video Player" tab with controls for Sharktopoda 2.

This can be used to specify the host/ports for Sharktopoda 2 and re-connect as needed when VARS Annotation grabs the connection.

[kevinsbarnard]

From discussion 12/06/2023:

First cut: load all annotations from a given video, not just the selected one.

Next version: load annotations from within a time window of the selected annotation. Will require reaching out to VARS for other annotations

[kevinsbarnard]

As of v0.10.0, all loaded localizations in the grid that map to the same video reference as the selected one will be added to Sharktopoda.

[lonnylundsten]

Appears to be completed @kevinsbarnard. Ok to close?