v3.1.0

What's Changed

• build(deps): bump github.com/prometheus/client_golang from 1.20.2 to 1.20.4 by @dependabot in #510
• build(deps): bump golangci/golangci-lint-action from 6.1.0 to 6.1.1 by @dependabot in #522
• Update docker-compose.yml by @wagyus in #534
• feat(multistream): Experimental multistream by @Entrivax in #355
  • feat(multistream): Experimental multistream by @Entrivax in Zibbp/ganymede-frontend#25
• fix(playlist): cascade delete multistream info by @Zibbp in #535

Features

Multistream

Huge thanks to @Entrivax for contributing this feature. See a full demonstration here: https://www.youtube.com/watch?v=aKQGbw2gb70.

Videos added to the same playlist can be viewed together in a "multistream" layout allowing you to see each stream's perspective, synced up. Non-livestreams use the streamed_at field while livestreams use the created_at field to determine the initial timing in the timeline.

Multistream.Demo.mp4

New Contributors

• @wagyus made their first contribution in #534
• @Entrivax made their first contribution in #355

Full Changelog: v3.0.2...v3.1.0

v3.0.2

Fixes

• Re-add database SSL support @Zibbp in #508.
• Fix needing to restart container when changing application settings (Admin > Settings).
  • The worker process now correctly loads the config from file when needing to fetch a config value preventing the need to restart the API container.

What's Changed

• build(deps): bump golang.org/x/oauth2 from 0.21.0 to 0.22.0 by @dependabot in #486
• build(deps): bump github.com/go-jose/go-jose/v4 from 4.0.3 to 4.0.4 by @dependabot in #488
• build(deps): bump github.com/prometheus/client_golang from 1.19.1 to 1.20.2 by @dependabot in #494
• build(deps): bump github.com/riverqueue/river/rivertype from 0.11.2 to 0.11.4 by @dependabot in #495
• build(deps): bump github.com/riverqueue/river from 0.11.2 to 0.11.4 by @dependabot in #496
• build(deps): bump github.com/testcontainers/testcontainers-go/modules/postgres from 0.32.0 to 0.33.0 by @dependabot in #499
• build(deps): bump golang.org/x/oauth2 from 0.22.0 to 0.23.0 by @dependabot in #503
• build(deps): bump github.com/jackc/pgx/v5 from 5.6.0 to 5.7.0 by @dependabot in #502
• feat: re-add database ssl support by @Zibbp in #508

Full Changelog: v3.0.1...v3.0.2

v3.0.1

Important

If you have not upgraded to v3.0.0 yet see the release notes for upgrade instructions!

What's Changed

• build(deps): bump golangci/golangci-lint-action from 6.0.1 to 6.1.0 by @dependabot in #479
• build(deps): bump golang.org/x/crypto from 0.25.0 to 0.26.0 by @dependabot in #481
• build(deps): bump github.com/docker/docker from 27.0.3+incompatible to 27.1.1+incompatible by @dependabot in #480
• fix arm64 builds by @Zibbp in #484
  • arm64 images will now work with the :latest tag

Full Changelog: v3.0.0...v3.0.1

v3.0.0

Important

Version 3.0.0 includes numerous breaking changes that are not backwards compatible!

Features and Changes

• Replace Temporal with a simpler task system: River.
  • Temporal provided a lot of nice features but ultimately is too difficult for a selfhosted application. Using a simpler task system with custom workflows allows for more customization.
  • Bring back the ability to restart any task from the queue page (except for live video and chat downloads).
    • This brings back the possibility to pull a new version of Ganymede to resolve an issue and then re-run failing archive tasks.
    • 
  • Migrate all but one scheduled tasks to River. The live channel check is not run in River as that will just pollute the queue and some users have a very low interval configured.
  • Optionally deploy River WebUI for a look into the tasks queue
    • 
  • Archive tasks have a heartbeat. If the task times out (crashes, Ganymede restarts, etc) a watchdog will attempt to re-queue the task.
    • Livestream video and chat downloads are not retried, instead they are cancelled and the subsequent workflows (post-process video and convert chat) started.
• Add support to customize all the directories inside the container using environment variables (#337).
  • On start a migration function will check if the paths change. If a change is detected it will attempt to update all paths in the database.
• Category restrictions can now optionally be applied to live streams.
  • On each channel check, the category of the live stream is compared against the list of categories configured in the watched channel settings. If a match is found then the stream is archived. It will not detect if the channel switches to a different category after the live stream archive is created. If this is something that interests you let me know!
  • 
• Add a "Blocked VODs" page in the admin panel.
  • Video IDs can be blocked from being archived.
• Ability to cancel a VOD archive in-progress.
  • Optionally can delete the video along with the video files.
  • Optionally can block the video ID from future archives.
• Add a button to regenerate the static thumbnail for videos.
  • A tasked is queued which uses ffmpeg to extract a screenshot at a random position for the full resolution thumbnail along with the web thumbnail.
  • Browsers along with the nginx config aggressively cache thumbnails so a hard reload may be required to see the updated thumbnail.
  • 

Notable Changes

Important changes that don't break functionality.

• The ganymede-nginx container isn't required anymore. The API container will serve the static files at the VIDEO_DIR env var route. Example: VIDEO_DIR=/data/videos then it would be available at localhost:4800/data/videos. There is no index so you can't browse files. Updating the frontend CDN_URL to localhost:4800/data/videos is the same as serving the files via Nginx.
  • I recommend still using Nginx, it provides caching and sets headers so your browser caches important images like thumbnails. This is mainly for people who don't like running a lot of containers.

Breaking Changes

• Completely different worker system (requires changes to the compose file)
  • Remove temporal containers
  • Add a new river-ui container
• Moved the following settings from the config.json to environment variables (set these as environment variables on the api container)
  • DEBUG
  • OAUTH_ENABLED
• Added environment variables to change the paths of important directory inside the container. Changing these will require changing the mounted volumes.
  • VIDEOS_DIR
  • TEMP_DIR
  • CONFIG_DIR
  • LOGS_DIR
    The defaults of these are:

	VideosDir string `env:"VIDEOS_DIR, default=/data/videos"`
	TempDir   string `env:"TEMP_DIR, default=/data/temp"`
	ConfigDir string `env:"CONFIG_DIR, default=/data/config"`
	LogsDir   string `env:"LOGS_DIR, default=/data/logs"`

• **You will need to update the paths for your containers to match these new defaults, or override the default by specifying the path in the environment variables. Example:

volumes:
-   - /mnt/nas/vods:/vods
+  - /mnt/nas/vods:/data/videos
-   - ./logs:/logs
+  - ./logs:/data/logs
+  - ./temp:/data/temp
+  - ./config:/data/config # be sure to move `config.json` if needed

• I recommend mounting the TEMP_DIR as a volume so temporary files are not lost on container restart. See the below docker-compose diff for an example.

Updating Your Instance to v3.0.0

Important

BACK UP YOUR CURRENT INSTANCE, INCLUDING THE DATABASE!!!

The upgrade is non-reversible. Be sure to create a backup of the database if something goes wrong. Rely on your standard backup procedure or create a database dump by running the following command.

docker exec ganymede-db pg_dump -U ganymede ganymede-prd | gzip > /tmp/dump.sql.gz

Upgrade Steps

Wait until you have no active archives. There is no archive comparability between versions!

It may be simpler to start with a fresh and updated docker-compose.yml file. You will need to update the paths and any env vars from your old compose file.

1. Bring down all containers docker compose down
2. Make modifications to the docker-compose.yml file. Optionally you can start with a fresh docker-compose.yml file. Be sure to update the volume mounts to account for the new *_DIR variables.
   1. Perform the changes to your docker-compose.yml outlined in the breaking changes section above. This includes...
   2. Removing the temporal containers, and any references to it.
   3. Optionally update the paths in the VIDEOS_DIR and TEMP_DIR environment variable. I recommend mounting the TEMP_DIR to a volume on your host. This is to prevent losing data if the container crashes or restarts. Any modification to these variables requires changing the volume mounts as well.
   4. Adding the river-ui container

version: "3.3"
services:
  ganymede-api:
    container_name: ganymede-api
    image: ghcr.io/zibbp/ganymede:latest
    restart: unless-stopped
    depends_on:
-      - ganymede-temporal
+      - ganymede-db
    environment:
      - TZ=America/Chicago # Set to your timezone
+      - DEBUG=false # set to true for debug logs 
+      - VIDEOS_DIR=/data/videos
+      - TEMP_DIR=/data/temp
+      - LOGS_DIR=/data/logs
+      - CONFIG_DIR=/data/config
      - DB_HOST=ganymede-db
      - DB_PORT=5432
      - DB_USER=ganymede
      - DB_PASS=PASSWORD
      - DB_NAME=ganymede-prd
      - DB_SSL=disable
      - JWT_SECRET=SECRET
      - JWT_REFRESH_SECRET=SECRET
      - TWITCH_CLIENT_ID=
      - TWITCH_CLIENT_SECRET=
      - FRONTEND_HOST=http://IP:PORT
-      - COOKIE_DOMAIN=http://domain.com
      # OPTIONAL
+     # - OAUTH_ENABLED=false
      # - OAUTH_PROVIDER_URL=
      # - OAUTH_CLIENT_ID=
      # - OAUTH_CLIENT_SECRET=
      # - OAUTH_REDIRECT_URL=http://IP:PORT/api/v1/auth/oauth/callback # Points to the API service
-     - TEMPORAL_URL=ganymede-temporal:7233
      # WORKER
      - MAX_CHAT_DOWNLOAD_EXECUTIONS=2
      - MAX_CHAT_RENDER_EXECUTIONS=2
      - MAX_VIDEO_DOWNLOAD_EXECUTIONS=3
      - MAX_VIDEO_CONVERT_EXECUTIONS=2
    volumes:
-   - /mnt/nas/vods:/vods
+   - /mnt/nas/vods:/data/videos
-   - ./logs:/logs
+   - ./logs:/data/logs
+   - ./temp:/data/temp
+   - ./config:/data/config # be sure to move `config.json` if needed
    ports:
      - 4800:4000
  ganymede-frontend:
    container_name: ganymede-frontend
    image: ghcr.io/zibbp/ganymede-frontend:latest
    restart: unless-stopped
    environment:
      - API_URL=http://IP:PORT # Points to the API service
      - CDN_URL=http://IP:PORT # Points to the CDN service
      - SHOW_SSO_LOGIN_BUTTON=true # show/hide SSO login button on login page
      - FORCE_SSO_AUTH=false # force SSO auth for all users (bypasses login page and redirects to SSO)
      - REQUIRE_LOGIN=false # require login to view videos
    ports:
      - 4801:3000
-  ganymede-temporal:
-    image: temporalio/auto-setup:1.23
-    container_name: ganymede-temporal
-    depends_on:
-      - ganymede-db
-    environment:
-      - DB=postgres12 # this tells temporal to use postgres (not the db name)
-      - DB_PORT=5432
-      - POSTGRES_USER=ganymede
-      - POSTGRES_PWD=PASSWORD
-      - POSTGRES_SEEDS=ganymede-db # name of the db service
-    ports:
-      - 7233:7233
-  # -- Uncomment below to enable temporal web ui --
-  # ganymede-temporal-ui:
-  #   image: temporalio/ui:latest
-  #   container_name: ganymede-temporal-ui
-  #   depends_on:
-  #     - ganymede-temporal
- #   environment:
-  #     - TEMPORAL_ADDRESS=ganymede-temporal:7233
-  #   ports:
-  #     - 8233:8080
  ganymede-db:
    container_name: ganymede-db
    image: postgres:14
    volumes:
      - ./ganymede-db:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=PASSWORD
      - POSTGRES_USER=ganymede
      - POSTGRES_DB=ganymede-prd
    ports:
      - 4803:5432
  ganymede-nginx: # this isn't necessary if you do not want to serve static files via Nginx. The API container will serve the `VIDEO_DIR` environment variable.
    container_name: ganymede-nginx
    image: nginx
    volumes:
      - /path/to/ng...

v2.3.2

Tip

Now is the best time to switch back to the :latest image tags if you are on :main for testing.

v2.3.2

Important

Required Change!

The following changes must be made to your docker-compose.yml file if you have not already done so. Do not make this change if you have archives running. Wait until nothing is being archived then perform the changes. See #441 for more information.

  ganymede-temporal:
-   image: temporalio/auto-setup:1
+   image: temporalio/auto-setup:1.23
    container_name: ganymede-temporal
    depends_on:
      - ganymede-db
    environment:
-     - DB=postgresql # this tells temporal to use postgres (not the db name)
+     - DB=postgres12 # this tells temporal to use postgres (not the db name)
      - DB_PORT=5432
      - POSTGRES_USER=ganymede
      - POSTGRES_PWD=PASSWORD

Features

Audio Only

Support for audio-only archives.

Bug Fixes

A bunch of bug fixes are included in this release, most important are #437 and #413.

What's Changed

• feat: audio only archive support by @Zibbp in #431
• build(deps): bump go.temporal.io/api from 1.32.0 to 1.34.0 by @dependabot in #432
• build(deps): bump golangci/golangci-lint-action from 4.0.0 to 6.0.1 by @dependabot in #428
• build(deps): bump github.com/go-playground/validator/v10 from 10.19.0 to 10.20.0 by @dependabot in #421
• feat(archive): chat conversion improvements by @Zibbp in #433
• fix(archive): delete postprocessed video file after converting to hls by @Zibbp in #438
• feat(activities): handle and kill multiple chat process ids by @Zibbp in #439
• fix(docker): pin temporal image by @Zibbp in #440
• fix: re-add tasks by @Zibbp in #442

Full Changelog: v2.3.1...v2.3.2

v2.3.1

Tip

Now is the best time to switch back to the :latest image tags if you are on :main for testing.

What's Changed

• Update TwitchDownloader version to fix new VOD ids being greater than what int32 supports lay295/TwitchDownloader#1057

  • You may need to restart the ArchiveTwitchChatWorkflow for any failed archives. Restarting ArchiveTwitchChatWorkflow will restart the entire chat archive flow (download, render, and move). To restart, click the failed workflow then click the "restart" button.
  • 

• fix : #416 oidc by @boringwolf in #417

New Contributors

• @boringwolf made their first contribution in #417

Full Changelog: v2.3.0...v2.3.1

v2.3.0

Tip

Now is the best time to switch back to the :latest image tags if you are on :main for testing.

New Features

Muted Segments

Going forward, all video (vod) downloads will also save the muted segments. These are saved to the database in the table muted_segments. They are also saved to the info.json file. These are not displayed in the frontend yet, I'm still trying to find a good way to display the segments. For now you can use the API or query the database.

/api/v1/vod/<id>?with_channel=true&with_chapters=true&with_muted_segments=true

Title Regex

Watched channels now have a new option called "title regex". Custom regex can be applied to either live streams or video downloads. This allows you to customize what is downloaded by applying regex to the title. The obvious use case is to ignore reruns as the title usually contains it, but it's all customizable to fit your needs.

To get started, edit a "watched channel" and click the plus button under Advanced > Title Regex

A new entry will be visible allowing you to enter a regex and apply some optional settings.

• negative: Invert the match, meaning you do not want the regex
• apply to video downloads: Apply the regex to video downloads. By default the regex is only applied to live streams. If this is checked then it will not be used for live streams. You need to make two regexes if you want to apply to both live stream and videos.

A title regex to not download any rerun livestreams would look like the following

For more information please see the wiki page https://github.com/Zibbp/ganymede/wiki/Watched-Channel-Title-Regex. If you have any useful regexes, please share in a discussion!

What's Changed

• feat: save muted segments by @Zibbp in #406
• feat: title regex filter by @Zibbp in #409
• build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in #410
• build: package updates by @Zibbp in #411
• build(deps): bump docker/setup-buildx-action from 3.2.0 to 3.3.0 by @dependabot in #403

Full Changelog: v2.2.0...v2.3.0

v2.2.0

What's Changed

• The Twitch API is slow to update when a stream goes offline, this is especially noticeable with a low "live check interval". #385 adds some logic to detect if a stream gets archived when it's truly offline, but the API hasn't updated yet. This includes stopping the queue/workflows and deleting the video entry.
• Add a "max video age" to watched channels when archiving previous VODs. VODs are only archived if they are less than X number of days old. (#395).
  • 
• Re-implement the dark/light mode theme toggle (#397).

Commits

• build(deps): bump docker/login-action from 3.0.0 to 3.1.0 by @dependabot in #393
• build(deps): bump docker/build-push-action from 5.1.0 to 5.3.0 by @dependabot in #392
• build(deps): bump docker/setup-buildx-action from 3.0.0 to 3.2.0 by @dependabot in #391
• build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 by @dependabot in #384
• build(deps): bump github.com/go-playground/validator/v10 from 10.16.0 to 10.19.0 by @dependabot in #381
• build(deps): bump github.com/google/uuid from 1.5.0 to 1.6.0 by @dependabot in #375
• build(deps): bump github.com/rs/zerolog from 1.31.0 to 1.32.0 by @dependabot in #374
• Delete archive if stream is ended but Twitch API is slow to update by @Zibbp in #385

Full Changelog: v2.1.0...v2.2.0

v2.1.0

Important

Temporal container changes

It's been reported that the sqlite version of the Temporal container that was previously shipped with Ganymede caused issues, see #339 and #353 for more information. To fix this, I'm now recommending everyone to switch to the more "production-like" temporal setup, which uses a database. To perform this change, follow the below.

No workflow data will be saved from the previous container, so ensure that nothing is actively being archived

1. Bring down Ganymede docker compose down
2. Perform the following change to your docker-compose.yml file. The new Temporal container requires access to your database, make sure to update the POSTGRES_USER and POSTGRES_PWD with the username and password of your ganymede-db container.

-  ganymede-temporal:
-    container_name: ganymede-temporal
-    image: ghcr.io/zibbp/ganymede-temporal:latest
-    restart: unless-stopped
-    volumes:
-      - ./temporal:/data
-    ports:
-      - 7233:7233
-      - 8233:8233 # web ui - optional
+  ganymede-temporal:
+    image: temporalio/auto-setup:1
+    container_name: ganymede-temporal
+    depends_on:
+      - ganymede-db
+    environment:
+      - DB=postgresql # this tells temporal to use postgres (not the db name)
+      - DB_PORT=5432
+      - POSTGRES_USER=ganymede
+      - POSTGRES_PWD=PASSWORD
+      - POSTGRES_SEEDS=ganymede-db # name of the db service
+    ports:
+      - 7233:7233

3. Optionally add the web-ui service if you wish to use that.

+  ganymede-temporal-ui:
+    image: temporalio/ui:latest
+    container_name: ganymede-temporal-ui
+    depends_on:
+      - ganymede-temporal
+    environment:
+      - TEMPORAL_ADDRESS=ganymede-temporal:7233
+    ports:
+      - 8233:8080

4. Delete the temporal directory
5. Bring everything back up docker compose up -d
6. Ensure the temporal container is working fine docker logs ganymede-temporal and check that the API container is able to connect docker logs ganymede-api

Refer to the docker-compose.yml file for a full example.

What's Changed

If you've been running the :main image for either the API or frontend, now is a good time to switch back to the :latest tag.

Frontend

• Bump package versions
• Place theater mode button next to full screen button
• Show thumbnail when video hasn't begun playing yet
• Add sharp and ensure /app directory is writable
• Fix bug the prevented the VOD title component from coming back after exiting theater mode

API

• Use debian as the x86 docker base image
• Re-run live thumbnail download 10 minutes after the archive starts

Lots of bug fixes and package version bumps:

• feat(workflow): update live stream archives with correct vod ids by @Zibbp in #351
• Fixes 354 by @Zibbp in #357
• Fixes 2024 02 04 by @Zibbp in #361
• fix(exec): check if livechatworkflowid is populated by @Zibbp in #364
• fix: add comments by @Zibbp in #365
• feat(workflow): re-download live thumbnail again after 10 minutes by @Zibbp in #372
• build(deps): bump golangci/golangci-lint-action from 3.7.0 to 4.0.0 by @dependabot in #368
• build(deps): bump golang.org/x/crypto from 0.17.0 to 0.19.0 by @dependabot in #367
• build(deps): bump docker/metadata-action from 5.0.0 to 5.5.1 by @dependabot in #363
• build(deps): bump github.com/mattn/go-sqlite3 from 1.14.17 to 1.14.22 by @dependabot in #362
• build(deps): bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 by @dependabot in #338
• build(deps): bump github.com/labstack/echo/v4 from 4.11.3 to 4.11.4 by @dependabot in #332
• build(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.17.0 by @dependabot in #366
• build(deps): bump actions/setup-go from 4 to 5 by @dependabot in #322
• build(deps): bump github/codeql-action from 2 to 3 by @dependabot in #326
• build(docker): use debian as base image by @Zibbp in #373

Full Changelog: v2.0.2...v2.1.0

v2.0.2

What's Changed

• feat(workflows): paginate executions by @Zibbp in #341
• fix(channel): allow specifying an external id when creating a channel… by @Zibbp in #344
• feat(activity/livevideo): attempt to update livestream archive with the external vod id by @Zibbp in #347
• Fix live stream chat update if the channel does not have any previous videos

Frontend

What's Changed

• feat(workflows): paginate workflows by @Zibbp in Zibbp/ganymede-frontend#19
• feat(admin/channel): allow setting external id when creating channel by @Zibbp in Zibbp/ganymede-frontend#20
• fix(chat): fallback emote tooltip text to fragment text when empty + use mantine tooltips by @Entrivax in Zibbp/ganymede-frontend#23
• fix(channel): add more elements per row on bigger screens by @Entrivax in Zibbp/ganymede-frontend#21
• fix(vod): try improve responsiveness of the vod page by @Entrivax in Zibbp/ganymede-frontend#22

New Contributors

• @Entrivax made their first contribution in Zibbp/ganymede-frontend#23

Full Changelog: v2.0.1...v2.0.2