v3 Beta - Changelog and Upgrade Instructions #474
Comments
|
I'm having an issue with HLS videos, ffmpeg is failing to write the output files. Note I'm running it manually here, but that was just to test it outside the API. it's the same command that the API attempted to run. |
Should be resolved in the latest build of the |
|
Frontend links to /vods path for profile pictures that doesn't seem to resolve anymore in nginx after the latest changes: http://hostname:4802/vods/channelname/profile.png needs to be changed to |
Thanks for the information. I didn't know where this path was coming from (wasn't aware it was stored in the DB), so I ended up just manually updating the paths in the channel info. (This is a new instance with no data other than channel info in it yet.) |
|
I worked around the error Steltek mentioned above by changing the nginx volume line to : |
|
I forgot to mention the nginx changes, I'll update the issue. If you're wanting to use the new default of ganymede-nginx:
container_name: ganymede-nginx
image: nginx
volumes:
- /path/to/nginx.conf:/etc/nginx/nginx.conf:ro
- /mnt/nas/vods:/data/videosThen the |
|
New changes:
|
I had seen those in the commit log and already applied them (but the /vods link from the frontend of course couldn't resolve with that config). |
|
Nevermind I'm tired and stupid, I just have to go into the queue and set them to not processing (for those of you who might have this problem I'll keep this here) Oh, I do have another question, is there a way to manually trigger a chat render assuming you have the json file downloaded already? |
Yes you can, assuming you still have a queue item for the archive. You will need to copy the chat json file to the path the queue expects it to be. This is If you don't have a queue item for this, you can manually create a render by running this command, updating the paths to the input and output. Optionally add any extra settings you have configured for chat renders. docker exec ganymede-api TwitchDownloaderCLI chatrender -i /path/to/chat.json --collision overwrite -h 1440 -w 340 --framerate 30 --font Inter --font-size 13 --log-level Verbose -o /path/to/output/chat.mp4 |
|
Not sure if anyone else can repro this. I have a watched channel with "Archive chat" turned on, and "Render chat" turned off. After a recording finishes, I still see a queue item processing for "Chat Render" (and an ffmpeg task doing the rendering to a video file in the temp directory). After the rendering completes, the mp4 file is left in the temp directory. |
I was able to reproduce this. The logic wasn't detecting if the chat should be rendered for live streams or not. This is fixed in the latest |
|
One thing that I have noticed after the update is that when I bulk add watched channels and de-select both Archive and Render chat, that it's still downloading chat. This in turn is causing the video to fail to process. I will attempt to grab some logs when a stream is finished recording. |
Latest |
|
Wow, after a little bit of playing to get this beta up and running on one of my instances... All I can say wow, this is the best update yet! Touch wood everything is working perfectly. I hope this ends in the main release soon as it's awesome, great work! 👌 |
|
I completely agree with the comment above—I'm really enjoying the beta as well. However, I do have one issue. The progress bar on all my videos is static. When I click on a VOD, it does take me to the latest point I watched, so it seems to be just a visual bug. Additionally, when I've fully watched a VOD or manually set it to "mark as watched," it doesn't get marked as watched.
|
I fixed this a couple days ago, are you running the latest build of the |
Ah good to hear, not running latest build as I am in the middle of recording a subathon. But good to hear it's fixed! Thank you |
|
v3.0.0 has been released. If you run into any other issues please open a new issue. |
|
Awsome! If we were on the beta, are there any specific steps to swap back to the main build now that's on 3.0? Or just swap back from :dev to :latest ? Thanks |
In general just switching to :latest should work, however it depends on when you went into the beta (the beta underwent a few changes to the docker-compose and nginx.conf for example that you'd need to follow). |
|
Ah, so I have the beta working on my AMD based server fine.. But trying to get the v3 working on one of my arm64 devices is proving a no go. Am I missing something or has v3 just not been compiled for arm64 yet? |
|
Images are being built and published in amd64 and arm64. Can try deleting the docker image and pulling again?
Then can you check what architecture the image is? Last I check it's a "multi-architecture" image in that you can simply pull the |
I did remove and re-pull the images, they do show "Architecture": "arm64".. but still the same "exec /usr/local/bin/entrypoint.sh: exec format" in the API frustratingly. I also removed the ganymede:latest-arm64 and pulled this again, but I still get the temporal errors in the API logs.. Very odd indeed |
|
I noticed an issue with the Dockerfile causing arm image builds to build as amd64 for the last stage. Can you try pulling the latest The |
Tried with the dev tag and i get something similar still even after stopping the stack, removing the images and re-pulling: error: exec failed: exec format error
|
V3 Beta
Changes
Replace Temporal with a simpler task system: River.
Add support to customize the video and temp directories inside the container with
VIDEOS_DIRandTEMP_DIRenvironment variables (Environment variables for paths used #337)Category restrictions can now optionally be applied to livestreams.
Add a "Blocked VODs" page in the admin panel.
Ability to cancel a VOD archive in-progress.
Add a button to regenerate the static thumbnail for videos.
ffmpegto extract a screenshot at a random position for the full resolution thumbnail along with the web thumbnail.A lot of backend code refactoring.
Miscellaneous bug fixes
Known Issues
Stopping the server during a live stream will not fully recover (continue with post-video and chat convert tasks)Breaking Changes
config.jsonto environment variables (set these as environment variables on the api container)DEBUGOAUTH_ENABLEDVIDEOS_DIRTEMP_DIRCONFIG_DIRLOGS_DIRThe defaults of these are:
TEMP_DIRas a volume so temporary files are not lost on container restart. See the below docker-compose diff for an example.Notable Changes
Changes that aren't breaking but are notable.
ganymede-nginxcontainer isn't required anymore. The API container will serve the static files at theVIDEO_DIRenv var route. Example:VIDEO_DIR=/vodsthen it would be available atlocalhost:4800/vods. There is no index so you can't browse files. Updating the frontendCDN_URLtolocalhost:4800/vodsis the same as serving the files via Nginx.Upgrading to the Beta
Important
BACK UP YOUR CURRENT INSTANCE, INCLUDING THE DATABASE!!!
The upgrade is non-reversible. Be sure to create a backup of the database if something goes wrong. Rely on your standard backup procedure or create a database dump by running the following command.
Upgrade Steps
Wait until you have no active archives. There is no archive comparability between versions!
The v3 beta image is tagged as
:devfor both the API and frontend. These are built frequently as I push changes. Be sure to periodically update!docker compose downdocker-compose.ymlfiledocker-compose.ymloutlined in the breaking changes section above. This includes...VIDEOS_DIRandTEMP_DIRenvironment variable. I recommend mounting theTEMP_DIRto a volume on your host. This is to prevent losing data if the container crashes or restarts. Any modification to these variables requires changing the volume mounts as well.:devversion: "3.3" services: ganymede-api: container_name: ganymede-api - image: ghcr.io/zibbp/ganymede:latest + image: ghcr.io/zibbp/ganymede:dev restart: unless-stopped depends_on: - - ganymede-temporal + - ganymede-db environment: - TZ=America/Chicago # Set to your timezone - DB_HOST=ganymede-db - DB_PORT=5432 - DB_USER=ganymede - DB_PASS=PASSWORD - DB_NAME=ganymede-prd - DB_SSL=disable - JWT_SECRET=SECRET - JWT_REFRESH_SECRET=SECRET - TWITCH_CLIENT_ID= - TWITCH_CLIENT_SECRET= - FRONTEND_HOST=http://IP:PORT # OPTIONAL + # - OAUTH_ENABLED=false # - OAUTH_PROVIDER_URL= # - OAUTH_CLIENT_ID= # - OAUTH_CLIENT_SECRET= # - OAUTH_REDIRECT_URL=http://IP:PORT/api/v1/auth/oauth/callback # Points to the API service - - TEMPORAL_URL=ganymede-temporal:7233 # WORKER - MAX_CHAT_DOWNLOAD_EXECUTIONS=5 - MAX_CHAT_RENDER_EXECUTIONS=3 - MAX_VIDEO_DOWNLOAD_EXECUTIONS=5 - MAX_VIDEO_CONVERT_EXECUTIONS=3 volumes: - - /mnt/nas/vods:/vods + - /mnt/nas/vods:/data/videos - - ./logs:/logs + - ./logs:/data/logs + - ./temp:/data/temp + - ./config:/data/config # be sure to move `config.json` if needed ports: - 4800:4000 ganymede-frontend: container_name: ganymede-frontend - image: ghcr.io/zibbp/ganymede-frontend:latest + image: ghcr.io/zibbp/ganymede-frontend:dev restart: unless-stopped environment: - API_URL=http://IP:PORT # Points to the API service - CDN_URL=http://IP:PORT # Points to the CDN service - SHOW_SSO_LOGIN_BUTTON=true # show/hide SSO login button on login page - FORCE_SSO_AUTH=false # force SSO auth for all users (bypasses login page and redirects to SSO) - REQUIRE_LOGIN=false # require login to view videos ports: - 4801:3000 - ganymede-temporal: - image: temporalio/auto-setup:1.23 - container_name: ganymede-temporal - depends_on: - - ganymede-db - environment: - - DB=postgres12 # this tells temporal to use postgres (not the db name) - - DB_PORT=5432 - - POSTGRES_USER=ganymede - - POSTGRES_PWD=PASSWORD - - POSTGRES_SEEDS=ganymede-db # name of the db service - ports: - - 7233:7233 - # -- Uncomment below to enable temporal web ui -- - # ganymede-temporal-ui: - # image: temporalio/ui:latest - # container_name: ganymede-temporal-ui - # depends_on: - # - ganymede-temporal - # environment: - # - TEMPORAL_ADDRESS=ganymede-temporal:7233 - # ports: - # - 8233:8080 ganymede-db: container_name: ganymede-db image: postgres:14 volumes: - ./ganymede-db:/var/lib/postgresql/data environment: - POSTGRES_PASSWORD=PASSWORD - POSTGRES_USER=ganymede - POSTGRES_DB=ganymede-prd ports: - 4803:5432 ganymede-nginx: # this isn't necessary if you do not want to serve static files via Nginx. The API container will serve the `VIDEO_DIR` environment variable. container_name: ganymede-nginx image: nginx volumes: - /path/to/nginx.conf:/etc/nginx/nginx.conf:ro - /pah/to/vod/stoage:/mnt/vods ports: - 4802:8080 + ganymede-river-ui: + image: ghcr.io/riverqueue/riverui:0.3 + environment: + - DATABASE_URL=postgres://ganymede:DB_PASSWORD@ganymede-db:5432/ganymede-prd # update with env settings from the ganymede-db container + ports: + - 4804:8080rm -rf temporal(if you still have it)docker compose up -ddocker logs ganymede-api -fQuestions / Issues
Please post any questions or issues you run into!
The text was updated successfully, but these errors were encountered: