Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[🚀 Feature]: Optimize video recorder containers (Phase 1) #2458

Open
VietND96 opened this issue Nov 5, 2024 Discussed in #2452 · 0 comments · Fixed by #2468
Open

[🚀 Feature]: Optimize video recorder containers (Phase 1) #2458

VietND96 opened this issue Nov 5, 2024 Discussed in #2452 · 0 comments · Fixed by #2468
Milestone
4.27.0

Comments

@VietND96
Copy link
Member

VietND96 commented Nov 5, 2024 •
edited
Loading

Phase 1

Build a static FFmpeg binary (~12MB) for recording with RCLONE binary (~27MB) for uploading.
Bring all scripts for functionalities and program binaries to Node base.
Node/Standalone has the capability to record/upload itself.

Discussed in #2452

Originally posted by saidoyk November 1, 2024
I have a Selenium Grid setup with 20 Chrome containers and 20 video recording containers monitoring each one. This setup consumes 80GB of memory. Do I need 20 video containers? Can a single video container monitor multiple Chrome pods? Also, my docker-compose file is too long may I simply it easily or do I need a scalable automation script?

version: '3'
services:
  jaeger:
    image: jaegertracing/all-in-one:1.60
    ports:
      - "16686:16686"
      - "4317:4317"
  chrome_01:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6900:5900"
  chrome_02:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6901:5900"
  chrome_03:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6902:5900"
  chrome_04:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6903:5900"
  chrome_05:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6904:5900"
  chrome_06:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6905:5900"
  chrome_07:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6906:5900"
  chrome_08:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6907:5900"
  chrome_09:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6908:5900"
  chrome_10:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6909:5900"
  chrome_11:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6910:5900"
  chrome_12:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6912:5900"
  chrome_13:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6913:5900"
  chrome_14:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6914:5900"
  chrome_15:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6915:5900"
  chrome_16:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6916:5900"
  chrome_17:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6917:5900"
  chrome_18:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6918:5900"
  chrome_19:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6919:5900"
  chrome_20:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6920:5900"


  chrome_01_video:
    image: selenium/video:latest
     
    depends_on:
      - chrome_01
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_01
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto

  chrome_02_video:
    image: selenium/video:latest
     
    depends_on:
      - chrome_02
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_02
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
       
  chrome_03_video:
    image: selenium/video:latest
    depends_on:
      - chrome_03
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_03
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_04_video:
    image: selenium/video:latest
    depends_on:
      - chrome_04
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_04
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_05_video:
    image: selenium/video:latest
    depends_on:
      - chrome_05
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_05
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_06_video:
    image: selenium/video:latest
    depends_on:
      - chrome_06
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_06
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_07_video:
    image: selenium/video:latest
    depends_on:
      - chrome_07
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_07
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_08_video:
    image: selenium/video:latest
    depends_on:
      - chrome_08
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_08
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_09_video:
    image: selenium/video:latest
    depends_on:
      - chrome_09
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_09
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_10_video:
    image: selenium/video:latest
    depends_on:
      - chrome_10
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_10
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_11_video:
    image: selenium/video:latest
    depends_on:
      - chrome_11
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_11
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_12_video:
    image: selenium/video:latest
    depends_on:
      - chrome_12
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_12
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_13_video:
    image: selenium/video:latest
    depends_on:
      - chrome_13
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_13
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_14_video:
    image: selenium/video:latest
    depends_on:
      - chrome_14
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_14
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_15_video:
    image: selenium/video:latest
    depends_on:
      - chrome_15
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_15
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_16_video:
    image: selenium/video:latest
    depends_on:
      - chrome_16
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_16
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_17_video:
    image: selenium/video:latest
    depends_on:
      - chrome_17
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_17
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_18_video:
    image: selenium/video:latest
    depends_on:
      - chrome_18
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_18
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_19_video:
    image: selenium/video:latest
    depends_on:
      - chrome_19
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_19
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_20_video:
    image: selenium/video:latest
    depends_on:
      - chrome_20
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_20
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto

  selenium-hub:
    image: selenium/hub:latest
    
      
      
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"
    depends_on:
      - jaeger
    environment:
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317


```</div>
@VietND96 VietND96 added this to the 4.27.0 milestone Nov 5, 2024
@VietND96 VietND96 mentioned this issue Nov 15, 2024
Open
@VietND96 VietND96 changed the title [🚀 Feature]: Optimize video recorder containers [🚀 Feature]: Optimize video recorder containers (Phase 1) Nov 18, 2024
@VietND96 VietND96 linked a pull request Dec 12, 2024 that will close this issue
Build static FFmpeg and copy binary to Node base #2468
Merged
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
4.27.0
Development

Successfully merging a pull request may close this issue.

Build static FFmpeg and copy binary to Node base SeleniumHQ/docker-selenium
1 participant
@VietND96