zgaetano
cf1fe136d0
YouTube imports were stuck in `ingesting` forever: after the capability-
routing split (fdec2e3), every worker's WORKER_QUEUES enumerated an explicit
list that omitted `import`, so index.js's `want('import')` was false on every
worker and nothing consumed the import queue. Add `import` to worker-p4
(heavy/primary worker; import is concurrency-1 and network-bound, one consumer
is enough).
Second defect that would have surfaced once routed: the baked yt-dlp came from
the distro package (Ubuntu 22.04 apt = 2022.04.08, ~4 years stale), which
current YouTube rejects. Pull the latest self-contained yt-dlp release binary
at image build instead (GPU image: yt-dlp_linux; alpine image: python zipapp).
Rebuild the worker image to refresh.
Verified on zampp1: import jobs drain ingesting -> processing -> ready,
failed=0, yt-dlp 2026.03.17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
33 lines
1.5 KiB
Text
33 lines
1.5 KiB
Text
# GPU-accelerated worker — requires NVIDIA Container Toolkit on the host
|
|
# Build: docker compose -f docker-compose.yml -f docker-compose.gpu.yml build worker
|
|
#
|
|
# Prerequisites:
|
|
# - NVIDIA drivers installed on host
|
|
# - NVIDIA Container Toolkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/
|
|
# - GPU capability in docker-compose.gpu.yml enables NVENC access
|
|
|
|
FROM nvcr.io/nvidia/cuda:12.3.1-base-ubuntu22.04
|
|
|
|
# Install Node.js 20, ffmpeg (Ubuntu's ffmpeg includes h264_nvenc/hevc_nvenc),
|
|
# and yt-dlp for the YouTube importer.
|
|
#
|
|
# yt-dlp is NOT installed from apt: Ubuntu 22.04's package is pinned to a 2022
|
|
# release, which YouTube has long since broken (extraction fails). yt-dlp must
|
|
# track YouTube's frequent changes, so we pull the latest self-contained
|
|
# release binary at build time. /usr/local/bin precedes /usr/bin on PATH, so
|
|
# `yt-dlp` resolves to this one. Rebuild the worker image to refresh it.
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
curl ca-certificates ffmpeg python3 \
|
|
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
|
|
&& apt-get install -y --no-install-recommends nodejs \
|
|
&& curl -fsSL https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux \
|
|
-o /usr/local/bin/yt-dlp \
|
|
&& chmod a+rx /usr/local/bin/yt-dlp \
|
|
&& apt-get clean && rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /app
|
|
COPY package*.json ./
|
|
RUN npm install --omit=dev
|
|
COPY src ./src
|
|
|
|
CMD ["node", "src/index.js"]
|