Compare commits
2 commits
674dccca4e
...
9ad88e4df4
| Author | SHA1 | Date | |
|---|---|---|---|
| 9ad88e4df4 | |||
| 7a2710dc9a |
13 changed files with 1277 additions and 226 deletions
272
docs/superpowers/specs/2026-05-23-youtube-importer-design.md
Normal file
272
docs/superpowers/specs/2026-05-23-youtube-importer-design.md
Normal file
|
|
@ -0,0 +1,272 @@
|
|||
# YouTube Importer — Design Spec
|
||||
|
||||
> Status: **design approved 2026-05-23**, awaiting user review of the spec before the implementation plan is written.
|
||||
|
||||
## Context
|
||||
|
||||
The Ingest group in Dragonflight today covers file Upload, Recorders (SRT/RTMP/SDI), Capture (DeckLink), Monitors, and Schedule. There is no path to bring in media that already lives on the public web. The frequent ask is: "I want to grab a YouTube link and have it become an asset in my project, with the same proxy/thumbnail pipeline as anything else." This spec adds a YouTube importer that mirrors the existing upload flow: paste a URL, pick a project, click Import, and the asset shows up in the Library once the worker is done.
|
||||
|
||||
The importer rides on the existing job pipeline. After the download lands in S3, the asset re-enters the same proxy → thumbnail → ready path as a regular upload, so there is no parallel "imported asset" lifecycle to maintain.
|
||||
|
||||
## Goals & non-goals
|
||||
|
||||
**Goals**
|
||||
- Paste a public YouTube URL, end up with a `ready` asset in the chosen project.
|
||||
- Reuse the existing `assets` table, S3 layout, BullMQ pipeline, and Jobs screen — no parallel state machine.
|
||||
- Progress visible from both the import screen (queue rows) and the Jobs screen.
|
||||
- Clear, actionable errors for the obvious failure modes (private, age-gated, removed, geo-blocked, network).
|
||||
|
||||
**Non-goals**
|
||||
- Playlists, channels, or batch-paste of multiple URLs. Single URL per submission. (Easy to add later.)
|
||||
- Cookies / login. Private, members-only, and age-gated videos are out of scope v1.
|
||||
- Quality picker. Always grabs best MP4 (with M4A audio merge fallback).
|
||||
- Non-YouTube sources (Vimeo, Twitch VODs, Dropbox links, etc.). The route is `/imports/youtube` precisely to leave room for siblings later.
|
||||
- Auto-update of yt-dlp inside the running container. Updates land via image rebuild.
|
||||
- Copyright enforcement. We surface a one-line "only import videos you have rights to use" note and stop there.
|
||||
|
||||
## Architecture
|
||||
|
||||
The importer threads through four existing layers:
|
||||
|
||||
```
|
||||
[web-ui] YouTube screen ──POST /imports/youtube──▶ [mam-api]
|
||||
│
|
||||
assets row (status='ingesting')
|
||||
jobs row (type='youtube_import')
|
||||
│
|
||||
BullMQ "import" queue
|
||||
▼
|
||||
[worker]
|
||||
yt-dlp download → S3 originals/
|
||||
ffprobe metadata → assets row
|
||||
status='processing'
|
||||
│
|
||||
BullMQ "proxy" queue ◀── existing path
|
||||
▼
|
||||
proxy → thumbnail → ready
|
||||
```
|
||||
|
||||
Once the worker hands off to the `proxy` queue, the asset is indistinguishable from one that came through Upload — same proxy worker, same thumbnail worker, same Library list.
|
||||
|
||||
## 1. UX
|
||||
|
||||
### Nav
|
||||
|
||||
A 6th child is added to the **Ingest** group in `shell.jsx`, between Upload and Recorders:
|
||||
|
||||
```js
|
||||
{ id: "youtube", label: "YouTube", icon: "download" },
|
||||
```
|
||||
|
||||
The `download` glyph already exists in `icons.jsx`. The matching `ingestChildren` array in `shell.jsx` and the crumbs map in `app.jsx` both gain `"youtube"`.
|
||||
|
||||
### Screen
|
||||
|
||||
A new `YouTubeImport` component lives in `screens-ingest.jsx` and is exported on `window` alongside `Upload`, `Recorders`, etc. It is registered as a route in `app.jsx`.
|
||||
|
||||
Layout — visually a sibling of the Upload screen:
|
||||
|
||||
- **Header**: title "YouTube", subtitle "Paste a link — we download and import the best available MP4."
|
||||
- **Project selector**: same `select` element as Upload's, pre-selected to the first project.
|
||||
- **URL input**: a single-line `field-input` with placeholder "Paste a YouTube URL (youtube.com/watch, youtu.be, or shorts)…" and an inline **Import** button. Enter submits. The button is disabled until a URL pattern matches.
|
||||
- **Subtitle line under the input**: "Only import videos you have rights to use. Private, age-gated, and members-only videos are not supported."
|
||||
- **Queue panel**: identical structure to Upload's queue — one row per submitted URL, showing:
|
||||
- Source icon (use `link` glyph) and the URL (truncated middle, full URL in `title` tooltip).
|
||||
- Title once known (filled in by a poll on the asset row).
|
||||
- Progress bar tied to job `progress` (0–100). The worker drives this between 5 and 60 % for download and 60 to 100 % for upload + DB writes.
|
||||
- Status pill: queued → downloading → processing → done / failed.
|
||||
- Error text if the job fails (red, one line).
|
||||
- A "Clear done" button at the top of the queue.
|
||||
|
||||
The queue persists for the session in component state only — no separate UI table. Jobs screen remains the canonical history.
|
||||
|
||||
### URL validation (client-side, before POST)
|
||||
|
||||
Accept (case-insensitive) any of these patterns:
|
||||
- `https?://(www\.|m\.)?youtube\.com/watch\?[^ ]*v=[A-Za-z0-9_-]{11}`
|
||||
- `https?://youtu\.be/[A-Za-z0-9_-]{11}`
|
||||
- `https?://(www\.)?youtube\.com/shorts/[A-Za-z0-9_-]{11}`
|
||||
|
||||
Anything else is rejected inline ("That doesn't look like a YouTube URL") without an API call. The server re-validates as a defense-in-depth check.
|
||||
|
||||
### Out-of-scope v1 (called out, not built)
|
||||
|
||||
- Pasting a playlist URL. Server returns 400 "Playlists aren't supported yet."
|
||||
- Multi-line paste. Single URL only.
|
||||
- Quality picker. yt-dlp format string is hard-coded.
|
||||
- Cookies upload. Private videos fail with a clear message.
|
||||
|
||||
## 2. API
|
||||
|
||||
### Route
|
||||
|
||||
New file `services/mam-api/src/routes/imports.js`, mounted at `/api/v1/imports` in `services/mam-api/src/index.js`.
|
||||
|
||||
**`POST /api/v1/imports/youtube`**
|
||||
|
||||
Request body:
|
||||
```json
|
||||
{ "url": "https://youtu.be/dQw4w9WgXcQ", "projectId": "uuid", "binId": "uuid?" }
|
||||
```
|
||||
|
||||
Behavior:
|
||||
1. Validate `url` against the same three regexes as the client. 400 on miss.
|
||||
2. Reject playlist URLs (URL contains `list=`) with 400 "Playlists aren't supported yet."
|
||||
3. Generate `assetId = uuidv4()`.
|
||||
4. Insert into `assets` with:
|
||||
- `status='ingesting'`
|
||||
- `media_type='video'`
|
||||
- `filename = url` (placeholder; worker overwrites with the sanitized title once yt-dlp prints metadata — keeps the row queryable in the meantime)
|
||||
- `display_name = url` (same; worker overwrites)
|
||||
- `original_s3_key = NULL` (worker fills in)
|
||||
- `source_url = url` (new column — see Schema)
|
||||
- `project_id`, `bin_id`, timestamps.
|
||||
5. Insert into `jobs` with `type='youtube_import'`, `asset_id`, `payload={ url }`, `status='queued'`, `progress=0`.
|
||||
6. Enqueue BullMQ job on the `import` queue:
|
||||
```js
|
||||
await importQueue.add('youtube', { assetId, url });
|
||||
```
|
||||
7. Respond `200 { assetId, jobId }`.
|
||||
|
||||
Errors:
|
||||
- Missing fields → 400.
|
||||
- Bad URL → 400 with `error: 'Invalid YouTube URL'`.
|
||||
- Playlist URL → 400 with `error: 'Playlists aren't supported yet'`.
|
||||
- Project not found → 404.
|
||||
- DB / queue failure → 500 (next(err)).
|
||||
|
||||
### Jobs screen integration
|
||||
|
||||
`services/web-ui/public/screens-jobs.jsx` already normalizes job types via a `kindMap`. Add one entry:
|
||||
```js
|
||||
const kindMap = { proxy: 'Proxy', thumbnail: 'Thumbnail', conform: 'Conform', transcode: 'Transcode', youtube_import: 'YouTube' };
|
||||
```
|
||||
Retry, delete, and the SSE event stream all work for the new type with no further changes because they key off `job.id`, not `job.type`.
|
||||
|
||||
## 3. Worker
|
||||
|
||||
### Container changes
|
||||
|
||||
`services/worker/Dockerfile` gains two packages:
|
||||
```dockerfile
|
||||
RUN apk add --no-cache ffmpeg yt-dlp python3
|
||||
```
|
||||
`yt-dlp` is in the Alpine `community` repo and pulls `python3` as a runtime dep — we list it explicitly for clarity. Image grows by ~25 MB.
|
||||
|
||||
### New worker
|
||||
|
||||
`services/worker/src/workers/youtube-import.js`, registered in `services/worker/src/index.js`:
|
||||
```js
|
||||
const workers = [
|
||||
createWorker('proxy', proxyWorker),
|
||||
createWorker('thumbnail', thumbnailWorker),
|
||||
createWorker('conform', conformWorker),
|
||||
createWorker('import', youtubeImportWorker),
|
||||
];
|
||||
```
|
||||
|
||||
### Job handler
|
||||
|
||||
For a job with `{ assetId, url }`:
|
||||
|
||||
1. `job.updateProgress(2)` — accepted.
|
||||
2. Build a temp directory `tmpdir()/yt-${jobId}`.
|
||||
3. Run yt-dlp:
|
||||
```sh
|
||||
yt-dlp \
|
||||
--no-playlist \
|
||||
--no-warnings \
|
||||
--restrict-filenames \
|
||||
-f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]/b" \
|
||||
--merge-output-format mp4 \
|
||||
--print-json \
|
||||
--newline \
|
||||
-o "<tmpdir>/<assetId>.%(ext)s" \
|
||||
"<url>"
|
||||
```
|
||||
- `--print-json` writes one JSON line at the end with title, duration, width, height, uploader, etc.
|
||||
- `--newline` makes progress lines newline-terminated so we can parse them.
|
||||
- `--restrict-filenames` prevents shell-special characters in temp paths.
|
||||
4. Stream stdout line-by-line. Lines matching `r'\[download\]\s+(\d+(\.\d+)?)%'` map to `job.updateProgress(5 + Math.floor(pct * 0.55))` so download takes us from 5 to 60 %.
|
||||
5. On yt-dlp non-zero exit: parse stderr for the first line containing `ERROR:` and use it as the job's error message. Mark the asset `status='error'`, mark the job failed, throw so BullMQ records it. Surface a friendly substitution for the common cases:
|
||||
- "Private video" → "Private video — not supported."
|
||||
- "Sign in to confirm your age" → "Age-restricted video — not supported."
|
||||
- "Video unavailable" → "Video unavailable or removed."
|
||||
- "This video is not available in your country" → "Video is geo-blocked from this region."
|
||||
- HTTP 429 → "YouTube rate-limited the importer — try again later."
|
||||
- Anything else → use yt-dlp's stderr line verbatim, truncated to 300 chars.
|
||||
6. Parse the last stdout line as JSON to read metadata. The resulting file is `<tmpdir>/<assetId>.mp4`.
|
||||
7. `getMediaInfo` (existing helper in `services/worker/src/ffmpeg/executor.js`) on that path. Use ffprobe's values for codec/fps/duration when yt-dlp's are missing or wrong.
|
||||
8. Sanitize the title for the S3 filename: keep `[A-Za-z0-9 ._-]`, collapse runs of whitespace, trim, cap at 120 chars, append `.mp4`. If the sanitized title is empty, fall back to `youtube-<videoId>.mp4`.
|
||||
9. Upload to `originals/{assetId}/{sanitized-title}.mp4` via the existing `uploadToS3` helper. Progress 60 → 90 %.
|
||||
10. UPDATE the assets row with:
|
||||
- `filename = <sanitized title>.mp4`
|
||||
- `display_name = <yt-dlp title untouched>`
|
||||
- `original_s3_key = originals/<assetId>/<sanitized-title>.mp4`
|
||||
- `codec`, `resolution`, `fps`, `duration_ms`, `file_size` from ffprobe.
|
||||
- `status = 'processing'`
|
||||
- `updated_at = NOW()`
|
||||
11. Enqueue a `proxy` job on the existing `proxy` queue with the same payload shape `upload.js` uses:
|
||||
```js
|
||||
await proxyQueue.add('generate', {
|
||||
assetId,
|
||||
inputKey: asset.original_s3_key,
|
||||
outputKey: `proxies/${assetId}.mp4`,
|
||||
});
|
||||
```
|
||||
12. `job.updateProgress(100)`. Return — BullMQ marks the import job done. The proxy job picks up the rest exactly like a regular upload.
|
||||
13. Always `rm -rf` the temp directory in a `finally`.
|
||||
|
||||
### Concurrency & retries
|
||||
|
||||
- Default BullMQ concurrency for this queue: **1** per worker process. Two simultaneous yt-dlp invocations risk YouTube rate-limiting more than they help throughput. Configurable later via env if needed.
|
||||
- No automatic BullMQ retry — yt-dlp failures are almost always permanent (private, geo, removed) and a silent retry storm would chew through quota. The Jobs screen's manual Retry button is the right knob for "this should be transient" cases.
|
||||
|
||||
## 4. Schema migration
|
||||
|
||||
New file `services/mam-api/src/db/migrations/011-youtube-import.sql`:
|
||||
|
||||
```sql
|
||||
-- 1. Add the new job type to the enum.
|
||||
-- Postgres requires ALTER TYPE ... ADD VALUE for enum changes.
|
||||
ALTER TYPE job_type ADD VALUE IF NOT EXISTS 'youtube_import';
|
||||
|
||||
-- 2. Remember where an asset came from. NULL for everything that
|
||||
-- pre-dates the importer; populated for any imported asset.
|
||||
ALTER TABLE assets ADD COLUMN IF NOT EXISTS source_url TEXT;
|
||||
```
|
||||
|
||||
`source_url` is exposed on the asset drawer as a "Source" line ("imported from youtu.be/…") in a follow-up PR — out of scope for this spec, but worth noting that the column exists for it.
|
||||
|
||||
## 5. Files touched
|
||||
|
||||
**New**
|
||||
- `services/mam-api/src/routes/imports.js`
|
||||
- `services/mam-api/src/db/migrations/011-youtube-import.sql`
|
||||
- `services/worker/src/workers/youtube-import.js`
|
||||
|
||||
**Edited**
|
||||
- `services/mam-api/src/index.js` — mount the new route.
|
||||
- `services/web-ui/public/screens-ingest.jsx` — add `YouTubeImport`, export on `window`.
|
||||
- `services/web-ui/public/shell.jsx` — add the nav child, extend `ingestChildren`.
|
||||
- `services/web-ui/public/app.jsx` — register the route and the crumb.
|
||||
- `services/web-ui/public/screens-jobs.jsx` — extend `kindMap` with `youtube_import: 'YouTube'`.
|
||||
- `services/worker/src/index.js` — register the `import` queue worker.
|
||||
- `services/worker/Dockerfile` — add `yt-dlp` and `python3` to the apk install line.
|
||||
|
||||
## 6. Risks & trade-offs
|
||||
|
||||
- **Worker egress**. The worker container needs outbound HTTPS to YouTube. Fine in the current homelab; will fail in a locked-down cluster. Documented in the implementation plan.
|
||||
- **yt-dlp drift**. YouTube changes break old yt-dlp versions every few weeks. The Alpine package lags upstream by days. Fix is to rebuild the worker image. We do not auto-update inside the running container — too risky for an offline / locked-down deploy. If imports start failing en masse, the runbook is `docker compose build worker && docker compose up -d worker`.
|
||||
- **Single-URL UX feels light**. That is deliberate for v1. Adding multi-URL paste and playlist expansion are both small follow-ups once the single-URL path is stable.
|
||||
- **No copyright enforcement**. We rely on the one-line notice in the UI. If misuse becomes a real concern, the next step would be an admin allowlist of domains or a per-user import quota — not in this spec.
|
||||
- **`filename = url` placeholder**. Briefly, the asset row in the Library shows the URL as the name. The worker overwrites it within seconds for a successful import. Acceptable; the Library already handles "ingesting" assets with placeholder names from the upload path.
|
||||
|
||||
## 7. Acceptance
|
||||
|
||||
The feature is done when:
|
||||
- A user can navigate to **Ingest → YouTube**, paste a public YouTube URL, pick a project, click Import, and within a minute or two see the asset appear in the Library with proxy and thumbnail.
|
||||
- A failed import (private video, removed video, bogus URL) shows a clear error message on both the YouTube screen's queue row and the Jobs screen.
|
||||
- The Jobs screen lists "YouTube" jobs alongside Proxy / Thumbnail / Conform, with Retry working.
|
||||
- `source_url` is populated on the imported asset row.
|
||||
- Image rebuild + `docker compose up -d worker` is the documented recovery path if YouTube changes break yt-dlp.
|
||||
15
services/mam-api/src/db/migrations/011-youtube-import.sql
Normal file
15
services/mam-api/src/db/migrations/011-youtube-import.sql
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
-- 2026-05: YouTube importer — new job type + remember source URL on assets.
|
||||
--
|
||||
-- Job type enum gains 'youtube_import' so the Jobs screen can show imports
|
||||
-- alongside proxy / thumbnail / conform. Assets gain source_url so an
|
||||
-- imported asset remembers where it came from (used by the Asset Detail
|
||||
-- page and, later, dedup checks).
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_enum WHERE enumlabel = 'youtube_import' AND enumtypid = 'job_type'::regtype) THEN
|
||||
ALTER TYPE job_type ADD VALUE 'youtube_import';
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
ALTER TABLE assets ADD COLUMN IF NOT EXISTS source_url TEXT;
|
||||
|
|
@ -30,6 +30,7 @@ import sdkRouter from './routes/sdk.js';
|
|||
import schedulesRouter from './routes/schedules.js';
|
||||
import metricsRouter from './routes/metrics.js';
|
||||
import commentsRouter from './routes/comments.js';
|
||||
import importsRouter from './routes/imports.js';
|
||||
import { startSchedulerLoop } from './scheduler.js';
|
||||
|
||||
const app = express();
|
||||
|
|
@ -83,6 +84,7 @@ app.use('/api/v1/sdk', sdkRouter);
|
|||
app.use('/api/v1/schedules', schedulesRouter);
|
||||
app.use('/api/v1/metrics', metricsRouter);
|
||||
app.use('/api/v1/assets/:assetId/comments', commentsRouter);
|
||||
app.use('/api/v1/imports', importsRouter);
|
||||
|
||||
// ── Error handler ─────────────────────────────────────────────────────────────
|
||||
app.use(errorHandler);
|
||||
|
|
|
|||
96
services/mam-api/src/routes/imports.js
Normal file
96
services/mam-api/src/routes/imports.js
Normal file
|
|
@ -0,0 +1,96 @@
|
|||
// External media imports — currently YouTube only.
|
||||
//
|
||||
// The flow mirrors upload.js: create the asset row up front with a placeholder
|
||||
// filename (the worker fills in the real title once yt-dlp prints metadata),
|
||||
// then enqueue a BullMQ job. The worker downloads, lands the file in S3 at the
|
||||
// same originals/{assetId}/... path uploads use, and hands off to the existing
|
||||
// proxy queue — so an imported asset travels the same lifecycle as any upload.
|
||||
|
||||
import express from 'express';
|
||||
import { Queue } from 'bullmq';
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import pool from '../db/pool.js';
|
||||
import { requireAuth } from '../middleware/auth.js';
|
||||
|
||||
const router = express.Router();
|
||||
router.use(requireAuth);
|
||||
|
||||
const parseRedisUrl = (url) => {
|
||||
try {
|
||||
const parsed = new URL(url);
|
||||
return { host: parsed.hostname, port: parseInt(parsed.port, 10) || 6379 };
|
||||
} catch {
|
||||
return { host: 'localhost', port: 6379 };
|
||||
}
|
||||
};
|
||||
|
||||
const importQueue = new Queue('import', {
|
||||
connection: parseRedisUrl(process.env.REDIS_URL || 'redis://queue:6379'),
|
||||
});
|
||||
|
||||
// Match the same three forms the client UI validates against. Server is the
|
||||
// authoritative check — never trust the client to have validated.
|
||||
const YT_PATTERNS = [
|
||||
/^https?:\/\/(?:www\.|m\.)?youtube\.com\/watch\?[^ ]*v=[A-Za-z0-9_-]{11}/i,
|
||||
/^https?:\/\/youtu\.be\/[A-Za-z0-9_-]{11}/i,
|
||||
/^https?:\/\/(?:www\.)?youtube\.com\/shorts\/[A-Za-z0-9_-]{11}/i,
|
||||
];
|
||||
|
||||
function isYouTubeUrl(url) {
|
||||
return typeof url === 'string' && YT_PATTERNS.some((re) => re.test(url));
|
||||
}
|
||||
|
||||
// POST /api/v1/imports/youtube — body { url, projectId, binId? }
|
||||
router.post('/youtube', async (req, res, next) => {
|
||||
try {
|
||||
const { url, projectId, binId } = req.body || {};
|
||||
|
||||
if (!url || !projectId) {
|
||||
return res.status(400).json({ error: 'url and projectId are required' });
|
||||
}
|
||||
if (!isYouTubeUrl(url)) {
|
||||
return res.status(400).json({ error: 'Invalid YouTube URL' });
|
||||
}
|
||||
// A playlist URL has `list=…` — yt-dlp's --no-playlist would still grab
|
||||
// the single video, but the operator probably meant "import the list" and
|
||||
// we don't support that yet. Reject so the intent is explicit.
|
||||
if (/[?&]list=/i.test(url)) {
|
||||
return res.status(400).json({ error: "Playlists aren't supported yet" });
|
||||
}
|
||||
|
||||
const projCheck = await pool.query('SELECT id FROM projects WHERE id = $1', [projectId]);
|
||||
if (projCheck.rows.length === 0) {
|
||||
return res.status(404).json({ error: 'Project not found' });
|
||||
}
|
||||
|
||||
const assetId = uuidv4();
|
||||
|
||||
// Placeholder filename/display_name — the worker overwrites both once
|
||||
// yt-dlp resolves the video title (usually within a second or two).
|
||||
await pool.query(
|
||||
`INSERT INTO assets (
|
||||
id, project_id, bin_id, filename, display_name, status,
|
||||
media_type, original_s3_key, source_url, created_at, updated_at
|
||||
)
|
||||
VALUES ($1, $2, $3, $4, $4, 'ingesting', 'video', NULL, $5, NOW(), NOW())`,
|
||||
[assetId, projectId, binId || null, url, url]
|
||||
);
|
||||
|
||||
const bullJob = await importQueue.add('youtube', {
|
||||
assetId,
|
||||
url,
|
||||
// Surface the URL in the Jobs screen until the worker fills in the title.
|
||||
assetName: url,
|
||||
});
|
||||
|
||||
res.status(202).json({
|
||||
assetId,
|
||||
jobId: `import:${bullJob.id}`,
|
||||
status: 'queued',
|
||||
});
|
||||
} catch (err) {
|
||||
next(err);
|
||||
}
|
||||
});
|
||||
|
||||
export default router;
|
||||
|
|
@ -21,11 +21,13 @@ const redisConn = parseRedisUrl(process.env.REDIS_URL || 'redis://queue:6379');
|
|||
const proxyQueue = new Queue('proxy', { connection: redisConn });
|
||||
const thumbnailQueue = new Queue('thumbnail', { connection: redisConn });
|
||||
const conformQueue = new Queue('conform', { connection: redisConn });
|
||||
const importQueue = new Queue('import', { connection: redisConn });
|
||||
|
||||
const QUEUES = [
|
||||
{ queue: proxyQueue, type: 'proxy' },
|
||||
{ queue: thumbnailQueue, type: 'thumbnail' },
|
||||
{ queue: conformQueue, type: 'conform' },
|
||||
{ queue: importQueue, type: 'import' },
|
||||
];
|
||||
|
||||
// BullMQ state → API status mapping
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@ function App() {
|
|||
library: ['Library'], projects: ['Projects'],
|
||||
upload: ['Ingest', 'Upload'], recorders: ['Ingest', 'Recorders'],
|
||||
schedule: ['Ingest', 'Schedule'],
|
||||
youtube: ['Ingest', 'YouTube'],
|
||||
capture: ['Ingest', 'Capture'], monitors: ['Ingest', 'Monitors'],
|
||||
jobs: ['Jobs'], editor: ['Editor'],
|
||||
users: ['Admin', 'Users & Groups'], tokens: ['Admin', 'Tokens'],
|
||||
|
|
@ -74,6 +75,7 @@ function App() {
|
|||
case 'upload': content = <Upload navigate={navigate} />; break;
|
||||
case 'recorders': content = <Recorders navigate={navigate} onNew={() => setShowNewRecorder(true)} />; break;
|
||||
case 'schedule': content = <Schedule navigate={navigate} />; break;
|
||||
case 'youtube': content = <YouTubeImport navigate={navigate} />; break;
|
||||
case 'capture': content = <Capture navigate={navigate} />; break;
|
||||
case 'monitors': content = <Monitors navigate={navigate} />; break;
|
||||
case 'jobs': content = <Jobs navigate={navigate} />; break;
|
||||
|
|
|
|||
|
|
@ -181,6 +181,198 @@ function Upload({ navigate }) {
|
|||
);
|
||||
}
|
||||
|
||||
/* ===== YouTube importer ===== */
|
||||
// Accept the same three URL shapes the API validates against.
|
||||
const _YT_PATTERNS = [
|
||||
/^https?:\/\/(?:www\.|m\.)?youtube\.com\/watch\?[^ ]*v=[A-Za-z0-9_-]{11}/i,
|
||||
/^https?:\/\/youtu\.be\/[A-Za-z0-9_-]{11}/i,
|
||||
/^https?:\/\/(?:www\.)?youtube\.com\/shorts\/[A-Za-z0-9_-]{11}/i,
|
||||
];
|
||||
function _looksLikeYouTube(s) {
|
||||
return typeof s === 'string' && _YT_PATTERNS.some(re => re.test(s.trim()));
|
||||
}
|
||||
|
||||
function YouTubeImport({ navigate }) {
|
||||
const PROJECTS = window.ZAMPP_DATA?.PROJECTS || [];
|
||||
const [projectId, setProjectId] = React.useState(PROJECTS[0]?.id || '');
|
||||
const [url, setUrl] = React.useState('');
|
||||
const [queue, setQueue] = React.useState([]); // { id, url, status, progress, title, error, assetId, jobId }
|
||||
const [submitting, setSubmitting] = React.useState(false);
|
||||
|
||||
const valid = _looksLikeYouTube(url);
|
||||
|
||||
const updateRow = React.useCallback((id, patch) => {
|
||||
setQueue(prev => prev.map(r => r.id === id ? { ...r, ...patch } : r));
|
||||
}, []);
|
||||
|
||||
// Poll the asset row to pick up the title once yt-dlp resolves it, and the
|
||||
// proxy job's progress so the queue row reflects the full lifecycle, not
|
||||
// just the import step.
|
||||
const pollRow = React.useCallback((row) => {
|
||||
if (!row.assetId) return;
|
||||
let stopped = false;
|
||||
const tick = async () => {
|
||||
if (stopped) return;
|
||||
try {
|
||||
const asset = await window.ZAMPP_API.fetch('/assets/' + row.assetId);
|
||||
const patch = {};
|
||||
if (asset.display_name && asset.display_name !== row.url) patch.title = asset.display_name;
|
||||
if (asset.status === 'ready') {
|
||||
patch.status = 'done';
|
||||
patch.progress = 100;
|
||||
} else if (asset.status === 'error') {
|
||||
patch.status = 'error';
|
||||
patch.error = patch.error || 'Import failed — check the Jobs screen for details.';
|
||||
} else if (asset.status === 'processing') {
|
||||
patch.status = 'processing';
|
||||
}
|
||||
if (Object.keys(patch).length) updateRow(row.id, patch);
|
||||
if (asset.status === 'ready' || asset.status === 'error') return;
|
||||
} catch { /* ignore */ }
|
||||
setTimeout(tick, 3000);
|
||||
};
|
||||
tick();
|
||||
return () => { stopped = true; };
|
||||
}, [updateRow]);
|
||||
|
||||
const submit = React.useCallback(async () => {
|
||||
if (!valid || !projectId || submitting) return;
|
||||
setSubmitting(true);
|
||||
const rowId = Date.now();
|
||||
const row = {
|
||||
id: rowId,
|
||||
url: url.trim(),
|
||||
status: 'queued',
|
||||
progress: 0,
|
||||
title: '',
|
||||
error: null,
|
||||
assetId: null,
|
||||
jobId: null,
|
||||
};
|
||||
setQueue(prev => [row, ...prev]);
|
||||
try {
|
||||
const res = await window.ZAMPP_API.fetch('/imports/youtube', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ url: row.url, projectId }),
|
||||
});
|
||||
updateRow(rowId, { assetId: res.assetId, jobId: res.jobId, status: 'downloading' });
|
||||
pollRow({ ...row, assetId: res.assetId, jobId: res.jobId });
|
||||
setUrl('');
|
||||
} catch (e) {
|
||||
updateRow(rowId, { status: 'error', error: e.message || 'Failed to start import' });
|
||||
} finally {
|
||||
setSubmitting(false);
|
||||
}
|
||||
}, [valid, projectId, submitting, url, updateRow, pollRow]);
|
||||
|
||||
return (
|
||||
<div className="page">
|
||||
<div className="page-header">
|
||||
<h1>YouTube</h1>
|
||||
<span className="subtitle">Paste a link — we download and import the best available MP4.</span>
|
||||
</div>
|
||||
<div className="page-body">
|
||||
<div style={{ display: 'grid', gridTemplateColumns: '1fr 1fr', gap: 12, marginBottom: 16 }}>
|
||||
<div>
|
||||
<label className="field-label">Project</label>
|
||||
<select className="field-input" value={projectId} onChange={e => setProjectId(e.target.value)}
|
||||
style={{ appearance: 'auto' }}>
|
||||
{PROJECTS.length === 0
|
||||
? <option value="">No projects</option>
|
||||
: PROJECTS.map(p => <option key={p.id} value={p.id}>{p.name}</option>)}
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="field" style={{ marginBottom: 8 }}>
|
||||
<label className="field-label">YouTube URL</label>
|
||||
<div style={{ display: 'flex', gap: 8 }}>
|
||||
<input
|
||||
className="field-input mono"
|
||||
value={url}
|
||||
onChange={e => setUrl(e.target.value)}
|
||||
onKeyDown={e => { if (e.key === 'Enter' && valid) submit(); }}
|
||||
placeholder="https://www.youtube.com/watch?v=… or https://youtu.be/…"
|
||||
style={{ flex: 1 }}
|
||||
autoFocus
|
||||
/>
|
||||
<button
|
||||
className="btn primary"
|
||||
onClick={submit}
|
||||
disabled={!valid || !projectId || submitting}
|
||||
title={!valid ? 'Paste a YouTube URL (youtube.com/watch, youtu.be, or shorts)' : ''}
|
||||
>
|
||||
<Icon name="download" />Import
|
||||
</button>
|
||||
</div>
|
||||
{url && !valid && (
|
||||
<div style={{ fontSize: 11.5, color: 'var(--danger)', marginTop: 4 }}>
|
||||
That doesn't look like a YouTube URL.
|
||||
</div>
|
||||
)}
|
||||
<div style={{ fontSize: 11.5, color: 'var(--text-3)', marginTop: 6 }}>
|
||||
Only import videos you have rights to use. Private, age-gated, and members-only videos are not supported.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{queue.length > 0 && (
|
||||
<div style={{ marginTop: 20 }}>
|
||||
<div style={{ fontSize: 13, fontWeight: 600, marginBottom: 12, display: 'flex', alignItems: 'center', gap: 8 }}>
|
||||
Queue <span className="badge neutral">{queue.length}</span>
|
||||
<span style={{ flex: 1 }} />
|
||||
<button className="btn ghost sm" onClick={() => setQueue(q => q.filter(r => r.status !== 'done' && r.status !== 'error'))}>
|
||||
Clear finished
|
||||
</button>
|
||||
</div>
|
||||
<div className="panel">
|
||||
{queue.map(r => {
|
||||
const statusColor =
|
||||
r.status === 'done' ? 'var(--success)' :
|
||||
r.status === 'error' ? 'var(--danger)' : 'var(--text-3)';
|
||||
return (
|
||||
<div key={r.id} className="upload-row">
|
||||
<Icon name="link" size={16} style={{ color: 'var(--text-3)' }} />
|
||||
<div style={{ flex: 1, minWidth: 0 }}>
|
||||
<div style={{ display: 'flex', gap: 8, alignItems: 'center' }}>
|
||||
<span style={{ fontWeight: 500, fontSize: 12.5, overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }}>
|
||||
{r.title || r.url}
|
||||
</span>
|
||||
</div>
|
||||
{r.title && (
|
||||
<div className="muted mono" style={{ fontSize: 10.5, marginTop: 2, overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }} title={r.url}>
|
||||
{r.url}
|
||||
</div>
|
||||
)}
|
||||
<div style={{ marginTop: 6, height: 4, background: 'var(--bg-3)', borderRadius: 99, overflow: 'hidden' }}>
|
||||
<div style={{
|
||||
width: (r.status === 'done' ? 100 : r.progress) + '%',
|
||||
height: '100%',
|
||||
background: r.status === 'done' ? 'var(--success)' : r.status === 'error' ? 'var(--danger)' : 'var(--accent)',
|
||||
transition: 'width 200ms',
|
||||
}} />
|
||||
</div>
|
||||
{r.error && (
|
||||
<div style={{ marginTop: 3, fontSize: 11, color: 'var(--danger)' }}>{r.error}</div>
|
||||
)}
|
||||
</div>
|
||||
<span className="mono" style={{ fontSize: 11.5, minWidth: 88, textAlign: 'right', color: statusColor }}>
|
||||
{r.status === 'done' ? '✓ done'
|
||||
: r.status === 'error' ? '✗ failed'
|
||||
: r.status === 'processing'? 'processing'
|
||||
: r.status === 'downloading' ? 'downloading'
|
||||
: 'queued'}
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
/* ===== Live preview (HLS) ====================================
|
||||
Shared by RecorderRow + MonitorTile. The capture container writes
|
||||
HLS segments to /live/{assetId}/index.m3u8 (see capture-manager.js
|
||||
|
|
@ -669,109 +861,281 @@ function _durationMin(startISO, endISO) {
|
|||
return Math.round((new Date(endISO) - new Date(startISO)) / 60000);
|
||||
}
|
||||
|
||||
// ── Calendar helpers ─────────────────────────────────────────────────────────
|
||||
function _ymd(d) {
|
||||
// Local-zone yyyy-mm-dd key for grouping events into day cells.
|
||||
const y = d.getFullYear();
|
||||
const m = String(d.getMonth() + 1).padStart(2, '0');
|
||||
const day = String(d.getDate()).padStart(2, '0');
|
||||
return y + '-' + m + '-' + day;
|
||||
// ── EPG (timeline) helpers ───────────────────────────────────────────────────
|
||||
//
|
||||
// The Schedule screen is a broadcast-control-room timeline: recorders are
|
||||
// rows, time is the horizontal axis. Helpers below convert between Date and
|
||||
// "minutes into local day" so we can position absolute-positioned event
|
||||
// blocks against a fixed --epg-pph (pixels-per-hour) CSS variable.
|
||||
//
|
||||
function _dayStart(d) {
|
||||
const x = new Date(d);
|
||||
x.setHours(0, 0, 0, 0);
|
||||
return x;
|
||||
}
|
||||
function _startOfMonth(d) { return new Date(d.getFullYear(), d.getMonth(), 1); }
|
||||
function _addMonths(d, n) { return new Date(d.getFullYear(), d.getMonth() + n, 1); }
|
||||
function _gridStart(viewMonth) {
|
||||
// Sunday before (or equal to) the 1st of viewMonth — gives a fixed 6-row grid.
|
||||
const first = _startOfMonth(viewMonth);
|
||||
return new Date(first.getFullYear(), first.getMonth(), 1 - first.getDay());
|
||||
function _dayEnd(d) {
|
||||
const x = _dayStart(d);
|
||||
x.setDate(x.getDate() + 1);
|
||||
return x;
|
||||
}
|
||||
function _addDays(d, n) {
|
||||
const x = new Date(d);
|
||||
x.setDate(x.getDate() + n);
|
||||
return x;
|
||||
}
|
||||
function _minutesIntoDay(date, dayStart) {
|
||||
return Math.max(0, Math.min(24 * 60, (date - dayStart) / 60000));
|
||||
}
|
||||
function _eventOverlapsDay(ev, dayStart, dayEnd) {
|
||||
const s = new Date(ev.start_at);
|
||||
const e = new Date(ev.end_at);
|
||||
return s < dayEnd && e > dayStart;
|
||||
}
|
||||
function _sameDay(a, b) {
|
||||
return a.getFullYear() === b.getFullYear()
|
||||
&& a.getMonth() === b.getMonth()
|
||||
&& a.getDate() === b.getDate();
|
||||
}
|
||||
function _fmtDay(d) {
|
||||
return d.toLocaleDateString(undefined, { weekday: 'short', month: 'short', day: 'numeric' });
|
||||
}
|
||||
function _fmtTime(d) {
|
||||
return d.toLocaleTimeString(undefined, { hour: 'numeric', minute: '2-digit' });
|
||||
}
|
||||
function _fmtHour(h) {
|
||||
// 0 → "12 AM", 12 → "12 PM", 18 → "6 PM"
|
||||
const ampm = h < 12 ? 'AM' : 'PM';
|
||||
const hr = ((h + 11) % 12) + 1;
|
||||
return hr + ' ' + ampm;
|
||||
}
|
||||
function _fmtCountdown(ms) {
|
||||
if (ms <= 0) return 'now';
|
||||
const s = Math.floor(ms / 1000);
|
||||
if (s < 60) return 'in ' + s + 's';
|
||||
const m = Math.floor(s / 60);
|
||||
if (m < 60) return 'in ' + m + 'm';
|
||||
const h = Math.floor(m / 60);
|
||||
return 'in ' + h + 'h ' + (m % 60) + 'm';
|
||||
}
|
||||
function _fmtElapsed(ms) {
|
||||
if (ms < 0) ms = 0;
|
||||
const s = Math.floor(ms / 1000);
|
||||
return String(Math.floor(s / 3600)).padStart(2, '0') + ':' +
|
||||
String(Math.floor((s % 3600) / 60)).padStart(2, '0') + ':' +
|
||||
String(s % 60).padStart(2, '0');
|
||||
}
|
||||
|
||||
function ScheduleCalendar({ schedules, viewMonth, onDayClick, onEventClick }) {
|
||||
const today = new Date();
|
||||
const gridStart = _gridStart(viewMonth);
|
||||
const days = [];
|
||||
for (let i = 0; i < 42; i++) {
|
||||
const d = new Date(gridStart);
|
||||
d.setDate(gridStart.getDate() + i);
|
||||
days.push(d);
|
||||
}
|
||||
// Pick a stable color for a project_id given the global PROJECTS list.
|
||||
function _projectColor(projectId, projects) {
|
||||
if (!projectId) return null;
|
||||
const p = (projects || []).find(p => p.id === projectId);
|
||||
return p?.color || null;
|
||||
}
|
||||
|
||||
const byDay = React.useMemo(() => {
|
||||
const m = {};
|
||||
(schedules || []).forEach(s => {
|
||||
const key = _ymd(new Date(s.start_at));
|
||||
(m[key] || (m[key] = [])).push(s);
|
||||
});
|
||||
Object.values(m).forEach(list => list.sort((a, b) => new Date(a.start_at) - new Date(b.start_at)));
|
||||
return m;
|
||||
}, [schedules]);
|
||||
// ── EPG components ───────────────────────────────────────────────────────────
|
||||
|
||||
function _StatusStrip({ schedules, recorders, now, projects }) {
|
||||
// What's recording: any schedule whose window contains `now` AND whose
|
||||
// status is 'running'. (Manually-started recorders without a schedule
|
||||
// surface on the Recorders screen; the schedule strip stays focused on
|
||||
// planned events so the operator can trust it.)
|
||||
const active = (schedules || []).filter(s => {
|
||||
const start = new Date(s.start_at);
|
||||
const end = new Date(s.end_at);
|
||||
return start <= now && now < end && (s.status === 'running' || s.status === 'pending');
|
||||
});
|
||||
|
||||
// Next up: earliest pending schedule strictly in the future.
|
||||
const upcoming = (schedules || [])
|
||||
.filter(s => s.status === 'pending' && new Date(s.start_at) > now)
|
||||
.sort((a, b) => new Date(a.start_at) - new Date(b.start_at));
|
||||
const next = upcoming[0];
|
||||
|
||||
const recMap = {};
|
||||
(recorders || []).forEach(r => { recMap[r.id] = r; });
|
||||
|
||||
return (
|
||||
<div className="cal">
|
||||
<div className="cal-weekheads">
|
||||
{['Sun','Mon','Tue','Wed','Thu','Fri','Sat'].map(w => <div key={w}>{w}</div>)}
|
||||
</div>
|
||||
<div className="cal-grid">
|
||||
{days.map(d => {
|
||||
const inMonth = d.getMonth() === viewMonth.getMonth();
|
||||
const isToday = _sameDay(d, today);
|
||||
const dayEvents = byDay[_ymd(d)] || [];
|
||||
const visible = dayEvents.slice(0, 3);
|
||||
const overflow = dayEvents.length - visible.length;
|
||||
return (
|
||||
<div key={d.toISOString()}
|
||||
className={'cal-cell' + (inMonth ? '' : ' off-month') + (isToday ? ' today' : '')}
|
||||
onClick={() => onDayClick(d)}
|
||||
title="Click to schedule on this day">
|
||||
<div className="cal-cell-head">
|
||||
<span className="cal-daynum">{d.getDate()}</span>
|
||||
{isToday && <span className="cal-today-pip">today</span>}
|
||||
</div>
|
||||
<div className="cal-events">
|
||||
{visible.map(s => {
|
||||
const badge = _STATUS_BADGE[s.status] || _STATUS_BADGE.pending;
|
||||
return (
|
||||
<button key={s.id}
|
||||
className={'cal-event ' + badge.cls}
|
||||
onClick={(e) => { e.stopPropagation(); onEventClick(s); }}
|
||||
title={s.name + ' · ' + _fmtWhen(s.start_at) + ' → ' + _fmtWhen(s.end_at) + ' · ' + badge.label}>
|
||||
<span className="cal-event-time mono">{_fmtTime(new Date(s.start_at))}</span>
|
||||
<span className="cal-event-name">{s.name}</span>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
{overflow > 0 && (
|
||||
<div className="cal-event-overflow">+{overflow} more</div>
|
||||
)}
|
||||
</div>
|
||||
<div className="epg-status">
|
||||
<div className="epg-status-row">
|
||||
{active.length === 0 ? (
|
||||
<>
|
||||
<span className="epg-status-dot idle" />
|
||||
<span className="epg-status-label">Nothing scheduled right now</span>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<span className="epg-status-dot live" />
|
||||
<span className="epg-status-label">On air</span>
|
||||
<div className="epg-status-active">
|
||||
{active.map(s => {
|
||||
const rec = recMap[s.recorder_id];
|
||||
const elapsed = _fmtElapsed(now - new Date(s.start_at));
|
||||
const endsAt = _fmtTime(new Date(s.end_at));
|
||||
const color = _projectColor(rec?.project_id, projects);
|
||||
return (
|
||||
<span key={s.id} className="epg-status-pill">
|
||||
{color && <span className="epg-status-pill-bar" style={{ background: color }} />}
|
||||
<span className="epg-status-pill-name">{s.name}</span>
|
||||
<span className="epg-status-pill-rec mono">{rec?.name || s.recorder_id.slice(0, 8)}</span>
|
||||
<span className="epg-status-pill-time mono">{elapsed} · ends {endsAt}</span>
|
||||
</span>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
<div className="epg-status-row sub">
|
||||
{next ? (
|
||||
<>
|
||||
<span className="epg-status-label muted">Next up</span>
|
||||
<span className="epg-status-next">{next.name}</span>
|
||||
<span className="epg-status-next-rec mono">{recMap[next.recorder_id]?.name || next.recorder_id.slice(0, 8)}</span>
|
||||
<span className="epg-status-next-time mono">{_fmtCountdown(new Date(next.start_at) - now)} · {_fmtTime(new Date(next.start_at))}</span>
|
||||
</>
|
||||
) : (
|
||||
<span className="epg-status-label muted">No upcoming schedules</span>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function Schedule({ navigate }) {
|
||||
const [schedules, setSchedules] = React.useState(null);
|
||||
const [recorders, setRecorders] = React.useState([]);
|
||||
const [showNew, setShowNew] = React.useState(false);
|
||||
const [newDefaults, setNewDefaults] = React.useState(null); // { start: Date, end: Date }
|
||||
const [editing, setEditing] = React.useState(null);
|
||||
const [filter, setFilter] = React.useState('upcoming');
|
||||
const [view, setView] = React.useState('calendar'); // 'calendar' | 'list'
|
||||
const [viewMonth, setViewMonth] = React.useState(() => _startOfMonth(new Date()));
|
||||
function _EpgRuler({ pph }) {
|
||||
// 25 ticks so the last one (24:00) labels the right edge. The 24h column
|
||||
// ends at 24*pph; the 25th tick is purely a label and has zero width.
|
||||
const hours = [];
|
||||
for (let h = 0; h <= 24; h++) hours.push(h);
|
||||
return (
|
||||
<div className="epg-ruler" style={{ width: 24 * pph }}>
|
||||
{hours.map(h => (
|
||||
<div key={h} className={'epg-ruler-tick ' + (h === 24 ? 'end' : '')}
|
||||
style={{ left: h * pph }}>
|
||||
<span>{h === 24 ? '' : _fmtHour(h)}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// Calendar mode wants every schedule in the visible window — the upcoming/past
|
||||
// filter only applies to the list view, so swap the API query accordingly.
|
||||
const apiFilter = view === 'calendar' ? 'all' : filter;
|
||||
function _EventBlock({ event, recorder, dayStart, dayEnd, pph, now, projects, onClick }) {
|
||||
const s = new Date(event.start_at);
|
||||
const e = new Date(event.end_at);
|
||||
const startMin = _minutesIntoDay(s, dayStart);
|
||||
const endMin = _minutesIntoDay(e, dayStart);
|
||||
const left = (startMin / 60) * pph;
|
||||
const width = Math.max(40, ((endMin - startMin) / 60) * pph);
|
||||
|
||||
const isLive = event.status === 'running' || (event.status === 'pending' && s <= now && now < e);
|
||||
const isFailed = event.status === 'failed';
|
||||
const isPast = (event.status === 'completed' || event.status === 'cancelled') || e < now;
|
||||
const color = _projectColor(recorder?.project_id, projects);
|
||||
|
||||
const classes = ['epg-block'];
|
||||
if (isLive) classes.push('live');
|
||||
if (isFailed) classes.push('failed');
|
||||
else if (isPast) classes.push('past');
|
||||
|
||||
return (
|
||||
<button
|
||||
className={classes.join(' ')}
|
||||
style={{ left, width, '--epg-block-color': color || 'var(--text-3)' }}
|
||||
onClick={(ev) => { ev.stopPropagation(); onClick(event); }}
|
||||
title={event.name + ' · ' + _fmtTime(s) + ' → ' + _fmtTime(e) + (event.error_message ? ' · ' + event.error_message : '')}>
|
||||
<span className="epg-block-bar" />
|
||||
<span className="epg-block-name">{event.name}</span>
|
||||
<span className="epg-block-time mono">{_fmtTime(s)}</span>
|
||||
{isLive && <span className="epg-block-glyph live" title="on air">●</span>}
|
||||
{isFailed && <span className="epg-block-glyph failed" title={event.error_message || 'failed'}>!</span>}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
function _EpgRow({ recorder, schedules, dayStart, dayEnd, pph, now, projects, onEventClick, onEmptyClick }) {
|
||||
const dayEvents = (schedules || []).filter(s => s.recorder_id === recorder.id && _eventOverlapsDay(s, dayStart, dayEnd));
|
||||
|
||||
const handleRowClick = (e) => {
|
||||
// Translate clicked x to a Date in this row's day. Snap to 15-minute
|
||||
// increments so the resulting modal pre-fill looks intentional.
|
||||
const rect = e.currentTarget.getBoundingClientRect();
|
||||
const x = e.clientX - rect.left;
|
||||
const minutes = Math.max(0, Math.min(24 * 60 - 30, Math.round((x / pph) * 60 / 15) * 15));
|
||||
const start = new Date(dayStart);
|
||||
start.setMinutes(minutes);
|
||||
onEmptyClick(recorder, start);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="epg-row" style={{ width: 24 * pph }} onClick={handleRowClick}>
|
||||
{dayEvents.map(s => (
|
||||
<_EventBlock
|
||||
key={s.id}
|
||||
event={s}
|
||||
recorder={recorder}
|
||||
dayStart={dayStart}
|
||||
dayEnd={dayEnd}
|
||||
pph={pph}
|
||||
now={now}
|
||||
projects={projects}
|
||||
onClick={onEventClick} />
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function _NowLine({ now, dayStart, pph }) {
|
||||
if (!_sameDay(now, dayStart)) return null;
|
||||
const min = _minutesIntoDay(now, dayStart);
|
||||
const x = (min / 60) * pph;
|
||||
return (
|
||||
<div className="epg-now" style={{ left: x }}>
|
||||
<span className="epg-now-pip" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function _RecorderGutter({ recorders, projects }) {
|
||||
return (
|
||||
<div className="epg-gutter-rows">
|
||||
{recorders.map(r => {
|
||||
const color = _projectColor(r.project_id, projects);
|
||||
const isLive = r.status === 'recording';
|
||||
const isErr = r.status === 'error';
|
||||
return (
|
||||
<div key={r.id} className="epg-gutter-row">
|
||||
<span className={'epg-gutter-status ' + (isLive ? 'live' : isErr ? 'err' : 'idle')} />
|
||||
<div className="epg-gutter-meta">
|
||||
<div className="epg-gutter-name">{r.name}</div>
|
||||
<div className="epg-gutter-sub mono">{(r.source_type || '—').toUpperCase()}{color ? ' · ' : ''}{color && <span className="epg-gutter-dot" style={{ background: color }} />}</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function Schedule({ navigate }) {
|
||||
const [schedules, setSchedules] = React.useState(null);
|
||||
const [recorders, setRecorders] = React.useState([]);
|
||||
const [showNew, setShowNew] = React.useState(false);
|
||||
const [newDefaults, setNewDefaults] = React.useState(null);
|
||||
const [editing, setEditing] = React.useState(null);
|
||||
const [view, setView] = React.useState('today'); // 'today' | 'week' | 'list'
|
||||
const [day, setDay] = React.useState(() => _dayStart(new Date()));
|
||||
const [listFilter, setListFilter] = React.useState('upcoming');
|
||||
const [now, setNow] = React.useState(() => new Date());
|
||||
|
||||
// Tick the now-line every second. We only re-render the components that
|
||||
// consume `now`; the rest are React.memo or insensitive.
|
||||
React.useEffect(() => {
|
||||
const id = setInterval(() => setNow(new Date()), 1000);
|
||||
return () => clearInterval(id);
|
||||
}, []);
|
||||
|
||||
// Schedule data — pull everything once and filter client-side for the
|
||||
// active view. /schedules caps at 200 rows so this stays cheap.
|
||||
const apiFilter = view === 'list' ? listFilter : 'all';
|
||||
const load = React.useCallback(() => {
|
||||
window.ZAMPP_API.fetch('/schedules?status=' + apiFilter)
|
||||
.then(d => setSchedules(d.schedules || []))
|
||||
|
|
@ -779,165 +1143,226 @@ function Schedule({ navigate }) {
|
|||
}, [apiFilter]);
|
||||
|
||||
React.useEffect(() => { load(); }, [load]);
|
||||
React.useEffect(() => { window.ZAMPP_API.fetch('/recorders').then(setRecorders).catch(() => setRecorders([])); }, []);
|
||||
|
||||
// Auto-refresh every 10s so the list reflects the tick loop's transitions
|
||||
React.useEffect(() => {
|
||||
const t = setInterval(load, 10_000);
|
||||
return () => clearInterval(t);
|
||||
}, [load]);
|
||||
window.ZAMPP_API.fetch('/recorders').then(setRecorders).catch(() => setRecorders([]));
|
||||
}, []);
|
||||
|
||||
const cancel = (s) => {
|
||||
if (!confirm(`Cancel scheduled recording "${s.name}"?`)) return;
|
||||
window.ZAMPP_API.fetch('/schedules/' + s.id + '/cancel', { method: 'POST' })
|
||||
.then(load)
|
||||
.catch(e => alert('Cancel failed: ' + e.message));
|
||||
};
|
||||
const remove = (s) => {
|
||||
if (!confirm(`Delete schedule "${s.name}"?`)) return;
|
||||
window.ZAMPP_API.fetch('/schedules/' + s.id, { method: 'DELETE' })
|
||||
.then(load)
|
||||
.catch(e => alert('Delete failed: ' + e.message));
|
||||
};
|
||||
// Auto-refresh: 10s in normal view, 4s if anything is live so the now-state
|
||||
// catches transitions promptly.
|
||||
React.useEffect(() => {
|
||||
const anyLive = (schedules || []).some(s => s.status === 'running');
|
||||
const id = setInterval(load, anyLive ? 4000 : 10_000);
|
||||
return () => clearInterval(id);
|
||||
}, [load, schedules]);
|
||||
|
||||
// Calendar: clicking a day pre-fills the new-schedule modal with a 30-minute
|
||||
// window starting at 10:00 AM that day (or +5min if the day is today and
|
||||
// 10:00 has already passed). Gives the operator a sensible starting point
|
||||
// instead of dropping them into empty datetime-local fields.
|
||||
const openNewOnDay = (day) => {
|
||||
const now = new Date();
|
||||
const isToday = _sameDay(day, now);
|
||||
const start = new Date(day);
|
||||
if (isToday && now.getHours() >= 10) {
|
||||
start.setHours(now.getHours(), now.getMinutes() + 5, 0, 0);
|
||||
} else {
|
||||
start.setHours(10, 0, 0, 0);
|
||||
// The Recorders screen broadcasts these on create/delete; refresh the
|
||||
// gutter so renamed or new recorders show up immediately.
|
||||
React.useEffect(() => {
|
||||
const refresh = () => window.ZAMPP_API.fetch('/recorders').then(setRecorders).catch(() => {});
|
||||
window.addEventListener('df:recorders-changed', refresh);
|
||||
return () => window.removeEventListener('df:recorders-changed', refresh);
|
||||
}, []);
|
||||
|
||||
const projects = window.ZAMPP_DATA?.PROJECTS || [];
|
||||
|
||||
// Pixels per hour — wider on Today (high-res operations view), tighter
|
||||
// when the user is scanning Week-at-a-glance.
|
||||
const pph = view === 'week' ? 44 : 88;
|
||||
|
||||
const dayStart = _dayStart(day);
|
||||
const dayEnd = _dayEnd(day);
|
||||
|
||||
// Scroll the canvas so "now" sits ~30% from the left edge on first paint
|
||||
// for Today view. Re-runs when the user jumps days via the Today button.
|
||||
const canvasRef = React.useRef(null);
|
||||
React.useLayoutEffect(() => {
|
||||
if (view !== 'today' || !canvasRef.current) return;
|
||||
if (!_sameDay(now, dayStart)) {
|
||||
canvasRef.current.scrollLeft = 0;
|
||||
return;
|
||||
}
|
||||
const min = _minutesIntoDay(now, dayStart);
|
||||
const x = (min / 60) * pph;
|
||||
const target = Math.max(0, x - canvasRef.current.clientWidth * 0.3);
|
||||
canvasRef.current.scrollLeft = target;
|
||||
// Deliberately only re-run on view/day change, not on `now` ticking.
|
||||
// Otherwise the canvas would re-scroll every second and trap the user.
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [view, day, pph, recorders.length]);
|
||||
|
||||
const openNewAt = (recorder, start) => {
|
||||
const end = new Date(start.getTime() + 30 * 60 * 1000);
|
||||
setNewDefaults({ start, end });
|
||||
setNewDefaults({ start, end, recorder_id: recorder.id });
|
||||
setShowNew(true);
|
||||
};
|
||||
const openNewBlank = () => { setNewDefaults(null); setShowNew(true); };
|
||||
|
||||
const monthLabel = viewMonth.toLocaleString(undefined, { month: 'long', year: 'numeric' });
|
||||
const cancel = (s) => {
|
||||
if (!confirm('Cancel scheduled recording "' + s.name + '"?')) return;
|
||||
window.ZAMPP_API.fetch('/schedules/' + s.id + '/cancel', { method: 'POST' }).then(load).catch(e => alert('Cancel failed: ' + e.message));
|
||||
};
|
||||
const remove = (s) => {
|
||||
if (!confirm('Delete schedule "' + s.name + '"?')) return;
|
||||
window.ZAMPP_API.fetch('/schedules/' + s.id, { method: 'DELETE' }).then(load).catch(e => alert('Delete failed: ' + e.message));
|
||||
};
|
||||
|
||||
// Days for Week view: the 7-day window starting at the Sunday of `day`.
|
||||
const weekDays = React.useMemo(() => {
|
||||
const sun = _addDays(dayStart, -dayStart.getDay());
|
||||
return Array.from({ length: 7 }, (_, i) => _addDays(sun, i));
|
||||
}, [dayStart]);
|
||||
|
||||
// List view filters schedules by client-side time bucket too.
|
||||
const listSchedules = React.useMemo(() => {
|
||||
if (!schedules) return null;
|
||||
return [...schedules].sort((a, b) => new Date(a.start_at) - new Date(b.start_at));
|
||||
}, [schedules]);
|
||||
|
||||
return (
|
||||
<div className="page">
|
||||
<div className="page-header">
|
||||
<h1>Schedule</h1>
|
||||
<span className="subtitle">
|
||||
{view === 'calendar'
|
||||
? 'Click a day to schedule · ' + (schedules?.length || 0) + ' total'
|
||||
: 'Plan recorder windows · ' + (schedules?.length || 0) + ' ' + filter}
|
||||
</span>
|
||||
<div className="spacer" />
|
||||
<div className="tab-group" style={{ marginRight: 8 }}>
|
||||
<button className={view === 'calendar' ? 'active' : ''} onClick={() => setView('calendar')} title="Month calendar"><Icon name="jobs" />Calendar</button>
|
||||
<button className={view === 'list' ? 'active' : ''} onClick={() => setView('list')} title="Linear list"><Icon name="list" />List</button>
|
||||
<div className="epg-page" style={{ '--epg-pph': pph + 'px' }}>
|
||||
<_StatusStrip schedules={schedules || []} recorders={recorders} now={now} projects={projects} />
|
||||
|
||||
<div className="epg-toolbar">
|
||||
<div className="epg-date">
|
||||
<button className="icon-btn" onClick={() => setDay(_addDays(day, -1))} title="Previous day"><Icon name="chevron" style={{ transform: 'rotate(90deg)' }} /></button>
|
||||
<div className="epg-date-label">{_sameDay(day, new Date()) ? 'Today · ' + _fmtDay(day) : _fmtDay(day)}</div>
|
||||
<button className="icon-btn" onClick={() => setDay(_addDays(day, 1))} title="Next day"><Icon name="chevron" style={{ transform: 'rotate(-90deg)' }} /></button>
|
||||
<button className="btn ghost sm" onClick={() => setDay(_dayStart(new Date()))} disabled={_sameDay(day, new Date())}>Today</button>
|
||||
</div>
|
||||
<div className="spacer" />
|
||||
<div className="tab-group">
|
||||
<button className={view === 'today' ? 'active' : ''} onClick={() => setView('today')}>Today</button>
|
||||
<button className={view === 'week' ? 'active' : ''} onClick={() => setView('week')}>Week</button>
|
||||
<button className={view === 'list' ? 'active' : ''} onClick={() => setView('list')}>List</button>
|
||||
</div>
|
||||
{view === 'list' && (
|
||||
<div className="tab-group" style={{ marginRight: 8 }}>
|
||||
<button className={filter === 'upcoming' ? 'active' : ''} onClick={() => setFilter('upcoming')}>Upcoming</button>
|
||||
<button className={filter === 'past' ? 'active' : ''} onClick={() => setFilter('past')}>Past</button>
|
||||
<button className={filter === 'all' ? 'active' : ''} onClick={() => setFilter('all')}>All</button>
|
||||
</div>
|
||||
)}
|
||||
<button className="btn ghost sm" onClick={load}><Icon name="refresh" />Refresh</button>
|
||||
<button className="btn primary" onClick={openNewBlank} disabled={recorders.length === 0}>
|
||||
<Icon name="plus" />New schedule
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="page-body">
|
||||
{view === 'calendar' && (
|
||||
<>
|
||||
<div className="cal-toolbar">
|
||||
<button className="btn ghost sm" onClick={() => setViewMonth(_addMonths(viewMonth, -1))} title="Previous month">
|
||||
<Icon name="chevron" style={{ transform: 'rotate(90deg)' }} />
|
||||
</button>
|
||||
<div className="cal-month-label">{monthLabel}</div>
|
||||
<button className="btn ghost sm" onClick={() => setViewMonth(_addMonths(viewMonth, 1))} title="Next month">
|
||||
<Icon name="chevron" style={{ transform: 'rotate(-90deg)' }} />
|
||||
</button>
|
||||
<button className="btn ghost sm" onClick={() => setViewMonth(_startOfMonth(new Date()))} style={{ marginLeft: 8 }}>
|
||||
Today
|
||||
</button>
|
||||
<div style={{ flex: 1 }} />
|
||||
{recorders.length === 0 && (
|
||||
<div style={{ fontSize: 11.5, color: 'var(--text-3)' }}>Create a recorder before scheduling.</div>
|
||||
)}
|
||||
</div>
|
||||
{schedules === null
|
||||
? <div style={{ textAlign: 'center', padding: '64px 0', color: 'var(--text-3)' }}>Loading…</div>
|
||||
: <ScheduleCalendar
|
||||
schedules={schedules}
|
||||
viewMonth={viewMonth}
|
||||
onDayClick={openNewOnDay}
|
||||
onEventClick={(s) => setEditing(s)} />}
|
||||
</>
|
||||
)}
|
||||
{schedules === null && (
|
||||
<div className="epg-empty">Loading…</div>
|
||||
)}
|
||||
|
||||
{view === 'list' && schedules === null && (
|
||||
<div style={{ textAlign: 'center', padding: '64px 0', color: 'var(--text-3)' }}>Loading…</div>
|
||||
)}
|
||||
{view === 'list' && schedules !== null && schedules.length === 0 && (
|
||||
<div style={{ textAlign: 'center', padding: '64px 0', color: 'var(--text-3)' }}>
|
||||
<div style={{ fontSize: 32, marginBottom: 12 }}>📅</div>
|
||||
<div style={{ fontWeight: 500, fontSize: 14 }}>No {filter} recordings</div>
|
||||
{filter === 'upcoming' && recorders.length === 0 && (
|
||||
<div style={{ fontSize: 12, marginTop: 6 }}>Create a recorder first, then schedule it here.</div>
|
||||
)}
|
||||
{filter === 'upcoming' && recorders.length > 0 && (
|
||||
<div style={{ fontSize: 12, marginTop: 6 }}>Click <em>New schedule</em> to plan a future recording.</div>
|
||||
)}
|
||||
{schedules !== null && recorders.length === 0 && (
|
||||
<div className="epg-empty">
|
||||
<div className="epg-empty-title">No recorders configured</div>
|
||||
<div className="epg-empty-sub">Create a recorder before scheduling.</div>
|
||||
<button className="btn primary sm" onClick={() => navigate('recorders')}>Go to Recorders</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{schedules !== null && recorders.length > 0 && view === 'today' && (
|
||||
<div className="epg" ref={canvasRef}>
|
||||
<div className="epg-corner">
|
||||
<span className="mono">{_fmtDay(day)}</span>
|
||||
</div>
|
||||
)}
|
||||
{view === 'list' && schedules !== null && schedules.length > 0 && (
|
||||
<div className="panel">
|
||||
<div className="schedule-row head">
|
||||
<div>Name</div>
|
||||
<div>Recorder</div>
|
||||
<div>Starts</div>
|
||||
<div>Duration</div>
|
||||
<div>Recurrence</div>
|
||||
<div>Status</div>
|
||||
<div></div>
|
||||
<div className="epg-gutter">
|
||||
<_RecorderGutter recorders={recorders} projects={projects} />
|
||||
</div>
|
||||
<div className="epg-canvas-head">
|
||||
<_EpgRuler pph={pph} />
|
||||
</div>
|
||||
<div className="epg-canvas" style={{ width: 24 * pph }}>
|
||||
<div className="epg-rows">
|
||||
{recorders.map(r => (
|
||||
<_EpgRow key={r.id}
|
||||
recorder={r}
|
||||
schedules={schedules}
|
||||
dayStart={dayStart}
|
||||
dayEnd={dayEnd}
|
||||
pph={pph}
|
||||
now={now}
|
||||
projects={projects}
|
||||
onEventClick={(s) => setEditing(s)}
|
||||
onEmptyClick={openNewAt} />
|
||||
))}
|
||||
</div>
|
||||
{schedules.map(s => {
|
||||
const badge = _STATUS_BADGE[s.status] || _STATUS_BADGE.pending;
|
||||
return (
|
||||
<div key={s.id} className="schedule-row">
|
||||
<div style={{ fontWeight: 500, fontSize: 13 }}>
|
||||
{s.name}
|
||||
{s.error_message && (
|
||||
<div style={{ fontSize: 11, color: 'var(--danger)', marginTop: 3 }}>{s.error_message}</div>
|
||||
)}
|
||||
</div>
|
||||
<div className="mono" style={{ fontSize: 11.5, color: 'var(--text-2)' }}>{s.recorder_name || s.recorder_id.slice(0, 8)}</div>
|
||||
<div className="mono" style={{ fontSize: 11.5 }}>{_fmtWhen(s.start_at)}</div>
|
||||
<div className="mono" style={{ fontSize: 11.5 }}>{_durationMin(s.start_at, s.end_at)} min</div>
|
||||
<div className="mono" style={{ fontSize: 11.5, color: 'var(--text-3)' }}>{s.recurrence === 'none' ? 'one-shot' : s.recurrence}</div>
|
||||
<div><span className={`badge ${badge.cls}`}>{badge.label}</span></div>
|
||||
<div style={{ display: 'flex', gap: 4, justifyContent: 'flex-end' }}>
|
||||
{s.status === 'pending' && (
|
||||
<button className="btn ghost sm" onClick={() => setEditing(s)} title="Edit name / time / recurrence">Edit</button>
|
||||
)}
|
||||
{(s.status === 'pending' || s.status === 'running') && (
|
||||
<button className="btn ghost sm" onClick={() => cancel(s)}>Cancel</button>
|
||||
)}
|
||||
{(s.status === 'completed' || s.status === 'failed' || s.status === 'cancelled' || s.status === 'pending') && (
|
||||
<button className="btn ghost sm" onClick={() => remove(s)} title="Delete schedule row">Delete</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
<_NowLine now={now} dayStart={dayStart} pph={pph} />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{schedules !== null && recorders.length > 0 && view === 'week' && (
|
||||
<div className="epg-week">
|
||||
{weekDays.map(d => {
|
||||
const dEnd = _dayEnd(d);
|
||||
const isToday = _sameDay(d, new Date());
|
||||
return (
|
||||
<div key={d.toISOString()} className={'epg-week-day' + (isToday ? ' today' : '')}>
|
||||
<div className="epg-week-dayhead">
|
||||
<span className="epg-week-dayname">{_fmtDay(d)}</span>
|
||||
{isToday && <span className="epg-week-todaypip">today</span>}
|
||||
</div>
|
||||
<div className="epg-week-row-wrap" style={{ width: 24 * pph }}>
|
||||
<_EpgRuler pph={pph} />
|
||||
<div className="epg-rows">
|
||||
{recorders.map(r => (
|
||||
<_EpgRow key={r.id}
|
||||
recorder={r}
|
||||
schedules={schedules}
|
||||
dayStart={d}
|
||||
dayEnd={dEnd}
|
||||
pph={pph}
|
||||
now={now}
|
||||
projects={projects}
|
||||
onEventClick={(s) => setEditing(s)}
|
||||
onEmptyClick={openNewAt} />
|
||||
))}
|
||||
</div>
|
||||
{isToday && <_NowLine now={now} dayStart={d} pph={pph} />}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{schedules !== null && recorders.length > 0 && view === 'list' && (
|
||||
<div className="epg-list">
|
||||
<div className="tab-group" style={{ marginBottom: 12 }}>
|
||||
<button className={listFilter === 'upcoming' ? 'active' : ''} onClick={() => setListFilter('upcoming')}>Upcoming</button>
|
||||
<button className={listFilter === 'past' ? 'active' : ''} onClick={() => setListFilter('past')}>Past</button>
|
||||
<button className={listFilter === 'all' ? 'active' : ''} onClick={() => setListFilter('all')}>All</button>
|
||||
</div>
|
||||
{(listSchedules || []).length === 0 ? (
|
||||
<div className="epg-empty"><div className="epg-empty-title">No {listFilter} schedules</div></div>
|
||||
) : (
|
||||
<div className="panel">
|
||||
<div className="schedule-row head">
|
||||
<div>Name</div><div>Recorder</div><div>Starts</div><div>Duration</div><div>Recurrence</div><div>Status</div><div></div>
|
||||
</div>
|
||||
{listSchedules.map(s => {
|
||||
const badge = _STATUS_BADGE[s.status] || _STATUS_BADGE.pending;
|
||||
return (
|
||||
<div key={s.id} className="schedule-row">
|
||||
<div style={{ fontWeight: 500, fontSize: 13 }}>
|
||||
{s.name}
|
||||
{s.error_message && <div style={{ fontSize: 11, color: 'var(--danger)', marginTop: 3 }}>{s.error_message}</div>}
|
||||
</div>
|
||||
<div className="mono" style={{ fontSize: 11.5, color: 'var(--text-2)' }}>{s.recorder_name || s.recorder_id.slice(0, 8)}</div>
|
||||
<div className="mono" style={{ fontSize: 11.5 }}>{_fmtWhen(s.start_at)}</div>
|
||||
<div className="mono" style={{ fontSize: 11.5 }}>{_durationMin(s.start_at, s.end_at)} min</div>
|
||||
<div className="mono" style={{ fontSize: 11.5, color: 'var(--text-3)' }}>{s.recurrence === 'none' ? 'one-shot' : s.recurrence}</div>
|
||||
<div><span className={'badge ' + badge.cls}>{badge.label}</span></div>
|
||||
<div style={{ display: 'flex', gap: 4, justifyContent: 'flex-end' }}>
|
||||
{s.status === 'pending' && <button className="btn ghost sm" onClick={() => setEditing(s)}>Edit</button>}
|
||||
{(s.status === 'pending' || s.status === 'running') && <button className="btn ghost sm" onClick={() => cancel(s)}>Cancel</button>}
|
||||
{(s.status === 'completed' || s.status === 'failed' || s.status === 'cancelled' || s.status === 'pending') &&
|
||||
<button className="btn ghost sm" onClick={() => remove(s)}>Delete</button>}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{showNew && <NewScheduleModal
|
||||
recorders={recorders}
|
||||
defaultRecorderId={newDefaults?.recorder_id}
|
||||
defaultStart={newDefaults?.start}
|
||||
defaultEnd={newDefaults?.end}
|
||||
onClose={() => { setShowNew(false); setNewDefaults(null); }}
|
||||
|
|
@ -947,6 +1372,7 @@ function Schedule({ navigate }) {
|
|||
);
|
||||
}
|
||||
|
||||
|
||||
function EditScheduleModal({ schedule, onClose, onSaved }) {
|
||||
const toLocalInput = (iso) => {
|
||||
const d = new Date(iso);
|
||||
|
|
@ -1036,7 +1462,7 @@ function EditScheduleModal({ schedule, onClose, onSaved }) {
|
|||
);
|
||||
}
|
||||
|
||||
function NewScheduleModal({ recorders, onClose, onCreated, defaultStart, defaultEnd }) {
|
||||
function NewScheduleModal({ recorders, onClose, onCreated, defaultStart, defaultEnd, defaultRecorderId }) {
|
||||
// If the user clicked a day on the calendar we honour that; otherwise default
|
||||
// to "start in 5 minutes, run for 30 min" so the modal is immediately usable.
|
||||
const toLocalInput = (d) => {
|
||||
|
|
@ -1050,7 +1476,7 @@ function NewScheduleModal({ recorders, onClose, onCreated, defaultStart, default
|
|||
|
||||
const [form, setForm] = React.useState({
|
||||
name: '',
|
||||
recorder_id: recorders[0]?.id || '',
|
||||
recorder_id: defaultRecorderId || recorders[0]?.id || '',
|
||||
start_at: toLocalInput(startDefault),
|
||||
end_at: toLocalInput(endDefault),
|
||||
recurrence: 'none',
|
||||
|
|
@ -1162,4 +1588,4 @@ function NewScheduleModal({ recorders, onClose, onCreated, defaultStart, default
|
|||
);
|
||||
}
|
||||
|
||||
Object.assign(window, { Upload, Recorders, Capture, Monitors, Schedule });
|
||||
Object.assign(window, { Upload, Recorders, Capture, Monitors, Schedule, YouTubeImport });
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@ function Jobs({ navigate }) {
|
|||
|
||||
const normalizeJob = (j) => {
|
||||
const statusMap = { waiting: 'queued', active: 'running', completed: 'done', failed: 'failed' };
|
||||
const kindMap = { proxy: 'Proxy', thumbnail: 'Thumbnail', conform: 'Conform', transcode: 'Transcode' };
|
||||
const kindMap = { proxy: 'Proxy', thumbnail: 'Thumbnail', conform: 'Conform', transcode: 'Transcode', import: 'YouTube' };
|
||||
const meta = j.metadata || {};
|
||||
return {
|
||||
...j,
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ const NAV_TREE = [
|
|||
id: "ingest", label: "Ingest", icon: "upload", group: true,
|
||||
children: [
|
||||
{ id: "upload", label: "Upload", icon: "upload" },
|
||||
{ id: "youtube", label: "YouTube", icon: "download" },
|
||||
{ id: "recorders", label: "Recorders", icon: "record" },
|
||||
{ id: "schedule", label: "Schedule", icon: "jobs" },
|
||||
{ id: "capture", label: "Capture", icon: "capture" },
|
||||
|
|
@ -89,7 +90,7 @@ function Sidebar({ active, onNavigate, me }) {
|
|||
};
|
||||
|
||||
React.useEffect(() => {
|
||||
const ingestChildren = ["upload", "recorders", "schedule", "capture", "monitors"];
|
||||
const ingestChildren = ["upload", "youtube", "recorders", "schedule", "capture", "monitors"];
|
||||
if (ingestChildren.includes(active)) {
|
||||
setOpenGroups(prev => prev.has("ingest") ? prev : new Set([...prev, "ingest"]));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
FROM node:20-alpine
|
||||
RUN apk add --no-cache ffmpeg
|
||||
# yt-dlp powers the YouTube importer; python3 is its runtime dep.
|
||||
RUN apk add --no-cache ffmpeg yt-dlp python3
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm install --omit=dev
|
||||
|
|
|
|||
|
|
@ -8,9 +8,10 @@
|
|||
|
||||
FROM nvcr.io/nvidia/cuda:12.3.1-base-ubuntu22.04
|
||||
|
||||
# Install Node.js 20 and ffmpeg (Ubuntu's ffmpeg includes h264_nvenc/hevc_nvenc)
|
||||
# Install Node.js 20, ffmpeg (Ubuntu's ffmpeg includes h264_nvenc/hevc_nvenc),
|
||||
# and yt-dlp (+ python3 runtime) for the YouTube importer.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl ca-certificates ffmpeg \
|
||||
curl ca-certificates ffmpeg yt-dlp python3 \
|
||||
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
|
||||
&& apt-get install -y --no-install-recommends nodejs \
|
||||
&& apt-get clean && rm -rf /var/lib/apt/lists/*
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import { Worker } from 'bullmq';
|
|||
import { proxyWorker } from './workers/proxy.js';
|
||||
import { thumbnailWorker } from './workers/thumbnail.js';
|
||||
import { conformWorker } from './workers/conform.js';
|
||||
import { youtubeImportWorker } from './workers/youtube-import.js';
|
||||
import { startPromotionWorker } from './workers/promotion.js';
|
||||
|
||||
const parseRedisUrl = (url) => {
|
||||
|
|
@ -16,7 +17,7 @@ const parseRedisUrl = (url) => {
|
|||
|
||||
const redisOptions = parseRedisUrl(process.env.REDIS_URL || 'redis://localhost:6379');
|
||||
|
||||
const createWorker = (queueName, handler) => {
|
||||
const createWorker = (queueName, handler, overrides = {}) => {
|
||||
const worker = new Worker(queueName, handler, {
|
||||
connection: redisOptions,
|
||||
// Stall detection: if a worker dies mid-job, BullMQ moves it back to wait
|
||||
|
|
@ -25,6 +26,7 @@ const createWorker = (queueName, handler) => {
|
|||
maxStalledCount: 1,
|
||||
lockDuration: 60000,
|
||||
lockRenewTime: 15000,
|
||||
...overrides,
|
||||
});
|
||||
|
||||
worker.on('completed', (job) => {
|
||||
|
|
@ -50,13 +52,21 @@ const workers = [
|
|||
createWorker('proxy', proxyWorker),
|
||||
createWorker('thumbnail', thumbnailWorker),
|
||||
createWorker('conform', conformWorker),
|
||||
// YouTube imports: keep concurrency at 1 so we don't burn through rate
|
||||
// limits when several jobs land back-to-back. Lock window is longer than
|
||||
// the default because a long video download can run for minutes.
|
||||
createWorker('import', youtubeImportWorker, {
|
||||
concurrency: 1,
|
||||
lockDuration: 10 * 60 * 1000,
|
||||
lockRenewTime: 60000,
|
||||
}),
|
||||
];
|
||||
|
||||
startPromotionWorker();
|
||||
|
||||
console.log('Wild Dragon Worker Service started');
|
||||
console.log(`Redis: ${redisOptions.host}:${redisOptions.port}`);
|
||||
console.log('Active queues: proxy, thumbnail, conform');
|
||||
console.log('Active queues: proxy, thumbnail, conform, import');
|
||||
console.log('Background scans: promotion (growing-files → S3)');
|
||||
|
||||
process.on('SIGTERM', async () => {
|
||||
|
|
|
|||
223
services/worker/src/workers/youtube-import.js
Normal file
223
services/worker/src/workers/youtube-import.js
Normal file
|
|
@ -0,0 +1,223 @@
|
|||
// YouTube importer worker — shells out to yt-dlp, lands the resulting MP4 in
|
||||
// S3 at the same originals/{assetId}/<title>.mp4 path uploads use, then hands
|
||||
// off to the existing proxy queue. From that point an imported asset is
|
||||
// indistinguishable from an uploaded one.
|
||||
|
||||
import { spawn } from 'node:child_process';
|
||||
import { join } from 'node:path';
|
||||
import { mkdtemp, rm, stat, readdir } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { Queue } from 'bullmq';
|
||||
import { query } from '../db/client.js';
|
||||
import { uploadToS3 } from '../s3/client.js';
|
||||
import { getMediaInfo } from '../ffmpeg/executor.js';
|
||||
|
||||
const S3_BUCKET = process.env.S3_BUCKET || 'wild-dragon';
|
||||
|
||||
const parseRedisUrl = (url) => {
|
||||
const parsed = new URL(url);
|
||||
return { host: parsed.hostname, port: parseInt(parsed.port, 10) };
|
||||
};
|
||||
|
||||
// Hand off to the existing proxy queue once the original is in S3.
|
||||
const proxyQueue = new Queue('proxy', {
|
||||
connection: parseRedisUrl(process.env.REDIS_URL || 'redis://queue:6379'),
|
||||
});
|
||||
|
||||
// Map yt-dlp stderr lines to short, operator-friendly messages. Anything that
|
||||
// doesn't match here falls back to the raw stderr (truncated).
|
||||
function friendlyError(stderr) {
|
||||
const s = stderr || '';
|
||||
if (/Private video/i.test(s)) return 'Private video — not supported.';
|
||||
if (/Sign in to confirm your age/i.test(s)) return 'Age-restricted video — not supported.';
|
||||
if (/members[- ]only/i.test(s)) return 'Members-only video — not supported.';
|
||||
if (/Video unavailable/i.test(s)) return 'Video unavailable or removed.';
|
||||
if (/not available in your country|geo[- ]?restricted/i.test(s)) return 'Video is geo-blocked from this region.';
|
||||
if (/HTTP Error 429/i.test(s)) return 'YouTube rate-limited the importer — try again later.';
|
||||
if (/Unable to extract|Unsupported URL/i.test(s)) return 'YouTube changed its API — worker image needs a rebuild.';
|
||||
|
||||
// Last-resort: the first ERROR: line from stderr, capped.
|
||||
const m = s.match(/ERROR:\s*([^\n]+)/i);
|
||||
const raw = (m ? m[1] : s).trim().slice(0, 300);
|
||||
return raw || 'yt-dlp failed with no error message';
|
||||
}
|
||||
|
||||
// Replace anything outside [A-Za-z0-9 ._-] with '-', collapse runs of
|
||||
// whitespace/dashes, trim, cap to 120 chars. The .mp4 extension is appended
|
||||
// by the caller. If the result is empty we fall back to the video ID.
|
||||
function sanitizeTitle(title, videoId) {
|
||||
if (!title || typeof title !== 'string') return `youtube-${videoId}`;
|
||||
let out = title
|
||||
.replace(/[^\w .\-]+/g, '-')
|
||||
.replace(/[-\s]+/g, ' ')
|
||||
.trim()
|
||||
.slice(0, 120);
|
||||
if (!out) out = `youtube-${videoId}`;
|
||||
return out;
|
||||
}
|
||||
|
||||
// Run yt-dlp with progress streaming. Returns the parsed --print-json line.
|
||||
// Throws if yt-dlp exits non-zero — the caller maps stderr to a friendly msg.
|
||||
async function runYtDlp({ url, outputTemplate, onProgress }) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const args = [
|
||||
'--no-playlist',
|
||||
'--no-warnings',
|
||||
'--restrict-filenames',
|
||||
'-f', "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]/b",
|
||||
'--merge-output-format', 'mp4',
|
||||
'--print-json',
|
||||
'--newline',
|
||||
'-o', outputTemplate,
|
||||
url,
|
||||
];
|
||||
|
||||
const proc = spawn('yt-dlp', args);
|
||||
let stdoutBuf = '';
|
||||
let stderrBuf = '';
|
||||
let lastJsonLine = null;
|
||||
|
||||
proc.stdout.on('data', (chunk) => {
|
||||
stdoutBuf += chunk.toString();
|
||||
let nl;
|
||||
while ((nl = stdoutBuf.indexOf('\n')) !== -1) {
|
||||
const line = stdoutBuf.slice(0, nl);
|
||||
stdoutBuf = stdoutBuf.slice(nl + 1);
|
||||
|
||||
// [download] 42.3% of 53.21MiB at 4.21MiB/s ETA 00:07
|
||||
const m = line.match(/\[download\]\s+(\d+(?:\.\d+)?)%/);
|
||||
if (m && onProgress) onProgress(parseFloat(m[1]));
|
||||
|
||||
// --print-json emits one JSON line at the end of a successful download.
|
||||
if (line.startsWith('{') && line.endsWith('}')) {
|
||||
try { lastJsonLine = JSON.parse(line); } catch { /* not the json line */ }
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
proc.stderr.on('data', (chunk) => { stderrBuf += chunk.toString(); });
|
||||
|
||||
proc.on('error', (err) => reject(new Error(`Failed to spawn yt-dlp: ${err.message}`)));
|
||||
|
||||
proc.on('close', (code) => {
|
||||
if (code === 0) {
|
||||
resolve(lastJsonLine || {});
|
||||
} else {
|
||||
const err = new Error(friendlyError(stderrBuf));
|
||||
err.stderr = stderrBuf;
|
||||
err.code = code;
|
||||
reject(err);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export const youtubeImportWorker = async (job) => {
|
||||
const { assetId, url } = job.data;
|
||||
|
||||
// Each job gets its own temp directory so concurrent jobs (if we ever bump
|
||||
// concurrency above 1) can't clobber each other's intermediate files.
|
||||
const workDir = await mkdtemp(join(tmpdir(), `yt-${job.id}-`));
|
||||
const outputTemplate = join(workDir, `${assetId}.%(ext)s`);
|
||||
|
||||
try {
|
||||
console.log(`[youtube] Asset ${assetId}: importing ${url}`);
|
||||
await job.updateProgress(2);
|
||||
|
||||
// yt-dlp does the work; progress 5..60 covers the download.
|
||||
const meta = await runYtDlp({
|
||||
url,
|
||||
outputTemplate,
|
||||
onProgress: async (pct) => {
|
||||
const mapped = 5 + Math.floor(pct * 0.55);
|
||||
try { await job.updateProgress(mapped); } catch { /* ignore */ }
|
||||
},
|
||||
});
|
||||
|
||||
await job.updateProgress(62);
|
||||
|
||||
// Find the resulting MP4 — yt-dlp's --merge-output-format ensures .mp4
|
||||
// but we scan the dir defensively in case the format string changes.
|
||||
const files = await readdir(workDir);
|
||||
const mp4 = files.find((f) => f.endsWith('.mp4'));
|
||||
if (!mp4) {
|
||||
throw new Error(`yt-dlp produced no .mp4 in ${workDir} (got ${files.join(', ') || 'nothing'})`);
|
||||
}
|
||||
const localPath = join(workDir, mp4);
|
||||
|
||||
const { size: fileSize } = await stat(localPath);
|
||||
if (fileSize < 4096) {
|
||||
throw new Error(`Downloaded file is suspiciously small (${fileSize} bytes) — aborting before upload.`);
|
||||
}
|
||||
|
||||
// ffprobe the file ourselves — yt-dlp's metadata is sometimes missing or
|
||||
// wrong (especially fps), and we already trust ffprobe everywhere else.
|
||||
let mediaInfo = {};
|
||||
try {
|
||||
mediaInfo = await getMediaInfo(localPath);
|
||||
} catch (err) {
|
||||
console.warn(`[youtube] getMediaInfo failed for ${assetId}: ${err.message}`);
|
||||
}
|
||||
|
||||
const videoId = meta.id || mp4.replace(/\..+$/, '').replace(/^.*-/, '');
|
||||
const sanitized = sanitizeTitle(meta.title || meta.fulltitle, videoId);
|
||||
const filename = `${sanitized}.mp4`;
|
||||
const originalKey = `originals/${assetId}/${filename}`;
|
||||
|
||||
await job.updateProgress(70);
|
||||
console.log(`[youtube] Uploading ${localPath} → s3://${S3_BUCKET}/${originalKey} (${fileSize} bytes)`);
|
||||
await uploadToS3(S3_BUCKET, originalKey, localPath);
|
||||
|
||||
await job.updateProgress(90);
|
||||
|
||||
// Backfill the asset row with the real title + S3 key + ffprobe metadata,
|
||||
// then flip to 'processing' so the rest of the UI treats it like any
|
||||
// freshly-uploaded asset.
|
||||
await query(
|
||||
`UPDATE assets
|
||||
SET filename = $1,
|
||||
display_name = $2,
|
||||
original_s3_key = $3,
|
||||
codec = COALESCE($4, codec),
|
||||
resolution = COALESCE($5, resolution),
|
||||
fps = COALESCE($6, fps),
|
||||
duration_ms = COALESCE($7, duration_ms),
|
||||
file_size = COALESCE($8, file_size),
|
||||
status = 'processing',
|
||||
updated_at = NOW()
|
||||
WHERE id = $9`,
|
||||
[
|
||||
filename,
|
||||
meta.title || meta.fulltitle || filename,
|
||||
originalKey,
|
||||
mediaInfo.codec ?? null,
|
||||
mediaInfo.resolution ?? null,
|
||||
mediaInfo.fps ?? null,
|
||||
mediaInfo.durationMs ?? null,
|
||||
mediaInfo.fileSizeBytes ?? fileSize,
|
||||
assetId,
|
||||
]
|
||||
);
|
||||
|
||||
// Hand off to the proxy queue — identical payload shape to upload.js so
|
||||
// the proxy worker doesn't need to know this came from an import.
|
||||
await proxyQueue.add('generate', {
|
||||
assetId,
|
||||
inputKey: originalKey,
|
||||
outputKey: `proxies/${assetId}.mp4`,
|
||||
});
|
||||
|
||||
console.log(`[youtube] Asset ${assetId} imported (${meta.title || 'untitled'}); proxy job queued`);
|
||||
await job.updateProgress(100);
|
||||
return { assetId, originalKey };
|
||||
} catch (error) {
|
||||
console.error(`[youtube] Import failed for asset ${assetId}:`, error.message);
|
||||
await query(
|
||||
`UPDATE assets SET status = 'error', updated_at = NOW() WHERE id = $1`,
|
||||
[assetId]
|
||||
);
|
||||
throw error;
|
||||
} finally {
|
||||
await rm(workDir, { recursive: true, force: true }).catch(() => {});
|
||||
}
|
||||
};
|
||||
Loading…
Reference in a new issue