zgaetano/ame-job-manager
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

docs: Add detailed architecture and troubleshooting guide

Browse source 
Explain the host-level SMB mounting approach:
- Why we mount SMB on the host, not in the container (security, reliability)
- Why we bind specific subdirectories instead of the root share (avoid ephemeral volumes)
- How the separation of concerns improves operations and portability

Add troubleshooting section covering:
- Files not appearing on SMB share after upload
- SMB mount failures on host
- Container startup issues
- Step-by-step verification commands

Document the .prproj path remapping rationale and AMPP proxy workflow context.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
Claude 2026-03-31 17:00:21 -04:00
parent 4f54305abf
commit 63b303fd01
1 changed files with 188 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

203
README.md
View file

@ -58,15 +58,31 @@ POLL_INTERVAL_MS=5000
JOB_TIMEOUT_MS=3600000 JOB_TIMEOUT_MS=3600000
``` ```
3. Mount your watch folder and AME log directory in `docker-compose.yml`, then start: 3. **Mount the SMB share on the Docker host** (one-time setup):
```bash
# Create mount point
sudo mkdir -p /mnt/smb-ame
# Mount the SMB share (adjust credentials/IP as needed)
sudo mount -t cifs //172.18.210.5/ame /mnt/smb-ame \
-o username=smb,password=Production2020!,uid=1000,gid=1000,file_mode=0755,dir_mode=0755,vers=3.0
# Verify mount succeeded
mount | grep cifs
```
4. Configure `docker-compose.yml` with bind-mounts to subdirectories, then start:
```bash ```bash
docker compose up -d docker compose up -d
``` ```
4. Open `http://localhost:3100` in your browser and log in. 5. Open `http://localhost:3100` in your browser and log in.
## Docker Compose Example ## Docker Compose Configuration
The key to this architecture is binding SMB subdirectories from the host into the container paths the app expects:
```yaml ```yaml
services: services:
@ -75,27 +91,52 @@ services:
ports: ports:
- "3100:3100" - "3100:3100"
volumes: volumes:
- /mnt/ame-watch:/watch # AME watch folder (SMB mount or local) # Bind SMB subdirectories from host into container
- /mnt/ame-output:/output # AME output folder # The host must have these mounted (e.g., at /mnt/smb-ame) from the SMB server
- /mnt/ame-logs:/ame-logs # AME log directory (contains AMEEncodingLog.txt) - /mnt/smb-ame/Watch:/watch # Where AME watches for new .prproj files
- job-data:/data - /mnt/smb-ame/Output:/output # Where AME writes encoded output
- /mnt/smb-ame/Logs:/ame-logs # Where AME writes AMEEncodingLog.txt
# Persistent storage for job records and session data
- app_data:/data
- upload_tmp:/tmp/uploads
env_file: .env env_file: .env
restart: unless-stopped restart: unless-stopped
volumes: volumes:
job-data: app_data:
upload_tmp:
``` ```
## SMB Network Share Configuration **Important**: The paths `/mnt/smb-ame/Watch`, `/mnt/smb-ame/Output`, and `/mnt/smb-ame/Logs` must exist on the Docker host after mounting. Create them if they don't exist:
The Docker container can automatically mount an SMB share on startup using credentials stored in the Settings GUI. This is useful when the watch folder, output folder, and logs are on a remote network share. ```bash
mkdir -p /mnt/smb-ame/{Watch,Output,Logs}
```
### How It Works ## Architecture: Docker Container with Host-Level SMB Mounting
1. The `entrypoint.sh` script reads SMB credentials from `settings.json` (persisted in `/data`) The job manager runs in Docker but needs access to SMB network shares for the watch folder, output folder, and AME logs. The architecture accomplishes this by:
2. On container startup, it mounts the SMB share at `/mnt/smb-share` using `mount -t cifs`
3. It then bind-mounts subdirectories to `/watch`, `/output`, and `/ame-logs` 1. **Host-level SMB mount**: The Docker host mounts the SMB share using native Linux `mount -t cifs`, making it available at a path like `/mnt/smb-ame`
4. The Node.js app accesses these paths as if they were local 2. **Bind-mounts from host to container**: `docker-compose.yml` binds specific subdirectories from the host into the container:
- `/mnt/smb-ame/Watch` → `/watch` (inside container)
- `/mnt/smb-ame/Output` → `/output` (inside container)
- `/mnt/smb-ame/Logs` → `/ame-logs` (inside container)
3. **Application accesses local paths**: The Node.js app reads/writes to `/watch`, `/output`, `/ame-logs` as if they were local, unaware of the SMB infrastructure
### Why This Approach?
We chose host-level SMB mounting over container-level mounting for several reasons:
- **Capability constraints**: Docker requires `SYS_ADMIN` capability and `apparmor=unconfined` to mount CIFS from within a container. This introduces security surface and may fail depending on host kernel/Docker daemon version.
- **Reliability**: SMB mounts are more stable and persistent when managed by the host OS (systemd, `/etc/fstab`) rather than ephemeral container entrypoint scripts.
- **Separation of concerns**: The container doesn't need to know or care about SMB credentials—the host handles authentication, the container just accesses mounted paths.
- **Volume flexibility**: If the SMB share is ever replaced with local storage or a different protocol, only the host mount needs to change; the container remains unaware.
### SMB Network Share Configuration
The watch folder, output folder, and logs are expected to be on a shared network location (SMB/CIFS). The Docker host must mount this share at the path specified in `docker-compose.yml` volumes.
### Configuration Steps ### Configuration Steps
@ -184,3 +225,135 @@ All endpoints except `/api/login` require an `x-session-id` header from a valid
| `POLL_INTERVAL_MS` | `5000` | How often to poll watch/output folders (ms) | | `POLL_INTERVAL_MS` | `5000` | How often to poll watch/output folders (ms) |
| `JOB_TIMEOUT_MS` | `3600000` | Time before a stuck job is marked as error (ms) | | `JOB_TIMEOUT_MS` | `3600000` | Time before a stuck job is marked as error (ms) |
| `AME_LOG_DIR` | `/ame-logs` | Directory containing `AMEEncodingLog.txt` | | `AME_LOG_DIR` | `/ame-logs` | Directory containing `AMEEncodingLog.txt` |
## Architecture Decisions
### Host-Level SMB Mounting (Final Approach)
**Decision**: Mount the SMB share on the Docker host at `/mnt/smb-ame`, then bind-mount individual subdirectories into the container.
**Why not container-level mounting?**
- Container-level CIFS mounting requires Docker `SYS_ADMIN` capability and disabling AppArmor, introducing security risks
- Host kernel version or Docker daemon configuration may not support container-level mounts
- SMB credentials in container entrypoint scripts are harder to manage and rotate
**Why not a single mount point?**
- Initial implementation mounted `/mnt/smb-ame` as `/smb-share` in the container, then created separate Docker volumes for `/watch`, `/output`, `/ame-logs`
- This caused uploaded files to go to ephemeral Docker volumes instead of the SMB share
- Fixed by binding each SMB subdirectory directly to its container path
**Final solution**:
- Host mounts SMB at `/mnt/smb-ame` (persists in `/etc/fstab` or systemd automount)
- `docker-compose.yml` specifies three bind-mounts:
```yaml
- /mnt/smb-ame/Watch:/watch
- /mnt/smb-ame/Output:/output
- /mnt/smb-ame/Logs:/ame-logs
```
- App reads/writes to `/watch`, `/output`, `/ame-logs` as local paths
- Files automatically appear on the SMB share where AME can access them
### Separation of Concerns
The Docker container intentionally does NOT handle SMB mounting. This separation ensures:
- **Security**: SMB credentials live on the host, never in container code or `.env` files
- **Reliability**: Host OS manages mount persistence; container restart doesn't affect SMB access
- **Portability**: Container works with any mounted filesystem (SMB, NFS, local, etc.) — no protocol assumptions
- **Operations**: Infrastructure team manages storage layer; application team manages app container
### Why .prproj Path Remapping?
The core value of this tool is translating proxy-resolution Premiere projects to high-resolution paths:
- **AMPP Workflow**: Editors on Grassvalley AMPP workstations edit with `.gves` proxy media for responsiveness
- **Problem**: AME on a dedicated render machine can't resolve `.gves` paths — they're relative to AMPP infrastructure
- **Solution**: Before handing the project to AME, rewrite all `.gves` references to their corresponding high-resolution UNC paths
- **Result**: AME can render the full-resolution media without the editor needing to manage two versions of the project
### Why Not Auto-Generate Remappings?
The remapping rules are configured manually in the app settings because:
- `.gves` files and their high-res equivalents may not follow a consistent naming pattern
- Different projects may use different proxy strategies
- Manual configuration is explicit and auditable — you can see what gets remapped and why
## Troubleshooting
### Files Don't Appear on SMB Share After Upload
**Symptom**: File appears in job queue but isn't visible at `//172.18.210.5/ame/Watch` on the network.
**Check 1**: Verify SMB is mounted on the host
```bash
mount | grep cifs
```
**Check 2**: Verify subdirectories exist and are accessible
```bash
ls -la /mnt/smb-ame/Watch
ls -la /mnt/smb-ame/Output
ls -la /mnt/smb-ame/Logs
```
**Check 3**: Verify container has the bind-mounts
```bash
docker inspect ame-job-manager | jq '.Mounts[] | {Source, Destination}'
```
**Check 4**: Verify file is actually written to `/watch` inside the container
```bash
docker exec ame-job-manager ls -la /watch/
```
**If file is in `/watch` inside container but not on host SMB share**: The bind-mount isn't working. Verify:
- `/mnt/smb-ame/Watch` exists on the host
- SMB is mounted (check `mount | grep cifs`)
- File permissions allow writing (`ls -la /mnt/smb-ame`)
### SMB Mount Fails on Host
**Symptom**: `mount -t cifs` command fails with "Permission denied" or "Connection refused"
**Check credentials**:
```bash
# Test with correct username/password/domain
sudo mount -t cifs //172.18.210.5/ame /mnt/smb-ame \
-o username=smb,password=YourPassword,domain=WORKGROUP
```
**Check network connectivity**:
```bash
# Can we reach the SMB server?
ping 172.18.210.5
# Can we list the share?
smbclient -L //172.18.210.5 -U smb -p YourPassword
```
**Check subdirectories exist on the share**:
```bash
smbclient //172.18.210.5/ame -U smb -p YourPassword
> ls
```
### Container Starts but No Files Appear
**Symptom**: Container is running, upload succeeds, but files don't show up anywhere.
**Check 1**: Are there any uploads at all?
```bash
docker exec ame-job-manager find /tmp/uploads -type f
```
**Check 2**: Are files being written to `/watch`?
```bash
docker exec ame-job-manager ls -la /watch/
```
**Check 3**: Check Docker logs for errors during startup
```bash
docker logs ame-job-manager
```
Look for mount errors or permission issues in the logs.