dragonflight/services/premiere-plugin/docs/TESTING_ADVANCED_FEATURES.md

End-to-End Testing: Advanced Features

Overview

This document covers the comprehensive testing plan for the FCP XML Export & Conform and Hi-Res Auto-Relink features.

Prerequisites

• Running MAM API server (services/mam-api)
• Running Worker service (services/worker)
• Redis instance (for BullMQ)
• S3-compatible storage
• Premiere Pro 2024 (Windows 10/11 or macOS 13+)
• Premiere Pro CEP panel installed and connected to the MAM server
• At least one project with assets in the MAM
• Assets should have hi-res originals uploaded to S3

---

FCP XML Export & Conform

Test 1: Simple Timeline Export

1. Create a new sequence in Premiere with a single video track and 3 clips
2. Each clip should be imported through the MAM plugin (to create the asset mapping)
3. Click Advanced > Export & Conform Timeline
4. Select the "Web Delivery" preset
5. Click Start Conform
6. Expected: Slide panel closes, progress bar shows job status
7. Monitor the job via the Jobs page in the web UI
8. Expected: Job transitions through parsing, processing, encoding, uploading
9. Expected: New asset appears in the grid with name "Conformed: [Sequence Name]"
10. Import the conformed asset and verify playback

Test 2: Complex Timeline (Multi-Track, Transitions)

1. Create a sequence with 3 video tracks and 6+ clips total
2. Include video transitions (cross-dissolve, dip to color)
3. Some clips should have speed/duration changes
4. Export and conform using "Broadcast Quality" preset (ProRes)
5. Expected: All clips are processed; transitions are rendered into the output
6. Expected: Output codec matches ProRes, high quality

Test 3: All Codec Presets

Preset	Codec	Expected Extension
Broadcast Quality	ProRes	.mov
Web Delivery	H.264	.mp4
Archive	H.265/HEVC	.mp4
Custom (H.264)	H.264	.mp4
Custom (ProRes)	ProRes	.mov

1. Run conform job for each preset
2. Verify the output codec via ffprobe or Premiere's project panel

Test 4: All Resolution Options

1. Test with "Match Sequence" - output should match the sequence resolution
2. Test "4K" - output should be 3840x2160
3. Test "1080p" - output should be 1920x1080
4. Test "720p" - output should be 1280x720

Test 5: Frame Rate Handling

Test with sequences at each frame rate:

Frame Rate	Type	Notes
23.976 fps	Non-drop	Common for cinema
24 fps	Non-drop	Film standard
25 fps	Non-drop	PAL standard
29.97 fps	Drop-frame	NTSC standard
30 fps	Non-drop	Progressive
59.94 fps	Drop-frame	High-frame rate NTSC
60 fps	Non-drop	High-frame rate progressive

Verify:

• Timecodes are correct in output
• No frame rate mismatch warnings
• Audio sync is maintained

Test 6: Job Cancellation

1. Start a conform job with a long timeline (5+ minutes)
2. Before it completes, navigate to the Jobs page
3. Cancel the job
4. Expected: Job status changes to "cancelled" or "failed"
5. Expected: No orphaned partial output in S3

Test 7: Network Failure During Upload

1. Start a conform job
2. Disconnect the network while the FCP XML is uploading
3. Expected: Error message displayed in the plugin
4. Reconnect and retry the job
5. Expected: Job completes on retry

Test 8: Server Failure During Conform

1. Start a conform job
2. Restart the worker service while the job is active
3. Expected: Job is stalled then moved back to "waiting" by BullMQ's stall detection
4. Expected: Worker picks it up on restart and completes it

Test 9: Verify Conformed Asset in MAM

1. After conform completes, locate the new asset in the MAM grid
2. Verify metadata: correct codec, resolution, frame rate, duration
3. Verify the conformed asset has the conform_source_sequence_id set
4. Import the asset into a new Premiere sequence
5. Verify playback and audio sync

Test 10: Re-Importing Conformed Asset

1. After conform, import the conformed asset into a new Premiere project
2. Expected: Asset imports and plays correctly
3. Verify the asset shows up in the imported assets list

---

Hi-Res Auto-Relink

Test 11: Relink Single Clip

1. Have a sequence with 1 clip that was imported via MAM proxy workflow
2. Click Advanced > Fetch & Relink All
3. Expected: Clip list shows 1 clip with checkbox checked
4. Click Fetch & Relink
5. Expected: Server trims the hi-res original to match the timeline clip
6. Expected: Clip is relinked in Premiere
7. Expected: Success summary shows 1 clip relinked

Test 12: Relink Multiple Clips (5-10)

1. Create a sequence with 5-10 clips from various assets
2. Run the relink workflow
3. Expected: All clips are processed in parallel (concurrency up to 4)
4. Expected: All clips are relinked successfully

Test 13: Same Asset, Multiple Instances

1. Create a sequence that uses the same source asset in 3 different positions
2. Each instance should have different in/out points (different trim ranges)
3. Run the relink workflow
4. Expected: Each instance gets a separate trimmed segment (different clipInstanceId)
5. Expected: Each instance is individually relinked to its trimmed segment

Test 14: Various Codecs

Test with source clips in each codec:

Codec	Format	Expected
ProRes 422	.mov	Segment trims correctly
ProRes 4444	.mov	Alpha channel preserved (if applicable)
DNxHR HQ	.mxf	Segment trims correctly
H.264	.mp4	Segment trims correctly
HEVC/H.265	.mp4	Segment trims correctly
XDCAM	.mxf	Segment trims correctly

Test 15: Selective Relink (Deselect Some Clips)

1. Create a sequence with 5 clips
2. Open the clip list and uncheck 2 clips
3. Click Fetch & Relink
4. Expected: Only the 3 checked clips are relinked
5. Expected: The 2 unchecked clips remain as proxies

Test 16: Retry Logic for Failed Relinks

1. Simulate a network error during segment download (disconnect Wi-Fi)
2. Start the relink workflow
3. Expected: Failed clip shows "retry" in the summary
4. Reconnect and retry
5. Expected: Failed clips are retried and succeed on second attempt

Test 17: Locked Premiere Project

1. With a collaborator's project open (if available), or a read-only project
2. Run the relink workflow
3. Expected: Appropriate error message about project being locked
4. Expected: Plugin does not crash

Test 18: Missing Hi-Res Originals

1. Create a sequence with a clip whose hi-res original has been deleted from S3
2. Run the relink workflow
3. Expected: The worker reports "Asset not found" for that clip
4. Expected: Summary shows that clip as failed with appropriate error

Test 19: Network Failure During Download

1. Start the relink workflow with 3+ clips
2. Disconnect the network during the download phase
3. Expected: Download fails with network error
4. Expected: Summary shows failed clips with "download error" message
5. Reconnect and retry
6. Expected: Succeeds on retry

Test 20: Temp Segment Cleanup After 24 Hours

1. Complete a relink workflow (creates temp segments)
2. Verify the temp segment records exist in the database
3. Fast-forward the system clock by 25 hours (or set expires_at to the past)
4. Wait for the cleanup task to run (within 1 hour)
5. Expected: Temp segments are deleted from S3 and database
6. Expected: Cleanup log shows "X deleted, 0 errors"

---

GUI Redesign

Test 21: Visual Consistency with Web-UI

1. Open the Premiere plugin panel
2. Compare colors with the web-ui (open in browser)
3. Verify OKLCH color tokens match:
   • Background surfaces (bg-panel, bg-surface, bg-raised)
   • Text hierarchy (primary, secondary, tertiary)
   • Accent color is the same blue-purple hue
   • Status colors (green, red, yellow) match
4. Expected: Consistent visual identity between web and Premiere panel

Test 22: Responsive Layout

1. Dock the panel at various widths (200px, 400px, 600px)
2. Verify the layout adjusts:
   • Asset grid columns adapt to available width
   • Buttons remain visible at all widths
   • Details panel is usable at all widths
   • Slide panels render correctly

Test 23: Color Contrast (WCAG AA)

1. Measure contrast ratios:
   • Text primary on bg-panel: should be ≥ 4.5:1
   • Text secondary on bg-panel: should be ≥ 3:1
   • Accent text on accent background: should be ≥ 3:1
2. Expected: All text meets WCAG AA contrast requirements

Test 24: Font Rendering

1. Verify body text renders in Inter (or system fallback)
2. Verify metadata and technical values render in JetBrains Mono
3. Check font loading from web-ui server (not bundled):
   • Look for 404s in the console
   • Verify fonts load after server connection is established

Test 25: Button States

Test each button for all states:

State	Visual Indicator
Default	Normal styling
Hover	Background lightens, subtle lift
Active/Pressed	Returns to default position
Disabled	38% opacity, no pointer events

Test 26: Slide Panel Animations

1. Open the Export & Conform slide panel
2. Expected: Panel slides in from the right (translateX animation)
3. Close the panel
4. Expected: Panel slides out to the right
5. Test the same with the Relink clip list panel
6. Expected: Smooth 250ms ease-out transitions

Test 27: Progress Indicators

1. Start any long-running operation (conform, relink)
2. Expected: Progress bar appears with percentage
3. Expected: Progress bar fills smoothly
4. Expected: Status label updates with current operation
5. Expected: Progress bar disappears on completion

Test 28: Toast Notifications

1. Perform actions that trigger messages:
   • Connect to server → "Connected..." (success)
   • Failed connection → "Failed to connect" (error)
   • Import complete → "Imported" (success)
   • Relink summary → shows success/failure counts (info)
2. Expected: Messages appear at the top of the panel
3. Expected: Messages auto-dismiss after 5 seconds
4. Expected: Success = green, Error = red, Info = accent blue

---

Performance Testing

Test 29: Conform Job with 30-Minute Timeline

1. Create a sequence with 30+ minutes of clips
2. Run the conform workflow with H.264 Web Delivery preset
3. Measure:
   • XML generation time (< 2 seconds)
   • Upload time (depends on network)
   • Server parsing time (< 5 seconds)
   • FFmpeg transcode time (varies by server GPU/CPU)
   • Total time
4. Expected: Completes within reasonable time (under 30 minutes on GPU-enabled server)

Test 30: Relink 50+ Clips Simultaneously

1. Create a sequence with 50+ clips (can duplicate assets)
2. Run the relink workflow
3. Measure:
   • Timeline analysis time
   • Server trim queue time
   • Total download time
4. Expected: All clips processed within reasonable time
5. Expected: No timeout errors

Test 31: Multiple Concurrent Conform Jobs

1. Start 3 conform jobs in rapid succession
2. Expected: Jobs are queued FIFO
3. Expected: Only 1 job is active at a time (concurrency: 1)
4. Expected: Queued jobs show "waiting" status
5. Expected: Each job completes in turn without data corruption

Test 32: Server Load During Peak Usage

1. While a conform job is running, perform other operations:
   • Browse assets in the web UI
   • Upload new files
   • Search the asset catalog
2. Expected: API remains responsive (under 500ms response time)
3. Expected: Conform job continues without error

Test 33: Premiere Pro Responsiveness

1. Start a relink workflow for 10 clips
2. During the operation, interact with Premiere:
   • Scrub the timeline
   • Open other panels
   • Play back the sequence
3. Expected: Premiere remains responsive (no freezing)

---

Test Environments

Environment	OS	Premiere Version	Notes
Workstation A	Windows 11	Premiere Pro 2024 (v24.x)	Primary test environment
Workstation B	Windows 10	Premiere Pro 2024 (v24.x)	Legacy Windows
Workstation C	macOS 14 Sonoma	Premiere Pro 2024 (v24.x)	macOS testing
Workstation D	macOS 13 Ventura	Premiere Pro 2024 (v24.x)	Legacy macOS

Network Profiles

Profile	Bandwidth	Latency	Notes
Local	1 Gbps	< 1ms	Server on same LAN
Fast WAN	100 Mbps	10ms	Remote server
Slow WAN	10 Mbps	50ms	VPN or limited connection

---

Acceptance Criteria

• All test scenarios pass
• No crashes or data loss in any tested configuration
• Clear error messages displayed for all failure modes
• Performance meets expectations across all tested environments
• UI matches the Wild Dragon design system specification
• Font loading from web-ui server works (no bundled fonts)
• Temp segment cleanup runs and removes expired data
• Slide panel animations are smooth and meet motion specs
• All button states render correctly
• Color contrast meets WCAG AA standards