Migrations silently swallow failures — partial schema upgrades leave server running on broken DB #107

New issue

Closed

opened 2026-05-26 18:19:01 -04:00 by zgaetano · 1 comment

zgaetano commented

2026-05-26 18:19:01 -04:00

Owner

Fixed in 04ce096. Migrations now run through schema_migrations (filename + applied_at), each migration in its own transaction. Failures process.exit(1) unless MIGRATIONS_ALLOW_FAILURES=1 is set, so the orchestrator restarts instead of serving traffic on a broken schema. MIGRATIONS_FORCE=1 re-runs migrations when iterating locally.

Fixed in 04ce096. Migrations now run through `schema_migrations` (filename + applied_at), each migration in its own transaction. Failures `process.exit(1)` unless `MIGRATIONS_ALLOW_FAILURES=1` is set, so the orchestrator restarts instead of serving traffic on a broken schema. `MIGRATIONS_FORCE=1` re-runs migrations when iterating locally.

~~zgaetano referenced this issue 2026-05-26 18:22:49 -04:00~~

Tracker: 2026-05-26 audit session — 27 bugs logged (#100-#127) #128

zgaetano commented

2026-05-26 19:31:13 -04:00

Author

Owner

Fix Plan — #107 Migrations silently swallow failures

Root cause: src/index.js:105-111 wraps each migration in try/catch, console.error, continues loop. Failed migration → partial schema → API starts on broken DB.

Fix:

// src/index.js:105-111
for (const migration of migrations) {
  try {
    await migration();
  } catch (err) {
    console.error("Migration failed:", err.message);
    console.error("Aborting startup. Run migrations manually.");
    process.exit(1);
  }
}

Also add retry-with-backoff (5 tries × 2s) for loadS3ConfigFromDb() at index.js:116.

Files: src/index.js:105-120
Effort: ~30min
**Priority: P0 — data integrity

## Fix Plan — #107 Migrations silently swallow failures **Root cause:** `src/index.js:105-111` wraps each migration in try/catch, console.error, continues loop. Failed migration → partial schema → API starts on broken DB. **Fix:** ```js // src/index.js:105-111 for (const migration of migrations) { try { await migration(); } catch (err) { console.error("Migration failed:", err.message); console.error("Aborting startup. Run migrations manually."); process.exit(1); } } ``` Also add retry-with-backoff (5 tries × 2s) for `loadS3ConfigFromDb()` at `index.js:116`. **Files:** `src/index.js:105-120` **Effort:** ~30min **Priority: P0 — data integrity

zgaetano referenced this issue

2026-05-26 19:42:16 -04:00

Tracker: 2026-05-26 audit session — 27 bugs logged (#100-#127) #128

zgaetano referenced this issue from a commit

2026-05-26 22:06:25 -04:00

chore: 1.2 ship-prep sweep — close 38 issues

zgaetano closed this issue

2026-05-26 22:09:15 -04:00