A recovery checklist you can rehearse

Define acceptable loss and outage

Write down the latest acceptable recovery point and target time to restore useful service. These are application objectives, not a PrivacyNodes SLA. If losing an hour of data is unacceptable, a daily copy alone cannot meet that objective. Choose a database backup method consistent with your workload and version.

Obtain a disposable target, the application artifact and authorized recovery credentials through protected storage. Confirm who approves restoring customer data and how the test environment prevents outside traffic. Do not restore over the running production database or an existing directory.

Inventory the full application state

For the reporting API, list PostgreSQL, uploads, exports that cannot be reproduced, configuration, encryption-key references, DNS and external dependencies. Distinguish a disposable cache from a queue holding customer work that needs reconciliation. A container image describes software, not all persistent state.

Understand what the backup includes

PostgreSQL documents logical dumps, filesystem backups and continuous archiving as different strategies. A pg_dump archive covers one database; cluster-wide roles and tablespaces require separate consideration. Its client cannot dump a newer major-version server. Match tool versions and strategy to the objective; copying a live data directory is not automatically consistent.

Inspect an archive before restoring. Selecting a table with pg_restore does not automatically include all its dependencies. Its --clean option drops existing objects; a single-transaction restore cannot combine with parallel jobs. Review options for the specifically identified empty test database.

Technical reference: PostgreSQL backup and restore · PostgreSQL pg_dump · PostgreSQL pg_restore.

Check integrity and restore independently

Repository checks and functional restores answer different questions. restic's ordinary check does not read every stored data pack; check --read-data adds those reads and can consume substantial bandwidth. Neither proves the snapshot contains everything the API needs.

Select a specific snapshot and a new target directory. An unqualified latest in a shared repository may select another workload. A restore can overwrite files and interruption can leave partial results. Verify the destination first, keeping production files and the original repository untouched.

Technical reference: restic repository checks · restic restore targets.

Test the restored behavior in dependency order

Recreate the environment, restore the database and matching uploads, then start the API with staging credentials. Keep workers paused until their backlog and side effects are understood. Disable production email, payment calls and webhooks; use a test export destination and never consume the production queue.

Use a known test account to read a record, open its upload, check access restrictions and run a small new export. Verify that another test account cannot read its data. Compare the selected recovery point with the latest expected record. Record actual restoration time after the drill; do not write an invented success result in advance.

Resolve write ownership before a real cutover

During an incident, decide which system may accept writes and how later changes are reconciled. Two writable copies can diverge. A drill documents this decision without switching real traffic. Remove disposable data under your retention rules when the exercise ends.

Correct missing credentials, dependencies and validation steps after each rehearsal. Keep the runbook outside the original host and ensure another authorized maintainer can find it. The selectable backup option is separate from this application process; confirm scope and restoration procedures in service facts. Use the release record for the deployment that follows.