The databases behind my applications are dumped nightly to network storage. The job had been running happily for weeks, the files were the right size, and I had never restored one.
So before letting a ledger with real bank connections depend on it, I did the boring exercise.
The test
Restore last night's dump into a scratch database on the same server, then compare it to the live one: row counts per table, the sum of every transaction, the number of tables. Then drop the scratch database.
Everything matched. That part was pleasant and uninformative — it is the part everybody assumes.
The part that mattered
The application stores credentials encrypted: bank access tokens, API secrets. They are ciphertext in the dump, which means restoring the file is only half the question. The other half is whether the key still opens them.
So the test also decrypts one field from the restored copy, inside the running application container, using the key from its environment. It came back as plaintext of the expected length.
That is the check worth having. A restore that produces a structurally perfect database full of unreadable secrets is not a restore — it is a very convincing decoy. And the failure mode is delayed: you discover it on the day you actually need the backup, which is by definition the worst day.
Two consequences
The key belongs somewhere other than the server. If the host is gone, the dumps are useless without it. That means a password manager, written down as a dependency of the backup rather than a detail of the application.
Test after the thing exists. The database in question was created after the previous night's run, so it was not in any dump at all. The job listed the databases it found; it found the ones that existed at 2am. Running the backup by hand, immediately, and checking the new name appeared took one minute and closed a real gap.