Troubleshooting

First, ask Rowsafe what's wrong. rowsafe status lists every problem with the command to run next:

rowsafe status             # the whole fleet
rowsafe status app         # one database: why isn't it protected?
rowsafe tasks app --status failed
rowsafe task show ID       # the full log of one task

On the server, the agent's log is in sudo journalctl -u rowsafe-agent, and pgBackRest's logs are in /var/log/rowsafe/.

archive_command is already set

archive_command is already set to "...": another archiver may be running; re-run with force to replace it

Another tool archives WAL today, for example WAL-G, Barman or an older pgBackRest setup. The plan refuses to replace it.

• Find out what it is before you go on. If it is your current backup, replacing it stops it.
• If you want Rowsafe to take over: rowsafe apply app --force. Keep the old tool's existing backups until Rowsafe has proven itself.
• The same applies to archive_library is set to "...".

In Docker, a plan also refuses an archive_command that runs pgBackRest itself (for example left over from a server install), and a server install refuses a Docker spool command. --force doesn't override these. Reset the setting first, then plan again, only if archive_mode is off (otherwise see Rollback):

ALTER SYSTEM RESET archive_command; SELECT pg_reload_conf();

Restart pending

rowsafe show app says Status: awaiting PostgreSQL restart, or rowsafe verify fails with:

archive_mode is still off: restart PostgreSQL so the adopt settings take effect, then verify again

The plan was applied, but archive_mode only takes effect after a restart, and Rowsafe never restarts PostgreSQL. Restart it in a maintenance window, then verify:

sudo systemctl restart postgresql@18-main     # your cluster's unit; in Docker: docker compose restart postgres
rowsafe verify app

After 24 hours you get the awaiting_restart alert as a reminder.

Not enough disk for a drill

not enough disk for a drill: 8.1 GiB free in /var/lib/rowsafe/drills, need about 14.0 GiB

A drill restores a full copy of the database, so it needs about 1.3 × the database size + 1 GiB free. Either:

• free up or add disk on the filesystem holding /var/lib/rowsafe/drills; or
• point drills at a bigger filesystem with ROWSAFE_DRILL_DIR in /etc/rowsafe/agent.env (a short absolute path, outside /home, owned by the postgres user), then restart the agent.

Until a drill passes again, the database isn't protected: an unproven backup isn't proof.

Agent offline

You got agent_offline, or rowsafe hosts list shows the host last seen minutes ago. PostgreSQL keeps archiving, but backups, drills and monitoring have stopped.

On the server:

sudo systemctl status rowsafe-agent
sudo journalctl -u rowsafe-agent -n 50

In Docker: docker compose ps and docker compose logs rowsafe-agent.

WAL archiving failing

You got wal_archiving_failing, or the status says WAL archiving failed at .... Point-in-time recovery stops at the last archived segment, and pg_wal grows until the disk fills. Fix it soon.

On the server, find the error:

sudo -u postgres psql -Xc "select last_failed_wal, last_failed_time, last_archived_time from pg_stat_archiver"
sudo ls -t /var/log/rowsafe/ | head          # the newest pgBackRest logs
sudo tail -50 /var/log/rowsafe/app-archive-push.log

Common causes:

• The bucket keys were rotated or revoked, or the token's IP allowlist no longer matches the server. Update /etc/rowsafe/agent.env, restart the agent, then run rowsafe verify app: it rewrites the configuration archive_command uses and proves it works.
• The bucket is unreachable from the server: firewall, DNS or a provider outage. Archiving catches up by itself once it's reachable again.
• A full disk on the server.
• In Docker, the agent container is stopped, or can't reach the bucket: WAL waits in the spool. docker compose logs rowsafe-agent shows the pgBackRest error. Don't delete spooled files.

When archiving works again, the alert resolves by itself.

Other messages

Still stuck? Write to hello@rowsafe.sh with the output of rowsafe status and the failing task's ID.