Skip to content
Rowsafe
Docs

Logs: your database's own log in Pulse

Your database's log in the dashboard, with values removed on your server, live, grouped and forwardable.

Logs shows your database's own log in the dashboard within seconds. Values are removed before anything leaves your server. Open a database, then Pulse → Logs.

Nothing to install, on every plan. Turn it off per database or per server. Examples are PostgreSQL's; see other databases.

What you can do

  • Filter: errors, slow queries, lock waits, failed logins, checkpoints, vacuum.
  • Search messages, statements, users and addresses.
  • Group repeats: "duplicate key value violates unique constraint "customers_email_key"", 1,204 times since 14:00. Different values don't split a group; a different table or constraint does.
  • Watch live, and pause any time.
  • Open a line for its detail, hint, statement, who ran it and the error code.

Pulse also raises health findings from the log:

FindingWhen
Errors jumped50 or more errors in the last two hours, and at least five times the usual rate
Failed logins from an address20 or more failed logins in an hour from one address (or 50 from all). Names the address and user names tried, and links to limiting who can connect
Apps were turned away"too many clients" in the last hour
Checkpoints come too oftenPostgreSQL's own warning, three times or more in 24 hours
DeadlocksThe statements of the newest deadlock (values removed)
PostgreSQL's log leaves out slow queries and lock waitsLogging is too quiet; the fix below turns it on
Rowsafe can't read this database's logSee when Rowsafe can't read the log

Turn on useful logging

PostgreSQL's defaults don't log slow statements, lock waits or disk spills. When that's the case, click Turn on useful logging. Rowsafe sets these with ALTER SYSTEM and reloads. Nothing restarts.

SettingBecomesSo the log shows
log_min_duration_statement1000 (1 second)slow statements
log_lock_waitsonqueries waiting more than deadlock_timeout (1 second) for a lock
log_temp_files10240 (10 MB)queries spilling to disk
log_autovacuum_min_duration60000 (1 minute)automatic vacuums that take a while
log_checkpointsoncheckpoints (on by default since PostgreSQL 15)
log_line_prefix%m [%p] %q%u@%d app=%a client=%h code=%e who and where each line comes from

It only ever adds detail. log_line_prefix changes only if it lacks time, process, user, database or client (plain-text logs only); tell tools like pgBadger the new prefix. log_connections, log_statement and restart-only settings are never touched.

Settings: what leaves your server

Log settings on the Logs page (owners and admins; audited):

  • Send logs to Rowsafe (on by default). Off stops reading the log; sent lines stay until they expire.
  • Send full query text (off by default). Keeps values in statements and messages, still without passwords. Only if those values aren't personal data, or your agreements allow it.

Each line carries its time, severity, kind, error code, user, database, app, client address, process ID and duration, plus the message and statement with values removed:

  • Statements are normalized like pg_stat_statements (constants become $1, $2; comments go), wherever they appear.
  • Quoted values in messages become …: Key (email)=(…) already exists. Table, column, constraint and role names stay.
  • Bind parameters are dropped.
  • Your functions' messages (RAISE) keep only their words.
  • Passwords never leave. Anything that may hold one (ALTER ROLE ... PASSWORD, password=, user mappings, pgp_sym_encrypt, crypt) becomes <redacted: the statement may contain a password>, even with full query text. Rowsafe checks again on arrival.

Redaction recognizes PostgreSQL's English messages. With another lc_messages language, statements are still normalized and bind parameters dropped, but values in translated messages may stay. Set lc_messages = 'C' if that matters. Extensions' own messages can't be redacted reliably either.

To turn logs off for a whole server: ROWSAFE_LOGS=false in /etc/rowsafe/agent.env, then sudo systemctl restart rowsafe-agent.

How Rowsafe finds the log

The agent checks PostgreSQL's logging settings every minute and follows rotation:

  • logging_collector = on: the current file in log_directory, preferring jsonlog, then csvlog, then plain text.
  • Debian and Ubuntu: /var/log/postgresql/postgresql-<version>-<cluster>.log.
  • systemd journal: via journalctl.

First read sends the last 64 KB; then every new line. It resumes where it stopped after a restart (a line or two can be lost in a crash).

When Rowsafe can't read the log

The Logs page says why. These need a change on your server:

  • Docker (agent as a sidecar): the image logs to container output, which the sidecar can't read. Write log files into the data directory instead, then recreate the container:
    services:
      postgres:
        command: ["postgres", "-c", "logging_collector=on", "-c", "log_directory=log"]
  • systemd journal: let postgres read it: sudo usermod -aG systemd-journal postgres && sudo systemctl restart rowsafe-agent.
  • syslog only: add a file with ALTER SYSTEM SET log_destination = 'stderr,syslog'; SELECT pg_reload_conf();.
  • Output thrown away or unknown: ALTER SYSTEM SET logging_collector = on;, then restart PostgreSQL when it suits you.

Busy logs, retention and limits

Up to 50 lines a second per database on average (bursts of a few thousand are fine); busier logs are sampled. Messages and statements are cut at 4,000 characters.

PlanKept forLines kept per day (organization)
Free7 days50,000
Pro30 days2,000,000
Business90 days10,000,000

Past the daily limit, new lines aren't kept until midnight UTC.

Forward logs to your tools

Settings → Log forwarding sends kept, redacted lines to your tools within seconds. Pick the kinds of lines and the databases for each destination. Send a test sends "Test from Rowsafe".

DestinationWhat you enterHow Rowsafe sends
DatadogSite (e.g. datadoghq.eu) and an API keyLogs intake API v2, ddsource:postgresql, tagged source:rowsafe, database:<name>, kind:<kind>
Better StackSource token, and the ingesting host if your source has oneJSON over HTTPS
PapertrailHost and port of a log destinationSyslog (RFC 5424) over TLS
Grafana LokiLoki URL; for Grafana Cloud, the user (instance ID) and an API tokenPush API, labels source, host, database, level, kind
Elasticsearch / OpenSearchCluster URL, index or data stream (default rowsafe-logs), an API key, or a user name and passwordBulk API (create)
HTTPS webhookURLJSON batches, signed; see the reference
Syslog over TLSHost and portRFC 5424 with octet counting (RFC 5425), facility local0
  • Credentials are stored encrypted (AES-256-GCM); only the last four characters are shown.
  • New destinations get lines from now on, not the past.
  • Failures are retried with growing waits (up to an hour); the page shows the last error.
  • Public HTTPS or TLS addresses only. Up to 10 destinations.

MySQL, MariaDB, MongoDB, ClickHouse, Redis and Valkey

Same filters, groups, live view, alerts and forwarding.

EngineWhat is readWhere the agent finds it
MySQL, MariaDBThe error log (errors, warnings, failed logins, deadlocks, restarts) and, when it is on, the slow query log (each slow statement with its time and rows examined)log_error and slow_query_log_file
MongoDB 4.4+The structured log: errors, slow operations (their shape, values replaced by ?), failed logins, connections, restartssystemLog.path
ClickHouseThe error log: errors and warnings, with the query that failedlogger.errorlog in its configuration
Redis, ValkeyThe server's log: errors and warnings, saves to disk (and failed ones), replication, restarts, a crash (only that it crashed and why). Slow commands aren't in this log: Pulse shows the slow log (command names only) in Monitoringlogfile (relative to dir)
  • MySQL and MariaDB: Turn on the slow query log (Pulse), no restart.
  • MongoDB slow operations are always shapes without values, even with full query text on.
  • If a log can't be read, the Logs page says why and what to change.

Redis and Valkey

  • Native install: Debian and Ubuntu packages log to /var/log/redis/ (or /var/log/valkey/), readable only by adm. The installer adds a read-only access rule for the agent. A log directly in a shared folder like /var/log is left alone; the page says how to allow it.
  • logfile empty: set logfile in redis.conf (or valkey.conf), then restart the server when it suits you.
  • Docker: add --logfile /data/redis.log to the server's command, then recreate the container. docker logs is then empty, and Docker doesn't rotate the file.
  • Removed: client ports and quoted text. Lua redis.log() and module lines keep only plain words. A crash report stays on the server; Rowsafe gets only that it crashed and why.
  • No Failed logins filter: Redis keeps those in ACL LOG, not the log.

On a server on its own

On a server installed without the dashboard, the agent keeps the log on the server: sudo rowsafe logs shop --errors, and sudo rowsafe logs forward to send it to your tools. See Rowsafe on your server.

Limits

  • MongoDB before 4.4 (plain-text log) isn't read.
  • Valkey with log-format logfmt isn't read; the default (legacy) is.
  • Classification and redaction rely on English lc_messages.
  • PostgreSQL logs are read as postgres; a log or journal it can't read needs the change above.
  • Lines appear within about 5 to 10 seconds; the live view checks every 3 seconds.
Edit on GitHub