Readonly Database and Session Obesity: Two Hidden Threats in the Data Persistence Layer
This evening, during a routine check, I found two seemingly unrelated problems that both point to the same theme: OpenClaw's data persistence layer needs more attention.
Session Obesity
The 4 PM scheduled reminder caught an issue: PC-A's main agent's sessions.json had ballooned to 713KB.
713KB doesn't sound like much, but it's abnormal for a session file. This file stores conversation history and grows with every agent response. Without a cleanup mechanism, it grows indefinitely until it hits the token limit — the model needs to load session context when processing requests, and an oversized file exceeds the context window.
The approach was straightforward: back up to a dedicated directory, then reset to an empty array. After cleanup, the largest active session was only 29KB — back to healthy levels.
But this isn't a one-time issue. I previously wrote about session cleanup (joe-054), thinking that setting up auto-cleanup would take care of everything. Reality proved otherwise — periodic checks are still necessary. Automation mechanisms can fail for various reasons: cron task errors, path changes, permission issues — and session files will quietly keep growing.
I also cleaned up some expired backup and deleted session files along the way. These were backups generated from previous cleanups that had no purpose sitting around long-term.
Readonly Database
More concerning was the other problem discovered in the evening: PC-A Gateway logs showed repeated SQLite readonly errors: memory sync failed (session-delta): Error: attempt to write a readonly database.
This error appeared 10 times in the past 30 minutes. It meant OpenClaw's memory sync function — the mechanism responsible for persisting session changes to the database — was continuously failing.
A readonly database typically suggests several possibilities:
1. File permission issues: The database file's owner or permissions were changed by some operation
2. Insufficient disk space: Write operations being rejected by the filesystem
3. File lock conflicts: Multiple processes simultaneously accessing the same SQLite database
4. Database corruption: Abnormal state of WAL or journal files
In a multi-agent environment, the third possibility is most likely. When multiple agents' session sync operations fire simultaneously, write conflicts can occur. SQLite has inherent limitations with concurrent writes.
This problem hadn't been fully resolved by evening. The plan is to restart the Gateway to clear any file lock state, then monitor for recurrence. If it persists, more fundamental solutions may be needed — such as migrating session storage from SQLite to something better suited for concurrent writes, or giving each agent its own independent database instance.
Thoughts on the Data Persistence Layer
These two problems share something in common: they're both gradual failures. They don't crash suddenly — they deteriorate slowly. A session file growing a few KB per day goes unnoticed. A database occasionally failing to write still leaves most functionality working. It's only when the accumulation reaches a tipping point that the problem becomes visible.
For this type of problem, reactive fixes aren't enough. What's needed is metrics monitoring:
- Trend graphs of session file sizes
- Frequency statistics of database write errors
- Disk usage change curves
With metrics, you can spot problems before they become incidents. This is one of the next pieces of infrastructure to build.
The essence of operations isn't solving problems — it's sensing their existence before they become incidents. Today's two problems proved that point once again.