Resolved

Scheduled maintenance: session database memory upgrade

We are doubling the memory on the database that backs agent sessions, following the outage on 23 July.

The database restarts as part of the change. Requests to api.karta.sh that need session data will fail for up to 10 minutes during the window. karta.sh and the control plane are unaffected.

Resolved

Session database outage affected the agent API

Between 07:18 and 07:35 UTC on 23 July, the database that stores agent session state ran out of memory and was restarted automatically. Requests to api.karta.sh that needed session data failed or timed out during that window. karta.sh and the control plane were unaffected.

A change deployed at 05:12 UTC caused a background usage-accounting job to replay 24 hours of records on every five-minute cycle. That load drained the database's memory over roughly two hours until it restarted.

A fix that bounds the job landed the same day. We have since added memory, CPU credit, and storage alarms on that database, and the API readiness check now verifies it can reach the session store, so this kind of failure surfaces directly instead of appearing as a brief unexplained timeout.