Write statistics

Problem

Neo4j reports what a write query changed in the stats entry of the final PULL summary (nodes-created, relationships-deleted, properties-set, ...). neo4j-browser shows these as "Created 2 nodes, set 4 properties". boltage sends no stats (BoltConnection.pull, boltage/bolt.py:327), so the browser says "(no changes, no records)". Drivers' summary.counters.contains_updates is always false, which also breaks code that checks it.

AGE's cypher() does not report counts, and AGE writes label tables without firing triggers, so triggers cannot count either.

Approach: PostgreSQL's transaction-local table statistics

pg_stat_xact_user_tables has, per table, the number of rows inserted, updated and deleted by the current transaction so far (n_tup_ins, n_tup_upd, n_tup_del). Every AGE label is a table in the graph's schema, and ag_catalog.ag_label says whether it holds vertices or edges. Snapshot those counters for the graph's schema before and after the query and take the difference.

Spike first. Confirm on the supported PostgreSQL versions (the image uses 18):
- the counters include the changes of a statement that has just finished, within the same transaction;
- a non-superuser can read them for tables they wrote to;
- how AGE's SET and REMOVE show up (an UPDATE of the row, so n_tup_upd), and DETACH DELETE (deletes on edge tables and the vertex table);
- MERGE that matches counts nothing, MERGE that creates counts inserts.
Transaction per auto-commit query. The counters reset at commit, and the connection runs in autocommit mode (boltage/backend.py:126). Wrap each auto-commit Cypher query in BEGIN / COMMIT in Session.run (see shared groundwork). Inside an explicit Bolt transaction nothing changes: the counters accumulate, and the diff per query still works.
Snapshot query. One query, run before and after:
```
SELECT l.kind, s.relname <> ALL(%(defaults)s) AS labelled,
       sum(s.n_tup_ins), sum(s.n_tup_upd), sum(s.n_tup_del)
FROM pg_stat_xact_user_tables s
JOIN ag_catalog.ag_label l ON l.relation = s.relid
JOIN ag_catalog.ag_graph g ON g.graphid = l.graph
WHERE g.name = %(graph)s
GROUP BY 1, 2
```
To avoid two extra round trips per query, send the "before" snapshot in the same pipeline as BEGIN (psycopg's conn.pipeline()), and skip both snapshots for queries that cannot write (no CREATE/MERGE/SET/DELETE/REMOVE keyword). The existing keyword check at boltage/backend.py:232 runs on the raw text, so a string literal like 'SET' counts as a write; run it on scan().masked instead. A false positive only costs the snapshots, a false negative loses the counts.

Mapping to Neo4j counters:

Neo4j counter	Source
`nodes-created` / `nodes-deleted`	inserts / deletes on vertex tables
`relationships-created` / `relationships-deleted`	inserts / deletes on edge tables
`labels-added`	inserts on vertex tables other than `_ag_label_vertex` (AGE has one label per node)
`labels-removed`	deletes on vertex tables other than `_ag_label_vertex`
`properties-set`	see below
`contains-updates`	true when any of the above is non-zero

Only send non-zero counters, like Neo4j, plus contains-updates.

properties-set has no exact source: an update rewrites a row's whole properties map. Start with an approximation and document it:
- for inserts: not counted (Neo4j counts the properties in CREATE; leaving this out under-reports, but never claims changes that did not happen);
- for updates: the number of updated rows. A more exact count would need comparing old and new properties maps, which AGE does not expose. Revisit if users ask for it.
System commands. CREATE DATABASE / DROP DATABASE (and later user management, plan 04) report system-updates: 1 and contains-system-updates: true.
Plumbing. Add a stats dict to Result, filled in Session._run_cypher and sent by BoltConnection.pull in the final SUCCESS. Use the counts to set query_type (r, w or rw, depending on whether there are columns and writes) instead of the regex.

With streaming (plan 06) the "after" snapshot must be taken once the cursor is exhausted or drained, just before the final summary.

Alternatives considered

Counting rows before and after with count(*) per label table: slow on large graphs, wrong under concurrent writers, and blind to updates.
Rewriting the query to count, for example appending a RETURN count(...): changes the query's result and does not work for queries that already return.
Statement-level triggers with transition tables: AGE does not fire triggers on its label tables.

Tests

Smoke test, using the driver's summary.counters:

CREATE (a:Person)-[:KNOWS]->(b:Person): 2 nodes, 1 relationship, 2 labels, contains_updates.
MATCH (n:Person) SET n.x = 1: properties set equals the number of persons.
MATCH (n:Person) DETACH DELETE n: nodes and relationships deleted.
MATCH (n) RETURN n: no updates.
Same queries in an explicit transaction, two queries in one transaction report their own counts.
CREATE DATABASE: system updates.

README changes

Remove the write statistics bullet; add a note that properties-set is approximate.

5.4 KiB Raw Blame History