Stream results instead of fetching the entire set

This commit is contained in:
Bart Broere committed 2026-11-04 14:34:57 +01:00
1 parent eb989c9155
commit dc978fd0fd
9 files changed
+324 -22

No files matched your search

+15 -2
View File
@@ -103,6 +103,12 @@ variables](https://www.postgresql.org/docs/current/libpq-envars.html), such as
- **Queries** are sent to AGE as `SELECT * FROM cypher('<graph>', $$ <query> $$) AS (...)`.
boltage works out the result columns from the final `RETURN` clause and inlines
query parameters as Cypher literals.
- **Results** of read queries are streamed through a PostgreSQL cursor as clients pull
them, so boltage holds at most a thousand rows of a result at a time. The query
keeps a transaction open until its result has been read. Queries that can write
(`CREATE`, `MERGE`, `SET`, `DELETE`, `REMOVE`, or `CALL` of a procedure) run to
completion first and are buffered, because AGE performs writes while their rows are
fetched.
- **Neo4j procedures** that neo4j-browser relies on (`dbms.components`, `SHOW DATABASES`,
`db.labels`, `db.schema.visualization`, `SHOW PROCEDURES`, ...) are emulated from the
AGE catalog.
@@ -522,8 +528,8 @@ addressing it in [`plans/`](plans/README.md):
([plan](plans/04-users-transactions-sysinfo.md)).
5. [TLS](#tls) certificates are only read at start-up, so renewing one needs a
restart ([plan](plans/05-tls.md)).
6. Each query's full result is fetched from PostgreSQL before it is streamed to the
client, so very large results use memory in boltage
6. The full result of a query that can write is fetched from PostgreSQL before it is
sent to the client, so very large results of such queries use memory in boltage
([plan](plans/06-streaming-results.md)).
## Development
@@ -544,6 +550,13 @@ pip install neo4j
python tests/smoke_test.py neo4j://localhost:7687 postgres secret
```
[`tests/streaming_test.py`](tests/streaming_test.py) checks result streaming by speaking
Bolt directly, which gives it control over `PULL`:
```sh
python tests/streaming_test.py localhost 7687 postgres secret neo4j
```
## License
boltage is free software, licensed under the GNU General Public License v3.0 or