Stream results instead of fetching the entire set
This commit is contained in:
1 parent
eb989c9155
commit
dc978fd0fd
9 files changed
+324
-22
No files matched your search
@@ -103,6 +103,12 @@ variables](https://www.postgresql.org/docs/current/libpq-envars.html), such as
|
||||
- **Queries** are sent to AGE as `SELECT * FROM cypher('<graph>', $$ <query> $$) AS (...)`.
|
||||
boltage works out the result columns from the final `RETURN` clause and inlines
|
||||
query parameters as Cypher literals.
|
||||
- **Results** of read queries are streamed through a PostgreSQL cursor as clients pull
|
||||
them, so boltage holds at most a thousand rows of a result at a time. The query
|
||||
keeps a transaction open until its result has been read. Queries that can write
|
||||
(`CREATE`, `MERGE`, `SET`, `DELETE`, `REMOVE`, or `CALL` of a procedure) run to
|
||||
completion first and are buffered, because AGE performs writes while their rows are
|
||||
fetched.
|
||||
- **Neo4j procedures** that neo4j-browser relies on (`dbms.components`, `SHOW DATABASES`,
|
||||
`db.labels`, `db.schema.visualization`, `SHOW PROCEDURES`, ...) are emulated from the
|
||||
AGE catalog.
|
||||
@@ -522,8 +528,8 @@ addressing it in [`plans/`](plans/README.md):
|
||||
([plan](plans/04-users-transactions-sysinfo.md)).
|
||||
5. [TLS](#tls) certificates are only read at start-up, so renewing one needs a
|
||||
restart ([plan](plans/05-tls.md)).
|
||||
6. Each query's full result is fetched from PostgreSQL before it is streamed to the
|
||||
client, so very large results use memory in boltage
|
||||
6. The full result of a query that can write is fetched from PostgreSQL before it is
|
||||
sent to the client, so very large results of such queries use memory in boltage
|
||||
([plan](plans/06-streaming-results.md)).
|
||||
|
||||
## Development
|
||||
@@ -544,6 +550,13 @@ pip install neo4j
|
||||
python tests/smoke_test.py neo4j://localhost:7687 postgres secret
|
||||
```
|
||||
|
||||
[`tests/streaming_test.py`](tests/streaming_test.py) checks result streaming by speaking
|
||||
Bolt directly, which gives it control over `PULL`:
|
||||
|
||||
```sh
|
||||
python tests/streaming_test.py localhost 7687 postgres secret neo4j
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
boltage is free software, licensed under the GNU General Public License v3.0 or
|
||||
|
||||
Reference in new issue
Block a user