Files
boltage/README.md
T

12 KiB

peo

neo4j-browser, cypher-shell and the Neo4j drivers on top of PostgreSQL, using Apache AGE as the graph engine.

peo is a small Python server that speaks Neo4j's Bolt protocol and translates Cypher into AGE cypher() calls. It can also serve neo4j-browser. A container image bundles everything, together with pgAdmin for working with the same data in SQL.

 neo4j-browser (static files)      peo (Python)                     PostgreSQL + AGE
 ┌──────────────────────┐  Bolt  ┌────────────────────────-──┐  SQL  ┌────────────────┐
 │ in your web browser  │───────▶│ Bolt server (TCP + WS)    │──────▶│ cypher('g',$$  │
 │ served on :7474      │ :7687  │ Cypher → AGE translation  │ :5432 │   ... $$)      │
 └──────────────────────┘        │ emulated Neo4j procedures │       └────────────────┘
                                 └─────────────────────────-─┘

Installation

peo needs Python 3.11 or later and a PostgreSQL server with the AGE extension installed and loaded (shared_preload_libraries=age).

pip install 'peo[binary]'
PEO_PG_HOST=localhost PEO_PG_DATABASE=postgres peo

The binary extra pulls in psycopg's prebuilt wheels; leave it out to use a psycopg built against your system's libpq. python -m peo works the same as peo.

Neo4j clients can now connect to neo4j://localhost:7687 with a PostgreSQL user name and password.

peo does not include neo4j-browser itself. To serve it, install the browser extra (Starlette and uvicorn), build neo4j-browser and point PEO_BROWSER_DIR at its dist directory; it is then served on http://localhost:7474/. The container images do all of this for you.

pip install 'peo[binary,browser]'
PEO_PG_HOST=localhost PEO_PG_DATABASE=postgres PEO_BROWSER_DIR=/path/to/neo4j-browser/dist peo

Configuration

peo is configured through environment variables:

Variable Default Purpose
PEO_PG_HOST 127.0.0.1 PostgreSQL host.
PEO_PG_PORT 5432 PostgreSQL port.
PEO_PG_DATABASE $POSTGRES_DB, else $POSTGRES_USER, else postgres PostgreSQL database holding the graphs.
PEO_DEFAULT_GRAPH neo4j Graph used when a client does not pick a database.
PEO_BOLT_PORT 7687 Port the Bolt server listens on (TCP and WebSocket).
PEO_HTTP_PORT 7474 Port neo4j-browser is served on, when PEO_BROWSER_DIR is set.
PEO_BROWSER_DIR (unset) neo4j-browser dist directory to serve; needs the browser extra.
PEO_ADVERTISED_BOLT_PORT $PEO_BOLT_PORT Bolt port advertised to the browser; set it when clients reach the Bolt port on another port.
PEO_ADVERTISED_HOST localhost Host reported in SHOW DATABASES and to clients that do not send a routing address.
PEO_LOG_LEVEL INFO DEBUG logs every generated SQL statement.

How it works

  • Databases are AGE graphs. Every AGE graph shows up as a Neo4j database, and :use <name> switches between them. Clients that do not pick a database get PEO_DEFAULT_GRAPH (neo4j); the full container image and the compose setup create that graph on first start. CREATE DATABASE x and DROP DATABASE x on the system database call create_graph / drop_graph; AGE requires names of at least 3 characters.
  • Queries are sent to AGE as SELECT * FROM cypher('<graph>', $$ <query> $$) AS (...). peo works out the result columns from the final RETURN clause and inlines query parameters as Cypher literals.
  • Neo4j procedures that neo4j-browser relies on (dbms.components, SHOW DATABASES, db.labels, db.schema.visualization, SHOW PROCEDURES, ...) are emulated from the AGE catalog.
  • Workarounds for AGE: neo4j-browser's "expand node" query uses a pattern comprehension, and its relationship lookup uses id(n) IN [...], which AGE rejects for lists of more than one element. peo rewrites both into equivalent Cypher that AGE accepts.
  • Authentication is passed straight through: every Bolt connection opens its own PostgreSQL connection with the credentials you log in with.

Querying the graph from SQL

AGE stores each node label and relationship type as a table in the graph's schema, with all properties in one agtype column, properties; properties ->> 'key'::text returns a property as text. For typed columns, add them yourself, either as a view or as a generated column:

CREATE VIEW neo4j.people AS
    SELECT id, properties ->> 'name'::text AS name, (properties ->> 'age'::text)::bigint AS age
    FROM neo4j."Person";

ALTER TABLE neo4j."Person"
    ADD COLUMN age bigint GENERATED ALWAYS AS ((properties ->> 'age'::text)::bigint) STORED;

PostgreSQL keeps generated columns up to date on every Cypher CREATE, MERGE, SET and REMOVE, and they can be indexed. Some things to keep in mind:

  • A value that does not fit the column's type makes the Cypher write fail: with the column above, CREATE (:Person {age: 'old'}) is rejected. A view only fails when it is read.
  • AGE does not allow dropping columns from its label tables, so a generated column cannot be removed again.
  • Adding a stored generated column rewrites the table and locks it while doing so.
  • AGE has no date or time types, so store those as ISO 8601 strings and cast them in SQL.

Container images

The Containerfile builds two images:

  • full (the default): PostgreSQL with AGE, peo, neo4j-browser and pgAdmin in one container.
  • slim (--target slim): only peo and neo4j-browser, for a PostgreSQL server with AGE that runs elsewhere. compose.yaml runs it next to separate PostgreSQL and pgAdmin containers.

Any OCI-compatible build tool works. Docker does not pick up a Containerfile by itself, so pass it explicitly: docker build -f Containerfile -t peo .. docker takes the same options as podman in the commands below.

Everything in one container

podman build -t peo .
podman run -d --name peo -e POSTGRES_PASSWORD=secret \
    -p 7474:7474 -p 7687:7687 -p 5050:5050 -p 5432:5432 \
    -v peo-data:/var/lib/postgresql -v peo-pgadmin:/var/lib/pgadmin peo
Port Service Log in with
7474 neo4j-browser: http://localhost:7474/ PostgreSQL user postgres and POSTGRES_PASSWORD
7687 Bolt, for Neo4j drivers and cypher-shell (neo4j://localhost:7687) same as above
5050 pgAdmin: http://localhost:5050/ admin@example.com and POSTGRES_PASSWORD
5432 PostgreSQL, for psql and other SQL clients same as neo4j-browser

pgAdmin comes with the bundled PostgreSQL server already registered as "peo"; it asks for the PostgreSQL password the first time you open it. AGE stores each graph as a schema of the same name, with one table per label.

Besides peo's own variables above, the image uses these:

Variable Default Purpose
POSTGRES_PASSWORD (required) Password of the PostgreSQL superuser, see the postgres image docs. POSTGRES_USER and POSTGRES_DB work too.
PEO_DEFAULT_GRAPH neo4j Also the graph created when the database is initialised.
PGADMIN_DEFAULT_EMAIL admin@example.com pgAdmin login, created on first start.
PGADMIN_DEFAULT_PASSWORD $POSTGRES_PASSWORD pgAdmin login password, set on first start.
PEO_PGADMIN true Set to false to not start pgAdmin.
PEO_PGADMIN_PORT 5050 Port pgAdmin listens on inside the container.

pgAdmin keeps its users and settings in /var/lib/pgadmin; changing PGADMIN_DEFAULT_* after the first start has no effect unless that directory is empty.

peo with a separate PostgreSQL server

The slim image runs peo and serves neo4j-browser, and is configured with the variables in Configuration. Point it at any PostgreSQL server that has AGE installed and loaded (shared_preload_libraries=age):

podman build --target slim -t peo:slim .
podman run -d --name peo -p 7474:7474 -p 7687:7687 \
    -e PEO_PG_HOST=db.example.com -e PEO_PG_DATABASE=graphs peo:slim

peo creates no graphs of its own. If the database has none yet, run CREATE DATABASE neo4j on the system database (:use system in neo4j-browser), or SELECT ag_catalog.create_graph('neo4j'); in SQL.

Compose: peo, PostgreSQL and pgAdmin as separate containers

compose.yaml builds the slim image and runs it with apache/age for PostgreSQL and the official dpage/pgadmin4 image:

echo POSTGRES_PASSWORD=secret > .env
podman compose up -d

It works with podman compose and docker compose, and publishes the same ports and logins as the full image: neo4j-browser on 7474, Bolt on 7687, pgAdmin on 5050 and PostgreSQL on 5432. Set PEO_POSTGRES_PORT to publish PostgreSQL on another host port. The default graph (PEO_DEFAULT_GRAPH, neo4j) is created when the database is initialised, and pgAdmin comes with the server already registered as "peo". PGADMIN_DEFAULT_EMAIL, PGADMIN_DEFAULT_PASSWORD and PEO_LOG_LEVEL can be set in .env too. Data is kept in the postgres-data and pgadmin-data volumes; podman compose down -v removes them.

Build arguments

AGE_IMAGE (base of the full image, default apache/age:release_PG18_1.8.0), SLIM_BASE_IMAGE (base of the slim image, default debian:trixie-slim), NEO4J_BROWSER_REF (neo4j-browser commit to build) and PGADMIN_VERSION (pgAdmin in the full image, default 9.18).

Limitations

This is a first version. Things that do not work (yet):

  • RETURN *, EXPLAIN and PROFILE.
  • Write statistics: write queries report "(no changes, no records)" even when they changed the graph.
  • Anything AGE does not support, for example multiple labels per node, pattern comprehensions, temporal and spatial types, indexes and constraints through Cypher.
  • User and role management, SHOW TRANSACTIONS, :sysinfo.
  • TLS (bolt+s://, HTTPS).
  • Each query's full result is fetched from PostgreSQL before it is streamed to the client, so very large results use memory in peo.

Development

peo depends on psycopg 3 and websockets, plus Starlette and uvicorn for the browser extra. To work on it, install it in editable mode; python -m build (or uv build) builds an sdist and wheel:

pip install -e '.[binary,browser]'

tests/smoke_test.py runs end-to-end checks through the official Neo4j Python driver against a running instance:

pip install neo4j
python tests/smoke_test.py neo4j://localhost:7687 postgres secret

License

peo is free software, licensed under the GNU General Public License v3.0 or later; see LICENSE. The container images bundle third-party software under their own licenses, including neo4j-browser (GPL-3.0), Apache AGE (Apache-2.0), PostgreSQL and pgAdmin (PostgreSQL License). See THIRD_PARTY_NOTICES.md for the full list and where each license text and source archive is in the image.

Neo4j is a trademark of Neo4j, Inc.; peo is not affiliated with or endorsed by Neo4j.