boltage/README.md

# peo4c

[neo4j-browser](https://github.com/neo4j/neo4j-browser) running on PostgreSQL, using
[Apache AGE](https://github.com/apache/age) as the graph engine.

Everything ships in one container image, together with
[pgAdmin](https://www.pgadmin.org/) for working with the same data in SQL:

```
 neo4j-browser (static files)      peo4c bridge (Python)              PostgreSQL + AGE
 ┌──────────────────────┐  Bolt   ┌──────────────────────────┐  SQL  ┌────────────────┐
 │ in your web browser  │───────▶│ Bolt server (TCP + WS)    │──────▶│ cypher('g',$$ │
 │ served on :7474      │ :7687  │ Cypher → AGE translation   │ :5432 │   ... $$)      │
 └──────────────────────┘        │ emulated Neo4j procedures │       └────────────────┘
                                 └──────────────────────────┘
```

## Quick start

```sh
podman build -t peo4c .
podman run -d --name peo4c -e POSTGRES_PASSWORD=secret \
    -p 7474:7474 -p 7687:7687 -p 5050:5050 -p 5432:5432 \
    -v peo4c-data:/var/lib/postgresql -v peo4c-pgadmin:/var/lib/pgadmin peo4c
```

| Port | Service | Log in with |
| --- | --- | --- |
| 7474 | neo4j-browser: <http://localhost:7474/> | PostgreSQL user `postgres` and `POSTGRES_PASSWORD` |
| 7687 | Bolt, for Neo4j drivers and cypher-shell (`neo4j://localhost:7687`) | same as above |
| 5050 | pgAdmin: <http://localhost:5050/> | `admin@example.com` and `POSTGRES_PASSWORD` |
| 5432 | PostgreSQL, for `psql` and other SQL clients | same as neo4j-browser |

pgAdmin comes with the bundled PostgreSQL server already registered as "peo4c";
it asks for the PostgreSQL password the first time you open it. AGE stores each
graph as a schema of the same name, with one table per label.

`docker` works as a drop-in replacement for `podman` in the commands above.

## How it works

- **Databases are AGE graphs.** Every AGE graph shows up as a Neo4j database, and
  `:use <name>` switches between them. A graph named `neo4j` is created on first start
  (`PEO4C_DEFAULT_GRAPH`). `CREATE DATABASE x` and `DROP DATABASE x` on the `system`
  database call `create_graph` / `drop_graph`; AGE requires names of at least 3 characters,
  and `peo4c` is reserved for the schema holding peo4c's SQL helpers.
- **Queries** are sent to AGE as `SELECT * FROM cypher('<graph>', $$ <query> $$) AS (...)`.
  The bridge works out the result columns from the final `RETURN` clause and inlines
  query parameters as Cypher literals.
- **Neo4j procedures** that neo4j-browser relies on (`dbms.components`, `SHOW DATABASES`,
  `db.labels`, `db.schema.visualization`, `SHOW PROCEDURES`, ...) are emulated from the
  AGE catalog.
- **Workarounds for AGE**: neo4j-browser's "expand node" query uses a pattern
  comprehension, and its relationship lookup uses `id(n) IN [...]`, which AGE rejects
  for lists of more than one element. The bridge rewrites both into equivalent Cypher
  that AGE accepts.
- **Authentication** is passed straight through: every Bolt connection opens its own
  PostgreSQL connection with the credentials you log in with.

## Querying the graph from SQL

AGE stores each node label and relationship type as a table in the graph's schema,
with all properties in one `agtype` column, `properties`. peo4c adds a generated
column for every property key it finds, so you can use plain SQL:

```sql
SELECT name, age FROM neo4j."Person" WHERE name LIKE 'A%';
CREATE INDEX ON neo4j."Person" (name);
```

AGE's Cypher writes do not fire PostgreSQL triggers, so new keys cannot be caught
the moment they are written. Instead the bridge calls
`CALL peo4c.sync_property_columns()` every few seconds
(`PEO4C_PROPERTY_COLUMNS_INTERVAL`), which you can also run yourself. It only scans
tables that had inserts or updates since its previous run, and adds all new keys of a
table with a single `ALTER TABLE`. Once a column exists, PostgreSQL keeps it in step
with every `CREATE`, `MERGE`, `SET` and `REMOVE`.

- Columns are `text`, the same as `properties->>'key'`: one key can hold values of
  different types in different rows, and a failing cast would make writes fail. Cast
  in your query, for example `age::int`. Lists and maps come out as JSON text.
- Keys that clash with an existing column (`id`, `properties`, `start_id`, `end_id`)
  get no column. Use `properties->>'id'` for those.
- Adding a column rewrites the table, which locks it for the duration. On a large
  table this blocks writes to that label for a while, but only once per new key.
- AGE does not allow dropping columns from label tables, so a column stays after its
  key disappears from the data.

## Configuration

| Variable | Default | Purpose |
| --- | --- | --- |
| `POSTGRES_PASSWORD` | (required) | Password of the PostgreSQL superuser, see the [postgres image docs](https://hub.docker.com/_/postgres). `POSTGRES_USER` and `POSTGRES_DB` work too. |
| `PEO4C_DEFAULT_GRAPH` | `neo4j` | Graph created at initialisation and used when a client does not pick a database. |
| `PEO4C_PG_DATABASE` | `$POSTGRES_DB` | PostgreSQL database holding the graphs. |
| `PEO4C_ADVERTISED_BOLT_PORT` | `7687` | Bolt port advertised to the browser; set it when you publish 7687 on another host port. |
| `PEO4C_ADVERTISED_HOST` | `localhost` | Host reported in `SHOW DATABASES` and to clients that do not send a routing address. |
| `PEO4C_LOG_LEVEL` | `INFO` | `DEBUG` logs every generated SQL statement. |
| `PEO4C_PROPERTY_COLUMNS` | `true` | Set to `false` to stop adding generated property columns. |
| `PEO4C_PROPERTY_COLUMNS_INTERVAL` | `5` | Seconds between checks for new property keys. |
| `PGADMIN_DEFAULT_EMAIL` | `admin@example.com` | pgAdmin login, created on first start. |
| `PGADMIN_DEFAULT_PASSWORD` | `$POSTGRES_PASSWORD` | pgAdmin login password, set on first start. |
| `PEO4C_PGADMIN` | `true` | Set to `false` to not start pgAdmin. |
| `PEO4C_PGADMIN_PORT` | `5050` | Port pgAdmin listens on inside the container. |

Build arguments: `AGE_IMAGE` (base image, default `apache/age:release_PG18_1.8.0`),
`NEO4J_BROWSER_REF` (neo4j-browser commit to build) and `PGADMIN_VERSION` (default `9.18`).

pgAdmin keeps its users and settings in `/var/lib/pgadmin`; changing
`PGADMIN_DEFAULT_*` after the first start has no effect unless that directory is empty.

## Limitations

This is a first version. Things that do not work (yet):

- `RETURN *`, `EXPLAIN` and `PROFILE`.
- Write statistics: write queries report "(no changes, no records)" even when they
  changed the graph.
- Anything AGE does not support, for example multiple labels per node, pattern
  comprehensions, temporal and spatial types, indexes and constraints through Cypher.
- User and role management, `SHOW TRANSACTIONS`, `:sysinfo`.
- TLS (`bolt+s://`, HTTPS).
- Each query's full result is fetched from PostgreSQL before it is streamed to the
  client, so very large results use memory in the bridge.

## Development

The bridge is a dependency-light Python package in [`bridge/peo4c`](bridge/peo4c)
(only `psycopg` 3). To run it outside the container against any AGE-enabled
PostgreSQL:

```sh
pip install 'psycopg[binary]'
cd bridge
PEO4C_PG_HOST=localhost PEO4C_PG_DATABASE=postgres PEO4C_BROWSER_DIR=/path/to/neo4j-browser/dist \
    python -m peo4c
```

[`tests/smoke_test.py`](tests/smoke_test.py) runs end-to-end checks through the official
Neo4j Python driver against a running instance:

```sh
pip install neo4j
python tests/smoke_test.py neo4j://localhost:7687 postgres secret
```

## License

peo4c is free software, licensed under the GNU General Public License v3.0 or
later; see [LICENSE](LICENSE). The container image bundles third-party software
under its own licenses, including neo4j-browser (GPL-3.0), Apache AGE
(Apache-2.0), PostgreSQL and pgAdmin (PostgreSQL License). See
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for the full list and where
each license text and source archive is in the image.

Neo4j is a trademark of Neo4j, Inc.; peo4c is not affiliated with or endorsed
by Neo4j.