Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions docs/cypher-compatibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Cypher compatibility

This note records the Cypher behavior verified for this change. It does not claim full openCypher or Neo4j
compatibility. The runtime used Java 17.0.20.1, TinkerPop 3.8.1, `org.opencypher.gremlin:translation:1.0.4`,
and RocksDB, rebased onto the Apache master that includes the Java 17 / TinkerPop 3.8.1 upgrade.

## Request forms

The endpoint is `/graphspaces/{graphspace}/graphs/{graph}/cypher`.

| Method and content type | Request | Verified behavior |
| --- | --- | --- |
| GET | `?cypher=<URL-encoded statement>` | Query-string form works: `testGet` |
| POST `application/json` | Raw Cypher text | Legacy raw-body form works: `testPost` |
| POST `text/plain` | Any body | Rejected with HTTP 415: `testRejectPlainTextPost` |
| POST `application/json` | JSON object below | Separate bindings work: `testParameters` |

```json
{
"cypher": "MATCH (n:person) WHERE n.name = $name RETURN n.name",
"parameters": {"name": "marko"}
}
```

POST bodies containing JSON arrays, strings, numbers, booleans, or null are rejected with HTTP 400.
Malformed JSON objects and trailing tokens are also request errors and do not fall back to raw Cypher.

For the JSON-object form, `cypher` must be a nonblank string and `parameters`, when present, must be an
object. Omitting `parameters` means an empty map. Missing bindings produce an execution error; an explicit
null value is accepted. Tests also cover bindings named `id` and `label`, and the fixed limit of 16
top-level parameters. This Cypher processor does not support a `maxParameters` configuration setting.
Values containing quotes and newlines remain bound data rather than changing query text.

A Map counts as one top-level parameter. Group related values into one Map and reference its fields
explicitly; the vertex label and required properties must match the graph's existing schema:

```json
{
"cypher": "CREATE (n:person {name:$props.name, age:$props.age, city:$props.city}) RETURN n.name",
"parameters": {"props": {"name": "new-person", "age": 20, "city": "Beijing"}}
}
```

Do not use the shorthand `CREATE (n:person $props)`: the pinned translator does not assign the bound
Map's properties in that path. Live REST verification confirmed that one Map containing 17 entries
works with explicit property references, with all 17 persisted values verified through native REST.
A 17-entry returned Map also succeeds, while 17 top-level bindings produce an execution error.

The translator's reserved null-marker string `" cypher.null"` (two leading spaces) is rejected in binding
values, including nested maps and lists, to avoid silently converting a user string to null. Actual JSON
null remains accepted. The same marker in query literals or stored properties is an existing translator
limitation and is outside this compatibility target.
Check the HTTP status and body `status.code`: execution failures retain HTTP 200 with `status.code` 400
and null result data.

Traversal iteration failures, including Java errors, attempt rollback on the execution thread and return
a terminal failure response. A rollback exception is attached to the original failure.

## Verified behavior

The API fixture uses a strong schema with a `SECONDARY` index on `city` and a `RANGE` index on `age`.
Acceptance tests verified:

| Area | Verified cases and test methods |
| --- | --- |
| Reads | Label scans, equality and range predicates, computed regex full-string matching, boolean combinations, directed one- and two-hop patterns, empty results: `testGet`, `testExactReadsAndPredicates`, `testComputedRegexExecutesExtensionPredicate`, `testComputedRegexRequiresWholeStringMatch`, `testRelationQuery` |
| Results | Aliases, scalar and node values, nested maps/lists, relationship ids, path shape, and null/missing-property semantics: `testReturnNodeIdAsPrimitiveValue`, `testReturnNodeDoesNotLeakInternalIdTypes`, `testReturnNestedIdDoesNotLeakInternalIdTypes`, `testReturnRelationIdDoesNotLeakInternalIdTypes`, `testReturnPathShape`, `testNullAndMissingProperty` |
| Aggregation and pagination | `DISTINCT`, `count`/`sum`/`min`/`max`/`avg`, `ORDER BY`, `SKIP`, and `LIMIT`: `testDuplicatesDistinctAndPagination`, `testAggregates` |
| Parameters | String, number, boolean, null, empty and missing bindings; quoting/newline safety: `testParameters`, `testRequestPreservesQueryAndParameterValues`, `testRequestAcceptsEmptyParameters`, `testAcceptCypherParameterNamesAndNullValues`, `testEnforceParameterCountBoundary` |
| Writes | Vertex and edge `CREATE`, property `SET`, edge and vertex `DELETE`; state checked through native REST reads: `testCreate`, `testCreateSetAndDeleteWithNativeReadback` |
| Failures and routing | Invalid syntax, request shape, binding keys, schema values, authentication, and graph routing: `testInvalidRequests`, `testRejectInvalidQuery`, `testRejectInvalidBindingShapeAndKeys`, `testAuthenticationAndGraphRouting`, `testSpecifiedGraphRouting` |
| Failed write | A rejected multi-vertex create left no residue in native reads or in 32 later query/native-read checks: `testFailedWriteDoesNotLeakIntoLaterRequests` |

Write verification describes how acceptance was checked; clients are not required to issue native REST reads
after each write. A successful Cypher response alone was not used as proof of persisted state.

## Verification record

The final verification covers the rebased tree, including non-object JSON request rejection, the
unsupported `text/plain` media type, computed regex fixture cases, and fatal-error rollback and response.
`CypherApiTest` passed all 23 cases, `CypherClientTest` all seven, and `CypherOpProcessorTest` all eight,
with no Cypher skips. The related predicate and Gremlin HTTP ownership/context tests also passed.
`AbstractRestClientTest` passed all nine cases, including charset serialization and an ASCII-default JVM.
Gremlin API tests passed ten cases with one existing non-shared-backend skip; Login API tests passed all
three. EditorConfig formatting, the root clean compile, and reactor installation passed on Java 17.
The live RocksDB server reported Gremlin 3.8.1; its packaged API JAR and compiled target JAR were
byte-identical before the API run.

Test sources: [`CypherApiTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/CypherApiTest.java),
[`CypherClientTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/cypher/CypherClientTest.java),
and [`CypherOpProcessorTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/opencypher/CypherOpProcessorTest.java).

## Outside this compatibility target

Advanced constructs including `MERGE`, `OPTIONAL MATCH`, `WITH`, `UNWIND`, `UNION`, and variable-length paths,
as well as HStore, Bolt, cross-request transactions, and performance, were not verified by this change.
13 changes: 13 additions & 0 deletions docs/upgrade-tinkerpop-3.8.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,19 @@ The response has rows under `result.data`; each row is a map and `nodeId` is a
JSON scalar, not a HugeGraph internal ID class. The Cypher API test covers this
shape, nested IDs and paths.

## HTTP request errors

The supplied WebSocket/HTTP channelizer releases each HTTP request buffer once,
including when a traversal alias is unknown. Such a request returns one HTTP 400,
and a later valid request on the same keep-alive connection receives its own
response. This avoids TinkerPop 3.8.1's duplicate release and second HTTP 500,
which could be mistaken for the next request's response by a connection pool.
The correction applies when the configured channelizer is TinkerPop's standard
`WsAndHttpChannelizer`; HugeGraph selects its compatibility implementation at
startup and retains TinkerPop's WebSocket handling. Custom channelizer classes
keep their configured behavior. Unknown aliases still require the caller to
use a graph or traversal-source name exposed by the server.

## Compatibility verification

The repository's checked-in tests establish the following boundaries:
Expand Down
4 changes: 4 additions & 0 deletions hugegraph-commons/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ numeric or collection util classes to simplify the development of HugeGraph and
- Util: performance analyzer, version checker, numeric and Collection utils, log and exception utils, etc.
- Rpc: rpc component for inner module communication, currently it's based on [Sofa-RPC](https://github.com/sofastack/sofa-rpc)

The REST client serializes object bodies as JSON for `application/json`, including media types
with parameters such as `application/json; charset=UTF-16`. Request bodies use the declared
charset, or UTF-8 when none is specified; gzip preserves these encoded bytes.

You could use import the dependencies in `maven` like this:

```xml
Expand Down
6 changes: 6 additions & 0 deletions hugegraph-commons/hugegraph-common/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ hugegraph-common encapsulates locks, configurations, events, iterators, rest and
numeric or collection util classes to simplify the development of HugeGraph and
its components.

## REST request encoding

The shared REST client encodes request bodies using the charset declared in
`Content-Type`, or UTF-8 when no charset is declared. JSON and text requests do
not depend on the JVM's default charset. Gzip compresses those same encoded bytes.

## Components

- Lock: atomic lock, key lock, lock group and lock manager
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,9 +125,10 @@ public AbstractRestClient(String url, RestClientConfig config) {
}

private static RequestBody buildRequestBody(Object body, RestHeaders headers) {
String contentType = parseContentType(headers);
MediaType contentType = MediaType.parse(parseContentType(headers));
String bodyContent;
if (RestHeaders.APPLICATION_JSON.equals(contentType)) {
if (contentType != null && "application".equals(contentType.type()) &&
"json".equals(contentType.subtype())) {
if (body == null) {
bodyContent = "{}";
} else if (body instanceof String) {
Expand All @@ -138,8 +139,7 @@ private static RequestBody buildRequestBody(Object body, RestHeaders headers) {
} else {
bodyContent = String.valueOf(body);
}
RequestBody requestBody = RequestBody.create(bodyContent,
MediaType.parse(contentType));
RequestBody requestBody = RequestBody.create(bodyContent, contentType);

if (headers != null &&
"gzip".equals(headers.get(RestHeaders.CONTENT_ENCODING))) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ public static void main(String[] args) throws IOException {
tests.testJsonRequestBodyUsesUtf8();
tests.testGzipRequestBodyUsesUtf8();
tests.testRequestBodyRespectsExplicitCharset();
tests.testJsonObjectRequestBodyRespectsExplicitCharset();
}
}

Expand Down Expand Up @@ -102,21 +103,39 @@ public void testRequestBodyRespectsExplicitCharset() throws IOException {
buffer.readByteArray());
}

@Test
public void testJsonObjectRequestBodyRespectsExplicitCharset() throws IOException {
String value = "\u4f60\u597d\ud83d\ude80";
String json = "{\"id\":\"" + value + "\"}";
for (Charset charset : new Charset[]{StandardCharsets.UTF_8, StandardCharsets.UTF_16}) {
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_TYPE,
"application/json; charset=" + charset.name());
assertBody(Collections.singletonMap("id", value), headers, json, charset, false);
headers.add(RestHeaders.CONTENT_ENCODING, "gzip");
assertBody(Collections.singletonMap("id", value), headers, json, charset, true);
}
}

private static void assertUtf8Body(Object value, RestHeaders headers,
String expected, boolean gzip) throws IOException {
assertBody(value, headers, expected, StandardCharsets.UTF_8, gzip);
}

private static void assertBody(Object value, RestHeaders headers, String expected,
Charset charset, boolean gzip) throws IOException {
RequestBody body = requestBody(value, headers);
Assert.assertEquals(StandardCharsets.UTF_8, body.contentType().charset());
Assert.assertEquals(charset, body.contentType().charset());
Buffer buffer = new Buffer();
body.writeTo(buffer);
if (gzip) {
try (GzipSource source = new GzipSource(buffer)) {
Buffer decoded = new Buffer();
decoded.writeAll(source);
Assert.assertArrayEquals(expected.getBytes(StandardCharsets.UTF_8),
Assert.assertArrayEquals(expected.getBytes(charset),
decoded.readByteArray());
}
} else {
Assert.assertArrayEquals(expected.getBytes(StandardCharsets.UTF_8),
Assert.assertArrayEquals(expected.getBytes(charset),
buffer.readByteArray());
}
}
Expand Down
2 changes: 2 additions & 0 deletions hugegraph-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ HugeGraph Server consists of two layers of functionality: the graph engine layer

- Graph Engine Layer:
- REST Server: Provides a RESTful API for querying graph/schema information, supports the [Gremlin](https://tinkerpop.apache.org/gremlin.html) and [Cypher](https://en.wikipedia.org/wiki/Cypher) query languages, and offers APIs for service monitoring and operations.
- See [Cypher compatibility notes](../docs/cypher-compatibility.md) for request forms, verified behavior, and
unverified scope.
- Graph Engine: Supports both OLTP and OLAP graph computation types, with OLTP implementing the [Apache TinkerPop3](https://tinkerpop.apache.org) framework.
- Backend Interface: Implements the storage of graph data to the backend.

Expand Down
Loading
Loading