Bolt Protocol Message Specification - Version 3
Note: Byte values are represented using hexadecimal notation unless otherwise specified.
Version 3
Deltas
The changes compared to Bolt protocol version 2 are listed below:
- The
INIT
request message has been replaced withHELLO
message. - The
ACK_FAILUER
request message has been removed. UseRESET
message instead. - Added
extra::Dictionary
field toRUN
message. - Added
extra::Dictionary
field toBEGIN
message. - New
HELLO
request message. - New
GOODBYE
request message. - New
BEGIN
request message. - New
COMMIT
request message. - New
ROLLBACK
request message. - New
RESET
request message.
Overview of Major Version Changes
Several new messages have been introduced to support new features.
The INIT
message have been replaced with the HELLO
message that have a more generic structure.
The new request message GOODBYE
have been introduced to be able to tell the server that the connection have been closed.
In Bolt protocol version 3 the concept of Auto-commit Transaction and Explicit Transaction have been introduced.
Auto-commit Transaction is the concept of being in the READY
server state and transition to the STREAMING
server state.
A new Explicit Transaction is created with the request message BEGIN
and closed with the request message COMMIT
.
See, Bolt Protocol Server State Specification Version 3
The request message ACK_FAILURE
has been removed and to reset the connection the new request message RESET
should be used.
The RESET
message will trigger an <INTERRUPT>
signal on the server.
Bolt Protocol Server State Specification
For the server, each connection using the Bolt Protocol will occupy one of several states throughout its lifetime.
This state is used to determine what actions may be undertaken by the client.
See, Bolt Protocol Server State Specification Version 3
Server Signals
Jump ahead is that the signal will imediatly be available before any messages are processed in the message queue.
Server Signal | Jump Ahead | Description |
---|---|---|
<INTERRUPT> |
x | an interrupt signal |
<DISCONNECT> |
a disconnect signal |
Protocol Errors
If a server or client receives a message type that is unexpected, according to the transitions described in this document, it must treat that as a protocol error.
Protocol errors are fatal and should immediately transition the server state to DEFUNCT
, closing any open connections.
Message Exchange
Messages are exchanged in a request-response pattern between client and server.
Each request consists of exactly one message and each response consists of zero or more detail messages followed by exactly one summary message. The presence or absence of detail messages in a response is directly related to the type of request message that has been sent. In other words, some request message types elicit a response that may contain detail messages, others do not.
Messages may also be pipelined. In other words, clients may send multiple requests eagerly without first waiting for responses.
When a failure occurs in this scenario, the server must ignore all subsequent requests until the connection from the server side is in the READY
state.
This prevents inadvertent execution of queries that may not be valid.
Serialization
Messages and their contents are serialized into network streams using PackStream Specification Version 1.
Each message is represented as a PackStream structure, that contains a fixed number of fields.
Note: conceptually BOLT messages and PackStream structures are not the same. Even though they have the same encoded form, the tags used might overlap. It is recommended to model them separately when implementing the protocol.
The message type is denoted by the PackStream structure tag byte and each message is defined in the Bolt protocol.
Serialization is specified with PackStream Version 1.
Some messages that contain Dictionary
fields, define default values for some of the keys. Bandwidth can be saved by omitting these entries if the value equals the default.
Chunking
A layer of chunking is also applied to message transmissions as a way to more predictably manage packets of data.
The chunking process allows the message to be broken into one or more pieces, each of an arbitrary size, and for those pieces to be transmitted within separate chunks.
Each chunk consists of a two-byte header, detailing the chunk size in bytes followed by the chunk data itself. Chunk headers are 16-bit unsigned integers, meaning that the maximum theoretical chunk size permitted is 65,535 bytes.
Each encoded message must be terminated with a chunk of zero size, i.e.
00 00
This is used to signal message boundaries to a receiving parties, allowing blocks of data to be fully received without requiring that the message is parsed immediately. This also allows for unknown message types to be received and handled without breaking the messaging chain.
The Bolt protocol encodes each message using a chunked transfer encoding.
- Each message is transferred as one or more chunks of data.
- Each chunk starts with a two-byte header, an unsigned big-endian 16-bit integer, representing the size of the chunk not including the header.
- A message can be divided across multiple chunks, allowing client and server alike to transfer large messages without having to determine the length of the entire message in advance.
- Chunking applies on each message individually.
- One chunk cannot contain more than one message.
- Each message ends with two bytes with the value
00 00
, these are not counted towards the chunk size. (Or you can think they are individual chunks of size 0)
This section provides some examples to illustrate how Bolt chunks messages.
Example: A message in one chunk
Message data containing 16 bytes,
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
The chunk,
00 10 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00 00
chunk header: 00 10
end marker: 00 00
Example: A message split in two chunks
Message data containig 20 bytes:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 01 02 03 04
chunk 1, chunk 2,
00 10 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00 04 01 02 03 04 00 00
chunk 1 header: 00 10
chunk 1 end marker: no end marker, still message data
chunk 2 header: 00 04
chunk 2 end marker: 00 00
Example: Two messages
Message 1 data containing 16 bytes:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
Message 2 data containing 8 bytes:
0F 0E 0D 0C 0B 0A 09 08
The two messages encoded with chunking,
00 10 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00 00
00 08 0F 0E 0D 0C 0B 0A 09 08 00 00
Pipelining
The client may send multiple requests eagerly without first waiting for responses.
Transaction
A transaction is the concept of atomic units of work.
An Auto-commit Transaction is a basic but limited form of transaction.
The concept of Auto-commit Transaction is when the server is in the READY
state and the transaction is opened with the request message RUN
and the response of a summary message SUCCESS
.
The Auto-commit Transaction is successfully closed with the summary message SUCCESS
for the request message PULL_ALL
or the request message DISCARD_ALL
.
Thus the Auto-commit Transaction can only contain one RUN
request message.
Example 1:
...
C: HELLO ...
S: SUCCESS ... // Server is in READY state
C: RUN ... // Open a new Auto-commit Transaction
S: SUCCESS ... // Server is in STREAMING state
C: PULL_ALL ...
S: RECORD ...
...
S: RECORD ...
S: SUCCESS ... // Server is in READY state and this implies that the Auto-commit Transaction is closed.
See, Bolt Protocol Server State Specification Version 3
An Explicit Transaction is a more generic transaction that can contain several RUN
request messages.
The concept of Explicit Transaction is when the server is in the READY
state and the transaction is opened with the request message BEGIN
and the response of a summary message SUCCESS
(thus transition into the TX_READY
server state).
The Explicit Transaction is successfully closed with the request message COMMIT
and the response of a summary message SUCCESS
.
The result stream (detail messages) must be fully consumed or discarded by a client before the server can transition to the TX_READY
state and thus be able to close the transaction with a COMMIT
request message.
The Explicit Transaction can be gracefully discarded and set to the initial server state of READY
with the request message ROLLBACK
.
Example 2:
...
C: HELLO ...
S: SUCCESS ... // Server is in READY state
C: BEGIN ... // Open a new Explicit Transaction
S: SUCCESS ... // Server is in TX_READY state
C: RUN ...
S: SUCCESS ... // Server is in TX_STREAMING state, first stream is open
C: PULL_ALL ...
S: RECORD ...
...
S: RECORD ...
S: SUCCESS ... // Server is in TX_READY state, first stream has been fully consumed
C: RUN ...
S: SUCCESS ... // Server is in TX_STREAMING state, second stream is open
C: PULL_ALL ...
S: RECORD ...
...
S: RECORD ...
S: SUCCESS ... // Server is in TX_READY state, second stream has been fully consumed
C: COMMIT // Close the Explicit Transaction
S: SUCCESS // Server is in READY state
See, Bolt Protocol Server State Specification Version 3
Messages
- Request Message, the client sends a message.
- Summary Message, the server will always respond with one summary message.
- Detail Message, the server will always repond with zero or more detail messages before sending a summary message.
Message | Signature | Request Message | Summary Message | Detail Message | Fields | Description |
---|---|---|---|---|---|---|
HELLO |
01 |
x | extra::Dictionary(user_agent::String, scheme::String) |
initialize connection | ||
GOODBYE |
02 |
x | close the connection, triggers a <DISCONNECT> signal |
|||
RESET |
0F |
x | reset the connection, triggers a <INTERRUPT> signal |
|||
RUN |
10 |
x | query::String , parameters::Dictionary , extra::Dictionary(bookmarks::List<String>, tx_timeout::Integer, tx_metadata::Dictionary, mode::String) |
execute a query | ||
DISCARD_ALL |
2F |
x | discard all records | |||
PULL_ALL |
3F |
x | fetch all records | |||
BEGIN |
11 |
x | extra::Dictionary(bookmarks::List<String>, tx_timeout::Integer, tx_metadata::Dictionary, mode::String) |
begin a new transaction | ||
COMMIT |
12 |
x | commit a transaction | |||
ROLLBACK |
13 |
x | rollback a transaction | |||
SUCCESS |
70 |
x | metadata::Dictionary |
request succeeded | ||
IGNORED |
7E |
x | request was ignored | |||
FAILURE |
7F |
x | metadata::Dictionary(code::String, message::String) |
request failed | ||
RECORD |
71 |
x | data::List |
data values |
Request Message - HELLO
The HELLO
message request the connection to be authorized for use with the remote database.
The server must be in the CONNECTED
state to be able to process a HELLO
message.
For any other states, receipt of an HELLO
request must be considered a protocol violation and lead to connection closure.
Clients should send HELLO
message to the server immediately after connection and process the corresponding response before using that connection in any other way.
If authentication fails, the server must respond with a FAILURE
message and immediately close the connection.
Clients wishing to retry initialization should establish a new connection.
Signature: 01
Fields:
extra::Dictionary(
user_agent::String,
scheme::String,
…
)
- The
user_agent
should conform to"Name/Version"
for example"Example/3.0.0"
. (see, developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) - The
scheme
is the authentication scheme. Predefined schemes are"none"
,"basic"
,"kerberos"
. - Further entries in
extra
are passed to the implementation of the chosen authentication scheme. Their names, types, and defaults depend on that choice.
The scheme"basic"
requires a user nameprincipal::String
and a passwordcredentials::String
.
Detail Messages:
No detail messages should be returned.
Valid Summary Messages:
SUCCESS
FAILURE
Synopsis
HELLO {user_agent::String, scheme::String}
Example:
HELLO {"user_agent": "Example/3.0.0", "scheme": "basic", "principal": "user-id-1", "credentials": "user-password"}
Server Response SUCCESS
A SUCCESS
message response indicates that the client is permitted to exchange further messages.
Servers can include metadata that describes details of the server environment and/or the connection.
The following fields are defined for inclusion in the SUCCESS
metadata.
server::String
, the server agent string. e.g."Neo4j/3.5.0"
.connection_id::String
, an unique identifier of the bolt connection used on the server side. e.g."bolt-61"
.
Example:
SUCCESS {"server": "Neo4j/3.5.0", "connection_id": "bolt-61"}
Server Response FAILURE
A FAILURE
message response indicates that the client is not permitted to exchange further messages.
Servers may choose to include metadata describing the nature of the failure but must immediately close the connection after the failure has been sent.
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Request Message - GOODBYE
The GOODBYE
message notifies the server that the connection is terminating gracefully.
On receipt of this message, the server should immediately shut down the socket on its side without sending a response.
A client may shut down the socket at any time after sending the GOODBYE
message.
This message interrupts the server current work if there is any.
Signature: 02
Fields:
No fields.
Detail Messages:
No detail messages should be returned.
Valid Summary Messages:
No summary message should be returned.
Synopsis
GOODBYE
Example:
GOODBYE
Request Message - RESET
The RESET
message requests that the connection should be set back to its initial state, as if a HELLO
message had just been successfully completed.
The RESET
message is unique in that it on arrival at the server, it jumps ahead in the message queue, stopping any unit of work that happens to be executing.
All the queued messages originally in front of the RESET
message will then be IGNORED
until the RESET
position is reached.
Then from this point, the server state is reset to a state that is ready for a new session.
Signature: 0F
Fields:
No fields.
Detail Messages:
No detail messages should be returned.
Valid Summary Messages:
No summary message should be returned.
Synopsis
RESET
Example:
RESET
Request Message - RUN
The RUN
message requests that a Cypher query is executed with a set of parameters and additional extra data.
This message could both be used in an Explicit Transaction or an Auto-commit Transaction. The transaction type is implied by the order of message sequence.
Signature: 10
Fields:
query::String,
parameters::Dictionary,
extra::Dictionary(
bookmarks::List<String>,
tx_timeout::Integer,
tx_metadata::Dictionary,
mode::String,
)
- The
query
can be a Cypher syntax or a procedure call. - The
parameters
is a dictionary of parameters to be used in thequery
string.
An Explicit Transaction (BEGIN
+RUN
) does not carry any data in the extra
field.
For Auto-commit Transaction (RUN
) the extra
field carries:
- The
bookmarks
is a list of strings containing some kind of bookmark identification e.g["neo4j-bookmark-transaction:1", "neo4j-bookmark-transaction:2"]
. Default:[]
. - The
tx_timeout
is an integer in that specifies a transaction timeout in ms. Default: server-side configured timeout. - The
tx_metadata
is a dictionary that can contain some metadata information, mainly used for logging. Default:null
. - The
mode
specifies what kind of server theRUN
message is targeting. For write access use"w"
and for read access use"r"
. Default:"w"
.
Detail Messages:
No detail messages should be returned.
Valid Summary Messages:
SUCCESS
IGNORED
FAILURE
Synopsis
RUN "query" {parameters} {extra}
Example 1:
RUN "RETURN $x AS x" {"x": 1} {bookmarks: [], "tx_timeout": 123, "tx_metadata": {"log": "example_message"}, mode: "r"}
Example 2:
RUN "RETURN $x AS x" {"x": 1} {}
Example 3:
RUN "CALL dbms.procedures()" {} {}
Server Response SUCCESS
A SUCCESS
message response indicates that the client is permitted to exchange further messages.
The following fields are defined for inclusion in the SUCCESS
metadata.
fields::List<String>
, the fields of the return result. e.g. [“name”, “age”, …]t_first::Integer
, the time, specified in ms, which the first record in the result stream is available after.
Example 1:
SUCCESS {"fields": ["x"], "t_first": 123}
Server Response IGNORED
Example:
IGNORED
Server Response FAILURE
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Request Message - DISCARD_ALL
The DISCARD_ALL
message requests that all of the result stream should be thrown away.
Signature: 2F
Fields:
No fields.
Detail Messages:
No detail messages should be returned.
Valid Summary Messages:
SUCCESS
IGNORED
FAILURE
Synopsis
DISCARD_ALL
Example 1:
DISCARD_ALL
Server Response SUCCESS
The following fields are defined for inclusion in the SUCCESS
metadata.
bookmark::String
, the bookmark after committing this transaction. (Auto-commit Transaction only).
Example:
SUCCESS {"bookmark": "example-bookmark:1"}
Server Response IGNORED
Example:
IGNORED
Server Response FAILURE
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Request Message - PULL_ALL
The PULL_ALL
message requests all data of the result stream.
Signature: 3F
Fields:
No fields.
Detail Messages:
Zero or more RECORD
Valid Summary Messages:
SUCCESS
IGNORED
FAILURE
Synopsis
PULL_ALL
Example:
PULL_ALL
Server Response SUCCESS
The following fields are defined for inclusion in the SUCCESS
metadata.
bookmark::String
, the bookmark after committing this transaction. (Autocommit Transaction only).t_last::Integer
, the time, specified in ms, which the last record in the result stream is consumed after.type::String
, the type of the statement, e.g."r"
for read-only statement,"w"
for write-only statement,"rw"
for read-and-write, and"s"
for schema only.stats::Dictionary
, counter information, such as db-hits etc.plan::Dictionary
, plan result.profile::Dictionary
, profile result.notifications::Dictionary
, any notification generated during execution of this statement.
Example:
SUCCESS {"bookmark": "example-bookmark:1", "t_last": 123}
Server Response IGNORED
Example:
IGNORED
Server Response FAILURE
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Request Message - BEGIN
The BEGIN
message request the creation of a new Explicit Transaction.
This message should then be followed by a RUN
message.
The Explicit Transaction is closed with either the COMMIT
message or ROLLBACK
message.
Signature: 11
Fields:
extra::Dictionary(
bookmarks::List<String>,
tx_timeout::Integer,
tx_metadata::Dictionary,
mode::String,
)
- The
bookmarks
is a list of strings containg some kind of bookmark identification e.g["neo4j-bookmark-transaction:1", "neo4j-bookmark-transaction:2"]
. Default:[]
. - The
tx_timeout
is an integer in that specifies a transaction timeout in ms. Default: server-side configured timeout. - The
tx_metadata
is a dictionary that can contain some metadata information, mainly used for logging. Default:null
. - The
mode
specifies what kind of server theRUN
message is targeting. For write access use"w"
and for read access use"r"
. Default:"w"
.
Detail Messages:
No detail messages.
Valid Summary Messages:
SUCCESS
FAILURE
Synopsis
BEGIN {extra}
Example 1:
BEGIN {"tx_timeout": 123, "mode": "r", "tx_metadata": {"log": "example_log_data"}}
Example 2:
BEGIN {"tx_metadata": {"log": "example_log_data"}, "bookmarks": ["example-bookmark:1", "example-bookmark2"]}
Server Response SUCCESS
Example:
SUCCESS {}
Server Response IGNORED
Example:
IGNORED
Server Response FAILURE
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Request Message - COMMIT
The COMMIT
message request that the Explicit Transaction is done.
Signature: 12
Fields:
No fields.
Detail Messages:
No detail messages.
Valid Summary Messages:
SUCCESS
IGNORED
FAILURE
Synopsis
COMMIT
Example:
COMMIT
Server Response SUCCESS
A SUCCESS
message response indicates that the Explicit Transaction was completed.
bookmark::String
, the bookmark after committing this transaction.
Example:
SUCCESS {"bookmark": "example-bookmark:1"}
Server Response IGNORED
Example:
IGNORED
Server Response FAILURE
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Request Message - ROLLBACK
The ROLLBACK
message requests that the Explicit Transaction rolls back.
Signature: 13
Fields:
No fields.
Detail Messages:
No detail messages.
Valid Summary Messages:
SUCCESS
IGNORED
FAILURE
Synopsis
ROLLBACK
Example:
ROLLBACK
Server Response SUCCESS
A SUCCESS
message response indicates that the Explicit Transaction was rolled back.
SUCCESS {}
Server Response IGNORED
Example:
IGNORED
Server Response FAILURE
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Summary Message - SUCCESS
The SUCCESS
message indicates that the corresponding request has succeeded as intended.
It may contain metadata relating to the outcome.
Metadata keys are described in the section of this document relating to the message that began the exchange.
Signature: 70
Fields:
metadata::Dictionary
Synopsis
SUCCESS {metadata}
Example:
SUCCESS {"example": "see specific message for server response metadata"}
Summary Message - IGNORED
The IGNORED
message indicates that the corresponding request has not been carried out.
Signature: 7E
Fields:
No fields
Synopsis
IGNORED
Example:
IGNORED
Summary Message - FAILURE
Signature: 7F
Fields:
metadata::Dictionary(
code::String,
message::String,
)
- The
code
is a textual code that uniquely identifies the type of failure. - The
message
is a textual description of the failure, intended for human consumption.
Synopsis
FAILURE {metadata}
Example:
FAILURE {"code": "Example.Failure.Code", "message": "example failure"}
Detail Message - RECORD
A RECORD
message carries a sequence of values corresponding to a single entry in a result.
Signature: 71
These messages are currently only ever received in response to a PULL_ALL
message and will always be followed by a summary message.
Fields:
data::List
Synopsis
RECORD [data]
Example 1:
RECORD ["1", "2", "3"]
Example 2:
RECORD [{"point": [1, 2]}, "example_data", 123]
Appendix - Message Exchange Examples
- The
C:
stands for client. - The
S:
stands for server.
Example 1
C: 60 60 B0 17
C: 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00
S: 00 00 00 03
C: HELLO {"user_agent": "Example/3.0.0", "scheme": "basic", "principal": "user", "credentials": "password"}
S: SUCCESS {"server": "Neo4j/3.5.0", "connection_id": "example-connection-id:1"}
C: GOODBYE
Example 2
C: 60 60 B0 17
C: 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00
S: 00 00 00 03
C: HELLO {"user_agent": "Example/3.0.0", "scheme": "basic", "principal": "user", "credentials": "password"}
S: SUCCESS {"server": "Neo4j/3.5.0", "connection_id": "example-connection-id:1"}
C: RUN "RETURN $x AS example" {"x": 123} {"mode": "r"}
S: SUCCESS {"fields": ["example"]}
C: PULL_ALL
S: RECORD [123]
S: SUCCESS {"bookmark": "example-bookmark:1", "t_last": 300, "type": "r"}
C: GOODBYE
Example 3
C: 60 60 B0 17
C: 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00
S: 00 00 00 03
C: HELLO {"user_agent": "Example/3.0.0", "scheme": "basic", "principal": "user", "credentials": "password"}
S: SUCCESS {"server": "Neo4j/3.5.0", "connection_id": "example-connection-id:1"}
C: RUN "RETURN $x AS example" {"x": 123} {"mode": "r"}
S: SUCCESS {"fields": ["example"]}
C: DISCARD_ALL
S: SUCCESS {"bookmark": "example-bookmark:1", "t_last": 300, "type": "r"}
C: GOODBYE
Example 4
C: 60 60 B0 17
C: 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00
S: 00 00 00 03
C: HELLO {"user_agent": "Example/3.0.0", "scheme": "basic", "principal": "user", "credentials": "password"}
S: SUCCESS {"server": "Neo4j/3.5.0", "connection_id": "example-connection-id:1"}
C: BEGIN {"mode": "r"}
S: SUCCESS {}
C: RUN "RETURN $x AS example" {"x": 123} {}
S: SUCCESS {"fields": ["example"]}
C: PULL_ALL
S: RECORD [123]
S: SUCCESS {"t_last": 300, "type": "r"}
C: COMMIT
S: SUCCESS {"bookmark": "example-bookmark:1"}
C: GOODBYE