=====================
MongoDB Wire Protocol
=====================
.. default-domain:: mongodb
.. include:: /includes/note-legacy.rst
Introduction
------------
The MongoDB Wire Protocol is a simple socket-based, request-response
style protocol. Clients communicate with the database server through a
regular TCP/IP socket.
.. important:: Default Socket Port
The default port is 27017, but this is configurable and will vary.
Clients should connect to the database with a regular TCP/IP socket.
Currently, there is no connection handshake.
.. important:: To describe the message structure, a C-like ``struct`` is
used. The types used in this document (``cstring``, ``int32``, etc.)
are the same as those defined in the
`BSON specification <http://bsonspec.org/#/specification>`_.
The standard message header is typed as ``MsgHeader``. Integer constants
are in capitals (e.g. ``ZERO`` for the integer value of 0).
In the case where more than one of something is possible (like in a
:ref:`OP_INSERT <wire-op-insert>` or :ref:`OP_KILL_CURSORS
<wire-op-kill-cursors>`), we again use the notation from the
`BSON specification <http://bsonspec.org/#/specification>`_.
(e.g. ``int64*``). This simply indicates that one or more
of the specified type can be written to the socket, one after
another.
.. admonition:: Byte Ordering
Like BSON documents, all data in the MongoDB wire protocol is little-endian.
Messages Types and Formats
--------------------------
There are two types of messages, client requests and database responses,
each having a slightly different structure.
Standard Message Header
~~~~~~~~~~~~~~~~~~~~~~~
In general, each message consists of a standard message header followed
by request-specific data. The standard message header is structured as
follows:
.. code-block:: sh
struct MsgHeader {
int32 messageLength; // total message size, including this
int32 requestID; // identifier for this message
int32 responseTo; // requestID from the original request
// (used in reponses from db)
int32 opCode; // request type - see table below
}
**messageLength**: This is the total size of the message in bytes. This
total includes the 4 bytes that holds the message length.
**requestID**: This is a client or database-generated identifier that
uniquely identifies this message. For the case of client-generated
messages (e.g. :ref:`OP_QUERY <wire-op-query>` and
:ref:`OP_GET_MORE <wire-op-get-more>`), it will be returned in
the ``responseTo`` field of the :ref:`OP_REPLY <wire-op-reply>`
message. Along with the ``reponseTo`` field in responses, clients can use
this to associate query responses with the originating query.
**responseTo**: In the case of a message from the database, this will be
the ``requestID`` taken from the :ref:`OP_QUERY <wire-op-query>`
or :ref:`OP_GET_MORE <wire-op-get-more>` messages from the
client. Along with the ``requestID`` field in queries, clients can use this
to associate query responses with the originating query.
**opCode**: Type of message. See the table below in the next section.
Request Opcodes
~~~~~~~~~~~~~~~
The following are the currently supported opcodes:
.. list-table:: Opcodes
:widths: 15 15 100
:header-rows: 1
* - Opcode Name
- opCode value
- Comment
* - OP_REPLY
- 1
- Reply to a client request. responseTo is set
* - OP_MSG
- 1000
- generic msg command followed by a string
* - OP_UPDATE
- 2001
- update document
* - OP_INSERT
- 2002
- insert new document
* - RESERVED
- 2003
- formerly used for OP_GET_BY_OID
* - OP_QUERY
- 2004
- query a collection
* - OP_GET_MORE
- 2005
- Get more data from a query. See Cursors
* - OP_DELETE
- 2006
- Delete documents
* - OP_KILL_CURSORS
- 2007
- Tell database client is done with a cursor
Client Request Messages
-----------------------
Clients can send all messages except for :ref:`OP_REPLY
<wire-op-reply>`. This is reserved for use by the database.
Only the :ref:`OP_QUERY <wire-op-query>` and
:ref:`OP_GET_MORE <wire-op-get-more>` messages result in a
response from the database. There will be no response sent for any other
message.
You can determine if a message was successful with a getLastError command.
.. _wire-op-update:
OP_UPDATE
~~~~~~~~~
The OP_UPDATE message is used to update a document in a collection. The
format of a OP_UPDATE message is the following:
.. code-block:: sh
struct OP_UPDATE {
MsgHeader header; // standard message header
int32 ZERO; // 0 - reserved for future use
cstring fullCollectionName; // "dbname.collectionname"
int32 flags; // bit vector. see below
document selector; // the query to select the document
document update; // specification of the update to perform
}
**fullCollectionName**: The full collection name. The full collection
name is the concatenation of the database name with the collection name,
using a ``.`` for the concatenation. For example, for the database ``foo``
and the collection ``bar``, the full collection name is ``foo.bar``.
.. list-table:: Flags
:widths: 15 15 100
:header-rows: 1
* - bit num
- name
- description
* - 0
- Upsert
- If set, the database will insert the supplied object into the
collection if no matching document is found.
* - 1
- MultiUpdate
- If set, the database will update all matching objects in the
collection. Otherwise only updates first matching doc.
* - 2-31
- Reserved
- Must be set to 0.
**selector**: BSON document that specifies the query for selection of the
document to update.
**update**: BSON document that specifies the update to be performed. For
information on specifying updates see the :manual:`Update Operations
</applications/update>` documentation from the MongoDB Manual.
There is no response to an OP_UPDATE message.
.. _wire-op-insert:
OP_INSERT
~~~~~~~~~
The OP_INSERT message is used to insert one or more documents into a
collection. The format of the OP_INSERT message is
.. code-block:: sh
struct {
MsgHeader header; // standard message header
int32 flags; // bit vector - see below
cstring fullCollectionName; // "dbname.collectionname"
document* documents; // one or more documents to insert into the collection
}
**fullCollectionName**: The full collection name. The full collection
name is the concatenation of the database name with the collection name,
using a ``.`` for the concatenation. For example, for the database ``foo``
and the collection ``bar``, the full collection name is ``foo.bar``.
**documents**: One or more documents to insert into the collection. If
there are more than one, they are written to the socket in sequence, one
after another.
.. list-table:: Flags
:widths: 15 15 100
:header-rows: 1
* - bit num
- name
- description
* - 0
- ContinueOnError
- If set, the database will not stop processing a bulk insert if
one fails (eg due to duplicate IDs). This makes bulk insert
behave similarly to a series of single inserts, except lastError
will be set if any insert fails, not just the last one. If
multiple errors occur, only the most recent will be reported by
getLastError. (new in 1.9.1)
* - 1-31
- Reserved
- Must be set to 0.
There is no response to an OP_INSERT message.
.. _wire-op-query:
OP_QUERY
~~~~~~~~
The OP_QUERY message is used to query the database for documents in a
collection. The format of the OP_QUERY message is:
.. code-block:: sh
struct OP_QUERY {
MsgHeader header; // standard message header
int32 flags; // bit vector of query options. See below for details.
cstring fullCollectionName; // "dbname.collectionname"
int32 numberToSkip; // number of documents to skip
int32 numberToReturn; // number of documents to return
// in the first OP_REPLY batch
document query; // query object. See below for details.
[ document returnFieldSelector; ] // Optional. Selector indicating the fields
// to return. See below for details.
}
.. list-table:: Flags
:widths: 15 15 100
:header-rows: 1
* - bit num
- name
- description
* - 0
- Reserved
- Must be set to 0.
* - 1
- TailableCursor
- Tailable means cursor is not closed when the last data is
retrieved. Rather, the cursor marks the final object's position.
You can resume using the cursor later, from where it was located,
if more data were received. Like any "latent cursor", the cursor
may become invalid at some point (CursorNotFound) – for example
if the final object it references were deleted.
* - 2
- SlaveOk
- Allow query of replica slave. Normally these return an error
except for namespace "local".
* - 3
- OplogReplay
- Internal replication use only - driver should not set
* - 4
- NoCursorTimeout
- The server normally times out idle cursors after an inactivity
period (10 minutes) to prevent excess memory use. Set this option
to prevent that.
* - 5
- AwaitData
- Use with TailableCursor. If we are at the end of the data, block
for a while rather than returning no data. After a timeout
period, we do return as normal.
* - 6
- Exhaust
- Stream the data down full blast in multiple "more" packages, on
the assumption that the client will fully read all data queried.
Faster when you are pulling a lot of data and know you want to
pull it all down. Note: the client is not allowed to not read all
the data unless it closes the connection.
* - 7
- Partial
- Get partial results from a mongos if some shards are down
(instead of throwing an error)
* - 8-31
- Reserved
- Must be set to 0.
**fullCollectionName**: The full collection name. The full collection name
is the concatenation of the database name with the collection name,
using a ``.`` for the concatenation. For example, for the database ``foo``
and the collection ``bar``, the full collection name is ``foo.bar``.
**numberToSkip**: Sets the number of documents to omit - starting from the
first document in the resulting dataset - when returning the result of
the query.
**numberToReturn**: Limits the number of documents in the first
:ref:`OP_REPLY <wire-op-reply>` message to the query. However,
the database will still establish a cursor and return the ``cursorID``
to the client if there are more results than ``numberToReturn``. If the
client driver offers 'limit' functionality (like the SQL LIMIT keyword),
then it is up to the client driver to ensure that no more than the
specified number of document are returned to the calling application. If
``numberToReturn`` is ``0``, the db will use the default return size. If
the number is negative, then the database will return that number and
close the cursor. No futher results for that query can be fetched. If
``numberToReturn`` is ``1`` the server will treat it as ``-1`` (closing the
cursor automatically).
**query**: BSON document that represents the query. The query will
contain one or more elements, all of which must match for a document to
be included in the result set. Possible elements include ``$query``,
``$orderby``, ``$hint``, ``$explain``, and ``$snapshot``.
**returnFieldsSelector**: OPTIONAL BSON document that limits the fields
in the returned documents. The ``returnFieldsSelector`` contains one or
more elements, each of which is the name of a field that should be
returned, and and the integer value ``1``. In JSON notation, a
``returnFieldsSelector`` to limit to the fields ``a``, ``b`` and ``c``
would be:
.. code-block:: sh
{ a : 1, b : 1, c : 1}
The database will respond to an OP_QUERY message with an
:ref:`OP_REPLY <wire-op-reply>` message.
.. _wire-op-get-more:
OP_GET_MORE
~~~~~~~~~~~
The OP_GET_MORE message is used to query the database for documents in a
collection. The format of the OP_GET_MORE message is:
.. code-block:: sh
struct {
MsgHeader header; // standard message header
int32 ZERO; // 0 - reserved for future use
cstring fullCollectionName; // "dbname.collectionname"
int32 numberToReturn; // number of documents to return
int64 cursorID; // cursorID from the OP_REPLY
}
**fullCollectionName**: The full collection name. The full collection name
is the concatenation of the database name with the collection name,
using a ``.`` for the concatenation. For example, for the database ``foo``
and the collection ``bar``, the full collection name is ``foo.bar``.
**numberToReturn**: Limits the number of documents in the first
:ref:`OP_REPLY <wire-op-reply>` message to the query. However,
the database will still establish a cursor and return the ``cursorID`` to
the client if there are more results than ``numberToReturn``. If the client
driver offers 'limit' functionality (like the SQL LIMIT keyword), then
it is up to the client driver to ensure that no more than the specified
number of document are returned to the calling application. If
``numberToReturn`` is ``0``, the db will used the default return size.
**cursorID**: Cursor identifier that came in the :ref:`OP_REPLY
<wire-op-reply>`. This must be the value that came from the database.
The database will respond to an OP_GET_MORE message with an
:ref:`OP_REPLY <wire-op-reply>` message.
.. _wire-op-delete:
OP_DELETE
~~~~~~~~~
The OP_DELETE message is used to remove one or more documents from a
collection. The format of the OP_DELETE message is:
.. code-block:: sh
struct {
MsgHeader header; // standard message header
int32 ZERO; // 0 - reserved for future use
cstring fullCollectionName; // "dbname.collectionname"
int32 flags; // bit vector - see below for details.
document selector; // query object. See below for details.
}
**fullCollectionName**: The full collection name. The full collection name
is the concatenation of the database name with the collection name,
using a ``.`` for the concatenation. For example, for the database ``foo``
and the collection ``bar``, the full collection name is ``foo.bar``.
.. list-table:: Flags
:widths: 15 15 100
:header-rows: 1
* - bit num
- name
- description
* - 0
- SingleRemove
- If set, the database will remove only the first matching document
in the collection. Otherwise all matching documents will be
removed.
* - 1-31
- Reserved
- Must be set to 0.
**selector**: BSON document that represent the query used to select the
documents to be removed. The selector will contain one or more elements,
all of which must match for a document to be removed from the
collection.
There is no response to an OP_DELETE message.
.. _wire-op-kill-cursors:
OP_KILL_CURSORS
~~~~~~~~~~~~~~~
The OP_KILL_CURSORS message is used to close an active cursor in the
database. This is necessary to ensure that database resources are
reclaimed at the end of the query. The format of the OP_KILL_CURSORS
message is:
.. code-block:: sh
struct {
MsgHeader header; // standard message header
int32 ZERO; // 0 - reserved for future use
int32 numberOfCursorIDs; // number of cursorIDs in message
int64* cursorIDs; // sequence of cursorIDs to close
}
**numberOfCursorIDs**: The number of cursors that are in the message.
**cursorIDs**: "array" of cursor IDs to be closed. If there are more than
one, they are written to the socket in sequence, one after another.
If a cursor is read until exhausted (read until :ref:`OP_QUERY <wire-op-query>`
or :ref:`OP_GET_MORE <wire-op-get-more>` returns zero
for the cursor id), there is no need to kill the cursor.
.. _wire-op-msg:
OP_MSG
~~~~~~
Deprecated. OP_MSG sends a diagnostic message to the database. The
database sends back a fixed response. The format is:
.. code-block:: sh
struct {
MsgHeader header; // standard message header
cstring message; // message for the database
}
Drivers do not need to implement OP_MSG.
Database Response Messages
--------------------------
.. _wire-op-reply:
OP_REPLY
~~~~~~~~
The OP_REPLY message is sent by the database in response to an
:ref:`OP_QUERY <wire-op-query>` or :ref:`OP_GET_MORE
<wire-op-get-more>` message. The format of an OP_REPLY message is:
.. code-block:: sh
struct {
MsgHeader header; // standard message header
int32 responseFlags; // bit vector - see details below
int64 cursorID; // cursor id if client needs to do get more's
int32 startingFrom; // where in the cursor this reply is starting
int32 numberReturned; // number of documents in the reply
document* documents; // documents
}
.. list-table:: responseFlags
:widths: 15 15 100
:header-rows: 1
* - bit num
- name
- description
* - 0
- CursorNotFound
- Set when getMore is called but the cursor id is not valid at the
server. Returned with zero results.
* - 1
- QueryFailure
- Set when query failed. Results consist of one document containing
an "$err" field describing the failure.
* - 2
- ShardConfigStale
- Drivers should ignore this. Only mongos will ever see this set,
in which case, it needs to update config from the server.
* - 3
- AwaitCapable
- Set when the server supports the AwaitData Query option. If it
doesn't, a client should sleep a little between getMore's of a
Tailable cursor. Mongod version 1.6 supports AwaitData and thus
always sets AwaitCapable.
* - 4-31
- Reserved
- Ignore
**cursorID**: The ``cursorID`` that this OP_REPLY is a part of. In the event
that the result set of the query fits into one OP_REPLY message,
``cursorID`` will be 0. This ``cursorID`` must be used in any
:ref:`OP_GET_MORE <wire-op-get-more>` messages used to get more
data, and also must be closed by the client when no longer needed via a
:ref:`OP_KILL_CURSORS <wire-op-kill-cursors>` message.