mongomirror¶

Download mongomirror¶

Syntax¶

To run mongomirror, you must specify the source replica set and the target Atlas replica set. For Atlas, you must specify a user in the Atlas cluster with appropriate privileges and the corresponding password. If the source replica set requires authentication, you must specify a user with appropriate privileges.

mongomirror --host <sourceReplSet> \
   --destination <atlasCluster> \
   --destinationUsername <atlasAdminUser> \
   --destinationPassword <atlasPassword> \
   [Additional options]

For details on the options, see Options.

Migration Process¶

When you start mongomirror:

1. First, mongomirror performs an initial sync, copying collections from the existing MongoDB replica set to the target cluster in Atlas.
2. After the initial sync, mongomirror continuously tails the replica set's oplog for incoming changes and replays them on the target cluster in MongoDB Atlas.

Once started, mongomirror runs continuously until you shut down the process.

• If you shut down mongomirror during the initial sync stage, either ensure that the target cluster is empty before you restart mongomirror or run mongomirror with the --drop option.
• If you shut down mongomirror during the oplog tailing stage, you can restart mongomirror to continue from the last oplog record processed. See the --bookmarkFile option.

2016-12-16T17:54:53.638-0800  [#....................]  park.events  2179/34184    (6.4%)
2016-12-16T17:54:53.638-0800  [#############........]  zoo.animals  29000/49778  (58.3%)

2016-12-12T16:22:17.027-0800 Current lag from source: 6s

Considerations¶

Options¶

--host <host>¶

The host information for the source replica set. Specify the replica set name and a seed list of the members, as in the following:

<RSname>/<host1>:<port1>,<host2>:<port2>,<host3>:<port3>

--username <username>¶

If the source replica set requires authentication, the name of a user in the source replica set with privileges to read any database, including the local database. A user with the backup role provides the appropriate privileges. For details on the specific privileges required, see Required Access on Source Replica Set.

--password <password>¶

Password for the user specified in --username.

--authenticationDatabase <authenticationDatabase>¶

The database in the source replica set where the user specified in --username was created.

--authenticationMechanism <authenticationMechanism>¶

The authentication mechanism to use to authenticate the user to the source replica set.

Value	Description
SCRAM-SHA-1	RFC 5802 standard Salted Challenge Response Authentication Mechanism using the SHA1 hash function.
SCRAM-SHA-256	RFC 5802 standard Salted Challenge Response Authentication Mechanism using the SHA256 hash function.
MONGODB-CR	MongoDB challenge/response authentication.
MONGODB-X509	MongoDB TLS/SSL certificate authentication.
GSSAPI (Kerberos)	External authentication using Kerberos. This mechanism is available only in MongoDB Enterprise.
PLAIN (LDAP SASL)	External authentication using LDAP. You can also use PLAIN for authenticating in-database users. PLAIN transmits passwords in plain text. This mechanism is available only in MongoDB Enterprise.
MONGODB-IAM	New in version 0.10.0 External authentication with AWS IAM. To authenticate with AWS IAM credentials, use the following options: --username <aws access key id> --password <secret access key id> --awsSessionToken <aws session token>

--awsSessionToken¶

New in version 0.10.0

An AWS session token for use with the MONGODB-IAM authentication mechanism.

--compressors <snappy,...>¶

New in version 0.9.0

Comma-separated list of compressors to enable. Use 'none' to disable. Default: snappy,zstd,zlib

--destination <destination>¶

The host information for the target Atlas replica set.

Specify the replica set name and a seed list of the members, as in the following:

<RSname>/<host1>:<port1>,<host2>:<port2>,<host3>:<port3>

--destinationUsername <Atlas user name>¶

Name of a database user in the Atlas cluster with privileges to read, write, and admin any database. A user with the Atlas admin role provides the appropriate privileges. For details on the specific privileges required, see Required Access on Destination Cluster.

--destinationPassword <password>¶

Password of the database user specified in --destinationUsername.

--drop¶

Flag that indicates that mongomirror should drop all but the system.* collections in the target cluster.

--noIndexRestore¶

New in version 0.10.0

Omit indexes when migrating data.

--includeNamespace <database.collection>¶

Specify a namespace on the source cluster to mirror to the target cluster. May be provided multiple times.

--includeDB <database>¶

Specify a database on the source cluster to mirror to the target cluster. May be provided multiple times.

--ssl¶

Enables TLS/SSL encrypted connections to the source replica set.

--sslPEMKeyFile <file>¶

The .pem file if the source replica set requires clients to present a certificate. The .pem file contains both the TLS/SSL certificate and key. Specify the file using relative or absolute paths.

--sslPEMKeyPassword <value>¶

Password to decrypt the certificate-key file specified in --sslPEMKeyFile. Use if the --sslPEMKeyFile is encrypted.

--sslCAFile <file>¶

The .pem file that contains the root certificate chain from the Certificate Authority(CA) for the source replica set. Specify the file using relative or absolute paths.

--sslCRLFile <filename>¶

The .pem file that contains the Certificate Revocation List for the source replica set. Specify the file using relative or absolute paths.

--sslAllowInvalidHostnames¶

Disables the validation of the hostnames in TLS/SSL certificates presented by the source replica set. Allows mongomirror to connect to the source replica set if the hostname in the certificates does not match the specified hostname.

--sslAllowInvalidCertificates¶

Bypasses the validation checks for certificates presented by the source replica set and allows the use of invalid certificates. When using the --allowInvalidCertificates setting, MongoDB logs as a warning the use of the invalid certificate.

--gssapiServiceName <name>¶

If the source replica set uses Kerberos authentication, the name of the service using GSSAPI/Kerberos. Only required if the service does not use the default name of mongodb.

This option is available only in MongoDB Enterprise.

--gssapiHostName <host>¶

If the source replica set uses Kerberos authentication, the hostname of a service using GSSAPI/Kerberos. Only required if the hostname of a machine does not match the hostname resolved by DNS.

This option is available only in MongoDB Enterprise.

--readPreference <read preference>¶

Deprecated since version 0.9.0

mongomirror always reads from the primary unless the source is a single host without a replica set name, in which case it makes a direct connection only to that host.

--writeConcern <write concern>¶

Deprecated since version 0.2.3: mongomirror always uses majority write concern.

--numParallelCollections <num>, -j <num>¶

Default: 4

The number of collections to copy and restore in parallel.

--bypassDocumentValidation¶

Deprecated since version 0.2.3: mongomirror always bypasses document validation.

--bookmarkFile <file>¶

Default: mongomirror.timestamp

Name of the oplog timestamp bookmark file.

--forceDump¶

Flag that indicates that mongomirror resync all source collections, even if a nonempty bookmark file exists.

--oplogPath <path>¶

New in version 0.5.0

Enables mongomirror to buffer the initial sync oplog window to disk. When you specify a value for this option, mongomirror streams the source oplog entries to the specified directory in a single file: <oplogPath>/oplog-mongomirror.bson.sz. After the entire oplog file is replayed to the destination cluster, mongomirror removes the file and starts tailing the source oplog without buffering.

By default, mongomirror streams oplog entries from the source and applies them to the destination cluster. However, the migration may fail if the source oplog is not large enough to contain the entire initial sync oplog window. To avoid this error, you can either increase the size of the source oplog, or specify this option to ensure that the source oplog will not run out of space during the migration process.

Important

There must be enough disk space to accommodate all of the source oplog entries that occur during the initial mongomirror sync.

Example

If the source oplog is 10 GB and covers 24 hours of changes, and mongomirror's sync is estimated to take 48 hours, there must be at least 20 GB of free disk space in the specified directory.

--oplogBatchSize <num>¶

New in version 0.8.0

Specify the number of oplog entries to send as a batch. Defaults to 10,000.

--httpStatusPort <num>¶

Directs mongomirror to start an HTTP server on the specified port. You can retrieve the current status of mongomirror by issuing an HTTP GET request to http://localhost:<num>.

When running with --httpStatusPort, mongomirror does not exit when it encounters an error. Instead, it logs the error as normal and reports the error over HTTP to the specified port.

mongomirror returns a document in response to the HTTP request. The following example syntax represents all the possible output fields - the actual response may only return a subset of these fields. See the subsequent table for a description of the fields and when to expect them.

{
   "stage" : "<stage Name>",
   "phase" : "<phase Name>",
   "details" : {
      "currentTimestamp" : "<BSON timestamp>",
      "latestTimestamp" : "<BSON timestamp>",
      "lastWriteOnSourceTimestamp" : "<BSON timestamp>",
      "<namespace>" : {
         "complete" : <boolean>,
         "copiedBytes" : <integer>,
         "totalBytes" : <integer>,
         "createIndexes" : <integer>
      },
      ...
   },
   "errorMessage" : "<error message>"
}

The following table describes each field and its possible values:

Field	Description
stage	The name of the stage in progress. Possible values are: initializing mongomirror has started but is not yet copying any data. initial sync mongomirror is copying documents and indexes that already exist on the source deployment. mongomirror also tails and applies entries from the oplog. oplog sync mongomirror is tailing and applying entries from the oplog.
phase	The name of the phase. Provides more specific details about what part of the stage is in progress.
details	A document providing a detailed description of the progress of the current phase. During the initial sync stage, each subdocument in details represents a single collection being copied by mongomirror. Depending on the stage or phase, mongomirror may not include this field in the response.
details.<namespace>	The full namespace of the collection being copied, displayed as <database>.<collection>. Only displays during the initial sync phase when copying documents or indexes.
details.<namespace>.complete	Displays true or false depending on whether or not mongomirror has copied all documents or indexes from the collection to the target Atlas cluster. Only displays during the initial sync phase when copying documents or indexes.
details.<namespace>.copiedBytes	The number of bytes copied so far. Note that this is a different measurement from the mongomirror logs, which report the current/total number of documents copied. Only displays during the initial sync phase when copying non-index data.
details.<namespace>.totalBytes	The total size (in bytes) of the collection. Only displays during the initial sync phase when copying non-index data.
details.<namespace>.createIndexes	The number of indexes that have been or will be created. Only displays during the initial sync stage when copying indexes.
details.currentTimestamp	The BSON timestamp value of the oplog entry most recently processed. mongomirror only refreshes this data point every 10 seconds, so mongomirror may be slightly further ahead of the reported time. Only displays during the initial sync or oplog sync stages when tailing or applying oplog entries.
details.latestTimestamp	During the initial sync stage, this represents the BSON timestamp value of the latest oplog entry available after the initial data was copied during initial sync. During the oplog sync stage, this represents the BSON timestamp value of the latest oplog entry available on the source deployment. Only displays during the initial sync or oplog sync stages when tailing or applying oplog entries.
details .lastWriteOnSourceTimestamp	The BSON timestamp value of the most recent oplog entry that is not a no-op. No-op entries are generally system-level operations such as heartbearts that do not write or edit data in the database. mongomirror refreshes this value every 10 seconds. Operations which write or edit data in the database may not be reported until the next refresh occurs. The lastWriteOnSourceTimestamp field is useful as a confirmation that no new writes are occurring on the source deployment before cutting over during a migration.
errorMessage	A string that describes any error encountered by mongomirror.

--collStatsThreshold <num>¶

New in version 0.9.0

Maximum number of collections which may exist before collStats is disabled. Use -1 to always run collStats or 0 to never run collStats. Default: -1

Examples¶

mongomirror --host  sourceRS/source-host1:27017,source-host2:27017 \
         --destination myAtlasRS/atlas-host1:27017,atlas-host2:27017 \
         --destinationUsername myAtlasUser \
         --destinationPassword myAtlasPwd

<replicaSetName>/<host1>:<port1>,<host2>:<port2>,<host3>:<port3>,...

mongomirror --host sourceRS/source-host1:27017,source-host2:27017,source-host3:27017 \
   --username mySourceUser \
   --password mySourcePassword \
   --authenticationDatabase admin \
   --destination myAtlasRS/atlas-host1:27017,atlas-host2:27017 \
   --destinationUsername myAtlasUser \
   --destinationPassword atlasPassw0Rd

<replicaSetName>/<replicaMember>,<replicaMember>,<replicaMember>,...

mongomirror <args> 2>&1 | tee -a mongomirror.log

mongomirror --host sourceRS/source-host1:27017,source-host2:27017,source-host3:27017 \
   --username "CN=myName,OU=myOrgUnit,O=myOrg,L=myLocality,ST=myState,C=myCountry" \
   --authenticationDatabase '$external' \
   --authenticationMechanism MONGODB-X509 \
   --ssl \
   --sslPEMKeyFile <path-to-my-client-certificate.pem> \
   --sslCAFile <path-to-my-certificate-authority-certificate.pem> \
   --destination myAtlasRS/atlas-host1:27017,atlas-host2:27017 \
   --destinationUsername myAtlasUser \
   --destinationPassword atlasPassw0Rd

<replicaSetName>/<replicaMember>,<replicaMember>,<replicaMember>,...

mongomirror --host sourceRS/source-host1:27017,source-host2:27017,source-host3:27017 \
   --username sourceUser/administrator@MYREALM.COM \
   --authenticationDatabase '$external' \
   --authenticationMechanism GSSAPI \
   --destination myAtlasRS/atlas-host1:27017,atlas-host2:27017,atlas-host3:27017 \
   --destinationUsername atlasUser \
   --destinationPassword atlasPass

<replicaSetName>/<replicaMember>,<replicaMember>,<replicaMember>,...