Navigation

Migrate with mongomirror

mongomirror is a utility for migrating data from an existing MongoDB replica set to a MongoDB Atlas replica set. mongomirror does not require you to shut down your existing replica set or applications.

Info With Circle IconCreated with Sketch.Note
support

mongomirror is intended to be used as a utility for performing a one-time migration of a dataset into a MongoDB Atlas cluster from a MongoDB deployment hosted outside of MongoDB Atlas. It can also be used for performing a one-time migration of a dataset from one Atlas cluster into another Atlas cluster. Any other use of this utility is not supported.

Important With Circle IconCreated with Sketch.Important

mongomirror does not import user/role data.

Operating SystemDownload
Amazon Linux 1mongomirror-linux-x86_64-enterprise-amzn64-0.11.0.tgz
Amazon Linux 2mongomirror-linux-x86_64-enterprise-amazon2-0.11.0.tgz
Debian 8mongomirror-linux-x86_64-debian81-0.11.0.tgz
Debian 9mongomirror-linux-x86_64-debian92-0.11.0.tgz
macOSmongomirror-osx-x86_64-0.11.0.tgz
RHEL 6.2mongomirror-linux-x86_64-rhel62-0.11.0.tgz
RHEL 7.0mongomirror-linux-x86_64-rhel70-0.11.0.tgz
Suse 12mongomirror-linux-x86_64-suse12-0.11.0.tgz
Ubuntu 14.04mongomirror-linux-x86_64-ubuntu1404-0.11.0.tgz
Ubuntu 16.04mongomirror-linux-x86_64-ubuntu1604-0.11.0.tgz
Ubuntu 18.04mongomirror-linux-x86_64-ubuntu1804-0.11.0.tgz
Windowsmongomirror-win32-x86_64-0.11.0.zip
  • The source MongoDB deployment must be a replica set. If the source is a standalone MongoDB deployment, convert to a replica set first to use mongomirror. See Convert a Standalone to a Replica Set.
  • The source replica set must run MongoDB version 2.6 or higher.
  • The source MongoDB replica set cannot be an M0 (Free Tier) or M2/M5 shared cluster.
  • You should not change the featureCompatibilityVersion flag at any time during the procedure.
  • You should not make any namespace changes during the migration process, such as using the renameCollection command or executing an aggregation pipeline that includes the $out aggregation stage.

    Contact MongoDB Support if you believe renaming or dropping a collection needs to occur during the migration process. From the Atlas UI, click Support to open a support ticket.

If the source replica set requires authentication, you must include user credentials when running mongomirror. You must specify a database user with, at a minimum, the following privileges:

  • Read all databases and collections. The readAnyDatabase role on the admin database covers this requirement.
  • Read the oplog.
  • Run the getParameter command.

If no such user exists, create the user in your source MongoDB replica set. Different MongoDB server versions have different built-in roles. Select a built-in role based on your MongoDB server version and execute the appropriate commands in the mongo shell:

  • For source clusters running MongoDB version 3.4 or later a user must have, at a minimum, both clusterMonitor and readAnyDatabase roles. For example:

    use admin
    db.createUser(
    {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "clusterMonitor", "readAnyDatabase" ]
    }
    )
  • For source clusters running MongoDB version 3.2 a user must have, at a minimum, both clusterManager and readAnyDatabase roles, as well as read access on the local database. This requires a custom role, which you can create with the following commands:

    use admin
    db.createRole(
    {
    role: "migrate",
    privileges: [
    { resource: { db: "local", collection: "" }, actions: [ "find" ] }
    ],
    roles: ["readAnyDatabase", "clusterManager"]
    }
    )
    db.createUser(
    {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "migrate" ]
    }
    )
  • For source clusters running MongoDB version 2.6 or 3.0 a user must have, at a minimum, the readAnyDatabase role. For example:

    use admin
    db.createUser(
    {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "readAnyDatabase" ]
    }
    )
  • The target Atlas cluster must be a replica set.
  • The target Atlas cluster should not contain any namespaces that overlap with the source cluster. If mongomirror detects overlapping namespaces on the target cluster it errors out before the migration process begins. If your target cluster contains overlapping namespaces, you can either:

    • Run mongomirror with the --drop option to delete all non-system data from the target cluster, or
    • Use the --includeNamespace option to specify particular namespaces to import from the source cluster.
  • The target Atlas cluster cannot be an M0 (Free Tier), M2, or M5 shared cluster.
  • The target Atlas cluster MongoDB version must be the same as or later than the source cluster MongoDB version.
  • You should not change the featureCompatibilityVersion flag at any time during the procedure.
  • mongomirror is not compatible with TTL indexes. Drop any existing TTL indexes and rebuild them when the migration process is complete.
1

If the source replica set requires authentication, you must include user credentials when running mongomirror. You must specify a database user with, at a minimum, the following privileges:

If no such user exists, create the user in your source MongoDB replica set. Different MongoDB server versions have different built-in roles. Select a built-in role based on your MongoDB server version and execute the appropriate commands in the mongo shell:

  • For source clusters running MongoDB version 3.4 or later a user must have, at a minimum, both clusterMonitor and readAnyDatabase roles. For example:

    use admin
    db.createUser(
    {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "clusterMonitor", "readAnyDatabase" ]
    }
    )
  • For source clusters running MongoDB version 3.2 a user must have, at a minimum, both clusterManager and readAnyDatabase roles, as well as read access on the local database. This requires a custom role, which you can create with the following commands:

    use admin
    db.createRole(
    {
    role: "migrate",
    privileges: [
    { resource: { db: "local", collection: "" }, actions: [ "find" ] }
    ],
    roles: ["readAnyDatabase", "clusterManager"]
    }
    )
    db.createUser(
    {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "migrate" ]
    }
    )
  • For source clusters running MongoDB version 2.6 or 3.0 a user must have, at a minimum, the readAnyDatabase role. For example:

    use admin
    db.createUser(
    {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "readAnyDatabase" ]
    }
    )

Make note of the username and password for this user, as you must specify these credentials when running mongomirror.

2

You must specify a database user with the Atlas admin role to run mongomirror. See Add Database Users for documentation on creating a database user.

If no such user exists, create the user:

  1. In the Security section of the left navigation, click Database Access. The Database Users tab displays.
  2. Click Add New Database User.
  3. Add an Atlas admin user.

Make note of the username and password selected for the new user, as you must specify these credentials when running mongomirror.

3

If the host where you will run mongomirror is not in your cluster's IP Access List, update the list. You can specify either:

  • The public IP address of the server on which mongomirror will run, or
  • If set up for VPC peering, either the peer's VPC CIDR block (or a subnet) or the peer VPC 's Security Group.
4

From the Clusters view, click Connect for the Atlas cluster into which you want to migrate data.

5

You can get your Atlas cluster's hostname information from the Atlas application connection assistance UI.

Info With Circle IconCreated with Sketch.Note

You don't need to use a driver to migrate data with mongomirror. The application connection assistance UI is simply a convenient way to copy your connection string to the clipboard.

  1. Click Connect Your Application.
  2. Select Python from the Driver dropdown menu.
  3. Select 3.3 or earlier from the Version dropdown menu.
  4. Copy the connection string and paste into your preferred text editor.

The connection string should look similar to the following example. This example has been broken into multiple lines for readability:

mongodb://<username>:<PASSWORD>@
00.foo.mongodb.net:27017,
01.foo.mongodb.net:27017,
02.foo.mongodb.net:27017/test?
ssl=true&replicaSet=myAtlasRS&authSource=admin

Copy the value of replicaSet and append the host list as comma separated values, similar to the following:

myAtlasRS/00.foo.mongodb.net:27017,01.foo.mongodb.net:27017,02.foo.mongodb.net:27017

Use this value for the --destination option in the next step.

6

Start mongomirror with the following options:

Your final command should resemble the following example:

mongomirror --host "MySourceRS/host1.example.net:27017,host2.example.net:27017,host3.example.net:27017" \
--ssl \
--username "mySourceUser" \
--password "mySourceP@$$word" \
--authenticationDatabase "admin" \
--destination "myAtlasRS/00.foo.mongodb.net:27017,01.foo.mongodb.net:27017,02.foo.mongodb.net:27017" \
--destinationUsername "myAtlasAdminUser" \
--destinationPassword "atlasPassword"

See Options for more mongomirror options.

When you start mongomirror:

  1. First, mongomirror performs an initial sync, copying collections from the existing MongoDB replica set to the target cluster in Atlas. mongomirror does not copy the config database.

    Info With Circle IconCreated with Sketch.Note

    Starting in version 0.5.0, mongomirror builds all indexes on the destination cluster in the foreground, regardless of how the indexes were built on the source cluster. Foreground index builds block all other operations on the database. For more information on foreground index builds, see Index Build Operations on a Populated Collection.

  2. After the initial sync, mongomirror continuously tails the replica set's oplog for incoming changes and replays them on the target cluster in MongoDB Atlas.

Once started, mongomirror runs continuously until you shut down the process. To fully migrate to the Atlas, continue on to Switch to Atlas steps.

After the mongomirror has performed the initial sync and has been tailing the replica set's oplog, to switch over to Atlas cluster:

1

This ensures that no additional writes occur on the source cluster.

2

This signifies that the source and destination clusters are in a consistent state.

3

Once the Atlas cluster is up to date, stop mongomirror.

4

Update your client application(s) with the Atlas connection string provided via the Connect button on the cluster panel.

For details on connections to Atlas, see Connect via Driver.

For more information on mongomirror, including behavior, options, and examples, see the mongomirror reference page.

Give Feedback