Navigation

Migrate with mongomirror

On this page

mongomirror is a utility for migrating data from an existing MongoDB replica set to a MongoDB Atlas replica set. mongomirror does not require you to shut down your existing replica set or applications.

Important

mongomirror does not import user/role data.

Prerequisites

Source

  • The source MongoDB deployment must be a replica set. If the source is a standalone MongoDB deployment, convert to a replica set first to use mongomirror. See Convert a Standalone to a Replica Set.

  • The source replica set must run MongoDB version 2.6 through 4.2.

  • The source MongoDB replica set cannot be an M0 (Free Tier) or M2/M5 shared cluster.

  • You should not change the featureCompatibilityVersion flag at any time during the procedure.

  • You should not make any namespace changes during the migration process, such as using the renameCollection command or executing an aggregation pipeline that includes the $out aggregation stage.

    Contact MongoDB Support if you believe renaming or dropping a collection needs to occur during the migration process. From the Atlas UI, click Support to open a support ticket.

Required Access on Source Replica Set

If the source replica set requires authentication, you must include user credentials when running mongomirror. You must specify a MongoDB user with, at a minimum, the following privileges:

  • Read all databases and collections. The readAnyDatabase role on the admin database covers this requirement.
  • Read the oplog.
  • Run the getParameter command.

If no such user exists, create the user in your source MongoDB replica set. Different MongoDB server versions have different built-in roles. Select a built-in role based on your MongoDB server version and execute the appropriate commands in the mongo shell:

  • For source clusters running MongoDB version 3.4 or later a user must have, at a minimum, both clusterMonitor and readAnyDatabase roles. For example:

    copy 
    use admin
db.createUser(
  {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "clusterMonitor", "readAnyDatabase" ]
  }
)

  • For source clusters running MongoDB version 3.2 a user must have, at a minimum, both clusterManager and readAnyDatabase roles, as well as read access on the local database. This requires a custom role, which you can create with the following commands:

    copy 
    use admin
db.createRole(
  {
    role: "migrate",
    privileges: [
      { resource: { db: "local", collection: "" }, actions: [ "find" ] }
    ],
    roles: ["readAnyDatabase", "clusterManager"]
  }
)
db.createUser(
  {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "migrate" ]
  }
)

  • For source clusters running MongoDB version 2.6 or 3.0 a user must have, at a minimum, the readAnyDatabase role. For example:

    copy 
    use admin
db.createUser(
  {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "readAnyDatabase" ]
  }
)

Target

  • The target Atlas cluster must be a replica set.
  • The target Atlas cluster should not contain any namespaces that overlap with the source cluster. If mongomirror detects overlapping namespaces on the target cluster it errors out before the migration process begins. If your target cluster contains overlapping namespaces, you can either:
    • Run mongomirror with the --drop option to delete all non-system data from the target cluster, or
    • Use the --includeNamespace option to specify particular namespaces to import from the source cluster.
  • The target Atlas cluster cannot be an M0 (Free Tier), M2, or M5 shared cluster.
  • The target Atlas cluster MongoDB version must be the same as or later than the source cluster MongoDB version.
  • You should not change the featureCompatibilityVersion flag at any time during the procedure.
  • mongomirror is not compatible with TTL indexes. Drop any existing TTL indexes and rebuild them when the migration process is complete.

Procedure

1

Set up MongoDB user in the source replica set.

If the source replica set requires authentication, you must include user credentials when running mongomirror. You must specify a MongoDB user with, at a minimum, the following privileges:

If no such user exists, create the user in your source MongoDB replica set. Different MongoDB server versions have different built-in roles. Select a built-in role based on your MongoDB server version and execute the appropriate commands in the mongo shell:

  • For source clusters running MongoDB version 3.4 or later a user must have, at a minimum, both clusterMonitor and readAnyDatabase roles. For example:

    copy 
    use admin
db.createUser(
  {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "clusterMonitor", "readAnyDatabase" ]
  }
)

  • For source clusters running MongoDB version 3.2 a user must have, at a minimum, both clusterManager and readAnyDatabase roles, as well as read access on the local database. This requires a custom role, which you can create with the following commands:

    copy 
    use admin
db.createRole(
  {
    role: "migrate",
    privileges: [
      { resource: { db: "local", collection: "" }, actions: [ "find" ] }
    ],
    roles: ["readAnyDatabase", "clusterManager"]
  }
)
db.createUser(
  {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "migrate" ]
  }
)

  • For source clusters running MongoDB version 2.6 or 3.0 a user must have, at a minimum, the readAnyDatabase role. For example:

    copy 
    use admin
db.createUser(
  {
    user: "mySourceUser",
    pwd: "mySourceP@$$word",
    roles: [ "readAnyDatabase" ]
  }
)

Make note of the username and password for this user, as you must specify these credentials when running mongomirror.

2

Set up MongoDB user in the target Atlas cluster.

You must specify an Atlas MongoDB user with the Atlas admin role to run mongomirror. See Add MongoDB Users for documentation on creating an Atlas MongoDB user.

If no such user exists, create the user:

  1. In the Security section of the left navigation, click Database Access. The MongoDB Users tab displays.
  2. Click plus icon Add New User.
  3. Add an Atlas admin user.

Make note of the username and password selected for the new user, as you must specify these credentials when running mongomirror.

3

Open the connect dialog.

From the Clusters view, click Connect for the Atlas cluster into which you want to migrate data.

4

Update IP Whitelist.

If the host where you will run mongomirror is not in the IP Whitelist, update the list. You can specify either:

  • The public IP address of the server on which mongomirror will run, or
  • If set up for VPC peering, either the peer’s VPC CIDR block (or a subnet) or the peer VPC’s Security Group.
5

Copy the target cluster host information.

  1. Click Connect Your Application.
  2. Click I am using driver 3.4 or earlier.
  3. Copy the connection string and paste into your preferred text editor.

The connection string should look similar to the following example. This example has been broken into multiple lines for readability:

mongodb://<username>:<PASSWORD>@
  00.foo.mongodb.net:27017,
  01.foo.mongodb.net:27017,
  02.foo.mongodb.net:27017/test?
  ssl=true&replicaSet=myAtlasRS&authSource=admin

Copy the value of replicaSet and append the host list as comma separated values, similar to the following:

myAtlasRS/00.foo.mongodb.net:27017,01.foo.mongodb.net:27017,02.foo.mongodb.net:27017

Use this value for the --destination option in the next step.

6

Start mongomirror.

Start mongomirror with the following options:

Your final command should resemble the following example:

copy 
mongomirror --host "MySourceRS/host1.example.net:27017,host2.example.net:27017,host3.example.net:27017" \
   --ssl \
   --username "mySourceUser" \
   --password "mySourceP@$$word" \
   --authenticationDatabase "admin" \
   --destination "myAtlasRS/00.foo.mongodb.net:27017,01.foo.mongodb.net:27017,02.foo.mongodb.net:27017" \
   --destinationUsername "myAtlasAdminUser" \
   --destinationPassword "atlasPassword"

See Options for more mongomirror options.

When you start mongomirror:

  1. First, mongomirror performs an initial sync, copying collections from the existing MongoDB replica set to the target cluster in Atlas. mongomirror does not copy the config database.

    Note

    Starting in version 0.5.0, mongomirror builds all indexes on the destination cluster in the foreground, regardless of how the indexes were built on the source cluster. Foreground index builds block all other operations on the database. For more information on foreground index builds, see Index Build Operations on a Populated Collection.

  2. After the initial sync, mongomirror continuously tails the replica set’s oplog for incoming changes and replays them on the target cluster in MongoDB Atlas.

Once started, mongomirror runs continuously until you shut down the process. To fully migrate to the Atlas, continue on to Switch to Atlas steps.

Switch to Atlas

After the mongomirror has performed the initial sync and has been tailing the replica set’s oplog, to switch over to Atlas cluster:

1

Stop your application.

This ensures that no additional writes occur on the source cluster.

2

Wait for mongomirror to report 0s of lag between the source and destination clusters.

This signifies that the source and destination clusters are in a consistent state.

3

Stop mongomirror.

Once the Atlas cluster is up to date, stop mongomirror.

4

Update client applications to use the Atlas cluster.

Update your client application(s) with the Atlas connection string provided via the Connect button on the cluster panel.

For details on connections to Atlas, see Connect via Driver.

Additional Information

For more information on mongomirror, including behavior, options, and examples, see the mongomirror reference page.

© MongoDB, Inc 2008-present. MongoDB, Mongo, and the leaf logo are registered trademarks of MongoDB, Inc.