Navigation

Migrate with mongomirror

mongomirror is a utility for migrating data from an existing MongoDB replica set to a MongoDB Atlas replica set. mongomirror does not require you to shut down your existing replica set or applications.

Important

mongomirror does not import user/role data.

Prerequisites

Source

  • The source MongoDB deployment must be a replica set. If the source is a standalone MongoDB deployment, convert to a replica set first.
  • The source replica set must be version 3.0 or greater.

Target

  • The target Atlas cluster must be a replica set.
  • The target Atlas cluster should be empty or run mongomirror with the --drop option.
  • The target Atlas cluster cannot be an M0 (Free Tier) cluster.

Procedure

1

Set up MongoDB user in the source replica set.

If the source replica set requires authentication, you must specify a source MongoDB user that has privileges to read any database, including the local database. For example, a user with backup role provides these privileges.

If the source replica set requires authentication and no such user exists, create the user in your source MongoDB replica set. For example, if the source replica set uses SCRAM-SHA1 authentication:

  1. Connect a mongo shell to the primary.

  2. Switch to the admin database and create a user with the backup role.

    use admin
db.createUser( { user: "mySourceUser", pwd: "mySourceP@$$word", roles: [ "backup" ] } )
2

Set up MongoDB user in the target Atlas cluster.

To run mongomirror, you must specify a MongoDB user that has readWriteAnyDatabase and dbAdminAnyDatabase privileges in the Atlas cluster. For example, a user with Atlas admin role provides these privileges.

If no such user exists, create the user:

  1. Go to Clusters view, select the Security tab, and then Users.
  2. Click Add New User.
  3. Add an Atlas admin user.
3

Open the connect dialog.

From the Clusters view, click Connect for the Atlas cluster into which you want to migrate data.

4

Update IP Whitelist.

If the host where you will run mongomirror is not in the IP Whitelist, update the list. You can specify either:

  • The public IP address of the server on which mongomirror will run, or
  • If set up for VPC peering, either the peer’s VPC CIDR block (or a subnet) or the peer VPC’s Security Group.
5

Copy the target cluster host information.

From the connect dialog, get the cluster host information. This information is specified to mongomirror in the --destination option.

You must also retrieve the replica set name. For example:

myAtlasRS/atlas-host1:27017,atlas-host2:27017,atlas-host3:27017
6

Start mongomirror.

From a terminal, start mongomirror with the appropriate options.

For example, if the source replica set uses SCRAM-SHA1 authentication,

mongomirror --host sourceRS/source-host1:27017,source-host2:27017,source-host3:27017 \
   --user mySourceUser \
   --password 'mySourceP@$$word' \
   --authenticationDatabase admin \
   --destination myAtlasRS/atlas-host1:27017,atlas-host2:27017,atlas-host3:27017 \
   --destinationUsername myAtlasAdminUser \
   --destinationPassword 'atlasP@$$assw0Rd'

See Options for more mongomirror options.

When you start mongomirror:

  1. First, mongomirror performs an initial sync, copying collections from the existing MongoDB replica set to the target cluster in Atlas.
  2. After the initial sync, mongomirror continuously tails the replica set’s oplog for incoming changes and replays them on the target cluster in MongoDB Atlas. See oplog Entries.

Once started, mongomirror runs continuously until you shut down the process. To fully migrate to the Atlas, continue on to Switch to Atlas steps.

Switch to Atlas

After the mongomirror has performed the initial sync and has been tailing the replica set’s oplog, to switch over to Atlas cluster:

1

Stop write operations to the source replica set.

2

Wait for the Atlas cluster to be up to date with the last writes.

Continue to run mongomirror to finish tailing the oplog.

3

Stop mongomirror.

Once the Atlas cluster is up to date, stop mongomirror.

4

Update client applications to use Atlas cluster.

Update client applications to connect to the Atlas cluster.

For details on connections to Atlas, see Connect via Driver.

oplog Entries

Because mongomirror tails the source oplog and applies the entries to the destination cluster, the destination oplog is not an exact duplicate of the source’s oplog. Instead, the tailed entries from the source oplog become part of an applyOps entry in the destination oplog.

For example, after mongomirror has performed the initial sync, the source replica set receives three insert operations and has the following oplog entries for these operations:

{"ts":<ts1>,"t":<t1>,"h":<h1>,"v":2,"op":"i","ns":"test.foo","o":{"_id":0,"a":0}}
{"ts":<ts2>,"t":<t2>,"h":<h2>,"v":2,"op":"i","ns":"test.foo","o":{"_id":1,"a":1}}
{"ts":<ts3>,"t":<t3>,"h":<h3>,"v":2,"op":"i","ns":"test.foo","o":{"_id":2,"a":2}}

As mongomirror tails the source oplog and applies these operations to the destination cluster, the three entries become part of a single entry in the destination cluster’s oplog:

{"ts":<ts>,"t":<t>,"h":<h>,"v":2,"op":"c","ns":"admin.$cmd","o":{"applyOps":
  [
    {"ts":<ts1>,"t":<t1>,"h":<h1>,"v":2,"op":"i","ns":"test.foo","o":{"_id":0,"a":0},"o2":{ }},
    {"ts":<ts2>,"t":<t2>,"h":<h2>,"v":2,"op":"i","ns":"test.foo","o":{"_id":1,"a":1},"o2":{ }},
    {"ts":<ts3>,"t":<t3>,"h":<h3>,"v":2,"op":"i","ns":"test.foo","o":{"_id":2,"a":2},"o2":{ }}
  ]
} }

For applications that tail or parse the oplog, if you switch these applications to read from the destination cluster’s oplog, you may need to modify the applications, depending on how you wish to handle these applyOps entries.

If you have not switched over to writing to the destination cluster, you can continue to read from the source without modifying these applications.

Additional Information

For more information on mongomirror, including behavior, options, and examples, see the mongomirror reference page.

© MongoDB, Inc 2011-2017. MongoDB, Mongo, and the leaf logo are registered trademarks of MongoDB, Inc.