Navigation

Configure Your Database and Collections

Beta

The Atlas Data Lake is available as a Beta feature. The product and the corresponding documentation may change at any time during the Beta stage. For support, see Atlas Support.

Estimated completion time: 15 minutes

This part of the tutorial walks you through connecting to your Data Lake using the Mongo shell and setting the database and collection.

Prerequisites

To complete this part of the tutorial, you will need to have completed:

You must be connected to your Data Lake with the Mongo shell before beginning the following procedure.

Procedure

1

Copy the following storageSetConfig command to your preferred text editor.

use admin;
db.runCommand( { "storageSetConfig": {
  "stores": [{
    "s3": {
      "name": "s3store",           // Creates an S3 store
      "region":"<bucket-region>",  // Update with the bucket region code
      "bucket": "<bucket-name>"    // Update with your bucket name
    }
  }],
  "databases": {
    "sample": {                    // Creates a database named sample
      "*": [{                      /* Works with the defintion to create a
                                      collection the data in each directory */
        "store": "s3store",        // Links to the S3 store above
        "definition": "/json/{collectionName()}"
      }]
    }
  }
}})

The storage configuration maps a set of data stores to a set of databases and collections. A store is a set of objects in an S3 bucket under a specific prefix.

Example

In this example, the s3store contains all objects in the S3 bucket. We configured a sample database, and created a collection for the data in each /json/ subdirectory. To learn more about how the definition field maps data, see Data Lake Configuration.

Note

When you dynamically generate collections from filenames, the number of collections is not accurately reported in the Data Lake view.

2

Change the two highlighted lines to match your configuration.

Key Type Description Example
stores.s3.region string The AWS region code. us-east-1
stores.s3.bucket string The bucket name you chose in Part 1: Prepare your S3 Bucket. atlas-sample-data
3

Execute the storageSetConfig command.

  1. Copy the filled-out command from your text editor.
  2. Paste the command into the mongo shell.
  3. Execute the command.

Upon successful configuration, the mongo shell outputs the following:

{ "ok" : 1 }
3

Verify your database and collection mapping.

  1. Run the following command to display the mapped database:

    show dbs
    

    Upon successful configuration, the mongo shell outputs the following:

    sample  (empty)
    

    Note

    The (empty) in the output is expected.

  2. Switch to the sample database:

    use sample
    
  3. Run the following command to display the mapped collections:

    show collections
    

    Upon successful configuration, the mongo shell outputs the following:

    airbnb
    weather
    

Next Steps

Now that you mapped your data store to Data Lake databases and collections, we’re ready to run some queries. Proceed to Run Queries Against Your Data Lake.