Navigation

Create an Online Archive

You can create an online archive from the API and Atlas UI. You must have the GROUP_DATA_ACCESS_ADMIN (Project Data Access Admin) role to create an online archive.

The Atlas API authenticates using HTTP Digest Authentication. Provide a programmatic API public key and corresponding private key as the username and password when constructing the HTTP request.

To learn how to configure API access for an Atlas project, see Configure Atlas API Access.

Info With Circle IconCreated with Sketch.Note

Groups and projects are synonymous terms. Your {GROUP-ID} is the same as your project ID. For existing groups, your group/project ID remains the same. The resource and corresponding endpoints use the term groups.

https://cloud.mongodb.com/api/atlas/v1.0

POST /groups/{GROUP-ID}/clusters/{CLUSTER-NAME}/onlineArchives
Path ElementNecessityDescription
GROUP-IDRequiredUnique identifier for the project that contains the specified cluster.
CLUSTER-NAMERequiredName of the cluster that contains the collection for which you want to create an online archive.

The following query parameters are optional:

Query ParameterTypeDescriptionDefault
prettybooleanDisplays response in a prettyprint format.false
envelopebooleanSpecifies whether or not to wrap the response in an envelope.false
NameTypeNecessityDescription
collNamestringRequiredName of the collection for which to create an online archive.
dbNamestringRequiredName of the database that contains the collection.
criteriaobjectRequiredCriteria to use for archiving data.
criteria.typestringRequired

Type of criteria. Value can be one of the following:

  • DATE - to select documents for archiving based on a date.
  • CUSTOM - to select documents for archiving based on a custom JSON query.
criteria.dateFieldstringConditional

Required if criteria.type is DATE.

Name of an already indexed date field from the documents. Data is archived when the current date is greater than the value of the date field specified here plus the number of days specified via the expireAfterDays parameter.

criteria.dateFormatenumOptional

If criteria.type is DATE, the date format. Value can be one of the following:

  • ISODATE - ISO-8601 format date (default)
  • EPOCH_SECONDS - Unix timestamp in seconds
  • EPOCH_MILLIS - Unix timestamp in milliseconds
  • EPOCH_NANOSECONDS - Unix timestamp in nanoseconds

If omitted, defaults to ISODATE.

criteria.expireAfterDaysintConditional

Required if criteria.type is DATE.

Number of days that specifies the age limit for the data in the live Atlas cluster. Data is archived when the current date is greater than the value of the date field specified via the dateField parameter plus the number of days specified here.

criteria.querystringConditional

Required if criteria.type is CUSTOM.

JSON query to use to select documents for archiving. Atlas uses the specified query with the db.collection.find(query) command. The empty document {} to return all documents is not supported.

partitionFieldsdocument arrayOptional

Fields to use to partition data. You can specify up to two frequently queried fields to use for partitioning data. Note that queries that don’t contain the specified fields will require a full collection scan of all archived documents, which will take longer and increase your costs. To learn more about how partition improves query performance, see Data Structure in S3.

The value of a partition field can be up to a maximum of 700 characters. Documents with values exceeding 700 characters are not archived.

partitionFields.fieldNamestringRequiredName of the field. To specify a nested field, use the dot notation.
partitionFields.orderintRequired

Position of the field in the partition. Value can be:

  • 0 - for the first position
  • 1 - for the second position
  • 2 - for the third position

By default, the date field specified in the criteria.dateField parameter is in the first position of the partition.

NameTypeDescription
idstringID of the online archive.
clusterNamestringName of the cluster that contains the collection.
collNamestringName of the collection.
criteriadocumentCriteria to use for archiving data.
criteria.typestring

Type of criteria. Value can be one of the following:

  • DATE - to select documents for archiving based on a date.
  • CUSTOM - to select documents for archiving based on a custom JSON query.
criteria.dateFieldstringIf criteria.type is DATE, name of the date field that the online archive is based on. Data is archived when the current date is after the date specified here plus the number of days specified via the expireAfterDays parameter.
criteria.dateFormatenum

If criteria.type is DATE, the date format. Value can be one of the following:

  • ISODATE - ISO-8601 format date (default)
  • EPOCH_SECONDS - Unix timestamp in seconds
  • EPOCH_MILLIS - Unix timestamp in milliseconds
  • EPOCH_NANOSECONDS - Unix timestamp in nanoseconds
criteria.expireAfterDaysintIf criteria.type is DATE, number of days that specifies the age limit for the data in the live Atlas cluster. Data is archived when the current date is greater than the value of the date field specified via the dateField parameter plus the number of days specified here.
criteria.queryintIf criteria.type is CUSTOM, JSON query used to select documents for archiving. The specified query is used with the db.collection.find(query) command.
dbNamestringName of the database that contains the collection.
groupIdstringUnique identifier of the project that contains the cluster.
partitionFieldsdocument arrayFields to use to partition data.
partitionFields.fieldNamestringName of the field.
partitionFields.fieldTypestringData type of the field.
partitionFields.orderint

Position of the field in the partition. Value can be:

  • 0 - for the first position
  • 1 - for the second position
  • 2 - for the third position
pausedboolean

State of the online archive. Value is:

  • true if the online archive is in paused state.
  • false if the online archive is in pending or active state.
statestring

Status of the online archive. Valid values are:

PendingIndicates documents are queued for archive, but archiving has not yet started.
ArchivingIndicates archiving has started. In this state, the documents that meet the criteria for archiving are being archived.
IdleIndicates online archive is waiting for the next archival job to start.
PausedIndicates archiving has been temporarily stopped. In this state, previously archived documents continue to be available on the cloud object storage, but the specified archiving operation on the active cluster is put on hold. You can resume archiving for paused archives at any time.
OrphanedIndicates collection associated with an active or paused online archive was deleted. Atlas will not automatically delete the archived data. You must manually delete the online archives associated with the deleted collection.
DeletedIndicates online archive was deleted. When you delete an online archive, associated archived documents are removed from the cloud object storage.

The following example request creates an online archive for an example people.employees collection in a cluster named myTestClstr. Data in the collection will be archived based on the created date field and partitioned based on the firstName and lastName fields. The criteria for archiving data specifies that data should be archived when current date is greater than the value of the created date field plus 5 days.

Example Request

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
--header "Content-Type: application/json" \
--include \
--request POST "https://cloud.mongodb.com/api/atlas/v1.0/groups/5e2211c17a3e5a48f5497de3/clusters/myTestClstr/onlineArchives?pretty=true" \
--data '
{
"dbName": "people",
"collName": "employees",
"partitionFields": [
{
"fieldType": "string",
"fieldName": "firstName",
"order": 0
},
{
"fieldType": "string",
"fieldName": "lastName",
"order": 1
}],
"criteria": {
"type": "DATE",
"dateField": "created",
"dateFormat": "ISODATE",
"expireAfterDays": 5
}
}'

Example Response

{
"_id": "5ebad3c1fe9c0ab8d37d61e1",
"clusterName": "myTestClstr",
"collName": "employees",
"criteria": {
"type": "DATE",
"dateField": "created",
"dateFormat": "ISODATE",
"expireAfterDays": 5
},
"dbName": "people",
"groupId": "5e2211c17a3e5a48f5497de3",
"partitionFields": [
{
"fieldName": "firstName",
"fieldType": "string",
"order": 0
},
{
"fieldName": "lastName",
"fieldType": "string",
"order": 1
}
],
"paused": false,
"state": "PENDING"
}

The following example request creates an online archive for an example people.employees collection in a cluster named myTestClstr. Documents in the collection will be selected for archiving based on a custom query and data is partitioned based on the firstName and lastName fields.

Example Request

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
--header "Content-Type: application/json" \
--include \
--request POST "https://cloud.mongodb.com/api/atlas/v1.0/groups/5e2211c17a3e5a48f5497de3/clusters/myTestClstr/onlineArchives?pretty=true" \
--data '
{
"dbName": "people",
"collName": "employees",
"partitionFields": [
{
"fieldType": "string",
"fieldName": "firstName",
"order": 0
},
{
"fieldType": "string",
"fieldName": "lastName",
"order": 1
}],
"criteria": {
"type": "CUSTOM",
"query": "{ department: engineering }"
}
}'

Example Response

{
"_id": "5f9856146b3c0c4e454b9409",
"clusterName": "osTest",
"collName": "employees",
"criteria": {
"type": "CUSTOM",
"query": "{department: engineering}"
},
"dbName": "people",
"groupId": "5e2211c17a3e5a48f5497de3",
"partitionFields": [
{
"fieldName": "name.firstName",
"fieldType": null,
"order": 0
},
{
"fieldName": "name.lastName",
"fieldType": null,
"order": 1
}
],
"paused": false,
"state": "PENDING"
}

If the request fails, it returns one of the following errors:

Error CodeDescription
INVALID_GROUP_IDThe specified project ID is not valid.
CLUSTER_NOT_FOUNDThere is no cluster with the specified name in the specified project.
CLUSTER_DELETE_REQUESTEDThe cluster for the online archive is being deleted.
ONLINE_ARCHIVE_ALREADY_EXISTSAn online archive with identical settings already exists.
NAMESPACE_HAS_ONLINE_ARCHIVEThe collection has another active online archive and so, an online archive with the specified criteria can't be created.
RESOURCE_NOT_FOUNDThe specified archive ID is not valid.
Give Feedback