Documentation Index

Fetch the complete documentation index at: https://docs.dataddo.com/llms.txt

Use this file to discover all available pages before exploring further.

Transformation Pipeline Stages

  • 3 minute(s) read
Prev Next

On this page, you will find a comprehensive list of transformation pipeline stages and their options.

Transformation Pipeline Stages

Stage Name Description
$addFields Adds new fields to documents.
$bucket Groups documents into specified buckets based on an expression and boundaries. Options available.
$bucketAuto Automatically distributes documents into a specified number of even buckets. Options available.
$changeStream Returns a Change Stream cursor; must be the first stage in the pipeline.
$collStats Provides statistics about a collection or view.
$count Counts the documents at this pipeline stage.
$currentOp Displays active and/or dormant operations. Requires the db.aggregate() method. Options available.
$facet Executes multiple aggregation pipelines in one stage, enabling multi-faceted aggregations.
$geoNear Orders documents based on geospatial proximity, incorporating $match, $sort, and $limit. Options available.
$graphLookup Conducts a recursive search, appending results to the output document as an array. Options available.
$group Groups documents by an expression, applying accumulator expressions to each group.
$indexStats Reports usage statistics for each index in the collection.
$limit Passes the first n documents unmodified to the pipeline.
$listLocalSessions Lists active sessions on the connected mongos or mongod instance.
$listSessions Lists sessions propagated to the system.sessions collection.
$lookup Performs a left outer join with another collection in the same database. Options available.
$match Filters documents, passing only matches to the next stage.
$merge Writes aggregation results to a collection; must be the last stage in the pipeline. Options available.
$out Writes aggregation results to a collection; must be the last stage. Options available.
$planCacheStats Provides plan cache data for a collection.
$project Reshapes each document, adding or removing fields.
$redact Reshapes documents based on their content, allowing field level redaction.
$replaceRoot Replaces a document with a specified embedded one, promoting the embedded document.
$replaceWith Alias for $replaceRoot. Replaces a document with a specified embedded one.
$sample Randomly selects a specified number of documents. Options available.
$search Executes a full-text search.
$set Alias for $addFields. Adds new fields to documents.
$skip Skips the first n documents, passing the rest unmodified.
$sort Reorders documents based on a specified sort key.
$sortByCount Groups documents by an expression's value and counts them.
$unionWith Combines results from two collections into one. Options available.
$unset Removes specified fields from documents.
$unwind Deconstructs an array field, outputting a document for each element. Options available.

Stages Options

Stage Option
$bucket groupBy: An expression to group by.
boundaries: An array specifying the boundaries to group documents.
default: A literal that specifies the _id of an additional group for all documents that do not match any boundary.
$bucketAuto groupBy: An expression to group by.
buckets: The number of buckets to create.
$currentOp allUsers: A boolean indicating if operations from all users should be returned.
localOps: A boolean indicating if only operations on the local mongod instance should be returned.
$geoNear near: The geospatial point to calculate distances from.
distanceField: The field that will contain the calculated distance.
spherical: A boolean specifying whether to treat the near point as a sphere.
maxDistance: The maximum distance from the near point that the documents can be.
$graphLookup from: The target collection for the recursive search.
startWith: The expression that specifies the value of the connectFromField with which to start the recursive search.
connectFromField: The field whose value startWith references.
connectToField: The field in the documents of the from collection that is equal to the connectFromField.
as: The array field added to each output document that contains the search results.
$lookup from: The target collection.
localField: The field from the input documents.
foreignField: The field from the documents of the from collection.
as: The name of the new array field to add to the input documents.
$merge into: The target collection.
on: The fields to match on.
whenMatched: The action to take when a matching document exists.
whenNotMatched: The action to take when no matching document exists.
$out collection: The target collection.
$sample size: The number of documents to randomly select.
$unionWith coll: The name of the collection to union with.
pipeline: An optional aggregation pipeline to apply to the specified collection before performing the union.
$unwind path: The path to the array field to unwind.
includeArrayIndex: The name of a new field to hold the array index of the element.
preserveNullAndEmptyArrays: A boolean indicating whether to include as an output document those documents that have an empty array or missing array field.