- 12 Minutes to read

- DarkLight

# Transformation Pipeline Expressions

- 12 Minutes to read

- DarkLight

On this page, you will find a comprehensive list of the following data transformation pipeline expressions:

For more information on types of expressions, refer to the article on Transformation Pipeline.

## Aggregation Variables

Variable | Description |
---|---|

`$$NOW` | Returns the current datetime value. Remains constant across all stages of the aggregation pipeline. |

`$$CLUSTER_TIME` | Provides the current timestamp in replica sets and sharded clusters. Consistent across all deployment members and throughout the pipeline stages. |

`$$ROOT` | Refers to the top-level document currently in the aggregation pipeline stage. |

`$$CURRENT` | Points to the start of the field path being processed. It starts the same as `ROOT` and is modifiable, which affects the interpretation of `$` field path accesses. |

`$$REMOVE` | Evaluates to a missing value to conditionally exclude fields. Used in `$project` to omit fields. |

`$$DESCEND` | An allowable outcome of a `$redact` stage. |

`$$PRUNE` | Another possible result of a `$redact` stage, indicating the removal of data. |

`$$KEEP` | An outcome of `$redact` stage indicating that the data should be retained. |

## Operator Expressions

**Basic Operations**

**Data Types and Structures**

**Advanced Operations**

**Accumulators**

### Arithmetic Expression Operators

Arithmetic expression operators are mathematical operations on numbers, with some supporting date arithmetic.

Operator | Description |
---|---|

`$abs` | Returns the absolute value of a number. |

`$add` | Adds numbers or adds milliseconds to a date. Accepts multiple arguments with at most one date. |

`$ceil` | Rounds up to the nearest integer. |

`$divide` | Divides the first number by the second. Requires two arguments. |

`$exp` | Raises e to a given exponent. |

`$floor` | Rounds down to the nearest integer. |

`$ln` | Computes the natural log of a number. |

`$log` | Computes the log of a number for a given base. |

`$log10` | Computes the base 10 logarithm of a number. |

`$mod` | Returns the remainder after division. Requires two arguments. |

`$multiply` | Multiplies numbers. Accepts multiple arguments. |

`$pow` | Raises a number to a given exponent. |

`$round` | Rounds a number to the nearest integer or specified decimal place. |

`$sqrt` | Computes the square root of a number. |

`$subtract` | Subtracts numbers, dates, or subtracts milliseconds from a date. Requires two arguments, with date first if used. |

`$trunc` | Truncates a number to an integer or specified decimal place. |

### Array Expression Operators

Operator | Description |
---|---|

`$arrayElemAt` | Retrieves the element at a specified array index. |

`$arrayToObject` | Transforms an array of key-value pairs into a document. |

`$concatArrays` | Concatenates multiple arrays. |

`$filter` | Filters an array based on a condition. |

`$first` | Retrieves the first array element (different from `$first` accumulator). |

`$in` | Checks if a value exists in an array; returns boolean. |

`$indexOfArray` | Finds the first occurrence of a value in an array or returns -1 if not found. |

`$isArray` | Checks if the operand is an array; returns boolean. |

`$last` | Retrieves the last array element (different from `$last` accumulator). |

`$map` | Applies a subexpression to each array element, returning an array of results. |

`$objectToArray` | Converts a document into an array of key-value pair documents. |

`$range` | Generates an array of integers based on user-defined parameters. |

`$reduce` | Combines array elements using an expression into a single value. |

`$reverseArray` | Reverses the order of array elements. |

`$size` | Counts elements in an array. |

`$slice` | Extracts a portion of an array. |

`$zip` | Merges two arrays into one. |

### Boolean Expression Operators

Boolean expressions return a boolean result based on their arguments. They interpret `null`

, `0`

, and `undefined`

as `false`

. All other values, including non-zero numbers and arrays, are interpreted as `true`

.

Operator | Description |
---|---|

`$and` | Returns true if all argument expressions are true. |

`$not` | Inverts the boolean value of its argument. |

`$or` | Returns true if at least one of its argument expressions is true. |

### Comparison Expression Operators

Comparison expressions compare both value and type of two arguments, using the BSON comparison order for different types. `$cmp`

uniquely returns a number, while others return a boolean.

Operator | Description |
---|---|

`$cmp` | Outputs 0 for equivalent values, 1 if the first is greater, and -1 if the first is less. |

`$eq` | True when values are equivalent. |

`$gt` | True when the first value exceeds the second. |

`$gte` | True when the first value is greater or matches the second. |

`$lt` | True when the first value is smaller than the second. |

`$lte` | True when the first value is smaller or matches the second. |

`$ne` | True when values are not equivalent. |

### Conditional Expression Operators

Operator | Description |
---|---|

`$cond` | Evaluates the first expression and, based on its result, returns the value from either the second or third expression. Can accept three ordered expressions or three named parameters. |

`$ifNull` | If the first expression is null, undefined, or missing, returns the result of the second expression. Accepts two expressions; the second one can also return null. |

`$switch` | Checks multiple case expressions. On the first true case, executes the corresponding expression and exits. |

### Custom Aggregation Expression Operators

Operator | Description |
---|---|

`$accumulator` | Allows creation of a custom accumulator for use in aggregation pipelines. |

`$function` | Allows creation of a custom JavaScript function for use in expressions. |

### Data Size Operators

Operator | Description |
---|---|

`$binarySize` | Returns the byte size of a string or binary data value's content. |

`$bsonSize` | Returns the byte size of a document when encoded as BSON (bsontype Object). |

### Date Expression Operators

Operator | Description |
---|---|

`$dateFromParts` | Constructs a BSON Date object from its constituent parts. |

`$dateFromString` | Converts a date/time string to a BSON Date object. |

`$dateToParts` | Returns the constituent parts of a date as a document. |

`$dateToString` | Formats the date as a string. |

`$dayOfMonth` | Returns the day of the month as a number (1-31). |

`$dayOfWeek` | Returns the weekday number (1 for Sunday to 7 for Saturday). |

`$dayOfYear` | Returns the day number of the year (1-366). |

`$hour` | Returns the hour component (0-23). |

`$isoDayOfWeek` | Returns the weekday number in ISO 8601 format (1 for Monday to 7 for Sunday). |

`$isoWeek` | Returns the ISO 8601 week number (1-53). |

`$isoWeekYear` | Returns the year number as per ISO 8601. |

`$millisecond` | Returns the milliseconds component (0-999). |

`$minute` | Returns the minute component (0-59). |

`$month` | Returns the month number (1 for January to 12 for December). |

`$second` | Returns the seconds component (0-60). |

`$toDate` | Converts a value to a Date. New in version 4.0. |

`$week` | Returns the week number (0 for the partial week before the first Sunday to 53 for a leap year). |

`$year` | Returns the year as a number. |

It is possible to use the following arithmetic operators to work with date operands.

Operator | Description |
---|---|

`$add` | Adds numbers to a date, treating numbers as milliseconds. At most, one argument can resolve to a date. |

`$subtract` | Subtracts values; if both are dates, returns difference in milliseconds. With a date and number, returns the resulting date. Specify the date first when subtracting a number from it. |

### Literal Expression Operator

Operator | Description |
---|---|

`$literal` | Returns a value without interpretation. Useful for values that may be mistaken as expressions, especially strings starting with $. Prevents accidental field path parsing. |

### Miscellaneous Operators

Operator | Description |
---|---|

`$rand` | Returns a random float between 0 and 1. |

`$sampleRate` | Selects documents randomly based on a given rate. The chosen count is roughly the rate's percentage of the total documents. |

### Object Expression Operators

Operator | Description |
---|---|

`$mergeObjects` | Combines multiple documents into one. |

`$objectToArray` | Converts a document into an array of documents representing key-value pairs. |

### Set Expression Operators

Set expressions operate on arrays, treating them as sets. They filter out any duplicate entries within each array and do not guarantee a specific order for the elements in the resulting array. Nested arrays are evaluated at the top level without delving into their individual elements.

Operator | Description |
---|---|

`$allElementsTrue` | Evaluates if all elements in a set are true. Accepts a single argument. |

`$anyElementTrue` | Evaluates if at least one element in a set is true. Accepts a single argument. |

`$setDifference` | Returns elements that are in the first set but not in the second, essentially performing a relative complement. Requires two arguments. |

`$setEquals` | Checks if the input sets contain the same distinct elements. Can accept two or more arguments. |

`$setIntersection` | Returns elements common to all input sets. Can accept any number of arguments. |

`$setIsSubset` | Checks if all elements of the first set are present in the second, but not necessarily vice versa. Requires two arguments. |

`$setUnion` | Combines and returns elements from all input sets without duplicates. |

### String Expression Operators

String expressions primarily operate with well-defined behavior on ASCII character strings. However, the `$concat`

function is an exception, exhibiting consistent behavior regardless of the character set.

Operator | Description |
---|---|

`$concat` | Concatenates any number of strings. |

`$dateFromString` | Converts a date/time string to a date object. |

`$dateToString` | Returns the date as a formatted string. |

`$indexOfBytes` | Searches a string for a substring and returns the UTF-8 byte index of the first occurrence. Returns -1 if not found. |

`$indexOfCP` | Searches a string and returns the UTF-8 code point index of the first occurrence of a substring. Returns -1 if not found. |

`$ltrim` | Removes whitespace or specified characters from the beginning of a string. |

`$regexFind` | Applies a regex to a string and returns information on the first matched substring. |

`$regexFindAll` | Applies a regex to a string and returns information on all matched substrings. |

`$regexMatch` | Applies a regex to a string and returns a boolean indicating if a match is found. |

`$replaceOne` | Replaces the first instance of a matched string in the input. |

`$replaceAll` | Replaces all instances of a matched string in the input. |

`$rtrim` | Removes whitespace or specified characters from the end of a string. New in version 4.0. |

`$split` | Splits a string based on a delimiter and returns an array of substrings. |

`$strLenBytes` | Returns the number of UTF-8 encoded bytes in a string. |

`$strLenCP` | Returns the number of UTF-8 code points in a string. |

`$strcasecmp` | Performs case-insensitive string comparison and returns 0 if equivalent, 1 if the first is greater, -1 if less. |

`$substr` | Deprecated. Use `$substrBytes` or `$substrCP` instead. |

`$substrBytes` | Returns a substring starting at a specified UTF-8 byte index and continues for a specified number of bytes. |

`$substrCP` | Returns a substring starting at a specified UTF-8 code point index and continues for a specified number of code points. |

`$toLower` | Converts a string to lowercase. |

`$toString` | Converts a value to a string. |

`$trim` | Removes whitespace or specified characters from the start and end of a string. |

`$toUpper` | Converts a string to uppercase. |

### Text Expression Operators

Operator | Description |
---|---|

`$meta` | Retrieves specific metadata about a document during aggregation, such as the relevance score from a text search. |

### Trigonometry Expression Operators

Trigonometry expressions handle trigonometric operations on numerical values. When working with angles, these expressions always use radians. For conversions between degrees and radians, utilize `$degreesToRadians`

and `$radiansToDegrees`

.

Operator | Description |
---|---|

`$sin` | Returns the sine of a value measured in radians. |

`$cos` | Returns the cosine of a value measured in radians. |

`$tan` | Returns the tangent of a value measured in radians. |

`$asin` | Returns the inverse sin (arc sine) of a value in radians. |

`$acos` | Returns the inverse cosine (arc cosine) of a value in radians. |

`$atan` | Returns the inverse tangent (arc tangent) of a value in radians. |

`$atan2` | Returns the inverse tangent (arc tangent) of y / x in radians, where y and x are the first and second values passed to the expression, respectively. |

`$asinh` | Returns the inverse hyperbolic sine (hyperbolic arc sine) of a value in radians. |

`$acosh` | Returns the inverse hyperbolic cosine (hyperbolic arc cosine) of a value in radians. |

`$atanh` | Returns the inverse hyperbolic tangent (hyperbolic arc tangent) of a value in radians. |

`$sinh` | Returns the hyperbolic sine of a value measured in radians. |

`$cosh` | Returns the hyperbolic cosine of a value measured in radians. |

`$tanh` | Returns the hyperbolic tangent of a value measured in radians. |

`$degreesToRadians` | Converts a value from degrees to radians. |

`$radiansToDegrees` | Converts a value from radians to degrees. |

### Type Expression Operators

Operator | Description |
---|---|

`$convert` | Converts a value to a specified type. |

`$isNumber` | Returns `true` if the expression resolves to an integer, decimal, double, or long; otherwise, returns `false` . |

`$toBool` | Converts a value to a boolean. |

`$toDate` | Converts a value to a date. |

`$toDecimal` | Converts a value to Decimal128. |

`$toDouble` | Converts a value to a double. |

`$toInt` | Converts a value to an integer. |

`$toLong` | Converts a value to a long. |

`$toObjectId` | Converts a value to an ObjectId. |

`$toString` | Converts a value to a string. |

`$type` | Returns the BSON data type of the field. |

### Accumulators ($group)

In the `$group`

stage{target=`_blank`

} of the aggregation pipeline, accumulators are essential operators that keep track of data states such as totals, peaks, troughs, and more. Each accumulator evaluates an expression for every document in its group. The unique feature of these operators is their ability to retain their state across documents with the same group key.

Operator | Description |
---|---|

`$accumulator` | Returns the result of a user-defined accumulator function. |

`$addToSet` | Returns an array of unique expression values for each group. |

`$avg` | Calculates the average of numerical values. Excludes non-numeric values. |

`$first` | Retrieves a value from the first document in each group. Defined order is necessary for consistent results. |

`$last` | Retrieves a value from the last document in each group. Defined order is necessary for consistent results. |

`$max` | Identifies the highest expression value in each group. |

`$mergeObjects` | Combines input documents to create a single document for each group. |

`$min` | Identifies the lowest expression value in each group. |

`$push` | Produces an array of expression values for each group. |

`$stdDevPop` | Calculates the population standard deviation of input values. |

`$stdDevSamp` | Calculates the sample standard deviation of input values. |

`$sum` | Sums up numerical values in each group. Excludes non-numeric values. |

### Accumulators (in Other Stages)

Some operators that are available as accumulators for the `$group`

stage{target=`_blank`

} are also available for use in other stages but not as accumulators. When used in these other stages, these operators do not maintain their state and can take as input either a single argument or multiple arguments.

The following accumulator operators are available in the `$project`

, `$addFields`

, and `$set`

stages.

Name | Description |
---|---|

`$avg` | Computes an average for the given expression or set of expressions per document, ignoring non-numeric values. |

`$first` | Retrieves a value from the initial document in each group. Order matters if documents follow a specific sequence. |

`$last` | Gleans a value from the last document in each group. Document sequence matters for a defined order. |

`$max` | Determines the peak value from the provided expression or set of expressions for every document. |

`$min` | Identifies the lowest value from the designated expression or collection of expressions per document. |

`$stdDevPop` | Computes the overall standard deviation from the input values. |

`$stdDevSamp` | Calculates the sample standard deviation from the input values. |

`$sum` | Aggregates the sum of numerical entries, excluding non-numeric ones. |

### Variable Expression Operators

Name | Description |
---|---|

`$let` | Defines variables for use in a subexpression's scope and returns its result. It accepts named parameters and can take any number of argument expressions. |