Introducing Transformations

Introducing Transformations

Transformations are used to perform a variety of operations on data as it moves through the dataflow pipeline. Astera Data Stack provides a full complement of built-in transformations enabling you to build sophisticated data maps.

Astera Data Stack transformations are divided into two types— record-level and set-level.

Record-level Transformations

Record-level transformations are used to create derived values by applying a Lookup, Function, or Expression to fields from a single record.

Examples of record-level transformations include Lookups, Expressions, and Function transformations.

In Astera, data records flow between set transformations. Record-level transformations are used to transform or augment individual fields during these movements.

A record-level transformation can be connected to only one set transformation. For instance, a Lookup or Expression cannot receive input from two different set transformations.

Set-level Transformations

Set-level transformations, on the other hand, operate on a group of records and may result in the Joining, Reordering, Eliminating, or aggregating of records. Examples of set transformations include Join, Sort, Route, Distinct, etc. Data Sources and Destinations are also considered set transformations.

Other than transformations that enable combining multiple data sources—such as Join, Merge, and Union—transformations can receive input from only one set transformation. Transformations, however, can receive input from any number of record-level transformations as long as all these record-level transformations receive input from the same transformation.