Using Output Variables in Astera
Last updated
Last updated
The Variables object in Astera allows users to create variables that can be used in multiple objects within a dataflow or workflow. The user can assign a fixed value to a variable or allocate it dynamically, depending on the requirement. An output variable is used when you need to pass a certain value from an inner flow (dataflow) to an outer flow (workflow).
In this case, we have a Customers dataset with LateArrivals and TotalOrders fields that contain the count of orders that arrived late and the total number of orders for each customer, respectively. We want to find out the following:
The total number of orders, by summing all the entries in the TotalOrders column, for each customer.
The total number of orders which were shipped late, by summing all the entries in the LateArrivals column.
The percentage of orders which were shipped late, by using the following expression: PercentageLate = (LateArrivals/TotalOrders) * 100.
Finally, we want to notify the administration by sending them an automated e-mail if the PercentageLate is greater than a fixed Threshold value.
To achieve our desired functionality, we have created a dataflow with the following objects:
Delimited File Source: To read the Customers dataset from the CSV file.
Aggregate Transformation: To sum the values in the LateArrivals and TotalOrders columns.
Expression Transformation: To calculate PercentageLate.
Variables: To set the output variables, PercentLate and Threshold.
Drag and drop the Variables object from the Resources section of the Toolbox onto the designer.
Right-click on the Variables object and select Properties. Alternatively, you can also double-click on the object’s header to open Properties.
A Variables Properties window will open, where you can specify the Name, Data Type, Variable Type, Default Value, and Parameter Path of the variable.
Specify appropriate values for Name, Data Type, Variable Type, and Default Value (optional) for the variables you want to create. In this case, we want to create output variables, PercentLate and Threshold.
Go to the drop-down menu in the Parameter Path column and expand the Expression section. Select PercentageLate.
PercentageLate will only appear in this drop-down menu if it is marked as a Variable in the properties of the Expression object, as shown below.
To set the value of the variable, Threshold, we will create a new variable, SetThreshold, and use the $SetVariable(String Variable, Any Value) function in the Layout Builder of the Expression object. Make sure to specify the parameters of the function with the correct syntax.
Here, we have set the value of Threshold to 9.99. Click OK to close this window.
Note: The Set Variable functionality works in two cases only:
When there is a set-level transformation (purple box) in the flow. Here, we have used an Aggregate transformation, which is a set transformation, in the dataflow. If you do not have a set-level transformation in your flow, you can add a Passthru transformation and pass the data from the Expression object to it.
Alternatively, in case you do not want to use a set-level transformation, you can uncheck all the checkboxes in the Output column in the Layout Builder of the Expression object for this functionality to work.
We have successfully configured the Variables object in the dataflow.
We will now complete our desired use case by using workflow tasks. We can run the dataflow in a workflow using the Run Dataflow object. In our case, we have made the following workflow (shown below) to notify the admin if the PercentLate exceeds the Threshold value. Notice that the PercentLate and Threshold variables appear in the Run Dataflow object because they are defined as output variables.
The output of the Decision object depends on the following conditions:
If the PercentLate is greater than the defined Threshold, our Decision object will return True, and the SendMail task will be executed. Here is a job trace of the workflow:
The Decision object has returned False as our PercentLate variable, 4.4%, is less than our specified Threshold of 9.99%.
This is how you can use the Variables object to create and use output variables in Astera.