AI Lookup

Astera Data Stack introduces an innovative AI Lookup feature which leverages the power of Artificial Intelligence to perform intelligent lookup. This feature works based on semantic similarity, ensuring more accurate and comprehensive lookup results.

AI Lookup in Astera Data Stack

In Astera Data Stack, the AI Lookup object can be found in the Toolbox and can be used within the scope of the Dataflow.

Use Case

For our use case, we have a Customers dataset from the sales department as shown below:

Sales Dataset

We want to replace the values in the Country column of the sales dataset by semantically looking them up with Country values from the Customers dataset provided by the marketing team, ensuring both departments follow a unified naming standard.

Marketing Dataset

1. To get started, let’s drag-and-drop an Excel Workbook Source object and configure it with the customers dataset provided by the sales department.

1. Next, drag-and-drop the AI Lookup object from the Toolbox onto the Dataflow and map the field you want to perform lookup on from the Excel Workbook Source onto the AI Lookup object Value field.

1. Once the lookup field has been mapped, right-click on the AI Lookup object and select Properties from the context menu.

1. A File Lookup Map Properties window will open where you can see an option for Source File Type. Here, you need to specify the type of your lookup file and click Next.

1. The AILookup Transformation Properties screen will open, let’s configure these properties.

• File Path: This is where we provide the path of the file on the basis of which we want to perform our semantic lookup.

• Worksheet: This is where we can define which Excel sheet data to use if there are multiple sheets.

• Lookup Field: This is the field based on which we are performing the lookup.

For our use case, let’s select the Country Field as our lookup field.

1. Once configured, click Next. The object will perform an AI call and automatically create a list of the Incoming Value and its corresponding Lookup Value. We can also add or remove any values from this list if needed.

1. On the next window, you will see various Lookup Options.

• If Value Is Not Found In The Source File: In case no lookup value is found against a source value, you can choose one of the following three options to be appended with your output.

  • No Message: There will be no message, and the output will be the same as the input value.

  • Add Error: An error message will appear with the output.

  • Add Warning: A warning will appear with the output.

• If Value Is Not Found In The Source File, Assign Value: If no lookup value is found against a source value, you can assign an output value of your choice.

  • Assign Source Value: Will return the incoming source value in the output.

  • Assign Null: Will assign null to your output value.

  • This Value: You can select this option and assign any value of your choice.

1. For our use-case, we have kept the used the Assign Source Value option. Once configured, click OK.

2. Now, let’s drag-and-drop a Delimited File Destination object and map the data onto it.

1. Right-click on the destination object’s header and select Preview Output.

1. As you can see below, the values in the Country field have been semantically replaced from the file, using AI. Any missing values have just been assigned the source value as selected.

Running this Dataflow will write the data to the delimited file.

This concludes the working of the AI Lookup object in Astera Data Stack.