# Database Lookup Transformation

The *Database Lookup* object in Astera is used to look up values from a source. It uses a database table that contains the lookup values as well as a set of corresponding output values.

When the lookup is performed, the object returns either a single output field or multiple output fields, depending on the nature of the lookup table. Similarly, the lookup can be performed based on one lookup field or multiple lookup fields. In each case, the output field or fields are returned from the records in which the lookup values match the incoming values.

{% embed url="<https://youtu.be/qJQLNRUgJLo>" %}

## **Use Case**

In this use case, we have a sample *Customers* dataset that is stored in a database table. Within this dataset, there is a field that contains the country of residence for each customer. We have another database table that contains all of these countries and their corresponding codes.

Our goal is to replace the full country names with codes while writing the customer dataset to an Excel file. To do this, we will use a *Database Lookup* object.

## **Working with Database Lookup**

1. Drag the [relevant source](https://documentation.astera.com/v/astera-data-stack-v8/dataflows/sources/database-table-source) object from the *Toolbox* and drop it onto the designer. In this case, we will select the *Database Table Source* object from *Toolbox > Sources > Database Table Source* and configure it so that it reads data from the *Customers* dataset.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F6LPUOSAIUEzhrwMyeA3u%2Fimage.png?alt=media&#x26;token=716a8641-9c22-442c-887a-de4e1eb13d9f" alt=""><figcaption></figcaption></figure>

To preview the data, right-click on the object header and select *Preview Output* from the context menu. Here, you can see that there is a field that contains each customer’s country of residence.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FQsn2Pjzg3I3oBedIi2Rt%2Fimage.png?alt=media&#x26;token=7a0762f2-8720-4e1f-a6e5-6687d9633c78" alt=""><figcaption></figcaption></figure>

2. Drag the *Database Lookup* object from *Toolbox > Transformations > Database Lookup* and drop it onto the designer.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FpsPqkSstWj89RmwdoW9Y%2FDatabase%20Lookup%20Gif%201.gif?alt=media&#x26;token=23a52de2-413f-42cc-8cc0-d19768c77fe9" alt=""><figcaption></figcaption></figure>

3. Right-click on the header of the *Database Lookup* object and select *Properties* from the context menu.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FXrLA3xFjK5zZXiDKmbQE%2Fimage.png?alt=media&#x26;token=82b1cead-ea23-445d-b146-9fec42f8162f" alt=""><figcaption></figcaption></figure>

This will open a new window on your screen.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FY6l4idcYz2vOyRk5mZmb%2Fimage.png?alt=media&#x26;token=28916f9f-af66-4cbb-9637-75865d548649" alt=""><figcaption></figcaption></figure>

Here, you are required to configure the properties for the *Database Lookup* object.

4. On the *Database Connection* window, enter the details for the database you wish to connect to.

* Use the *Data Provider* drop-down list to specify which database provider you wish to connect to. The required credentials will vary according to your chosen provider.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F506lWts6GSTXWeCMpPea%2Fimage.png?alt=media&#x26;token=395b4bbf-aef2-4150-8a8f-551ff2405082" alt=""><figcaption></figcaption></figure>

* Provide the required credentials. Alternatively, use the *Recently Used* drop-down list to connect to a recently connected database.
* *Test Connection* to ensure that you have successfully connected to the database. A new window will open, showing whether your test is successful or has ended in an error. When the connection has been successfully established, close it by clicking *OK*, and then click *Next*.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FSPILcuU4INtB6PKy8m55%2Fimage.png?alt=media&#x26;token=9f71792c-237c-4ee0-a086-6c44f4d11a92" alt=""><figcaption></figcaption></figure>

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FePk77gVza9W2JpM2pvcR%2Fimage.png?alt=media&#x26;token=92d9eede-9fce-4e1e-9829-5445ec8202ea" alt=""><figcaption></figcaption></figure>

5. The next window is the *Database Lookup Map Properties* window. Here, you can pick a table from the database that you have connected to.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F2MtKygpmpF5lQPXXooKI%2Fimage.png?alt=media&#x26;token=4c351043-8584-4c64-a845-d8920ba1cad3" alt=""><figcaption></figcaption></figure>

In this case, we will select the table named *CountryCodeLookup*. This table contains the code for each country and will serve as the lookup table in our use case.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2Fga57gVStKPoehwgVS7K9%2Fimage.png?alt=media&#x26;token=89c51dca-cf94-4691-b96d-bbe67898f585" alt=""><figcaption></figcaption></figure>

In the text box provided under the *Pick Table* option, you can enter a *where clause* to modify the lookup query. In this case, we will leave it empty.

Once you have chosen a table, click *Next*.

6. On the next window, you can choose a *Lookup Cache Type* from the following options:

* *No Caching*: No data will be stored in a cache. This option is selected by default.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F9RPbd7h4R9bcHMS1PSUF%2Fimage.png?alt=media&#x26;token=d6e43d4c-a5ff-4648-84e4-2da0b032c5b1" alt=""><figcaption></figcaption></figure>

* *Static*: The lookup values are stored in a cache. Once the cache is created, the transformation object will always query the cache instead of the lookup table. When you select this option, the following sub-options are enabled:
  * *Fill Cache With All Lookup Values at Start*: Fills the cache with all of the lookup values at the start and continues to use this cache for every lookup.
  * *Cache After First Use*: Uses the database table for the first lookup and fills the cache right after it is done. This cache is then used for every subsequent lookup. Checking this option enables another sub-option:
    * *Cache Commit Count*: Defines the number of records collected per cache chunk before they are committed to the cache.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FRk3iqNUDXbcR0AmxU02x%2Fimage.png?alt=media&#x26;token=a57763d7-e923-4b9c-b908-31f934c079d3" alt=""><figcaption></figcaption></figure>

* *Dynamic*: The lookup values are stored in a temporary cache file, which is deleted once the dataflow has been executed. When you select this option, the following sub-options are enabled:
  * *Fill Cache With All Lookup Values at Start*: Fills the cache with all of the lookup values at the start and continues to use this cache for every lookup.
  * *Cache After First Use*: Uses the database table for the first lookup and fills the cache right after it is done. This cache is then used for every subsequent lookup. Checking this option enables other sub-options:
    * *Cache Commit Count*: Defines the number of records collected per cache chunk before they are committed to the cache.
    * *Cache Key Column*: Defines a matching key field to check whether a record already exists in the cache.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FLOMY0n4vvZ2057WmD1o4%2Fimage.png?alt=media&#x26;token=a5241601-58a2-4536-bbea-0d37df5ac1ab" alt=""><figcaption></figcaption></figure>

* *Persistent*: Saves the lookup values in a cache file that can be reused for future lookups. When you choose this option, the following sub-options are enabled:
  * *Rebuild Persistent Cache on Next Run*: Checking this option will allow the contents of the cache file to be modified after every run.
  * *Cache File Name*: Here, you can enter a name for your cache file.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FU357wBqgjAJ5hzPtQmD2%2Fimage.png?alt=media&#x26;token=91583c57-c6c9-4aec-baa0-3f86462977c7" alt=""><figcaption></figcaption></figure>

In this case, we will select the *No Caching* option. Once you are done, click *Next*.

7. On the *Lookup Options* window, you can choose between multiple lookup options.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FumlfRnoIoecwM9hN1KWu%2Fimage.png?alt=media&#x26;token=25e6e759-7525-4ba5-8bb2-989a573c1c5d" alt=""><figcaption></figcaption></figure>

This page provides a set of options for different scenarios that could be faced during a lookup.

If multiple values are found, then opt for the *Multiple Matches Found Option.*

* *Multiple Matches Found Option*: This option provides the flexibility to choose the output value if more than one match is found for a single value in your lookup file. The option expands into a drop-down list where you can select one from the following three options:
  * *Return First*: Will return the first matched value found.
  * *Return Last*: Will return the last value among all the matched values.
  * *Return All*: Will return all the values in the lookup file that match a source value.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F2PRYTUbXDgFCFeGsS8x3%2Fimage.png?alt=media&#x26;token=806732ea-2351-41a4-bcf5-c9cb82119695" alt=""><figcaption></figcaption></figure>

* *If Value Is Not Found In The Lookup List:* In case no lookup value is found against a source value, you can choose one of the following three options to be appended to your output.
  * *No Message:* There will be no message and the output will be the same as the input value.
  * *Add Error:* An error message will appear with the output.
  * *Add Warning:* A warning will appear with the output.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FDg5luLTs8jFYo51PRtKK%2Fimage.png?alt=media&#x26;token=53d8656d-f659-4c50-bdbd-932945aa276f" alt=""><figcaption></figcaption></figure>

* *If Value Is Not Found In The Lookup List, Assign Value:* If no lookup value is found against a source value, you can assign an output value of your choice.
  * *Assign Source Value:* Will return the source value in the output.
  * *Assign Null:* Will assign null to your output value.
  * *This Value:* You can select this option and assign any value of your choice.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FbGo5PrAtEQ931xsqL9tm%2Fimage.png?alt=media&#x26;token=82c715a8-5404-4913-a423-0548da2105fc" alt=""><figcaption></figcaption></figure>

In this case, there is only one code for each country. Therefore, we will choose *Return First* from the drop-down list in the *Multiple Matches Found Option*.

Moreover, we will leave the other options at their default selection i.e. *No Message* under *If Value Is Not Found in the Lookup List*, and *Assign Null* under *If Value Is Not Found, Assign Value*.

Once you are done choosing the options, click *Next*.

8. On the *Config Parameters* window, you can define certain parameters for the *Database Lookup* object.

These parameters facilitate an easier deployment of flows by excluding hardcoded values and providing a more convenient method of configuration. If left blank, they will assume the default values that were initially assigned to them.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F9QGzyEXiYwAUU4ey5npe%2Fimage.png?alt=media&#x26;token=99d88943-ff43-4e9f-b16c-6158f4bcc9c3" alt=""><figcaption></figcaption></figure>

In this case, we will leave them blank. Click *Next*.

9. On the last window, which is the *General Options* window, you will be provided with a text box to add *Comments*. The *General Options* in this window have been disabled.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FXRspCx8WhqFqCcgSA137%2Fimage.png?alt=media&#x26;token=36380a2e-69e3-45ff-8752-87074a0f72eb" alt=""><figcaption></figcaption></figure>

You are now done configuring the *Database Lookup* object. Click *OK* to close the configuration window.

10. Expand the *Database Lookup* object to view the layout of the lookup table. In this case, it contains two fields, *Country* and *Code*. The former contains the full name of each country and the latter contains each country’s code.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FZ0FshJXfTDJirEkQeigY%2Fimage.png?alt=media&#x26;token=a35371a5-367f-42fe-a001-d9970dfbcf45" alt=""><figcaption></figcaption></figure>

11. Map the Country field from the *Database Table Source* object to its counterpart in the *Database Lookup* object.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FkEudDWyOaQ5W3mfw0Bxr%2Fimage.png?alt=media&#x26;token=bafc6511-7758-425b-81d2-2bd83fd57f01" alt=""><figcaption></figcaption></figure>

12. Drag an [*Excel Workbook Destination*](https://documentation.astera.com/v/astera-data-stack-v8/dataflows/destinations/excel-workbook-destination) object from *Toolbox > Destinations > Excel Workbook Destination* and drop it onto the designer. Configure the object by providing a name and the path to the directory and folder where you want to save your destination file.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2Fa9toduY4F1BmKBIhTUkG%2Fimage.png?alt=media&#x26;token=39377917-4bd2-47fb-8ddd-224d256e4f23" alt=""><figcaption></figcaption></figure>

13. Auto-map the source dataset to the destination object.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FXEKJFKNc16wpwv4bHOOi%2Fimage.png?alt=media&#x26;token=745536f5-e544-403c-b179-3661eb96385f" alt=""><figcaption></figcaption></figure>

14. Delete the mapping link between the *Country* fields in the source and destination. To do this, right-click on the mapping link and select *Delete* from the context menu.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2Ffuu4XH2dxfbAv8rmixid%2Fimage.png?alt=media&#x26;token=e2bf796e-0f0d-40f6-b2c5-863aa30a53e0" alt=""><figcaption></figcaption></figure>

15. Map the *Code* field from the *Database Lookup* object to the *Country* field in the destination object. This is what the final dataflow should look like:

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FaKOSzGU9v0MKq63S2O4f%2Fimage.png?alt=media&#x26;token=4ea05133-e60a-4917-9026-69f9fb74a184" alt=""><figcaption></figcaption></figure>

16. Right-click on the destination object’s header and select *Preview Output* from the context menu.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2F99c1R53xQX8x8rOVqwaw%2Fimage.png?alt=media&#x26;token=4df656ce-b934-42a1-9b84-490977e52222" alt=""><figcaption></figcaption></figure>

In the *Data Preview* window, you will see that each county name has been replaced by its corresponding code.

<figure><img src="https://750977703-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FqHxyVNGb7tSdIWecl6Ru%2Fuploads%2FMGr7EWuVGis7awe7oNPG%2Fimage.png?alt=media&#x26;token=ff9ca3b3-74b6-4c9c-aa74-abc353149622" alt=""><figcaption></figcaption></figure>

This concludes using the *Database Lookup* Transformation object in the Astera Data Stack.
