# Using Dynamic Layout & Template Mapping in Astera

The *Dynamic Layout/Template Mapping* feature in Astera enables automatic and dynamic changes to be implemented in flows in case of any alterations in a database source system. Further, this feature also enables users to create a template dataflow that can be used to process multiple files and generate copies of the same dataflow, without having to change the mapping links individually for each source dataset. Upon execution, Astera will replace the values dynamically and these changes will be visible in the destination file or table.

### Benefits of Template Mapping and Dynamic Layout

* Running your routine processes does not require you to design, modify, and map your ETL flows every time.
* ETL pipelines can be run without any dependence on the development environment.
* Template mapping also facilitates scheduling repetitive, routine jobs for automation.
* Using runtime parameters, you can customize your templates to suit your changing requirements.
* Template mapping makes it more convenient for non-technical users to customize and deploy these templates in different business scenarios.

In this document, we will learn how to use the *Dynamic Layouts/Template Mapping* functionality in Astera Data Stack.

### Use-Case

For this use-case, we will look at how the *Dynamic Layout* and *Template Mapping* features in Astera may behave in case fields from users’ database sources are deleted or if additional fields need to be added.

1. First, let’s drag-and-drop a *Database Table Source* object from the Toolbox onto the designer.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FLP4io7ZnwQdi2mPIGVNc%2F0.png?alt=media)

2. Now, right-click the object header and select *Properties* from the context menu. In the *Database Connection* window, configure a connection to a database and click *Next.*

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FuLaiUIP0HQIR8Cc5mLLP%2F1.png?alt=media)

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2Fvo8oWCOyzqOucdg7JWuf%2F03-DL-TM.PNG?alt=media&#x26;token=a5ba2ea7-0577-49f2-b86a-0bbd19f6c793" alt=""><figcaption></figcaption></figure>

3. In the *Pick Source Table and Reading Option* window, select a table and check the *Dynamic Layout* option in the *Dynamic Layout* section.

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FGyXxZa14XaSATg2yTqq5%2F04-DL-TM.PNG?alt=media&#x26;token=9f6a97c9-7c28-4631-afe3-f9fa92192d23" alt=""><figcaption></figcaption></figure>

* The *Dynamic Layout* option essentially allows for altered fields to be catered for without impacting an automated Workflow or ETL pipeline. Therefore, if, for example, a field in a database table is being deleted or added by the user, the tool will not halt any flows due to errors, instead it will simply ignore the field in case of deletion and add a field in case of addition of a new field to the database table.

4. Checking the *Dynamic Layout* option enables the two following options, *Delete fields in Subsequent Objects,* and *Add Fields in Subsequent Objects.* These options can also be unchecked by users.

* *Delete Fields in Subsequent Objects:* Checking this option ensures that a field is definitely deleted from subsequent objects in a flow in case of deletion from the source database.
* *Add Fields in Subsequent Objects:* Checking this option ensures that a field is definitely added in subsequent objects in a flow in case additional fields are manually added in the source database by the user or need to be added into the source database.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FTpBK4tMXnS9oXaY1mCnZ%2F4.png?alt=media)

Essentially, these options ensure that any alterations in the source database are reflected further down the flow. For now, we will leave these as default as we will be discussing both the deletion and addition of fields.

5. Once done, we will click *OK* to close the window.

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FBWti8n1FmmY3rxDTLFnh%2F06-DL-TM.PNG?alt=media&#x26;token=d393a9d7-abd2-4468-ac36-ec85f585b50e" alt=""><figcaption></figcaption></figure>

5. As evident in the screenshot below, the source object has been given a “DYN” label, indicating the now dynamic nature of the source object.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2Fr1u60VSTwh6wBVJaqgWT%2F6.png?alt=media)

7. Now, we will add an *Excel* *Workbook Destination* object to the designer and configure it. Once done, we will map a single field from the *Customers* source object to the *Destination\_Guide* destination object.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FgbZUR3Fw4i7V8NnHfdzu%2F7.png?alt=media)

8. Next, we will right-click the mapping and select the *Template Map Properties* option from the menu. A *Template Map Properties* window will open.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2F00rKJkHKRsdgoW2QNOiI%2F8.png?alt=media)

* There are two types of Template Mappings: Guide Mapping and Conditional Mapping.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FCpkUMik8rvx0s9Zt1iQT%2F9.png?alt=media)

* *Guide Map:* This mapping link establishes a connection between two objects in a Dataflow, specifying how data moves between them. The Guide Map operates independently of specific fields in the source object, focusing solely on defining the data path from the source to the destination, without concern for the particular field it connects to.
* *Conditional Map:* Conditional Maps are employed to define specific conditions, which, when met, trigger automatic modifications to the destination database or file.

### Template Mapping - Guide Maps - Adding and Deleting Fields

9. In the *Template Map Properties* window, select *Guide Map* from the *Template Map type* drop-down. Once done, click *OK* to close the window.

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FT4Djl3SsvlpSeEyEM8bm%2F11-DL-TM.PNG?alt=media&#x26;token=cd0c4640-9c1c-4c03-9e16-ae2860df1bc2" alt=""><figcaption></figcaption></figure>

Now, we will manually delete the *Fax* field that exists in our source database and add an *AlternatePhone* field in the database table.

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FQ8nvXKiRhjNu38NF6fga%2Fimage.png?alt=media&#x26;token=0015f5ff-2402-4414-af2d-12bd93dcbdff" alt=""><figcaption></figcaption></figure>

10. Now, let’s run this flow by selecting the *Start Dataflow* icon.

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2F8pko18LG4OpAFtMSMPth%2F14-DL-TM.PNG?alt=media&#x26;token=3fe30a31-5ba6-49e2-b4ea-5d89be423a89" alt=""><figcaption></figcaption></figure>

10. A *Job Progress* window will pop up. Here you can see that the tool has not halted the flow due to any errors. Instead, if we open the newly created Excel file, we can see that the tool has ignored the deleted *Fax* field and added an *AlternatePhone* field.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FxVxICIWMVcSuMFBrxSH1%2F14.png?alt=media)

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FmAwvmxclCjEyLlR3vEaZ%2F15.png?alt=media)

### Template Mapping - Conditional Maps - Adding a Conditional Field

Now, we will use conditional mapping in such a way that if a field name consists of the word “Title”, it must be written to a *Title\_New* column in the destination file, while ensuring that the *CustomerID* and *Phone* fields are also present in the destination file as they are.

12. To do this, let’s first get rid of any previous mappings. Once done, we will map the *Customer ID* and *Phone* fields to the destination object.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FxxeZZmtYaon9bn8KFC5k%2F16.png?alt=media)

13. Next, we will create a new field, labeled *Title\_New,* in the destination object and map the *ContactName* field from the source object to the destination object node labeled *Title\_New.*

{% hint style="info" %}
**Note:** For this use-case, we are mapping the *ContactName* field, however, any field can be mapped to the destination object for specifying conditions through mappings.
{% endhint %}

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FdLA2UxPy3GSwbeXq7Z6U%2F17.png?alt=media)

14. Now, we will right-click the *ContactName* mapping and select the *Template Map Properties* option from the drop-down.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2Fgn1TKCCT47GYy7jJs8mX%2F18.png?alt=media)

15. In this window, we will select *Conditional Map* from the *Template Map type* drop-down.

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FXYTBD9LllXkfNVPtxyvt%2F19.png?alt=media)

16. Next, we will specify the following condition in the *Start Condition* box: <mark style="color:blue;">Contains</mark> (<mark style="color:red;">"Title"</mark>, <mark style="color:green;">Name</mark>)

![](https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2F6B7qJn5e1U7mEdA9PJ1Y%2F20.png?alt=media)

{% hint style="info" %}
**Note:** If we select the ellipses option next to the *Start Condition* box, in the Expression Builder, we can see the various parameters related to the metadata of the source and destination files, that can be used in conditions.
{% endhint %}

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FLy4xqa5IDBf0DE5m4pYD%2F22-DL-TM.PNG?alt=media&#x26;token=aa5fb44b-71cb-41c0-aebe-db24c825f3d9" alt=""><figcaption></figcaption></figure>

17. Once done, click *OK* to close the window.

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FSUQtgVJXCfX4fHXhLbNs%2F23-DL-TM.PNG?alt=media&#x26;token=a1f17168-ae89-4896-b7d5-10c996a208d5" alt=""><figcaption></figcaption></figure>

17. Now, let’s run the Dataflow by selecting the *Start Dataflow* icon in the toolbar. As we can see in the newly created *Excel* destination file, when compared to the source data previewed in the *Data Preview* window, the *CustomerID* and *Phone* fields have been mapped as they are, while all the data in the *ContactTitle* field has been written to the field labeled *Title\_New.*

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FpnUKc8XV1YqD2iyoWwGr%2Fimage.png?alt=media&#x26;token=5e7f30e2-fcbb-4b9e-b431-c9a9c78246be" alt=""><figcaption></figcaption></figure>

<figure><img src="https://3083465318-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FsR50Wa7EwZGlmPSAMkkf%2Fuploads%2FRg5znQ71QOSEIo6kXP6Z%2Fimage.png?alt=media&#x26;token=b27d0727-76c9-4fe4-824d-56ae31550e7b" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
**Note:** Conditional mapping always maps the first instance of the specified rule being met. Hence, if our source file consisted of another field with a name containing “Title”, conditional mapping will prompt the tool to only map data from the first field containing “Title” in the name, thus the *ContactTitle* field in our use-case.
{% endhint %}

This concludes using the *Dynamic Layout* and *Template Mapping* features in Astera.

{% hint style="info" %}
**Note:** Please note that this is a beta version of the feature. The functionality is still in the testing phase and could change as we get user feedback on this feature. Should you have any suggestions or feedback, or come across a bug, please report to our [Support Site](https://astera.zendesk.com/hc/en-us) or email us at <support@astera.com>.
{% endhint %}
