Delimited File Destination

Astera's Delimited File Destination provides the functionality to write data to a delimited file. Delimited File Destination gives you the ability to control the structure and content of a file including numeric, date, and Boolean formats, encodings, text qualifiers (quotes), and character sets. You can choose to create a new file or append data to an existing file.

Configuring the Delimited File Destination Object

To get a Delimited File Destination object from the Toolbox, go to Toolbox > Destinations > Delimited File Destination. If you are unable to see the Toolbox, go to View > Toolbox or press Ctrl + Alt + X.

Drag and drop the Delimited File Destination object onto the designer.

The dragged destination object is empty right now. This is because the object has not been configured yet.

Mapping

For a Delimited Destination object to work, data fields should be mapped to the object so that the mapped data can be written to the destination.

Configure the source object and place it onto the designer next to the Delimited File Destination object.

Note: In this case, a Customers sample table has been used from a Database Table Source to write to the Delimited File Destination.

Now map the source object to the destination object. The following ways can be used for mapping:

i. By dragging and dropping the parent nodes onto each other for automatic mapping.

ii. By mapping the source parent node by dragging it to the destination parent node manually.

iii. By directly writing the source layout to a Delimited File Destination through the source context menu of its parent node.

The fields are now mapped.

Configuration

To configure the Delimited File Destination object, right-click on the header, select Properties from the context menu and a dialog box will open.

Select the Properties option from the context menu and a dialog box will open.

Provide the File Path. This is where the delimited destination file will be saved.

The dialog box has some other configuration options. Let’s go over these options:
- Options:

If the source First Row Contains Header, check the box to read headers from the source file.
Field Delimiter - Allows you to select a delimiter from the drop-down list for the fields.
Record Delimiter - Allows you to select the delimiter for the records in the fields. The choices available are carriage-return and line-feed combination, carriage-return and line-feed. You can also type the record delimiter of your choice instead of choosing the available options.
Encoding - Allows you to choose the encoding scheme for the delimited file from a list of choices. The default value is Unicode (UTF-8)
A Text Qualifier is a symbol that identifies where text begins and ends. It is used specifically when importing data.
Apply Text Qualifier to all Fields will add the specified qualifier to all the fields that have been mapped

Say you need to import a text file that is comma delimited (commas separate the different fields that will be placed in adjacent cells).

Use Null Text to specify a certain value that you do not want in your data and want it to be replaced by a null value.
Choose Append to File (If Exists) to append to an existing file or create a new file. Creating a new file will overwrite any existing file data.
Check on Hierarchical Destination when the data in the source file needs to be sorted into hierarchies in the destination file.
Check on Write to Multiple Files for the data to be saved to multiple files instead of one single file. This can be done within a single data flow through the destination object and supporting transformations.

Once the data reading options have been specified in this window, click Next.

The next window is the Layout Builder. On this window, the layout of the delimited destination file can be modified.

To add a new field to the layout, go to the last row of the layout (Name column), which will be blank and double-click on it, and a blinking text cursor will appear. Type in the name of the field to be added and select the subsequent properties for it. A new field will be added to the source layout.

Note: Adding a new field (Email) to the layout.

To delete a field from the layout, click on the serial column of the row that is to be deleted. The selected row will be highlighted in blue.

Note: Deleting the Fax field from the layout.

Right-click on the highlighted line and a context menu will appear which will have the option to Delete.

Selecting Delete will delete the entire row.

The field is now deleted from the layout and will not appear in the output.

To change the position of any field and move it below or above another field in the layout, select the row and use the Move up/Move down keys.

Note: Find the Move up/Move down icons on the top left of the builder.

For example: To move the Country field right below the Region field, select the row and use the Move up key from the 9th row to the 8th row.

Once the object layout is configured, click Next. A new window will appear, Config Parameters, which allows us to further configure and define the parameters for the delimited destination file.

Parameters can provide easier deployment of flows by eliminating hardcoded values and provide an easier way of changing multiple configurations with a simple value change.

Note: If you want to specify parametrized values for Field/Record Delimiter, make sure to follow the same syntax of delimiters as in the dropdown menus of these options on the Properties window.

Parameters left blank will use their default values assigned on the properties page.

Next a General Options window will appear. In this window:
- Comments can be added.
- General Options are given, which relate to the processing of records in the destination file.
  - Clear Incoming Record Messages: For any messages coming in from objects preceding the current object to be cleared.
  - Do Not Process Records With Errors: Will not let erroneous records to proceed further for the output.
  - Do Not Overwrite Default Values with Nulls: Ensure that values are not overwritten with null values in the output.