> For the complete documentation index, see [llms.txt](https://documentation.astera.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.astera.com/astera-data-stack-v7/dataflows/data-logging-and-profiling/data-quality-mode.md).

# Data Quality Mode

In addition to the standard logging functionality, Astera provides a special *Data Quality Mode* option, useful for advanced profiling and debugging.

When a dataflow is created/opened in the *Data Quality Mode*, most objects on the dataflow show the *Messages* node with output ports..

## **Sample Use-Case**

In this case, we have a simple dataflow designed to perform a data quality check. It contains customers’ data coming in from an [*Excel Workbook Source*](https://documentation.astera.com/v/astera-data-stack-v7/dataflows/sources/excel-workbook-source). A [*Data Quality Rule*](/astera-data-stack-v7/dataflows/data-logging-and-profiling/using-data-quality-rules-in-astera.md) object is added to validate data for null values and perform warning checks.

![](/files/nOlrPxkTh8QCSFYUHxNu)

If you preview the *Customers* dataset output at this stage, you will see that some of the records have missing values in the *Region* and *Fax* fields.

![](/files/h8HCDeGuxLaOHH7CMZWX)

Data quality rules are set so that records with empty *Region* values are marked as errors and records with empty *Fax* values are marked as warnings.

A red exclamation sign in the *Data Preview* window identifies the records that have failed to match the rule and returned an error or a warning as a result.

![](/files/O5lY9WCcL2xrXn4ql1t5)

Now, for instance, we want to collect information regarding the number of errors/warnings in a single record along with the error/warning messages attached to these records, and write this information to a destination. For this purpose, we will use *Data Quality Mode*.

{% hint style="info" %}
**Note:** The *Record Level Log* feature also collects and records this information but we can not further process it in the dataflow.
{% endhint %}

## **Activating Data Quality Mode**

1. To activate this feature, click on the *Data Quality Mode* ![](/files/iyqZ8wY6xsp6KlV8WMhU)icon located at the top of the dataflow designer.

![](/files/ddp0f4jvuGgOu6xzOga2)

Once the *Data Quality Mode* is activated, a *Messages* node will be added to all the objects in the dataflow.

![](/files/tNEFJRbJ7RUKNMVikKLk)

The *Messages* node captures the following statistical information:

* *TotalCount*
* *ErrorCount*
* *WarningCount*
* *InfoCount*
* *MessagesText*
* *DbAction*
* *Custom*

In addition, *FirstItem*, *LastItem*, and *Items* sub-nodes provide a way to collect quality control data for each of the records. The quality control data includes *ElementName*, *MessageType*, *Action,* etc., and can be written to a destination object for record-keeping purposes.

## **Writing to a Destination**

Connecting the *Messages* node’s output ports to another object’s input ports on the dataflow makes it possible to get both - summary statistics and record-level statistics for the dataset, which are useful for analysis and debugging. To do this:

1. Drag-and-drop *Delimited File Destination* onto the dataflow designer by going to *Toolbox > Destinations > Delimited File Destination*.

![](/files/cNH2SKDk3j9rL9jMQ6L2)

2. Auto-map fields under the *Messages* node onto the destination object.

![](/files/Q7O0UrO7eGiYWY4UnBUf)

3. Configure settings for the [*Delimited File Destination*](https://documentation.astera.com/v/astera-data-stack-v7/dataflows/destinations/delimited-file-destination) to save this data.
4. Right-click on the header of the destination object and select *Preview Output* from the context menu.

![](/files/nYPpn2ODGHJin3ZyUFnR)

A *Data Preview* window will open, showing error and warning information.

![](/files/3QYXxhvjIrDwCnNIo0hk)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://documentation.astera.com/astera-data-stack-v7/dataflows/data-logging-and-profiling/data-quality-mode.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.