> For the complete documentation index, see [llms.txt](https://documentation.astera.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.astera.com/data-model/deployment-and-usage/using-astera-data-models-in-etl-pipelines.md).

# Using Astera Data Models in ETL Pipelines

A deployed data model is available as a virtual database that can be used in ETL pipelines. In Astera Data Stack, you can use the tables in these virtual databases as sources or destinations in dataflows, which represent the ETL or ELT functionality of the product. To learn how you can get started with dataflows, click [here](https://docs.astera.com/projects/centerprise/en/7/getting-started/introducing-dataflows.html#introducing-dataflows).

### Video

{% embed url="<https://youtu.be/ThxwHLPfHAs>" %}

### Dataflows

The dataflow toolbox contains an extensive variety of objects that can be used as part of an ETL pipeline via the drag and drop interface of the dataflow designer. These include sources, destinations, transformations, and numerous other object types.

<img src="/files/xMZDpxtffJ8VxeuM5ED2" alt="01-dataflow-toolbox" width="275">

You’ll notice a section titled *Data Warehouse* in the toolbox. This contains the *Fact Loader* and *Dimension Loader* objects, which can be used to load data into fact and dimension tables, respectively. We will discuss these objects in further detail at a later stage in the article.

<img src="/files/1ZvStnv6dLr15re3Mi4G" alt="03-dataflow-data-warehouse" width="280">

Here is an example of a simple ETL pipeline designed in a dataflow:

![02-simple-dataflow](/files/B1hf6FwRBvQXn2dqCRsU)

In this dataflow, we’ve done the following:

* Extracted data from a database table using the *Database Table Source* object.
* Transformed the incoming dataset by filtering out certain records using the *Filter Transformation* object.
* Loaded the transformed dataset into an Excel worksheet using the *Excel Workbook Destination* object.

### How to Use Astera Data Models in Dataflows

Multiple objects in the dataflow toolbox can be used to access data models that have been deployed on the server. These include:

* *Database Table Source*
* *Data Model Query Source*
* *SQL Query Source*
* *Fact Loader*
* *Dimension Loader*

Of these, the first three can be used as source objects and the other two can be used as destination objects. However, the *Database Connection* screen in the *Properties* configuration window is common in all of these objects. Let’s take a look at this *Database Table Source* object as an example.

![04-database-table-source](/files/iG3aufCvH1YvZntabYCY)

To access the properties of an object, right-click on the object header and select *Properties* from the context menu.

<img src="/files/NGxvnCccatbf9ANAV0mi" alt="05-database-table-source-properties" width="348">

A configuration window will appear on your screen.

![06-database-connection](/files/FFJ7J7EpDM4XmBOuy5hG)

The *Database Connection* screen on this window can be used to access the virtual databases representing the deployed data models.

The *Data Provider* dropdown menu features a comprehensive list of data providers available from within Astera DWB. Among these, you’ll notice an option by the name of *Astera Data Model*.

![07-astera-data-model-data-provider](/files/X8wE4eE7NskV28vkfUik)

On selecting the *Astera Data Model* option, a list of required credentials will appear on the screen.

![08-astera-data-model-credentials](/files/0GNVYVXCwySx8V8jfkL6)

*User*: Here, you must provide your Astera DWB username. The default username is *admin*.

*Password*: Here, you must provide your Astera DWB password. The default password is *Admin123*.

*Server*: Here, you must provide the server connection in the given format. This is the server where you’ve deployed your data models.

![09-astera-data-model-credentials](/files/fHZBSTaTV6ATXahjSb40)

Once you’ve filled in the required credentials, the *Database* dropdown menu will provide a list of deployments that are running on the server.

![10-astera-data-model-database](/files/BbWqwnTBQy5S7HQuKQe6)

Choose the deployment that you wish to access. In this case, we’ll select *Demo\_Source*. You can test the connection by clicking on the *Test* icon.

![11-test-database-connection](/files/WWQ61Z4AnN4JbyHwqqqr)

A pop-up window will show you the test results.

<img src="/files/b9F1XgkUp7c2fX6IoHbA" alt="12-test-connection-results" width="330">

This *Database Table Source* object now has access to the tables present in *Demo\_Source*. Similarly, you can connect to a deployed data model from the *Database Connection* screen in the *Data Model Query Source*, *SQL Query Source*, *Fact Loader*, and *Dimension Loader* objects.

Click *Next* to move to the *Pick Source Table and Reading Option* screen.

![13-astera-data-model-pick-table](/files/0rY1k5ncNkZQ9DAXnfmu)

This screen is specific to the *Database Table Source* object and will vary according to the object you’re using.

Here, you can pick a table from the virtual database representing the data model.

![14-astera-data-model-pick-table](/files/zGKxWxeKyetXzttzrW1O)

For reference, here is a look at the *Demo\_Source* data model.

![15-deployed-astera-data-model](/files/lqnQCFhbDOMERqjP7Met)

In this case, we’ll select the *Customers* table from the dropdown menu.

![16-astera-data-model-pick-table](/files/y1o8IExzrMaaKffsPVBc)

Click *OK* to close the configuration window. You’ll notice that the layout of the *Customers* table is now visible in the *Database Table Source* object.

<img src="/files/oB1HDniPK6NdPAJJjsvK" alt="17-database-table-source-layout" width="251">

You’ve successfully accessed an *Astera Data Model* table in a dataflow and can use it as part of an ETL pipeline.

### Fact Loader & Dimension Loader

As mentioned earlier, the *Fact Loader* and *Dimension Loader* objects can be used as destination objects to load data into fact and dimension tables, respectively. These tables are represented by fact and dimension entities in a dimensional model, which denotes the schema of a data warehouse. Collectively, the *Fact Loader* and *Dimension Loader* objects are used to populate a data warehouse.

You can use the *Database Connection* screen in the properties of both objects to access the virtual databases representing deployed data models, in the same way as we saw earlier in the *Database Table Source* object. Once you’ve done so, click next to move to the *Pick Table* screen, where you can pick a table from the virtual database.

Let’s assume that we’ve deployed the following dimensional model:

![18-dimensional-model](/files/w7Fpq8bhtXT6INrvcRsT)

The *Sale* entity represents the fact table, and the rest of the entities represent dimension tables.

#### Fact Loader – Pick Table

![19-fact-loader-pick-table](/files/cWivlLhGt3fLphk3fJlj)

On expanding the *Pick Table* dropdown menu, you’ll notice that it only shows tables that are marked as facts in the dimensional model. In this case, the only table which fits that criterion is the *Sale* table.

![20-fact-loader-pick-table](/files/mxyTQK3Td3ONH9T4iRfc)

![21-fact-loader-pick-table](/files/qPDDQE6VTETmrtEQw9Cl)

Click *Next* to move to the *Layout Builder* screen. Here, you can view the layout of the fact table.

![22-fact-loader-layout-builder](/files/6foWvndnbU8w7wwuImKG)

Click *OK* to the close the configuration window. The layout will now be visible in the *Fact Loader* object.

![23-fact-loader-object](/files/xA49351u6C1Sga2MW5fm)

You can now make the required mappings from a relevant source object and run the dataflow in pushdown mode to populate the fact table.

#### Dimension Loader – Pick Table

The *Pick Table* dropdown menu in the *Dimension Loader* object only shows tables that are marked as dimensions in the dimensional model.

![24-dimension-loader-pick-table](/files/C5mjHyS7c1H0DhKf8bqT)

In this case, we’ll select the *Customer* table.

![25-dimension-loader-pick-table](/files/olLfcAFjdWPR7Fwq4wqn)

Click *Next* to move to the *Layout Builder* screen.

![26-dimension-loader-layout-builder](/files/FnvD0nnpvYO77QsbatdN)

Here, you can view the layout of the dimension table.

Click *OK* to close the configuration window. The layout will now be visible in the *Dimension Loader* object.

![28-dimension-loader-object](/files/KRMQf7Hq8JEi9hEOg0Km)

You can now make the required mappings from a relevant source object and run the dataflow to load data into the dimensional table.

This concludes our discussion on using *Astera Data Models* in ETL pipelines.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://documentation.astera.com/data-model/deployment-and-usage/using-astera-data-models-in-etl-pipelines.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.