Astera 9.0 was a major release in Astera’s line of ETL solutions. It comes with the tried and tested features that were available in previous versions of the Data Stack, along with some improvements and additions. The highlight of this release is the data warehousing module that includes a comprehensive list of features and functionalities.

Here’s an overview of what’s was added and improved in this version of Astera.

Data Warehousing Module

We’ve introduced a brand-new data warehousing automation module that offers an agile, meta-data-driven approach to building enterprise data warehouses. This module is available as an extension of the existing feature set.

This state-of-the-art platform puts data modeling and dimensional modeling at the front and center of the entire data warehousing process, enabling users to build on-prem or cloud data warehouses with ease. Moreover, it also allows users to expedite and automate many of the standard and repetitive tasks involved in the data warehousing lifecycle, from design and development, all the way to publishing data.

The addition of this tool is what makes this release stand out among all previous versions of Astera. Our ultimate data-management solution is now capable of much more than it was at any time before!

Server-Related Enhancements

We’ve made some specific fixes to address scenarios where the client application is not connected to the server, or the server is down. Certain options that were not needed in such a situation have either been removed or are disabled until connectivity is restored.

For instance, the grid and toolbar buttons in the Job Scheduler, Job Monitor, and Server Log become disabled if the server is disconnected or unavailable. Moreover, we’ve also eliminated any unnecessary pop-ups that notify the user that the server is down.

Some server-related changes have also been made to the UI. We’ve removed some icons from the top of the Server Explorer to reduce clutter. The options represented by these icons are now available within right-click context menus.

Also, we’ve changed the appearance of the server status indicator on the right side of the main toolbar. This has been done to improve visibility and make the server connection status clearer to the user.

Change Password

Any user who is logged into their Astera account can now change their account password from within the client. This feature can be accessed via the user dropdown menu on the right side of the main toolbar.

Moreover, it is also accessible via Server Explorer > Default > Change Password.

Once you click on this option, a pop-up window will prompt you to enter your current and new passwords, and then confirm your new password.

Here are some criteria you need to follow while choosing a new password:

• It must contain a minimum of eight characters.

• It must contain at least one capital letter.

• It must contain at least one digit.

• It must contain at least six unique letters.

Client and Server Share the Same File System

A checkbox has been added to the Cluster Settings tab for you to indicate whether the client and server exist on the same machine or network. This option would have to be unchecked in a scenario where they exist on separate file systems.

For instance, when submitting a job from the client on a local network that connects to a Centerprise server running on a Cloud VM instance. Checking this option will cause the Centerprise client to create a mini archive with the flow document and any associated dependencies (subflows, shared actions, etc.) when running a job or previewing on the Cloud server.

Job Scheduler

We’ve made some changes to the Job Scheduler to disallow editing the schedule grid while saving a schedule. This is particularly useful in remote servers, where the save request may take a few seconds to process.

Data Models

You can create data models from scratch via the toolbox entities and user-friendly drag-and-drop interface. Alternatively, you can use the data model context menu to add new entities to a model. The data model toolbar gives you the option to create identifying, non-identifying, and self-reference relationships between and within these entities.

Reverse Engineer

We also provide users with the ability to construct a data model from an existing database by using the Reverse Engineer option. This option will enable you to point to an existing database and create a logical structure that incorporates the tables in this database and the relationships between them. All that with just a few simple clicks!

Dimensional Modeling

Astera's dimensional modeling functionality allows you to convert a data model to a dimensional model by conveniently assigning fact and dimension types.

Dimension

Once you’ve assigned the dimension type to an entity, you can also allot specified Dimension Roles to the fields present in the entity. The following roles are available:

• Surrogate Key

• Business Key

• SCD1, SCD2, SCD3, and SCD6 – We now support multiples sets of SCD3 and SCD6 values.

• Row Identifiers – Effective Date, Expiration Date, Current Record Designator, and Version Number.

• Placeholder Dimension

Fact

Once you’ve assigned the fact type to an entity, you can allot the Transaction Date Key role to a field in the fact entity layout.

Date & Time Dimension

The Date Dimension and Time Dimension objects in the data model toolbox can be added to a dimensional model to aid in analyzing and reporting data more efficiently.

Generate DDL Script

You can use the Generate DDL Script option available under the Forward Engineer functionality to transform a logical data model into a physical model by generating the database schema.

Indexes

For every data model entity, you can create indexes to increase the speed of data retrieval based on a field or a set of fields.

Data Model Deployment & OData

Users may also deploy or publish a model to the server for consumption. The product’s built-in OData module makes deployed models available to be used in ETL pipelines via dataflows or for data analytics and visualization in industry-leading tools like Power BI, Tableau, Domo, etc.

In a dataflow, you can access a deployed model through multiple objects via the newly added Astera Data Model data provider.

Verification

The Verification option allows you to check a data model for any possible errors or warnings. There are two sub-options available within the Verification option:

• Verify for Read and Write Deployment: To check if the model is ready to be deployed on the server.

• Verify for Forward Engineering: To check if any changes made to a model are ready to be mirrored to a target database.

Validate Metadata and Data Integrity

Using this option, you can check the integrity of the metadata and data present within the entities in a deployed dimensional model. This allows you to ensure that the data present in dimensions is correct and valid. You can access this option from within the Data Source Browser when you connect it to an Astera Data Model (ADM).

Dataflow Objects

We’ve added three new objects to the already extensive variety of accessories available in the dataflow toolbox. In the Sources section, we’ve added the Data Model Query object. Moreover, we’ve included a new section titled Data Warehouse in the dataflow toolbox. This new section contains two objects, a Fact Loader and a Dimension Loader.

Data Model Query

The Data Model Query object allows you to extract multiple tables from a deployed data model. The contents of these tables are organized in a hierarchical structure and can be mapped to any further objects in the dataflow. This is particularly useful when a destination contains fields from multiple tables in a source model.

Fact Loader

The Fact Loader object can be used to load data into a fact table by mapping the relevant fields from a source object.

Dimension Loader

The Dimension Loader object, previously available as SCD in the dataflow toolbox, can be used to load data into a dimension table by mapping the relevant fields from a source object.

What’s Coming?

Data Providers

At the moment, we support four data providers in Astera:

• SQL

• PostgreSQL

• Snowflake

More providers will be added to this list in upcoming releases.

Forward Engineer

The Forward Engineer functionality is currently restricted to the Generate DDL Script option. However, in future releases, the option will allow you to directly mirror any changes made to a data model into a target database.

Indexes

Astera currently supports a generalized indexing protocol. However, in future releases, you will be able to create and edit data provider-specific indexes.

Known Issues

• There are a couple of transient errors that may appear during data model deployment. These will go away when you save the data model once. They are indicated by the following error messages:

  • Please log in to perform this action.

  • There is already an open DataReader associated with this Command which must be closed first…

• When reverse engineering from an Oracle database, you may experience an inconsistency in the number of relationships that appear between two entities.

Astera's report model module also comes with some significant improvements in the UI and in the existing features. The overall look and feel of Astera's report models have been refreshed with a complete UI overhaul to enable seamless navigation and improve user experience. Aside from these improvements, we have also introduced new features such as raw data cleansing options and auto-determine field name to facilitate you in data extraction processes.

UI Improvements

We have introduced dedicated panels for Report Options, and Region, Field and Pattern properties in Astera. These panels appear on the right-side of the client window as you click on the respective region, field or pattern and on the Record node in the model layout. Unlike other panels in Astera, these panels are not fixed, and user can move them around in the client window.

Each of these panels has a toolbar with relevant shortcut icons to perform different actions. With this improvement, users will be able to specify the properties for report, region, field or pattern from within a single panel and see the changes being made simultaneously in the report model. Let’s look at these panels one by one.

Report Options Panel

This panel will allow users to apply or change the extraction properties by staying on the same window. It has an option for providing a file path to your source file. It also has multiple options such as Reading Options and Other Options which will enable users to specify the relevant settings to extract data from their source file.

This is how Report Options panel looks like in Astera.

Region Properties Panel

This panel gives you an exposure of all the properties relevant to the selected region as well as change them according to your report model. You can specify the region properties such as the name of your data region, the number of lines your region spans over, how it ends and whether it is an overlapping or container region. There are many other options present in the toolbar for auto-creating fields, auto-determining their names, as well as for deleting the entire data region.

This is how Region Properties panel looks like in Astera.

Field Properties Panel

This panel allows to refine the field properties to make the extraction of a data field more accurate. Thus, making it easier for users to modify them by introducing all the necessary features. Along with options in the toolbar for determining the field length and the general options specifying the name and data type of a field, or size and position options for specifying the formatting of a data field, now, it has also added Remove section for scrubbing of raw data.

This is how Field Properties panel looks like in Astera.

Pattern Properties Panel

Through this panel, users can go over the options to specify the properties of a selected pattern. For example, if the pattern is floating, it is applied on a multi-column region, it is a regular expression, on how many lines the pattern is applied to, and whether you want the pattern match to be case sensitive or not.

This is how Pattern Properties Panel looks like in Astera.

New Options for Raw Data Cleansing

Raw data cleansing is an important part of the data extraction process which brings the extracted data into a presentable, easy-to-comprehend, enterprise-ready format. We have added a couple of new options to the Remove options used for cleansing and scrubbing the extracted data.

In addition to removing extra spaces inside text, text qualifiers (surrounding quotes), and leading and trailing spaces, you can now remove all white spaces and punctuations from within the extracted data by simply checking the All White Spaces and Punctuations options under the Remove section in the Field Properties panel. These options can be applied to data irrespective of its type. This is because during extraction, the data in a field is read as text and the data cleanse options are applied at that time. Once the data is read and cleansed, it can be converted into appropriate data type.

The Remove options for data cleansing can be found in the Field Properties panel as soon as you add a data field.

Auto-Determine Field Name

We have introduced a new feature called Auto Determine Field Names in the Region Properties panel. This feature helps cut down the repetitive steps involved in individually naming every data field to just one click. Once you have defined a data region and added fields, right-click on that region and click Auto Determine Field Name in the context menu. This will automatically identify and name your data fields as they appear in the report.

You can also find this option in the Region Properties panel.

If you have any questions, feedback, or suggestions, feel free to reach out to us at support@astera.com.