Astera 10.3 is here, brimming with excitement and a sea of new features!
With Astera's new AI Automapping feature, field mapping becomes easier, alongside seamless connectivity through providers such as Azure SQL, Google BigQuery, and many more.
Astera’s AI-powered data extraction is a game-changer, while Astera Data Stack's warehousing component introduces AI-Select and more. Unlock the full potential of your APIs with enhanced connectivity options and new features in Astera Data Stack.
Finally, experience the power of the Data Analytics Workbench, and refine your data with ease using Astera Data Prep. Elevate your data game with Astera 10.3!
Windows authentication is a security feature that allows users to log into a system using their Windows credentials.
Astera Data Stack leverages this authentication method and provides the option to register new users using Windows authentication through its Server Browser interface.
In response to Microsoft Outlook discontinuing support for Basic Authentication and advocating the use of Modern Authentication, the SendMail object in workflow tasks has been updated.
You can view what Microsoft had to say about this here.
It now incorporates a new feature of Modern Authentication, enabling users to seamlessly add authentication credentials directly from within the SendMail object.
This enhancement simplifies the process and ensures compatibility with the latest authentication standards recommended by Microsoft.
The option has also been added in the Cluster Settings as seen below:
Existing Astera customers can easily upgrade to version 10.3.1 by executing an exe. script, which automates the repository update to the latest release.
This streamlined approach enhances the efficiency and effectiveness of the upgrade process, ensuring a smoother transition for users.
Note: This upgrade applies to v10.0 and later ones. Previous versions cannot be upgraded and will need a clean repository as part of the upgrade.
The Catalog feature in Astera Data Stack is a centralized repository where one can store artifacts and share them with users as per the application.
The artifacts within the catalog are stored in a Catalog table. The security aspect of the Resource Catalog lets the user give permission to only the people whom they wish to share the artifacts in the catalog with.
For more information on Resource Catalog, please visit the documentation site here.
With the release of Astera 10.3, there have been quite a few new developments in the connectors’ domain.
Microsoft SharePoint is an enterprise document management and collaboration platform that helps organizations manage critical content. Its enterprise content management capabilities streamline flows and centralize important content to enhance collaboration.
The SharePoint provider is present within the Cloud Connection object dropdown.
Simply drag and drop the Cloud Storage Connection object from the toolbox onto the designer.
Right-click on the object header and select Properties from the context menu.
This will open a new window.
Select Microsoft SharePoint Document Library from the Providers dropdown menu.
Note: The SharePoint connection can be accessed in any object where Cloud files are available.
You can learn more about the SharePoint connector here.
The Azure SQL Database is a fully managed platform as a service (PaaS) engine that handles most database management functions such as upgrading, patching, backups, and monitoring without user involvement.
In Astera Data Stack, users can access Azure SQL Databases using Database Table Source or Database Table Destination, DB lookup, SQL Statement Lookup, and Database Write Strategies objects. They can also connect with the Run SQL Script task in a workflow.
Google BigQuery is a serverless, highly scalable data warehouse that comes with a built-in query engine. The query engine can run SQL queries on terabytes of data in a matter of seconds, and on petabytes in minutes.
This kind of performance is achieved without having to manage any infrastructure and without having to create or rebuild indexes.
In Astera Data Stack, users can connect with Google BigQuery as a database source or destination. As a source, flat and hierarchical data both can be read. Only flat data can be written to the Google BigQuery destination.
Azure Data Lake Gen 2 is a cloud-based big data storage and analytics solution provided by Microsoft Azure. It offers scalable and cost-effective storage for structured, semi-structured, and unstructured data.
The Azure Data Lake Gen 2 provider is present within the Cloud Connector object in Astera.
Improvements have been made to the user-friendliness of the Cloud Browser.
Changing the Cloud Connection, browsing cloud folders has been made easier in Astera 10.3. Cloud Browser’s functionality with SharePoint has also been improved.
In the File System Items Source object, multiple filters are supported for both local and cloud connections.
In Astera 10.3, the overall user interface of Astera has been improved and revamped. There are quite a lot of new features being introduced in this version. Let us take a look at them.
To provide users with greater control over their report editor, we have introduced a new feature called Pages to Read. This feature enables users to filter and selectively display specific PDF pages within the report editor, ensuring a focused and efficient document analysis experience.
The Auto Create Table option empowers users to select and create tables within the document seamlessly. With this addition, users can automate the table creation process, saving valuable time and effort.
The Report Model Path Parameterization feature introduces enhanced flexibility to dataflows by enabling users to specify a customizable Report Model path in addition to the Report Source path. This crucial enhancement allows for runtime parameterization, granting the ability to dynamically change the Report Model or apply different templates to various report sources.
As a result, automating data extraction from Report Models becomes significantly easier, streamlining the overall data processing and analysis workflows. This feature empowers users with increased control and adaptability, facilitating more efficient and versatile data-driven decision-making processes.
Astera now uses AI to recommend report model templates, allowing you to automatically generate models for multiple source files at once. By specifying the layout and document type, Astera recommends the most suitable model templates, saving you valuable time and energy when building your data extraction processes.
With this new feature, you can streamline your workflow and eliminate the need for manual data extraction. In this document, we will see how to use this feature to create the report models.
To view more information on AI-Powered Data Extraction, click here.
Astera now provides the functionality to extract data from PDFs that contain scanned documents using Optical Character Recognition.
When provided such a PDF, the tool recognizes it as an image PDF. However, the Use OCR option must be enabled manually by the users first. This option has now made scanned documents available for extraction to users, minimizing the effort of manual data entry from such documents. Users can select the Resolution for OCR, allowing them to get the best result for their documents.
Additionally, to ensure correct data extraction, as noise elements can cause erroneous data to be extracted, an Edit Mode is also available for the users to clean and tweak the extracted data.
Edit Mode allows you to deal with the data as a text file and make changes accordingly.
To learn more about loading PDFs with OCR, click here.
Report Models now have the functionality to verify if the data fields have been captured properly for all data instances by checking for any non-blank character being adjacent to instances of data fields. This option gives users a one-click check for the data fields they have created.
Additionally, to allow users better visibility of the erroneous fields, navigation between instances of the data field is also provided along with an option to auto-adjust field lengths for all data fields within the selected data region.
To learn more about Data field verification, click here.
Users can now access wildcards and other additional features for patterns in a report model through a context menu by right-clicking on the pattern box.
Now, if need be, users can change a data region to an append region and vice versa within the Model Layout panel.
This allows users flexibility in changing the model layout as they are creating their extraction template. To learn more about this feature, click here.
Astera Dataprep is a dynamic platform designed for rigorous data cleansing, transformation, and preparation activities. With its user-oriented and preview-focused interface, it offers significant functionality and visibility to streamline the preparation process.
The system allows for quick operations by seamlessly interchanging between scripting and point-and-click methodologies. Serving as a crucial intersection of data engineering and data science, Astera Data Prep is an invaluable tool in any data-driven operation.
Astera Dataprep is a data manipulation tool that offers interactive data correction, ATL scripting with auto-completion, and UI-ATL synchronization. It provides smooth navigation, action history tracking, and comprehensive data quality assurance.
The tool promotes scripting efficiency and reusability with template scripts and supports real-time visual insights for data analysis. It enables rich data transformations, including resolving cardinalities and merging datasets. Astera Dataprep streamlines data preparation, enhances productivity, and ensures data accuracy and integrity.
Dataprep offers 60+ ATL commands to enable a comprehensive set of data preparation strategies. ATL is a smart scripting language integrated with IntelliSense, where data engineers benefit from script auto-completion, reducing the need for constant reference to documentation. This propriety language can generate code snippets, enabling users to fill in the required fields effortlessly.
This streamlined approach enhances productivity, enabling data engineers to focus on the specific requirements of their data preparation tasks without the burden of repetitive syntax or command structure.
It hosts all ATL commands and command-related operations. It is a multipurpose artifact that also serves as a preparation process navigation browser.
The Data Source Browser, while not a new feature, is a vital part of Dataprep. It hosts all file sources, catalog sources, cloud sources, and project sources to be imported into the Dataprep artifact.
Note: While the Data Source Browser is essential in Dataprep, it is not specific to it.
Dataprep scripts are reusable and hence can be used as a source as well as a transformation in other artifacts such as dataflow, workflow, and analytics workbench.
The grid view hosts a preview-centric interactive grid that automatically updates in real-time to display the transformed data upon each transformation/modification. It is a dynamic grid that provides instant feedback on data quality.
The Dataprep Profile Browser is a side window providing a comprehensive view of the data with graphs, charts, and field-level profile tables. It keeps a check on data health and highlights the presence of invalid entries, missing values, duplicates, etc.
Within Dataprep, this is a borderless, and headerless 2x2 grid that enhances the experience of data reading, joining, union, and lookup.
Users can navigate smoothly using point-and-click actions in the ATL editor. This includes action history tracking, allowing users to review and backtrack changes made during the data preparation process for transparency, editing, and control.
Astera Data Stack's Data Model component has also introduced a handful of new features for the Astera 10.3 release.
The AI Select feature in Astera Data Stack assists users in identifying potential Fact and Dimension candidates from their selected entities. To do that, first users can select Build Dimensional Model from the main menu bar.
In cases where users are unsure about classifying entities as Facts or dimensions, this feature leverages AI capabilities to automatically determine the appropriate classification, streamlining the data modeling process.
