Astera 8.0 - What's New, What's Fixed, and What's Improved
Astera 8.0 is a major release in Astera’s line of ETL solutions and is shipped with new features, existing feature updates, as well as a brand-new UI. Here’s a short summary of what’s new, what’s fixed, and what’s improved:
Astera 8.0 – What’s New
REST Server Design
We are now shipping a REST-based Astera server with the release of version 8.0. We have transitioned from .NET remoting to RESTful web services.
Introducing the Astera Lean client – A Modern, Scalable Architecture
You asked for an easier way to deploy Astera across a company network, and we delivered! This is good news for your IT administrators. We have introduced a lightweight version of the Astera client with the 8.0 upgrade. Assemblies and DLLs (Dynamic Link Libraries) for most database drivers now reside on the Astera server instead of the client machines. Now you can scale horizontally and add more clients to an existing cluster of servers with minimum IT overhead.
SmartMatch Functionality
We have introduced the SmartMatch functionality in Astera 8.0 to simplify complex mapping jobs in ETL flows. It uses a Synonym Dictionary File created by the user to resolve mapping issues that arise due to naming inconsistencies in the incoming source data. The Synonym Dictionary file is a simple txt file in which users can specify variations of the header name in a pipe delimited format and the Astera integration server will automatically read and match field names at run-time based on the predefined information in the dictionary.
Data Cleanse Transformation
Data Cleanse is a new addition to Astera's library of built-in transformations. Data cleansing is an essential step in the data preparation process and the Data Cleanse Transformation makes it all the more convenient for users to cleanse raw data and present it in a more refined, standardized, and enterprise-ready format. Using the Data Cleanse transformation, users can:
Replace null strings and numeric values with blanks and zeros respectively
Remove extra spaces, unnecessary text and numeric characters, punctuations, line breaks, etc.
Change letter cases to upper, lower, or title case
Apply regular expressions and find-and-replace actions on datasets
Prepare raw data for transformation, validation, profiling, and record-matching functions
Dynamic Layout
Dynamic Layout feature has been added to some source objects in Astera 8.0. It makes it all more convenient for users to ensure layout consistency in integration flows. The Dynamic Layout feature eliminates the need and hassle of manually updating the mappings between different objects in a dataflow and ensures they’re all in sync with the source schema.
Job Optimizer
Delivering on our promise of optimizing data integration processes for maximum agility and seamless performance, we have added the Job Optimizer functionality to the latest release of Astera 8.0. The Job Optimizer enables users to optimize ETL performance and minimize job execution time.
The Job Optimizer issues recommendations to optimize integration flows, at runtime in job trace. These recommendations can be removing repetitive sort operations on presorted data, applying an ORDER BY clause on database sources to run sort operations in the database, or any other suggestion to optimize your flow.
As a result, it helps bring a significant increase in performance by efficiently managing the RAM, CPU, network, and disk utilization while executing jobs.
The New Search Feature
A Search feature has been added to the Toolbox, Expression Builder, Server Log, and Job Monitor. This new Search bar enables better navigation through different features and components, resulting in better experience and improved user-friendliness.
In the Toolbox, the Search feature enables easy navigation across different sources, destinations, transformations, text processors, resources, and other components of the toolbox.
In the Expression Builder, the Search features make it convenient for users to find a relevant expression function without having to browse through the extensive library of Astera's built-in expressions.
In the Server Log, this feature allows users to look for particular jobs deployed, running, or executed on the Server with respect to JobID, Server Name, Component, Severity, and Error Info.
In the Job Monitor, the search feature allows users to look for jobs based on different criteria such as Date, Job Type, Job File, Server, etc.
Astera REST Web Service APIs
We have published an extensive library of Astera REST web service APIs. Now, you can perform a wide range of data integration functions by using APIs to directly communicate with the Astera server and get the required data. Here are the resources that are available for the users in the first round of the beta release:
Account APIs: Resources related to account login and user authentication.
Server APIs: Resources related to job monitoring, schedules, and server information.
User Roles and Access Control
Astera 8.0 comes with enhanced role-based security features, enabling users to manage data pipelines in heterogeneous and multi-user environments. Administrators can grant or restrict access to various users within the organization, based on their role in the entire data management cycle. The three-fold security mechanism incorporated in Astera 8.0 includes:
Secure Domain Configuration – encrypts information exchange between Astera client and server through an SSL protocol
User Authentication – prevents unauthorized access to data
Role Management – enables administrators to centrally manage user access by creating custom roles and access controls
Astera 8.0 – What’s Improved
Server Monitoring and Job Management
Astera 8.0 includes advanced server monitoring and job management features to enable administrators to proactively manage job monitoring, prevent downtimes, and identify the causes of server failure. Moreover, in case of server failure or outage, users can generate a diagnostic file to get details about the state of the server and the machine where it’s deployed.
This diagnostic file contains information such as server status, server health, active connections, process memory size, thread count, installed drivers, etc. which you can use for troubleshooting and improvements. Furthermore, you can track, monitor, and manage the jobs scheduled, running, or executed on the server in the Job Monitor window.
Job Trace Improvements
The Job Trace window has been updated with an improved UI response time and better trace navigation. The trace refresh on the client is decoupled from the server writing to the trace. As a result, the client may not display all entries while loading a very fast and long-lived trace, which will ultimately improve the load time for long trace outputs. The server performance has also been optimized since the server uses bulk insert to write the trace output.
Here’s a rundown of the improvements in the job trace:
Refresh and Clear Trace buttons have been added to the monitoring toolbar.
Suggestions for job optimization are always displayed in the job trace.
Verification errors warnings or suggestions can be traced in the job trace.
The status of the job is also displayed by an icon depending on whether it is completed, queued, terminated, etc.
Workflow Performance Optimization
Another significant highlight of Astera 8.0 is the improvement in Astera server code to optimize the performance, reliability, and scalability of workflows. The workflow execution logic has been optimized to eliminate any chance of a workflow hanging amidst a session in any state (queued, running, or terminating), even if there are transient communication errors between the main server or cluster servers and the repository.
Significant improvements have been made to Resume and Terminate functionalities for workflows in Astera 8.0. The limitations in resuming complex, nested workflows found in previous releases have been resolved in Astera's latest release.
In Astera 8.0, while working with nested workflows (workflows that call other workflows within a master workflow), if a child component returns an error and is not executed successfully, the parent flow will terminate automatically. Likewise, when you resume a child flow that returned an error(s) and led to the termination of the parent flow, the parent flow will also get resumed automatically.
Parametrization
The parametrization functionality in Astera 8.0 has been improved to support expression compilation in variables. The syntax has been changed from ‘$ ()’ notation to ‘{}’. However, this update won’t affect the parameters incorporated in existing flows.
The current flows will continue to run seamlessly as before, without having to manually update the syntax. And once you save any flow that was using the previous notation ‘$ ()’ in version 8, it will automatically get converted to ‘{}’.
Other Highlights
Improvements in List Lookup and DB Lookup transformations: New options added to the layout builder screen of List Lookup and DB Lookup transformations. Now you can keep the source value for the values that do not match the replacing criteria in a list or a database table.
Path Mappings tab added: A new tab for specifying client and server path mappings has been added to the Cluster Settings. This is particularly useful when the client and server reside on different networks or network file systems and their paths have to be mapped correctly to transfer files at run-time.
Option added to repair file paths: For better project management, we have added a new option to repair the file paths for files in any particular folder. This option is useful if you are working in a case-sensitive environment.
Shortcut for replacing parameters information for consolidated parameters added: Now you can replace the common parameter information for all objects in a flow with this command right from the dataflow screen.
Pushdown mode is expanded and improved: More transformations and expressions are now supported in the Pushdown mode. We have also improved the explanations for the verification errors for easier troubleshooting.
Salesforce UPSERT is fully supported: Salesforce UPSERT is now fully supported. UPSERT uses an external ID field in the Salesforce table to match incoming data with existing records.
Keyboard shortcuts added and displayed: Keyboard shortcuts are enabled for various task windows. You can view the shortcut keys next to their respective commands.
Performance improvements when mapping complex trees: Runtime performance of the dataflows containing complex tree structures is improved.
New additions to the data providers list: We have added Tableau to the list of supported data providers.
SCD write strategy improved: SCD type 3 and type 6 are now supported.
REST call retries: For the REST calls running in timeout error, Astera will automatically make up to three retry attempts to process the call.
Migrating from Astera 7.x to Astera 8.0
If you’re using an older version of the Astera Data Stack and are planning to upgrade to Astera 8.0, follow through the steps mentioned in this document for a smooth migration.
Have any queries, issues, or suggestions? Reach out to us at support@astera.com.
Last updated
