Deployment is a way for setting up Astera projects to run on the Scheduler.

When you are deploying a project, you can also specify parameters by creating a config file for project archive (.car) files to run project flows independent of any local parameters.

In this document, we will learn how to deploy projects in Astera.

How to deploy a project on the Astera Server

1. Go to Server > Server Explorer. This will open a Server Explorer panel.

1. On the Server Explorer panel, right-click on the DEFAULT server node, and select Cluster Settings.

This will direct you to the general settings tab.

1. In the general settings tab, specify the Deployment Directory path. The Deployment Directory is the location where the deployment files will be stored for the server to access when a deployment schedule is run.

1. Now, open the project that you want to deploy on Astera’s Server by going to Project > Open and direct it to the .cprj file. This will open the selected project in the Project Explorer panel.

1. In the Project Explorer panel, right-click on the Project node, and select Build Archive (.Car) for the Project from the context menu.

1. Save this archive file on your system.

2. Now, open the Deployment window. Go to Server > Deployment.

This will direct you to the main Deployment screen.

1. Click the Add Deployment icon to add a new deployment.

1. Configure deployment settings by providing the following details:

• Name

• Archive File Path (*.car)

• Config File Path (an optional file to specify project parameters)

• Comment (optional)

Multiple deployments can be configured on the cluster from the deployment window. The top part of the deployment screen shows the configured deployment files.

Astera features integration with Team Foundation Server and VSTS to manage version control over your projects.

To connect to a Source Control server, navigate to

Project > Source Control > Connect to Source Control Server

Connecting to TFS

Windows Authentication can be used to connect to a Team Foundation Server. This uses the current user which is logged on to Windows and uses the windows credentials to authenticate the user.

Connecting to VSTS

Various authentication methods can be used to connect to Visual Studio Team Services (Visual Studio Online). When using these authentication methods, the Path can be set as empty with Port set to 0.

Alternate Authentication can be enabled inside the Security settings for your Visual Studio account. This enables the account to have a manageable separate username and password.

Personal Access Tokens can also be enabled and uses a generated token to authenticate the account.

Rest Authentication provides a built in prompt to login using your Microsoft account.

The Cluster Monitor and Cluster Settings help in managing the cluster database that you have made for Astera.

Cluster Monitor

To open the Cluster Monitor, go to the Server Explorer > Right Click on Default > Cluster Monitor.

The cluster monitor shows us details about all clusters on our server. It also tells us information about which product is running on a particular server, it's version, operating system, type of license as well details such as how many jobs are running and queued and more.

Cluster Settings

To open the Cluster Settings panel, go to the Server Explorer > Right Click on Default > Cluster Settings.

In the Cluster Settings panel, there are different tabs available where users can:

• General: Configure their general cluster settings

• Mail Setup: Setup their mailing service

• Path Mappings: Provide alternate paths for directory

• User Authentication: Setup different user authentication methods.

Let's look at all the configuration options available in these tabs.

General

• Name: Users can set their cluster's name here, by default the name 'DEFAULT' is set.

• Staging Directory: The staging directory path is specified here.

• Deployment Directory: The deployment directory path is specified here.

• Cloud File System:

  • If a user unchecks the option client and server share the same file system, then this indicates that the client and server does not exist on the same machine or network (for example their client is on a Local Network and their server is on a Cloud VM instance). In this case a temporary CAR file will be sent to the server to allow the flow to be executed easily on the server.

  • If a user unchecks the option Allow local file access, then it will restrict the user to only use system local paths to read/write data from files (like pdf, excel, csv etc)

• Purge Frequency:

  • Purge Job Info After: Users can specify the number of days until jobs information should be saved on the server, after which they will be purged.

  • Purge Server Events After: Users can specify the number of days until server events information should be saved on the server, after which they will be purged.

  • Purge Rest Request Info After: Users can specify the number of days until Rest request information should be saved on the server, after which they will be purged.

• Server Pause Options: Users can check this option when they want to pause all activity on their servers and specify the time duration until this is necessary. This option is usually useful when there is a Maintenance happening on a server.

• Usage Data and Error Reporting: Users can uncheck this option if they do not want to share usage and error reporting data with Astera.

Mail Setup

Path Mappings

User Authentication

CAR Convert Utility Guide

Introduction

Welcome to the CarConvert Utility guide for Astera Software. This document provides comprehensive instructions on using the CarConvert utility, designed to convert Centerprise Project files (*.cprj) into Centerprise Archive files (*.car). This guide will help you understand the utility's functionality and how to effectively use it to optimize your data integration processes.

Accessing the CAR Convert Utility

The CarConvert utility is provided as part of a downloadable zip file. Follow these steps to set up and use the utility.

Setting Up the CAR Convert Utility

Step 1: Download and Extract the Zip File

Download the Tool: Download the zip file.

Extract the Zip File: Extract the contents of the zip file to a preferred location on your system.

Step 2: Open the C# Project

Choose an IDE: To open the project, you will need an Integrated Development Environment (IDE) that supports C# projects. Examples include:

• Visual Studio Community Edition

• Visual Studio Code

• JetBrains Rider

Note: In this guide we are using Visual Studio Community Edition.

Open the Project: Locate the extracted files and open the C# project in your chosen IDE.

Using the CAR Convert Utility

Method 1: Running the CAR Convert Tool via IDE

1. Set Debug Properties:

   • In your IDE, navigate to Debug > CarConvert Debug Properties.

• • In the Debug Properties, locate the Command line arguments text area.

• • The CarConvert utility requires two parameters:

    1. cprj: Path to the existing Centerprise Project file.

    2. car: Path where the output Centerprise Archive file will be saved.

Example:

1. cprj: `C:\test\testproject.cprj`

2. car: `C:\test\testarchive.car`

   • Enter the paths for the .cprj file and the output directory as command-line arguments.

3. Run the Project:

   • Save the CarConvert file.

   • Run the file from the IDE.

• • The converted .car file will be created in the output directory you specified.

Method 2: Running the CAR Convert Tool via CLI

1. Build the Project:

   • In your IDE, navigate to Build > Build Solution.

• • This will create an executable file for the CarConvert utility.

Note: The CarConvert Project has to be built first so that the carconvert.exe executable is created.

1. Locate the Executable:

   • After building the project, navigate to the path where the executable is created. This is typically found at CarConvert > bin > Debug > CarConvert.exe.

1. Run the Executable via CLI:

   • Open your Command-Line Interface (CLI) (e.g., Command Prompt, PowerShell).

   • Navigate to the directory containing carconvert.exe.

   • Run the utility using the following command:

carconvert.exe <path_to_project_file.cprj> <path_to_output_file.car>

• • The converted .car file will be created in the output directory you specified.

By following these steps, you can effectively use the CarConvert utility to convert Centerprise Project files into Centerprise Archive files.

Advanced server monitoring and job management features have made troubleshooting server related errors and managing jobs easier for administrators. Proactively monitoring server health, preventing server downtimes, and identifying the causes of server failure is also made possible with the improved server monitoring and logging capabilities.

Moreover, a diagnostic file can be generated in case of server failure that contains information such as server status, server health, active connections, installed drivers etc. Administrators can use this information to troubleshoot and fix the problems.

The server monitoring features that we will discuss in this article are:

• Using Job Monitor to monitor jobs scheduled on the server

• Viewing Server Properties

• Creating Diagnostics File to troubleshoot server related issues

• Viewing Server Information

• Managing cluster databases through Cluster Monitor and Settings

Let’s go over each of these one by one.

Job Monitor

The Job Monitor will show you information regarding all the jobs (running, scheduled, or executed) on the Astera server.

The Job Monitor provides users with the following details:

1. Job ID

2. Parent

3. Priority

4. Job Type

5. Server

6. Job File

7. Job Name

8. Queued

9. Started

10. Duration

11. Status

12. Scheduled Job

13. Email Status

14. Records Read

15. Error Records

16. Warning Records

17. User

To open the Job Monitor, go to Server Explorer > Right-Click on Default > Job Monitor.

Server Properties

The Server Connection Properties window gives information about the cluster database and options for the diagnostics file. It can also be used to change the cluster database connection at any instance.

To open the Server Properties, go to Server Explorer > Right-click on the connected Server node > Server Properties. A Server Connection Properties screen will open.

Generating a Diagnostics File

A diagnostics file is generated in case of server failure or downtime. The diagnostics file is used to troubleshoot the causes for failure or downtime. It works as a performance and health monitor for Astera’s server.

Information included in the diagnostics file is as follows:

• Operating System

• Machine Information

• Error Information

• Server Information

To generate a diagnostics file, go to Server Explorer > Right-Click on the connected server node > Generate Diagnostics File. It will ask you for a file path where you want to save the file.

Once the file is saved, you can open the file from the given file path, and an XML file will open.

Server Information

To view details related to the server, Astera provides information regarding the connected server and machine through this option for easier management.

The Server Information screen will show the following specifics:

1. Server Information

2. Licensing Information

3. System Information

4. Server Snapshot

To access Server Information, go to Server Explorer > Right-Click on the connected server node > Show Server Information.

Cluster Monitor and Settings

The Cluster Monitor and settings help in managing the cluster database that you have made for Astera.

To open the Cluster Monitor, go to the Server Explorer > Right Click on Default > Cluster Monitor.

In the Cluster Settings panel, you can:

• General: Give staging and deployment directory paths.

• Mail Setup: Provide credentials to support mailing for job automation.

• Path Mapping: Provide alternate paths for directory.

To open the Cluster Settings panel, go to the Server Explorer > Right Click on Default > Cluster Settings.

The purpose of creating a project is to organize your Astera work-items in files and folders. This is particularly helpful when one is dealing with a large project which consists of numerous flows. It also helps in implementing a use-case in a more effective way and makes the process of deployment easier.

In this article, we will learn how to create a new Integration Project and use some of the options available in the Project Explorer panel.

Steps to Create a New Project

1. To create a new project, go to the menu bar and click on the following options:

Project > New > Integration Project

There are 3 different kinds of projects you can create by either clicking on its name, or pressing the given shortcut keys to the right.

• Integration Project: Creates an empty project which does not contain any pre-defined folders. It is ideal to use when you want to develop an integration project from scratch.

• Data Warehousing Project: Allows user to create a data warehousing project.

1. Next, navigate to the folder where you want to save your project, name your project, and click Save. In this example, we are saving our project with the name SampleProject.cprj.

1. You can view your project in the Project Explorer panel, as shown below.

Project Explorer Options

There are several options in the Project Explorer panel such as:

• Close Project: Closes a currently open project.

• Build Archive (.Car) for the Project: Creates an archive (.Car) file.

• Verify Project: Verifies design time errors of the project (such as missing fields, syntax errors in the expressions, etc.).

  Note: It does not verify the runtime errors.

• Refresh Project: Refreshes and synchronizes changes within the project.

• Replace parameter values for all the distinct parameters: Replaces all the distinct directory paths and the database connection information used in the project with new values provided.

• Replace parameters infos for all distinct and consolidated parameters: Replaces all the distinct and consolidated directory paths and the database connection information used in the project with new values provided (this option removes the repeats and shows consolidated parameters).

• Add Project to Source Control: Enables Source Control to keep track of new changes and save all previous commits in case the user wants to revert to the last saved version.

• Automatically re-build lineage when project is loaded or modified: Represents connections among different files such as dataflows, subflows, etc. in the project graphically.

Steps to Add New Item or Add New Folder to a Project

1. Right-click on the project name inside the Project Explorer, and from the context menu, select Add New Item.

1. Choose the type of item from the available options, provide an appropriate name for the item, and click Add. In this example, we are adding a new dataflow with the name SampleDataflow.Df.

1. The new item, SampleDataflow.Df, is now a part of your project.

1. Similarly, to add a new folder, right-click on the project name inside the Project Explorer, and click Add New Folder.

1. The new folder is now a part of your project. You can rename the folder by right-clicking on Folder and selecting Rename.

Steps to Add Existing Items or Add Existing Folder to a Project

1. Right-click on the project name in the Project Explorer and click Add Existing Items or Add Existing Folder, depending on what you want to add. We will see how to Add Existing Items first.

1. Locate the item/folder you want to add to the project in the local directory, select it, and click Open. In this case, we are adding SampleDataflow2.Df to our project.

1. The item, SampleDataflow2.Df, is now a part of your project.

1. You can try the Add Existing Folder option by following the same steps mentioned above if you wish to add an existing folder instead of an existing item to the Project.

To conclude, this is how you can create and manage a project in Astera Data Stack using the tools provided in the Project Explorer panel.