Astera Data Stack
Version 7
Version 7
  • Welcome to Astera Data Stack Documentation
  • Release Notes
    • Upgrading Astera 6 to Version 7
    • Release Notes for Astera 7.1
    • What is New in Astera 7.4
    • What’s New in Astera 7.4.1.221
    • What’s New in Astera 7.6
    • Upgrading to Astera 7.6
  • Setting Up
    • System Requirements
    • Installing Astera Data Integrator
    • Setting up Astera Integration Server 7
    • UI Walkthrough - Astera Data Integrator
  • Dataflows
    • Introducing Dataflows
    • Sources
      • Setting Up Sources
      • Raw Text Filters in File Sources
      • COBOL File Source
      • Database Table Source
      • Data Model Source
      • Delimited File Source
      • Email Source
      • Excel Workbook Source
      • File System Items Source
      • Fixed Length File Source
      • PDF Form Source
      • Report Source
      • SQL Query Source
      • XML/JSON File Source
    • Transformations
      • Aggregate Transformation
      • Apply To All Transformation
      • Constant Value Transformation
      • Data Cleanse Transformation
      • Data Quality Rules Transformation
      • Denormalize Transformation
      • Distinct Transformation
      • Expression Transformation
      • Filter Transformation
      • Function Transformations
      • Join Transformation
      • Merge Transformation
      • Normalize Transformation
      • Passthru Transformation
      • Rest Client
      • Route Transformation
      • Sequence Generator Transformation
      • Sort Transformation
      • Sources as Transformations
      • Subflow Transformation
      • Tree Join Transformation
      • Union Transformation
      • Web Service Transformation
    • Destinations
      • Setting Up Destinations
      • Database Table Destination
      • Delimited File Destination
      • Excel Workbook Destination
      • Fixed Length File Destination
      • SQL Statement Destination
      • XML/JSON File Destination
    • Data Logging and Profiling
      • Creating Data Profile
      • Creating Field Profile
      • Data Quality Mode
      • Record Level Log
      • Using Data Quality Rules in Astera
    • Database Write Strategies
      • Database Write Strategies
    • Text Processors
      • Fixed Length Parser
      • Fixed Length Serializer
  • Workflows
    • Adding Workflow Tasks
    • Creating Workflows
    • Using Workflow Designer
    • Customizing Workflows With Parameters
    • Workflows with a Dynamic Destination Path
    • Resuming and Rerunning Workflows in Astera
  • Subflows
    • Using Subflows
  • Functions
    • Functions Glossary
    • Introducing Function Transformations
    • Custom Functions
    • Logical
      • Coalesce (Any value1, Any value2)
      • IsNotNull (AnyValue)
      • IsRealNumber (AnyValue)
      • IsValidSqlDate (Date)
      • IsDate (AnyValue)
      • If (Boolean)
      • If (DateTime)
      • If (Double)
      • Exists
      • If (Int64)
      • If (String)
      • IsDate (str, strformat)
      • IsInteger (AnyValue)
      • IsNullOrWhitespace (StringValue)
      • IsNullorEmpty (StringValue)
      • IsNull (AnyValue)
      • IsNumeric (AnyValue)
    • Conversion
      • GetDateComponents (DateWithOffset)
      • ParseDate (Formats, Str)
      • GetDateComponents (Date)
      • HexToInteger (Any Value)
      • ToInteger (Any value)
      • ToDecimal (Any value)
      • ToReal (Any value)
      • ToDate (String dateStr)
      • TryParseDate (String, UnknownDate)
      • ToString (Any value)
      • ToString (DateValue)
      • ToString (Any data, String format)
    • Math
      • Abs (Double)
      • Abs (Decimal)
      • Ceiling (Real)
      • Ceiling(Decimal)
      • Floor (Decimal)
      • Floor (Real)
      • Max (Decimal)
      • Max (Date)
      • Min (Decimal)
      • Min (Date)
      • Max (Real)
      • Max (Integer)
      • Min (Real)
      • Pow (BaseExponent)
      • Min (Integer)
      • RandomReal (Int)
      • Round (Real)
      • Round (Real Integer)
      • Round (Decimal Integer)
      • Round (Decimal)
    • Financial
      • DDB
      • FV
      • IPmt
      • IPmt (FV)
      • Pmt
      • Pmt (FV)
      • PPmt
      • PPmt (FV)
      • PV (FV)
      • Rate
      • Rate (FV)
      • SLN
      • SYD
    • String
      • Center (String)
      • Chr (IntAscii)
      • Asc (String)
      • AddCDATAEnvelope
      • Concatenate (String)
      • ContainsAnyChar (String)
      • Contains (String)
      • Compact (String)
      • Find (Int64)
      • EndsWith (String)
      • FindIntStart (Int32)
      • Extract (String)
      • GetFindCount (Int64)
      • FindLast (Int64)
      • GetDigits (String)
      • GetLineFeed
      • Insert (String)
      • IsAlpha
      • GetToken
      • IndexOf
      • IsBlank
      • IsLower
      • IsUpper
      • IsSubstringOf
      • Length (String)
      • LeftOf (String)
      • Left (String)
      • IsValidName
      • Mid (String)
      • PadLeft
      • Mid (String Chars)
      • LSplit (String)
      • PadRight
      • ReplaceAllSpecialCharsWithSpace
      • RemoveChars (String str, StringCharsToRemove)
      • ReplaceLast
      • RightAlign
      • Reverse
      • Right (String)
      • RSplit (String)
      • SplitStringMultipleRecords
      • SplitStringMultipleRecords (2 Separators)
      • SplitString (3 separators)
      • SplitString
      • SplitStringMultipleRecords (3 Separators)
      • Trim
      • SubString (NoOfChars)
      • StripHtml
      • Trim (Start)
      • TrimExtraMiddleSpace
      • TrimEnd
      • PascalCaseWithSpace (String str)
      • Trim (String str)
      • ToLower(String str)
      • ToProper(String str)
      • ToUpper (String str)
      • Substring (String str, Integer startAt)
      • StartsWith (String str, String value)
      • RemoveAt (String str, Integer startAt, Integer noofChars)
      • Proper (String str)
      • Repeat (String str, Integer count)
      • ReplaceAll (String str, String lookFor, String replaceWith)
      • ReplaceFirst (String str, String lookFor, String replaceWith)
      • RightOf (String str, String lookFor)
      • RemoveChars (String str, String charsToRemove)
      • SplitString (String str, String separator1, String separator2)
    • Date Time
      • AddMinutes (DateTime)
      • AddDays (DateTimeOffset)
      • AddDays (DateTime)
      • AddHours (DateTime)
      • AddSeconds (DateTime)
      • AddMonths (DateTime)
      • AddMonths (DateTimeOffset)
      • AddMinutes (DateTimeOffset)
      • AddSeconds (DateTimeOffset)
      • AddYears (DateTimeOffset)
      • AddYears (DateTime)
      • Age (DateTime)
      • Age (DateTimeOffset)
      • CharToSeconds (Str)
      • DateDifferenceDays (DateTimeOffset)
      • DateDifferenceDays (DateTime)
      • DateDifferenceHours (DateTimeOffset)
      • DateDifferenceHours (DateTime)
      • DateDifferenceMonths (DateTimeOffset)
      • DateDifferenceMonths (DateTime)
      • DatePart (DateTimeOffset)
      • DatePart (DateTime)
      • DateDifferenceYears (DateTimeOffset)
      • DateDifferenceYears (DateTime)
      • Month (DateTime)
      • Month (DateTimeOffset)
      • Now
      • Quarter (DateTime)
      • Quarter (DateTimeOffset)
      • Second (DateTime)
      • Second (DateTimeOffset)
      • SecondsToChar (String)
      • TimeToInteger (DateTime)
      • TimeToInteger (DateTimeOffset)
      • ToDate Date (DateTime)
      • ToDate DateTime (DateTime)
      • ToDateString (DateTime)
      • ToDateTimeOffset-Date (DateTimeOffset)
      • ToDate DateTime (DateTimeOffset)
      • ToDateString (DateTimeOffset)
      • Today
      • ToLocal (DateTime)
      • ToJulianDate (DateTime)
      • ToJulianDayNumber (DateTime)
      • ToTicks (Date dateTime)
      • ToTicks (DateTimeWithOffset dateTime)
      • ToUnixEpoc (Date dateTime)
      • ToUtc (Date dateTime)
      • UnixTimeStampToDateTime (Real unixTimeStamp)
      • UtcNow ()
      • Week (Date dateTime)
      • Week (DateTimeWithOffset dateTime)
      • Year (Date dateTime)
      • Year (DateTimeWithOffset dateTime)
      • DateToJulian (Date dateTime, Integer length)
      • DateTimeOffsetUtcNow ()
      • DateTimeOffsetNow ()
      • Day (DateTimeWithOffset dateTime)
      • Day (Date dateTime)
      • DayOfWeekStr (DateTimeWithOffset dateTime)
      • DayOfWeek (DateTimeWithOffset dateTime)
      • DayOfWeek (Date dateTime)
      • DateToJulian (DateTimeWithOffset dateTime, Integer length)
      • DayOfWeekStr (Date dateTime)
      • FromJulianDate (Real julianDate)
      • DayOfYear (Date dateTime)
      • DaysInMonth(Integer year, Integer month)
      • DayOfYear (DateTimeWithOffset dateTime)
      • FromUnixEpoc
      • FromJulianDayNumber (Integer julianDayNumber)
      • FromTicksUtc(Integer ticks)
      • FromTicksLocal(Integer ticks)
      • Hour (Date dateTime)
      • Hour (DateTimeWithOffset dateTime)
      • Minute (Date dateTime)
      • JulianToDate (String julianDate)
      • Minute (DateTimeWithOffset dateTime)
      • DateToIntegerYYYYMMDD (DateTimeWithOffset dateTime)
      • DateToIntegerYYYYMMDD (Date dateTime)
    • Files
      • AppendTextToFile (String filePath, String text)
      • CopyFile (String sourceFilePath, String destFilePath, Boolean overWrite)
      • CreateDateTime (String filePath)
      • DeleteFile (String filePath)
      • DirectoryExists (String filePath)
      • FileExists (String filePath)
      • FileLength (String filePath)
      • FileLineCount (String filePath)
      • GetDirectory (String filePath)
      • GetEDIFileMetaData (String filePath)
      • GetExcelWorksheets (String excelFilePath)
      • GetFileExtension (String filePath)
      • GetFileInfo (String filePath)
      • GetFileName (String filePath)
      • GetFileNameWithoutExtension (String filePath)
      • LastUpdateDateTime (String filePath)
      • MoveFile (String filePath, String newDirectory)
      • ReadFileBytes (String filePath)
      • ReadFileFirstLine (String filePath)
      • ReadFileText (String filePath)
      • ReadFileText (String filePath, String codePage)
      • WriteBytesToFile (String filePath, ByteArray bytes)
      • WriteTextToFile (String filePath, String text)
    • Date Time With Offset
      • ToDateTimeOffsetFromDateTime (dateTime String)
      • ToUtc (DateTimeWithOffset)
      • ToDateTimeOffsetFromDateTime
      • ToDateTimeOffset (String dateTimeOffsetStr)
      • ToDateTimeFromDateTimeOffset
    • GUID
      • NewGuid
    • Encoding
      • ToBytes
      • FromBytes
      • UrlEncode
      • UrlDecode
    • Regular Expressions
      • ReplaceRegEx
      • ReplaceRegEx (Integer StartAt)
    • TimeSpan
      • Minutes
      • Hours
      • Days
      • Milliseconds
    • Matching
      • Soundex
      • DoubleMetaphone
      • RefinedSoundex
  • Report Model
    • User Guide
      • Report Model Tutorial
    • Report Model Interface
      • Field Properties
      • Pattern Properties
      • Region Properties
      • Report Browser
      • Report Options
    • Use Cases
      • Applying Pattern to Line
      • Auto-Creating Data Regions and Fields
      • Auto-Parsing
      • Creating Multi-Column Data Regions
      • Floating Patterns and Floating Fields
      • How to Work with Microsoft Word (Doc/Docx) Files in a Report Model
      • How to Work With OCR Scanned Files in a Report Model
      • How to Work With PDF Scaling Factor in a Report Model
      • Line Count
      • Pattern Count
      • Pattern is a Regular Expression
    • Exporting Options
      • Exporting a Report Model
      • Exporting Report Model to a Dataflow
    • Miscellaneous
      • Astera Report Model: Performance on Different Hardware Settings and Case Complexities
      • Microsoft Word and Rich Text Format Support
      • Importing Monarch Models
  • Project Management
    • Project Management
      • Deployment
      • Parameterization
    • Job Scheduler
      • Scheduling and Running Jobs on a Server
  • Web Services
    • Configuring Google Drive API through REST Client in Astera
    • Connecting to Eloqua using Astera REST API
    • POSTing Data Using the REST Client
    • Using the REST Client to Download a Text File
  • Metadata Management
    • Lineage and Impact Analysis
  • Connectors and Providers
    • Setting Up Oracle ODP.Net Connectivity in Astera
    • Running Microsoft Access Database Engine with Astera
    • Oracle Client Tools Setup
    • Oracle Data Load Options
  • Miscellaneous
    • Job Trace Improvements
    • SmartMatch Feature
    • Synonym Dictionary File
    • Using the Data Source Browser in Astera
  • Best Practices
    • Astera Best Practices - Dataflows
    • Cardinality Errors FAQs
    • Overview of Cardinality in Data Modeling
Powered by GitBook

© Copyright 2025, Astera Software

On this page
  • Creating a New Workflow
  • Using Workflow Designer
  • Adding Objects
  • Unlimited Undo/Redo
  • Copying Objects
  • Managing Workflow Layout
  • Linking Objects
  • Ports
  • Links
  • Setting Object Properties
  • Verifying Workflow
  • Running Workflow
  • Setting up Sources
  • Flat File Sources:
  • Tree File Sources:
  • Database Sources:
  1. Workflows

Creating Workflows

PreviousAdding Workflow TasksNextUsing Workflow Designer

Last updated 9 months ago

A workflow enables automated and repeated execution of a sequence of tasks, such as running a program, sending email, uploading a file, running a transfer setting or batch, executing some SQL code, and many others.

The tasks run according to some predefined path and following the custom logic that controls in what circumstances specific paths should be activated. You can add any number of tasks on a single visual workflow diagram, as well as specify what should happen at each step depending on whether the task has been completed successfully, or returned an error.

You can also route the workflow one way or another by using Boolean conditions that are suitable for your scenario. You can even start another workflow, or a dataflow, to run within your main workflow.

A workflow can be created in a short time and with little effort by using a graphical user interface with a drag-and-drop capability. The Astera workflow editor allows you to copy or move workflow tasks, source objects, or workflow parameters; change their properties, and do a number of other things, all of this with the capability of an unlimited undo-redo of any previous action.

A workflow makes it easy to visualize and implement complex sequences of tasks because pieces of a workflow can be used like ‘building blocks’. These building blocks can be pasted into a different workflow, or even into the same workflow, which allows you to quickly replicate tasks, or even sequences of tasks, with similar properties.

Objects can be added to a workflow in several ways, such as direct drag-and-drop of files from any Explorer window, drag-and-drop of tables or views from the built-in Data Source Browser, or by adding an object from the Flow toolbox.

A workflow that you create in Astera can run on local or remote servers. The workflow can also be ported to run in any number of target environments. This is achieved by using run-time Parameters, which change their values depending on the current context in which the workflow runs. Finally, to make sure that your workflow is continuously up to date with your changing environment, you can easily update data connections, such as server names, login credentials etc, throughout the entire workflow, using a single easy-to-use interface. This means that your workflow will stay in sync with the current requirements, even if the data connections have changed since the workflow was created.

Creating a New Workflow

To create a new workflow, go to File -> New -> Workflow on the main menu. Or you can expand the dropdown on the main toolbar and select Workflow as shown below.

Using Workflow Designer

Adding Objects

A workflow always has at least one workflow task and may have zero or more sources. Sources, workflow tasks, and resources are represented as objects in the workflow.

Depending on the type of object, an object can be added to the workflow in one of the following ways:

For Flat File Sources

  1. Using Drag-and-Drop. You can drag a file of the type listed below from an Explorer window and drop it onto an open workflow tab in Astera.

    • Excel

    • Delimited

    • Fixed-Length File

The advantage of using drag-and-drop compared to other methods is that many of the object’s properties are pre-populated for you based on the file’s content. For example, the field layout is automatically filled out so that there is no need to manually create it.

Note: XML files cannot be added to a dataflow using the drag-and-drop method.

  1. Using Toolbox. You can add a source object by selecting it from the appropriate category in the Flow toolbox.

For example, to add a source comma-delimited file object, expand the Sources group in the Flow toolbox, and the drag and drop the Delimited File Source tool on the workflow.

Note that an object added this way initially does not have any properties defined. To define its properties, double-click on the object title, or right-click and select Properties from the context menu.

In the Properties screen that opens, select the File Path of the file that will be associated with the object. Field layout and other properties can then be populated based on the file’s content.

  1. Copying and pasting an existing object from the same or different workflow or dataflow. I

If your source is already defined in the same or different workflow (or dataflow), you can copy the existing object and paste it into your workflow. The object being copied retains the properties of the original object and is assigned a unique new name to distinguish it from the original object.

For XML Sources

  1. Using Toolbox. To add an XML source to the workflow, use the XML File Sourcetool in the Sources group in the Flow toolbox.

Note that the XML File object initially will not have any properties defined. To define its properties, double-click on the object’s title or right-click and select Properties from the context menu.

In the Properties screen that opens, select the File Path of the XML file that will be associated with the object. Additionally, please provide the path to the XSD schema that controls the layout of your XML file.

As with flat files, you can copy and paste an existing XML object from the same or different workflow or dataflow. The object being copied retains the properties of the original object and is assigned a unique new name to distinguish it from the original object.

For Database Sources

  1. Using drag-and-drop. You can drag a database table or view it from the Data Source Browser and drop it on an open workflow tab.

To open the Data Source Browser, go to View -> Data Source Browser. Connect to the appropriate server, then expand the Database tree and expand Tables (or Views) tree to select your table (or view). Drag-and-drop the selected table or view to the workflow.

By default, the database table is added as a Database Table Source object.

To add a data model source, press and hold Ctrl key while dragging and dropping a table (or view) from the Data Source Browser.

As with files, you can copy and paste an existing database table object from the same or different workflow or dataflow. The object being copied retains the properties of the original object, and is assigned a unique new name to distinguish it from the original object.

For Any Other Types of Objects (for example, workflow tasks or resources objects):

  1. Using Toolbox. You can add an object by selecting it from the appropriate category in the Flow toolbox.

Note that an object added this way initially does not have any properties defined. To define its properties, double-click on the object title or right-click and select Properties from the context menu.

  1. Copying and pasting an existing object from the same or different workflow or dataflow.

If your object is already defined in the same or different workflow (or dataflow), you can copy the existing object and paste it into your workflow. The object being copied retains the properties of the original object and is assigned a unique new name to distinguish it from the original object.

Unlimited Undo/Redo

The workflow editor supports unlimited undo-redo capability. You can quickly undo or redo the last action done, or undo/redo several actions at once.

Copying Objects

Using the copy and paste feature, you can replicate an object on your workflow by copying it into a new object with a different name to distinguish it from the original object.

You can paste this object into the same workflow, or a different workflow.

You can also copy several objects at once. To do so, click the objects you wish to copy while pressing Ctrl key. Or, you can draw a rectangle with your mouse while holding down the left mouse button. The objects inside the rectangle will be selected. Right-click on a selected object and select Copy from the context menu. Then right-click on white space in the same or different workflow, and select Paste from the context menu.

To move an object or a set of objects, use the Cut and Paste sequence similar to the one described above.

Note: When you move objects, they keep their original names.

Note: You can use Ctrl + C shortcut to copy the selected object to the clipboard. Ctrl + V will paste it from the clipboard. Ctrl + X will cut it into the clipboard.

Managing Workflow Layout

The Auto Layout feature allows you to arrange objects on the workflow improving its visual representation.

Note: You can manually move the object around the workflow by holding the left mouse button down over the object title, and moving it to a new location.

The following tools are available from the Workflow menu to help you adjust the display size of the dataflow:

  • Zoom In

  • Zoom Out

  • Fit To Screen

Additionally, you can select a custom zoom percentage using the Zoom % input on the Workflow toolbar.

Linking Objects

Ports

Any Workflow Tasks objects you add to the workflow will show Input and Output ports. Ports make it possible to connect an object to another object via links, creating the required workflow path.

The Input port on an object allows you to connect it to an upstream object on the workflow. Conversely, the Output port allows you to connect your object to a downstream object on the workflow. The downstream object representing a task will be processed after the upstream object it is connected to has finished running (with some exceptions, such as the Run Program task, which may not require waiting for the program to exit).

Source objects and Resources objects have no ports and cannot be linked to other objects on the workflow. These types of objects cannot be linked to any specific object as they provide values that can be used by any task throughout the workflow.

All source objects added to the workflow are used in Singleton mode only. A Singleton mode object only returns the first record of data (excluding the header) from the object.

This makes it possible for you to get the values from the first record anywhere on the workflow, using the following notation: ObjectName.FieldName.

Similarly, Source objects and Resources Objects such as Parameters or Context Info, provide parameters used throughout the workflow. Parameters have a similar notation of ObjectName.ParameterName.

Links

To create links between two objects, drag the Output port of the upstream object and drop it on the Input port of the downstream object.

To remove a link between two objects, right-click on the link and select Delete, or left-click on the link and press the Del key on the keyboard.

Note: By default, the link of Normal type is created between two objects. Normal link means that the downstream object will be processed in the event of a successful completion of the upstream object to which it is connected. Normal links are displayed in green.

In contrast, an Error link can be created between two objects, meaning that the downstream object will be processed only in the event of a failed (error) status of the upstream object to which it is connected. Error links are displayed in orange.

To change the link type, right-click on the link and select Change Link Type.

An example of linked workflow tasks is shown below.

Setting Object Properties

To open an object’s properties, double-click on the object’s title. Or right-click on the object and select Properties from the context menu.

Note: While in the Properties screen, you can switch to the Properties of another object on the workflow. To do so, select an object in the Editing dropdown.

The following functions are common for many types of objects on the workflow. These functions are available via the context menu.

  • Rename – renames the object.

  • Edit File – only applies to files. It opens the file for editing in a new tab.

  • Edit Schema – only applies to XML files. It opens the XSD file for editing in a new tab.

  • View Table Data – only applies to database tables. It opens the table for viewing in a new tab.

  • View Table Schema – only applies to database tables. It opens the database schema for viewing in a new tab.

  • Delete – deletes the object from the workflow. This will also remove any links to and from the object.

  • Cut – removes the object from the workflow and places it into the clipboard. The object can then be ‘pasted’.

  • Copy – copies the object into the clipboard leaving the original object as it is. A copy of the object can then be ‘pasted’.

  • Paste - pastes the object from the clipboard.

Verifying Workflow

Verifying a workflow will list any errors or warnings present in the workflow design. Correct any such errors or warnings, and verify your workflow again to ensure there are no errors.

To verify a workflow, click the window.

Running Workflow

Setting up Sources

Each source on the workflow is represented as a source object. You can have any number of sources of the workflow, but they can only be used in Singleton mode. A Singleton mode source only returns the first record of data (excluding the header) from the source. This makes it possible for you to get the values from the first record anywhere on the workflow, using the following notation: ObjectName.FieldName.

The following source types are supported by the workflow engine:

Flat File Sources:

  • Delimited File

  • Excel File

  • Fixed Length File

Tree File Sources:

  • COBOL

  • XML File

Database Sources:

  • Data Model

  • Database Table

  • SQL Query

All sources can be added to the workflow by picking a source type on the Flow toolbox and dropping it on the workflow. File sources can also be added by dragging and dropping a file from an Explorer window. Database sources can be drag-and-dropped from the Data Source Browser.

To undo the last action, open the View menu and select Undo. You can also click the icon on the Workflow toolbar. You can also use Ctrl + Z shortcut.

To redo the last action, open the View menu and select Redo. You can also click theicon on the Workflow toolbar. You can also use Ctrl + Y shortcut.

To undo several actions at once, select the first action you wish to undo from the dropdown. The actions following the selected action will also be undone.

To redo several actions at once, select the first action you wish to redo from the dropdown. The actions following the selected action will also be redone.

To invoke the Auto Layout feature, click the icon on the Workflow toolbar. Or you can open the Workflow menu and select Auto Layout.

To stop verification while it is still in process, click the icon on the Verify window Toolbar.

To run your workflow, click the icon on the main Toolbar. The workflow will run on the server that is selected in the Server input on the main Toolbar.

To stop a workflow that is currently running, click the icon on the Job Progress window Toolbar.