Astera Data Stack
Version 6
Version 6
  • Welcome to Astera Data Stack Documentation
  • Release Notes
    • Upgrading from Astera 5.1 to 6.0
  • Getting Started
    • Introduction to Astera 6 Webinar Video
  • Dataflows
    • Introducing Dataflows
    • Dataflow Examples
    • Parameterizing Dataflows
    • Sources
      • Setting Up Sources
      • ADO.Net Metadata Collections Source
      • COBOL File Source
      • Data Model Source
      • Database Table Source
      • Delimited File Source
      • Excel File Source
      • File System Entries Source
      • Fixed Length File Source
      • SQL Query Source
      • XML/JSON File Source
    • Transformations
      • Introducing Transformations
      • Aggregate Transformation
      • Apply To All Transformation
      • Constant Value Transformation
      • Data Quality Rules Transformation
      • Denormalize Transformation
      • Distinct Transformation
      • Expression Transformation
      • Filter Transformation
      • Join Transformation
      • Merge Transformation
      • Normalize Transformation
      • Passthru Transformation
      • REST Client
      • Route Transformation
      • Sequence Generator
      • Sort Transformation
      • Subflow Transformation
      • Tree Join Transformation
      • Union Transformation
      • Web Service Transformation
    • Destinations
      • Setting Up Destinations
      • Database Table Destination
      • Delimited File Destination
      • Excel File Destination
      • Fixed Length File Destination
      • SQL Statement Destination
      • XML File Destination
    • Maps
      • Linking and Mapping Objects
      • Creating Constant Value Maps
      • Creating Database Lookup Maps
      • Creating Direct Maps
      • Creating Expression Maps
      • Creating Function Maps
      • Creating List Lookup Maps
      • Creating SQL Statement Lookup Maps
    • Data Logging and Profiling
      • Creating Data Profiles
      • Creating Field Profiles
      • Using Data Quality Mode
      • Using Data Quality Rules
      • Using Record Level Log
    • Database Write Strategies
      • Database Write Strategies
  • Workflows
    • Adding Workflow Tasks
    • Creating Workflows
    • Customizing Workflows With Parameters
    • Using Workflow Designer
  • Subflows
    • Using Subflows in Astera
  • Report Model
    • Report Model Tutorial
    • Applying an Existing Model to a New Report
    • Using Report Models to Extract Business Information from Printed Documents
  • Functions
    • Introducing Function Transformations
    • Functions Glossary
    • Custom Functions
    • Logical
      • Coalesce (Any value1, Any value2)
      • IsNotNull (AnyValue)
      • IsRealNumber (AnyValue)
      • IsValidSqlDate (Date)
      • IsDate (AnyValue)
      • If (Boolean)
      • If (DateTime)
      • If (Double)
      • Exists
      • If (Int64)
      • If (String)
      • IsDate (str, strformat)
      • IsInteger (AnyValue)
      • IsNullOrWhitespace (StringValue)
      • IsNullorEmpty (StringValue)
      • IsNull (AnyValue)
      • IsNumeric (AnyValue)
    • Conversion
      • GetDateComponents (DateWithOffset)
      • ParseDate (Formats, Str)
      • GetDateComponents (Date)
      • HexToInteger (Any Value)
      • ToInteger (Any value)
      • ToDecimal (Any value)
      • ToReal (Any value)
      • ToDate (String dateStr)
      • TryParseDate (String, UnknownDate)
      • ToString (Any value)
      • ToString (DateValue)
      • ToString (Any data, String format)
    • Math
      • Abs (Double)
      • Abs (Decimal)
      • Ceiling (Real)
      • Ceiling(Decimal)
      • Floor (Decimal)
      • Floor (Real)
      • Max (Decimal)
      • Max (Date)
      • Min (Decimal)
      • Min (Date)
      • Max (Real)
      • Max (Integer)
      • Min (Real)
      • Pow (BaseExponent)
      • Min (Integer)
      • RandomReal (Int)
      • Round (Real)
      • Round (Real Integer)
      • Round (Decimal Integer)
      • Round (Decimal)
    • Financial
      • DDB
      • FV
      • IPmt
      • IPmt (FV)
      • Pmt
      • Pmt (FV)
      • PPmt
      • PPmt (FV)
      • PV (FV)
      • Rate
      • Rate (FV)
      • SLN
      • SYD
    • String
      • Center (String)
      • Chr (IntAscii)
      • Asc (String)
      • AddCDATAEnvelope
      • Concatenate (String)
      • ContainsAnyChar (String)
      • Contains (String)
      • Compact (String)
      • Find (Int64)
      • EndsWith (String)
      • FindIntStart (Int32)
      • Extract (String)
      • GetFindCount (Int64)
      • FindLast (Int64)
      • GetDigits (String)
      • GetLineFeed
      • Insert (String)
      • IsAlpha
      • GetToken
      • IndexOf
      • IsBlank
      • IsLower
      • IsUpper
      • IsSubstringOf
      • Length (String)
      • LeftOf (String)
      • Left (String)
      • IsValidName
      • Mid (String)
      • PadLeft
      • Mid (String Chars)
      • LSplit (String)
      • PadRight
      • ReplaceAllSpecialCharsWithSpace
      • RemoveChars (String str, StringCharsToRemove)
      • ReplaceLast
      • RightAlign
      • Reverse
      • Right (String)
      • RSplit (String)
      • SplitStringMultipleRecords
      • SplitStringMultipleRecords (2 Separators)
      • SplitString (3 separators)
      • SplitString
      • SplitStringMultipleRecords (3 Separators)
      • Trim
      • SubString (NoOfChars)
      • StripHtml
      • Trim (Start)
      • TrimExtraMiddleSpace
      • TrimEnd
      • PascalCaseWithSpace (String str)
      • Trim (String str)
      • ToLower(String str)
      • ToProper(String str)
      • ToUpper (String str)
      • Substring (String str, Integer startAt)
      • StartsWith (String str, String value)
      • RemoveAt (String str, Integer startAt, Integer noofChars)
      • Proper (String str)
      • Repeat (String str, Integer count)
      • ReplaceAll (String str, String lookFor, String replaceWith)
      • ReplaceFirst (String str, String lookFor, String replaceWith)
      • RightOf (String str, String lookFor)
      • RemoveChars (String str, String charsToRemove)
      • SplitString (String str, String separator1, String separator2)
    • Date Time
      • AddMinutes (DateTime)
      • AddDays (DateTimeOffset)
      • AddDays (DateTime)
      • AddHours (DateTime)
      • AddSeconds (DateTime)
      • AddMonths (DateTime)
      • AddMonths (DateTimeOffset)
      • AddMinutes (DateTimeOffset)
      • AddSeconds (DateTimeOffset)
      • AddYears (DateTimeOffset)
      • AddYears (DateTime)
      • Age (DateTime)
      • Age (DateTimeOffset)
      • CharToSeconds (Str)
      • DateDifferenceDays (DateTimeOffset)
      • DateDifferenceDays (DateTime)
      • DateDifferenceHours (DateTimeOffset)
      • DateDifferenceHours (DateTime)
      • DateDifferenceMonths (DateTimeOffset)
      • DateDifferenceMonths (DateTime)
      • DatePart (DateTimeOffset)
      • DatePart (DateTime)
      • DateDifferenceYears (DateTimeOffset)
      • DateDifferenceYears (DateTime)
      • Month (DateTime)
      • Month (DateTimeOffset)
      • Now
      • Quarter (DateTime)
      • Quarter (DateTimeOffset)
      • Second (DateTime)
      • Second (DateTimeOffset)
      • SecondsToChar (String)
      • TimeToInteger (DateTime)
      • TimeToInteger (DateTimeOffset)
      • ToDate Date (DateTime)
      • ToDate DateTime (DateTime)
      • ToDateString (DateTime)
      • ToDateTimeOffset-Date (DateTimeOffset)
      • ToDate DateTime (DateTimeOffset)
      • ToDateString (DateTimeOffset)
      • Today
      • ToLocal (DateTime)
      • ToJulianDate (DateTime)
      • ToJulianDayNumber (DateTime)
      • ToTicks (Date dateTime)
      • ToTicks (DateTimeWithOffset dateTime)
      • ToUnixEpoc (Date dateTime)
      • ToUtc (Date dateTime)
      • UnixTimeStampToDateTime (Real unixTimeStamp)
      • UtcNow ()
      • Week (Date dateTime)
      • Week (DateTimeWithOffset dateTime)
      • Year (Date dateTime)
      • Year (DateTimeWithOffset dateTime)
      • DateToJulian (Date dateTime, Integer length)
      • DateTimeOffsetUtcNow ()
      • DateTimeOffsetNow ()
      • Day (DateTimeWithOffset dateTime)
      • Day (Date dateTime)
      • DayOfWeekStr (DateTimeWithOffset dateTime)
      • DayOfWeek (DateTimeWithOffset dateTime)
      • DayOfWeek (Date dateTime)
      • DateToJulian (DateTimeWithOffset dateTime, Integer length)
      • DayOfWeekStr (Date dateTime)
      • FromJulianDate (Real julianDate)
      • DayOfYear (Date dateTime)
      • DaysInMonth(Integer year, Integer month)
      • DayOfYear (DateTimeWithOffset dateTime)
      • FromUnixEpoc
      • FromJulianDayNumber (Integer julianDayNumber)
      • FromTicksUtc(Integer ticks)
      • FromTicksLocal(Integer ticks)
      • Hour (Date dateTime)
      • Hour (DateTimeWithOffset dateTime)
      • Minute (Date dateTime)
      • JulianToDate (String julianDate)
      • Minute (DateTimeWithOffset dateTime)
      • DateToIntegerYYYYMMDD (DateTimeWithOffset dateTime)
      • DateToIntegerYYYYMMDD (Date dateTime)
    • Files
      • AppendTextToFile (String filePath, String text)
      • CopyFile (String sourceFilePath, String destFilePath, Boolean overWrite)
      • CreateDateTime (String filePath)
      • DeleteFile (String filePath)
      • DirectoryExists (String filePath)
      • FileExists (String filePath)
      • FileLength (String filePath)
      • FileLineCount (String filePath)
      • GetDirectory (String filePath)
      • GetEDIFileMetaData (String filePath)
      • GetExcelWorksheets (String excelFilePath)
      • GetFileExtension (String filePath)
      • GetFileInfo (String filePath)
      • GetFileName (String filePath)
      • GetFileNameWithoutExtension (String filePath)
      • LastUpdateDateTime (String filePath)
      • MoveFile (String filePath, String newDirectory)
      • ReadFileBytes (String filePath)
      • ReadFileFirstLine (String filePath)
      • ReadFileText (String filePath)
      • ReadFileText (String filePath, String codePage)
      • WriteBytesToFile (String filePath, ByteArray bytes)
      • WriteTextToFile (String filePath, String text)
    • Date Time With Offset
      • ToDateTimeOffsetFromDateTime (dateTime String)
      • ToUtc (DateTimeWithOffset)
      • ToDateTimeOffsetFromDateTime
      • ToDateTimeOffset (String dateTimeOffsetStr)
      • ToDateTimeFromDateTimeOffset
    • GUID
      • NewGuid
    • Encoding
      • ToBytes
      • FromBytes
      • UrlEncode
      • UrlDecode
    • Regular Expressions
      • ReplaceRegEx
      • ReplaceRegEx (Integer StartAt)
    • TimeSpan
      • Minutes
      • Hours
      • Days
      • Milliseconds
    • Matching
      • Soundex
      • DoubleMetaphone
      • RefinedSoundex
  • Integration Server
    • Job Scheduling
      • Scheduling and Running Jobs on a Server
  • Miscellaneous
    • Astera API User Guide
    • Astera Best Practices Dataflows
    • Astera Options
    • Astera Webinar Series
    • MySQL date/time conversion error
    • Report Model Monthly Online Training
    • Server Command Line Utility
    • Shared Actions
    • Shared Connections
    • Working with Data Formats
    • Using the Data Source Browser
    • Using the Query Editor
Powered by GitBook

© Copyright 2025, Astera Software

On this page
  • Steps
  • Usage
  1. Dataflows
  2. Data Logging and Profiling

Using Record Level Log

PreviousUsing Data Quality RulesNextDatabase Write Strategies

Last updated 9 months ago

Using a Record Level Log in a dataflow is similar to using the data profiler in a transfer setting. A record level log captures the status of each record transferred, such as Success, Error, Warning, or Skip, and includes snapshots of the source record and the destination record, as well as provides additional details, such as error messages.

You can have any number of record level logs on the dataflow; each record level log will collect the status of the records in the object that the record level log is connected to.

Steps

A record level log is saved to a .prof file, which by default is stored in the Temp folder of the user profile.

To display a record level log, click on the log link in Transfer Progress window. Or you can double click on a .prof file in Windows Explorer.

To add a record level log, drag the Record Level Log object from the Logs and Profiles group in the Flow toolbox and drop it on the dataflow.

An example of what a Record Level Log object might look like is shown below;

.

To configure the properties of a Record Level Log object, right-click on it and select Properties. The following properties are available:

  • Profile File: specifies the path to the profile file that stores the Record Level Log.

  • Log Level Type – select among the following types of events to log:

    • All – all records (including Success records) are logged

    • Errors – only error records are logged

    • Warnings – only warning records are logged

    • Errors and Warnings – both error and warning records are logged

    • Off – no logging.

  • Stop Logging After … Records with Errors – allows you to limit excessive logging by setting a cap on the maximum number of errors to be logged. The logging stops after the cap has been reached.

The default value is 1000 errors.

  • General Options screen: The Comments input allows you to enter comments associated with this object.

Usage

An example of a dataflow with a Record Level Log object is shown below:

An example of a what a record level log could look like is shown below:

The Record Level Detail tab in the record level log displays successes, errors, and warnings down at the record level. Using Record Level Filter, available via the dropdown, you can view a subset of records meeting your search criteria, such as All, Success, Errors, Skipped, Warnings, or Errors and Warnings.

Note: While viewing a record level log, you can optionally export source records into a Delimited File or Excel file, or you can export the profile itself into an Excel file, all by using the dropdown on the Profile toolbar.