Using Report Models to Extract Business Information from Printed Documents
Last updated
Last updated
© Copyright 2023, Astera Software
n this tutorial, we will explore Astera’s Report Model features.
To mine a report, you need to create a report model containing the definition of the report’s structure, and then use your report source object in a dataflow just as you would with any other hierarchical source object.
Let’s demonstrate how this can be accomplished.
We will start by creating a report model.
Note: A report model normally has several regions and fields belonging to those regions. An example of a region is the header, footer, Data region, and any additional ‘append’ regions. An example of a field within a region is CompanyName, AccountNo, Quantity etc. A region may have child regions located within that region. A field can only belong to one region at a time, and fields cannot overlap.
To create a new report layout, go to File > New > Report Model.
Select a sample report file in the Open dialog box. We will use this sample report to create our report model. Using a sample of an actual report will allow to visualize our report showing regions and fields making up the report as well as their actual values from the sample.
Note: Astera supports reading flat text reports, PRN reports and PDF reports.
In the screenshot above, we selected a sample report file for Orders. The selected sample is loaded to the Report Definition Editor:
Let’s take a look at this report. At the top of our sample is general Order information, such as Company Name, Order Date and time, Customer Name, Account Number and others. Following it is the detailed Order info, such as order items making up the order.
Our sample report has two logical regions, the Header, the Data region. Unlike some other common reports, this report has no Footer.
The header is at the very top of the report, spanning three lines starting at the line with the order date.
So, the first step in creating our report model will be to define the header for our report.
In the Report Definition Editor, select the top three lines. This is the area that covers the Header. Right-click on your selection and using the context menu select one of the following options, shown in the context menu below:
Since we are creating the Header, select Add Page Header Region.
The Report Browser on the left-hand side of Astera now shows a new node Header.
Any time this pattern occurs inside the report, Astera will treat it as the starting point of the Header.
Notice that the Report Definition Editor now highlights the header in purple. The header spans 3 lines, as shown by the purple block in the editor. The height of the header or any other region, (i.e. the number of lines that the header spans) is controlled by the Line Count input below the Report toolbar.
The next step is to create fields making up the header.
There are two ways to create fields.
Highlight a field, right-click and select Add Field.
Right-click within the Header area, and select Auto Create Fields.
Astera will scan our sample report and identify any changing values within any occurrences of the Header. These changing values will be marked as fields.
In our example, the Auto Create Fields feature added five fields. They are now displayed in the Report Browser under the Header node. Notice that our new fields are also highlighted in darker purple in the Report Definition Editor.
The fields created this way are assigned unique names, such as FIELD_0, FIELD_1 and so on.
You can rename a field if needed. Let’s rename our newly created fields to make them more descriptive.
Select a field in the Report Browser, double click and enter the new name.
Or:
Select a field in the Report Browser, right-click it and select Rename.
Or:
Select a field in Report Definition Editor (the selected field is highlighted in yellow), right-click and select Rename from the context menu.
Note: The selected field is always highlighted in yellow in the report definition editor.
We can also change the field’s data type if needed. In our example, Astera was able to correctly assign fields data types from our sample report:
Now that we created the definition of the Header, let’s look into the main region of the report. As we saw earlier, the main region starts with the Customer Name and then includes Account Number, Contact Name, and finally specific order details.
Let’s select the main region in the report definition editor, then right-click it and select Add Data Region from the context menu.
This will add a new node Data in the Report Browser. This new node has no fields at this point.
Notice that Astera assigned the default vertical size of this region as 23 lines based on our selection. We can adjust this number as needed by using the Line Count input under the toolbar.
Now we will identify the starting point of the region. Place the cursor at the position where the text ‘CUSTOMER:’ begins as shown in the image below and enter CUSTOMER: in the pattern text input.
Report Definition Editor highlights any occurrences of the Data region in report. Remember that we can easily adjust the height of the region by using the Line Count input.
Let’s now rename our region CustomerData. Now our report has two regions: Header, and CustomerData.
Now, let’s identify the fields making up CustomerData region.
You can either manually assign fields, or you can use the Auto Create Fields feature.
To manually add a field, highlight a field with the mouse cursor, right-click it and select Add Field. A new field is added to the Report Browser. The Report Definition Editor shows all the occurrences of this field in the report.
Note: To automatically add fields, right-click within the header area, and select Auto Create Fields. You can then modify, rename, add or delete fields as necessary.
Next, let’s take a closer look at CustomerData. Notice that each Customer can have one or more orders, and each order may have several items in it. In Astera terms, we say that the region has a collection of items, or to put it simply, is a Collection. Also note that order data in located within the CustomerData region we defined above. In other words, CustomerData region is also a container for order details.
Select CustomerData node in ReportBrowser. Right-click it and select Add Collection Data Region. This will add a new region under the CustomerData node. The default name here is Data, which we will rename OrderData to make it more descriptive.
Now, let’s now define the starting point of our new region.
Type ORDER NUMBER: in the text pattern input.
The report definition editor highlights all instances of OrderData region.
Right-click anywhere within our region, and select Auto Create Fields. This creates Order Number field and Ship Date field, named Field_0 and Field_1 respectively. Let’s give these fields more user-friendly names.
As we saw earlier, a Customer can have more than one order (which in Astera is called a Collection of items). Whenever a node has a collection of items, we need to turn on its Is Collection property as shown below. Notice that the appearance of the icon for ORDER node in the report browser changes to help identify this node as collection. Note that when we add a Collection Data Region via the context menu, the Is Collection property is enabled automatically.
Now, let’s create the definition of Order Items. Select CustomerData node in the Report Browser. Add a new Collection Data Region in the report definition editor in the same manner we did earlier.
Next we will rename our new region OrderDetails, and auto create fields using the protocol we demonstrated earlier.
This action adds 6 fields, such as media type, quantity, description, label/no, unit price and amount. The fields are named by default Field_0, Field_1, so we will need to rename them as desired.
Our sample report does not have the footer. If necessary, a footer can be added in the same way we added the Header and Data regions above.
We have now created the report definition.
Note: Report definitions are used by Astera to correctly parse, interpret and assign data as it is fed from the report source. Report definitions are assigned *.rmd extension.
Let’s save our report model, by clicking on the Save icon on the main toolbar. Now we can preview our data to see how it is parsed by Astera .
Let’s now take a look at some additional functionality that Astera offers to help you customize your report.
To select a field, left-click on it in the Report Browser’s tree. The field is highlighted in yellow in the Report Definition Editor. Some of the more common field properties are displayed in the top pane of the editor:
To select a region, click on it in the Report Browser’s tree. The region is highlighted in light purple in the Report Definition Editor, and the fields in the selected region are also highlighted in darker purple. The top pane shows the properties that are applicable for the region.
To view and update all other properties of a field or a region, right-click on a field (or region) inside Report Browser, and select Edit Field (or Edit Region) from the context menu.
You can also access field properties by right-clicking the field in the Report Definition Editor and selecting Field Properties… from the context menu.
To rename a field, double click it on the tree in the Report Browser and enter a new name.
To rename a region, double click it on the tree in the Report Browser and enter a new name.
You can also rename a field or a region by entering the new name in the Name input on the top pane.
To delete a field, right-click it in Report Browser or Report Definition Editor and select Delete Field.
To delete a region, right-click on a region (or a field inside the region), and select Delete Region from the context menu. Note that this action will also delete any fields in that region.
After your field has been created, you can change its start position by moving it a number of characters to the left or to the right. Right-click on a field and select Move Field Marker Right One Character or “Move Field Marker Left One Character from the context menu. Repeat as needed to move the field the desired number of characters. Note that the same functionality is also accessible from the top toolbar via the icons accordingly.
You can also change the field length, by selecting Decrease Field Length by one character and Increase Field Length by one character from the context menu. Repeat as many times as needed to change the field length by the desired number of characters.
Note that the same functionality is also accessible from the top toolbar via the icons accordingly.
To auto determine field length based on the available sample data, right-click a field and select Auto determine field length from the context menu. Or click the icon on the top toolbar.
Alternatively, you can also move all fields within the same region left or right by a specified number of characters. To do it, right-click on a region or field, and select Move All field markers left one character or Move All field markers right one character. You can also use the icons on the top toolbar.
Note: To undo any action in the editor, use the Undo dropdown menu on the toolbar or press Ctrl + Z.
The following options are available to help you create a text pattern that will identify the starting point of a field, or a region:
The following options are available:
Sample File Path – provides the path to the sample report file that you want to use for creating your report model.
Line Count – controls how many lines are loaded from the sample file.
Other useful options are Tab Size (default value of 8), Font and Numeric Format.
Now that we created and previewed our report, let’s add it to a dataflow so we can read the entire source report and feed it to a destination object.
Go to File > New > Dataflow. This creates a new dataflow.
Using the Toolbox pane, expand the Sources category and select Report Source.
Drag and drop Report Source onto the Designer.
Double click ReportModel1 object that we just added (or right-click it and select Properties) to open the Properties dialog.
Using the Properties dialog, enter the path to the report source file and the report model. The report model location should point to the report model we created and saved earlier.
Click OK to close the dialog. ReportModel1 object shows the report structure according to the report model we created:
Note: You may need to expand the tree nodes to see all the child nodes under the root node.
Our new report source is ready to feed data to the downstream objects in our dataflow.
Note: You can also load a different sample file in the report definition editor at a later time. Click the icon on the toolbar and navigate to the file you want to load.
Now, let’s take a closer look at the header. The header in our sample report always starts with a date, shown at the very first line and in the very first character position of the header. We can use the date as an identifying pattern for the header. Any time the pattern occurs in the report file, Astera will treat it as the beginning of the Header.
Let’s enter the wildcard characters denoting digits as shown below:
Specify the text pattern that will identify our order items. In our example, we will use part of the Quantity data followed by a space character to identify a line with the order item. To that end, enter "Match any digit” and then “Match any blank character” as shown below:
Click the icon on the top toolbar. This opens the Data Preview window showing the entire report structure with the actual values for all the fields we have defined above.
The same functionality is also available on the top toolbar, by pressing the icon.
Match any alphabet
Match any digit
Match any alphabet or digit
Match any non-blank character
Match any blank character
For example, to match a date 12/15/2011, you can use the pattern is “match any digit”.
To change report options, click on the icon on the report toolbar.
Let’s now preview our sample report based on the report model we have created. To preview it, click on the icon on the Report toolbar. Our report displays in the Data Preview pane as shown in the screenshot below:
