Resuming and Rerunning Workflows in Astera
Last updated
Last updated
A workflow enables the automated running of a sequence of tasks, such as starting a program, sending an email, uploading a file, running a dataflow, workflow or SQL code, and more. The tasks run according to some predefined logic and follow the workflow path that controls how different paths in the workflow should be activated at run-time. You can add any number of tasks to the visual workflow diagram, and connect them as needed to control what should happen at each step depending on whether the task is completed successfully or returns an error. You can even create nested levels of workflows or dataflows to run as part of your master workflow.
Note: A workflow that you create in Astera can run on local or remote servers. The workflow can also be ported to run in multiple target environments. This is achieved by using run-time Parameters that change their values depending on the current context in which the workflow runs.
Astera allows you to resume a workflow from the step where it failed (with some limitations specific to nested workflows). If a particular task in the workflow fails, the subsequent tasks will not run unless that task is linked via the Run Always or On Error link to the task that failed. Resuming a workflow in this scenario saves the need to rerun the entire workflow in case of an error at some intermediate step.
If the workflow is terminated due to an error, you have the option to resume it to make sure the failed step has a chance to complete successfully.
To resume a failed workflow, open the Server menu and click Monitor.
You will be able to see all the jobs that are running in this Monitor. Right-click the failed workflow that you want to resume.
You will get a pop-up message for confirmation. Click Yes.
This will resume your job.
Now, let’s examine how this feature works in case there are several levels of nested flows.
Let’s say you have a master workflow that has a File System Action task, a Decision task and two other workflow tasks, RunWorkflow1 and RunWorkflow2. The File System Action task copies a file from one location to another, the Decision task decides whether the previous task ran successfully, and if it did, it triggers RunWorkflow1, which is followed by RunWorkflow2.
RunWorkflow1 calls two dataflows – RunDataflow1 and RunDataflow2.
Dataflow1 transfers records from an Excel Source File to a Delimited Destination File.
Dataflow2 transfers records from a Database Table Source to a Database Table Destination.
Finally, RunWorkflow2 runs an SQL Script.
Let’s say RunDataflow2, which is a part of RunWorkflow1, failed due to an error. This error will trickle up to the RunWorkflow1 task because RunDataflow2 is a part of that workflow, and then it will trickle all the way up to the master workflow, which will then show the Error status.
You can see that in the trace output below. Also, notice that RunWorkflow2 never started because RunWorkflow1 had an error.
Let’s assume we corrected the original issue that caused the error in RunDataflow2, and now we Resume the master workflow:
In the screenshot of the Job Progress window below, you can see that the two tasks, FileSystemAction1 and Decision1, were skipped from the master workflow because those tasks already ran successfully in the previous run.
Then, the child flow RunWorkflow1 will start from the beginning and will run both RunDataflow1 (which was completed successfully during the previous run) and RunDataflow2, which previously failed. Notice that the workflow will not skip the RunDataflow1 task during the Resume operation in this scenario.
In this scenario, the RunDataflow1 task has thrown an error in the RunWorkflow1 task. If you resume the master workflow, it will skip FileSystemAction1 and Decision1 tasks and then start RunWorkflow1 from beginning to end, followed by RunWorkflow2.
In this example, the child workflow RunWorkflow1 had no errors in it. However, the Run SQL Script task in RunWorkflow2 terminated due to a verification error. For example, the query should have been ‘Select * from Orders’, but it was written as follows:
When you run the master workflow, it will fail at the RunWorkflow2 step due to the issue with the query above. You can see the error in RunWorkflow2 in the Job Progress window below. All the tasks preceding this one ran successfully.
If you Resume the master workflow, it will skip FileSystemAction1, Decision1 and RunWorkflow1 tasks because they all completed successfully in the previous run. It will then start the RunWorkflow2 task. This task should be completed successfully now, assuming that we have corrected the above syntax error with the query.
Astera allows you to rerun any flow after the flow is finished running. Whether it is a failed flow or one that ended successfully, you can rerun it by right-clicking it in the Monitor grid and selecting the Rerun command. The Rerun command will run the entire workflow from the beginning if you rerun the master workflow. If you rerun an inner-level workflow, it will rerun that level only, and will not proceed to rerunning the remaining steps in its parent flow, if any.
To rerun any flow, open Monitor from the Server menu, right-click on the flow that you want to rerun and select Rerun selected job.
You will get a pop-up message for confirmation. Click Yes.
You can see in the Job Progress window that it has run the entire master workflow in this scenario. All the tasks that reran are highlighted in the trace below.
In addition to the workflow rerun features discussed above, you can also schedule a job to automatically rerun after an error in Astera, eliminating the need to rerun it manually in case of an error. You can do so by setting the Schedule frequency as Continuous and enabling the Rerun After Abnormal Termination feature.
To schedule a job, open the Job Schedules from the Server menu.
Specify the Name, provide the File Path, point to the Server, and set the Frequency, Scope, and Continuous Settings. You can see in the screenshot below that we have set the frequency to Continuous and checked Rerun After Abnormal Job Termination.
When this option is checked, the flow will keep on running repeatedly even if the previous run ends in error. You can also specify the hours between which you want the flow to run, and you can add maximum and minimum seconds to wait before a job is scheduled to rerun.
Note: For temporal schedules, such as Hourly, Daily, Monthly, etc, the next instance of the scheduled job will always start on schedule, regardless of whether the previous run was completed successfully or failed.
Once you save this schedule, the job starts running. You can see in the Monitor that it is rerunning continuously despite errors.
