Resource Catalog
Was this helpful?
Was this helpful?
The Catalog feature in Astera Data Stack is a centralized repository where one can store artifacts and share them with users as per the application.
The artifacts within the catalog are stored in a Catalog table. The security aspect of the Resource Catalog lets the user give permission to only the people whom they wish to share the catalog with.
Open a project, then head to the project explorer.
Right-click on any artifact present within the project and select Enable for Catalog in the context menu,
This will add a new node to the artifact,
This is where information regarding the catalog for this artifact is going to be stored.
Double-click on this new node or right-click and select Open from the context menu,
This will open a new window.
Artifact Name: The name of the artifact whose catalog has been enabled.
Namespace: The path of the artifact in the project to keep track of.
Description: A description that explains the artifact and its catalog.
Enable For Catalog: This checkbox is selected if this artifact is going to be included in the catalog deployment.
Right-click on the window header and select Save and Close from the context menu,
Artifacts present in the project explorer can be deployed individually or they can be grouped and deployed if they are all present under a single folder.
For our use case, we will be deploying a shared action file and a data flow present within a single folder,
Right-Click on the folder and select Deploy Selected Folder from the context menu,
Select the option and a new window will pop up,
Click Ok and the selected folder will be deployed.
Once deployed, it can be viewed in the Server Browser under the Resource Catalog node,
Within the Server Browser, there are two views present that can be filtered based on Catalog,
View by Artifact Type: This option is going to arrange the Resource Catalog artifacts based on artifact type.
View by Namespace: Selecting this option is going to arrange the Resource Catalog Artifacts based on Namespace.
We can provide access to this catalog to any of the users.
Right-click any role under the Security node and select Edit Role Resources,
This will open a new window.
Simply select the checkboxes of whichever artifact you wish to give access to,
Additionally, the resource window also has a filter based on Catalog Artifacts,
Once the resources have been selected, right-click on the window and select Save & Close.
Once the artifacts within the catalog have been deployed, they can then be consumed in various ETL pipelines.
These deployments can also be viewed in the Data Source Browser,
Here, we have the data source browser present from where the artifacts can be easily dragged and dropped.
For our use case, we have the following workflow present,
However, we need another Run Dataflow object for our use case to make sure this flow runs accurately. We want the other Run Dataflow object to run on success.
Thus, we will drag and drop our dataflow artifact present in the Resource Catalog section of the Data Source Browser.
If we open the Run Dataflow object, we can see what the catalog path looks like,
We will then connect it to our OR object.
We can also access our catalog artifact through the Browse Cloud Files option present in all the objects,
This concludes the configuration, deployment, and consumption of the Resource Catalog in Astera Data Stack.