> For the complete documentation index, see [llms.txt](https://documentation.astera.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.astera.com/astera-data-stack-v10/dataflows/testing-and-diagnostics/correlation-analysis.md). # Correlation Analysis The *Correlation Analysis* object enables users to compute a two-way association between several numeric data fields. The strength of association is measured by computing correlation coefficients. In Astera, users have the option to compute three different types of correlation coefficients: 1. *Pearson Correlation*: A measure for investigating the strength of association between two quantitative continuous variables, for example, age and height. 2. *Spearman’s Rank Correlation*: A measure for investigating the strength of association between two ordinal scale rank variables with mutually exclusive ranks. For example, two physicians ranking a patient’s condition from worst to best. 3. *Kendall’s Tau Correlation*: A non-parametric measure of the relationship between columns of ranked data. For example, two interviewers ranked ten candidates as per their performance in the interview. The coefficient’s value varies between -1 to 1, with ±1 indicating a perfect or 100% association, while a coefficient value of 0 suggests zero association between the numeric data fields. The higher the magnitude of the coefficient, the stronger the correlation between two variables. The mathematical sign of coefficient value determines the slope inclination; positive (+) sign indicates an upward slope whereas negative (-) sign indicates a downward slope. Therefore, a positive coefficient indicates that the two variables are directly related, whereas a negative coefficient indicates that the two variables are inversely proportional. In this document, we will learn to perform *Correlation Analysis* on a set of numeric data in Astera. ### Sample Use-Case In this case, we are using a [*Delimited File Source*](/astera-data-stack-v10/dataflows/sources/delimited-file-source.md) object to extract the source data. You can download this sample data file from [here](https://asterasoftware.sharepoint.com/:x:/s/ProductionWebsite/EYKyEZ8dLlBGuumnZKrqSC4Bk2guOZ_sJaugxMnJpoSAsQ?download=1). ![](/files/C7sRlUyMaTmj8RhLQ41J) The source file contains information on the demographics of individuals who bought the insurance policy as well as those who chose otherwise. You can preview this data by right-clicking on source object’s header and selecting *Preview Output*. ![](/files/05aUsxON5RpLuMqFsYRT) A *Data Preview* window will open, previewing the insurance data. ![](/files/6OSpzMmWQFfAd2IEYJV5) Now, we want to figure out the strength of association between these numeric data fields to assess which fields are highly correlated and which fields have low correlation. For this purpose, we will use the *Correlation Analysis* object in Astera. #### Using Correlation Analysis 1\. To get a *Correlation Analysis* object from the Toolbox, go to *Toolbox > Testing and Diagnostics > Correlation Analysis* and drag-and-drop the testing object onto the dataflow designer. ![](/files/Rz3biu2lBG1sLCEpWnlM) 2\. The testing object contains two sub-nodes, *Input* and *Output*. Auto-map the source fields by dragging-and-dropping the root node of the source object onto the *Input* node of the *Correlation Analysis* object. ![](/files/VCAJPmEZelAbaCc6yB3B) 3\. Right-click on the object’s header and select *Properties* from the context menu. ![](/files/fgRSWPm8LvNzqng4ruU3) 4\. A configuration window will open, as shown below. This is the *Layout Builder*, where users can delete fields, change the *Name* or *Data Type* and modify fields with *Calculation*. Click *Next*. ![](/files/wWnfB39t4M2ToN6EJpjF) 5\. This is the *Properties* window that contains configuration options. ![](/files/3Xrla9KJxt0E6iN7o46j) * *Method Type*: Specifies the type of correlation coefficient to compute, depending on the content of the data and its type. The options available are *Pearson*, *Spearman* or *Kendall*. ![](/files/y11DVifMlM2W4JWR17Vw) * *Treatment of Null*: Specifies the handling of missing data. By default, it is set to *Assumes No Missing Value*, as missing data produces an error. There are two more options: ![](/files/FzrJ3DC29o5tPMtxZmNb) * *Listwise Deletion*: Removes only the missing cell from the data. * *Pairwise Deletion*: Removes the entire row of data if any one fields contains an empty cell. In this example, the source data is neither ranked nor on an ordinal scale. Hence, we will select *Pearson’s Correlation Coefficient* and leave *Treatment of Null* option as default. Click *OK*. 6\. Right-click on the *Output* node of *Correlation Analysis* object and go to *Write to > Excel Workbook Destination*. ![](/files/L6cCkYpOeHyLYaozLokQ) 7\. Configure the *Excel Workbook Destination* object to save the correlation output. To perform this step, refer to the article [here](/astera-data-stack-v10/dataflows/destinations/excel-workbook-destination.md). 8\. Right-click on the *Correlation Analysis* object and select *Preview Output*. In the *Data Preview* window, the preview shows the *Pearson’s Correlation Coefficients* computed for each variable with the rest of the numeric data fields. ![](/files/RsmPMaM4LtGhI0Rdktf6) By looking at the magnitude and the sign of the coefficient, we can deduce that the highest positive correlation exists between the variables *Age* and *Retire*. This means that the higher the age, the higher the chances of a person being retired. Whereas, the highest negatively correlated variables are *EducYear* and *Hisp*. Meaning, if the person is Hispanic, he is more likely to have fewer number of years of education, according to our *Correlation Analysis*. This concludes the document on using *Correlation Analysis* in Astera. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://documentation.astera.com/astera-data-stack-v10/dataflows/testing-and-diagnostics/correlation-analysis.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.