Root cause analysis with multiple personasSteven We will start with a common situation wherein a dashboard is broken. The data that normally comes in for the days since prior order graph is skewing to the right, seeming like it is adding a full month to the number of days. In this situation, we will begin from the perspective of a data analyst. They are the owners of the dashboard and have just been notified of the break that a data consumer noticed. Logging into Atlan we go to the dashboard that we were notified on the break of.
That notification may come in through email, slack, or in this case through a ticket in Jira, which has been linked to the asset through the Jira integration. As the analyst, I can click on the ticket and review the issue, along with clicking on the Preview icon for Tableau to actually validate the issue, all from Atlan. In this case we do find that there is an issue with the dashboard, and that for some reason the days since prior order graph is skewed by a large amount to the right. Since we are already in Atlan on the dashboard, we simply click on the lineage tab.
Here, we see a graphical representation of the lineage of our dashboard and can start working upstream of our asset. The initial graph expands both one level upstream and one level downstream of the asset that we start with, so in this case we can immediately see the sources of our dashboard.
We know that the days since prior order is a field that is fed from our data warehouse presentation layer. Immediately we see our data source, and by clicking on the view fields drop down, we can look to find our source field. There is the days since prior order column. Clicking on it will show the lineage visually. Next, we will expand further upstream to find where this data source field is coming from. We click the plus button to expand back and find the days since prior order comes from another data source, and again click the plus to find the Snowflake table that we are pulling from.
Since we are able to view sample data on our Snowflake assets inside of Atlan, we can click on the table and actually validate the underlying data immediately. Clicking on the sample data tab will return the first 100 rows of data for this table. Once returned, we can see the values on the days since prior order and find that the values are also skewed here. As a data analyst, we have hit the end of our investigation into the root cause of this issue, as we do not own the data warehouse assets and transformations.
However, we do want to make sure that there aren't other dashboards that are impacted by this issue. In this case, we will go back to the lineage tab, click on the column and then click on the view impact report icon to see all downstream assets. The impact report allows us to not only see where this column is being used downstream but also, we can immediately see the owner of those assets, if other than ourselves, and notify those owners of the current issue. We could also get this information as a CSV immediately by clicking on the download icon.
Additionally, for each of the impacted dashboards, we can go create an announcement so that our end data consumers know of the issue. By simply clicking on the Food and Beverage Order Analysis dashboard from the impact analysis, we are taken to the dashboard. Here, we will create a issue announcement that the days since prior order is currently broken. Our final step is to get the data warehouse team involved in the root cause analysis. By going back to the Snowflake table and simply checking the ownership in Atlan, we can clearly see that Duane, a data engineer, is the owner of this data set.
Here, we can either create a slack message directly from the asset notifying Duane of the issue, create a new issue in Jira or update our existing ticket to include this asset, or email Duane.
Once notified, our part as the data analyst in this root cause analysis is completed. We will start this next part of the R C A as the data engineer, Duane. We have received notification of the issue on the Snowflake table from our data analyst and it is now our job to find the root cause of this issue. We go directly into the table in Atlan and go right to the lineage tab. The graph opens one level upstream and downstream of our table. We click show columns and click on days since prior order.
We go to the table that is producing our days since prior order column. Here, we click into the table and open up it's asset overview. In the same way the data analyst did, we will click on the sample data to validate the values for days since prior order inside of this table. After validating we find that the data in this table appears to be correct, without the skewing that is occurring in the downstream table. We now know that the source of our issue is in the transformation from this table. Since Atlan actually shows the seequill logic for transformations in our Snowflake connection, we can click on the process asset between this asset and the one downstream that is causing the issue.
We select the Snowflake process and on the right hand side we can clearly see the sequel. We expand and immediately see the issue in the sequel. In this case we are incorrectly transforming our field and need to make changes. We need to make changes to the DBT model, so we go ahead and create a pull request in Git Hub for this. Luckily, we also have the Atlan Git Hub integration setup in Git Hub and we can immediately see the downstream impact of any changes that we are making. This allows us to make sure that there are no unintended consequences of making changes to this table right from Git Hub.
It also gives us the metadata we need to contact owners of downstream assets if they are unaware of the fix needed. Finally, we make the changes, check in the new code, and once it runs we validate the results. In looking back through this process, from both a data analyst and data engineer role, we were not only able identify the source of the issue inside of Atlan, but also validate there was an issue, get ownership and notify users, and also even get critical metadata information from Atlan in our source tools. With this, we see how Atlan makes it easier to tackle root cause analysis.
