ALERTS
Notify downstream consumers of impacts
Make consumers aware of potential problems in advance.🔄 Tasks
Do the following whenever a data store changes.
- Refresh metadata: Crawl the data store to retrieve its updated metadata.
- Detect changes: Compare the new metadata against the previous metadata. Identify any changes that could cause an impact — the addition or removal of columns, for example.
- Find dependencies: Use lineage to find who uses the data store. These could include analyzing transformation processes, other data stores, BI dashboards, and so on.
- Notify consumers: Notify each consumer through their preferred communication channel — Slack, Jira, etc.
This process could also be incorporated as part of the testing phases of changing a data store. For example, the CI/CD process that changes the data store itself could trigger these steps. Orchestration can then notify affected consumers before production systems change.
🎉 Outcome
Consumers of data are aware of potential problems in advance.