Updated November 16th, 2023

5 Types of Data Lineage: Understand All Ways to View Your Data

Improve data trust, cut costs, and make better decisions with Atlan's actionable lineage.
Book a Demo →

Share this article

Quick answer:

Strapped for time? Here’s a quick summary outlining what to expect from this article:

  • Data lineage can be classified based on the documentation method (descriptive, automated), technique (design, business, operational), or persona-specific use cases.
  • This article explores the various classification methods and delves into the 5 most common types of data lineage available:
    • Descriptive
    • Automated
    • Design
    • Business
    • Operational
  • Looking for an automated data lineage solution? Make sure to check out Atlan. Book a demo or take a guided product tour.

Types of Data Lineage #

There are five most common types of data lineage available. They are

  1. Descriptive data lineage
  2. Automated data lineage
  3. Design lineage
  4. Business lineage
  5. Operational lineage

Data lineage has different types that are classified based on how it is generated, who the intended user of the lineage is, and how the generated data lineage is documented.

Different types of data lineage exist because there are multiple questions you may want to ask about data and multiple stakeholders who can benefit from data lineage visibility.

For example, a lineage view that solves for compliance may not essentially be the same one that solves for root-cause analysis or quality. So, how do we start thinking about the types of data lineage? Start with understanding how they’re classified.

3 ways in which data lineage is typically classified #


  1. The way lineage is documented
  2. The techniques used to derive the lineage
  3. The requirements of the stakeholders who work with the lineage

Here we discuss multiple types of data lineage – e.g., descriptive, automated, design, operational, etc. and the importance of each.

Classification of data lineage and associated types #

  1. Lineage based on the method of documentation
    • Descriptive data lineage
    • Automated data lineage
  2. Lineage based on the choice of technique
    • Design lineage
    • Business lineage
    • Operational lineage
  3. Lineage based on persona-specific use cases
    • Business data lineage
    • Technical and design data lineage
  4. Data provenance

To understand these different types of data lineage, consider the example of a report that tracks the performance of a marketing campaign. The report is updated each week and records data about ad spending and user engagement, which the business, in turn, uses to measure marketing ROI.

Throughout its lifetime, this data source may undergo a number of transformations. Not only is the report updated on a weekly basis, but information about the most recent sales is also appended periodically; and after the campaign ends, the report may be exported into a data warehouse for longer-term storage. These changes form the basis for tracking the data’s lineage.

Depending on how and why you decide to trace that lineage, you may end up with a different type of lineage.

Data lineage based on the method of documentation #

As noted in this resource by lineage expert Irina Steenbeek, we can have two types of data lineage from the perspective of how the lineage is documented: Descriptive data lineage and automated data lineage.

1. Descriptive data lineage #


A descriptive data lineage is one that is generated manually. In the context of the marketing report described above, a descriptive data lineage could be a Word document or text file that records information about how the report was updated over time and how its contents were later exported to a data warehouse.

2. Automated data lineage #


Alternatively, you could create an automated data lineage based on the report. This type of data lineage would be generated by data lineage tools that automatically trace the report’s changes and transformations throughout its lifecycle, then make that information available including data change details, along with visualizations to help stakeholders understand how the data changed.

Lineages based on choice of technique #

Mandy Chessell notes that you can also classify types of data lineage based on the techniques used to generate them. The three main types include design lineage, business lineage, and operational lineage.

1. Design lineage #


Design lineage focuses on identifying the data sources and flows that result in a given data state. For the marketing report, a design lineage would record details about which data sources form the report, how new data was appended to the report each week, and how the report contents are moved between different reporting systems.

2. Business lineage #


A business lineage describes the origins and evolution of data in terms of business information. Instead of showing every component of each data flow, it filters and focuses on those that have direct business relevance – such as the source of data about ad spending, user engagement, and conversions.

While this is similar to design lineage in some respects, the main difference is that business lineage focuses on helping make business-focused decisions instead of design decisions about how to acquire and process information.

Screenshot illustrating business lineage

Business lineage helps in data discovery and verifying the integrity of data. Source: Atlan

3. Operational lineage #


Operational lineage describes data movement and transformation based on which technical operations take place.

Technical lineage helps track data at deeper levels of granularity: Systems (database, applications, services), APIs, transformations, SQL queries, and table columns. Technical lineage helps with root cause analysis, debugging pipeline issues, guiding testing, and refactoring.

Screenshot showing operational lineage

Operational data lineage helps debug issues, guide testing, and refactoring pipelines. Source: Atlan

None of these types of data lineage is better or worse than the others. Instead, think of them as serving different purposes and offering different types of information.

Lineages based on persona-specific use cases #

We have referred to Irina Steebeek’s article on data lineage above. The same article also talks about a third way of classifying data lineage - which is to think in terms of who uses them.

This approach is similar to categorizing data lineage types based on the lineage generation technique because different techniques align with different use cases.

In general, there are two main types of personas – and, by extension, two types of data lineage – to consider here:

1. Business data lineage #


If the data consumers are non-technical business users whose main goal is to understand how data impacts the business, you’ll typically produce a business lineage.

As noted above, business data lineage avoids technical details and focuses on enabling easy data discovery, verifying the freshness and integrity of the data, following data flow into BI dashboards, and tracking changes to data and its downstream impact. These are instances of information that matter to business stakeholders, as opposed to technical teams.

2. Technical and design data lineage #


In contrast, technical stakeholders, such as IT engineers and data scientists, are typically more interested in the technical and operational details of the data lineage. Technical lineage helps identify where the data originated from (systems, processes, datasets APIs), and where it’s used (BI/reporting, ML datasets).

This helps data architects to build better pipeline designs, understand dependencies, optimize ETL jobs, and ensure compliance with regulatory requirements related to data processing.

Because most businesses include both business-centric and technical stakeholders, you’ll usually need to produce both types of data lineage tailored to multiple types of personas.

Business lineage vs. technical lineage #

The main difference between technical lineage and business lineage is that business lineage focuses on aspects of data origins and processing that affect business priorities, such as which business unit produced, consumed, or updated data. In contrast, technical lineage trace data lifecycles based on technical operations, like ETL logs, root cause analysis, impact analysis, and pipeline workflows.

What is data provenance? #

It’s difficult to discuss types of data lineage without also thinking about data provenance.

Data provenance is information about the original source of data, such as who created the data, when it was created, and why it was created.

Data provenance details may be included as metadata that accompanies a file, database, or other data sources — the source of data, data types, size of data, version ids, and transformation steps.

Data lineage vs. data provenance #


Data provenance identifies the origins of data. In contrast, data lineage records the complete journey undertaken by data to arrive at its present form. Data provenance is therefore one component of data lineage. But it’s not the only component.

For example, for the marketing report described above, the data lineage would include full details about where the report originated as well as how the data in it was expanded over time and later exported to a database.

But the report’s data provenance would detail only the report’s original creation. It would lack information about data appendages or the export operations that moved the data to a data warehouse.

FAQ on Types of Data Lineage #

What are the different types of data lineage? #


There are five most common types of data lineage available. They are 1. Descriptive data lineage, 2. Automated data lineage, 3. Design lineage, 4. Business lineage, 5. Operational lineage.

Conclusion #

Different types of data lineages serve different purposes, and they are generated in different ways. In many cases, businesses will need to leverage a variety of data lineage types and generation techniques to get the most value out of their data assets.

If you are evaluating a data lineage solution for your data stack, take Atlan for a spin. Atlan makes data lineage effortless, it provides an all-new way to experience data lineage — that’s smooth, intuitive, and interactive.

Share this article

[Website env: production]