1
---
2
name: data-lineage-agent
3
description: A skill for creating an agent to analyze data lineage and linkage across database scripts and stored procedures.
4
---
5
6
# Data Lineage Agent Skill
7
8
## Purpose
9
This skill assists in creating an agent that can analyze and report on the data lineage and linkage within a database system. It is ideal for understanding how changes to tables can affect the overall system and helps in uncovering the dependencies across different platforms.
10
11
## Steps to Create the Agent
12
1. **Access the Repository:**
13
- Link to the GitHub repository: [GitHub Repo](https://github.com/optuminsight-payer/COB-PARS_DB_SCRIPTS)
14
- Clone the repository to access all database scripts and stored procedures.
15
16
2. **Analyze Data Lineage:**
17
- Use tools to parse SQL scripts to identify table relationships and dependencies.
18
- Map out the data flow from source tables to final tables.
19
20
3. **Identify Changes Impact:**
21
- Implement logic to trace changes in intermediate tables to see which final tables are affected.
22
- Use graph databases or lineage analysis tools for better visualization and impact assessment.
23
24
4. **Host the Agent:**
25
- Choose a hosting platform (e.g., AWS, Azure) to deploy the agent for continuous analysis and reporting.
26
27
## Use Cases
28
- **Impact Analysis:** Determine the impact of changes in any table across the system.
29
- **Data Flow Mapping:** Visualize how data moves through the system from source to final tables.
30
- **Dependency Reporting:** Generate reports on table dependencies and affected platforms.
31
32
## Additional Features
33
- **Automated Alerts:** Notify users when potential impacts are detected.
34
- **Version Control Integration:** Link changes to specific commits in the repository for traceability.
35
36
## Example Variables
37
- `${repositoryUrl}`: The URL of the GitHub repository.
38
- `${platforms}`: List of platforms involved in the data flow.
39
40
This skill provides a structured approach to building an agent capable of comprehensive data lineage analysis, which can be crucial for database management and optimization tasks.