1
---
2
name: csv-data-quality-profiler
3
description: Profiles CSV and spreadsheet exports before they are trusted for analysis, imports, or dashboards - infers column types, measures missing values, duplicates, mixed types, outliers, whitespace and encoding problems, and key candidates - then writes a prioritized data quality report with concrete fixes. Use when a user shares a CSV, asks "is this data clean?", prepares a data import or migration, or sees numbers that look wrong in a report.
4
---
5
6
# CSV Data Quality Profiler
7
8
You check a tabular dataset the way a careful analyst would before building anything on top of it. You measure first, then explain what matters for the user's goal, then suggest the smallest safe fixes.
9
10
## Files in this skill
11
12
- `scripts/profile_csv.py` - column profiler and issue finder (Python 3 standard library only)
13
- `references/quality-dimensions.md` - the six dimensions you score and what counts as a problem
14
- `references/fix-playbook.md` - safe fixes per issue type, and what never to do automatically
15
- `templates/quality-report.md` - report format
16
- `examples/example-orders-report.md` - a worked report on a small orders export
17
18
## Workflow
19
20
### 1. Understand the purpose
21
Ask (or infer) what the data is for: a one-off analysis, a recurring import, a dashboard, or a migration. Ask which column should be unique (the key) and which columns matter most. The same issue can be critical for an import and harmless for a rough analysis.
22
23
### 2. Profile
24
```bash
25
python3 scripts/profile_csv.py data.csv
26
python3 scripts/profile_csv.py data.csv --key order_id
27
python3 scripts/profile_csv.py data.csv --delimiter ";" --json > profile.json
28
```
29
The script reports per column: inferred type, missing count and percent, distinct count, top values, min and max, and issues (mixed types, leading or trailing spaces, outliers by the IQR rule, inconsistent date formats, inconsistent casing). It also reports duplicate rows, ragged rows, and whether the key is unique. Exit code is 1 when any HIGH issue is found.
30
31
If the user cannot run scripts, read the first 200 rows yourself and apply the same checks by hand, and say the result is a sample.
32
33
### 3. Interpret
34
For each finding, use `references/quality-dimensions.md` to decide:
35
1. Which dimension it affects (completeness, validity, uniqueness, consistency, accuracy signals, structure).
36
2. Severity for this purpose: HIGH (wrong results or failed import), MEDIUM (misleading in some views), LOW (cosmetic).
37
3. Whether it is a real problem or expected (for example, an optional "coupon_code" column is allowed to be mostly empty).
38
39
### 4. Recommend fixes
40
Use `references/fix-playbook.md`. Prefer fixes at the source system over cleaning downstream. Give each fix as a concrete step (a formula, a pandas or SQL snippet, or a source-system change) and say what it changes and how many rows.
41
42
### 5. Write the report
43
Fill `templates/quality-report.md` the way `examples/example-orders-report.md` does: verdict first, then the top issues, then the column table.
44
45
## Verdicts
46
- **READY** - no HIGH issues for the stated purpose.
47
- **READY WITH CAVEATS** - usable if the listed caveats are accepted.
48
- **NOT READY** - at least one HIGH issue that would produce wrong numbers or a failed import.
49
50
## Rules
51
- Never silently drop, impute, or deduplicate rows; always state the rule and the affected row count, and keep the original file.
52
- Do not guess what a code or abbreviation means; ask or mark it as an assumption.
53
- Treat personal data with care: show at most a few example values, and mask emails, phone numbers, and IDs in the report.
54
- Outliers are leads, not errors. Ask before removing them.
55