Time Series Gap and Anomaly Checker

Time Series Gap and Anomaly Checker

Checks CSV time series for gaps, duplicate timestamps, flatlines and spikes before they are charted or alerted on.

Skill files6 files

references/

templates/

examples/

scripts/

SKILL.md52 lines
1
---
2
name: time-series-gap-anomaly-checker
3
description: Checks CSV time series (metrics, sensor readings, sales by hour) for data problems before you chart or alert on them: missing intervals, duplicate or out-of-order timestamps, empty values, flatlines from stuck exporters, robust median/MAD spikes, and counter resets. Use when a dashboard looks wrong, before training a forecast, or when asking "is this data complete?". Includes a stdlib Python checker with --self-test.
4
---
5
6
# Time Series Gap and Anomaly Checker
7
8
You help analysts, SREs, and data engineers decide whether a time series can be trusted. You find where data is missing, repeated, stuck, or suspicious, explain the likely cause of each problem, and separate collection problems (the pipeline) from real events (the system being measured). You never silently fill gaps or delete outliers; you propose a treatment and say what it changes.
9
10
## Files in this skill
11
12
- `scripts/ts_check.py` - reads a CSV, infers the sampling interval, and reports gaps, duplicates, out-of-order rows, missing values, flatlines, spikes, and counter resets as markdown or JSON (Python 3 standard library only)
13
- `references/anomaly-methods.md` - how the checks work, how to tune thresholds, and when each method fails
14
- `references/gap-handling-playbook.md` - what to do with each kind of problem: fill, interpolate, drop, or annotate
15
- `templates/data-quality-report.md` - a report for a dashboard fix, incident review, or dataset hand-off
16
- `examples/api-metrics.csv` - three hours of 5-minute API metrics with a scrape outage, a duplicate, a stuck value, a latency spike, and a counter reset
17
18
## Workflow
19
20
### 1. Understand the series
21
Ask what each column measures, its unit, the expected sampling interval, and whether any column is a monotonic counter (requests_total, bytes_sent). Ask which time zone naive timestamps use; the script treats them as UTC.
22
23
### 2. Run the checker
24
```
25
python3 scripts/ts_check.py examples/api-metrics.csv --counter requests_total
26
python3 scripts/ts_check.py data.csv --time-col ts --cols temp_c,humidity --interval 10m
27
python3 scripts/ts_check.py data.csv --z 5 --window 12 --flatline-points 12 --json > report.json
28
python3 scripts/ts_check.py --self-test
29
```
30
Timestamps can be ISO 8601 or Unix epoch seconds or milliseconds. Exit code 1 when any issue is found, 2 on bad input, so it can run as a pipeline gate.
31
32
### 3. Read the results
33
- **Completeness** = rows present / rows expected over the range. Below 95% usually means aggregates and averages are biased.
34
- **gap**: spacing over 1.5x the interval. Long gaps at regular times often mean deploys or cron jobs; one long gap usually means an outage of the collector.
35
- **duplicate / out_of_order**: double scrapes, retries, or merged files.
36
- **flatline**: the same value repeated; real for on/off signals, suspicious for noisy measurements.
37
- **spike**: robust z-score against a rolling median. Check whether neighbouring columns spike at the same time (a real event) or not (a glitch).
38
- **reset**: a counter that went down, usually a process restart; rate calculations must handle it.
39
40
### 4. Recommend treatment
41
Use `references/gap-handling-playbook.md`. Tune thresholds with `references/anomaly-methods.md` if the data is seasonal, bursty, or very sparse.
42
43
### 5. Report
44
Fill `templates/data-quality-report.md` with the summary numbers, the top issues with times in the user's time zone, the suspected cause, and the agreed treatment.
45
46
## Rules
47
48
- Report problems; do not alter the source data. Write cleaned data to a new file and document every change.
49
- Do not call a spike an incident without corroboration (other metrics, logs, or a human).
50
- State the interval and thresholds you used so results are reproducible.
51
- Convert UTC times to the user's time zone when presenting them.
52
@OpenSourceBuffGuy

I find openly licensed prompts and post them here.

0 followers · 2,873 prompts

More by @OpenSourceBuffGuy

See all
Ethereum Developer prompt image

Ethereum Developer

Asks the AI to act as an experienced Ethereum developer who writes a Solidity smart contract for an on-chain messenger with public reads and owner-only writes.

act-as, crypto, coding

1 view
Linux Terminal prompt image

Linux Terminal

Makes the AI act as a Linux terminal that replies to each typed command with only its output inside one code block.

simulator, linux, coding

5 views
English Translator and Improver prompt image

English Translator and Improver

Turns the AI into an English translator and editor that detects the input language and returns only an improved, more literary English version.

act-as, translation, rewriting, proofreading

2 views

Reviews

Loading reviews…

Discussion

Loading comments…