Hi, I am a senior Data Analyst and have been working with data and its anomalies all my career . Recently in my current company which is an utilities based one , i have seen the metering team struggling hard with the data discrepency
and data analmoly as they are dealing with miliions of rows in excel.
They do all the maths calculations to find if the voltage has seen a sudden drop or if current has gone too up too much. They use all the methods and formulaes in excel to derive this.
It has taken whole of their to fix the excel to identify one KPI such as reverse current.
This propelled me to think in building this ThresholdIQ, a browser based anomaly detection tool for operational data. I have made that users can drop excel/CSV/XML/JSON. It is extended Google Sheets Integration with the input data.
Everything is processed in Client side in browser, there is no backend server involved in processing of data except for storing static assets like email address for the subscription purpose.
ThresholdIQ detection engine works based 9 ML models as below
1. Multi window Z-score - Detect the spikes and drops based on 4 rolling window in parallel( W50,W100,W200,W500)
2. EWMA - Exponentially Weighted Moving Average , that identifies the drifts that multi window misses.
3. SARIMA - Seasonal Pattern Modelling. It raises deviations that is different from expected behaviour
4. Isolation forest Multivariate outlier dtection - Very useful methods that combines measures that are deviated.
5. Correlation deviation - It detects metrics that normaly go togther , but a sudden spike or drop together.
6. DBSCAN - Density based clustering- It raises to any points that doesnt fall in normal cluster.
7. Seasonal decomposition - Timeline analysis . It compares data against history for an hour.
8. Trend dtection - It identifies any failure that are so gradually moving towards a failure
9. Stuck/Zero Detection - It catches complete drop , for example in if voltage or current drops to 0.
i tested this on a 1,500-row AMI demo dataset — 60 raw metric alerts grouped down to 7 named incidents, with the voltage sag at 186V and the meter fault was correctly identified.
Technical stack
Cloudflare Pages + Workers + KV. Detection runs in a Web Worker for datasets above 1K rows so the UI stays responsive. The ML methods (EWMA, SARIMA, Isolation Forest, DBSCAN, correlation scoring) are loaded as ES modules and bundled into a blob URL at runtime to satisfy CSP without needing unsafe-eval.
The incident classification uses a priority-ordered rule chain — domain-specific named rules first, then generic statistical fallbacks (zero-from-nonzero, inverse correlation, systemic event).
The severity scoring combines the worst single-metric z-score with a breadth bonus for how many metrics deviated simultaneously — so a 41V voltage drop alone scores Emergency, and 3 metrics deviating together also scores Emergency even if each one individually would only be Critical
Honest limitations
Seasonal baseline is based on time-of-day grouping, not the STL decomposition - daily load data may still produce false positives especially on W50 window
We dont store data or data doesnt stay more then that session . If you refresh the page, data resets ( It is intentional for the Privacy Policy)
We dont have the live data connections for now. once this succeeds , the roadmap is to built this a data anomaly tool, that can be installed with all data connections.
Try it: thresholdiq.app (free, no account)
We have demo dataset to run and check how it works.
Looking for a honest feedback.
I built ThresholdIQ after getting paged at 3am because of badly tuned alert thresholds. The classic problem: set them too low → alert fatigue, set them too high → miss critical issues.
The tool lets you test thresholds on historical data before deploying:
- Upload CSV/Excel or paste from clipboard (even works with PDFs!)
- Set warning/critical/emergency thresholds
- See exactly which alerts would have fired
- Visualize on a timeline
- Export reports
Key features:
100% client-side - your data never leaves your browser
Handles millions of rows using Web Workers
Visualize ALL metrics - set rules only for the ones you want to monitor
Interactive tutorial built in
Works offline after first load
NEW: Paste Data feature - copy tables from anywhere (PDFs, Excel, web pages) and paste directly. No file upload needed.
COMING SOON: Google Sheets integration
Tech stack: Vanilla JS, Web Workers, Chart.js. No frameworks, no server.
Thanks! Client-side is the only way that made sense for this - I wouldn't upload production metrics to a random SaaS either.
Great question on Web Workers - yes, I'm using them! The simulation engine has three modes:
1. *Web Worker* (default for large files): Runs threshold evaluation off-thread. UI stays completely responsive. This is what kicks in for your scenario with millions of rows.
2. *Chunked processing* (fallback): Processes data in batches with setTimeout between chunks if Workers aren't available. Slower but still keeps UI alive.
3. *Synchronous* (small files only): Direct processing for datasets where the overhead of spawning a worker isn't worth it.
The threshold logic itself is pretty straightforward - for each data point, check value against warn/crit/emrg thresholds based on the operator (>, <, >=, etc.). The Worker handles iterating through potentially millions of rows and building the alert timeline.
Progress updates come back from the Worker every ~1000 rows so users see the progress bar move in real-time.