User Guide¶
This guide is written for practitioners who need calibrated anomaly decisions without working through the full conformal-inference literature first.
Start with the task you have, then read only the theory needed to understand the assumptions behind that task.
Getting Started¶
If you are new to conformal prediction, start here:
| Page | Description |
|---|---|
| Statistical Concepts | Plain-language reference for p-values, FDR, exchangeability, power, and shift |
| Conformal Inference | Practitioner-level explanation of what the guarantees mean and when they apply |
Core Concepts¶
| Page | Description |
|---|---|
| Conformalization Strategies | Split, Cross-Validation, Jackknife+, and Bootstrap strategies explained |
| Choosing Strategies | Decision framework: which strategy to use for your dataset and requirements |
| Detector Compatibility | How to use PyOD, scikit-learn, or your own custom detectors |
Advanced Topics¶
| Page | Description |
|---|---|
| FDR Control | Control false discovery rates when testing many observations |
| Weighted Conformal | Handle distribution shift between training and test data |
Practical Usage¶
| Page | Description |
|---|---|
| Input Validation | Parameter constraints and what error messages mean |
| Batch Evaluation | Evaluate performance on labeled test sets |
| Streaming Evaluation | Online evaluation for real-time detection |
| Exchangeability Martingales | Sequential evidence monitoring on streaming conformal p-values |
| Best Practices | Production patterns, data preparation, and model selection |
| Logging | Configure progress bars and debug output |
| Troubleshooting | Solutions to common issues |
Recommended Reading Order¶
- New to conformal prediction? Start with Statistical Concepts, then Conformal Inference
- Choosing a strategy? Read Choosing Strategies
- Going to production? Review Best Practices and Troubleshooting
- Dealing with distribution shift? Study Weighted Conformal
Validity Checklist¶
Before relying on a statistical guarantee, confirm:
- The calibration data is representative of normal data for the target workflow.
- The detector score direction is configured correctly.
- Multiple anomaly decisions go through
select(...)or another documented FDR procedure. - Distribution shift is handled explicitly instead of ignored.
- Any reported empirical FDR or power uses labels that were not used for training, calibration, or threshold selection.