Alpha Spending Methods¶
Alpha spending functions provide flexible control of Type I error rate in sequential hypothesis testing, allowing the number and timing of interim analyses to be determined adaptively during the trial.
Overview¶
The alpha spending approach, developed by Lan and DeMets (1983), overcomes key limitations of traditional group sequential methods by not requiring the total number of analyses or their exact timing to be specified in advance. This flexibility is particularly valuable in clinical trials where interim analyses may be needed at unplanned times.
Available Methods¶
Alpha Spending Function¶
online_fdr.spending.alpha_spending.AlphaSpending
¶
Bases: AbstractSequentialTest
Alpha Spending Function for sequential hypothesis testing with flexible interim analyses.
The alpha spending function approach, developed by Lan and DeMets (1983), provides flexible group sequential boundaries that control Type I error rate while allowing the number and timing of interim analyses to be determined adaptively during the trial.
This method overcomes key limitations of traditional group sequential methods by not requiring the total number of analyses or their exact timing to be specified in advance. It's particularly valuable in clinical trials where interim analyses may be needed at unplanned times for ethical or scientific reasons.
The approach works by "spending" portions of the overall alpha budget at each interim analysis according to a pre-specified spending function, ensuring that the cumulative Type I error rate never exceeds the target level.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
alpha | float | Target Type I error rate (e.g., 0.05 for 5% error rate). Must be in (0, 1). | required |
spend_func | AbstractSpendFunc | Alpha spending function that determines how to allocate alpha across interim analyses. Must inherit from AbstractSpendFunc. | required |
Attributes:
Name | Type | Description |
---|---|---|
alpha0 | float | Original target Type I error rate. |
rule | AbstractSpendFunc | The spending function used to determine alpha allocation. |
num_test | int | Number of tests/analyses conducted so far. |
alpha | float | Current alpha level for the next test. |
Examples:
>>> from online_fdr.spending.functions.bonferroni import BonferroniSpendFunc
>>> # Create Bonferroni spending function for 5 planned analyses
>>> spend_func = BonferroniSpendFunc(max_analyses=5)
>>> alpha_spending = AlphaSpending(alpha=0.05, spend_func=spend_func)
>>> # Test p-values sequentially
>>> decision1 = alpha_spending.test_one(0.01) # First interim analysis
>>> decision2 = alpha_spending.test_one(0.03) # Second interim analysis
References
Lan, K. K. Gordon, and D. L. DeMets (1983). "Discrete Sequential Boundaries for Clinical Trials." Biometrika, 70(3):659-663.
DeMets, D. L., and K. K. Gordon Lan (1994). "Interim Analysis: The Alpha Spending Function Approach." Statistics in Medicine, 13(13-14):1341-1352.
Jennison, C., and B. W. Turnbull (1999). "Group Sequential Methods with Applications to Clinical Trials." Chapman and Hall/CRC.
Source code in online_fdr/spending/alpha_spending.py
Functions¶
test_one(p_val)
¶
Test a single p-value using the alpha spending approach.
Determines the alpha level for the current analysis based on the spending function and the analysis number, then tests whether the p-value meets the significance threshold.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
p_val | float | P-value to test. Must be in [0, 1]. | required |
Returns:
Type | Description |
---|---|
bool | True if the null hypothesis is rejected (p_val < alpha), False otherwise. |
Raises:
Type | Description |
---|---|
ValueError | If p_val is not in [0, 1]. |
Examples:
>>> alpha_spending = AlphaSpending(alpha=0.05, spend_func=my_spend_func)
>>> alpha_spending.test_one(0.01) # First analysis
True
>>> alpha_spending.test_one(0.04) # Second analysis, stricter threshold
False
Source code in online_fdr/spending/alpha_spending.py
Online Fallback Procedure¶
online_fdr.spending.online_fallback.OnlineFallback
¶
Bases: AbstractSequentialTest
Online Fallback procedure for controlling the familywise error rate (FWER) online.
The online fallback procedure, developed by Tian and Ramdas (2021), provides strong FWER control for testing an a priori unbounded sequence of hypotheses one by one over time without knowing the future. It ensures that with high probability there are no false discoveries in the entire sequence.
This procedure is uniformly more powerful than traditional Alpha-spending methods and strongly controls the FWER even under arbitrary dependence among p-values. It uses a gamma sequence to allocate alpha budget over time and includes a "fallback" mechanism that increases future testing power after making discoveries.
The method unifies algorithm design concepts from offline FWER control and online false discovery rate control, providing a powerful adaptive approach for sequential hypothesis testing scenarios.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
alpha | float | Target familywise error rate (e.g., 0.05 for 5% FWER). Must be in (0, 1). | required |
Attributes:
Name | Type | Description |
---|---|---|
alpha0 | float | Original target FWER level. |
last_rejected | bool | Whether the previous hypothesis was rejected. |
seq | Gamma sequence for alpha allocation over time. | |
num_test | int | Number of hypotheses tested so far. |
alpha | float | Current alpha level for the next test. |
Examples:
>>> # Create online fallback instance
>>> fallback = OnlineFallback(alpha=0.05)
>>> # Test p-values sequentially
>>> decision1 = fallback.test_one(0.01) # First test
>>> decision2 = fallback.test_one(0.03) # Higher power if previous rejected
>>> print(f"Decisions: {decision1}, {decision2}")
>>> # Sequential testing with FWER guarantee
>>> p_values = [0.001, 0.8, 0.02, 0.9, 0.005]
>>> decisions = [fallback.test_one(p) for p in p_values]
>>> print(f"FWER-controlled decisions: {decisions}")
References
Tian, J., and A. Ramdas (2021). "Online control of the familywise error rate." Statistical Methods in Medical Research, 30(4):976-993.
ArXiv preprint: https://arxiv.org/abs/1910.04900
Source code in online_fdr/spending/online_fallback.py
Functions¶
test_one(p_val)
¶
Test a single p-value using the online fallback procedure.
The online fallback procedure adjusts the alpha level based on whether the previous test was rejected (fallback mechanism) and allocates additional alpha using a gamma sequence. This creates higher power for future tests when discoveries are made.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
p_val | float | P-value to test. Must be in [0, 1]. | required |
Returns:
Type | Description |
---|---|
bool | True if the null hypothesis is rejected (discovery), False otherwise. |
Raises:
Type | Description |
---|---|
ValueError | If p_val is not in [0, 1]. |
Examples:
>>> fallback = OnlineFallback(alpha=0.05)
>>> fallback.test_one(0.01) # First test, uses base alpha
True
>>> fallback.test_one(0.03) # Second test, higher power after discovery
True
>>> fallback.test_one(0.04) # Third test, even higher power
False
Note
The alpha level increases when the previous test was rejected, implementing the "fallback" mechanism that provides additional testing power.
Source code in online_fdr/spending/online_fallback.py
Spending Function Types¶
The alpha spending framework supports various spending functions that determine how to allocate the alpha budget across interim analyses:
- O'Brien-Fleming type: Conservative early spending, allowing larger effects to be detected later
- Pocock type: Equal spending at each analysis
- Linear spending: Proportional to analysis timing
- Custom functions: User-defined spending patterns
Key Concepts¶
Alpha Budget Allocation¶
The fundamental principle is to "spend" portions of the overall alpha budget at each interim analysis according to a pre-specified spending function, ensuring cumulative Type I error never exceeds the target level.
Flexibility vs. Control¶
Alpha spending provides: - Flexibility: Adapt timing and number of analyses during the trial - Control: Maintain strict Type I error rate control - Efficiency: Stop early for efficacy or futility
Comparison with Other Methods¶
Method | Flexibility | Power | Complexity |
---|---|---|---|
Alpha Spending | High | Good | Medium |
Group Sequential | Low | High | Low |
Online FDR | High | Variable | High |
Usage Guidelines¶
- Choose appropriate spending function based on trial characteristics
- Monitor spending carefully to avoid early alpha exhaustion
- Plan for unscheduled analyses that may be required
- Consider futility bounds in addition to efficacy boundaries
References¶
Lan, K. K. Gordon, and D. L. DeMets (1983). "Discrete Sequential Boundaries for Clinical Trials." Biometrika, 70(3):659-663.
DeMets, D. L., and K. K. Gordon Lan (1994). "Interim Analysis: The Alpha Spending Function Approach." Statistics in Medicine, 13(13-14):1341-1352.
Jennison, C., and B. W. Turnbull (1999). Group Sequential Methods with Applications to Clinical Trials. Chapman and Hall/CRC.