Flexible Safe Anytime-Valid Tests for Practical Scenarios11 days ago
Installation | 1. Designing savi t-test experiments | Type I error and power | Case (I): Designing experiments where the minimal clinically relevant effect size and the targeted power are known. Target: The sample size to plan for | Case (II): Minimal clinically relevant effect size and a budget constrained number of samples known. Target: Power | Case (III): A desirable power and a budget constrained number of samples known. Target: Detectable effect size. | 2. Inference with savi tests: Optional stopping | Savi tests conserve the type I error rate: Batch analysis | Savi tests allow for early stopping without inflating the type I error rate above the tolerable $\alpha$-level | Optional stopping is problematic for p-values | Optional stopping does not cause savi tests to over-reject the null | Savi tests detect the effect early if it is present: deltaTrue equal to deltaMin | Savi tests detect the effect even earlier if deltaTrue is larger than deltaMin | 3. Optional Continuation | Optional continuation is problematic for p-values and leads to overinflating the type I error rate | i. Optional continuation by extending the experiment does not result in savi tests exceeding the tolerable $\alpha$-level | When the effect is present optional continuation results in savi tests correctly rejecting the null | 4. Optional contination and meta-analysis: Extending the research endeavours through replication studies | Type I error is guaranteed when multiplying e-values | Meta-analysis e-values under the alternative | Conclusion