No articles match
Flexible Safe Anytime-Valid Tests for Practical Scenarios11 days ago
Installation | 1. Designing savi t-test experiments | Type I error and power | Case (I): Designing experiments where the minimal clinically relevant effect size and the targeted power are known. Target: The sample size to plan for | Case (II): Minimal clinically relevant effect size and a budget constrained number of samples known. Target: Power | Case (III): A desirable power and a budget constrained number of samples known. Target: Detectable effect size. | 2. Inference with savi tests: Optional stopping | Savi tests conserve the type I error rate: Batch analysis | Savi tests allow for early stopping without inflating the type I error rate above the tolerable $\alpha$-level | Optional stopping is problematic for p-values | Optional stopping does not cause savi tests to over-reject the null | Savi tests detect the effect early if it is present: deltaTrue equal to deltaMin | Savi tests detect the effect even earlier if deltaTrue is larger than deltaMin | 3. Optional Continuation | Optional continuation is problematic for p-values and leads to overinflating the type I error rate | i. Optional continuation by extending the experiment does not result in savi tests exceeding the tolerable $\alpha$-level | When the effect is present optional continuation results in savi tests correctly rejecting the null | 4. Optional contination and meta-analysis: Extending the research endeavours through replication studies | Type I error is guaranteed when multiplying e-values | Meta-analysis e-values under the alternative | Conclusion
Installing the safestats package 1 months ago
1. Installing the development version of the safestats package | a. Install using the remotes package | b. Install using a downloaded tar.gz file | 2. First example: Designing a savi z-test experiment | A first example
I. Testing: P-values and e-variables1 months ago
The two-sample z-test | 1. Sequential use of p-values over-inflates the type I error | Sequential analysis | One sample path | The behaviour of the sequential p-value procedure under multiple/repeated use | 2. Sequential use of a savi test/e-variable will not over-inflate the type I error | The behaviour of the e-variable procedure under multiple/repeated use | Concluding remarks
I. Testing: P-values, subjective Bayes factor, and e-variables1 months ago
The two-sample z-test | 1. Sequential use of p-values over-inflates the type I error | Sequential analysis | One sample path | The behaviour of the sequential p-value procedure under multiple/repeated use | 2. Sequential use of Bayes factors based on subjectively chosen priors can also over-inflate the type I error | The behaviour of the subjective Bayes factor procedure under multiple/repeated use | 3. Sequential use of a savi test/e-variable will not over-inflate the type I error | The behaviour of the e-variable procedure under multiple/repeated use | Concluding remarks
II. Design of experiments: Behaviour of e-variables under alternatives1 months ago
1. Designing savi z-test experiments | One sample path | Early stopping: True effect size equals the minimal clinical relevant mean effect size | Even earlier stopping: True effect size larger than the minimal clinical relevant mean effect size | Optional continuation: True effect size smaller than the minimal clinical relevant mean effect size | Meta-analysis e-values: Combining the evidence across studies by multiplication | Concluding remarks
II. Interval estimation: Classical confidence intervals and anytime-valid confidence sequences1 months ago
Interval estimation for a normal mean | 1. Classical 95% confidence intervals fail to cover the true mean with 95% chance across time | One sample path | The behaviour of classical confidence intervals across time under multiple use | 2. The 95% anytime-valid confidence sequence/intervals will always cover the true mean with at least 95% chance | The behaviour of anytime-valid confidence sequences across time under multiple use | Sensitivity wrt to the parameter gMom: Width of the confidence intervals versus anytime-valid confidence intervals
III. Design of experiments: Behaviour of e-variables under alternatives1 months ago
1. Designing safe z-test experiments | One sample path | Early stopping: True effect size equals the minimal clinical relevant mean effect size | Even earlier stopping: True effect size larger than the minimal clinical relevant mean effect size | Optional continuation: True effect size smaller than the minimal clinical relevant mean effect size | Meta-analysis e-values: Combining the evidence across studies by multiplication | Concluding remarks
III. Interval estimation: Classical confidence intervals, (Bayesian) credible interval, and anytime-valid confidence sequences1 months ago
Interval estimation for a normal mean | 1. Classical 95% confidence intervals fail to cover the true mean with 95% chance across time | One sample path | The behaviour of classical confidence intervals across time under multiple use | 2. Bayesian 95% credible intervals fail to cover the true mean with 95% chance across time | The behaviour of (Bayesian) credible intervals across time under multiple use | 3. The 95% anytime-valid confidence sequence/intervals will always cover the true mean with at least 95% chance | The behaviour of anytime-valid confidence sequences across time under multiple use | Sensitivity wrt to the parameter g and gMom: Credible intervals versus anytime-valid confidence intervals
Savi Tests and Confidence Intervals for Tests of Two Proportions: Practical Scenarios1 months ago
Introduction | 1. Designing Savi Experiments | Case (I): Designing experiments with the minimal clinically relevant effect size known | Case (II): Minimal clinically relevant effect size unknown, but maximum number of samples known. | Short examples of non-standard design settings | Unequal group sizes | Non-standard Beta priors | Restricting the alternative hypothesis | 2. Inference with Savi Tests and Optional Stopping | True effect size equals minimal clinically relevant effect size | True effect size larger than the minimal clinically relevant effect size | Data under the null: True effect size is zero, thus, much smaller than the minimal clinically relevant effect size | Classical test "Fisher's exact test" under the null with optional stopping | 3. Optional Continuation for tests of two proportions | 4. Confidence intervals | Confidence sequence for difference between success rates | Confidence sequence for log of the odds ratio | References