| Title: | Safe Anytime-Valid Inference |
|---|---|
| Description: | Functions to design and apply tests that are anytime valid. The functions can be used to design hypothesis tests in the prospective/randomised control trial setting or in the observational/retrospective setting. The resulting tests remain valid under both optional stopping and optional continuation. The current version includes safe t-tests and safe tests of two proportions. For details on the theory of safe tests, see Ly, A, Boehm, Grunwald, Ramdas and van Ravenzwaaij (2024). "Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values" (<doi:10.31234/osf.io/h5vae>) and Grunwald, de Heide and Koolen (2024) "Safe Testing" (<doi:10.1093/jrsssb/qkae011>), for details on safe logrank tests see ter Schure, Perez-Ortiz, Ly and Grunwald (2024) "The Anytime-Valid Logrank Test: Error Control under Continuous Monitoring with Unlimited Horizon" (<doi:10.51387/24-NEJSDS65>), and Turner, Ly and Grunwald (2024) "Generic E-variables for exact sequential k-sample tests that allow for optional stopping" (<doi:10.1016/j.jspi.2023.106116>) for details on safe contingency table tests. |
| Authors: | Alexander Ly [cre, aut], Rosanne Turner [aut], Yongqi Wang [aut], Muriel Felipe Perez-Ortiz [ctb], Udo Boehm [ctb], Judith ter Schure [ctb], Peter Grunwald [ctb] |
| Maintainer: | Alexander Ly <[email protected]> |
| License: | LGPL (>= 3) |
| Version: | 0.8.9 |
| Built: | 2026-10-05 01:42:20 UTC |
| Source: | https://github.com/alexanderlynl/safestats |
Keys are of the form firstAuthorYearFirstWord
addCite(..., breakLine = TRUE)addCite(..., breakLine = TRUE)
... |
further arguments to be passed to or from methods. |
breakLine |
logical, default TRUE to break up the references |
a character string
addCite(grunwald2024safe)addCite(grunwald2024safe)
A ceiling function with a tolerance at the 13th digit
ceil(x, digits = 13)ceil(x, digits = 13)
x |
numeric, that needs |
digits |
integer, position of the digit to round of to |
integer
ceiling(27/21*21) ceil(27/21*21)ceiling(27/21*21) ceil(27/21*21)
Checks consistency between the sided of the hypothesis and the minimal clinically relevant effect size or savi test defining parameter. Throws an error if the one-sided hypothesis is incongruent with the
checkAndReturnEsMinParameterSide( paramToCheck, alternative = c("twoSided", "greater", "less"), esMinName = c("noName", "meanDiffMin", "meanDiffTrue", "phiS", "deltaMin", "deltaS", "hrMin", "thetaS", "deltaTrue", "g", "kappaG", "propDiffMin", "logOddsMin"), paramDomain = NULL )checkAndReturnEsMinParameterSide( paramToCheck, alternative = c("twoSided", "greater", "less"), esMinName = c("noName", "meanDiffMin", "meanDiffTrue", "phiS", "deltaMin", "deltaS", "hrMin", "thetaS", "deltaTrue", "g", "kappaG", "propDiffMin", "logOddsMin"), paramDomain = NULL )
paramToCheck |
numeric. Either a named savi test defining parameter such as phiS, or thetaS, or a minimal clinically relevant effect size called with a non-null esMinName name |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
esMinName |
provides the name of the effect size. Either "meanDiffMin" for the z-test, "deltaMin" for the t-test, "hrMin" for the logrank test, or "propDiffMin" and "logOddsMin" for the 2x2 test (both A minus B) |
paramDomain |
Domain of the paramToCheck, typically, positiveNumbers. Default |
paramToCheck after checking, perhaps with a change in sign
Check consistency between nPlan and the testType for one and two-sample z and t-tests
checkAndReturnNPlan( nPlan, ratio = 1, testType = c("oneSample", "paired", "twoSample") )checkAndReturnNPlan( nPlan, ratio = 1, testType = c("oneSample", "paired", "twoSample") )
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
testType |
either one of "oneSample", "paired", "twoSample". |
nPlan a vector of sample sizes of length 1 or 2
Checks consistency between the sided of the hypothesis and the minimal clinically relevant effect size or safe test defining parameter. Throws an error if the one-sided hypothesis is incongruent with the
checkAndReturnsEsMinParameterSide( paramToCheck, alternative = c("twoSided", "greater", "less"), esMinName = c("noName", "meanDiffMin", "phiS", "deltaMin", "deltaS", "hrMin", "thetaS", "deltaTrue"), paramDomain = NULL )checkAndReturnsEsMinParameterSide( paramToCheck, alternative = c("twoSided", "greater", "less"), esMinName = c("noName", "meanDiffMin", "phiS", "deltaMin", "deltaS", "hrMin", "thetaS", "deltaTrue"), paramDomain = NULL )
paramToCheck |
numeric. Either a named safe test defining parameter such as phiS, or thetaS, or a minimal clinically relevant effect size called with a non-null esMinName name |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
esMinName |
provides the name of the effect size. Either "meanDiffMin" for the z-test, "deltaMin" for the t-test, or "hrMin" for the logrank test |
paramDomain |
Domain of the paramToCheck, typically, positiveNumbers. Default |
paramToCheck after checking, perhaps with a change in sign
Helper function to check whether arguments are specified in a function at a higher level and already provided in the design object.
checkDoubleArgumentsDesignObject(designObj, ...)checkDoubleArgumentsDesignObject(designObj, ...)
designObj |
an object of class "safeDesign". |
... |
arguments that need checking. |
Returns nothing only used for its side-effects to produces warnings if needed.
designObj <- designSaviZ(0.4) checkDoubleArgumentsDesignObject(designObj, "alpha"=NULL, alternative=NULL) # Throws a warning checkDoubleArgumentsDesignObject(designObj, "alpha"=0.4, alternative="d")designObj <- designSaviZ(0.4) checkDoubleArgumentsDesignObject(designObj, "alpha"=NULL, alternative=NULL) # Throws a warning checkDoubleArgumentsDesignObject(designObj, "alpha"=0.4, alternative="d")
Repeated calls of genhypergeo,
which provides further details.
compute1F1AllVersions(U, L, z, tol = 0, maxiter = 2000)compute1F1AllVersions(U, L, z, tol = 0, maxiter = 2000)
U |
Upper arguments respectively (real or complex) |
L |
Lower arguments respectively (real or complex) |
z |
Primary complex argument |
tol |
tolerance with default zero meaning to iterate until additional terms to not change the partial sum |
maxiter |
Maximum number of iterations to perform |
a vector
compute1F1AllVersions(U=-359, L=1/2, z=-0.1891234)compute1F1AllVersions(U=-359, L=1/2, z=-0.1891234)
Helper function: Computes 1-power based on meanDiffTrue and nPlan
computeBetaBatchSaviZ( meanDiffTrue, nPlan, alpha = 0.05, sigma = 1, kappa = sigma, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow") )computeBetaBatchSaviZ( meanDiffTrue, nPlan, alpha = 0.05, sigma = 1, kappa = sigma, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow") )
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
numeric that represents the type II error
computeBetaBatchSafeZ(): Deprecated version of computeBetaBatchSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Helper function to compute uncertainty regarding nPlan estimates
computeBetaBootstrapper(samplingResult, parameter, nPlan, nBoot)computeBetaBootstrapper(samplingResult, parameter, nPlan, nBoot)
samplingResult |
output from sampling functions such as computeNPlanSaviZ and computeNPlanSaviT |
parameter |
numeric > 0, the savi test defining parameter. |
nPlan |
vector of max length 2 representing the planned sample sizes. |
nBoot |
integer > 0 representing the number of bootstrap samples to estimate the uncertainty of various estimates. |
list with bootstrap objects
samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20) result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20) result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)
Helper function: Computes the power of the saviTTest based on deltaMin and nPlan
computePowerSaviT( deltaTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, deltaMin = deltaTrue, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, nuMin = 2, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, ... )computePowerSaviT( deltaTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, deltaMin = deltaTrue, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, nuMin = 2, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, ... )
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi t test under continuous monitoring,or for the computation of the logarithm of the implied target. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
wantSimData |
logical, if |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
a list which contains at least power and an adapted bootObject of class
boot().
computeBetaSafeT(): Deprecated version of computeBetaSaviT
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computePowerSaviT(deltaTrue=0.7, 27, nSim=10)computePowerSaviT(deltaTrue=0.7, 27, nSim=10)
Helper function: Computes the power based on meanDiffTrue and nPlan
computePowerSaviZ( meanDiffTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, meanDiffMin = meanDiffTrue, testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, ... )computePowerSaviZ( meanDiffTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, meanDiffMin = meanDiffTrue, testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, ... )
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi z test under continuous monitoring,or for the computation of the logarithm of the implied target. |
... |
further arguments to be passed to or from methods. |
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
wantSimData |
logical, if |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
a list which contains at least power and an adapted bootObject of class boot.
computeBetaSafeZ(): Deprecated version of computeBetaSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computePowerSaviZ(meanDiffTrue=0.7, 20, nSim=10)computePowerSaviZ(meanDiffTrue=0.7, 20, nSim=10)
Computes the bootObj for sequential sampling procedures regarding nPlan, beta, the implied target
computeBootObj( values, power = NULL, nPlan = NULL, nBoot = 1000L, alpha = NULL, beta = NULL, objType = c("nPlan", "nMean", "power", "beta", "betaFromEValues", "logImpliedTarget", "expectedStopTime") )computeBootObj( values, power = NULL, nPlan = NULL, nBoot = 1000L, alpha = NULL, beta = NULL, objType = c("nPlan", "nMean", "power", "beta", "betaFromEValues", "logImpliedTarget", "expectedStopTime") )
values |
numeric vector. If objType equals "nPlan" or "beta" then values should be stopping times, if objType equals "logImpliedTarget" then values should be eValues. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
nPlan |
numeric vector of length at most 2 representing the planned sample size(s). |
nBoot |
integer > 0 representing the number of bootstrap samples to estimate the uncertainty of various estimates. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
objType |
character string either "nPlan", "nMean", "beta", "betaFromEValues", "expectedStopTime" or "logImpliedTarget". |
bootObj
computeBootObj(1:100, objType="nPlan", beta=0.3)computeBootObj(1:100, objType="nPlan", beta=0.3)
Estimate an upper or lower bound for a savi confidence sequence on the logarithm of the odds ratio for two proportions.
computeConfidenceBoundForLogOddsTwoProportions( ya, yb, saviDesign, bound = c("lower", "upper"), deltaStart, deltaStop, precision )computeConfidenceBoundForLogOddsTwoProportions( ya, yb, saviDesign, bound = c("lower", "upper"), deltaStart, deltaStop, precision )
ya |
positive observations/ events per data block in group a: a numeric with integer values
between (and including) 0 and |
yb |
positive observations/ events per data block in group b: a numeric with integer values
between (and including) 0 and |
saviDesign |
a 'saviDesign' object obtained through
|
bound |
type of bound to calculate; "lower" to get a lower bound on positive delta, "upper" to get an upper bound on negative delta. |
deltaStart |
starting value of the grid to search over for the bound on the confidence sequence (in practice: the interval). Numeric >0 when searching for a lower bound, numeric < 0 when searching for an upper bound. |
deltaStop |
end value of the grid to search over for the bound on the confidence sequence (in practice: the interval). Numeric >0 when searching for a lower bound, numeric < 0 when searching for an upper bound. |
precision |
precision of the grid between deltaStart and deltaStop. |
numeric: the established lower- or upper bound on the logarithm of the odds ratio between the groups
balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 10, alpha = 0.05) #hypothesize OR < 1 (i.e., log OR < 0) ya <- c(1,1,1,1,1,1,1,1,0,1) yb <- c(0,0,0,0,1,0,0,0,0,0) #one-sided CI for OR-, establish upper bound on log odds ratio computeConfidenceBoundForLogOddsTwoProportions(ya = ya, yb = yb, saviDesign = balancedSaviDesign, bound = "upper", deltaStart = -0.01, deltaStop = -4, precision = 20)balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 10, alpha = 0.05) #hypothesize OR < 1 (i.e., log OR < 0) ya <- c(1,1,1,1,1,1,1,1,0,1) yb <- c(0,0,0,0,1,0,0,0,0,0) #one-sided CI for OR-, establish upper bound on log odds ratio computeConfidenceBoundForLogOddsTwoProportions(ya = ya, yb = yb, saviDesign = balancedSaviDesign, bound = "upper", deltaStart = -0.01, deltaStop = -4, precision = 20)
Estimate Lower and Upper Bounds on the Confidence Sequence (Interval) for the Difference Divergence Measure for Two Proportions
computeConfidenceBoundsForDifferenceTwoProportions( ya, yb, precision, saviDesign )computeConfidenceBoundsForDifferenceTwoProportions( ya, yb, precision, saviDesign )
ya |
positive observations/ events per data block in group a: a numeric with integer values
between (and including) 0 and |
yb |
positive observations/ events per data block in group b: a numeric with integer values
between (and including) 0 and |
precision |
precision of the grid of differences to search over for the lower and upper bounds. |
saviDesign |
a 'saviDesign' object obtained through
|
list with found lower and upper bound.
balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 10, alpha = 0.05) ya <- c(1,1,1,1,1,1,1,1,0,1) yb <- c(0,0,0,0,1,0,0,0,0,0) computeConfidenceBoundsForDifferenceTwoProportions(ya = ya, yb = yb, precision = 20, saviDesign = balancedSaviDesign)balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 10, alpha = 0.05) ya <- c(1,1,1,1,1,1,1,1,0,1) yb <- c(0,0,0,0,1,0,0,0,0,0) computeConfidenceBoundsForDifferenceTwoProportions(ya = ya, yb = yb, precision = 20, saviDesign = balancedSaviDesign)
Inverts the eGauss numerator: the interval is the set of logOdds
values at which the numerator over the conditional likelihood of
logLikelihoodFNCH() stays below 1 / alpha, found by one minimum and
two roots on domain.
computeConfidenceInterval2x2LogOdds( ya, yb, na, nb, logPTotal, alpha, domain = c(-40, 40) )computeConfidenceInterval2x2LogOdds( ya, yb, na, nb, logPTotal, alpha, domain = c(-40, 40) )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
logPTotal |
numeric, the eGauss log numerator on all blocks: the plain cumulative log e-value plus the cumulative hypergeometric log likelihood. |
alpha |
numeric in (0, 1), one minus the confidence level. |
domain |
|
numeric vector that contains the lower and upper bound of the
savi confidence interval, named lowerBound, upperBound; NA with
a warning for a bound that lies beyond domain, and the whole
domain with a warning when the set is empty.
Inverts the eBeta e-process: the predictive Beta numerator of
predictiveThetas2x2() against, per block, the reverse information
projection of solveRIPr2x2PropDiff() onto each candidate propDiff.
The interval is the set of candidates whose e-value stays below
1 / alpha, found by one minimum and two roots on domain.
computeConfidenceInterval2x2PropDiff( ya, yb, na, nb, betaParameter, alpha, domain = c(-1, 1) )computeConfidenceInterval2x2PropDiff( ya, yb, na, nb, betaParameter, alpha, domain = c(-1, 1) )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
betaParameter |
|
alpha |
numeric in (0, 1), one minus the confidence level. |
domain |
|
numeric vector that contains the lower and upper bound of the
savi confidence interval, named lowerBound, upperBound; the
domain edge when that edge is still inside, and c(-1, 1) with a
warning when the set is empty.
Helper function: Computes the savi confidence sequence for the mean in a T-test
computeConfidenceIntervalT( meanObs, sdObs, nEff, nu, parameter, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), alternative = c("twoSided", "greater", "less"), ciValue = 0.95, maxRoot = 11, nuMin = 2 )computeConfidenceIntervalT( meanObs, sdObs, nEff, nu, parameter, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), alternative = c("twoSided", "greater", "less"), ciValue = 0.95, maxRoot = 11, nuMin = 2 )
meanObs |
numeric, the observed mean. For two sample tests this is difference of the means. |
sdObs |
numeric, the observed standard deviation. For a two-sample test this is the root of the pooled variance. |
nEff |
numeric > 0, the effective sample size. For one sample tests, this is just n. |
nu |
numeric > 0, the degrees of freedom. |
parameter |
numeric > 0, the savi test defining parameter,
see |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
maxRoot |
Used to bound the candidate set of width of the confidence interval/ |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
numeric vector that contains the upper and lower bound of the savi confidence sequence
computeConfidenceIntervalT(meanObs=0.3, sdObs=2, nEff=12, nu=11, parameter=0.4)computeConfidenceIntervalT(meanObs=0.3, sdObs=2, nEff=12, nu=11, parameter=0.4)
Helper function: Computes the savi confidence sequence for a Z-test
computeConfidenceIntervalZ( nEff, meanObs, parameter, sigma = 1, ciValue = 0.95, alternative = "twoSided", a = NULL, g = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "freq", "credibleInterval"), maxRoot = 20 )computeConfidenceIntervalZ( nEff, meanObs, parameter, sigma = 1, ciValue = 0.95, alternative = "twoSided", a = NULL, g = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "freq", "credibleInterval"), maxRoot = 20 )
nEff |
numeric > 0, the effective sample size. |
meanObs |
numeric, the observed mean. For two sample tests this is difference of the means. |
parameter |
numeric > 0, the savi test defining parameter,
see |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
a |
numeric, only used for eType="credibleInterval". a specifies the centre of the Gaussian prior on population mean. |
g |
numeric > 0, only used for eType="credibleInterval". g specifies the variance of the Gaussian prior on population mean. |
eType |
character string, one of "eCauchy", "eGauss", "grow", "freq", "credibleInterval". This is somewhat a misnomer as "freq" and "credibleInterval" do not correspond to e-value tests. "eCauchy" yields an anytime-valid confidence interval based on a Cauchy mixture, whereas "eGauss" yields an anytime-valid confidence interval based on a Gaussian mixture. "grow" yields an anytime-valid confidence interval based on a mixture of point masses at the minimal clinically relevant mean difference. This confidence interval unfortunately does not shrink as the sample size tends to infinity. "freq" yields the standard unsafe frequentist confidence interval, which is not safe. "credibleInterval" yields the Bayesian credible interval based on a conjugate prior as is usual in Bayesian analysis. This interval is also not safe. |
maxRoot |
Used to bound the candidate set of width of the confidence interval, whenever eType="eCauchy" |
numeric vector that contains the upper and lower bound of the savi confidence sequence
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computeConfidenceIntervalZ(nEff=15, meanObs=0.3, parameter=0.2)computeConfidenceIntervalZ(nEff=15, meanObs=0.3, parameter=0.2)
Row i is the interval of computeConfidenceInterval2x2LogOdds() on
blocks 1..i. The eGauss numerator at block i is that of the test on
blocks 1..i, as every factor is fixed given its block's total or
predictable, so the test's numerator vector serves all rows; the root
finding is repeated per prefix.
computeConfidenceSequence2x2LogOdds( ya, yb, na, nb, logNumerator, alpha, runningIntersection )computeConfidenceSequence2x2LogOdds( ya, yb, na, nb, logNumerator, alpha, runningIntersection )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
logNumerator |
numeric vector, the eGauss log numerator on blocks
|
alpha |
numeric in (0, 1), one minus the confidence level. |
runningIntersection |
logical, if |
an nBlocks x 2 matrix that contains the lower and upper bound
of the savi confidence sequence per block; an empty set is an NA
row, and with runningIntersection the rows stay NA after the first
empty row.
Row i is the interval of computeConfidenceInterval2x2PropDiff() on
blocks 1..i, found by advancing one e-process per grid candidate and
reading off the run of candidates below 1 / alpha, each bound refined
outward by a secant.
computeConfidenceSequence2x2PropDiff( ya, yb, na, nb, betaParameter, alpha, runningIntersection, nGrid = 2000L )computeConfidenceSequence2x2PropDiff( ya, yb, na, nb, betaParameter, alpha, runningIntersection, nGrid = 2000L )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
betaParameter |
|
alpha |
numeric in (0, 1), one minus the confidence level. |
runningIntersection |
logical, if |
nGrid |
integer > 0, the number of candidates strictly inside |
an nBlocks x 2 matrix that contains the lower and upper bound
of the savi confidence sequence per block; an empty set is an NA
row, and with runningIntersection the rows are nested and stay NA
after the first empty row.
Computes the credible interval of a two-sample t-test based on conjugate priors
computeConjugateCredibleIntervalTwoSampleT( x1, sdObs1, n1, x2, sdObs2, n2, a1 = 3.98, g1 = 0.03, a2 = 4.02, g2 = 0.05, aGamma = 2, bGamma = 1/2, ciValue = 0.95 )computeConjugateCredibleIntervalTwoSampleT( x1, sdObs1, n1, x2, sdObs2, n2, a1 = 3.98, g1 = 0.03, a2 = 4.02, g2 = 0.05, aGamma = 2, bGamma = 1/2, ciValue = 0.95 )
x1 |
numeric, sample mean of group 1 |
sdObs1 |
numeric, the observed standard deviation of the first group. |
n1 |
integer sample size of group 1 |
x2 |
numeric, sample mean of group 2 |
sdObs2 |
numeric, the observed standard deviation of the second group. |
n2 |
integer sample size of group 2 |
a1 |
numeric, prior mean of the population mean mu1 of group 1 |
g1 |
numeric > 0, conditional prior variance of the population mean
mu1 of group 1 is given by |
a2 |
numeric, prior mean of the population mean mu2 of group 2 |
g2 |
numeric > 0, conditional prior variance of the population mean
mu2 of group 2 is given by |
aGamma |
numeric > 0, shape parameter of the prior on the standard deviation sigma |
bGamma |
numeric > 0, rate parameter of the prior on the standard deviation sigma |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
a vector of length two representing the credible interval
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computeConjugateCredibleIntervalTwoSampleT(1, 1, 3, 1, 1, 3)computeConjugateCredibleIntervalTwoSampleT(1, 1, 3, 1, 1, 3)
The smallest propDiffMin at which the worst-case power of
computePowerSavi2x2() at nBlocks = nBlocksPlan reaches power,
found by stats::uniroot() on propDiffBounds. Every candidate is
simulated with the same seed, so the target is a deterministic step
function of the candidate. There is no planning on logOdds.
computeEsMinSavi2x2( na, nb, nBlocksPlan, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), nSim = 1000L, seed = NULL, pb = TRUE, propDiffBounds = c(0.01, 0.9), tol = 1e-05 )computeEsMinSavi2x2( na, nb, nBlocksPlan, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), nSim = 1000L, seed = NULL, pb = TRUE, propDiffBounds = c(0.01, 0.9), tol = 1e-05 )
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
nBlocksPlan |
planned number of data blocks collected, see scenario 2 and 3 above. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi 2x2 test under continuous monitoring. |
seed |
integer, seed number. Default seed=NULL yields seed=2026. |
pb |
logical, if |
propDiffBounds |
search interval for the magnitude of |
tol |
tolerance of the root on the |
numeric that represents the minimal detectable difference in
proportions, negative for "less", or NA when not found.
Helper function: Computes the type II error under optional stopping based on the minimal clinically relevant hazard ratio and the maximum number of nEvents.
computeLogrankBetaFrom( hrMin, nEvents, alpha = 0.05, alternative = c("twoSided", "greater", "less"), m0 = 50000L, m1 = 50000L, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, groupSizePerTimeFunction = returnOne, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, ... )computeLogrankBetaFrom( hrMin, nEvents, alpha = 0.05, alternative = c("twoSided", "greater", "less"), m0 = 50000L, m1 = 50000L, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, groupSizePerTimeFunction = returnOne, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, ... )
hrMin |
numeric that defines the minimal relevant hazard ratio, the smallest hazard ratio that we want to detect. |
nEvents |
numeric > 0, targetted number of events. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis, which must be one of "twoSided" (default),"greater" or "less". The alternative is pitted against the null hypothesis of equality of the survival distributions. More specifically, let lambda1 be the hazard rate of group 1 (i.e., placebo), and lambda2 the hazard ratio of group 2 (i.e., treatment), then the null hypothesis states that the hazard ratio theta = lambda2/lambda1 = 1. If alternative = "less", the null hypothesis is compared to theta < 1, thus, lambda2 < lambda1, that is, the hazard of group 2 (i.e., treatment) is less than that of group 1 (i.e., placebo), hence, the treatment is beneficial. If alternative = "greater", then the null hypothesis is compared to theta > 1, thus, lambda2 > lambda1, hence, harm. |
m0 |
Number of subjects in the control group 0/1 at the beginning of the trial, i.e., |
m1 |
Number of subjects in the treatment group 1/2 at the beginning of the trial, i.e., |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 (Treatment) over condition 1 (Placebo),
thus, m1/m0. Note that m1 and m0 are not used to specify ratio. Ratio is only used when |
parameter |
Numeric > 0, represents the savi tests defining thetaS. Default NULL so it's decided by the algorithm, typically, this equals hrMin, which corresponds to the GROW choice. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
groupSizePerTimeFunction |
A function without parameters and integer output. This function provides the number
of events at each time step. For instance, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of events for the exact savi logrank test under continuous monitoring |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximation of power or nEvents for the exact savi logrank test under continuous monitoring |
relevanceTest |
logical, if |
... |
further arguments to be passed to or from methods. |
a list which contains at least beta and an adapted bootObject of class boot.
Muriel Felipe Perez-Ortiz and Alexander Ly
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
computeLogrankBetaFrom(hrMin=0.7, 300, nSim=10)computeLogrankBetaFrom(hrMin=0.7, 300, nSim=10)
Helper function: Computes the planned sample size based on the minimal clinical relevant hazard ratio, alpha and beta under optional stopping.
computeLogrankNEvents( hrMin, beta, m0 = 50000, m1 = 50000, alpha = 0.05, alternative = c("twoSided", "greater", "less"), nSim = 1000L, nBoot = nSim, groupSizePerTimeFunction = returnOne, nMax = 1000L, parameter = NULL, digits = getOption("digits"), pb = TRUE )computeLogrankNEvents( hrMin, beta, m0 = 50000, m1 = 50000, alpha = 0.05, alternative = c("twoSided", "greater", "less"), nSim = 1000L, nBoot = nSim, groupSizePerTimeFunction = returnOne, nMax = 1000L, parameter = NULL, digits = getOption("digits"), pb = TRUE )
hrMin |
numeric that defines the minimal relevant hazard ratio, the smallest hazard ratio that we want to detect. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
m0 |
Number of subjects in the control group 0/1 at the beginning of the trial, i.e., |
m1 |
Number of subjects in the treatment group 1/2 at the beginning of the trial, i.e., |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis, which must be one of "twoSided" (default),"greater" or "less". The alternative is pitted against the null hypothesis of equality of the survival distributions. More specifically, let lambda1 be the hazard rate of group 1 (i.e., placebo), and lambda2 the hazard ratio of group 2 (i.e., treatment), then the null hypothesis states that the hazard ratio theta = lambda2/lambda1 = 1. If alternative = "less", the null hypothesis is compared to theta < 1, thus, lambda2 < lambda1, that is, the hazard of group 2 (i.e., treatment) is less than that of group 1 (i.e., placebo), hence, the treatment is beneficial. If alternative = "greater", then the null hypothesis is compared to theta > 1, thus, lambda2 > lambda1, hence, harm. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of events for the exact savi logrank test under continuous monitoring |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximation of power or nEvents for the exact savi logrank test under continuous monitoring |
groupSizePerTimeFunction |
A function without parameters and integer output. This function provides the number
of events at each time step. For instance, if |
nMax |
An integer. Once nEvents hits nMax the experiment terminates, if it didn't stop due to threshold crossing crossing already. Default set to Inf. |
parameter |
Numeric > 0, represents the savi tests defining thetaS. Default NULL so it's decided by the algorithm, typically, this equals hrMin, which corresponds to the GROW choice. |
digits |
number of significant digits to be used. |
pb |
logical, if |
a list which contains at least nEvents and an adapted bootObject of class boot.
Muriel Felipe Perez-Ortiz and Alexander Ly
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
computeLogrankNEvents(0.7, 0.2, nSim=10)computeLogrankNEvents(0.7, 0.2, nSim=10)
This function was created to complement survdiff from the
'survival' package, which is restricted to 'Surv' objects of type "right". Most likely
survdiff is much faster
computeLogrankZ( survObj, group, computeZ = TRUE, computeExactE = FALSE, theta0 = 1, thetaS = NULL, ... )computeLogrankZ( survObj, group, computeZ = TRUE, computeExactE = FALSE, theta0 = 1, thetaS = NULL, ... )
survObj |
a Surv object that is either of type |
group |
a grouping factor with 2 levels |
computeZ |
logical. If |
computeExactE |
logical. If |
theta0 |
numeric > 0 used only for the e-value, i.e., if computeExactE is |
thetaS |
numeric > 0 used only for the e-value, i.e., if computeExactE is |
... |
further arguments to be passed to or from methods. |
Returns a list containing at least the following components:
the number of events.
the observed logrank statistic.
vector of observed minus expected.
vector of hypergeometric variances.
vector at which the events occurred.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
Schoenfeld, D. (1981). The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika, 68(1), 316-319, https://doi.org/10.2307/2335833.
data <- generateSurvData(nP = 5, nT = 5, lambdaP = 0.03943723, lambdaT = 0.5*0.03943723, endTime = 40, seed = 2006) survObj <- survival::Surv(data$time, data$status) survObj <- survival::Surv(data$time, data$status) result <- computeLogrankZ(survObj, data$group) result$z sqrt(survival::survdiff(survObj~data$group)$chisq)data <- generateSurvData(nP = 5, nT = 5, lambdaP = 0.03943723, lambdaT = 0.5*0.03943723, endTime = 40, seed = 2006) survObj <- survival::Surv(data$time, data$status) survObj <- survival::Surv(data$time, data$status) result <- computeLogrankZ(survObj, data$group) result$z sqrt(survival::survdiff(survObj~data$group)$chisq)
Computes the meanDiffMin that is detectable with probability "power" for given nPlan
computeMinEsBatchSaviZ( nPlan, alpha = 0.05, power = 0.8, sigma = 1, kappa = sigma, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), lowEsTrue = 0.01, highEsTrue = 0.002, ... )computeMinEsBatchSaviZ( nPlan, alpha = 0.05, power = 0.8, sigma = 1, kappa = sigma, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), lowEsTrue = 0.01, highEsTrue = 0.002, ... )
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
numeric > 0 that represents the minimal detectable mean difference
computeMinEsBatchSafeZ(): Deprecated version of computeMinEsBatchSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computeMinEsBatchSaviZ(27)computeMinEsBatchSaviZ(27)
Computes the smallest detectable deltaMin with power probability, for the provided sample size
computeMinEsBatchSaviT( nPlan, alpha = 0.05, power = 0.8, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), lowEsTrue = 0.01, highEsTrue = 3, ... )computeMinEsBatchSaviT( nPlan, alpha = 0.05, power = 0.8, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), lowEsTrue = 0.01, highEsTrue = 3, ... )
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
numeric > 0 that represents the minimal detectable effect size
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computeMinEsBatchSaviT(27)computeMinEsBatchSaviT(27)
Help function to compute the effective sample size based on a length 2 vector of samples
computeNEff(n, testType = c("oneSample", "paired", "twoSample"), silent = TRUE)computeNEff(n, testType = c("oneSample", "paired", "twoSample"), silent = TRUE)
n |
vector of length at most 2 representing the sample sizes of the first and second group |
testType |
either one of "oneSample", "paired", "twoSample". |
silent |
logical, if true, then turn off warnings |
a numeric that represents the effective sample size.
Helper function: Computes nPlan based on meanDiffTrue, alpha and power.
computeNPlanBatchSaviZ( meanDiffTrue, alpha = 0.05, power = 0.8, sigma = 1, kappa = sigma, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), parameter = NULL, meanDiffMin = NULL, beta = NULL, highN = 10000L, ratio = 1, ... )computeNPlanBatchSaviZ( meanDiffTrue, alpha = 0.05, power = 0.8, sigma = 1, kappa = sigma, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), parameter = NULL, meanDiffMin = NULL, beta = NULL, highN = 10000L, ratio = 1, ... )
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
highN |
integer, largest possible sampling horizon. This might be the
largest n that we are able to fund, which by default is set to 1e4L.
Typically, highN is not used, as the function
|
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
a list which contains at least nPlan and the savi test defining parameter.
computeNPlanBatchSafeZ(): Deprecated version of computeNPlanBatchSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Helper function: Computes the planned sample size for the savi T-test based deltaMin, alpha and power
computeNPlanBatchSaviT( deltaTrue, alpha = 0.05, power = 0.8, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), sigma = 1, sigma2 = 1, parameter = NULL, beta = NULL, ratio = 1, deltaMin = NULL, ... )computeNPlanBatchSaviT( deltaTrue, alpha = 0.05, power = 0.8, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), sigma = 1, sigma2 = 1, parameter = NULL, beta = NULL, ratio = 1, deltaMin = NULL, ... )
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
sigma |
numeric > 0 representing the population standard deviation used for the test. |
sigma2 |
numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test |
parameter |
numeric, an optional savi test defining parameter. Default set to |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
a list which contains at least nPlan and the savi test defining parameter
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Helper function to compute uncertainty regarding nPlan estimates
computeNPlanBootstrapper( samplingResult, parameter, power, nPlanBatch, nBoot, beta = NULL )computeNPlanBootstrapper( samplingResult, parameter, power, nPlanBatch, nBoot, beta = NULL )
samplingResult |
output from sampling functions such as computeNPlanSaviZ and computeNPlanSaviT |
parameter |
numeric > 0, the savi test defining parameter. |
power |
numeric in (0, 1) that specifies the desirable power necessary to calculate both "n" and the minimum detectable effect size. |
nPlanBatch |
integer, the sample size needed in a batch design to reach the targeted power=1-beta with tolerable type I error alpha |
nBoot |
integer > 0 representing the number of bootstrap samples to estimate the uncertainty of various estimates. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
list with bootstrap objects
samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20) result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20) result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)
Helper function: Computes nPlan based on deltaMin and power
computeNPlanSaviT( deltaTrue, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), deltaMin = NULL, beta = NULL, ratio = 1, parameter = NULL, nMax = 1e+08, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, nuMin = 2, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, sigma = 1, sigma2 = 1, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, ... )computeNPlanSaviT( deltaTrue, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), deltaMin = NULL, beta = NULL, ratio = 1, parameter = NULL, nMax = 1e+08, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, nuMin = 2, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, sigma = 1, sigma2 = 1, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, ... )
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
nMax |
integer > 0, maximum sample size of the (first) sample in each sample path. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi t test under continuous monitoring,or for the computation of the logarithm of the implied target. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
wantSimData |
logical, if |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
sigma |
numeric > 0 representing the population standard deviation used for the test. |
sigma2 |
numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
a list which contains at least nPlan and an adapted bootObject of class boot().
computeNPlanSafeT(): Deprecated version of computeNPlanSaviT
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computeNPlanSaviT(0.7, 0.2, nSim=10)computeNPlanSaviT(0.7, 0.2, nSim=10)
Helper function: Computes nPlan based on meanDiffTrue, alpha and power
computeNPlanSaviZ( meanDiffTrue, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), sigma = 1, kappa = sigma, meanDiffMin = NULL, beta = NULL, ratio = 1, parameter = NULL, nMax = 1e+08, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, highN = 10000L, ... )computeNPlanSaviZ( meanDiffTrue, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), sigma = 1, kappa = sigma, meanDiffMin = NULL, beta = NULL, ratio = 1, parameter = NULL, nMax = 1e+08, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, highN = 10000L, ... )
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
nMax |
integer > 0, maximum sample size of the (first) sample in each sample path. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi z test under continuous monitoring,or for the computation of the logarithm of the implied target. |
relevanceTest |
logical, if |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
wantSimData |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
highN |
integer, largest possible sampling horizon. This might be the
largest n that we are able to fund, which by default is set to 1e4L.
Typically, highN is not used, as the function
|
a list which contains at least nPlan and an adapted bootObject of class boot.
computeNPlanSafeZ(): Deprecated version of computeNPlanSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
computeNPlanSaviZ(0.7, 0.2, nSim=10)computeNPlanSaviZ(0.7, 0.2, nSim=10)
Runs sampleStoppingTimesSavi2x2() and reports its nPlan, the
power quantile of the stopping time at the hardest baseline, with a
bootstrap SE and the mean stopping time of paths capped at nPlan,
both at that baseline.
computeNPlanSavi2x2( propDiffMin, na, nb, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), nSim = 1000L, nBoot = nSim, nMax = 10000L, seed = NULL, wantSamplePaths = FALSE, pb = TRUE )computeNPlanSavi2x2( propDiffMin, na, nb, power = 0.8, alpha = 0.05, alternative = c("twoSided", "less", "greater"), nSim = 1000L, nBoot = nSim, nMax = 10000L, seed = NULL, wantSamplePaths = FALSE, pb = TRUE )
propDiffMin |
numeric in (-1, 1) that defines the minimal relevant
difference in proportions |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi 2x2 test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, or the number of blocks for the savi 2x2 test under continuous monitoring. |
nMax |
integer > 0, maximum number of data blocks in each sample path. |
seed |
integer, seed number. Default seed=NULL yields seed=2026. |
wantSamplePaths |
logical, if |
pb |
logical, if |
a list which contains at least nPlan (the worst-case block
count, Inf with a warning when the worst baseline crossed too
rarely) and adapted bootObjects bootObjNPlan and bootObjNMean of
class boot::boot() on the worst baseline (NULL when nPlan is
Inf); further nPlanVec (the quantile per curve), nMean,
worstCaseThetaA, worstCaseThetaB, and the sampler's thetaA,
thetaB, stoppingTimes, breakVector, eValuesStopped,
samplePaths, n1Vector.
Helper function to compute uncertainty regarding nPlan estimates
computePowerBootstrapper(samplingResult, parameter, nPlan, nBoot)computePowerBootstrapper(samplingResult, parameter, nPlan, nBoot)
samplingResult |
output from sampling functions such as computeNPlanSaviZ and computeNPlanSaviT |
parameter |
numeric > 0, the savi test defining parameter. |
nPlan |
vector of max length 2 representing the planned sample sizes. |
nBoot |
integer > 0 representing the number of bootstrap samples to estimate the uncertainty of various estimates. |
list with bootstrap objects
samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20) result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20) result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)
Runs sampleStoppingTimesSavi2x2() with nMax = nBlocks and reports,
per curve, the fraction of paths that cross 1 / alpha within
nBlocks; the worst case is the smallest, since the test must meet its
target whatever the baseline is.
computePowerSavi2x2( propDiffMin, na, nb, nBlocks, alpha = 0.05, alternative = c("twoSided", "less", "greater"), nSim = 1000L, nBoot = nSim, seed = NULL, wantSamplePaths = FALSE, pb = TRUE )computePowerSavi2x2( propDiffMin, na, nb, nBlocks, alpha = 0.05, alternative = c("twoSided", "less", "greater"), nSim = 1000L, nBoot = nSim, seed = NULL, wantSamplePaths = FALSE, pb = TRUE )
propDiffMin |
numeric in (-1, 1) that defines the minimal relevant
difference in proportions |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
nBlocks |
integer > 0, the planned number of data blocks. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi 2x2 test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, or the number of blocks for the savi 2x2 test under continuous monitoring. |
seed |
integer, seed number. Default seed=NULL yields seed=2026. |
wantSamplePaths |
logical, if |
pb |
logical, if |
a list which contains at least power (the worst case over the
curves) and an adapted bootObject bootObjPower of class
boot::boot() on that baseline; further powerVec (one per curve),
worstCaseThetaA, worstCaseThetaB, nBlocks, and the sampler's
thetaA, thetaB, stoppingTimes, breakVector, eValuesStopped,
samplePaths, n1Vector.
Computes the sufficient statistics needed to compute 'logrankSingleZ'
computeStatsForLogrank( survDataFrame, y0Index, y1Index, timeNow, timeBefore, survType = "right", ... )computeStatsForLogrank( survDataFrame, y0Index, y1Index, timeNow, timeBefore, survType = "right", ... )
survDataFrame |
a 'Surv' object converted to a matrix, then to a data.frame |
y0Index |
vector of integers corresponding to the control group |
y1Index |
vector of integers corresponding to the treatment group |
timeNow |
numeric, current time |
timeBefore |
numeric, previous time |
survType |
character, either "right" or "counting" (left truncated, right censored) |
... |
further arguments to be passed to or from methods. |
Returns a list containing at least the following components:
number of observations in the control group.
number of observations in the treatment group.
total number of participants in the control group.
total number of participants in the treatment group.
#'
data <- generateSurvData(nP = 5, nT = 5, lambdaP = 0.03943723, lambdaT = 0.5*0.03943723, endTime = 40, seed = 2006) survObj <- survival::Surv(data$time, data$status) survDataFrame <- as.data.frame(as.matrix(survObj)) y0Index <- which(data$group=="P") y1Index <- which(data$group=="T") timeNow <- 4 timeBefore <- 0 computeStatsForLogrank(survDataFrame, y0Index, y1Index, timeNow, timeBefore) timeNow <- 13 timeBefore <- 4 computeStatsForLogrank(survDataFrame, y0Index, y1Index, timeNow, timeBefore)data <- generateSurvData(nP = 5, nT = 5, lambdaP = 0.03943723, lambdaT = 0.5*0.03943723, endTime = 40, seed = 2006) survObj <- survival::Surv(data$time, data$status) survDataFrame <- as.data.frame(as.matrix(survObj)) y0Index <- which(data$group=="P") y1Index <- which(data$group=="T") timeNow <- 4 timeBefore <- 0 computeStatsForLogrank(survDataFrame, y0Index, y1Index, timeNow, timeBefore) timeNow <- 13 timeBefore <- 4 computeStatsForLogrank(survDataFrame, y0Index, y1Index, timeNow, timeBefore)
Helper function to compute the relevant summary statistics in z and t-tests
computeZTSumStats( x, y = NULL, sequential = NULL, paired = FALSE, varEqual = TRUE, testType = NULL )computeZTSumStats( x, y = NULL, sequential = NULL, paired = FALSE, varEqual = TRUE, testType = NULL )
x |
a (non-empty) numeric vector of data values. |
y |
an optional (non-empty) numeric vector of data values. |
sequential |
a logical indicating whether a sequential analysis should be performed. |
paired |
a logical, if |
varEqual |
a logical variable indicating whether to treat the two variances as being equal. Default varEqual=TRUE. |
testType |
either one of "oneSample", "paired", "twoSample". |
a list of summary statistics
n1 <- 23 n2 <- 39 x <- rnorm(n1, sd=7) y <- rnorm(n2, sd=3) computeZTSumStats(x=x, y=y)n1 <- 23 n2 <- 39 x <- rnorm(n1, sd=7) y <- rnorm(n2, sd=3) computeZTSumStats(x=x, y=y)
Based on conjugate priors with a total of 8 hyperparameters.
conjugateBfTStat( x1, sdObs1, n1, x2, sdObs2, n2, a1 = 3.98, g1 = 0.03, a2 = 4.02, g2 = 0.05, a0 = 4, g0 = 2, aGamma = 2, bGamma = 1/2, log = FALSE )conjugateBfTStat( x1, sdObs1, n1, x2, sdObs2, n2, a1 = 3.98, g1 = 0.03, a2 = 4.02, g2 = 0.05, a0 = 4, g0 = 2, aGamma = 2, bGamma = 1/2, log = FALSE )
x1 |
numeric, sample mean of group 1 |
sdObs1 |
numeric, the observed standard deviation of the first group. |
n1 |
integer sample size of group 1 |
x2 |
numeric, sample mean of group 2 |
sdObs2 |
numeric, the observed standard deviation of the second group. |
n2 |
integer sample size of group 2 |
a1 |
numeric, prior mean of the population mean mu1 of group 1 |
g1 |
numeric > 0, conditional prior variance of the population mean
mu1 of group 1 is given by |
a2 |
numeric, prior mean of the population mean mu2 of group 2 |
g2 |
numeric > 0, conditional prior variance of the population mean
mu2 of group 2 is given by |
a0 |
numeric, prior mean of the overall population mean mu0 of both groups |
g0 |
numeric > 0, conditional prior variance of the population mean
mu0 of both groups is given by |
aGamma |
numeric > 0, shape parameter of the prior on the standard deviation sigma |
bGamma |
numeric > 0, rate parameter of the prior on the standard deviation sigma |
log |
logical, default FALSE, if TRUE then return logarithm of the subjective Bayes factor outcome |
numeric > 0 representing the subjective Bayes factor outcome in favour of the alternative over the null
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
conjugateBfTStat(5.2, 2, 3, 3.4, 2, 12)conjugateBfTStat(5.2, 2, 3, 3.4, 2, 12)
Based on conjugate priors with a total of 8 hyperparameters.
conjugateBfTStatOld( x1, sdObs1, n1, x2, sdObs2, n2, a1 = 3.98, g1 = 0.5, a2 = 4.02, g2 = 0.1, a0 = 4, g0 = 2, aGamma = 2, bGamma = 1/2, log = FALSE )conjugateBfTStatOld( x1, sdObs1, n1, x2, sdObs2, n2, a1 = 3.98, g1 = 0.5, a2 = 4.02, g2 = 0.1, a0 = 4, g0 = 2, aGamma = 2, bGamma = 1/2, log = FALSE )
x1 |
numeric, sample mean of group 1 |
sdObs1 |
numeric, the observed standard deviation of the first group. |
n1 |
integer sample size of group 1 |
x2 |
numeric, sample mean of group 2 |
sdObs2 |
numeric, the observed standard deviation of the second group. |
n2 |
integer sample size of group 2 |
a1 |
numeric, prior mean of the population mean mu1 of group 1 |
g1 |
numeric > 0, conditional prior variance of the population mean
mu1 of group 1 is given by |
a2 |
numeric, prior mean of the population mean mu2 of group 2 |
g2 |
numeric > 0, conditional prior variance of the population mean
mu2 of group 2 is given by |
a0 |
numeric, prior mean of the overall population mean mu0 of both groups |
g0 |
numeric > 0, conditional prior variance of the population mean
mu0 of both groups is given by |
aGamma |
numeric > 0, shape parameter of the prior on the standard deviation sigma |
bGamma |
numeric > 0, rate parameter of the prior on the standard deviation sigma |
log |
logical, default FALSE, if TRUE then return logarithm of the subjective Bayes factor outcome |
numeric > 0 representing the subjective Bayes factor outcome in favour of the alternative over the null
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
conjugateBfTStat(5.2, 2, 3, 3.4, 2, 12)conjugateBfTStat(5.2, 2, 3, 3.4, 2, 12)
Construct a savi design object to be set in the design function
constructSaviDesignObj(testName)constructSaviDesignObj(testName)
testName |
The name of the analysis that is performed such as "Z-Test", and "Test of Two Proportions". |
a savi design object
constructSafeDesignObj(): Deprecated version of constructSaviDesignObj
obj <- constructSaviDesignObj("Z-Test")obj <- constructSaviDesignObj("Z-Test")
Construct a savi test object to be set in the savi testing function
constructSaviTestObj(testName)constructSaviTestObj(testName)
testName |
The name of the analysis that is performed such as "Z-Test", and "Test of Two Proportions". |
a savi test object
constructSafeTestObj(): Deprecated version of constructSaviTestObj
obj <- constructSaviTestObj("Z-Test")obj <- constructSaviTestObj("Z-Test")
Construct a list to be set in the sampleStoppingTimes... function
constructSampleStoppingTimesList( nSim = 1000L, nMax = 1000L, wantEValuesAtNMax = FALSE, wantSamplePaths = TRUE )constructSampleStoppingTimesList( nSim = 1000L, nMax = 1000L, wantEValuesAtNMax = FALSE, wantSamplePaths = TRUE )
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
nMax |
integer > 0, maximum sample size of the (first) sample in each sample path. |
wantEValuesAtNMax |
logical. If |
wantSamplePaths |
logical. If |
a list with names
obj <- constructSampleStoppingTimesList()obj <- constructSampleStoppingTimesList()
Helper function that outputs a sequence of sample sizes, effective sample sizes, and the degrees of freedom depending on the type of T-test. Also used for Z-tests.
defineTTestN( lowN = 3, highN = 100, ratio = 1, testType = c("oneSample", "paired", "twoSample") )defineTTestN( lowN = 3, highN = 100, ratio = 1, testType = c("oneSample", "paired", "twoSample") )
lowN |
integer that defines the smallest n of our search space for n. |
highN |
integer largest sample size of the (first) sample. Default set to 100. |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
testType |
either one of "oneSample", "paired", "twoSample". |
Returns the sample sizes and degrees of freedom.
Computes the number of samples necessary to reach a tolerable type I and desired power for the frequentist T-test.
designFreqT( deltaMin, alpha = 0.05, power = 0.8, alternative = c("twoSided", "greater", "less"), h0 = 0, testType = c("oneSample", "paired", "twoSample"), ... )designFreqT( deltaMin, alpha = 0.05, power = 0.8, alternative = c("twoSided", "greater", "less"), h0 = 0, testType = c("oneSample", "paired", "twoSample"), ... )
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
h0 |
numeric, representing the null value, default h0=0. |
testType |
either one of "oneSample", "paired", "twoSample". |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
Returns an object of class 'freqTDesign'. An object of class 'freqTDesign' is a list containing at least the following components:
the planned sample size(s).
the minimal clinically relevant standardised effect size provided by the user.
the tolerable type I error provided by the user.
the desired power provided by the user.
the smallest n of the search space for n provided by the user.
the largest n of the search space for n provided by the user.
any of "oneSample", "paired", "twoSample" provided by the user.
any of "twoSided", "greater", "less" provided by the user.
designFreqT(0.5)designFreqT(0.5)
Computes the number of samples necessary to reach a tolerable type I and desired power for the frequentist Z-test.
designFreqZ( meanDiffMin, alternative = c("twoSided", "greater", "less"), alpha = 0.05, power = 0.8, testType = c("oneSample", "paired", "twoSample"), ratio = 1, sigma = 1, h0 = 0, kappa = sigma, lowN = 3L, highN = 100L, ... )designFreqZ( meanDiffMin, alternative = c("twoSided", "greater", "less"), alpha = 0.05, power = 0.8, testType = c("oneSample", "paired", "twoSample"), ratio = 1, sigma = 1, h0 = 0, kappa = sigma, lowN = 3L, highN = 100L, ... )
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
h0 |
numeric, representing the null value, default h0=0. |
kappa |
the true population standard deviation. Default kappa=sigma. |
lowN |
integer that defines the smallest n of our search space for n. |
highN |
integer that defines the largest n of our search space for n. This might be the largest n that we are able to fund. |
... |
further arguments to be passed to or from methods. |
returns a 'freqZDesign' object.
freqDesign <- designFreqZ(meanDiffMin = 0.5, highN = 100) freqDesign$nPlan freqDesign2 <- designFreqZ(meanDiffMin = 0.2, lowN = 32, highN = 200) freqDesign2$nPlanfreqDesign <- designFreqZ(meanDiffMin = 0.5, highN = 100) freqDesign$nPlan freqDesign2 <- designFreqZ(meanDiffMin = 0.2, lowN = 32, highN = 200) freqDesign2$nPlan
Helper function that extract the results for the design scenario 1a: Target nPlan
designSavi1aHelper( samplingResult, esMin, power, ratio, beta = NULL, testType = c("oneSample", "paired", "twoSample") )designSavi1aHelper( samplingResult, esMin, power, ratio, beta = NULL, testType = c("oneSample", "paired", "twoSample") )
samplingResult |
output from sampling functions such as computeNPlanSaviZ and computeNPlanSaviT |
esMin |
numeric that defines the minimal clinically relevant effect size, e.g. meanDiffMin for the z-test, or deltaMin for the t-test. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
testType |
either one of "oneSample", "paired", "twoSample". |
power |
numeric in (0, 1) that specifies the desirable power necessary to calculate both "n" and the minimum detectable effect size. |
a list of partial results for the design scenario 1a
designSafe1aHelper(): Deprecated version of designSavi1aHelper
samplingResult <- computeNPlanSaviZ(0.7, nSim=10, nMax=20) result <- designSavi1aHelper(samplingResult, 0.7, 0.2, 1)samplingResult <- computeNPlanSaviZ(0.7, nSim=10, nMax=20) result <- designSavi1aHelper(samplingResult, 0.7, 0.2, 1)
Helper function that extract the results for the design scenario 1a: Target nPlan
designSavi2Helper( samplingResult, esMin, nPlan, ratio, testType = c("oneSample", "paired", "twoSample") )designSavi2Helper( samplingResult, esMin, nPlan, ratio, testType = c("oneSample", "paired", "twoSample") )
samplingResult |
output from sampling functions such as computeNPlanSaviZ and computeNPlanSaviT |
esMin |
numeric that defines the minimal clinically relevant effect size, e.g. meanDiffMin for the z-test, or deltaMin for the t-test. |
nPlan |
vector of max length 2 representing the planned sample sizes. |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
testType |
either one of "oneSample", "paired", "twoSample". |
a list of partial results for the design scenario 1a
designSafe2Helper(): Deprecated version of designSavi2Helper
samplingResult <- computeNPlanSaviZ(0.7, nSim=10, nMax=20) result <- designSavi1aHelper(samplingResult, 0.7, 0.2, 1)samplingResult <- computeNPlanSaviZ(0.7, nSim=10, nMax=20) result <- designSavi1aHelper(samplingResult, 0.7, 0.2, 1)
A designed experiment requires (1) an anticipated number of events nEvents, or even better nPlan, the number of
participants to be recruited in the study, and (2) the parameter of the savi test, i.e., thetaS. Provided with a
clinically relevant minimal hazard ratio hrMin, this function outputs thetaS = hrMin as the savi test defining
parameter in accordance to the GROW criterion. If a tolerable type II error beta is provided then nEvents can be
sampled. The sampled nEvents is then the smallest nEvents for which hrMin is found with power of at least 1 - beta
under optional stopping. If exact equal FALSE, then the computations exploit the local asymptotic normal
approximation to sampling distribution of the logrank test derived by Schoenfeld (1981).
designSaviLogrank( hrMin = NULL, power = NULL, nEvents = NULL, alpha = 0.05, h0 = 1, alternative = c("twoSided", "greater", "less"), m0 = 50000L, m1 = 50000L, testType = c("exactLogrank", "gaussianLogrank"), ratio = 1, exact = TRUE, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, groupSizePerTimeFunction = returnOne, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, beta = NULL, relevanceTest = FALSE, relevanceSize = NULL, wantEValuesAtNMax = NULL, wantSimData = FALSE, ... )designSaviLogrank( hrMin = NULL, power = NULL, nEvents = NULL, alpha = 0.05, h0 = 1, alternative = c("twoSided", "greater", "less"), m0 = 50000L, m1 = 50000L, testType = c("exactLogrank", "gaussianLogrank"), ratio = 1, exact = TRUE, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, groupSizePerTimeFunction = returnOne, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, beta = NULL, relevanceTest = FALSE, relevanceSize = NULL, wantEValuesAtNMax = NULL, wantSimData = FALSE, ... )
hrMin |
numeric that defines the minimal relevant hazard ratio, the smallest hazard ratio that we want to detect. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
nEvents |
numeric > 0, targetted number of events. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
h0 |
numeric > 0, represents the null hypothesis, default h0=1. |
alternative |
a character string specifying the alternative hypothesis, which must be one of "twoSided" (default),"greater" or "less". The alternative is pitted against the null hypothesis of equality of the survival distributions. More specifically, let lambda1 be the hazard rate of group 1 (i.e., placebo), and lambda2 the hazard ratio of group 2 (i.e., treatment), then the null hypothesis states that the hazard ratio theta = lambda2/lambda1 = 1. If alternative = "less", the null hypothesis is compared to theta < 1, thus, lambda2 < lambda1, that is, the hazard of group 2 (i.e., treatment) is less than that of group 1 (i.e., placebo), hence, the treatment is beneficial. If alternative = "greater", then the null hypothesis is compared to theta > 1, thus, lambda2 > lambda1, hence, harm. |
m0 |
Number of subjects in the control group 0/1 at the beginning of the trial, i.e., |
m1 |
Number of subjects in the treatment group 1/2 at the beginning of the trial, i.e., |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 (Treatment) over condition 1 (Placebo),
thus, m1/m0. Note that m1 and m0 are not used to specify ratio. Ratio is only used when |
exact |
a logical indicating whether the design should be based on the exact savi logrank test based on the
hypergeometric likelihood. Default is |
parameter |
Numeric > 0, represents the savi tests defining thetaS. Default NULL so it's decided by the algorithm, typically, this equals hrMin, which corresponds to the GROW choice. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
groupSizePerTimeFunction |
A function without parameters and integer output. This function provides the number
of events at each time step. For instance, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of events for the exact savi logrank test under continuous monitoring |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximation of power or nEvents for the exact savi logrank test under continuous monitoring |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop for the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. This overrides the "beta" argument |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant standardised mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
wantEValuesAtNMax |
logical. If |
wantSimData |
logical. If |
Returns a saviDesign object that includes:
the anticipated number of events, either (1) specified by the user, or (2) computed based on beta and thetaMin.
the parameter that defines the savi test. Here log(thetaS).
the minimally clinically relevant hazard ratio specified by the user.
the tolerable type I error provided by the user.
the tolerable type II error provided by the user.
any of "twoSided", "greater", "less" provided by the user.
"logrank".
default is 1. It defines the ratio between the planned randomisation of condition 2 over condition 1.
FALSE to indicate that the design is not a pilot study.
the expression with which this function is called.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
Schoenfeld, D. (1981). The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika, 68(1), 316-319, https://doi.org/10.2307/2335833.
designSaviLogrank(hrMin=0.7) designSaviLogrank(hrMin=0.7, exact=FALSE) designSaviLogrank(hrMin=0.7, beta=0.3, nSim=10) designSaviLogrank(hrMin=0.7, nEvents=190, nSim=10)designSaviLogrank(hrMin=0.7) designSaviLogrank(hrMin=0.7, exact=FALSE) designSaviLogrank(hrMin=0.7, beta=0.3, nSim=10) designSaviLogrank(hrMin=0.7, nEvents=190, nSim=10)
Finds the parameter and beta when provided with only alpha, esMin, and nPlan
designSaviLogrank2WantBeta( hrMin, nEvents, alpha = 0.05, alternative = c("twoSided", "greater", "less"), m0 = 50000L, m1 = 50000L, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, groupSizePerTimeFunction = returnOne, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, ... )designSaviLogrank2WantBeta( hrMin, nEvents, alpha = 0.05, alternative = c("twoSided", "greater", "less"), m0 = 50000L, m1 = 50000L, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, groupSizePerTimeFunction = returnOne, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, ... )
hrMin |
numeric that defines the minimal relevant hazard ratio, the smallest hazard ratio that we want to detect. |
nEvents |
numeric > 0, targetted number of events. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis, which must be one of "twoSided" (default),"greater" or "less". The alternative is pitted against the null hypothesis of equality of the survival distributions. More specifically, let lambda1 be the hazard rate of group 1 (i.e., placebo), and lambda2 the hazard ratio of group 2 (i.e., treatment), then the null hypothesis states that the hazard ratio theta = lambda2/lambda1 = 1. If alternative = "less", the null hypothesis is compared to theta < 1, thus, lambda2 < lambda1, that is, the hazard of group 2 (i.e., treatment) is less than that of group 1 (i.e., placebo), hence, the treatment is beneficial. If alternative = "greater", then the null hypothesis is compared to theta > 1, thus, lambda2 > lambda1, hence, harm. |
m0 |
Number of subjects in the control group 0/1 at the beginning of the trial, i.e., |
m1 |
Number of subjects in the treatment group 1/2 at the beginning of the trial, i.e., |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 (Treatment) over condition 1 (Placebo),
thus, m1/m0. Note that m1 and m0 are not used to specify ratio. Ratio is only used when |
parameter |
Numeric > 0, represents the savi tests defining thetaS. Default NULL so it's decided by the algorithm, typically, this equals hrMin, which corresponds to the GROW choice. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
groupSizePerTimeFunction |
A function without parameters and integer output. This function provides the number
of events at each time step. For instance, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of events for the exact savi logrank test under continuous monitoring |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximation of power or nEvents for the exact savi logrank test under continuous monitoring |
... |
further arguments to be passed to or from methods. |
A list with the parameter and beta amongst other items
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
Schoenfeld, D. (1981). The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika, 68(1), 316-319, https://doi.org/10.2307/2335833.
designSaviLogrank2WantBeta(hrMin=0.9, nEvents=7, nSim=10)designSaviLogrank2WantBeta(hrMin=0.9, nEvents=7, nSim=10)
A designed experiment requires (1) a sample size nPlan to plan for, and (2) a savi test defining parameter. The design involves alpha and the three quantities: (1) nPlan, (2) power, and (3) a minimal clinically relevant standarised mean difference deltaMin.
Goal: "nPlan" and optimal E-variable. Given: deltaMin and power.
Goal: an optimal E-variable. Given: deltaMin only.
Goal: "power" and optimal E-variable. Given: deltaMin and nPlan.
Goal: "deltaMin" and optimal E-variable. Given: power and nPlan.
Goal: an optimal E-variable. Given: nPlan only.
designSaviT( deltaMin = NULL, power = NULL, nPlan = NULL, alpha = 0.05, h0 = 0, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, deltaTrue = NULL, sigma = 1, sigma2 = sigma, lowEsTrue = 0.01, highEsTrue = 3, varEqual = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, betaDefault = 0.2, highN = 10000L, wantSampling = TRUE, nuMin = 2, ... )designSaviT( deltaMin = NULL, power = NULL, nPlan = NULL, alpha = 0.05, h0 = 0, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, deltaTrue = NULL, sigma = 1, sigma2 = sigma, lowEsTrue = 0.01, highEsTrue = 3, varEqual = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, betaDefault = 0.2, highN = 10000L, wantSampling = TRUE, nuMin = 2, ... )
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
h0 |
numeric, representing the null value, default h0=0. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi t test under continuous monitoring,or for the computation of the logarithm of the implied target. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
wantSimData |
logical, if |
sigma |
numeric > 0 representing the population standard deviation used for the test. |
sigma2 |
numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test |
varEqual |
a logical variable indicating whether to treat the two variances as being equal. Default varEqual=TRUE. |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
betaDefault |
numeric, defaulting value for 1-power and alphaRelevance |
highN |
integer, largest possible sampling horizon. This might be the
largest n that we are able to fund, which by default is set to 1e4L.
Typically, highN is not used, as the function
|
wantSampling |
logical, default TRUE so sampling paths are drawn. For instance, if meanDiffMin and power, are given, then nPlan (scenario 1a) is derived by sampling. Set this to FALSE, whenever we want to run a minimal efficacy test without needing to know nPlan |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
Every scenario returns an E-variable adapted to the input. Scenario 1.a,
for instance, outputs the parameter of the provided eType (default mom)
savi test, see matchEParameterWith for details, and nPlan.
The nPlan is based on samples paths drawn under deltaTrue (if not specified,
then deltaTrue=deltaMin by default). The resulting nPlan corresponds to the
power (say 80%) quantile of the first-passage time distribution associated
with E crossing threshold 1/alpha.
Returns an object of class 'saviDesign'. An object of class 'saviDesign' is a list containing at least the following components:
the savi test defining parameter, see matchEParameterWith.
the tolerable type I error provided by the user.
logical, specifying whether it's a pilot design, which occurs when saviTTest is called without a designObj.
"T-Test".
the expression with which this function is called.
designSafeT(): Deprecated version of designSaviT
designSafeT2WantBeta(): Deprecated version of designSaviT2WantBeta
designSafeT3WantEsMin(): Deprecated version of designSafeT3WantEsMin
designSafeT3bWantParameter(): Deprecated version of designSaviT3bWantParameter
computeNPlanBatchSafeT(): Deprecated version of computeNPlanBatchSaviT
computeMinEsBatchSafeT(): Deprecated version of computeMinEsBatchSaviT
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
# Scenario 1.b: Goal: an E-variable designObj <- designSaviT(deltaMin=0.8) # Scenario 1.a: Goal: "nPlan" and optimal E-variable. designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100) plot(designObj) # Scenario 1a. with relevance testing, also stopping for practically null designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100, relevanceTest=TRUE) plot(designObj) # Scenario 2: Goal: "power" and optimal E-variable designObj <- designSaviT(deltaMin=0.8, nPlan=16, nSim=100) # Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable designObj <- designSaviT(power=0.7, nPlan=16) # Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only. designObj <- designSaviT(nPlan=16)# Scenario 1.b: Goal: an E-variable designObj <- designSaviT(deltaMin=0.8) # Scenario 1.a: Goal: "nPlan" and optimal E-variable. designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100) plot(designObj) # Scenario 1a. with relevance testing, also stopping for practically null designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100, relevanceTest=TRUE) plot(designObj) # Scenario 2: Goal: "power" and optimal E-variable designObj <- designSaviT(deltaMin=0.8, nPlan=16, nSim=100) # Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable designObj <- designSaviT(power=0.7, nPlan=16) # Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only. designObj <- designSaviT(nPlan=16)
Finds the parameter and power when provided with only alpha, esMin, and nPlan
designSaviT1aWantNPlan( deltaMin, power, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, deltaTrue = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, sigma = 1, sigma2 = 1, alphaRelevance = NULL, nuMin = 2, ... )designSaviT1aWantNPlan( deltaMin, power, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, deltaTrue = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, sigma = 1, sigma2 = 1, alphaRelevance = NULL, nuMin = 2, ... )
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi t test under continuous monitoring,or for the computation of the logarithm of the implied target. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
wantSimData |
logical, if |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
sigma |
numeric > 0 representing the population standard deviation used for the test. |
sigma2 |
numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
A list with the parameter and the targeted nPlan amongst other items
designSafeT1aWantNPlan(): Deprecated version of designSaviT1aWantNPlan
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviT1aWantNPlan(deltaMin=0.9, power=0.7, nSim=10)designSaviT1aWantNPlan(deltaMin=0.9, power=0.7, nSim=10)
The design requires the number of observations one expects to collect in each group in each data block.
I.e., when one expects balanced data, one could choose na = nb = 1 and would be allowed to analyse
the data stream each time a new observation in both groups has come in. The best results in terms of power
are achieved when the data blocks are chosen as small as possible, as this allows for analysing and updating
the savi test as often as possible, to fit the data best.
Further, the design requires two out of the following three parameters to be known:
the power one aims to achieve (1 - beta),
the minimal relevant difference between the groups (delta)
the number of blocks planned (nBlocksPlan),
where the unknown out of the three will be estimated. In the case of an exploratory "pilot" analysis, one can also only provide the number of blocks planned.
designSaviTwoProportions( na, nb, nBlocksPlan = NULL, beta = NULL, delta = NULL, alternativeRestriction = c("none", "difference", "logOddsRatio"), alpha = 0.05, pilot = "FALSE", hyperParameterValues = NULL, previousSaviTestResult = NULL, M = 1000, simThetaAMin = NULL, simThetaAMax = NULL )designSaviTwoProportions( na, nb, nBlocksPlan = NULL, beta = NULL, delta = NULL, alternativeRestriction = c("none", "difference", "logOddsRatio"), alpha = 0.05, pilot = "FALSE", hyperParameterValues = NULL, previousSaviTestResult = NULL, M = 1000, simThetaAMin = NULL, simThetaAMax = NULL )
na |
number of observations in group a per data block |
nb |
number of observations in group b per data block |
nBlocksPlan |
planned number of data blocks collected |
beta |
numeric in (0, 1) that specifies the tolerable type II error control necessary to calculate both "nBlocksPlan" and "delta". Note that 1-beta defines the power. |
delta |
a priori minimal relevant divergence between group means b and a, either a numeric between -1 and 1 for no alternative restriction or a restriction on difference, or a real for a restriction on the log odds ratio. |
alternativeRestriction |
a character string specifying an optional restriction on the alternative hypothesis; must be one of "none" (default), "difference" (difference group mean b minus group b) or "logOddsRatio" (the log odds ratio between group means b and a). |
alpha |
numeric in (0, 1) that specifies the tolerable type I error control –independent on n– that the designed test has to adhere to. Note that it also defines the rejection rule e10 >= 1/alpha. |
pilot |
logical, specifying whether it's a pilot design. |
hyperParameterValues |
named list containing numeric values for hyperparameters betaA1, betaA2, betaB1 and betaB2, with betaA1 and betaB1 specifying the parameter
equivalent to |
M |
number of simulations used to estimate power or nBlocksPlan. Default |
simThetaAMin |
minimal event rate in control group to simulate nPlan or power for.
Can be specified when specifically interested in planning studies for specific event rates.
Default |
simThetaAMax |
maximal event rate in control group to simulate nPlan or power for. Default |
previousSaviTestResult |
optionally, a previous savi test result can be provided. The posterior of the hyperparameters of this test is then used for the hyperparameter settings. Default NULL. |
Returns a 'saviDesign' object that includes:
the sample size(s) to plan for. Computed based on beta and meanDiffMin, or provided by the user if known.
the savi test defining parameter: here the hyperparameters.
the minimally clinically relevant effect size provided by the user.
the tolerable type I error provided by the user.
the tolerable type II error specified by the user.
any of "twoSided", "greater", "less" based on the alternativeRestriction provided by the user.
here 2x2
logical, specifying whether it's a pilot design.
the expression with which this function is called.
#plan for an experiment to detect minimal difference of 0.6 with a balanced design set.seed(3152021) designSaviTwoProportions(na = 1, nb = 1, alpha = 0.1, beta = 0.20, delta = 0.6, alternativeRestriction = "none", M = 75) #savi analysis of a pilot: number of samples already known designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 20, pilot = TRUE) #specify own hyperparameters hyperParameterValues <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) designSaviTwoProportions(na = 1, nb = 1, alpha = 0.1, beta = 0.20, delta = 0.6, hyperParameterValues = hyperParameterValues, alternativeRestriction = "none", M = 75) #restrict range of proportions for estimating nPlan in the control group designSaviTwoProportions(na = 1, nb = 1, beta = 0.20, delta = 0.3, alternativeRestriction = "none", M = 75, simThetaAMin = 0.1, simThetaAMax = 0.2)#plan for an experiment to detect minimal difference of 0.6 with a balanced design set.seed(3152021) designSaviTwoProportions(na = 1, nb = 1, alpha = 0.1, beta = 0.20, delta = 0.6, alternativeRestriction = "none", M = 75) #savi analysis of a pilot: number of samples already known designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 20, pilot = TRUE) #specify own hyperparameters hyperParameterValues <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) designSaviTwoProportions(na = 1, nb = 1, alpha = 0.1, beta = 0.20, delta = 0.6, hyperParameterValues = hyperParameterValues, alternativeRestriction = "none", M = 75) #restrict range of proportions for estimating nPlan in the control group designSaviTwoProportions(na = 1, nb = 1, beta = 0.20, delta = 0.3, alternativeRestriction = "none", M = 75, simThetaAMin = 0.1, simThetaAMax = 0.2)
A designed experiment requires (1) a sample size nPlan to plan for, and (2) a savi test defining parameter. The design involves alpha and the three quantities: (1) nPlan, (2) power, and (3) a minimal clinically relevant mean difference meanDiffMin.
Goal: "nPlan" and optimal E-variable. Given: meanDiffMin and power.
Goal: an optimal E-variable. Given: meanDiffMin only.
Goal: "power" and optimal E-variable. Given: meanDiffMin and nPlan.
Goal: "meanDiffMin" and optimal E-variable. Given: power and nPlan.
Goal: an optimal E-variable. Given: nPlan only.
designSaviZ( meanDiffMin = NULL, power = NULL, nPlan = NULL, alpha = 0.05, h0 = 0, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, meanDiffTrue = NULL, beta = NULL, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, lowEsTrue = 0.01, highEsTrue = 3, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, betaDefault = 0.2, highN = 10000L, wantSampling = TRUE, ... )designSaviZ( meanDiffMin = NULL, power = NULL, nPlan = NULL, alpha = 0.05, h0 = 0, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, meanDiffTrue = NULL, beta = NULL, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, lowEsTrue = 0.01, highEsTrue = 3, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, betaDefault = 0.2, highN = 10000L, wantSampling = TRUE, ... )
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
h0 |
numeric, representing the null value, default h0=0. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi z test under continuous monitoring,or for the computation of the logarithm of the implied target. |
relevanceTest |
logical, if |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
wantSimData |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
betaDefault |
numeric, defaulting value for 1-power and alphaRelevance |
highN |
integer, largest possible sampling horizon. This might be the
largest n that we are able to fund, which by default is set to 1e4L.
Typically, highN is not used, as the function
|
wantSampling |
logical, default TRUE so sampling paths are drawn. For instance, if meanDiffMin and power, are given, then nPlan (scenario 1a) is derived by sampling. Set this to FALSE, whenever we want to run a minimal efficacy test without needing to know nPlan |
Every scenario returns an E-variable adapted to the input. Scenario 1.a,
for instance, outputs the parameter of the provided eType (default mom)
savi test, see matchEParameterWith for details, and nPlan.
The nPlan is based on samples paths drawn under meanDiffTrue (if not specified,
then meanDiffTrue=meanDiffMin by default). The resulting nPlan corresponds to the
power (say 80%) quantile of the first-passage time distribution associated
with E crossing threshold 1/alpha.
Returns a saviDesign object that includes:
the savi test defining parameter, see matchEParameterWith.
the tolerable type I error provided by the user.
logical, specifying whether it's a pilot design, which occurs when saviZTest is called without a designObj.
"Z-Test".
the expression with which this function is called.
designSafeZ(): Deprecated version of designSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
# Scenario 1.b: Goal: an E-variable designObj <- designSaviZ(meanDiffMin=0.8) # Scenario 1.a: Goal: "nPlan" and optimal E-variable. designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100) plot(designObj) # Scenario 1a. with relevance testing, also stopping for practically null designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100, relevanceTest=TRUE) plot(designObj) # Scenario 2: Goal: "power" and optimal E-variable designObj <- designSaviZ(meanDiffMin=0.8, nPlan=16, nSim=100) # Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable designObj <- designSaviZ(power=0.7, nPlan=16) # Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only. designObj <- designSaviZ(nPlan=16)# Scenario 1.b: Goal: an E-variable designObj <- designSaviZ(meanDiffMin=0.8) # Scenario 1.a: Goal: "nPlan" and optimal E-variable. designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100) plot(designObj) # Scenario 1a. with relevance testing, also stopping for practically null designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2, alternative="greater", nSim=100, relevanceTest=TRUE) plot(designObj) # Scenario 2: Goal: "power" and optimal E-variable designObj <- designSaviZ(meanDiffMin=0.8, nPlan=16, nSim=100) # Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable designObj <- designSaviZ(power=0.7, nPlan=16) # Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only. designObj <- designSaviZ(nPlan=16)
Finds the parameter and power when provided with only alpha, esMin, and nPlan
designSaviZ1aWantNPlan( meanDiffMin, power, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, beta = NULL, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, meanDiffTrue = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, highN = 10000L, ... )designSaviZ1aWantNPlan( meanDiffMin, power, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, beta = NULL, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, meanDiffTrue = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, highN = 10000L, ... )
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi z test under continuous monitoring,or for the computation of the logarithm of the implied target. |
relevanceTest |
logical, if |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
wantSimData |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
highN |
integer, largest possible sampling horizon. This might be the
largest n that we are able to fund, which by default is set to 1e4L.
Typically, highN is not used, as the function
|
A list with the parameter and the targeted nPlan amongst other items
designSafeZ1aWantNPlan(): Deprecated version of designSaviZ1aWantNPlan
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviZ1aWantNPlan(meanDiffMin=0.9, power=0.7, nSim=10)designSaviZ1aWantNPlan(meanDiffMin=0.9, power=0.7, nSim=10)
Finds the parameter and power when provided with only alpha, esMin, and nPlan
designSaviZ2WantPower( meanDiffTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, meanDiffMin = meanDiffTrue, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, ... )designSaviZ2WantPower( meanDiffTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, meanDiffMin = meanDiffTrue, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantSamplePaths = TRUE, wantSimData = TRUE, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, ... )
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi z test under continuous monitoring,or for the computation of the logarithm of the implied target. |
... |
further arguments to be passed to or from methods. |
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
wantSimData |
logical, if |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
A list with the parameter and power amongst other items
designSafeZ2WantBeta(): Deprecated version of designSaviZ2WantBeta
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviZ2WantPower(meanDiffTrue=0.9, nPlan=7, nSim=10)designSaviZ2WantPower(meanDiffTrue=0.9, nPlan=7, nSim=10)
Finds the parameter and meanDiffMin when provided with only alpha, nPlan
designSaviZ3bWantParameter( nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), ... )designSaviZ3bWantParameter( nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), ... )
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
... |
further arguments to be passed to or from methods. |
A list with the parameter and the parameter amongst other items
designSafeZ3bWantParameter(): Deprecated version of designSaviZ3bWantParameter
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviZ3bWantParameter(nPlan=13)designSaviZ3bWantParameter(nPlan=13)
Finds the parameter and esMin when provided with only alpha, power, and nPlan
designSaviZ3WantEsMin( power, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), lowEsTrue = 0.01, highEsTrue = 3, ... )designSaviZ3WantEsMin( power, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), sigma = 1, kappa = sigma, testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), lowEsTrue = 0.01, highEsTrue = 3, ... )
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
... |
further arguments to be passed to or from methods. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
A list with the parameter and the targeted esMin amongst other items
designSafeZ3WantEsMin(): Deprecated version of designSaviZ3WantEsMin
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviZ3WantEsMin(power=0.7, nPlan=10)designSaviZ3WantEsMin(power=0.7, nPlan=10)
A designed experiment requires (1) a number of data blocks nBlocksPlan
to plan for, and (2) a savi test defining parameter: eType and, with
it, the effect measure. Both effects are A minus B and anchored on
thetaB: propDiff = thetaA - thetaB and logOdds = logit(thetaA) - logit(thetaB), so "greater" means group A has the larger proportion,
as x - y > 0 does in stats::t.test().
designSavi2x2( na, nb, nBlocksPlan = NULL, propDiffMin = NULL, logOddsMin = NULL, alpha = 0.05, power = NULL, h0 = 0, alternative = c("twoSided", "greater", "less"), eType = c("eBeta", "grow", "eGauss"), betaParameter = NULL, gaussParameter = NULL, runningIntersection = NULL, nSim = 1000L, nBoot = 1000L, nMax = 10000L, seed = NULL, wantSamplePaths = FALSE, pb = TRUE )designSavi2x2( na, nb, nBlocksPlan = NULL, propDiffMin = NULL, logOddsMin = NULL, alpha = 0.05, power = NULL, h0 = 0, alternative = c("twoSided", "greater", "less"), eType = c("eBeta", "grow", "eGauss"), betaParameter = NULL, gaussParameter = NULL, runningIntersection = NULL, nSim = 1000L, nBoot = 1000L, nMax = 10000L, seed = NULL, wantSamplePaths = FALSE, pb = TRUE )
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
nBlocksPlan |
planned number of data blocks collected, see scenario 2 and 3 above. |
propDiffMin |
numeric in (-1, 1) that defines the minimal relevant
difference in proportions |
logOddsMin |
numeric that defines the minimal relevant log odds
ratio |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. |
h0 |
numeric, representing the null value, default h0=0. Only
|
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
eType |
character one of "eBeta", "grow", and "eGauss". "eBeta" is default and uses a Beta prior on each proportion, "grow" uses a point prior at the minimal effect, "eGauss" a normal prior on the log odds ratio. |
betaParameter |
|
gaussParameter |
|
runningIntersection |
logical, if |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi 2x2 test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, or the number of blocks for the savi 2x2 test under continuous monitoring. |
nMax |
integer > 0, maximum number of data blocks in each sample path. |
seed |
integer, seed number. Default seed=NULL yields seed=2026. |
wantSamplePaths |
logical, if |
pb |
logical, if |
"eBeta" (propDiff) and "eGauss" (logOdds) are unrestricted: a
minimal effect is an error, a power is dropped with a warning, and
nBlocksPlan is kept as the planned block count. Each reads one
prior, betaParameter or gaussParameter; NULL keeps the default,
which savi2x2TestStat() fills in. A one-sided alternative acts in
the UMP first block, and for "eGauss" also restricts the prior to
that side.
"grow" plugs in exactly one of propDiffMin, logOddsMin as the
fixed alternative and reads no prior. As in designSaviZ(), a value
whose sign contradicts a one-sided alternative is flipped with a
warning; "twoSided" uses the magnitude. Zero is an error.
For "grow" on propDiff the design involves alpha and the three
quantities: (1) nBlocksPlan, (2) power, and (3) a minimal relevant
difference in proportions propDiffMin, simulated at the worst-case
baseline of sampleStoppingTimesSavi2x2().
Goal: "nBlocksPlan" and optimal E-variable. Given: propDiffMin and power.
Goal: "power" and optimal E-variable. Given: propDiffMin and nBlocksPlan.
Goal: "propDiffMin" and optimal E-variable. Given: power and nBlocksPlan.
A minimal effect alone designs without simulation. Any other
combination is an error, except that logOddsMin drops a power with
a warning. Per-block size vectors supply nBlocksPlan when it is NULL.
Other arguments are taken as given; only alpha and power in
(0, 1), equal lengths of na and nb, at most one minimal effect,
and the minimal effects' ranges are checked.
Returns a saviDesign object that includes:
the planned sample size(s): list(na, nb), with
nBlocksPlan when given or planned.
for "eBeta" and "eGauss", the prior as one named
string, or "default" when NULL; absent for "grow", whose minimal
effect is esMin.
the minimal relevant effect size provided by the user, or
found in scenario 3, signed and named propDiff or logOdds; NULL
for "eBeta" and "eGauss".
the tolerable type I error provided by the user.
the desired power provided by the user, or the worst-case power found in scenario 2.
any of "twoSided", "greater", "less" provided by the user.
any of "eBeta", "grow", "eGauss" provided by the user.
the null value, 0.
for "eBeta" and "grow" on propDiff, the Beta prior
given, NULL for the default.
for "eGauss", the Normal prior given, NULL for
the default.
logical, as provided by the user.
"1a", "2" or "3" when planned.
scenario 1a: the bootstrap summaries of the planned block count and of the mean stopping time.
scenario 2: the bootstrap summaries of the power.
scenarios 1a and 2: the worst baseline and its simulated paths.
here 2x2
"Two Proportions".
the expression with which this function is called.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Turner, R., Ly, A., & Grünwald, P. D. (2024). Generic e-variables for exact sequential k-sample tests that allow for optional stopping. Journal of Statistical Planning and inference 230, 106116, https://doi.org/10.1016/j.jspi.2023.106116.
# Unrestricted test on propDiff with the default Beta prior designSavi2x2(na = 10, nb = 10, eType = "eBeta") # Grow on a minimal difference, no planning designSavi2x2(na = 10, nb = 10, propDiffMin = 0.2, eType = "grow") # Scenario 2: worst-case power at a planned block count designSavi2x2(na = 10, nb = 10, propDiffMin = 0.3, nBlocksPlan = 8, eType = "grow", nSim = 50, pb = FALSE)# Unrestricted test on propDiff with the default Beta prior designSavi2x2(na = 10, nb = 10, eType = "eBeta") # Grow on a minimal difference, no planning designSavi2x2(na = 10, nb = 10, propDiffMin = 0.2, eType = "grow") # Scenario 2: worst-case power at a planned block count designSavi2x2(na = 10, nb = 10, propDiffMin = 0.3, nBlocksPlan = 8, eType = "grow", nSim = 50, pb = FALSE)
Finds the parameter and power when provided with only alpha, esMin, and nPlan
designSaviT2WantPower( deltaTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), deltaMin = deltaTrue, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, nuMin = 2, ... )designSaviT2WantPower( deltaTrue, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), deltaMin = deltaTrue, testType = c("oneSample", "paired", "twoSample"), ratio = 1, parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantSamplePaths = TRUE, wantSimData = TRUE, relevanceTest = FALSE, relevanceSize = NULL, alphaRelevance = NULL, pb = TRUE, seed = NULL, nSim = 1000L, nBoot = nSim, nuMin = 2, ... )
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantSamplePaths |
logical, if |
wantSimData |
logical, if |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
nBoot |
integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi t test under continuous monitoring,or for the computation of the logarithm of the implied target. |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
A list with the parameter and beta amongst other items
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviT2WantPower(deltaTrue=0.9, nPlan=7, nSim=10)designSaviT2WantPower(deltaTrue=0.9, nPlan=7, nSim=10)
Finds the parameter and deltaMin when provided with only alpha and nPlan
designSaviT3bWantParameter( nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), ... )designSaviT3bWantParameter( nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), ... )
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
A list with the parameter and the parameter amongst other items
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviT3bWantParameter(nPlan=20)designSaviT3bWantParameter(nPlan=20)
Finds the parameter and esMin when provided with only alpha, power, and nPlan
designSaviT3WantEsMin( power, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), lowEsTrue = 0.01, highEsTrue = 3, ... )designSaviT3WantEsMin( power, nPlan, alpha = 0.05, alternative = c("twoSided", "greater", "less"), testType = c("oneSample", "paired", "twoSample"), parameter = NULL, beta = NULL, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), lowEsTrue = 0.01, highEsTrue = 3, ... )
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
parameter |
numeric, an optional savi test defining parameter. Default set to |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
A list with the parameter and the targeted esMin amongst other items
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
designSaviT3WantEsMin(power=0.7, nPlan=10)designSaviT3WantEsMin(power=0.7, nPlan=10)
Helper function: Get all names as entered by the user
extractNameFromArgs(list, name)extractNameFromArgs(list, name)
list |
list from which the element needs retrieving |
name |
character string, name of the item that need retrieving |
returns a character string
Log partition function of Fisher's noncentral hypergeometric distribution
fnchLogPartition(na, nb, totalSuccesses, logOdds)fnchLogPartition(na, nb, totalSuccesses, logOdds)
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
totalSuccesses |
nonnegative integer, the total successes |
logOdds |
numeric, one log odds ratio |
A numeric, the log of the sum over the feasible k of
choose(na, k) * choose(nb, totalSuccesses - k) * exp(logOdds * k).
The designs supported are "oneSample", "paired", "twoSample".
generateNormalData( nPlan, nSim = 1000L, deltaTrue = NULL, muGlobal = 0, sigma = 1, sigma2 = 1, paired = FALSE, seed = NULL, meanDiffTrue = NULL )generateNormalData( nPlan, nSim = 1000L, deltaTrue = NULL, muGlobal = 0, sigma = 1, sigma2 = 1, paired = FALSE, seed = NULL, meanDiffTrue = NULL )
nPlan |
optional numeric vector of length at most 2, see scenario 2 and 3 above. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
deltaTrue |
numeric, the value of the true standardised effect size (test-relevant parameter).
This argument is used by |
muGlobal |
numeric, population grand mean |
sigma |
numeric > 0, population standard deviation |
sigma2 |
numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test |
paired |
a logical, if |
seed |
integer, seed number. |
meanDiffTrue |
numeric, data governing parameter value |
Returns a list of two data matrices contains at least the following components:
a matrix of data dimension nSim by nPlan[1].
a matrix of data dimension nSim by nPlan[2].
generateNormalData(20, 15, deltaTrue=0.3)generateNormalData(20, 15, deltaTrue=0.3)
survival PackageGenerate Survival Data which Can Be Analysed With the survival Package
generateSurvData( nP, nT, alpha = 1, lambdaP, lambdaT, seed = NULL, nDigits = 0, startTime = 1, endTime = 180, orderTime = TRUE, competeRatio = 0 )generateSurvData( nP, nT, alpha = 1, lambdaP, lambdaT, seed = NULL, nDigits = 0, startTime = 1, endTime = 180, orderTime = TRUE, competeRatio = 0 )
nP |
integer > 0 representing the number of of patients in the placebo group. |
nT |
integer > 0 representing the number of of patients in the treatment group. |
alpha |
numeric > 0, representing the shape parameter of the Weibull distribution. If alpha=1, then data are generated from the exponential, i.e., constant hazard. For alpha > 1 the hazard increases, if alpha < 1, the hazard decreases. |
lambdaP |
The (relative) hazard of the placebo group. |
lambdaT |
The (relative) hazard of the treatment group. |
seed |
A seed number. |
nDigits |
numeric, the number of digits to round of the random time to |
startTime |
numeric, adds this to the random times. Default 1, so the startTime is not 0, which
is the start time of |
endTime |
The endtime of the experiment. |
orderTime |
logical, if |
competeRatio |
The ratio of the data that is due to competing risk. |
A data set with time, status and group.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
generateSurvData(800, 800, alpha=1, lambdaP=0.008, lambdaT=0.008/2)generateSurvData(800, 800, alpha=1, lambdaP=0.008, lambdaT=0.008/2)
Helper function: Get all arguments as entered by the user
getArgs()getArgs()
a list of variable names of class "call" that can be changed into names
Helper function that outputs the alternative hypothesis of the analysis.
getNameAlternative( alternative = c("twoSided", "greater", "less"), testType, h0 = 0 )getNameAlternative( alternative = c("twoSided", "greater", "less"), testType, h0 = 0 )
alternative |
A character string. "twoSided", "greater", "less". |
testType |
A character string either "oneSample", "paired", "twoSample", "gLogrank", or "eLogrank". |
h0 |
the value of the null hypothesis |
Returns a character string with the name of the analysis.
Helper function that outputs the name of the analysis.
getNameTestType(testType, testName, relevanceTest = FALSE)getNameTestType(testType, testName, relevanceTest = FALSE)
testType |
A character string. For the t-tests: "oneSample", "paired", "twoSample". |
testName |
The name of the analysis that is performed such as "Z-Test", and "Test of Two Proportions". |
relevanceTest |
logical, if |
Returns a character string with the name of the analysis.
Checks whether any of the provided objects contains a try error.
isTryError(...)isTryError(...)
... |
objects that need testing. |
Returns TRUE if there's some object that's a try-error, FALSE when all objects are
not try-errors.
x <- 1 y <- "a" z <- try(integrate(exp, -Inf, Inf)) isTryError(x, y) isTryError(x, y, z)x <- 1 y <- "a" z <- try(integrate(exp, -Inf, Inf)) isTryError(x, y) isTryError(x, y, z)
Logarithmic hyperbolic cosine
lcosh(x)lcosh(x)
x |
numeric |
numeric value
lcosh(7)lcosh(7)
The conditional likelihood of Fisher's noncentral hypergeometric
distribution on blocks 1..i, mixed under the Normal prior
gaussParameter on logOddsGrid restricted to the side of a one-sided
alternative, over the hypergeometric null. The mixture for block i
includes block i, which is valid because each factor conditions on
its block's total.
logEValueVec2x2LogOddsEGauss( ya, yb, na, nb, gaussParameter = NULL, alternative = c("twoSided", "greater", "less"), logOddsGrid = seq(-20, 20, length.out = 2000) )logEValueVec2x2LogOddsEGauss( ya, yb, na, nb, gaussParameter = NULL, alternative = c("twoSided", "greater", "less"), logOddsGrid = seq(-20, 20, length.out = 2000) )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
gaussParameter |
|
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
logOddsGrid |
numeric vector, the grid the prior is normalised on. |
A numeric vector, element i the log e-value on blocks 1..i.
The cumulative conditional log likelihood ratio of logLikelihoodFNCH()
at logOddsMin, signed by alternative, over the hypergeometric null.
"twoSided" runs both signs of the magnitude and averages their
cumulative e-processes.
logEValueVec2x2LogOddsGrow( ya, yb, na, nb, logOddsMin, alternative = c("twoSided", "greater", "less") )logEValueVec2x2LogOddsGrow( ya, yb, na, nb, logOddsMin, alternative = c("twoSided", "greater", "less") )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
logOddsMin |
numeric that defines the minimal relevant log odds
ratio |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
A numeric vector, element i the log e-value on blocks 1..i.
The cumulative log likelihood ratio of the predictive Beta means of
predictiveThetas2x2() over the pooled null mean
(na * thetaA + nb * thetaB) / (na + nb), block 1 included.
logEValueVec2x2PropDiffEBeta(ya, yb, na, nb, betaParameter)logEValueVec2x2PropDiffEBeta(ya, yb, na, nb, betaParameter)
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
betaParameter |
|
A numeric vector, element i the log e-value on blocks 1..i.
The alternative is restricted to the curve thetaA = thetaB + propDiffMin, signed by alternative, which leaves one free parameter;
its posterior on a grid of nWeight points under a Beta prior, uniform
by default, gives the predictable plug-in for each block, against the
pooled null mean.
"twoSided" runs both signs of the magnitude and averages their
cumulative e-processes.
logEValueVec2x2PropDiffGrow( ya, yb, na, nb, propDiffMin, alpha, alternative = c("twoSided", "greater", "less"), betaParameter = NULL, earlyStopping = FALSE, nWeight = 1000L )logEValueVec2x2PropDiffGrow( ya, yb, na, nb, propDiffMin, alpha, alternative = c("twoSided", "greater", "less"), betaParameter = NULL, earlyStopping = FALSE, nWeight = 1000L )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
propDiffMin |
numeric in (-1, 1) that defines the minimal relevant
difference in proportions |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
betaParameter |
|
earlyStopping |
logical, if |
nWeight |
integer > 0, the number of grid points. |
A numeric vector, element i the log e-value on blocks 1..i.
Given its total ya + yb, ya is Fisher's noncentral hypergeometric
with odds exp(logOdds) on group A; logOdds = 0 is the hypergeometric
null. Evaluated on the log scale, binomial coefficients plus logOdds
times the feasible count normalised by log-sum-exp, so extreme logOdds
give finite values where the plain density underflows to 0.
logLikelihoodFNCH(ya, yb, na, nb, logOdds)logLikelihoodFNCH(ya, yb, na, nb, logOdds)
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
logOdds |
numeric, one log odds ratio |
A numeric vector, one log density per block.
Helper function computes single component of the exact logrank e-value
logrankSingleEExact(obs0, obs1, y0, y1, thetaS, theta0 = 1, ...)logrankSingleEExact(obs0, obs1, y0, y1, thetaS, theta0 = 1, ...)
obs0 |
integer, number of observations in the control group. |
obs1 |
integer, number of observations in the treatment group. |
y0 |
integer, total number of participants in the control group. |
y1 |
integer, total number of participants in the treatment group. |
thetaS |
numeric > 0 represents the savi test defining (GROW) alternative
hypothesis obtained from |
theta0 |
numeric > 0 represents the null hypothesis. Default theta0=1. |
... |
further arguments to be passed to or from methods. |
Returns a list containing at least the following components:
Log likelihood of Fisher's hypergeometric at the null
Log likelihood of Fisher's hypergeometric at the alternative
Log likelihood of Fisher's hypergeometric at 1/alternative
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
#' y0Vector <- c(5, 4, 3, 3, 2, 1) y1Vector <- c(5, 5, 4, 2, 2, 0) obs0Vector <- c(1, 1, 0, 1, 0, 1) obs1Vector <- c(0, 0, 1, 0, 1, 0) logEValueGreater <- logEValueLess <- vector("numeric", length(y0Vector)) for (i in seq_along(y0Vector)) { tempResult <- logrankSingleEExact(obs0=obs0Vector[i], obs1=obs1Vector[i], y0=y0Vector[i], y1=y1Vector[i], thetaS=0.7, theta0=1) logEValueLess[i] <- tempResult[["logEValueLess"]] logEValueGreater[i] <- tempResult[["logEValueGreater"]] } eValueLess <- exp(sum(logEValueLess)) eValueLess #1.116161 eValueGreater <- exp(sum(logEValueGreater)) eValueGreater # 0.7665818 eValue <- 1/2*eValueLess + 1/2*eValueGreater eValue # 0.9413714#' y0Vector <- c(5, 4, 3, 3, 2, 1) y1Vector <- c(5, 5, 4, 2, 2, 0) obs0Vector <- c(1, 1, 0, 1, 0, 1) obs1Vector <- c(0, 0, 1, 0, 1, 0) logEValueGreater <- logEValueLess <- vector("numeric", length(y0Vector)) for (i in seq_along(y0Vector)) { tempResult <- logrankSingleEExact(obs0=obs0Vector[i], obs1=obs1Vector[i], y0=y0Vector[i], y1=y1Vector[i], thetaS=0.7, theta0=1) logEValueLess[i] <- tempResult[["logEValueLess"]] logEValueGreater[i] <- tempResult[["logEValueGreater"]] } eValueLess <- exp(sum(logEValueLess)) eValueLess #1.116161 eValueGreater <- exp(sum(logEValueGreater)) eValueGreater # 0.7665818 eValue <- 1/2*eValueLess + 1/2*eValueGreater eValue # 0.9413714
Helper function computes single component of the logrank statistic
logrankSingleZ(obs0, obs1, y0, y1, ...)logrankSingleZ(obs0, obs1, y0, y1, ...)
obs0 |
integer, number of observations in the control group |
obs1 |
integer, number of observations in the treatment group |
y0 |
integer, total number of participants in the control group |
y1 |
integer, total number of participants in the treatment group |
... |
further arguments to be passed to or from methods. |
Returns a list containing at least the following components:
observed minus expected.
hypergeometric variance.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
y0Vector <- c(6, 4, 4, 1, 0) y1Vector <- c(6, 6, 5, 2, 2) obs0Vector <- c(1, 0, 2, 1, 0) obs1Vector <- c(0, 1, 1, 0, 1) varVector <- oMinEVector <-y0Vector for (i in seq_along(y0Vector)) { tempResult <- logrankSingleZ(obs0=obs0Vector[i], obs1=obs1Vector[i], y0=y0Vector[i], y1=y1Vector[i]) oMinEVector[i] <- tempResult[["oMinE"]] varVector[i] <- tempResult[["v"]] } sum(oMinEVector)/sqrt(sum(varVector))y0Vector <- c(6, 4, 4, 1, 0) y1Vector <- c(6, 6, 5, 2, 2) obs0Vector <- c(1, 0, 2, 1, 0) obs1Vector <- c(0, 1, 1, 0, 1) varVector <- oMinEVector <-y0Vector for (i in seq_along(y0Vector)) { tempResult <- logrankSingleZ(obs0=obs0Vector[i], obs1=obs1Vector[i], y0=y0Vector[i], y1=y1Vector[i]) oMinEVector[i] <- tempResult[["oMinE"]] varVector[i] <- tempResult[["v"]] } sum(oMinEVector)/sqrt(sum(varVector))
Helper function to create running intersections
makeRunningIntersection(x, upper = TRUE)makeRunningIntersection(x, upper = TRUE)
x |
vector of numeric, representing a sequence of upper or lower bounds of a confidence sequence |
upper |
logic, by default |
a sequence of numerics representing the running minimum or maximum
makeRunningIntersection(c(6, -1, 3, 12))makeRunningIntersection(c(6, -1, 3, 12))
Checks and outputs a threshold for a minimal efficacy analysis
matchAlphaRelevanceWith( alphaRelevance, alpha = NULL, power = NULL, beta = NULL, betaDefault = 0.2 )matchAlphaRelevanceWith( alphaRelevance, alpha = NULL, power = NULL, beta = NULL, betaDefault = 0.2 )
alphaRelevance |
numeric > 0 and < 1, used to set the threshold of a minimal efficacy procedure |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
beta |
numeric > 0 and < 1, a tolerable type II error, used to set the threshold if alphaRelevance is not given |
betaDefault |
numeric > 0 and < 1 a default value (0.2) to run a minimal efficacy procedure |
a alphaRelevance threshold
matchAlphaRelevanceWith(0.3)matchAlphaRelevanceWith(0.3)
Based on the minimal clinically relevant effect size esMin, sigma (for z-tests), alternative and eType
matchEParameterWith( esMin, analysisType = c("z", "t", "logRank"), sigma = 1, alternative = c("twoSided", "greater", "less"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), parameter = NULL )matchEParameterWith( esMin, analysisType = c("z", "t", "logRank"), sigma = 1, alternative = c("twoSided", "greater", "less"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), parameter = NULL )
esMin |
numeric: meanDiffMin for z-tests, or deltaMin for t-tests |
analysisType |
character. Either "z", "t", or "logRank", currently. |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
the parameter, a numeric value
matchEParameterWith(0.4)matchEParameterWith(0.4)
Based on the parameter, sigma, alternative and eType
matchEsMinWith( parameter, analysisType = c("z", "t"), sigma = 1, alternative = c("twoSided", "greater", "less"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow") )matchEsMinWith( parameter, analysisType = c("z", "t"), sigma = 1, alternative = c("twoSided", "greater", "less"), eType = c("mom", "eGauss", "imom", "eCauchy", "grow") )
parameter |
numeric, an optional savi test defining parameter. Default set to |
analysisType |
character. Either "z", "t", or "logRank", currently. |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
the parameter, a numeric value
matchEsMinWith(parameter=0.4)matchEsMinWith(parameter=0.4)
Helper function to check whether the power or beta (redundant now) argument is given
matchPowerWith(power, beta = NULL)matchPowerWith(power, beta = NULL)
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
numeric representing power
matchPowerWith(0.8)matchPowerWith(0.8)
Based on the relevanceSize, meanDiffMin, alternative and eType
matchRelevanceParameterWith(relevanceSize, esMin, esTrue)matchRelevanceParameterWith(relevanceSize, esMin, esTrue)
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
esMin |
numeric: meanDiffMin for z-tests, or deltaMin for t-tests |
esTrue |
numeric. meanDiffTrue for z-tests, or deltaTrue for t-tests |
the parameter, a numeric value
matchRelevanceParameterWith(0.4)matchRelevanceParameterWith(0.4)
Plots Results of Simulations for Comparing Hyperparameters for Savi Tests of Two Proportions
## S3 method for class 'savi2x2Sim' plot(x, ...)## S3 method for class 'savi2x2Sim' plot(x, ...)
x |
a result object obtained through |
... |
further arguments to be passed to or from methods. |
Plot data, mainly called for side effects, the plot of simulation results.
priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18) priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1) simResult <- simulateTwoProportions( hyperparameterList = list(priorList1, priorList2, priorList3), alternativeRestriction = "none", alpha = 0.1, beta = 0.2, na = 1, nb = 1, deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3, M = 10 ) plot(simResult)priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18) priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1) simResult <- simulateTwoProportions( hyperparameterList = list(priorList1, priorList2, priorList3), alternativeRestriction = "none", alpha = 0.1, beta = 0.2, na = 1, nb = 1, deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3, M = 10 ) plot(simResult)
Plots the saviDesign object for designs with sample paths
## S3 method for class 'saviDesign' plot( x, main = NULL, xlab = NULL, ylab = NULL, xlim = NULL, ylim = NULL, maxNBins = 40, numSamplePaths = 100, wantStepLines = FALSE, wantQuantiles = NULL, breaks = NULL, lwd = 2, pch = 15, colQuant = "#AA0000", overColour = "#8FC6E3", overColourBorder = "#1F78B4E6", underColour = "#E7A72199", underColourBorder = "#BD8E17FF", continueColour = "#8CA252E6", continueColourBorder = "#637939E6", histInnerColour = col, col = overColour, border = overColourBorder, cex = 1.3, yLabPAdj = -1, wantNotStoppedHist = FALSE, wantNotStoppedSamplePaths = TRUE, wantLegend = TRUE, legendAdjRelevance = c(-0.1, 0.5, 1), legendAdj = c(0.2, 0.8), legendCexFactor = 0.85, histFewestAtTop = FALSE, density = NULL, wantTitle = TRUE, ... )## S3 method for class 'saviDesign' plot( x, main = NULL, xlab = NULL, ylab = NULL, xlim = NULL, ylim = NULL, maxNBins = 40, numSamplePaths = 100, wantStepLines = FALSE, wantQuantiles = NULL, breaks = NULL, lwd = 2, pch = 15, colQuant = "#AA0000", overColour = "#8FC6E3", overColourBorder = "#1F78B4E6", underColour = "#E7A72199", underColourBorder = "#BD8E17FF", continueColour = "#8CA252E6", continueColourBorder = "#637939E6", histInnerColour = col, col = overColour, border = overColourBorder, cex = 1.3, yLabPAdj = -1, wantNotStoppedHist = FALSE, wantNotStoppedSamplePaths = TRUE, wantLegend = TRUE, legendAdjRelevance = c(-0.1, 0.5, 1), legendAdj = c(0.2, 0.8), legendCexFactor = 0.85, histFewestAtTop = FALSE, density = NULL, wantTitle = TRUE, ... )
x |
designObj |
main |
character string for the title of plot |
xlab |
character string for the x-axis |
ylab |
character string for the y-axis |
xlim |
vector of length 2 specifying the range of the x-axis |
ylim |
vector of length 2 specifying the range of the y-axis |
maxNBins |
integer, maximum number of bins of the histogram |
numSamplePaths |
integer, number of sample paths to plot |
wantStepLines |
logical, if TRUE, then plot the sample paths as step functions (realistic). |
wantQuantiles |
a vector of numerics between zero and one representing the quantile levels |
border |
the color of the border around the bars. The default is to use blue |
breaks |
Break points of the histogram see |
lwd |
The line width, a positive number, defaulting to 2. |
pch |
An integer specifying a symbol. |
histInnerColour |
A colour to be used to fill the bars. |
col |
colour of the lines |
colQuant |
colour of the quantiles |
cex |
size of the labels and the quantile text |
... |
further arguments to be passed to or from methods. |
overColour |
colour of an e-value sequence that cross the upper boundary of 1/alpha |
overColourBorder |
colour for the border of a rectangle in the histogram corresponding to e-values that cross the upper boundary of 1/alpha |
underColour |
colour of an e-value sequence that cross the lower boundary of alphaRelevance |
underColourBorder |
colour for the border of a rectangle in the histogram corresponding to e-values that cross the lower boundary of alphaRelevance |
continueColour |
colour for lines of e-values that did not (yet) cross a threshold |
continueColourBorder |
colour for the border of a rectangle in the histogram corresponding to e-values that did not (yet) cross a threshold |
yLabPAdj |
numeric to adjust the position of the y-axis label |
wantNotStoppedHist |
logical, if |
wantNotStoppedSamplePaths |
logical, if |
wantLegend |
logical, if |
legendAdjRelevance |
vector of 3 numerics that allow for the adjustment of the percentages shown at the bottom of the plot, when relevance testing is on |
legendAdj |
vector of 2 numerics that allow for the adjustment of the percentages shown at the bottom of the plot |
legendCexFactor |
numeric, additional factor to control the text size of the additional information such as the percentages at the bottom of the plot |
histFewestAtTop |
logical, if TRUE, then fewest counts (of against the null or against minimal efficacy) at the top of the histogram |
density |
the density of shading lines, in lines per inch. The default value of NULL means that no shading lines are drawn. Non-positive values of density also inhibit the drawing of shading lines. |
wantTitle |
logical, if |
Nothing it only plots
plot(safeDesign): Deprecated version of plot.saviDesign
Plots the saviTest object for sequential analyses
## S3 method for class 'saviTest' plot( x, main = NULL, xlab = NULL, ylab = NULL, xlim = NULL, ylim = NULL, lwd = 3, cex = 1.3, fillPlot = NULL, switchNFill = 10000, logScale = NULL, switchNLog = 30, h0Colour = "darkgrey", overColour = "#8FC6E3", overColourBorder = "#1F78B4E6", underColour = "#E7A72199", underColourBorder = "#BD8E17FF", continueColour = "#8CA252E6", continueColourBorder = "#637939E6", col = overColour, border = overColourBorder, wantConfSeqPlot = FALSE, add = FALSE, density = NULL, angle = 45, xaxt = NULL, yaxt = NULL, fillOddEven = FALSE, runningIntersection = NULL, wantRelevance = NULL, ... )## S3 method for class 'saviTest' plot( x, main = NULL, xlab = NULL, ylab = NULL, xlim = NULL, ylim = NULL, lwd = 3, cex = 1.3, fillPlot = NULL, switchNFill = 10000, logScale = NULL, switchNLog = 30, h0Colour = "darkgrey", overColour = "#8FC6E3", overColourBorder = "#1F78B4E6", underColour = "#E7A72199", underColourBorder = "#BD8E17FF", continueColour = "#8CA252E6", continueColourBorder = "#637939E6", col = overColour, border = overColourBorder, wantConfSeqPlot = FALSE, add = FALSE, density = NULL, angle = 45, xaxt = NULL, yaxt = NULL, fillOddEven = FALSE, runningIntersection = NULL, wantRelevance = NULL, ... )
x |
designObj |
main |
character string for the title of plot |
xlab |
character string for the x-axis |
ylab |
character string for the y-axis |
xlim |
vector of length 2 specifying the range of the x-axis |
ylim |
vector of length 2 specifying the range of the y-axis |
lwd |
The line width, a positive number, defaulting to 2. |
cex |
size of the labels and the quantile text |
fillPlot |
logical, if TRUE then plot the confidence sequence with a background colour |
switchNFill |
integer, if |
logScale |
logical, if TRUE then plot on the logscale |
switchNLog |
integer, if |
h0Colour |
Colour to indicate the null hypothesis. |
col |
The colour for filling the anytime-valid confidence interval. |
border |
The colour to draw the border. |
wantConfSeqPlot |
logical, if |
add |
logical, default |
density |
the density of shading lines, in lines per inch.
The default value of |
angle |
the slope of shading lines, given as an angle in degrees (anti-clockwise). |
fillOddEven |
logical controlling the polygon shading mode: see
|
... |
further arguments to be passed to or from methods. |
overColour |
colour of an e-value sequence that cross the upper boundary of 1/alpha |
overColourBorder |
colour for the border of a rectangle in the histogram corresponding to e-values that cross the upper boundary of 1/alpha |
underColour |
colour of an e-value sequence that cross the lower boundary of alphaRelevance |
underColourBorder |
colour for the border of a rectangle in the histogram corresponding to e-values that cross the lower boundary of alphaRelevance |
continueColour |
colour for lines of e-values that did not (yet) cross a threshold |
continueColourBorder |
colour for the border of a rectangle in the histogram corresponding to e-values that did not (yet) cross a threshold |
xaxt |
default NULL. If "n" then suppresses plotting of the x-axis. |
yaxt |
default NULL. If "n" then suppresses plotting of the y-axis. |
runningIntersection |
logical, if |
wantRelevance |
logical, if |
Returns nothing just plots
plot(safeTest): Deprecated version of plot.saviTest
Plot bounds of a savi confidence sequence of the difference or log odds ratio for two proportions against the number of data blocks in two data streams ya and yb.
plotConfidenceSequenceTwoProportions( ya, yb, saviDesign, differenceMeasure = c("difference", "odds"), precision = 100, deltaStart = 0.001, deltaStop = 3, trueDifference = NA )plotConfidenceSequenceTwoProportions( ya, yb, saviDesign, differenceMeasure = c("difference", "odds"), precision = 100, deltaStart = 0.001, deltaStop = 3, trueDifference = NA )
ya |
positive observations/ events per data block in group a: a numeric with integer values
between (and including) 0 and |
yb |
positive observations/ events per data block in group b: a numeric with integer values
between (and including) 0 and |
saviDesign |
a savi test design for two proportions retrieved through |
differenceMeasure |
the difference measure to construct the confidence interval for: one of "difference" and "odds". |
precision |
precision of the grid to search over for the confidence sequence bounds. |
deltaStart |
for the odds difference measure: the (absolute value of the) smallest log odds ratio to assess for in- or exclusion in the confidence sequence. Default 0.001. |
deltaStop |
for the odds difference measure: the (absolute value of the) highest log odds ratio to assess for in- or exclusion in the confidence sequence. Default 3. |
trueDifference |
true difference or log odds ratio in groups A and B: added to the plot. |
no return value; called for its side effects, a plot of the confidence sequence.
set.seed(39413) ya <- rbinom(n = 30, size = 1, prob = 0.1) yb <- rbinom(n = 30, size = 1, prob = 0.8) balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 30) plotConfidenceSequenceTwoProportions(ya = ya, yb = yb, saviDesign = balancedSaviDesign, differenceMeasure = "difference", precision = 15, trueDifference = 0.7) #log odds ratio difference measure plotConfidenceSequenceTwoProportions(ya = ya, yb = yb, saviDesign = balancedSaviDesign, differenceMeasure = "odds", precision = 15, deltaStop = 5, trueDifference = log(36)) #switch ya and yb: observe negative log odds ratio in the data, plot mirrored in x-axis plotConfidenceSequenceTwoProportions(ya = yb, yb = ya, saviDesign = balancedSaviDesign, differenceMeasure = "odds", precision = 15, deltaStop = 5, trueDifference = -log(36))set.seed(39413) ya <- rbinom(n = 30, size = 1, prob = 0.1) yb <- rbinom(n = 30, size = 1, prob = 0.8) balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 30) plotConfidenceSequenceTwoProportions(ya = ya, yb = yb, saviDesign = balancedSaviDesign, differenceMeasure = "difference", precision = 15, trueDifference = 0.7) #log odds ratio difference measure plotConfidenceSequenceTwoProportions(ya = ya, yb = yb, saviDesign = balancedSaviDesign, differenceMeasure = "odds", precision = 15, deltaStop = 5, trueDifference = log(36)) #switch ya and yb: observe negative log odds ratio in the data, plot mirrored in x-axis plotConfidenceSequenceTwoProportions(ya = yb, yb = ya, saviDesign = balancedSaviDesign, differenceMeasure = "odds", precision = 15, deltaStop = 5, trueDifference = -log(36))
Helper function to display the histogram of stopping times.
plotHistogramDistributionStoppingTimes( saviSim, nPlan, deltaTrue, showOnlyNRejected = FALSE, nBin = 25L, ... )plotHistogramDistributionStoppingTimes( saviSim, nPlan, deltaTrue, showOnlyNRejected = FALSE, nBin = 25L, ... )
saviSim |
A saviSim object |
nPlan |
numeric > 0, the planned sample size(s). |
deltaTrue |
numeric, that represents the true underlying standardised effect size delta. |
showOnlyNRejected |
logical, when |
nBin |
numeric > 0, the minimum number of bins in the histogram. |
... |
further arguments to be passed to or from methods. |
a histogram object, and called for its side-effect to plot the histogram.
Independent Beta posteriors on thetaA and thetaB with prior shapes
betaParameter, updated with the blocks before each block: block i
uses blocks 1..i-1 only, and block 1 the prior.
predictiveThetas2x2(ya, yb, na, nb, betaParameter)predictiveThetas2x2(ya, yb, na, nb, betaParameter)
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
betaParameter |
|
A list of numeric vectors thetaA and thetaB, one value per block.
Prints Results of Simulations for Comparing Hyperparameters for Savi Tests of Two Proportions
## S3 method for class 'savi2x2Sim' print(x, ...)## S3 method for class 'savi2x2Sim' print(x, ...)
x |
a result object obtained through |
... |
further arguments to be passed to or from methods. |
The data frame with simulation results, called for side effects to pretty print the simulation results.
priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18) priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1) simResult <- simulateTwoProportions( hyperparameterList = list(priorList1, priorList2, priorList3), alternativeRestriction = "none", alpha = 0.1, beta = 0.2, na = 1, nb = 1, deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3, M = 10 )priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18) priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1) simResult <- simulateTwoProportions( hyperparameterList = list(priorList1, priorList2, priorList3), alternativeRestriction = "none", alpha = 0.1, beta = 0.2, na = 1, nb = 1, deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3, M = 10 )
Printing objects of class 'saviTest' modelled after print.power.htest().
## S3 method for class 'saviDesign' print(x, digits = getOption("digits"), prefix = "\t", ...)## S3 method for class 'saviDesign' print(x, digits = getOption("digits"), prefix = "\t", ...)
x |
a saviTest object. |
digits |
number of significant digits to be used. |
prefix |
string, passed to strwrap for displaying the method components. |
... |
further arguments to be passed to or from methods. |
No returned value, called for side effects.
print(safeDesign): Deprecated version of print.saviDesign
designSaviZ(meanDiffMin=0.5) designSaviT(deltaMin=0.5) designSaviLogrank(hrMin=0.7)designSaviZ(meanDiffMin=0.5) designSaviT(deltaMin=0.5) designSaviLogrank(hrMin=0.7)
Printing objects of class 'saviTest' modelled after print.htest().
## S3 method for class 'saviTest' print( x, digits = getOption("digits"), prefix = "\t", runningIntersection = NULL, ... )## S3 method for class 'saviTest' print( x, digits = getOption("digits"), prefix = "\t", runningIntersection = NULL, ... )
x |
a saviTest object. |
digits |
number of significant digits to be used. |
prefix |
string, passed to strwrap for displaying the method components. |
... |
further arguments to be passed to or from methods. |
runningIntersection |
logical, if |
No returned value, called for side effects.
print(safeTest): Deprecated version of print.saviTest
saviTTest(rnorm(19))saviTTest(rnorm(19))
Computes the p-value for the Z-test
pValueFromZStat(z, alternative = c("twoSided", "less", "greater"), ...) pValueZTest( x, y = NULL, paired = FALSE, ciValue = NULL, alpha = 0.05, sigma = 1, h0 = 0, alternative = c("twoSided", "greater", "less"), ... )pValueFromZStat(z, alternative = c("twoSided", "less", "greater"), ...) pValueZTest( x, y = NULL, paired = FALSE, ciValue = NULL, alpha = 0.05, sigma = 1, h0 = 0, alternative = c("twoSided", "greater", "less"), ... )
z |
numeric that represents the observed z-statistic. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
... |
further arguments to be passed to or from methods. |
x |
a (non-empty) numeric vector of data values. |
y |
an optional (non-empty) numeric vector of data values. |
paired |
a logical indicating whether you want the paired Z-test. |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
h0 |
numeric, representing the null value, default h0=0. |
pValueTest object
pValueZTest(rnorm(10))pValueZTest(rnorm(10))
Auxiliary function for sampling of the logrank simulations to return the integer 1 event per time.
returnOne()returnOne()
1
returnOne()returnOne()
Draws a number of occurrences in group 1 (treatment) out of obsTotal number of occurrences.
rLogrank(n = 1, y0, y1, obsTotal, theta)rLogrank(n = 1, y0, y1, obsTotal, theta)
n |
integer, number of observations to be sampled. |
y0 |
Size of the risk set of group 0 (Placebo). |
y1 |
Size of the risk set of group 1 (Treatment). |
obsTotal |
Total number of observations. |
theta |
Odds of group 1 over group 0 (treatment over placebo). |
integer representing the number of occurrences in group 1 out of obsTotal number of occurrences.
Muriel Felipe Perez-Ortiz and Alexander Ly
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
rLogrank(y0=360, y1=89, obsTotal=12, theta=3.14)rLogrank(y0=360, y1=89, obsTotal=12, theta=3.14)
A savi test to test whether there is a difference between two survival curves. This function builds on the Mantel-Cox version of the logrank test.
saviLogrankTest( formula, designObj = NULL, ciValue = NULL, data = NULL, survTime = NULL, group = NULL, pilot = FALSE, exact = TRUE, computeZ = TRUE, ... ) saviLogrankTestStat( z, nEvents, designObj, ciValue = NULL, dataNull = 1, sigma = 1 )saviLogrankTest( formula, designObj = NULL, ciValue = NULL, data = NULL, survTime = NULL, group = NULL, pilot = FALSE, exact = TRUE, computeZ = TRUE, ... ) saviLogrankTestStat( z, nEvents, designObj, ciValue = NULL, dataNull = 1, sigma = 1 )
formula |
a formula expression as for other survival models, of the form Surv(time, status) ~ groupingVariable,
see |
designObj |
a savi logrank design obtained from |
ciValue |
numeric, represents the ciValue-level of the confidence sequence. Default ciValue=NULL, and ciValue = 1 - alpha, where alpha is taken from the design object. |
data |
an optional data frame in which to interpret the variables occurring in survTime and group. |
survTime |
an optional survival time object of class 'Surv' created with |
group |
an optional factor, a grouping variable. Currently, only two levels allowed. Does not need specifying
if a formula is provided, therefore set to |
pilot |
a logical indicating whether a pilot study is run. If |
exact |
a logical indicating whether the exact savi logrank test needs to be performed based on
the hypergeometric likelihood. Default is |
computeZ |
logical. If |
... |
further arguments to be passed to or from methods. |
z |
numeric representing the observed logrank z statistic. |
nEvents |
numeric > 0, observed number of events. |
dataNull |
numeric > 0, the null hypothesis corresponding to the z statistics. By default dataNull = 1 representing equality of the hazard ratio. |
sigma |
numeric > 0, scaling in the data. |
Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:
the value of the summary, i.e., z-statistic or the e-value.
The number of observed events.
the e-value of the savi test.
An anytime-valid confidence sequence.
To be implemented: An estimate of the hazard ratio.
"logrank".
a character string giving the name(s) of the data.
an object of class "saviDesign" obtained from designSaviLogrank.
a list containing.the time of events, the progression of the risk sets and events.
the expression with which this function is called.
saviLogrankTestStat(): Savi Logrank Test based on Summary Statistic Z
All provided data (i.e., z-scores) are assumed to be centred on a hazard ratio = 1, thus, log(hr) = 0 ,
and the proper (e.g., hypergeometric) scaling is applied to the data, so sigma = 1. The null hypothesis
in the design object pertains to the population and is allowed to differ from log(theta) = 0.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
# Example taken from survival::survdiff designObj <- designSaviLogrank(hrMin=1/2) ovData <- survival::ovarian ovData$survTime <- survival::Surv(ovData$futime, ovData$fustat) saviLogrankTest(formula=survTime~ rx, data=ovData, designObj=designObj) saviLogrankTest(survTime=survTime, group=rx, data=ovData, designObj=designObj) # Examples taken from coin::logrank_test ## Example data (Callaert, 2003, Tab. 1) #' callaert <- data.frame( time = c(1, 1, 5, 6, 6, 6, 6, 2, 2, 2, 3, 4, 4, 5, 5), group = factor(rep(0:1, c(7, 8))) ) designObj <- designSaviLogrank(hrMin=1/2) saviLogrankTest(survival::Surv(callaert$time)~callaert$group, designObj = designObj) saviLogrankTest(survTime=survival::Surv(callaert$time), group=callaert$group, designObj = designObj) result <- saviLogrankTest(survTime=survival::Surv(callaert$time), group=callaert$group, designObj = designObj) result ## Sequentially # Greater eValueGreater <- exp(cumsum(result$sumStats$logEValueGreater)) # Less eValueLess <- exp(cumsum(result$sumStats$logEValueLess)) # twoSided eValueTwoSided <- 1/2*eValueGreater+1/2*eValueLess eValueTwoSided result$eValue ###### Example switching between savi exact and savi Gaussian logrank test designObj <- designSaviLogrank(0.8, alternative="less") dat <- safestats::generateSurvData(300, 300, 2, 0.0065, 0.0065*0.8, seed=1) survTime <- survival::Surv(dat$time, dat$status) resultE <- saviLogrankTest(survTime ~ dat$group, designObj = designObj) resultG <- saviLogrankTest(survTime ~ dat$group, designObj = designObj, exact=FALSE) resultE resultG ###### Example switching between savi exact and savi Gaussian logrank test other side designObj <- designSaviLogrank(1/0.8, alternative="greater") resultE <- saviLogrankTest(survTime ~ dat$group, designObj = designObj) resultG <- saviLogrankTest(survTime ~ dat$group, designObj = designObj, exact=FALSE) if (log(resultE$eValue) >= 0 && log(resultG$eValue) >= 0 ) stop("one-sided wrong")# Example taken from survival::survdiff designObj <- designSaviLogrank(hrMin=1/2) ovData <- survival::ovarian ovData$survTime <- survival::Surv(ovData$futime, ovData$fustat) saviLogrankTest(formula=survTime~ rx, data=ovData, designObj=designObj) saviLogrankTest(survTime=survTime, group=rx, data=ovData, designObj=designObj) # Examples taken from coin::logrank_test ## Example data (Callaert, 2003, Tab. 1) #' callaert <- data.frame( time = c(1, 1, 5, 6, 6, 6, 6, 2, 2, 2, 3, 4, 4, 5, 5), group = factor(rep(0:1, c(7, 8))) ) designObj <- designSaviLogrank(hrMin=1/2) saviLogrankTest(survival::Surv(callaert$time)~callaert$group, designObj = designObj) saviLogrankTest(survTime=survival::Surv(callaert$time), group=callaert$group, designObj = designObj) result <- saviLogrankTest(survTime=survival::Surv(callaert$time), group=callaert$group, designObj = designObj) result ## Sequentially # Greater eValueGreater <- exp(cumsum(result$sumStats$logEValueGreater)) # Less eValueLess <- exp(cumsum(result$sumStats$logEValueLess)) # twoSided eValueTwoSided <- 1/2*eValueGreater+1/2*eValueLess eValueTwoSided result$eValue ###### Example switching between savi exact and savi Gaussian logrank test designObj <- designSaviLogrank(0.8, alternative="less") dat <- safestats::generateSurvData(300, 300, 2, 0.0065, 0.0065*0.8, seed=1) survTime <- survival::Surv(dat$time, dat$status) resultE <- saviLogrankTest(survTime ~ dat$group, designObj = designObj) resultG <- saviLogrankTest(survTime ~ dat$group, designObj = designObj, exact=FALSE) resultE resultG ###### Example switching between savi exact and savi Gaussian logrank test other side designObj <- designSaviLogrank(1/0.8, alternative="greater") resultE <- saviLogrankTest(survTime ~ dat$group, designObj = designObj) resultG <- saviLogrankTest(survTime ~ dat$group, designObj = designObj, exact=FALSE) if (log(resultE$eValue) >= 0 && log(resultG$eValue) >= 0 ) stop("one-sided wrong")
Savi one- and two-sample T-tests. Takes as input vector(s) of data and a designObj from
designSaviT. The function is modelled after t.test().
saviTTest(x, ...) ## Default S3 method: saviTTest( x, y = NULL, designObj = NULL, paired = FALSE, varEqual = TRUE, ciValue = NULL, maxRoot = 10, sequential = NULL, tDensity = FALSE, nuMin = 2, wantCi = TRUE, ... ) ## S3 method for class 'formula' saviTTest(formula, data, subset, na.action, ...) savi.t.test( x, y = NULL, paired = FALSE, designObj = NULL, varEqual = TRUE, ciValue = NULL, ... )saviTTest(x, ...) ## Default S3 method: saviTTest( x, y = NULL, designObj = NULL, paired = FALSE, varEqual = TRUE, ciValue = NULL, maxRoot = 10, sequential = NULL, tDensity = FALSE, nuMin = 2, wantCi = TRUE, ... ) ## S3 method for class 'formula' saviTTest(formula, data, subset, na.action, ...) savi.t.test( x, y = NULL, paired = FALSE, designObj = NULL, varEqual = TRUE, ciValue = NULL, ... )
tDensity |
Uses the the representation of the savi T-test as the likelihood ratio of t densities. |
paired |
a logical, if |
... |
further arguments to be passed to or from methods. |
x |
a (non-empty) numeric vector of data values. |
y |
an optional (non-empty) numeric vector of data values. |
designObj |
an object obtained from |
varEqual |
a logical variable indicating whether to treat the two variances as being equal. Default varEqual=TRUE. |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
maxRoot |
Used to bound the candidate set of width of the confidence interval/ |
sequential |
a logical indicating whether a sequential analysis should be performed. |
formula |
a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs either 1 for a one-sample or paired test or a factor with two levels giving the corresponding groups. If lhs is of class "Pair" and rhs is 1, a paired test is done |
data |
an optional matrix or data frame (or similar:
see |
subset |
an optional vector specifying a subset of observations to be used. |
na.action |
a function which indicates what should
happen when the data contain |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
wantCi |
default TRUE |
Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:
the value of the t-statistic.
The realised sample size(s).
the realised e-value from the savi test.
A savi confidence interval for the mean (difference)
the estimated means or mean (difference) depending on whether it was a one-sample test or a two-sample test.
the standard error of the mean (difference), used as denominator in the t-statistic formula.
a character string giving the name(s) of the data.
an object of class "saviDesign" obtained from designSaviT().
the expression with which this function is called.
safeTTestStat(): Deprecated version of saviTTestStat
safeTTestStatNEffNu(): Deprecated version of saviTTestStatNEffNu
#nu >0 degrees of freedom
safeTTestStatNEffNuMom(): Deprecated version of saviTTestStatNEffNuMom
safeTTestStatTDensity(): Deprecated version of saviTTestStatTDensity
safeTTest(): Deprecated version of saviTTest
safeTTest.default(): Deprecated version of saviTTest.default
safeTTest.formula(): Deprecated version of saviTTest.formula
safe.t.test(): Deprecated version of savi.t.test
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.
Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.
## Examples with simulated data ---- set.seed(1) x <- rnorm(30, mean=0) y <- rnorm(40, mean=0) # Because no designObj is specified, a default # designObj is used with deltaMin = 1/2, # which can be thought of as a medium effect size # according to Cohen. res <- saviTTest(x=x, y=y) # By default sequential=TRUE, because length(x) <= 200. # This allows us to visualise the e-value as a function of # the n1 and associated n2, where the ratio of sample sizes, # ratio=n2/n1 is maintained. Here the e-value at n1=6 uses data # x[1:6] and y[1:ceil(ratio*6)] plot(res) # Plots the confidence sequence plot(res, wantConfSeqPlot=TRUE) # See ?designSaviT for more info # This designObj also allows for # evidence quantification that the # mean difference is minimal clinically # relevant, here, larger than # relevanceSize=meanDiffMin designObj <- designSaviT(deltaMin=0.7, alpha=0.05, alternative="twoSided", testType="twoSample", relevanceTest=TRUE) res <- saviTTest(x=x, y=y, designObj=designObj) plot(res) # Note that the e-value against relevance falls below alphaRelevance # We can reject the hypothesis that the effect is relevantly large, # larger than designObj$relevanceTestSim$parameter, after a sample size of min(which(res$eRelevanceVec <= designObj$relevanceTestSim$alpha)) set.seed(2) x <- rnorm(30, mean=0.6) y <- rnorm(40, mean=0) res <- saviTTest(x, y, designObj=designObj) plot(res) # We could have stopped sampling after min(which(res$eValueVec >= 1/designObj$alpha)) # the yellow curve crosses 1/alpha sooner, # but we do **not** compare eRelevance >= 1/alpha, only eRelevance <= alphaRelevance. ## Classical example: Student's sleep data ----- plot(extra ~ group, data = sleep) designObj <- designSaviT(deltaMin=0.6, testType="twoSample") ## Traditional interface with(sleep, saviTTest(extra[group == 1], extra[group == 2], designObj=designObj)) ## Formula interface saviTTest(extra ~ group, data = sleep, designObj=designObj) ## Formula interface to one-sample test designObj1 <- designSaviT(deltaMin=0.6, testType="oneSample", sigma=2) saviTTest(extra ~ 1, data = sleep, designObj=designObj1) ## Formula interface to paired test ## The sleep data are actually paired, so could have been in wide format: designObjPaired <- designSaviT(deltaMin=0.6, testType="paired", sigma=1.4) sleep2 <- reshape(sleep, direction = "wide", idvar = "ID", timevar = "group") saviTTest(Pair(extra.1, extra.2) ~ 1, data = sleep2, designObj=designObjPaired)## Examples with simulated data ---- set.seed(1) x <- rnorm(30, mean=0) y <- rnorm(40, mean=0) # Because no designObj is specified, a default # designObj is used with deltaMin = 1/2, # which can be thought of as a medium effect size # according to Cohen. res <- saviTTest(x=x, y=y) # By default sequential=TRUE, because length(x) <= 200. # This allows us to visualise the e-value as a function of # the n1 and associated n2, where the ratio of sample sizes, # ratio=n2/n1 is maintained. Here the e-value at n1=6 uses data # x[1:6] and y[1:ceil(ratio*6)] plot(res) # Plots the confidence sequence plot(res, wantConfSeqPlot=TRUE) # See ?designSaviT for more info # This designObj also allows for # evidence quantification that the # mean difference is minimal clinically # relevant, here, larger than # relevanceSize=meanDiffMin designObj <- designSaviT(deltaMin=0.7, alpha=0.05, alternative="twoSided", testType="twoSample", relevanceTest=TRUE) res <- saviTTest(x=x, y=y, designObj=designObj) plot(res) # Note that the e-value against relevance falls below alphaRelevance # We can reject the hypothesis that the effect is relevantly large, # larger than designObj$relevanceTestSim$parameter, after a sample size of min(which(res$eRelevanceVec <= designObj$relevanceTestSim$alpha)) set.seed(2) x <- rnorm(30, mean=0.6) y <- rnorm(40, mean=0) res <- saviTTest(x, y, designObj=designObj) plot(res) # We could have stopped sampling after min(which(res$eValueVec >= 1/designObj$alpha)) # the yellow curve crosses 1/alpha sooner, # but we do **not** compare eRelevance >= 1/alpha, only eRelevance <= alphaRelevance. ## Classical example: Student's sleep data ----- plot(extra ~ group, data = sleep) designObj <- designSaviT(deltaMin=0.6, testType="twoSample") ## Traditional interface with(sleep, saviTTest(extra[group == 1], extra[group == 2], designObj=designObj)) ## Formula interface saviTTest(extra ~ group, data = sleep, designObj=designObj) ## Formula interface to one-sample test designObj1 <- designSaviT(deltaMin=0.6, testType="oneSample", sigma=2) saviTTest(extra ~ 1, data = sleep, designObj=designObj1) ## Formula interface to paired test ## The sleep data are actually paired, so could have been in wide format: designObjPaired <- designSaviT(deltaMin=0.6, testType="paired", sigma=1.4) sleep2 <- reshape(sleep, direction = "wide", idvar = "ID", timevar = "group") saviTTest(Pair(extra.1, extra.2) ~ 1, data = sleep2, designObj=designObjPaired)
Perform a savi test for two proportions (a 2x2 contingency table test) with a
result object retrieved through the design function for planning an experiment to compare
two proportions in this package, designSaviTwoProportions().
saviTwoProportionsTest( ya, yb, designObj = NULL, wantConfidenceSequence = FALSE, ciValue = NULL, confidenceBoundGridPrecision = 20, logOddsConfidenceSearchBounds = c(0.01, 5), pilot = FALSE ) savi.prop.test( ya, yb, designObj = NULL, wantConfidenceSequence = FALSE, ciValue = NULL, confidenceBoundGridPrecision = 20, logOddsConfidenceSearchBounds = c(0.01, 5), pilot = FALSE )saviTwoProportionsTest( ya, yb, designObj = NULL, wantConfidenceSequence = FALSE, ciValue = NULL, confidenceBoundGridPrecision = 20, logOddsConfidenceSearchBounds = c(0.01, 5), pilot = FALSE ) savi.prop.test( ya, yb, designObj = NULL, wantConfidenceSequence = FALSE, ciValue = NULL, confidenceBoundGridPrecision = 20, logOddsConfidenceSearchBounds = c(0.01, 5), pilot = FALSE )
ya |
positive observations/ events per data block in group a: a numeric with integer values
between (and including) 0 and |
yb |
positive observations/ events per data block in group b: a numeric with integer values
between (and including) 0 and |
designObj |
a savi test design for two proportions retrieved through |
wantConfidenceSequence |
logical that can be set to true when the user wants a savi confidence sequence to be estimated. |
ciValue |
coverage of the savi confidence sequence; default |
confidenceBoundGridPrecision |
integer specifying the grid precision used to search for the confidence bounds. Default 20. |
logOddsConfidenceSearchBounds |
vector of to positive doubles specifying the upper and lower bound of the grid
to search over for finding the confidence bound for the logOddsRatio restriction. Default |
pilot |
logical that can be set to true when performing an exploratory analysis
without a |
Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:
The realised sample size(s).
the e-value of the savi test.
a character string giving the name(s) of the data.
an object of class "saviDesign" described in designSaviTwoProportions().
#balanced design yb <- c(1,0,1,1,1,0,1) ya <- c(1,0,1,0,0,0,1) saviDesign <- designSaviTwoProportions(na = 1, nb = 1, beta = 0.20, delta = 0.6, alternativeRestriction = "none", M = 1e1) saviTwoProportionsTest(ya = ya, yb = yb, designObj = saviDesign) #pilot saviTwoProportionsTest(ya = ya, yb = yb, pilot = TRUE) #unbalanced design yb <- c(1,0,1,1,1,0,1) ya <- c(2,2,1,2,0,2,2) saviDesign <- designSaviTwoProportions(na = 2, nb = 1, beta = 0.20, delta = 0.6, alternativeRestriction = "none", M = 1e1) saviTwoProportionsTest(ya = ya, yb = yb, designObj = saviDesign)#balanced design yb <- c(1,0,1,1,1,0,1) ya <- c(1,0,1,0,0,0,1) saviDesign <- designSaviTwoProportions(na = 1, nb = 1, beta = 0.20, delta = 0.6, alternativeRestriction = "none", M = 1e1) saviTwoProportionsTest(ya = ya, yb = yb, designObj = saviDesign) #pilot saviTwoProportionsTest(ya = ya, yb = yb, pilot = TRUE) #unbalanced design yb <- c(1,0,1,1,1,0,1) ya <- c(2,2,1,2,0,2,2) saviDesign <- designSaviTwoProportions(na = 2, nb = 1, beta = 0.20, delta = 0.6, alternativeRestriction = "none", M = 1e1) saviTwoProportionsTest(ya = ya, yb = yb, designObj = saviDesign)
Savi one- and two-sample Z-tests. Takes as input vector(s) of data and a designObj from
designSaviZ. The function is modelled after t.test().
saviZTest(x, ...) ## Default S3 method: saviZTest( x, y = NULL, paired = FALSE, designObj = NULL, ciValue = NULL, maxRoot = 10, sequential = NULL, ... ) ## S3 method for class 'formula' saviZTest(formula, data, subset, na.action, ...) savi.z.test(x, y = NULL, paired = FALSE, designObj = NULL, ...)saviZTest(x, ...) ## Default S3 method: saviZTest( x, y = NULL, paired = FALSE, designObj = NULL, ciValue = NULL, maxRoot = 10, sequential = NULL, ... ) ## S3 method for class 'formula' saviZTest(formula, data, subset, na.action, ...) savi.z.test(x, y = NULL, paired = FALSE, designObj = NULL, ...)
x |
a (non-empty) numeric vector of data values. |
... |
further arguments to be passed to or from methods. |
y |
an optional (non-empty) numeric vector of data values. |
paired |
a logical indicating whether you want the paired Z-test. |
designObj |
an object obtained from |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
maxRoot |
Used to bound the candidate set of width of the confidence interval, whenever eType="eCauchy" |
sequential |
a logical indicating whether a sequential analysis should be performed. |
formula |
a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs either 1 for a one-sample or paired test or a factor with two levels giving the corresponding groups. If lhs is of class "Pair" and rhs is 1, a paired test is done |
data |
an optional matrix or data frame (or similar:
see |
subset |
an optional vector specifying a subset of observations to be used. |
na.action |
a function which indicates what should
happen when the data contain |
Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:
the value of the test statistic. Here the z-statistic.
The realised sample size(s).
the e-value of the savi test.
a savi confidence interval for the mean (difference).
the estimated means or mean (difference) depending on whether it was a one-sample test or a two-sample test.
a character string giving the name(s) of the data.
an object of class "saviDesign" described in designSaviZ().
the expression with which this function is called.
safeZTest(): Deprecated version of saviZTest
safeZTest(default): Deprecated version of saviZTest.default
safe.z.test(): Deprecated version of saviZTest
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
## Examples with simulated data ---- set.seed(1) x <- rnorm(30, mean=0) y <- rnorm(40, mean=0) # Because no designObj is specified, a default # designObj is used with meanDiffMin = 1/2 and sigma=1, # which can be thought of as a medium effect size # according to Cohen. res <- saviZTest(x=x, y=y) # By default sequential=TRUE, because length(x) <= 200. # This allows us to visualise the e-value as a function of # the n1 and associated n2, where the ratio of sample sizes, # ratio=n2/n1 is maintained. Here the e-value at n1=6 uses data # x[1:6] and y[1:ceil(ratio*6)] plot(res) # Plots the confidence sequence plot(res, wantConfSeqPlot=TRUE) # See ?designSaviZ for more info # This designObj also allows for # evidence quantification that the # mean difference is minimal clinically # relevant, here, larger than # relevanceSize=meanDiffMin designObj <- designSaviZ(meanDiffMin=0.7, alpha=0.05, alternative="twoSided", testType="twoSample", relevanceTest=TRUE) res <- saviZTest(x=x, y=y, designObj=designObj) plot(res) # Note that the e-value against relevance falls below alphaRelevance # We can reject the hypothesis that the effect is relevantly larger, # larger than designObj$relevanceTestSim$parameter after a sample size of min(which(res$eRelevanceVec <= designObj$relevanceTestSim$alpha)) set.seed(2) x <- rnorm(30, mean=0.6) y <- rnorm(40, mean=0) res <- saviZTest(x, y, designObj=designObj) plot(res) # We could have stopped sampling after min(which(res$eValueVec >= 1/designObj$alpha)) # the yellow curve crosses 1/alpha sooner, # but we do **not** compare eRelevance >= 1/alpha, only eRelevance <= alphaRelevance ## Classical example: Student's sleep data ----- plot(extra ~ group, data = sleep) designObj <- designSaviZ(meanDiffMin=0.6, sigma=2, testType="twoSample") ## Traditional interface with(sleep, saviZTest(extra[group == 1], extra[group == 2], designObj=designObj)) ## Formula interface saviZTest(extra ~ group, data = sleep, designObj=designObj) ## Formula interface to one-sample test designObj1 <- designSaviZ(meanDiffMin=0.6, testType="oneSample", sigma=2) saviZTest(extra ~ 1, data = sleep, designObj=designObj1) ## Formula interface to paired test ## The sleep data are actually paired, so could have been in wide format: designObjPaired <- designSaviZ(meanDiffMin=0.6, testType="paired", sigma=1.4) sleep2 <- reshape(sleep, direction = "wide", idvar = "ID", timevar = "group") saviZTest(Pair(extra.1, extra.2) ~ 1, data = sleep2, designObj=designObjPaired)## Examples with simulated data ---- set.seed(1) x <- rnorm(30, mean=0) y <- rnorm(40, mean=0) # Because no designObj is specified, a default # designObj is used with meanDiffMin = 1/2 and sigma=1, # which can be thought of as a medium effect size # according to Cohen. res <- saviZTest(x=x, y=y) # By default sequential=TRUE, because length(x) <= 200. # This allows us to visualise the e-value as a function of # the n1 and associated n2, where the ratio of sample sizes, # ratio=n2/n1 is maintained. Here the e-value at n1=6 uses data # x[1:6] and y[1:ceil(ratio*6)] plot(res) # Plots the confidence sequence plot(res, wantConfSeqPlot=TRUE) # See ?designSaviZ for more info # This designObj also allows for # evidence quantification that the # mean difference is minimal clinically # relevant, here, larger than # relevanceSize=meanDiffMin designObj <- designSaviZ(meanDiffMin=0.7, alpha=0.05, alternative="twoSided", testType="twoSample", relevanceTest=TRUE) res <- saviZTest(x=x, y=y, designObj=designObj) plot(res) # Note that the e-value against relevance falls below alphaRelevance # We can reject the hypothesis that the effect is relevantly larger, # larger than designObj$relevanceTestSim$parameter after a sample size of min(which(res$eRelevanceVec <= designObj$relevanceTestSim$alpha)) set.seed(2) x <- rnorm(30, mean=0.6) y <- rnorm(40, mean=0) res <- saviZTest(x, y, designObj=designObj) plot(res) # We could have stopped sampling after min(which(res$eValueVec >= 1/designObj$alpha)) # the yellow curve crosses 1/alpha sooner, # but we do **not** compare eRelevance >= 1/alpha, only eRelevance <= alphaRelevance ## Classical example: Student's sleep data ----- plot(extra ~ group, data = sleep) designObj <- designSaviZ(meanDiffMin=0.6, sigma=2, testType="twoSample") ## Traditional interface with(sleep, saviZTest(extra[group == 1], extra[group == 2], designObj=designObj)) ## Formula interface saviZTest(extra ~ group, data = sleep, designObj=designObj) ## Formula interface to one-sample test designObj1 <- designSaviZ(meanDiffMin=0.6, testType="oneSample", sigma=2) saviZTest(extra ~ 1, data = sleep, designObj=designObj1) ## Formula interface to paired test ## The sleep data are actually paired, so could have been in wide format: designObjPaired <- designSaviZ(meanDiffMin=0.6, testType="paired", sigma=1.4) sleep2 <- reshape(sleep, direction = "wide", idvar = "ID", timevar = "group") saviZTest(Pair(extra.1, extra.2) ~ 1, data = sleep2, designObj=designObjPaired)
Computes e-values using the z-statistic and the sample sizes based on the test defining parameter phiS.
Evidence for practical equivalence requires the e-value to be small, i.e. smaller than alphaRelevance. If the alternative holds true, i.e. meanDiffTrue >= relevanceSize, then there is no more than alphaRelevance probability of ever seeing eRelevance <= alphaRelevance.
saviZTestStat( z, n1, n2 = NULL, parameter, alternative = c("twoSided", "less", "greater"), paired = FALSE, sigma = 1, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), ... ) saviRelevanceZStat( z, n1, n2 = NULL, parameter = NULL, alternative = c("twoSided", "less", "greater"), paired = FALSE, sigma = 1, eType = "grow", relevanceSize = NULL, ... )saviZTestStat( z, n1, n2 = NULL, parameter, alternative = c("twoSided", "less", "greater"), paired = FALSE, sigma = 1, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), ... ) saviRelevanceZStat( z, n1, n2 = NULL, parameter = NULL, alternative = c("twoSided", "less", "greater"), paired = FALSE, sigma = 1, eType = "grow", relevanceSize = NULL, ... )
z |
numeric that represents the observed z-statistic. |
n1 |
integer that represents the size in a one-sample Z-test, (n2= |
n2 |
an optional integer that specifies the size of the second sample. If it's left unspecified, thus,
|
parameter |
numeric > 0, the savi test defining parameter,
see |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
paired |
a logical indicating whether you want the paired Z-test. |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
... |
further arguments to be passed to or from methods. |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
Returns an e-value.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
saviZTestStat(z=3, n1=100, parameter=0.4, eType="grow") saviZTestStat(z=3, n1=100, parameter=0.4^2, eType="eGauss") saviZTestStat(z=3, n1=100, parameter=0.4, eType="eCauchy") # evidence for the alternative over minimal efficacy saviRelevanceZStat(z=3, n1=100, parameter=0.4) # evidence for minimal efficacy over the alternative saviRelevanceZStat(z=0.35, n1=100, parameter=0.4)saviZTestStat(z=3, n1=100, parameter=0.4, eType="grow") saviZTestStat(z=3, n1=100, parameter=0.4^2, eType="eGauss") saviZTestStat(z=3, n1=100, parameter=0.4, eType="eCauchy") # evidence for the alternative over minimal efficacy saviRelevanceZStat(z=3, n1=100, parameter=0.4) # evidence for minimal efficacy over the alternative saviRelevanceZStat(z=0.35, n1=100, parameter=0.4)
Simulate stopping times for the exact savi logrank test
sampleLogrankStoppingTimes( hrTrue, alpha = 0.05, alternative = c("twoSided", "less", "greater"), m0 = 50000L, m1 = 50000L, nSim = 1000L, groupSizePerTimeFunction = returnOne, seed = NULL, power = NULL, beta = NULL, relevanceTest = FALSE, relevanceSize = NULL, wantEValuesAtNMax = FALSE, wantSamplePaths = TRUE, wantSimData = TRUE, parameter = NULL, nMax = Inf, pb = TRUE, hrMin = NULL, ... )sampleLogrankStoppingTimes( hrTrue, alpha = 0.05, alternative = c("twoSided", "less", "greater"), m0 = 50000L, m1 = 50000L, nSim = 1000L, groupSizePerTimeFunction = returnOne, seed = NULL, power = NULL, beta = NULL, relevanceTest = FALSE, relevanceSize = NULL, wantEValuesAtNMax = FALSE, wantSamplePaths = TRUE, wantSimData = TRUE, parameter = NULL, nMax = Inf, pb = TRUE, hrMin = NULL, ... )
hrTrue |
numeric that defines the data generating hazard ratio with which data are sampled. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis, which must be one of "twoSided" (default),"greater" or "less". The alternative is pitted against the null hypothesis of equality of the survival distributions. More specifically, let lambda1 be the hazard rate of group 1 (i.e., placebo), and lambda2 the hazard ratio of group 2 (i.e., treatment), then the null hypothesis states that the hazard ratio theta = lambda2/lambda1 = 1. If alternative = "less", the null hypothesis is compared to theta < 1, thus, lambda2 < lambda1, that is, the hazard of group 2 (i.e., treatment) is less than that of group 1 (i.e., placebo), hence, the treatment is beneficial. If alternative = "greater", then the null hypothesis is compared to theta > 1, thus, lambda2 > lambda1, hence, harm. |
m0 |
Number of subjects in the control group 0/1 at the beginning of the trial, i.e., |
m1 |
Number of subjects in the treatment group 1/2 at the beginning of the trial, i.e., |
nSim |
integer > 0, the number of simulations needed to compute power or the number of events for the exact savi logrank test under continuous monitoring |
groupSizePerTimeFunction |
A function without parameters and integer output. This function provides the number
of events at each time step. For instance, if |
seed |
integer, seed number. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop for the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. This overrides the "beta" argument |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant standardised mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
wantEValuesAtNMax |
logical. If |
wantSamplePaths |
logical, if |
wantSimData |
logical. If |
parameter |
Numeric > 0, represents the savi tests defining thetaS. Default NULL so it's decided by the algorithm, typically, this equals hrMin, which corresponds to the GROW choice. |
nMax |
An integer. Once nEvents hits nMax the experiment terminates, if it didn't stop due to threshold crossing crossing already. Default set to Inf. |
pb |
logical, if |
hrMin |
numeric that defines the minimal relevant hazard ratio, the smallest hazard ratio that we want to detect. |
... |
further arguments to be passed to or from methods. |
a list with stoppingTimes and breakVector. Entries of breakVector are 0, 1. A 1 represents stopping due to exceeding nMax, and 0 due to 1/alpha threshold crossing, or running out of participants, which implies that the corresponding stopping time is Inf.
Muriel Felipe Perez-Ortiz and Alexander Ly
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.
sampleLogrankStoppingTimes(0.7, nSim=10, nMax=30)sampleLogrankStoppingTimes(0.7, nSim=10, nMax=30)
Simulate stopping times for the savi T-test
sampleStoppingTimesSaviT( deltaTrue, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), ratio = 1, deltaMin = NULL, parameter = NULL, lowN = 3L, nMax = 100000000L, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantEValuesAtNMax = FALSE, nuMin = 2, power = NULL, wantSamplePaths = TRUE, wantSimData = TRUE, sigma = 1, sigma2 = 1, pb = TRUE, seed = NULL, nSim = 1000L, relevanceTest = FALSE, relevanceSize = NULL, beta = NULL, alphaRelevance = NULL, ... )sampleStoppingTimesSaviT( deltaTrue, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), ratio = 1, deltaMin = NULL, parameter = NULL, lowN = 3L, nMax = 100000000L, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), wantEValuesAtNMax = FALSE, nuMin = 2, power = NULL, wantSamplePaths = TRUE, wantSimData = TRUE, sigma = 1, sigma2 = 1, pb = TRUE, seed = NULL, nSim = 1000L, relevanceTest = FALSE, relevanceSize = NULL, beta = NULL, alphaRelevance = NULL, ... )
deltaTrue |
numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
lowN |
integer, smallest sample size (of the first group). |
nMax |
integer > 0, maximum sample size of the (first) sample in each sample path. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantEValuesAtNMax |
logical. If |
wantSamplePaths |
logical. If |
wantSimData |
logical. If |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
deltaMin |
numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power). |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
sigma |
numeric > 0 representing the population standard deviation used for the test. |
sigma2 |
numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test |
relevanceTest |
logical, if |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
a list with stoppingTimes and breakVector. Entries of breakVector are 0, 1. A 1 represents stopping due to exceeding nMax, and 0 due to 1/alpha threshold crossing, which implies that in corresponding stopping time is Inf.
sampleStoppingTimesSafeT(): Deprecated version of sampleStoppingTimesSaviT
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20)sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20)
Simulate stopping times for the savi Z-test
sampleStoppingTimesSaviZ( meanDiffTrue, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), sigma = 1, kappa = sigma, ratio = 1, meanDiffMin = NULL, parameter = NULL, nMax = 1000L, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantEValuesAtNMax = FALSE, power = NULL, wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, relevanceTest = FALSE, relevanceSize = NULL, beta = NULL, alphaRelevance = NULL, ... )sampleStoppingTimesSaviZ( meanDiffTrue, alpha = 0.05, alternative = c("twoSided", "less", "greater"), testType = c("oneSample", "paired", "twoSample"), sigma = 1, kappa = sigma, ratio = 1, meanDiffMin = NULL, parameter = NULL, nMax = 1000L, eType = c("mom", "eGauss", "imom", "eCauchy", "grow"), wantEValuesAtNMax = FALSE, power = NULL, wantSamplePaths = TRUE, wantSimData = TRUE, pb = TRUE, seed = NULL, nSim = 1000L, relevanceTest = FALSE, relevanceSize = NULL, beta = NULL, alphaRelevance = NULL, ... )
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
testType |
either one of "oneSample", "paired", "twoSample". |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
kappa |
the true population standard deviation. Default kappa=sigma. |
ratio |
numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1. |
parameter |
numeric, an optional savi test defining parameter. Default set to |
nMax |
integer > 0, maximum sample size of the (first) sample in each sample path. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
wantEValuesAtNMax |
logical. If |
wantSamplePaths |
logical. If |
wantSimData |
logical. If |
pb |
logical, if |
seed |
integer, seed number. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
relevanceTest |
logical, if |
beta |
numerical in (0,1). Old parameter now replaced by the power parameter |
... |
further arguments to be passed to or from methods. |
meanDiffMin |
numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power). |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore. |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
alphaRelevance |
numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power. |
a list with stoppingTimes and breakVector. Entries of breakVector are 0, 1. A 1 represents stopping due to exceeding nMax, and 0 due to 1/alpha threshold crossing, which implies that in corresponding stopping time is Inf.
sampleStoppingTimesSafeZ(): Deprecated version of sampleStoppingTimesSaviZ
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
sampleStoppingTimesSaviZ(0.7, nSim=10, nMax=20)sampleStoppingTimesSaviZ(0.7, nSim=10, nMax=20)
Draws nSim streams of tables on the curve thetaA = thetaB + |propDiffMin| for "greater", thetaA = thetaB - |propDiffMin| for
"less", and on both curves for "twoSided", at the worst-case
baseline of each curve from solveWorstCaseTheta2x2PropDiff(), and
records when the plain grow e-process first crosses 1 / alpha.
nPlan is the worst power quantile of the stopping time over the
curves; with power = NULL it is skipped. There is no planning on
logOdds.
sampleStoppingTimesSavi2x2( propDiffMin, na, nb, power = NULL, alpha = 0.05, alternative = c("twoSided", "less", "greater"), eType = c("grow"), nSim = 1000L, nMax = 10000L, nBoot = 10000L, seed = NULL, wantEValuesAtNMax = FALSE, wantSamplePaths = FALSE, wantSimData = TRUE, pb = TRUE )sampleStoppingTimesSavi2x2( propDiffMin, na, nb, power = NULL, alpha = 0.05, alternative = c("twoSided", "less", "greater"), eType = c("grow"), nSim = 1000L, nMax = 10000L, nBoot = 10000L, seed = NULL, wantEValuesAtNMax = FALSE, wantSamplePaths = FALSE, wantSimData = TRUE, pb = TRUE )
propDiffMin |
numeric in (-1, 1) that defines the minimal relevant
difference in proportions |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
power |
numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
eType |
"grow", the only e-variable with planning. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi 2x2 test under continuous monitoring. |
nMax |
integer > 0, maximum number of data blocks in each sample path. |
nBoot |
integer > 0, the number of bootstrap samples; not used by
the sampler itself, see |
seed |
integer, seed number. Default seed=NULL yields seed=2026. |
wantEValuesAtNMax |
logical. If |
wantSamplePaths |
logical, if |
wantSimData |
logical. If |
pb |
logical, if |
a list with thetaA, thetaB (one per curve) and, one row per
curve, stoppingTimes and breakVector. Entries of breakVector are
0, 1. A 1 represents stopping due to exceeding nMax, which implies
that the corresponding stopping time is Inf, and 0 due to 1/alpha
threshold crossing. Further eValuesStopped, samplePaths (a list of
nSim x nMax sparse matrices, or NULL), n1Vector (the block
index), nPlan, and the baseline it was taken at, worstCaseThetaA,
worstCaseThetaB.
Tests thetaA = thetaB on a stream of 2x2 tables, one per block, with
the e-process chosen by designObj[["eType"]]. Both effects are A
minus B, propDiff = thetaA - thetaB and logOdds = logit(thetaA) - logit(thetaB), so "greater" means group A has the larger proportion,
as in stats::t.test(). Block 1 of every e-process is replaced by the
UMP conditional e-value of savi2x2TestStatUmp() when one exists at
level alpha; otherwise it keeps its plain factor, with a warning.
savi2x2TestStat( ya, yb, designObj = NULL, wantCi = FALSE, wantConfidenceSequence = FALSE, ciValue = NULL )savi2x2TestStat( ya, yb, designObj = NULL, wantCi = FALSE, wantConfidenceSequence = FALSE, ciValue = NULL )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
designObj |
an object obtained from |
wantCi |
default |
wantConfidenceSequence |
logical that can be set to true when the
user wants a savi confidence sequence to be estimated, one row per
block; takes precedence over |
ciValue |
numeric representing the confidence level. Default ciValue=NULL yields ciValue = 1 - alpha |
"eBeta" (propDiff): independent Beta posterior means of thetaA
and thetaB from the previous blocks, against the pooled null mean.
"grow" (propDiff or logOdds, whichever minimal effect the design
holds): the alternative restricted to the signed minimal effect;
"twoSided" averages the cumulative e-processes of both signs.
"eGauss" (logOdds): the conditional likelihood of Fisher's
noncentral hypergeometric distribution mixed under the design's Normal
prior on a logOdds grid, against the hypergeometric null. The
mixture may include the current block because it conditions on the
block's total.
Only "eBeta" and "eGauss" give a confidence interval or sequence,
each on its own effect.
Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:
The realised sample size(s):
c(na = sum(na), nb = sum(nb), nBlocks = length(ya)).
the e-value of the savi test on all blocks; reject when
it is at least 1 / alpha.
the realised e-values after each block, element i
on blocks 1..i.
the block index, used for plotting.
the estimated proportions c(thetaA, thetaB), pooled
over all blocks.
a savi confidence interval for the design's effect at
level ciValue on all blocks; NA when the set is empty, NULL for
"grow" or when no interval was requested.
with wantConfidenceSequence, the savi confidence
sequence as an nBlocks x 2 matrix, row i the interval on blocks
1..i; confSeq is its last row. With runningIntersection the
rows are nested and stay NA after the first empty row.
the confidence level used.
for "eBeta" only, list(betaA1, betaA2, betaB1, betaB2), the Beta prior updated with all blocks.
any of "twoSided", "greater", "less" copied from the design.
the null value copied from the design.
a character string giving the name(s) of the data.
an object of class "saviDesign" described in
designSavi2x2().
the expression with which this function is called.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Turner, R., Ly, A., & Grünwald, P. D. (2024). Generic e-variables for exact sequential k-sample tests that allow for optional stopping. Journal of Statistical Planning and inference 230, 106116, https://doi.org/10.1016/j.jspi.2023.106116.
Turner, R., & Grünwald, P. D. (2024). Exact anytime-valid confidence intervals for contingency tables and beyond. Statistics and Probability letters, 198, 109835, https://doi.org/10.1016/j.spl.2023.109835.
designObj <- designSavi2x2(na = 10, nb = 10, eType = "eBeta") ya <- c(8, 7, 9, 6) yb <- c(4, 5, 3, 5) savi2x2TestStat(ya, yb, designObj = designObj)designObj <- designSavi2x2(na = 10, nb = 10, eType = "eBeta") ya <- c(8, 7, 9, 6) yb <- c(4, 5, 3, 5) savi2x2TestStat(ya, yb, designObj = designObj)
The e-value of the conditional test on one table: the likelihood of
logLikelihoodFNCH() at the uniformly most powerful log odds ratio of
solveUmpLogOdds() over the hypergeometric null. "twoSided" averages
the two one-sided e-values. The test uses it in place of block 1's
plain factor.
savi2x2TestStatUmp( ya, yb, na, nb, alpha, alternative = c("twoSided", "greater", "less") )savi2x2TestStatUmp( ya, yb, na, nb, alpha, alternative = c("twoSided", "greater", "less") )
ya |
positive observations/ events per data block in group A: a
numeric with integer values between (and including) 0 and |
yb |
positive observations/ events per data block in group B: a
numeric with integer values between (and including) 0 and |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
A numeric, or NULL when no one-sided UMP log odds ratio exists
at level alpha.
To compute equality of all means in K groups. Still in beta stage.
saviAnovaOneB(formula, data, subset, na.action, parameter, sigma = NULL, ...)saviAnovaOneB(formula, data, subset, na.action, parameter, sigma = NULL, ...)
formula |
a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs a factor with at least two levels/groups. |
data |
a data frame (or similar: see model.frame()) containing the variables in the formula. By default the variables are taken from environment(formula). |
subset |
an optional vector specifying a subset of observations to be used. |
na.action |
a function which indicates what should happen when the data contain NAs. Defaults to getOption("na.action"). |
parameter |
numeric > 0 representing the scaling the in the multivatiate Gaussian prior |
sigma |
numeric > 0 if a population standard deviation is assumed
to be known. Default |
... |
further arguments to be passed to or from methods. |
a list
# Data plantData <- data.frame( treatment = factor(rep(c("control", "fertiliserA", "fertiliserB"), each = 10)), height = c( c(12, 14, 11, 13, 12, 15, 13, 11, 14, 12), # Control group c(18, 20, 19, 22, 21, 20, 17, 19, 21, 23), # Fertiliser A group c(15, 17, 16, 15, 18, 14, 16, 17, 15, 16) # Fertiliser B group ) ) saviAnovaOneB(height ~ treatment, data = plantData, parameter=1)# Data plantData <- data.frame( treatment = factor(rep(c("control", "fertiliserA", "fertiliserB"), each = 10)), height = c( c(12, 14, 11, 13, 12, 15, 13, 11, 14, 12), # Control group c(18, 20, 19, 22, 21, 20, 17, 19, 21, 23), # Fertiliser A group c(15, 17, 16, 15, 18, 14, 16, 17, 15, 16) # Fertiliser B group ) ) saviAnovaOneB(height ~ treatment, data = plantData, parameter=1)
To compute equality of all means in K groups. Still in beta stage.
saviAnovaStat(sumX, n, sumSquares = NULL, parameter, sigma = NULL)saviAnovaStat(sumX, n, sumSquares = NULL, parameter, sigma = NULL)
sumX |
vector of numerics representing the sum of the observations of the K groups |
n |
vector of integers representing the sample sizes of the K groups |
sumSquares |
vector of numerics representing the sum of the square of the observations of the K groups |
parameter |
numeric > 0 representing the scaling the in the multivatiate Gaussian prior |
sigma |
numeric > 0 if a population standard deviation is assumed
to be known. Default |
a list with the e-value
saviAnovaStat(1:6, 1:6, (1:6)^2, parameter=1)saviAnovaStat(1:6, 1:6, (1:6)^2, parameter=1)
A summary stats version of saviTTest() with
the data replaced by t, n1 and n2, and the design object by parameter
This is saviTTestStat() based on t-densities instead of
hypergeometric functions.
Evidence for practical equivalence requires the e-value to be small, i.e. smaller than alphaRelevance. If the alternative holds true, i.e. deltaTrue >= relevanceSize, then there is no more than alphaRelevance probability of ever seeing eRelevance <= alphaRelevance.
saviTTestStat( t, n1, n2 = NULL, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, nuMin = 2, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), ... ) saviTTestStatNEffNu( t, nEff, nu, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, nuMin = 2, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), ... ) saviTTestStatNEffNuMom( t, nEff, nu, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, k = 1, ... ) saviTTestStatNEffNuGrow( t, nEff, nu, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, ... ) saviTTestStatTDensity( t, parameter, nu, nEff, alternative = c("twoSided", "less", "greater"), paired = FALSE, ... ) saviRelevanceTStatNEffNu( t, nEff, nu, parameter = NULL, alternative = c("twoSided", "less", "greater"), eType = "grow", tDensity = FALSE, paired = FALSE, relevanceSize, nuMin = 2, ... )saviTTestStat( t, n1, n2 = NULL, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, nuMin = 2, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), ... ) saviTTestStatNEffNu( t, nEff, nu, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, nuMin = 2, eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"), ... ) saviTTestStatNEffNuMom( t, nEff, nu, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, k = 1, ... ) saviTTestStatNEffNuGrow( t, nEff, nu, parameter, alternative = c("twoSided", "less", "greater"), tDensity = FALSE, paired = FALSE, ... ) saviTTestStatTDensity( t, parameter, nu, nEff, alternative = c("twoSided", "less", "greater"), paired = FALSE, ... ) saviRelevanceTStatNEffNu( t, nEff, nu, parameter = NULL, alternative = c("twoSided", "less", "greater"), eType = "grow", tDensity = FALSE, paired = FALSE, relevanceSize, nuMin = 2, ... )
t |
numeric that represents the observed t-statistic. |
n1 |
integer that represents the size of the (first) sample.
Default n2= |
n2 |
an optional integer that represents the size of the second sample, which implies a two-sample t-statistic |
parameter |
numeric > 0, the savi test defining parameter,
see |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
tDensity |
Uses the the representation of the savi T-test as the likelihood ratio of t densities. |
paired |
a logical, if |
nuMin |
numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1. |
eType |
character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior. |
... |
further arguments to be passed to or from methods, but mainly to perform do.calls. |
nEff |
numeric > 0, the effective sample size. For one sample tests, this is just n. |
nu |
numeric > 0, the degrees of freedom. |
k |
the moment used for the non-local moment prior. Default 1 |
relevanceSize |
numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin) |
Returns a numeric that represent the e10, that is, the e-value in favour of the alternative over the null
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.
Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.
Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.
Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.
Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.
Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.
Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.
saviTTestStat(t=1, n1=100, parameter=0.4) saviTTestStat(t=3, n1=100, parameter=0.3) # evidence for the alternative over minimal efficacy saviRelevanceTStatNEffNu(t=3, nEff=100, nu=60, parameter=0.4) # evidence for minimal efficacy over the alternative saviRelevanceTStatNEffNu(t=0.35, nEff=100, nu=60, parameter=0.4)saviTTestStat(t=1, n1=100, parameter=0.4) saviTTestStat(t=3, n1=100, parameter=0.3) # evidence for the alternative over minimal efficacy saviRelevanceTStatNEffNu(t=3, nEff=100, nu=60, parameter=0.4) # evidence for minimal efficacy over the alternative saviRelevanceTStatNEffNu(t=0.35, nEff=100, nu=60, parameter=0.4)
Helper function to demonstrate the selective continuation of paired z-tests
selectivelyContinueZOrTTestData( designObj, deltaTrue = NULL, testName = c("Z-Test", "T-Test"), n1New, muGlobal, sigma, meanDiffTrue = NULL, nSim = 1000L, eValuesOld, eOverOld, trackCrossingOld, firstPassageTimeOld, eStoppedOld, seed = NULL )selectivelyContinueZOrTTestData( designObj, deltaTrue = NULL, testName = c("Z-Test", "T-Test"), n1New, muGlobal, sigma, meanDiffTrue = NULL, nSim = 1000L, eValuesOld, eOverOld, trackCrossingOld, firstPassageTimeOld, eStoppedOld, seed = NULL )
designObj |
an object obtained from |
deltaTrue |
numeric, the value of the true standardised effect size (test-relevant parameter).
This argument is used by |
testName |
character string either "Z-Test" or "T-Test". |
n1New |
Number of samples in the follow-up study |
muGlobal |
numeric, population grand mean |
sigma |
numeric > 0 representing the assumed population standard deviation used to scale the data. |
meanDiffTrue |
numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin. |
nSim |
integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring. |
eValuesOld |
Matrix of eValues of the previous studies with nSim number of rows |
eOverOld |
Vector of 0s and 1s, where 1 indicates that |
trackCrossingOld |
Vector of same number of columns as eValuesOld providing the number of studies that got rejected at the indeced time |
firstPassageTimeOld |
Vector indicating the first passage time. If Inf, then not yet passed the threshold of 1/alpha |
eStoppedOld |
The stopped e-value either at the time it crosses the threshold, or at the previous nPlan |
seed |
integer, seed number. |
a list with the new e-values amongst other
Helper function to determine the lower and upper bound for the search space for standardised meanDiffMin
setLowAndHighEsTrueZ( nEff, eType = "mom", alternative = "twoSided", lowEsTrue = 1e-07, highEsTrue = 0.001 )setLowAndHighEsTrueZ( nEff, eType = "mom", alternative = "twoSided", lowEsTrue = 1e-07, highEsTrue = 0.001 )
nEff |
numeric > 0, the effective sample size. |
eType |
character string, one of "eCauchy", "eGauss", "grow", "freq", "credibleInterval". This is somewhat a misnomer as "freq" and "credibleInterval" do not correspond to e-value tests. "eCauchy" yields an anytime-valid confidence interval based on a Cauchy mixture, whereas "eGauss" yields an anytime-valid confidence interval based on a Gaussian mixture. "grow" yields an anytime-valid confidence interval based on a mixture of point masses at the minimal clinically relevant mean difference. This confidence interval unfortunately does not shrink as the sample size tends to infinity. "freq" yields the standard unsafe frequentist confidence interval, which is not safe. "credibleInterval" yields the Bayesian credible interval based on a conjugate prior as is usual in Bayesian analysis. This interval is also not safe. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less". |
lowEsTrue |
numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
highEsTrue |
numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a. |
a list of low and high values for the parameter
setLowAndHighEsTrueZ(10)setLowAndHighEsTrueZ(10)
Sets 'safestats' Plot Options and Returns the Current Plot Options.
setSafeStatsPlotOptionsAndReturnOldOnes(...)setSafeStatsPlotOptionsAndReturnOldOnes(...)
... |
further arguments to be passed to or from methods. |
Returns a list with the user specified plot options.
oldPar <- setSafeStatsPlotOptionsAndReturnOldOnes() graphics::plot(1:10, 1:10) setPar <- graphics::par(oldPar)oldPar <- setSafeStatsPlotOptionsAndReturnOldOnes() graphics::plot(1:10, 1:10) setPar <- graphics::par(oldPar)
Simulate the coverage of a savi confidence sequence for differences between proportions for a given distribution and savi design.
simulateCoverageDifferenceTwoProportions( successProbabilityA, trueDelta, saviDesign, precision = 100, M = 1000, numberForSeed = NA )simulateCoverageDifferenceTwoProportions( successProbabilityA, trueDelta, saviDesign, precision = 100, M = 1000, numberForSeed = NA )
successProbabilityA |
probability of observing a success in group A. |
trueDelta |
difference in probability between group A and B. |
saviDesign |
a savi test design for two proportions retrieved through |
precision |
precision of the grid to search over for the confidence sequence bounds. Default 100. |
M |
number of simulations to carry out. Default 1000. |
numberForSeed |
number for seed to set, default NA. |
the proportion of simulations where the trueDelta was included in the confidence sequence.
balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 20) simulateCoverageDifferenceTwoProportions(successProbabilityA = 0.2, trueDelta = 0, saviDesign = balancedSaviDesign, M = 100, precision = 20, numberForSeed = 1082021)balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 20) simulateCoverageDifferenceTwoProportions(successProbabilityA = 0.2, trueDelta = 0, saviDesign = balancedSaviDesign, M = 100, precision = 20, numberForSeed = 1082021)
Simulate incorrect optional stopping with fisher's exact test's p-value as the stopping rule.
simulateIncorrectStoppingTimesFisher( thetaA, thetaB, alpha, na, nb, maxSimStoptime = 10000, M = 1000, numberForSeed = NULL )simulateIncorrectStoppingTimesFisher( thetaA, thetaB, alpha, na, nb, maxSimStoptime = 10000, M = 1000, numberForSeed = NULL )
thetaA |
Bernoulli distribution parameter in group A |
thetaB |
Bernoulli distribution parameter in group B |
alpha |
Significance level |
na |
number of observations in group a per data block |
nb |
number of observations in group b per data block |
maxSimStoptime |
maximal number of blocks to sample in each experiment |
M |
Number of simulations to carry out, default 1e3. |
numberForSeed |
number for seed to set, default NULL. |
list with stopping times and rejection decisions.
simulateIncorrectStoppingTimesFisher(thetaA = 0.3, thetaB = 0.3, alpha = 0.05, na = 1, nb = 1, M = 10, maxSimStoptime = 100, numberForSeed = 251)simulateIncorrectStoppingTimesFisher(thetaA = 0.3, thetaB = 0.3, alpha = 0.05, na = 1, nb = 1, M = 10, maxSimStoptime = 100, numberForSeed = 251)
Simulate an optional stopping scenario according to a savi design for two proportions
simulateOptionalStoppingScenarioTwoProportions(saviDesign, M, thetaA, thetaB)simulateOptionalStoppingScenarioTwoProportions(saviDesign, M, thetaA, thetaB)
saviDesign |
a 'saviDesign' object obtained through |
M |
integer, the number of data streams to sample. |
thetaA |
Bernoulli distribution parameter in group A |
thetaB |
Bernoulli distribution parameter in group B |
list with the simulation results of the savi test under optional stopping with the following components:
Proportion of sequences where H0 was rejected
Mean stopping time
Proportion of experiments stopped before nBlocksPlan was reached
Minimum stopping time
All achieved E values
All stopping times
Decisions on rejecting H0 for each M
Stopping times of experiments where H0 was rejected
balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 30) optionalStoppingSimulationResult <- simulateOptionalStoppingScenarioTwoProportions( saviDesign = balancedSaviDesign, M = 1e2, thetaA = 0.2, thetaB = 0.5 )balancedSaviDesign <- designSaviTwoProportions(na = 1, nb = 1, nBlocksPlan = 30) optionalStoppingSimulationResult <- simulateOptionalStoppingScenarioTwoProportions( saviDesign = balancedSaviDesign, M = 1e2, thetaA = 0.2, thetaB = 0.5 )
Simulates for a range of divergence parameter values (differences or log odds ratios) the worst-case stopping times (i.e., number of data blocks collected) and expected stopping times needed to achieve the desired power for each hyperparameter setting provided.
simulateTwoProportions( hyperparameterList, alternativeRestriction = c("none", "difference", "logOddsRatio"), deltaDesign = NULL, alpha, beta, na, nb, deltamax = 0.9, deltamin = 0.1, deltaGridSize = 8, M = 100, maxSimStoptime = 10000, thetaAgridSize = 8 )simulateTwoProportions( hyperparameterList, alternativeRestriction = c("none", "difference", "logOddsRatio"), deltaDesign = NULL, alpha, beta, na, nb, deltamax = 0.9, deltamin = 0.1, deltaGridSize = 8, M = 100, maxSimStoptime = 10000, thetaAgridSize = 8 )
hyperparameterList |
list object, its components hyperparameter lists with a format as described in |
alternativeRestriction |
a character string specifying an optional restriction on the alternative hypothesis; must be one of "none" (default), "difference" (difference group mean b minus group b) or "logOddsRatio" (the log odds ratio between group means b and a). |
deltaDesign |
optional; when using a restricted alternative, the value of the divergence measure used. Either a numeric between -1 and 1 for a restriction on difference, or a real for a restriction on the log odds ratio. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error control –independent on n– that the designed test has to adhere to. Note that it also defines the rejection rule e10 >= 1/alpha. |
beta |
numeric in (0, 1) that specifies the tolerable type II error control in the study. Necessary to calculate the worst case stopping time. |
na |
number of observations in group a per data block |
nb |
number of observations in group b per data block |
deltamax |
maximal effect size to calculate power for; between -1 and 1 for designs without restriction or a restriction on difference;
real number for a restriction on the log odds ratio. Default |
deltamin |
minimal effect size to calculate power for; between -1 and 1 for designs without restriction or a restriction on difference;
real number for a restriction on the log odds ratio. Default |
deltaGridSize |
numeric, positive integer: size of grid of delta values worst case and expected sample sizes are simulated for. |
M |
number of simulations used to estimate sample sizes. Default |
maxSimStoptime |
maximal stream length in simulations; when the e value does not reach the rejection threshold before the end of the stream,
the maximal stream length is returned as the stopping time. Default |
thetaAgridSize |
numeric, positive integer: size of the grid of probability distributions examined for each delta value to find the worst case sample size over. |
Returns an object of class "savi2x2Sim". An object of class "savi2x2Sim" is a list containing at least the following components:
A data frame containing simulation results with worst case and expected stopping times for each hyperparameter setting, for the specified or default range of effect sizes.
the significance threshold used in the simulations
the type-II error control used in the simulations
the value of restriction on the alternative hypothesis parameter space used for the E variables in the simulations
the type of restriction used for the E variables in the simulation
list of the hyperparameters tested in the simulation
priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18) priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1) simResult <- simulateTwoProportions( hyperparameterList = list(priorList1, priorList2, priorList3), alternativeRestriction = "none", alpha = 0.1, beta = 0.2, na = 1, nb = 1, deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3, M = 10 ) print(simResult) plot(simResult)priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10) priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18) priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1) simResult <- simulateTwoProportions( hyperparameterList = list(priorList1, priorList2, priorList3), alternativeRestriction = "none", alpha = 0.1, beta = 0.2, na = 1, nb = 1, deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3, M = 10 ) print(simResult) plot(simResult)
The null pair (nullThetaA, nullThetaA - propDiff) closest in KL
divergence to (thetaA, thetaB) for one block: the root of the
derivative of na KL(thetaA || nullThetaA) + nb KL(thetaB || nullThetaB).
solveRIPr2x2PropDiff(thetaA, thetaB, na, nb, propDiff, tol = 1e-12)solveRIPr2x2PropDiff(thetaA, thetaB, na, nb, propDiff, tol = 1e-12)
thetaA, thetaB
|
numeric in (0, 1), the alternative's proportions. |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
propDiff |
numeric in (-1, 1), the null difference |
tol |
numeric > 0, the tolerance of |
A numeric, nullThetaA.
Solves the conditional KL divergence of the alternative at logOdds
from logOddsNull equal to log(1 / alpha), searching up to
searchBound from logOddsNull in the direction of alternative.
solveUmpLogOdds( na, nb, totalSuccesses, alpha, alternative = c("greater", "less"), logOddsNull = 0, searchBound = 100 )solveUmpLogOdds( na, nb, totalSuccesses, alpha, alternative = c("greater", "less"), logOddsNull = 0, searchBound = 100 )
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
totalSuccesses |
nonnegative integer, the total successes |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
alternative |
|
logOddsNull |
numeric, the null log odds ratio. |
searchBound |
numeric > 0, the width of the search interval. |
A numeric, or NULL with a warning when the target is not
reached on the search interval.
The pair on the curve of each alternative at which the e-process grows slowest, so a block count planned there holds for every baseline. The expected log e-increment per block is the KL divergence of the truth from its pooled null projection,
R(theta) = nLow KL(theta || theta0) + nHigh KL(theta + d || theta0), theta0 = theta + nHigh d / (nLow + nHigh),
with theta the lower proportion, nLow its group size and nHigh the
size of the group at theta + d. Its minimiser is the root of
nLow logit(theta) + nHigh logit(theta + d) = (nLow + nHigh) logit(theta0),
summed over blocks when the sizes vary per block. Equal sizes give
exactly the midpoint (1 - d) / 2; in general the root is
(1 - d) / 2 + d (nHigh - nLow) / (6 (nLow + nHigh)) + O(d^3). The rate
ignores the first-block UMP factor, which moves the simulated worst case
when one group is very small.
solveWorstCaseTheta2x2PropDiff( propDiffMin, na, nb, alternative = c("twoSided", "greater", "less") )solveWorstCaseTheta2x2PropDiff( propDiffMin, na, nb, alternative = c("twoSided", "greater", "less") )
propDiffMin |
numeric in (-1, 1) that defines the minimal relevant
difference in proportions |
na |
number of observations in group A per data block, one value or one per block. |
nb |
number of observations in group B per data block, one value or one per block. |
alternative |
a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion. |
"greater": thetaA = thetaB + d, so thetaB is the lower proportion
and nLow = nb. "less": thetaA = thetaB - d, so thetaA is the
lower one and nLow = na. "twoSided" gives both, greater first.
A data frame with one row per curve and columns thetaA, thetaB.
Based on conjugate priors with a total of 6 hyperparameters.
subjectiveBfZStat( x1, s21, n1, x2, s22, n2, sigma = 1, a1 = 5, b1 = 2, a2 = -5, b2 = 2, a0 = 0, b0 = 1, log = FALSE )subjectiveBfZStat( x1, s21, n1, x2, s22, n2, sigma = 1, a1 = 5, b1 = 2, a2 = -5, b2 = 2, a0 = 0, b0 = 1, log = FALSE )
x1 |
numeric, sample mean of group 1 |
s21 |
numeric, sample variance of group 1 |
n1 |
integer sample size of group 1 |
x2 |
numeric, sample mean of group 2 |
s22 |
numeric, sample variance of group 2 |
n2 |
integer sample size of group 2 |
sigma |
numeric > 0, population standard deviation |
a1 |
numeric, prior mean of the population mean mu1 of group 1 |
b1 |
numeric > 0, prior standard deviation of the population mean mu1 of group 1 |
a2 |
numeric, prior mean of the population mean mu2 of group 2 |
b2 |
numeric > 0, prior standard deviation of the population mean mu2 of group 2 |
a0 |
numeric, prior mean of the overall population mean mu0 of both groups |
b0 |
numeric > 0, prior standard deviation of the population mean mu0 of both groups |
log |
logical, default FALSE, if TRUE then return logarithm of the subjective Bayes factor outcome |
numeric > 0 representing the subjective Bayes factor outcome in favour of the alternative over the null
subjectiveBfZStat(5.2, 2, 3, 3.4, 2, 12)subjectiveBfZStat(5.2, 2, 3, 3.4, 2, 12)
NA
The evaluation fails with NA by default, but it is also able to fail with other values.
tryOrFailWithNA(expr, value = NA_real_)tryOrFailWithNA(expr, value = NA_real_)
expr |
Expression to be evaluated. |
value |
Return value if there is an error, default is |
Returns the evaluation of the expression, or value if it doesn't work out.
Internal function to solve the smallest width of an eGauss t-test
tTestWidthDerivative(g, nEff, nu, alpha = 0.05)tTestWidthDerivative(g, nEff, nu, alpha = 0.05)
g |
prior variance of the eGauss t-test |
nEff |
numeric > 0, the effective sample size. For one sample tests, this is just n. |
nu |
numeric > 0, the degrees of freedom. |
alpha |
numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha. |
a number that should be zero when g is optimal
Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.
tTestWidthDerivative(1, 1, 1)tTestWidthDerivative(1, 1, 1)
Logarithmic marginal likelihood of the normal with conjugate priors
zMarg(n, x, s2, a = 0, b = 1, sigma = 1)zMarg(n, x, s2, a = 0, b = 1, sigma = 1)
n |
integer, sample size |
x |
numeric, sample mean |
s2 |
numeric > 0, sample variance |
a |
numeric, prior mean of the population mean mu |
b |
numeric > 0, prior standard deviation of the population mean mu |
sigma |
numeric > 0, population standard deviation |
numeric, the logarithm of the marginal likelihood
zMarg(12, 0.3, 2)zMarg(12, 0.3, 2)