Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
86,887 characters · 38 sections · 63 citation commands
Optimal tests following sequential experiments
Recent years have seen tremendous advances in the theory and application of sequential/adaptive experiments. Such experiments are now used being in a wide variety of fields, ranging from online advertising russo2017tutorial, to dynamic pricing ferreira2018online, drug discovery wassmer2016group, public health athey2021shared, and economic interventions kasy2019adaptive. Compared to traditional randomized trials, these experiments allow one to target and achieve a more efficient balance of welfare, ethical, and economic considerations. In fact, starting from the Critical Path Initiative in 2006, the FDA has actively promoted the use of sequential designs in clinical trials for reducing trial costs and risks for participants guidance2018adaptive. For instance, group-sequential designs, wherein researchers conduct interim analyses at predetermined stages of the experiment, are now routinely used in clinical trials. If the analysis suggests a significant positive or negative effect from the treatment, the trial may be stopped early. Other examples of sequential experiments include bandit experiments lattimore2020bandit, best-arm identification russo2016information and costly sampling adusumilli2022sample, among many others.
Although hypothesis testing is not always the primary goal of sequential experiments, one may still desire to conduct a hypothesis test after the experiment is completed. For example, a pharmaceutical company may conduct an adaptive trial for drug testing with the explicit goal of maximizing welfare or minimizing costs, but may nevertheless be required to test the null hypothesis of a zero average treatment effect for the drug after the trial. Despite the practical importance of such inferential methods, there are currently few results characterizing optimal tests, or even identifying which sample statistics to use when conducting tests after sequential experiments. This paper aims to fill this gap.
To this end, we follow the standard approach in econometrics and statistics (see, e.g., van2000asymptotic) of studying the properties of various candidate tests by characterizing their power against local alternatives, also known as Pitman alternatives. These are alternatives that converge to the null at the parametric, i.e., $1/\sqrt{n}$ rate, leading to non-trivial asymptotic power. Here, $n$ is typically the sample size, although it can have other interpretations in experiments which are open-ended, see Section (ref) for a discussion. The main finding of this paper is that the asymptotic power function of any test can be matched by that of a test in a limit experiment where one observes a Gaussian process for each treatment, and the aim is to conduct inference on the drifts of the Gaussian processes.
As a by-product of this equivalence, we show that the power function of any candidate test (which may employ additional information beyond the sufficient statistics) can be matched asymptotically by one that only depends on a finite set of sufficient statistics. In the most general scenario, the sufficient statistics are the number of times each treatment has been sampled by the end of the experiment, along with final value of the score (for parametric models) or efficient influence function (for non-parametric models) process for each treatment. However, even these statistics can be further reduced under additional assumptions on the sampling and stopping rules. Our results thus show that a substantial dimension reduction is possible, and only a few statistics are relevant for conducting tests.
Furthermore, we characterize the optimal tests in the limit experiment. We then show that finite sample analogues of these are asymptotically optimal under the original sequential experiment. Our results can also be used to compute the power envelope, i.e., an upper bound on the asymptotic power function of any test. Although a uniformly most powerful test in the limit experiment may not always exist, some positive results are obtained for testing linear combinations under unbiasedness or $\alpha$-spending restrictions. Alternatively, one may impose less stringent criteria for optimality, like weighted average power, and we show how to compute optimal tests under such criteria as well.
We provide two new asymptotic representation theorems (ARTs) for formalizing the equivalence of tests between the original and limit experiments. The first applies to `stopping-time experiments', where the sampling rule is fixed beforehand but the stopping rule (which describes when the experiment is to be terminated) is fully adaptive (i.e., it can be updated after every new observation). Our second ART allows for the sampling rule to be adaptive as well, but we require the sampling and stopping decision to be updated only a finite number of times, after observing the data in batches. While constraining attention to batched experiments is undoubtedly a limitation, practical considerations often necessitate conducting sequential experiments in batches anyway. Also, as shown in adusumilli2021risk, any fully adaptive experiment can be approximated by a batched experiment with a sufficiently large number of batches. Our second ART builds on, and extends, the recent work of hirano2023asymptotic on asymptotic representations. We refer to Sections (ref) and (ref) for a detailed comparison.
Importantly, our framework covers both parametric and non-parametric settings. Finally, we apply our results to three important examples of sequential experiments: costly sampling, group sequential trials and bandit experiments, and suggest new inferential procedures for these experiments that are asymptotically optimal under different scenarios.
Despite the vast amount of work on the development of sequential learning algorithms, the literature on inference following the use of such algorithms is relatively sparse. One approach gaining some popularity in computer-science is called `any-time inference'. Here, one seeks to construct tests and confidence intervals that are correctly sized no matter how, or when, the experiment is stopped. We refer to ramdas2022game for a survey and to grunwald2020safe, howard2021time, johari2022always for some recent contributions. The uniform-in-time size constraint is a stringent requirement, and this comes at the expense of lower power than could be achieved otherwise. By contrast, our focus in this paper is on classical notions of testing, where size control is only achieved when the experimental protocol, i.e., the specific sampling rule and stopping time, is followed exactly. In essence, this requires the decision maker to pre-register the experiment and fully commit to the protocol. We believe this is valid assumption in most applications; adaptive experiments are usually constructed with the explicit goal of welfare maximization, so there is little incentive to deviate from the protocol as long as the preferences of the experimenter and the end-user of the experiment are aligned (e.g., in the case of online marketplaces they would be the same entity). In other situations, pre-registration of the experimental design is usually mandatory, see, e.g., the FDA guidance on sequential designs guidance2018adaptive.
There are other recent papers which propose inferential methods under the `classical' hypothesis-testing framework. zhang2020inference and hadad2021confidence suggest asymptotically normal tests for some specific classes of sequential experiments. These tests are based on re-weighing the observations. There are also a number of methods for group sequential and linear boundary designs commonly used in clinical trials, see hall2013analysis for a review. However, it is not clear if any of them are optimal even within their specific use cases.
Finally, in prior and closely related work to our own, hirano2023asymptotic obtain an Asymptotic Representation Theorem (ART) for batched sequential experiments that is different from ours and apply this to testing. The ART of hirano2023asymptotic is a lot more general than our own, e.g., it can be used to determine optimal conditional tests given outcomes from previous stages. However, this generality comes at a price as the state variables increase linearly with the number of batches. Here, we build on and extend these results to show that only a fixed number of sufficient statistics are needed to match the unconditional asymptotic power of any test, irrespective of the number of batches (our results also apply to asymptotic power conditional on stopping times). We also derive a number of additional results that are new to this literature: First, our ART for stopping-time experiments applies to fully adaptive experiments (this result is not based on hirano2023asymptotic; rather, it makes use of a representation theorem for stopping times due to LeCam1979). Second, our analysis covers non-parametric models, which is important for applications. Third, we characterize the properties of optimal tests in a number of different scenarios, e.g., for testing linear combinations of parameters, or under unbiased and $\alpha$-spending requirements. This is useful as UMP tests do not generally exist otherwise.
As noted earlier, this paper employs the local asymptotic power criterion to rank tests. This criterion naturally leads to `diffusion asymptotics', where the limit experiment consists of Gaussian diffusions. Diffusion asymptotics were first introduced by wager2021diffusion and fan2021diffusion to study the properties of a class of sequential algorithms. In previous work adusumilli2021risk, this author demonstrated some asymptotic equivalence results for comparing the Bayes and minimax risk of bandit experiments. Here, we apply the techniques devised in those papers to study inference.
Before describing our procedures, it can be instructive to consider some examples of sequential experiments.
Consider a sequential experiment in which sampling is costly, and the aim is to select the best of two possible treatments. Previous work by this author (adusumilli2022sample) showed that the minimax optimal strategy in this setting involves a fixed sampling rule (the Neyman allocation) and stopping when the average difference in treatment outcomes multiplied by the number of observations exceeds a specific threshold. In fact, the stopping rule here has the same form as the SPRT procedure of wald1947sequential, even though the latter is motivated by very different considerations. SPRT is itself a special case of `fully sequential linear boundary designs', as discussed, e.g., in whitehead1997design. Typically these procedures recommend sampling the two treatments in equal proportions instead of the Neyman allocation. In Section (ref), we show that for `horizontal fully sequential boundary designs' with any fixed sampling rule (including, but not restricted to, the Neyman allocation), the most powerful unbiased test for treatment effects depends only on the stopping time and rejects when it is below a specific threshold.
In many applications, it is not feasible to employ continuous-time monitoring designs that update the decision rule after each observation. Instead, one may wish to stop the experiment only at a limited number of pre-specified times. Such designs are known as group-sequential trials, see wassmer2016group for a textbook treatment. Recently, these experiments have become very popular for conducting clinical trials; they have been used, e.g., to test the efficacy of Coronavirus vaccines zaks2020phase. While a number of methods have been proposed for inference following these experiments, as reviewed, e.g., in hall2013analysis, it is not clear which, if any, are optimal. In Section (ref), we derive optimal non-parametric tests and confidence intervals for such designs under an $\alpha$-spending size criterion (see, Section (ref)).
In the previous two examples, the decision maker could choose when to end the experiment, but the sampling strategy was fixed beforehand. In many experiments however, the sampling rule can also be modified based on the information revealed from past data. Bandit experiments are a canonical example of these. Previously, hirano2023asymptotic derived asymptotic power envelopes for any test following batched parametric bandit experiments. In this paper, we refine the results of hirano2023asymptotic further by showing that only a finite number of sufficient statistics are needed for testing, irrespective of the number of batches. Our results apply to non-parametric models as well.
In this section we study the asymptotic properties of tests for parametric stopping-time experiments, i.e., sequential experiments that involve a pre-determined stopping time.
Consider a decision-maker (DM) who wishes to conduct an experiment involving some outcome variable $Y$. Before starting the experiment, the DM registers a stopping time, $\hat{\tau}$, that describes the eventual sample size in multiples of $n$ observations (see below for the interpretation of $n$). The choice of $\hat{\tau}$ may involve a balancing a number of considerations such as costs, ethics, welfare etc. Here, we abstract away from these issues and take $\hat{\tau}$ as given. In the course of the experiment, the DM observes a sequence of outcomes $Y_{1},Y_{2},\dots$ . The experiment ends in accordance with $\hat{\tau}$, which we assume to be adapted to the filtration generated by the outcome observations. Let $P_{\theta}$ denote a parametric model for the outcomes. Our interest in this section is in testing $H_{0}:\theta=\Theta_{0}$ vs $H_{1}:\theta\in\Theta_{1}$ where $\Theta_{0}\cap\Theta_{1}=\emptyset$. Let $\theta_{0}\in\Theta_{0}$ denote some reference parameter in the null set.
There are two notions of asymptotics one could employ in this setting, and consequently, two different interpretations of $n$. In many settings, e.g., group sequential trials, there is a limit on the maximum number of observations that can be collected; this limit is pre-specified and we take it to be $n$. Consequently, in these experiments, $\hat{\tau}\in[0,1]$. Alternatively, we may have open-ended experiments where the stopping time is determined by balancing the benefit of experimentation with the cost for sampling each additional unit of observation. In this case, we employ small-cost asymptotics and $n$ then indexes the rate at which the sampling costs go to $0$ (alternatively, we can relate $n$ to the population size in the implementation phase following the experiment, see adusumilli2022sample). The results in this section apply to both asymptotic regimes.
Let $\varphi_{n}\in[0,1]$ denote a candidate test. It is required to be measurable with respect to $\sigma\{Y_{1},\dots,Y_{\left\lfloor n\tau\right\rfloor }\}$. Now, it is fairly straightforward to construct tests that have power 1 against any fixed alternative as $n\to\infty$. Consequently, to obtain a more fine-grained characterization of tests, we consider their performance against local perturbations of the form $\{\theta_{0}+h/\sqrt{n};h\in\mathbb{R}^{d}\}$. Denote $P_{h}:=P_{\theta_{0}^ {}+h/\sqrt{n}}$ and let $\mathbb{E}_{h}^{(a)}[\cdot]$ denote its corresponding expectation. Also, let $\nu$ denote a dominating measure for $\{P_{\theta}:\theta\in\mathbb{R}\}$, and set $p_{\theta}:=dP_{\theta}/d\nu$. We impose the following regularity conditions on the family $P_{\theta}$, and the stopping time $\hat{\tau}$:
\begin{asm1} The class $\{P_{\theta}:\theta\in\mathbb{R}^{d}\}$ is differentiable in quadratic mean around $\theta_{0}$, i.e., there exists a score function $\psi(\cdot)$ such that for each $h\in\mathbb{R}^{d},$
\end{asm1}
\begin{asm2} There exists $T<\infty$ independent of $n$ such that $\hat{\tau}\le T$.\end{asm2}
Both assumptions are fairly innocuous. As noted previously, in many examples we already have $\tau\le1$.
Let $P_{nt,h}$ denote the joint probability measure over the iid sequence of outcomes $Y_{1},\dots,Y_{nt}$ and take $\mathbb{E}_{nt,\bm{h}}[\cdot]$ to be its corresponding expectation. Define the (standardized) score process $x_{n}(t)$ as \[ x_{n}(t)=\frac{I^{-1/2}}{\sqrt{n}}\sum_{i=1}^{\left\lfloor nt\right\rfloor }\psi(Y_{i}), \] where $I:=\mathbb{E}_{0}[\psi(Y_{i})\psi(Y_{i})^{\intercal}]$ is the information matrix. It is well known, see e.g., van2000asymptotic, that quadratic mean differentiability implies $\mathbb{E}_{nT,0}[\psi(Y_{i})]=0$ and that $I$ exists. Then, by a functional central limit theorem,
Here, and in what follows, $W(\cdot)$ denotes the standard $d$-dimensional Brownian motion. Assumption 1 also implies the important property of Sequential Local Asymptotic Normality (SLAN; adusumilli2021risk): for any given $h\in\mathbb{R}^{d}$,
The above states that the likelihood ratio admits a quadratic approximation uniformly over all $t$.
In what follows, take $U$ to be a $\textrm{Uniform}[0,1]$ random variable that is independent of the process $x(\cdot)$, and define $\mathcal{F}_{t}:=\sigma\{x(s),U;s\le t\}$ to be the filtration generated by $U$ and the stochastic process $x(\cdot)$ until time $t$.
Consider a limit experiment where one observes $U$ and a Gaussian diffusion $x(t):=I^{1/2}ht+W(t)$ with some unknown $h$, and constructs a test statistic $\varphi$ based on knowledge only of (i) an $\mathcal{F}_{t}$-adapted stopping time $\tau$ that is the limiting version of $\hat{\tau}$ (in a sense made precise below); and (ii) the stopped process $x(\tau)$. Let $\mathbb{P}_{h}$ denote the induced probability over the sample paths of $x(\cdot)$ given $h$, and $\mathbb{E}_{h}[\cdot]$ its corresponding expectation. The following theorem relates the original testing problem to the one in such a limit experiment:
The first part of Theorem (ref) is essentially due to LeCam1979.
To the best of our knowledge, the second part of Theorem (ref) is new. Previously, LeCam1979 showed that for $\{P_{\theta}\}$ in the exponential family of distributions, \[ \ln\frac{dP_{n\hat{\tau}.,h}}{dP_{n\hat{\tau},0}}\left({\bf y}_{n\hat{\tau}}\right)\xrightarrow[P_{nT,0}]{d}h^{\intercal}I^{1/2}x(\tau)-\frac{\tau}{2}h^{\intercal}Ih. \] Here, we extend the above to general families of distributions satisfying Assumption 1. We then derive an asymptotic representation theorem for $\varphi_{n}$ as a consequence of this result.
Note that in the second part of Theorem (ref), $\tau$ is taken as given (this mirrors how $\hat{\tau}$ is taken as given in the context of the original experiment). It is chosen so that the first part of the theorem is satisfied. In order to derive optimal tests, one would need to know the joint distribution of $\tau,x(\tau)$. Unfortunately, the first part of Theorem (ref) does not provide a characterization of $\tau$; it only asserts that such a stopping time must exist. Fortunately, in practice, most stopping times are functions, $\hat{\tau}=\tau(x_{n}(\cdot))$, of the score process, e.g., the optimal stopping time under costly sampling is given by $\hat{\tau}=\inf\{t:\vert x_{n}(t)\vert\ge\gamma\}$. Indeed, previous work by this author (adusumilli2022sample) and others has shown that if the stopping time is to be chosen according some notion of Bayes or minimax risk, then it is sufficient to restrict attention to stopping times that depend only on $x_{n}(\cdot)$. In such cases, the continuous mapping theorem allows us to determine $\tau$ as $\tau=\tau(x(\cdot))$.
The simplest hypothesis testing problem in the limit experiment concerns testing $H_{0}:h=0$ vs $H_{1}:h=h_{1}$. By the Neyman-Pearson lemma, the uniformly most powerful (UMP) test is \[ \varphi_{h_{1}}^{*}=\mathbb{I}\left\{ h_{1}^{\intercal}I^{1/2}x(\tau)-\frac{\tau}{2}h_{1}^{\intercal}Ih_{1}\ge\gamma_{h_{1}}\right\} , \] where $\gamma_{h_{1}}\in\mathbb{R}$ is chosen by the size requirement. Let $\beta^{*}(h_{1})$ denote the power function of $\varphi_{h_{1}}^{*}$. Then, by Theorem (ref), $\beta^{*}(\cdot)$ is an upper bound on the limiting power function of any test of $H_{0}:\theta=\theta_{0}$.
We now consider tests of linear combinations of $h$, i.e., $H_{0}:a^{\intercal}h=0$, in the limit experiment. In this case, a further dimension reduction is possible if the stopping time is also dependent on a reduced set of statistics.
Define $\sigma^{2}:=a^{\intercal}I^{-1}a$, $\tilde{x}(t):=\sigma^{-1}a^{\intercal}I^{-1/2}x(t)$, let $U_{1}$ denote a $\textrm{Uniform}[0,1]$ random variable independent of $\tilde{x}(\cdot)$, and take $\tilde{\mathcal{F}}_{t}$ to be the filtration generated by $\sigma\{U_{1},\tilde{x}(s):s\le t\}$. Note that $\tilde{x}(\cdot)\sim W(\cdot)$ under the null; hence, it is pivotal.
The above result suggests that $\tilde{x}(\tau)$ and $\tau$ are sufficient statistics for the optimal test. An important caveat, however, is that the class of stopping times are further constrained to only depend on $\tilde{x}(t)$ in the limit. In practice, this would happen if the stopping time $\hat{\tau}$ in the original experiment is a function only of $\hat{\tilde{x}}_{n}(\cdot):=\sigma^{-1}a^{\intercal}I^{-1/2}x_{n}(\cdot)$. Fortunately, this is the case in a number of examples.
It is straightforward to show that the same power envelope, $\beta^{*}(\cdot)$, also applies to tests of the composite hypothesis $H_{0}:a^{\intercal}\theta\le0$.
A test is said to be unbiased if its power is greater than size under all alternatives. The following result describes a useful property of unbiased tests in the limit experiment:
See Section (ref) for an application of the above result.
Suppose we specify a weight function, $w(\cdot)$, over alternatives $h\neq0$. Then, the test of $H_{0}:h=0$ in the limit experiment that maximizes weighted average power is given by \[ \varphi_{w}^{*}(\tau,x(\tau))=\mathbb{I}\left\{ \int e^{h^{\intercal}I^{1/2}x(\tau)-\frac{\tau}{2}h^{\intercal}Ih}dw(h)\ge\gamma\right\} . \] The value of $\gamma$ is determined by the size requirement.
In this section, we study inference under a stronger version of the size constraint, inspired by the $\bm{\alpha}$-spending approach in group sequential trials (gordon1983discrete). Suppose that the stopping time is discrete, taking only the values $t=1,2,\dots,T$. Then, instead of an overall size constraint of the form $\mathbb{E}_{nT,\bm{0}}[\varphi_{n}]\le\alpha$, we may specify a `spending-vector' $\bm{\alpha}:=(\alpha_{1},\dots,\alpha_{T})$ satisfying $\sum_{t=1}^{T}\alpha_{t}=\alpha$, and require
In what follows, we call a test, $\varphi_{n}$, satisfying ((ref)) a level-$\bm{\alpha}$ test (with a boldface $\bm{\alpha}$). Intuitively, if each $t$ corresponds to a different stage of the experiment, the $\bm{\alpha}$-spending constraint prescribes the maximum amount of Type-I error that may be expended at stage $t$. As a practical matter, it enables us to characterize a UMP or UMP unbiased test in settings where such tests do not otherwise exist. We also envision the criterion as a useful conceptual device: even if we are ultimately interested in a standard level-$\alpha$ test, we can obtain this by optimizing a chosen power criterion (average power, etc.) over the spending vectors $\bm{\alpha}:=(\alpha_{1},\dots,\alpha_{K})$ satisfying $\sum_{k}\alpha_{k}\le\alpha$.
A particularly interesting example of an $\bm{\alpha}$-spending vector is $(\alpha P_{nT,0}(\hat{\tau}=1),\dots,\alpha P_{nT,0}(\hat{\tau}=k))$; this corresponds to the requirement that $\mathbb{E}_{nT,\bm{0}}\left[\left.\varphi_{n}\right|\hat{\tau}=t\right]\le\alpha$ for all $t$, i.e., the test be conditionally level-$\alpha$ given any realization of the stopping time. This may have some intuitive appeal, though it does disregard any information provided by the stopping time for discriminating between the hypotheses.
Under the $\bm{\alpha}$-spending constraint, a test that maximizes expected power also maximizes expected power conditional on each realization of stopping time. This is a simple consequence of the law of iterated expectations. Consequently, we focus on conditional power in this section. Our main result here is a generalization of Theorem (ref) to $\bm{\alpha}$-spending restrictions. The limit experiment is the same as in Section (ref).
It may be possible to extend the above result to continuous stopping times using Le Cam's discretization device, though we do not take this up here.
By the Neyman-Pearson lemma, the uniformly most powerful level-$\bm{\alpha}$ (UMP-$\bm{\alpha}$) test of $H_{0}:h=0$ vs $H_{1}:h=h_{1}$ in the limit experiment is given by \[ \varphi_{h_{1}}^{*}(t,x(t))=
. \] Here, $\gamma(t)\in\mathbb{R}$ is chosen by the $\bm{\alpha}$-spending requirement that $\mathbb{E}_{0}[\varphi_{h_{1}}^{*}(\tau,x(\tau))\vert\tau=t]\le\alpha_{t}/\mathbb{P}_{0}(\tau=t)$ for each $t$. If we take $\beta^{*}(h_{1}\vert t)$ to be the power function of $\varphi_{h_{1}}^{*}(\cdot)$, Theorem (ref) implies $\beta^{*}(\cdot\vert t)$ is an upper bound on the limiting conditional power function of any level-$\bm{\alpha}$ test of $H_{0}:\theta=\theta_{0}$.
A stronger result is possible for tests of linear combinations of $\theta$. Recall the definitions of $\tilde{x}(t)$ and $\tilde{\mathcal{F}_{t}}$ from Section (ref). If the limiting stopping time is $\tilde{\mathcal{F}_{t}}$ -adapted, we have, as in Proposition (ref), that the sufficient statistics are only $\tilde{x}(\tau),\tau$, and the UMP-$\bm{\alpha}$ test of $H_{0}:a^{\intercal}h=0$ vs $H_{1}:a^{\intercal}h=c\ (>0)$ in the limit experiment is \[ \breve{\varphi}^{*}(t,\tilde{x}(t))=
. \] Here, $\tilde{\gamma}(t)$ is chosen such that $\mathbb{E}_{0}[\breve{\varphi}^{*}(\tau,\tilde{x}(\tau))\vert\tau=t]=\alpha_{t}/\mathbb{P}_{0}(\tau=t)$. Clearly, $\tilde{\gamma}(t)$ it is independent of $c$ for $c>0$. Since $\breve{\varphi}^{*}(\cdot)$ is thereby also independent of $c$ for $c>0$, we conclude that it is UMP-$\bm{\alpha}$ for testing the composite one-sided alternative $H_{0}:a^{\intercal}h=0$ vs $H_{1}:a^{\intercal}h>0$. Thus, a UMP-$\bm{\alpha}$ test exists in this scenario even as a UMP test doesn't. What is more, by Theorem (ref), the conditional power function, $\breve{\beta}^{*}(c\vert t)$, of $\breve{\varphi}^{*}(\cdot)$ is an asymptotic upper bound on the conditional power of any level-$\bm{\alpha}$ test, $\varphi_{n}$, of $H_{0}:a^{\intercal}\theta=0$ vs $H_{1}:a^{\intercal}\theta>0$ in the original experiment against local alternatives $\theta\equiv\theta_{0}+h/\sqrt{n}$ satisfying $a^{\intercal}\theta=c/\sqrt{n}$.
We call a test conditionally unbiased if it is unbiased conditional on any possible realization of the stopping time. In analogy with Proposition (ref), a necessary condition for $\varphi(\cdot)$ being conditionally unbiased in the limit experiment is that
Then, by a similar argument as in lehmann2005testing, the UMP conditionally unbiased (level-$\bm{\alpha}$) test of $H_{0}:a^{\intercal}h=0$ vs $H_{1}:a^{\intercal}h\neq0$ in the limit experiment can be shown to be \[ \bar{\varphi}^{*}(t,\tilde{x}(t))=
. \] The quantities $\gamma_{L}(t),\gamma_{U}(t)$ are chosen to satisfy both ((ref)) and ((ref)). In practice, this requires simulating the distribution of $\tilde{x}(\tau)$ given $\tau=t$. Also, $\gamma_{L}(\cdot)=-\gamma_{U}(\cdot)$ if the distribution of $\tilde{x}(\tau)$ given $\tau=t$ is symmetric around 0 under the null.
Earlier in this section, we took $\theta_{0}\in\Theta_{0}$ to be some reference parameter in the null set. However, such a choice may result in the limiting stopping time, $\tau$, collapsing to $0$. Consider, for example, the case of costly sampling (Example 1 in Section (ref)). In this experiment, the stopping time, $\hat{\tau}$, is itself chosen around a reference parameter $\theta_{0}$ (typically chosen so that the effect of interest is $0$ at $\theta_{0}$). But suppose we are interested in testing $H_{0}:\theta=\bar{\theta}_{0}$, for some $\bar{\theta}_{0}\neq\theta_{0}$. Under this null, $\hat{\tau}$ converges to $0$ in probability as $\bar{\theta}_{0}$ is a fixed distance away from $\theta_{0}$. This issue with the stopping time arises because the null hypothesis and the stopping time are not centered around the same reference parameter.
One way to still provide inference in such settings is to set the reference parameter to $\theta_{0}$, but employ a drifting null $H_{0}:h=h_{0}/\sqrt{n}$, where $h_{0}$ is taken to be fixed over $n$, and is calibrated as $\sqrt{n}(\bar{\theta}-\theta_{0})$. The null, $H_{0}$, thus changes with $n$, but for the observed sample size we are still testing $\theta=\bar{\theta}_{0}$. It is then straightforward to show that Theorems (ref) and (ref) continue to apply in this setting; asymptotically, the inference problem is equivalent to testing that the drift of $x(\cdot)$ is $I^{1/2}h_{0}$ in the limit experiment. The asymptotic approximation is expected to be more accurate the closer $\bar{\theta}_{0}$ is to $\theta_{0}$; but for distant values of $\bar{\theta}_{0}$, we caution that local asymptotics may not provide a good approximation.
So far we have described upper bounds on the asymptotic power functions of tests. Now, given a UMP test, $\varphi^{*}(\tau,x(\tau))$, in the limit experiment, we can construct a finite sample version of this, $\varphi_{n}^{*}:=\varphi^{*}(\hat{\tau},x_{n}(\hat{\tau}))$, by replacing $\tau,x(\tau)$ with $\hat{\tau},x_{n}(\hat{\tau})$. Since $x_{n}(\hat{\tau})$ depends on the information matrix, $I$, one would need to either calibrate it to $I(\theta_{0})$ (if $\theta_{0}$ is known), or replace it with a consistent estimate. We discuss variance estimators in Appendix (ref).
The test, $\varphi_{n}^{*}$, would then be asymptotically optimal, in the sense of attaining the power envelope, under mild assumptions. In particular, we only require that $\varphi^{*}(\cdot,\cdot)$ satisfy the conditions for an extended continuous mapping theorem. Together with ((ref)) and the first part of Theorem (ref), this implies \[ \left(
\right)\xrightarrow[P_{nT,0}]{d}\left(
\right), \] for any $h\in\mathbb{R}^{d}$. Then, a similar argument as in the proof of Theorem (ref) shows that the local power of $\varphi_{n}^{*}$ converges to that of $\varphi^{*}$ in the limit experiment.
We now turn to the setting where the distribution of outcomes is non-parametric. Let $\mathcal{P}$ denote a candidate class of probability measures for the outcome $Y$, with bounded variance, and dominated by some measure $\nu$. We are interested in conducting inference on some regular functional, $\mu:=\mu(P)$, of the unknown data distribution $P\in\mathcal{P}$. We assume for simplicity that $\mu$ is scalar. Let $P_{0}\in\mathcal{P}$ denote some reference probability distribution on the boundary of the null hypothesis so that $\mu(P_{0})=0$. Following van2000asymptotic, we consider the power of tests against smooth one-dimensional sub-models of the form $\{P_{s,h}:s\le\eta\}$ for some $\eta>0$, where $h(\cdot)$ is a measurable function satisfying
By van2000asymptotic, ((ref)) implies $\int hdP_{0}=0$ and $\int h^{2}dP_{0}<\infty$. The set of all such candidate $h$ is termed the tangent space $T(P_{0})$. This is a subset of the Hilbert space $L^{2}(P_{0})$, endowed with the inner product $\left\langle f,g\right\rangle =\mathbb{E}_{P_{0}}[fg]$ and norm $\left\Vert f\right\Vert =\mathbb{E}_{P_{0}}[f^{2}]^{1/2}$. For any $h\in T(P_{0})$, let $P_{nT,h}$ denote the joint probability measure over $Y_{1},\dots,Y_{nT}$, when each $Y_{i}$ is an iid draw from $P_{1/\sqrt{n},h}$. Also, take $\mathbb{E}_{nT,h}[\cdot]$ to be its corresponding expectation. An important implication of ((ref)) is the SLAN property that for all $h\in T(P_{0})$,
See adusumilli2021risk for the proof.
Let $\psi\in T(P_{0})$ denote the efficient influence function corresponding to estimation of $\mu$, in the sense that for any $h\in T(P_{0})$,
Denote $\sigma^{2}=\mathbb{E}_{P_{0}}[\psi^{2}]$. The analogue of the score process in the non-parametric setting is the efficient influence function process \[ x_{n}(t):=\frac{\sigma^{-1}}{\sqrt{n}}\sum_{i=1}^{\left\lfloor nt\right\rfloor }\psi(Y_{i}). \]
At a high level, the theory for inference in non-parametric settings is closely related to that for testing linear combinations in parametric models (see, Section (ref)). It is not entirely surprising, then, that the assumptions described below are similar to those used in Proposition (ref):
\begin{asm3} (i) The sub-models $\{P_{s,h};h\in T(P_{0})\}$ satisfy ((ref)). Furthermore, they admit an efficient influence function, $\psi$, such that ((ref)) holds.
(ii) The stopping time $\hat{\tau}$ is a continuous function of $x_{n}(\cdot)$ in the sense that $\hat{\tau}=\tau(x_{n}(\cdot))$, where $\tau(\cdot)$ satisfies the conditions for an extended continuous mapping theorem van1996weak.\end{asm3}
Assumption 3(i) is a mild regularity condition that is common in non-parametric analysis. Assumption 3(ii), which is substantive, states that the stopping times depend only on the efficient influence function process. This is indeed the case for the examples considered in Section (ref). More generally, however, it may be that $\hat{\tau}$ depends on other statistics beyond $x_{n}(\cdot)$. In such situations, the set of asymptotically sufficient statistics should be expanded to include these additional ones. We remark that an extension of our results to these situations is straightforward, see Section (ref) for an illustration.
We call a test, $\varphi_{n}$, of $H_{0}:\mu=0$ asymptotically level-$\alpha$ if \[ \sup_{\left\{ h\in T(P_{0}):\left\langle \psi,h\right\rangle =0\right\} }\limsup_{n}\int\varphi_{n}dP_{nT,h}\le\alpha. \] Our first result in this section is a power envelope for asymptotically level-$\alpha$ tests. Consider a limit experiment where one observes a stopping time $\tau$, which is the weak limit of $\hat{\tau}$, and a Gaussian process $x(\cdot)\sim\sigma^{-1}\mu\cdot+W(\cdot)$, where $W(\cdot)$ denotes 1-dimensional Brownian motion. By Assumption 3(ii), $\tau$ is adapted to the filtration generated by the sample paths of $x(\cdot)$. For any $\mu\in\mathbb{R}$, let $\mathbb{E}_{\mu}[\cdot]$ denote the induced distribution over the sample paths of $x(\cdot)$ between $[0,T]$. Also, define
with $\gamma$ being determined by the requirement $\mathbb{E}_{0}[\varphi_{\mu}^{*}]=\alpha$, and set $\beta^{*}(\mu):=\mathbb{E}_{\mu}[\varphi_{\mu}^{*}]$.
A similar result holds for unbiased tests. Following choi1996asymptotically, we say that a test $\varphi_{n}$ of $H_{0}:\mu=0$ vs $H_{1}:\mu\neq0$ is asymptotically unbiased if
The next result states that the local power of such a test is bounded by that of a best unbiased in the limit experiment, assuming one exists.
The proof is analogous to that of Proposition (ref), and is therefore omitted. Also, both propositions can be extended to $\bm{\alpha}$-spending constraints but we omit formal statements for brevity.
By similar reasoning as in Section (ref) (using parametric sub-models), it follows that we can attain the power bounds $\beta^{*}(\cdot),\tilde{\beta}^{*}(\cdot)$ by employing plug-in versions of the corresponding UMP tests. This process simply involves replacing $\tau,x(\tau)$ with $\hat{\tau},x_{n}(\hat{\tau})$. The statistic $x_{n}(\hat{\tau})$ depends on the variance, $\sigma$, so we must substitute it with a consistent estimate. We discuss various estimators for $\sigma$ in Appendix (ref).
In many sequential experiments it is common to test two treatments simultaneously. We may then be interested in conducting inference on the difference between some regular functionals of the two treatments. A salient example of this is inference on the expected treatment effect.
To make matters precise, let $a\in\{0,1\}$ denote the two treatments, with $P^{(a)}$ being the corresponding outcome distribution. Suppose that at each period, the experimenter samples treatment 1 at some fixed proportion $\pi$. It is without loss of generality to suppose that the outcomes from the two treatments are independent as we can only ever observe the effect of a single treatment. We are interested in conducting inference on the difference, $\mu(P^{(1)})-\mu(P^{(0)})$, where $\mu(\cdot)$ is some regular functional of the data distribution. As before, we take $\mu$ to be scalar.
Let $P_{0}^{(1)},P_{0}^{(0)}$ denote some reference probability distributions on the boundary of the null hypothesis so that $\mu(P_{0}^{(1)})-\mu(P_{0}^{(0)})=0$. Following van2000asymptotic, we consider the power of tests against smooth one-dimensional sub-models of the form $\left\{ \left(P_{s,h_{1}}^{(1)},P_{s,h_{0}}^{(0)}\right):s\le\eta\right\} $ for some $\eta>0$, where $h_{a}(\cdot)$ is a measurable function satisfying
As before, the set of all possible $h_{a}$ satisfying $\int h_{a}dP_{0}^{(a)}=0$ and $\int h_{a}^{2}dP_{0}^{(a)}<\infty$ forms a tangent space $T(P_{0}^{(a)})$. This is a subset of the Hilbert space $L^{2}(P_{0}^{(a)})$, endowed with the inner product $\left\langle f,g\right\rangle _{a}=\mathbb{E}_{P_{0}^{(a)}}[fg]$ and norm $\left\Vert f\right\Vert _{a}=\mathbb{E}_{P_{0}^{(a)}}[f^{2}]^{1/2}$. Let $\psi_{a}\in T(P_{0}^{(a)})$ denote the efficient influence function satisfying
for any $h_{a}\in T(P_{0}^{(a)})$. Denote $\sigma_{a}^{2}=\mathbb{E}_{P_{0}^{(a)}}[\psi_{a}^{2}]$. The sufficient statistic here is the differenced efficient influence function process
where $\sigma^{2}:=\left(\frac{\sigma_{1}^{2}}{\pi}+\frac{\sigma_{0}^{2}}{1-\pi}\right)$. Note that the number of observations from each treatment at time $t$ is $\left\lfloor n\pi t\right\rfloor ,\left\lfloor n(1-\pi)t\right\rfloor $. The assumptions below are analogous to Assumption 3:
\begin{asm4} (i) The sub-models $\{P_{s,h_{a}}^{(a)};h_{a}\in T(P_{0}^{(a)})\}$ satisfy ((ref)). Furthermore, they admit an efficient influence function, $\psi_{a}$, such that ((ref)) holds.
(ii) The stopping time $\hat{\tau}$ is a continuous function of $x_{n}(\cdot)$ in the sense that $\hat{\tau}=\tau(x_{n}(\cdot))$, where $\tau(\cdot)$ satisfies the conditions for an extended continuous mapping theorem van1996weak.\end{asm4}
Set $\mu_{a}:=\mu(P^{(a)})$. A test, $\varphi_{n}$, of $H_{0}:\mu_{1}-\mu_{0}=0$ is asymptotically level-$\alpha$ if
Similarly, a test, $\varphi_{n}$, of $H_{0}:\mu_{1}-\mu_{0}=0$ vs $H_{1}:\mu_{1}-\mu_{0}\neq0$ is asymptotically unbiased if
Consider the limit experiment where one observes $x(\cdot)\sim\sigma^{-1}(\mu_{1}-\mu_{0})\cdot+W(\cdot)$ and a $\mathcal{F}_{t}\equiv\sigma\{x(s);s\le t\}$ adapted stopping time $\tau$ that is the weak limit of $\hat{\tau}$. Then, setting $\mu:=\mu_{1}-\mu_{0}$, define the power functions $\beta^{*}(\cdot),\tilde{\beta}^{*}(\cdot)$ as in the previous section. The following results provide upper bounds on asymptotically level-$\alpha$ and asymptotically unbiased tests.
We prove Proposition (ref) in Appendix A. The proof of Proposition (ref) is similar and therefore omitted. Both Propositions (ref) and (ref) can be extended to $\bm{\alpha}$-spending constraints. We omit the formal statements for brevity.
We now analyze sequential experiments with multiple treatments and where the sampling rule, i.e., the number of units allocated to each treatment, also changes over the course of the experiment. Since our results here draw on hirano2023asymptotic, we restrict attention to batched experiments, where the sampling strategy is only allowed to be changed at some fixed, discrete set of times.
Suppose there are $K$ treatments under consideration. We take $K=2$ to simplify the notation, but all our results extend to any fixed $K$. The outcomes, $Y^{(a)}$, under treatment $a\in\{0,1\}$ are distributed according to some parametric model $\{P_{\theta^{(a)}}^{(a)}\}$. Here $\theta^{(a)}\in\mathbb{R}^{d}$ is some unknown parameter vector; we assume for simplicity that the dimension of $\theta^{(1)},\theta^{(0)}$ is the same, but none of our results actually require this. It is without loss of generality to suppose that the outcomes from each treatment are independent conditional on $\theta^{(1)},\theta^{(0)}$, as we only ever observe one of the two potential outcomes for any given observation. In the batch setting, the DM divides the observations into batches of size $n$, and registers a sampling rule $\{\hat{\pi}_{j}^{(a)}\}_{j}$ that prescribes the fraction of observations allocated to treatment $a$ in batch $j$ based on information from the previous batches $1,\dots,j-1$. The experiment ends after $J$ batches. It is possible to set $\pi_{j}^{(a)}=0$ for some or all treatments (e.g., the experiment may be stopped early); we only require $\sum_{a}\hat{\pi}_{j}^{(a)}\le1$ for each $j$. We develop asymptotic representation theorems for tests of $H_{0}:\theta=\Theta_{0}$ vs $H_{1}:\theta\in\Theta_{1}$, where $\theta:=(\theta^{(1)},\theta^{(0)})$. Let $(\theta_{0}^{(1)},\theta_{0}^{(0)})\in\Theta_{0}$ denote some reference parameter in the null set.
Take $\hat{q}_{j}^{(a)}$ to be the proportion of observations allocated to treatment $a$ up-to batch $j$, as a fraction of $n$. Let $Y_{j}^{(a)}$ denote the $j$-th observation of treatment $a$ in the experiment. Any candidate test, $\delta(\cdot)$, is required to be \[ \sigma\left\{ \left(Y_{1}^{(0)},\dots,Y_{nq_{J}^{(0)}}^{(0)}\right),\left(Y_{1}^{(1)},\dots,Y_{nq_{J}^{(1)}}^{(1)}\right)\right\} \] measurable. As in the previous sections, we measure the performance of tests against local perturbations of the form $\{\theta_{0}^{(a)}+h_{a}/\sqrt{n};h_{a}\in\mathbb{R}^{d}\}$. Let $\nu$ denote a dominating measure for $\{P_{\theta}^{(a)}:\theta\in\mathbb{R}^{d},a\in\{0,1\}\}$, and set $p_{\theta}^{(a)}:=dP_{\theta}^{(a)}/d\nu$. We require $\{P_{\theta}^{(a)}\}$ to be quadratically mean differentiable (qmd):
\begin{asm5} The class $\{P_{\theta}^{(a)}:\theta\in\mathbb{R}^{d}\}$ is qmd around $\theta_{0}^{(a)}$ for each $a\in\{0,1\}$, i.e., there exists a score function $\psi_{a}(\cdot)$ such that for each $h_{a}\in\mathbb{R}^{d},$ \[ \int\left[\sqrt{p_{\theta_{0}^{(a)}+h_{a}}^{(a)}}-\sqrt{p_{\theta_{0}^{(a)}}^{(a)}}-\frac{1}{2}h_{a}^{\intercal}\psi_{a}\sqrt{p_{\theta_{0}^{(a)}}}\right]^{2}d\nu=o(\vert h_{a}\vert^{2}). \] Furthermore, the information matrix $I_{a}:=\mathbb{E}_{0}[\psi_{a}\psi_{a}^{\intercal}]$ is invertible for $a\in\{0,1\}$. \end{asm5}
Define $z_{j,n}^{(a)}(\hat{\pi}_{j})$ as the standardized score process from each batch, where \[ z_{j,n}^{(a)}(t):=\frac{I_{a}^{-1/2}}{\sqrt{n}}\sum_{i=1}^{\left\lfloor nt\right\rfloor }\psi_{a}(Y_{i,j}^{(a)}) \] for each $t\in[0,1]$. Let $Y_{i,j}^{(a)}$ denote the $i$-th outcome observation from arm $a$ in batch $j$. At each batch $j$, one can imagine that there is a potential set of outcomes, $\{{\bf y}_{j}^{(1)},{\bf y}_{j}^{(0)}\}$ with ${\bf y}_{j}^{(a)}:=\{Y_{i,j}^{(a)}\}_{i=1}^{n}$, that could be sampled from both arms, but only a sub-collection, $\{Y_{i,j}^{(a)};i=1,\dots,n\hat{\pi}_{j}^{(a)}\}$, of these are actually sampled. Let $\bm{h}:=(h_{1},h_{0})$, take $P_{n,\bm{h}}$ to be the joint probability measure over \[ \{{\bf y}_{1}^{(1)},{\bf y}_{1}^{(0)},\dots,{\bf y}_{J}^{(1)},{\bf y}_{J}^{(0)}\} \] when each $Y_{i,j}^{(a)}\sim P_{\theta_{0}^{(a)}+h_{a}/\sqrt{n}}$, and take $\mathbb{E}_{n,\bm{h}}[\cdot]$ to be its corresponding expectation. Then, by a standard functional central limit theorem,
where $\{W_{j}^{(a)}\}_{j,a}$ are independent $d$-dimensional Brownian motions.
Consider a limit experiment where $\bm{h}:=(h_{1},h_{0})$ is unknown, and for each batch $j$, one observes the stopped process $z_{j}^{(a)}(\pi_{j}^{(a)})$, where
and $\{W_{j}^{(a)};j=1,\dots,J;a=0,1\}$ are independent Brownian motions. Each $\pi_{j}^{(a)}$ is required to satisfy $\sum_{a}\pi_{j}^{(a)}\le1$ and also to be \[ \sigma\left\{ (z_{1}^{(1)},z_{1}^{(0)},U_{1}),\dots,(z_{j-1}^{(1)},z_{j-1}^{(0)},U_{j-1})\right\} \] measurable, where $U_{j}\sim\textrm{Uniform}[0,1]$ is exogenous to all the past values $\left\{ z_{j^{\prime}}^{(a)},U_{j^{\prime}}:j^{\prime}<j\right\} $. Let $\varphi$ denote a test statistic for $H_{0}:h=0$ that depends only on: (i) $q_{a}=\sum_{j}\pi_{j}^{(a)}$, i.e., the number of times each arm was pulled; and (ii) $x_{a}=\sum_{j}z_{j}^{(a)}(\pi_{j}^{(a)})$, i.e., the sum of outcomes from each arm. Let $\mathbb{P}_{\bm{h}}$ denote the joint probability measure over $\{z_{j}^{(a)}(\cdot);a\in\{0,1\},j\in\{1,\dots,J\}\}$ when each $z_{j}^{(a)}(\cdot)$ is distributed as in ((ref)), and take $\mathbb{E}_{\bm{h}}[\cdot]$ to be its corresponding expectation.
The following theorem shows that the power function of any test $\varphi_{n}$ in the original testing problem can be matched by one such test, $\varphi$, in the limit experiment.
The first part of Theorem (ref) is due to hirano2023asymptotic; we only modify the terminology slightly. Note that the results of hirano2023asymptotic already imply that any $\varphi_{n}$ can be asymptotically matched by a test $\varphi$ in the limit experiment that is $\sigma\left\{ (z_{1}^{(1)},z_{1}^{(0)},U_{1}),\dots,(z_{J}^{(1)},z_{J}^{(0)},U_{J})\right\} $ measurable. The novel result here is the second part of Theorem (ref), which shows that a further dimension reduction is possible. A naive application of hirano2023asymptotic would require sufficient statistics that grow linearly with the number of batches, leading to a vector of dimension $2dJ+1$ (the uniform random variables $U_{1},\dots,U_{J}$ can be subsumed into a single $U\sim\textrm{Uniform}[0,1]$). Here, we show that one only need condition on $q_{1},q_{0},x_{1},x_{0}$, which are of a fixed dimension $2d+2$ (or $2d+1$ if we impose $q^{(1)}+q^{(0)}=J$). This is a substantial reduction in dimension.
From the distribution of $z_{j}^{(a)}(\cdot)$ given in ((ref)), it is easy to verify that \[ z_{j}^{(a)}(\pi_{j}^{(a)})\sim I_{a}^{1/2}h_{a}\pi_{j}^{(a)}+W_{j}^{(a)}(\pi_{j}^{(a)}). \] Combined with the definition $q_{a}=\sum_{j}\pi_{j}^{(a)}$ and the fact $\{W_{j}^{(a)};j=1,\dots,J;a=0,1\}$ are independent Brownian motions, we obtain
where $W_{1}(\cdot).W_{0}(\cdot)$ are standard $d$-dimensional Brownian motions that are again independent of each other. In view of the above, we can alternatively think of the limit experiment as observing $\{q_{a}\}_{a}$ along with $\{x_{a}\}_{a}$, with the latter distributed as in ((ref)). The advantage of this formulation is that it is independent of the number of batches. It therefore provides suggestive evidence that the asymptotic representation in Theorem (ref) would remain valid under continuous experimentation (however, our proof only applies to a finite number of batches).
It is generally unrealistic in batched sequential experiments for the sampling rule to depend on fewer statistics than $q_{1},q_{0},x_{1},x_{0}$. Consequently, we do not have sharp results for testing linear combinations as in Proposition (ref). We do, however, have analogues to the other results in Section (ref).
Consider testing $H_{0}:\bm{h}=0$ vs $H_{1}:\bm{h}=\bm{h}_{1}$ in the limit experiment. By the Neyman-Pearson lemma, and the Girsanov theorem applied on ((ref)), the optimal test is given by
where $\gamma_{\bm{h}_{1}}$ is chosen such that $\mathbb{E}_{0}[\varphi_{h_{1}}^{*}]=\alpha$. Take $\beta^{*}(\bm{h}_{1})$ to be the power function of $\varphi_{\bm{h}_{1}}^{*}$ against $H_{1}:\bm{h}=\bm{h}_{1}$. Theorem (ref) shows that $\beta^{*}(\cdot)$ is an asymptotic power envelope for any test of $H_{0}:\theta=\theta_{0}$ in the original experiment.
Suppose $\varphi(q_{1},q_{0},x_{1},x_{0})$ is an unbiased test of $H_{0}:\bm{h}=0$ vs $H_{1}:\bm{h}\neq0$ in the limit experiment. Then, in analogy with Proposition (ref), it needs to satisfy the following property:
Let $w(\cdot)$ denote a weight function over alternatives $\bm{h}\neq0$. Then, the uniquely optimal test of $H_{0}:\bm{h}=0$ that maximizes weighted average power over $w(\cdot)$ is given by \[ \varphi_{w}^{*}=\mathbb{I}\left\{ \int\exp\left\{ \sum_{a\in\{0,1\}}\left(h_{a}^{\intercal}I_{a}^{1/2}x_{a}-\frac{q_{a}}{2}h_{a}^{\intercal}I_{a}h_{a}\right)\right\} dw(\bm{h})\ge\gamma\right\} . \] The value of $\gamma$ is chosen to satisfy $\mathbb{E}_{0}[\varphi_{w}^{*}]=\alpha$. In practice, it can be computed by simulation.
For the non-parametric setting, we make use of the same notation as in Section (ref). We are interested in conducting inference on some regular vector of functionals, $\left(\mu(P^{(1)}),\mu(P^{(0)})\right)$, of the outcome distributions $P^{(1)},P^{(0)}$ for the two treatments. To simplify matters, we take $\mu_{a}:=\mu(P^{(a)})$ to be scalar. The definition of asymptotically level-$\alpha$ and unbiased tests is unchanged from ((ref)) and ((ref)).
Let $\psi_{a},\sigma_{a}$ be defined as in Section (ref). Set \[ z_{j,n}^{(a)}:=\frac{1}{\sigma_{a}\sqrt{n}}\sum_{i=1}^{\left\lfloor nt\right\rfloor }\psi_{a}(Y_{i,j}^{(a)}), \] and take $s_{n}(\cdot)=\left\{ x_{n,1}(\cdot),x_{n,0}(\cdot),q_{n,1}(\cdot),q_{n,0}(\cdot)\right\} $ to be the vector of state variables, where
\begin{asm6}(i) The sub-models $\{P_{s,h_{a}}^{(a)};h_{a}\in T(P_{0}^{(a)})\}$ satisfy ((ref)). Furthermore, they admit an efficient influence function, $\psi_{a}$, such that ((ref)) holds.
(ii) The sampling rule $\hat{\pi}_{j+1}$ in batch $j$ is a continuous function of $s_{n}(j)$ in the sense that $\hat{\pi}_{j+1}=\pi_{j+1}(s_{n}(j))$, where $\pi_{j+1}(\cdot)$ satisfies the conditions for an extended continuous mapping theorem van1996weak for each $j=0,\dots,K-1$.\end{asm6}
Assumption 6(i) is standard. Assumption 6(ii) implies that the sampling rule depends on a vector of four state variables. This is in contrast to the single sufficient statistic used in Section (ref). We impose Assumption 6(ii) as it is more realistic; many commonly used algorithms, e.g., Thompson sampling, depend on all four statistics. The assumption still imposes a dimension reduction as it requires the sampling rule to be independent of the data conditional on knowing $s_{n}(\cdot)$. In practice, any Bayes or minimax optimal algorithm would only depend on $s_{n}(\cdot)$ anyway, as noted in adusumilli2021risk. In fact, we are not aware of any commonly used algorithm that requires more statistics beyond these four.
The reliance of the sampling rule on the vector $s_{n}(\cdot)$ implies that the optimal test should also depend on the full vector, and cannot be reduced further. The relevant limit experiment is the one described in Section (ref), with $\mu_{a}$ replacing $h_{a}$. Also, let \[ \varphi_{\bar{\mu_{1}},\bar{\mu_{0}}}=\mathbb{I}\left\{ \sum_{a\in\{0,1\}}\left(\frac{\bar{\mu_{a}}}{\sigma_{a}}x_{a}-\frac{q_{a}}{2\sigma_{a}^{2}}\bar{\mu}_{a}^{2}\right)\ge\gamma_{\bar{\mu}_{1},\bar{\mu}_{0}}\right\} \] denote the Neyman-Pearson test of $H_{0}:(\mu_{1},\mu_{0})=(0,0)$ vs $H_{1}:(\mu_{1},\mu_{0})=(\bar{\mu}_{1},\bar{\mu}_{0})$ in the limit experiment, with $\gamma_{\bar{\mu}_{1},\bar{\mu}_{0}}$ determined by the size requirement. Take $\beta(\bar{\mu}_{1},\bar{\mu}_{0}$) to be its corresponding power.
Proposition (ref) describes the power envelope for testing that the parameter vector takes on a given value. Suppose, however, that one is only interested in providing inference for single component of that vector, say $\mu_{1}$. Then $\mu_{0}$ is a nuisance parameter under the null, and one would need to employ the usual strategies for getting rid of the dependence on $\mu_{0}$, e.g., through conditional inference or minimax tests. We leave the discussion of these possibilities for future research.
As a first illustration of our methods, consider the class of horizontal boundary designs with a fixed sampling rule, $\pi$, and the stopping time $\hat{\tau}=\inf\left\{ t:\vert x_{n}(t)\vert\ge\gamma\right\} $, where $x_{n}(t)$ is defined as in ((ref)). As a concrete example, suppose $\mu_{1},\mu_{0}$ denote the mean values of outcomes from each treatment, with $\sigma_{1},\sigma_{0}$ their corresponding standard deviations. If the goal of the experiment is to determine the treatment with the largest mean while minimizing the number of samples, which are costly, then, as shown in adusumilli2022sample, the minimax optimal sampling strategy is the Neyman allocation $\pi_{1}^{*}=\sigma_{1}/(\sigma_{1}+\sigma_{0})$, and optimal stopping rule is $\hat{\tau}=\inf\left\{ t:\vert x_{n}(t)\vert\ge\gamma\right\} $ with the efficient influence functions $\psi_{1}(Y)=\psi_{0}(Y)=Y$.
We are interested in testing the null of no treatment effect, $H_{0}:\mu_{1}-\mu_{0}=0$ vs $H_{1}:\mu_{1}-\mu_{0}\neq0$. Let $F_{\mu}(\cdot)$ denote the distribution of $\tau$ in the limit experiment where $x(t)\sim\sigma^{-1}\mu t+W(t)$ and $\tau=\inf\{t:\vert x(t)\vert\ge\gamma\}$. In adusumilli2022sample, this author suggested employing the test function $\hat{\varphi}=\mathbb{I}\{\hat{\tau}\le F_{0}^{-1}(\alpha)\}$. This corresponds to the test $\varphi^{*}=\mathbb{I}\{\tau\le F_{0}^{-1}(\alpha)\}$ in the limit experiment. However, no argument was given as to its optimality. The following result, proved in Appendix (ref), shows that $\hat{\varphi}$ is in fact the UMP asymptotically unbiased test.
To illustrate the finite sample performance of this test, we ran Monte-Carlo simulations with $Y_{i}^{(1)}=\delta+\epsilon_{i}^{(1)}$ and $Y_{i}^{(0)}=\epsilon_{i}^{(0)}$ where $\epsilon_{i}^{(1)},\epsilon_{i}^{(0)}\sim\sqrt{3}\times\textrm{Uniform}[-1,1]$. The threshold, $\gamma$, was taken to be $0.536$ (this corresponds to a sampling cost of $c=1$ for each observation in the costly sampling framework), and the treatments were sampled in equal proportions $(\pi=1/2$). Figure (ref), Panel A plots the size of the test for different values of $n$ under the nominal $5\%$ significance level. Even for relatively small values of $n$, the size is close to nominal. We also plot the size of the standard two-sample test for comparison; due to the adaptive stopping rule, this test is not valid and its actual size is close to 9%. Panel B of the same figure plots the finite sample power functions for $\hat{\varphi}$ under different $n$. The power is computed against local alternatives; the reward gap in the figure is the scaled one, $\mu=\sqrt{n}\vert\delta\vert$. But for any given $n$, the actual difference in mean outcomes is $\mu/\sqrt{n}$. The same plot also displays the asymptotic power envelope for unbiased tests, obtained as the power function of the best unbiased test, $\varphi{}^{*}=\mathbb{I}\{\tau\le F_{0}^{-1}(\alpha)\}$, in the limit experiment. Even for small samples, the power function of $\hat{\varphi}$ is close to the asymptotic upper bound.
In this application, we suggest methods for inference on treatment effects following group sequential experiments. To simplify matters, suppose that the researchers assign the two treatments with equal probability in each stage. Let $\mu_{1},\mu_{0}$ denote the expectation of outcomes from the two treatments. Also, take $x_{n}(\cdot)$ to be the scaled difference in sample means, i.e., it is the quantity defined in ((ref)) with $\psi_{1}(Y)=\psi_{0}(Y)=Y$. While there are a number of different group sequential designs, see, e.g., wassmer2016group for a textbook overview, the general construction is that the experiment is terminated at the end of stage $t$ if $x_{n}(t)$ is outside some interval $\mathcal{I}_{t}$. The stopping time $\hat{\tau}$ thus satisfies $\{\hat{\tau}>t-1\}\equiv\cap_{l=1}^{t-1}\left\{ x_{n}(l)\in\mathcal{I}_{l}\right\} $. The intervals $\{\mathcal{I}_{t}\}_{t=1}^{T}$ are pre-determined and chosen by balancing various ethical, cost and power criteria. We take them as given.
We are interested in testing the drifting hypotheses $H_{0}:\mu_{1}-\mu_{0}=\bar{\mu}/\sqrt{n}$ vs $H_{1}:\mu_{1}-\mu_{0}>\bar{\mu}/\sqrt{n}$ at some spending level $\bm{\alpha}$ that is chosen by experimenter.\footnote{In most examples of group sequential designs, the intervals $\mathcal{I}_{t}$ are themselves chosen to maximize power under some $\bar{\bm{\alpha}}$-spending criterion, given the null of $\mu_{1}=\mu_{0}$. In general, our $\bm{\alpha}$ here may be different from $\bar{\bm{\alpha}}$. Furthermore, we are interested in conducting inference on general null hypotheses of the form $H_{0}:\mu_{1}-\mu_{0}=\bar{\mu}/\sqrt{n}$; these are different from the null hypothesis of no average treatment effect used to motivate the group sequential design.} We can then invert these tests to obtain one-sided confidence intervals for the treatment effect $\mu_{1}-\mu_{0}$. The limit experiment in this setting consists of observing $x(t)\sim\sigma^{-1}\mu t+W(t)$, where $\mu:=\mu_{1}-\mu_{0}$, along with a discrete stopping time $\tau\in\{1,\dots,T\}$ such that $\{\tau>t-1\}$ if and only if $x(l)\in\mathcal{I}_{l}$ for all $l=1,\dots,t-1$. Let $\mathbb{P}_{\mu}(\cdot)$ denote the induced probability measure over the sample paths of $x(\cdot)$ between $0$ and $T$, and $\mathbb{E}_{\mu}[\cdot]$ its corresponding expectation. In view of the results in Section (ref), the optimal level-$\bm{\alpha}$ test $\varphi^{*}(\cdot)$ of $H_{0}:\mu=\bar{\mu}$ vs $H_{1}:\mu>\bar{\mu}$ in the limit experiment is given by
where $\gamma(t)$ is chosen such that $\mathbb{E}_{\bar{\mu}}[\varphi^{*}(\tau,x(\tau))\vert\tau=t]=\alpha_{t}/\mathbb{P}_{\bar{\mu}}(\tau=t)$.
A finite sample version, $\hat{\varphi}$, of this test can be constructed by replacing $\tau,x(\tau)$ in $\varphi^{*}$ with $\hat{\tau},x_{n}(\hat{\tau})$. The resulting test would be asymptotically optimal under a suitable non-parametric version of the $\bm{\alpha}$-spending requirement. We refer to Appendix (ref) for the details and for the proof that $\hat{\varphi}$ is asymptotically optimal, in the sense that it attains the power of $\varphi^{*}$ in the limit experiment. A two-sided test for $H_{0}:\mu_{1}-\mu_{0}=\bar{\mu}/\sqrt{n}$ vs $H_{1}:\mu_{1}-\mu_{0}\neq\bar{\mu}/\sqrt{n}$ can be similarly constructed by imposing a conditional unbiasedness restriction as in Section (ref).
To illustrate the methodology, consider a group sequential trial based on the widely-used design of o1979multiple, with $T=2$ stages. This corresponds to setting $\mathcal{I}_{1}=[-2.797,2.797]$. We would like to test $H_{0}:\mu_{1}-\mu_{0}=\bar{\mu}/\sqrt{n}$ vs $H_{1}:\mu_{1}-\mu_{0}>\bar{\mu}/\sqrt{n}$ at the spending level $(\alpha/\mathbb{P}_{\bar{\mu}}(\tau=1),\alpha/\mathbb{P}_{\bar{\mu}}(\tau=2))$, equivalent to a conditional size constraint, $\mathbb{P}_{\bar{\mu}}(\varphi=1\vert\tau=t)=\alpha\ \forall\ t$. Figure (ref) Panel A plots the thresholds, $(\gamma(1),\gamma(2))$, for this test under $\alpha=0.05$ and $\sigma_{1}=\sigma_{0}=1$. Unsurprisingly, the thresholds are increasing in $\bar{\mu}$ , but it is interesting to observe that they cross at some $\bar{\mu}$.
To describe the finite sample performance of this test, we ran Monte-Carlo simulations with $Y_{i}^{(1)}=\bar{\mu}/\sqrt{n}+\epsilon_{i}^{(1)}$ and $Y_{i}^{(0)}=\epsilon_{i}^{(0)}$ where $\epsilon_{i}^{(1)},\epsilon_{i}^{(0)}\sim\sqrt{3}\times\textrm{Uniform}[-1,1]$. The treatments were sampled in equal proportions $(\pi=1/2$). Since $\sigma_{1},\sigma_{0}$ are unknown in practice, we estimate them using data from the first stage. Figure (ref), Panel B plots the overall size of the test (which is the sum of the $\alpha$-spending values at each stage) for different values of $n$ and $\bar{\mu}$ under the nominal $\alpha$-spending level of $(0.05/\mathbb{P}_{\bar{\mu}}(\tau=1),0.05/\mathbb{P}_{\bar{\mu}}(\tau=2))$. We see that the asymptotic approximation worsens for larger values of $\bar{\mu}$, but overall, the size is close to nominal even for relatively small values of $n$.
Here, we describe inferential procedures for the batched Thompson-sampling algorithm. For illustration, we employ $K=2$ treatments and $J=10$ batches. Let $(\bar{\mu}_{1},\bar{\mu}_{0})$ and $(\sigma_{1}^{2},\sigma_{0}^{2})$ denote the population means and variances for each treatment. For simplicity, we take $\sigma_{1}^{2}=\sigma_{0}^{2}=1$. The limit experiment can be described as follows: Suppose the decision maker (DM) employs the sampling rule $\pi_{j}^{(a)}$ in batch $j$. The DM then observes $Z_{j}^{(a)}\sim\mathcal{N}\left(\bar{\mu}_{a}\pi_{a},\pi_{a}\right)$ for $a\in\{0,1\}$ and updates the state variables $x_{a},q_{a}$ (which are initially set to $0$) as \[ x_{a}\leftarrow x_{a}+Z_{j}^{(a)},\quad q_{a}\leftarrow q_{a}+\pi_{a}. \] Under an under-smoothed prior, suggested by wager2021diffusion, the Thompson sampling rule in batch $j+1$ is \[ \pi_{j+1}^{(1)}=\Phi\left(\frac{q_{1}^{-1}x_{1}-q_{0}^{-1}x_{0}}{\sqrt{j/q_{1}q_{0}}}\right). \] We set $\pi_{1}^{(a)}=1/2$ for first batch. In what follows, we let $\mu_{a}:=J\bar{\mu}_{a}$. We are interested in testing $H_{0}:(\mu_{1},\mu_{0})=(0,0)$.
Figure (ref), Panel A plots the asymptotic power envelope for testing $H_{0}:(\mu_{1},\mu_{2})=(0,0)$. Clearly, the envelope is not symmetric; distinguishing $(a,0)$ from $(0,0)$ is easier than distinguishing $(-a,0)$ from $(0,0)$ for any $a>0$. This is because of the asymmetry in treatment allocation under Thompson sampling; under $(-a,0)$, treatment 1 is sampled more often than treatment $0$ but the data from treatment $1$ is uninformative for distinguishing $(-a,0)$ from $(0,0)$.
To determine the accuracy of our asymptotic approximations, we ran Monte-Carlo simulations with $Y_{i}^{(a)}=\mu_{a}+\epsilon_{i}^{(a)}$ where $\epsilon_{i}^{(1)},\epsilon_{i}^{(0)}\sim\sqrt{3}\times\textrm{Uniform}[-1,1]$. Figure (ref), Panel A plots the finite sample performance of the Neyman-Pearson tests in the limit experiment for testing $H_{0}:(\mu_{1},\mu_{0})=(0,0)$ vs $H_{1}:(\mu_{1},\mu_{0})=(\mu,\mu)$ under various values of $\mu$ (due to symmetry, we only report the results for positive $\mu$). Panel B repeats the same calculation, but against alternatives of the form $H_{1}:(\mu,0)$. As noted earlier, power is higher here for $\mu>0$ as opposed to $\mu<0$. Both plots show that the asymptotic approximation is quite accurate even for $n$ as small as $20$ (note that the number of batches is $10$, so this corresponds to $200$ observations overall). The approximation is somewhat worse for testing $\mu<0$; this is because Thompson-sampling allocates much fewer units to treatment 0 in this instance, even though it is only data from this treatment that is informative for distinguishing the two hypotheses.
Conducting inference after sequential experiments is a challenging task. However, significant progress can be made by analyzing the optimal inference problem under an appropriate limit experiment. We showed that the data from any sequential experiment can be condensed into a finite number of sufficient statistics, while still maintaining the power of tests. Furthermore, we were able to establish uniquely optimal tests under reasonable constraints such as unbiasedness and $\bm{\alpha}$-spending, in both parametric and non-parametric regimes. Taken together, these findings offer a comprehensive framework for conducting optimal inference following sequential experiments.
Despite these results, there are still several avenues for future research. While we believe that our results for experiments with adaptive sampling rules apply without batching, this needs be formally verified. Our characterization of uniquely optimal tests is also limited in this context, as $\bm{\alpha}$-spending restrictions are not feasible. Therefore, exploring other types of testing considerations such as invariance or conditional inference may be worthwhile. We believe that the techniques developed in this paper will prove useful for analyzing these other types of tests.