Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
40,424 characters · 15 sections · 19 citation commands
Reserve Price Optimization for First Price Auctions
\affil[a]{John A. Paulson School of Engineering and Applied Sciences, Harvard University \authorcr zhe\[email removed]} \affil[b]{Google Inc, NYC \authorcr slahaie, jschnei, [email removed]}
A reserve price in an auction specifies a minimum acceptable winning bid, below which the item remains with the seller. The reserve price may correspond to some outside offer, or the value of the item to the seller itself, and more generally may be set to maximize expected revenue Myerson1981. In a data-rich environment like online advertising auctions it becomes possible to learn a revenue-optimal reserve price over time, and there is a substantial literature on optimizing reserve prices for second-price auctions, which have been commonly used to allocate ad space paes2016field,MM2016,MV2017.
In this work we examine the problem of reserve price optimization in first-price (i.e., pay-your-bid) auctions, motivated by the fact that all the major ad exchanges have recently transitioned to this auction format as their main ad allocation mechanism digidayFirstprice,BiglerFP. First-price auctions have grown in favor because they are considered more transparent, in the sense that there is no uncertainty in the final price upon winning digidayTransparency.\footnote{The full reasons for the transition are complex, and include the rise of “header bidding” digidayHeader. A header bidding auction is a first-price auction usually triggered by code in a webpage header (hence the name).} Unless restrictive assumptions are met, there is in theory no revenue ranking between first- and second-price auctions krishna2009auction, and there is no guarantee that reserve prices optimized for second-price auctions will continue to be effective in a first-price setting.
From a learning standpoint the shift from second- to first-price auctions introduces several new challenges. In a second-price auction, truthful bidding is a dominant strategy no matter what the reserve. The bidders' value distributions are therefore readily available, and bids stay static (in principle) as the reserve is varied. In a first-price auction, in contrast, bidders have an incentive to shade their values when placing their bids, and bid-shading strategies can vary by bidder. The gain from setting a reserve price now comes if (and only if) it induces higher bidding, so an understanding of bidder responsiveness becomes crucial to setting effective reserves.
Bid adjustments in response to a reserve price can occur at different timescales. If a bidder observes that it wins too few auctions because of the reserve price, it may increase its bid in the long-term (in a matter of hours up to weeks). Our focus here is on setting reserves prices by taking into account immediate bidder responses to reserves. We assume that each bidder has a fixed, unknown bidding function $b(r, v)$ that depends on its private value $v$ and the observed auction reserve $r$. This agrees with practice in display ad auctions because the reserve $r$ is normally sent out in the 'bid request' message to potential bidders OpenRTB. To the extent that the bid function responds to $r$, first-price reserves can potentially show an immediate positive effect on revenue.
We propose a gradient-based approach to adaptively improve and optimize reserve prices, where we perturb current reserves upwards and downwards (e.g., by 10%) on random slices of traffic to obtain gradient estimates.
Our key innovation is to draw on the inherent structure of the revenue objective in order to reduce the variance of gradient estimates and improve convergence rates in both theory (e.g., see Corollary (ref)) and practice. We show that revenue in a first-price auction can be usefully decomposed into two terms: a demand curve component which depends only on the bidder's value distribution; and a bidding component whose variance can be reduced based on natural assumptions on bidding functions.
A demand curve is a simpler, more structured object than the original revenue objective (e.g., it is downward-sloping), so the demand component lends itself to parametric modeling to reduce the variance. We offer two variance reduction techniques for the bidding component\footnote{Variance reduction of the bidding component relies on the insight that bids far above the reserves are little affected by them (under natural bidding models), so these bids can be filtered out when computing gradient estimates---changes in such bids are likely due to noise rather than any effect of reserves.}, referred to as bid truncation and quantile truncation. Bid truncation can strictly decrease variance with no additional bias assuming the right bidding model, whereas quantile truncation may introduce bias but is less sensitive to assumptions on the bidding model.
We evaluate our approach over synthetic data where bidder values are drawn uniformly, and also over real bid distributions collected from the logs of the Google ad exchange with different bidder response models. Our experimental results confirm that the combination of variance reduction on both objective components leads to the fastest convergence rate. For the demand component, a simple logistic model works well over the synthetic (i.e., uniform) data, but a flexible neural net is needed over the semi-synthetic data. For the bidding component, we find that quantile truncation is much more robust to assumptions on the bidding model.
This paper connects with the rich literature on reserve price optimization for auctions, e.g., Myerson1981, riley1981optimal. How to set optimal reserve prices in second price auctions based on access to bidders' historical bid data has been an increasingly popular research direction in Machine Learning community, e.g., OS2011, MM2016, MV2017. Another related line of work uses no-regret learning in second price auctions with partial information feedback to optimize reserve prices, e.g., Blum2004, CGM2015. All of the works cited so far rely on the fact that the seller can directly learn the valuation distribution from historical bid data, since the second price auction is truthful.
For first-price auctions, we have found little work on setting optimal reserves for asymmetric bidders, since there are no characterizations of equilibrium strategies for this case. Results are only available for limited environments, such as bidders with uniform valuation distributions krishna2009auction, Matthews1995. Recently, there has been a line of work regarding revenue optimization against strategic bidders in repeated auctions, e.g., Amin13,Huang18. In this paper, instead of assuming bidders act strategically, we assume each bidder has a fixed bidding function in response to reserves. This is a common assumption in large market settings and in the dynamic pricing literature Mao18.
The algorithms developed in this paper are related to the literature on online convex optimization with bandit feedback FKM2005, HL2014, ADX2010, AFH2011. However, there are two key differences with our work: (1) the revenue function in a first price auction is non-convex, and (2) the seller cannot obtain perfect revenue feedback under perturbed reserves with just a single query (i.e., auction)---the seller needs multiple queries to achieve accurate estimates with high confidence. Our algorithm is also related to zeroth-order stochastic gradient methods GL2013, BG2018, Ghadimi2019, LLCHA2018, which we discuss in detail later in Section (ref).
We consider a setting where a seller repeatedly sells a single item to a set of $m$ bidders via a first price auction. In such an auction, the seller first sends out a reserve price $r$ to all bidders. Each bidder $i$ then submits a bid $b_i$. The bidder with the highest bid larger than $r$ wins the item and pays their bid; if no bidder bids above $r$, the item goes unallocated. Note that the type of reserve price we consider in this work is anonymous in the sense that each bidder sees the same reserve price.
Each bidder $i$ has a private valuation $v_i \in [0, 1]$ for the item, where each value $v_i$ is drawn independently (but not necessarily identically) from some unknown distribution.\footnote{This is without loss of generality, our analysis can easily be applied to any bounded valuation setting.} In a first-price auction, only the highest bid matters for both allocation and pricing. Thus, to simplify the notation, we write $v = \max_i v_i$ to denote the maximum value and $v$ is drawn i.i.d.\ from an unknown distribution $F$ across each auction. Our analysis from here on will refer to this `representative' highest bidder. (See Appendix (ref) for a rigorous justification of why we can reduce multiple bidders to a single bidder.)
We write $b(r, v)$ to denote the maximum bid when the reserve price is $r$ and the maximum value is $v$, and $\mathcal{B}(r)$ to denote the distribution of $b(r, v)$ for a fixed $r$ when $v$ is drawn according to $\mathcal{F}$. The main goal of the seller considered in this work is to learn the optimal reserve price $r \in [0,1]$ that maximizes expected revenue:
Note that there is no reason for a bidder to bid a positive value less than the reserve $r$: such a bid is guaranteed to lose. Therefore, without loss of generality we can assume that if $b(r, v) < r$, then $b(r, v) = 0$. This allows us to write the revenue simply as: $$\mu(r) = \mathbb{E}_{b \sim \mathcal{B}(r)} \left[ b \right] = \mathbb{E}_{v \sim \mathcal{F}} \left[b(r, v)\right].$$ In this paper, we focus on maximizing the revenue function $\mu(r)$ in the steady state.
We begin by describing some general properties of bidding functions that hold for any utility-maximizing bidders (see Matthews1995 for further discussion).
In some of our algorithms, we would like to impose additional constraints on the response model which, while not a consequence of utility-maximizing behavior, are likely to hold in practice. One such constraint is the diminishing sensitivity in value of bid to reserve. This says that bidders with a larger value will change their bid less in response to a change in reserves.
One natural and concrete example of a response model is a bidder that increases its bid to the reserve as long as the reserve is below its value. We refer to this as the perfect response model, formally defined as follows.
Note that the perfect response model is based on the original bid of the bidder under reserve price $0$, namely $b(0, v)$. If $b(0, v)$ is already above the reserve, then this bidder is unaffected by the reserve. Note that the perfect response model satisfies the diminishing sensitivity property.
In practice, bidders are unlikely to exactly follow the perfect response model; for example, bidders will often increase their bid to some amount strictly above the reserve $r$ so as to remain competitive with other bidders. For this reason, we propose a relaxation of the perfect response model which we call the $\varepsilon$-bounded response model: the bid is at most $\varepsilon$ greater than what it would have been under the perfect response model if $b(0, v) < r \leq v$ (see also Definition (ref)). Note that the $\varepsilon$-bounded response model becomes the perfect response model when $\varepsilon = 0$.
The first-price auction setting introduces several challenges for setting reserve prices. First, the seller cannot observe true bidder values because truthful bidding is not a dominant strategy in a first-price auction. Second, how the bidders will react to different reserves is unknown to the seller---the only information that the seller receives is bids drawn from distribution $\mathcal{B}(r)$ when the seller sets a reserve price $r$.
One natural idea, and the approach we take in this paper, is to optimize the reserve price via gradient descent. Gradient descent is only guaranteed to converge to the optimal reserve when our objective is convex (or at least, unimodal), which is not necessarily true for an arbitrary revenue function. However, gradient descent has a number of practical advantages for reserve price optimization, including:
More specifically, since the seller has no direct access to the gradients (i.e, first-order information) of $\mu(r)$, we consider approaches that fit in the framework of zeroth-order stochastic optimization. Our framework, summarized in Algorithm (ref), proceeds in rounds. In round $t$ where the current reserve is $r_t$, the seller selects a perturbation size $\beta_t$ and randomly sets the reserve price to either $(1 + \beta_t)r_t$ or $(1-\beta_t)r_t$ on separate slices of experiment traffic, until it has received $n_t$ samples from both $\mathcal{B}((1+\beta_t)r_t)$ and $\mathcal{B}((1-\beta_t)r_t)$. The seller then uses these $2n_t$ samples to estimate the gradient $\hat{G}_t$ of the revenue curve $\mu(r)$ at $r_t$ and updates the reserve price based on this gradient estimate using learning rate (step size) $\alpha_t$.
We assume that we have access to a fixed total number of samples $N=\sum_{t=1}^T n_t$ (the number of iterations $T$ is a variable that will be fixed later). There is then a trade-off between $n_t$ (i.e, the number of samples per iteration) and $T$ (the number of iterations available to optimize the reserve price).
Zeroth-order stochastic gradient descent is a well-studied problem GL2013, BG2018, Ghadimi2019, LLCHA2018. In this paper, we focus on taking advantage of the structure of $b(r, v)$ to construct good discrete gradient estimates $\hat{G}_t$, as this aspect is specific to the problem of reserve price optimization. Specifically, we tackle the following problem which we term the discrete gradient problem:
Solutions to the discrete gradient problem with small bias and variance directly translate into faster convergence rates for our gradient descent. We provide a detailed convergence result in Theorem (ref) in Appendix (ref). We summarize this result informally as follows.
Intuitively, we want to design an estimator and choose our parameters $\alpha_t, \beta_t, n_t$, so as to trade off between $\delta$, $B$, and $V$. In the following sections, we show how to do this for a variety of bidder response models.
The simplest method for estimating the discrete gradient is to take the difference between the average revenue from bids from $\mathcal{B}(r^{+})$ and the average revenue from bids from $\mathcal{B}(r^{-})$. More formally, we compute discrete gradient as, {
We show that $\hat{G}$ has the following properties. }
This leads to the following convergence rate via Theorem (ref).
Although there are no matching lower bounds, this is the best known asymptotic convergence rate for zeroth-order optimization over a non-convex objective GL2013, BG2018. The naive gradient estimation approach has the advantage that it works regardless of response model, is simple to compute (it uses only revenue information and not individual bids), and leads to an unbiased estimator for the discrete derivative. The disadvantage is that the variance of this estimator can be large (especially as we take $\delta$ small). In the following section, we show how to address this by taking into account the inherent structure of the revenue objective based on an underlying bidder response model.
In this section, we first introduce another representation of the revenue formula by decomposing it into a demand component and a bidding component. We then propose techniques to reduce the variance of the discrete gradient of each component.
We can decompose the revenue $\mu(r)$ in the following way.
Define $E(r) = \mathbb{E}_{v \sim \mathcal{F}}[\max(b(r, v) - r, 0)]$ and $D(r) = \Pr_{v \sim \mathcal{F}}[v \geq r]$, so that $\mu(r) = E(r) + rD(r)$. These two terms capture two different aspects of bidder behavior which contribute to revenue. The function $D(r)$ amounts to a “demand curve” which gives the proportion of values that clear the reserve $r$, and therefore the proportion of auctions that are bid on at $r$. If the auction were just a simple posted-price auction (i.e., the winner is charged the quoted price $r$), then the demand component $r D(r)$ would be the associated revenue. However, in a first-price auction the winning bidder pays its bid, not the reserve. Therefore the bidding component $E(r)$ captures the excess contribution from bids greater than the reserve.
To construct a good estimator $\hat{G}$ for the discrete gradient of $\mu(r)$, it suffices to construct good estimators $\hat{G}_{E}$ and $\hat{G}_{D}$ for the discrete gradients of $E(r)$ and $rD(r)$ respectively, and then output $\hat{G} = \hat{G}_{E} + \hat{G}_{D}$. Note that $\mathrm{Bias}(\hat{G}) \leq \mathrm{Bias}(\hat{G}_{E}) + \mathrm{Bias}(\hat{G}_{D})$ and $\mathrm{Var}(\hat{G}) \leq 2(\mathrm{Var}(\hat{G}_{D}) + \mathrm{Var}(\hat{G}_{E}))$, so it suffices to bound the bias and variance of each component separately.
We begin by discussing how to estimate the gradient $\hat{G}_{D}$ of the demand component of revenue. One method of doing so is by estimating $D(r)$ with a parametric function $f_{\theta}(r)$, and using this approximation to estimate the gradient $\hat{G}_D$. (See Appendix (ref) for additional justification for why this is likely to be possible and helpful). Suppose that we have access to additional historical data $\mathcal{S}$ with which we can fit our parametric class to $D(r)$; let $\hat{\theta}$ be the resulting learned parameter. This learned demand function gives rise to the following estimator $\hat{G}_D$:
Note that this decreases overall variance, the variance of $\hat{G}_D$ is 0 because the randomness of $\hat{G}_D$ only comes from historical samples $\mathcal{S}$, which are independent of the samples obtained in the current round, at the cost of a possible increase in bias (due to inaccuracy in estimating $D(r)$).
In this section we propose a variance reduction method to achieve a better estimator for $\hat{G}_E$ for a variety of bidder models. \paragraph{Variance reduction via bid truncation.}
We first consider the special case of the perfect response (and more generally, the $\varepsilon$-bounded response) bidding model. In the perfect response model, if you were going to bid $b > r^{+}$ when the reserve was $r^{+}$, you will bid the same bid $b$ when the reserve is $r^{-}$. This means that large bids (bids larger than $r^{+}$) do not contribute in expectation to $\mu(r^{+}) - \mu(r^{-})$, but they do add noise to our gradient estimation. By filtering these out, we can reduce the variance of our estimator while keeping our estimator unbiased.
Since we only apply this filtering when estimating the bidding component $E(r)$ but not the demand component $r D(r)$, we must be careful when implementing this. Note that a large bid $b > r^{+}$ contributes $b - r^{+}$ to $E(r^{+})$ and $b - r^{-}$ to $E(r^{-})$, and therefore $r^{+} - r^{-}$ to $E(r^{+}) - E(r^{-})$. We can therefore construct an unbiased estimator for $E(r^{+}) - E(r^{-})$ by computing the contribution of unfiltered bids ($b < r^{+}$) from both $\mathcal{B}(r^{+})$ or $\mathcal{B}(r^{-})$ and then adding $r^{+} - r^{-}$ for each filtered bid in $\mathcal{B}(r^{-})$ (or equivalently, each filtered bid in $\mathcal{B}(r^{+})$; under perfect response, the fraction of filtered bids is equal in both models in expectation). Note that every bid from $\mathcal{B}(r^{+})$ is either filtered or has excess $0$, so we can write this gradient $\hat{G}_E$ entirely in terms of bids from $\mathcal{B}(r^{-})$. Formally, we define truncated bid $Y_i^-$ as { $$ Y^{-}_i = \left\{
\right. $$ } Our estimate for the gradient of $E(r)$ is then given by
Since any bid in an $\varepsilon$-bounded model only differs from one in the perfect response model by at most $\varepsilon$, we can apply this same estimator to an $\varepsilon$-bounded response model. The following theorem characterizes the bias and variance of the estimator for the $\varepsilon$-bounded response model.
Note that the bias of estimator $\hat{G}_E$ is 0 for the perfect response model. The complete proof is given in Appendix (ref). Combining the above results for $\hat{G}_E$ and $\hat{G}_D$, we have the following improved convergence result for the $\varepsilon$-bounded response model.
For perfect response bidding models, the above convergence rate is strictly faster than the convergence rate of naive estimator in Corollary (ref) (state-of-the-art convergence rate for zeroth-order stochastic gradient descent), but with additional bias coming from demand estimation. However, we show this bias has practically negligible effect on the revenue in our experiments.
\paragraph{Variance reduction via quantile truncation.}
In Eq. ((ref)), we reduced the variance of $\hat{G}_E$ by truncating all bids at the fixed threshold of $t = r^{+}$. In general, this does not quite work: for bidder response models that are far from perfect response, this truncation can introduce a very large bias. Here we demonstrate one technique for constructing good estimators $\hat{G}_E$ as long as the bidding function $b(r,v)$ possesses diminishing sensitivity in value to reserve.
Instead of truncating in bid space, we will instead want to truncate in value space to reduce the variance. Even though we cannot directly truncate by values, since $b(r, v)$ is monotonically increasing in $v$, quantiles of bids (e.g., of $\mathcal{B}(r^{+})$ and $\mathcal{B}(r^{-})$) directly correspond to quantiles of values (of $\mathcal{F}$). Instead of setting a threshold $t$ directly on the value, it is therefore equivalent to truncate at a fixed quantile of the bid distribution.
To achieve this, we first sort $X_i^+$ and $X_i^-$ in ascending order. Then we compute $\hat{G}_E$ as
where $q\in [0, 1]$ is the quantile threshold used to truncate bids. The following theorem characterizes the bias and variance of the above $\hat{G}_E$,
Unlike with bid truncation, with quantile truncation we have a clear bias-variance tradeoff as we change $q$: larger values of $q$ decrease the bias (both by decreasing $(1-q)$ and $b(r^{+}, t) - b(r^{-}, t)$, which is decreasing due to diminishing sensitivity) but lead to larger variance. Since one can estimate this bound on the bias (by approximating $b(r^{+}, t) - b(r^{-}, t)$ via $Y^{+}_{qn} - Y^{-}_{qn}$), it is possible to choose $q$ to optimize this bias-variance tradeoff as one sees fit (for example, to minimize $B^2 + V$ in Theorem (ref)). We show a convergence rate result for this quantile truncation approach in Corollary (ref) in Appendix (ref).
We evaluate the performance of our algorithms on synthetic and semi-synthetic data sets. Due to space limitations, we present the complete experimental results in Appendix (ref).
The data generation process consists of two parts: a base bid distribution specifying the distribution of bids when no reserve is set, and a response model describing how a bidder with bid $b$ would update its bid in response to a reserve of $r$.
{\bf Response models. } We assume that in the absence of a reserve bidders bid a constant fraction $\gamma$ of their value $v$ (i.e., $b = \gamma v$), which we refer to as linear shading. We consider linear shading combined with perfect response and with $\epsilon$-bounded response, which we implement by adding a uniform $[0, \epsilon]$ random variable to the bid. We also examine equilibrium bidding for $n$ i.i.d.\ bidders with uniformly distributed valuation krishna2009auction: $b = \frac{r^n + (n-1)v^n}{n v^{n-1}}$.
{\bf Synthetic data. } In our synthetic data sets, the (base) bid distribution is the uniform $[0,1]$ distribution. We apply the perfect response model, $\varepsilon$-bounded response model and equilibrium bidding model. In the simulations, we apply a constant shading factor of $0.4$ for the perfect response model and $\varepsilon$-bounded response model. For equilibrium bidding, we assume that each auction contains $n = 2$ bidders.
{\bf Semi-synthetic data. } For our semi-synthetic data sets, we separately collected the empirical distributions of winning bids over one day for 20 large publishers on a major display ad exchange. Each distribution was filtered for outliers and normalized to the interval $[0, 1]$. For this semi-synthetic data we only test the perfect-response model and $\varepsilon$-bounded response model, since there is no closed-form solution for the equilibrium bidding strategy. We use 0.3 as the constant shading factor for semi-synthetic data.
{\bf Gradient descent algorithms.} We examine five different gradient descent algorithms: (I) Naive GD: naive gradient descent using the gradient estimator in Eq. ((ref)); (II) Naive GD with bid truncation: gradient descent using the gradient estimator in Eq. ((ref)) for the bidding component, and a naive estimate\footnote{We can form a naive unbiased estimator $\hat{G}_D = \frac{\hat{D}(r^+) - \hat{D}(r^-)}{r^+ - r^-}$, where $\hat{D}(r^+) = \frac{1}{n}\sum_i \mathbb{I}\{x^+_i \geq r^+\}$ and similarly for $\hat{D}(r^-)$.} of the demand component; (III) Naive GD with quantile truncation: gradient descent using the gradient estimator in Eq. ((ref)) for the bidding component, and naive estimate of the demand component; (IV) Demand modeling with bid truncation: Same as the second variant, but with a parametric model of the demand curve to estimate demand component of gradient; (V) Demand modeling with quantile truncation: Same as the third variant, but with a parametric model of the demand curve to estimate demand component of gradient. The parameters used in these algorithms are specified in Appendix (ref).
{\bf Demand curve estimation. } To reduce variance following the ideas of Section (ref), we need a model $\hat{G}_D$ for the demand component of the discrete gradient. Instead of estimating $\hat{G}_D$ from historical data, we adaptively learn the demand curve during the training process. Concretely, at each round $t$, we observe new (reserve, demand) pairs from $2n_t$ samples and retrain our demand curve using all the samples observed up to the current round. We use this trained demand curve to compute $\hat{G}_D$ based on (ref). For the synthetic data, a simple logistic regression can effectively learn the demand curve. However, the semi-synthetic data required a more flexible model so for this case we model demand using a fully connected neural network with 1 hidden layer, 15 hidden nodes and ReLU activations.
{\bf Effectiveness of gradient descent.} First, we confirm that gradient descent can effectively find optimal reserves in our models. For each semi-synthetic model, we construct the revenue curve as a function of reserve with assumed response models. We find that 19 out of the 20 revenue curves have a clear single local maximum (the remaining curve has 2). In all cases (synthetic and semi-synthetic models), the revenue learned by the naive gradient descent algorithm is at least 95% of the revenue at the optimal reserve, which indicates that gradient descent can efficiently find the optimal reserve in these cases despite the lack of convexity.
{\bf Effectiveness of variance reduction methods.} We first evaluate the performance of the quantile-based variance reduction method. We run the algorithm variants (I), (III) and (V) under synthetic data and semi-synthetic data with multiple bidder response models. Figures ((ref)) and ((ref)) show the revenue achieved by the three algorithms over time under the perfect response model. We find that quantile-based variance reduction leads to a more stable training process which converges faster than naive gradient descent. Figure ((ref)) evaluates the performance of the three algorithm variants under synthetic data and an equilibrium response model, with similar conclusions. Overall, quantile-based variance reduction outperforms naive gradient descent. Moreover, with the addition of demand curve estimation, algorithm variant (V) achieves better revenue and converges to an optimal reserve faster than the other two algorithms, in agreement with our theoretical guarantees.
We next consider variance reduction using bid truncation, which is used in algorithm variants (II) and (IV). Bid truncation is tailored to perfect response and performs the best overall for this response model, in accordance with the theoretical guarantees, but quantile truncation is competitive and often performs as well over the semi-synthetic data (see Appendix (ref) for a detailed comparison). Under the equilibrium response model, bid truncation can in fact hinder the training process and lead to a substantially suboptimal reserve price (see Figure (ref)). In summary, quantile-based variance reduction coupled with a good demand-curve estimation is the method of choice to achieve good reserve prices under a range of different bid distributions and bidder response models.
\onecolumn