arXiv 14 Jul 2024 · Finance — Trading · 3 citations (OpenAlex)
arXiv:2407.21025 · PDF · DOI · OpenAlex · Extracted main text
This paper establishes a new and comprehensive theoretical analysis for the application of reinforcement learning (RL) in high-frequency market making. We bridge the modern RL theory and the continuous-time statistical models in high-frequency financial economics. Different with most existing literature on methodological research about developing various RL methods for market making problem, our work is a pilot to provide the theoretical analysis. We target the effects of sampling frequency, and find an interesting tradeoff between error and complexity of RL algorithm when tweaking the values of the time increment $\Delta$ $-$ as $\Delta$ becomes smaller, the error will be smaller but the complexity will be larger. We also study the two-player case under the general-sum game framework and establish the convergence of Nash equilibrium to the continuous-time game equilibrium as $\Delta\rightarrow0$. The Nash Q-learning algorithm, which is an online multi-agent RL method, is applied to solve the equilibrium. Our theories are not only useful for practitioners to choose the sampling frequency, but also very general and applicable to other high-frequency financial decision making problems, e.g., optimal executions, as long as the time-discretization of a continuous-time markov decision process is adopted. Monte Carlo simulation evidence support all of our theories.
appendix boundary found by appendix_command · 65% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Avellaneda, M. and Stoikov, S (2008) High-frequency trading in a limit order book | 0.874 | 6 | 2 | 100% |
| 2 | Cont, R. and Xiong, W (2022) Dynamics of market making algorithms in dealer markets: Learning and tacit collusion | 0.874 | 5 | 2 | 100% |
| 3 | Even-Dar, E., Mansour, Y., and Bartlett, P (2003) Learning rates for q-learning | 0.811 | 5 | 2 | 80% |
| 4 | Filar, J. and Vrieze, K (2012) Competitive Markov decision processes | 0.781 | 7 | 2 | 71% |
| 5 | Guéant, O., Lehalle, C.-A., and Fernandez-Tapia, J (2013) Dealing with the inventory risk: a solution to the market making problem | 0.737 | 3 | 2 | 100% |
| 6 | Hu, J. and Wellman, M. P (2003) Nash q-learning for general-sum stochastic games | 0.737 | 3 | 2 | 100% |
| 7 | Luo, J. and Zheng, H (2021) Dynamic equilibrium of market making with price competition | 0.737 | 3 | 2 | 100% |
| 8 | Guo, X. and Hernández-Lerma, O (2005) Nonzero-sum games for continuous-time markov chains with unbounded discounted payoffs | 0.644 | 5 | 2 | 40% |
| 9 | Gihman, I. I. and Skorohod, A. V (2012) Controlled stochastic processes | 0.644 | 3 | 2 | 67% |
| 10 | Sutton, R. S. and Barto, A. G (2018) Reinforcement learning: An introduction | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 41 scored citations.