Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
60,047 characters · 12 sections · 33 citation commands
A network simulation of OTC markets with multiple agents
Agent-based modelling (ABM) has become a popular method for simulating financial markets since its initial use in the late 1980s takahashiJASSS, kim_markowitz_1989, palmer_brian, samanidou_zschischang_stauffer_lux_2007, levy_2008. Financial risk modelling intuitively lends itself to this approach. Firstly, real world markets are naturally defined by the interaction of multiple agents, each with a diverse set of functions and behaviours. Secondly, systems with many interacting sub-units often exhibit scale-free phenomena, as is the fractal nature of market prices mandelbrot_fisher_calvet_1997. Emergent phenomena within markets are often difficult to model analytically, and whilst such models exist hagan2014arbitrage, hagansmile, bartlett2006hedging, they are inherently limited by their high-level assumptions of a predefined stochastic process governing the price movements. The complexity and nature of the financial environment has encouraged the use of ABM, which typically makes fewer assumptions about pricing dynamics. Now a staple component of financial risk analysis hassan2018survey, the multi-agent approach allows a better understanding of how certain macroscopic phenomena emerge from microscopic trader behaviours.
Prior works into agent-based financial modelling have used a variety of agents and mechanisms to define the system's behavior vanfossan_dagli_kwasa_2020, palmer_brian, amine, samanidou_zschischang_stauffer_lux_2007, levy_2008, kizaki_saito_takahashi_2021, grothmann, toth_scalas_huber_kirchler_2007, johnson_lamper_jefferies_hart_howison_2001, preis_golke_paul_schneider_2006, jefferies_hart_hui_johnson_2001, thurner_farmer_geanakoplos_2010. Often, these models are specialised for specific investigations, such as exploring the effect of insurance methods on market crashes kim_markowitz_1989 or increased leverage on the kurtosis of the resulting price distribution thurner_farmer_geanakoplos_2010. Substantially all investigations have neglected over-the-counter (OTC) systems in favor of exchange traded markets or auctions preis_golke_paul_schneider_2006, toth_scalas_huber_kirchler_2007, jefferies_hart_hui_johnson_2001, and although some of these studies have considered market makers kirilenko2017flash, paddrik2017effects, paddrik2012agent, bookstaber2015agent, they are considered symmetrically to the other investor classes and do not behave directly as an intermediary, typically trading through a limit-order book via an exchange. This is partly because the additional requirement to model market maker behaviour, whose reactions to trades can be difficult to quantify. However, market makers play a vital role in the financial system by providing additional liquidity to investors. Their function is especially central to OTC markets in products where no exchange exists. In addition, prior works have never considered a financial market with limited visibility between participants. This is another important factor that applies most relevantly to OTC markets, in which not every trade that occurs is visible to all of the participants.
We present a new, open-source agent-based model that simulates price changes within a financial market outside of the exchange setting, whilst also incorporating a parameterized visibility between investors. The model uses three types of agents distributed in a network: value investors, trend investors and market makers. Value investors are initialised with price targets, such that they will seek to buy the product if its current price is below the price target, and vice versa if its current price is higher. Trend investors represent those that rely on technical analysis, using a deep Q-learning approach mnih2013playing to inform investment decisions by analysing the product's visible price history. Trend investors incorporate a convolutional neural network in this process. Market makers intermediate all of the trades, offering to take the opposite end but at a price of their choosing. Market makers are in direct competition with each-other as trend and value investors always select the most competitive market maker to trade with. In order to keep their prices as accurate as possible, market makers adjust their prices based on the trades they see happening, ensuring that their mid-price reflects the current clearing level by always moving it to the last visible trade price. In order to reflect the market makers' goal to recycle risk, they will additionally move their prices higher or lower depending on their inventory. We also incorporate a soft risk limit to market makers, which has been shown to be an important factor in financial markets, and has empirically been linked to flash crashes kirilenko2017flash.
In this paper, we explore the ability of the proposed model to reproduce emergent phenomena observed in real financial markets. In particular, we demonstrate that the model produces fat-tailed distributions of price changes that follow a power-law rank distribution kim_yoon_chang_2004, warusawitharana_2016; that prices converge to the mean views of value investors; and that the model exhibits realistic, reproducible and predictable features within its price patterns (often referred to as “technicals"). In addition, we use the network feature to demonstrate that our market exhibits higher kurtosis when distributed across increasingly sparse networks, and also a critical point of fragmentation, beyond which arbitrage between the prices of different market makers becomes possible.
We use the NetLogo platform wilensky1 to simulate our agent-based model. NetLogo is a widely used platform for agent-based modelling, and has been used to study complex group dynamics from flocking to predator-prey interactions wilensky4, wilensky5. NetLogo provides an intuitive interface that allows users to adjust parameters, run experiments and visualise the phenomena being modelled, and as such, is perfect for an open-sourced model such as the one presented in this paper.
The model comprises three types of agent: market makers, value investors and trend investors distributed on a network topology. The behaviours of market makers and value investors are pre-determined, whilst that of trend investors is determined using the machine learning algorithm deep Q-learning mnih2013playing, mnih_human-level_2015. At each time step, an investor (which comprises a value investor, a trend investor, or a market maker in breach of their soft limit) is selected at random, and allowed to execute a trade versus a market maker that is connected via the network. Once the trade has occurred, the agent's inventory is updated, along with that of the market maker acting as counter party. At each time step, all market makers are allowed to update their prices. The exact mechanism will be laid out in full.
Within the model, interaction and visibility of all agents is constrained to occur along the edges of a network. We formalise this using the network model outlined in Figure (ref).
Reflecting that all interactions within our model must occur through market makers, we enforce that at least one market maker must be involved in each network edge, which is a constraint that deviates our network topology from that of a typical random network. The process outlined in figure (ref) connects each agent to each market maker with a user-defined probability $p$ that also parameterizes the network. Market makers are also connected to one another as part of this process, but no self-connections are allowed. We also enforce that at minimum, each agent is connected to at least one other agent that is a market maker to mitigate the possibility of an agent being entirely isolated from the system.
The network defines the visibility of each agent with the goal of inhibiting the transfer of information throughout the market. As such, within our model, market makers will only react to trades that occur with a market maker that is a direct network neighbor. This visibility in the real world would be representative of price discovery through the inter-dealer broker market. Similarly, trend investors that analyse the price-history only consider those prices that were advertised by market makers that are direct neighbors on the network.
We notice that the network architecture implied by a system with a central limit order book, as has been used in the majority of the prior literature, resembles a star network, with all agents connected to one single intermediator. Our inclusion of multiple intermediators (market maker agents) drastically increases the complexity within our model, allowing us to perform investigations into the effect of market structure on pricing dynamics that would be beyond the capability of a limit order book model. We perform such investigations ourselves in section (ref).
Market makers always advertise two prices - a bid price at which they are willing to buy, and an offer price at which they are willing to sell. The difference between these prices is referred to as the bid-offer, and is a fixed parameter determined by the user that is constant across all market makers within our system. We refer to the middle of these prices as the mid price. Defining a market maker's mid price as $P_{mid}$, with bid-offer $b$, a market maker's price when either buying or selling is given by equation (ref).
where
Equation (ref) formalises that a market maker's buy price is lower than their sell price. Market makers calculate their mid by considering both the last trade price that they observed (through a neighboring market maker on the network) and their inventory. Parameterizing the sensitivity of market maker prices to their inventory level with the proportionality $a$, we define a market maker's mid price as in equation (ref).
where $P^{m}_{last}$ is the last trade price seen by market maker $m$, and $I^m$ is their current inventory level.
The simulation begins by setting an initial price of 100, and market makers initialise their mid to this level. On the first iteration, all market makers begin by advertising the same prices, and the first transacting agent will choose their counter-party randomly from the symmetrical market makers. However, the market maker behaviours will diverge as their inventories evolve. When executing a trade, we consider the market maker's inventory following the transaction, so that if market maker $m$ had an initial inventory $I^m$, and was transacting $S$ units with a bid offer $b$, they propose a transaction price given by $P^m$ in equation (ref).
The value of $I^m + S$ is positive if the market maker is net long following the transaction, and negative if the market maker is net short. Our choice of a linear relationship between price and resulting inventory level is arbitrary, and a number of alternative functional forms were also tested (specifically quadratic and exponential forms along with varying values for $a$). However, no significant effect was observed, and so we progress with the functionality outlined in equations (ref) and (ref), and our value of $a$ being 0.001. This formulation implies that market makers will alter their prices lower when looking to reduce inventory, and higher when looking to increase inventory, which is qualitatively what we desire. The transacting counter party will always select the market maker with the most competitive price. If the most competitive price is shared by two or more market makers, one is chosen at random to trade with.
At initialisation, each value investor is given a target price of the product, which is drawn from a bimodal Gaussian distribution with a mean of 100. Our consideration of a bimodal distribution allows us to model common market scenarios where investors take views on a binary outcome, as is common around events such as political elections. The mean target price of 100 is arbitrarily chosen, and the user has full control over the distance between the centroid of each Gaussian mode so that a conventional, single-mode Gaussian can be used by setting this distance to zero. The targets of the value investors are static, and never change. Heuristically, this allows us to interpret the value investor target as a long-term view on the price that doesn't change over the time frame of our simulations.
A value investor’s action is to compare the prices of market makers, and choose to buy if the most competitive offer is lower than their target price and to sell if the most competitive bid is higher than their target price. In scenarios where a value investor can profit from both buying and selling versus different market maker prices, the investor will choose the action that maximises the difference between the trade price and their target price, thus maximising their expected profit. As is typical in a real financial market, the trade size is variable, and to reflect value investors' reaction to differing opportunity sizes, we determine that their trade size grows linearly as the difference between the investor’s target and the market price increases. Specifically, value investors calculate their trade size as
Where $P$ is the most competitive transaction price provided by the market makers, $T_{v}$ is the value investor's target, and we choose a proportionality of $\sigma=5$ which was found to generate an appropriate distribution of trade sizes involving value investors. The trade size is capped to a maximum value determined by the user. This is an intuitive restriction found within real financial markets. Equations (ref) and (ref) are simultaneous equations with an analytic solution for a trade price and size that satisfies both the value investor and the market makers behavior when they interact. Heuristically, this approach formalises the negotiation mechanics that would typically occur between a market maker and their client during a trade. In practice, we use the solution to determine the trade characteristics that consistently satisfy the behaviour of both market makers and value investors whilst transacting. Equation (ref) presents this solution.
Meanwhile, market makers are also allowed to trade versus one another in order to distribute risk between them via the inter-broker market. Each market maker that is beyond their position limit is allowed to act as an investor, asking the other market makers that are also network neighbours to trade a size that would reduce their inventory to zero in a single trade, capped at the maximum allowed trade size defined by the user. The logic here is similar to before, with the transaction price calculated using equation (ref). This behaviour can be toggled by the user, as outlined in section (ref).
Recently, interest in the intersection between reinforcement learning (RL) and agent-based modelling has exploded in popularity tianxu2022applications, canese2021multi, terry2021pettingzoo, vinyals2019grandmaster, wiering2000multi. RL has offered the means to explore multi-agent environments that require complex and nuanced agent strategies. In turn, the ABM approach has become an integral component of leading RL systems, where it is commonplace to use ABM as part of the training process. These synergies have led to the sub-field of Multi-Agent Reinforcement Learning (MARL) zhang2021multi, which find use in the state-of-the-art systems solving autonomous driving shalev2016safe, the game of Go silver2016mastering, silver2017mastering, and questions in finance lee2007multiagent.
We choose to construct our trend investors as artificially intelligent agents, with the goal of simulating investors that use technical analysis to guide their trading decisions. As in the real market, these trend-spotting investors act to remove inefficiencies within the system by profiting from their exploitation. Repeatable price patterns within the system can evolve over time as trend investors dynamically learn to profit from them. Once enough trend investors have recognized this same strategy, their collective investment activity can reduce, and even eradicate the pattern altogether. As such, the inclusion of artificially intelligent trend investors opens the doors to exciting and new research investigating the interplay of dynamically competing RL agents, the temporal evolution of stock price patterns, and the impact that different trading strategies can have on the underlying market.
We use deep Q-learning mnih2013playing, mnih_human-level_2015 with a convolutional neural network as outlined in figure (ref). Within this framework, our state vector is defined as the most recent 512 historical price data points. Each element within the state vector, for a given trend investor, is formed from the mean of all network-connected market maker mid prices at that snapshot in time. This implies that the state vectors of two trend investors are not necessarily equal, even if they are sampled at the same moment in time. This set-up allows us to define the Q-value associated with taking action $a$ from state $s_t$ using Bellman's equation, outlined in equation (ref).
where $R(s_t, a)$ is the reward, which we measure as the profit resulting from executing a trade defined by action $a$, from state $s_t$, $g$ is a discount rate used to attenuate the value of future rewards, and $s_{t+1}$ is the next state vector, which is the state vector for the agent at the moment that a trade is closed and the reward is realised. The central theme to deep Q-learning is to use a neural network to estimate the values of $Q$.
We further simplify the deep-Q paradigm by enforcing that trend investors consider trades individually, and the Q-network aims at choosing the action that maximizes the profit on this particular trade, without regard of the trend investor's other positions. This is a preferable approach to model our trend investors with, because it presents a simpler training paradigm for the model to learn in. In defining the Q value as the profit associated with a trade, plus a discounted optimal value of the resulting state, we achieve consistent training stability, and meaningful trading strategies being adopted by the trend investors.
The Q-network takes the most recent 512 items of the price history as input, which is passed through the convolutional and max-pooling layers and a four-layered fully-connected head with one output for each possible agent action.
A trend investor can choose between seven actions, when prompted to trade. These seven actions span the full six combinations of trade-directions (buy or sell) and holding-times (100 ticks, 250 ticks or 500 ticks); and also the additional action allowing them to do nothing, which can be useful in the case where all trades are considered unprofitable. All trading actions have the same trade size. This is because variable trade sizes should produce rewards that are direct multiples of one another, such that a well trained agent would always choose the action with the largest allowed trade size. Therefore, we choose to define all trend investor trades to use the maximum trade size allowed within the system, which is a user defined parameter, defaulted at 3 units for this paper's results. The trade time is specified as the number of time steps that must pass before a trend investor's trade can be closed. If the trade-time for a specific trade becomes zero (or negative) during the model's progress, the trend investor will deterministically close this trade at their next turn, at which point the reward (the profit) for the trade is calculated and used to train the agent. At this point, we save four observations in the agent's replay memory: the initial state that was present at the time of the trade, the action chosen, the reward that resulted, and the state vector of the environment when the trade was closed.
Following each action of a given agent, we sample a batch from the agent's memory, and train the Q-network using a mean-squared error loss on equation (ref) so that our Q-network learns to predict the value associated with actions given an initial state. Each time this update occurs, the parameter $\epsilon$, which is responsible for varying the agent's proportions of exploration and exploitation, is reduced by a constant factor (we chose to use a factor of $0.995$, which produced stable training).
The chaotic nature of price movements within the model poses a challenge for stable training, especially in the context that deep Q-learning has notorious instability. In the context of our trend investors, unstable training would result in unprofitable and non-meaningful investment strategies. In order to address these stability issues during training, we incorporate a soft-update to the Q-network as first presented by Mnih et. al, whose architecture closely resembles that of our own mnih_human-level_2015. For completeness, our algorithm is outlined in (ref).
Before generating each set of results, we train the newly-initialised trend investors in the environment until they have reached convergence, as defined by the value of $\epsilon$ reaching a minimum allowed value of $0.05$, and the mean-squared error of the Q-network converging as displayed in figure (ref). Some prior work paddrik2017effects has also introduced an additional set of agents that act randomly in order to capture the effect of additional investor behaviour, and so we note that our trend investors fulfill this same role because of their non-zero terminal value of $\epsilon$, which leaves them infrequently performing random actions as the simulation progresses.
Figure (ref) shows the mean-squared error loss of the Q-network on logarithmic axes, with a decay rate of $\epsilon$ of $0.995$, a learning rate of $1 \times 10^{-3}$ with an Adam optimizer kingma2014adam, and a soft-update parameter $\tau=1e-3$. For the environment, the default parameters outlined in (ref) were used. To put an intuition on the scale of the MSE loss shown in figure (ref), the loss associated with a randomly acting agent trading with trade times of 250 time steps is approximately $27\%$ (price standard deviation is approximately 9%, and using a maximum trade size of 3), which translates into a standard deviation of rewards being $0.27$. Thus, MSE errors below this level, such as those shown in figure (ref) which reach below levels of $0.01$ indicate the trend investors having developed good and economically-profitable strategies. We prove this empirically by analysing the profit of trend investors over time, as is shown in figure (ref).
Value investors are initialised with a static target of the market price. We distribute these value investor targets according to a mixture model of two equally-proportioned normal distributions with means separated by a model parameter $\Delta \mu$, which extends the model to consider a market that is split between two expectations of the price. This scenario is of high relevance to the financial market which is often active around binary events, such as elections.
Equation (ref) outlines the process by which we draw value investor targets. We choose to use a value of $\sigma=5$ arbitrarily based on our typical ranges for $\Delta \mu$, which varies up to 20.
The model has the following list of parameters that can be determined by the user.
The model also includes the following additional functionalities.
We examine the effectiveness of the model through a number of statistical tests that are prevalent throughout the literature kim_yoon_chang_2004, warusawitharana_2016. Unless specified otherwise, we use the default parameters for the model outlined in (ref), and only consider results from where all trend investors are fully trained. Specifically, we look for quantitative features in the price distribution that reflects the fat-tailed distributions observed in nature, such as a kurtosis above 3 and a Zipfian power-law between price movement sizes and ranks. We additionally illustrate a number of qualitative features exhibited by the model that highlight further similarities to reality.
Firstly, the distribution of price changes within the model can be shown to follow fat-tailed distributions, as illustrated in Fig (ref).
We can see from Fig (ref) that the model produces price returns that are inadequate to be described with a normal distribution: a fact that is particularly evident when looking at the tails of the distribution on a logarithmic scale. Both these fatter tails and the clustering of price-changes that are closer to the distribution mean explicate a distribution with a kurtosis greater than 3. Our model consistently produces price distributions with kurtosises above 3. We also note the nature of the distribution tails, which becomes clear in the lower plot of Fig. (ref). As the tails deviate from the normal distribution, they become approximately linear in the logarithmic scale, alluding to a power law relationship. This behaviour has been noted to exist in the observed price movements of the Dow Jones warusawitharana_2016.
We can directly show that the price changes produced by our model follow a power law, also known as Zipf's law ZipfOrig. Although power laws are found throughout nature, particularly in systems comprising multiple agents, this is a comparison that has not been included in the previous literature on agent based models of financial markets. Zipf's law has been demonstrated to apply in practice to stock price movements kim_yoon_chang_2004, and it is promising that our model replicates this phenomenon as shown in Fig. (ref).
We also find that our model exhibits phenomena that qualitatively match what we would expect in a real financial market. For example, the skew within the price distribution is negatively proportional to the positioning of market makers. Since data on market maker positioning is highly confidential, it is impossible to make quantitative comparisons to data. However, this relationship is highly intuitive, reflects the effect of the model's market makers competing to reduce their inventory, and is a notable feature of the proposed model. This relationship is shown in fig. (ref).
Further, we note the tendency of the market price to settle at-or-close-to the numerical mean of value investor targets. Again, although this phenomena is difficult to prove in the real world (because value investor target prices are proprietary and confidential), it intuitively exists within real-world markets, with participants routinely interpreting price levels between binary outcomes as an indicator of probability. This feature is a necessary component of the efficient market hypothesis, and in this regard, the market acts as an ensemble model with a mean estimator across its participants. As in real financial markets, our model demonstrates this property for the majority of situations but not all: there are situations where the market price can diverge largely from this mean target and the behaviour is lost, particularly in extreme instances of high volatility.
The price within the system converges to the mean regardless of our chosen distribution of value investor targets. As is illustrated in Fig. (ref), the price quickly converges to the mean view in a system where value investor targets are drawn from a bimodal mixture distribution with two peaks. This builds on our previous findings, suggesting that the mean target of value investors acts as an attractor even when the targets are distributed arbitrarily. This behaviour is caused by the value investors scaling their trade size by distance-from-target, as outlined in (ref). The varying trade size with price gives rise to an equilibrium level. We hypothesise that the convergence to the unweighted mean is a direct result of all value investors sharing the same proportionality between difference-to-price-target and trade size. This parameter, $\sigma_{vi}$, is introduced in (ref).
We also note the existence of patterns and trends within the model's price movements. These are particularly interesting from the perspective of technical analysis (the practice of predicting future prices using only historical price information), which is used by our trend investors. One example is during large price movements, when the market price will typically overshoot the equilibrium level before retracing a smaller amount. This is a widely observed phenomena in the real world that is visible following the vast majority of market crashes. We recover this same technical within our model, as shown in figure (ref).
In order to induce the market crash displayed in figure (ref), we instantaneously reduce the price targets of all value investors by 20%. Our prior findings that the market's equilibrium level is strongly influenced by the value investor targets imply that this should be an effective approach to instigating a market crash. Additionally, this is a realistic approach, since sharp drops in financial prices are often prompted in the real world when new information is revealed (via news updates or the publication of financial results) which moves investor expectations. As can be seen in fig. (ref), the model exhibits a characteristic rebound following the initial overshoot of the price. The overshoot occurs entirely through a combination of market maker positioning as they overshoot the equilibrium price in their plight to recycle inventory, along with trend investor strategy. This behaviour has been noted in prior ABM literature, which through the empirical study of liquidity-driven crises has concluded that financial crashes exhibit this artifact because of market makers competing to reduce their inventory against the direction of the market kirilenko2017flash. Following the initial drop, market makers within our model are eventually able to reduce their inventory during the initial price overshoot, in which value investors actively play a role in purchasing market maker inventory that is being offered at a lower price than their target level. Whilst the market price is finding its new equilibrium level, the scale of price movements is visibly increased, demonstrating the auto-correlation and clustering of volatility. This is also a significant feature present in the real financial markets, with volatility clustering being used as a key parameter in sophisticated models such as the SABR model hagan2002managing - one of the most widely used options-pricing models in the modern world.
Within our model, each trend investor is initialised with their own deep-Q network, therefore allowing dynamically evolving strategies and competition between them and their environment. Whilst the convergence of the models was verified as in section (ref), we aim here to present an analysis of the strategies being learned by these artificially intelligent agents.
As can be seen in figure (ref), two distinct clusters of weight optimums appear. We have visualised these two clusters using principal component analysis. Plotting the principal component of the fully-trained trend investor's weights, we are able to visualise a low-dimensional representation of the weights similarities on a scatter plot.
The lower chart in figure (ref) shows that the trained trend-investors select actions corresponding to selling (which have action IDs of 0, 1 and 2) when the price is at the upper end of it's historical range, and select actions corresponding to buying (which have action IDs of 4, 5 and 6) when the price is at the lower end of its historical range. This is a strong indication that the trend investors are primarily learning a strategy that corresponds to mean-reversion trading. This is the most intuitive strategy for our given set-up, since the deterministic behaviour of value investors has the effect of bounding the price range. The trend investors within our market compete over profit, since each trend investor trade that follows a mean-reversion strategy will implicitly reduce the size of the available profit following the same strategy that is available to the other trend investors. For example, in the act of selling when the price is at the top of its historical range in anticipation of the price returning towards the center of its trading range, the trend investor would transact with a market maker thus placing downward pressure on the market price, and reducing the scale of the price move. This is central to a trend investor's function as a dynamic learner: by continually adapting to profit from the market's movements, the trend investors reduce the size of the market's inefficiencies. We take the empirical evidence that the trend investors develop profitable strategies from figure (ref) as validation that they are acting towards this end.
By altering the network structure within the model, we are able to analyse the system under different market structures. This is facilitated both through our novel consideration of a network topology, and also our use of market makers as the sole intermediaries within the system. Prior literature that has used an order book or exchange as the intermediary cannot perform these investigations, since with one central intermediator, all agents are only connected through one edge to the central node.
We focus on varying the probability of a link forming within our network structure outlined in section (ref), denoted as $p$. Decreasing the value of $p$ has the effect of limiting the visibility of agents within the model. This is an important consideration of an OTC market, which has not been considered in prior work. We observe that in more fragmented network structures, with fewer links and a lower value of $p$, market maker prices begin to diverge from one another. This is illustrated in figure (ref).
In an efficient market, market makers' prices should differ by a relatively small amount. However, in reality, arbitrage opportunities can emerge when one market maker's buy price drifts above the sell price of another market maker. The arbitrage in this case is defined as the profit that would be made from instantaneously buying from one market maker, and selling to the other. Whilst arbirage is not directly capitalisable by any agents within our model (since the system is limited to one trade per time-step), we can directly observe this arbitrage opportunity. Defining the arbitrage opportunity as the difference in price between the most competitive bid price, and the most competitive offer price across all market makers, we find that as the value for $p$ decreases towards approximately 30%, arbitrage opportunities increase in frequency. These results are displayed in figure (ref).
As can be seen in figure (ref), when $p$ is greater than 30%, these arbitrage opportunities typically last for short periods of time, and are relatively small. However, for values of $p$ lower than 30%, the market undergoes a phase change, where these arbitrage opportunities rapidly grow in size. This phase change is not only characterized by a dramatic increase in arbitrage opportunity, but also by the nature of the market prices that ensue. As shown in figure (ref), this phase change involves a separation of the market into multiple, distinct clusters. The example shown in figure (ref) shows an example where the market is split into two clusters. Each cluster behaves as a sub-market within the overall system, with it's own prevailing price, complex behaviour and supply and demand dynamics. We infer that each cluster that forms when the market fragments comprise a group of agents with relatively high inter-connectivity, and there is a minimal, but not necessarily zero, number of edges between different clusters. Figure (ref) illustrates the simulation's network topology, which demonstrates that the behaviour seen in figure (ref) corresponds with the market markers being separated into two unconnected clusters. Figure (ref) also demonstrates that each of the two prices produces a fat-tailed price change distribution, with a kurtosis above 3.
The phase change also has an effect on the overall kurtosis of the resulting price change distribution. As shown in figure (ref), we can see that for values of $p$ lower than approximately 40%, kurtosis steeply rises from values of around 3, to be increasingly fat-tailed. We can conclude from this that limited visibility within financial markets can be a contributing factor to their characteristic fat-tailed distribution. We hypothesise two reasons for this: firstly, that limited visibility impairs the market makers' ability to recycle risk to the investors. As such, it is plausible that a build-up of inventory within the market maker's portfolios could give rise to more frequent dramatic price changes within the market. Secondly, once the market has fragmented into separate clusters (as is relevant for values of $p$ below approximately 30%), it is plausible that each of the clusters (in isolation) exhibits a price change distribution that differs from the other clusters. Our measure of kurtosis throughout this paper is from a distribution comprised of all market maker price movements. Thus, we are implicitly summing the price change distributions across all of the clusters, which we hypothesise could have an increasing effect on the kurtosis value.
We have presented a novel agent-based model for simulating an OTC financial market, that is unique in its consideration of limited market visibility, and its implementation of market makers as the sole intermediaries. With this approach, combined with a network topology to the underlying market, we are able to perform investigations into the effect of a market's structure on it's function. The models mechanics have been outlined, and demonstrated to replicate a wide range of features that are observed in real financial markets. For example, price changes follow fat-tailed distributions that follow Zipf's law when ranked kim_yoon_chang_2004, and a power distribution attenuation in the tails warusawitharana_2016. The model also reproduces a number of qualitatively desirable features, such as a negative correlation between market maker positioning and the skew of the price change distribution, a tendency to converge to the mean view of investors, and the exhibition of characteristic and repeatable price patterns.
We have used our model to examine the effect of network topology on the market function. This is a particularly relevant line of examination in OTC markets which often have lower visibility and liquidity than exchange traded environments. We find that markets that conform to the same network model as we have outlined in section (ref) exhibit a critical point of fragmentation, beyond which a phase change occurs. Following such a phase change, market function quickly deteriorates as the opportunity to profit from arbitrage between different market maker prices rapidly grows in size. We have demonstrated that this phase change is often characterised by the market fragmenting into distinct clusters of agents. Each cluster in this paradigm behaves heuristically like an independent market, with its own price, agents, and complex behaviour. Further, we have demonstrated that within this model, the kurtosis of the market's price change distribution increases as market visibility decreases. We demonstrated that this increase occurs at the aforementioned phase change of the market, where the market is characterised by a value of $p$ lower than approximately 30%.
There is room for further work in investigating the market clusters that form beyond the aforementioned phase change. Whilst each cluster contains its own subset of agents, instances can occur when one sole agent is connected to two distinct clusters. In this case, often this sole agent is not able to trade enough volume to pull the prices of both clusters together. As such, an agent such as a value-investor would effectively act to profit from the arbitrage between the prices of both clusters. However, the influence of these bridging agents does have an effect on the prices of each cluster, and we hypothesise that complex dynamics and correlations can exist between the two prices as a result. This would translate naturally to a physical interpretation, where markets are separated by time zone and geography, and it is both intuitive and common (especially for illiquid products) that prices can differ between the financial markets in Asia and Europe or the United States of America.
Currently, our trend investor implementation is based on deep Q learning, which limits the scope of the trend investors' to a discrete action space, which in part facilitated our analyses of the trend investor strategies. Extending these agents to use reinforcement learning algorithms designed for continuous action spaces (for example, actor-critic bahdanau2016actor), would be a natural progression. Market makers could also be adapted to also use a machine learning algorithm, and it is particularly relevant that a market maker's bid-offer spread is not always constant in the real world, but varies with a number of factors including volatility. Artificially intelligent market makers could have the ability to choose their bid offer spread independently, with the goal of maximising their profit in a competitive environment. This is an exciting and new paradigm to consider within the MARL literature.