The Physics of Finance

Natural models of markets

2012-01-17T06:16:00.000-08:00

I've written quite a bit about the shortcomings of traditional economic models of markets. Most notably, such models -- typically characterized by rational agents the actions of whom lead (by assumption) to a market equilibrium of some sort -- generally fail to explain basic dynamical features of real markets. These include, most prominently, 1) a pronounced tendency in all markets to large price fluctuations reflected in "fat tailed" distributions of returns, and 2) vigorous and persistent volatility with delicate long-memory features, which is closely linked to the way that episodes of volatility "cluster" in the market.

These phenomena show up in specific mathematical signatures in the statistics of price movements, and these signatures present some of the most obvious details that any decent model of markets ought to reproduce quite naturally. Models capable of doing this have only emerged in the past two decades, and only succeed by making a clear break with the neo-classical tradition of equilibrium. This short review from 2006 by economist Blake LeBaron gives a nice introduction to the motivation for this kind of work, which very much follows in the tradition of natural science (as opposed to much of modern economics) in pursuing explanations in models with make plausible assumptions and explore the conclusions which follow from them.

This is a topic I'll be writing about more in the next few months (in connection with a longer term project I'm working on). Le Baron's review is a little old (5 years) and needs updating, but he describes some crucial points very effectively. A few highlights are worth mentioning:

Le Baron first outlines some of what he calls the "major puzzles of financial markets," which are directly linked to the things I just mentioned. First is simply the existence of pronounced volatility:

Volatility is the most obvious and probably the most important puzzle in finance. Why do financial prices and foreign exchange rates move around so much relative to other macro series both on a short term and long term basis? The difficulty of overall financial volatility was first demonstrated in Shiller (1981), and an update is in Shiller (2003). The issue has been that it is difficult to find financial or macro economic fundamentals that move around enough to justify the large swings observed in financial markets. As a potential policy problem, and an issue for long range investors, this might be the most important puzzle faced by financial modelers.

As Le Baron notes, lots of models (ARCH, GARCH, etc) have been produced which reproduce the mathematical character of volatility but without even attempting to understanding its origin. Moreover, most models in traditional economics, by sticking to the view that individuals are more or less identical and have rational expectations, simply shy away from the very phenomenon needing to be modeled:

The persistence of volatility in many financial markets has led to an entire industry of models, and is an area of intense interest both in academic and commercial areas. However, although there is a lot of empirical activity, the underlying microeconomic motives for volatility persistence are still not well understood. There are very few models which have even tackled this problem. This is probably due to the fact that in a homogeneous agent framework this is simply a very difficult problem.

The second puzzle of markets that Le Baron lists is what I mentioned first above: the fat tails of market returns or "excess kurtosis" in the statistical lingo:

Financial returns at relatively high frequencies (less than one month) are not normally distributed. There is not much of a strong theoretical reason that they need to be, but the hope has often been that some form of the central limit theorem should drive returns close to normality when aggregated over time. Recently, a new field, Econophysics, has appeared which stresses that returns also have additional structure that can be described using power laws. The determination and testing of power laws remains a somewhat open area, and the set of processes that generate acceptable power law pictures is also not well understood.

Aside from the long memory of volatility and fat tailed returns, he also mentions the rich dynamics of trading volume, which is equally as interesting as those of prices. Unfortunately, when it comes to the dynamics of volume, Le Baron notes, "Most traditional financial models remain completely silent."

Le Baron rightly points out that these rich dynamics in time are almost certainly linked directly to two things: 1) the fact that different market participants have different expectations at any moment (indeed, it is such differences which drive trading) and 2) that these differences undergo perpetual evolution through time. This suggests that a decent explanation of the origin of these so-called "stylised facts" (and others like them) will likely emerge from models which attempt to follow and capture something about the dynamics of beliefs and expectations in a population of diverse interacting agents:

Many of the most puzzling results from finance deal with problems of behavioral heterogeneity, and the dynamics of heterogeneity. The study of market heterogeneity as a kind of complicated dynamic state variable that needs to be modeled is probably one of the defining features of agent-based models. Empirical features such as trading volume are directly related to the amount of heterogeneity in the market, and demand models that can speak to this issue. Other empirical features are probably indirectly related. Large moves, excess kurtosis, and market crashes all probably stem from some type of strategy correlation that keeps the law of large numbers from functioning well across the market. These changing patterns can only be explored in a framework that allows agent strategies to adapt and adjust over time...

I think this point about the failure of the law of large numbers is, while obvious, still worth emphasizing. Something in the market ruins the simple picture of normal statistics, which would emerge from independent factors driving prices changes. Somehow the actions of different market participants must come to be correlated, thereby leading to non-normal fat tailed returns and long memory effects. In principle, of course, there may be many mechanisms contributing to this lack of independence (some participants following trends could be enough, for example).

Now, I doubt anyone actually working in finance would find this observation anything but banal. Yet many economists seem determined to continue modelling markets as if this were not the case. I don't know enough economic history to know why the homogeneous rational expectations view has such a following, but it seems very weird to me indeed.

The rest of Le Baron's review explores an example of the kind of model -- an agent based model for a market very much out of equilibrium -- which reproduces the above features quite naturally, at least qualitatively. The idea is simply to respect the fact that agents in a market are different and change their behaviour over time, adapting to what happens in the market. There is no presupposition that market prices must settle down to an equilibrium; the market does what it does as people interact and trade and try to profit as well as they can. The simple model explored here is one developed by Le Baron in 2002, but shares basic features with many other models developed by others. Agents can choose between a risky asset (a stock) and a risk free bond, and they use a variety of information to make their trading strategies:

Agents chose over a set of portfolio strategies that map current asset market information into a recommended portfolio fraction of wealth in the risky asset. This fraction can vary from zero to one since short selling and borrowing are not allowed. Information includes lagged returns, dividend price ratios, and several trend indicators. Agents must evaluate rules using past performance, and it is in this dimension where they are assumed to be heterogeneous. Agents use differing amounts of past information to evaluate rules. In other words, they have different memory lengths when it comes to evaluating strategies. Some agents use 30 years worth of data, while others might use only 6 months. In this way this model implements to behavioral features. First, agents are clearly boundedly rational in that they do not attempt to determine the entire
state space of the economy, which would be unwieldy if they attempted this. Also, they are assumed to have “small sample bias” since they don’t all choose to use as much data as possible.

The paper is an easy read so I won't get into much detail, but what emerges from the interaction of learning and adapting agents in a setting like this is immediately much more realistic and interesting than anything coming from traditional models. For example, the figure below shows the price of the risky asset as a function of time. In the model, there is a true equilibrium price which is made to fluctuate in a normal, Gaussian way (this price being linked to dividends). The actual price in the market is rarely at this equilibrium, but instead has large fluctuations around it, being some times far too high and at others too low, and often moving very rapidly from one point to another.

Now there are some features of this time series that don't look realistic. There seems to be a periodicity of sorts, for example. But this is a very simple model and the exciting thing is what it gets right -- easily giving a model of a market which never settles down to a prices, has persisting volatility, fat tails in returns, rich dynamics for trading volume and so on. Details can be found in the paper.

As Le Baron concludes the review,

Agent-based models make more progress than other frameworks in explaining these features due to that fact that at their core is a world of people who process information differently, and try hard to continually adjust and adapt their behavior over time. This market may never reach anything that looks like an equilibrium efficient market, but it is in a continual struggle toward this. The range of facts these models explain, and the robustness of their explanations to different structures and parameters, is impressive. At the moment, no other models can capture this many facts with this kind of simplicity and style.

As I said, that was 5 years ago. The final statement remains very much true, however. I think it is becoming increasingly clear, even to most economists, that agent-based models present currently the best technique to go beyond the restricting and unrealistic assumption of equilibrium economics and to build models of markets which get their basic behaviour right.

I intend this post just to be a beginning of an exploration of what is currently happening in the development of more realistic non-equilibrium market models. The field is currently exploding and there are many interesting questions to tackle. For example, models of this kind often involve quite a large number of parameters describing agents strategies and so forth. Yet many models work very similarly despite large differences in such parameters. An open question is whether it may be possible to develop a kind of map of the space of possible models, showing a set of classes into which different models fall. Physicist Luciano Pietronero has done some interesting work in this direction.

In any event, I find it hard to understand why anyone today would go on studying homogeneous rational equilibrium, at least in application to the financial markets.

Rational -- by definition and ideology

2012-01-09T06:45:00.000-08:00

I've been doing some background reading on rationality in economics, and came across this fairly unique perspective offered by economist Duncan Foley. It's from 2003. What sets it apart from most other reviews of the role of the rationality assumption in economics, is that Foley tries to trace the history of this approach as it emerged out of the tradition of Hobbes and Locke in political philosophy. As Foley notes, the idea of rationality is in many ways beyond question for most economists, and not at all an empirical matter:

...an orientation toward situating explanations of economic phenomena in relation to rationality has increasingly become the touchstone by which mainstream economists identify themselves and recognize each other. This is not so much a question of adherence to any particular conception of rationality, but of taking rationality of individual behavior as the unquestioned starting point of economic analysis.

It has become so, he asserts, because this way of thinking has emerged from the "just so" story developed by Hobbes, Locke and others which allegedly explains how property rights and political institutions solve problems arising from the anarchic struggle of man against man in the original state of nature. They place reason at the core of this project, and essentially use this story to explain why things are as they are -- this is the rational world and the only way things can be, if we are to avoid the chaos of anarchy. In essence, the rationality assumption is part of a propaganda campaign. Foley:

A hallmark of these [rationally designed] institutions is that they are in themselves in principle democratic and egalitarian (everyone has an equal right to vote or to hold property) but lead
inexorably to sharp inequalities in economic well-being. It is not hard to see that an economic science whose philosophical starting point was not rational individual action would create an embarrassing discord with this political tradition. The whole point of the Hobbes-Locke “discourse” (to use the jargon of post-modernism) is to rationalize the existing inequalities of power and economic well-being that arise from the institutions of modern society as being unavoidable consequences of the interaction of naturally constituted rational individuals
confronting each other as equals, given the natural and unalterable conditions of human existence. Economic science has a place in this grand project only insofar as it can relate itself to the same philosophical foundations.

I think there's a strong current of truth here. There is in today's economic theory a standing presumption that people should be modelled as rational decision makers (optimizers), and the argument often seems to boil down (in some disguised form) to "we must, because if we do not, we will not be able to prove theorems about equilibrium and its efficiency." This is of course too strong, and research programs in behavioural economics, information asymmetries and so on seem to be working to correct this, but the effort required reflects how much resistance there is to such change and how much intellectual inertia still resides in the idea of thorough-going rationality. Foley suggests that efforts to bring more realistic perspectives such as bounded rationality into core theory have been resisted precisely because they cannot be used to justify the just-so story of the efficient equilibrium:

...in its pragmatic focus on understanding and explaining how people actually behave in modern society, bounded rationality loses contact with the underlying project of rationalizing the institutions of modern society. For example, there really is no logical place in the discourse of bounded rationality for the Fundamental Theorems of Welfare Economics that purport to establish a connection between competitive market equilibrium and an efficient allocation of resources.

I think he may largely be right. If so, this would go a long way to explaining why economics has persisted with such a narrow set of theoretical concepts for such a long time. Maybe it's not actually trying to explain and understand the world at all, but to rationalize why it is OK that it is as we see it. And that's not encouraging for those of us hoping it will change in a big way:

It will not be easy to create a social science that transcends the antinomies and limitations of rational-actor theory. Certainly we cannot depend on the “usual” processes of scientific self-criticism to accomplish much in this direction. No accumulation of its empirical anomalies, or demonstration of its logical inadequacies will somehow magically dispel the power of rational-actor theory, because its power does not rest in the last instance on the adequacy of its
explanations or the consistency of its logic.

Why we need to simulate global financial markets

2012-01-03T03:08:00.000-08:00

I highly recommend this essay by computer scientist Dave Cliff on the complexity of today's global markets, their inherent dependence on technology and potential for catastrophic breakdown. This is based on a presentation he made to the UK government not long after the Flash Crash of 6 May 2010. He offers a valuable perspective as someone who has been both closely involved with developing automated trading algorithms, and thought a great deal about the stability and potential for breakdown in large scale socio-technological systems -- that is, systems in which people and technology interact on a vast scale.

Most importantly, Cliff argues that we're facing a crisis of uncontrolled complexity that we really lack an ability to understand. Blind faith in the self-regulation of the financial system is totally unwarranted, and the only way to avoid potential catastrophes is probably to use technology itself -- through very large scale simulations -- to try to foresee the kinds of problems likely to emerge. A few highlights of the argument:

In a more general discussion, Cliff also argues that the growth of computing technology has led us toward an era of crises through a fundamental dynamic, as our ability to build and depend on complex systems is vastly outpacing out ability to understand and manage such systems. This I think is beyond dispute. Cliff illustrates it with a suggestive graph, below:

Given this problem and it's obvious special relevance to finance (in view of the transformation of HFT over the past decade), Cliff then spends the rest of the text outlining a plausible approach to doing something about it. Mainly, he argues that economics and finance really need to turn to the approach that has been used already in the rest of science and engineering. That is, building plausible large scale simulation models to gain some insight into complex systems (such as the weather). As he notes, computation is widely used in finance, but usually to do things like pricing options, and less frequently to modelling entire markets. This is slowly changing, however, despite the resistance of the traditional economics community.

Cliff's basic suggestion:

I've argued similar things elsewhere (in less detail and not as convincingly). The question is whether governments on a global scale have the will to do this in the face of the financial industry, which seems to quite enjoy there being less understanding of global risks rather than more.

The Higgs Boson -- and the value of money

2011-12-22T06:10:00.000-08:00

What does the (tentatively discovered) Higgs Boson have to do with finance? Nothing. At least nothing obvious. Still, it's a fascinating topic with grand sweeping themes. I've written an essay on it for Bloomberg Views, which will appear later tonight or tomorrow. I'll add the link when it does.

The interesting thing to me about the Higgs Boson, and the associated Higgs field (the boson is an elementary excitation of this field), is the intellectual or conceptual history of the idea. It seems crazy to think that as the universe cooled (very shortly after the big bang) a new field, the Higgs field, suddenly appeared, filling all space, and giving particles mass in proportion to how strongly they interact with that field. It would be a crazy idea if it were just a proposal pulled out of thin air. But the history is that Higgs' work (and the work of many others at the same time, the early 1960s) had very strong stimulation from the BCS theory of superconductivity of ordinary metals, which appeared in 1957.

That theory explained how superconductivity originates through the emergence below a critical temperature of a condensate of paired electrons (hence, bosons) which acts as an extremely sensitive electromagnetic medium. Try to impose a magnetic field inside a superconductor (by bringing a magnet close, for example) and this condensate or field will respond by stirring up currents which act precisely to cancel the field inside the superconductor. This is the essence of superconductivity -- its appearance changes physics inside the superconductor in such a way that electromagnetic fields cannot propagate. In quantum terms (from quantum electrodynamics), this is equivalent to saying that the photon -- the carrier of the electromagnetic fields -- comes to have a mass. It does so because it interacts very strongly with the condensate.

This idea from superconductivity is pretty much identical to the Higgs mechanism for giving the W and Z particles (the carriers of the weak force) mass. This is what I think is fascinating. The Higgs prediction arose not so much from complex mathematics, but from the use of analogy and metaphor -- I wonder if the universe is in some ways like a superconductor? If we're living in a superconductor (not for ordinary electrical charge, but for a different kind of charge of the electroweak field), then it's easy to understand why the W and Z particles have big masses (more than 100 times the mass of the proton). They're just like photons traveling inside an ordinary superconductor -- inside an ordinary metal, lead or tin or aluminum, cooled down to low temperatures.

I think it's fitting that physics theory so celebrated for bewildering mathematics and abstraction beyond ordinary imagination actually has its roots in the understanding of grubby things like magnets and metals. That's where the essential ideas were born and found their initial value.

Having said that none of this has anything to do with finance, however, I should mention a fascinating proposal from 2000 by Per Bak, Simon Nørrelykke and Martin Shubik, which draws a very close analogy between the process which determines the value of money and any Higgs-like mechanism. They made the observation that the absolute value of money is essentially undetermined:

The value of money represents a “continuous symmetry”. If, at some point, the value of money was globally redefined by a certain factor, this would have no consequences whatsoever. Thus, in order to arrive at a specific value of money, the continuous symmetry must be broken.

In other words, a loaf of bread could be worth $1, $10, or $100 -- it doesn't matter. But here and now in the real world it does have one specific value. The symmetry is broken.

This idea of continuous symmetry is something that arises frequently in physics. And it is indeed the breaking of a continuous symmetry that underlies the onset of superconductivity. The mathematics of field theory shows that, anytime a continuous symmetry is broken (so that some variables comes to take on one specific value), there appears in the theory a new dynamical mode -- a so-called Goldstone Mode -- corresponding to fluctuations along the direction of the continuous symmetry. This isn't quite the appearance of mass -- that takes another step in the mathematics, but this Goldstone business is a part of the Higgs mechanism.

I'll try to return to this paper again. It offers a seemingly plausible dynamical model for how a value of money can emerge in an economy, and also why it should be subject to strong inherent fluctuations (because of the Goldstone mode). None of this comes out of equilibrium theory, nor should one expect it to as money is an inherently dynamical thing -- we use it as a tool to manage activities through time, selling our services today to buy food next week, for example.

a little more on power laws

2011-12-13T07:33:00.000-08:00

I wanted to respond to several insightful comments on my recent post on power laws in finance. And, after that, pose a question on the economics/finance history of financial time series that I hope someone out there might be able to help me with.

First, comments:

ivansml said...
Why exactly is power-law distribution for asset returns inconsistent with EMH? It is trivial to write "standard" economic model where returns have fat tails, e.g. if we assume that stochastic process for dividends / firm profits has fat tails. That of course may not be very satisfactory explanation, but it still shows that EMH != normal distribution. In fact, Fama wrote about non-gaussian returns back in 1960's (and Mandelbrot before him), so the idea is not exactly new. The work you describe here is certainly useful and interesting, but pure patterns in data (or "stylized facts", as economists would call them) by themselves are not enough - we need some theory to make sense of them, and it would be interesting to hear more about contributions from econophysics in that area.

James Picerno said...
It's also worth pointing out that EMH, as I understand it, doesn't assume or dismiss that returns follow some specific distribution. Rather, EMH simply posits that prices reflect known information. For many years, analysts presumed that EMH implies a random distribution, but the empirical record says otherwise. But the random walk isn't a condition of EMH. Andrew Lo of MIT has discussed this point at length. The market may or may not be efficient, but it's not conditional on random price fluctuations. Separately, ivansmi makes a good point about models. You need a model to reject EMH. But that only brings you so far. Let's say we have a model of asset pricing that rejects EMH. Then the question is whether EMH or the model is wrong? That requires another model. In short, it's ultimately impossible to reject or accept EMH, unless of course you completely trust a given model. But that brings us back to square one. Welcome to economics.

I actually agree with these statements. Let me try to clarify. In my post I said, referring to the fat tails in returns and 1/t decay of volatility correlations, that "None of these patterns can be explained by anything in the standard economic theories of markets (the EMH etc)." The key word is of course "explained."

The EMH has so much flexibility and is so loosely linked to real data that it is indeed consistent with these observations, as Ivansml (Mark) and James rightly point out. I think it is probably consistent with any conceivable time series of prices. But "being consistent with" isn't a very strong claim, especially if the consistency comes from making further subsidiary assumptions about how these fat tails might come from fluctuations in fundamental values. This seems like a "just so" story (even if the idea that fluctuations in fundamental values could have fat tails is not at all preposterous).

The point I wanted to make is that nothing (that I know of) in traditional economics/finance (i.e. coming out of the EMH paradigm) gives a natural and convincing explanation of these statistical regularities. Such an explanation would start from simple well accepted facts about the behaviour of individuals, firms, etc., market structures and so on, and then demonstrate how -- because of certain logical consequences following from these facts and their interactions -- we should actually expect to find just these kinds of power laws, with the same exponents, etc., and in many different markets. Reading such an explanation, you would say "Oh, now I see where it comes from and how it works!"

To illustrate some possibilities, one class of proposed explanations sees large market movements as having inherently collective origins, i.e. as reflecting large avalanches of trading behaviour coming out of the interactions of market participants. Early models in this class include the famous Santa Fe Institute Stock Market model developed in the mid 1990s. This nice historical summary by Blake LeBaron explores the motivations of this early agent-based model, the first of which was to include a focus on the interactions among market participants, and so go beyond the usual simplifying assumptions of standard theories which assume interactions can be ignored. As LeBaron notes, this work began in part...

... from a desire to understand the impact of agent interactions and group learning dynamics in a financial setting. While agent-based markets have many goals, I see their first scientific use as a tool for understanding the dynamics in relatively traditional economic models. It is these models for which economists often invoke the heroic assumption of convergence to rational expectations equilibrium where agents’ beliefs and behavior have converged to a self-consistent world view. Obviously, this would be a nice place to get to, but the dynamics of this journey are rarely spelled out. Given that financial markets appear to thrive on diverse opinions and behavior, a first level test of rational expectations from a heterogeneous learning perspective was always needed.

I'm going to write posts on this kind of work soon looking in much more detail. This early model has been greatly extended and had many diverse offspring; a more recent review by LeBaron gives an updated view. In many such models one finds the natural emergence of power law distributions for returns, and also long-term correlations in volatility. These appear to be linked to various kinds of interactions between participants. Essentially, the market is an ecology of interacting trading strategies, and it has naturally rich dynamics as new strategies invade and old strategies, which had been successful, fall into disuse. The market never settles into an equilibrium, but has continuous ongoing fluctuations.

Now, these various models haven't yet explained anything, but they do pose potentially explanatory mechanisms, which need to be tested in detail. Just because these mechanisms CAN produce the right numbers doesn't mean this is really how it works in markets. Indeed, some physicists and economists working together have proposed a very different kind of explanation for the power law with exponent 3 for the (cumulative) distribution of returns which links it to the known power law distribution of the wealth of investors (and hence the size of the trades they can make). This model sees large movements as arising in the large actions of very wealthy market participants. However, this is more than merely attributing the effect to unknown fat tails in fundamentals, as would be the case with EMH based explanations. It starts with empirical observations of tail behaviour in several market quantities and argues that these together imply what we see for market returns.

There are more models and proposed explanations, and I hope to get into all this in some detail soon. But I hope this explains a little why I don't find the EMH based ideas very interesting. Being consistent with these statistical regularities is not as interesting as suggesting clear paths by which they arise.

Of course, I might make one other point too, and maybe this is, deep down, what I find most empty about the EMH paradigm. It essentially assumes away any dynamics in the market. Fundamentals get changed by external forces and the theory supposes that this great complex mass of heterogenous humanity which is the market responds instantaneously to find the new equilibrium which incorporates all information correctly. So, it treats the non-market part of the world -- the weather, politics, business, technology and so on -- as a rich thing with potentially complicated dynamics. Then it treats the market as a really simply dynamical thing which just gets driven in slave fashion by the outside. This to me seems perversely unnatural and impossible to take seriously. But it is indeed very difficult to rule out with hard data. The idea can always be contorted to remain consistent with observations.

Finally, another valuable comment:

David K. Waltz said...

In one of Taleeb's books, didn't he make mention that something cannot be proven true, only disproven? I think it was the whole swan thing - if you have an appropriate sample and count 100% white swans does not prove there are ONLY white swans, while a sample that has a black one proves that there are not ONLY white swans.

Again, I agree completely. This is a basic point about science. We don't ever prove a theory, only disprove it. And the best science works by trying to find data to disprove a hypothesis, not by trying to prove it.

I assume David is referring to my discussion of the empirical cubic power law for market returns. This is indeed a tentative stylized fact which seems to hold with appreciable accuracy in many markets, but there may well be markets in which it doesn't hold (or periods in which the exponent changes). Finding such deviations would be very interesting as it might offer further clues as to the mechanism behind this phenomenon.

NOW, for the question I wanted to pose. I've been doing some research on the history of finance, and there's something I can't quite understand. Here's the problem:

1. Mandelbrot in the early 1960s showed that market returns had fat tails; he conjectured that they fit the so-called Stable Paretian (now called Stable Levy) distributions which have power law tails. These have the nice property (like the Gaussian) that the composition of the returns for longer intervals, built up from component Stable Paretian distributions, also has the same form. The market looks the same at different time scales.
2. However, Mandelbrot noted in that same paper a shortcoming of his proposal. You can't think of returns as being independent and identically distributed (i.i.d.) over different time intervals because the volatility clusters -- high volatility predicts more to follow, and vice versa. We don't just have an i.i.d. process.
3. Lots of people documented volatility clustering over the next few decades, and in the 1980s Robert Engle and others introduced ARCH/GARCH and all that -- simple time series models able to reproduce the realistic properties of financial times, including volatility clustering.
4. But today I found several papers from the 1990s (and later) still discussing the Stable Paretian distribution as a plausible model for financial time series.

My question is simply -- why was anyone even 20 years ago still writing about the Stable Paretian distribution when the reality of volatility clustering was so well known? My understanding is that this distribution was proposed as a way to save the i.i.d. property (by showing that such a process can still create market fluctuations having similar character on all time scales). But volatility clustering is enough on its own to rule out any i.i.d. process.

Of course, the Stable Paretian business has by now been completely ruled out by empirical work establishing the value of the exponent for returns, which is too large to be consistent with such distributions. I just can't see why it wasn't relegated to the history books long before.

The only possibility, it just dawns on me, is that people may have thought that some minor variation of the original Mandelbrot view might work best. That is, let the distribution over any interval be Stable Paretian, but let the parameters vary a little from one moment to the next. You give up the i.i.d. but might still get some kind of nice stability properties as short intervals get put together into longer ones. You could put Mandelbrot's distribution into ARCH/GARCH rather than the Gaussian. But this is only a guess. Does anyone know?

Prosecuting Wall St.

2011-12-09T01:44:00.000-08:00

By way of Simolean Sense:

The following is a script of "Prosecuting Wall Street" (CBS) which aired on Dec. 4, 2011. Steve Kroft is correspondent, James Jacoby, producer.

It's been three years since the financial crisis crippled the American economy, and much to the consternation of the general public and the demonstrators on Wall Street, there has not been a single prosecution of a high-ranking Wall Street executive or major financial firm even though fraud and financial misrepresentations played a significant role in the meltdown. We wanted to know why, so nine months ago we began looking for cases that might have prosecutorial merit. Tonight you'll hear about two of them. We begin with a woman named Eileen Foster, a senior executive at Countrywide Financial, one of the epicenters of the crisis.

Steve Kroft: Do you believe that there are people at Countrywide who belong behind bars?

Eileen Foster: Yes.

Kroft: Do you want to give me their names?

Foster: No.

Kroft: Would you give their names to a grand jury if you were asked?

Foster: Yes.

But Eileen Foster has never been asked - and never spoken to the Justice Department - even though she was Countrywide's executive vice president in charge of fraud investigations...

See the video and transcript here.

Power laws in finance

2011-12-06T04:36:00.000-08:00

My latest column in Bloomberg looks very briefly at some of the basic mathematical patterns we know about in finance. Science has a long tradition of putting data and observation first. Look very carefully at what needs to be explained -- mathematical patterns that show up consistently in the data -- and then try to build simple models able to reproduce those patterns in a natural way.

This path has great promise in economic finance, although it hasn't been pursued very far until recently. My Bloomberg column gives a sketch of what is going on, but I'd like to give a few more details here and some links.

The patterns we find in finance are statistical regularities -- broad statistical patterns which show up in all markets studied, with an impressive similarity across markets in different countries and for markets in different instruments. The first regularity is the distribution of returns over various time intervals, which has been found generically to have broad power law tails -- "fat tails" -- implying that large fluctuations up or down are much more likely than they would be if markets fluctuated in keeping with normal Gaussian statistics. Anyone who read The Black Swan knows this.

This pattern has been established in a number of studies over the past 15 years or so, mostly by physicist Eugene Stanley of Boston University and colleagues. This paper from 1999 is perhaps the most notable, as it used enormous volumes of historical data to establish the fat tailed pattern for returns over times ranging from one minute up to about 4 days. One of the most powerful things about this approach is that it doesn't begin with any far reaching assumptions about human behaviour, the structure of financial markets or anything else, but only asks -- are there patterns in the data? As the authors note:

The most challenging difficulty in the study of a financial market is that the nature of the interactions between the different elements comprising the system is unknown, as is the way in which external factors affect it. Therefore, as a starting point, one may resort to empirical studies to help uncover the regularities or “empirical laws” that may govern financial markets.

This strategy seems promising to physicists because it has worked in building theories of complex physical systems -- liquids, gases, magnets, superconductors -- for which it is also often impossible to know anything in great detail about the interactions between the molecules and atoms within. This hasn't prevented the development of powerful theories because, as it turns out, many of the precise details at the microscopic level DO NOT influence the large scale collective properties of the system. This has inspired physicists to think that the same may be true in financial markets -- at least some of the collective behaviour we see in markets, their macroscopic behaviour, may be quite insensitive to details about human decision making, market structure and so on.

The authors of this 1999 study summarized their findings as follows:

Several points of clarification. First, the result for the power law with exponent close to 3 is a result for the cumulative distribution. That is, the probability that a return will be greater than a certain value (not just equal to that value). Second, the fact that this value lies outside of the range [0,2] means that the process generating these fluctuations isn't a simple stationary random process with an identical and independent distribution for each time period. This was the idea initially proposed by Benoit Mandelbrot on the basis of the so-called Levy Stable distributions. This study and others have established that this idea can't work -- something more complicated is going on.

That complication is also referred to in the second paragraph above. If you take the data on returns at the one minute level, and randomize the order in which it appears, then you still get the same power law tails in the distribution of returns over one minute. That's the same data. But this new time series has different returns over longer times, generated by combining sequences of the one minute returns. The distribution over longer and longer times turns out to converge slowly to a Gaussian for the randomized data, meaning that the true fat tailed distribution over longer times has its origin in some rich and complex correlations in market movements at different times (which gets wiped out by the randomization). Again, we're not just dealing with a fixed probability distribution and independent changes over different intervals.

To read more about this, see this nice review by Xavier Gabaix of MIT. It covers this and many other power laws in finance and economics.

Now, the story gets even more interesting if you look past the mere distribution of returns and study the correlations between market movements at different times. Market movements are, of course, extremely hard to predict. But it is very interesting where the unpredictability comes in.

The so-called autocorrelation of the time series of market returns decays to zero after a few minutes. This is essentially a measure of how much the return now can be used to predict a return in the future. After a few minutes, there's nothing. This is the sense in which the markets are unpredictable. However, there are levels of predictability. It was discovered in the early 1990s, and has been confirmed many times since in different markets, that the time series of volatility -- the absolute value of the market return -- has long-term correlations, a kind of long-term memory. Technically, the autocorrelation of this time series only decays to zero very slowly.

This is shown below in the following figure (from a representative paper, again from the Boston University group) which shows the autocorrelation of the return time series g(t) and also of the volatility, which is the absolute value of g(t):

Clearly, whereas the first signal shows no correlations after about 10 minutes, the second shows correlations and predictability persisting out to times as long as 10,000 minutes, which is on the order of 10 days or so.

So, its the directionality of price movements which has very little predictability, whereas the magnitude of changes follows a process with much more interesting structure. It is in the record of this volatility that one sees potentially deep links to other physical processes, including earthquakes. A particularly interesting paper is this one, again by the Boston group, quantifying several ways in which market volatility obeys several quantitative laws known from earthquake science, especially the Omori Law describing how the probability of aftershocks decays following a main earthquake. This probability decays quite simply in proportion to 1/time since the main quake, meaning that aftershocks are most likely immediately afterward, and become progressively less likely with time. Episodes of high volatility appear to follow similar behaviour quite closely.

Perhaps even better is another study, which looks at the link to earthquakes with a somewhat tighter focus. The abstract captures the content quite well:

We analyze the memory in volatility by studying volatility return intervals, defined as the time between two consecutive fluctuations larger than a given threshold, in time periods following stock market crashes. Such an aftercrash period is characterized by the Omori law, which describes the decay in the rate of aftershocks of a given size with time t by a power law with exponent close to 1. A shock followed by such a power law decay in the rate is here called Omori process. We find self-similar features in the volatility. Specifically, within the aftercrash period there are smaller shocks that themselves constitute Omori processes on smaller scales, similar to the Omori process after the large crash. We call these smaller shocks subcrashes, which are followed by their own aftershocks. We also show that the Omori law holds not only after significant market crashes as shown by Lillo and Mantegna [Phys. Rev. E 68, 016119 2003], but also after “intermediate shocks.” ...

These are only a few of the power law type regularities now known to hold for most markets, with only very minor differences between markets. An important effort is to find ways to explain these regularities in simple and plausible market models. None of these patterns can be explained by anything in the standard economic theories of markets (the EMH etc). They can of course be reproduced by suitably generating time series using various methods, but that hardly counts as explanation -- that's just using time series generators to reproduce certain kinds of data.

The promise of finding these kinds of patterns is that they may strongly constrain the types of theories to be considered for markets, by ruling out all those which do not naturally give rise to this kind of statistical behaviour. This is where data matters most in science -- by proving that certain ideas, no matter how plausible they seem, don't work. This data has already stimulated the development of a number of different avenues for building market theories which can explain the basic statistics of markets, and in so doing go well beyond the achievements of traditional economics.

I'll have more to say on that in the near future.

Interview with Dave Cliff

2011-12-02T09:12:00.000-08:00

Dave Cliff of the University of Bristol is someone whose work I've been meaning to look at much more closely for a long time. Essentially he's an artificial intelligence expert, but has has devoted some of his work to developing trading algorithms. He suggests that many of these algorithms, even one working on extremely simple rules, consistently outperform human beings, which rather undermines the common economic view that people are highly sophisticated rational agents.

I just noticed tht Moneyscience is beginning a several part interview with Cliff, the first part having just appeared. I'm looking forward to the rest. Some highlights from Part I, beginning with Cliff's early work, mid 1990s, on writing algorithms for trading:

I wrote this piece of software called ZIP, Zero Intelligence Plus. The intention was for it to be as minimal as possible, so it is a ridiculously simple algorithm, almost embarrassingly so. It’s essentially some nested if-then rules, the kind of thing that you might type into an Excel spreadsheet macro. And this set of decisions determines whether the trader should increase or decrease a margin. For each unit it trades, has some notion of the price below which it shouldn’t sell or above which it shouldn’t buy and that is its limit price. However, the price that it actually quotes into the market as a bid or an offer is different from the limit price because obviously, if you’ve been told you can buy something and spend no more than ten quid, you want to start low and you might be bidding just one or two pounds. Then gradually, you’ll approach towards the ten quid point in order to get the deal, so with each quote you’re reducing the margin on the trade. The key innovation I introduced in my ZIP algorithm was that it learned from its experience. So if it made a mistake, it would recognize that mistake and be better the next time it was in the same situation.

HFTR: When was this exactly?

DC: I did the research in 1996 and HP published the results, and the ZIP program code, in 1997. I then went on to do some other things, like DJ-ing and producing algorithmic dance music (but that’s another story!)

Fast-forward to 2001, when I started to get a bunch of calls because a team at IBM’s Research Labs in the US had just completed the first ever systematic experimental tests of human traders competing against automated, adaptive trading systems. Although IBM had developed their own algorithm called MGD, (Modified Gjerstad Dickhaut), it did the same kind of thing as my ZIP algorithm, using different methods. They had tested out both their MGD and my ZIP against human traders under rigorous experimental conditions and found that both algorithms consistently beat humans, regardless of whether the humans or robots were buyers or sellers. The robots always out-performed the humans.

IBM published their findings at the 2001 IJCAI conference (the International Joint Conference on AI) and although IBM are a pretty conservative company, in the opening paragraphs of this paper they said that this was a result that could have financial implications measured in billions of dollars. I think that implicitly what they were saying was there will always be financial markets and there will always be the institutions (i.e. hedge funds, pension management funds, banks, etc). But the traders that do the business on behalf of those institutions would cease to be human at some point in the future and start to be machines.

Personally, I think there are two important things here. One is that, yes, trading will probably soon become almost all algorithmic. This may tend to make you think the markets will become more mechanical, their collective behaviour emerging out of the very simple actions of so many crude programs.

But the second thing is what this tells us about people -- that traders and investors and people in general aren't so clever or rational, and most of them have probably been following fairly simple rules all along, rules that machines can easily beat. So there's really no reason to think the markets should become more mechanical as they become more algorithmic. They've probably been quite mechanical all along, and algorithmic too -- it's just that non-rational zero intelligence automatons running the algorithms were called people.

Alan Greenspan's nirvana

2011-11-30T08:44:00.000-08:00

I wrote a post a while back exploring some of the silliest things economists were saying before the crisis about how financial engineering was making our economy more robust, stable, efficient, wonderful, beautiful, intelligent, self-regulating, and so on. The markets were, R. Glenn Hubbard and William Dudley were convinced, even leading to better governance by punishing bad governmental decisions. [How that could be the case when markets have a relentless focus on the very short term is hard to fathom, but they indeed did assert this]..

Paul Krugman has recently undertaken a similar exercise in silliness mining -- in this case going through the hallucinations of Alan Greenspan. The Chairman of the Fed was evidently drinking the very same Kool-Aid:

Deregulation and the newer information technologies have joined, in the United States and elsewhere, to advance flexibility in the financial sector. Financial stability may turn out to have been the most important contributor to the evident significant gains in economic stability over the past two decades.

Historically, banks have been at the forefront of financial intermediation, in part because their ability to leverage offers an efficient source of funding. But in periods of severe financial stress, such leverage too often brought down banking institutions and, in some cases, precipitated financial crises that led to recession or worse. But recent regulatory reform, coupled with innovative technologies, has stimulated the development of financial products, such as asset-backed securities, collateral loan obligations, and credit default swaps, that facilitate the dispersion of risk.

Conceptual advances in pricing options and other complex financial products, along with improvements in computer and telecommunications technologies, have significantly lowered the costs of, and expanded the opportunities for, hedging risks that were not readily deflected in earlier decades. The new instruments of risk dispersal have enabled the largest and most sophisticated banks, in their credit-granting role, to divest themselves of much credit risk by passing it to institutions with far less leverage. Insurance companies, especially those in reinsurance, pension funds, and hedge funds continue to be willing, at a price, to supply credit protection.

These increasingly complex financial instruments have contributed to the development of a far more flexible, efficient, and hence resilient financial system than the one that existed just a quarter-century ago.

As Krugman notes, this can all be translated into ordinary language: "Thanks to securitization, CDOs, and AIG, nothing bad can happen!"

Bail out everyone

2011-11-30T04:05:00.000-08:00

I had wondered about this idea a couple years ago -- but that's all I did, wondered about it. The idea is that when banks need bailing out -- and sadly, we seem stuck with that problem for the moment -- we shouldn't bail them out directly, but indirectly. For example, just give every single person in the US $1,000. Or maybe a voucher for $1,000 that they have to spend somewhere, or put in a bank. This quickly amounts to $300 billion infused into the economy, a large portion of which would end up in banks. So cash would be pumped into the banks too, but only through people first.

You can imagine all kinds of ways to play around with such a scheme. Paying off some of peoples' mortgages. The amount injected could be much larger. Perhaps similar funds would be injected directly into banks and other businesses as well. Mark Thoma has thought through some of the details. But I'm quite surprised this is the first I've heard about any idea even remotely like this. It seems like a much better idea than just giving money to the bankers who created the problem in the first place. Why don't we hear more about such possibilities?

Modern European Tragedy

2011-11-30T03:35:00.000-08:00

The endgame playing out in Europe is a tragedy in the usual sense, but also in the sense of Greek tragedy -- downfall brought about ironically through the very efforts, perhaps even well intentioned, of those ultimately afflicted. It's terrible to see Europe looming toward disaster, but also utterly fascinating that everyone involved -- Greeks, Germans, French, the European Central Bank -- has acted in what they thought was their own interest, yet those very actions have led the collective to a likely outcome much worse for all. A tragedy of the commons.

Philosopher Simon Critchley has written a brilliant essay exploring this theme more generally. Among the most poetic analyses of the situation I have seen:

The euro was the very project that was meant to unify Europe and turn a rough amalgam of states in a free market arrangement into a genuine social, cultural and economic unity. But it has ended up disunifying the region and creating perverse effects, such as the spectacular rise of the populist right in countries like the Netherlands, for just about every member state, even dear old Finland.

What makes this a tragedy is that we knew some of this all along — economic seers of various stripes had so prophesied — and still we conspired with it out of arrogance, dogma and complacency. European leaders — technocrats whom Paul Krugman dubbed this week “boring cruel romantics” — ignored warnings that the euro was a politically motivated project that would simply not work given the diversity of economies that the system was meant to cover. The seers, indeed, said it would fail; politicians across Europe ignored the warnings because it didn’t fit their version of the fantasy of Europe as a counterweight to United States’ hegemony. Bad deals were made, some lies were told, the peoples of the various member countries were bludgeoned into compliance often without being consulted, and now the proverbial chickens are coming home to roost.

But we heard nothing and saw nothing, for shame. The tragic truth that we see unspooling in the desperate attempts to shore up the European Union while accepting no responsibility for the unfolding disaster is something that we both willed and that threatens to now destroy the union in its present form.

The euro is a vast boomerang that is busy knocking over millions of people. European leaders, in their blindness, continue to act as if that were not the case.

The end of the Euro?

2011-11-28T08:32:00.000-08:00

Three interesting articles on what now seems to be considered an increasingly likely event -- the end of the Euro (in its current form, although some version might arise from the ashes).

First, Gavyn Davies speculates on several possible scenarios for the collapse of the Euro. It might persist as the new currency of a smaller union including Germany and The Netherlands (in which case the value of the Euro would rise significantly), or it might persist as the new currency of the periphery countries after Germany bolts (in which case the value of the Euro would fall significantly). Or the Europeans might finally find a way through the ongoing nightmare. Not betting on that one.

Second, Satyajit Das goes into a little more detail, and I think rightly sees some cultural issues as ultimately being most important. The three logical possibilities are easy to list:

The latest plan has bought time, though far less than generally assumed. The European debt endgame remains the same: fiscal union (greater integration of finances where Germany and the stronger economies subsidise the weaker economies); debt monetisation (the ECB prints money); or sovereign defaults.

Germany may be largely in favour of solution number 1. But the smaller periphery countries, and perhaps France as well, will favour solution number 2. Hence, we may by default find Europe hurtling inexorably into "solution" number 3 -- sovereign defaults:

The accepted view is that, in the final analysis, Germany will embrace fiscal integration or allow printing money. This assumes that a cost-benefit analysis indicate that this would be less costly than a disorderly break-up of the Euro-zone and an integrated European monetary system. This ignores a deep-seated German mistrust of modern finance as well as a strong belief in a hard currency and stable money. Based on their own history, Germans believe that this is essential to economic and social stability. It would be unsurprising to see Germany refuse the type of monetary accommodation and open-ended commitment necessary to resolve the crisis by either fiscal union or debt monetisation.

Unless restructuring of the Euro, fiscal union or debt monetisation can be considered, sovereign defaults may be the only option available.

Perhaps it betrays a little bit of anarchy in my own soul, but I'm rooting quite hard for sovereign defaults. I wish the Greeks had gone ahead with their referendum. For all the complaining about the slack morals of the Greek taxpayer, every debt-creating transaction has two sides -- and the creditors (French and German banks) bear as much responsibility as the debtors.

Then again, the end is likely to bring some severe social misery, not to mention riots (the UK is already advising its European embassies on the likelihood). A third article by Simon Johnson and Peter Boone points ominously in this direction, essentially echoing Davies' analysis in bleaker language:

The path of the euro zone is becoming clear. As conditions in Europe worsen, there will be fewer euro-denominated assets that investors can safely buy. Bank runs and large-scale capital flight out of Europe are likely.

Devaluation can help growth but the associated inflation hurts many people and the debt restructurings, if not handled properly, could be immensely disruptive. Some nations will need to leave the euro zone. There is no painless solution.

Ultimately, an integrated currency area may remain in Europe, albeit with fewer countries and more fiscal centralization. The Germans will force the weaker countries out of the euro area or, more likely, Germany and some others will leave the euro to form their own currency. The euro zone could be expanded again later, but only after much deeper political, economic and fiscal integration.

Tragedy awaits. European politicians are likely to stall until markets force a chaotic end upon them. Let’s hope they are planning quietly to keep disorder from turning into chaos.

Are economists good scientists?

2011-11-18T01:39:00.000-08:00

I've had no time to post recently for several reasons, mostly the urgent need to work on a book closely related to this blog. The deadline is getting closer. I hope to resume something like my previous posting frequency soon.

But I would like to point everyone to a fascinating recent analysis of economists' opinions about the scientific method (that seems the best term for it, at least). Ole Rogeberg, a reader of this blog, alerted me to some work by himself and Hans Melberg in which they surveyed economists to see how much they looked to actual empirical tests of a theory's predictions in judging the value of a theory. The answer, it turns out, is -- not much. Internal consistency seems to be more important than empirical test.

This even for a theory -- the theory of "rational addiction", which seeks to explain heroin addiction and other life destroying addictions as the consequence of fully rational choices on the part of individuals as they maximize their expected utility over their lifetimes -- which on the face of it seems highly unlikely, making the burden of empirical evidence (one would think) even higher. Some history. Gary Becker (Nobel Prize) of the University of Chicago is famous for his efforts to push the neo-classical framework into every last corner of human life. He (and many followers) have applied the trusted old recipe of utility maximization to understand (they claim) everything from crime to patterns of having children to addiction. You may see a slobbering shivering drunk or junkie in an alleyway in winter and think -- like most people -- there goes someone trapped in some very destructive behavioural feedback controlled by the interaction of addictive physical substances, emotions and so on. Not Becker. It's all quite rational, he argues.

Now, Rogeberg and Melberg. Here's their abstract:

This paper reports on results from a survey of views on the theory of rational addiction among academics who have contributed to this research. The topic is important because if the literature is viewed by its participants as an intellectual game, then policy makers should be aware of this so as not to derive actual policy from misleading models. A majority of the respondents believe the literature is a success story that demonstrates the power of economic reasoning. At the same time, they also believe the empirical evidence to be weak, and they disagree both on the type of evidence that would validate the theory and the policy implications. These results shed light on how many economists think about model building, evidence requirements and the policy relevance of their work.

Now, in any area of science there are disgreements over what evidence really counts as important. I've certainly learned this from following 20 years of research on high temperature superconductivity, where every new paper with "knock down" evidence for some claim tends to be immediately countered by someone else claiming this evidence actually shows something quite different. The materials are complex as is the physics, and so far it just doesn't seem possible to bring clarity to the subject.

But in high-Tc research, theorists are under no illusion that they understand. They readily admit that they have no good theory. The same attitude doesn't seem to have been common in economics. Rogeberg and Melberg have also described their survey work in this clearly written paper in a less technical style.

A few more choice excerpts from their (full) paper below:

The core of the causal insight claims from rational addiction research is that people behave in a certain way (i.e. exhibit addictive behavior) because they face and solve a specific type of choice problem. Yet rational addiction researchers show no interest in empirically examining the actual choice problem – the preferences, beliefs, and choice processes – of the people whose behavior they claim to be explaining. Becker has even suggested that the rational choice process occurs at some subconscious level that the acting subject is unaware of, making human introspection irrelevant and leaving us no known way to gather relevant data...

The claim of causal insight, then, involves the claim that a choice problem people neither face nor would be able to solve prescribes an optimal consumption plan no one is aware of having. The gradual implementation of this unknown plan is then claimed to be the actual explanation for why people over time smoke more than they should according to the plans they actually thought they had. To quote Bertrand Russell out of context, this ‘is one of those views which are so absurd that only very learned men could possibly adopt them’ (Russell 1995, p. 110).

On the nature of reasoning in rational addiction models (this is Nobel Prize winning stuff, by the way):

[The addict]... looks strange because he sits down at (the first) period, surveys future income, production technologies, investment/addiction functions and consumption preferences over his lifetime to period T, maximizes the discounted value of his expected utility and decides to be an alcoholic. That’s the way he will get the greatest satisfaction out of life. (Winston 1980, p. 302)

ISDA: Stop Making Sense

2011-11-07T09:44:00.000-08:00

The following is a response I just posted on Bloomberg to some criticism last week coming from the International Swaps and Derivatives Association. They took issue with some things I had written in my latest Bloomberg column. I think their comment was partially fair, and also partially misleading, so I thought some clarification would be useful. The text below is identical to what appears (or will very shortly) in Bloomberg:

*********************************

My most recent Bloomberg column on the network of credit default swaps contracts provoked a comment from the International Swaps and Derivatives Association, Inc. The group objected to my characterization of the network of outstanding CDS contracts as "hidden" and potentially a source of trouble. I'd like to address their concerns, and also raise some questions.

Contrary to the association's claim, I am aware of the existence of the Depository Trust & Clearing Corporation. I'll admit to having underestimated how much their project to create a warehouse of information on CDS contracts has developed in the past few years; my statement that these contracts are not "recorded by any central repository" was too strong, as a partial repository does exist, and the DTCC deserves great credit for creating it.

However, it is not clear that this repository gives such a complete picture of outstanding CDS linkages that we can all relax.

For example, DTCC's repository covers 98 percent of all outstanding CDS contracts, not 100 percent. Asking why may or may not be a quibble. After all, a map showing 98 percent of the largest 300 cities in the U.S. could leave out New York, Los Angeles, Chicago, Houston and Philadelphia. Moreover, the simple number of contracts tells us nothing about the values listed on those contracts. In principle, the missing 2 percent of contracts could represent a significant fraction of the outstanding value of CDS contracts.

More importantly, when thinking about potentially cascading risks in a complex network, fine details of the network topology -- its architecture or wiring diagram -- matter a lot. Indeed, the CDS contracts that put American Insurance Group Inc. in grave danger in 2008 represented a tiny fraction -- much less than 1 percent -- of the total number of outstanding CDS contracts.

Hence, it would be interesting to know why the repository holds only 98 percent rather than 100 percent. There may be a very simple and reassuring answer, but it's not readily apparent from DTCC's description of the repository.

Also, there is another issue which makes "fully transparent" not quite the right phrase for this network of contracts, even if we suppose the 98 percent leaves out nothing of importance.

The DTCC commendably makes its data available to regulators. Still, it appears that the full network of interdependencies created by CDS contracts may remain opaque to regulators, because DTCC, according to its own description, enables...

"... each regulator to access reports tailored to their specific entitlements as a market regulator, prudential or primary supervisor, or central bank. These detailed reports are created for each regulator to show only the CDS data relevant to its jurisdiction, regulated entities or currency, at the appropriate level of aggregation."

This would imply, for example, that regulators in the U.S. can look and see which of their banks have sold CDS on, say, a big German bank. But the health of the U.S. banks then depends directly on the health of that German bank, which may in turn have sold CDS on Greek or Italian debt or any number of other things. The DTCC data on the latter CDS contracts would, apparently, not be available to U.S. regulators, being out of their jurisdiction.

The point is that a financial institution is at risk not only from contracts it has entered into, but also from contracts that its many counterparties have entered into (this is the whole idea of systemic risk linked to the possibility of contagion). Credible tests of the financial network's resilience require a truly global analysis of the potential pathways along which distress (particularly from outright counterparty failures) may spread. It's not clear that any regulator has the full data on which such an analysis can be based.

None of this, by any means, is meant as a criticism of DTCC or what it has done in the past few years. The 98 percent figure is impressive, and let's hope the 98 percent soon becomes 100 percent and the DTCC finds a way to make ALL information in the repository available to regulators everywhere. Even better would be full disclosure to the public.

Of course, nothing in the comment from the International Swaps and Derivatives Association changes the main point of my column, which was that it is incorrect to believe that more CDS contracts -- or, more generally, more financial interdependencies of any kind, including links created by other derivatives such as interest-rate swaps -- automatically lead to better risk-sharing and a safer banking system. More apparent risk-sharing can actually mean more systemic risk and less overall banking safety.

(Mark Buchanan is a Bloomberg View columnist.)

Building a financial system that works for the real economy

2011-11-02T06:21:00.000-07:00

I highly recommend this video of a talk given recently by Sony Kapoor at the Global Systems Dynamics Workshop in Berlin. As it happens, I was there and got to see the talk in person; it's funny and very insightful. Kapoor used to work at Lehman Bros (well before it's collapse), and eventually quit investment banking to do more useful things -- he now works at Re-define, a think tank on public policy.

I tried to embed the video here but failed. There seems to be embed protection on it for some reason.

There were a number of other great talks at the meeting as well, all available on video here.

There are markets... and markets...

2011-11-02T03:13:00.000-07:00

John Kay makes a very good point -- that the ideology and rhetoric surrounding the allegedly wonderful properties of markets has taken us a long way from where we ought to be. We need a more balanced perspective on what markets do well and what they do not do well, where they are useful and where they are not:

A semantic confusion leads us to use the word market to describe both the process which puts food on our table and the activity of gambling in credit default swaps. That confusion has enabled people to claim the virtues of the former for the latter.

In his book Extreme Money, Satyajit Das makes a closely related point which, I'm sure, many economists and finance people will probably find incomprehensible:

Banks are utilities matching borrowers and savers, providing payment services, facilitating hedging etc. The value added comes from reducing the cost of doing so. Paul Volcker questioned the role of finance: “I wish someone would give me one shred of neutral evidence that financial innovation has led to economic growth — one shred of evidence. US financial services increased its share of value added from 2% to 6.5% but Is that a reflection of your financial innovation, or just a reflection of what you’re paid?”

The idea of financial services as a driver of economic growth is absurd – it’s a bit like looking at a car’s gearbox as the basis for propulsion. But financiers don’t necessarily agree with this assessment, unsurprisingly.