Are You Using Good Risk Models?

The current recession has forced a re-evaluation of modeling in the financial services arena. Leaders in the modeling profession have jointly issued, and are actively promoting, a manifesto for responsible modeling. This is good news for banks. Unfortunately, much of the modeling mind-set that banks are now trying to shed is migrating to corporate risk management.

Perhaps there is no better example to illustrate what can go wrongwith modeling than the pricing of collateralized debt obligations (CDOs). Models that were meant to predict the likelihood of a particular mortgage defaulting actually oversimplified and obscured risks in the housing market, ultimately providing a false sense of security. Onthe belief that the value of CDOs could be easily and accurately assessed, the housing market boomed and bonuses swelled. But reality refused to follow the predictions of the models.

Predictive models are successful for physicists and engineers because physical laws, once written in mathematical equations, are accurate to several decimal places. Such precision comes from the immutablenature of the physical world and these models can predict reality, over and over again.

But risk and value are byproducts of human behavior. As such, riskand value are determined by events in our lives, by our feelings about such events, by how our feelings make us react to those events andby our expectations of other people's reactions to the same events. While we can, at times, discern the pattern of events, we are at a loss when it comes to how we and others are likely to react to them. Intruth, there are no fundamental laws in risk and value. So then, what makes for a good risk model?

First, a good risk model translates the manager's intuition of linear relationships into nonlinear dollar values. Good models do not replace managers by spitting out the answer. Good models help managers understand the monetary impact of risk and value tradeoffs for the enterprise. For example, a manager may need to decide whether to bundlelosses from the enterprise into a captive insurer under a master reinsurance treaty. A model can simulate a broad range of realistic, daily situations as well as extreme scenarios for the captive. Such modeling will then reveal the confluence of events required to make the captive profitable. Some of the results will be intuitive and come as no surprise to the manager, but there is not much value gained from modeling in this fashion. The model, however, will also produce some surprises. It is these surprises that make modeling a valuable risk management tool. The surprises inform and shape the final decision of the risk manager.

Second, a good risk model discloses how badly it misses reality, especially during times of stress. Because the process that creates such disclosure is imperfect and complex, risk models often skip this step in the model's development cycle. At a minimum, a good risk modelclearly states the assumptions that it is built on in a language that the manager can understand. Then, he or she can figure out when reality is most likely to violate the model's assumptions. The easiest of the complex processes available for validating a model is to go back in time, use the model to make a decision, and then, with the benefit of hindsight, check how the decision would have differed if the model was not available. Then repeat these steps at different intervals. Even though validation of a risk model is an imperfect process, it is well worth the trouble. The manager then knows when the model is likely to underperform and when it is likely to overperform in relation to the unfolding reality.

Third, a good risk model recognizes that there is utility to risk.This theory supports the fact that $100 means less to a wealthy shareholder than it does to a struggling actor--even though it buys exactly the same goods. The implication is that two identical assets, in terms of cash flow, may in fact have different values depending on whoholds them. This implication is well established in the literature written about incomplete markets, and it is the underlying reason why insurance contracts, which are fundamentally options, are priced the way they are instead of using option pricing methodology.