Reprinted with permission of RIA
Model Risk and Valuation
By Susan M. Mangiero
Reprinted with permission of RIA.
This article first appeared in the March/April 2003 issue of Valuation
Because the stakes are large when valuation model issues end up in court,
valuation professionals should pay close attention to model risk.
Models are used in business all the time and for
a variety of reasons. Without models, it would be hard to forecast earnings,
simulate cash flows, measure risk, assess competitors, analyze economic
conditions, evaluate management effectiveness, determine value of an ownership
stake or an entire company, price an individual security, determine optimal
capital structure, and so on. Models clearly play a big role in everyday
but perhaps never more so than now. There are many reasons for this, not the
least of which is a clamor for
added financial transparency. People are tired of seeing the markets gyrate in
response to one headline after another about fraud, corporate excess, and hidden
risks. Shareholders, lenders, regulators, and policy-makers want change now and
are no longer willing to accept, without scrutiny, sweet-sounding reassurances
from senior management.
As shown in Exhibit 1, the mandate for better
numbers comes from several places. Major exchanges support improved corporate
governance and they recently asked that listed companies get shareholder
approval before management can implement or change stock option plans. Laudable
and long overdue, this plan calls for informed shareholders, owners who
understand what an option represents, alternative valuation models, and the
dynamic relationship between plan characteristics and the bottom line. The
exchanges are not alone. Business valuators must similarly understand how an
option plan affects a company’s worth and be able to clearly and concisely
explain this to interested parties. On a broader front, accountants are
revisiting existing standards, many of which involve valuation models. Auditors
and financial statement users alike must comprehend how model choice affects the
quality of published information.
The Congressional response includes the
Sarbanes-Oxley Act of 2002, ordering executives to certify that reported numbers
“fairly present in all material respects the financial condition and results of
operations of the issuer…”1 The prospect of stiff penalties for
falsification should encourage a greater focus on the models used to derive
published financial data. Regulators like the SEC are also advocates of enhanced
disclosure. Witness its recent proposal to have companies provide additional
information about offbalance sheet items and various contingencies, including
derivatives, “to the extent that the fair value thereof is not fully reflected
as a liability or asset in the financial statements.”2 Combined with
FAS 133, Accounting for Derivative Instruments and Hedging Activities, modeling
experts are in demand because valuation models are a critical part of
determining what, how much, and when valuation changes hit earnings.
Banks have known for some time that modeling is a
big deal, as they gear up for changes in capital adequacy standards that are
directly tied to valuation. Model choice, good or bad, will determine the size
of pledged reserves for loans and traded assets. Corporations and individuals
who borrow from banks could feel the pinch in the form of higher fees if banks
get it wrong.3 Expert witnesses encounter model risk in the courtroom
as unhappy shareholders and lenders, beset with losses, cry foul, alleging
improper valuation in the form of inflated purchase prices. For securities
that seldom trade or are part of an investment pool about which is little known,
calculation methodology takes on an altogether different meaning with respect to
assessment of damages.4 Bad or inappropriately used models affect
legal outcomes in yet another way if they fail to meet the standards set out in
Daubert v.Merrell Dow Pharmaceuticals, Inc.5 and testimony is
Without a doubt, model-related issues are
relevant as never before. Anyone using a financial model must be prepared to
defend it, warts and all. No one can afford to look at output alone. Valuation
professionals will be under even more pressure to explain what goes into the
black box, how it gets assembled, and whether the output makes sense.
Anatomy of a Financial Model
shown in Exhibit 2, all good models share certain characteristics, starting with
a set of generalized assumptions that reflect economic reality most of the time.
Moreover, a model must be able to be tested to discern whether the output makes
sense, falling within the range of expected values. A stock valuation model that
spits out negative prices makes no sense. A model that generates a company value
that falls significantly below the sum of prices for fungible assets merits
Assuming tests on the model reflect accuracy,
repeated runs with different sets of data should generate consistent results.
Extreme data points should not lead to wildly different numbers. Generally
speaking, the model should be relatively insensitive to the variation of inputs.
Otherwise, data quality dominates the integrity of the model selection process.
Business valuators when choosing from a variety of vendors will want to know
something about each provider’s data-generation methodology. Ignorance is not
bliss. Inappropriately used data or use of inappropriate data costs time, money,
and reputations later on. The model must be cost-effective to use or it will
remain on the shelf, collecting dust. Finally, the model must be easy enough to
explain to others. Brilliant models that cost a fortune in processing time or
cannot be explained to a client, judge, regulator, or programmer are bad news.
There is no perfect model; all have problems, some worse than others. Stated
another way, model risk is a fact of life. Though experts disagree on a precise
formal definition, model risk occurs in situations such as:
Importantly, model risk may exist when applied in
one way but not another.7 For instance, a single-variable regression
is seldom the best way to
• Inappropriate use of an otherwise
• Bad data.
• Hard-to-obtain data.
• Incorrect form of data.
• Computational trouble.
• Incomplete or over-specified model.
It is necessary to recognize model risk before there can be any chance of
improvement. Some common examples
are shown in Exhibit 3.