|
Reprinted with permission of RIA
Model Risk and Valuation
By Susan M. Mangiero
Reprinted with permission of RIA.
This article first appeared in the March/April 2003 issue of Valuation
Strategies (RIA)
Because the stakes are large when valuation model issues end up in court,
valuation professionals should pay close attention to model risk.
Models are used in business all the time and for
a variety of reasons. Without models, it would be hard to forecast earnings,
simulate cash flows, measure risk, assess competitors, analyze economic
conditions, evaluate management effectiveness, determine value of an ownership
stake or an entire company, price an individual security, determine optimal
capital structure, and so on. Models clearly play a big role in everyday
commerce,
but perhaps never more so than now. There are many reasons for this, not the
least of which is a clamor for
added financial transparency. People are tired of seeing the markets gyrate in
response to one headline after another about fraud, corporate excess, and hidden
risks. Shareholders, lenders, regulators, and policy-makers want change now and
are no longer willing to accept, without scrutiny, sweet-sounding reassurances
from senior management.
As shown in Exhibit 1, the mandate for better
numbers comes from several places. Major exchanges support improved corporate
governance and they recently asked that listed companies get shareholder
approval before management can implement or change stock option plans. Laudable
and long overdue, this plan calls for informed shareholders, owners who
understand what an option represents, alternative valuation models, and the
dynamic relationship between plan characteristics and the bottom line. The
exchanges are not alone. Business valuators must similarly understand how an
option plan affects a company’s worth and be able to clearly and concisely
explain this to interested parties. On a broader front, accountants are
revisiting existing standards, many of which involve valuation models. Auditors
and financial statement users alike must comprehend how model choice affects the
quality of published information.
The Congressional response includes the
Sarbanes-Oxley Act of 2002, ordering executives to certify that reported numbers
“fairly present in all material respects the financial condition and results of
operations of the issuer…”1 The prospect of stiff penalties for
falsification should encourage a greater focus on the models used to derive
published financial data. Regulators like the SEC are also advocates of enhanced
disclosure. Witness its recent proposal to have companies provide additional
information about offbalance sheet items and various contingencies, including
derivatives, “to the extent that the fair value thereof is not fully reflected
as a liability or asset in the financial statements.”2 Combined with
FAS 133, Accounting for Derivative Instruments and Hedging Activities, modeling
experts are in demand because valuation models are a critical part of
determining what, how much, and when valuation changes hit earnings.
Banks have known for some time that modeling is a
big deal, as they gear up for changes in capital adequacy standards that are
directly tied to valuation. Model choice, good or bad, will determine the size
of pledged reserves for loans and traded assets. Corporations and individuals
who borrow from banks could feel the pinch in the form of higher fees if banks
get it wrong.3 Expert witnesses encounter model risk in the courtroom
as unhappy shareholders and lenders, beset with losses, cry foul, alleging
improper valuation in the form of inflated purchase prices. For securities
that seldom trade or are part of an investment pool about which is little known,
calculation methodology takes on an altogether different meaning with respect to
assessment of damages.4 Bad or inappropriately used models affect
legal outcomes in yet another way if they fail to meet the standards set out in
Daubert v.Merrell Dow Pharmaceuticals, Inc.5 and testimony is
excluded.6
Without a doubt, model-related issues are
relevant as never before. Anyone using a financial model must be prepared to
defend it, warts and all. No one can afford to look at output alone. Valuation
professionals will be under even more pressure to explain what goes into the
black box, how it gets assembled, and whether the output makes sense.
Anatomy of a Financial Model
 As
shown in Exhibit 2, all good models share certain characteristics, starting with
a set of generalized assumptions that reflect economic reality most of the time.
Moreover, a model must be able to be tested to discern whether the output makes
sense, falling within the range of expected values. A stock valuation model that
spits out negative prices makes no sense. A model that generates a company value
that falls significantly below the sum of prices for fungible assets merits
inspection.
Assuming tests on the model reflect accuracy,
repeated runs with different sets of data should generate consistent results.
Extreme data points should not lead to wildly different numbers. Generally
speaking, the model should be relatively insensitive to the variation of inputs.
Otherwise, data quality dominates the integrity of the model selection process.
Business valuators when choosing from a variety of vendors will want to know
something about each provider’s data-generation methodology. Ignorance is not
bliss. Inappropriately used data or use of inappropriate data costs time, money,
and reputations later on. The model must be cost-effective to use or it will
remain on the shelf, collecting dust. Finally, the model must be easy enough to
explain to others. Brilliant models that cost a fortune in processing time or
cannot be explained to a client, judge, regulator, or programmer are bad news.
Model RiskThere is no perfect model; all have problems, some worse than others. Stated
another way, model risk is a fact of life. Though experts disagree on a precise
formal definition, model risk occurs in situations such as:
• Inappropriate use of an otherwise
valid model.
• Bad data.
• Hard-to-obtain data.
• Incorrect form of data.
• Computational trouble.
• Incomplete or over-specified model.
It is necessary to recognize model risk before there can be any chance of
improvement. Some common examples
are shown in Exhibit 3.
Importantly, model risk may exist when applied in
one way but not another.7 For instance, a single-variable regression
is seldom the best way to quantify an individual’s lifetime expected earnings
stream for product liability cases. On the other hand, a single-variable
regression may be appropriate in a lost profits case that estimates equity costs
as a function of a single factor, returns on a market basket of investments. A
single-variable regression used to compute a hedge ratio that determines how
much of a derivative instrument to buy or sell is just as logical. Although
never easy, a business valuator must be able to discern:
• What risks are likely to plague a particular model.
• The cost of these risks.
• Whether adjustments can easily and logically be made.
• When the model should be jettisoned in favor of something else. A failure to
do so can be disastrous and costly in more ways than one.
Model Risk Example
One could write a book about model risk and still have plenty to say.8
The case of employee stock options is apropos, given the current debate
about whether and how to expense them. A popular valuation approach is the
familiar Black-Scholes (“B-S”) option pricing model, for which the Nobel Prize
in Economics was awarded to Robert C. Merton and Myron S. Scholes in 1997.9
This model is based on a set of assumptions including, but not limited to:
(1) zero transaction costs, (2) no dividends paid during the life of the option,
(3) no short-selling of the underlying security, (4) a constant riskfree rate of
interest, and (5) unchanging volatility of the underlying security.10
As stated before, evaluating a model to determine its appropriateness for the
task at hand is an important first step. Applied to employee stock options,
critics cite several factors that make the B-S model less than ideal, including
the long-term nature of stock options, the fact that few stock options trade in
an open market, if at all, their exercise is restricted, and issuance by closely
held companies complicates stock price determination.11 Nevertheless,
the B-S model shows up in many places, including tax rules. According to Rev.
Proc. 2002-13,12 a “taxpayer may value a compensatory stock opt ion
using any valuation method that is consistent with generally accepted accounting
principles (such as FAS 123),” adjusted for factors cited in other sections of
the Code. Rev. Proc. 2002-13 goes on to say in section 4 that “the safe harbor
valuation method provided by this revenue procedure is based on the Black-Scholes
model…”
Input selection is a critical second step and
often overlooked in terms of its impact on final numbers. The B-S option pricing
model takes five inputs: (1) price of the underlying stock, (2) option strike
price, (3) risk-free interest rate, (4) time to expiration, and (5) volatility.
A discussion of volatility is beyond the scope of this article but is typically
defined as the annualized standard deviation of the stock’s daily return.13
While it is possible to examine the impact of any of the five variables on opt
ion price, Exhibit 4 considers the three volatility categories described in Rev
Proc. 2002-13, i.e., “low, medium or high.” Although not reflected here, the
filer is further advised to consider whether a company is privately held or
publicly traded as part of settling on a volatility number.14
Even when the B-S model is established as a
suitable model, model risk still arises due to volatility choice, as seen in big
value differences across categories.15 Notice that the relationship
between volatility and option value is neither proportional, nor constant across
the range of all possible volatilities.16 A jump to $12.10 from $7.34
is not exactly the same percentage change as the move from 50% to 85%
volatility.17
Why does any of this matter? A mistake in
volatility choice has a cost. Suppose that a low estimate is used when a medium
category volatility number should have been used instead. For the taxpayer, this
would ultimately result in additional taxes and penalties on discovery of the
error. For an employee seeking damages as part of a termination lawsuit,
insufficiently low volatility would result in underestimating sought-after
damages. For a company that issues options, use of low volatility would lead to
insufficient expense recognition, which in turn would cause inflated earnings.
Volatility is singled out for illustrative
purposes in this example. Other examples can and should be created to
demonstrate each type of model risk. Running sensitivity tests is a good way to
evaluate pitfalls. Unfortunately, model risk is sometimes left undetected until
too late. Being proactive goes a long way towards keeping things on track. Tip
of the Iceberg Model risk is a reality that is here to stay. Ignoring it or
paying scant attention is a luxury few can afford. Generating good numbers is
codified in regulations, “best practices” management, and public expectations.
By the time valuation model issues end up in court, the stakes are large.
Business valuation associations can help by offering educational programs about
model risk, making the topic accessible to those who do not ordinarily look
behind the curtains with the requisite critical eye.
1 “Sarbanes-Oxley Act of 2002,
Section 302, Corporate Responsibility for Financial Reports,” Public Law No:
107-204, 7/30/02.
2 “Proposed Rules: Disclosure In Management’s Discussion and Analysis
About Off-Balance Sheet Arrangements, Contractual Obligations and Contingent
Liabilities and Commitments,” SEC, 67 Fed. Reg. 68054 (2002).
3 “Consultative Document: The New Basel Capital Accord, Issued for
Comment by 31 May 2001,” Basel Committee on Banking Supervision, Bank for
International Settlements, January 2001.
4 “Up Front: Lipper II: Judgment Day,” edited by Sheridan Prasso, cites debate
over valuation methodology to determine investor restitution. See BusinessWeek,
1/20/03, page 10.
5 509 U.S. 579 (1993). See also Kumho Tire Co. v. Carmichael, 526 U.S. 137
(1999), which extends Daubert to persons other than scientists.
6 Mangiero, “Financial Model Risk Looms Large,” 9 Investment Lawyer 1 (November
2002).
7 Emanuel Derman cites inapplicability as a primary type of model risk. See
“Model Risk,” Goldman Sachs Quantitative Strategies Research Notes (April 1996).
8 The author is currently working on a book about financial model risk that
looks at commonly used models, including derivative instrument models.
9 The third key person, Fischer Black, died in 1995, making him ineligible for
this honor since
the award is not given posthumously. See
http://www.ams.org/new-in-math/nobel1997econ.html.
10 See http://www.riskglossary.com for one of many introductory expositions
about the Black- Scholes option pricing model.
11 The International Employee Stock Options Coalition recently praised the FASB
for requesting that information about stock options be provided to the public on
a quarterly basis. At the same time, it cited problems with the B-S model,
applied to employee stock options. See “Statement by International Employee
Stock Options Coalition,” Business Wire, 10/7/02.
12 2002-8 IRB 549.
13 The logarithmic form of returns is often used. See http://www.investorwords.com
for one of many definitions of volatility. Historical volatility is mainly used
as the output. However, a market option price can be used to estimate what is
known as “implied” volatility.
14 According to Rev. Proc. 2002-13, “If the stock is not publicly traded on an
established securities market or otherwise, but the stock is required to be
registered under the Securities Exchange Act of 1934, the volatility for such
stock is assumed to be the same as the volatility for a comparable corporation
that is publicly traded” and is similar with respect to “industry, corporate
size, earnings, market capitalization, and debt-equity structure.”
15 Employee stock options are call options that entitle the employee to buy the
stock at a stated price. A European call is exercised at expiration. An American
call can be exercised at or prior to expiration.
16 Vega measures the volatility-option value relationship and is actively used
for risk management purposes.
17 Moving from 50% to 85% volatility does not result in a price that is 1.7
times greater. In other words, 85% is 1.7 times a 50% value, but the resulting
price of $12.10 is 1.65 times greater than $7.34, the price associated with the
50% value. Moreover, 50% is 3.3 times the 15% volatility level but generates an
option price of $7.34 that is 2.95 times as big as the $2.49 price associated
with the 15% volatility input.
|
