‘Garbage In, Gospel Out’

Karen Clark, former CEO of AIR Worldwide, now heads her own firm Karen Clark & Co. (www.karenclarkandco.comwww.karenclarkandco.com). She made the statement, “garbage in, gospel out,” at the European Insurance Forum in Dublin last April. But as the hurricane season approaches and with it the efforts of the insurance industry to gain insights into the chaotic world of weather the statement takes on added importance.

Despite her youthful appearance, she has few peers with as much experience in the arcane world of catastrophe modeling. Clark established the original concept of catastrophe models and has been in the forefront of their development ever since. Most recently, she was honored with an award certificate for the 2007 Nobel Peace Prize from the Intergovernmental Panel on Climate Change (IPCC).

No Substitute for Risk Analysis

Clark told the EIF delegates that cat models are no substitutes for risk analysis. She also noted that the current concerns over climate change don’t necessarily mean more or bigger hurricanes. There have been events similar to Katrina in past years, notably the 1926 hurricane that devastated Miami, and is still, when adjusted for inflation, the costliest hurricane in U.S. history.

Her presentation began with a summary of the findings published by IPCC (www.ipcc.ch) in February. First here’s what’s known: 1) atmospheric buildup of greenhouse gases since early 1900s is largely the result of human activities; 2) a warming trend of 1 degree to 1.7 degrees Fahrenheit occurred from 1906 to 2005; and 3) increasing greenhouse gas concentrations tend to warm the planet.

Then there’s what’s very likely: 1) Most of the observed increase in global average temperatures is due to increased anthropogenic (i.e., human activity generated) greenhouse gas concentrations; 2) as atmospheric concentrations continue to rise, average global temperatures and sea levels will continue to rise and precipitation patterns will change.

And, finally, what’s not known: 1) how much warming will occur; 2) how fast it will occur; and 3) how warming will affect precipitation patterns and storms.

According to Clark, the IPCC has more or less projected the following: 1) the best estimated range of projected temperature increase by the end of this century is from 3.1 to 7.2 degrees Fahrenheit (the total range is 2 to 11.5); 2) tropical cyclones are likely (by about 66 percent) to become more intense, with higher peak wind speeds and heavier precipitation (the most likely range is a 2 to 5 percent increase in peak wind speeds over the next 20 years), and 3) most climate models project a global decrease in tropical cyclone frequency.

In addition she looked at past hurricane strength and frequency predictions, which haven’t been entirely accurate. For instance, although the number of named storms fell within the predictions in 2006 and 2007, the number of actual hurricanes in both years seven to 10 were predicted was less, as were the predictions for “major hurricanes. There were only two in each year, where between three and five had been predicted. “In the words of Yogi Berra, ‘Prediction is very hard especially when it’s about the future,'” Clark said.

She also pointed out that hurricane strength and frequency aren’t really the most important issues the insurance industry has to deal with. “The biggest driver of increasing hurricane losses are the increases in numbers, values and sizes of properties in harm’s way.” A look at Miami in 1920 before the 1926 hurricane and what it looks like today reveals two very different cities. The growth all along the Gulf and Atlantic Coasts has made these regions far more costly to insure, regardless of how many hurricanes may come ashore there.

Finding Solutions

There are solutions, and they have to be adopted, as the “status quo is not sustainable,” said Clark. She pointed to two: 1) avoid the risk and let governments take over, and 2) proactively work on private market solutions.

The first involves expanding the National Flood Insurance Program to cover wind damage, providing federal reinsurance, and setting up state catastrophe programs. To some extent this is being done.

The second alternative she outlined is to adopt better risk assessment practices, more efficient processes for accessing the capital markets, and risk mitigation initiatives. This, too, is advancing, but the problem for both solutions is with the first step. Better risk analysis essentially means producing more accurate models.

“Cat models are guides,” Clark said, noting that the use of computers is still subject to the “garbage in, garbage out” rule, as the results are only as good as the data that is analyzed. In too many cases this has now become “garbage in, gospel out,” Clark continued. She defined the phrase as “the tendency to put excessive faith in computer output and to blindly trust what the model says.”

She pointed out, that, as a result, companies “are implementing, and sometimes radically changing, business strategies based on catastrophe model output that: is highly uncertain due to many simplifying assumptions; is highly prone to errors and inaccuracies,” and that ultimately they don’t understand.

“Since the companies themselves don’t understand,” she continued, “and can’t explain the model output, they can’t effectively explain their decisions and strategies to their customers, the policyholders. This is causing unintended and undesirable consequences, including regulatory actions, loss of credibility and good faith, and new competition/loss of customers.”

Catastrophe models, Clark maintained are inherently limited by limitations in scientific knowledge. “Every model is based on simplifying assumptions and there is significant uncertainty around many of those assumptions. Catastrophe model results are also very sensitive to certain assumptions, which is why the model loss estimates vary so significantly for different, scientifically valid assumptions.”

As “guidelines” on how to employ cat models, Clark gave the following advice: 1) no model gives the right answer, all models are wrong; 2) model input and output must be rigorously benchmarked for credibility before it is used for financial decision-making there are many opportunities for errors in the modeling process; 3) other independent information must be used to assess a company’s catastrophe loss potential; 4) models should be used in conjunction with other information to develop an independent view of the risk for developing business strategies and policies that are more effective and robust over time, and 5) standards of practice need to be put into place along with independent audits and reviews of company data, modeling processes, and model results.

Clark urged that “cat models need to be ‘stress tested’ to take into account new possibilities,” as relying on historical data alone, when the climate is changing, isn’t enough.

As an example of one area where real progress can be made, Clark urged passing and then enforcing adequate building codes for areas at risk. “Engineers have given us the means to build hurricane resistant buildings since 1981,” she said. “Much of the damage from hurricanes is due to substandard construction and is avoidable.” She summed up her message by citing a formula from Pilkey’s engineering text published the same year: “Put the building or dwelling at a high enough elevation where the highest waters will not reach it, and make sure the fastest winds will not destroy it.”