741

This chapter should be cited as:

Flato, G., J. Marotzke, B. Abiodun, P. Braconnot, S.C. Chou, W. Collins, P. Cox, F. Driouech, S. Emori, V. Eyring, C.

Forest, P. Gleckler, E. Guilyardi, C. Jakob, V. Kattsov, C. Reason and M. Rummukainen, 2013: Evaluation of Climate

Models. In: Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assess-

ment Report of the Intergovernmental Panel on Climate Change [Stocker, T.F., D. Qin, G.-K. Plattner, M. Tignor,

S.K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex and P.M. Midgley (eds.)]. Cambridge University Press, Cambridge,

United Kingdom and New York, NY, USA.

Coordinating Lead Authors:

Gregory Flato (Canada), Jochem Marotzke (Germany)

Lead Authors:

Babatunde Abiodun (South Africa), Pascale Braconnot (France), Sin Chan Chou (Brazil), William

Collins (USA), Peter Cox (UK), Fatima Driouech (Morocco), Seita Emori (Japan), Veronika

Eyring (Germany), Chris Forest (USA), Peter Gleckler (USA), Eric Guilyardi (France), Christian

Jakob (Australia), Vladimir Kattsov (Russian Federation), Chris Reason (South Africa), Markku

Rummukainen (Sweden)

Contributing Authors:

Krishna AchutaRao (India), Alessandro Anav (UK), Timothy Andrews (UK), Johanna Baehr

(Germany), Nathaniel L. Bindoff (Australia), Alejandro Bodas-Salcedo (UK), Jennifer Catto

(Australia), Don Chambers (USA), Ping Chang (USA), Aiguo Dai (USA), Clara Deser (USA),

Francisco Doblas-Reyes (Spain), Paul J. Durack (USA/Australia), Michael Eby (Canada), Ramon

de Elia (Canada), Thierry Fichefet (Belgium), Piers Forster (UK), David Frame (UK/New Zealand),

John Fyfe (Canada), Emiola Gbobaniyi (Sweden/Nigeria), Nathan Gillett (Canada), Jesus Fidel

González-Rouco (Spain), Clare Goodess (UK), Stephen Grifﬁes (USA), Alex Hall (USA), Sandy

Harrison (Australia), Andreas Hense (Germany), Elizabeth Hunke (USA), Tatiana Ilyina (Germany),

Detelina Ivanova (USA), Gregory Johnson (USA), Masa Kageyama (France), Viatcheslav Kharin

(Canada), Stephen A. Klein (USA), Jeff Knight (UK), Reto Knutti (Switzerland), Felix Landerer

(USA), Tong Lee (USA), Hongmei Li (Germany/China), Natalie Mahowald (USA), Carl Mears

(USA), Gerald Meehl (USA), Colin Morice (UK), Rym Msadek (USA), Gunnar Myhre (Norway),

J. David Neelin (USA), Jeff Painter (USA), Tatiana Pavlova (Russian Federation), Judith Perlwitz

(USA), Jean-Yves Peterschmitt (France), Jouni Räisänen (Finland), Florian Rauser (Germany),

Jeffrey Reid (USA), Mark Rodwell (UK), Benjamin Santer (USA), Adam A. Scaife (UK), Jörg

Schulz (Germany), John Scinocca (Canada), David Sexton (UK), Drew Shindell (USA), Hideo

Shiogama (Japan), Jana Sillmann (Canada), Adrian Simmons (UK), Kenneth Sperber (USA),

David Stephenson (UK), Bjorn Stevens (Germany), Peter Stott (UK), Rowan Sutton (UK), Peter

W. Thorne (USA/Norway/UK), Geert Jan van Oldenborgh (Netherlands), Gabriel Vecchi (USA),

Mark Webb (UK), Keith Williams (UK), Tim Woollings (UK), Shang-Ping Xie (USA), Jianglong

Zhang (USA)

Review Editors:

Isaac Held (USA), Andy Pitman (Australia), Serge Planton (France), Zong-Ci Zhao (China)

Evaluation of

Climate Models

742

Table of Contents

Executive Summary ..................................................................... 743

9.1 Climate Models and Their Characteristics ............... 746

9.1.1 Scope and Overview of this Chapter ......................... 746

9.1.2 Overview of Model Types to Be Evaluated ................ 746

9.1.3 Model Improvements ................................................ 748

Box 9.1: Climate Model Development and Tuning ................ 749

9.2 Techniques for Assessing Model Performance ....... 753

9.2.1 New Developments in Model Evaluation

Approaches ............................................................... 753

9.2.2 Ensemble Approaches for Model Evaluation ............. 754

9.2.3 The Model Evaluation Approach Used in this

Chapter and Its Limitations ....................................... 755

9.3 Experimental Strategies in Support of Climate

Model Evaluation ............................................................ 759

9.3.1 The Role of Model Intercomparisons ......................... 759

9.3.2 Experimental Strategy for Coupled Model

Intercomparison Project Phase 5 ............................... 759

9.4 Simulation of Recent and Longer-Term Records

in Global Models ............................................................. 760

9.4.1 Atmosphere .............................................................. 760

Box 9.2: Climate Models and the Hiatus in Global Mean

Surface Warming of the Past 15 Years .................................... 769

9.4.2 Ocean ........................................................................ 777

9.4.3 Sea Ice ...................................................................... 787

9.4.4 Land Surface, Fluxes and Hydrology .......................... 790

9.4.5 Carbon Cycle ............................................................. 792

9.4.6 Aerosol Burdens and Effects on Insolation ................ 794

9.5 Simulation of Variability and Extremes .................... 795

9.5.1 Importance of Simulating Climate Variability ............ 795

9.5.2 Diurnal-to-Seasonal Variability .................................. 796

9.5.3 Interannual-to-Centennial Variability ........................ 799

9.5.4 Extreme Events ......................................................... 806

Box 9.3: Understanding Model Performance ......................... 809

9.6 Downscaling and Simulation of Regional-Scale

Climate ............................................................................... 810

9.6.1 Global Models ........................................................... 810

9.6.2 Regional Climate Downscaling ................................. 814

9.6.3 Skill of Downscaling Methods ................................... 814

9.6.4 Value Added through RCMs ...................................... 815

9.6.5 Sources of Model Errors and Uncertainties ............... 815

9.6.6 Relating Downscaling Performance to Credibility

of Regional Climate Information ............................... 816

9.7 Climate Sensitivity and Climate Feedbacks ............ 817

9.7.1 Equilibrium Climate Sensitivity, Idealized Radiative

Forcing, and Transient Climate Response in the

Coupled Model Intercomparison Project

Phase 5 Ensemble ..................................................... 817

9.7.2 Understanding the Range in Model Climate

Sensitivity: Climate Feedbacks .................................. 819

9.7.3 Climate Sensitivity and Model Performance .............. 820

9.8 Relating Model Performance to Credibility of

Model Applications ......................................................... 821

9.8.1 Synthesis Assessment of Model Performance ............ 821

9.8.2 Implications of Model Evaluation for Climate

Change Detection and Attribution ............................ 825

9.8.3 Implications of Model Evaluation for Model

Projections of Future Climate .................................... 825

References .................................................................................. 828

Appendix 9.A: Climate Models Assessed

in Chapter 9 .................................................................................. 854

Frequently Asked Questions

FAQ 9.1 Are Climate Models Getting Better, and How

Would We Know? ................................................... 824

743

Evaluation of Climate Models Chapter 9

In this Report, the following summary terms are used to describe the available evidence: limited, medium, or robust; and for the degree of agreement: low, medium, or high.

A level of conﬁdence is expressed using ﬁve qualiﬁers: very low, low, medium, high, and very high, and typeset in italics, e.g., medium conﬁdence. For a given evidence and

agreement statement, different conﬁdence levels can be assigned, but increasing levels of evidence and degrees of agreement are correlated with increasing conﬁdence (see

Section 1.4 and Box TS.1 for more details).

Executive Summary

Climate models have continued to be developed and improved

since the AR4, and many models have been extended into Earth

System models by including the representation of biogeochem-

ical cycles important to climate change. These models allow for

policy-relevant calculations such as the carbon dioxide (CO

) emissions

compatible with a speciﬁed climate stabilization target. In addition, the

range of climate variables and processes that have been evaluated has

greatly expanded, and differences between models and observations

are increasingly quantiﬁed using ‘performance metrics’. In this chapter,

model evaluation covers simulation of the mean climate, of historical

climate change, of variability on multiple time scales and of regional

modes of variability. This evaluation is based on recent internationally

coordinated model experiments, including simulations of historic and

paleo climate, specialized experiments designed to provide insight into

key climate processes and feedbacks and regional climate downscal-

ing. Figure 9.44 provides an overview of model capabilities as assessed

in this chapter, including improvements, or lack thereof, relative to

models assessed in the AR4. The chapter concludes with an assessment

of recent work connecting model performance to the detection and

attribution of climate change as well as to future projections. {9.1.2,

9.8.1, Table 9.1, Figure 9.44}

The ability of climate models to simulate surface temperature

has improved in many, though not all, important aspects rel-

ative to the generation of models assessed in the AR4. There

continues to be very high conﬁdence

that models reproduce observed

large-scale mean surface temperature patterns (pattern correlation of

~0.99), though systematic errors of several degrees are found in some

regions, particularly over high topography, near the ice edge in the

North Atlantic, and over regions of ocean upwelling near the equa-

tor. On regional scales (sub-continental and smaller), the conﬁdence

in model capability to simulate surface temperature is less than for

the larger scales; however, regional biases are near zero on average,

with intermodel spread of roughly ±3°C. There is high conﬁdence that

regional-scale surface temperature is better simulated than at the time

of the AR4. Current models are also able to reproduce the large-scale

patterns of temperature during the Last Glacial Maximum (LGM), indi-

cating an ability to simulate a climate state much different from the

present. {9.4.1, 9.6.1, Figures 9.2, 9.6, 9.39, 9.40}

There is very high conﬁdence that models reproduce the gener-

al features of the global-scale annual mean surface temperature

increase over the historical period, including the more rapid

warming in the second half of the 20th century, and the cooling

immediately following large volcanic eruptions. Most simulations

of the historical period do not reproduce the observed reduction in

global mean surface warming trend over the last 10 to 15 years. There

is medium conﬁdence that the trend difference between models and

observations during 1998–2012 is to a substantial degree caused by

internal variability, with possible contributions from forcing error and

some models overestimating the response to increasing greenhouse

gas (GHG) forcing. Most, though not all, models overestimate the

observed warming trend in the tropical troposphere over the last 30

years, and tend to underestimate the long-term lower stratospheric

cooling trend. {9.4.1, Box 9.2, Figure 9.8}

The simulation of large-scale patterns of precipitation has

improved somewhat since the AR4, although models continue

to perform less well for precipitation than for surface tempera-

ture. The spatial pattern correlation between modelled and observed

annual mean precipitation has increased from 0.77 for models availa-

ble at the time of the AR4 to 0.82 for current models. At regional scales,

precipitation is not simulated as well, and the assessment remains dif-

ﬁcult owing to observational uncertainties. {9.4.1, 9.6.1, Figure 9.6}

The simulation of clouds in climate models remains challeng-

ing. There is very high conﬁdence that uncertainties in cloud processes

explain much of the spread in modelled climate sensitivity. However,

the simulation of clouds in climate models has shown modest improve-

ment relative to models available at the time of the AR4, and this has

been aided by new evaluation techniques and new observations for

clouds. Nevertheless, biases in cloud simulation lead to regional errors

on cloud radiative effect of several tens of watts per square meter.

{9.2.1, 9.4.1, 9.7.2, Figures 9.5, 9.43}

Models are able to capture the general characteristics of storm

tracks and extratropical cyclones, and there is some evidence of

improvement since the AR4. Storm track biases in the North Atlantic

have improved slightly, but models still produce a storm track that is

too zonal and underestimate cyclone intensity. {9.4.1}

Many models are able to reproduce the observed changes in

upper ocean heat content from 1961 to 2005 with the mul-

ti-model mean time series falling within the range of the avail-

able observational estimates for most of the period. The ability

of models to simulate ocean heat uptake, including variations imposed

by large volcanic eruptions, adds conﬁdence to their use in assessing

the global energy budget and simulating the thermal component of

sea level rise. {9.4.2, Figure 9.17}

The simulation of the tropical Paciﬁc Ocean mean state has

improved since the AR4, with a 30% reduction in the spurious

westward extension of the cold tongue near the equator, a per-

vasive bias of coupled models. The simulation of the tropical Atlan-

tic remains deﬁcient with many models unable to reproduce the basic

east–west temperature gradient. {9.4.2, Figure 9.14}

744

Chapter 9 Evaluation of Climate Models

Current climate models reproduce the seasonal cycle of Arctic

sea ice extent with a multi-model mean error of less than about

10% for any given month. There is robust evidence that the

downward trend in Arctic summer sea ice extent is better sim-

ulated than at the time of the AR4, with about one quarter of

the simulations showing a trend as strong as, or stronger, than

in observations over the satellite era (since 1979). There is a ten-

dency for models to slightly overestimate sea ice extent in the Arctic

(by about 10%) in winter and spring. In the Antarctic, the multi-model

mean seasonal cycle agrees well with observations, but inter-model

spread is roughly double that for the Arctic. Most models simulate a

small decreasing trend in Antarctic sea ice extent, albeit with large

inter-model spread, in contrast to the small increasing trend in obser-

vations. {9.4.3, Figures 9.22, 9.24}

Models are able to reproduce many features of the observed

global and Northern Hemispher (NH) mean temperature vari-

ance on interannual to centennial time scales (high conﬁdence),

and most models are now able to reproduce the observed peak

in variability associated with the El Niño (2- to 7-year period)

in the Tropical Paciﬁc. The ability to assess variability from millennial

simulations is new since the AR4 and allows quantitative evaluation of

model estimates of low-frequency climate variability. This is important

when using climate models to separate signal and noise in detection

and attribution studies (Chapter 10). {9.5.3, Figures 9.33, 9.35}

Many important modes of climate variability and intraseasonal

to seasonal phenomena are reproduced by models, with some

improvements evident since the AR4. The statistics of the global

monsoon, the North Atlantic Oscillation, the El Niño-Southern Oscilla-

tion (ENSO), the Indian Ocean Dipole and the Quasi-Biennial Oscilla-

tion are simulated well by several models, although this assessment is

tempered by the limited scope of analysis published so far, or by limited

observations. There are also modes of variability that are not simulated

well. These include modes of Atlantic Ocean variability of relevance

to near term projections in Chapter 11 and ENSO teleconnections

outside the tropical Paciﬁc, of relevance to Chapter 14. There is high

conﬁdence that the multi-model statistics of monsoon and ENSO have

improved since the AR4. However, this improvement does not occur in

all models, and process-based analysis shows that biases remain in the

background state and in the strength of associated feedbacks. {9.5.3,

Figures 9.32, 9.35, 9.36}

There has been substantial progress since the AR4 in the assess-

ment of model simulations of extreme events. Based on assess-

ment of a suite of indices, the inter-model range of simulated climate

extremes is similar to the spread amongst observationally based esti-

mates in most regions. In addition, changes in the frequency of extreme

warm and cold days and nights over the second half of the 20th centu-

ry are consistent between models and observations, with the ensemble

global mean time series generally falling within the range of observa-

tional estimates. The majority of models underestimate the sensitivity

of extreme precipitation to temperature variability or trends, especially

in the tropics, which implies that models may underestimate the pro-

jected increase in extreme precipitation in the future. Some high-res-

olution atmospheric models have been shown to reproduce observed

year-to-year variability of Atlantic hurricane counts when forced with

observed sea surface temperatures, though so far only a few studies of

this kind are available. {9.5.4, Figure 9.37}

An important development since the AR4 is the more wide-

spread use of Earth System models, which include an interac-

tive carbon cycle. In the majority of these models, the simulated

global land and ocean carbon sinks over the latter part of the

20th century fall within the range of observational estimates.

However, the regional patterns of carbon uptake and release are less

well reproduced, especially for NH land where models systematically

underestimate the sink implied by atmospheric inversion techniques

The ability of models to simulate carbon ﬂuxes is important because

these models are used to estimate ‘compatible emissions’ (carbon

dioxide emission pathways compatible with a particular climate

change target; see Chapter 6). {9.4.5, Figure 9.27}

The majority of Earth System models now include an interac-

tive representation of aerosols, and make use of a consistent

speciﬁcation of anthropogenic sulphur dioxide emissions. How-

ever, uncertainties in sulphur cycle processes and natural sources and

sinks remain and so, for example, the simulated aerosol optical depth

over oceans ranges from 0.08 to 0.22 with roughly equal numbers of

models over- and under-estimating the satellite-estimated value of

0.12. {9.1.2, 9.4.6, Table 9.1, Figure 9.29}

Time-varying ozone is now included in the latest suite of models,

either prescribed or calculated interactively. Although in some

models there is only medium agreement with observed changes in

total column ozone, the inclusion of time-varying stratospheric ozone

constitutes a substantial improvement since the AR4 where half of the

models prescribed a constant climatology. As a result, there is robust

evidence that the representation of climate forcing by stratospheric

ozone has improved since the AR4. {9.4.1, Figure 9.10}

Regional downscaling methods are used to provide climate

information at the smaller scales needed for many climate

impact studies, and there is high conﬁdence that downscaling

adds value both in regions with highly variable topography and

for various small-scale phenomena. Regional models necessar-

ily inherit biases from the global models used to provide boundary

conditions. Furthermore, the ability to systematically evaluate region-

al climate models, and statistical downscaling schemes, is hampered

because coordinated intercomparison studies are still emerging. How-

ever, several studies have demonstrated that added value arises from

higher resolution of stationary features like topography and coastlines,

and from improved representation of small-scale processes like con-

vective precipitation. {9.6.4}

Earth system Models of Intermediate Complexity (EMICs) pro-

vide simulations of millennial time-scale climate change, and are

used as tools to interpret and expand upon the results of more

comprehensive models. Although they are limited in the scope and

resolution of information provided, EMIC simulations of global mean

surface temperature, ocean heat content and carbon cycle response

over the 20th century are consistent with the historical records and

with more comprehensive models, suggesting that they can be used to

provide calibrated projections of long-term transient climate response

745

Evaluation of Climate Models Chapter 9

and stabilization, as well as large ensembles and alternative, policy-rel-

evant, scenarios. {9.4.1, 9.4.2, 9.4.5, Figures 9.8, 9.17, 9.27}

The Coupled Model Intercomparison Project Phase 5 (CMIP5)

model spread in equilibrium climate sensitivity ranges from

2.1°C to 4.7°C and is very similar to the assessment in the AR4.

No correlation is found between biases in global mean surface tem-

perature and equilibrium climate sensitivity, and so mean temperature

biases do not obviously affect the modelled response to GHG forcing.

There is very high conﬁdence that the primary factor contributing to

the spread in equilibrium climate sensitivity continues to be the cloud

feedback. This applies to both the modern climate and the LGM. There

is likewise very high conﬁdence that, consistent with observations,

models show a strong positive correlation between tropospheric tem-

perature and water vapour on regional to global scales, implying a pos-

itive water vapour feedback in both models and observations. {9.4.1,

9.7.2, Figures 9.9, 9.42, 9.43}

Climate and Earth System models are based on physical princi-

ples, and they reproduce many important aspects of observed

climate. Both aspects contribute to our conﬁdence in the

models’ suitability for their application in detection and attri-

bution studies (Chapter 10) and for quantitative future predic-

tions and projections (Chapters 11 to 14). In general, there is no

direct means of translating quantitative measures of past performance

into conﬁdent statements about ﬁdelity of future climate projections.

However, there is increasing evidence that some aspects of observed

variability or trends are well correlated with inter-model differences

in model projections for quantities such as Arctic summertime sea

ice trends, snow albedo feedback, and the carbon loss from tropical

land. These relationships provide a way, in principle, to transform an

observable quantity into a constraint on future projections, but the

application of such constraints remains an area of emerging research.

There has been substantial progress since the AR4 in the methodol-

ogy to assess the reliability of a multi-model ensemble, and various

approaches to improve the precision of multi-model projections are

being explored. However, there is still no universal strategy for weight-

ing the projections from different models based on their historical per-

formance. {9.8.3, Figure 9.45}

746

Chapter 9 Evaluation of Climate Models

9.1 Climate Models and Their Characteristics

9.1.1 Scope and Overview of this Chapter

Climate models are the primary tools available for investigating the

response of the climate system to various forcings, for making climate

predictions on seasonal to decadal time scales and for making projec-

tions of future climate over the coming century and beyond. It is crucial

therefore to evaluate the performance of these models, both individu-

ally and collectively. The focus of this chapter is primarily on the models

whose results will be used in the detection and attribution Chapter 10

and the chapters that present and assess projections (Chapters 11 to

14; Annex I), and so this is necessarily an incomplete evaluation. In

particular, this chapter draws heavily on model results collected as part

of the Coupled Model Intercomparison Projects (CMIP3 and CMIP5)

(Meehl et al., 2007; Taylor et al., 2012b), as these constitute a set of

coordinated and thus consistent and increasingly well-documented cli-

mate model experiments. Other intercomparison efforts, such as those

dealing with Regional Climate Models (RCMs) and those dealing with

Earth System Models of Intermediate Complexity (EMICs) are also used.

It should be noted that the CMIP3 model archive has been extensively

evaluated, and much of that evaluation has taken place subsequent to

the AR4. By comparison, the CMIP5 models are only now being evalu-

ated and so there is less published literature available. Where possible

we show results from both CMIP3 and CMIP5 models so as to illustrate

changes in model performance over time; however, where only CMIP3

results are available, they still constitute a useful evaluation of model

performance in that for many quantities, the CMIP3 and CMIP5 model

performances are broadly similar.

The direct approach to model evaluation is to compare model output

with observations and analyze the resulting difference. This requires

knowledge of the errors and uncertainties in the observations, which

have been discussed in Chapters 2 through 6. Where possible, aver-

ages over the same time period in both models and observations are

compared, although for many quantities the observational record is

rather short, or only observationally based estimates of the climato-

logical mean are available. In cases where observations are lacking,

we resort to intercomparison of model results to provide at least some

quantiﬁcation of model uncertainty via inter-model spread.

After a more thorough discussion of the climate models and meth-

ods for evaluation in Sections 9.1 and 9.2, we describe climate model

experiments in Section 9.3, evaluate recent and longer-term records as

simulated by climate models in Section 9.4, variability and extremes

in Section 9.5, and regional-scale climate simulation including down-

scaling in Section 9.6. We conclude with a discussion of model perfor-

mance and climate sensitivity in Section 9.7, and the relation between

model performance and the credibility of future climate projections in

Section 9.8.

9.1.2 Overview of Model Types to Be Evaluated

The models used in climate research range from simple energy balance

models to complex Earth System Models (ESMs) requiring state of the

art high-performance computing. The choice of model depends directly

on the scientiﬁc question being addressed (Held, 2005; Collins et al.,

2006d). Applications include simulating palaeo or historical climate,

sensitivity and process studies for attribution and physical understand-

ing, predicting near-term climate variability and change on seasonal

to decadal time scales, making projections of future climate change

over the coming century or more and downscaling such projections

to provide more detail at the regional and local scale. Computational

cost is a factor in all of these, and so simpliﬁed models (with reduced

complexity or spatial resolution) can be used when larger ensembles

or longer integrations are required. Examples include exploration of

parameter sensitivity or simulations of climate change on the millenni-

al or longer time scale. Here, we provide a brief overview of the climate

models evaluated in this chapter.

9.1.2.1 Atmosphere–Ocean General Circulation Models

Atmosphere–Ocean General Circulation Models (AOGCMs) were the

‘standard’ climate models assessed in the AR4. Their primary function

is to understand the dynamics of the physical components of the cli-

mate system (atmosphere, ocean, land and sea ice), and for making

projections based on future greenhouse gas (GHG) and aerosol forcing.

These models continue to be extensively used, and in particular are run

(sometimes at higher resolution) for seasonal to decadal climate pre-

diction applications in which biogeochemical feedbacks are not critical

(see Chapter 11). In addition, high-resolution or variable-resolution

AOGCMs are often used in process studies or applications with a focus

on a particular region. An overview of the AOGCMs assessed in this

chapter can be found in Table 9.1 and the details in Table 9.A.1. For

some speciﬁc applications, an atmospheric component of such a model

is used on its own.

9.1.2.2 Earth System Models

ESMs are the current state-of-the-art models, and they expand on

AOGCMs to include representation of various biogeochemical cycles

such as those involved in the carbon cycle, the sulphur cycle, or ozone

(Flato, 2011). These models provide the most comprehensive tools

available for simulating past and future response of the climate system

to external forcing, in which biogeochemical feedbacks play an impor-

tant role. An overview of the ESMs assessed in this chapter can be

found in Table 9.1 and details in Table 9.A.1.

9.1.2.3 Earth System Models of Intermediate Complexity

EMICs attempt to include relevant components of the Earth system,

but often in an idealized manner or at lower resolution than the

models described above. These models are applied to certain scientiﬁc

questions such as understanding climate feedbacks on millennial time

scales or exploring sensitivities in which long model integrations or

large ensembles are required (Claussen et al., 2002; Petoukhov et al.,

2005). This class of models often includes Earth system components

not yet included in all ESMs (e.g., ice sheets). As computing power

increases, this model class has continued to advance in terms of reso-

lution and complexity. An overview of EMICs assessed in this chapter

and in the AR5 WG1 is provided in Table 9.2 with additional details in

Table 9.A.2.

747

Evaluation of Climate Models Chapter 9

Table 9.1 | Main features of the Atmosphere–Ocean General Circulation Models (AOGCMs) and Earth System Models (ESMs) participating in Coupled Model Intercomparison

Project Phase 5 (CMIP5), and a comparison with Coupled Model Intercomparison Project Phase 3 (CMIP3), including components and resolution of the atmosphere and the ocean

models. Detailed CMIP5 model description can be found in Table 9.A.1 (* refers to Table 9.A.1 for more details). Ofﬁcial CMIP model names are used. HT stands for High-Top

atmosphere, which has a fully resolved stratosphere with a model top above the stratopause. AMIP stands for models with atmosphere and land surface only, using observed sea

surface temperature and sea ice extent. A component is coloured when it includes at least a physically based prognostic equation and at least a two-way coupling with another

component, allowing climate feedbacks. For aerosols, lighter shading means ‘semi-interactive’ and darker shading means ‘fully interactive’. The resolution of the land surface usually

follows that of the atmosphere, and the resolution of the sea ice follows that of the ocean. In moving from CMIP3 to CMIP5, note the increased complexity and resolution as well

as the absence of artiﬁcial ﬂux correction (FC) used in some CMIP3 models.

748

Chapter 9 Evaluation of Climate Models

Signiﬁcant advances in EMIC capabilities are inclusion of ice sheets

(UVic 2.9, Weaver et al., 2001; CLIMBER-2.4, Petoukhov et al., 2000;

LOVECLIM, Goosse et al., 2010) and ocean sediment models (DCESS,

Shaffer et al., 2008; UVic 2.9, Weaver et al., 2001; Bern3D-LPJ, Ritz

et al., 2011). These additional interactive components provide criti-

cal feedbacks involved in sea level rise estimates and carbon cycle

response on millennial time scales (Zickfeld et al., 2013). Further, the

ﬂexibility and efﬁciency of EMICs allow calibration to speciﬁc climate

change events to remove potential biases.

9.1.2.4 Regional Climate Models

RCMs are limited-area models with representations of climate process-

es comparable to those in the atmospheric and land surface compo-

nents of AOGCMs, though typically run without interactive ocean and

sea ice. RCMs are often used to dynamically ‘downscale’ global model

simulations for some particular geographical region to provide more

detailed information (Laprise, 2008; Rummukainen, 2010). By contrast,

empirical and statistical downscaling methods constitute a range of

techniques to provide similar regional or local detail.

9.1.3 Model Improvements

The climate models assessed in this report have seen a number of

improvements since the AR4. Model development is a complex and

iterative task: improved physical process descriptions are developed,

new model components are introduced and the resolution of the

models is improved. After assembly of all model components, model

parameters are adjusted, or tuned, to provide a stable model climate.

Table 9.2 | Main features of the EMICs assessed in the AR5, including components and complexity of the models. Model complexityfor four componentsis indicated by colour

shading. Further detailed descriptions of the models are contained in Table 9.A.2.

The overall approach to model development and tuning is summarized

in Box 9.1.

9.1.3.1 Parameterizations

Parameterizations are included in all model components to represent

processes that cannot be explicitly resolved; they are evaluated both

in isolation and in the context of the full model. The purpose of this

section is to highlight recent developments in the parameterizations

employed in each model component. Some details for individual

models are listed in Table 9.1.

9.1.3.1.1 Atmosphere

Atmospheric models must parameterize a wide range of processes,

including those associated with atmospheric convection and clouds,

cloud-microphysical and aerosol processes and their interaction,

boundary layer processes, as well as radiation and the treatment of

unresolved gravity waves. Advances made in the representation of

cloud processes, including aerosol–cloud and cloud–radiation interac-

tions, and atmospheric convection are described in Sections 7.2.3 and

7.4.

Improvements in representing the atmospheric boundary layer since

the AR4 have focussed on basic boundary layer processes, the rep-

resentation of the stable boundary layer, and boundary layer clouds

(Teixeira et al., 2008). Several global models have successfully adopt-

ed new approaches to the parameterization of shallow cumulus con-

vection and moist boundary layer turbulence that acknowledge their

749

Evaluation of Climate Models Chapter 9

Box 9.1 | Climate Model Development and Tuning

The Atmosphere–Ocean General Circulation Models, Earth System Models and Regional Climate Models evaluated here are based on

fundamental laws of nature (e.g., energy, mass and momentum conservation). The development of climate models involves several

principal steps:

1. Expressing the system’s physical laws in mathematical terms. This requires theoretical and observational work in deriving and sim-

plifying mathematical expressions that best describe the system.

2. Implementing these mathematical expressions on a computer. This requires developing numerical methods that allow the solution

of the discretized mathematical expressions, usually implemented on some form of grid such as the latitude–longitude–height grid

for atmospheric or oceanic models.

3. Building and implementing conceptual models (usually referred to as parameterizations) for those processes that cannot be rep-

resented explicitly, either because of their complexity (e.g., biochemical processes in vegetation) or because the spatial and/or

temporal scales on which they occur are not resolved by the discretized model equations (e.g., cloud processes and turbulence). The

development of parameterizations has become very complex (e.g., Jakob, 2010) and is often achieved by developing conceptual

models of the process of interest in isolation using observations and comprehensive process models. The complexity of each process

representation is constrained by observations, computational resources and current knowledge (e.g., Randall et al., 2007).

The application of state-of-the-art climate models requires signiﬁcant supercomputing resources. Limitations in those resources lead to

additional constraints. Even when using the most powerful computers, compromises need to be made in three main areas:

1. Numerical implementations allow for a choice of grid spacing and time step, usually referred to as ‘model resolution’. Higher model

resolution generally leads to mathematically more accurate models (although not necessarily more reliable simulations) but also to

higher computational costs. The ﬁnite resolution of climate models implies that the effects of certain processes must be represented

through parameterizations (e.g., the carbon cycle or cloud and precipitation processes; see Chapters 6 and 7).

2. The climate system contains many processes, the relative importance of which varies with the time scale of interest (e.g., the carbon

cycle). Hence compromises to include or exclude certain processes or components in a model must be made, recognizing that an

increase in complexity generally leads to an increase in computational cost (Hurrell et al., 2009).

3. Owing to uncertainties in the model formulation and the initial state, any individual simulation represents only one of the possible

pathways the climate system might follow. To allow some evaluation of these uncertainties, it is necessary to carry out a number of

simulations either with several models or by using an ensemble of simulations with a single model, both of which increase compu-

tational cost.

Trade-offs amongst the various considerations outlined above are guided by the intended model application and lead to the several

classes of models introduced in Section 9.1.2.

Individual model components (e.g., the atmosphere, the ocean, etc.) are typically ﬁrst evaluated in isolation as part of the model devel-

opment process. For instance, the atmospheric component can be evaluated by prescribing sea surface temperature (SST) (Gates et al.,

1999) or the ocean and land components by prescribing atmospheric conditions (Barnier et al., 2006; Grifﬁes et al., 2009). Subsequently,

the various components are assembled into a comprehensive model, which then undergoes a systematic evaluation. At this stage, a small

subset of model parameters remains to be adjusted so that the model adheres to large-scale observational constraints (often global aver-

ages). This ﬁnal parameter adjustment procedure is usually referred to as ‘model tuning’. Model tuning aims to match observed climate

system behaviour and so is connected to judgements as to what constitutes a skilful representation of the Earth’s climate. For instance,

maintaining the global mean top of the atmosphere (TOA) energy balance in a simulation of pre-industrial climate is essential to prevent

the climate system from drifting to an unrealistic state. The models used in this report almost universally contain adjustments to param-

eters in their treatment of clouds to fulﬁl this important constraint of the climate system (Watanabe et al., 2010; Donner et al., 2011; Gent

et al., 2011; Golaz et al., 2011; Martin et al., 2011; Hazeleger et al., 2012; Mauritsen et al., 2012; Hourdin et al., 2013).

With very few exceptions (Mauritsen et al., 2012; Hourdin et al., 2013) modelling centres do not routinely describe in detail how they

tune their models. Therefore the complete list of observational constraints toward which a particular model is tuned is generally not

(continued on next page)

750

Chapter 9 Evaluation of Climate Models

close mutual coupling. One new development is the Eddy-Diffusivi-

ty-Mass-Flux (EDMF) approach (Siebesma et al., 2007; Rio and Hour-

din, 2008; Neggers, 2009; Neggers et al., 2009; Rio et al., 2010). The

EDMF approach, like the shallow cumulus scheme of Park and Breth-

erton (2009), determines the cumulus-base mass ﬂux from the statis-

tical distribution of boundary layer updraft properties, a conceptual

advance over the ad hoc closure assumptions used in the past. Realistic

treatment of the stable boundary layer remains difﬁcult (Beare et al.,

2006; Cuxart et al., 2006; Svensson and Holtslag, 2009) with implica-

tions for modelling of the diurnal cycle of temperature even under clear

skies (Svensson et al., 2011).

Parameterizations of unresolved orographic and non-orographic gravi-

ty-wave drag (GWD) have seen only a few changes since the AR4 (e.g.,

Richter et al., 2010; Geller et al., 2011). In addition to new formula-

tions, the estimation of the parameters used in the GWD schemes has

recently been advanced through the availability of satellite and ground-

based observations of gravity-wave momentum ﬂuxes, high-resolution

numerical modelling, and focussed process studies (Alexander et al.,

2010). Evidence from the Numerical Weather Prediction community

that important terrain-generated features of the atmospheric circu-

lation are better represented at higher model resolution has recently

been conﬁrmed (Watanabe et al., 2008; Jung et al., 2012).

9.1.3.1.2 Ocean

Ocean components in contemporary climate models generally have

horizontal resolutions that are too coarse to admit mesoscale eddies.

Consequently, such models typically employ some version of the Redi

(Redi, 1982) neutral diffusion and Gent and McWilliams (Gent and

McWilliams, 1990) eddy advection parameterization (see also Gent

et al., 1995; McDougall and McIntosh, 2001). Since the AR4, a focus

has been on how parameterized mesoscale eddy ﬂuxes in the ocean

interior interact with boundary layer turbulence (Gnanadesikan et al.,

2007; Danabasoglu et al., 2008; Ferrari et al., 2008, 2010). Another

focus concerns eddy diffusivity, with many CMIP5 models employing

ﬂow-dependent schemes. Both of these reﬁnements are important for

the mean state and the response to changing forcing, especially in

the Southern Ocean (Hallberg and Gnanadesikan, 2006; Boning et al.,

2008; Farneti et al., 2010; Farneti and Gent, 2011; Gent and Danabaso-

glu, 2011; Hofmann and Morales Maqueda, 2011).

In addition to mesoscale eddies, there has been a growing awareness

of the role that sub-mesoscale eddies and fronts play in restratifying

the mixed layer (Boccaletti et al., 2007; Fox-Kemper et al., 2008; Klein

and Lapeyre, 2009), and the parameterization of Fox-Kemper et al.

(2011) is now used in some CMIP5 models.

There is an active research effort on the representation of dianeutral

mixing associated with breaking gravity waves (MacKinnon et al.,

2009), with this work adding rigour to the prototype energetically con-

sistent abyssal tidal mixing parameterization of Simmons et al. (2004)

now used in several climate models (e.g., Jayne, 2009; Danabasoglu et

al., 2012). The transport of dense water down-slope by gravity currents

(e.g., Legg et al., 2008, 2009) has also been the subject of focussed

efforts, with associated parameterizations making their way into some

CMIP5 models (Jackson et al., 2008b; Legg et al., 2009; Danabasoglu

et al., 2010).

9.1.3.1.3 Land

Land surface properties such as vegetation, soil type and the amount

of water stored on the land as soil moisture, snow and groundwa-

ter all strongly inﬂuence climate, particularly through their effects on

surface albedo and evapotranspiration. These climatic effects can be

profound; for example, it has been suggested that changes in the state

of the land surface may have played an important part in the severity

and length of the 2003 European drought (Fischer et al., 2007), and

Box 9.1 (continued)

available. However, it is clear that tuning involves trade-offs; this keeps the number of constraints that can be used small and usually

focuses on global mean measures related to budgets of energy, mass and momentum. It has been shown for at least one model that

the tuning process does not necessarily lead to a single, unique set of parameters for a given model, but that different combinations

of parameters can yield equally plausible models (Mauritsen et al., 2012). Hence the need for model tuning may increase model uncer-

tainty. There have been recent efforts to develop systematic parameter optimization methods, but owing to model complexity they

cannot yet be applied to fully coupled climate models (Neelin et al., 2010).

Model tuning directly inﬂuences the evaluation of climate models, as the quantities that are tuned cannot be used in model evalua-

tion. Quantities closely related to those tuned will provide only weak tests of model performance. Nonetheless, by focusing on those

quantities not generally involved in model tuning while discounting metrics clearly related to it, it is possible to gain insight into model

performance. Model quality is tested most rigorously through the concurrent use of many model quantities, evaluation techniques, and

performance metrics that together cover a wide range of emergent (or un-tuned) model behaviour.

The requirement for model tuning raises the question of whether climate models are reliable for future climate projections. Models are

not tuned to match a particular future; they are tuned to reproduce a small subset of global mean observationally based constraints.

What emerges is that the models that plausibly reproduce the past, universally display signiﬁcant warming under increasing green-

house gas concentrations, consistent with our physical understanding.

751

Evaluation of Climate Models Chapter 9

that more than 60% of the projected increase in interannual summer

temperature variability in Europe is due to soil moisture–temperature

feedbacks (Seneviratne et al., 2006).

Land surface schemes calculate the ﬂuxes of heat, water and momen-

tum between the land and the atmosphere. At the time of the AR4,

even the more advanced land surface schemes suffered from obvious

simpliﬁcations, such as the need to prescribe rather than simulate the

vegetation cover and a tendency to ignore lateral ﬂows of water and

sub-gridscale heterogeneity in soil moisture (Pitman, 2003). Since the

AR4, a number of climate models have included some representation

of vegetation dynamics (see Sections 9.1.3.2.5 and 9.4.4.3), land–

atmosphere CO

exchange (see Section 9.4.5), sub-gridscale hydrology

(Oleson et al., 2008b) and changes in land use (see Section 9.4.4.4).

9.1.3.1.4 Sea ice

Most large-scale sea ice processes, such as basic thermodynamics and

dynamics, are well understood and well represented in models (Hunke

et al., 2010). However, important details of sea ice dynamics and defor-

mation are not captured, especially at small scales (Coon et al., 2007;

Girard et al., 2009; Hutchings et al., 2011). Currently, sea ice model

development is focussed mainly on (1) more precise descriptions of

physical processes such as microstructure evolution and anisotropy

and (2) including biological and chemical species. Many models now

include some representation of sub-grid-scale thickness variations,

along with a description of mechanical redistribution that converts

thinner ice to thicker ice under deformation (Hunke et al., 2010).

Sea ice albedo has long been recognized as a critical aspect of the

global heat balance. The average ice surface albedo on the scale of a

climate model grid cell is (as on land) the result of a mixture of surface

types: bare ice, melting ice, snow-covered ice, open water, etc. Many

sea ice models use a relatively simple albedo parameterization that

speciﬁes four albedo values: cold snow; warm, melting snow; cold,

bare ice; and warm, melting ice, and the speciﬁc values may be subject

to tuning (e.g., Losch et al., 2010). Some parameterizations take into

account the ice and snow thickness, spectral band and surface melt

(e.g., Pedersen et al., 2009; Vancoppenolle et al., 2009). Solar radiation

may be distributed within the ice column assuming exponential decay

or via a more complex multiple-scattering radiative transfer scheme

(Briegleb and Light, 2007).

Snow model development for sea ice has lagged behind terrestrial

snow models. Lecomte et al. (2011) introduced vertically varying snow

temperature, density and conductivity to improve vertical heat con-

duction and melting in a 1D model intended for climate simulation,

but many physical processes affecting the evolution of the snow pack,

such as redistribution by wind, moisture transport (including ﬂooding

and snow ice formation) and snow grain size evolution, still are not

included in most climate models.

Salinity affects the thermodynamic properties of sea ice, and is used

in the calculation of fresh water and salt exchanges at the ice–ocean

interface (Hunke et al., 2011). Some models allow the salinity to vary

in time (Schramm et al., 1997), while others assume a salinity proﬁle

that is constant (e.g., Bitz and Lipscomb, 1999). Another new thrust is

the inclusion of chemistry and biogeochemistry (Piot and von Glasow,

2008; Zhao et al., 2008; Vancoppenolle et al., 2010; Hunke et al., 2011),

with dependencies on the ice microstructure and salinity proﬁle.

Melt ponds can drain through interconnected brine channels when

the ice becomes warm and permeable. This ﬂushing can effectively

clean the ice of salt, nutrients, and other inclusions, which affect the

albedo, conductivity and biogeochemical processes and thereby play a

role in climate change. Advanced parameterizations for melt ponds are

making their way into sea ice components of global climate models

(e.g., Flocco et al., 2012; Hunke et al., 2013).

9.1.3.2 New Components and Couplings: Emergence of

Earth System Modelling

9.1.3.2.1 Carbon cycle

The omission of internally consistent feedbacks among the physical,

chemical and biogeochemical processes in the Earth’s climate system

is a limitation of AOGCMs. The conceptual issue is that the physical

climate inﬂuences natural sources and sinks of CO

and methane (CH

the two most important long-lived GHGs. ESMs incorporate many of

the important biogeochemical processes, making it possible to sim-

ulate the evolution of these radiatively active species based on their

emissions from natural and anthropogenic sources together with their

interactions with the rest of the Earth system. Alternatively, when

forced with speciﬁed concentrations, a model can be used to diagnose

these sources with feedbacks included (Hibbard et al., 2007). Given the

large natural sources and sinks of CO

relative to anthropogenic emis-

sions, and given the primacy of CO

among anthropogenic GHGs, some

of the most important enhancements are the addition of terrestrial

and oceanic carbon cycles. These cycles have been incorporated into

many models (Christian et al., 2010; Tjiputra et al., 2010) used to make

projections of climate change (Schurgers et al., 2008; Jungclaus et al.,

2010). Several ESMs now include coupled carbon and nitrogen cycles

(Thornton et al., 2007; Gerber et al., 2010; Zaehle and Dalmonech,

2011) in order to simulate the interactions of nitrogen compounds with

ecosystem productivity, GHGs including nitrous oxide (N

O) and ozone

), and global carbon sequestration (Zaehle and Dalmonech, 2011).

Oceanic uptake of CO

is highly variable in space and time, and is deter-

mined by the interplay between the biogeochemical and physical pro-

cesses in the ocean. About half of CMIP5 models make use of schemes

that partition marine ecosystems into nutrients, plankton, zooplankton

and detritus (hence called NPZD-type models) while others use a more

simpliﬁed representation of ocean biogeochemistry (see Table 9.A.1).

These NPZD-type models allow simulation of some of the important

feedbacks between climate and oceanic CO

uptake, but are limited by

the lack of marine ecosystem dynamics. Some efforts have been made

to include more plankton groups or plankton functional types in the

models (Le Quere et al., 2005) with as-yet uncertain implications for

Earth system response.

Ocean acidiﬁcation and the associated decrease in calciﬁcation in

many marine organisms provides a negative feedback on atmospheric

increase (Ridgwell et al., 2007a). New-generation models there-

fore include various parameterizations of calcium carbonate (CaCO

)

752

Chapter 9 Evaluation of Climate Models

production as a function of the saturation state of seawater with

respect to calcite (Gehlen et al., 2007; Ridgwell et al., 2007a; Ilyina et

al., 2009) or partial pressure CO

(pCO

; Heinze, 2004). On centennial

to multi-millennial scales, deep-sea carbonate sediments neutralize

atmospheric CO

. Some CMIP5 models include the sediment carbon

reservoir, and progress has been made toward reﬁned sediment rep-

resentation in the models (Heinze et al., 2009).

9.1.3.2.2 Aerosol particles

The treatment of aerosol particles has advanced since the AR4. Many

AOGCMs and ESMs now include the basic features of the sulphur

cycle and so represent both the direct effect of sulphate aerosol, along

with some of the more complex indirect effects involving cloud drop-

let number and size. Further, several AOGCMs and ESMs are currently

capable of simulating the mass, number, size distribution and mixing

state of interacting multi-component aerosol particles (Bauer et al.,

2008b; Liu et al., 2012b). The incorporation of more physically com-

plete representations of aerosol often improves the simulated climate

under historical and present-day conditions, including the mean pat-

tern and interannual variability in continental rainfall (Rotstayn et al.,

2010, 2011). However, despite the addition of aerosol–cloud interac-

tions to many AOGCMs and ESMs since the AR4, the representation

of aerosol particles and their interaction with clouds and radiative

transfer remains an important source of uncertainty (see Sections 7.3.5

and 7.4). Additional aerosol-related topics that have received attention

include the connection between dust aerosol and ocean biogeochemis-

try, the production of oceanic dimethylsulphide (DMS, a natural source

of sulphate aerosol), and vegetation interactions with organic atmos-

pheric chemistry (Collins et al., 2011).

9.1.3.2.3 Methane cycle and permafrost

In addition to CO

, an increasing number of ESMs and EMICs are also

incorporating components of the CH

cycle, for example, atmospheric

chemistry and wetland emissions, to quantify some of the feed-

backs from changes in CH

sources and sinks under a warming climate

(Stocker et al., 2012). Some models now simulate the evolution of the

permafrost carbon stock (Khvorostyanov et al., 2008a, 2008b), and in

some cases this is integrated with the representation of terrestrial and

oceanic CH

cycles (Volodin, 2008b; Volodin et al., 2010).

9.1.3.2.4 Dynamic global vegetation models and wildland ﬁres

One of the potentially more signiﬁcant effects of climate change is the

alteration of the distribution, speciation and life cycle of vegetated

ecosystems (Bergengren et al., 2001, 2011). Vegetation has a signiﬁ-

cant inﬂuence on the surface energy balance, exchanges of non-CO

GHGs and the terrestrial carbon sink. Systematic shifts in vegetation, for

example, northward migration of boreal forests, would therefore impose

biogeophysical feedbacks on the physical climate system (Clark et al.,

2011). In order to include these effects in projections of climate change,

several dynamic global vegetation models (DGVMs) have been devel-

oped and deployed in ESMs (Cramer et al., 2001; Sitch et al., 2008; Ostle

et al., 2009). Although agriculture and managed forests are not yet gen-

erally incorporated, DGVMs can simulate the interactions among natu-

ral and anthropogenic drivers of global warming, the state of terrestrial

ecosystems and ecological feedbacks on further climate change. The

incorporation of DGVMs has required considerable improvement in the

physics of coupled models to produce stable and realistic distributions of

ﬂora (Oleson et al., 2008b). The improvements include better treatments

of surface, subsurface and soil hydrological processes; the exchange of

water with the atmosphere; and the discharge of water into rivers and

streams. Whereas the ﬁrst DGVMs have been coupled primarily to the

carbon cycle, the current generation of DGVMs are being extended

to include ecological sources and sinks of other non-CO

trace gases

including CH

, N

O, biogenic volatile organic compounds (BVOCs) and

nitrogen oxides collectively known as NO

(Arneth et al., 2010). BVOCs

and NO

can alter the lifetime of some GHGs and act as precursors

for secondary organic aerosols (SOAs) and ozone. Disturbance of the

natural landscape by ﬁre has signiﬁcant climatic effects through its

impact on vegetation and through its emissions of GHGs, aerosols and

aerosol precursors. Because the frequency of wildland ﬁres increases

rapidly with increases in ambient temperature (Westerling et al., 2006),

the effects of ﬁres are projected to grow over the 21st century (Kloster

et al., 2012). The interactions of ﬁres with the rest of the climate system

are now being introduced into ESMs (Arora and Boer, 2005; Pechony

and Shindell, 2009; Shevliakova et al., 2009).

9.1.3.2.5 Land use/land cover change

The impacts of land use and land cover change on the environment

and climate are explicitly included as part of the Representative Con-

centration Pathways (RCPs; cf. Chapters 1 and 12) used for climate

projections to be assessed in later chapters (Moss et al., 2010). Several

important types of land use and land cover change include effects of

agriculture and changing agricultural practices, including the poten-

tial for widespread introduction of biofuel crops; the management of

forests for preservation, wood harvest and production of woody bio-

fuel stock; and the global trends toward greater urbanization. ESMs

include increasingly detailed treatments of crops and their interaction

with the landscape (Arora and Boer, 2010; Smith et al., 2010a, 2010b),

forest management (Bellassen et al., 2010, 2011) and the interactions

between urban areas and the surrounding climate systems (Oleson et

al., 2008a).

9.1.3.2.6 Chemistry–climate interactions and stratosphere–

troposphere coupling

Important chemistry–climate interactions such as the impact of the

ozone hole and recovery on Southern Hemisphere (SH) climate or the

radiative effects of stratospheric water vapour changes on surface

temperature have been conﬁrmed in multiple studies (SPARC-CCMVal,

2010; WMO, 2011). In the majority of the CMIP5 simulations strato-

spheric ozone is prescribed. The main advance since the AR4 is that

time-varying rather than constant stratospheric ozone is now generally

used. In addition, several CMIP5 models treat stratospheric chemistry

interactively, thus prognostically calculating stratospheric ozone and

other chemical constituents. Important chemistry–climate interactions

such as an increased inﬂux of stratospheric ozone in a warmer climate

that results in higher ozone burdens in the troposphere have also been

identiﬁed (Young et al., 2013). Ten of the CMIP5 models simulate trop-

ospheric chemistry interactively whereas it is prescribed in the remain-

ing models (see Table 9.1 and Eyring et al. (2013)).

753

Evaluation of Climate Models Chapter 9

It is now widely accepted that in addition to the inﬂuence of trop-

ospheric circulation and climate change on the stratosphere, strato-

spheric dynamics can in turn inﬂuence the tropospheric circulation and

its variability (SPARC-CCMVal, 2010; WMO, 2011). As a result, many

climate models now have the ability to include a fully resolved strato-

sphere with a model top above the stratopause, located at around 50

km. The subset of CMIP5 models with high-top conﬁgurations is com-

pared to the set of low-top models with a model top below the strat-

opause in several studies (Charlton-Perez et al., 2012; Hardiman et al.,

2012; Wilcox et al., 2012), although other factors such as differences

in tropospheric warming or ozone could affect the two sub-ensembles.

9.1.3.2.7 Land ice sheets

The rate of melt water release from the Greenland and Antarctic ice

sheets in response to climate change remains a major source of uncer-

tainty in projections of sea level rise (see Sections13.4.3 and13.4.4).

Until recently, the long-term response of these ice sheets to alterations

in the surrounding atmosphere and ocean has been simulated using

ofﬂine models. Several ESMs currently have the capability to have ice

sheet component models coupled to the rest of the climate system

(Driesschaert et al., 2007; Charbit et al., 2008; Vizcaino et al., 2008;

Huybrechts et al., 2011; Robinson et al., 2012) although these capabil-

ities are not exercised for CMIP5.

9.1.3.2.8 Additional features in ocean–atmosphere coupling

Several features in the coupling between the ocean and the atmos-

phere have become more widespread since the AR4. The bulk formulae

used to compute the turbulent ﬂuxes of heat, water and momentum

at the air–sea interface, have been revised. A number of models now

consider the ocean surface current when calculating wind stress (e.g.,

Luo et al., 2005; Jungclaus et al., 2006). The coupling frequency has

been increased in some cases to include the diurnal cycle, which was

shown to reduce the SST bias in the tropical Paciﬁc (Schmidt et al.,

2006; Bernie et al., 2008; Ham et al., 2010). Several models now repre-

sent the coupling between the penetration of the solar radiation into

the ocean and light-absorbing chlorophyll, with some implications on

the representation of the mean climate and climate variability (Murtu-

gudde et al., 2002; Wetzel et al., 2006). This coupling is achieved either

by prescribing the chlorophyll distribution from observations, or by

computing the chlorophyll distribution with an ocean biogeochemical

model (e.g., Arora et al., 2009).

9.1.3.3 Resolution

The typical horizontal resolution (deﬁned here as horizontal grid spac-

ing) for current AOGCMs and ESMs is roughly 1 to 2 degrees for the

atmospheric component and around 1 degree for the ocean (Table 9.1).

The typical number of vertical layers is around 30 to 40 for the atmos-

phere and around 30 to 60 for the ocean (note that some ‘high-top’

models may have 80 or more vertical levels in the atmosphere). There

has been some modest increase in model resolution since the AR4,

especially for the near-term simulations (e.g., around 0.5 degree for

the atmosphere in some cases), based on increased availability of more

powerful computers. For the models used in long-term simulations

with interactive biogeochemistry, the resolution has not increased

substantially due to the trade-off against higher complexity in such

models. Since the AR4, typical regional climate model resolution has

increased from around 50 km to around 25 km (see Section 9.6.2.2),

and the impact of this has been explored with multi-decadal regional

simulations (e.g., Christensen et al., 2010). In some cases, RCMs are

being run at 10 km resolution or higher (e.g., Kanada et al., 2008;

Kusaka et al., 2010; van Roosmalen et al., 2010; Kendon et al., 2012).

Higher resolution can sometimes lead to a stepwise, rather than incre-

mental, improvement in model performance (e.g., Roberts et al., 2004;

Shaffrey et al., 2009). For example, ocean models undergo a transition

from laminar to eddy-permitting when the computational grid contains

more than one or two grid points per ﬁrst baroclinic Rossby radius (i.e.,

ﬁner than 50 km at low latitudes and 10 km at high latitudes) (Smith

et al., 2000; McWilliams, 2008). Such mesoscale eddy-permitting ocean

models better capture the large amount of energy contained in fronts,

boundary currents, and time dependent eddy features (e.g., McClean

et al., 2006). Models run at such resolution have been used for some

climate simulations, though much work remains before they are as

mature as the coarser models currently in use (Bryan et al., 2007; Bryan

et al., 2010; Farneti et al., 2010; McClean et al., 2011; Delworth et al.,

2012).

Similarly, atmospheric models with grids that allow the explicit rep-

resentation of convective cloud systems (i.e., ﬁner than a few kilo-

metres) avoid employing a parameterization of their effects—a long-

standing source of uncertainty in climate models. For example, Kendon

et al. (2012) simulated the climate of the UK region over a 20-year

period at 1.5 km resolution, and demonstrated several improvements

of errors typical of coarser resolution models. Further discussion of this

is provided in Section 7.2.2.

9.2 Techniques for Assessing Model

Performance

Systematic evaluation of models through comparisons with observa-

tions is a prerequisite to applying them conﬁdently. Several signiﬁcant

developments in model evaluation have occurred since the AR4 and

are assessed in this section. This is followed by a description of the

overall approach to evaluation taken in this chapter and a discussion

of its known limitations.

9.2.1 New Developments in Model Evaluation

Approaches

9.2.1.1 Evaluating the Overall Model Results

The most straightforward approach to evaluate models is to compare

simulated quantities (e.g., global distributions of temperature, precip-

itation, radiation etc.) with corresponding observationally based esti-

mates (e.g., Gleckler et al., 2008; Pincus et al., 2008; Reichler and Kim,

2008). A signiﬁcant development since the AR4 is the increased use of

quantitative statistical measures, referred to as performance metrics.

The use of such metrics simpliﬁes synthesis and visualization of model

performance (Gleckler et al., 2008; Pincus et al., 2008; Waugh and

Eyring, 2008; Cadule et al., 2010; Sahany et al., 2012) and enables the

754

Chapter 9 Evaluation of Climate Models

quantitative assessment of model improvements over time (Reichler

and Kim, 2008). Recent work has addressed redundancy of multiple

performance metrics through methods such as cluster analysis (Yokoi

et al., 2011; Nishii et al., 2012).

9.2.1.2 Isolating Processes

To understand the cause of model errors it is necessary to evaluate the

representation of processes both in the context of the full model and

in isolation. A number of evaluation techniques to achieve both pro-

cess and component isolation have been developed. One involves the

so-called ‘regime-oriented’ approach to process evaluation. Instead of

averaging model results in time (e.g., seasonal averages) or space (e.g.,

global averages), results are averaged within categories that describe

physically distinct regimes of the system. Applications of this approach

since the AR4 include the use of circulation regimes (Bellucci et al.,

2010; Brown et al., 2010b; Brient and Bony, 2012; Ichikawa et al.,

2012), cloud regimes (Williams and Brooks, 2008; Chen and Del Genio,

2009; Williams and Webb, 2009; Tsushima et al., 2013) and thermody-

namic states (Sahany et al., 2012; Su, 2012). The application of new

observations, such as vertically resolved cloud and water vapour infor-

mation from satellites (Jiang et al., 2012a; Konsta et al., 2012; Quaas,

2012) and water isotopes (Risi et al., 2012a; Risi et al., 2012b), has also

enhanced the ability to evaluate processes in climate models.

Another approach involves the isolation of model components or

parameterizations in off-line simulations, such as Single Column

Models of the atmosphere. Results of such simulations are compared

to measurements from ﬁeld studies or to results of more detailed pro-

cess models (Randall et al., 2003). Numerous process evaluation data

sets have been collected since the AR4 (Redelsperger et al., 2006; Ill-

ingworth et al., 2007; Verlinde et al., 2007; May et al., 2008; Wood et

al., 2011) and have been applied to the evaluation of climate model

processes (Xie et al., 2008; Boone et al., 2009; Boyle and Klein, 2010;

Hourdin et al., 2010). These studies are crucial to test the realism of the

process formulations that underpin climate models.

9.2.1.3 Instrument Simulators

Satellites provide nearly global coverage, sampling across many mete-

orological conditions. This makes them powerful tools for model eval-

uation. The conventional approach has been to convert satellite-ob-

served radiation information to ‘model-equivalents’ (Stephens and

Kummerow, 2007), and these have been used in numerous studies

(Allan et al., 2007; Gleckler et al., 2008; Li et al., 2008; Pincus et al.,

2008; Waliser et al., 2009b; Li et al., 2011a, 2012a; Jiang et al., 2012a).

A challenge is that limitations of the satellite sensors demand various

assumptions in order to convert a satellite measurement into a ‘model

equivalent’ climate variable.

An alternative approach is to calculate ‘observation-equivalents’ from

models using radiative transfer calculations to simulate what the

satellite would provide if the satellite system were ‘observing’ the

model. This approach is usually referred to as an ‘instrument simula-

tor’. Microphysical assumptions (which differ from model to model)

can be included in the simulators, avoiding inconsistencies. A simulator

for cloud properties from the International Cloud Satellite Climatology

Project (ISCCP) (Yu et al., 1996; Klein and Jakob, 1999; Webb et al.,

2001) has been widely used for model evaluation since the AR4 (Chen

and Del Genio, 2009; Marchand et al., 2009; Wyant et al., 2009; Yoko-

hata et al., 2010), often in conjunction with statistical techniques to

separate model clouds into cloud regimes (e.g., Field et al., 2008; Wil-

liams and Brooks, 2008; Williams and Webb, 2009). New simulators for

other satellite products have also been developed and are increasingly

applied for model evaluation (Bodas-Salcedo et al., 2011). Although

often focussed on clouds and precipitation, the simulator approach has

also been used successfully for other variables such as upper tropo-

spheric humidity (Allan et al., 2003; Iacono et al., 2003; Ringer et al.,

2003; Brogniez et al., 2005; Brogniez and Pierrehumbert, 2007; Zhang

et al., 2008b; Bodas-Salcedo et al., 2011). Although providing an alter-

native to the use of model-equivalents from observations, instrument

simulators have limitations (Pincus et al., 2012) and are best applied in

combination with other model evaluation techniques.

9.2.1.4 Initial Value Techniques

To be able to forecast the weather a few days ahead, knowledge of the

present state of the atmosphere is of primary importance. In contrast,

climate predictions and projections simulate the statistics of weather

seasons to centuries in advance. Despite their differences, both weath-

er predictions and projections of future climate are performed with

very similar atmospheric model components. The atmospheric com-

ponent of climate models can be integrated as a weather prediction

model if initialized appropriately (Phillips et al., 2004). This allows

testing parameterized sub-grid scale processes without the compli-

cation of feedbacks substantially altering the underlying state of the

atmosphere.

The application of these techniques since the AR4 has led to some

new insights. For example, many of the systematic errors in the mod-

elled climate develop within a few days of simulation, highlighting

the important role of fast, parameterized processes (Klein et al., 2006;

Boyle et al., 2008; Xie et al., 2012). Errors in cloud properties for exam-

ple were shown to be present within a few days in a forecast in at least

some models (Williams and Brooks, 2008), although this was not the

case in another model (Boyle and Klein, 2010; Zhang et al., 2010b).

Other studies have highlighted the advantage of such methodologies

for the detailed evaluation of model processes using observations that

are available only for limited locations and times (Williamson and

Olson, 2007; Bodas-Salcedo et al., 2008; Xie et al., 2008; Hannay et al.,

2009; Boyle and Klein, 2010), an approach that is difﬁcult to apply to

long-term climate simulations.

9.2.2 Ensemble Approaches for Model Evaluation

Ensemble methods are used to explore the uncertainty in climate

model simulations that arise from internal variability, boundary con-

ditions, parameter values for a given model structure or structural

uncertainty due to different model formulations (Tebaldi and Knutti,

2007; Hawkins and Sutton, 2009; Knutti et al., 2010a). Since the AR4,

techniques have been designed to speciﬁcally evaluate model per-

formance of individual ensemble members. Although this is typically

done to better characterize uncertainties, the methods and insights are

applicable to model evaluation in general. The ensembles are generally

755

Evaluation of Climate Models Chapter 9

of two types: Multi-model Ensembles (MMEs) and Perturbed Parameter

(or Physics) Ensembles (PPEs).

9.2.2.1 Multi-Model Ensembles

The MME is created from existing model simulations from multiple

climate modelling centres. MMEs sample structural uncertainty and

internal variability. However, the sample size of MMEs is small, and

is confounded because some climate models have been developed

by sharing model components leading to shared biases (Masson and

Knutti, 2011a). Thus, MME members cannot be treated as purely inde-

pendent, which implies a reduction in the effective number of inde-

pendent models (Tebaldi and Knutti, 2007; Jun et al., 2008; Knutti,

2010; Knutti et al., 2010a; Pennell and Reichler, 2011).

9.2.2.2 Perturbed-Parameter Ensembles

In contrast, PPEs are created to assess uncertainty based on a single

model and beneﬁt from the explicit control on parameter perturba-

tions. This allows statistical methods to determine which parameters

are the main drivers of uncertainty across the ensemble (e.g., Rougier

et al., 2009). PPEs have been used frequently in simpler models such

as EMICs (Xiao et al., 1998; Forest et al., 2002, 2006, 2008; Stott and

Forest, 2007; Knutti and Tomassini, 2008; Sokolov et al., 2009; Loutre et

al., 2011) and are now being applied to more complex models (Murphy

et al., 2004; Annan et al., 2005; Stainforth et al., 2005; Collins et al.,

2006a, 2007; Jackson et al., 2008a; Brierley et al., 2010; Klocke et al.,

2011; Lambert et al., 2012). The disadvantage of PPEs is that they do

not explore structural uncertainty, and thus the estimated uncertainty

will depend on the underlying model that is perturbed (Yokohata et al.,

2010) and may be too narrow (Sakaguchi et al., 2012). Several stud-

ies (Sexton et al., 2012; Sanderson, 2013) recognize the importance

of sampling both parametric and structural uncertainty by combining

information from both MMEs and PPEs. However, even these approach-

es cannot account for the effect on uncertainty of systematic errors.

9.2.2.3 Statistical Methods Applied to Ensembles

The most common approach to characterize MME results is to calcu-

late the arithmetic mean of the individual model results, referred to

as an unweighted multi-model mean. This approach of ‘one vote per

model’ gives equal weight to each climate model regardless of (1) how

many simulations each model has contributed, (2) how interdependent

the models are or (3) how well each model has fared in objective eval-

uation. The multi-model mean will be used often in this chapter. Some

climate models share a common lineage and so share common biases

(Frame et al., 2005; Tebaldi and Knutti, 2007; Jun et al., 2008; Knutti,

2010; Knutti et al., 2010a, 2013; Annan and Hargreaves, 2011; Pennell

and Reichler, 2011; Knutti and Sedlácek, 2013). As a result, collections

such as the CMIP5 MME cannot be considered a random sample of

independent models. This complexity creates challenges for how best

to make quantitative inferences of future climate as discussed further

in Chapter 12 (Knutti et al., 2010a; Collins et al., 2012; Stephenson et

al., 2012; Sansom et al., 2013).

Annan and Hargreaves (2010) have proposed a ‘rank histogram’

approach to evaluate model ensembles as a whole, rather than

individual models, by diagnosing whether observations can be con-

sidered statistically indistinguishable from a model ensemble. Studies

based on this approach have suggested that MMEs (CMIP3/5) are ‘reli-

able’ in that they are not too narrow or too dispersive as a sample of

possible models, but existing single-model-based ensembles tend to

be too narrow (Yokohata et al., 2012, 2013). Although initial work has

analysed the current mean climate state, further work is required to

study the relationships between simulation errors and uncertainties in

ensembles of future projections (Collins et al., 2012).

Bayesian methods offer insights into how to account for model inad-

equacies and combine information from several metrics in both MME

and PPE approaches (Sexton and Murphy, 2012; Sexton et al., 2012),

but they are complex. A simpler strategy of screening out some model

variants on the basis of some observational comparison has been used

with some PPEs (Lambert et al., 2012; Shiogama et al., 2012). Edwards

et al. (2011) provided a statistical framework for ‘pre-calibrating’ out

such poor model variants. Screening techniques have also been used

with MMEs (Santer et al., 2009).

Additional Bayesian methods are applied to the MMEs so that past

model performance is combined with prior distributions to estimate

uncertainty from the MME (Furrer et al., 2007; Tebaldi and Knutti, 2007;

Milliff et al., 2011). Similar to Bayesian PPE methods, common biases

can be assessed within the MME to determine effective independence

of the climate models (Knutti et al., 2013) (see Section 12.2.2 for a

discussion of the assumptions in the Bayesian approaches).

9.2.3 The Model Evaluation Approach Used in this

Chapter and Its Limitations

This chapter applies a variety of evaluation techniques ranging from

visual comparison of observations and the multi-model ensemble

and its mean, to application of quantitative performance metrics (see

Section 9.2.2). No individual evaluation technique or performance

measure is considered superior; rather, it is the combined use of many

techniques and measures that provides a comprehensive overview of

model performance.

Although crucial, the evaluation of climate models based on past cli-

mate observations has some important limitations. By necessity, it

is limited to those variables and phenomena for which observations

exist. Table 9.3 provides an overview of the observations used in this

chapter. In many cases, the lack or insufﬁcient quality of long-term

observations, be it a speciﬁc variable, an important processes, or a

particular region (e.g., polar areas, the upper troposphere/lower strat-

osphere (UTLS), and the deep ocean), remains an impediment. In addi-

tion, owing to observational uncertainties and the presence of internal

variability, the observational record against which models are assessed

is ‘imperfect’. These limitations can be reduced, but not entirely elim-

inated, through the use of multiple independent observations of the

same variable as well as the use of model ensembles.

The approach to model evaluation taken in the chapter reﬂects the

need for climate models to represent the observed behaviour of

past climate as a necessary condition to be considered a viable tool

for future projections. This does not, however, provide an answer to

756

Chapter 9 Evaluation of Climate Models

the much more difﬁcult question of determining how well a model

must agree with observations before projections made with it can be

deemed reliable. Since the AR4, there are a few examples of emer-

gent constraints where observations are used to constrain multi-model

Quantity

CMIP5 Output

Variable Name

Observations

(Default/Alternates)

Reference Figure and Section Number(s)

ATMOSPHERE

Surface (2 m) Air

Temperature (°C)

Tas

(2 m)

ERA-Interim

NCEP-NCAR

ERA40

CRU TS 3.10

HadCRUT4

GISTEMP

MLOST

Dee et al. (2011)

Kalnay et al. (1996)

Uppala et al. (2005)

Mitchell and Jones (2005)

Morice et al. (2012)

Hansen et al. (2010)

Vose et al. (2012)

Figures 9.2, 9.3, 9.6

, 9.7

, Section 9.4.1;

Figures 9.38, 9.40, Section 9.6.1

Figures 9.6

, 9.7

, Section 9.4.1

Figure 9.38, Section 9.6.1

Figures 9.38, 9.39, Section 9.6.1

Figure 9.8, Section 9.4.1

Temperature (ºC) Ta

(200, 850 hPa)

ERA-Interim

NCEP-NCAR

Dee et al. (2011)

Kalnay et al. (1996)

Figure 9.9

Section 9.4.

Figure 9.9

Section 9.4.1

Zonal mean

wind ( m s

–1

)

(200, 850 hPa)

ERA-Interim

NCEP-NCAR

Dee et al. (2011)

Kalnay et al. (1996)

Figure 9.7

, Section 9.4.1

Figure 9.7

, Section 9.4.1

Zonal wind

stress ( m s

–1

)

Tauu QuikSCAT satellite measurements

NCEP/NCAR reanalysis

ERA-Interim

Risien and Chelton (2008)

Kalnay et al. (1996)

Dee et al. (2011)

Figures 9.19–9.20, Section 9.4.2

Meridional wind (m s

–1

) Va

(200, 850 hPa)

ERA-Interim

NCEP-NCAR

Dee et al. (2011)

Kalnay et al. (1996)

Figure 9.7

, Section 9.4.1

Figure 9.7

, Section 9.4.1

Geopotential

height (m)

(500 hPa)

ERA-Interim

NCEP-NCAR

Dee et al. (2011)

Kalnay et al. (1996)

Figure 9.7

, Section 9.4.1

Figure 9.7

, Section 9.4.1

TOA reﬂected short-

wave radiation (W m

–2

)

Rsut CERES EBAF 2.6

ERBE

Loeb et al. (2009)

Barkstrom (1984)

Figure 9.9

Section 9.4.1

Figure 9.9

, Section 9.4.1

TOA longwave

radiation (W m

–2

)

Rlut CERES EBAF 2.6

ERBE

Loeb et al. (2009)

Barkstrom (1984)

Figure 9.9

Section 9.4.1

Figure 9.9

, Section 9.4.1

Clear sky TOA short-

wave cloud radiative

effect (W m

–2

)

SW CRE

Derived from CMIP5

rsut and rsutcs

CERES EBAF 2.6

CERES ES-4 ERBE

Loeb et al. (2009)

Barkstrom (1984)

Figures 9.5

, 9.6

, 9.7

, Section 9.4.1

Figure 9.5

, Section 9.4.1

Figure 9.7

, Section 9.4.1

Clear sky TOA long-

wave cloud radiative

effect (W m

–2

)

LW CRE

Derived from CMIP5

rsut and rsutcs

CERES EBAF 2.6

CERES ES-4 ERBE

Loeb et al. (2009)

Figure 9.9

, Section 9.4.1

Figure 9.5

, Section 9.4.1

Total precipitation

(mm day

–1

)

Pr GPCP

CMAP

CRU TS3.10.1

Adler et al. (2003)

Xie and Arkin (1997)

Mitchell and Jones (2005)

Figures 9.4, 9.6

, 9.7

Section 9.4.1;

Figures 9.38, 9.40, Section 9.6.1

Figures 9.6

, 9.7

, Section 9.4.1;

Figures 9.38, 9.40, Section 9.6.1

Figures 9.38, 9.39, Section 9.6.1

ensemble projections. These examples, which are discussed further in

Section 9.8.3, remain part of an area of active and as yet inconclusive

research.

Table 9.3 | Overview of observations that are used to evaluate climate models in this chapter. The quantity and CMIP5 output variable name are given along with references

for the observations. Superscript (1) indicates this observations-based data set is obtained from atmospheric reanalysis. Superscript (D) indicates default reference; superscript (A)

alternate reference.

(continued on next page)

757

Evaluation of Climate Models Chapter 9

Quantity

CMIP5 Output

Variable Name

Observations

(Default/Alternates)

Reference Figure and Section Number(s)

ATMOSPHERE (continued)

Precipitable water PRW RSS V7 SSM/I

ERA-INTERIM

MERRA

JRA-25

Wentz et al. (2007)

Dee et al. (2011)

Rienecker et al. (2011)

Onogi et al. (2007)

Figure 9.9, Section 9.4.1

Lower-tropospheric

temperature

TLT RSS V3.3 MSU/AMSU

UAH V5.4 MSU/AMSU

ERA-INTERIM

MERRA

JRA-25

Mears et al. (2011)

Christy et al. (2007)

Dee et al. (2011)

Rienecker et al. (2011)

Onogi et al. (2007)

Figure 9.9, Section 9.4.1

Snow albedo

feedback (%/K)

tas, rsds, rsus Advanced Very High Resolu-

tion Radiometer (AVHRR), Polar

Pathﬁnder-x (APP-x), all-sky albedo

and ERA40 surface air temperature

Hall and Qu (2006); Fern-

andes et al. (2009)

Figure 9.43, Section 9.8.3

Reconstruction of bio-

climatic variables for

the mid-Holocene

and the Last Glacial

Maximum

Tas, pr,, tcold, twarm,

GDD5, alpha

Bartlein et al. (2010) Figure 9.11, Section 9.4.1

Figure 9.12, Section 9.4.1

OZONE and AEROSOLS

Aerosol optical depth aod MODIS

MISR

Shi et al. (2011)

Zhang and Reid (2010); Ste-

vens and Schwartz (2012)

Figures 9.28, 9.29, Section 9.4.6

Figure 9.29, Section 9.4.6

Total column ozone

(DU)

tro3 Ground-based measurements

NASA TOMS/OMI/SBUV(/2)

merged satellite data

NIWA combined total

column ozone database

Solar Backscatter Ultraviolet

(SBUV, SBUV/2) retrievals

DLR GOME/SCIA/GOME-2

updated from Fioletov et al. (2002)

Stolarski and Frith (2006)

Bodeker et al. (2005)

Updated from Miller et al. (2002)

Loyola et al. (2009); Loyola and

Coldewey-Egbers (2012)

Figure 9.10, Section 9.4.1

CARBON CYCLE

Atmospheric CO

(ppmv)

co2

Masarie and Tans (1995);

Meinshausen et al. (2011)

Figure 9.45, Section 9.8.3

Global Land Carbon

Sink (PgC yr

–1

)

NBP GCP Le Quere et al. (2009) Figure 9.26, 9.27, Section 9.4.5

Global Ocean Carbon

Sink (PgC yr

–1

)

fgCO2 GCP Le Quere et al. (2009) Figure 9.26, 9.27, Section 9.4.5

Regional Land Sinks

(PgC yr

–1

)

NBP JAM Gurney et al. (2003) Figure 9.27, Section 9.4.5

Regional Ocean

Sinks (PgC yr

–1

)

fgCO2 JAM

Gurney et al. (2003);

Takahashi et al. (2009)

Figure 9.27, Section 9.4.5

(continued on next page)

Table 9.3 (continued)

758

Chapter 9 Evaluation of Climate Models

Quantity CMIP5 Output

Variable Name

Observations

(Default/Alternates)

Reference Figure and Section Number(s)

OCEAN

Annual mean

temperature

thetao Levitus et al. (2009) Figure 9.13, Section 9.4.2

Annual mean salinity so Antonov et al. (2010) Figure 9.13, Section 9.4.2

Sea Surface

Temperature

tos HadISST1.1

HadCRU 4

ERA40

Rayner et al. (2003)

Jones et al. (2012)

Uppala et al. (2005)

Figure 9.14, Section 9.4.2

Figure 9.35, Section 9.5.3

Figure 9.36, Section 9.5.3

Global ocean heat

content (0 to 700 m)

OHC Levitus

Ishii

Domingues

Levitus et al. (2009)

Ishii and Kimoto, 2009)

Domingues et al. (2008)

Figure 9.17, Section 9.4.2

Dynamic Sea

surface height

SSH AVISO AVISO Figure 9.16, Section 9.4.2

Meridional heat

transport

hfnorth (1) Using surface and TOA heat ﬂuxes:

NCEP/NCAR

ERA40

Updated NCEP reanalysis

(2) Direct estimates using

WOCE and inverse models

Trenberth and Fasullo (2008)

Large and Yeager (2009)

Kalnay et al. (1996)

Uppala et al. (2005)

Kistler et al. (2001)

Ganachaud and Wunsch (2003)

Figure 9.21, Section 9.4.2

Annual mean tem-

perature and salinity

Palaeoclimate reconstruction

of temperature and salinity

Adkins et al. (2002) Figure 9.18, Section 9.4.2

Total area (km

) of grid

cells where Sea Ice

Area Fraction (%) is

>15%. Boundary of sea

ice where Sea Ice Area

Fraction (%) is >15%

HadISST

NSIDC

NASA

Rayner et al. (2003)

Fetterer et al. (2002)

Comiso and Nishio (2008)

Figure 9.22, Section 9.4.3

Figure 9.23, Section 9.4.3

Figure 9.24, Section 9.4.3

MISC

Total area (km

) of grid

cells where Surface

Snow Area Fraction

(%) is 15% or Surface

Snow Amount (kg

–2

) is >5 kg m

–2

Robinson and Frei (2000) Figure 9.25, Section 9.4.4

3-hour precipitation

ﬁelds

15,000 stations and cor-

rected Ta from COADS (Dai

and Deser, 1999; Dai, 2001

Dai (2006) Figure 9.30, Section 9.5.2

Absolute value of

MJJAS minus NDJFM

precipitation exceeding

375 mm

GPCP

(Adler et al., 2003)

Wang et al. (2011a) Figure 9.32, Section 9.5.2

EXTREMES

Daily maximum and

minimum surface air

temperature ﬁelds (ºC)

Daily precipitation

ﬁelds (mm day

–1

)

for calculating

extremes indices

tas, precip ERA40

ERA-Interim,

NCEP/NCAR Reanalysis 1,

NCEP-DOE, Reanalysis 2

Uppala et al. (2005)

Dee et al. (2011)

Kistler et al. (2001)

Kanamitsu et al. (2002)

Calculation of indices is based

on Sillmann et al. (2013)

Figure 9.37, Section 9.5.4

Temperature extremes

indices based on

station observations

HadEX2 Donat et al. (2013) Figure 9.37, Section 9.5.4

Table 9.3 (continued)

Notes:

This observationally constrained data set is obtained from atmospheric reanalysis.

Default reference.

Alternate reference.

759

Evaluation of Climate Models Chapter 9

9.3 Experimental Strategies in Support of

Climate Model Evaluation

9.3.1 The Role of Model Intercomparisons

Systematic model evaluation requires a coordinated and well-doc-

umented suite of model simulations. Organized Model Intercompar-

ison Projects (MIPs) provide this via standard or benchmark experi-

ments that represent critical tests of a model’s ability to simulate

the observed climate. When modelling centres perform a common

experiment, it offers the possibility to compare their results not just

with observations, but with other models as well. This intercompari-

son enables researchers to explore the range of model behaviours, to

isolate the various strengths and weaknesses of different models in a

controlled setting, and to interpret, through idealized experiments, the

inter-model differences. Benchmark MIP experiments offer a way to

distinguish between errors particular to an individual model and those

that might be more universal and should become priority targets for

model improvement.

9.3.2 Experimental Strategy for Coupled Model

Intercomparison Project Phase 5

9.3.2.1 Experiments Utilized for Model Evaluation

CMIP5 includes a much more comprehensive suite of model experi-

ments than was available in the preceding CMIP3 results assessed in

the AR4 (Meehl et al., 2007). In addition to a better constrained speci-

ﬁcation of historical forcing, the CMIP5 collection also includes initial-

ized decadal-length predictions and long-term experiments using both

ESMs and AOGCMs (Taylor et al., 2012b) (Figure 9.1). The CO

forcing

additional predictions

Initialized in all years from

1960-present

prediction with

2010 Pinatubo-

like eruption

alternative

initialization

strategies

AMIP

30-year hindcast & prediction

ensembles: initialized 1960,

1980 & 2005

10-year hindcast & prediction

ensembles: initialized 1960,

1965, …, 2005

AMIP & historical

ensembles

Control,

AMIP &

historical

RCP4.5,

RCP8.5

E-driven Control

& historical

E-driven

RCP8.5

1%/yr CO

(140 yrs)

abrupt 4xCO

(150 yrs)

fixed SST with 1x &

4x CO

of these experiments is prescribed as a time series of either global

mean concentrations or spatially resolved anthropogenic emissions

(Section 9.3.2.2). The analyses of model performance in this chapter

are based on the concentration-based experiments with the exception

of the evaluation of the carbon cycle (see Section9.4.5).

Most of the model diagnostics are derived from the historical simula-

tions that span the period 1850– 2005. In some cases, these histori-

cal simulations are augmented by results from a scenario run, either

RCP4.5 or RCP8.5 (see Section 9.3.2.2), so as to facilitate comparison

with more recent observations. CMIP5/Paleoclimate Modelling Inter-

comparison Project version 3 (PMIP3) simulations for the mid-Holo-

cene and last glacial maximum are used to evaluate model response

to palaeoclimatic conditions. Historical emissions-driven simulations

are used to evaluate the prognostic carbon cycle. The analysis of global

surface temperature variability is based in part on long pre-industrial

control runs to facilitate calculation of variability on decadal to cen-

tennial time scales. Idealized simulations with 1% per year increases

in CO

are utilized to derive transient climate response. Equilibrium cli-

mate sensitivities are derived using results of specialized experiments,

with fourfold CO

increase, designed speciﬁcally for this purpose.

9.3.2.2 Forcing of the Historical Experiments

Under the protocols adopted for CMIP5 and previous assessments, the

transient climate experiments are conducted in three phases. The ﬁrst

phase covers the start of the modern industrial period through to the

present day, years 1850–2005 (van Vuuren et al., 2011). The second

phase covers the future, 2006–2100, and is described by a collection

of RCPs (Moss et al., 2010). As detailed in Chapter 12, the third phase

is described by a corresponding collection of Extension Concentration

Figure 9.1 | Left: Schematic summary of CMIP5 short-term experiments with tier 1 experiments (yellow background) organized around a central core (pink background). (From

Taylor et al., 2012b, their Figure 2). Right: Schematic summary of CMIP5 long-term experiments with tier 1 experiments (yellow background) and tier 2 experiments (green back-

ground) organized around a central core (pink background). Green font indicates simulations to be performed only by models with carbon cycle representations, and ‘E-driven’

means ‘emission-driven’. Experiments in the upper semicircle either are suitable for comparison with observations or provide projections, whereas those in the lower semicircle are

either idealized or diagnostic in nature, and aim to provide better understanding of the climate system and model behaviour. (From Taylor et al., 2012b, their Figure 3.)

760

Chapter 9 Evaluation of Climate Models

Pathways (Meinshausen et al., 2011). The forcings for the historical

simulations evaluated in this section and are described brieﬂy here

(with more details in Annex II).

In the CMIP3 20th century experiments, the forcings from radiatively

active species other than long-lived GHGs and sulphate aerosols were

left to the discretion of the individual modelling groups (IPCC, 2007).

By contrast, a comprehensive set of historical anthropogenic emissions

and land use and land cover change data have been assembled for

the CMIP5 experiments in order to produce a relatively homogeneous

ensemble of historical simulations with common time series of forcing

agents. Emissions of natural aerosols including soil dust, sea salt and

volcanic species are still left to the discretion of the individual model-

ling groups.

For AOGCMs without chemical and biogeochemical cycles, the forcing

agents are prescribed as a set of concentrations. The concentrations

for GHGs and related compounds include CO

, CH

, N

O, all ﬂuori-

nated gases controlled under the Kyoto Protocol (hydroﬂuorocarbons

(HFCs), perﬂuorocarbons (PFCs), and sulphur hexaﬂuoride (SF

)), and

ozone-depleting substances controlled under the Montreal Proto-

col (chloroﬂuorocarbons (CFCs), hydrochloroﬂuorocarbons (HCFCs),

Halons, carbon tetrachloride (CCl

), methyl bromide (CH

Br), methyl

chloride (CH

Cl)). The concentrations for aerosol species include sul-

phate (SO

), ammonium nitrate (NH

), hydrophobic and hydrophilic

black carbon, hydrophobic and hydrophilic organic carbon, secondary

organic aerosols (SOAs) and four size categories of dust and sea salt.

For ESMs that include chemical and biogeochemical cycles, the forc-

ing agents are prescribed both as a set of concentrations and as a

set of emissions with provisions to separate the forcing by natural

and anthropogenic CO

(Hibbard et al., 2007). The emissions include

time-dependent spatially resolved ﬂuxes of CH

, NO

, CO, NH

, black

and organic carbon, and volatile organic compounds (VOCs). For

models that treat the chemical processes associated with biomass

burning, emissions of additional species such as C

O (acetaldehyde),

OH (ethanol), C

S (dimethylsulphide) and C

O (acetone) are

also prescribed. Historical land use and land cover change is described

in terms of the time-evolving partitioning of land surface area among

cropland, pasture, primary land and secondary (recovering) land,

including the effects of wood harvest and shifting cultivation, as well

as land use changes and transitions from/to urban land (Hurtt et al.,

2009). These emissions data are aggregated from empirical recon-

structions of grassland and forest ﬁres (Schultz et al., 2008; Mieville

et al., 2010); international shipping (Eyring et al., 2010); aviation (Lee

et al., 2009), sulphur (Smith et al., 2011b), black and organic carbon

(Bond et al., 2007); and NO

, CO, CH

and non methane volatile organic

compounds (NMVOCs) (Lamarque et al., 2010) contributed by all other

sectors.

For the natural forcings a recommended monthly averaged total solar

irradiance time series was given, but there was no recommended treat-

ment of volcanic forcing. Both integrated solar irradiance and its spec-

trum were available, but not all CMIP5 models used the spectral data.

The data employed an 1850-2008 reconstruction of the solar cycle and

its secular trend using observations of sunspots and faculae, the 10.7

cm solar irradiance measurements and satellite observations (Frohlich

and Lean, 2004).For volcanic forcing CMIP5 models typically employed

one of two prescribed volcanic aerosol data sets (Sato et al., 1993)

or (Ammann et al., 2003) but at least one ESM employed interactive

aerosol injection (Driscoll et al., 2012). The prescribed data sets did not

incorporate injection from explosive volcanoes after 2000.

9.3.2.3 Relationship of Decadal and Longer-Term Simulations

The CMIP5 archive also includes a new class of decadal-prediction

experiments (Meehl et al., 2009, 2013b) (Figure 9.1). The goal is to

understand the relative roles of forced changes and internal variability

in historical and near-term climate variables, and to assess the predict-

ability that might be realized on decadal time scales. These experiments

comprise two sets of hindcast and prediction ensembles with initial

conditions spanning 1960 through 2005. The set of 10-year ensembles

are initialized starting at 1960 in 1-year increments through the year

2005 while the 30-year ensembles are initialized at 1960, 1980 and

2005. The same physical models are often used for both the short-term

and long-term experiments (Figure 9.1) despite the different initiali-

zation of these two sets of simulations. Results from the short-term

experiments are described in detail in Chapter 11.

9.4 Simulation of Recent and Longer-Term

Records in Global Models

9.4.1 Atmosphere

Many aspects of the atmosphere have been more extensively evaluat-

ed than other climate model components. One reason is the availability

of near-global observationally based data for energy ﬂuxes at the TOA,

cloud cover and cloud condensate, temperature, winds, moisture, ozone

and other important properties. As discussed in Box 2.3, atmospheric

reanalyses have also enabled integrating independent observations in

a physically consistent manner. In this section we use this diversity of

data (see Table 9.3) to evaluate the large-scale atmospheric behaviour.

9.4.1.1 Temperature and Precipitation Spatial Patterns of

the Mean State

Surface temperature is perhaps the most routinely examined quantity

in atmospheric models. Many processes must be adequately represent-

ed in order for a model to realistically capture the observed temper-

ature distribution. The dominant external inﬂuence is incoming solar

radiation, but many aspects of the simulated climate play an important

role in modulating regional temperature such as the presence of clouds

and the complex interactions between the atmosphere and the under-

lying land, ocean, snow, ice and biosphere.

The annual mean surface air temperature (at 2 m) is shown in Figure

9.2(a) for the mean of all available CMIP5 models, and the error, rela-

tive to an observationally constrained reanalysis (ECMWF reanalysis of

the global atmosphere and surface conditions (ERA)-Interim; Dee et al.,

2011) is shown in Figure 9.2(b). In most areas the multi-model mean

agrees with the reanalysis to within 2°C, but there are several loca-

tions where the biases are much larger, particularly at high elevations

over the Himalayas and parts of both Greenland and Antarctica, near

the ice edge in the North Atlantic, and over ocean upwelling regions

761

Evaluation of Climate Models Chapter 9

( )

( ) ( )

( )

off the west coasts of South America and Africa. Averaging the abso-

lute error of the individual CMIP5 models (Figure 9.2c) yields similar

magnitude as the multi-model mean bias (Figure 9.2b), implying that

compensating errors across models is limited. The inconsistency across

the three available global reanalyses (Figure 9.2d) that have assimilat-

ed temperature data at two metres (Onogi et al., 2007; Simmons et al.,

2010) provides an indication of observational uncertainty. Although

the reanalysis inconsistency is smaller than the mean absolute bias in

almost all regions, areas where inconsistency is largest (typically where

observations are sparse) tend to be the same regions where the CMIP5

models show largest mean absolute error.

Seasonal performance of models can be evaluated by examining the

difference between means for December–January–February (DJF) and

June–July–August (JJA). Figures 9.3(a) and (b) show the CMIP5 mean

model seasonal cycle amplitude in surface air temperature (as meas-

ured by the difference between the DJF and JJA and the absolute value

of this difference). The seasonal cycle amplitude is much larger over

land where the thermal inertia is much smaller than over the oceans,

Figure 9.2 | Annual-mean surface (2 m) air temperature (°C) for the period 1980–2005. (a) Multi-model (ensemble) mean constructed with one realization of all available models

used in the CMIP5 historical experiment. (b) Multi-model-mean bias as the difference between the CMIP5 multi-model mean and the climatology from ECMWF reanalysis of the

global atmosphere and surface conditions (ERA)-Interim (Dee et al., 2011); see Table 9.3. (c) Mean absolute model error with respect to the climatology from ERA-Interim. (d) Mean

inconsistency between ERA-Interim, ERA 40-year reanalysis (ERA40) and Japanese 25-year ReAnalysis (JRA-25) products as the mean of the absolute pairwise differences between

those ﬁelds for their common period (1979–2001).

and it is generally larger at higher latitudes as a result of the larger

seasonal amplitude in insolation. Figures 9.3(c) and (d) show the mean

model bias of the seasonal cycle relative to the ERA-Interim reanaly-

sis (Dee et al., 2011). The largest biases correspond to areas of large

seasonal amplitude, notably high latitudes over land, but relatively

large biases are also evident in some lower latitude regions such as

over northern India. Over most land areas the amplitude of the mod-

elled seasonal cycle is larger than observed, whereas over much of the

extratropical oceans the modelled amplitude is too small.

The simulation of precipitation is a more stringent test for models as it

depends heavily on processes that must be parameterized. Challenges

are compounded by the link to surface ﬁelds (topography, coastline,

vegetation) that lead to much greater spatial heterogeneity at regional

scales. Figure 9.4 shows the mean precipitation rate simulated by the

CMIP5 multi-model ensemble, along with measures of error relative to

precipitation analyses from the Global Precipitation Climatology Pro-

ject (Adler et al., 2003). The magnitude of observational uncertainty for

precipitation varies with region, which is why many studies make use

762

Chapter 9 Evaluation of Climate Models

Figure 9.3 | Seasonality (December–January–February minus June–July–August ) of surface (2 m) air temperature (°C) for the period 1980–2005. (a) Multi-model mean, calcu-

lated from one realization of all available CMIP5 models for the historical experiment. (b) Multi-model mean of absolute seasonality. (c) Difference between the multi-model mean

and the ECMWF reanalysis of the global atmosphere and surface conditions (ERA)-Interim seasonality. (d) Difference between the multi-model mean and the ERA-Interim absolute

seasonality.

( )

( ) ( )

( )

of several estimates of precipitation. Known large-scale features are

reproduced by the multi-model mean, such as a maximum precipita-

tion just north of the equator in the central and eastern tropical Paciﬁc,

dry areas over the eastern subtropical ocean basins, and the minimum

rainfall in Northern Africa (Dai, 2006).

While many large-scale fea-

tures of the tropical circulation are reasonably well simulated, there

are persistent biases. These include too low precipitation along the

equator in the Western Paciﬁc associated with ocean–atmosphere

feedbacks maintaining the equatorial cold tongue (Collins et al.,

2010) and excessive precipitation in tropical convergence zones

south of the equator in the Atlantic and the Eastern Paciﬁc (Lin, 2007;

Pincus et al., 2008). Other errors occurring in several models include

an overly zonal orientation of the South-Paciﬁc Convergence Zone

(Brown et al., 2013) as well as an overestimate of the frequency of

occurrence of light rain events (Stephens et al., 2010). Regional-scale

precipitation simulation has strong parameter dependence (Rougi-

er et al., 2009; Chen et al., 2010; Neelin et al., 2010), and in some

models substantial improvements have been shown through increas-

es in resolution (Delworth et al., 2012) and improved representa-

tions of sub-gridscale processes, particularly convection (Neale et al.,

2008)

. Judged by similarity with the spatial pattern of observations,

the overall quality of the simulation of the mean state of precipitation

in the CMIP5 ensemble is slightly better than in the CMIP3 ensemble

(see FAQ 9.1 and Figure 9.6).

In summary, there is high conﬁdence that large-scale patterns of sur-

face temperature are well simulated by the CMIP5 models. In certain

regions this agreement with observations is limited, particularly at

elevations over the Himalayas and parts of both Greenland and Ant-

arctica. The broad-scale features of precipitation as simulated by the

CMIP5 models are in modest agreement with observations, but there

are systematic errors in the Tropics.

9.4.1.2 Atmospheric Moisture, Clouds and Radiation

The global annual mean precipitable water is a measure of the total

moisture content of the atmosphere. For the CMIP3 ensemble, the

values of precipitable water agreed with one another and with multi-

ple estimates from the National Centers for Environmental Prediction/

National Center for Atmospheric Research (NCEP/NCAR) and ECMWF

763

Evaluation of Climate Models Chapter 9

( )

Figure 9.4 | Annual-mean precipitation rate (mm day

–1

) for the period 1980–2005. (a) Multi-model-mean constructed with one realization of all available AOGCMs used in the

CMIP5 historical experiment. (b) Difference between multi-model mean and precipitation analyses from the Global Precipitation Climatology Project (Adler et al., 2003). (c) Multi-

model-mean absolute error with respect to observations. (d) Multi-model-mean error relative to the multi-model-mean precipitation itself.

ERA40 meteorological reanalyses to within approximately 10% (Walis-

er et al., 2007). Initial analysis of the CMIP5 ensemble shows the model

results are within the uncertainties of the observations (Jiang et al.,

2012a).

Modelling the vertical structure of water vapour is subject to great-

er uncertainty since the humidity proﬁle is governed by a variety of

processes. The CMIP3 models exhibited a signiﬁcant dry bias of up to

25% in the boundary layer and a signiﬁcant moist bias in the free

troposphere of up to 100% (John and Soden, 2007). Upper tropospher-

ic water vapour varied by a factor of three across the multi-model

ensemble (Su et al., 2006). Many models have large biases in lower

stratospheric water vapour (Gettelman et al., 2010), which could have

implications for surface temperature change (Solomon et al., 2010).

The limited number of studies available for the CMIP5 model ensem-

ble broadly conﬁrms the results from the earlier model generation. In

tropical regions, the models are too dry in the lower troposphere and

too moist in the upper troposphere, whereas in the extratropics they

are too moist throughout the troposphere (Tian et al., 2013). However,

many of the model values lie within the observational uncertainties.

Jiang et al. (2012a) show that the largest biases occur in the upper

troposphere, with model values up to twice that observed, while in the

middle and lower troposphere models simulate water vapour to within

10% of the observations.

The spatial patterns and seasonal cycle of the radiative ﬂuxes at the

TOA are fundamental energy balance quantities. Both the CMIP3 and

CMIP5 model ensembles reproduce these patterns with considerable

ﬁdelity relative to the National Aeronautics and Space Adminsitration

(NASA) Clouds and the Earth’s Radiant Energy System (CERES) data

sets (Pincus et al., 2008; Wang and Su, 2013). Globally averaged TOA

shortwave and longwave components of the radiative ﬂuxes in 12

atmosphere-only versions of the CMIP5 models were within 2.5 W m

–2

of the observed values (Wang and Su, 2013).

Comparisons against surface components of radiative ﬂuxes show

that, on average, the CMIP5 models overestimate the global mean

downward all-sky shortwave ﬂux at the surface by 2 ± 6 W m

–2

(1 ±

3%) and underestimate the global downward longwave ﬂux by 6 ± 9

W m

–2

(2 ± 2%) (Stephens et al., 2012). Although in tropical regions

764

Chapter 9 Evaluation of Climate Models

( )

Figure 9.5 | Annual-mean cloud radiative effects of the CMIP5 models compared against the Clouds and the Earth’s Radiant Energy System Energy Balanced and Filled 2.6 (CERES

EBAF 2.6) data set (in W m

–2

; top row: shortwave effect; middle row: longwave effect; bottom row: net effect). On the left are the global distributions of the multi-model-mean

biases, and on the right are the zonal averages of the cloud radiative effects from observations (solid black: CERES EBAF 2.6; dashed black: CERES ES-4), individual models (thin

grey lines), and the multi-model mean (thick red line). Model results are for the period 1985–2005, while the available CERES data are for 2001–2011. For a deﬁnition and maps

of cloud radiative effect, see Section 7.2.1.2 and Figure 7.7.

between 1 and 3 W m

–2

of the bias may be due to systematic omission

of precipitating and/or convective core ice hydrometeors (Waliser et al.,

2011), the correlation between the biases in the all-sky and clear-sky

downwelling ﬂuxes suggests that systematic errors in clear-sky radia-

tive transfer calculations may be a primary cause for these biases. This

is consistent with an analysis of the global annual mean estimates

of clear-sky atmospheric absorption from the CMIP3 ensemble and

the systematic underestimation of clear-sky solar absorption by radi-

ative transfer codes (Oreopoulos et al., 2012). The underestimation of

absorption can be attributed to the omission or underestimation of

765

Evaluation of Climate Models Chapter 9

Figure 9.6 | Centred pattern correlations between models and observations for the annual mean climatology over the period 1980–1999. Results are shown for individual CMIP3

(black) and CMIP5 (blue) models as thin dashes, along with the corresponding ensemble average (thick dash) and median (open circle). The four variables shown are surface air

temperature (TAS), top of the atmosphere (TOA) outgoing longwave radiation (RLUT), precipitation (PR) and TOA shortwave cloud radiative effect (SW CRE). The observations used

for each variable are the default products and climatological periods identiﬁed in Table 9.3. The correlations between the default and alternate (Table 9.3) observations are also

shown (solid green circles). To ensure a fair comparison across a range of model resolutions, the pattern correlations are computed at a resolution of 4º in longitude and 5º in

latitude. Only one realization is used from each model from the CMIP3 20C3M and CMIP5 historical simulations.

TAS

RLUT PR

SW CRE

0.5

0.6

0.7

0.8

0.9

Correlation

CMIP3

OBS

CMIP5

absorbing aerosols, in particular carbonaceous species (Kim and Ram-

anathan, 2008), or to the omission of weak-line (Collins et al., 2006b)

or continuum (Ptashnik et al., 2011) absorption by water vapour (Wild

et al., 2006).

One of the major inﬂuences on radiative ﬂuxes in the atmosphere is

the presence of clouds and their radiative properties. To measure the

inﬂuence of clouds on model deﬁciencies in the TOA radiation budget,

Figure 9.5 shows maps of deviations from observations in annual mean

shortwave (top left), longwave (middle left) and net (bottom left) cloud

radiative effect (CRE) for the CMIP5 multi-model mean. The ﬁgure (right

panels) also shows zonal averages of the same quantities from two

sets of observations, the individual CMIP5 models, and the multi-model

average. The deﬁnition of CRE and observed mean ﬁelds for these quan-

tities can be found in Chapter 7 (Section 7.2.1.2, Figure 7.7).

Models show large regional biases in CRE in the shortwave component,

and these are particularly pronounced in the subtropics with too weak

an effect (positive error) of model clouds on shortwave radiation in

the stratocumulus regions and too strong an effect (negative error) in

the trade cumulus regions. This error has been shown to largely result

from an overestimation of cloud reﬂectance, rather than cloud cover

(Nam et al., 2012). A too weak cloud inﬂuence on shortwave radia-

tion is evident over the subpolar oceans of both hemispheres and the

Northern Hemisphere (NH) land areas. It is evident in the zonal mean

graphs that there is a wide range in both longwave and shortwave CRE

between individual models. As is also evident, a signiﬁcant reduction

in the difference between models and observations has resulted from

changes in the observational estimates of CRE, in particular at polar

and subpolar as well as subtropical latitudes (Loeb et al., 2009).

Understanding the biases in CRE in models requires a more in-depth

analysis of the biases in cloud properties, including the fractional cov-

erage of clouds, their vertical distribution as well as their liquid water

and ice content. Major progress in this area has resulted from both the

availability of new observational data sets and improved diagnostic

techniques, including the increased use of instrument simulators (e.g.,

Cesana and Chepfer, 2012; Jiang et al., 2012a). Many models have

particular difﬁculties simulating upper tropospheric clouds (Jiang et al.,

2012a), and low and mid-level cloud occurrence are frequently under-

estimated (Cesana and Chepfer, 2012; Nam et al., 2012; Tsushima et

al., 2013). Global mean values of both simulated ice and liquid water

path vary by factors of 2 to 10 between models (Jiang et al., 2012a; Li

et al., 2012a). The global mean fraction of clouds that can be detected

with conﬁdence from satellites (optical thickness >1.3, Pincus et al.

(2012)) is underestimated by 5 to 10 % (Klein et al., 2013). Some of the

above errors in clouds compensate to provide the global mean balance

in radiation required by model tuning (Tsushima et al., 2013; Wang and

Su, 2013; Box 9.1).

In-depth analysis of several global and regional models (Karlsson et al.,

2008; Teixeira et al., 2011) has shown that the interaction of boundary

layer and cloud processes with the larger scale circulation systems that

ultimately drive the observed subtropical cloud distribution remains

poorly simulated. Large errors in subtropical clouds have been shown

to negatively affect SST patterns in coupled model simulations (Hu

766

Chapter 9 Evaluation of Climate Models

et al., 2011; Wahl et al., 2011). Several studies have highlighted the

potential importance and poor simulation of subpolar clouds in the

Arctic and Southern Oceans (Karlsson and Svensson, 2010; Trenberth

and Fasullo, 2010b; Haynes et al., 2011; Bodas-Salcedo et al., 2012). A

particular challenge for models is the simulation of the correct phase

of the cloud condensate, although very few observations are available

to evaluate models particularly with respect to their representation

of cloud ice (Waliser et al., 2009b; Li et al., 2012a). Regime-oriented

approaches to the evaluation of model clouds (see Section 9.2.1) have

identiﬁed that compensating errors in the CRE are largely a result of

misrepresentations of the frequency of occurrence of key observed

cloud regimes, while the radiative properties of the individual regimes

contribute less to the overall model deﬁciencies (Tsushima et al., 2013).

Several studies have identiﬁed progress in the simulation of clouds in

the CMIP5 models compared to their CMIP3 counterparts. Particular

examples include the improved simulation of vertically integrated ice

water path (Jiang et al., 2012a; Li et al., 2012a) as well as a reduction

of overabundant optically thick clouds in the mid-latitudes (Klein et al.,

2013; Tsushima et al., 2013).



























































 



!" 



#! $

#!

#! 

#! 

 

 

%

% 

% 

% 



&!!

&!

&!!





'



&!

&&



()

*'&

 '&

*'&

 '&

+*'&

+ '&

,'&



&

!





!



























































 



!" 



#! $

#!

#! 

#! 

 

 

%

% 

% 

% 



&!!

&!

&!!





'



&!

&&



()

*'&

 '&

*'&

 '&

+*'&

+ '&

,'&



&

!





!

Figure 9.7 | Relative error measures of CMIP5 model performance, based on the global seasonal-cycle climatology (1980–2005) computed from the historical experiments. Rows

and columns represent individual variables and models, respectively. The error measure is a space–time root-mean-square error (RMSE), which, treating each variable separately, is

portrayed as a relative error by normalizing the result by the median error of all model results (Gleckler et al., 2008). For example, a value of 0.20 indicates that a model’s RMSE is

20% larger than the median CMIP5 error for that variable, whereas a value of –0.20 means the error is 20% smaller than the median error. No colour (white) indicates that model

results are currently unavailable. A diagonal split of a grid square shows the relative error withrespect to both the default reference data set (upper left triangle) and the alternate

(lower right triangle). The relative errors are calculated independently for the default and alternate data sets. All reference data used in the diagram are summarized in Table 9.3.

In summary, despite modest improvements there remain signiﬁcant

errors in the model simulation of clouds. There is very high conﬁdence

that these errors contribute signiﬁcantly to the uncertainties in esti-

mates of cloud feedbacks (see Section 9.7.2.3; Section 7.2.5, Figure

7.10) and hence the spread in climate change projections reported in

Chapter 12.

9.4.1.3 Quantifying Model Performance with Metrics

Performance metrics were used to some extent in the Third Assessment

Report (TAR) and the Fourth Assessment Report (AR4), and are expand-

ed upon here because of their increased appearance in the recent lit-

erature. As a simple example, Figure 9.6 illustrates how the pattern

correlation between the observed and simulated climatological annual

mean spatial patterns depends very much on the quantity examined.

All CMIP3 and CMIP5 models capture the mean surface temperature

distribution quite well, with correlations above 0.95, which are large-

ly determined by the meridional temperature gradient. Correlations

for outgoing longwave radiation are somewhat lower. For precipita-

tion and the TOA shortwave cloud radiative effect, the correlations

767

Evaluation of Climate Models Chapter 9

between models and observations are below 0.90, and there is con-

siderable scatter among model results. This example quantiﬁes how

some aspects of the simulated large-scale climate agree with observa-

tions better than others. Some of these differences are attributable to

smoothly varying ﬁelds (e.g., temperature, water vapour) often agree-

ing better with observations than ﬁelds that exhibit ﬁne structure (e.g.,

precipitation) (see also Section 9.6.1.1). Incremental improvement in

each ﬁeld is also evident in Figure 9.6, as gauged by the mean and

median results in the CMIP5 ensemble having higher correlations than

CMIP3. This multi-variate quantiﬁcation of model improvement across

development cycles is evident in several studies (e.g., Reichler and Kim,

2008; Knutti et al., 2013)

Figure 9.7 (following Gleckler et al., 2008) depicts the space–time root-

mean-square error (RMSE) for the 1980–2005 climatological season-

al cycle of the historically forced CMIP5 simulations. For each of the

ﬁelds examined, this ‘portrait plot’ depicts relative performance, with

blue shading indicating performance being better, and red shading

worse, than the median of all model results. In each case, two obser-

vations-based estimates are used to demonstrate the impact of the

selection of reference data on the results. Some models consistently

compare better with observations than others, some exhibit mixed

performance and some stand out with relatively poor agreement with

observations. For most ﬁelds, the choice of the observational data set

does not substantially change the result for global error measures (e.g.,

between a state-of-the-art and an older-generation reanalysis), indi-

cating that inter-model differences are substantially larger than the

differences between the two reference data sets or the impact of two

different climatological periods (e.g., for radiation ﬁelds: Earth Radia-

tion Budget Experiment (ERBE) 1984–1988; CERES EBAF, 2001–2011).

Nevertheless, it is important to recognize that different data sets often

rely on the same source of measurements, and that the results in this

ﬁgure can have some sensitivity to a variety of factors such as instru-

ment uncertainty, sampling errors (e.g., limited record length of obser-

vations), the spatial scale of comparison, the domain considered and

the choice of metric.

Another notable feature of Figure 9.7 is that in most cases the mul-

ti-model mean agrees more favourably with observations than any

individual model. This has been long recognized to hold for surface

temperature and precipitation (e.g., Lambert and Boer, 2001). However,

since the AR4, it has become clear that this holds for a broad range of

climatological ﬁelds (Gleckler et al., 2008; Pincus et al., 2008; Knutti

et al., 2010a) and is theoretically better understood (Annan and Harg-

reaves, 2011). It is worth noting that when most models suffer from a

common error, such as the cold bias at high latitudes in the upper trop-

osphere (see TA 200 hPa of Figure 9.7), individual models can agree

better with observations than the multi-model mean.

Correlations between the relative errors for different quantities in

Figure 9.7 are known to exist, reﬂecting physical relationships in the

model formulations and in the real world. Cluster analysis methods

have recently been used in an attempt to reduce this redundancy (e.g.,

Yokoi et al., 2011; Nishii et al., 2012), thereby providing more succinct

summaries of model performance. Some studies have attempted an

overall skill score by averaging together the results from multiple met-

rics (e.g., Reichler and Kim, 2008). Although this averaging process is

largely arbitrary, combining the results of multiple metrics can reduce

the chance that a poorer performing model will score well for the

wrong reasons. Recent work (Nishii et al., 2012) has demonstrated that

different methods used to produce a multi-variate skill measure for the

CMIP3 models did not substantially alter the conclusions about the

better and lesser performing models.

Large scale performance metrics are a typical ﬁrst-step toward quan-

tifying model agreement with observations, and summarizing broad

characteristics of model performance that are not focussed on a par-

ticular application. More specialized performance tests target aspects

of a simulation believed to be especially important for constraining

model projections, although to date the connections between particu-

lar performance metrics and reliability of future projections are not

well established. This important topic is addressed in Section 9.8.3,

which highlights several identiﬁed relationships between model per-

formance and projection responses.

9.4.1.4 Long-Term Global-Scale Changes

The comparison of observed and simulated climate change is compli-

cated by the fact that the simulation results depend on both model

formulation and the time-varying external forcings imposed on the

models (Allen et al., 2000; Santer et al., 2007). De-convolving the

importance of model and forcing differences in the historical simula-

tions is an important topic that is addressed in Chapter 10; however,

in this section a direct comparison is made to illustrate the ability of

models to reproduce past changes.

9.4.1.4.1 Global surface temperature

Figure 9.8 compares the observational record of 20th century changes

in global surface temperature to that simulated by each CMIP5 and

EMIC model and the respective multi-model means. The inset on the

right of the ﬁgure shows the climatological mean temperature for each

model, averaged over the 1961–1990 reference period. Although biases

in mean temperature are apparent, there is less conﬁdence in observa-

tional estimates of climatological temperature than in variations about

this mean (Jones et al. (1999). For the CMIP5 models, interannual varia-

bility in most of the simulations is qualitatively similar to that observed

although there are several exceptions. The magnitude of interannual

variations in the observations is noticeably larger than the multi-mod-

el mean because the averaging of multiple model results acts to ﬁlter

much of the simulated variability. On the other hand, the episodic

volcanic forcing that is applied to most models (see Section 9.3.2.2)

is evident in the multi-model agreement with the observed cooling

particularly noticeable after the 1991 Pinatubo eruption. The gradual

warming evident in the observational record, particularly in the more

recent decades, is also evident in the simulations, with the multi-model

mean tracking the observed value closely over most of the century, and

individual model results departing by less than about 0.5

C. Because

the interpretation of differences in model behaviour can be confounded

by internal variability and forcing, some studies have attempted to iden-

tify and remove dominant factors such as El Niño-Southern Oscillation

(ENSO) and the impacts of volcanic eruptions (e.g., Fyfe et al., 2010).

Figure 9.8 shows the similar capability for EMICs to simulate the glob-

al-scale response to the 20th century forcings (Eby et al. 2013). These

768

Chapter 9 Evaluation of Climate Models

Figure 9.8 | Observed and simulated time series of the anomalies in annual and global mean surface temperature. All anomalies are differences from the 1961–1990 time-mean

of each individual time series. The reference period 1961–1990 is indicated by yellow shading; vertical dashed grey lines represent times of major volcanic eruptions. (a) Single

simulations for CMIP5 models (thin lines); multi-model mean (thick red line); different observations (thick black lines). Observational data (see Chapter 2) are Hadley Centre/Climatic

Research Unit gridded surface temperature data set 4 (HadCRUT4; Morice et al., 2012), Goddard Institute for Space Studies Surface Temperature Analysis (GISTEMP; Hansen et

al., 2010) and Merged Land–Ocean Surface Temperature Analysis (MLOST; Vose et al., 2012) and are merged surface temperature (2 m height over land and surface temperature

over the ocean). All model results have been sub-sampled using the HadCRUT4 observational data mask (see Chapter 10). Following the CMIP5 protocol (Taylor et al., 2012b), all

simulations use speciﬁed historical forcings up to and including 2005 and use RCP4.5 after 2005 (see Figure 10.1 and note different reference period used there; results will differ

slightly when using alternative RCP scenarios for the post-2005 period). (a) Inset: the global mean surface temperature for the reference period 1961–1990, for each individual

model (colours), the CMIP5 multi-model mean (thick red), and the observations (thick black: Jones et al., 1999). (Bottom) Single simulations from available EMIC simulations (thin

lines), from Eby et al. (2013). Observational data are the same as in (a). All EMIC simulations ended in 2005 and use the CMIP5 historical forcing scenario. (b) Inset: Same as in (a)

but for the EMICs.

results demonstrate a level of consistency between the EMICs with both

the observations and the CMIP5 ensemble.

In summary, there is very high conﬁdence that models reproduce the

general features of the global-scale annual mean surface temperature

increase over the historical period, including the more rapid warming

in the second half of the 20th century, and the cooling immediately

following large volcanic eruptions. The disagreement apparent over the

most recent 10 to 15 years is discussed in detail in Box 9.2.

769

Evaluation of Climate Models Chapter 9

Box 9.2 | Climate Models and the Hiatus in Global Mean Surface Warming of the Past 15 Years

The observed global mean surface temperature (GMST) has shown a much smaller increasing linear trend over the past 15 years than

over the past 30 to 60 years (Section 2.4.3, Figure 2.20, Table 2.7; Figure 9.8; Box 9.2 Figure 1a, c). Depending on the observational

data set, the GMST trend over 1998–2012 is estimated to be around one-third to one-half of the trend over 1951–2012 (Section 2.4.3,

Table 2.7; Box 9.2 Figure 1a, c). For example, in HadCRUT4 the trend is 0.04ºC per decade over 1998–2012, compared to 0.11ºC per

decade over 1951–2012. The reduction in observed GMST trend is most marked in Northern Hemisphere winter (Section 2.4.3; Cohen

et al., 2012). Even with this “hiatus” in GMST trend, the decade of the 2000s has been the warmest in the instrumental record of GMST

(Section 2.4.3, Figure 2.19). Nevertheless, the occurrence of the hiatus in GMST trend during the past 15 years raises the two related

questions of what has caused it and whether climate models are able to reproduce it.

Figure 9.8 demonstrates that 15-year-long hiatus periods are common in both the observed and CMIP5 historical GMST time series

(see also Section 2.4.3, Figure 2.20; Easterling and Wehner, 2009; Liebmann et al., 2010). However, an analysis of the full suite of

CMIP5 historical simulations (augmented for the period 2006–2012 by RCP4.5 simulations, Section 9.3.2) reveals that 111 out of

114 realizations show a GMST trend over 1998–2012 that is higher than the entire HadCRUT4 trend ensemble (Box 9.2 Figure 1a;

CMIP5 ensemble mean trend is 0.21ºC per decade). This difference between simulated and observed trends could be caused by some

combination of (a) internal climate variability, (b) missing or incorrect radiative forcing and (c) model response error. These potential

sources of the difference, which are not mutually exclusive, are assessed below, as is the cause of the observed GMST trend hiatus.

Internal Climate Variability

Hiatus periods of 10 to 15 years can arise as a manifestation of internal decadal climate variability, which sometimes enhances and

sometimes counteracts the long-term externally forced trend. Internal variability thus diminishes the relevance of trends over periods

as short as 10 to 15 years for long-term climate change (Box 2.2, Section 2.4.3). Furthermore, the timing of internal decadal climate

variability is not expected to be matched by the CMIP5 historical simulations, owing to the predictability horizon of at most 10 to 20

years (Section 11.2.2; CMIP5 historical simulations are typically started around nominally 1850 from a control run). However, climate

models exhibit individual decades of GMST trend hiatus even during a prolonged phase of energy uptake of the climate system (e.g.,

Figure 9.8; Easterling and Wehner, 2009; Knight et al., 2009), in which case the energy budget would be balanced by increasing

subsurface–ocean heat uptake (Meehl et al., 2011, 2013a; Guemas et al., 2013).

Owing to sampling limitations, it is uncertain whether an increase in the rate of subsurface–ocean heat uptake occurred during the

past 15 years (Section 3.2.4). However, it is very likely

that the climate system, including the ocean below 700 m depth, has continued

to accumulate energy over the period 1998–2010 (Section 3.2.4, Box 3.1). Consistent with this energy accumulation, global mean sea

level has continued to rise during 1998–2012, at a rate only slightly and insigniﬁcantly lower than during 1993–2012 (Section 3.7). The

consistency between observed heat-content and sea level changes yields high conﬁdence in the assessment of continued ocean energy

accumulation, which is in turn consistent with the positive radiative imbalance of the climate system (Section 8.5.1; Section 13.3, Box

13.1). By contrast, there is limited evidence that the hiatus in GMST trend has been accompanied by a slower rate of increase in ocean

heat content over the depth range 0 to 700 m, when comparing the period 2003–2010 against 1971–2010. There is low agreement on

this slowdown, since three of ﬁve analyses show a slowdown in the rate of increase while the other two show the increase continuing

unabated (Section 3.2.3, Figure 3.2).

During the 15-year period beginning in 1998, the ensemble of HadCRUT4 GMST trends lies below almost all model-simulated trends

(Box 9.2 Figure 1a), whereas during the 15-year period ending in 1998, it lies above 93 out of 114 modelled trends (Box 9.2 Figure

1b; HadCRUT4 ensemble-mean trend 0.26°C per decade, CMIP5 ensemble-mean trend 0.16°C per decade). Over the 62-year period

1951–2012, observed and CMIP5 ensemble-mean trends agree to within 0.02ºC per decade (Box 9.2 Figure 1c; CMIP5 ensemble-mean

trend 0.13°C per decade). There is hence very high conﬁdence that the CMIP5 models show long-term GMST trends consistent with

observations, despite the disagreement over the most recent 15-year period. Due to internal climate variability, in any given 15-year

period the observed GMST trend sometimes lies near one end of a model ensemble (Box 9.2, Figure 1a, b; Easterling and Wehner, 2009),

an effect that is pronounced in Box 9.2, Figure 1a, b because GMST was inﬂuenced by a very strong El Niño event in 1998.

(continued on next page)

In this Report, the following terms have been used to indicate the assessed likelihood of an outcome or a result: Virtually certain 99–100% probability, Very likely

90–100%, Likely 66–100%, About as likely as not 33–66%, Unlikely 0–33%, Very unlikely 0–10%, Exceptionally unlikely 0–1%. Additional terms (Extremely likely:

95–100%, More likely than not >50–100%, and Extremely unlikely 0–5%) may also be used when appropriate. Assessed likelihood is typeset in italics, e.g., very

likely (see Section 1.4 and Box TS.1 for more details).

770

Chapter 9 Evaluation of Climate Models

Box 9.2 (continued)

Unlike the CMIP5 historical simulations referred to above, some CMIP5 predictions were initialized from the observed climate state

during the late 1990s and the early 21st century (Section 11.1, Box 11.1; Section 11.2). There is medium evidence that these initialized

predictions show a GMST lower by about 0.05ºC to 0.1ºC compared to the historical (uninitialized) simulations and maintain this lower

GMST during the ﬁrst few years of the simulation (Section 11.2.3.4, Figure 11.3 top left; Doblas-Reyes et al., 2013; Guemas et al.,

2013). In some initialized models this lower GMST occurs in part because they correctly simulate a shift, around 2000, from a positive

to a negative phase of the Interdecadal Paciﬁc Oscillation (IPO, Box 2.5; e.g., Meehl and Teng, 2012; Meehl et al., 2013a). However,

the improvement of this phasing of the IPO through initialization is not universal across the CMIP5 predictions (cf. Section 11.2.3.4).

Moreover, while part of the GMST reduction through initialization indeed results from initializing at the correct phase of internal

variability, another part may result from correcting a model bias that was caused by incorrect past forcing or incorrect model response

to past forcing, especially in the ocean. The relative magnitudes of these effects are at present unknown (Meehl and Teng, 2012);

moreover, the quality of a forecasting system cannot be evaluated from a single prediction (here, a 10-year prediction within the period

1998–2012; Section 11.2.3). Overall, there is medium conﬁdence that initialization leads to simulations of GMST during 1998–2012

that are more consistent with the observed trend hiatus than are the uninitialized CMIP5 historical simulations, and that the hiatus is

in part a consequence of internal variability that is predictable on the multi-year time scale.

Radiative Forcing

On decadal to interdecadal time scales and under continually increasing effective radiative forcing (ERF), the forced component of

the GMST trend responds to the ERF trend relatively rapidly and almost linearly (medium conﬁdence, e.g., Gregory and Forster, 2008;

Held et al., 2010; Forster et al., 2013). The expected forced-response GMST trend is related to the ERF trend by a factor that has been

estimated for the 1% per year CO

increases in the CMIP5 ensemble as 2.0 [1.3 to 2.7] W m

–2

°C

–1

(90% uncertainty range; Forster et

al., 2013). Hence, an ERF trend can be approximately converted to a forced-response GMST trend, permitting an assessment of how

much of the change in the GMST trends shown in Box 9.2 Figure 1 is due to a change in ERF trend.

The AR5 best-estimate ERF trend over 1998–2011 is 0.22 [0.10 to 0.34] W m

–2

per decade (90% uncertainty range), which is substantially

lower than the trend over 1984–1998 (0.32 [0.22 to 0.42] W m

–2

per decade; note that there was a strong volcanic eruption in 1982)

and the trend over 1951–2011 (0.31 [0.19 to 0.40] W m

–2

per decade; Box 9.2, Figure 1d–f; numbers based on Section 8.5.2, Figure

8.18; the end year 2011 is chosen because data availability is more limited than for GMST). The resulting forced-response GMST trend

would approximately be 0.12 [0.05 to 0.29] °C per decade, 0.19 [0.09 to 0.39] °C per decade, and 0.18 [0.08 to 0.37] °C per decade

for the periods 1998–2011, 1984–1998 and 1951–2011, respectively (the uncertainty ranges assume that the range of the conversion

factor to GMST trend and the range of ERF trend itself are independent). The AR5 best-estimate ERF forcing trend difference between

1998–2011 and 1951–2011 thus might explain about one-half (0.05°C per decade) of the observed GMST trend difference between

these periods (0.06 to 0.08°C per decade, depending on observational data set).

The reduction in AR5 best-estimate ERF trend over 1998–2011 compared to both 1984–1998 and 1951–2011 is mostly due to

decreasing trends in the natural forcings,–0.16 [–0.27 to –0.06] W m

–2

per decade over 1998–2011 compared to 0.01 [–0.00 to 0.01]

W m

–2

per decade over 1951–2011 (Section 8.5.2, Figure 8.19). Solar forcing went from a relative maximum in 2000 to a relative

minimum in 2009, with a peak-to-peak difference of around 0.15 W m

–2

and a linear trend over 1998–2011 of around –0.10 W m

–2

per decade (cf. Section 10.3.1, Box 10.2). Furthermore, a series of small volcanic eruptions has increased the observed stratospheric

aerosol loading after 2000, leading to an additional negative ERF linear-trend contribution of around –0.06 W m

–2

per decade over

1998–2011 (cf. Section 8.4.2.2, Section 8.5.2, Figure 8.19; Box 9.2 Figure 1d, f). By contrast, satellite-derived estimates of tropospheric

aerosol optical depth (AOD) suggests little overall trend in global mean AOD over the last 10 years, implying little change in ERF due

to aerosol-radiative interaction (low conﬁdence because of low conﬁdence in AOD trend itself, Section 2.2.3; Section 8.5.1; Murphy,

2013). Moreover, because there is only low conﬁdence in estimates of ERF due to aerosol–cloud interaction (Section 8.5.1, Table 8.5),

there is likewise low conﬁdence in its trend over the last 15 years.

For the periods 1984–1998 and 1951–2011, the CMIP5 ensemble-mean ERF trend deviates from the AR5 best-estimate ERF trend

by only 0.01 W m

–2

per decade (Box 9.2 Figure 1e, f). After 1998, however, some contributions to a decreasing ERF trend are missing

in the CMIP5 models, such as the increasing stratospheric aerosol loading after 2000 and the unusually low solar minimum in 2009.

Nonetheless, over 1998–2011 the CMIP5 ensemble-mean ERF trend is lower than the AR5 best-estimate ERF trend by 0.03 W m

–2

per

decade (Box 9.2 Figure 1d). Furthermore, global mean AOD in the CMIP5 models shows little trend over 1998–2012, similar to the

observations (Figure 9.29). Although the forcing uncertainties are substantial, there are no apparent incorrect or missing global mean

forcings in the CMIP5 models over the last 15 years that could explain the model–observations difference during the warming hiatus.

(continued on next page)

771

Evaluation of Climate Models Chapter 9

Box 9.2 (continued)

Model Response Error

The discrepancy between simulated and observed GMST trends during 1998–2012 could be explained in part by a tendency for some

CMIP5 models to simulate stronger warming in response to increases in greenhouse gas (GHG) concentration than is consistent

with observations (Section 10.3.1.1.3, Figure 10.4). Averaged over the ensembles of models assessed in Section 10.3.1.1.3, the best-

estimate GHG and other anthropogenic (OA) scaling factors are less than one (though not signiﬁcantly so, Figure 10.4), indicating

that the model-mean GHG and OA responses should be scaled down to best match observations. This ﬁnding provides evidence that

some CMIP5 models show a larger response to GHGs and other anthropogenic factors (dominated by the effects of aerosols) than

the real world (medium conﬁdence). As a consequence, it is argued in Chapter 11 that near-term model projections of GMST increase

should be scaled down by about 10% (Section 11.3.6.3). This downward scaling is, however, not sufﬁcient to explain the model-mean

overestimate of GMST trend over the hiatus period.

Another possible source of model error is the poor representation of water vapour in the upper atmosphere (Section 9.4.1.2). It has

been suggested that a reduction in stratospheric water vapour after 2000 caused a reduction in downward longwave radiation and

hence a surface-cooling contribution (Solomon et al., 2010), possibly missed by the models, However, this effect is assessed here to be

small, because there was a recovery in stratospheric water vapour after 2005 (Section 2.2.2.1, Figure 2.5). (continued on next page)

1998-2012

0.0 0.2 0.4 0.6

(°C per decade) (°C per decade)(°C per decade)

Normalized density

CMIP5

HadCRUT4

(a)

1984-1998

0.0 0.2 0.4 0.6

(b)

1951-2012

0.0 0.2 0.4 0.6

(c)

1998-2011

-0.3 0.0 0.3 0.6 0.9

(W m

-2

per decade) (W m

-2

per decade) (W m

-2

per decade)

Normalized density

(d)

1984-1998

-0.3 0.0 0.3 0.6 0.9

(e)

1951-2011

-0.3 0.0 0.3 0.6 0.9

(f)

Box 9.2, Figure 1 | (Top) Observed and simulated global mean surface temperature (GMST) trends in degrees Celsius per decade, over the periods 1998–2012

(a), 1984–1998 (b), and 1951–2012 (c). For the observations, 100 realizations of the Hadley Centre/Climatic Research Unit gridded surface temperature data set 4

(HadCRUT4) ensemble are shown (red, hatched: Morice et al., 2012). The uncertainty displayed by the ensemble width is that of the statistical construction of the

global average only, in contrast to the trend uncertainties quoted in Section 2.4.3, which include an estimate of internal climate variability. Here, by contrast, internal

variability is characterized through the width of the model ensemble. For the models, all 114 available CMIP5 historical realizations are shown, extended after 2005 with

the RCP4.5 scenario and through 2012 (grey, shaded: after Fyfe et al., 2010). (Bottom) Trends in effective radiative forcing (ERF, in W m

–2

per decade) over the periods

1998–2011 (d), 1984–1998 (e), and 1951–2011 (f). The ﬁgure shows AR5 best-estimate ERF trends (red, hatched; Section 8.5.2, Figure 8.18) and CMIP5 ERF (grey,

shaded: from Forster et al., 2013). Black lines are smoothed versions of the histograms. Each histogram is normalized so that its area sums up to one.

772

Chapter 9 Evaluation of Climate Models

Box 9.2 (continued)

In summary, the observed recent warming hiatus, deﬁned as the reduction in GMST trend during 1998–2012 as compared to the trend

during 1951–2012, is attributable in roughly equal measure to a cooling contribution from internal variability and a reduced trend in

external forcing (expert judgment, medium conﬁdence). The forcing trend reduction is primarily due to a negative forcing trend from

both volcanic eruptions and the downward phase of the solar cycle. However, there is low conﬁdence in quantifying the role of forcing

trend in causing the hiatus, because of uncertainty in the magnitude of the volcanic forcing trend and low conﬁdence in the aerosol

forcing trend.

Almost all CMIP5 historical simulations do not reproduce the observed recent warming hiatus. There is medium conﬁdence that the

GMST trend difference between models and observations during 1998–2012 is to a substantial degree caused by internal variability,

with possible contributions from forcing error and some CMIP5 models overestimating the response to increasing GHG and other

anthropogenic forcing. The CMIP5 model trend in ERF shows no apparent bias against the AR5 best estimate over 1998–2012. However,

conﬁdence in this assessment of CMIP5 ERF trend is low, primarily because of the uncertainties in model aerosol forcing and processes,

which through spatial heterogeneity might well cause an undetected global mean ERF trend error even in the absence of a trend in the

global mean aerosol loading.

The causes of both the observed GMST trend hiatus and of the model–observation GMST trend difference during 1998–2012 imply

that, barring a major volcanic eruption, most 15-year GMST trends in the near-term future will be larger than during 1998–2012

(high conﬁdence; see 11.3.6.3. for a full assessment of near-term projections of GMST). The reasons for this implication are fourfold:

ﬁrst, anthropogenic greenhouse-gas concentrations are expected to rise further in all RCP scenarios; second, anthropogenic aerosol

concentration is expected to decline in all RCP scenarios, and so is the resulting cooling effect; third, the trend in solar forcing is

expected to be larger over most near-term 15-year periods than over 1998–2012 (medium conﬁdence), because 1998–2012 contained

the full downward phase of the solar cycle; and fourth, it is more likely than not that internal climate variability in the near-term will

enhance and not counteract the surface warming expected to arise from the increasing anthropogenic forcing.

9.4.1.4.2 Tropical tropospheric temperature trends

Most climate model simulations show a larger warming in the tropical

troposphere than is found in observational data sets (e.g., McKitrick et

al., 2010; Santer et al., 2013). There has been an extensive and some-

times controversial debate in the published literature as to whether

this difference is statistically signiﬁcant, once observational uncertain-

ties and natural variability are taken into account (e.g., Douglass et

al., 2008; Santer et al., 2008, 2013; Christy et al., 2010; McKitrick et

al., 2010; Bengtsson and Hodges, 2011; Fu et al., 2011; McKitrick et

al., 2011; Thorne et al., 2011). For the period 1979–2012, the various

observational data sets ﬁnd, in the tropical lower troposphere (LT),

a linear warming trend ranging from 0.06°C to 0.13°C per decade

(Section 2.4.4, Figure 2.27). In the tropical middle troposphere (MT),

the linear warming trend ranges from 0.02°C to 0.12°C per decade

(Section 2.4.4, Figure 2.27). Uncertainty in these trend values arises

from different methodological choices made by the groups deriving

satellite products (Mears et al., 2011) and radiosonde compilations

(Thorne et al., 2011), and from ﬁtting a linear trend to a time series

containing substantial interannual and decadal variability (Box 2.2;

Section 2.4.4; (Santer et al., 2008; McKitrick et al., 2010)). Although

there have been substantial methodological debates about the calcu-

lation of trends and their uncertainty, a 95% conﬁdence interval of

around ±0.1°C per decade has been obtained consistently for both LT

and MT (e.g., Section 2.4.4; McKitrick et al., 2010). In summary, despite

unanimous agreement on the sign of the observed trends, there exists

substantial disagreement between available estimates as to the rate

of temperature changes in the tropical troposphere, and there is only

low conﬁdence in the rate of change and its vertical structure (Section

2.4.4).

For the 30-year period 1979–2009 (sometimes updated through 2010

or 2011), the CMIP3 models simulate a tropical warming trend ranging

from 0.1°C to somewhat above 0.4°C per decade for both LT and MT

(McKitrick et al., 2010), while the CMIP5 models simulate a tropical

warming trend ranging from slightly below 0.15°C to somewhat above

0.4°C per decade for both LT and MT (Santer et al., 2013; see also Po-

Chedley and Fu, 2012, who considered the period 1979–2005). Both

model ensembles show trends that on average are higher than in the

observational estimates, although both model ensembles overlap the

observational ensemble. Because the differences between the various

observational estimates are largely systematic and structural (Section

2.4.4; Mears et al., 2011), the uncertainty in the observed trends cannot

be reduced by averaging the observations as if the differences between

the data sets were purely random. Likewise, to properly represent inter-

nal climate variability, the full model ensemble spread must be used in

a comparison against the observations (e.g., Box 9.2; Section 11.2.3.2;

Raftery et al., 2005; Wilks, 2006; Jolliffe and Stephenson, 2011). The

very high signiﬁcance levels of model–observation discrepancies in LT

and MT trends that were obtained in some studies (e.g., Douglass et

al., 2008; McKitrick et al., 2010) thus arose to a substantial degree from

using the standard error of the model ensemble mean as a measure

of uncertainty, instead of the ensemble standard deviation or some

other appropriate measure for uncertainty arising from internal climate

773

Evaluation of Climate Models Chapter 9

variability (e.g., Box 9.2; Section 11.2.3.2; Raftery et al., 2005; Wilks,

2006; Jolliffe and Stephenson, 2011). Nevertheless, almost all model

ensemble members show a warming trend in both LT and MT larger

than observational estimates (McKitrick et al., 2010; Po-Chedley and

Fu, 2012; Santer et al., 2013).

The CMIP3 models show a 1979–2010 tropical SST trend of 0.19°C

per decade in the multi-model mean, signiﬁcantly larger than the

various observational trend estimates ranging from 0.10°C to 0.14°C

per decade (including the 95% conﬁdence interval; Fu et al., 2011).

As a consequence, simulated tropospheric temperature trends are

also too large because models attempt to maintain static stability. By

contrast, atmospheric models that are forced with the observed SST

are in better agreement with observations, as was found in the CMIP3

model ECHAM5 (Bengtsson and Hodges, 2011) and the CMIP5 atmos-

phere-only runs. In the latter, the LT trend range for the period 1981–

2008 is 0.13 to 0.19ºC per decade—less than in the CMIP5 coupled

models, but still an overestimate (Po-Chedley and Fu, 2012). The inﬂu-

ence of SST trend errors on the analysis can be reduced by considering

trends in tropospheric static stability, measured by the ampliﬁcation

of MT trends against LT trends; another approach is to consider the

ampliﬁcation of tropospheric trends against SST trends. The results of

such analyses strongly depend on the time scale considered. Month-

to-month variations are consistent between observations and models

concerning ampliﬁcation aloft against SST variations (Santer et al.,

2005) and concerning ampliﬁcation of MT against LT variations (Po-

Chedley and Fu, 2012). By contrast, the 30-year trend in tropical static

stability has been found to be larger than in the satellite observations

for almost all ensemble members in both CMIP3 (Fu et al., 2011) and

CMIP5 (Po-Chedley and Fu, 2012). However, if the radiosonde compi-

lations are used for the comparison, the trends in static stability in the

CMIP3 models agree much better with the observations, and inconsist-

ency cannot be diagnosed unambiguously (Seidel et al., 2012) . What

caused the remaining trend overestimate in static stability is not clear

but has been argued recently to result from an upward propagation of

bias in the model climatology (O’Gorman and Singh, 2013).

In summary, most, though not all, CMIP3 and CMIP5 models overesti-

mate the observed warming trend in the tropical troposphere during

the satellite period 1979–2012. Roughly one-half to two-thirds of this

difference from the observed trend is due to an overestimate of the

SST trend, which is propagated upward because models attempt to

maintain static stability. There is low conﬁdence in these assessments,

however, due to the low conﬁdence in observed tropical tropospheric

trend rates and vertical structure (Section 2.4.4).

9.4.1.4.3 Extratropical circulation

The AR4 concluded that models, when forced with observed SSTs,

are capable of producing the spatial distribution of storm tracks, but

generally show deﬁciencies in the numbers and depth of cyclones and

the exact locations of the storm tracks. The ability to represent extra-

tropical cyclones in climate models has been improving, partly due to

increases in horizontal resolution.

Storm track biases over the North Atlantic have decreased in CMIP5

models compared to CMIP3 (Zappa et al., 2013) although models

still produce too zonal a storm track in this region and most models

underestimate cyclone intensity (Colle et al., 2013; Zappa et al., 2013).

Chang et al. (2012) also ﬁnd the storm tracks in the CMIP5 models

to be too weak and too equatorwards in their position, similar to the

CMIP3 models. The performance of the CMIP5 models in representing

North Atlantic cyclones was found to be strongly dependent on model

resolution (Colle et al., 2013). Studies based on individual models typi-

cally ﬁnd that models capture the general characteristics of storm tracks

and extratropical cyclones (Ulbrich et al., 2008; Catto et al., 2010) and

their associated fronts (Catto et al., 2013) and show improvements over

earlier model versions (Loptien et al., 2008). However, some models

have deﬁciencies in capturing the location of storm tracks (Greeves et

al., 2007; Catto et al., 2011), in part owing to problems related to the

location of warm waters such as the Gulf Stream and Kuroshio Current

(Greeves et al., 2007; Keeley et al., 2012). This is an important issue

because future projections of storm tracks are sensitive to changes in

SSTs (Catto et al., 2011; Laine et al., 2011; McDonald, 2011; Woollings

et al., 2012). Some studies ﬁnd that storm track and cyclone biases are

strongly related to atmospheric processes and parameterizations (Bauer

et al., 2008a; Boer and Lambert, 2008; Zappa et al., 2013). Representa-

tion of the Mediterranean storm track has been shown to be particularly

dependent on model resolution (Pinto et al., 2006; Raible et al., 2007;

Bengtsson et al., 2009; Ulbrich et al., 2009), as is the representation

of storm intensity and associated extremes in this area (Champion et

al., 2011). Most studies have focussed on NH storm tracks. However,

recently two CMIP3 models were found to differ signiﬁcantly in their

simulation of extratropical cyclones affecting Australia (Dowdy et al.,

2013) and only about a third of the CMIP3 models were able to capture

the observed changes and trends in Southern Hemisphere (SH) baro-

clinicity responsible for a reduction in the growth rate of the leading

winter storm track modes (Frederiksen et al., 2011). There is still a lack

of information on SH storm track evaluation for the CMIP5 models.

9.4.1.4.4 Tropical circulation

Earlier assessments of a weakening Walker circulation (Vecchi et al.,

2006; Vecchi and Soden, 2007; DiNezio et al., 2009) from models and

reanalyses (Yu and Zwiers, 2010) have been tempered by subsequent

evidence that tropical Paciﬁc Trade winds may have strengthened

since the early 1990s

(e.g., Merriﬁeld and Maltrud, 2011)

. Models

suggest that the width of the Hadley cell should increase

(Frierson

et al., 2007; Lu et al., 2007)

, and there are indications that this has

been observed over the past 25 years

(Seidel et al., 2008)

but at an

apparent rate (2 to 5 degrees of latitude since 1979) that is faster

than in the CMIP3 models

(Johanson and Fu, 2009)

The tendency in a warming climate for wet areas to receive more

precipitation and subtropical dry areas to receive less, often termed

the ‘rich-get richer’ mechanism

(Chou et al., 2006; Held and Soden,

2006)

is simulated in CMIP3 models

(Chou and Tu, 2008),

and obser-

vational support for this is found from ocean salinity observations

(Durack et al., 2012) and

precipitation gauge data over land

(Zhang

et al., 2007)

There is medium conﬁdence that models are correct in

simulating precipitation increases in wet areas and decreases in dry

areas on broad spatial scales in a warming climate based on agree-

ment among models and some evidence that this has been detected in

observed trends (see Section 2.5.1).

774

Chapter 9 Evaluation of Climate Models

Several recent studies have examined the co-variability of tropical cli-

mate variables as a further means of evaluating climate models. Spe-

ciﬁcally, there are observed relationships between lower tropospheric

temperature and total column precipitable water (Mears et al., 2007),

and between surface temperature and relative humidity (Willett et al.,

2010). Figure 9.9 (updated from Mears et al., 2007) shows the relation-

ship between 25-year (1988–2012) linear trends in tropical precipita-

ble water and lower tropospheric temperature for individual historical

simulations (extended by appending RCP8.5 simulations after 2005,

see Santer et al., 2013). As described by Mears et al. (2007), the ratio

between changes in these two quantities is fairly tightly constrained

in the model simulations and similar across a range of time scales,

indicating that relative humidity is close to invariant in each model. In

the updated ﬁgure, the Remote Sensing System (RSS) observations are

in fairly good agreement with model expectations, and the University

of Alabama in Huntsville (UAH) observations less so. The points asso-

ciated with two of the reanalyses are also relatively far from the line,

consistent with long-term changes in relative humidity. It is not known

whether thesediscrepancies are due to remaining inhomogeneity in

the observational data and/orreanalysis results, or due to problems

with the climate simulations. All of the observational and reanalysis

points lie at the lower end of the model distribution, consistent with

the ﬁndings of (Santer et al., 2013).

9.4.1.4.5 Ozone and lower stratospheric temperature trends

Stratospheric ozone has been subject to a major perturbation since

the late 1970s due to anthropogenic emissions of ozone-depleting

Trend in precitable water (% per decade)

Trend in TLT (°C per decade)

Slope = 5.7 (% °C

-1

)

Figure 9.9 | Scatter plot of decadal trends in tropical (20ºS to 20ºN) precipitable water as a function of trends in lower tropospheric temperature (TLT) over the world’s oceans.

Coloured symbols are from CMIP5 models; black symbols are from satellite observations or from reanalysis output. Trends are calculated over the 1988–2012 period, so CMIP5

historical runs, which typically end in December 2005, were extended using RCP8.5 simulations initialized using these historical runs. Figure updated from Mears et al. (2007).

substances (see also Section 2.2.2.2 and Figure 2.6). Since the AR4,

there is increasing evidence that the ozone hole has led to a poleward

shift and strengthening of the SH mid-latitude tropospheric jet during

summer (Perlwitz et al., 2008; Son et al., 2008, 2010; SPARC-CCMVal,

2010; McLandress et al., 2011; Polvani et al., 2011; WMO, 2011; Swart

and Fyfe, 2012b). These trends are well captured in both chemistry–cli-

mate models (CCMs) with interactive stratospheric chemistry and in

CMIP3 models with prescribed time-varying ozone (Son et al., 2010;

SPARC-CCMVal, 2010). However, around half of the CMIP3 models pre-

scribe ozone as a ﬁxed climatological value, and so these models are

not able to simulate trends in surface climate attributable to changing

stratospheric ozone amount (Karpechko et al., 2008; Son et al., 2008,

2010; Fogt et al., 2009). For CMIP5, a new time-varying ozone data

set (Cionni et al., 2011) was developed and prescribed in the majority

of models without interactive chemistry. This zonal mean data set is

based on observations by Randel and Wu (2007) and CCM projections

in the future (SPARC-CCMVal, 2010). Further, nine of the CMIP5 models

include interactive chemistry and so compute their own ozone evolu-

tion. As a result, all CMIP5 models consider stratospheric ozone deple-

tion and capture associated effects on SH surface climate, a signiﬁcant

advance over CMIP3. Figure 9.10 shows the global annual mean and

Antarctic October mean of total column ozone in the CMIP5 models.

The simulated trends in total column ozone are in medium agreement

with observations, noting that some models that calculate ozone inter-

actively show signiﬁcant deviations from observation (Eyring et al.,

2013). The multi-model mean agrees well with observations, and there

is robust evidence that this constitutes a signiﬁcant improvement over

CMIP3, where around half of the models did not include stratospheric

775

Evaluation of Climate Models Chapter 9

ozone trends. Correspondingly, there is high conﬁdence that the rep-

resentation of associated effects on high-latitude surface climate and

lower stratospheric cooling trends has improved compared to CMIP3.

Lower stratospheric temperature change is affected by ozone, and

since 1958 the change is characterized by a long-term global cooling

trend interrupted by three 2-year warming episodes following large

volcanic eruptions (Figure 2.24). During the satellite era (since 1979)

the cooling occurred mainly in two step-like transitions in the after-

math of the El Chichón eruption in 1982 and the Mt Pinatubo eruption

in 1991, with each cooling transition followed by a period of relatively

steady temperatures (Randel et al., 2009; Seidel et al., 2011). This spe-

ciﬁc evolution of global lower stratosphere temperatures since 1979 is

well captured in the CMIP5 models when forced with both natural and

anthropogenic climate forcings, although the models tend to underes-

Figure 9.10 | Time series of area-weighted total column ozone from 1960 to 2005 for (a) annual and global mean (90°S to 90°N) and (b) Antarctic October mean (60°S to 90°S).

Individual CMIP5 models with interactive or semi-interactive chemistry are shown in thin coloured lines, their multi-model mean (CMIP5Chem) in thick red and their standard

deviation as the blue shaded area. Further shown are the multi-model mean of the CMIP5 models that prescribe ozone (CMIP5noChem, thick green), the International Global

Atmospheric Chemistry/Stratospheric Processes and their Role in Climate (IGAC/SPARC) ozone database (thick pink), the Chemistry Climate Model Validation-2 (CCMVal-2) multi-

model mean (thick orange), and observations from ﬁve different sources (black symbols). These sources include ground-based measurements (updated from Fioletov et al., 2002),

National Aeronautics and Space Administration (NASA) Total Ozone Mapping Spectrometer/Ozone Monitoring Instrument/Solar Backscatter Ultraviolet(/2) (TOMS/OMI/SBUV(/2))

merged satellite data (Stolarski and Frith, 2006), the National Institute of Water and Atmospheric Research (NIWA) combined total column ozone database (Bodeker et al., 2005),

Solar Backscatter Ultraviolet (SBUV, SBUV/2) retrievals (updated from Miller et al. 2002), and Deutsches Zentrum für Luft- und Raumfahrt/ Global Ozone Monitoring Experiment/

SCanning Imaging Absorption spectrometer for atmospheric chartography /GOME-2 (DLR GOME/SCIA/GOME-2; Loyola et al., 2009; Loyola and Coldewey-Egbers, 2012). Note that

the IGAC/SPARC database over Antarctica (and thus the majority of the CMIP5noChem models) is based on ozonesonde measurements at the vortex edge (69°S) and as a result

underestimates Antarctic ozone depletion compared to the observations shown. Ozone depletion was more pronounced after 1960 as equivalent stratospheric chlorine values

steadily increased throughout the stratosphere. (Adapted from Figure 2 of Eyring et al., 2013.)

timate the long-term cooling trend (Charlton-Perez et al., 2012; Eyring

et al., 2013; Santer et al., 2013) (see Chapter 10).

Tropospheric ozone is an important GHG and as such needs to be

well represented in climate simulations. In the historical period it has

increased due to increases in ozone precursor emissions from anthro-

pogenic activities (see Chapters 2 and 8). Since the AR4, a new emis-

sion data set has been developed (Lamarque et al., 2010), which has

led to some differences in tropospheric ozone burden compared to pre-

vious studies, mainly due to biomass burning emissions (Lamarque et

al., 2010; Cionni et al., 2011; Young et al., 2013). Climatological mean

tropospheric ozone in the CMIP5 simulations generally agrees well

with satellite observations and ozonesondes, although as in the strato-

sphere, biases exist for individual models (Eyring et al., 2013; Young et

al., 2013) (see also Chapter 8).

776

Chapter 9 Evaluation of Climate Models

9.4.1.5 Model Simulations of the Last Glacial Maximum and

the Mid-Holocene

Simulations of past climate can be used to test a model’s response

to forcings larger than those of the 20th century (see Chapter 5), and

the CMIP5 protocol includes palaeoclimate simulations referred to as

PMIP3 (Paleoclimate Model Intercomparison Project, version 3) (Taylor

et al., 2012b). Speciﬁcally, the Last Glacial Maximum (LGM, 21000

years BP) allows testing of the modelled climate response to the pres-

ence of a large ice sheet in the NH and to lower concentrations of

radiatively active trace gases, whereas the mid-Holocene (MH, 6000

years BP) tests the response to changes in seasonality of insolation in

the NH (see Chapter 5). For these periods, palaeoclimate reconstruc-

tions allow quantitative model assessment (Braconnot et al., 2012).

( )

(mm year

-1

)

(mm year

-1

)

In addition the CMIP5/PMIP3 simulations can compared to previous

palaeoclimate intercomparisons (Joussaume and Taylor, 1995; Bracon-

not et al., 2007c).

Figure 9.11 compares model results to palaeoclimate reconstructions

for both LGM (left) and MH (right). For most models the simulated LGM

cooling is within the range of the climate reconstructions (Braconnot

et al., 2007c; Izumi et al., 2013), however Hargreaves et al. (2011) ﬁnd

a global mean model warm bias over the ocean of about 1°C for this

period (Hargreaves et al., 2011). LGM simulations tend to overestimate

tropical cooling and underestimate mid-latitude cooling (Kageyama et

al., 2006; Otto-Bliesner et al., 2009). They thus underestimate polar

ampliﬁcation which is a feature also found for the mid-Holocene (Mas-

son-Delmotte et al., 2006; Zhang et al., 2010a) and other climatic con-

Figure 9.11 | Reconstructed and simulated conditions for the Last Glacial Maximum (LGM, 21,000 years BP, left) and the mid-Holocene (MH, 6000 years BP, right). (a) LGM change

in annual mean surface temperature (°C) over land as shown by palaeo-environmental climate reconstructions from pollen, macrofossils, and ice cores (Bartlein et al., 2010; Bracon-

not et al., 2012), and in annual mean sea surface temperature (°C) over the ocean from different type of marine records (Waelbroeck et al., 2009). (b) MH change in annual mean

precipitation (mm yr

–1

) over land (Bartlein et al., 2010). In (a) and (b), the size of the dots is proportional to the uncertainties at the different sites as provided in the reconstructions.

(c) Annual mean temperature changes over land against changes over the ocean, in the tropics (downward triangles) and over the North Atlantic and Europe (upward triangles).

The mean and range of the reconstructions are shown in black, the Paleoclimate Modelling Intercomparison Project version 2 (PMIP2) simulations as grey triangles, and the CMIP5/

PMIP3 simulations as coloured triangles. The 5 to 95% model ranges are in red for the tropics and in blue for the North Atlantic/Europe. (d) Changes in annual mean precipitation in

different data-rich regions. Box plots for reconstructions provide the range of reconstructed values for the region. For models, the individual model average over the region is plotted

for PMIP2 (small grey circle) and CMIP5/PMIP3 simulations (coloured circles). Note that in PMIP2, ‘ESM’ indicates that vegetation is computed using a dynamical vegetation model,

whereas in CMIP5/PMIP3 it indicates that models have an interactive carbon cycle with different complexity in dynamical vegetation (see Table 9.A.1). The limits of the boxes are

as follows: Western Europe (40°N to 50°N, 10°W to 30°E); northeast America (35°N to 60°N, 95°W to 60°W); North Africa (10°N to 25°N, 20°W to 30°W), and East Asia (25°N

to 40°N, 75°E to 105°E). (Adapted from Braconnot et al., 2012.)

777

Evaluation of Climate Models Chapter 9

Figure 9.12 | Relative model performance for the Last Glacial Maximum (LGM, about 21,000 yr BP) and the mid-Holocene (MH, about 6000 yr BP) for seven bioclimatic variables:

annual mean sea surface temperature, mean annual temperature (over land), mean temperature of the coldest month, mean temperature of the warmest month, growing degree

days above a threshold of 5°C, and ratio of actual to equilibrium evapotranspiration. Model output is compared to the Bartlein et al. (2010) data set over land, including ice core data

over Greenland and Antarctica (Braconnot et al., 2012) and the Margo data set (Waelbroeck et al., 2009) over the ocean. The CMIP5/Paleoclimate Modelling Intercomparison Project

version 3 (PMIP3) ensemble of Ocean–Atmosphere (OA) and Earth System Model (ESM) simulations are compared to the respective PMIP2 ensembles in the ﬁrst four columns of

each panel. A diagonal divides each cell in two parts to show in the upper triangle a measure of the distance between model and data, taking into account the uncertainties in the

palaeoclimate reconstructions (Guiot et al., 1999), and in the lower triangle the normalized mean-square error (NMSE) that indicates how well the spatial pattern is represented.

In this graph all the values have been normalized following (Gleckler et al., 2008) using the median of the CMIP5/PMIP3 ensemble. The colour scale is such that blue colours mean

that the result is better than the median CMIP5 model and red means that it is worse.

texts (Masson-Delmotte et al., 2010). Part of this can be attributed to

uncertainties in the representation of sea ice and vegetation feedbacks

that have been shown to amplify the response at the LGM and the MH

in these latitudes (Braconnot et al., 2007b; Otto et al., 2009; O’ishi and

Abe-Ouchi, 2011). Biases in the representation of the coupling between

vegetation and soil moisture are also responsible for excessive conti-

nental drying at the LGM (Wohlfahrt et al., 2008) and uncertainties in

vegetation feedback in monsoon regions (Wang et al., 2008; Dallmeyer

et al., 2010). Nevertheless, the ratio between the simulated change

in temperature over land and over the ocean (Figure 9.11c) is rather

similar in different models, resulting mainly from simulation of the

hydrological cycle over land and ocean (Sutton et al., 2007; Laine et al.,

2009). At a regional scale, models tend to underestimate the changes

in the north-south temperature gradient over Europe both at the LGM

(Ramstein et al., 2007) and at the mid-Holocene (Brewer et al., 2007;

Davis and Brewer, 2009).

The large-scale pattern of precipitation change during the MH (Figure

9.11d) is reproduced, but models tend to underestimate the magnitude

of precipitation change in most regions. In the SH (not shown in the

ﬁgure), the simulated change in atmospheric circulation is consistent

with precipitation records in Patagonia and New Zealand, even though

the differences between model results are large and the reconstruc-

tions have large uncertainties (Rojas et al., 2009; Rojas and Moreno,

2011).

A wider range of model performance metrics is provided in Figure 9.12

(Guiot et al., 1999; Brewer et al., 2007; Annan and Hargreaves, 2011;

Izumi et al., 2013). Results for the MH are less reliable than for the

LGM, because the forcing is weaker and involves smaller scale respons-

es over the continent (Hargreaves et al., 2013). As is the case for the

simulations of present day climate, there is only modest improvement

between the results of the more recent models (CMIP5/PMIP3) and

those of earlier model versions (PMIP2) despite higher resolution and

sophistication.

9.4.2 Ocean

Accurate simulation of the ocean in climate models is essential for the

correct estimation of transient ocean heat uptake and transient climate

response, ocean CO

uptake, sea level rise, and coupled climate modes

such as ENSO. In this section model performance is assessed for the

mean state of ocean properties, surface ﬂuxes and their impact on the

simulation of ocean heat content and sea level, and aspects of impor-

tance for climate variability. Simulations of both the recent and more

distant past are evaluated against available data. Following Chapter 3,

ocean reanalyses are not used for model evaluation as many of their

properties depend on the model used to build the reanalysis.

9.4.2.1 Simulation of Mean Temperature and Salinity Structure

Potential temperature and salinity are the main ocean state variables

and their zonal distribution offers an evaluation of climate models in

different parts of the ocean (upper ocean, thermocline, deep ocean).

Over most latitudes, at depths ranging from 200 m to 2000 m, the

CMIP5 multi-model mean zonally averaged ocean temperature is too

warm (Figure 9.13a), albeit with a cooler deep ocean. Similar biases

were evident in the CMIP3 multi-model mean. Above 200 m, however,

778

Chapter 9 Evaluation of Climate Models

the CMIP5 (and CMIP3) multi-model mean is too cold, with maximum

cold bias (more than 1°C) near the surface at mid-latitudes of the NH

and near 200 m at 15°S. Zonal salinity errors (Figure 9.13b) exhibit a

different pattern from those of the potential temperature indicating

that most do not occur via density compensation. Some near surface

structures in the tropics and in the northern mid-latitude are indicative

of density compensation and are presumably due to surface ﬂuxes

errors. At intermediate depths, errors in water mass formation translate

into errors in both salinity and potential temperature.

In the AR4 it was noted that the largest errors in SST in CMIP3 were

found in mid and high latitudes. While this is still the case in CMIP5,

there is marginal improvement with fewer individual models exhibiting

serious bias—the inter-model zonal mean SST error standard deviation

is signiﬁcantly reduced at all latitudes north of 40

S—even though the

multi-model mean is only slightly improved (Figure 9.14a, c). Near the

equator, the cold tongue error in the Paciﬁc (see Section 9.4.2.5.1) is

reduced by 30% in CMIP5; the Atlantic still exhibits serious errors and

the Indian is still well simulated (Figure 9.14b,d). In the Tropics, Li and

Xie (2012) have shown that SST errors could be classiﬁed into those

exhibiting broad meridional structures that are due to cloud errors, and

those associated with Paciﬁc and Atlantic cold tongue errors that are

due to thermocline depth errors.

Sea surface salinity (SSS) is more challenging to observe, even though

the last decade has seen substantial improvements in the development

of global salinity observations, such as those from the Array for Real-

time Geostrophic Oceanography (ARGO) network (see Chapter 3).

Whereas SST is strongly constrained by air–sea interactions, the sources

of SSS variations (surface forcing via evaporation minus precipitation,

Figure 9.13 | (a) Potential temperature (oC) and (b) salinity (PSS-78); shown in colour are the time-mean differences between the CMIP5 ensemble mean and observations, zonally

averaged for the global ocean (excluding marginal and regional seas). The observed climatological values are sourced from the World Ocean Atlas 2009 (WOA09; Prepared by the

Ocean Climate Laboratory, National Oceanographic Data Center, Silver Spring, MD, USA), and are shown as labelled black contours. White contours show regions in (a) where poten-

tial temperature differences exceed positive or negative 1, 2 or 3°C, and in (b) where salinity differences exceed positive or negative 0.25, 0.5, 0.75 or 1 (PSS-78). The simulated

annual mean climatologies are obtained for 1975 to 2005 from available historical simulations, whereas WOA09 synthesizes observed data from 1874 to 2008 in calculations of the

annual mean; however, the median time for gridded observations most closely resembles the 1980–2010 period (Durack and Wijffels, 2012). Multiple realizations from individual

models are ﬁrst averaged to form a single-model climatology, before the construction of the multi-model ensemble mean. A total of 43 available CMIP5 models have contributed

to the temperature panel (a) and 41 models to the salinity panel (b).

Depth (m)

200

400

600

800

Latitude

Temperature

90S 60S 30S EQU 30N 60N 90N

1000

2000

3000

4000

5000

−3 −2 −1 0 1 2 3

34.

34.5

35.5

34.5

Latitude

Salinity

90S 60S 30S EQU 30N 60N 90N

−1 −0.75 −0.5 −0.25 0 0.25 0.5 0.75 1

sea ice formation/melt and river runoff) are only loosely related to the

SSS itself, allowing errors to develop unchecked in coupled models. An

analysis of CMIP3 models showed that, whereas the historical trend in

global mean SSS is well captured by the models, regional SSS biases

are as high as ±2.5 psu (Terray et al., 2012). Comparisons of modelled

versus observed estimates of evaporation minus precipitation suggest

that model biases in surface freshwater ﬂux play a role in some regions

(e.g., double Intertropical Convergence Zone (ITCZ) in the East Paciﬁc;

Lin, 2007) or over the Indian ocean (Pokhrel et al., 2012).

The performance of coupled climate models in simulating hydrograph-

ic structure and variability were assessed in two important regions,

the Labrador and Irminger Seas and the Southern Ocean (de Jong et

al., 2009) and (Sloyan and Kamenkovich, 2007). Eight CMIP3 models

produce simulations of the intermediate and deep layers in the Lab-

rador and Irminger Seas that are generally too warm and saline, with

biases up to 0.7 psu and 2.9°C. The biases arise because the convective

regime is restricted to the upper 500 m; thus, intermediate water that

in reality is formed by convection is, in the models, partly replaced

by warmer water from the south. In the Southern Ocean, Subantarctic

Mode Water (SAMW) and Antarctic Intermediate Water (AAIW), two

water masses indicating very efﬁcient ocean ventilation, are found to

be well simulated in some CMIP3 and CMIP5 models but not in others,

some having a signiﬁcant fresh bias (Sloyan and Kamenkovich, 2007;

Salle et al., 2013). McClean and Carman (2011) found biases in the

properties of the North Atlantic mode waters and their formation rates

in the CMIP3 models. Errors in Subtropical Mode Water (STMW) forma-

tion rate and volume produce a turnover time of 1 to 2 years, approx-

imately half of that observed. Bottom water properties assessment in

CMIP5 shows that about half of the models create dense water on

779

Evaluation of Climate Models Chapter 9

Figure 9.14 | (a) Zonally averaged sea surface temperature (SST) error in CMIP5 models. (b) Equatorial SST error in CMIP5 models. (c) Zonally averaged multi-model mean SST

error for CMIP5 (red curve) and CMIP3 (blue curve), together with inter-model standard deviation (shading). (d) Equatorial multi-model mean SST in CMIP5 (red curve), CMIP3 (blue

curve) together with inter-model standard deviation (shading) and observations (black). Model climatologies are derived from the 1979–1999 mean of the historical simulations.

The Hadley Centre Sea Ice and Sea Surface Temperature (HadISST) (Rayner et al., 2003) observational climatology for 1979–1999 is used as reference for the error calculation (a),

(b), and (c); and for observations in (d).

the Antarctic shelf, but it mixes with lighter water and is not export-

ed as bottom water. Instead most models create deep water by open

ocean deep convection, a process occurring rarely in reality (Heuzé et

al., 2013) which leads to errors in deep water formation and properties

in the Southern Ocean as shown in Figure 9.15.

Few studies have assessed the performance of models in simulating

Mixed Layer Depth (MLD). In the North East Paciﬁc region, Jang et al.

(2011) found that the CMIP3 models exhibit the observed deep MLD

in the Kuroshio Extension, though with a deep bias and only one large

deep MLD region, rather than the observed two localized maxima.

Other studies have noted MLD biases near sea ice edges (Capotondi

et al., 2012).

9.4.2.2 Simulation of Sea Level and Ocean Heat Content

Steric and dynamic components of the mean dynamic topography (MDT)

and sea surface height (SSH) patterns can be compared to observations

(Maximenko et al., 2009). Pattern correlations between simulated and

observed MDT are above 0.95 for all of the CMIP5 models (Figure

9.16), an improvement compared to CMIP3. MDT biases over tropical

ocean regions are consistent with surface wind stress biases (Lee et al.,

2013). Over the Antarctic Circumpolar Current, the parameterization of

eddy-induced transports is essential for the models’ density structure

and thus MDT (Kuhlbrodt et al., 2012). High-resolution eddy resolving

ocean models show improved SSH simulations over coarser resolution

versions (McClean et al., 2006). Chapter 13 provides a more extensive

780

Chapter 9 Evaluation of Climate Models

Figure 9.15 | Time-mean bottom potential temperature in the Southern Ocean, observed (a) and the differences between individual CMIP5 models and observations (b–p); left

colour bar corresponds to the observations, right colour bar to the differences between model and observations (same unit). Thick dashed black line is the mean August sea ice

extent (concentration >15%); thick continuous black line is the mean February sea ice extent (concentration >15%). Numbers indicate the area-weighted root-mean-square (RMS)

error for all depths between the model and the climatology (unit °C); mean RMS error = 0.97 °C. (After Heuzé et al., 2013.)

a) b) c) d)

e) f) g) h)

i) j) k) l)

-2 -1 0123-2 -1 012

(°C)

n) o) p)

Clim.

Obs.

CNRM-CM5

1.90

HiGEM

0.67

GISS-E2-R

1.24

NorESM1-M

0.73

INMCM4

1.30

MIROC4h

1.14

HadGM2-ES

0.62

CanESM2

0.54

CSIRO

Mk3-6-0

0.72

GFDL

ESM2M

1.52

IPSL

CM5A-LR

0.66

GFDL

ESM2G

0.72

MIROC

ESM-CHEM

0.52

MPI

ESM-LR

0.94

MRI

CGCM3

1.34

assessment of sea level changes in CMIP5 simulations, including com-

parisons with century-scale historical records.

Ocean heat content (OHC) depends only on ocean temperature, where-

as absolute changes in sea level are also inﬂuenced by processes that

are only now being incorporated into global models (e.g., mass loss

from large ice sheets discussed in Section 9.1.3.2.7). However, glob-

al-scale changes in OHC are highly correlated with the thermosteric

contribution to global SSH changes (Domingues et al., 2008). Approx-

imately half of the historical CMIP3 simulations did not include the

effects of volcanic eruptions, resulting in substantially greater than

observed ocean heat uptake during the late 20th century (Gleckler et

781

Evaluation of Climate Models Chapter 9

0.1

0.2

0.5

0.6

0.4

0.5

0.57

0.7

0.8

0.9

0.8

0.9

0.95

0.99

Standard deviation

Observations

CMIP3 mean

CMIP5 mean

Observations

CMIP5 mean

CMIP3 mean

ACCESS1.0

ACCESS1.3

BCC−CSM1.1

CCSM4

CESM1(BGC)

CESM1(WACCM)

CMCC−CM

CNRM−CM5

CSIRO−Mk3.6.0

CanCM4

CanESM2

EC−EARTH

GFDL−CM2p1

GFDL−CM3

GFDL−ESM2G

GFDL−ESM2M

GISS−E2−R

HadGEM2−CC

HadGEM2−ES

INM−CM4

IPSL−CM5A−LR

IPSL−CM5A−MR

IPSL−CM5B−LR

MIROC−ESM−CHEM

MIROC−ESM

MIROC4h

MIROC5

MPI−ESM−LR

MPI−ESM−MR

MPI−ESM−P

MRI−CGCM3

NorESM1−ME

NorESM1−M

Figure 9.16 | Taylor diagram for the dynamic sea surface height climatology (1987–2000). The radial coordinate shows the standard deviation of the spatial pattern, normalized

by the observed standard deviation. The azimuthal variable shows the correlation of the modelled spatial pattern with the observed spatial pattern. The root-mean square error with

bias removed is indicated by the dashed grey circles about the observational point. Analysis is for the global ocean, 50°S to 50°N. The reference data set is Archiving, Validation

and Interpretation of Satellite Oceanographic data (AVISO), a merged satellite product (Ducet et al., 2000), which is described in Chapter 3. One realization per model is shown for

each CMIP5 and CMIP3 model result. Grey ﬁlled circles are for individual CMIP3 models; other symbols as in legend.

al., 2006; Domingues et al., 2008). Figure 9.17 shows observed and

simulated global 0 to 700 m and total OHC changes during the overlap

period of the observational record and the CMIP5 historical experiment

(1961–2005). Three upper-ocean observational estimates, assessed in

Chapter 3, are also shown to indicate observational uncertainty. The

CMIP5 multi-model mean falls within the range of observations for

most of the period, and the intermodel spread is reduced relative to

CMIP3 (Gleckler et al., 2006; Domingues et al., 2008). This may result

from most CMIP5 models including volcanic forcings. When the deep

ocean is included, the CMIP5 multi-model mean also agrees well with

the observations, although the deeper ocean estimates are much more

uncertain (Chapter 3). There is high conﬁdence that many CMIP5

models reproduce the observed increase in ocean heat content since

1960.

EMIC results for changes in total OHC are also compared with obser-

vations in Figure 9.17. (Note: results in this ﬁgure are based on Eby

et al. (2013) who show OHC changes for 0 to 2000 m, whereas here

the time-integrated net heat ﬂux into the ocean surface is shown to

compare with CMIP5 results (Figure 9.17b)). There is a tendency for

the EMICs to overestimate total OHC changes and this could alter the

temperature related feedbacks on the oceanic carbon cycle, and affect

the long-term millennium projections in Chapter 12. However, it should

be noted that high OHC changes can compensate for biases in climate

sensitivity or RF so as to reproduce surface temperature changes over

the 20th century. This will result in biased thermosteric sea level rise

for millennial projections. Calibrated EMICs (Meinshausen et al., 2009;

Sokolov et al., 2010) would remove such biases.

In idealized CMIP5 experiments (CO

increasing 1% yr

–1

), the heat

uptake efﬁciency of the CMIP5 models varies by a factor of two,

explaining about 50% of the model spread (Kuhlbrodt and Gregory,

2012). Despite observational uncertainties, this recent work also pro-

vides limited evidence that in the upper 2000 m, most CMIP5 models

are less stratiﬁed (in the global mean) than is observed, which sug-

gests that these models transport heat downwards more efﬁciently

than the real ocean. These results are consistent with earlier studies

(Forest et al., 2006, 2008; Boe et al., 2009a; Sokolov et al., 2010) that

conclude the CMIP3 models may overestimate oceanic mixing efﬁcien-

cy and therefore underestimate the Transient Climate Response (TCR)

and its impact on future surface warming. However, Kuhlbrodt and

Gregory (2012) also ﬁnd that this apparent bias explains very little of

the model spread in TCR. Although some progress has been made in

understanding mixing deﬁciencies in ocean models (Grifﬁes and Great-

batch, 2012; Ilicak et al., 2012), this remains a key challenge in improv-

ing the representation of physical processes that impact the evolution

of ocean heat content and thermal expansion.

782

Chapter 9 Evaluation of Climate Models

Figure 9.17 | Time series of simulated and observed global ocean heat content anomalies (with respect to 1971). CMIP5 historical simulations and observations for both the

upper 700 meters of the ocean (a) as well as for the total ocean heat content (b). Total ocean heat content results are also shown for EMICs and observations (c). EMIC estimates

are based on time-integrated surface heat ﬂux into the ocean. The 0 to 700 m and total heat content observational estimates (thick lines) are respectively described in Figure 3.2

and Box 3.1, Figure 1. Simulation drift has been removed from all CMIP5 runs with a contemporaneous portion of a quadratic ﬁt to each corresponding pre-industrial control run

(Gleckler et al., 2012). Units are 10

Joules.

9.4.2.3 Simulation of Circulation Features Important for

Climate Response

9.4.2.3.1 Simulation of recent ocean circulation

Atlantic Meridional Overturning Circulation

The Atlantic Meridional Overturning Circulation (AMOC) consists of

northward transport of shallow warm water overlying a southward

transport of deep cold water and is responsible for a considerable part

of the northward oceanic heat transport. Long-term AMOC estimates

have had to be inferred from hydrographic measurements sporadically

available over the last decades (e.g., Bryden et al., 2005; Lumpkin et

al., 2008, Chapter 3.6.3). Continuous AMOC monitoring at 26.5°N was

started in 2004 (Cunningham et al., 2007) and now provides a 5-year

mean value of 18.5 Sv with annual means having a standard devia-

tion of 1 Sv (McCarthy et al., 2012). The ability of models to simulate

this important circulation feature is tied to the credibility of simulated

AMOC weakening during the 21st century because the magnitude of

783

Evaluation of Climate Models Chapter 9

the weakening is correlated with the initial AMOC strength (Gregory

et al., 2005). The mean AMOC strength in CMIP5 models ranges from

15 to 30 Sv for the historical period which is comparable to the CMIP3

models (Weaver et al., 2012; see Figure 12.35). The variability of the

AMOC is assessed in Section 9.5.3.3.1.

Southern Ocean circulation

The Southern Ocean is an important driver for the meridional over-

turning circulation and is closely linked to the zonally continuous

Antarctic Circumpolar Current (ACC). Gupta et al. (2009) noted that

relatively small deﬁciencies in the position of the ACC lead to more

obvious biases in the SST in the models. The ability of CMIP3 models

to adequately represent Southern Ocean circulation and water masses

seems to be affected by several factors (Russell et al., 2006). The most

important are the strength of the westerlies at the latitude of the Drake

Passage, the heat ﬂux gradient over this region, and the change in

salinity with depth across the ACC. Kuhlbrodt et al. (2012) found that

the strongest inﬂuence on ACC transport in the CMIP3 models was the

Gent-McWilliams thickness diffusivity. The ACC has a typical transport

through the Drake Passage of about 135 Sv (e.g., Cunningham et al.,

2003). A comparison of CMIP5 models (Meijers et al., 2012) shows

that, ﬁrstly, the ACC transport through Drake Passage is improved as

compared to the CMIP3 models, and secondly, that the inter-model

44.8

45.2

45.6

46.4 46.8 47.2

47.6

48.4

48.

Salinity(psu)

34.5 35.534.035.036.

.735.630.73

42-

Present LGM

N. Atl.

N. Atl. = Site 981, 55.5N 14.5W 2184m

SO = Site 1093, 50S, 6E 3626m

Observed

Black circles

CMIP5/PMIP3

Big colored triangles

CCSM4

CNRM-CM5

FGOALS-g2

GISS-E2-R_p150

GISS-E2-R_p151

IPSL-CM5A-LR

MIROC-ESM

MPI-ESM-P_p1

MPI-ESM-P_p2

MRI-CGCM3

PMIP2

Small grey triangles

CCSM

ECBILT

ECHAM_oav

FGOALS

HadCM

HadCM_oav

IPSL

MIROC

Potential temperature (°C)

range in the zonal mean ACC position is smaller than in the CMIP3

ensemble (in CMIP5, the mean transport is 148 Sv and the standard

deviation is 50 Sv across an ensemble of 21 models).

Simulation of glacial ocean conditions

Reconstructions of the last glacial maximum from sediment cores dis-

cussed in Chapter 5 indicate that the regions of deep water forma-

tion in the North Atlantic were shifted southward, that the boundary

between North Atlantic Deep Water (NADW) and Antarctic Bottom

Water (AABW) was substantially shallower than today, and that

NADW formation was less intense (Duplessy et al., 1988; Dokken

and Jansen, 1999; McManus et al., 2004; Curry and Oppo, 2005). This

signal, although estimated from a limited number of sites, is robust

(see Chapter 5). The AR4 reported that model simulations showed a

wide range of AMOC response to LGM forcing (Weber et al., 2007),

with some models exhibiting reduced strength of the AMOC and its

extension at depth and other showing no change or an increase. Figure

9.18 provides an update of the diagnosis proposed by Otto-Bliesner

et al. (2007) to compare model results with the deep ocean data from

Adkins et al. (2002) using PMIP2 and CMIP5/PMIP3 pre-industrial and

LGM simulations (Braconnot et al., 2012). These models reproduce the

modern deep ocean temperature–salinity (T–S) structure in the Atlan-

tic basin, but most of them do not capture the cold and salty bottom

Figure 9.18 | Temperature and salinity for the modern period (open symbols) and the Last Glacial Maximum (LGM, ﬁlled symbols) as estimated from proxy data at Ocean Drilling

Program (ODP) sites (black symbols, from Adkins et al., 2002) and simulated by the Paleoclimate Modelling Intercomparison Project version 2 (PMIP2, small triangles) and PMIP3/

CMIP5 (big triangles) models. The isolines represent lines of equal density. Site 981 (triangles) is located in the North Atlantic (Feni Drift, 55ºN, 15ºW, 2184 m). Site 1093 (upside-

down triangles) is located in the South Atlantic (Shona Rise, 50ºS, 6ºE, 3626 m). In PMIP2, only Community Climate System Model (CCSM) included a 1 psu adjustment of ocean

salinity at initialization to account for freshwater frozen into LGM ice sheets; the other PMIP2 model-simulated salinities have been adjusted to allow a comparison. In PMIP3, all

simulations include the 1 psu adjustment as required in the PMIP2/CMIP5 protocol (Braconnot et al., 2012). The dotted lines allow a comparison of the values at the NH and SH

sites for a same model. This ﬁgure is adapted from Otto-Bliesner et al. (2007).

784

Chapter 9 Evaluation of Climate Models

water suggested by the LGM reconstructions, providing evidence that

processes responsible for such palaeoclimate changes may not be well

reproduced in contemporary climate models. This is expected to also

affect projected changes in deep ocean properties.

9.4.2.4 Simulation of Surface Fluxes and Meridional Transports

Surface ﬂuxes play a large part in determining the ﬁdelity of ocean

simulations. As noted in the AR4, large uncertainties in surface heat

and fresh water ﬂux observations (usually obtained indirectly) do not

allow useful evaluation of models. This is still the case and so the focus

here is on an integrated quantity, meridional heat transport, which is

less prone to errors. Surface wind stress is better observed and models

are evaluated against observed products below.

The zonal component of wind stress is particularly important in driv-

ing ocean surface currents; modelled and observed values are shown

in Figure 9.19. At middle to high latitudes, the model-simulated wind

stress maximum lies 5 to 10 degree equatorward of that in the obser-

vationally based estimates, and so mid-latitude westerly winds are

Figure 9.19 | Zonal-mean zonal wind stress over the oceans in (a) CMIP5 models and (b) multi-model mean comparison with CMIP3. Shown is the time-mean of the period

1970–1999 from the historical simulations. The black solid, dashed, and dotted curves represent ECMWF reanalysis of the global atmosphere and surface conditions (ERA)-Interim

(Dee et al., 2011), National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) reanalysis I (Kalnay et al., 1996), and QuikSCAT satellite

measurements (Risien and Chelton, 2008), respectively. In (b) the shading indicates the inter-model standard deviation.

785

Evaluation of Climate Models Chapter 9

Figure 9.20 | Equatorial (2°S to 2°N averaged) zonal wind stress for the Indian, Paciﬁc, and Atlantic oceans in (a) CMIP5 models and (b) multi-model mean comparison with

CMIP3. Shown is the time-mean of the period 1970–1999 from the historical simulations. The black solid, dashed, and dotted curves represent ERA-Interim (Dee et al., 2011),

National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) reanalysis I (Kalnay et al., 1996) and QuikSCAT satellite measurements

(Risien and Chelton, 2008), respectively. In (b) the shading indicates the inter-model standard deviation.

too strong in models. This equatorward shift in the southern ocean

is slightly reduced in CMIP5 relative to CMIP3. At these latitudes, the

largest near surface wind speed biases in CMIP5 are located over the

Paciﬁc sector and the smallest are in the Atlantic sector (Bracegirdle et

al., 2013). Such wind stress errors may adversely affect oceanic heat

and carbon uptake (Swart and Fyfe, 2012a). At middle to low latitudes,

the CMIP3 and CMIP5 model spreads are smaller than at high lati-

tudes, although near the equator this can occur through compensating

errors (Figure 9.20). The simulated multi-model mean equatorial zonal

wind stress is too weak in the Atlantic and Indian Oceans and too

strong in the western Paciﬁc, with no major improvement from CMIP3

to CMIP5.

The CMIP5 model simulations qualitatively agree with the various

observational estimates on the most important features of ocean heat

transport (Figure 9.21) and, in a multi-model sense, no major change

from CMIP3 can be seen. All CMIP5 models are able to the represent the

strong north-south asymmetry, with the largest values in the NH, con-

sistent with the observational estimates. At most latitudes the majority

of CMIP5 model results fall within the range of observational estimates,

786

Chapter 9 Evaluation of Climate Models

Figure 9.21 | Annual- and zonal-mean oceanic heat transport implied by net heat ﬂux imbalances at the sea surface for CMIP5 simulations, under an assumption of negligible

changes in oceanic heat content. Observational estimates include: the data set from Trenberth and Caron (2001) for the period February 1985 to April 1989, derived from reanalysis

products from the National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR; Kalnay et al., 1996; dash-dotted black) and European

Centre for Medium Range Weather Forecasts 40-year reanalysis (ERA40; Uppala et al., 2005; short-dashed black), an updated version by Trenberth and Fasullo (2008) with

improved top of the atmosphere (TOA) radiation data from the Clouds and Earth’s Radiant Energy System (CERES) for March 2000 to May 2004, and updated NCEP reanalysis

(Kistler et al., 2001) up to 2006 (solid black), the Large and Yeager (2009) analysis based on the range of annual mean transport estimated over the years 1984–2006, computed

from air–sea surface ﬂuxes adjusted to agree in the mean with a variety of satellite and in situ measurements (long-dashed black), and direct estimates by Ganachaud and Wunsch

(2003) obtained from hydrographic sections during the World Ocean Circulation Experiment combined with inverse models (black diamonds). The model climatologies are derived

from the years 1986 to 2005 in the historical simulations in CMIP5. The multi-model mean is shown as a thick red line. The CMIP3 multi-model mean is added as a thick blue line.

although there is some suggestion of modest underestimate between

15°N and 25°N and south of about 60°S. Some models show an equa-

torward transport at Southern-Hemisphere mid-latitudes that is also

featured in the observation estimate of Large and Yeager (2009). This

highlights the difﬁculties in representing large-scale energy processes

in the Southern ocean as discussed by Trenberth and Fasullo (2010b).

Note that climate models should exhibit a vanishing net energy balance

when long time averages are considered but unphysical sources and

sinks lead to energy biases (Trenberth and Fasullo, 2009, 2010a; Luca-

rini and Ragone, 2011) that are also found in reanalysis constrained by

observations (Trenberth et al., 2009). When correcting for the imperfect

closure of the energy cycle, as done here, comparison between models

and observational estimates become possible.

9.4.2.5 Simulation of Tropical Mean State

9.4.2.5.1 Tropical Paciﬁc Ocean

Although the basic east–west structure of the tropical Paciﬁc is well

captured, models have shown persistent biases in important proper-

ties of the mean state (AchutaRao and Sperber, 2002; Randall et al.,

2007; Guilyardi et al., 2009b) with severe local impacts (Brown et al.,

2012). Among these biases are the mean thermocline depth and slope

along the equator, the structure of the equatorial current system, and

the excessive equatorial cold tongue (Reichler and Kim, 2008; Brown

et al., 2010a; Zheng et al., 2012). Many reasons for these biases have

been proposed, such as: too strong trade winds; a too diffusive ther-

mocline; deﬁcient horizontally isotropic mixing coefﬁcients; insufﬁcient

penetration of solar radiation; and too weak tropical instability waves

(Meehl et al., 2001; Wittenberg et al., 2006; Lin, 2007). It is noteworthy

that CMIP5 models exhibit some improvements in the western equato-

rial Paciﬁc when compared to CMIP3, with reduced SST and trade wind

errors (Figures 9.14 and 9.20). Because of strong interactions between

the processes involved, it is difﬁcult to identify the ultimate source of

these errors, although new approaches using the rapid adjustment of

initialized simulations hold promise (Vannière et al., 2011).

A particular problem in simulating the seasonal cycle in the tropical

Paciﬁc arises from the ‘double ITCZ’, deﬁned as the appearance of a

spurious ITCZ in the SH associated with excessive tropical precipita-

tion. Further problems are too strong a seasonal cycle in simulated

SST and winds in the eastern Paciﬁc and the appearance of a spurious

semi-annual cycle. The latter has been attributed to meridional asym-

metry in the background state that is too weak, possibly in conjunc-

tion with incorrect regional water vapour feedbacks (Li and Philander,

1996; Guilyardi, 2006; Timmermann et al., 2007; De Szoeke and Xie,

2008; Wu et al., 2008a; Hirota et al., 2011).

787

Evaluation of Climate Models Chapter 9

A further persistent problem is insufﬁcient marine stratocumulus cloud

in the eastern tropical Paciﬁc, caused presumably by weak coastal

upwelling off South America leading to a warm SST bias (Lin, 2007).

Although the problem persists, improvements are being made (Achuta-

Rao and Sperber, 2006).

9.4.2.5.2 Tropical Atlantic Ocean

CMIP3 and CMIP5 models exhibit severe biases in the tropical Atlantic

Ocean, so severe that some of the most fundamental features—the

east–west SST gradient and the eastward shoaling thermocline along

the equator—cannot be reproduced (Figure 9.14; (Chang et al., 2007;

Chang et al., 2008; Richter and Xie, 2008; Richter et al., 2013). In many

models, the warm SST bias along the Benguela coast is in excess of

5°C and the Atlantic warm pool in the western basin is grossly under-

estimated (Liu et al., 2013a). As in the Paciﬁc, CMIP3 models suffer the

double ITCZ error in the Atlantic. Hypotheses for the complex Atlantic

bias problem tend to draw on the fact that the Atlantic Ocean has a far

smaller basin, and thus encourages a tighter and more complex land–

atmosphere–ocean interaction. A recent study using a high-resolution

coupled model suggests that the warm eastern equatorial Atlantic SST

bias is more sensitive to the local rather than basin-wide trade wind

bias and to a wet Congo basin instead of a dry Amazon—a ﬁnding

that differs from previous studies (Patricola et al., 2012). Recent ocean

model studies show that a warm subsurface temperature bias in the

eastern equatorial Atlantic is common to virtually all ocean models

forced with ‘best estimated’ surface momentum and heat ﬂuxes, owing

to problems in parameterization of vertical mixing (Hazeleger and

Haarsma, 2005). Toniazzo and Woolnough (2013) show that among a

variety of causes for the initial bias development, ocean–atmosphere

coupling is key for their maintenance.

9.4.2.5.3 Tropical Indian Ocean

CMIP3 and CMIP5 models simulate equatorial Indian Ocean climate

reasonably well (e.g., Figure 9.14), though most models produce weak

westerly winds and a ﬂat thermocline on the equator. The models show

a large spread in the modelled depth of the 20°C isotherm in the east-

ern equatorial Indian Ocean (Saji et al., 2006). The reasons are unclear

but may be related to differences in the various parameterizations of

vertical mixing as well as the wind structure (Schott et al., 2009).

CMIP3 models generally simulate the Seychelles Chagos thermocline

ridge in the Southwest Indian Ocean, a feature important for the Indian

monsoon and tropical cyclone activity in this basin (Xie et al., 2002). The

models, however, have signiﬁcant problems in accurately representing

its seasonal cycle because of the difﬁculty in capturing the asymmetric

nature of the monsoonal winds over the basin, resulting in too weak

a semi-annual harmonic in the local Ekman pumping over the ridge

region compared to observations (Yokoi et al., 2009b). In about half

of the models, the thermocline ridge is displaced eastward associated

with the easterly wind biases on the equator (Nagura et al., 2013).

9.4.2.6 Summary

There is high conﬁdence that the CMIP3 and CMIP5 models simulate

the main physical and dynamical processes at play during transient

ocean heat uptake, sea level rise, and coupled modes of variability.

There is little evidence that CMIP5 models differ signiﬁcantly from

CMIP3, although there is some evidence of modest improvement. Many

improvements are seen in individual CMIP5 ocean components (some

now including interactive ocean biogeochemistry) and the number of

relatively poor-performing models has been reduced (thereby reducing

inter-model spread). New since the AR4, process-based model evalua-

tion is now helping identify the cause of some speciﬁc biases, helping

to overcome the limits set by the short observational records available.

9.4.3 Sea Ice

Evaluation of sea ice performance requires accurate information on ice

concentration, thickness, velocity, salinity, snow cover and other fac-

tors. The most reliably measured characteristic of sea ice remains sea

ice extent (usually understood as the area covered by ice with a con-

centration above 15%). Caveats, however, exist related to the uneven

reliability of different sources of sea ice extent estimates (e.g., satellite

vs. pre-satellite observations; see Chapter 4), as well as to limitations of

this characteristic as a metric of model performance (Notz et al., 2013).

Sea ice extent (10

)Sea ice extent (10

)

Figure 9.22 | Mean (1980–1999) seasonal cycle of sea ice extent (the ocean area with

a sea ice concentration of at least 15%) in the Northern Hemisphere (upper) and the

Southern Hemisphere (lower) as simulated by 42 CMIP5 and 17 CMIP3 models. Each

model is represented with a single simulation. The observed seasonal cycles (1980–

1999) are based on the Hadley Centre Sea Ice and Sea Surface Temperature (HadISST;

Rayner et al., 2003), National Aeronautics and Space Administration (NASA; Comiso

and Nishio, 2008) and the National Snow and Ice Data Center (NSIDC; Fetterer et al.,

2002) data sets. The shaded areas show the inter-model standard deviation for each

ensemble. (Adapted from Pavlova et al., 2011.)

788

Chapter 9 Evaluation of Climate Models

Figure 9.23 | Sea ice distribution (1986–2005) in the Northern Hemisphere (upper

panels) and the Southern Hemisphere (lower panels) for February (left) and Septem-

ber (right). AR5 baseline climate (1986–2005) simulated by 42 CMIP5 AOGCMs. Each

model is represented with a single simulation. For each 1° × 1° longitude-latitude grid

cell, the ﬁgure indicates the number of models that simulate at least 15% of the area

covered by sea ice. The observed 15% concentration boundaries (red line) are based on

the Hadley Centre Sea Ice and Sea Surface Temperature (HadISST) data set (Rayner et

al., 2003). (Adapted from Pavlova et al., 2011.)

The CMIP5 multi-model ensemble exhibits improvements over CMIP3

in simulation of sea ice extent in the both hemispheres (Figure 9.22).

In the Arctic, the multi-model mean error do not exceed 10% of the

observationally based estimates for any given month. In the Antarc-

tic, the corresponding multi-model mean error exceeds 10% (but is

less than 20%) near the annual minimum of sea ice extent; around

the annual maximum, the CMIP5 multi-model mean shows a clear

improvement over CMIP3.

In many models the regional distribution of sea ice concentration is

poorly simulated, even if the hemispheric extent is approximately cor-

rect. In Figure 9.23, however, one can see that the median ice edge

position (indicated by the colour at which half of the models have ice

of 15% concentration) agrees reasonably well with observations in

both hemispheres (except austral summer in Antarctica), as was the

case for the CMIP3 models.

A widely discussed feature of the CMIP3 models as a group is a pro-

nounced underestimation of the trend in the September (annual min-

imum) sea ice extent in the Arctic over the past several decades (e.g.,

Stroeve et al., 2007; Zhang, 2010; Rampal et al., 2011; Winton, 2011).

Possible reasons for the discrepancy include variability inherent to high

latitudes, model shortcomings, and observational uncertainties (e.g.,

Kattsov et al., 2010; Kay et al., 2011; Day et al., 2012). Compared to

CMIP3, the CMIP5 models better simulate the observed trend of Sep-

tember Arctic ice extent (Figure 9.24). It has been suggested (Stroeve

et al., 2012) that in some cases model improvements, such as new sea

ice albedo parameterization schemes (e.g., Pedersen et al., 2009; Hol-

land et al., 2012), have been responsible. (Holland et al., 2010) show

that models with initially thicker ice generally retain more extensive ice

throughout the 21st century, and indeed several of the CMIP5 models

start the 20th century with rather thin winter ice cover promoting more

rapid melt (Stroeve et al., 2012). Notz et al. (2013) caution, however,

against direct comparison of modelled trends with observations unless

the models’ internal variability is carefully taken into account. Their

analysis of the MPI-ESM ensemble shows that internal variability in

the Arctic can result in individual model realizations exhibiting a range

of trends (negative, or even positive) for the 29-year-long period start-

ing in 1979, even if the background climate is warming. According to

the distribution of sea ice extent trends over the period 1979–2010

obtained in an ensemble of simulations with individual CMIP5 models

(Figure 9.24) about one quarter of the simulations shows a September

trend in the Arctic as strong as, or stronger, than in observations.

The majority of CMIP5 (and CMIP3) models exhibit a decreasing trend

in SH austral summer sea ice extent over the satellite era, in contrast

to the observed weak but signiﬁcant increase (see Chapter 4). A large

spread in the modelled trends is present, and a comparison of multi-

ple ensemble members from the same model suggests large internal

variability during the late 20th century and the ﬁrst decade of the 21st

century (e.g., Landrum et al., 2012; Zunz et al., 2013). Compared to

observations, CMIP5 models strongly overestimate the variability of

sea ice extent, at least in austral winter (Zunz et al., 2013).Therefore,

using the models to assess the potential role of the internal variability

in the trend of sea ice extent in the Southern Ocean over the last three

decades presents a signiﬁcant challenge.

Sea ice is a product of atmosphere–ocean interaction. There are a

number of ways in which sea ice is inﬂuenced by and interacts with the

atmosphere and ocean, and some of these feedbacks are still poorly

quantiﬁed. As noted in the AR4, among the primary causes of biases

in simulated sea ice extent, especially its geographical distribution, are

problems with simulating high-latitude winds, ocean heat advection

and mixing. For example, Koldunov et al. (2010) have shown, for a par-

ticular CMIP3 model, that signiﬁcant ice thickness errors originate from

biases in the atmospheric component. Similarly, Melsom et al. (2009)

note sea ice improvements associated with improved description of

heat transport by ocean currents. Biases imparted on modelled sea ice,

common to many models, may also be related to representation of

high-latitude processes (e.g., polar clouds) or processes not yet com-

monly included in models (e.g., deposition of carbonaceous aerosols

on snow and ice). Some CMIP5 models show improvements in simula-

tion of sea ice that are connected to improvements in simulation of the

atmosphere (e.g., Notz et al., 2013).

9.4.3.1 Summary

CMIP5 models reproduce the seasonal cycle of sea ice extent in both

hemispheres. There is robust evidence that the downward trend in

Arctic summer sea ice extent is better simulated than at the time of

the AR4, with about one quarter of the simulations showing a trend

as strong as, or stronger than, that observed over the satellite era. The

performance improvements are not only a result of improvements in

789

Evaluation of Climate Models Chapter 9

(d) CMIP5 Antarctic ice extent February trends (1979 - 2010)

(10

per decade) (10

per decade)

Sea ice extent (10

)Sea ice extent (10

)

Figure 9.24 | (Top and middle rows) Time series of sea ice extent from 1900 to 2012 for (a) the Arctic in September and (b) the Antarctic in February, as modelled in CMIP5

(coloured lines) and observations-based (NASA; Comiso and Nishio, 2008) and NSIDC; (Fetterer et al., 2002), solid and dashed thick black lines, respectively). The CMIP5 multi-

model ensemble mean (thick red line) is based on 37 CMIP5 models (historical simulations extended after 2005 with RCP4.5 projections). Each model is represented with a single

simulation. The dotted black line for the Arctic in (a) relates to the pre-satellite period of observation-based time series (Stroeve et al., 2012). In (a) and (b) the panels on the right

are based on the corresponding 37-member ensemble means from CMIP5 (thick red lines) and 12-model ensemble means from CMIP3 (thick blue lines). The CMIP3 12-model

means are based on CMIP3 historical simulations extended after 1999 with Special Report on Emission Scenarios (SRES) A2 projections. The pink and light blue shadings denote

the 5 to 95 percentile range for the corresponding ensembles. Note that these are monthly means, not yearly minima. (Adapted from Pavlova et al., 2011.) (Bottom row) CMIP5 sea

ice extent trend distributions over the period 1979–2010 for (c) the Arctic in September and (d) the Antarctic in February. Altogether 66 realizations are shown from 26 different

models (historical simulations extended after 2005 with RCP4.5 projections). They are compared against the observations-based estimates of the trends (green vertical lines in

(c) and (d) from Comiso and Nishio (2008); blue vertical line in (d) from Parkinson and Cavalieri (2012)). In (c), the observations-based estimates (Cavalieri and Parkinson, 2012;

Comiso and Nishio, 2008) coincide.

790

Chapter 9 Evaluation of Climate Models

sea ice components themselves but also in atmospheric circulation.

Most CMIP5 models simulate a decrease in Antarctic sea ice extent

over the past few decades compared to the small but signiﬁcant

increase observed.

9.4.4 Land Surface, Fluxes and Hydrology

The land surface determines the partitioning of precipitation into evap-

otranspiration and runoff, and the partitioning of surface net radiation

into sensible and latent heat ﬂuxes. Land surface processes therefore

impact strongly on both the climate and hydrological resources. This

subsection summarizes recent studies on the evaluation of land sur-

face models, wherever possible emphasizing their performance in

CMIP3 and CMIP5 climate models.

9.4.4.1 Snow Cover and Near-Surface Permafrost

The modelling of snow and near-surface permafrost (NSP) processes

has received increased attention since the AR4, in part because of the

recognition that these processes can provide signiﬁcant feedbacks on

climate change (e.g., Koven et al., 2011; Lawrence et al., 2011). The

SnowMIP2 project compared results from 33 snowpack models of vary-

ing complexity, including some snow models that are used in AOGCMs,

using driving data from ﬁve NH locations (Rutter et al., 2009). Most

snow models were found to be consistent with observations at open

sites, but there was much greater discrepancy at forested sites due

to the complex interactions between plant canopy and snow cover.

Despite these difﬁculties, the CMIP5 multi-model ensemble reproduces

key features of the large-scale snow cover (Figure 9.25). In the NH,

models are able to simulate the seasonal cycle of snow cover over

the northern parts of continents, with more disagreement in south-

erly regions where snow cover is sparse, particularly over China and

Mongolia (Brutel-Vuilmet et al., 2013). The latter weaknesses are asso-

ciated with incorrect timing of the snow onset and melt, and possibly

with the choice of thresholds for diagnosing snow cover in the model

output. In spite of the good performance of the multi-model mean,

there is a signiﬁcant inter-model scatter of spring snow cover extent

in some regions. There is a strong linear correlation between North-

ern-Hemisphere spring snow cover extent and annual mean surface

air temperature in the models, consistent with available observations.

The recent negative trend in spring snow cover is underestimated by

the CMIP5 (and CMIP3) models (Derksen and Brown, 2012), which is

associated with an underestimate of the boreal land surface warming

(Brutel-Vuilmet et al., 2013).

Some CMIP5 models now represent NSP and frozen soil process-

es (Koven et al., 2013), but this is not generally the case. Therefore

it is difﬁcult to make a direct quantitative evaluation of most CMIP5

models against permafrost observations. A less direct but more inclu-

sive approach is to diagnose NSP extent using snow depths and skin

temperatures generated by climate models to drive a stand-alone mul-

ti-layer permafrost model (Pavlova et al., 2007). Figure 9.25 shows the

result of using this approach on the CMIP5 ensemble. The multi-model

mean is able to simulate the approximate location of the NSP bound-

ary (as indicated by the 0°C soil temperature isotherm). However, the

range of present-day (1986–2005) NSP area inferred from individu-

al models spans a factor of more than six (~4 to 25 × 10

) due

to differences in simulated surface climate and to varying abilities of

the underlying land surface models. Even though many CMIP5 models

include some representation of soil freezing in mineral soils, very few

include key processes necessary to accurately model NSP changes,

such as the distinct properties of organic soils, the existence of local

water tables and the heat released by microbial respiration (Nicolsky et

al., 2007; Wania et al., 2009; Koven et al., 2011, 2013).

Despite large differences in the absolute NSP area, the relationship

between the decrease in NSP area and the warming air temperature

over the present-day NSP region is similar, and approximately linear, in

many models (Slater and Lawrence, 2013).

9.4.4.2 Soil Moisture and Surface Hydrology

The partitioning of precipitation into evapotranspiration and runoff is

highly dependent on the moisture status of the land surface, especially

the amount of soil moisture available for evapotranspiration, which in

turn depends on properties of the land cover such as the rooting depth

of plants.

There has been a long history of off-line evaluation of land surface

schemes, aided more recently by the increasing availability of site-spe-

ciﬁc data (Friend et al., 2007; Blyth et al., 2010). Throughout this time,

Figure 9.25 | Terrestrial snow cover distribution (1986–2005) in the Northern Hemi-

sphere (NH) as simulated by 30 CMIP5 models for February, updated for CMIP5 from

Pavlova et al. (2007). For each 1° × 1° longitude-latitude grid cell, the ﬁgure indicates

the number of models that simulate at least 5 kg m

–2

of snow-water equivalent. The

observations-based boundaries (red line) mark the territory with at least 20% of the

days per month with snow cover (Robinson and Frei, 2000) over the period 1986–2005.

The annual mean 0°C isotherm at 3.3 m depth averaged across 24 CMIP5 models

(yellow line) is a proxy for the near-surface permafrost boundary. Observed permafrost

extent in the NH (magenta line) is based on Brown et al. (1997, 1998).

791

Evaluation of Climate Models Chapter 9

representations of the land surface have signiﬁcantly increased in

complexity, allowing the representation of key processes such as links

between stomatal conductance and photosynthesis, but at the cost of

increasing the number of poorly known internal model parameters.

These more sophisticated land surface models are based on physical

principles that should make them more appropriate for projections of

future climate and increased CO

However for speciﬁc data-rich sites,

current land surface models still struggle to perform as well as statis-

tical models in predicting year-to-year variations in latent and sensible

heat ﬂuxes (Abramowitz et al., 2008) and runoff (Materia et al., 2010).

There are few evaluations of the performance of land surface schemes

in coupled climate models, but those that have been undertaken ﬁnd

major limitations associated with the atmospheric forcing rather than

the land surface schemes themselves. For example, an evaluation of

the soil moisture simulations of CMIP3 models found that long-term

soil moisture trends could only be reproduced in models that simu-

lated the reduction in solar radiation at the surface associated with

‘global dimming’ (Li et al., 2007). A comparison of simulated evapo-

transpiration ﬂuxes from CMIP3 against large-scale observation-based

estimates, showed underestimates in India and parts of eastern South

America, and overestimates in the western USA, Australia and China

(Mueller et al., 2011).

Land–atmosphere coupling determines the ability of climate models

to simulate the inﬂuence of soil moisture anomalies on rainfall,

droughts and high-temperature extremes (Fischer et al., 2007; Lorenz

et al., 2012). The coupling strength depends both on the sensitivity of

evapotranspiration to soil moisture, which is determined by the land

surface scheme, and the sensitivity of precipitation to evapotranspi-

ration, which is determined by the atmospheric model (Koster et al.,

2004; Seneviratne et al., 2010). Comparison of climate model simu-

lations to observations suggests that the models correctly represent

the soil-moisture impacts on temperature extremes in southeastern

Europe, but overestimate them in central Europe (Hirschi et al., 2011).

The inﬂuence of soil moisture on rainfall varies signiﬁcantly with region,

and with the lead-time between a soil moisture anomaly and a rainfall

event (Seneviratne et al., 2010). In some regions, such as the Sahel,

enhanced precipitation can even be induced by dry anomalies (Taylor

et al., 2011). Recent analyses of CMIP5 models reveals considerable

spread in the ability of the models to reproduce observed correlations

between precipitation and soil moisture in the tropics (Williams et al.,

2012), and a systematic failure to simulate the positive impact of dry

soil moisture anomalies on rainfall in the Sahel (Taylor et al., 2012a).

9.4.4.3 Dynamic Global Vegetation and Nitrogen Cycling

At the time of the AR4 very few climate models included dynamic veg-

etation, with vegetation being prescribed and ﬁxed in all but a handful

of coupled climate–carbon cycle models (Friedlingstein et al., 2006).

Dynamic Global Vegetation Models (DGVMs) certainly existed at the

time of the AR4 (Cramer et al., 2001) but these were not typically

incorporated in climate models. Since the AR4 there has been continual

development of ofﬂine DGVMs, and some climate models incorporate

dynamic vegetation in at least a subset of the runs submitted to CMIP5

(also see Section 9.1.3.2.4), with likely consequences for climate model

biases and regional climate projection (Martin and Levine, 2012).

DGVMs are designed to simulate the large-scale geographical distri-

bution of plant functional types and how these patterns will change

in response to climate change, CO

increases, and other forcing factors

(Cramer et al., 2001). These models typically include rather detailed

representations of plant photosynthesis but less sophisticated treat-

ments of soil carbon, with a varying number of soil carbon pools. In

the absence of nitrogen limitations on CO

fertilization, ofﬂine DGVMs

agree qualitatively that CO

increase alone will tend to enhance

carbon uptake on the land while the associated climate change will

tend to reduce it. There is also good agreement on the degree of CO

fertilization in the case of no nutrient limitation (Sitch et al., 2008).

However, under more extreme emissions scenarios the responses of

the DGVMs diverge markedly. Large uncertainties are associated with

the responses of tropical and boreal ecosystems to elevated tempera-

tures and changing soil moisture status. Particular areas of uncertainty

are the high-temperature response of photosynthesis (Galbraith et al.,

2010), and the extent of CO

fertilization (Rammig et al., 2010) in the

Amazonian rainforest.

Most of the land surface models and DGVMs used in the CMIP5

models continue to neglect nutrient limitations on plant growth (see

Section 6.4.6.2), even though these may signiﬁcantly moderate the

response of photosynthesis to CO

(Wang and Houlton, 2009). Recent

extensions of two land surface models to include nitrogen limitations

improve the ﬁt to ‘Free-Air CO

Enrichment Experiments’, and suggest

that models without these limitations are expected to overestimate

the land carbon sink in the nitrogen-limited mid and high latitudes

(Thornton et al., 2007; Zaehle et al., 2010).

9.4.4.4 Land Use Change

A major innovation in the land component of ESMs since the AR4 is the

inclusion of the effects of land use change associated with the spread

of agriculture, urbanization and deforestation. These affect climate

by altering the biophysical properties of the land surface, such as its

albedo, aerodynamic roughness and water-holding capacity (Bondeau

et al., 2007; Bonan, 2008; Bathiany et al., 2010; Levis, 2010). Land

use change has also contributed almost 30% of total anthropogen-

ic CO

emissions since 1850 (see Table 6.1), and affects emissions of

trace gases, and VOCs such as isoprene. The latest ESMs used in CMIP5

attempt to model the CO

emissions implied by prescribed land use

change and many also simulate the associated changes in the biophys-

ical properties of the land surface. This represents a major advance on

the CMIP3 models which typically neglected land use change, aside

from its assumed contribution to anthropogenic CO

emissions.

However, the increasing sophistication of the modelling of the impacts

of land use change has introduced additional spread in climate model

projections. The ﬁrst systematic model intercomparison demonstrated

that large-scale land cover change can signiﬁcantly affect regional cli-

mate (Pitman et al., 2009) and showed a large spread in the response

of different models to the same imposed land cover change (de Nob-

let-Ducoudre et al., 2012). This uncertainty arises from the often coun-

teracting effects of evapotranspiration and albedo changes (Boisier et

al., 2012) and has consequences for the simulation of temperature and

rainfall extremes (Pitman et al., 2012b).

792

Chapter 9 Evaluation of Climate Models

9.4.5 Carbon Cycle

An important development since the AR4 is the more widespread

implementation of ESMs that include an interactive carbon cycle.

Coupled climate-carbon cycle models are used extensively for the pro-

jections presented in Chapter 12. The evaluation of the carbon cycle

within coupled models is discussed here, while the performance of the

individual land and ocean carbon models, together with the detailed

analysis of climate–carbon cycle feedbacks, is presented in Chapter 6

(Section 6.4 and Box 6.4).

The transition from AOGCMs to ESMs was motivated in part by the

results from the ﬁrst generation coupled climate–carbon cycle models,

which suggested that feedbacks between the climate and the carbon

cycle were uncertain but potentially very important in the context of

21st century climate change (Cox et al., 2000; Friedlingstein et al.,

2001). The ﬁrst-generation models used in the Coupled Climate Carbon

Cycle Model Intercomparison Project (C

MIP) included both extended

AOGCMS and EMICs. The C

MIP experimental design involved running

each model under a common emission scenario (SRES A2) and cal-

culating the evolution of the global atmospheric CO

concentration

interactively within the model. The impacts of climate–carbon cycle

feedbacks were diagnosed by carrying out parallel “uncoupled” sim-

ulations in which increases in atmospheric CO

did not inﬂuence cli-

mate. Analysis of the C

MIP runs highlighted a greater than 200 ppmv

range in the CO

concentration by 2100 due to uncertainties in cli-

mate–carbon cycle feedbacks, and that the largest uncertainties were

associated with the response of land ecosystems to climate and CO

(Friedlingstein et al., 2006).

For CMIP5 a different experimental design was proposed in which the

core simulations use prescribed RCPs of atmospheric CO

and other

GHGs (Moss et al., 2010). Under a prescribed CO

scenario, ESMs calcu-

late land and ocean carbon ﬂuxes interactively, but these ﬂuxes do not

affect the evolution of atmospheric CO

. Instead the modelled land and

ocean ﬂuxes, along with the prescribed increase in atmospheric CO

can be used to diagnose the ‘compatible’ emissions of CO

consistent

with the simulation (see Section 6.3; Miyama and Kawamiya, 2009;

Arora et al., 2011). The compatible emissions for each model can then

be evaluated against the best estimates of the actual historical CO

emissions. Parallel model experiments in which the carbon cycle does

not respond to the simulated climate change (which are equivalent to

the ‘uncoupled’ simulations in C

MIP) provide a means to diagnose

climate–carbon cycle feedbacks in terms of their impact on the com-

patible emissions of CO

(Hibbard et al., 2007).

Carbon cycle model evaluation is limited by the availability of direct

observations at appropriately large spatial scales. Field studies and

eddy-covariance ﬂux measurements provide detailed information on

the land carbon cycle over short-time scales and for speciﬁc locations,

and ocean inventories are able to constrain the long-term uptake of

anthropogenic CO

by the ocean (Sabine et al., 2004; Takahashi et al.,

2009). However the stores of carbon on the land are less well-known,

even though these are important determinants of the CO

ﬂuxes from

land use change. ESM simulations vary by a factor of at least six in

global soil carbon (Anav et al., 2013; Todd-Brown et al., 2013) and by

a factor of four in global vegetation carbon, although about two thirds

of models are within 50% of the uncertain observational estimates

(Anav et al., 2013).

Large-scale land–atmosphere and global atmosphere ﬂuxes are not

directly measured, but global estimates can be made from the carbon

balance, and large-scale regional ﬂuxes can be estimated from the

inversion of atmospheric CO

measurements (see Section 6.3.2). Figure

9.26 shows modelled annual mean ocean–atmosphere and net land–

atmosphere CO

ﬂuxes from the historical simulations in the CMIP5

archive (Anav et al., 2013). Also shown are estimates derived from

ofﬂine ocean carbon cycle models, measurements of atmospheric CO

and best estimates of the CO

ﬂuxes from fossil fuels and land use

change (Le Quere et al., 2009). Uncertainties in these latter annual

estimates are approximately ±0.5 PgC yr

–1

, arising predominantly from

the uncertainty in the model-derived ocean CO

uptake. The conﬁdence

limits for the ensemble mean are derived by assuming that the CMIP5

models form a t-distribution centred on the ensemble mean (Anav et

al., 2013).

The evolution of the global ocean carbon sink is shown in the top

panel of Figure 9.26. The CMIP5 ensemble mean global ocean uptake

(± standard deviation of the multi-model ensemble), computed using

all the 23 models that reported ocean CO

ﬂuxes, increases from 0.47

± 0.32 PgC yr

–1

over the period 1901–1930 to 1.53 ± 0.36 PgC yr

–1

for the period 1960–2005. For comparison, the Global Carbon Project

(GCP) estimates a stronger ocean carbon sink of 1.92 ± 0.3 PgC yr

–1

for

1960–2005 (Anav et al., 2013). The bottom panel of Figure 9.26 shows

the variability in global land carbon uptake evident in the GCP esti-

mates, with the global land carbon sink being strongest during La Niña

years and after volcanoes, and turning into a source during El Niño

years. The CMIP5 models cannot be expected to precisely reproduce

this year-to-year variability as these models will naturally simulate cha-

otic ENSO variability that is out of phase with the historical variability.

However, the ensemble mean does successfully simulate a strengthen-

ing global land carbon sink during the 1990s, especially after the Mt

Pinatubo eruption in 1991. The CMIP5 ensemble mean land–atmos-

phere ﬂux (± standard deviation of the multi-model ensemble) evolves

from a small source of –0.34 ± 0.49 PgC yr

–1

over the period 1901–

1930, predominantly due to land use change, to a sink of 0.47 ± 0.72

PgC yr

–1

in the period 1960–2005. The GCP estimates give a weaker

sink of 0.36 ± 1 PgC yr

–1

for the 1960–2005 period.

Figure 9.27 shows the ocean–atmosphere ﬂuxes (top panel) and mean

land–atmosphere ﬂuxes (bottom panel) simulated by ESMs and EMICs

(Eby et al., 2013) for the period 1986–2005, and compares these

to observation-based estimates from GCP and Atmospheric Tracer

Transport Model Intercomparison Project (TRANSCOM3) atmospheric

inversions (Gurney et al., 2003). Unlike Figure 9.26, only models that

reported both land and ocean carbon ﬂuxes are included in this ﬁgure.

The atmospheric inversions results are taken from the Japanese Meteo-

rological Agency (JMA) as this was the only TRANSCOM3 model that

reported results for all years of the 1986–2005 reference period. The

error bars on the observational estimates (red triangles) and the ESM

simulations (black diamonds) represent the interannual variation in

the form of the standard deviation of the annual ﬂuxes. EMICs do

not typically simulate interannual variability, so only mean values are

shown for these models (green boxes). Here, as in Figure 9.26, the net

793

Evaluation of Climate Models Chapter 9

Year

Atmosphere−Ocean CO

Flux (Pg C yr

-1

)

1900 1915 1930 1945 1960 1975 1990 2005

−1

CMIP5

GCP

Confidence (%)

10 20 30 40 50 60 70 80 90

Sink

Source

Year

Atmosphere−Land CO

Flux (Pg C yr

-1

)

1900 1915 1930 1945 1960 1975 1990 2005

−9

−7

−5

−3

−1

CMIP5

GCP

Confidence (%)

10 20 30 40 50 60 70 80 90

Source

Sink

Figure 9.26 | Ensemble-mean global ocean carbon uptake (top) and global land carbon uptake (bottom) in the CMIP5 ESMs for the historical period 1900–2005. For comparison,

the observation-based estimates provided by the Global Carbon Project (Le Quere et al., 2009) are also shown (thick black line). The conﬁdence limits on the ensemble mean are

derived by assuming that the CMIP5 models come from a t-distribution. The grey areas show the range of annual mean ﬂuxes simulated across the model ensemble. This ﬁgure

includes results from all CMIP5 models that reported land CO

ﬂuxes, ocean CO

ﬂuxes, or both (Anav et al., 2013).

land–atmosphere ﬂux is ‘Net Biome Productivity (NBP)’ which includes

the net CO

emissions from land use change as well as the changing

carbon balance of undisturbed ecosystems.

For the period 1986–2005 the observation-based estimates of the

global ocean carbon sink are 1.71 PgC yr

–1

(JMA), 2.19 PgC yr

–1

(GCP)

and 2.33 PgC yr

–1

(Takahashi et al., 2009). Taking into account the

uncertainties in the mean values of these ﬂuxes associated with inter-

annual variability, the observationally constrained range is approxi-

mately 1.4 to 2.4 PgC yr

–1

. All of the ESMs, and all but one of the

EMICs, simulate ocean sinks within this range. The observation-based

estimates of GCP and JMA agree well on the mean global land carbon

sink over the period 1986–2005, and most ESMs ﬁt within the uncer-

tainty bounds of these estimates (i.e., 1.17 ± 1.06 PgC yr

–1

for JMA).

The exceptions are two ESMs sharing common atmosphere and land

components (CESM1-BGC and NorESM1-ME) which model a net land

carbon source rather than a sink over this period. The EMICs tend to

systematically underestimate the contemporary land carbon sink (Eby

et al., 2013). Some ESMs (notably GFDL-ESM2M and GFDL-ESM2G)

signiﬁcantly overestimate the interannual variation in the global land–

atmosphere CO

ﬂux, with a possible consequence being an overes-

timate of the vulnerability of tropical ecosystems to future climate

change (Cox et al., 2013), and see Figure 9.45). All ESMs qualitatively

simulate the expected pattern of ocean CO

ﬂuxes, with outgassing in

the tropics and uptake in the mid and high latitudes (Anav et al., 2013).

However, there are systematic differences between the ESMs and the

JMA inversion estimates for the zonal land CO

ﬂuxes, with the ESMs

tending to produce weaker uptake in the NH, and simulating a net land

carbon sink rather than a source in the tropics.

In summary, there is high conﬁdence that CMIP5 ESMs can simu-

late the global mean land and ocean carbon sinks within the range

of observation-based estimates. Overall, EMICs reproduce the recent

global ocean CO

ﬂuxes uptake as well as ESMs, but estimate a lower

794

Chapter 9 Evaluation of Climate Models

1.3

1.5

1.7

1.9

2.1

2.3

2.5

a) Global Atmosphere-Ocean CO

Flux

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

GCP

JMA

BNU-ESM

CanESM2

CESM1-BGC

GFDL-ESM2G

GFDL-ESM2M

HadGEM2-CC

HadGEM2-ES

IPSL-CM5A-LR

IPSL-CM5A-MR

IPSL-CM5B-LR

MIROC-ESM-CHEM

MIROC-ESM

MPI-ESM-LR

MPI-ESM-MR

MRI-ESM1

NorESM1-ME

Bern3D

DCESS

GENIE

IGSM

MESMO

UMD

UVic

b) Global Atmosphere-Land CO

Flux

(Pg C yr

-1

)(Pg C yr

-1

)

land carbon sink compared with most ESMs while remaining consist-

ent with the observations (Eby et al., 2013). With few exceptions, the

CMIP5 ESMs also reproduce the large-scale pattern of ocean–atmos-

phere CO

ﬂuxes, with uptake in the Southern Ocean and northern

mid-latitudes, and outgassing in the tropics. However, the geographical

pattern of simulated land–atmosphere ﬂuxes agrees much less well

with inversion estimates, which suggest a larger sink in the northern

mid-latitudes, and a net source rather than a sink in the tropics. While

there are also inherent uncertainties in atmospheric inversions, dis-

crepancies like this might be expected from known deﬁciencies in the

CMIP5 generation of ESMs—namely the failure to correctly simulate

nitrogen fertilization in the mid-latitudes, and a rudimentary treat-

ment of the net CO

emissions arising from land use change and forest

regrowth.

9.4.6 Aerosol Burdens and Effects on Insolation

9.4.6.1 Recent Trends in Global Aerosol Burdens and Effects

on Insolation

The ability of CMIP5 models to simulate the current burden of tropo-

spheric aerosol and the decadal trends in this burden can be assessed

using observations of aerosol optical depth (AOD, see Section 7.3.1.2).

The historical data used to drive the CMIP5 20th century simulations

reﬂect recent trends in anthropogenic SO

emissions, and hence these

trends should be manifested in the modelled and observed AOD. During

the last three decades, anthropogenic emissions of SO

from North

America and Europe have declined due to the imposition of emission

controls, while the emissions from Asia have increased. The combina-

tion of the European, North American, and Asians trends has yielded a

global reduction in SO

emissions of 20 Gg(SO

), or 15% between 1970

and 2000 although emissions subsequently increased by 9 Gg(SO

)

Figure 9.27 | Simulation of global mean (a) atmosphere–ocean CO

ﬂuxes (‘fgCO2’)

and (b) net atmosphere–land CO

ﬂuxes (‘NBP’), by ESMs (black diamonds) and EMICs

(green boxes), for the period 1986–2005. For comparison, the observation-based esti-

mates provided by Global Carbon Project (GCP; Le Quere et al., 2009), and the Japa-

nese Meteorological Agency (JMA) atmospheric inversion (Gurney et al., 2003) are also

shown as the red triangles. The error bars for the ESMs and observations represent

interannual variability in the ﬂuxes, calculated as the standard deviation of the annual

means over the period 1986–2005.

between 2000 and 2005 (Smith et al., 2011b). For the period 2001 to

2005, CMIP5 models underestimate the mean AOD at 550nm relative

to satellite-retrieved AOD by at least 20% over virtually all land surfac-

es (Figure 9.28). The differences between the modelled and measured

AODs exceed the errors in the Multi-angle Imaging Spectro-Radiome-

ter (MISR) retrievals over land of ±0.05 or 0.2×AOD (Kahn et al., 2005)

and the RMS errors in the corrected Moderate Resolution Imaging

Spectrometer (MODIS) retrievals over ocean of 0.061(Shi et al., 2011).

The effects of sulphate and other aerosol species on surface insolation

through direct and indirect forcing appear to be one of the principal

causes of the ‘global dimming’ between the 1960s and 1980s and

subsequent ‘global brightening’ in the last two decades (see Section

2.3.3.1). This inference is supported by trends in aerosol optical depth

and trends in surface insolation under cloud-free conditions. Thirteen

out of 14 CMIP3 models examined by Ruckstuhl and Norris (2009) pro-

duce a transition from “dimming” to ‘brightening’ that is consistent

with the timing of the transition from increasing to decreasing global

anthropogenic aerosol emissions. The transition from ‘dimming’ to

‘brightening’ in both Europe and North America is well simulated with

the HadGEM2 model (Haywood et al., 2011).

Figure 9.28 | (a): Annual mean visible aerosol optical depth (AOD) for 2001 through

2005 using the Moderate Resolution Imaging Spectrometer (MODIS) version5 satellite

retrievals for ocean regions (Remer et al., 2008) with corrections (Zhang et al., 2008a;

Shi et al., 2011) and version31 of MISR retrievals over land (Zhang and Reid, 2010;

Stevens and Schwartz, 2012). (b): The absolute error in visible AOD from the median of a

subset of CMIP5 models’ historical simulations relative to the satellite retrievals of AOD

shown in (a). The model outputs for 2001 through 2005 are from 21 CMIP5 models with

interactive or semi-interactive aerosol representation.

795

Evaluation of Climate Models Chapter 9

Figure 9.29 | Time series of global oceanic mean aerosol optical depth (AOD) from individual CMIP5 models’ historical (1850–2005) and RCP4.5 (2006–2010) simulations, cor-

rected Moderate Resolution Imaging Spectrometer (MODIS) satellite observations by Shi et al. (2011) and Zhang et al. (2008a), and the Atmospheric Chemistry and Climate Model

Intercomparison Project (ACCMIP) simulations for the 1850s by Shindell et al. (2013b).

These recent trends are superimposed on a general upward trend in

aerosol loading since 1850 reﬂected by an increase in global mean

oceanic AODs from the CMIP5 historical and RCP 4.5 simulations from

1850 to 2010 (Figure 9.29). Despite the use of common anthropogenic

aerosol emissions for the historical simulations (Lamarque et al., 2010),

the simulated oceanic AODs for 2010 range from 0.08 to 0.215, with

nearly equal numbers of models over and underestimating the satellite

retrieved AOD of 0.12 (Figure 9.29). This range in AODs results from

differing estimates of the trends and of the initial global mean oceanic

AOD at 1850 across the CMIP5 ensemble (Figure 9.29).

9.4.6.2 Principal Sources of Uncertainty in Projections of

Sulphate Burdens

Natural sources of sulphate from oxidation of dimethylsulphide (DMS)

emissions from the ocean surface are not speciﬁed under the RCP pro-

tocol and therefore represent a source of uncertainty in the sulphur

cycle simulated by the CMIP5 ensemble. In simulations of present-day

conditions, DMS emissions span a 5 to 95% conﬁdence interval of

10.7 to 28.1 TgS yr

–1

(Faloona, 2009). After chemical processing, DMS

contributes between 18 and 42% of the global atmospheric sulphate

burden and up to 80% of the sulphate burden over most the SH (Car-

slaw et al., 2010). Several CMIP5 models include prognostic calculation

of the biogenic DMS source; however, the effects from differences in

DMS emissions on modelled sulphate burdens remain to be quantiﬁed.

In contrast to CMIP3, the models in the CMIP5 ensemble are provid-

ed with a single internally consistent set of future anthropogenic SO

emissions. The use of a single set of emissions removes an important,

but not dominant, source of uncertainty in the AR5 simulations of the

sulphur cycle. In experiments based on a single chemistry–climate

model with perturbations to both emissions and sulphur-cycle process-

es, uncertainties in emissions accounted for 53.3% of the ensemble

variance (Ackerley et al., 2009). The next largest source of uncertain-

ty was associated with the wet scavenging of sulphate (see Section

7.3.2), which accounted for 29.5% of the intra-ensemble variance and

represents the source/sink term with the largest relative range in the

aerosol models evaluated by Faloona (2009). Similarly, simulations run

with heterogeneous or harmonized emissions data sets yielded approx-

imately the same intermodel standard deviation in sulphate burden of

25 Tg. These results show that a dominant source of the spread among

the sulphate burdens is associated with differences in the treatment

of chemical production, transport, and removal from the atmosphere

(Liu et al., 2007; Textor et al., 2007). Errors in modelled aerosol burden

systematically affect anthropogenic RF (Shindell et al., 2013b).

9.5 Simulation of Variability and Extremes

9.5.1 Importance of Simulating Climate Variability

The ability of a model to simulate the mean climate, and the slow,

externally forced change in that mean state, was evaluated in the pre-

vious section. However, the ability to simulate climate variability, both

unforced internal variability and forced variability (e.g., diurnal and

seasonal cycles) is also important. This has implications for the signal-

796

Chapter 9 Evaluation of Climate Models

to-noise estimates inherent in climate change detection and attribu-

tion studies where low-frequency climate variability must be estimat-

ed, at least in part, from long control integrations of climate models

(Section 10.2). It also has implications for the ability of models to

make quantitative projections of changes in climate variability and the

statistics of extreme events under a warming climate. In many cases,

the impacts of climate change will be experienced more profoundly in

terms of the frequency, intensity or duration of extreme events (e.g.,

heat waves, droughts, extreme rainfall events; see Section 12.4). The

ability to simulate climate variability is also central to achieving skill

in climate prediction by initializing models from the observed climate

state (Sections 11.1 and 11.2).

Evaluating model simulations of climate variability also provides a

means to explore the representation of certain processes, such as the

coupled processes underlying the ENSO and other important modes of

variability. A model’s representation of the diurnal or seasonal cycle—

both of which represent responses to external (rotational or orbital)

forcing – may also provide some insight into a model’s ‘sensitivity’ and

by extension, the ability to respond correctly to GHG, aerosol, volcanic

and solar forcing.

9.5.2 Diurnal-to-Seasonal Variability

9.5.2.1 Diurnal Cycles of Temperature and Precipitation

The diurnally varying solar radiation received at a given location

drives, through complex interactions with the atmosphere, land sur-

face and upper ocean, easily observable diurnal variations in surface

and near-surface temperature, precipitation, level stability and winds.

The AR4 noted that climate models simulated the global pattern of

the diurnal temperature range, zonally and annually averaged over the

continent, but tended to underestimate its magnitude in many regions

(Randall et al., 2007). New analyses over land indicate that model

deﬁciencies in surface–atmosphere interactions and the planetary

boundary layer are also expected to contribute to some of the diurnal

cycle errors and that model agreement with observations depends on

region, vegetation type and season (Lindvall et al., 2012). Analyses of

CMIP3 simulations show that the diurnal amplitude of precipitation is

realistic, but most models tend to start moist convection prematurely

over land (Dai, 2006; Wang et al., 2011a). Many CMIP5 models also

have peak precipitation several hours too early compared to surface

observations and TRMM satellite observations (Figure 9.30). This

and the so-called ‘drizzling bias’ (Dai, 2006) can have large adverse

impacts on surface evaporation and runoff (Qian et al., 2006). Over

the ocean, models often rain too frequently and underestimated the

diurnal amplitude (Stephens et al., 2010). It has also been suggested

that a weak diurnal cycle of surface air temperature is produced over

the ocean because of a lack of diurnal variations in SST (Bernie et al.,

2008), and most models have difﬁculty with this due to coarse vertical

resolution and coupling frequency (Dai and Trenberth, 2004; Danaba-

soglu et al., 2006).

Improved representation of the diurnal cycle has been found with

increased atmospheric resolution (Sato et al., 2009; Ploshay and Lau,

2010) or with improved representation of cloud physics (Khairoutdinov

et al., 2005), but the reasons for these improvements remain poorly

understood. Other changes such as the representation of entrainment

in deep convection (Stratton and Stirling, 2012), improved coupling

between shallow and deep convection, and inclusion of density cur-

rents (Peterson et al., 2009) have been shown to greatly improve the

diurnal cycle of convection over tropical land and provide a good rep-

resentation of the timing of convection over land in coupled ocean–

atmosphere simulations (Hourdin et al., 2013). Thanks to improve-

ments like this, the best performing models in Figure 9.30 appear now

to be able to capture the land and ocean diurnal phase and amplitude

quite well.

9.5.2.2 Blocking

In the mid latitudes, climate is often characterized by weather regimes

(see Chapter 2), amongst which blocking regimes play a role in the

occurrence of extreme weather events (Buehler et al., 2011; Sillmann

et al., 2011; Pfahl and Wernli, 2012). During blocking, the prevailing

mid-latitude westerly winds and storm systems are interrupted by a

local reversal of the zonal ﬂow. Climate models in the past have uni-

versally underestimated the occurrence of blocking, in particular in the

Euro-Atlantic sector (Scaife et al., 2010).

There are important differences in methods used to identify blocking

(Barriopedro et al., 2010a), and the diagnosed blocking frequency can

be very sensitive to details such as the choice of latitude (Barnes et al.,

2012). Blocking indices can be sensitive to biases in the representa-

tion of mean state (Scaife et al., 2010) or in variability (Barriopedro

et al., 2010b; Vial and Osborn, 2012). When blocking is measured via

anomaly ﬁelds, rather than reversed absolute ﬁelds, model skill can

be high even in relatively low-resolution models (e.g., Sillmann and

Croci-Maspoli, 2009).

Recent work has shown that models with high horizontal (Matsueda,

2009; Matsueda et al., 2009, 2010) or vertical resolution (Anstey et al.,

2013) are better able to simulate blocking. These improvement arise

from increased representation of orography and atmospheric dynamics

(Woollings et al., 2010b; Jung et al., 2012; Berckmans et al., 2013), as

well as reduced ocean surface temperature errors in the extra tropics

(Scaife et al., 2011). Improved physical parameterizations have also

been shown to improve simulations of blocking (Jung et al., 2010).

However, as in CMIP3 (Scaife et al., 2010; Barnes et al., 2012), most of

the CMIP5 models still signiﬁcantly underestimate winter Euro-Atlantic

blocking (Anstey et al., 2013; Masato et al., 2012; Dunn-Sigouin and

Son, 2013). These new results show that the representation of blocking

events is improving in models, even though the overall quality of CMIP5

ensemble is medium. There is high conﬁdence that model representa-

tion of blocking is improved through increases in model resolution.

9.5.2.3 Madden–Julian Oscillation

During the boreal winter the eastward propagating feature known

as the Madden–Julian Oscillation (MJO; (Madden and Julian, 1972,

1994) predominantly affects the deep tropics, while during the boreal

summer there is also northward propagation over much of southern

Asia (Annamalai and Sperber, 2005). The MJO has received much

attention given the prominent role it plays in tropical climate variabil-

ity (e.g., monsoons, ENSO, and mid-latitudes; Lau and Waliser, 2011)

797

Evaluation of Climate Models Chapter 9

Figure 9.30 | Composite diurnal cycle of precipitation averaged over land (left) and ocean (right) for three different latitude bands at each local time and season (June–July–August

(JJA), December–January–February (DJF), or their sum). For most of the CMIP5 models, data from 1980–2005 from the historical runs were averaged to derived the composite cycle;

however, a few models had the required 3-hourly data only for 1990–2005 or 1996–2005. For comparison with the model results, a similar diagnosis from observations are shown

(black solid line: surface-observed precipitation frequency; black dashed line: TRMM 3B42 data set, 1998–2003 mean). (Update of Figure 17 of Dai, 2006.)

Phenomenological diagnostics (Waliser et al., 2009a) and process-ori-

ented diagnostics (e.g., Xavier, 2012) have been used to evaluate MJO

in climate models. An important reason for model errors in repre-

senting the MJO is that convection parameterizations do not provide

sufﬁcient build-up of moisture in the atmosphere for the large scale

organized convection to occur (Kim et al., 2012; Mizuta et al., 2012).

Biases in the model mean state also contribute to poor MJO simulation

(Inness et al., 2003). High-frequency coupling with the ocean is also

an important factor (Bernie et al., 2008). While new parameterizations

of convection may improve the MJO (Hourdin et al., 2013), this some-

times occurs at the expense of a good simulation of the mean tropical

climate (Kim et al., 2012). In addition, high resolution models with an

improved diurnal cycle do not necessarily produce an improved MJO

(Mizuta et al., 2012).

Most models underestimate the strength and the coherence of convec-

tion and wind variability (Lin et al., 2006; Lin and Li, 2008). The sim-

pliﬁed metric shown in Figure 9.31 provides a synthesis of CMIP3 and

CMIP5 model results (Sperber and Kim, 2012). It shows that simulation

of the MJO is still a challenge for climate models (Lin et al., 2006; Kim

et al., 2009; Xavier et al., 2010). Most models have weak coherence in

their MJO propagation (smaller maximum positive correlation). Even

so, relative to CMIP3 there has been improvement in CMIP5 in simu-

lating the eastward propagation of boreal winter MJO convection from

798

Chapter 9 Evaluation of Climate Models

the Indian Ocean into the western Paciﬁc (Hung et al., 2013) and north-

ward propagation during boreal summer (Sperber et al., 2012). In addi-

tion there is evidence that models reproduce MJO characteristics in the

east Paciﬁc (Jiang et al., 2012b), and that, overall, there is improvement

compared to previous generations of climate models (Waliser et al.,

2003; Lin et al., 2006; Sperber and Annamalai, 2008).

9.5.2.4 Large-Scale Monsoon Rainfall and Circulation

Monsoons are the dominant modes of annual variation in the tropics

(Trenberth et al., 2000; Wang and Ding, 2008), and affect weather and

climate in numerous regions (Chapter 14). High-ﬁdelity simulation of

the mean monsoon and its variability is of great importance for simulat-

ing future climate impacts (Wang, 2006; Sperber et al., 2010; Colman et

al., 2011; Turner and Annamalai, 2012). The monsoon is characterized by

an annual reversal of the low level winds and well deﬁned dry and wet

seasons (Wang and Ding, 2008), and its variability is primarily connect-

ed to the MJO and ENSO (Section 9.5.3). The AR4 reported that most

CMIP3 models poorly represent the characteristics of the monsoon and

monsoon teleconnections (Randall et al., 2007), though improvement in

CMIP5 has been noted for the mean climate, seasonal cycle, intrasea-

sonal and interannual variability (Sperber et al., 2012).

Figure 9.31 | (a, b) The two leading Empirical Orthogonal Functions (EOFs) of outgoing longwave radiation (OLR) from years of strong Madden–Julian Oscillation (MJO) variability

computed following Sperber (2003). The 20- to 100-day ﬁltered OLR from observations and each of the CMIP5 historical simulations and the CMIP3 simulations of 20th century

climate is projected on these two leading EOFs to obtain MJO Principal Component time series. The scatterplot (c) shows the maximum positive correlation between the resulting

MJO Principal Components and the time lag at which it occurred for all winters (November to March). The maximum positive correlation is an indication of the coherence with

which the MJO convection propagates from the Indian Ocean to the Maritime Continent/western Paciﬁc, and the time lag is approximately one fourth of the period of the MJO.

(Constructed following Sperber and Kim, 2012.)

The different monsoon systems are connected through the large-scale

tropical circulation, offering the possibility to evaluate a models’ rep-

resentation of monsoon domain and intensity through the global mon-

soon concept (Wang and Ding, 2008; Wang et al., 2011a). The CMIP5

multi-model ensemble generally reproduces the observed spatial pat-

terns but somewhat underestimates the extent and intensity, especial-

ly over Asia and North America (Figure 9.32). The best model has simi-

lar performance to the multi-model mean, whereas the poorest models

fail to capture the monsoon precipitation domain and intensity over

Asia and the western Paciﬁc, Central America, and Australia. Fan et al.

(2010) also show that CMIP3 simulations capture the observed trend

of weakening of the South Asian summer circulation over the past half

century, but are unable to reproduce the magnitude of the observed

trend in precipitation. On longer time scales, mid-Holocene simulations

show that even though models capture the sign of the monsoon pre-

cipitation changes, they tend to underestimate its magnitude (Bracon-

not et al., 2007b; Zhao and Harrison, 2012)

Poor simulation of the monsoon has been attributed to cold SST biases

over the Arabian Sea (Levine and Turner, 2012), a weak meridional

temperature gradient (Joseph et al., 2012), unrealistic development

of the Indian Ocean dipole (Achuthavarier et al., 2012; Boschat et al.,

799

Evaluation of Climate Models Chapter 9

2012) and changes to the circulation through excessive precipitation

over the southwest equatorial Indian Ocean (Bollasina and Ming,

2013). These biases lead to too weak inland moisture transport and an

underestimate of monsoon precipitation over India. Similar SST biases

contribute to model-data mismatch in the simulation of the mid-Hol-

ocene Asian monsoon (Ohgaito and Abe-Ouchi, 2009), even though

the representation of atmospheric processes such as convection seems

to dominate the model spread in this region (Ohgaito and Abe-Ouchi,

2009) or over Africa (Zheng and Braconnot, 2013). Factors that have

contributed to improved representation of the monsoon in some

CMIP5 models include better simulation of topography-related mon-

soon precipitation due to higher horizontal resolution (Mizuta et al.,

2012), a more realistic ENSO–monsoon teleconnection (Meehl et al.,

2012) and improved propagation of intraseasonal variations (Sperber

and Kim, 2012). The impact of aerosols on monsoon precipitation and

its variability is the subject of ongoing investigation (Lau et al., 2008).

These results provide robust evidence that CMIP5 models simulate

more realistic monsoon climatology and variability than their CMIP3

predecessors, but they still suffer from biases in the representation of

the monsoon domain and intensity leading to medium model quality at

the global scale and declining quality at the regional scale.

9.5.3 Interannual-to-Centennial Variability

In addition to the annual, intra-seasonal and diurnal cycles described

above, a number of other modes of variability arise on multi-annual to

multi-decadal time scales (see also Box 2.5). Most of these modes have

a particular regional manifestation whose amplitude can be larger

Figure 9.32 | Monsoon precipitation intensity (shading, dimensionless) and monsoon precipitation domain (lines) are shown for (a) observation-based estimates from Global Pre-

cipitation Climatology Project (GPCP), (b) the CMIP5 multi-model mean, (c) the best model and (d) the worst model in terms of the threat score for this diagnostic. These measures

are based on the seasonal range of precipitation using hemispheric summer (May through September in the Northern Hemisphere (NH)) minus winter (November through March in

the NH) values. The monsoon precipitation domain is deﬁned where the annual range is >2.5 mm day

–1

, and the monsoon precipitation intensity is the seasonal range divided by

the annual mean. The threat scores (Wilks, 1995) indicate how well the models represent the monsoon precipitation domain compared to the GPCP data. The threat score in panel

(a) is between GPCP and CMAP rainfall to indicate observational uncertainty, whereas in the other panel it is between the simulations and the GPCP observational data set. A threat

score of 1.0 would indicate perfect agreement between the two data sets. See Wang and Ding (2008),Wang et al. (2011a), and Kim et al. (2011) for details of the calculations.

than that of human-induced climate change. The observational record

is usually too short to fully evaluate the representation of variability

in models and this motivates the use of reanalysis or proxies, even

though these have their own limitations.

9.5.3.1 Global Surface Temperature Multi-Decadal Variability

The AR4 concluded that modelled global temperature variance at dec-

adal to inter-decadal time scales was consistent with 20th century

observations. In addition, results from the last millennium suggest that

simulated variability is consistent with indirect estimates (Hegerl et

al., 2007).

Figure 9.33a shows simulated internal variability of mean surface tem-

perature from CMIP5 pre-industrial control simulations. Model spread

is largest in the tropics and mid to high latitudes (Jones et al., 2012),

where variability is also large; however, compared to CMIP3, the spread

is smaller in the tropics owing to improved representation of ENSO var-

iability (Jones et al., 2012). The power spectral density of global mean

temperature variance in the historical simulations is shown in Figure

9.33b and is generally consistent with the observational estimates.

At longer time scale of the spectra estimated from last millennium

simulations, performed with a subset of the CMIP5 models, can be

assessed by comparison with different NH temperature proxy records

(Figure 9.33c; see Chapter 5 for details). The CMIP5 millennium sim-

ulations include natural and anthropogenic forcings (solar, volcanic,

GHGs, land use) (Schmidt et al., 2012). Signiﬁcant differences between

unforced and forced simulations are seen for time scale larger than

800

Chapter 9 Evaluation of Climate Models

Figure 9.33 | Global climate variability as represented by: (a) Standard deviation of zonal-mean surface temperature of the CMIP5 pre-industrial control simulations (after Jones

et al., 2012). (b) Power spectral density for 1901–2010 global mean surface temperature for both historical CMIP5 simulations and the observations (after Jones et al., 2012). The

grey shading provides the 5 to 95% range of the simulations. (c) Power spectral density for Northern Hemisphere surface temperature from the CMIP5/ Paleoclimate Modelling

Intercomparison Project version 3 (PMIP3) last-millennium simulations (colour, solid) using common external forcing conﬁgurations (Schmidt et al., 2012), together with the cor-

responding pre-industrial simulations (colour, dashed), previous last-millennium AOGCM simulations (black: Fernandez-Donado et al., 2013), and temperature reconstructions from

different proxy records (see Section 5.3.5). For comparison between model results and proxy records, the spectra in (c) have been computed from normalized Northern Hemisphere

time series. The small panel included in the bottom panel shows for the different models and reconstructions the percentage of spectral density cumulated for periods longer than

50 years, to highlight the differences between unforced (pre-industrial control) and forced (PMIP3 and pre-PMIP3) simulations, compared to temperature reconstruction for the

longer time scales. In (b) and (c) the spectra have been computed using a Tukey–Hanning ﬁlter of width 97 and 100 years, respectively. The model outputs were not detrended,

except for the MIROC-ESM millennium simulation. The 5 to 95% intervals (vertical lines) provide the accuracy of the power spectra estimated given a typical length of 110 years

for (b) and 1150 years for (c).

801

Evaluation of Climate Models Chapter 9

50 years, indicating the importance of forced variability at these time

scales (Fernandez-Donado et al., 2013). It should be noted that a few

models exhibit slow background climate drift which increases the

spread in variance estimates at multi-century time scales. Nevertheless,

the lines of evidence above suggest with high conﬁdence that models

reproduce global and NH temperature variability on a wide range of

time scales.

9.5.3.2 Extratropical Circulation, North Atlantic Oscillation

and Other Related Dipolar and Annular Modes

Based on CMIP3 models, Gerber et al. (2008) conﬁrmed the AR4 assess-

ment that climate models are able to capture the broad spatial and

temporal features of the North Atlantic Oscillation (NAO), but there

are substantial differences in the spatial patterns amongst individual

models (Casado and Pastor, 2012; Handorf and Dethloff, 2012). Climate

models tend to have patterns of variability that are more annular in

character than observed (Xin et al., 2008). Models substantially overes-

timate persistence on sub-seasonal and seasonal time scales, and have

difﬁculty simulating the seasonal cycle of annular mode time scales

found in reanalyses (Gerber et al., 2008). The unrealistically long time

scale of variability is worse in models with particularly strong equator-

ward biases in the mean jet location, a result which has been found to

hold in the North Atlantic and in the SH (Barnes and Hartmann, 2010;

Kidston and Gerber, 2010).

As described in the AR4, climate models have generally been unable

to simulate changes as strong as the observed NAO trend over the

period 1965–1995, either in coupled mode (Gillett, 2005; Stephenson

et al., 2006; Stoner et al., 2009) or forced with observed boundary con-

ditions (Scaife et al. (2009). However, there are a few exceptions to

this (e.g., Selten et al., 2004; Semenov et al., 2008), so it is unclear to

what extent the underestimation of late 20th century trends reﬂects

model shortcomings versus internal variability. Further evidence has

emerged of the coupling of NAO variability between the troposphere

and the stratosphere, and even models with improved stratospheric

resolution appear to underestimate the vertical coupling (Morgenstern

et al., 2010) with consequences for the NAO response to anthropogen-

ic forcing (Sigmond and Scinocca, 2010; Karpechko and Manzini, 2012;

Scaife et al., 2012).

The Paciﬁc basin analogue of the NAO, the North Paciﬁc Oscillation

(NPO) is a prominent pattern of wintertime atmospheric circulation

variability characterized by a north–south dipole in sea level pressure

(Linkin and Nigam, 2008). Although climate models simulate the main

spatial features of the NPO, many of them are unable to capture the

observed linkages with tropical variability and the ocean (Furtado et

al., 2011).

Raphael and Holland (2006) showed that coupled models produce a

clear Southern Annular Mode (SAM) but that there are relatively large

differences between models in terms of the exact shape and orien-

tation of this pattern. Karpechko et al. (2009) found that the CMIP3

models have problems representing linkages between the SAM and

SST, surface air temperature, precipitation and particularly sea ice in

the Antarctic region.

9.5.3.3 Atlantic Modes

9.5.3.3.1 Atlantic Meridional Overturning Circulation variability

Previous comparisons of the observed and simulated AMOC were

restricted to its mean strength, as it had only been sporadically

observed (see Chapter 3 and Section 9.4.2.3.1). Continuous AMOC

time series now exist for latitudes 41°N (reconstructions since 1993)

and 26.5°N (estimate based on direct observations since 2004) (Cun-

ningham et al., 2010; Willis, 2010). At 26.5°N, CMIP3 and CMIP5 model

simulations show total AMOC variability that is within the observa-

tional uncertainty (Baehr et al., 2009; Marsh et al., 2009; Balan Saro-

jini et al., 2011; Msadek et al., 2013). However, the total AMOC is the

sum of a wind-driven component and a mid-ocean geostrophic com-

ponent. While both CMIP3 and CMIP5 models tend to overestimate

the wind-driven variability, they tend to underestimate the mid-ocean

geostrophic variability (Baehr et al., 2009; Balan Sarojini et al., 2011;

Msadek et al., 2013). The latter is suggested to result from deﬁcien-

cies in the simulation of the hydrographic characteristics (Baehr et al.,

2009), speciﬁcally the Nordic Seas overﬂows (Yeager and Danaba-

soglu, 2012; Msadek et al., 2013).

9.5.3.3.2 Atlantic multi-decadal variability/Atlantic

Multi-decadal Oscillation

The Atlantic Multi-decadal Variability (AMV), also known as Atlantic

Multi-decadal Oscillation (AMO), is a mode of climate variability with

an apparent period of about 70 years, and a pattern centred in the

North Atlantic Ocean (see Section 14.7.6). In the AR4, it was shown

that a number of climate models produced AMO-like multidecadal var-

iability in the North Atlantic linked to variability in the strength of the

AMOC. Subsequent analyses has conﬁrmed this, however simulated

time scales range from 40 to 60 years (Frankcombe et al., 2010; Park

and Latif, 2010; Kavvada et al., 2013), to a century or more (Msadek

and Frankignoul, 2009; Menary et al., 2012). In addition, the spatial

patterns of variability related to the AMOC differ in many respects from

one model to another as shown in Figure 9.34.

The presence of AMO-like variability in unforced simulations, and the

fact that forced 20th century simulations in the CMIP3 multi-mod-

el ensemble produce AMO variability that is not in phase with that

observed, implies the AMO is not predominantly a result of the forc-

ings imposed on the models (Kravtsov and Spannagle, 2008; Knight,

2009; Ting et al., 2009). Results from the CMIP5 models also show a

key role for internal variability, alongside a contribution from external

forcings in recent decades (Terray, 2012). Historical AMO ﬂuctuations

have been better reproduced in a model with a more sophisticated aer-

osol treatment than was typically used in CMIP3 (Booth et al., 2012a),

albeit at the expense of introducing other observational inconsisten-

cies (Zhang et al., 2013). This could suggest that at least part of the

AMO may in fact be forced, and that aerosols play a role. In addition

to tropospheric aerosols, Otterå et al. (2010) showed the potential for

simulated volcanic forcing to have inﬂuenced AMO ﬂuctuations over

the last 600 years.

802

Chapter 9 Evaluation of Climate Models

( )

(

)

(

)

9.5.3.3.3 Tropical zonal and meridional modes

The Atlantic Meridional Mode (AMM) is the dominant mode of inter-

annual variability in the tropical Atlantic, is characterized by an anom-

alous meridional shift in the ITCZ (Chiang and Vimont, 2004), and has

impacts on hurricane tracks over the North Atlantic (Xie et al., 2005;

Smirnov and Vimont, 2011). Virtually all CMIP models simulate AMM-

like SST variability in their 20th century climate simulations. However,

Figure 9.34 | Sequence of physical links postulated to connect Atlantic Meridional Overturning Circulation (AMOC) and Atlantic Multi-decadal Variability (AMV), and how they are

represented in three climate models. Shown are regression patters for the following quantities (from top to bottom): sea surface temperature (SST) composites using AMOC time

series; precipitation composites using cross-equatorial SST difference time series; equatorial salinity composites using Intertropical Convergence Zone (ITCZ)-strength time series;

subpolar-gyre depth-averaged salinity (top 800 to 1000 m) using equatorial salinity time series; subpolar gyre depth averaged density using subpolar gyre depth averaged salinity

time series. From left to right: the two CMIP3 models HadCM3 and ECHAM/MPI-OM (MPI), and the non-CMIP model KCM. Black outlining signiﬁes areas statistically signiﬁcant at

the 5% level for a two-tailed t test using the moving-blocks bootstrapping technique (Wilks, 1995). (Figure 3 from Menary et al., 2012.)

most models underestimate the SST variance associated with the

AMM, and position the north tropical Atlantic SST anomaly too far

equatorward. More problematic is the fact that the development of

the AMM in many models is led by a zonal mode during boreal win-

ter—a feature that is not observed in nature (Breugem et al., 2006).

This spurious AMM behaviour in the models is expected to be associ-

ated with the severe model biases in simulating the ITCZ (see Section

9.4.2.5.2).

803

Evaluation of Climate Models Chapter 9

Atlantic Niño

CMIP3 models have considerable difﬁculty simulating Atlantic Niño

in their 20th century climate simulations. For many models the so-

called ‘Atl-3’ SST index (20°W to 0°W, 3°S to 3°N) displays the wrong

seasonality, with the maximum value in either DJF or SON instead of

JJA as is observed (Breugem et al., 2006). Despite large biases in the

simulated climatology (Section 9.4.2.5.2), about one third of CMIP5

models capture some aspects of Atlantic Niño variability, including

amplitude, spatial pattern and seasonality (Richter et al., 2013). This

represents an improvement over CMIP3.

9.5.3.4 Indo-Paciﬁc Modes

9.5.3.4.1 El Niño-Southern Oscillation

The ENSO phenomenon is the dominant mode of climate variability on

seasonal to interannual time scales (see Wang and Picaut (2004) and

Chapter 14). The representation of ENSO in climate models has stead-

ily improved and now bears considerable similarity to observed ENSO

properties (AchutaRao and Sperber, 2002; Randall et al., 2007; Guil-

yardi et al., 2009b). However, as was the case in the AR4, simulations

Figure 9.35 | Maximum entropy power spectra of surface air temperature averaged over the NINO3 region (5°N to 5°S, 150°W to 90°W) for (a) the CMIP5 models and (b) the

CMIP3 models. ECMWF reanalysis in (a) refers to the European Centre for Medium Range Weather Forecasts (ECMWF) 15-year reanalysis (ERA15). The vertical lines correspond

to periods of two and seven years. The power spectra from the reanalyses and for SST from the Hadley Centre Sea Ice and Sea Surface Temperature (HadISST) version 1.1, Hadley

Centre/Climatic Research Unit gridded surface temperature data set 4 (HadCRU 4), ECMWF 40-year reanalysis (ERA40) and National Centers for Environmental Prediction/National

Center for Atmospheric Research (NCEP/NCAR) data set are given by the series of black curves. (Adapted from AchutaRao and Sperber, 2006.)

804

Chapter 9 Evaluation of Climate Models

Figure 9.36 | ENSO metrics for pre-industrial control simulations in CMIP3 and CMIP5. (a) and (b): SST anomaly standard deviation (°C) in Niño 3 and Niño 4, respectively, (c)

precipitation response (standard deviation, mm/day) in Niño4. Reference data sets, shown as dashed lines: Hadley Centre Sea Ice and Sea Surface Temperature (HadISST) version

1.1 for (a) and (b), CPC Merged Analysis of Precipitation (CMAP) for (c). The CMIP5 and CMIP3 multi-model means are shown as squares on the left of each panel with the whiskers

representing the model standard deviation. Individual CMIP3 models shown as ﬁlled grey circles, and individual CMIP5 models are identiﬁed in the legend.

CMIP5

CMIP

BCC

BCCR

BNU

CCCma

CMCC

CNRM

CSIRO

FIO

GFDL

GISS

IAP

IPSL

MIROC

INM

MIUB

MOHC

MPI

MRI

NCAR

NCC

NSF-DO

INGV

Obs.

per

(

)

(°C) (°C)

of both background climate (time mean and seasonal cycle, see Section

9.4.2.5.1) and internal variability exhibit serious systematic errors (van

Oldenborgh et al., 2005; Capotondi et al., 2006; Guilyardi, 2006; Wit-

tenberg et al., 2006; Watanabe et al., 2011; Stevenson et al., 2012; Yeh

et al., 2012), many of which can be traced to the representation of

deep convection, trade wind strength and cloud feedbacks, with little

improvement from CMIP3 to CMIP5 (Braconnot et al., 2007a; L’Ecuyer

and Stephens, 2007; Guilyardi et al., 2009a; Lloyd et al., 2009, 2010;

Sun et al., 2009; Zhang and Jin, 2012).

While a number of CMIP3 models do not exhibit an ENSO variability

maximum at the observed 2- to 7- year time scale, most CMIP5 models

do have a maximum near the observed range and fewer models have

the tendency for biennial oscillations (Figure 9.35; see also Stevenson,

2012). In CMIP3 the amplitude of El Niño ranged from less than half

to more than double the observed amplitude (van Oldenborgh et al.,

2005; AchutaRao and Sperber, 2006; Guilyardi, 2006; Guilyardi et al.,

2009b). By contrast, the CMIP5 models show less inter-model spread

(Figure 9.36; Kim and Yu, 2012). The CMIP5 models still exhibit errors

in ENSO amplitude, period, irregularity, skewness, spatial patterns (Lin,

2007; Leloup et al., 2008; Guilyardi et al., 2009b; Ohba et al., 2010; Yu

and Kim, 2011; Su and Jiang, 2012) or teleconnections (Watanabe et

al., 2012; Weller and Cai, 2013a).

Since AR4, new analysis methods have emerged and are now being

applied. For example, Jin et al. (2006) and Kim and Jin (2011a) iden-

tiﬁed ﬁve different feedbacks affecting the Bjerknes (or BJ) index,

which in turn characterizes ENSO stability. Kim and Jin (2011b) applied

this process-based analysis to the CMIP3 multi-model ensemble and

demonstrated a signiﬁcant positive correlation between ENSO ampli-

tude and the BJ index. When respective components of the BJ index

obtained from the coupled models were compared with those from

observations, it was shown that most coupled models underestimated

the negative thermal damping feedback (Lloyd et al., 2012; Chen et

al., 2013) and the positive zonal advective and thermocline feedbacks.

Detailed quantitative evaluation of ENSO performance is hampered by

the short observational record of key processes (Wittenberg, 2009; Li

et al., 2011b; Deser et al., 2012) and the complexity and diversity of

the processes involved (Wang and Picaut, 2004). While shortcomings

805

Evaluation of Climate Models Chapter 9

remain (Guilyardi et al., 2009b), the CMIP5 model ensemble shows

some improvement compared to CMIP3, but there has been no major

breakthrough and the multi-model improvement is mostly due to a

reduced number of poor-performing models.

9.5.3.4.2 Indian Ocean basin and dipole modes

Indian Ocean SST displays a basin-wide warming following El Niño

(Klein et al., 1999). This Indian Ocean basin (IOB) mode peaks in boreal

spring and persists through the following summer. Most CMIP5 models

capture this IOB mode, an improvement over CMIP3 (Du et al., 2013).

However, only about half the CMIP5 models capture its long tempo-

ral persistence, and these models tend to simulate ENSO-forced ocean

Rossby waves in the tropical south Indian Ocean (Zheng et al., 2011).

The Indian Ocean zonal dipole mode (IOD) (Saji et al., 1999; Webster

et al., 1999) appears to be part of a hemispheric response to tropi-

cal atmospheric forcing (Fauchereau et al., 2003; Hermes and Reason,

2005). Most CMIP3 models are able to reproduce the general features

of the IOD, including its phase lock onto the July to November season

(Saji et al., 2006). The modelled SST anomalies, however, tend to show

too strong a westward extension along the equator in the eastern

Indian Ocean. CMIP3 models exhibit considerable spread in IOD ampli-

tude, some of which can be explained by differences in the strength

of the simulated Bjerknes feedback (Liu et al., 2011; Cai and Cowan,

2013). No substantial change is seen in CMIP5 (Weller and Cai, 2013a).

Many models simulate the observed correlation between IOD and

ENSO. The magnitude of this correlation varies substantially between

models, but is apparently not tied to the amplitude of ENSO (Saji et

al., 2006). A subset of CMIP3 models show a spurious correlation with

ENSO following the decay of ENSO events, instead of during the ENSO

developing phase, possibly due to erroneous representation of oceanic

pathways connecting the equatorial Paciﬁc and Indian Oceans (Cai et

al., 2011).

9.5.3.4.3 Tropospheric biennial oscillation

The tropospheric biennial oscillation (TBO, Section 14.7.4) is a bien-

nial tendency of many phenomena in the Indo-Paciﬁc region that

affects droughts and ﬂoods over large areas of south Asia and Aus-

tralia (e.g., Chang and Li, 2000; Li et al., 2001; Meehl et al., 2003).

The IOD involves regional patterns of SST anomalies in the TBO in the

Indian Ocean during the northern fall season following the south Asian

monsoon (Loschnigg et al., 2003). The TBO has been simulated in a

number of global coupled climate model simulations (e.g., Ogasawara

et al., 1999; Loschnigg et al., 2003; Nanjundiah et al., 2005; Turner et

al., 2007; Meehl and Arblaster, 2011).

9.5.3.5 Indo-Paciﬁc Teleconnections

Tropical SST variability provides a signiﬁcant forcing of atmospheric

teleconnections and drives a large portion of the climate variability

over land (Goddard and Mason, 2002; Shin et al., 2010). Although

local forcings and feedbacks can play an important role (Pitman et

al., 2012a), the simulation of land surface temperatures and precip-

itation requires accurate predictions of SST patterns (Compo and

Sardeshmukh, 2009; Shin et al., 2010) as well as zonal wind variability

patterns (Handorf and Dethloff, 2012). Teleconnections hence play a

central role in regional climate change (see Chapter 14).

9.5.3.5.1 Teleconnections affecting North America

The Paciﬁc North American (PNA) pattern is a wavetrain-like pattern in

mid-level geopotential heights. The majority of CMIP3 models simulate

the spatial structure of the PNA pattern in wintertime (Stoner et al.,

2009). The PNA pattern has contributions from both internal atmos-

pheric variability (Johnson and Feldstein, 2010) and ENSO and PDO

teleconnections (Deser et al., 2004). The power spectrum of this tempo-

ral behaviour is generally captured by the CMIP3 models, although the

level of year-to-year autocorrelation varies according to the strength of

the simulated ENSO and PDO (Stoner et al., 2009).

9.5.3.5.2 Tropical ENSO teleconnections

These moist teleconnection pathways involve mechanisms related to

those at play in the precipitation response to global warming (Chiang

and Sobel, 2002; Neelin et al., 2003) and provide challenging test sta-

tistics for model precipitation response. Compared to earlier genera-

tion climate models, CMIP3 and CMIP5 models tend to do somewhat

better (Neelin, 2007; Cai et al., 2009; Coelho and Goddard, 2009; Lan-

genbrunner and Neelin, 2013) at precipitation reductions associated

with El Niño over equatorial South America and the Western Paciﬁc,

although CMIP5 offers little further improvement over CMIP3 (see for

instance the standard deviation of precipitation in the western Paciﬁc

in Figure 9.36). CMIP5 models simulate the sign of the precipitation

change over broad regions, and do well at predicting the amplitude of

the change (for a given SST forcing) (Langenbrunner and Neelin, 2013).

A regression of the West African monsoon precipitation index with

global SSTs reveals two major teleconnections (Fontaine and Janicot,

1996). The ﬁrst highlights the strong inﬂuence of ENSO, while the

second reveals a relationship between the SST in the Gulf of Guinea

and the northward migration of the monsoon rain belt over West

Africa. Most CMIP3 models show a single dominant Paciﬁc telecon-

nection, which is, however, of the wrong sign for half of the models

(Joly et al., 2007). Only one model shows a signiﬁcant second mode,

emphasizing the difﬁculty in simulating the response of the African

rain belt to Atlantic SST anomalies that are not synchronous with Pacif-

ic anomalies.

Both CMIP3 and CMIP5 models have been evaluated and found to vary

in their abilities to represent both the seasonal cycle of correlations

between the Niño 3.4 and North Australian SSTs (Catto et al., 2012a,

2012b) with little change in quality from CMIP3 to CMIP5. Generally

the models do not capture the strength of the negative correlations

during the second half of the year. The models also still struggle to

capture the SST evolution in the North Australian region during El Niño

and La Niña. Teleconnection patterns from both ENSO and the Indian

Ocean Dipole to precipitation over Australia are reasonably well simu-

lated in the key September-November season (Cai et al., 2009; Weller

and Cai, 2013b) in the CMIP3 and CMIP5 multi-model mean.

806

Chapter 9 Evaluation of Climate Models

9.5.3.6 Paciﬁc Decadal Oscillation and Interdecadal

Paciﬁc Oscillation

The Paciﬁc Decadal Oscillation (PDO) refers to a mode of variabil-

ity involving SST anomalies over the North Paciﬁc (north of 20°N)

(Mantua et al., 1997). Although the PDO time series exhibits consider-

able decadal variability, it is difﬁcult to ascertain whether there are any

robust spectral peaks given the relatively short observational record

(Minobe, 1997, 1999; Pierce, 2001; Deser et al., 2004). The ability of

climate models to represent the PDO has been assessed by Stoner et

al. (2009) and Furtado et al. (2011). Their results indicate that approxi-

mately half of the CMIP3 models simulate the observed spatial pattern

and temporal behaviour (e.g., enhanced variance at low frequencies);

however, spectral peaks are consistently higher in frequency than

those suggested by the short observational record. The modelled PDO

correlation with SST anomalies in the tropical Indo-Paciﬁc are strongly

underestimated by the CMIP3 models (Wang et al., 2010; Deser et al.,

2011; Furtado et al., 2011; Lienert et al., 2011). Climate models have

been shown to simulate features of the closely related Interdecadal

Paciﬁc Oscillation (IPO, based on SSTs over the entire Paciﬁc basin; see

Section 14.7.3; Power and Colman, 2006; Power et al., 2006; Meehl et

al., 2009), although deﬁciencies remain in the strength of the tropical–

extratropical connections.

9.5.3.7 The Quasi-Biennial Oscillation

Signiﬁcant progress has been made in recent years to model and

understand the impacts of the Quasi-Biennial Oscillation (QBO; Bald-

win et al., 2001). Many climate models have now increased their

vertical domain and/or improved their physical parameterizations

(see Tables 9.1 and 9.A.1), and some of these reproduce a QBO (e.g.,

Short

Name

Level of

Conﬁdence

Level of

Evidence for

Evaluation

Degree of

Agreement

Model

Quality

Difference with AR4

(including CMIP5

vs. CMIP3)

Section

Global sea surface tem-

perature (SST) variability

SST-var High Robust Medium Medium Slight improvement in the tropics 9.5.3.1

North Atlantic Oscillation

and Northern Annular Mode

NAO Medium Medium Medium High No assessment 9.5.3.2

Southern Annular Mode SAM Low Limited Medium Medium No assessment 9.5.3.2

Atlantic Meridional Overturn-

ing Circulation Variability

AMOC-var Low Limited Medium Medium No improvement 9.5.3.3

Atlantic Multi-decadal

Variability

AMO Low Limited Medium Medium No improvement 9.5.3.3

Atlantic Meridional Mode AMM High Medium High Low No assessment 9.5.3.3

Atlantic Niño AN Low Limited Medium Low Slight improvement 9.5.3.3

El Niño Southern Oscillation ENSO High Medium High Medium Slight improvement 9.5.3.4

Indian Ocean Basin mode IOB Medium Medium Medium High Slight improvement 9.5.3.4

Indian Ocean Dipole IOD Medium Medium Medium Medium No improvement 9.5.3.4

Paciﬁc North American PNA High Medium High Medium Slight improvement 9.5.3.5

Tropical ENSO tele-

connections

ENSOtele High Robust Medium Medium No improvement 9.5.3.5

Paciﬁc Decadal Oscillation PDO Low Limited Medium Medium No assessment 9.5.3.6

Interdecadal Paciﬁc

Oscillation

IPO Low Limited Medium High No assessment 9.5.3.6

Quasi-Biennial Oscillation QBO Medium Medium Medium High No assessment 9.5.3.7

HadGEM2, MPI-ESM-LR, MIROC). Many features of the QBO such as

its width and phase asymmetry also appear spontaneously in these

simulations due to internal dynamics (Dunkerton, 1991; Scaife et al.,

2002; Haynes, 2006). Some of the QBO effects on the extratropical cli-

mate (Holton and Tan, 1980; Hamilton, 1998; Naoe and Shibata, 2010)

as well as ozone (Butchart et al., 2003; Shibata and Deushi, 2005) are

also reproduced in models.

9.5.3.8 Summary

In summary, most modes of interannual to interdecadal variability are

now present in climate models. As in AR4, their assessment presents

a varied picture and CMIP5 models only show a modest improvement

over CMIP3, mostly due to fewer poor-performing models. New since

the AR4, process-based model evaluation is now helping identify

sources of speciﬁc biases, although the observational record is often

too short or inaccurate to offer strong constraints. The assessment of

modes and patterns is summarized in Table 9.4.

9.5.4 Extreme Events

Extreme events are realizations of the tail of the probability distribu-

tion of weather and climate variability. They are higher-order statistics

and thus generally more difﬁcult to realistically represent in climate

models. Shorter time scale extreme events are often associated with

smaller scale spatial structure, which may be better represented as

model resolution increases. In the AR4, it was concluded that models

could simulate the statistics of extreme events better than expected

from the generally coarse resolution of the models at that time, espe-

cially for temperature extremes (Randall et al., 2007).

Table 9.4 | Summary of assessment of interannual to interdecadal variability in climate models. See also Figure 9.44.

807

Evaluation of Climate Models Chapter 9

The IPCC has conducted an assessment of extreme events in the con-

text of climate change—the Special Report on Managing the Risks

of Extreme Events and Disasters to Advance Climate Change Adap-

tation (SREX) (IPCC, 2012). Although there is no comprehensive cli-

mate model evaluation with respect to extreme events in SREX, there

is some consideration of model performance taken into account in

assessing uncertainties in projections.

9.5.4.1 Extreme Temperature

Since the AR4, evaluation of CMIP3 and CMIP5 models has been

undertaken with respect to temperature extremes. Both model ensem-

bles simulate present-day warm extremes, in terms of 20-year return

values, reasonably well, with errors typically within a few degrees Cel-

sius over most of the globe (Kharin et al., 2007; Kharin et al., 2012). The

CMIP5 and CMIP3 models perform comparably for various tempera-

ture extreme indices, but with smaller inter-model spread in CMIP5.The

inter-model range of simulated indices is similar to the spread amongst

observationally based estimates in most regions (Sillmann et al., 2013).

Figure 9.37 shows relative error estimates of available CMIP5 models

for various extreme indices based on Sillmann et al. (2013). Although

the relative performance of an individual model may depend on the

choice of the reference data set (four different reanalyses are used),

the mean and median models tend to outperform individual models.

According to the standardized multi-model median errors (RMSE

std

) for

CMIP3 and CMIP5 shown on the right side of Figure 9.37, the perfor-

mance of the two ensembles is similar.

In terms of historical trends, CMIP3 and CMIP5 models generally cap-

ture observed trends in temperature extremes in the second half of

the 20th century (Sillmann et al., 2013), as illustrated in Figure 9.37.

The modelled trends are consistent with both reanalyses and sta-

tion-based estimates. It is also clear in the ﬁgure that model-based

indices respond coherently to major volcanic eruptions. Detection and

attribution studies based on CMIP3 models suggest that models tend

to overestimate the observed warming of warm temperature extremes

and underestimate the warming of cold extremes in the second half

of 20th century (Christidis et al., 2011; Zwiers et al., 2011) as noted

in SREX (Seneviratne et al., 2012). See also Chapter 10. This is not as

obvious in the CMIP5 model evaluation (Figure 9.37 and Sillmann et al.

(2013)) and needs further investigation.

9.5.4.2 Extreme Precipitation

For extreme precipitation, observational uncertainty is much larger

than for temperature, making model evaluation more challenging. Dis-

crepancies between different reanalyses for extreme precipitation are

substantial, whereas station-based observations have limited spatial

coverage (Kharin et al., 2007, 2012; Sillmann et al., 2013). Moreover,

a station-based observational data set, which is interpolated from sta-

tion measurements, has a potential mismatch of spatial scale when

compared to model results or reanalyses (Chen and Knutson, 2008).

Uncertainties are especially large in the tropics. In the extratropics,

precipitation extremes in terms of 20-year return values simulated by

CMIP3 and CMIP5 models compared relatively well with the observa-

tional data sets, with typical discrepancies in the 20% range (Kharin

et al., 2007, 2012). Figure 9.37 shows relative errors of CMIP5 models

for ﬁve precipitation-related indices. Darker grey shadings in the RMSE

columns for precipitation indicate larger discrepancies between models

and reanalyses for precipitation extremes in general. Sillmann et al.

(2013) found that the CMIP5 models tend to simulate more intense

precipitation and fewer consecutive wet days than the CMIP3, and

thus are closer to the observationally based indices.

It is known from sensitivity studies that simulated extreme precipita-

tion is strongly dependent on model resolution. Growing evidence has

shown that high-resolution models (50 km or ﬁner in the atmosphere)

can reproduce the observed intensity of extreme precipitation (Wehner

et al., 2010; Endo et al., 2012; Sakamoto et al., 2012), though some of

these results are based on models with observationally constrained

surface or lateral boundary conditions (i.e., Atmospheric General Circu-

lation Models (AGCMs) or Regional Climate Models (RCMs)).

In terms of historical trends, a detection and attribution study by Min et

al. (2011) found consistency in sign between the observed increase in

heavy precipitation over NH land areas in the second half of the 20th

century and that simulated by CMIP3 models, but they found that the

models tend to underestimate the magnitude of the trend (see also

Chapter 10). Related to this, it has been pointed out from comparisons

to satellite-based data sets that the majority of models underestimate

the sensitivity of extreme precipitation intensity to temperature in the

tropics (Allan and Soden, 2008; Allan et al., 2010; O’Gorman, 2012)

and globally (Liu et al., 2009; Shiu et al., 2012). O’Gorman (2012)

showed that this implies possible underestimation of the projected

future increase in extreme precipitation in the tropics.

9.5.4.3 Tropical Cyclones

It was concluded in the AR4 that high-resolution AGCMs generally

reproduced the frequency and distribution, but underestimated inten-

sity of tropical cyclones (Randall et al., 2007). Since then, Mizuta et

al. (2012) have shown that a newer version of the MRI-AGCM with

improved parameterizations (at 20 km horizontal resolution) simulates

tropical cyclones as intense as those observed with improved distri-

bution as well. Another remarkable ﬁnding since the AR4 is that the

observed year-to-year count variability of Atlantic hurricanes can be

well simulated by modestly high resolution (100 km or ﬁner) AGCMs

forced by observed SST, though with less skill in other basins (Larow et

al., 2008; Zhao et al., 2009; Strachan et al., 2013). Vortices that have

some characteristics of tropical cyclones can also be detected and

tracked in AOGCMs in CMIP3 and 5, but their intensities are generally

too weak (Yokoi et al., 2009a; Yokoi et al., 2012; Tory et al., 2013; Walsh

et al., 2013).

9.5.4.4 Droughts

Drought is caused by long time scale (months or longer) variability of

both precipitation and evaporation. Shefﬁeld and Wood (2008) found

that models in the CMIP3 ensemble simulated large-scale droughts in

the 20th century reasonably well, in the sense that multi-model spread

includes the observational estimate in each of several regions. Howev-

er, it should be noted that there are various deﬁnitions of drought (see

Chapter 2 and Seneviratne et al., 2012) and the performance of simu-

lated drought can depend on the deﬁnition. Moreover, different models

808

Chapter 9 Evaluation of Climate Models

Figure 9.37 | (a) Portrait plot of relative error metrics for the CMIP5 temperature and precipitation extreme indices based on Sillmann et al. (2013). (b)–(e) Time series of global

mean temperature extreme indices over land from 1948 to 2010 for CMIP3 (blue) and CMIP5 (red) models, ECMWF 40-year reanalysis (ERA40, green dashed) and National Centers

for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR, green dotted) reanalyses and HadEX2 station-based observational data set (black) based on

Sillmann et al. (2013). In (a), reddish and bluish colours indicate, respectively, larger and smaller root-mean-square (RMS) errors for an individual model relative to the median model.

The relative error is calculated for each observational data set separately. The grey-shaded columns on the right side indicate the RMS error for the multi-model median standardized

by the spatial standard deviation of the index climatology in the reanalysis, representing absolute errors for CMIP3 and CMIP5 ensembles. Results for four different reference data

sets, ERA-interim (top), ERA40 (left), NCEP/NCAR (right) and NCEP- Department of Energy (DOE) (bottom) reanalyses, are shown in each box. The analysis period is 1981–2000, and

only land areas are considered. The indices shown are simple daily precipitation intensity index (SDII), very wet days (R95p), annual maximum 5-day/1-day precipitation (RX5day/

RX1day), consecutive dry days (CDD), tropical nights (TR), frost days (FD), annual minimum/maximum daily maximum surface air temperature (TXn/TXx), and annual minimum/

maximum daily minimum surface air temperature (TNn/TNx). See Box 2.4 for the deﬁnitions of indices. Note that only a small selection of the indices analysed in Sillmann et al.

(2013) is shown, preferentially those that appear in Chapters 2, 10, 11, 12, 14. Also note that the NCEP/NCAR reanalysis has a known defect for TXx (Sillmann et al., 2013), but its

impact on this ﬁgure is small. In (b)–(e), shading for model results indicates the 25th to 75th quantile range of inter-model spread. Grey shading along the horizontal axis indicates

the evolution of globally averaged volcanic forcing according to Sato et al. (1993). The indices shown are the frequency of daily minimum/maximum surface air temperature below

the 10th percentile (b: Cold nights/c: Cold days) and that above 90th percentile (d: Warm nights/e: Warm days) of the 1961–1990 base period. Note that, as these indices essentially

represent changes relative to the base period, they are particularly suitable for being shown in time series and not straightforward for being shown in (a).

1950 1960 1970 1980 1990 2000 2010

Year

Exceedance rate (%)

Cold nights

CMIP3

CMIP5

ERA−40

NCEP/NCAR

HadEX2

1950 1960 1970 1980 1990 20002010

Year

Exceedance rate (%)

Cold days

CMIP3

CMIP5

ERA−40

NCEP/NCAR

HadEX2

1950 1960 1970 1980 1990 2000 2010

Year

Exceedance rate (%)

Warm nights

CMIP3

CMIP5

ERA−40

NCEP/NCAR

HadEX2

1950 1960 1970 1980 1990 2000 2010

Year

Exceedance rate (%)

Warm days

CMIP3

CMIP5

ERA−40

NCEP/NCAR

HadEX2

)c()b(

)e()d(

(a)

ENSMEAN

ENSMEDIAN

ACCESS1−0

BCC−CSM1−1

BCC−CSM1−1−M

BNU−ESM

CanCM4

CanESM2

CCSM4

CESM1−BGC

CMCC−CM

CNRM−CM5

CSIRO−Mk3−6−0

EC−EARTH

GFDL−CM3

GFDL−ESM2G

GFDL−ESM2M

GISS−E2−R

HadCM3

HadGEM2−CC

HadGEM2−ES

INMCM4

IPSL−CM5A−LR

IPSL−CM5B−LR

IPSL−CM5A−MR

MIROC4h

MIROC5

MIROC−ESM

IROC−ESM−CHEM

MPI−ESM−P

MPI−ESM−LR

MPI−ESM−MR

MRI−CGCM3

NorESM1−M

TNx

TNn

TXx

TXn

CDD

RX1day

RX5day

R95p

SDII

CMIP5 global land 1981−2000

CMIP5 RMSE

std

CMIP3 RMSE

std

−0.5

−0.4

−0.3

−0.2

−0.1

0.1

0.2

0.3

0.4

0.5

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.1

809

Evaluation of Climate Models Chapter 9

can simulate drought with different mechanisms (McCrary and Ran-

dall, 2010; Taylor et al., 2012a). A comprehensive evaluation of CMIP5

models for drought is currently not available, although Sillmann et al.

(2013) found that consecutive dry days simulated by CMIP5 models are

comparable to observations in magnitude and distribution.

9.5.4.5 Summary

There is medium evidence (i.e., a few multi-model studies) and high

agreement that the global distribution of temperature extremes are

represented well by CMIP3 and CMIP5 models. The observed global

warming trend of temperature extremes in the second half of the 20th

century is reproduced in models, but there is medium evidence (a few

CMIP3 studies) and medium agreement (not evident in a preliminary

look at CMIP5) that models tend to overestimate the warming of warm

temperature extremes and underestimate the warming of cold temper-

ature extremes.

There is medium evidence (single multi-model study) and medium

agreement (as inter-model difference is large) that CMIP5 models

Box 9.3 | Understanding Model Performance

This Box provides a synthesis of ﬁndings on understanding model performance based on the model evaluations discussed in this chapter.

Uncertainty in Process Representation

Some model errors can be traced to uncertainty in representation of processes (parameterizations). Some of these are long-standing

issues in climate modelling, reﬂecting our limited, though gradually increasing, understanding of very complex processes and the

inherent challenges in mathematically representing them. For the atmosphere, cloud processes, including convection and its interaction

with boundary layer and larger-scale circulation, remain major sources of uncertainty (Section 9.4.1). These in turn cause errors or

uncertainties in radiation which propagate through the coupled climate system. Distribution of aerosols is also a source of uncertainty

arising from modelled microphysical processes and transport (Sections 9.4.1 and 9.4.6). Ocean models are subject to uncertainty in

parameterizations of vertical and horizontal mixing and convection (Sections 9.4.2, 9.5.2 and 9.5.3), and ocean errors in turn affect the

atmosphere through resulting SST biases. Simulation of sea ice is also affected by errors in both the atmosphere and the ocean as well

as the parameterization of sea ice itself (Section 9.4.3). With respect to biogeochemical components in Earth System Models (ESMs),

parameterizations of nitrogen limitation and forest ﬁres are thought to be important for simulating the carbon cycle, but very few ESMs

incorporate these so far (Sections 9.4.4 and 9.4.5).

Error Propagation

Causes of one model bias can sometimes be associated with another. Although the root cause of those biases is often unclear, knowl-

edge of the causal chain or a set of interrelated biases can provide a key to further understanding and improvement of model per-

formance. For example, biases in storm track position are partly due to a SST biases in the Gulf Stream and Kuroshio Current (Section

9.4.1). Some biases in variability or trend can be partly traced back to biases in mean states. The decreasing trend in September Arctic

ice extent tends to be underestimated when sea ice thickness is overestimated (Section 9.4.3). In such cases, improvement of the mean

state may improve simulated variability or trend.

Sensitivity to Resolution

Some phenomena or aspects of climate are found to be better simulated with models run at higher horizontal and/or vertical resolution.

In particular, increased resolution in the atmosphere has improved, at least in some models, storm track and extratropical cyclones

(Section 9.4.1), diurnal variation of precipitation over land (Section 9.5.2), extreme precipitation, and tropical cyclone intensity and

structure (Section 9.5.4). Similarly, increased horizontal resolution in the ocean is shown to improve sea surface height variability,

western boundary currents, tropical instability waves and coastal upwelling (Section 9.4.2), and variability of Atlantic meridional over-

turning circulation (Section 9.5.3). High vertical resolution and a high model top, as well as high horizontal resolution, are important for

simulating lower stratospheric climate variability (Section 9.4.1), blocking (Section 9.5.2), the Quasi-Biennial Oscillation and the North

Atlantic Oscillation (Section 9.5.3). (continued on next page)

tend to simulate more intense and thus more realistic precipitation

extremes than CMIP3, which could be partly due to generally higher

horizontal resolution. There is medium evidence and high agreement

that CMIP3 models tend to underestimate the sensitivity of extreme

precipitation intensity to temperature. There is medium evidence and

high agreement that high resolution (50 km or ﬁner) AGCMs tend to

simulate the intensity of extreme precipitation comparable to observa-

tional estimates.

There is medium evidence and high agreement that year-to-year count

variability of Atlantic hurricanes can be well simulated by modestly

high resolution (100 km or ﬁner) AGCMs forced by observed SSTs.

There is medium evidence and medium agreement (as inter-model

difference is large) that the intensity of tropical cyclones is too weak

in CMIP3 and CMIP5 models. Finally, there is medium evidence (a

few multi-model studies) and medium agreement (as it might depend

on deﬁnitions of drought) that models can simulate aspects of large-

scale drought.

810

Chapter 9 Evaluation of Climate Models

9.6 Downscaling and Simulation of

Regional-Scale Climate

Regional-scale climate information can be obtained directly from

global models; however, their horizontal resolution is often too low to

resolve features that are important at regional scales. High-resolution

AGCMs, variable-resolution global models, and statistical and dynam-

ical downscaling (i.e., regional climate modelling) are used to comple-

ment AOGCMs, and to generate region-speciﬁc climate information.

These approaches are evaluated in the following.

9.6.1 Global Models

9.6.1.1 Regional-Scale Simulation by Atmosphere–Ocean

General Circulation Models

A comparison of CMIP3 and CMIP5 seasonal cycles of temperature

and precipitation for different regions (Figure 9.38) shows that tem-

perature is generally better simulated than precipitation in terms of

the amplitude and phase of the seasonal cycle. The multi-model mean

is closer to observations than most of the individual models. The sys-

tematic difference between the CMIP5 and CMIP3 ensembles is small

in most regions, although there is evident improvement in South Asia

(SAS) and Tropical South America (TSA) in the rainy seasons. In some

cases the spread amongst observational estimates can be of compa-

rable magnitude to the model spread, e.g., winter in the Europe and

Mediterranean (EUM) region.

There are as yet rather few published studies in which regional behav-

iour of the CMIP5 models is evaluated in great detail. Cattiaux et al.

(2013) obtained results for Europe similar to those discussed above.

Joetzjer et al. (2013) considered 13 models that participated in both

CMIP3 and in CMIP5 and found that the seasonal cycle of precipitation

over the Amazon improved in the latter.

Based on the CMIP archives, regional biases in seasonal and annual

mean temperature and precipitation are shown for several land

Box 9.3 (continued)

Uncertainty in Observational Data

In some cases, insufﬁcient length or quality of observational data makes model evaluation challenging, and is a frequent problem in the

evaluation of simulated variability or trends. This is evident for evaluation of upper tropical tropospheric temperature, tropical atmos-

pheric circulation (Section 9.4.1), the Atlantic meridional overturning circulation, the North Atlantic Oscillation and the Paciﬁc Decadal

Oscillation (Section 9.5.3). Data quality has been pointed out as an issue for arctic cloud properties (Section 9.4.1), ocean heat content,

heat and fresh water ﬂuxes over the ocean (Section 9.4.2) and extreme precipitation (Section 9.5.4). Palaeoclimate reconstructions

also have large inherent uncertainties (Section 9.5.2). It is clear therefore that updated or newly available data affect model evaluation

conclusions.

Other Factors

Model evaluation can be affected by how models are forced. Uncertainties in speciﬁed greenhouse gases, aerosols emissions, land use

change, etc. will all affect model results and hence evaluation of model quality. Different statistical methods used in model evaluation

can also lead to subtle or substantive differences in the assessment of model quality.

regions in Figure 9.39, and for polar and oceanic regions in Figure

9.40. The CMIP5 median temperature biases range from about –3°C

to 1.5°C. Substantial cold biases over NH regions are more prevalent

in winter (December to February) than summer (June to August). The

median biases appear in most cases slightly less negative for CMIP5

than CMIP3. The spread amongst models, as characterized by the 25

to 75% and 5 to 95% ranges, is slightly reduced from CMIP3 to CMIP5

in a majority of the regions and is roughly ±3ºC. The RMS error of

individual CMIP5 models is smaller than that for CMIP3 in 24 of the 26

regions in Figure 9.39 in DJF, JJA and the annual mean. The absolute

value of the ensemble mean bias has also been reduced in most cases.

The inter-model spread remains large, particularly in high-latitude

regions in winter and in regions with steep orography (such as CAS,

SAS, TIB and WSA). The inter-model temperature spread has decreased

from CMIP3 to CMIP5 over most of the oceans and over the Arctic and

Antarctic land regions. The cold winter bias over the Arctic has been

reduced. There is little systematic inter-ensemble difference in temper-

ature over lower latitude oceans.

Biases in seasonal total precipitation are shown in the right column

of Figures 9.39 and 9.40 for the NH winter (October to March) and

summer (April to September) half years as well as the annual mean.

The largest systematic biases over land regions occur in ALA, WSA and

TIB, where the annual precipitation exceeds that observed in all CMIP5

models, with a median bias on the order of 100%. All these regions are

characterized by high orography and / or a large fraction of solid pre-

cipitation, both of which are expected to introduce a negative bias in

gauge-based precipitation (Yang and Ohata, 2001; Adam et al., 2006)

that may amplify the model-observation discrepancy. A large negative

relative bias in SAH occurs in October to March, but it is of negligible

magnitude in absolute terms. In nearly all other seasonal and region-

al cases over land, the observational estimate falls within the range

of the CMIP5 simulations. Compared with CMIP3, the CMIP5 median

precipitation is slightly higher in most regions. In contrast with tem-

perature, the seasonal and annual mean ensemble mean and the root-

mean square precipitation biases are larger for CMIP5 than for CMIP3

in a slight majority of land regions (Figure 9.39) and in most of the

811

Evaluation of Climate Models Chapter 9

Figure 9.38 | Mean seasonal cycle of (a) temperature (ºC) and (b) precipitation (mm day

–1

). The average is taken over land areas within the indicated regions, and over the period

1980–1999. The red line is the average over 45 CMIP5 models; the blue line is the average over 22 CMIP3 models. The standard deviation of the respective data set is indicated

with shading. The different line styles in black refer to observational and reanalysis data: Climatic Research Unit (CRU) TS3.10, ECMWF 40-year reanalysis (ERA40) and ERA-Interim

for temperature; CRU TS3.10.1, Global Precipitation Climatology Project (GPCP), and CPC Merged Analysis of Precipitation (CMAP) for precipitation. Note the different axis-ranges

for some of the sub-plots. The 15 regions shown are: Western North America (WNA), Eastern North America (ENA), Central America (CAM), Tropical South America (TSA), Southern

South America (SSA), Europe and Mediterranean (EUM), North Africa (NAF), Central Africa (CAF), South Africa (SAF), North Asia (NAS), Central Asia (CAS), East Asia (EAS), South

Asia (SAS), Southeast Asia (SEA) and Australia (AUS).

812

Chapter 9 Evaluation of Climate Models

14 other regions (Figure 9.40). However, considering the observational

uncertainty, the performance of the CMIP3 and CMIP5 ensembles is

assessed to be broadly similar. The inter-model spreads are similar and

typically largest in arid areas when expressed in relative terms.

Figure 9.39 | Seasonal- and annual mean biases of (left) temperature (°C) and (right) precipitation (%) in the IPCC Special Report on Managing the Risks of Extreme Events and

Disasters to Advance Climate Change Adaptation (SREX) land regions (cf. Seneviratne et al., 2012, p. 12. The region’s coordinates can be found from their online Appendix 3.A). The

5th, 25th, 50th, 75th and 95th percentiles of the biases in 42 CMIP5 models are shown in box-and-whisker format, and corresponding values for 23 CMIP3 models with crosses.

The CMIP3 20C3M simulations are complemented with the corresponding A1B runs for the 2001–2005 period. The biases are calculated over 1986–2005, using Climatic Research

Unit (CRU) T3.10 as the reference for temperature and CRU TS 3.10.01 for precipitation. The regions are labelled with red when the root-mean-square error for the individual CMIP5

models is larger than that for CMIP3 and blue when it is smaller. The regions are: Alaska/NW Canada (ALA), Eastern Canada/Greenland/Iceland (CGI), Western North America

(WNA), Central North America (CNA), Eastern North America (ENA), Central America/Mexico (CAM), Amazon (AMZ), NE Brazil (NEB), West Coast South America (WSA), South-

Eastern South America (SSA), Northern Europe (NEU), Central Europe (CEU), Southern Europe/the Mediterranean (MED), Sahara (SAH), Western Africa (WAF), Eastern Africa (EAF),

Southern Africa (SAF), Northern Asia (NAS), Western Asia (WAS), Central Asia (CAS), Tibetan Plateau (TIB), Eastern Asia (EAS), Southern Asia (SAS), Southeast Asia (SEA), Northern

Australia (NAS) and Southern Australia/New Zealand (SAU). Note that the region WSA is poorly resolved in the models.

Especially over the oceans and polar regions (Figure 9.40), the scarci-

ty of observations and their uncertainty complicates the evaluation of

simulated precipitation. Of two commonly used data sets, CMAP indi-

cates systematically more precipitation than GPCP over low-latitude

813

Evaluation of Climate Models Chapter 9

Figure 9.40 | As Figure 9.39, but for polar and ocean regions, with ECMWF reanalysis

of the global atmosphere and surface conditions (ERA)-Interim reanalysis as the refer-

ence for temperature and Global Precipitation Climatology Project (GPCP) for precipi-

tation. Global land, ocean and overall means are also shown. The regions are: Arctic:

67.5 to 90°N, Caribbean (area deﬁned by the following coordinates): 68.8°W, 11.4°N;

85.8°W, 25°N; 60°W, 25°N, 60°W, 11.44°N; Western Indian Ocean: 25°S to 5°N, 52°E

to 75°E; Northern Indian Ocean: 5°N to 30°N, 60°E to 95°E; Northern Tropical Paciﬁc:

5°N to 25°N, 155°E to 150°W; Equatorial Tropical Paciﬁc: 5°S to 5°N, 155°E to 130°W;

Southern Tropical Paciﬁc: 5°S to 25°S, 155°E to 130°W; Antarctic: 50°S to 90°S. The

normalized difference between CPC Merged Analysis of Precipitation (CMAP) and GPCP

precipitation is shown with dotted lines.

oceans and less over many high-latitude regions (Yin et al., 2004; Shin

et al., 2011). Over most low-latitude ocean regions, annual precipita-

tion in most CMIP3 and CMIP5 models exceeds GPCP. The difference

relative to CMAP is smaller although mostly of the same sign. In Arctic

and Antarctic Ocean areas, simulated precipitation is much above

CMAP, but more similar to GPCP. Over Antarctic land, precipitation in

most models is below CMAP, but close to or above GPCP.

Continental to sub-continental mean values may not be representative

for smaller-scale biases, as biases generally increase with decreasing

spatial averaging (Masson and Knutti, 2011b; Raisanen and Ylhaisi,

2011). A typical order of magnitude for grid-box-scale annual mean

biases in individual CMIP3 models was 2°C for temperature and 1 mm

day

–1

for precipitation (Raisanen, 2007; Masson and Knutti, 2011b),

with some geographical variation. This has been noted also in studies

on how much spatial averaging would be needed in order to ﬁlter out

the most unreliable small-scale features (e.g., Räisänen and Ylhäisi,

2011). In order to reduce such errors while still retaining information

on small scales, Masson and Knutti (2011b) found, depending on the

variable and the region, that smoothing needed to vary from the grid-

point scale to around 2000 km.

On the whole, based on analysis of both ensemble means and

inter-model spread, there is high conﬁdence that the CMIP5 models

simulate regional-scale temperature distributions somewhat better

than the CMIP3 models did. This improvement is evident for most

regions. For precipitation, there is medium conﬁdence that there is no

systematic change in model performance. In many regions, precipita-

tion biases relative to CRU TS 3.10.01 and GPCP (and CMAP) are larger

for CMIP5 than for CMIP3, but given observational uncertainty, the

two ensembles are broadly similar.

9.6.1.2 Regional-Scale Simulation by Atmospheric General

Circulation Models

Stand-alone global atmospheric models (AGCMs) run at higher res-

olution than AOGCMs provide complementary regional-scale climate

information, sometimes referred to as ‘global downscaling’. One

important example of this is the simulation of tropical cyclones (e.g.,

Zhao et al., 2009, 2012; Murakami and Sugi, 2010; Murakami et al.,

2012). A number of advantages of high-resolution AGCMs have been

identiﬁed, including improved regional precipitation (Zhao et al., 2009;

Kusunoki et al., 2011) and blocking (Matsueda et al., 2009, 2010). As

AGCMs do not simulate interactions with the ocean, their ability to

capture some high-resolution phenomena, such as the cold wake in the

surface ocean after a tropical cyclone, is limited (e.g., Hasegawa and

Emori, 2007). As in lower-resolution models, performance is affected by

the quality of physical parameterizations (Lin et al., 2012; Mizuta et al.,

2012; Zhao et al., 2012).

9.6.1.3 Regional-Scale Simulation by Variable-Resolution

Global Climate Models

An alternative to global high resolution is the use of variable reso-

lution (so-called ‘stretched grid’) models with higher resolution over

the region of interest. Some examples are Abiodun et al. (2011) who

showed that such simulations improve the simulation of West African

814

Chapter 9 Evaluation of Climate Models

monsoon systems and African easterly jets, and White et al. (2013)

who demonstrated improvements in temperature and precipitation

related extreme indices. Fox-Rabinovitz et al. (2008) showed that

regional biases in the high-resolution portion of a stretched grid model

were similar to that of a global model with the same high resolution

everywhere. Markovic et al. (2010) and Déqué (2010) reported similar

results. Although not widely used, such methods can complement more

conventional climate models.

9.6.2 Regional Climate Downscaling

Regional Climate Models (RCMs) are applied over a limited-area

domain with boundary conditions either from global reanalyses or

global climate model output. The use of RCMs for ‘dynamical down-

scaling’ has grown since the AR4, their resolution has increased,

process-descriptions have developed further, new components have

been added, and coordinated experimentation has become more

widespread (Laprise, 2008; Rummukainen, 2010). Statistical downscal-

ing (SD) involves deriving empirical relationships linking large-scale

atmospheric variables (predictors) and local/regional climate variables

(predictands). These relationships may then be applied to equivalent

predictors from global models. SD methods have also been applied

to RCM output (e.g., Boe et al., 2007; Déqué, 2007; Segui et al., 2010;

Paeth, 2011; van Vliet et al., 2011). A signiﬁcant constraint in a com-

prehensive evaluation of regional downscaling is that available studies

often involve different methods, regions, periods and observational

data for evaluation. Thus, evaluation results are difﬁcult to generalize.

9.6.2.1 Recent Developments of Statistical Methods

The development of SD since the AR4 has been quite vigorous (e.g.,

Fowler et al., 2007; Maraun et al., 2010b), and many state-of-the-art

approaches combine different methods (e.g., Vrac and Naveau, 2008;

van Vliet et al., 2011). There is an increasing number of studies on

extremes (e.g., Vrac and Naveau, 2008; Wang and Zhang, 2008), and

on features such as hurricanes (Emanuel et al., 2008), river ﬂow and

discharge, sediment, soil erosion and crop yields (e.g., Zhang, 2007;

Prudhomme and Davies, 2009; Lewis and Lamoureux, 2010). Tech-

niques have also been developed to consider multiple climatic vari-

ables simultaneously in order to preserve some physical consistency

(e.g., Zhang and Georgakakos, 2011). The methods used to evaluate

SD approaches vary with the downscaled variable and include metrics

related to intensities (e.g., Ning et al., 2011; Tryhorn and DeGaetano,

2011), temporal behaviour (e.g., May, 2007; Timbal and Jones, 2008;

Maraun et al., 2010a; Brands et al., 2011), and physical processes (Len-

derink and Van Meijgaard, 2008; Maraun et al., 2010a). SD capabilities

are also examined through secondary variables like runoff, river dis-

charge and stream ﬂow (e.g., Boe et al., 2007; Teutschbein et al., 2011).

9.6.2.2 Recent Developments of Dynamical Methods

Since the AR4, typical RCM resolution has increased from around 50

km to around 25 km (e.g., Christensen et al., 2010). Long RCM runs

at very high resolution are still, however, rather few (e.g., Yasutaka et

al., 2008; Chan et al., 2012; Kendon et al., 2012). Coupled RCMs, with

interactive ocean and, when appropriate, also sea ice have also been

developed (Somot et al., 2008; Dorn et al., 2009; Artale et al., 2010;

Doscher et al., 2010). Smith et al. (2011a) added vegetation dynamics–

ecosystem biogeochemistry in an RCM.

At the time of the AR4, RCMs were typically used for time-slice experi-

ments. Since then, multi-decadal and centennial RCM simulations have

emerged in larger numbers (e.g., Diffenbaugh et al., 2011; Kjellstrom

et al., 2011; de Elia et al., 2013). Coordinated RCM experiments and

ensembles have also become much more common and today, with

domains covering Europe (e.g., Christensen et al., 2010; Vautard et

al., 2013), North America (e.g., Gutowski et al., 2010; Lucas-Picher et

al., 2012a; Mearns et al., 2012), South America (e.g., Menendez et al.,

2010; Chou et al., 2012; Krüger et al., 2012), Africa (e.g., Druyan et al.,

2010; Ruti et al., 2011; Nikulin et al., 2012; Paeth et al., 2012; Hernán-

dez-Díaz et al., 2013), the Arctic (e.g., Inoue et al., 2006) and Asian

regions (e.g., Feng and Fu, 2006; Shkolnik et al., 2007; Feng et al., 2011;

Ozturk et al., 2012; Suh et al., 2012).

9.6.3 Skill of Downscaling Methods

Downscaling skill varies with location, season, parameter and bounda-

ry conditions (see Section 9.6.5) (e.g., Schmidli et al., 2007; Maurer and

Hidalgo, 2008). Although there are indications that model skill increas-

es with higher resolution, it does not do so linearly. Rojas (2006) found

more improvement when increasing resolution from 135 km to 45 km

than from 45 km to 15 km. Walther et al. (2013) found that the diurnal

precipitation cycle and light precipitation improved more when going

from 12 km to 6 km resolution than when going from 50 km to 25 km

or from 25 km to 12 km. Higher resolution does enable better sim-

ulation of extremes (Seneviratne et al., 2012). For example, Pryor et

al. (2012) noted that an increase in RCM resolution from 50 km to 6

km increased extreme wind speeds more than the mean wind speed.

Kawazoe and Gutowski (2013) compared six RCMs and the two GCMs

to high resolution observations, concluding that precipitation extremes

were more representative in the RCMs than in the GCMs. Vautard et al.

(2013) found that warm extremes in Europe were generally better sim-

ulated in RCMs with 12 km resolution compared to 50 km. Kendon et

al. (2012) and Chan et al. (2012) found mixed results in daily precipita-

tion simulated at 12 km and 1.5 km resolution, although the latter had

improved sub-daily features, perhaps as convection could be explicitly

resolved.

Coupled RCMs, with an interactive ocean, offer further improvements.

Döscher et al. (2010) reproduced empirical relationships between

Arctic sea ice extent and sea ice thickness and NAO in a coupled RCM.

Zou and Zhou (2013) found that a regional ocean–atmosphere model

improved the simulation of precipitation over the western North

Paciﬁc compared to an uncoupled model. Samuelsson et al. (2010)

showed that coupling a lake model with an RCM captured the effect

of lakes on the air temperature over adjacent land. Lenaerts et al.

(2012) added drifting snow in an RCM run for the Antarctica, which

increased the area of ablation and improved the ﬁt to observations.

Smith et al. (2011a) added vegetation dynamics–ecosystem biogeo-

chemistry into an RCM, and found some evidence of local feedback

to air temperature.

Applying an RCM developed for a speciﬁc region to other regions

exposes it to a wider range of conditions and therefore provides an

815

Evaluation of Climate Models Chapter 9

opportunity for more rigorous evaluation. Transferability experiments

target this by running RCMs for different regions while holding their

process-descriptions constant (cf. Takle et al., 2007; Gbobaniyi et al.,

2011; Jacob et al., 2012). Suh et al. (2012) noted that 10 RCMs run

for Africa did well overall for average and maximum temperature, but

systematically overestimated the daily minimum temperature. Precip-

itation was generally simulated betted for wet regions than for dry

regions. Similarly, Nikulin et al. (2012) reported on 10 RCMs over

Africa, run with boundary conditions from ERA-Interim, and evaluated

against different observational data sets. Many of the RCMs simulated

precipitation better than the ERA-Interim reanalysis itself.

Christensen et al. (2010) examined a range metrics related to simula-

tion of extremes, mesoscale features, trends, aspects of variability and

consistency with the driving boundary conditions. Only one of these

metrics led to clear differentiation among RCMs (Lenderink, 2010). This

may imply a general skilfulness of models, but may also simply indicate

that the metrics were not very informative. Nevertheless, using some of

these metrics, Coppola et al. (2010) and Kjellström et al. (2010) found

that weighted sets of RCMs outperformed sets without weighting for

both temperature and precipitation. Sobolowski and Pavelsky (2012)

demonstrated a similar impact.

9.6.4 Value Added through RCMs

RCMs are regularly tested to evaluate whether they show improve-

ments over global models (Laprise et al., 2008), that is, whether they

do indeed ‘add value’. In essence, added value is a measure of the

extent to which the downscaled climate is closer to observations than

the model from which the boundary conditions were obtained. Differ-

ences between RCM and GCM simulations are not always very obvious

for time-averaged quantities on larger scales or in fairly homogeneous

regions. RCM ﬁelds are, however, richer in spatial and temporal detail.

Indeed, the added value of RCMs is mainly expected in the simulation

of topography-inﬂuenced phenomena and extremes with relatively

small spatial or short temporal character (e.g., Feser et al., 2011; Feser

and Barcikowska, 2012; Shkol’nik et al., 2012). As an example, RCM

downscaling led to better large-scale monsoon precipitation patterns

(Gao et al., 2012) for East Asia than in the global models used for

boundary conditions. In the few instances where RCMs have been

interactively coupled to global models (i.e., ‘two-way’ coupling), the

effects of improved small scales propagate to larger scales and this

has been found to improve the simulation of larger scale phenomena

(Lorenz and Jacob, 2005; Inatsu and Kimoto, 2009; Inatsu et al., 2012).

Other examples include improved simulation of convective precipita-

tion (Rauscher et al., 2010), near-surface temperature (Feser, 2006),

near-surface temperature and wind (Kanamaru and Kanamitsu, 2007),

temperature and precipitation (Lucas-Picher et al., 2012b), extreme

precipitation (Kanada et al., 2008), coastal climate features (Winter-

feldt and Weisse, 2009; Winterfeldt et al., 2011; Kawazoe and Gutows-

ki, 2013; Vautard et al., 2013), Atlantic hurricanes (Bender et al.,

2010), European storm damage (Donat et al., 2010), strong mesoscale

cyclones (Cavicchia and Storch, 2011), cutoff lows (Grose et al., 2012),

polar lows (Zahn and von Storch, 2008) and higher statistical moments

of the water budget (e.g., Bresson and Laprise, 2011).

In summary, there is high conﬁdence that downscaling adds value to

the simulation of spatial climate detail in regions with highly varia-

ble topography (e.g., distinct orography, coastlines) and for mesoscale

phenomena and extremes. Regional downscaling is therefore comple-

mentary to results obtained directly from global climate models. These

results are from a variety of distinct studies with different RCMs.

9.6.5 Sources of Model Errors and Uncertainties

In addition to issues related to resolution and model complexity (see

Section 9.6.3), errors and uncertainties arise from observational uncer-

tainty in evaluation data and parameterizations (see Box 9.3), choice of

model domain and application of boundary conditions (driving data).

In the case of SD, sources of model errors and uncertainties depend on

the choice of method, including the choice of the predictors, the esti-

mation of empirical relationships between predictors and predictands

from limited data sets, and also the data used to estimate the predic-

tors (Frost et al., 2011). There are numerous different SD methods, and

the ﬁndings are difﬁcult to generalize.

Small domains allow less freedom for RCMs to generate the small-scale

features that give rise to added value (e.g., Leduc and Laprise, 2009).

Therefore large domains –covering entire continents– have become

more common. Køltzow et al. (2008) found improvements with the

use of a larger domain, but the RCM solution can become increasingly

‘decoupled’ from the driving data (e.g., Rockel et al., 2008), which can

introduce inconsistencies. Large domains also introduce large internal

variability, which can signiﬁcantly contaminate interannual variability

of seasonal means (Kanamitsu et al., 2010). Techniques such as spec-

tral nudging (Misra, 2007; Separovic et al., 2012) can be used to con-

strain such inconsistencies (Feser et al., 2011). Winterfeldt and Weisse

(2009) concluded that nudging improved the simulation of marine

wind climate, while Otte et al. (2012) demonstrated improvements in

temperature and precipitation. Nudging may, however, also lead to

deterioration of features such as precipitation extremes (Alexandru et

al., 2009; Kawazoe and Gutowski, 2013). Veljovic et al. (2010) showed

that an RCM can in fact improve the large scales with respect to those

inherent in the boundary conditions, and argued that nudging may be

undesirable.

The quality of RCM results may vary according to the synoptic situation,

season, and the geographic location of the lateral boundaries (Alexan-

dru et al., 2007; Xue et al., 2007; Laprise et al., 2008; Separovic et al.,

2008; Leduc and Laprise, 2009; Nikiema and Laprise, 2010; Rapaić et

al., 2010). In addition to lateral boundary conditions, RCMs also need

sea surface information. Few studies have explored the dependency

of RCM results on the treatment of the SSTs and sea ice, although

Koltzow et al. (2011) found that the speciﬁcation of SSTs was less inﬂu-

ential than was the domain or the lateral boundaries. Woollings et al.

(2010a) investigated the effect of speciﬁed SST on the simulation of

the Atlantic storm track and found that it was better simulated with

high-resolution SSTs, whereas increasing temporal resolution gave

mixed results.

As is the case in global models, RCM errors are directly related to

shortcomings in process parameterizations. Examples include the

816

Chapter 9 Evaluation of Climate Models

representation of clouds, convection and land surface–atmosphere

interactions, the planetary boundary layer, horizontal diffusion, and

microphysics (Tjernstrom et al., 2008; Wyser et al., 2008; Lynn et al.,

2009; Pfeiffer and Zängl, 2010; Axelsson et al., 2011; Crétat et al., 2012;

Evans et al., 2012; Roy et al., 2012; Solman and Pessacg, 2012). The

representation of land surface and atmosphere coupling is also impor-

tant, particularly for simulating monsoon regions (Cha et al., 2008;

Yhang and Hong, 2008; Boone et al., 2010; Druyan et al., 2010; van

den Hurk and van Meijgaard, 2010).

9.6.6 Relating Downscaling Performance to Credibility

of Regional Climate Information

A fundamental issue is how the performance of a downscaling method

relates to its ability to provide credible future projections (Raisanen,

2007). This subject is discussed further in Section 9.8. The credibility of

downscaled information of course depends on the quality of the down-

scaling method itself (e.g., Dawson et al., 2012; Déqué et al., 2012;

Eum et al., 2012), and on the quality of the global climate models pro-

viding the large-scale boundary conditions (e.g., van Oldenborgh et al.,

2009; Diaconescu and Laprise, 2013).

Speciﬁc to SD is the statistical stationarity hypothesis, that is, that the

relationships inferred from historical data remain valid under a chang-

ing climate (Maraun, 2012). Vecchi et al. (2008) note that a statistical

method that captures interannual hurricane variability gives very dif-

ferent results for projections compared to RCMs. Such results suggest

that good performance of statistical downscaling as assessed against

observations does not guarantee credible regional climate information.

Some recent studies have proposed ways to evaluate SD approaches

using RCM outputs (e.g., Vrac and Naveau, 2008; Driouech et al., 2010)

or long series of observations (e.g. Schmith, 2008).

Giorgi and Coppola (2010) argued that regional-scale climate pro-

jections over land in the CMIP3 models were not sensitive to their

temperature biases. For precipitation, the same was found for about

two thirds of the global land area. However, there is some recent evi-

dence that regional biases may be nonlinear for temperature extremes

(Christensen et al., 2008; Boberg and Christensen, 2012; Christensen

and Boberg, 2013) in both global and regional models. A mechanism

at play may be that models tend to dry out the soil too effectively at

high temperatures, which can lead to systematic biases in projected

warm summertime conditions (Christensen et al., 2008; Kostopoulou

et al., 2009). This is illustrated in Figure 9.41 for the Mediterranean

region, which suggests a tendency in RCMs, CMIP3 and CMIP5 models

towards an enhanced warm bias in the warmer months. The implica-

tion is that the typically large warming signal in these regions could

be biased (Boberg and Christensen, 2012; Mearns et al., 2012). Find-

ings such as these stress the importance of a thorough assessment of

models’ biases when they are applied for projections (e.g., de Elia and

Cote, 2010; Boberg and Christensen, 2012; Christensen and Boberg,

2013).

Di Luca et al. (2012) analysed downscaled climate change projections

from six RCMs run over North America. The climate change signals for

seasonal precipitation and temperature were similar to those in the

driving AOGCMs, and the spatial detail gained by downscaling was

comparable in both present and future climate. Déqué et al. (2012)

studied projections with several combinations of AOGCM and RCM

for Europe. A larger part of the spread in winter temperature and

Figure 9.41 | Ranked modelled versus observed monthly mean temperature for the Mediterranean region for the 1961–2000 period. The Regional Climate Model (RCM) data (a)

are from Christensen et al. (2008) and are adjusted to get a zero mean in model temperature with respect to the diagonal. The smaller insert shows uncentred data. The General

Circulation Model (GCM) data (b) are from CMIP5 and CMIP3 and adjusted in the same way. (After Boberg and Christensen, 2012.)

817

Evaluation of Climate Models Chapter 9

precipitation projections was explained by the differences in global

model boundary conditions, although much of the spread in project-

ed summer precipitation was explained by RCM. This underlines the

importance of both the quality of the boundary conditions and the

downscaling method.

9.7 Climate Sensitivity and Climate Feedbacks

An overall assessment of climate sensitivity and transient climate

response is given in Box 12.2. Observational constraints based on

observed warming over the last century are discussed in Section 10.8.2

and shown in Box 12.2, Figure 2.

9.7.1 Equilibrium Climate Sensitivity, Idealized Radiative

Forcing, and Transient Climate Response in the

Coupled Model Intercomparison Project Phase 5

Ensemble

Equilibrium climate sensitivity (ECS) is the equilibrium change in global

and annual mean surface air temperature after doubling the atmos-

pheric concentration of CO

relative to pre-industrial levels. In the AR4,

the range in equilibrium climate sensitivity of the CMIP3 models was

2.1°C to 4.4°C, and the single largest contributor to this spread was

differences among modelled cloud feedbacks. These assessments carry

over to the CMIP5 ensemble without any substantial change (Table

9.5).

The method of diagnosing climate sensitivity in CMIP5 differs funda-

mentally from the method employed in CMIP3 and assessed in the

AR4 (Randall et al., 2007). In CMIP3, an AGCM was coupled to a

non-dynamic mixed-layer (slab) ocean model with prescribed ocean

heat transport convergence. CO

concentration was then instantane-

ously doubled, and the model was integrated to a new equilibrium

with unchanged implied ocean heat transport. While computationally

efﬁcient, this method had the disadvantage of employing a different

model from that used for the historical simulations and climate projec-

tions. However, in the few comparisons that were made, the resulting

disagreement in ECS was less than about 10% (Boer and Yu, 2003;

Williams et al., 2008; Danabasoglu and Gent, 2009; Li et al., 2013a).

In CMIP5, climate sensitivity is diagnosed directly from the AOGCMs

following the approach of Gregory etal. (2004). In this case the CO

concentration is instantaneously quadrupled and kept constant for 150

years of simulation, and both equilibrium climate sensitivity and RF

are diagnosed from a linear ﬁt of perturbations in global mean surface

temperature to the instantaneous radiative imbalance at the TOA.

The transient climate response (TCR) is the change in global and annual

mean surface temperature from an experiment in which the CO

con-

centration is increased by 1% yr

–1

, and calculated using the difference

between the start of the experiment and a 20-year period centred on

the time of CO

doubling. TCR is smaller than ECS because ocean heat

uptake delays surface warming. TCR is linearly correlated with ECS in

the CMIP5 ensemble (Figure 9.42), although the relationship may be

nonlinear outside the range spanned in Table 9.5 (Knutti et al., 2005).

Based on the methods outlined above and explained in Section

9.7.2 below, Table 9.5 shows effective ERF, ECS, TCR and feedback

strengths for the CMIP5 ensemble. The two estimates of ERF agree

with each other to within 5% for six models (CanESM2, INM-CM4,

IPSL-CM5A-LR, MIROC5, MPI-ESM-LR and MPI-ESM-P), although the

deviation exceeds 10% for four models (CCSM4, CSIRO-Mk3-6-0,

HadGEM2-ES, and MRI-CGCM3) and is indicative of deviations from

the basic assumptions underlying one or both ERF estimation methods.

However, the mean difference of 0.3 W m

–2

between the two meth-

ods for diagnosing ERF is only about half of the ensemble standard

deviation of 0.5 W m

–2

, or 15% of the mean value for ERF by CO

using ﬁxed SSTs. ECS and TCR vary across the ensemble by a factor

of approximately 2. The multi-model ensemble mean in ECS is 3.2°C,

a value nearly identical to that for CMIP3, while the CMIP5 ensemble

range is 2.1°C to 4.7°C, a spread which is also nearly indistinguishable

from that for CMIP3. While every CMIP5 model whose heritage can

Figure 9.42 | (a) Equilibrium climate sensitivity (ECS) against the global mean surface temperature of CMIP5 models, both for the period 1961–1990 (larger symbols, cf. Figure

9.8, Table9.5) and for the pre-industrial control runs (smaller symbols). (b) Equilibrium climate sensitivity against transient climate response (TCR). The ECS and TCR information are

based on Andrews et al. (2012) and Forster et al. (2013) and updated from the CMIP5 archive.

818

Chapter 9 Evaluation of Climate Models

Table 9.5 | Effective radiative forcing (ERF), climate sensitivity and climate feedbacks estimated for the CMIP5 AOGCMs (see Table 9.1 for model details). ERF, equilibrium climate sensitivity (ECS) and transient climate response (TCR) are based

on Andrews et al. ( 2012) and Forster et al. (2013) and updated from the CMIP5 archive. The ERF entries are calculated according to Hansen etal. (2005) using ﬁxed sea surface temperatures (SSTs) and Gregory etal. (2004) using regression. ECS

is calculated using regressions following Gregory etal. (2004). TCR is calculated from the CMIP5 simulations with 1% CO

increase per year (Taylor et al., 2012b), using the 20-year mean centred on the year of CO

doubling. The climate sensitivity

parameter and its inverse, the climate feedback parameter, are calculated from the regression-based ERF and the ECS. Strengths of the individual feedbacks are taken from Vial et al. (2013), following Soden etal. (2008) and using radiative kernel

methods with two different kernels. The sign convention is such that a positive entry for an individual feedback marks a positive feedback; the sum of individual feedback strengths must hence be multiplied by –1 to make it comparable to the

climate feedback parameter. The entries for radiative forcing and equilibrium climate sensitivity were obtained by dividing by two the original results, which were obtained for CO

quadrupling. ERF and ECS for BNU-ESM are from Vial et al. (2013).

Model

Effective Radiative Forcing

2 × CO

(W m

–2

)

Equilibrium

Climate

Sensitivity

(°C)

Transient

Climate

Response

(°C)

Climate

Sensitivity

Parameter

(°C(W m

–2

)

–1

)

Climate Feed-

back Parameter

(W m

–2

°C

–1

)

Planck Feedback

(W m

–2

°C

–1

)

Water Vapour

Feedback

(W m

–2

°C

–1

)

Lapse Rate

Feedback

(W m

–2

°C

–1

)

Surface Albedo

Feedback

(W m

–2

°C

–1

)

Cloud Feedback

(W m

–2

°C

–1

)

Fixed SST Regression

ACCESS1.0 n.a. 3.0 3.8 2.0 1.3 0.8 n.a. n.a. n.a. n.a. n.a.

ACCESS1.3 n.a. n.a. n.a. 1.7 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

BCC–CSM1.1 n.a. 3.2 2.8 1.7 0.9 1.1 n.a. n.a. n.a. n.a. n.a.

BCC–CSM1.1(m) n.a. 3.6 2.9 2.1 0.8 1.2 n.a. n.a. n.a. n.a. n.a.

BNU–ESM n.a. 3.9 4.1 2.6 1.1 1.0 –3.1 1.4 –0.2 0.4 0.1

CanESM2 3.7 3.8 3.7 2.4 1.0 1.0 –3.2 1.7 –0.6 0.3 0.5

CCSM4 4.4 3.6 2.9 1.8 0.8 1.2 –3.2 1.5 –0.4 0.4 –0.4

CESM1(BGC) n.a. n.a. n.a. 1.7 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

CESM1(CAM5) n.a. n.a. n.a. 2.3 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

CNRM–CM5 n.a. 3.7 3.3 2.1 0.9 1.1 n.a. n.a. n.a. n.a. n.a.

CSIRO–Mk3.6.0 3.1 2.6 4.1 1.8 1.6 0.6 n.a. n.a. n.a. n.a. n.a.

FGOALS–g2 n.a. n.a. n.a. 1.4 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

GFDL–CM3 n.a. 3.0 4.0 2.0 1.3 0.8 n.a. n.a. n.a. n.a. n.a.

GFDL–ESM2G n.a. 3.1 2.4 1.1 0.8 1.3 n.a. n.a. n.a. n.a. n.a.

GFDL–ESM2M n.a. 3.4 2.4 1.3 0.7 1.4 n.a. n.a. n.a. n.a. n.a.

GISS–E2–H n.a. 3.8 2.3 1.7 0.6 1.7 n.a. n.a. n.a. n.a. n.a.

GISS–E2–R n.a. 3.8 2.1 1.5 0.6 1.8 n.a. n.a. n.a. n.a. n.a.

HadGEM2–ES 3.5 2.9 4.6 2.5 1.6 0.6 –3.2 1.4 –0.5 0.3 0.4

INM–CM4 3.1 3.0 2.1 1.3 0.7 1.4 –3.2 1.7 –0.7 0.3 0

IPSL–CM5A–LR 3.2 3.1 4.1 2.0 1.3 0.8 –3.3 1.9 –1 0.2 1.2

IPSL–CM5A–MR n.a. n.a. n.a. 2.0 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

IPSL–CM5B–LR n.a. 2.7 2.6 1.5 1.0 1.0 n.a. n.a. n.a. n.a. n.a.

MIROC5 4.0 4.1 2.7 1.5 0.7 1.5 –3.2 1.7 –0.6 0.3 0.1

MIROC–ESM n.a. 4.3 4.7 2.2 1.1 0.9 n.a. n.a. n.a. n.a. n.a.

MPI–ESM–LR 4.3 4.1 3.6 2.0 0.9 1.1 –3.3 1.8 –0.9 0.3 0.5

MPI–ESM–MR n.a. n.a. n.a. 2.0 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

MPI–ESM–P 4.3 4.3 3.5 2.0 0.8 1.2 n.a. n.a. n.a. n.a. n.a.

MRI–CGCM3 3.6 3.2 2.6 1.6 0.8 1.2 –3.2 1.6 –0.6 0.3 0.2

NorESM1–M n.a. 3.1 2.8 1.4 0.9 1.1 –3.2 1.6 –0.5 0.3 0.2

NorESM1–ME n.a. n.a. n.a. 1.6 n.a. n.a. n.a. n.a. n.a. n.a. n.a.

Model mean 3.7 3.4 3.2 1.8 1.0 1.1 –3.2 1.6 –0.6 0.3 0.3

90% uncertainty ±0.8 ±0.8 ±1.3 ±0.6 ±0.5 ±0.5 ±0.1 ±0.3 ±0.4 ±0.1 ±0.7

819

Evaluation of Climate Models Chapter 9

Figure 9.43 | (a) Strengths of individual feedbacks for CMIP3 and CMIP5 models (left and right columns of symbols) for Planck (P), water vapour (WV), clouds (C), albedo (A), lapse

rate (LR), combination of water vapour and lapse rate (WV+LR) and sum of all feedbacks except Planck (ALL), from Soden and Held (2006) and Vial et al. (2013), following Soden

etal. (2008). CMIP5 feedbacks are derived from CMIP5 simulations for abrupt fourfold increases in CO

concentrations (4 × CO

). (b) ECS obtained using regression techniques by

Andrews et al. (2012) against ECS estimated from the ratio of CO

ERF to the sum of all feedbacks. The CO

ERF is one-half the 4 × CO

forcings from Andrews et al. (2012), and

the total feedback (ALL + Planck) is from Vial et al. (2013).

be traced to CMIP3 shows some change in ECS, there is no discernible

systematic tendency. This broad similarity between CMIP3 and CMIP5

and the good agreement between different methods where they were

applied to the same atmospheric GCM indicate that the uncertainty

in methodology is minor compared to the overall spread in ECS. The

change in TCR from CMIP3 to CMIP5 is generally of the same sign but

of smaller magnitude compared to the change in ECS. The relationship

between ECS and an estimates derived from total feedbacks are dis-

cussed in Section 9.7.2.

Although ECS can vary with global mean surface temperature owing to

the temperature dependencies of the various feedbacks (Colman and

McAvaney, 2009; cf. Section 9.7.2), Figure 9.42 shows no discernible

correlation for the CMIP5 historical temperature ranges, a fact that

suggests that ECS is less sensitive to errors in the current climate than

to other sources of uncertainty.

9.7.2 Understanding the Range in Model Climate

Sensitivity: Climate Feedbacks

The strengths of individual feedbacks for the CMIP3 and CMIP5

models are compared in Figure 9.43. The feedbacks are generally

similar between CMIP3 and CMIP5, and the water vapour, lapse rate,

and cloud feedbacks are assessed in detail in Chapter 7. The surface

albedo feedback is assessed here to be likely positive. There is high

conﬁdence that the sum of all feedbacks (excluding the Planck feed-

back) is positive. Advances in estimating and understanding each of

the feedback parameters in Table 9.5 are described in detail below (see

also Chapters 7 and 8).

9.7.2.1 Role of Humidity and Lapse Rate Feedbacks in

Climate Sensitivity

The compensation between the water vapour and lapse-rate feed-

backs noted in the CMIP3 models is still present in the CMIP5 models,

and possible explanations of the compensation have been developed

(Ingram, 2010; Ingram, 2013). New formulations of the feedbacks,

replacing speciﬁc with relative humidity, eliminate most of the cancel-

lation between the water vapour and lapse rate feedbacks and reduce

the inter-model scatter in the individual feedback terms (Held and

Shell, 2012).

9.7.2.2 Role of Surface Albedo in Climate Sensitivity

Analysis of observed declines in sea ice and snow coverage from 1979

to 2008 suggests that the NH albedo feedback is between 0.3 and 1.1

W m

–2

°C

–1

(Flanner et al., 2011). This range is substantially above the

global feedback of 0.3 ± 0.1 W m

–2

°C

–1

of the CMIP5 models ana-

lysed in Table9.5. One possible explanation is that the CMIP5 models

underestimate the strength of the feedback as did the CMIP3 models

based upon the systematic errors in simulated sea ice coverage decline

relative to observed rates (Boe et al., 2009b).

9.7.2.3 Role of Cloud Feedbacks in Climate Sensitivity

Cloud feedbacks represent the main cause for the range in modelled

climate sensitivity (Chapter7). The spread due to inter-model differenc-

es in cloud feedbacks is approximately 3 times larger than the spread

contributed by feedbacks due to variations in water vapour and lapse

820

Chapter 9 Evaluation of Climate Models

rate combined (Dufresne and Bony, 2008), and is a primary factor

governing the range of climate sensitivity across the CMIP3 ensem-

ble (Volodin, 2008a). Differences in equilibrium and effective climate

sensitivity are due primarily to differences in the shortwave cloud feed-

back (Yokohata et al., 2008).

In perturbed ensembles of three different models, the primary cloud-re-

lated factor contributing to the spread in equilibrium climate sensitivity

is the low-level shortwave cloud feedback (Yokohata et al., 2010; Klocke

et al., 2011). Changes in the high-altitude clouds also induce climate

feedbacks due to the large areal extent and signiﬁcant longwave cloud

radiative effects of tropical convective cloud systems. In experiments

with perturbed physics ensembles of AOGCMs, the parameterization

of ice fall speed also emerges as one of the most important determi-

nants of climate sensitivity (Sanderson et al., 2008a, 2010; Sexton et

al., 2012). Other non-microphysical feedback mechanisms are detailed

in Chapter 7.

Cloud feedbacks in AOGCMs are generally positive or near neutral

(Shell et al., 2008; Soden et al., 2008), as evidenced by the net positive

or neutral cloud feedbacks in all of the models examined in a mul-

ti-thousand member ensemble of AOGCMs constructed by parameter

perturbations (Sanderson et al., 2010). The sign of cloud feedbacks in

the current climate deduced from observed relationships between SSTs

and TOA radiative ﬂuxes are discussed further in Section 7.2.5.7.

9.7.2.4 Relationship of Feedbacks to Modelled Climate

Sensitivity

The ECS can be estimated from the ratio of forcing to the total cli-

mate feedback parameter. This approach is applicable to simulations

in which the net radiative balance is much smaller than the forcing

and hence the modelled climate system is essentially in equilibrium.

This approach can also serve to check the internal consistency of esti-

mates of the ECS, forcing, and feedback parameters obtained using

independent methods. The relationship between ECS from Andrews et

al. (2012) and estimates of ECS obtained from the ratio of forcings to

feedbacks is shown in Figure 9.43b. The forcings are estimated using

both regression and ﬁxed SST techniques (Gregory et al., 2004; Hansen

et al., 2005) by Andrews et al. (2012) and the feedbacks are calculated

using radiative kernels (Soden et al., 2008). On average, the ECS from

forcing to feedback ratios underestimate the ECS from Andrews et al.

(2012) by 25% and 35%, or up to 50% for individual models, using

ﬁxed-SST and regression forcings, respectively.

9.7.2.5 Relationship of Feedbacks to Uncertainty in

Modelled Climate Sensitivity

Objective methods for perturbing uncertain model parameters to

optimize performance relative to a set of observational metrics have

shown a tendency toward an increase in the mean and a narrowing

of the spread of estimated climate sensitivity (Jackson et al., 2008a).

This tendency is opposed by the effects of structural biases related

to incomplete process representations in GCMs. If common structur-

al biases are replicated across models in a MME (cf. Section 9.2.2.7),

the most likely sensitivity for the MME tends to shift towards lower

sensitivities while the possibility of larger sensitivities increases at the

same time (Lemoine, 2010). Following Schlesinger and Mitchell (1987),

Roe and Baker (2007) suggest that symmetrically distributed uncer-

tainties in feedbacks lead to inherently asymmetrical uncertainties in

climate sensitivity with increased probability in extreme positive values

of the sensitivity. Roe and Baker (2007) conclude that this relationship

makes it extremely difﬁcult to reduce uncertainties in climate sensitiv-

ity through incremental improvements in the speciﬁcation of feedback

parameters. While subsequent analysis has suggested that this ﬁnding

could be an artifact of the statistical formulation (Hannart et al., 2009)

and linearization (Zaliapin and Ghil, 2010) of the relationship between

feedback and sensitivity adopted by (Roe and Baker, 2007), these

issues remain unsettled (Roe and Armour, 2011; Roe and Baker, 2011).

9.7.3 Climate Sensitivity and Model Performance

Despite the range in equilibrium sensitivity of 2.1°C to 4.4°C for CMIP3

models, they reproduce the global surface air temperature anomaly

of 0.76°C over 1850–2005 to within 25% relative error. The relative-

ly small range of historical climate response suggests that there is

another mechanism, for example a compensating non-GHG forcing,

present in the historical simulations that counteracts the relatively

large range in sensitivity obtained from idealized experiments forced

only by increasing CO

. One possible mechanism is a systematic neg-

ative correlation across the multi-model ensemble between ECS and

anthropogenic aerosol forcing (Kiehl, 2007; Knutti, 2008; Anderson et

al., 2010). A second possible mechanism is a systematic overestimate

of the mixing between the oceanic mixed layer and the full depth

ocean underneath (Hansen et al., 2011). However, despite the same

range of ECS in the CMIP5 models as in the CMIP3 models, there is

no signiﬁcant relationship across the CMIP5 ensemble between ECS

and the 20th-century ERF applied to each individual model (Forster et

al., 2013). This indicates a lesser role of compensating ERF trends from

GHGs and aerosols in CMIP5 historical simulations than in CMIP3. Dif-

ferences in ocean heat uptake also do not appreciably affect the spread

in projected changes in global mean temperature by 2095 (Forster et

al., 2013).

9.7.3.1 Constraints on Climate Sensitivity from Earth System

Models of Intermediate Complexity

An EMIC intercomparison (Eby et al., 2013; Zickfeld et al., 2013) allows

an assessment of model response characteristics, including ECS, TCR,

and heat uptake efﬁciency (Table 9.6). In addition, Bayesian methods

applied to PPE experiments using EMICs have estimated uncertainty

in model response characteristics (see Box 12.2) based on simulated

climate change in 20th century, past millennia, and LGM scenarios.

Here, the range of response metrics (Table 9.6) described for default

model conﬁgurations (Eby et al., 2013) indicates consistency with the

CMIP5 ensemble.

9.7.3.2 Climate Sensitivity During the Last Glacial Maximum

Climate sensitivity can also be explored in another climatic context.

The AR4 assessed attempts to relate simulated LGM changes in trop-

ical SST to global climate sensitivity (Hegerl et al., 2007; Knutti and

Hegerl, 2008). LGM temperature changes in the tropics (Hargreaves

et al., 2007), but not in Antarctica (Hargreaves et al., 2012), have been

821

Evaluation of Climate Models Chapter 9

shown to scale well with climate sensitivity because the signal is mostly

dominated by CO

forcing in these regions (Braconnot et al., 2007b;

Jansen et al., 2007). The analogy between the LGM climate sensitivity

and future climate sensitivity is, however, not perfect (Cruciﬁx, 2006).

In a single-model ensemble of simulations, the magnitudes of the LGM

cooling and the warming induced by a doubling of CO

are nonline-

ar in the forcings applied (Hargreaves et al., 2007). Differences in the

cloud radiative feedback are at the origin of this asymmetric response

to equivalent positive and negative forcings (Yoshimori et al., 2009).

There is thus still low conﬁdence that the regional LGM model-data

comparisons can be used to evaluate model climate sensitivity. How-

ever, even if the results do not scale perfectly with equilibrium or tran-

sient climate sensitivity, the LGM simulations allow the identiﬁcation

of the different feedback factors that contributed to the LGM global

cooling (Yoshimori et al., 2011) and model spread in these feedbacks.

The largest spread in LGM model feedbacks is found for the shortwave

cloud feedback, just as for the modern climate. This correspondence

between LGM and modern climates adds to the high conﬁdence that

the shortwave cloud feedback is the dominant source of model spread

in climate sensitivity (cf. Section 5.3.3).

9.7.3.3 Constraints on Equilibrium Climate Sensitivity from

Climate-Model Ensembles and Observations

The large scale climatological information available has so far been

insufﬁcient to constrain model behaviour to a range tighter than

CMIP3, at least on a global scale. Sanderson and Knutti (2012) sug-

gest that much of the available and commonly used large scale obser-

vations have already been used to develop and evaluate models and

Table 9.6 | Model response metrics for EMICs in Table 9.2. TCR

, TCR

and ECS

are the changes in global average model surface air temperature from the decades centred

at years 70, 140 and 995 respectively, from the idealized 1% increase to 4 × CO

experiment. The ocean heat uptake efﬁciency, κ

, is calculated from the global average heat

ﬂux divided by TCR

for the decade centred at year 140, from the same idealized experiment. ECS

was calculated from the decade centred about year 995 from a 2 × CO

pulse

experiment. (Data from Eby et al., 2013.)

Model TCR

(°C) ECS

(°C) TCR

(°C) ECS

(°C) κ

(W m

–2

°C

–1

)

Bern3D 2.0 3.3 4.6 6.8 0.58

CLIMBER2 2.1 3.0 4.7 5.8 0.84

CLIMBER3 1.9 3.2 4.5 5.9 0.93

DCESS 2.1 2.8 3.9 4.8 0.72

FAMOUS 2.3 3.5 5.2 8.0 0.55

GENIE 2.5 4.0 5.4 7.0 0.51

IAP RAS CM 1.6 — 3.7 4.3 —

IGSM2 1.5 1.9 3.7 4.5 —

LOVECLIM1.2 1.2 2.0 2.1 3.5 1.17

MESMO 2.4 3.7 5.3 6.9 0.55

MIROC-lite 1.6 2.4 3.6 4.6 0.66

MIROC-lite-LCM 1.6 2.8 3.7 5.5 1.00

SPEEDO 0.8 3.6 2.9 5.2 0.84

UMD 1.6 2.2 3.2 4.3 —

Uvic 1.9 3.5 4.3 6.6 0.92

EMIC mean 1.8 3.0 4.0 5.6 0.8

EMIC range 0.8–2.5 1.9–4.0 2.1–5.4 3.5–8.0 0.5–1.2

are therefore of limited value to further constrain climate sensitivity

or TCR. The assessed literature suggests that the range of climate

sensitivities and transient responses covered by CMIP3/5 cannot be

narrowed signiﬁcantly by constraining the models with observations

of the mean climate and variability, consistent with the difﬁculty of

constraining the cloud feedbacks from observations (see Chapter 7).

Studies based on PPE and CMIP3 support the conclusion that a credi-

ble representation of the mean climate and variability is very difﬁcult

to achieve with equilibrium climate sensitivities below 2°C (Piani et al.,

2005; Stainforth et al., 2005; Sanderson et al., 2008a, 2008b; Huber et

al., 2011; Klocke et al., 2011; Fasullo and Trenberth, 2012). High climate

sensitivity values above 5°C (in some cases above 10°C) are found in

the PPE based on HadAM/HadCM3. Several recent studies ﬁnd that

such high values cannot be excluded based on climatological con-

straints, but comparison with observations shows the smallest errors

for many ﬁelds if ECS is between 3 and 4°C (Piani et al., 2005; Knutti

et al., 2006; Rodwell and Palmer, 2007; Sanderson et al., 2008a, 2008b,

2010; Sanderson, 2011, 2013).

9.8 Relating Model Performance to

Credibility of Model Applications

9.8.1 Synthesis Assessment of Model Performance

This chapter has assessed the performance of individual climate

models as well as the multi-model mean. In addition, changes between

models available now and those that were available at the time of the

AR4 have been documented. The models display a range of abilities to

822

Chapter 9 Evaluation of Climate Models

simulate climate characteristics, underlying processes, and phenome-

na. No model scores high or low in all performance metrics, but some

models perform substantially better than others for speciﬁc climate

variables or phenomena. For a few climate characteristics, the assess-

ment has shown that some classes of models, for example, those with

higher horizontal resolution, higher model top or a more complete

representation of the carbon cycle, aerosols or chemistry, agree better

with observations, although this is not universally true.

Figure 9.44 provides a synthesis of key model evaluation results for

AOGCMs and ESMs. The ﬁgure makes use of the calibrated language

as deﬁned in Mastrandrea et al. (2011). The x-axis refers to the level

of conﬁdence which increases towards the right as suggested by the

increasing strength of shading. The level of conﬁdence is a combina-

tion of the level of evidence and the degree of agreement. The level of

Very low Low Medium High Very high

Model Performance

Confidence in Assessment

Low Medium High

(a) Mean State

TAS

TropO3

TAS-diur

PR-diur

Blocking

Monsoon

NBP

fgCO2

fgCO2-sp

NBP-sp

SAF

ArcSIE

AntSIE

SST

SSS

SNC

MHT

TrPacific

TrInOcean

TrAtlantic CRE

ZTaux

TAS-RS

PR-RS

AMOC EqTaux

SMO

VAR-diur TrSST

SSS-RS

AOD

-t

Low Medium High

(b) Trends

OHC-t

TotalO3-t

TTT-t

LST-t

ArcSIE-t

AntSIE-t

TAS-t

Model Performance

Confidence in Assessment

NBP-t

fgCO2-t

Very low Low Medium High Very high

Low Medium High

NAO

SAM

AMO

CO2-iav

ENSO

IOD

QBO

PDO

PNA

ENSOtele

MJO

AMM

AMOC-var

SST-var

IPO

Model Performance

Confidence in Assessment

dCO2-iav

Very low Low Medium High Very high

IOB

Low Medium High

(d) Extremes

TAS-ext

Droughts

TAS-ext-t PR-ext-t

PR-ext PR-ext-hr

TC-hr

Hurric-hr

Model Performance

Confidence in Assessment

Very low Low Medium High Very high

Degradation since CMIP3

No changes since CMIP3

Improvements since CMIP3

No relative assessment CMIP3 vs. CMIP5

evidence includes the number of studies and quality of observation-

al data. Generally, evidence is most robust when there are multiple,

independent studies that evaluate multiple models using high-quality

observations. The degree of agreement measures whether different

studies come to the same conclusions or not. The ﬁgure shows that

several important aspects of the climate are simulated well by con-

temporary models, with varying levels of conﬁdence. The colour coding

provides an indication of how model quality has changed from CMIP3

to CMIP5. For example, there is high conﬁdence that the model perfor-

mance for global mean surface air temperature (TAS) is high, and it is

shown in green because there is robust evidence of improvement since

CMIP3. By contrast, the diurnal cycle of global mean surface air tem-

perature (TAS-diur) is simulated with medium performance, but there

is low conﬁdence in this assessment owing to as yet limited analy-

ses. It should be noted that there are no instances in the ﬁgure for

Figure 9.44 | Summary of the ﬁndings of Chapter 9 with respect to how well the CMIP5 models simulate important features of the climate of the 20th century. Conﬁdence in the

assessment increases towards the right as suggested by the increasing strength of shading. Model performance improves from bottom to top. The colour coding indicates changes

since CMIP3 (or models of that generation) to CMIP5. The assessment of model performance is expert judgment based on the agreement with observations of the multi-model

mean and distribution of individual models around the mean, taking into account internal climate variability. Note that assessed model performance is simpliﬁed for representation

in the ﬁgure and it is referred to the text for details of each assessment. The ﬁgure highlights the following key features, with the sections that back up the assessment added in

parentheses:

823

Evaluation of Climate Models Chapter 9

PANEL a:

AMOC Atlantic Meridional Overturning Circulation mean

(Section 9.4.2.3)

AntSIE Seasonal cycle Antarctic sea ice extent (Section 9.4.3)

AOD Aerosol Optical Depth (Section 9.4.6)

ArctSIE Seasonal cycle Arctic sea ice extent (Section 9.4.3)

Blocking Blocking events (Section 9.5.2.2)

CRE Cloud radiative effects (Section 9.4.1.2)

EqTaux Equatorial zonal wind stress (Section 9.4.2.4)

fgCO2 Global ocean carbon sink (Section 9.4.5)

fgCO2-sp Spatial pattern of ocean–atmosphere CO

ﬂuxes (Section 9.4.5)

MHT Meridional heat transport (Section 9.4.2.4)

Monsoon Global monsoon (Section 9.5.2.4)

NBP Global land carbon sink (Section 9.4.5)

NBP-sp Spatial pattern of land–atmosphere CO

ﬂuxes (Section 9.4.5)

PR Large scale precipitation (Sections 9.4.1.1, 9.4.1.3)

PR-diur Diurnal cycle precipitation (Section 9.5.2.1)

PR-RS Regional scale precipitation (Section 9.6.1.1)

SAF Snow albedo feedbacks (Section 9.8.3)

SMO Soil moisture (Section 9.4.4)

SNC Snow cover (Section 9.4.4)

SSS Sea surface salinity (Section 9.4.2.1)

SSS-RS Regional Sea surface salinity (Section 9.4.2.1)

SST Sea surface temperature (Section 9.4.2.1)

TAS Large scale surface air temperature (Sections 9.4.1.1, 9.4.1.3)

TAS-diur Diurnal cycle surface air temperature (Section 9.5.2.1)

TAS-RS Regional scale surface air temperature (Section 9.6.1.1)

TrSST Tropical sea surface temperature (Section 9.4.2.1)

TropO3 Tropospheric column ozone climatology (Section 9.4.1.4.5)

TrAtlantic Tropical Atlantic mean state (Section 9.4.2.5)

TrInOcean Tropical Indian Ocean mean state (Section 9.4.2.5)

TrPaciﬁc Tropical Paciﬁc mean state (Section 9.4.2.5)

VAR-diur Diurnal cycle other variables (Section 9.5.2.1)

ZTaux Zonal mean zonal wind stress (Section 9.4.2.4)

PANEL b (Trends)

AntSIE-t Trend in Antarctic sea ice extent (Section 9.4.3)

ArctSIE-t Trend in Arctic sea ice extent (Section 9.4.3)

fgCO2-t Global ocean carbon sink trends (Section 9.4.5)

LST-t Lower stratospheric temperature trends (Section 9.4.1.4.5)

NBP-t Global land carbon sink trends (Section 9.4.5)

OHC-t Global ocean heat content trends (Section 9.4.2.2)

TotalO3-t Total column ozone trends (Section 9.4.1.4.5)

TAS-t Surface air temperature trends (Section 9.4.1.4.1)

TTT-t Tropical tropospheric temperature trends (Section 9.4.1.4.2)

PANEL c (Variability):

AMM Atlantic Meridional Mode (Section 9.5.3.3)

AMO Atlantic Multi-decadal Variability (Section 9.5.3.3)

AMOC-var Atlantic Meridional Overturning Circulation (Section 9.5.3.3)

AN Atlantic Niño (Section 9.5.3.3)

CO2-iav Interannual variability of atmospheric CO

(Section 9.8.3)

dCO2-iav Sensitivity of CO

growth rate to tropical temperature

(Section 9.8.3)

ENSO El Niño Southern Oscillation (Section 9.5.3.4)

ENSOtele Tropical ENSO teleconnections (Section 9.5.3.5)

IOB Indian Ocean basin mode (Section 9.5.3.4)

IOD Indian Ocean dipole (Section 9.5.3.4)

IPO Interdecadal Paciﬁc Oscillation (Section 9.5.3.6)

MJO Madden-Julian Oscillation (Section 9.5.2.3)

NAO North Atlantic Oscillation and Northern annular mode

(Section 9.5.3.2)

PDO Paciﬁc Decadal Oscillation (Section 9.5.3.6)

PNA Paciﬁc North American (Section 9.5.3.5)

QBO Quasi-Biennial Oscillation (Section 9.5.3.7)

SAM Southern Annular Mode (Section 9.5.3.2)

SST-var Global sea surface temperature variability (Section 9.5.3.1)

PANEL d (Extremes):

Hurric-hr Year-to-year counts of Atlantic hurricanes in high-resolution

AGCMs (Section 9.5.4.3)

PR-ext Global distributions of precipitation extremes (Section 9.5.4.2)

PR-ext-hr Global distribution of precipitation extremes in high-resolution

AGCMs (Section 9.5.4.2)

PR-ext-t Global trends in precipitation extremes (Section 9.5.4.2)

TAS-ext Global distributions of surface air temperature extremes

(Section 9.5.4.1)

TAS-ext-t Global trends in surface air temperature extremes

(Section 9.5.4.1)

TC Tropical cyclone tracks and intensity (Section 9.5.4.3)

TC-hr Tropical cyclone tracks and intensity in high-resolution

AGCMs (Section 9.5.4.3)

Droughts Droughts (Section 9.5.4.4)

which CMIP5 models perform worse than CMIP3 models (something

that would have been indicated by the red colour). A description that

explains the expert judgment for each of the results presented in Figure

9.44 can be found in the body of this chapter, with a link to the speciﬁc

sections given in the ﬁgure caption.

EMICs have also been evaluated to some extent in this chapter as they

are used to provide long-term projections (in Chapter 12) beyond year

2300, and to provide large ensembles emulating the response of more

comprehensive ESMs and allowing probabilistic estimates. Results

from the EMIC intercomparison project (Eby et al., 2013; Zickfeld et al.,

2013) illustrate the ability to reproduce the large-scale climate chang-

es in GMST (Figure 9.8) and OHC (Figure 9.17) during the 20th century.

The models also estimate CO

ﬂuxes for land and oceans, which are as

consistent with observations as are ﬂuxes estimated by ESMs (Figure

9.27). This gives conﬁdence that the EMICs, albeit limited in the scope

and resolution of information they can provide, can be used for long-

term projections compatible with those of ESMs (Plattner et al., 2008;

Eby et al., 2013). Overall, these studies imply that EMICs are well suited

for simulations extending beyond the CMIP5 ensemble.

824

Chapter 9 Evaluation of Climate Models

Frequently Asked Questions

FAQ 9.1 | Are Climate Models Getting Better, and How Would We Know?

Climate models are extremely sophisticated computer programs that encapsulate our understanding of the climate

system and simulate, with as much ﬁdelity as currently feasible, the complex interactions between the atmosphere,

ocean, land surface, snow and ice, the global ecosystem and a variety of chemical and biological processes.

The complexity of climate models—the representation of physical processes like clouds, land surface interactions

and the representation of the global carbon and sulphur cycles in many models—has increased substantially since

the IPCC First Assessment Report in 1990, so in that sense, current Earth System Models are vastly ‘better’ than the

models of that era. This development has continued since the Fourth Assessment, while other factors have also

contributed to model improvement. More powerful supercomputers allow current models to resolve ﬁner spatial

detail. Today’s models also reﬂect improved understanding of how climate processes work—understanding that has

come from ongoing research and analysis, along with new and improved observations.

Climate models of today are, in principle, better than their predecessors. However, every bit of added complexity,

while intended to improve some aspect of simulated climate, also introduces new sources of possible error (e.g., via

uncertain parameters) and new interactions between model components that may, if only temporarily, degrade a

model’s simulation of other aspects of the climate system. Furthermore, despite the progress that has been made,

scientiﬁc uncertainty regarding the details of many processes remains.

An important consideration is that model performance

can be evaluated only relative to past observations,

taking into account natural internal variability. To have

conﬁdence in the future projections of such models, his-

torical climate—and its variability and change—must be

well simulated. The scope of model evaluation, in terms

of the kind and quantity of observations available, the

availability of better coordinated model experiments,

and the expanded use of various performance met-

rics, has provided much more quantitative information

about model performance. But this alone may not be

sufﬁcient. Whereas weather and seasonal climate pre-

dictions can be regularly veriﬁed, climate projections

spanning a century or more cannot. This is particularly

the case as anthropogenic forcing is driving the climate

system toward conditions not previously observed in the

instrumental record, and it will always be a limitation.

Quantifying model performance is a topic that has fea-

tured in all previous IPCC Working Group I Reports.

Reading back over these earlier assessments provides

a general sense of the improvements that have been

made. Past reports have typically provided a rather

broad survey of model performance, showing differenc-

es between model-calculated versions of various climate

quantities and corresponding observational estimates.

Inevitably, some models perform better than others for

certain climate variables, but no individual model clear-

ly emerges as ‘the best’ overall. Recently, there has been

progress in computing various performance metrics,

which synthesize model performance relative to a range

of different observations according to a simple numeri-

cal score. Of course, the deﬁnition of such a score, how

it is computed, the observations used (which have their

CMIP2 CMIP3

CMIP5

0.5

0.6

0.7

0.8

0.9

Pattern correlation

Precipitation

CMIP2 CMIP3

CMIP5

0.95

0.96

0.97

0.98

0.99

Pattern correlation

Surface Temperature

FAQ 9.1, Figure 1 | Model capability in simulating annual mean temperature

and precipitation patterns as illustrated by results of three recent phases of

the Coupled Model Intercomparison Project (CMIP2, models from about year

2000; CMIP3, models from about 2005; and CMIP5, the current generation

of models). The ﬁgure shows the correlation (a measure of pattern similarity)

between observed and modelled temperature (upper panel) and precipitation

(lower panel). Larger values indicate better correspondence between modelled

and observed spatial patterns. The black symbols indicate correlation coefﬁcient

for individual models, and the large green symbols indicate the median value

(i.e., half of the model results lie above and the other half below this value).

Improvement in model performance is evident by the increase in correlation for

successive model generations.

(continued on next page)

825

Evaluation of Climate Models Chapter 9

FAQ 9.1 (continued)

own uncertainties), and the manner in which various scores are combined are all important, and will affect the end

result.

Nevertheless, if the metric is computed consistently, one can compare different generations of models. Results

of such comparisons generally show that, although each generation exhibits a range in performance, the aver-

age model performance index has improved steadily between each generation. An example of changes in model

performance over time is shown in FAQ 9.1, Figure 1, and illustrates the ongoing, albeit modest, improvement. It

is interesting to note that both the poorest and best performing models demonstrate improvement, and that this

improvement comes in parallel with increasing model complexity and an elimination of artiﬁcial adjustments to

atmosphere and ocean coupling (so-called ‘ﬂux adjustment’). Some of the reasons for this improvement include

increased understanding of various climate processes and better representation of these processes in climate

models. More comprehensive Earth observations are also driving improvements.

So, yes, climate models are getting better, and we can demonstrate this with quantitative performance metrics

based on historical observations. Although future climate projections cannot be directly evaluated, climate models

are based, to a large extent, on veriﬁable physical principles and are able to reproduce many important aspects of

past response to external forcing. In this way, they provide a scientiﬁcally sound preview of the climate response to

different scenarios of anthropogenic forcing.

9.8.2 Implications of Model Evaluation for Climate

Change Detection and Attribution

The evaluation of model simulations of historical climate is of direct

relevance to detection and attribution (D&A) studies (Chapter 10)

since these rely on model-derived patterns (or ‘ﬁngerprints’) of climate

response to external forcing, and on the ability of models to simulate

decadal and longer-time scale internal variability (Hegerl and Zwiers,

2011). Conversely, D&A research contributes to model evaluation

through estimation of the amplitude of modeled response to vari-

ous forcings (Section 10.3.1.1.3). The estimated ﬁngerprint for some

variables such as water vapor is governed by basic physical process-

es that are well represented in models and are rather insensitive to

model uncertainties (Santer et al., 2009). Figure 9.44 illustrates slight

improvements in the representation of some of the modes of variability

and climate phenomena discussed in Sections 9.5.2 and 9.5.3, sug-

gesting with medium conﬁdence that models now better reproduce

internal variability. On the other hand, biases that affect D&A studies

remain. An example is the warm bias of lower-stratosphere tempera-

ture trends during the satellite period (Section 9.4.1.4.5) that can be

linked to uncertainties in stratospheric ozone forcing (Solomon et al.,

2012; Santer et al., 2013). Recent studies of climate extremes (Sec-

tion 9.5.4) also provide evidence that models have reasonable skill in

these important attributes of a changing climate; however, there is an

indication that models have difﬁculties in reproducing the right bal-

ance between historical changes in cold and warm extremes. They also

conﬁrm that resolution affects the conﬁdence that can be placed in the

analyses of extreme in precipitation. D&A studies focussed on extreme

events are therefore constrained by current model limitations. Lastly,

some D&A studies have incorporated model quality results by repeat-

ing a multi-model analysis with only the models that agree best with

observations (Santer et al., 2009). This model discrimination or weight-

ing is less problematic for D&A analysis than it is for model projections

of future climate (Section 9.8.3), because D&A research is focussed on

historical and control-run simulations which can be directly evaluated

against observations.

9.8.3 Implications of Model Evaluation for Model

Projections of Future Climate

Conﬁdence in climate model projections is based on physical under-

standing of the climate system and its representation in climate

models, and on a demonstration of how well models represent a wide

range of processes and climate characteristics on various spatial and

temporal scales (Knutti et al., 2010b). A climate model’s credibility is

increased if the model is able to simulate past variations in climate,

such as trends over the 20th century and palaeoclimatic changes. Pro-

jections from previous IPCC assessments can also be directly compared

to observations (see Figures 1.4 and 1.5), with the caveat that these

projections were not intended to be predictions over the short time

scales for which observations are available to date. Unlike shorter lead

forecasts, longer-term climate change projections push models into

conditions outside the range observed in the historical period used for

evaluation.

In some cases, the spread in climate projections can be reduced by

weighting of models according to their ability to reproduce past

observed climate. Several studies have explored the use of unequally

weighted means, with the weights based on the models’ performance

in simulating past variations in climate, typically using some perfor-

mance metric or collection of metrics (Connolley and Bracegirdle,

2007; Murphy et al., 2007; Waugh and Eyring, 2008; Pierce et al., 2009;

Reifen and Toumi, 2009; Christensen et al., 2010; Knutti et al., 2010b;

Raisanen et al., 2010; Abe et al., 2011; Shiogama et al., 2011; Wat-

terson and Whetton, 2011; Tsushima et al., 2013). When applied to

projections of Arctic sea ice, averages in which extra weight is given

826

Chapter 9 Evaluation of Climate Models

to models with the most realistic historical sea ice do give different

results than the unweighted mean (Stroeve et al., 2007, 2012; Scher-

rer, 2011; Massonnet et al., 2012; Wang and Overland, 2012; Overland

and Wang, 2013). Another frequently used approach is the re-calibra-

tion of model outputs to a given observed value (Boe et al., 2009b;

Mahlstein and Knutti, 2012; Wang and Overland, 2012), see further

discussion in Section 12.4.6.1. Some studies explicitly formulate a

statistical frameworks that relate future observables to climate model

output (reviewed in Knutti et al. (2010b) and Stephenson et al. (2012)).

Such frameworks not only provide weights for the mean response but

also allow the uncertainty in the predicted response to be quantiﬁed

(Bracegirdle and Stephenson, 2012).

There are several encouraging examples of ‘emergent constraints’,

which are relationships across an ensemble of models between some

aspect of Earth System sensitivity and an observable trend or varia-

tion in the contemporary climate (Allen and Ingram, 2002; Hall and

Qu, 2006; Eyring et al., 2007; Boe et al., 2009a, 2009b; Mahlstein

and Knutti, 2010; Son et al., 2010; Huber et al., 2011; Schaller et al.,

2011; Bracegirdle and Stephenson, 2012; Fasullo and Trenberth, 2012;

O’Gorman, 2012). For example, analyzing the CMIP3 ensemble, Hall

and Qu (2006) showed that inter-model variations of snow albedo

feedback in the contemporary seasonal cycle strongly correlate with

comparably large inter-model variations in this feedback under future

climate change. An update of this analysis with CMIP5 models added

is shown in Figure 9.45 (left panel). This relationship presumably arises

from the fact that surface albedo values in areas covered by snow vary

widely across the models, particularly in the heavily vegetated boreal

forest zone. Models with higher surface albedos in these areas have a

larger contrast between snow-covered and snow-free areas, and hence

a stronger snow albedo feedback whether the context is the seasonal

variation in sunshine or anthropogenic forcing. Comparison with an

observational estimate of snow albedo feedback reveals a large spread

with both high and low biases.

The right panel of Figure 9.45 shows another example of an emergent

constraint, where the sensitivity of tropical land carbon to warming

(i.e., without CO

fertilization effects) is related to the sensitivity of the

annual CO

growth rate to tropical temperature anomalies (Cox et al.,

2013) ). The horizontal axis is the regression of the atmospheric CO

growth rate on the tropical temperature anomaly for each model. The

strong statistical relationship between these two variables is consist-

ent with the fact that interannual variability in the CO

growth-rate is

known to be dominated by the response of tropical land to climatic

anomalies, associated particularly with ENSO. Thus the relationship has

a physical as well as a statistical basis. The interannual sensitivity of

the CO

growth rate to tropical temperature can be estimated from

observational data. Like the snow albedo feedback example, this inter-

model relationship provides a credible means to reduce model spread

in the sensitivity of tropical land carbon to tropical climate change.

On the other hand, many studies have failed to ﬁnd strong relation-

ships between observables and projections. Whetton et al. (2007) and

Knutti et al. (2010a) found that correlations between local to region-

al climatological values and projected changes are small except for

a few regions. Scherrer (2011) ﬁnds no robust relationship between

the ability of the CMIP3 models to represent interannual variability

of near-surface air temperature and the amplitude of future warm-

ing.Raisanen et al. (2010) report only small (10–20%) reductions in

cross-validation error of simulated 21st century temperature changes

when weighting the CMIP3 models based on their simulation of the

present-day climatology. The main difﬁculties are sparse coverage in

Figure 9.45 | (Left) Scatterplot of simulated springtime snow–albedo feedback (Δα

/ΔT

) values in climate change (y-axis) versus simulated springtime Δα

/ΔT

values in the

seasonal cycle (x-axis) in transient climate change experiments from 17 CMIP3 (blue) and 24 CMIP5 models (α

and T

are surface albedo and surface air temperature, respectively).

(Adapted from Hall and Qu, 2006.) (Right) Constraint on the climate sensitivity of land carbon in the tropics (30°N to 30°S) from interannual variability in the growth rate of global

atmospheric CO

(Cox et al., 2013). This is based on results from Earth System Models (ESMs) with free-running CO

; Coupled Climate Carbon Cycle Model Intercomparison Project

General Circulation Models (C4MIP GCMs, black labels; Friedlingstein et al., 2006), and three land carbon ‘physics ensembles’ with HadCM3 (red labels; Booth et al., 2012b) . The

values on the y-axis are calculated over the period 1960–2099 inclusive, and those on the x-axis over the period 1960–2010 inclusive. In both cases the temperature used is the

mean (land+ocean) temperature over 30°N to 30°S. The width of the vertical yellow bands in both (a) and (b) shows the observation-based estimate of the variable on the x-axis.

827

Evaluation of Climate Models Chapter 9

many observed variables, short time series for observed trends, lack of

correlation between observed quantities and projected past or future

trends, and systematic errors in the models (Tebaldi and Knutti, 2007;

Jun et al., 2008; Knutti, 2010; Knutti et al., 2010a), the ambiguity of

possible performance metrics and the difﬁculty of associating them

with predictive skill (Parker et al., 2007; Gleckler et al., 2008; Pincus

et al., 2008; Reichler and Kim, 2008; Pierce et al., 2009; Knutti et al.,

2010a).

Emergent constraints can be difﬁcult to identify if climate models are

structurally similar and share common biases, thereby reducing the

effective ensemble size. Comparison of emergent constraints in MMEs

from different modelling experiments can help reveal which constraints

are robust (Massonnet et al., 2012; Bracegirdle and Stephenson, 2013).

Another issue is that testing of large numbers of predictors will ﬁnd

statistically signiﬁcant correlations that do not remain signiﬁcant in

a different ensemble. This is particularly important if many predictors

are tested using only small ensembles like CMIP3 (DelSole and Shukla,

2009; Raisanen et al., 2010; Huber et al., 2011; Masson and Knutti,

2013). All of these potential pitfalls underscore the need for analysis

of the mechanism underpinning the statistical relationship between

current and future climate parameters for any proposed emergent

constraint.

828

Chapter 9 Evaluation of Climate Models

References

Abe, M., H. Shiogama, T. Nozawa, and S. Emori, 2011: Estimation of future surface

temperature changes constrained using the future-present correlated modes

in inter-model variability of CMIP3 multimodel simulations. J. Geophys. Res.

Atmos., 116, D18104.

Abiodun, B., W. Gutowski, A. Abatan, and J. Prusa, 2011: CAM-EULAG: A non-

hydrostatic atmospheric climate model with grid stretching. Acta Geophys., 59,

1158–1167.

Abramowitz, G., R. Leuning, M. Clark, and A. Pitman, 2008: Evaluating the

performance of land surface models. J. Clim., 21, 5468–5481.

AchutaRao, K., and K. Sperber, 2002: Simulation of the El Niño Southern Oscillation:

Results from the coupled model intercomparison project. Clim. Dyn., 19, 191–

209.

AchutaRao, K., and K. Sperber, 2006: ENSO simulations in coupled ocean-atmosphere

models: Are the current models better? Clim. Dyn., 27, 1–16.

Achuthavarier, D., V. Krishnamurthy, B. P. Kirtman, and B. H. Huang, 2012: Role of the

Indian Ocean in the ENSO-Indian Summer Monsoon Teleconnection in the NCEP

Climate Forecast System. J. Clim., 25, 2490–2508.

Ackerley, D., E. J. Highwood, and D. J. Frame, 2009: Quantifying the effects of

perturbing the physics of an interactive sulfur scheme using an ensemble of

GCMs on the climateprediction.net platform. J. Geophys. Res. Atmos., 114,

D01203

Adachi, Y., et al., 2013: Basic performance of a new earth system model of the

Meteorological Research Institute (MRI-ESM1). Papers Meteorol. Geophys.,

doi:10.2467/mripapers.64.1.

Adam, J., E. Clark, D. Lettenmaier, and E. Wood, 2006: Correction of global

precipitation products for orographic effects. J. Clim., 19, 15–38.

Adkins, J. F., K. McIntyre, and D. P. Schrag, 2002: The salinity, temperature, and delta

O-18 of the glacial deep ocean. Science, 298, 1769–1773.

Adler, R. F., et al., 2003: The Version 2 Global Precipitation Climatology Project (GPCP)

Monthly Precipitation Analysis (1979–Present). J. Hydrometeor., 4, 1147–1167.

Alekseev, V. A., E. M. Volodin, V. Y. Galin, V. P. Dymnikov, and V. N. Lykossov, 1998:

Modeling of the present-day climate by the atmospheric model of INM RAS

DNM GCM. Description of the model version A5421 and results of AMIP2

simulations. Institute of Numerical Mathematics, Moscow, Russia, 200 pp.

Alessandri, A., P. G. Fogli, M. Vichi, and N. Zeng, 2012: Strengthening of the

hydrological cycle in future scenarios: Atmospheric energy and water balance

perspective. Earth Syst. Dyn., 3, 199–212.

Alexander, M. J., et al., 2010: Recent developments in gravity-wave effects in climate

models and the global distribution of gravity-wave momentum ﬂux from

observations and models. Q. J. R. Meteorol. Soc., 136, 1103–1124.

Alexandru, A., R. de Elia, and R. Laprise, 2007: Internal variability in regional climate

downscaling at the seasonal scale. Mon. Weather Rev., 135 3221–3238.

Alexandru, A., R. de Elia, R. Laprise, L. Separovic, and S. Biner, 2009: Sensitivity study

of regional climate model simulations to large-scale nudging parameters. Mon.

Weather Rev., 137, 1666–1686.

Allan, R. P., and B. J. Soden, 2008: Atmospheric warming and the ampliﬁcation of

precipitation extremes. Science, 321, 1481–1484.

Allan, R. P., M. A. Ringer, and A. Slingo, 2003: Evaluation of moisture in the Hadley

Centre climate model using simulations of HIRS water-vapour channel radiances.

Q. J. R. Meteorol. Soc., 129, 3371–3389.

Allan, R. P., A. Slingo, S. F. Milton, and M. E. Brooks, 2007: Evaluation of the Met

Ofﬁce global forecast model using Geostationary Earth Radiation Budget

(GERB) data. Q. J. R. Meteorol. Soc., 133, 1993–2010.

Allan, R. P., B. J. Soden, V. O. John, W. Ingram, and P. Good, 2010: Current changes in

tropical precipitation. Environ. Res. Lett., 5, 025205.

Allen, M., P. Stott, J. Mitchell, R. Schnur, and T. Delworth, 2000: Quantifying the

uncertainty in forecasts of anthropogenic climate change. Nature, 407, 617–

620.

Allen, M. R., and W. J. Ingram, 2002: Constraints on future changes in climate and

the hydrologic cycle. Nature, 419, 224–232.

Ammann, C. M., G. A. Meehl, W. M. Washington, and C. S. Zender, 2003: A monthly

and latitudinally varying volcanic forcing dataset in simulations of 20th century

climate. Geophys. Res. Lett., 30, 1657.

Anav, A., et al., 2013: Evaluating the land and ocean components of the global

carbon cycle in the CMIP5 Earth System Models. J. Clim., 26, 6801–6843.

Anderson, B. T., J. R. Knight, M. A. Ringer, C. Deser, A. S. Phillips, J. H. Yoon, and

A. Cherchi, 2010: Climate forcings and climate sensitivities diagnosed from

atmospheric global circulation models. Clim. Dyn., 35, 1461–1475.

Andrews, T., J. M. Gregory, M. J. Webb, and K. E. Taylor, 2012: Forcing, feedbacks

and climate sensitivity in CMIP5 coupled atmosphere-ocean climate models.

Geophys. Res. Lett., 39, L09712.

Annamalai, H., and K. R. Sperber, 2005: Regional heat sources and the active and

break phases of boreal summer intraseasonal (30–50 day) variability. J. Atmos.

Sci., 62, 2726–2748.

Annan, J., and J. Hargreaves, 2011: Understanding the CMIP3 Multimodel Ensemble.

J. Clim., 24, 4529–4538.

Annan, J. D., and J. C. Hargreaves, 2010: Reliability of the CMIP3 ensemble. Geophys.

Res. Lett., 37, L02703.

Annan, J. D., D. J. Lunt, J. C. Hargreaves, and P. J. Valdes, 2005: Parameter estimation in

an atmospheric GCM using the Ensemble Kalman Filter. Nonlin. Proc. Geophys.,

12, 363–371.

Anstey, J. A., et al., 2013: Multi-model analysis of Northern Hemisphere winter

blocking and its relation to the stratosphere. J. Geophys. Res. Atmos., 118,

3956–3971.

Antonov, J. I., et al., 2010: World Ocean Atlas 2009, Vol. 2: Salinity. [S. Levitus (eds.)].

NOAA Atlas NESDIS 69, U.S. Gov. Printing Ofﬁce, Washington, D.C., 184 pp.

Archer, D. E., G. Eshel, A. Winguth, W. Broecker, R. Pierrehumbert, M. Tobis, and R.

Jacob, 2000: Atmospheric pCO(2) sensitivity to the biological pump in the ocean.

Global Biogeochem. Cycles, 14, 1219–1230.

Arneth, A., et al., 2010: From biota to chemistry and climate: Towards a

comprehensive description of trace gas exchange between the biosphere and

atmosphere. Biogeosciences, 7, 121–149.

Arora, V. K., and G. J. Boer, 2005: Fire as an interactive component of dynamic

vegetation models. J. Geophys. Res.-Biogeosciences, 110, G02008.

Arora, V. K., and G. J. Boer, 2010: Uncertainties in the 20th century carbon budget

associated with land use change. Global Change Biol., 16, 3327–3348.

Arora, V. K., et al., 2011: Carbon emission limits required to satisfy future

representative concentration pathways of greenhouse gases. Geophys. Res.

Lett., 38, L05805.

Arora, V. K., et al., 2009: The effect of terrestrial photosynthesis down regulation on

the twentieth-century carbon budget simulated with the CCCma Earth System

Model. J. Clim., 22, 6066–6088.

Artale, V., et al., 2010: An atmosphere–ocean regional climate model for the

Mediterranean area: Assessment of a present climate simulation. Clim. Dyn.,

35, 721–740.

Arzhanov, M. M., P. F. Demchenko, A. V. Eliseev, and I. I. Mokhov, 2008: Simulation of

characteristics of thermal and hydrologic soil regimes in equilibrium numerical

experiments with a Climate Model of Intermediate Complexity. Izvestiya Atmos.

Ocean. Phys., 44, 548–566.

Assmann, K. M., M. Bentsen, J. Segschneider, and C. Heinze, 2010: An isopycnic

ocean carbon cycle model. Geosci. Model Dev., 3, 143–167.

Aumont, O., and L. Bopp, 2006: Globalizing results from ocean in situ iron fertilization

studies. Global Biogeochem. Cycles, 20, Gb2017.

Aumont, O., E. Maier-Reimer, S. Blain, and P. Monfray, 2003: An ecosystem model

of the global ocean including Fe, Si, P colimitations. Global Biogeochem. Cycles,

17, 1060.

Austin, J., and R. J. Wilson, 2006: Ensemble simulations of the decline and recovery

of stratospheric ozone. J. Geophys. Res. Atmos., 111, D16314.

Axelsson, P., M. Tjernström, S. Söderberg, and G. Svensson, 2011: An ensemble of

Arctic simulations of the AOE-2001 ﬁeld experiment. Atmosphere, 2, 146–170.

Baehr, J., S. Cunnningham, H. Haak, P. Heimbach, T. Kanzow, and J. Marotzke, 2009:

Observed and simulated estimates of the meridional overturning circulation at

26.5 N in the Atlantic. Ocean Sci., 5, 575–589.

Balan Sarojini, B., et al., 2011: High frequency variability of the Atlantic meridional

overturning circulation. Ocean Science, 7, 471–486.

Baldwin, M. P., et al., 2001: The quasi-biennial oscillation. Rev. Geophys., 39, 179–

229.

Balsamo, G., P. Viterbo, A. Beljaars, B. van den Hurk, M. Hirschi, A. K. Betts, and K.

Scipal, 2009: A revised hydrology for the ECMWF Model: Veriﬁcation from ﬁeld

site to terrestrial water storage and impact in the Integrated Forecast System. J.

Hydrometeorol., 10, 623–643.

829

Evaluation of Climate Models Chapter 9

Bao, Q., G. Wu, Y. Liu, J. Yang, Z. Wang, and T. Zhou, 2010: An introduction to the

coupled model FGOALS1.1-s and its performance in East Asia. Adv. Atmos. Sci.,

27, 1131–1142.

Bao, Q., et al., 2013: The Flexible Global Ocean-Atmosphere-Land System model

Version: FGOALS-s2. Adv. Atmos. Sci., doi:10.1007/s00376-012-2113-9.

Bao, Y., F. L. Qiao, and Z. Y. Song, 2012: Historical simulation and twenty-ﬁrst century

prediction of oceanic CO

sink and pH change. Acta Ocean. Sin., 31, 87–97.

Barkstrom, B. R., 1984: The Earth Radiation Budget Experiment (ERBE). Bull. Am.

Meteorol. Soc., 65, 1170–1185.

Barnes, E. A., and D. L. Hartmann, 2010: Inﬂuence of eddy-driven jet latitude on

North Atlantic jet persistence and blocking frequency in CMIP3 integrations.

Geophys. Res. Lett., 37, L23802.

Barnes, E. A., J. Slingo, and T. Woollings, 2012: A methodology for the comparison of

blocking climatologies across indices, models and climate scenarios. Clim. Dyn.,

38, 2467–2481.

Barnier, B., et al., 2006: Impact of partial steps and momentum advection schemes

in a global ocean circulation model at eddy-permitting resolution. Ocean Dyn.,

56, 543–567.

Barriopedro, D., R. Garcia-Herrera, and R. M. Trigo, 2010a: Application of blocking

diagnosis methods to General Circulation Models. Part I: A novel detection

scheme. Clim. Dyn., 35, 1373–1391.

Barriopedro, D., R. Garcia-Herrera, J. F. Gonzalez-Rouco, and R. M. Trigo, 2010b:

Application of blocking diagnosis methods to General Circulation Models. Part

II: Model simulations. Clim. Dyn., 35, 1393–1409.

Bartlein, P. J., et al., 2010: Pollen-based continental climate reconstructions at 6 and

21 ka: A global synthesis. Clim. Dyn., 37, 775–802.

Bathiany, S., M. Claussen, V. Brovkin, T. Raddatz, and V. Gayler, 2010: Combined

biogeophysical and biogeochemical effects of large-scale forest cover changes

in the MPI earth system model. Biogeosciences, 7, 1383–1399.

Bauer, H. S., V. Wulfmeyer, and L. Bengtsson, 2008a: The representation of a synoptic-

scale weather system in a thermodynamically adjusted version of the ECHAM4

general circulation model. Meteorol. Atmos. Phys., 99, 129–153.

Bauer, S. E., D. Koch, N. Unger, S. M. Metzger, D. T. Shindell, and D. G. Streets, 2007:

Nitrate aerosols today and in 2030: A global simulation including aerosols and

tropospheric ozone. Atmos. Chem. Phys., 7, 5043–5059.

Bauer, S. E., et al., 2008b: MATRIX (Multiconﬁguration Aerosol TRacker of mIXing

state): An aerosol microphysical module for global atmospheric models. Atmos.

Chem. Phys., 8, 6003–6035.

Beare, R., et al., 2006: An intercomparison of large-eddy simulations of the Stable

Boundary Layer. Boundary-Layer Meteorol., 118, 247–272.

Bellassen, V., G. Le Maire, J. F. Dhote, P. Ciais, and N. Viovy, 2010: Modelling forest

management within a global vegetation model Part 1: Model structure and

general behaviour. Ecol. Model., 221, 2458–2474.

Bellassen, V., G. le Maire, O. Guin, J. F. Dhote, P. Ciais, and N. Viovy, 2011: Modelling

forest management within a global vegetation model-Part 2: Model validation

from a tree to a continental scale. Ecol. Model., 222, 57–75.

Bellouin, N., J. Rae, A. Jones, C. Johnson, J. Haywood, and O. Boucher, 2011: Aerosol

forcing in the Climate Model Intercomparison Project (CMIP5) simulations by

HadGEM2–ES and the role of ammonium nitrate. J. Geophys. Res., 116, 1–25.

Bellucci, A., S. Gualdi, and A. Navarra, 2010: The Double-ITCZ Syndrome in Coupled

General Circulation Models: The role of large-scale vertical circulation regimes.

J. Clim., 23, 1127–1145.

Bender, M. A., T. R. Knutson, R. E. Tuleya, J. J. Sirutis, G. A. Vecchi, S. T. Garner, and I.

M. Held, 2010: Modeled impact of anthropogenic warming on the frequency of

intense Atlantic hurricanes. Science, 327, 454–458.

Bengtsson, L., and K. Hodges, 2011: On the evaluation of temperature trends in the

tropical troposphere. Clim. Dyn., 36, 419–430.

Bengtsson, L., K. I. Hodges, and N. Keenlyside, 2009: Will extratropical storms

intensify in a warmer climate? J. Clim., 22, 2276–2301.

Berckmans, J., T. Woollings, M.-E. Demory, P.-L. Vidal, and M. Roberts, 2013:

Atmospheric blocking in a high resolution climate model: Inﬂuences of mean

state, orography and eddy forcing. Atmos. Sci. Lett., 14, 34–40.

Bergengren, J., D. Waliser, and Y. Yung, 2011: Ecological sensitivity: A biospheric view

of climate change. Clim. Change, 107, 433–457.

Bergengren, J., S. Thompson, D. Pollard, and R. DeConto, 2001: Modeling global

climate-vegetation interactions in a doubled CO

world. Clim. Change, 50,

31–75.

Bernie, D. J., E. Guilyardi, G. Madec, J. M. Slingo, S. Woolnough, and J. Cole, 2008:

Impact of resolving the diurnal cycle in an ocean-atmosphere GCM. Part 2: A

diurnally coupled CGCM. Clim. Dyn., 31, 909–925.

Bi, D., et al., 2013a: ACCESS-OM: The Ocean and Sea ice Core of the ACCESS Coupled

Model. Aust. Meteorol. Oceanogr. J., 63, 213–232.

Bi, D., et al., 2013b: The ACCESS Coupled Model: Description, control climate and

evaluation. Aust. Meteorol. Oceanogr. J., 63, 41–64.

Bitz, C. M., and W. H. Lipscomb, 1999: An energy-conserving thermodynamic sea ice

model for climate study. J. Geophys. Res.. Oceans, 104, 15669–15677.

Blyth, E., J. Gash, A. Lloyd, M. Pryor, G. P. Weedon, and J. Shuttleworth, 2010:

Evaluating the JULES Land Surface Model Energy Fluxes Using FLUXNET Data. J.

Hydrometeorol., 11, 509–519.

Boberg, F., and J. H. Christensen, 2012: Overestimation of Mediterranean summer

temperature projections due to model deﬁciencies. Nature Clim. Change, 2,

433–436.

Boccaletti, G., R. Ferrari, and B. Fox-Kemper, 2007: Mixed layer instabilities and

restratiﬁcation. J. Phys. Oceanogr., 37, 2228–2250.

Bodas-Salcedo, A., K. D. Williams, P. R. Field, and A. P. Lock, 2012: The surface

downwelling solar radiation surplus over the Southern Ocean in the Met Ofﬁce

Model: The role of midlatitude cyclone clouds. J. Clim., 25, 7467–7486.

Bodas-Salcedo, A., M. Webb, M. Brooks, M. Ringer, K. Williams, S. Milton, and D.

Wilson, 2008: Evaluating cloud systems in the Met Ofﬁce global forecast model

using simulated CloudSat radar reﬂectivities. J. Geophys. Res. Atmos., 113,

D00A13.

Bodas-Salcedo, A., et al., 2011: COSP: Satellite simulation software for model

assessment. Bull. Am. Meteorol. Soc., 92, 1023–1043.

Bodeker, G., H. Shiona, and H. Eskes, 2005: Indicators of Antarctic ozone depletion.

Atmos. Chem. Phys., 5, 2603–2615.

Boe, J., A. Hall, and X. Qu, 2009a: Deep ocean heat uptake as a major source of

spread in transient climate change simulations. Geophys. Res. Lett., 36, L22701.

Boe, J., L. Terray, F. Habets, and E. Martin, 2007: Statistical and dynamical downscaling

of the Seine basin climate for hydro-meteorological studies. Int. J. Climatol. , 27,

1643–1655.

Boe, J. L., A. Hall, and X. Qu, 2009b: September sea-ice cover in the Arctic Ocean

projected to vanish by 2100. Nature Geosci., 2, 341–343.

Boer, G., and S. Lambert, 2008: The energy cycle in atmospheric models. Clim. Dyn.,

30, 371–390.

Boer, G. J., and B. Yu, 2003: Climate sensitivity and climate state. Clim. Dyn., 21,

167–176.

Boisier, J.-P., et al., 2012: Attributing the biogeophysical impacts of land-use induced

Land-Cover Changes on surface climate to speciﬁc causes. Results from the ﬁrst

LUCID set of simulations. J. Geophys. Res., 117, D12116.

Bollasina, M. A., and Y. Ming, 2013: The general circulation model precipitation

bias over the southwestern equatorial Indian Ocean and its implications for

simulating the South Asian monsoon. Clim. Dyn., 40, 823–838.

Bonan, G. B., 2008: Forests and climate change: Forcings, feedbacks, and the climate

beneﬁts of forests. Science, 320, 1444–1449.

Bond, T. C., et al., 2007: Historical emissions of black and organic carbon aerosol

from energy-related combustion, 1850–2000. Global Biogeochem. Cycles, 21,

GB2018.

Bondeau, A., P. C. Smith, S. Zaehle, S. Schaphoff, W. Lucht, W. Cramer, and D. Gerten,

2007: Modelling the role of agriculture for the 20th century global terrestrial

carbon balance. Global Change Biol., 13, 679–706.

Boning, C. W., A. Dispert, M. Visbeck, S. R. Rintoul, and F. U. Schwarzkopf, 2008: The

response of the Antarctic Circumpolar Current to recent climate change. Nature

Geosci., 1, 864–869.

Boone, A., et al., 2009: THE AMMA Land Surface Model Intercomparison Project

(ALMIP). Bull. Am. Meteorol. Soc., 90, 1865–1880.

Boone, A. A., I. Poccard-Leclercq, Y. K. Xue, J. M. Feng, and P. de Rosnay, 2010:

Evaluation of the WAMME model surface ﬂuxes using results from the AMMA

land-surface model intercomparison project. Clim. Dyn., 35, 127–142.

Booth, B. B. B., N. J. Dunstone, P. R. Halloran, T. Andrews, and N. Bellouin, 2012a:

Aerosols implicated as a prime driver of twentieth-century North Atlantic

climate variability. Nature, 484, 228–232.

Booth, B. B. B., et al., 2012b: High sensitivity of future global warming to land carbon

cycle processes. Environ. Res. Lett., 7, 024002.

Boschat, G., P. Terray, and S. Masson, 2012: Robustness of SST teleconnections and

precursory patterns associated with the Indian summer monsoon. Clim. Dyn.,

38, 2143–2165.

830

Chapter 9 Evaluation of Climate Models

Boyle, J., and S. A. Klein, 2010: Impact of horizontal resolution on climate model

forecasts of tropical precipitation and diabatic heating for the TWP-ICE period.

J. Geophys. Res., 115, D23113.

Boyle, J., S. Klein, G. Zhang, S. Xie, and X. Wei, 2008: Climate Model Forecast

Experiments for TOGA COARE. Mon. Weather Rev., 136, 808–832.

Bracegirdle, T., et al., 2013: Assessment of surface winds over the Atlantic, Indian

and Paciﬁc Ocean sectors of the Southern Hemisphere in CMIP5 models:

Historical bias, forcing response, and state dependence. J. Geophys. Res. Atmos.,

doi:10.1002/jgrd.50153.

Bracegirdle, T. J., and D. B. Stephenson, 2012: Higher precision estimates of regional

polar warming by ensemble regression of climate model projections. Clim. Dyn.,

39, 2805–2821.

Bracegirdle, T. J., and D. B. Stephenson, 2013: On the robustness of emergent

constraints used in multi-model climate change projections of Arctic warming.

J. Clim., 26, 669–678.

Braconnot, P., F. Hourdin, S. Bony, J. Dufresne, J. Grandpeix, and O. Marti, 2007a:

Impact of different convective cloud schemes on the simulation of the tropical

seasonal cycle in a coupled ocean-atmosphere model. Clim. Dyn., 29, 501–520.

Braconnot, P., et al., 2012: Evaluation of climate models using palaeoclimatic data.

Nature Clim. Change, 2, 417–424.

Braconnot, P., et al., 2007b: Results of PMIP2 coupled simulations of the Mid-

Holocene and Last Glacial Maximum - Part 2: Feedbacks with emphasis on the

location of the ITCZ and mid- and high latitudes heat budget. Clim. Past, 3,

279–296.

Braconnot, P., et al., 2007c: Results of PMIP2 coupled simulations of the Mid-

Holocene and Last Glacial Maximum - Part 1: Experiments and large-scale

features. Clim. Past, 3, 261–277.

Brands, S., J. Taboada, A. Coﬁno, T. Sauter, and C. Schneider, 2011: Statistical

downscaling of daily temperatures in the NW Iberian Peninsula from global

climate models: Validation and future scenarios. Clim. Res., 48, 163–176.

Bresson, R., and R. Laprise, 2011: Scale-decomposed atmospheric water budget over

North America as simulated by the Canadian Regional Climate Model for current

and future climates. Clim. Dyn., 36, 365–384.

Breugem, W. P., W. Hazeleger, and R. J. Haarsma, 2006: Multimodel study of tropical

Atlantic variability and change. Geophys. Res. Lett., 33, L23706.

Brewer, S., J. Guiot, and F. Torre, 2007: Mid-Holocene climate change in Europe: A

data-model comparison. Clim. Past, 3, 499–512.

Briegleb, B. P., and B. Light, 2007: A Delta-Eddington multiple scattering

parameterization for solar radiation in the sea ice component of the Community

Climate System Model. NCAR Technical Note, National Center for Atmospheric

Research, 100 pp.

Briegleb, B. P., C. M. Blitz, E. C. Hunke, W. H. Lipscomb, M. M. Holland, J. L. Schramm,

and R. E. Moritz, 2004: Scientiﬁc description of the sea ice component in the

Community Climate System Model, Version 3. NCAR Technical Note, National

Center for Atmospheric Research, 70 pp.

Brient, F., and S. Bony, 2012: Interpretation of the positive low-cloud feedback

predicted by a climate model under global warming. Clim. Dyn., doi:10.1007/

s00382–011–1279–7.

Brierley, C. M., M. Collins, and A. J. Thorpe, 2010: The impact of perturbations to

ocean-model parameters on climate and climate change in a coupled model.

Clim. Dyn., 34, 325–343.

Brogniez, H., and R. T. Pierrehumbert, 2007: Intercomparison of tropical tropospheric

humidity in GCMs with AMSU-B water vapor data. Geophys. Res. Lett., 34,

L17812

Brogniez, H., R. Roca, and L. Picon, 2005: Evaluation of the distribution of subtropical

free tropospheric humidity in AMIP-2 simulations using METEOSAT water vapor

channel data. Geophys. Res. Lett., 32, L19708.

Brovkin, V., J. Bendtsen, M. Claussen, A. Ganopolski, C. Kubatzki, V. Petoukhov, and A.

Andreev, 2002: Carbon cycle, vegetation, and climate dynamics in the Holocene:

Experiments with the CLIMBER-2 model. Global Biogeochem. Cycles, 16, 1139.

Brown, A., S. Milton, M. Cullen, B. Golding, J. Mitchell, and A. Shelly, 2012: Uniﬁed

modeling and prediction of weather and climate: A 25-year journey. Bull. Am.

Meteorol. Soc., 93, 1865–1877.

Brown, J., A. Fedorov, and E. Guilyardi, 2010a: How well do coupled models replicate

ocean energetics relevant to ENSO? Clim. Dyn., 36, 2147–2158.

Brown, J., O. J. Ferrians, J. A. Heginbottom, and E. S. E.S. Melnikov, 1997: International

Permafrost Association Circum-Arctic Map of Permafrost and Ground Ice

Conditions. Geological Survey (U.S.), Denver, CO, USA.

Brown, J., O. J. Ferrians, J. A. Heginbottom, and E. S. E.S. Melnikov, 1998: Digital

circum-arctic map of permafrost and ground ice conditions. In: Circumpolar

Active-Layer Permafrost System (CAPS). CD-ROM. 1.0 ed., University of Colorado

at Boulder National Snow and Ice Data Center. Boulder, CO, USA.

Brown, J. R., C. Jakob, and J. M. Haynes, 2010b: An evaluation of rainfall frequency

and intensity over the Australian region in a Global Climate Model. J. Clim., 23,

6504–6525.

Brown, J. R., A. F. Moise, and R. A. Colman, 2013: The South Paciﬁc Convergence Zone

in CMIP5 simulations of historical and future climate. Clim. Dyn., doi:10.1007/

s00382-012-1591–x.

Brutel-Vuilmet, C., M. Menegoz, and G. Krinner, 2013: An analysis of present and

future seasonal Northern Hemisphere land snow cover simulated by CMIP5

coupled climate models. Cryosphere, 7, 67–80.

Bryan, F. O., M. W. Hecht, and R. D. Smith, 2007: Resolution convergence and

sensitivity studies with North Atlantic circulation models. Part I: The western

boundary current system. Ocean Model., 16, 141–159.

Bryan, F. O., R. Tomas, J. M. Dennis, D. B. Chelton, N. G. Loeb, and J. L. McClean, 2010:

Frontal scale air-sea interaction in high-resolution coupled climate models. J.

Clim., 23, 6277–6291.

Bryan, K., and L. J. Lewis, 1979: Water mass model of the world ocean. J. Geophys.

Res. Oceans Atmos., 84, 2503–2517.

Bryden, H. L., H. R. Longworth, and S. A. Cunningham, 2005: Slowing of the Atlantic

meridional overturning circulation at 25° N. Nature, 438, 655–657.

Buehler, T., C. C. Raible, and T. F. Stocker, 2011: The relationship of winter season

North Atlantic blocking frequencies to extreme cold or dry spells in the ERA-40.

Tellus A, 63, 212–222.

Butchart, N., A. A. Scaife, J. Austin, S. H. E. Hare, and J. R. Knight, 2003: Quasi-biennial

oscillation in ozone in a coupled chemistry-climate model. J. Geophys. Res., 108,

4486.

Cadule, P., et al., 2010: Benchmarking coupled climate-carbon models against long-

term atmospheric CO

measurements. Global Biogeochem. Cycles, 24, Gb2016.

Cai, W., and T. Cowan, 2013: Why is the amplitude of the Indian Ocean Dipole overly

large in CMIP3 and CMIP5 climate models? . Geophys. Res. Lett., doi:10.1002/

grl.50208.

Cai, W., A. Sullivan, and T. Cowan, 2011: Interactions of ENSO, the IOD, and the SAM

in CMIP3 Models. J. Clim., 24, 1688–1704.

Cai, W. J., A. Sullivan, and T. Cowan, 2009: Rainfall teleconnections with Indo-Paciﬁc

variability in the WCRP CMIP3 models. J. Clim., 22, 5046–5071.

Calov, R., A. Ganopolski, V. Petoukhov, M. Claussen, and R. Greve, 2002: Large-scale

instabilities of the Laurentide ice sheet simulated in a fully coupled climate-

system model. Geophys. Res. Lett., 29, 2216.

Cameron-Smith, P., J. F. Lamarque, P. Connell, C. Chuang, and F. Vitt, 2006: Toward an

Earth system model: Atmospheric chemistry, coupling, and petascale computing.

Scidac 2006: Scientiﬁc Discovery through Advanced Computing [W. M. Tang

(ed.)]. Journal of Physics: Conference Series, Vol. 46, Denver, Colorado, USA.

Capotondi, A., A. Wittenberg, and S. Masina, 2006: Spatial and temporal structure

of Tropical Paciﬁc interannual variability in 20th century coupled simulations.

Ocean Model., 15, 274–298.

Capotondi, A., M. A. Alexander, N. A. Bond, E. N. Curchitser, and J. D. Scott, 2012:

Enhanced upper ocean stratiﬁcation with climate change in the CMIP3 models.

J. Geophys. Res. Oceans 117, C04031.

Cariolle, D., and H. Teyssedre, 2007: A revised linear ozone photochemistry

parameterization for use in transport and general circulation models: Multi-

annual simulations. Atmos. Chem. Phys., 7, 2183–2196.

Carslaw, K. S., O. Boucher, D. V. Spracklen, G. W. Mann, J. G. L. Rae, S. Woodward, and

M. Kulmala, 2010: A review of natural aerosol interactions and feedbacks within

the Earth system. Atmos. Chem. Phys., 10, 1701–1737.

Casado, M. J., and M. A. Pastor, 2012: Use of variability modes to evaluate AR4

climate models over the Euro-Atlantic region. Clim. Dyn., 38, 225–237.

Cattiaux, J., H. Douville, and Y. Peings, 2013: European temperatures in CMIP5:

Origins of present-day biases and future uncertainties. Clim. Dyn., doi:10.1007/

s00382-013-1731-y.

Catto, J., N. Nicholls, and C. Jakob, 2012a: North Australian sea surface temperatures

and the El Niño Southern Oscillation in observations and models. J. Clim., 25,

5011–5029.

Catto, J., N. Nicholls, and C. Jakob, 2012b: North Australian sea surface temperatures

and the El Niño Southern Oscillation in the CMIP5 models. J. Clim., 25, 6375–

6382.

831

Evaluation of Climate Models Chapter 9

Catto, J. L., L. C. Shaffrey, and K. I. Hodges, 2010: Can climate models capture the

structure of extratropical cyclones? J. Clim., 23, 1621–1635.

Catto, J. L., L. C. Shaffrey, and K. I. Hodges, 2011: Northern Hemisphere Extratropical

cyclones in a warming climate in the HiGEM high-resolution climate Model. J.

Clim., 24, 5336–5352.

Catto, J. L., C. Jakob, and N. Nicholls, 2013: A global evaluation of fronts and

precipitation in the ACCESS model. Aust. Meteorol. Oceanogr. J., 63,191-203.

Cavalieri, D. J., and C. L. Parkinson, 2012: Arctic sea ice variability and trends, 1979–

2010. Cryosphere, 6, 881–889.

Cavicchia, L., and H. von Storch, 2011: The simulation of medicanes in a high-

resolution regional climate model. Clim. Dyn., 39 2273–2290.

Cesana, G., and H. Chepfer, 2012: How well do climate models simulate cloud vertical

structure? A comparison between CALIPSO-GOCCP satellite observations and

CMIP5 models. Geophys. Res. Lett., 39, L20803.

Cha, D., D. Lee, and S. Hong, 2008: Impact of boundary layer processes on seasonal

simulation of the East Asian summer monsoon using a Regional Climate Model.

Meteorol. Atmos. Phys., 100, 53–72.

Champion, A. J., K. I. Hodges, L. O. Bengtsson, N. S. Keenlyside, and M. Esch, 2011:

Impact of increasing resolution and a warmer climate on extreme weather from

Northern Hemisphere extratropical cyclones. Tellus A, 63, 893–890.

Chan, S. C., E. J. Kendon, H. J. Fowler, S. Blenkinsop, C. A. T. Ferro, and D. B. Stephenson,

2012: Does increasing resolution improve the simulation of United Kingdom

daily precipitation in a regional climate model? Clim. Dyn., doi:10.1007/s00382-

012-1568-9.

Chang, C. P., and T. Li, 2000: A theory for the tropical tropospheric biennial oscillation.

J. Atmos. Sci., 57, 2209–2224.

Chang, C. Y., S. Nigam, and J. A. Carton, 2008: Origin of the springtime westerly

bias in equatorial Atlantic surface winds in the Community Atmosphere Model

version 3 (CAM3) simulation. J. Clim., 21, 4766–4778.

Chang, C. Y., J. A. Carton, S. A. Grodsky, and S. Nigam, 2007: Seasonal climate of the

tropical Atlantic sector in the NCAR community climate system model 3: Error

structure and probable causes of errors. J. Clim., 20, 1053–1070.

Chang, E. K. M., Y. Guo, and X. Xia, 2012: CMIP5 multi-model ensemble projection

of storm track change under global warming. J. Geophys. Res., 117, D23118.

Charbit, S., D. Paillard, and G. Ramstein, 2008: Amount of CO

emissions irreversibly

leading to the total melting of Greenland. Geophys. Res. Lett., 35, L12503.

Charlton-Perez, A. J., et al., 2012: Mean climate and variability of the stratosphere in

CMIP5 models. J. Geophys. Res., doi:10.1002/jgrd.50125.

Chen, C. T., and T. Knutson, 2008: On the veriﬁcation and comparison of extreme

rainfall indices from climate models. J. Clim., 21, 1605–1621.

Chen, H. M., T. J. Zhou, R. B. Neale, X. Q. Wu, and G. J. Zhang, 2010: Performance of

the New NCAR CAM3.5 in East Asian summer monsoon simulations: Sensitivity

to modiﬁcations of the Convection Scheme. J. Clim., 23, 3657–3675.

Chen, L., Y. Yu, and D. Sun, 2013: Cloud and water vapor feedbacks to the El Niño

warming: Are they still biased in CMIP5 models? J. Clim., doi:10.1175/JCLI-D-

12-00575.1.

Chen, Y. H., and A. D. Del Genio, 2009: Evaluation of tropical cloud regimes in

observations and a general circulation model. Clim. Dyn., 32, 355–369.

Chiang, J. C. H., and A. H. Sobel, 2002: Tropical tropospheric temperature variations

caused by ENSO and their inﬂuence on the remote tropical climate. J. Clim., 15,

2616–2631.

Chiang, J. C. H., and D. J. Vimont, 2004: Analogous Paciﬁc and Atlantic meridional

modes of tropical atmosphere-ocean variability. J. Clim., 17, 4143–4158.

Chou, C., and J. Y. Tu, 2008: Hemispherical asymmetry of tropical precipitation in

ECHAM5/MPI-OM during El Niño and under global warming. J. Clim., 21, 1309–

1332.

Chou, C., J. D. Neelin, J. Y. Tu, and C. T. Chen, 2006: Regional tropical precipitation

change mechanisms in ECHAM4/OPYC3 under global warming. J. Clim., 19,

4207–4223.

Chou, S., et al., 2012: Downscaling of South America present climate driven by

4-member HadCM3 runs. Clim. Dyn., 38, 635–653.

Christensen, J., F. Boberg, O. Christensen, and P. Lucas-Picher, 2008: On the need

for bias correction of regional climate change projections of temperature and

precipitation. Geophys. Res. Lett., 35, L20709.

Christensen, J., E. Kjellstrom, F. Giorgi, G. Lenderink, and M. Rummukainen, 2010:

Weight assignment in regional climate models. Clim. Res., 44, 179–194.

Christensen, J. H., and F. Boberg, 2013: Temperature dependent climate projection

deﬁciencies in CMIP5 models. Geophys. Res. Lett., 39, L24705.

Christian, J. R., et al., 2010: The global carbon cycle in the Canadian Earth system

model (CanESM1): Preindustrial control simulation. J. Geophys. Res. Biogeosci.,

115, G03014

Christidis, N., P. A. Stott, and S. J. Brown, 2011: The role of human activity in the recent

warming of extremely warm daytime temperatures. J. Clim., 24, 1922–1930.

Christy, J. R., W. B. Norris, R. W. Spencer, and J. J. Hnilo, 2007: Tropospheric temperature

change since 1979 from tropical radiosonde and satellite measurements. J.

Geophys. Res. Atmos., 112, D06102.

Christy, J. R., et al., 2010: What do observational datasets say about modeled

tropospheric temperature trends since 1979? Remote Sens., 2, 2148–2169.

Cimatoribus, A. A., S. S. Drijfhout, and H. A. Dijkstra, 2012: A global hybrid coupled

model based on atmosphere-SST feedbacks. Clim. Dyn., 38, 745–760.

Cionni, I., et al., 2011: Ozone database in support of CMIP5 simulations: Results and

corresponding radiative forcing. Atmos. Chem. Phys., 11, 11267–11292.

Clark, D. B., et al., 2011: The Joint UK Land Environment Simulator (JULES), model

description - Part 2: Carbon ﬂuxes and vegetation dynamics. Geosci. Model Dev.,

4, 701–722.

Claussen, M., et al., 2002: Earth system models of intermediate complexity: Closing

the gap in the spectrum of climate system models. Clim. Dyn., 18, 579–586.

Coelho, C. A. S., and L. Goddard, 2009: El Niño-induced tropical droughts in climate

change projections. J. Clim., 22, 6456–6476.

Cohen, J. L., J. C. Furtado, M. Barlow, V. A. Alexeev, and J. E. Cherry, 2012: Asymmetric

seasonal temperature trends. Geophys. Res. Lett., 39, L04705.

Collatz, G. J., M. Ribas-Carbo, and J. A. Berry, 1992: Coupled photosynthesis-stomatal

conductance model for leaves of C4 Plants. Aust. J. Plant Physiol., 19, 519–538.

Collatz, G. J., J. T. Ball, C. Grivet, and J. A. Berry, 1991: Physiological and environmental

regulation of stomatal conductance, photosynthesis and transpiration: A model

that includes a laminar boundary layer. Agr. Forest Meteorol., 54, 107–136.

Colle, B. A., Z. Zhang, K. A. Lombardo, E. Chang, P. Liu, and M. Zhang, 2013: Historical

evaluation and future prediction of eastern North America and western Atlantic

extratropical cyclones in the CMIP5 models during the cool season. J. Clim.,

doi:10.1175/JCLI-D-12–00498.1.

Collins, M., S. Tett, and C. Cooper, 2001: The internal climate variability of HadCM3,

a version of the Hadley Centre coupled model without ﬂux adjustments. Clim.

Dyn., 17, 61–81.

Collins, M., C. M. Brierley, M. MacVean, B. B. B. Booth, and G. R. Harris, 2007: The

sensitivity of the rate of transient climate change to ocean physics perturbations.

J. Clim., 20, 2315–2320.

Collins, M., B. B. B. Booth, G. R. Harris, J. M. Murphy, D. M. H. Sexton, and M. J. Webb,

2006a: Towards quantifying uncertainty in transient climate change. Clim. Dyn.,

27, 127–147.

Collins, M., R. Chandler, P. Cox, J. Huthnance, J. Rougier, and D. Stephenson, 2012:

Quantifying future climate change. Nature Clim. Change, 2, 403–409.

Collins, M., B. Booth, B. Bhaskaran, G. Harris, J. Murphy, D. Sexton, and M. Webb,

2010: Climate model errors, feedbacks and forcings: A comparison of perturbed

physics and multi-model ensembles. Clim. Dyn., 36, 1737–1766.

Collins, W. D., J. M. Lee-Taylor, D. P. Edwards, and G. L. Francis, 2006b: Effects of

increased near-infrared absorption by water vapor on the climate system. J.

Geophys. Res. Atmos., 111, D18109.

Collins, W. D., et al., 2006c: The formulation and atmospheric simulation of the

Community Atmosphere Model version 3 (CAM3). J. Clim., 19, 2144–2161.

Collins, W. D., et al., 2006d: The Community Climate System Model version 3

(CCSM3). J. Clim., 19, 2122–2143.

Collins, W. J., et al., 2011: Development and evaluation of an Earth-System model-

HadGEM2. Geosci. Model Dev., 4, 1051–1075.

Colman, R., and B. McAvaney, 2009: Climate feedbacks under a very broad range of

forcing. Geophys. Res. Lett., 36, L01702.

Colman, R. A., A. F. Moise, and L. I. Hanson, 2011: Tropical Australian climate and the

Australian monsoon as simulated by 23 CMIP3 models. J. Geophys. Res. Atmos.,

116, D10116.

Comiso, J. C., and F. Nishio, 2008: Trends in the sea ice cover using enhanced and

compatible AMSR-E, SSM/I, and SMMR data. J. Geophys. Res. Oceans, 113,

C02s07.

Compo, G. P., and P. D. Sardeshmukh, 2009: Oceanic inﬂuences on recent continental

warming. Clim. Dyn., 32, 333–342.

Connolley, W., and T. Bracegirdle, 2007: An Antarctic assessment of IPCC AR4

coupled models. Geophys. Res. Lett., 34 L22505.

832

Chapter 9 Evaluation of Climate Models

Coon, M., R. Kwok, G. Levy, M. Pruis, H. Schreyer, and D. Sulsky, 2007: Arctic

Ice Dynamics Joint Experiment (AIDJEX) assumptions revisited and found

inadequate. J. Geophys. Res., 112, C11S90.

Coppola, E., F. Giorgi, S. Rauscher, and C. Piani, 2010: Model weighting based

on mesoscale structures in precipitation and temperature in an ensemble of

regional climate models. Clim. Res., 44 121–134.

Cox, P., 2001: Description of the “TRIFFID” Dynamic GlobalVegetation ModelHadley

Centre, Met Ofﬁce Hadley Centre, Berks, United Kingdom, 16 pp.

Cox, P. M., R. A. Betts, C. D. Jones, S. A. Spall, and I. J. Totterdell, 2000: Acceleration

of global warming due to carbon-cycle feedbacks in a coupled climate model.

Nature, 408, 184–187.

Cox, P. M., R. A. Betts, C. B. Bunton, R. L. H. Essery, P. R. Rowntree, and J. Smith, 1999:

The impact of new land surface physics on the GCM simulation of climate and

climate sensitivity. Clim. Dyn., 15, 183–203.

Cox, P. M., D. Pearson, B. B. B. Booth, P. Friedlingstein, C. Huntingford, C. D. Jones, and

C. M. Luke, 2013: Sensitivity of tropical carbon to climate change constrained by

carbon dioxide variability. Nature, 494, 341–344.

Cramer, W., et al., 2001: Global response of terrestrial ecosystem structure and

function to CO

and climate change: Results from six dynamic global vegetation

models. Global Change Biol., 7, 357–373.

Crétat, J., B. Pohl, Y. Richard, and P. Drobinski, 2012: Uncertainties in simulating

regional climate of Southern Africa: Sensitivity to physical parameterizations

using WRF. Clim. Dyn., 38, 613–634.

Croft, B., U. Lohmann, and K. von Salzen, 2005: Black carbon ageing in the Canadian

Centre for Climate modelling and analysis atmospheric general circulation

model. Atmos. Chem. Phys., 5, 1931–1949.

Cruciﬁx, M., 2006: Does the Last Glacial Maximum constrain climate sensitivity?

Geophys. Res. Lett., 33, L18701.

Cunningham, S., et al., 2010: The present and future system for measuring the

Atlantic meridional overturning circulation and heat transport. In: Proceedings

of OceanObs’09: Sustained Ocean Observations and Information for Society (Vol.

2), Venice, Italy, 21–25 September 2009, ESA Publication.

Cunningham, S. A., S. G. Alderson, B. A. King, and M. A. Brandon, 2003: Transport and

variability of the Antarctic Circumpolar Current in Drake Passage. J. Geophys.

Res.-Oceans, 108, 8084.

Cunningham, S. A., et al., 2007: Temporal variability of the Atlantic meridional

overturning circulation at 26.5°N. Science, 317, 935–938.

Curry, W. B., and D. W. Oppo, 2005: Glacial water mass geometry and the distribution

of delta C-13 of sigma CO

in the western Atlantic Ocean. Paleoceanography,

20, Pa1017.

Cuxart, J., et al., 2006: Single-column model intercomparison for a stably stratiﬁed

atmospheric boundary layer. Boundary-Layer Meteorol., 118, 273–303.

Dai, A., 2001: Global precipitation and thunderstorm frequencies. Part II: Diurnal

variations. J. Clim., 14, 1112–1128.

Dai, A., 2006: Precipitation characteristics in eighteen coupled climate models. J.

Clim., 19, 4605–4630.

Dai, A., and C. Deser, 1999: Diurnal and semidiurnal variations in global surface wind

and divergence ﬁelds. J. Geophys. Res. Atmos., 104, 31109–31125.

Dai, A., and K. E. Trenberth, 2004: The diurnal cycle and its depiction in the Community

Climate System Model. J. Clim., 17, 930–951.

Dai, Y. J., R. E. Dickinson, and Y. P. Wang, 2004: A two-big-leaf model for canopy

temperature, photosynthesis, and stomatal conductance. J. Clim., 17, 2281–

2299.

Dai, Y. J., et al., 2003: The Common Land Model. Bull. Am. Meteorol. Soc., 84, 1013–

1023.

Dallmeyer, A., M. Claussen, and J. Otto, 2010: Contribution of oceanic and vegetation

feedbacks to Holocene climate change in monsoonal Asia. Clim. Past, 6, 195–

218.

Danabasoglu, G., and P. R. Gent, 2009: Equilibrium climate sensitivity: Is it accurate

to use a Slab Ocean Model? J. Clim., 22, 2494–2499.

Danabasoglu, G., R. Ferrari, and J. C. McWilliams, 2008: Sensitivity of an ocean

general circulation model to a parameterization of near-surface eddy ﬂuxes. J.

Clim., 21, 1192–1208.

Danabasoglu, G., W. G. Large, and B. P. Briegleb, 2010: Climate impacts of

parameterized Nordic Sea overﬂows. J. Geophys. Res. Oceans, 115, C11005.

Danabasoglu, G., W. G. Large, J. J. Tribbia, P. R. Gent, B. P. Briegleb, and J. C.

McWilliams, 2006: Diurnal coupling in the tropical oceans of CCSM3. J. Clim.,

19, 2347–2365.

Danabasoglu, G., et al., 2012: The CCSM4 Ocean Component. J. Clim., 25, 1361–

1389.

Davies, T., M. J. P. Cullen, A. J. Malcolm, M. H. Mawson, A. Staniforth, A. A. White, and

N. Wood, 2005: A new dynamical core for the Met Ofﬁce’s global and regional

modelling of the atmosphere. Q. J. R. Meteorol. Soc., 131, 1759–1782.

Davis, B. A. S., and S. Brewer, 2009: Orbital forcing and role of the latitudinal

insolation/temperature gradient. Clim. Dyn., 32, 143–165.

Dawson, A., T. N. Palmer, and S. Corti, 2012: Simulating regime structures in weather

and climate prediction models. Geophys. Res. Lett., 39, L21805.

Day, J. J., J. C. Hargreaves, J. D. Annan, and A. Abe-Ouchi, 2012: Sources of multi-

decadal variability in Arctic sea ice extent. Environ. Res. Lett., 7, 034011.

de Elia, R., and H. Cote, 2010: Climate and climate change sensitivity to model

conﬁguration in the Canadian RCM over North America. Meteorol. Z., 19, 325–

339.

de Elia, R., S. Biner, and A. Frigon, 2013: Interannual variability and expected regional

climate change over North America. Clim. Dyn., doi:10.1007/s00382-013-1717-

de Jong, M. F., S. S. Drijfhout, W. Hazeleger, H. M. van Aken, and C. A. Severijns,

2009: Simulations of hydrographic properties in the Northwestern North Atlantic

Ocean in Coupled Climate Models. J. Clim., 22, 1767–1786.

de Noblet-Ducoudre, N., et al., 2012: Determining robust impacts of land-use

induced land-cover changes on surface climate over North America and Eurasia;

Results from the ﬁrst set of LUCID experiments. J. Clim., 25, 3261–3281.

De Szoeke, S. P., and S. P. Xie, 2008: The tropical eastern Paciﬁc seasonal cycle:

Assessment of errors and mechanisms in IPCC AR4 coupled ocean - atmosphere

general circulation models. J. Clim., 21, 2573–2590.

Dee, D. P., et al., 2011: The ERA-Interim reanalysis: Conﬁguration and performance of

the data assimilation system. Q. J. R. Meteorol. Soc., 137, 553–597.

DelSole, T., and J. Shukla, 2009: Artiﬁcial skill due to predictor screening. J. Clim.,

22, 331–345.

Delworth, T. L., et al., 2012: Simulated climate and climate change in the GFDL

CM2.5 High-Resolution Coupled Climate Model. J. Clim., 25, 2755–2781.

Delworth, T. L., et al., 2006: GFDL’s CM2 global coupled climate models. Part I:

Formulation and simulation characteristics. J. Clim., 19, 643–674.

Déqué, M., 2007: Frequency of precipitation and temperature extremes over France

in an anthropogenic scenario: Model results and statistical correction according

to observed values. Global Planet. Change, 57, 16–26.

Déqué, M., 2010: Regional climate simulation with a mosaic of RCMs. Meteorol. Z.,

19, 259–266.

Déqué, M., C. Dreveton, A. Braun, and D. Cariolle, 1994: The ARPEGE/IFS atmosphere

model: A contribution to the French community climate modelling. Clim. Dyn.,

10, 249–266.

Déqué, M., S. Somot, E. Sanchez-Gomez, C. Goodess, D. Jacob, G. Lenderink, and

O. Christensen, 2012: The spread amongst ENSEMBLES regional scenarios:

Regional climate models, driving general circulation models and interannual

variability. Clim. Dyn., 38, 951–964.

Derksen, C., and R. Brown, 2012: Spring snow cover extent reductions in the

2008–2012 period exceeding climate model projections. Geophys. Res. Lett., 39,

L19504.

Deser, C., A. S. Phillips, and J. W. Hurrell, 2004: Paciﬁc interdecadal climate variability:

Linkages between the tropics and the North Paciﬁc during boreal winter since

1900. J. Clim., 17, 3109–3124.

Deser, C., A. S. Phillips, V. Bourdette, and H. Teng, 2011: Uncertainty in climate change

projections: The role of internal variability. Clim. Dyn., 38, 527–546.

Deser, C., et al., 2012: ENSO and Paciﬁc decadal variability in Community Climate

System Model Version 4. J. Clim., 25, 2622–2651.

Deushi, M., and K. Shibata, 2011: Development of a Meteorological Research

Institute Chemistry-Climate Model version 2 for the Study of Tropospheric and

Stratospheric Chemistry. Papers Meteorol. Geophys., 62, 1–46.

Di Luca, A., R. Elía, and R. Laprise, 2012: Potential for small scale added value of

RCM’s downscaled climate change signal. Clim. Dyn., 40, 601–618.

Diaconescu, E. P., and R. Laprise, 2013: Can added value be expected in RCM-

simulated large scales? Clim. Dyn., doi:10.1007/s00382-012-1649-9.

Diffenbaugh, N., M. Ashfaq, and M. Scherer, 2011: Transient regional climate

change: Analysis of the summer climate response in a high-resolution, century-

scale ensemble experiment over the continental United States. J. Geophys. Res.

Atmos., 116, D24111.

DiNezio, P. N., A. C. Clement, G. A. Vecchi, B. J. Soden, and B. P. Kirtman, 2009: Climate

response of the equatorial Paciﬁc to global warming. J. Clim., 22, 4873–4892.

833

Evaluation of Climate Models Chapter 9

Dix, M., et al., 2013: The ACCESS Coupled Model: Documentation of core CMIP5

simulations and initial results. Aust. Meteorol. Oceanogr. J., 63, 83–99.

Doblas-Reyes, F. J., et al., 2013: Initialized near-term regional climate change

prediction. Nature Commun., 4, 1715.

Dokken, T. M., and E. Jansen, 1999: Rapid changes in the mechanism of ocean

convection during the last glacial period. Nature, 401, 458–461.

Domingues, C., J. Church, N. White, P. Gleckler, S. Wijffels, P. Barker, and J. Dunn, 2008:

Improved estimates of upper-ocean warming and multi-decadal sea-level rise.

Nature, 453, 1090–1093.

Donat, M., G. Leckebusch, S. Wild, and U. Ulbrich, 2010: Beneﬁts and limitations

of regional multi-model ensembles for storm loss estimations. Clim. Res., 44,

211–225.

Donat, M. G., et al., 2013: Updated analyses of temperature and precipitation

extreme indices since the beginning of the twentieth century: The HadEX2

dataset. J. Geophys. Res., doi:10.1002/2012JD018606.

Donner, L. J., et al., 2011: The dynamical core, physical parameterizations, and basic

simulation characteristics of the atmospheric component AM3 of the GFDL

Global Coupled Model CM3. J. Clim., 24, 3484–3519.

Dorn, W., K. Dethloff, and A. Rinke, 2009: Improved simulation of feedbacks between

atmosphere and sea ice over the Arctic Ocean in a coupled regional climate

model. Ocean Model., 29, 103–114.

Doscher, R., K. Wyser, H. E. M. Meier, M. W. Qian, and R. Redler, 2010: Quantifying

Arctic contributions to climate predictability in a regional coupled ocean-ice-

atmosphere model. Clim. Dyn., 34, 1157–1176.

Douglass, D., J. Christy, B. Pearson, and S. Singer, 2008: A comparison of tropical

temperature trends with model predictions. Int. J. Climatol. , 28, 1693–1701.

Dowdy, A. J., G. A. Mills, B. Timbal, and Y. Wang, 2013: Changes in the risk of

extratropical cyclones in Eastern Australia. J. Clim., 26, 1403–1417.

Driesschaert, E., et al., 2007: Modeling the inﬂuence of Greenland ice sheet melting

on the Atlantic meridional overturning circulation during the next millennia.

Geophys. Res. Lett., 34, L10707.

Driouech, F., M. Deque, and E. Sanchez-Gomez, 2010: Weather regimes-Moroccan

precipitation link in a regional climate change simulation. Global Planet.

Change, 72, 1–10.

Driscoll, S., A. Bozzo, L. J. Gray, A. Robock, and G. Stenchikov, 2012: Coupled Model

Intercomparison Project 5 (CMIP5) simulations of climate following volcanic

eruptions. J. Geophys. Res. Atmos., 117, D17105.

Druyan, L. M., et al., 2010: The WAMME regional model intercomparison study. Clim.

Dyn., 35, 175–192.

Du, Y., S.-P. Xie, Y.-L. Yang, X.-T. Zheng, L. Liu, and G. Huang, 2013: Indian Ocean

variability in the CMIP5 multi-model ensemble: The basin mode. J. Clim., 26,

7240–7266.

Ducet, N., P. Y. Le Traon, and G. Reverdin, 2000: Global high-resolution mapping

of ocean circulation from TOPEX/Poseidon and ERS-1 and-2. J. Geophys. Res.

Oceans, 105, 19477–19498.

Dufresne, J.-L., et al., 2012: Climate change projections using the IPSL-CM5 Earth

System Model: From CMIP3 to CMIP5. Clim. Dyn., doi:10.1007/s00382-012-

1636-1.

Dufresne, J. L., and S. Bony, 2008: An assessment of the primary sources of spread

of global warming estimates from coupled atmosphere-ocean models. J. Clim.,

21, 5135–5144.

Dunkerton, T. J., 1991: Nonlinear propagation of zonal winds in an atmosphere with

Newtonian cooling and equatorial wavedriving. J. Atmos. Sci., 48, 236–263.

Dunn-Sigouin, E., and S.-W. Son, 2013: Northern Hemisphere blocking frequency and

duration in the CMIP5 models. J. Geophys. Res., 118, 1179–1188.

Dunne, J. P., et al., 2013: GFDL’s ESM2 global coupled climate-carbon Earth

System Models Part II: Carbon system formulation and baseline simulation

characteristics. J. Clim., doi:10.1175/JCLI-D-12-00150.1.

Dunne, J. P., et al., 2012: GFDL’s ESM2 Global coupled climate-carbon Earth System

models. Part I: Physical formulation and baseline simulation characteristics. J.

Clim., 25, 6646–6665.

Duplessy, J. C., N. J. Shackleton, R. Fairbanks, L. Labeyrie, D. Oppo, and N. Kallel, 1988:

Deep water source variation during the last climatic cycle and their impact on th

global deep water circulation. Paleoceanography, 3, 343–360.

Durack, P. J., and S. E. Wijffels, 2010: Fifty-year trends in global ocean salinities and

their relationship to broad-scale warming. J. Clim., 23, 4342-4362.

Durack, P. J., S. E. Wijffels, and R. J. Matear, 2012: Ocean salinities reveal strong

global water cycle intensiﬁcation during 1950 to 2000. Science, 336, 455–458.

Easterling, D. R., and M. F. Wehner, 2009: Is the climate warming or cooling?

Geophys. Res. Lett., 36, L08706.

Eby, M., K. Zickfeld, A. Montenegro, D. Archer, K. J. Meissner, and A. J. Weaver, 2009:

Lifetime of anthropogenic climate change: Millennial time scales of potential

and surface temperature perturbations. J. Clim., 22, 2501–2511.

Eby, M., et al., 2013: Historical and idealized climate model experiments:An EMIC

intercomparison. Clim. Past, 9, 1111–1140.

Edwards, N., and R. Marsh, 2005: Uncertainties due to transport-parameter

sensitivity in an efﬁcient 3-D ocean-climate model. Clim. Dyn., 24, 415–433.

Edwards, N. R., D. Cameron, and J. Rougier, 2011: Precalibrating an intermediate

complexity climate model. Clim. Dyn., 37, 1469–1482.

Ek, M. B., et al., 2003: Implementation of Noah land surface model advances in the

National Centers for Environmental Prediction operational mesoscale Eta model.

J. Geophys. Res. Atmos., 108, 8851.

Eliseev, A. V., and I. I. Mokhov, 2011: Uncertainty of climate response to natural and

anthropogenic forcings due to different land use scenarios. Adv. Atmos. Sci., 28,

1215–1232.

Emanuel, K., R. Sundararajan, and J. Williams, 2008: Hurricanes and global

warming—Results from downscaling IPCC AR4 simulations. Bull. Am. Meteorol.

Soc., 89, 347–367.

Endo, H., A. Kitoh, T. Ose, R. Mizuta, and S. Kusunoki, 2012: Future changes and

uncertainties in Asian precipitation simulated by multiphysics and multi-sea

surface temperature ensemble experiments with high-resolution Meteorological

Research Institute atmospheric general circulation models (MRI-AGCMs). J.

Geophys. Res. Atmos., 117, D16118.

Essery, R. L. H., M. J. Best, R. A. Betts, P. M. Cox, and C. M. Taylor, 2003: Explicit

representation of subgrid heterogeneity in a GCM land surface scheme. J.

Hydrometeorol., 4, 530–543.

Eum, H., P. Gachon, R. Laprise, and T. Ouarda, 2012: Evaluation of regional climate

model simulations versus gridded observed and regional reanalysis products

using a combined weighting scheme. Clim. Dyn., 38, 1433–1457.

Evans, J. P., M. Ekstroem, and F. Ji, 2012: Evaluating the performance of a WRF

physics ensemble over South-East Australia. Clim. Dyn., 39, 1241–1258.

Eyring, V., et al., 2010: Transport impacts on atmosphere and climate: Shipping.

Atmos. Environ., 44, 4735–4771.

Eyring, V., et al., 2013: Long-term ozone changes and associated climate impacts in

CMIP5 simulations. J. Geophys. Res., doi:10.1002/jgrd.50316.

Eyring, V., et al., 2007: Multimodel projections of stratospheric ozone in the 21st

century. J. Geophys. Res. Atmos., 112, D16303.

Faloona, I., 2009: Sulfur processing in the marine atmospheric boundary layer: A

review and critical assessment of modeling uncertainties. Atmos. Environ., 43,

2841–2854.

Fan, F. X., M. E. Mann, S. Lee, and J. L. Evans, 2010: Observed and modeled changes

in the South Asian summer monsoon over the Historical Period. J. Clim., 23,

5193–5205.

Fanning, A. F., and A. J. Weaver, 1996: An atmospheric energy-moisture balance

model: Climatology, interpentadal climate change, and coupling to an ocean

general circulation model. J. Geophys. Res. Atmos., 101, 15111–15128.

Farneti, R., and P. R. Gent, 2011: The effects of the eddy-inducedadvection coefﬁcient

in a coarse-resolution coupled climate model. Ocean Model., 39, 135–145.

Farneti, R., T. L. Delworth, A. J. Rosati, S. M. Grifﬁes, and F. R. Zeng, 2010: The role of

mesoscale eddies in the rectiﬁcation of the Southern Ocean response to climate

change. J. Phys. Oceanogr., 40, 1539–1557.

Fasullo, J., and K. E. Trenberth, 2012: A less cloudy future: The role of subtropical

subsidence in climate sensitivity. Science, 338, 792–794.

Fauchereau, N., S. Trzaska, Y. Richard, P. Roucou, and P. Camberlin, 2003: Sea-surface

temperature co-variability in the southern Atlantic and Indian Oceans and its

connections with the atmospheric circulation in the Southern Hemisphere. Int. J.

Climatol. , 23, 663–677.

Felzer, B., D. Kicklighter, J. Melillo, C. Wang, Q. Zhuang, and R. Prinn, 2004: Effects of

ozone on net primary production and carbon sequestration in the conterminous

United States using a biogeochemistry model. Tellus B, 56, 230–248.

Feng, J., and C. Fu, 2006: Inter-comparison of 10–year precipitation simulated by

several RCMs for Asia. Adv. Atmos. Sci., 23 531–542.

Feng, J., et al., 2011: Comparison of four ensemble methods combining regional

climate simulations over Asia. Meteorol. Atmos. Phys., 111, 41–53.

Fernandes, R., H. X. Zhao, X. J. Wang, J. Key, X. Qu, and A. Hall, 2009: Controls on

Northern Hemisphere snow albedo feedback quantiﬁed using satellite Earth

observations. Geophys. Res. Lett., 36, L21702.

834

Chapter 9 Evaluation of Climate Models

Fernandez-Donado, L., et al., 2013: Large-scale temperature response to external

forcing in simulations and reconstructions of the last millennium. Clim. Past, 9,

393–421.

Ferrari, R., J. C. McWilliams, V. M. Canuto, and M. Dubovikov, 2008: Parameterization

of eddy ﬂuxes near oceanic boundaries. J. Clim., 21, 2770–2789.

Ferrari, R., S. M. Grifﬁes, A. J. G. Nurser, and G. K. Vallis, 2010: A boundary-value

problem for the parameterized mesoscale eddy transport. Ocean Model., 32,

143–156.

Feser, F., 2006: Enhanced detectability of added value in limited-area model results

separated into different spatial scales. Mon. Weather Rev., 134, 2180–2190.

Feser, F., and M. Barcikowska, 2012: The inﬂuence of spectral nudging on typhoon

formation in regional climate models. Environ. Res. Lett., 7, 014024.

Feser, F., B. Rockel, H. von Storch, J. Winterfeldt, and M. Zahn, 2011: Regional climate

models add value to global model data: A review and selected examples. Bull.

Am. Meteorol. Soc., 92, 1181–1192.

Fetterer, F., K. Knowles, W. Meier, and M. Savoie, 2002: Sea Ice Index. National Snow

and Ice Data Center. Boulder, CO, USA.

Fichefet, T., and M. A. M. Maqueda, 1997: Sensitivity of a global sea ice model to

the treatment of ice thermodynamics and dynamics. J. Geophys. Res., 102,

12609–12646.

Fichefet, T., and M. A. M. Maqueda, 1999: Modelling the inﬂuence of snow

accumulation and snow-ice formation on the seasonal cycle of the Antarctic

sea-ice cover. Clim. Dyn., 15, 251–268.

Field, P. R., A. Gettelman, R. B. Neale, R. Wood, P. J. Rasch, and H. Morrison, 2008:

Midlatitude cyclone compositing to constrain climate model behavior using

satellite observations. J. Clim., 21, 5887–5903.

Fioletov, V., G. Bodeker, A. Miller, R. McPeters, and R. Stolarski, 2002: Global and

zonal total ozone variations estimated from ground-based and satellite

measurements: 1964–2000. J. Geophys. Res. Atmos., 107, 4647.

Fischer, E. M., S. I. Seneviratne, D. Lüthi, and C. Schär, 2007: Contribution of land-

atmosphere coupling to recent European summer heat waves. Geophys. Res.

Lett., 34, L06707.

Flanner, M. G., K. M. Shell, M. Barlage, D. K. Perovich, and M. A. Tschudi, 2011:

Radiative forcing and albedo feedback from the Northern Hemisphere

cryosphere between 1979 and 2008. Nature Geosci., 4, 151–155.

Flato, G., 2011: Earth system models: an overview. Wiley Interdisciplinary Reviews,

Climate Change, 2, 783–800.

Flocco, D., D. Schroeder, D. L. Feltham, and E. C. Hunke, 2012: Impact of melt ponds

on Arctic sea ice simulations from 1990 to 2007. J. Geophys. Res.Oceans, 117,

C09032.

Fogli, P. G., et al., 2009: INGV-CMCC Carbon (ICC): A Carbon Cycle Earth System

Model. CMCC Res. Papers. Euro-Mediterranean Center on Climate Change,

Bologna, Italy, 31 pp.

Fogt, R. L., J. Perlwitz, A. J. Monaghan, D. H. Bromwich, J. M. Jones, and G. J. Marshall,

2009: Historical SAM variability. Part II: Twentieth-century variability and trends

from reconstructions, observations, and the IPCC AR4 models. J. Clim., 22,

5346–5365.

Fontaine, B., and S. Janicot, 1996: Sea surface temperature ﬁelds associated with

West African rainfall anomaly types. J. Clim., 9, 2935–2940.

Forest, C. E., P. H. Stone, and A. P. Sokolov, 2006: Estimated PDFs of climate system

properties including natural and anthropogenic forcings. Geophys. Res. Lett., 33,

L01705.

Forest, C. E., P. H. Stone, and A. P. Sokolov, 2008: Constraining climate model

parameters from observed 20th century changes. Tellus A, 60, 911–920.

Forest, C. E., P. H. Stone, A. P. Sokolov, M. R. Allen, and M. D. Webster, 2002:

Quantifying uncertainties in climate system properties with the use of recent

climate observations. Science, 295, 113–117.

Forster, P. M., T. Andrews, P. Good, J. M. Gregory, L. S. Jackson, and M. Zelinka, 2013:

Evaluating adjusted forcing and model spread for historical and future scenarios

in the CMIP5 generation of climate models. J. Geophys. Res. Atmos., 118, 1139–

1150.

Fowler, H., S. Blenkinsop, and C. Tebaldi, 2007: Linking climate change modelling to

impacts studies: Recent advances in downscaling techniques for hydrological

modelling. Int. J. Climatol., 27, 1547–1578.

Fox-Kemper, B., R. Ferrari, and R. Hallberg, 2008: Parameterization of mixed layer

eddies. Part I: Theory and diagnosis. J. Phys. Oceanogr., 38, 1145–1165.

Fox-Kemper, B., et al., 2011: Parameterization of mixed layer eddies. III:

Implementation and impact in global ocean climate simulations. Ocean Model.,

39, 61–78.

Fox-Rabinovitz, M., J. Cote, B. Dugas, M. Deque, J. McGregor, and A. Belochitski,

2008: Stretched-grid Model Intercomparison Project: Decadal regional climate

simulations with enhanced variable and uniform-resolution GCMs. Meteorol.

Atmos. Phys., 100, 159–177.

Frame, D., B. Booth, J. Kettleborough, D. Stainforth, J. Gregory, M. Collins, and M.

Allen, 2005: Constraining climate forecasts: The role of prior assumptions.

Geophys. Res. Lett., 32, L09702.

Frankcombe, L. M., A. von der Heydt, and H. A. Dijkstra, 2010: North Atlantic

multidecadal climate variability: An investigation of dominant time scales and

processes. J. Clim., 23, 3626–3638.

Frederiksen, C. S., J. S. Frederiksen, J. M. Sisson, and S. L. Osbrough, 2011: Australian

winter circulation and rainfall changes and projections. Int. J. Clim. Change Strat.

Manage., 3, 170–188.

Friedlingstein, P., et al., 2001: Positive feedback between future climate change and

the carbon cycle. Geophys. Res. Lett., 28, 1543–1546.

Friedlingstein, P., et al., 2006: Climate-carbon cycle feedback analysis: Results from

the (CMIP)-M-4 model intercomparison. J. Clim., 19, 3337–3353.

Friend, A. D., et al., 2007: FLUXNET and modelling the global carbon cycle. Global

Change Biol., 13, 610–633.

Frierson, D. M. W., J. Lu, and G. Chen, 2007: Width of the Hadley cell in simple and

comprehensive general circulation models. Geophys. Res. Lett., 34, L18804.

Frohlich, C., and J. Lean, 2004: Solar radiative output and its variability: Evidence and

mechanisms. Astron. Astrophys. Rev., 12, 273–320.

Frost, A. J., et al., 2011: A comparison of multi-site daily rainfall downscaling

techniques under Australian conditions. J. Hydrol., 408, 1–18.

Fu, Q., S. Manabe, and C. M. Johanson, 2011: On the warming in the tropical upper

troposphere: Models versus observations. Geophys. Res. Lett., 38, L15704.

Furrer, R., R. Knutti, S. Sain, D. Nychka, and G. Meehl, 2007: Spatial patterns of

probabilistic temperature change projections from a multivariate Bayesian

analysis. Geophys. Res. Lett., 34, L06711.

Furtado, J., E. Di Lorenzo, N. Schneider, and N. A. Bond, 2011: North Paciﬁc decadal

variability and climate change in the IPCC AR4 models. J. Clim., 24, 3049–3067

Fyfe, J. C., N. P. Gillett, and D. W. J. Thompson, 2010: Comparing variability and trends

in observed and modelled global-mean surface temperature. Geophys. Res. Lett.,

37, L16802.

Fyke, J. G., A. J. Weaver, D. Pollard, M. Eby, L. Carter, and A. Mackintosh, 2011: A new

coupled ice sheet/climate model: Description and sensitivity to model physics

under Eemian, Last Glacial Maximum, late Holocene and modern climate

conditions. Geosci. Model Dev., 4, 117–136.

Galbraith, D., P. E. Levy, S. Sitch, C. Huntingford, P. Cox, M. Williams, and P. Meir,

2010: Multiple mechanisms of Amazonian forest biomass losses in three

dynamic global vegetation models under climate change. New Phytologist, 187,

647–665.

Ganachaud, A., and C. Wunsch, 2003: Large-scale ocean heat and freshwater

transports during the World Ocean Circulation Experiment. J. Clim., 16, 696–705.

Gangsto, R., F. Joos, and M. Gehlen, 2011: Sensitivity of pelagic calciﬁcation to ocean

acidiﬁcation. Biogeosciences, 8, 433–458.

Gao, X., Y. Shi, D. Zhang, J. Wu, F. Giorgi, Z. Ji, and Y. Wang, 2012: Uncertainties in

monsoon precipitation projections over China: Results from two high-resolution

RCM simulations. Clim. Res., 52, 213–226.

Gates, W. L., et al., 1999: An overview of the results of the Atmospheric Model

Intercomparison Project (AMIP I). Bull. Am. Meteorol. Soc., 80, 29–55.

Gbobaniyi, E. O., B. J. Abiodun, M. A. Tadross, B. C. Hewitson, and W. J. Gutowski,

2011: The coupling of cloud base height and surface ﬂuxes: A transferability

intercomparison. Theor. Appl. Climatol., 106, 189–210.

Gehlen, M., R. Gangsto, B. Schneider, L. Bopp, O. Aumont, and C. Ethe, 2007: The fate

of pelagic CaCO

production in a high CO

ocean: A model study. Biogeosciences,

4, 505–519.

Geller, M. A., et al., 2011: New gravity wave treatments for GISS climate models. J.

Clim., 24, 3989–4002.

Gent, P. R., and J. C. McWilliams, 1990: Isopycnal mixing in ocean circulation models.

J. Phys. Oceanogr., 20, 150–155.

Gent, P. R., and G. Danabasoglu, 2011: Response toincreasing Southern Hemisphere

winds in CCSM4. J. Clim., 24, 4992–4998.

Gent, P. R., J. Willebrand, T. J. McDougall, and J. C. McWilliams, 1995: Parameterizing

eddy-induced tracer transports in ocean circulation models. J. Phys. Oceanogr.,

25, 463–474.

Gent, P. R., et al., 2011: The Community Climate System Model Version 4. J. Clim.,

24, 4973–4991.

835

Evaluation of Climate Models Chapter 9

Gerber, E. P., L. M. Polvani, and D. Ancukiewicz, 2008: Annular mode time scales

in the Intergovernmental Panel on Climate Change Fourth Assessment Report

models. Geophys. Res. Lett., 35, L22707.

Gerber, S., L. O. Hedin, M. Oppenheimer, S. W. Pacala, and E. Shevliakova, 2010:

Nitrogen cycling and feedbacks in a global dynamic land model. Global

Biogeochem. Cycles, 24, Gb1001.

Gettelman, A., et al., 2010: Multimodel assessment of the upper troposphere and

lower stratosphere: Tropics and global trends. J. Geophys. Res. Atmos., 115,

D00m08.

Ghan, S., X. Liu, R. Easter, P. Rasch, J. Yoon, and B. Eaton, 2012: Toward a minimal

representation of aerosols in climate models: Comparative decomposition of

aerosol direct, semi-direct and indirect radiative forcing. J. Clim., doi:10.1175/

JCLI-D-11-00650.1.

Gillett, N. P., 2005: Climate modelling —Northern Hemisphere circulation. Nature,

437, 496–496.

Giorgi, F., and E. Coppola, 2010: Does the model regional bias affect the projected

regional climate change? An analysis of global model projections. Clim. Change,

100, 787–795.

Girard, L., J. Weiss, J. M. Molines, B. Barnier, and S. Bouillon, 2009: Evaluation of

high-resolution sea ice models on the basis of statistical and scaling properties

of Arctic sea ice drift and deformation. J. Geophys. Res., 114, C08015.

Gleckler, P., K. Taylor, and C. Doutriaux, 2008: Performance metrics for climate

models. J. Geophys. Res. Atmos., 113, D06104.

Gleckler, P., K. AchutaRao, J. Gregory, B. Santer, K. Taylor, and T. Wigley, 2006:

Krakatoa lives: The effect of volcanic eruptions on ocean heat content and

thermal expansion. Geophys. Res. Lett., 33, L17702.

Gleckler, P. J., et al., 2012: Human-induced global ocean warming on multidecadal

timescales, Nature Climate Change, 2, 524–529.

Gnanadesikan, A., S. M. Grifﬁes, and B. L. Samuels, 2007: Effects in a climate model

of slope tapering in neutral physics schemes. Ocean Model., 16, 1–16.

Goddard, L., and S. J. Mason, 2002: Sensitivity of seasonal climate forecasts to

persisted SST anomalies. Clim. Dyn., 19, 619–631.

Golaz, J.-C., M. Salzmann, L. J. Donner, L. W. Horowitz, Y. Ming, and M. Zhao, 2011:

Sensitivity of the aerosol indirect effect to subgrid variability in the cloud

parameterization of the GFDL Atmosphere General Circulation Model AM3. J.

Clim., 24, 3145–3160.

Goosse, H., and T. Fichefet, 1999: Importance of ice-ocean interactions for the global

ocean circulation: A model study. J. Geophys. Res. Oceans, 104, 23337–23355.

Goosse, H., et al., 2010: Description of the Earth system model of intermediate

complexity LOVECLIM version 1.2. Geosci. Model Dev., 3, 603–633.

Gordon, C., et al., 2000: The simulation of SST, sea ice extents and ocean heat

transports in a version of the Hadley Centre coupled model without ﬂux

adjustments. Clim. Dyn., 16, 147–168.

Gordon, H., et al., 2010: The CSIRO Mk3.5 Climate Model. CAWCR Technical

Report,21, 1–74.

Gordon, H. B., et al., 2002: The CSIRO Mk3 Climate System Model. Technical Paper

No. 60. CSIRO Atmospheric Research, Aspendale, Vic., Australia.

Greeves, C. Z., V. D. Pope, R. A. Stratton, and G. M. Martin, 2007: Representation

of Northern Hemisphere winter storm tracks in climate models. Clim. Dyn., 28,

683–702.

Gregory, J. M., and P. M. Forster, 2008: Transient climate response estimated from

radiative forcing and observed temperature change. J. Geophys. Res. Atmos.,

113, D23105.

Gregory, J. M., et al., 2004: A new method for diagnosing radiative forcing and

climate sensitivity. Geophys. Res. Lett., 31, L03205.

Gregory, J. M., et al., 2005: A model intercomparison of changes in the Atlantic

thermohaline circulation in response to increasing atmospheric CO

concentration. Geophys. Res. Lett., 32, L12703.

Grifﬁes, S. M., 2009: Elements of MOM4p1. GFDL Ocean Group Technical Report No.

6. NOAA/GFDL. Princeton, USA, 371 pp.

Grifﬁes, S. M., and R. J. Greatbatch, 2012: Physical processes that impact the

evolution of global mean sea level in ocean climate models. Ocean Model., 51,

37–72.

Grifﬁes, S. M., M. J. Harrison, R. C. Pacanowski, and A. Rosati, 2004: A Technical Guide

to MOM4. GFDL Ocean Group Technical Report No. 5, Princeton, USA,337 pp.

Grifﬁes, S. M., et al., 2005: Formulation of an ocean model for global climate

simulations. Ocean Sci., 1, 45–79.

Grifﬁes, S. M., et al., 2009: Coordinated Ocean-ice Reference Experiments (COREs).

Ocean Model., 26, 1–46.

Grose, M., M. Pook, P. McIntosh, J. Risbey, and N. Bindoff, 2012: The simulation of

cutoff lows in a regional climate model: Reliability and future trends. Clim. Dyn.,

39, 445–459.

Guemas, V., F. J. Doblas-Reyes, I. Andreu-Burillo, and M. Asif, 2013: Retrospective

prediction of the global warming slowdown in the last decade. Nature Clim.

Change, doi:10.1038/nclimate1863.

Guilyardi, E., 2006: El Niño - mean state - seasonal cycle interactions in a multi-

model ensemble. Clim. Dyn., 26, 229–348.

Guilyardi, E., P. Braconnot, F. F. Jin, S. T. Kim, M. Kolasinski, T. Li, and I. Musat,

2009a: Atmosphere feedbacks during ENSO in a coupled GCM with a modiﬁed

atmospheric convection scheme. J. Clim., 22, 5698–5718.

Guilyardi, E., et al., 2009b: Understanding El Niño in ocean–atmosphere general

circulation models: Progress and challenges. Bull. Am. Meteorol. Soc., 90, 325–

340.

Guiot, J., J. J. Boreux, P. Braconnot, F. Torre, and P. Participants, 1999: Data-model

comparison using fuzzy logic in paleoclimatology. Clim. Dyn., 15, 569–581.

Gupta, A. S., A. Santoso, A. S. Taschetto, C. C. Ummenhofer, J. Trevena, and M. H.

England, 2009: Projected changes to the Southern Hemisphere ocean and sea

ice in the IPCC AR4 climate models. J. Clim., 22, 3047–3078.

Gurney, K. R., et al., 2003: TransCom 3 CO

inversion intercomparison: 1. Annual

mean control results and sensitivity to transport and prior ﬂux information.

Tellus B, 55, 555–579.

Gutowski, W., et al., 2010: Regional extreme monthly precipitation simulated by

NARCCAP RCMs. J. Hydrometeorol., 11, 1373–1379.

Hall, A., and X. Qu, 2006: Using the current seasonal cycle to constrain snow albedo

feedback in future climate change. Geophys. Res. Lett., 33, L03502.

Hallberg, R., and A. Gnanadesikan, 2006: The role of eddies in determining the

structure and response of the wind-driven southern hemisphere overturning:

Results from the Modeling Eddies in the Southern Ocean (MESO) project. J. Phys.

Oceanogr., 36, 2232–2252.

Hallberg, R., and A. Adcroft, 2009: Reconciling estimates of the free surface height

in Lagrangian vertical coordinate ocean models with mode-split time stepping.

Ocean Model., 29, 15–26.

Halloran, P. R., 2012: Does atmospheric CO

seasonality play an important role in

governing the air-sea ﬂux of CO

? Biogeosciences, 9, 2311–2323.

Ham, Y.-G., J. S. Kug, I. S. Kang, F. F. Jin, and A. Timmermann, 2010: Impact of diurnal

atmospher-ocean coupling on tropical climate simulations using a coupled

GCM. Clim. Dyn., 34, 905–917.

Hamilton, K., 1998: Effects of an imposed Quasi-Biennial Oscillation in

a comprehensive troposphere-stratosphere-mesosphere general

circulationmodel. J. Atmos. Sci., 55, 2393– 2418.

Handorf, D., and K. Dethloff, 2012: How well do state-of-the-art atmosphere-ocean

general circulation models reproduce atmospheric teleconnection patterns?

Tellus A, 64, 19777.

Hannart, A., J. L. Dufresne, and P. Naveau, 2009: Why climate sensitivity may not be

so unpredictable. Geophys. Res. Lett., 36, L16707.

Hannay, C., et al., 2009: Evaluation of Forecasted Southeast Paciﬁc Stratocumulus in

the NCAR, GFDL, and ECMWF Models. J. Clim., 22, 2871–2889.

Hansen, J., R. Ruedy, M. Sato, and K. Lo, 2010: Global surface temperature change.

Rev. Geophys., 48, Rg4004.

Hansen, J., M. Sato, P. Kharecha, and K. von Schuckmann, 2011: Earth’s energy

imbalance and implications. Atmos. Chem. Phys., 11, 13421–13449.

Hansen, J., et al., 1983: Efﬁcient Three-Dimensional Global Models for Climate

Studies: Models I and II. Mon. Weath. Rev., 111, 609–662.

Hansen, J., et al., 1984: Climate Sensitivity: Analysis of Feedback Mechanisms. Clim.

Proc. Clim. Sens. Geophys. Monogr., 29, 130–163.

Hansen, J., et al., 2005: Efﬁcacy of climate forcings. J. Geophys. Res. Atmos., 110,

D18104.

Hardiman, S. C., N. Butchart, T. J. Hinton, S. M. Osprey, and L. J. Gray, 2012: The effect

of a well-resolved stratosphere on surface climate: Differences between CMIP5

simulations with high and low top versions of the Met Ofﬁce Climate Model. J.

Clim., 25, 7083–7099.

Hargreaves, J. C., A. Abe-Ouchi, and J. D. Annan, 2007: Linking glacial and future

climates through an ensemble of GCM simulations. Clim. Past, 3, 77–87.

Hargreaves, J. C., J. D. Annan, M. Yoshimori, and A. Abe-Ouchi, 2012: Can the Last

Glacial Maximum constrain climate sensitivity? Geophys. Res. Lett., 39, L24702.

Hargreaves, J. C., A. Paul, R. Ohgaito, A. Abe-Ouchi, and J. D. Annan, 2011: Are

paleoclimate model ensembles consistent with the MARGO data synthesis?

Clim. Past, 7, 917–933.

836

Chapter 9 Evaluation of Climate Models

Hargreaves, J. C., J. D. Annan, R. Ohgaito, A. Paul, and A. Abe-Ouchi, 2013: Skill and

reliability of climate model ensembles at the Last Glacial Maximum and mid

Holocene. Clim. Past, 9, 811–823.

Hasegawa, A., and S. Emori, 2007: Effect of air-sea coupling in the assessment of

-induced intensiﬁcation of tropical cyclone activity. Geophys. Res. Lett., 34,

L05701.

Hasumi, H., 2006: CCSR Ocean Component Model (COCO) Version 4.0. CCSR Report.

Centre for Climate System Research, University of Tokyo, Tokyo, Japan, 68 pp.

Hasumi, H., and S. Emori, 2004: K-1 Coupled GCM (MIROC) Description. Center for

Climate System Research, University of Tokyo, Tokyo, Japan, 34 pp.

Hawkins, E., and R. Sutton, 2009: The potential to narrow uncertainty in regional

climate predictions. Bull. Am. Meteorol. Soc., 90, 1095–1107.

Haynes, J. M., C. Jakob, W. B. Rossow, G. Tselioudis, and J. Brown, 2011: Major

characteristics of Southern Ocean cloud regimes and their effects on the energy

budget. J. Clim., 24, 5061–5080.

Haynes, P. H., 2006: The latitudinal structure of the QBO. Q. J. R. Meteorol. Soc., 124,

2645–2670.

Haywood, J. M., N. Bellouin, A. Jones, O. Boucher, M. Wild, and K. P. Shine, 2011: The

roles of aerosol, water vapor and cloud in future global dimming/brightening. J.

Geophys. Res. Atmos., 116, D20203.

Hazeleger, W., and R. J. Haarsma, 2005: Sensitivity of tropical Atlantic climate to

mixing in a coupled ocean-atmosphere model. Clim. Dyn., 25, 387–399.

Hazeleger, W., et al., 2012: EC-Earth V2.2: Description and validation of a new

seamless earth system prediction model. Clim. Dyn., 39, 2611–2629.

Hegerl, G., and F. Zwiers, 2011: Use of models in detection and attribution of climate

change. Clim. Change, 2, 570–591.

Hegerl, G. C., et al., 2007: Understanding and attributing climate change. In: Climate

Change 2007: The Physical Science Basis. Contribution of Working Group I to the

Fourth Assessment Report of the Intergovernmental Panel on Climate Change

[Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K. B. Averyt, M. Tignor

and H. L. Miller (eds.)] Cambridge University Press, Cambridge, United Kingdom

and New York, NY, USA, pp. 665–775.

Heinze, C., 2004: Simulating oceanic CaCO

export production in the greenhouse.

Geophys. Res. Lett., 31, L16308.

Heinze, C., I. Kriest, and E. Maier-Reimer, 2009: Age offsets among different biogenic

and lithogenic components of sediment cores revealed by numerical modeling.

Paleoceanography, 24, PA4214.

Held, I. M., 2005: The gap between simulation and understanding in climate

modeling. Bull. Am. Meteorol. Soc., 86, 1609–1614.

Held, I. M., and B. J. Soden, 2006: Robust responses of the hydrological cycle to

global warming. J. Clim., 19, 5686–5699.

Held, I. M., and K. M. Shell, 2012: Using Relative Humidity as a State Variable in

Climate Feedback Analysis. J. Clim., 25, 2578–2582.

Held, I. M., M. Winton, K. Takahashi, T. Delworth, F. R. Zeng, and G. K. Vallis, 2010:

Probing the fast and slow components of global warming by returning abruptly

to preindustrial forcing. J. Clim., 23, 2418–2427.

Henson, S. A., D. Raitsos, J. P. Dunne, and A. McQuatters-Gollop, 2009: Decadal

variability in biogeochemical models: Comparison with a 50-year ocean colour

dataset. Geophys. Res. Lett., 36, L21601.

Hermes, J. C., and C. J. C. Reason, 2005: Ocean model diagnosis of interannual

coevolving SST variability in the South Indian and South Atlantic Oceans. J. Clim.,

18, 2864–2882.

Hernández-Díaz, L., R. Laprise, L. Sushama, A. Martynov, K. Winger, and B. Dugas,

2013: Climate simulation over CORDEX Africa domain using the ﬁfth-generation

Canadian Regional Climate Model (CRCM5). Clim. Dyn., 40, 1415–1433.

Heuzé, C., K. J. Heywood, D. P. Stevens, and J. K. Ridley, 2013: Southern Ocean

bottom water characteristics in CMIP5 models. Geophys. Res. Lett., doi:10.1002/

grl.50287.

Hewitt, H. T., et al., 2011: Design and implementation of the infrastructure of

HadGEM3: The next-generation Met Ofﬁce climate modelling system. Geosci.

Model Dev., 4, 223–253.

Hibbard, K. A., G. A. Meehl, P. M. Cox, and P. Friedlingstein, 2007: A strategy for

climate change stabilization experiments. Eos Trans. Am. Geophys. Union, 88,

217–221.

Hirai, M., T. Sakashita, H. Kitagawa, T. Tsuyuki, M. Hosaka, and M. Oh’Izumi, 2007:

Development and validation of a new land surface model for JMA’s operational

global model using the CEOP observation dataset. J. Meteorol. Soc. Jpn., 85A,

1–24.

Hirota, N., Y. N. Takayabu, M. Watanabe, and M. Kimoto, 2011: Precipitation

reproducibility over tropical oceans and its relationship to the double ITCZ

problem in CMIP3 and MIROC5 climate models. J. Clim., 24, 4859–4873.

Hirschi, M., et al., 2011: Observational evidence for soil-moisture impact on hot

extremes in southeastern Europe. Nature Geosci., 4, 17–21.

Hofmann, M., and M. A. Morales Maqueda, 2011: The response of Southern Ocean

eddies to increased midlatitude westerlies: A non-eddy resolving model study.

Geophys. Res. Lett., 38, L03605.

Holden, P. B., N. R. Edwards, D. Gerten, and S. Schaphoff, 2013: A model based

constraint of CO

fertilisation. Biogeosciences, 10, 339–355.

Holian, G. L., A. P. Sokolov, and R. G. Prinn, 2001: Uncertainty in atmospheric CO

predictions from a parametric uncertainty analysis of a Global Ocean Carbon

Cycle Model. Joint Program Report Series. MIT Joint Program on the Science and

Policy of Global Change, Cambridge, MA, USA, 25 pp.

Holland, M., D. Bailey, B. Briegleb, B. Light, and E. Hunke, 2012: Improved sea ice

shortwave radiation physics in CCSM4: The impact of melt ponds and aerosols

on arctic aea ice. J. Clim., 25, 1413–1430.

Holland, M. M., M. C. Serreze, and J. Stroeve, 2010: The sea ice mass budget of the

Arctic and its future change as simulated by coupled climate models. Clim. Dyn.,

doi:10.1007/s00382-008-0493-4.

Holton, J. R., and H. C. Tan, 1980: The inﬂuence of the equatorial Quasi-Biennial

Oscillation on the global circulation at 50 mb. J. Atmos. Sci., 37, 2200–2208.

Horowitz, L. W., et al., 2003: A global simulation of tropospheric ozone and related

tracers: Description and evaluation of MOZART, version 2. J. Geophys. Res.

Atmos., 108, 4784.

Hourdin, F., et al., 2012: Impact of the LMDZ atmospheric grid conﬁguration on the

climate and sensitivity of the IPSL-CM5A coupled model. Clim. Dyn., doi:10.1007/

s00382–012–1411–3.

Hourdin, F., et al., 2013: LMDZ5B: The atmospheric component of the IPSL climate

model with revisited parameterizations for clouds and convection. Clim. Dyn.,

40, 2193–2222.

Hourdin, F., et al., 2010: AMMA-Model Intercomparison Project. Bull. Am. Meteorol.

Soc., 91, 95–104.

Hu, Z.-Z., B. Huang, Y.-T. Hou, W. Wang, F. Yang, C. Stan, and E. Schneider, 2011:

Sensitivity of tropical climate to low-level clouds in the NCEP climate forecast

system. Clim. Dyn., 36, 1795–1811.

Huang, C. J., F. Qiao, Q. Shu, and Z. Song, 2012: Evaluating austral summer mixed-

layer response to surface wave-induced mixing in the Southern Ocean. J.

Geophys. Res.-Oceans, 117, C00j18.

Huber, M., I. Mahlstein, M. Wild, J. Fasullo, and R. Knutti, 2011: Constraints on climate

sensitivity from radiation patterns in climate models. J. Clim., 24, 1034–1052.

Hung, M., J. Lin, W. Wang, D. Kim, T. Shinoda, and S. Weaver, 2013: MJO and

convectively coupled equatorial waves simulated by CMIP5 climate models. J.

Clim., doi:10.1175/JCLI-D-12-00541.1.

Hunke, E. C., and J. K. Dukowicz, 1997: An elastic-viscous-plastic model for sea ice

dynamics. J. Phys. Oceanogr., 27, 1849–1867.

Hunke, E. C., and W. H. Lipscomb, 2008: CICE: The Los Alamos Sea Ice

ModelDocumentation and Software User’s ManualVersion 4.1. Los Alamos

National Laboratory, Los Alamos, NM, USA, 76 pp.

Hunke, E. C., W. H. Lipscomb, and A. K. Turner, 2010: Sea ice models for climate study:

Retorspective and new directions. J. Glaciol., 56, 1162–1172.

Hunke, E. C., D. A. Hebert, and O. Lecomte, 2013: Level-ice melt ponds in the Los

Alamos sea ice model, CICE. Ocean Model., doi:10.1016/j.ocemod.2012.11.008.

Hunke, E. C., D. Notz, A. K. Turner, and M. Vancoppenolle, 2011: The multiphase

physics of sea ice : A review for model developers. Cryosphere, 5, 989–1009.

Hurrell, J., G. A. Meehl, D. Bader, T. L. Delworth, B. Kirtman, and B. Wielicki, 2009:

A uniﬁed modeling approach to climate system prediction. Bull. Am. Meteorol.

Soc., 90, 1819–1832.

Hurrell, J., et al., 2013: The Community Earth System Model: A framework for

collaborative research. Bull. Am. Meteorol. Soc., doi:10.1175/BAMS-D-12–

00121.

Hurtt, G. C., et al., 2009: Harmonization of global land-use scenarios for the period

1500–2100 for IPCC-AR5. iLEAPS Newsl., 7, 6–8.

Hutchings, J. K., A. Roberts, C. A. Geiger, and J. Richter-Menge, 2011: Spatial and

temporal characterization of sea-ice deformation. Ann. Glaciol., 52, 360–368.

Huybrechts, P., 2002: Sea-level changes at the LGM from ice-dynamic reconstructions

of the Greenland and Antarctic ice sheets during the glacial cycles. Quat. Sci.

Rev., 21, 203–231.

837

Evaluation of Climate Models Chapter 9

Huybrechts, P., H. Goelzer, I. Janssens, E. Driesschaert, T. Fichefet, H. Goosse, and M.

F. Loutre, 2011: Response of the Greenland and Antarctic ice sheets to multi-

millennial greenhouse warming in the Earth System Model of Intermediate

Complexity LOVECLIM. Surv. Geophys., 32, 397–416.

Iacono, M. J., J. S. Delamere, E. J. Mlawer, and S. A. Clough, 2003: Evaluation of upper

tropospheric water vapor in the NCAR Community Climate Model (CCM3) using

modeled and observed HIRS radiances. J. Geophys. Res., 108, 4037.

Ichikawa, H., H. Masunaga, Y. Tsushima, and H. Kanzawa, 2012: Reproducibility by

climate models of cloud radiative forcing associated with tropical convection. J.

Clim., 25, 1247–1262.

Ilicak, M., A. J. Adcroft, S. M. Grifﬁes, and R. W. Hallberg, 2012: Spurious dianeutral

mixing and the role of momentum closure. Ocean Model., 45–46, 37–58.

Illingworth, A. J., et al., 2007: Cloudnet. Bull. Am. Meteor. Soc., 88, 883–898.

Ilyina, T., R. E. Zeebe, E. Maier-Reimer, and C. Heinze, 2009: Early detection of ocean

acidiﬁcation effects on marine calciﬁcation. Global Biogeochem. Cycles, 23,

Gb1008.

Ilyina, T., K. Six, J. Segschneider, J. Maier-Reimer, H. Li, and I. Nunez-Riboni, 2013:

The global ocean biogeochemistry model HAMOCC: Model architecture

and performance as component of the MPI-Earth System Model in different

CMIP5experimental realizations. J. Adv. Model. Earth Syst., 5, 287–315.

Inatsu, M., and M. Kimoto, 2009: A scale interaction study on East Asian cyclogenesis

using a General Circulation Model coupled with an Interactively Nested Regional

Model. Mon. Weather Rev., 137, 2851–2868.

Inatsu, M., Y. Satake, M. Kimoto, and N. Yasutomi, 2012: GCM bias of the western

Paciﬁc summer monsoon and its correction by two-way nesting system. J.

Meteorol. Soc. Jpn., 90B, 1–10.

Ingram, W., 2010: A very simple model for the water vapour feedback on climate

change. Q. J. R. Meteorol. Soc., 136, 30–40.

Ingram, W., 2013: Some implications of a new approach to the water vapour

feedback. Clim. Dyn., 40, 925–933.

Inness, P. M., J. M. Slingo, E. Guilyardi, and J. Cole, 2003: Simulation of the Madden-

Julian oscillation in a coupled general circulation model. Part II: The role of the

basic state. J. Clim., 16, 365–382.

Inoue, J., J. P. Liu, J. O. Pinto, and J. A. Curry, 2006: Intercomparison of Arctic Regional

Climate Models: Modeling clouds and radiation for SHEBA in May 1998. J. Clim.,

19, 4167–4178.

IPCC, 2007: Climate Change 2007: The Physical Science Basis. Contribution of

Working Group I to the Fourth Assessment Report of the Intergovernmental

Panel on Climate Change [Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis,

K. B. Averyt, M. Tignor and H. L. Miller (eds.)] Cambridge University Press,

Cambridge, United Kingdom and New York, NY, USA, 996 pp.

IPCC, 2012: IPCC WGI/WGII Special Report on Managing the Risks of Extreme

Events and Disasters to Advance Climate Change Adaptation (SREX). [Field,

C.B., V. Barros, T.F. Stocker, D. Qin, D.J. Dokken, K.L. Ebi, M.D. Mastrandrea, K.J.

Mach, G.-K. Plattner, S.K. Allen, M. Tignor, and P.M. Midgley (Eds.)]. Cambridge

University Press, The Edinburgh Building, Shaftesbury Road, Cambridge CB2 8RU

ENGLAND, 582 pp.

Ishii, M., and M. Kimoto, 2009: Reevaluation of historical ocean heat content

variations with time-varying XBT and MBT depth bias corrections. J. Oceanogr.,

65, 287–299.

Ito, A., and T. Oikawa, 2002: A simulation model of the carbon cycle in land

ecosystems (Sim-CYCLE): A description based on dry-matter production theory

and plot-scale validation. Ecol. Model., 151, 143–176.

Iversen, T., et al., 2013: The Norwegian Earth System Model, NorESM1–M. Part

2:Climate response and scenario projections.Geosci. Model Dev., 6, 1–27.

Izumi, K., P. J. Bartlein, and S. P. Harrison, 2013: Consistent large-scale temperature

responses in warm and cold climates. Geophys. Res. Lett., doi:2013GL055097.

Jackson, C. S., M. K. Sen, G. Huerta, Y. Deng, and K. P. Bowman, 2008a: Error reduction

and convergence in climate prediction. J. Clim., 21, 6698–6709.

Jackson, L., R. Hallberg, and S. Legg, 2008b: A parameterization of shear-driven

turbulence for ocean climate models. J. Phys. Oceanogr., 38, 1033–1053.

Jacob, D., et al., 2012: Assessing the transferability of the Regional Climate Model

REMO to Different COordinated Regional Climate Downscaling EXperiment

(CORDEX) regions. Atmosphere, 3, 181–199.

Jakob, C., 2010: Accelerating progress in Global Atmospheric Model development

through improved parameterizations: Challenges, opportunities, and strategies.

Bull. Am. Meteorol. Soc., 91, 869–875.

Jang, C. J., J. Park, T. Park, and S. Yoo, 2011: Response of the ocean mixed layer depth

to global warming and its impact on primary production: A case for the North

Paciﬁc Ocean. Ices J. Mar. Sci., 68, 996–1007.

Jansen, E., et al., 2007: Paleoclimate. In: Climate Change 2007: The Physical Science

Basis. Contribution of Working Group I to the Fourth Assessment Report of the

Intergovernmental Panel on Climate Change [Solomon, S., D. Qin, M. Manning,

Z. Chen, M. Marquis, K. B. Averyt, M. Tignor and H. L. Miller (eds.)] Cambridge

University Press, Cambridge, United Kingdom and New York, NY, USA, pp. 433–

498.

Jayne, S. R., 2009: The impact of abyssal mixing parameterizations in an ocean

General Circulation Model. J. Phys. Oceanogr., 39, 1756–1775.

Ji, J., M. Huang, and K. Li, 2008: Prediction of carbon exchanges between China

terrestrial ecosystem and atmosphere in 21st century. Sci. China D, 51, 885–898.

Ji, J. J., 1995: A climate-vegetation interaction model: Simulating physical and

biological processes at the surface. J. Biogeogr., 22, 445–451.

Jiang, J. H., et al., 2012a: Evaluation of cloud and water vapor simulations in CMIP5

climate models using NASA “A-Train” satellite observations. J. Geophys. Res.,

117, D14105.

Jiang, X., et al., 2012b: Simulation of the intraseasonal variability over the Eastern

Paciﬁc ITCZ in climate models. Clim. Dyn., 39, 617–636.

Jin, F. F., S. T. Kim, and L. Bejarano, 2006: A coupled-stability index for ENSO. Geophys.

Res. Let., 33, L23708.

Joetzjer, E., H. Douville, C. Delire, and P. Ciais, 2013: Present-day and future

Amazonian precipitation in global climate models: CMIP5 versus CMIP3. Clim.

Dyn., doi: 10.1007/s00382–012-1644-1.

Johanson, C. M., and Q. Fu, 2009: Hadley Cell Widening: Model Simulations versus

Observations. J. Clim., 22, 2713–2725.

John, V., and B. Soden, 2007: Temperature and humidity biases in global climate

models and their impact on climate feedbacks. Geophys. Res. Lett., 34, L18704.

Johns, T. C., et al., 2003: Anthropogenic climate change for 1860 to 2100 simulated

with the HadCM3 model under updated emissions scenarios. Clim. Dyn., 20,

583–612.

Johns, T. C., et al., 2006: The new Hadley Centre Climate Model (HadGEM1):

Evaluation of coupled simulations. J. Clim., 19, 1327–1353

Johnson, N. C., and S. B. Feldstein, 2010: The continuum of north Paciﬁc sea level

pressure patterns: Intraseasonal, interannual, and interdecadal variability. J.

Clim., 23, 851–867.

Jolliffe, I. T., and D. B. Stephenson, 2011: Forecast Veriﬁcation: A Practitioner’s Guide

in Atmospheric Science. 2nd ed. John Wiley & Sons, Hoboken, NJ, 292 pp.

Joly, M., A. Voldoire, H. Douville, P. Terray, and J. F. Royer, 2007: African monsoon

teleconnections with tropical SSTs: Validation and evolution in a set of IPCC4

simulations. Clim. Dyn., 29, 1–20.

Jones, A., D. L. Roberts, M. J. Woodage, and C. E. Johnson, 2001: Indirect sulphate

aerosol forcing in a climate model with an interactive sulphur cycle. J. Geophys.

Res. Atmos., 106, 20293–20310.

Jones, G. S., P. A. Stott, and N. Christidis, 2012: Attribution of observed historical

near surface temperature variations to anthropogenic and natural causes using

CMIP5 simulations. J. Geophys. Res., doi:10.1002/jgrd.50239.

Jones, P. D., M. New, D. E. Parker, S. Martin, and I. G. Rigor, 1999: Surface air

temperature and its changes over the past 150 years. Rev. Geophys., 37, 173–

199.

Joseph, S., A. K. Sahai, B. N. Goswami, P. Terray, S. Masson, and J. J. Luo, 2012: Possible

role of warm SST bias in the simulation of boreal summer monsoon in SINTEX-F2

coupled model. Clim. Dyn., 38, 1561–1576.

Joussaume, S., and K. E. Taylor, 1995: Status of the Paleoclimate Modeling

Intercomparison Project. In: Proceedings of the ﬁrst international AMIP scientiﬁc

conference, WCRP-92, Monterey, USA, 425–430.

Jun, M., R. Knutti, and D. Nychka, 2008: Spatial analysis to quantify numerical model

bias and dependence: How many climate models are there? J. Am. Stat. Assoc.,

103, 934–947.

Jung, T., et al., 2010: The ECMWF model climate: Recent progress through improved

physical parametrizations. Q. J. R. Meteorol. Soc., 136, 1145–1160.

Jung, T., et al., 2012: High-resolution global climate simulations with the ECMWF

Model in Project Athena: Experimental design, model climate, and seasonal

forecast skill. J. Clim., 25, 3155–3172.

Jungclaus, J. H., et al., 2006: Ocean circulation and tropical variability in the coupled

model ECHAM5/MPI-OM. J. Clim., 19, 3952–3972.

838

Chapter 9 Evaluation of Climate Models

Jungclaus, J. H., et al., 2013: Characteristics of the ocean simulations in MPIOM,

the ocean componentof the MPI-Earth System Model. J. Adv. Model. Earth Syst.,

doi:10.1002/jame.20023.

Jungclaus, J. H., et al., 2010: Climate and carbon-cycle variability over the last

millennium. Clim. Past, 6, 723–737.

Kageyama, M., et al., 2006: Last Glacial Maximum temperatures over the North

Atlantic, Europe and western Siberia: A comparison between PMIP models,

MARGO sea-surface temperatures and pollen-based reconstructions. Quat. Sci.

Rev., 25, 2082–2102.

Kahn, R. A., B. J. Gaitley, J. V. Martonchik, D. J. Diner, K. A. Crean, and B. Holben, 2005:

Multiangle Imaging Spectroradiometer (MISR) global aerosol optical depth

validation based on 2 years of coincident Aerosol Robotic Network (AERONET)

observations. J. Geophys. Res. Atmos., 110, D10s04.

Kalnay, E., et al., 1996: The NCEP/NCAR 40-year reanalysis project. Bull. Am.

Meteorol. Soc., 77, 437–471.

Kanada, S., M. Nakano, S. Hayashi, T. Kato, M. Nakamura, K. Kurihara, and A. Kitoh,

2008: Reproducibility of Maximum Daily Precipitation Amount over Japan by a

High-resolution Non-hydrostatic Model. Sola, 4, 105–108.

Kanamaru, H., and M. Kanamitsu, 2007: Fifty-seven-year California reanalysis

downscaling at 10 km (CaRD10). Part II: Comparison with North American

regional reanalysis. J. Clim., 20, 5572–5592.

Kanamitsu, M., K. Yoshimura, Y. B. Yhang, and S. Y. Hong, 2010: Errors of interannual

variability and trend in dynamical downscaling of reanalysis. J. Geophys. Res.

Atmos., 115, D17115.

Kanamitsu, M., W. Ebisuzaki, J. Woollen, S. K. Yang, J. J. Hnilo, M. Fiorino, and G. L.

Potter, 2002: NCEP-DOE AMIP-II reanalysis (R-2). Bull. Am. Meteorol. Soc., 83,

1631–1643.

Karlsson, J., and G. Svensson, 2010: The simulation of Arctic clouds and their

inﬂuence on the winter surface temperature in present-day climate in the CMIP3

multi-model dataset. Clim. Dyn., 36, 623–635.

Karlsson, J., G. Svensson, and H. Rodhe, 2008: Cloud radiative forcing of subtropical

low level clouds in global models. Clim. Dyn., 30, 779–788.

Karpechko, A., N. Gillett, G. Marshall, and A. Scaife, 2008: Stratospheric inﬂuence on

circulation changes in the Southern Hemisphere troposphere in coupled climate

models. Geophys. Res. Lett., 35, L20806.

Karpechko, A. Y., and E. Manzini, 2012: Stratospheric inﬂuence on tropospheric

climate change in the Northern Hemisphere. J. Geophys. Res. Atmos., 117,

D05133.

Karpechko, A. Y., N. P. Gillett, G. J. Marshall, and J. A. Screen, 2009: Climate impacts

of the southern annular mode simulated by the CMIP3 models. J. Clim., 22,

6149–6150.

Kattsov, V. M., et al., 2010: Arctic sea-ice change: A grand challenge of climate

science. J. Glaciol., 56, 1115–1121.

Kavvada, A., A. Ruiz-Barradas, and S. Nigam, 2013: AMO’s structure and climate

footprint in observations and IPCC AR5 climate simulations. Clim. Dyn.,

doi:10.1007/s00382–013–1712–1.

Kawazoe, S., and W. Gutowski, 2013: Regional, very heavy daily precipitation in

NARCCAP simulations. J. Hydrometeorol., doi:10.1175/JHM-D-12-068.1.

Kay, J. E., M. M. Holland, and A. Jahn, 2011: Inter-annual to multi-decadal Arctic sea

ice extent trends in a warming world. Geophys. Res. Lett., 38, L15708.

Keeley, S. P. E., R. T. Sutton, and L. C. Shaffrey, 2012: The impact of North Atlantic sea

surface temperature errors on the simulation of North Atlantic European region

climate. Q. J. R. Meteorol. Soc., doi:10.1002/qj.1912.

Kendon, E. J., N. M. Roberts, C. A. Senior, and M. J. Roberts, 2012: Realism of rainfall

in a very high resolution regional climate model. J. Clim., 25, 5791–5806.

Khairoutdinov, M. F., D. A. Randall, and C. DeMott, 2005: Simulations of the

Atmospheric general circulation using a cloud-resolving model as a

superparameterization of physical processes. J. Atmos. Sci., 62, 2136–2154.

Kharin, V. V., F. W. Zwiers, X. B. Zhang, and G. C. Hegerl, 2007: Changes in temperature

and precipitation extremes in the IPCC ensemble of global coupled model

simulations. J. Clim., 20, 1419–1444.

Kharin, V. V., F. W. Zwiers, X. Zhang, and M. Wehner, 2012: Changes in temperature

and precipitation extremes in the CMIP5 ensemble. Clim. Change, doi:10.1007/

s10584-013-0705-8.

Khvorostyanov, D. V., G. Krinner, P. Ciais, M. Heimann, and S. A. Zimov, 2008a:

Vulnerability of permafrost carbon to global warming. Part I: Model description

and role of heat generated by organic matter decomposition. Tellus B, 60, 250–

264.

Khvorostyanov, D. V., P. Ciais, G. Krinner, S. A. Zimov, C. Corradi, and G. Guggenberger,

2008b: Vulnerability of permafrost carbon to global warming. Part II: Sensitivity

of permafrost carbon stock to global warming. Tellus B, 60, 265–275.

Kidston, J., and E. P. Gerber, 2010: Intermodel variability of the poleward shift of the

austral jet stream in the CMIP3 integrations linked to biases in 20th century

climatology. Geophys. Res. Lett., 37, L09708.

Kiehl, J. T., 2007: Twentieth century climate model response and climate sensitivity.

Geophys. Res. Lett., 34.

Kim, D., and V. Ramanathan, 2008: Solar radiation budget and radiative forcing due

to aerosols and clouds. J. Geophys. Res. Atmos., 113, D02203.

Kim, D., et al., 2012: The tropical subseasonal variability simulated in the NASA GISS

general circulation model. J. Clim., 25, 4641–4659.

Kim, D., et al., 2009: Application of MJO simulation diagnostics to climate models. J.

Clim., 22, 6413–6436.

Kim, H.-J., K. Takata, B. Wang, M. Watanabe, M. Kimoto, T. Yokohata, and T. Yasunari,

2011: Global monsoon, El Niño, and their interannual linkage simulated by

MIROC5 and the CMIP3 CGCMs. J. Clim., 24, 5604–5618.

Kim, S., and F.-F. Jin, 2011a: An ENSO stability analysis. Part I: Results from a hybrid

coupled model. Clim. Dyn., 36, 1593–1607.

Kim, S., and F.-F. Jin, 2011b: An ENSO stability analysis. Part II: Results from the

twentieth and twenty-ﬁrst century simulations of the CMIP3 models. Clim. Dyn.,

36, 1609–1627.

Kim, S. T., and J.-Y. Yu, 2012: The two types of ENSO in CMIP5 models. Geophys. Res.

Lett., 39, L11704.

Kirkevåg, K., et al., 2013: Aerosol-climate interactions in the Norwegian Earth

System Model – NorESM1–M. Geophys. Model Dev., 6, 207–244.

Kistler, R., et al., 2001: The NCEP-NCAR 50–year reanalysis: Monthly means CD-ROM

and documentation. Bull. Am. Meteorol. Soc., 82, 247–267.

Kjellstrom, E., G. Nikulin, U. Hansson, G. Strandberg, and A. Ullerstig, 2011: 21st

century changes in the European climate: Uncertainties derived from an

ensemble of regional climate model simulations. Tellus A, 63, 24–40.

Kjellstrom, E., F. Boberg, M. Castro, J. Christensen, G. Nikulin, and E. Sanchez, 2010:

Daily and monthly temperature and precipitation statistics as performance

indicators for regional climate models. Clim. Res., 44 135–150.

Klein, P., and G. Lapeyre, 2009: The oceanic vertical pump induced by mesoscale and

submesoscale turbulence. Annu. Rev. Mar. Sci., 1, 351–375.

Klein, S. A., and C. Jakob, 1999: Validation and sensitivities of frontal clouds

simulated by the ECMWF model. Mon. Weather Rev., 127, 2514–2531.

Klein, S. A., B. J. Soden, and N. C. Lau, 1999: Remote sea surface temperature

variations during ENSO: Evidence for a tropical atmospheric bridge. J. Clim., 12,

917–932.

Klein, S. A., X. Jiang, J. Boyle, S. Malyshev, and S. Xie, 2006: Diagnosis of the

summertime warm and dry bias over the U.S. Southern Great Plains in the GFDL

climate model using a weather forecasting approach. Geophys. Res. Lett., 33,

L18805.

Klein, S. A., Y. Zhang, M. D. Zelinka, R. Pincus, J. S. Boyle, and P. J. Glecker, 2013: Are

climate model simulations of clouds improving? An evaluation using the ISCCP

simulator. J. Geophys. Res., doi:10.1002/jgrd.50141.

Klocke, D., R. Pincus, and J. Quaas, 2011: On constraining estimates of climate

sensitivity with present-day observations through model weighting. J. Clim., 24,

6092–6099.

Kloster, S., N. M. Mahowald, J. T. Randerson, and P. J. Lawrence, 2012: The impacts of

climate, land use, and demography on ﬁres during the 21st century simulated by

CLM-CN. Biogeosciences, 9, 509–525.

Knight, J., et al., 2009: Do global temperature trends over the last decade falsify

climate predictions? [In: State of the Climate in 2008]. Bull. Am. Meteorol. Soc.,

90, S22–S23.

Knight, J. R., 2009: The Atlantic Multidecadal Oscillation inferred from the forced

climate response in Coupled General Circulation Models. J. Clim., 22, 1610–

1625.

Knutti, R., 2008: Why are climate models reproducing the observed global surface

warming so well? Geophys. Res. Lett., 35, L18704

Knutti, R., 2010: The end of model democracy? Clim. Change, 102, 395–404.

Knutti, R., and G. C. Hegerl, 2008: The equilibrium sensitivity of the Earth’s

temperature to radiation changes. Nature Geosci., 1, 735–743.

Knutti, R., and L. Tomassini, 2008: Constraints on the transient climate response

from observed global temperature and ocean heat uptake. Geophys. Res. Lett.,

35, L09701.

839

Evaluation of Climate Models Chapter 9

Knutti, R., and J. Sedlácek, 2013: Robustness and uncertainties in the new CMIP5

climate model projections. Nature Clim. Change, 3, 369–373.

Knutti, R., D. Masson, and A. Gettelman, 2013: Climate model genealogy: Generation

CMIP5 and how we got there. Geophys. Res. Lett., 40, 1194–1199.

Knutti, R., G. A. Meehl, M. R. Allen, and D. A. Stainforth, 2006: Constraining climate

sensitivity from the seasonal cycle in surface temperature. J. Clim., 19, 4224–

4233.

Knutti, R., F. Joos, S. A. Muller, G. K. Plattner, and T. F. Stocker, 2005: Probabilistic

climate change projections for CO

stabilization proﬁles. Geophys. Res. Lett.,

32, L20707.

Knutti, R., R. Furrer, C. Tebaldi, J. Cermak, and G. A. Meehl, 2010a: Challenges in

combining projections from multiple climate models. J. Clim., 23, 2739–2758.

Knutti, R., G. Abramowitz, M. Collins, V. Eyring, P. J. Gleckler, B. Hewitson, and L.

Mearns, 2010b: Good practice guidance paper on assessing and combining

multi model climate projections. In: Meeting Report of the Intergovernmental

Panel on Climate Change Expert Meeting on Assessing and Combining Multi

Model Climate Projections [T. F. Stocker, T.F., D. Qin, G.-K. Plattner, M. Tignor, and

P.M. Midgley (eds.)]. IPCC Working Group I Technical Support Unit, University of

Bern, Bern, Switzerland.

Koch, D., et al., 2011: Coupled Aerosol-Chemistry-Climate Twentieth-Century

Transient Model investigation: Trends in short-lived species and climate

responses. J. Clim., 24, 2693–2714.

Koldunov, N. V., D. Stammer, and J. Marotzke, 2010: Present-day Arctic sea ice

variability in the coupled ECHAM5/MPI-OM model. J. Clim., 23, 2520–2543.

Koltzow, M., T. Iversen, and J. Haugen, 2008: Extended Big-Brother experiments: The

role of lateral boundary data quality and size of integration domain in regional

climate modelling. Tellus A, 60, 398–410.

Koltzow, M. A. O., T. Iversen, and J. E. Haugen, 2011: The importance of lateral

boundaries, surface forcing and choice of domain size for dynamical downscaling

of global climate simulations. Atmosphere, 2, 67–95.

Komuro, Y., et al., 2012: Sea-ice in twentieth-century simulations by new MIROC

Coupled Models: A comparison between models with high resolution and with

ice thickness distribution. J. Meteorol. Soc. Jpn,, 90A, 213–232.

Konsta, D., H. Chepfer, and J.-L. Dufresne, 2012: A process oriented characterization

of tropical oceanic clouds for climate model evaluation, based on a statistical

analysis of daytime A-train observations. Clim. Dyn., 39, 2091–2108.

Koster, R., et al., 2004: Regions of strong coupling between soil moisture and

precipitation. Science, 305, 1138–1140.

Kostopoulou, E., K. Tolika, I. Tegoulias, C. Giannakopoulos, S. Somot, C.

Anagnostopoulou, and P. Maheras, 2009: Evaluation of a regional climate model

using in situ temperature observations over the Balkan Peninsula. Tellus A, 61,

357–370.

Koven, C. D., W. J. Riley, and A. Stern, 2013: Analysis of permafrost thermal dynamics

and response to climate change in the CMIP5 Earth System Models. J. Clim., 26,

1877–1900.

Koven, C. J., et al., 2011: Permafrost carbon-climate feedbacks accelerate global

warming. Proc. Natl. Acad. Sci. U.S.A., 108, 14769–14774.

Kowalczyk, E. A., Y. P. Wang, R. M. Law, H. L. Davies, J. L. McGregor, and G. Abramowitz

2006: The CSIRO Atmosphere Biosphere LandExchange (CABLE) model for use

in climate models and as an ofﬂine model. CSIRO Marine and Atmospheric

Research paper 013,Victoria, Australia, 37 pp.

Kowalczyk, E. A., et al., 2013: The land surface model component of ACCESS:

Description and impact on the simulated surface climatology. Aust. Meteorol.

Oceanogr. J., 63, 65–82.

Kravtsov, S., and C. Spannagle, 2008: Multidecadal climate variability in observed

and modeled surface temperatures. J. Clim., 21, 1104–1121.

Krinner, G., et al., 2005: A dynamic global vegetation model for studies of the coupled

atmosphere-biosphere system. Global Biogeochem. Cycles, 19, GB1015.

Krüger, L., R. da Rocha, M. Reboita, and T. Ambrizzi, 2012: RegCM3 nested in

HadAM3 scenarios A2 and B2: Projected changes in extratropical cyclogenesis,

temperature and precipitation over the South Atlantic Ocean. Clim. Change,

113, 599–621.

Kuhlbrodt, T., and J. Gregory, 2012: Ocean heat uptake and its consequences

for the magnitude of sea level rise and climate change. Geophys. Res. Lett.,

doi:10.1029/2012GL052952.

Kuhlbrodt, T., R. S. Smith, Z. Wang, and J. M. Gregory, 2012: The inﬂuence of eddy

parameterizations on the transport of the Antarctic Circumpolar Current in

coupled climate models. Ocean Model., 52–53, 1–8.

Kusaka, H., T. Takata, and Y. Takane, 2010: Reproducibility of regional climate in

central Japan using the 4-km Resolution WRF Model. Sola, 6, 113–116.

Kusunoki, S., R. Mizuta, and M. Matsueda, 2011: Future changes in the East Asian

rain band projected by global atmospheric models with 20-km and 60-km grid

size. Clim. Dyn., 37, 2481–2493.

L’Ecuyer, T., and G. Stephens, 2007: The tropical atmospheric energy budget from the

TRMM perspective. Part II: Evaluating GCM representations of the sensitivity

of regional energy and water cycles to the 1998–99 ENSO Cycle. J. Clim., 20,

4548–4571.

Laine, A., G. Lapeyre, and G. Riviere, 2011: A quasigeostrophic model for moist storm

tracks. J. Atmos. Sci., 68, 1306–1322.

Laine, A., M. Kageyama, P. Braconnot, and R. Alkama, 2009: Impact of greenhouse

gas concentration changes on surface energetics in IPSL-CM4: Regional

warming patterns, land-sea warming ratios, and glacial-interglacial differences.

J. Clim., 22, 4621–4635.

Lamarque, J. F., et al., 2012: CAM-chem: Description and evaluation of interactive

atmospheric chemistry in the Community Earth System Model. Geosci. Model

Dev., 5, 369–411.

Lamarque, J. F., et al., 2010: Historical (1850–2000) gridded anthropogenic and

biomass burning emissions of reactive gases and aerosols: Methodology and

application. Atmos. Chem. Phys., 10, 7017–7039.

Lambert, F. H., G. R. Harris, M. Collins, J. M. Murphy, D. M. H. Sexton, and B. B.

B. Booth, 2012: Interactions between perturbations to different Earth system

componentssimulated by a fully-coupled climate model. Clim. Dyn., doi:10.1007/

s00382-012-1618-3.

Lambert, S., and G. Boer, 2001: CMIP1 evaluation and intercomparison of coupled

climate models. Clim. Dyn., 17, 83–106.

Landrum, L., M. M. Holland, D. P. Schneider, and E. Hunke, 2012: Antarctic sea ice

climatology, variability and late 20th century change in CCSM4. J. Clim., 25,

4817–4838.

Langenbrunner, B., and J. D. Neelin, 2013: Analyzing ENSO teleconnections in CMIP

models as a measure of model ﬁdelity in simulating precipitation. J. Clim.,

doi:10.1175/JCLI-D-12-00542.1.

Laprise, R., 2008: Regional climate modelling. J. Comput. Phys., 227, 3641–3666.

Laprise, R., et al., 2008: Challenging some tenets of regional climate modelling.

Meteorol. Atmos. Phys., 100, 3–22.

Large, W., and S. Yeager, 2009: The global climatology of an interannually varying

air-sea ﬂux data set. Clim. Dyn., 33, 341–364.

Larow, T. E., Y. K. Lim, D. W. Shin, E. P. Chassignet, and S. Cocke, 2008: Atlantic basin

seasonal hurricane simulations. J. Clim., 21, 3191–3206.

Lau, K. M., et al., 2008: The Joint Aerosol-Monsoon Experiment —A new challenge

for monsoon climate research. Bull. Am. Meteorol. Soc., 89, 369–383.

Lau, W. K. M., and D. E. Waliser, 2011: Intraseasonal Variability of the Atmosphere-

Ocean Climate System. Springer Science+Business Media, New York, NY, USA,

and Heidelberg, Germany.

Lawrence, D. M., et al., 2012: The CCSM4 Land Simulation, 1850–2005: Assessment

of surface climate and new capabilities. J. Clim., 25, 2240–2260.

Lawrence, D. M., et al., 2011: Parameterization improvements and functional and

structural advances in version 4 of the Community Land Model. J. Adv. Model.

Earth Syst., 3, 2011MS000045.

Le Quere, C., et al., 2005: Ecosystem dynamics based on plankton functional types

for global ocean biogeochemistry models. Global Change Biol., 11, 2016–2040.

Le Quere, C., et al., 2009: Trends in the sources and sinks of carbon dioxide. Nature

Geosci., 2, 831–836.

Lecomte, O., T. Fichefet, M. Vancoppenolle, and M. Nicolaus, 2011: A new snow

thermodynamic scheme for large-scale sea-ice models. Ann. Glaciol., 52, 337–

346.

Leduc, M., and R. Laprise, 2009: Regional climate model sensitivity to domain size.

Clim. Dyn., 32, 833–854.

Lee, D. S., et al., 2009: Aviation and global climate change in the 21st century. Atmos.

Environ., 43, 3520–3537.

Lee, T., D. E. Waliser, J.-L. F. Li, F. W. Landerer, and M. M. Gierach, 2013: Evaluation of

CMIP3 and CMIP5 wind stress climatology using satellite measurements and

atmospheric reanalysis products. J. Clim., doi:10.1175/JCLI-D-12-00591.1.

Legg, S., L. Jackson, and R. W. Hallberg, 2008: Eddy-resolving modelingof overﬂows.

In: Eddy Resolving Ocean Models, 177 ed. [M. Hecht, and H. Hasumi (eds.)].

American Geophysical Union, Washington, DC, pp. 63–82.

840

Chapter 9 Evaluation of Climate Models

Legg, S., et al., 2009: Improving oceanic overﬂow representation in climate models:

The Gravity Current Entrainment Climate Process Team. Bull. Am. Meteorol. Soc.,

90, 657–670.

Leloup, J., M. Lengaigne, and J.-P. Boulanger, 2008: Twentieth century ENSO

characteristics in the IPCC database. Clim. Dyn., 30, 277–291.

Lemoine, D. M., 2010: Climate sensitivity distributions dependence on the possibility

that models share biases. J. Clim., 23, 4395–4415.

Lenaerts, J., M. van den Broeke, S. Dery, E. van Meijgaard, W. van de Berg, S. Palm, and

J. Rodrigo, 2012: Modeling drifting snow in Antarctica with a regional climate

model: 1. Methods and model evaluation. J. Geophys. Res. Atmos., 117, D05108.

Lenderink, G., 2010: Exploring metrics of extreme daily precipitation in a large

ensemble of regional climate model simulations. Clim. Res., 44 151–166.

Lenderink, G., and E. Van Meijgaard, 2008: Increase in hourly precipitation extremes

beyond expectations from temperature changes. Nature Geosci., 1, 511–514.

Levine, R. C., and A. G. Turner, 2012: Dependence of Indian monsoon rainfall on

moisture ﬂuxes across the Arabian Sea and the impact of coupled model sea

surface temperature biases. Clim. Dyn., 38, 2167–2190.

Levis, S., 2010: Modeling vegetation and land use in models of the Earth System.

Clim. Change, 1, 840–856.

Levitus, S., J. I. Antonov, T. P. Boyer, R. A. Locarnini, H. E. Garcia, and A. V. Mishonov,

2009: Global ocean heat content 1955–2008 in light of recently revealed

instrumentation problems. Geophys. Res. Lett., 36, L07608

Levy, H., L. W. Horowitz, M. D. Schwarzkopf, Y. Ming, J.-C. Golaz, V. Naik, and V.

Ramaswamy, 2013: The roles of aerosol direct and indirect effects in pastand

future climate change. J. Geophys. Res., doi:10.1002/jgrd.50192.

Lewis, T., and S. Lamoureux, 2010: Twenty-ﬁrst century discharge and sediment yield

predictions in a small high Arctic watershed. Global Planet. Change, 71, 27–41.

Li, C., J.-S. von Storch, and J. Marotzke, 2013a: Deep-ocean heat uptake and

equilibrium climate response. Clim. Dyn., 40, 1071–1086.

Li, G., and S.-P. Xie, 2012: Origins of tropical-wide SST biases in CMIP multi-model

ensembles. Geophys. Res. Lett., 39, L22703.

Li, H. B., A. Robock, and M. Wild, 2007: Evaluation of Intergovernmental Panel on

Climate Change Fourth Assessment soil moisture simulations for the second half

of the twentieth century. J. Geophys. Res. Atmos., 112, D06106

Li, J.-L. F., D. E. Waliser, and J. H. Jiang, 2011a: Correction to “Comparisons of satellites

liquid water estimates to ECMWF and GMAO analyses, 20th century IPCC AR4

climate simulations, and GCM simulations”. Geophys. Res. Lett., 38, L24807.

Li, J.-L. F., et al., 2008: Comparisons of satellites liquid water estimates to ECMWF

and GMAO analyses, 20th century IPCC AR4 climate simulations, and GCM

simulations. Geophys. Res. Lett., 35, L19710.

Li, J., S.-P. X. and A. Mestas-Nunez, E. R. C. and Gang Huang, R. D’Arrigo, F. Liu, J. Ma,

and X. Zheng, 2011b: Interdecadal modulation of ENSO amplitude during the

last millennium. Nature Clim. Change, 1, 114–118.

Li, J. L. F., et al., 2012a: An observationally-based evaluation of cloud ice water in

CMIP3 and CMIP5 GCMs and contemporary reanalyses using contemporary

satellite data. J. Geophys. Res., 117, D16105.

Li, L., et al., 2013b: Development and Evaluation of Grid-point Atmospheric Model of

IAP LASG, Version 2.0 (GAMIL 2.0). Adv. Atmos. Sci., 30, 855–867.

Li, L., et al., 2012b: The Flexible Global Ocean-Atmosphere-Land System Model: Grid-

point Version 2: FGOALS-g2. Adv. Atmos. Sci., doi:10.1007/s00376–012–2140–6.

Li, T., and G. H. Philander, 1996: On the annual cycle in the eastern equatorial Paciﬁc.

J. Clim., 9, 2986–2998.

Li, T., C. W. Tham, and C. P. Chang, 2001: A coupled air-sea-monsoon oscillator for the

tropospheric biennial oscillation. J. Clim., 14, 752–764.

Liebmann, B., R. M. Dole, C. Jones, I. Blade, and D. Allured, 2010: Inﬂuence of

choice of time period on global surface temperature trend wstimates. Bull. Am.

Meteorol. Soc., 91, 1485–1491.

Lienert, f., J. C. Fyfe, and W. J. Merryﬁeld, 2011: Do climate models capture the

tropical inﬂuences on North Paciﬁc sea surface temperature variability? J. Clim.,

24, 6203–6209.

Lin, A. L., and T. Li, 2008: Energy spectrum characteristics of Boreal Summer

Intraseasonal Oscillations: Climatology and variations during the ENSO

developing and decaying phases. J. Clim., 21, 6304–6320.

Lin, J.-L., 2007: The double-ITCZ problem in IPCC AR4 Coupled GCMs: Ocean-

atmosphere feedback analysis. J. Clim., 20, 4497–4525.

Lin, J. L., et al., 2006: Tropical intraseasonal variability in 14 IPCC AR4 climate models.

Part I: Convective signals. J. Clim., 19, 2665–2690.

Lin, P., Y. Yongqiang, and H. Liu, 2013: Long-term stability and oceanic mean state

simulated by the coupled model FGOALS-s2. Adv. Atmos. Sci., 30, 175–192.

Lin, Y., et al., 2012: TWP-ICE global atmospheric model intercomparison: Convection

responsiveness and resolution impact. J. Geophys. Res., 117, D09111.

Lindvall, J., G. Svensson, and C. Hannay, 2012: Evaluation of near-surface parameters

in the two versions of the atmospheric model in CESM1 using ﬂux station

observations. J. Clim., 26 26–44.

Linkin, M., and S. Nigam, 2008: The north paciﬁc oscillation-west Paciﬁc

teleconnection pattern: Mature-phase structure and winter impacts. J. Clim., 21,

1979–1997.

Liu, H., C. Wang, S. K. Lee, and D. Enﬁeld, 2013a: Atlantic Warm Pool Variability in the

CMIP5 Simulations. J. Clim., doi:10.1175/JCLI-D-12–00556.1.

Liu, H. L., P. F. Lin, Y. Q. Yu, and X. H. Zhang, 2012a: The baseline evaluation of LASG/

IAP Climate system Ocean Model (LICOM) version 2.0. Acta Meteorol. Sin., 26,

318–329.

Liu, J., 2010: Sensitivity of sea ice and ocean simulations to sea ice salinity in a

coupled global climate model. Science China Earth Sci., 53, 911–918.

Liu, L., W. Yu, and T. Li, 2011: Dynamic and thermodynamic air–sea coupling

associated with the Indian Ocean dipole diagnosed from 23 WCRP CMIP3

Models. J. Clim., 24, 4941–4958.

Liu, S. C., C. B. Fu, C. J. Shiu, J. P. Chen, and F. T. Wu, 2009: Temperature dependence

of global precipitation extremes. Geophys. Res. Lett., 36, L17702.

Liu, X., et al., 2012b: Toward a minimal representation of aerosols in climate models:

Description and evaluation in the Community Atmosphere Model CAM5.

Geophys. Model Dev., 5, 709–739.

Liu, X. H., et al., 2007: Uncertainties in global aerosol simulations: Assessment using

three meteorological data sets. J. Geophys. Res. Atmos., 112, D11212

Liu, Y., 1996: Modeling the emissions of nitrous oxide and methane from the

terrestrial biosphere to the atmosphere. In: Joint Program Report Series. MIT

Joint Program on the Science and Policy of Global Change, Cambridge, MA,

USA, 219 pp.

Liu, Y., J. Hu, B. He, Q. Bao, A. Duan, and G. X. Wu, 2013b: Seasonal evolution of

subtropical anticyclones in the Climate System Model FGOALS-s2. Adv. Atmos.

Sci., 30, 593–606.

Lloyd, J., E. Guilyardi, and H. Weller, 2010: The role of atmosphere feedbacks during

ENSO in the CMIP3 models. Part II: Using AMIP runs to understand the heat ﬂux

feedback mechanisms. Clim. Dyn., 37, 1271–1292.

Lloyd, J., E. Guilyardi, and H. Weller, 2012: The role of atmosphere feedbacks during

ENSO in the CMIP3 Models. Part III: The Shortwave Flux Feedback. J. Clim., 25,

4275–4293.

Lloyd, J., E. Guilyardi, H. Weller, and J. Slingo, 2009: The role of atmosphere feedbacks

during ENSO in the CMIP3 models. Atmos. Sci. Lett., 10, 170–176.

Loeb, N. G., et al., 2009: Toward optimal closure of the Earth’s top-of-atmosphere

radiation budget. J. Clim., 22, 748–766.

Lohmann, U., K. von Salzen, N. McFarlane, H. G. Leighton, and J. Feichter, 1999:

Tropospheric sulfur cycle in the Canadian general circulation model. J. Geophys.

Res. Atmos., 104, 26833–26858.

Long, M. C., K. Lindsay, S. Peacock, J. K. Moore, and S. C. Doney, 2012: Twentieth-

century oceanic carbon uptake and storage in CESM1(BGC). J. Clim., doi:10.1175/

JCLI-D-12-00184.1.

Loptien, U., O. Zolina, S. Gulev, M. Latif, and V. Soloviov, 2008: Cyclone life cycle

characteristics over the Northern Hemisphere in coupled GCMs. Clim. Dyn., 31,

507–532.

Lorenz, P., and D. Jacob, 2005: Inﬂuence of regional scale information on the global

circulation: A two-way nesting climate simulation. Geophys. Res. Lett., 32,

L18706.

Lorenz, R., E. L. Davin, and S. I. Seneviratne, 2012: Modeling land-climate coupling

in Europe: Impact of land surface representation on climate variability and

extremes. J. Geophys. Res., 117, doi:10.1029/2012JD017755.

Losch, M., D. Menemenlis, J.-M. Campin, P. Heimbach, and C. Hill, 2010: On the

formulation of sea-ice models. Part 1: Effects of different solver implementations

and parameterizations. Ocean Model., 33, 129–144.

Loschnigg, J., G. A. Meehl, P. J. Webster, J. M. Arblaster, and G. P. Compo, 2003: The

Asian monsoon, the tropospheric biennial oscillation, and the Indian Ocean

zonal mode in the NCAR CSM. J. Clim., 16, 1617–1642.

Loutre, M. F., A. Mouchet, T. Fichefet, H. Goosse, H. Goelzer, and P. Huybrechts,

2011: Evaluating climate model performance with various parameter sets using

observations over the recent past. Clim. Past, 7, 511–526.

Loyola, D., and M. Coldewey-Egbers, 2012: Multi-sensor data merging with stacked

neural networks for the creation of satellite long-term climate data records.

Eurasip J. Adv. Signal Proc., doi:10.1186/1687–6180–2012–91.

841

Evaluation of Climate Models Chapter 9

Loyola, D., et al., 2009: Global long-term monitoring of the ozone layer—a

prerequisite for predictions. Int. J. Remote Sens., 30, 4295–4318.

Lu, J., G. A. Vecchi, and T. Reichler, 2007: Expansion of the Hadley cell under global

warming. Geophys. Res. Lett., 34, L06805.

Lu, J. H., and J. J. Ji, 2006: A simulation and mechanism analysis of long-term

variations at land surface over arid/semi-arid area in north China. J. Geophys.

Res. Atmos., 111, D09306.

Lucarini, V., and F. Ragone, 2011: Energetics of climate models: Net energy balance

and meridional enthalpy transport. Rev. Geophys., 49, RG1001.

Lucas-Picher, P., S. Somot, M. Déqué, B. Decharme, and A. Alias, 2012a: Evaluation of

the regional climate model ALADIN to simulate the climate over North America

in the CORDEX framework. Clim. Dyn., doi:10.1007/s00382-012-1613-8.

Lucas-Picher, P., M. Wulff-Nielsen, J. Christensen, G. Adalgeirsdottir, R. Mottram, and

S. Simonsen, 2012b: Very high resolution regional climate model simulations

over Greenland: Identifying added value. J. Geophys. Res. Atmos., 117, D02108.

Lumpkin, R., K. G. Speer, and K. P. Koltermann, 2008: Transport across 48°N in the

Atlantic Ocean. J. Phys. Oceanogr., 38, 733–752.

Luo, J. J., S. Masson, E. Roeckner, G. Madec, and T. Yamagata, 2005: Reducing

climatology bias in an ocean-atmosphere CGCM with improved coupling

physics. J. Clim., 18, 2344–2360.

Lynn, B., R. Healy, and L. Druyan, 2009: Quantifying the sensitivity of simulated

climate change to model conﬁguration. Clim. Change, 92, 275–298.

MacKinnon, J., et al., 2009: Using global arrays to investigate internal-wavesand

mixing. In: OceanObs09: Sustained Ocean Observations and Information for

Society, Venice, Italy, ESA.

Madden, R. A., and P. R. Julian, 1972: Description of global-scale circulation ells in

tropics with a 40–50 day period. J. Atmos. Sci., 29, 1109–1123.

Madden, R. A., and P. R. Julian, 1994: Observations of the 40–50-Day Tropical

Oscillation—a Review. Mon. Weather Rev., 122, 814–837.

Madec, G., 2008: NEMO ocean engine. Technical Note. Institut Pierre-Simon Laplace

(IPSL), France, 300pp.

Madec, G., P. Delecluse, M. Imbard, and C. Levy, 1998: OPA 8.1 ocean general

circulation model reference manual. IPSL Note du Pole de Modelisation, Institut

Pierre-Simon Laplace (IPSL), France, 91 pp.

Mahlstein, I., and R. Knutti, 2010: Regional climate change patterns identiﬁed by

cluster analysis. Clim. Dyn., 35, 587–600.

Mahlstein, I., and R. Knutti, 2012: September Arctic sea ice predicted to disappear

near 2C global warming above present. J. Geophys. Res., 117, D06104.

Maier-Reimer, E., I. Kriest, J. Segschneider, and P. Wetze, 2005: The HAMburg Ocean

Carbon Cycle Model HAMOCC 5.1-Technical Description Release 1.1. Tech.

Rep. 14, Rep. Earth Syst. Sci., Max Planck Institute for Meteorology, Hamburg,

Germany, 50 pp.

Mantua, N. J., S. R. Hare, Y. Zhang, J. M. Wallace, and R. C. Francis, 1997: A Paciﬁc

interdecadal climate oscillation with impacts on salmon production. Bull. Am.

Meteorol. Soc., 78, 1069–1079.

Manzini, E., C. Cagnazzo, P. G. Fogli, A. Bellucci, and W. A. Muller, 2012: Stratosphere-

troposphere coupling at inter-decadal time scales: Implications for the North

Atlantic Ocean. Geophys. Res. Lett., 39, L05801.

Maraun, D., 2012: Nonstationarities of regional climate model biases in European

seasonal mean temperature and precipitation sums. Geophys. Res. Lett., 39,

L06706.

Maraun, D., H. Rust, and T. Osborn, 2010a: Synoptic airﬂow and UK daily precipitation

extremes: Development and validation of a vector generalised linear model.

Extremes, 13, 133–153.

Maraun, D., et al., 2010b: Precipitation downscaling under climate change: Recent

developments to bridge the gap between dynamical models and the end user.

Rev. Geophys., 48, RG3003.

Marchand, R., N. Beagley, and T. P. Ackerman, 2009: Evaluation of hydrometeor

occurrence proﬁles in the Multiscale Modeling Framework Climate Model using

atmospheric classiﬁcation. J. Clim., 22, 4557–4573.

Markovic, M., H. Lin, and K. Winger, 2010: Simulating global and North American

climate using the Global Environmental Multiscale Model with a Variable-

Resolution Modeling Approach. Mon. Weather Rev., 138, 3967–3987.

Marsh, R., S. A. Mueller, A. Yool, and N. R. Edwards, 2011: Incorporation of the

C-GOLDSTEIN efﬁcient climate model into the GENIE framework: “eb_go_gs”

conﬁgurations of GENIE. Geophys. Model Dev., 4, 957–992.

Marsh, R., et al., 2009: Recent changes in the North Atlantic circulation simulated

with eddy-permitting and eddy-resolving ocean models. Ocean Model., 28,

226–239.

Marsland, S. J., et al., 2013: Evaluation of ACCESS Climate Model ocean diagnostics

in CMIP5 simulations. Aust. Meteorol. and Oceanogr. J., 63,101–119.

Martin, G. M., and R. C. Levine, 2012: The inﬂuence of dynamic vegetation on

the present-day simulation and future projectons of the South Asian summer

monsoon in the HadGEM2 family. Earth Syst. Dyn., 2, 245–261.

Martin, G. M., et al., 2011: The HadGEM2 family of Met Ofﬁce Uniﬁed Model climate

conﬁgurations. Geophys. Model Dev., 4, 723–757.

Masarie, K. A., and P. P. Tans, 1995: Extension and integration of atmospheric carbon

dioxide data into a globally consistent measurement record. J. Geophys. Res.

Atmos., 100, 11593–11610.

Masato, G., B. Hoskins, and T. Woollings, 2012: Winter and summer Northern

Hemisphere blocking in CMIP5 models. J. Clim., doi:10.1175/JCLI-D-12-00466.1.

Masson-Delmotte, V., et al., 2010: EPICA Dome C record of glacial and interglacial

intensities. Quat. Sci. Rev., 29, 113–128.

Masson-Delmotte, V., et al., 2006: Past and future polar ampliﬁcation of climate

change: Climate model intercomparisons and ice-core constraints. Clim. Dyn.,

27, 437–440.

Masson, D., and R. Knutti, 2011a: Climate model genealogy. Geophys. Res. Lett.,

38, L08703.

Masson, D., and R. Knutti, 2011b: Spatial-scale dependence of climate model

performance in the CMIP3 ensemble. J. Clim., 24, 2680–2692.

Masson, D., and R. Knutti, 2013: Predictor screening, calibration and observational

constraints in climate model ensembles: An illustration using climate sensitivity.

J. Clim., 26, 887–898.

Massonnet, F., T. Fichefet, H. Goosse, C. M. Bitz, G. Philippon-Berthier, M. M. Holland,

and P.-Y. Barriat, 2012: Constraining projections of summer Arctic sea ice.

Cryosphere, 6, 1383–1394.

Mastrandrea, M. D., et al., 2011: Guidance Note for Lead Authors of the IPCC Fifth

Assessment Report on Consistent Treatment of Uncertainties. Intergovernmental

Panel on Climate Change (IPCC). IPCC guidance note, Jasper Ridge, CA, USA, 7

pp.

Materia, S., P. A. Dirmeyer, Z. C. Guo, A. Alessandri, and A. Navarra, 2010: The

sensitivity of simulated river discharge to land surface representation and

meteorological forcings. J. Hydrometeorol., 11, 334–351.

Matsueda, M., 2009: Blocking predictability in operational medium-range ensemble

forecasts. Sola, 5, 113–116.

Matsueda, M., R. Mizuta, and S. Kusunoki, 2009: Future change in wintertime

atmospheric blocking simulated using a 20-km-mesh atmospheric global

circulation model. J. Geophys. Res. Atmos., 114, D12114.

Matsueda, M., H. Endo, and R. Mizuta, 2010: Future change in Southern Hemisphere

summertime and wintertime atmospheric blockings simulated using a

20-km-mesh AGCM. Geophys. Res. Lett., 37, L02803.

Matsumoto, K., K. S. Tokos, A. R. Price, and S. J. Cox, 2008: First description of the

Minnesota Earth System Model for Ocean biogeochemistry (MESMO 1.0).

Geophys. Model Dev., 1, 1–15.

Maurer, E., and H. Hidalgo, 2008: Utility of daily vs. monthly large-scale climate data:

An intercomparison of two statistical downscaling methods. Hydrol. Earth Syst.

Sci., 12, 551–563.

Mauritsen, T., et al., 2012: Tuning the climate of a global model. J. Adv. Model. Earth

Syst., 4, M00A01.

Maximenko, N., et al., 2009: Mean dynamic topography of the ocean derived from

satellite and drifting buoy data using three different techniques. J. Atmos. Ocean.

Technol., 26, 1910–1919.

May, P. T., J. H. Mather, G. Vaughan, K. N. Bower, C. Jakob, G. M. McFarquhar, and G.

G. Mace, 2008: The Tropical Warm Pool International Cloud Experiment. Bull. Am.

Meteorol. Soc., 89, 629–645.

May, W., 2007: The simulation of the variability and extremes of daily precipitation

over Europe by the HIRHAM regional climate model. Global Planet. Change,

57, 59–82.

McCarthy, G., et al., 2012: Observed interannual variability of the Atlantic meridional

overturning circulation at 26.5 degrees N. Geophys. Res. Lett., 39, L19609.

McClean, J. L., and J. C. Carman, 2011: Investigation of IPCC AR4 coupled climate

model North Atlantic modewater formation. Ocean Model., 40, 14–34.

McClean, J. L., M. E. Maltrud, and F. O. Bryan, 2006: Measures of the ﬁdelity of

eddying ocean models. Oceanography, 19, 104–117.

McClean, J. L., et al., 2011: A prototype two-decade fully-coupled ﬁne-resolution

CCSM simulation. Ocean Model. 39, 10–30.

842

Chapter 9 Evaluation of Climate Models

McCormack, J. P., S. D. Eckermann, D. E. Siskind, and T. J. McGee, 2006: CHEM2D-OPP:

A new linearized gas-phase ozone photochemistry parameterization for high-

altitude NWP and climate models. Atmos. Chem. Phys., 6, 4943–4972.

McCrary, R. R., and D. A. Randall, 2010: Great plains drought in simulations of the

twentieth century. J. Clim., 23, 2178–2196.

McDonald, R. E., 2011: Understanding the impact of climate change on Northern

Hemisphere extra-tropical cyclones. Clim. Dyn., 37, 1399–1425.

McDougall, T. J., and P. C. McIntosh, 2001: The temporal-residual-mean velocity. Part

II: Isopycnal interpretation and the tracer and momentum equations. J. Phys.

Oceanogr., 31, 1222–1246.

McKitrick, R., S. McIntyre, and C. Herman, 2010: Panel and multivariate methods for

tests of trend equivalence in climate data series. Atmos. Sci. Lett., 11, 270–277.

McKitrick, R., S. McIntyre, and C. Herman, 2011: Panel and multivariate methods for

tests of trend equivalence in climate data series. Atmos. Sci. Lett., 12, 386–388.

McLandress, C., T. Shepherd, J. Scinocca, D. Plummer, M. Sigmond, A. Jonsson, and

M. Reader, 2011: Separating the dynamical effects of climate change and ozone

depletion. Part II Southern Hemisphere troposphere. J. Clim., 24, 1850–1868.

McLaren, A. J., et al., 2006: Evaluation of the sea ice simulation in a new coupled

atmosphere-ocean climate model (HadGEM1). J. Geophys. Res. Oceans, 111,

C12014.

McManus, J. F., R. Francois, J. M. Gherardi, L. D. Keigwin, and S. Brown-Leger, 2004:

Collapse and rapid resumption of Atlantic meridional circulation linked to

deglacial climate changes. Nature, 428, 834–837.

McWilliams, J. C., 2008: The nature and consequences of oceanic eddies. In: Ocean

Modeling in an Eddying Regime [M. Hecht and H. Hasumi (eds.)]. American

Geophysical Union, Washington, DC, pp. 5–15.

Mearns, L. O., et al., 2012: The North American Regional Climate Change Assessment

Program: Overview of Phase I Results. Bull. Am. Meteorol. Soc., 93, 1337–1362.

Mears, C. A., F. J. Wentz, P. Thorne, and D. Bernie, 2011: Assessing uncertainty in

estimates of atmospheric temperature changes from MSU and AMSU using a

Monte-Carlo estimation technique. J. Geophys. Res., 116, D08112.

Mears, C. A., B. D. Santer, F. J. Wentz, K. E. Taylor, and M. F. Wehner, 2007: Relationship

between temperature and precipitable water changes over tropical oceans.

Geophys. Res. Lett., 34, L24709.

Meehl, G. A., and J. M. Arblaster, 2011: Decadal variability of Asian-Australian

Monsoon-ENSO-TBO relationships. J. Clim., 24, 4925–4940.

Meehl, G. A., and H. Teng, 2012: Case studies for initialized decadal hindcasts and

predictions for the Paciﬁc region. Geophys. Res. Lett., 39, L22705.

Meehl, G. A., J. M. Arblaster, and J. Loschnigg, 2003: Coupled ocean-atmosphere

dynamical processes in the tropical Indian and Paciﬁc Oceans and the TBO. J.

Clim., 16, 2138–2158.

Meehl, G. A., J. M. Arblaster, J. T. Fasullo, A. X. Hu, and K. E. Trenberth, 2011: Model-

based evidence of deep-ocean heat uptake during surface-temperature hiatus

periods. Nature Clim. Change, 1, 360–364.

Meehl, G. A., A. Hu, J. Arblaster, J. Fasullo, and K. E. Trenberth, 2013a: Externally

forced and internally generated decadal climate variability associated with the

Interdecadal Paciﬁc Oscillation. J. Clim., doi:10.1175/JCLI-D-12-00548.1.

Meehl, G. A., P. R. Gent, J. M. Arblaster, B. L. Otto-Bliesner, E. C. Brady, and A. Craig,

2001: Factors that affect the amplitude of El Niño in global coupled climate

models. Clim. Dyn., 17, 515–526.

Meehl, G. A., J. M. Arblaster, J. M. Caron, H. Annamalai, M. Jochum, A. Chakraborty,

and R. Murtugudde, 2012: Monsoon regimes and processes in CCSM4. Part I: The

Asian-Australian Monsoon. J. Clim., 25, 2583–2608.

Meehl, G. A., et al., 2007: The WCRP CMIP3 multimodel dataset —A new era in

climate change research. Bull. Am. Meteorol. Soc., 88, 1383–1394.

Meehl, G. A., et al., 2009: Decadal prediction: Can it be skillful? Bull. Am. Meteorol.

Soc., 90, 1467–1485.

Meehl, G. A., et al., 2013b: Decadal climate prediction: An update from the trenches.

Bull. Am. Meteorol. Soc., doi:10.1175/BAMS-D-12-00241.1.

Meijers, A., E. Shuckburgh, N. Bruneau, J.-B. Sallee, T. Bracegirdle, and Z. Wang,

2012: Representation of the Antarctic Circumpolar Current in the CMIP5 climate

models and future changes under warming scenarios. J. Geophys. Res. Oceans,

117, C12008.

Meinshausen, M., et al., 2009: Greenhouse-gas emission targets for limiting global

warming to 2 degrees C. Nature, 458, 1158–1162.

Meinshausen, M., et al., 2011: The RCP greenhouse gas concentrations and their

extensions from 1765 to 2300. Clim. Change, 109, 213–241.

Meissner, K. J., A. J. Weaver, H. D. Matthews, and P. M. Cox, 2003: The role of land

surface dynamics in glacial inception: a study with the UVic Earth System Model.

Clim. Dyn., 21, 515–537.

Melillo, J. M., A. D. McGuire, D. W. Kicklighter, B. Moore, C. J. Vorosmarty, and A.

L. Schloss, 1993: Global climate-change and terrestrial net primary production.

Nature, 363, 234–240.

Melsom, A., V. Lien, and W. P. Budgell, 2009: Using the Regional Ocean Modeling

System (ROMS) to improve the ocean circulation from a GCM 20th century

simulation. Ocean Dyn., 59, 969–981.

Menary, M., W. Park, K. Lohmann, M. Vellinga, D. Palmer, M. Latif, and J. H. Jungclaus,

2012: A multimodel comparison of centennial Atlantic meridional overturning

circulation variability. Clim. Dyn., 38, 2377–2388.

Menendez, C., M. de Castro, A. Sorensson, J. Boulanger, and C. M. Grp, 2010: CLARIS

Project: Towards climate downscaling in South America. Meteorol. Z., 19, 357–

362.

Menon, S., D. Koch, G. Beig, S. Sahu, J. Fasullo, and D. Orlikowski, 2010: Black carbon

aerosols and the third polar ice cap. Atmos. Chem. Phys., 10, 4559–4571.

Mercado, L. M., C. Huntingford, J. H. C. Gash, P. M. Cox, and V. Jogireddy, 2007:

Improving the representation of radiation interception and photosynthesis for

climate model applications. Tellus B, 59, 553–565.

Merriﬁeld, M. A., and M. E. Maltrud, 2011: Regional sea level trends due to a Paciﬁc

trade wind intensiﬁcation. Geophys. Res. Lett., 38, L21605.

Merryﬁeld, W. J., et al., 2013: The Canadian Seasonal to Interannual Prediction

System. Part I: Models and Initialization. Mon. Weather Rev., doi:10.1175/MWR-

D-12-00216.1.

Mieville, A., et al., 2010: Emissions of gases and particles from biomass burning

during the 20th century using satellite data and an historical reconstruction.

Atmos. Environ., 44, 1469–1477.

Miller, A., et al., 2002: A cohesive total ozone data set from the SBUV(/2) satellite

system. J. Geophys. Res. Atmos., 107, 4701.

Milliff, R., A. Bonazzi, C. Wikle, N. Pinardi, and L. Berliner, 2011: Ocean ensemble

forecasting. Part I: Ensemble Mediterranean winds from a Bayesian hierarchical

model. Q. J. R. Meteorol. Soc., 137, 858–878.

Milly, P. C. D., and A. B. Shmakin, 2002: Global modeling of land water and energy

balances. Part I: the land dynamics (LaD) model. J. Hydrometeorol., 3, 283–299.

Min, S. K., X. B. Zhang, F. W. Zwiers, and G. C. Hegerl, 2011: Human contribution to

more-intense precipitation extremes. Nature, 470, 376–379.

Minobe, S., 1997: A 50–70 year climatic oscillation over the North Paciﬁc and North

America. Geophys. Res. Lett., 24, 683–686.

Minobe, S., 1999: Resonance in bidecadal and pentadecadal climate oscillations

over the North Paciﬁc: Role in climatic regime shifts. Geophys. Res. Lett., 26,

855–858.

Misra, V., 2007: Addressing the issue of systematic errors in a regional climate

model. J. Clim., 20, 801–818.

Mitchell, T. D., and P. D. Jones, 2005: An improved method of constructing a database

of monthly climate observations and associated high-resolution grids. Int. J.

Climatol. , 25, 693–712.

Miyama, T., and M. Kawamiya, 2009: Estimating allowable carbon emission for

CO(2) concentration stabilization using a GCM-based Earth system model.

Geophys. Res. Lett., 36, L19709.

Mizuta, R., et al., 2012: Climate simulations using MRI-AGCM3.2 with 20-km grid. J.

Meteorol. Soc. Jpn., 90A, 233–258.

Molteni, F., 2003: Atmospheric simulations using a GCM with simpliﬁed physical

parameterizations. I: Model climatology and variability in multi-decadal

experiments. Clim. Dyn., 20, 175–191.

Montoya, M., A. Griesel, A. Levermann, J. Mignot, M. Hofmann, A. Ganopolski, and S.

Rahmstorf, 2005: The earth system model of intermediate complexity CLIMBER-3

alpha. Part 1: description and performance for present-day conditions. Clim.

Dyn., 25, 237–263.

Morgenstern, O., et al., 2010: Anthropogenic forcing of the Northern Annular Mode

in CCMVal-2 models. J. Geophys. Res., 115, D00M03.

Morice, C. P., J. J. Kennedy, N. A. Rayner, and P. D. Jones, 2012: Quantifying

uncertainties in global and regional temperature change using an ensemble of

observational estimates: The HadCRUT4 data set. J. Geophys. Res. Atmos., 117,

D08101.

Moss, R. H., et al., 2010: The next generation of scenarios for climate change research

and assessment. Nature, 463, 747–756.

843

Evaluation of Climate Models Chapter 9

Mouchet, A., and L. M. François, 1996: Sensitivity of a global oceanic carbon cycle

model to the circulation and the fate of organic matter: Preliminary results. Phys.

Chem. Earth, 21, 511–516.

Msadek, R., and C. Frankignoul, 2009: Atlantic multidecadal oceanic variability and

its inﬂuence on the atmosphere in a climate model. Clim. Dyn., 33, 45–62.

Msadek, R., W. E. Johns, S. G. Yeager, G. Danabasoglu, T. Delworth, and T. Rosati,

2013: The Atlantic meridional heat transport at 26.5ºN and its relationship with

the MOC in the RAPID-array and GFDL and NCAR coupled models. J. Clim.,

doi:10.1175/JCLI-D-12–00081.1.

Mueller, B., et al., 2011: Evaluation of global observations-based evapotranspiraion

datasets and IPCC AR4 simulations. Geophys. Res. Lett., 38, L06402.

Muller, S. A., F. Joos, N. R. Edwards, and T. F. Stocker, 2006: Water mass distribution

and ventilation time scales in a cost-efﬁcient, three-dimensional ocean model.

J. Clim., 19, 5479–5499.

Murakami, H., and M. Sugi, 2010: Effect of model resolution on tropical cyclone

climate projections. Sola, 6, 73–76.

Murakami, H., et al., 2012: Future changes in tropical cyclone activity projected by

the new high-resolution MRI-AGCM. J. Clim., 25, 3237–3260.

Murphy, D. M., 2013: Little net clear-sky radiative forcing from recent regional

redistribution of aerosols. Nature Geosci., 6, 258–262.

Murphy, J., B. Booth, M. Collins, G. Harris, D. Sexton, and M. Webb, 2007: A

methodology for probabilistic predictions of regional climate change from

perturbed physics ensembles. Philos.Trans. R. Soc. London A, 365 1993–2028.

Murphy, J. M., D. M. H. Sexton, D. N. Barnett, G. S. Jones, M. J. Webb, M. Collins,

and D. A. Stainforth, 2004: Quantiﬁcation of modelling uncertainties in a large

ensemble of climate change simulations. Nature, 430, 768–772.

Murtugudde, R., J. Beauchamp, C. R. McClain, M. Lewis, and A. J. Busalacchi, 2002:

Effects of penetrative radiation on the upper tropical ocean circulation. J. Clim.,

15, 470–486.

Muryshev, K. E., A. V. Eliseev, I. I. Mokhov, and N. A. Diansky, 2009: Validating and

assessing the sensitivity of the climate model with an ocean general circulation

model developed at the Institute of Atmospheric Physics, Russian Academy of

Sciences. Izvestiya Atmos. Ocean. Phys., 45, 416–433.

Nagura, M., W. Sasaki, T. Tozuka, J.-J. Luo, S. K. Behera, and T. Yamagata, 2013:

Longitudinal biases in the Seychelles Dome simulated by 35 ocean-atmosphere

coupled general circulation models. J. Geophys. Res., doi:10.1029/2012JC008352.

Nakano, H., H. Tsujino, M. Hirabara, T. Yasuda, T. Motoi, M. Ishii, and G. Yamanaka,

2011: Uptake mechanism of anthropogenic CO

in the Kuroshio Extension

region in an ocean general circulation model. J. Oceanogr., 67, 765–783.

Nam, C., S. Bony, J. L. Dufresne, and H. Chepfer, 2012: The ‘too few, too bright’

tropical low-cloud problem in CMIP5 models. Geophys. Res. Lett., 39, L21801.

Nanjundiah, R. S., V. Vidyunmala, and J. Srinivasan, 2005: The impact of increase

in CO

on the simulation of tropical biennial oscillations (TBO) in 12 coupled

general circulation models. Atmos. Sci. Lett., 6, 183–191.

Naoe, H., and K. Shibata, 2010: Equatorial quasi-biennial oscillation inﬂuence on

northern winter extratropical circulation. J. Geophys. Res. Atmos., 115, D19102.

Neale, R. B., J. H. Richter, and M. Jochum, 2008: The Impact of Convection on ENSO:

From a delayed oscillator to a series of events. J. Clim., 21, 5904–5924.

Neale, R. B., J. Richter, S. Park, P. H. Lauritzen, S. J. Vavrus, P. J. Rasch, and M. Zhang,

2013: The Mean Climate of the Community Atmosphere Model (CAM4) in forced

SST and fully coupled experiments. J. Clim., doi:10.1175/JCLI-D-12-00236.1.

Neale, R. B., et al., 2010: Description of the NCAR Community Atmosphere Model

(CAM 4.0). NCAR Technical Note NCAR/TN-486+STR, National Center for

Atmopsheric Research, Boulder, CO, 268 pp.

Neelin, J. D., 2007: Moist dynamics of tropical convection zones in monsoons,

teleconnections and global warming. In: The Global Circulation of the

Atmosphere [T. Schneider and A. Sobel (eds.)]. Princeton University Press,

Princeton, NJ. 385 pp.

Neelin, J. D., and N. Zeng, 2000: A quasi-equilibrium tropical circulation model—

Formulation. J. Atmos. Sci., 57, 1741–1766.

Neelin, J. D., C. Chou, and H. Su, 2003: Tropical drought regions in global warming

and El Niño teleconnections. Geophys. Res. Lett., 30, 2275.

Neelin, J. D., A. Bracco, H. Luo, J. C. McWilliams, and J. E. Meyerson, 2010:

Considerations for parameter optimization and sensitivity in climate models.

Proc. Nat. Acad. Sci. U.S.A., 107, 21349–21354.

Neggers, R. A. J., 2009: A dual mass ﬂux framework for boundary layer convection.

Part II: Clouds. J. Atmos. Sci., 66, 1489–1506.

Neggers, R. A. J., M. Kohler, and A. C. M. Beljaars, 2009: A dual mass ﬂux framework

for boundary layer convection. Part I: Transport. J. Atmos. Sci., 66, 1465–1487.

Nicolsky, D. J., V. E. Romanovsky, V. A. Alexeev, and D. M. Lawrence, 2007: Improved

modeling of permafrost dynamics in a GCM land-surface scheme. J. Geophys.

Res., 34, L08501.

Nikiema, O., and R. Laprise, 2010: Diagnostic budget study of the internal variability

in ensemble simulations of the Canadian RCM. Clim. Dyn., 36 2313–2337.

Nikulin, G., et al., 2012: Precipitation climatology in an ensemble of CORDEX-Africa

regional climate simulations. J. Clim., doi:10.1175/jcli-d-11–00375.1.

Ning, L., M. E. Mann, R. Crane, and T. Wagener, 2011: Probabilistic projections of

climate change for the Mid-Atlantic region of the United States—Validation of

precipitation downscaling during the Historical Era. J. Clim., 25, 509–526.

Nishii, K., et al., 2012: Relationship of the reproducibility of multiple variables among

Global Climate Models. J. Meteorol. Soc. Jpn., 90A, 87–100.

Notz, D., F. A. Haumann, H. Haak, J. H. Jungclaus, and J. Marotzke, 2013: Sea-ice

evolution in the Arctic as modeled by MPI-ESM. J. Adv. Model. Earth Syst.,

doi:10.1002/jame.20016.

O’Connor, F. M., C. E. Johnson, O. Morgenstern, and W. J. Collins, 2009: Interactions

between tropospheric chemistry and climate model temperature and humidity

biases. Geophys. Res. Lett., 36, L16801.

O’Farrell, S. P., 1998: Investigation of the dynamic sea ice component of a coupled

atmosphere sea ice general circulation model. J. Geophys. Res.-Oceans, 103,

15751–15782.

O’Gorman, P. A., 2012: Sensitivity of tropical precipitation extremes to climate

change. Nature Geosci., 5, 697–700.

O’Gorman, P. A., and M. S. Singh, 2013: Vertical structure of warming consistent

with an upward shift in the middle and upper troposphere. Geophys. Res. Lett.,

doi:10.1002/grl.50328.

O’ishi, R., and A. Abe-Ouchi, 2011: Polar ampliﬁcation in the mid-Holocene derived

from dynamical vegetation change with a GCM. Geophys. Res. Lett., 38, L14702.

Ogasawara, N., A. Kitoh, T. Yasunari, and A. Noda, 1999: Tropospheric biennial

oscillation of ENSO-monsoon system in the MRI coupled GCM. J. Meteorol. Soc.

Jpn., 77, 1247–1270.

Ohba, M., D. Nohara, and H. Ueda, 2010: Simulation of asymmetric ENSO transition

in WCRP CMIP3 Multimodel Experiments. J. Clim., 23, 6051–6067.

Ohgaito, R., and A. Abe-Ouchi, 2009: The effect of sea surface temperature bias in

the PMIP2 AOGCMs on mid-Holocene Asian monsoon enhancement. Clim. Dyn.,

33, 975–983.

Oka, A., E. Tajika, A. Abe-Ouchi, and K. Kubota, 2011: Role of the ocean in controlling

atmospheric CO

concentration in the course of global glaciations. Clim. Dyn.,

37, 1755–1770.

Oleson, K. W., 2004: Technical description of the Community Land Model (CLM).

NCAR Technical Note NCAR/TN-461+STR, National Center for Atmospheric

Research, Boulder, CO, 174 pp.

Oleson, K. W., G. B. Bonan, J. Feddema, M. Vertenstein, and C. S. B. Grimmond, 2008a:

An urban parameterization for a global climate model. Part I: Formulation and

evaluation for two cities. J. Appl. Meteorol. Climatol., 47, 1038–1060.

Oleson, K. W., et al., 2010: Technical Description of version 4.0 ofthe Community

Land Model (CLM)NCAR Technical Note NCAR/TN-478+STR, National Center

for Atmospheric Research, Boulder, CO, 257 pp.

Oleson, K. W., et al., 2008b: Improvements to the Community Land Model and their

impact on the hydrological cycle. J. Geophys. Res. Biogeosci., 113, G01021

Onogi, K., et al., 2007: The JRA-25 reanalysis. J. Meteorol. Soc. Jpn., 85, 369–432.

Opsteegh, J. D., R. J. Haarsma, F. M. Selten, and A. Kattenberg, 1998: ECBILT: A

dynamic alternative to mixed boundary conditions in ocean models. Tellus A,

50, 348–367.

Oreopoulos, L., et al., 2012: The continual intercomparison of radiation codes:

Results from Phase I. J. Geophys. Res. Atmos., 117, D06118.

Ostle, N. J., et al., 2009: Integrating plant-soil interactions into global carbon cycle

models. J. Ecol., 97, 851–863.

Otte, T. L., C. G. Nolte, M. J. Otte, and J. H. Bowden, 2012: Does nudging squelch the

extremes in Regional Climate Modeling? J. Clim., 25, 7046–7066.

Ottera, O. H., M. Bentsen, H. Drange, and L. Suo, 2010: External forcing as a

metronome for Atlantic multidecadal variability. Nature Geosci., 3, 688–694.

Otto-Bliesner, B. L., et al., 2007: Last Glacial Maximum ocean thermohaline

circulation: PMIP2 model intercomparisons and data constraints. Geophys. Res.

Lett., 34, L12706.

Otto-Bliesner, B. L., et al., 2009: A comparison of PMIP2 model simulations and

the MARGO proxy reconstruction for tropical sea surface temperatures at last

glacial maximum. Clim. Dyn., 32, 799–815.

844

Chapter 9 Evaluation of Climate Models

Otto, J., T. Raddatz, M. Claussen, V. Brovkin, and V. Gayler, 2009: Separation of

atmosphere-ocean-vegetation feedbacks and synergies for mid-Holocene

climate. Global Biogeochem. Cycles, 23, L09701.

Overland, J. E., and M. Wang, 2013: When will the summer Arctic be nearly sea ice

free? Geophys. Res. Lett., doi:10.1002/grl.50316, doi:10.1002/grl.50316.

Ozturk, T., H. Altinsoy, M. Turkes, and M. Kurnaz, 2012: Simulation of temperature

and precipitation climatology for the central Asia CORDEX domain using RegCM

4.0. Clim. Res., 52, 63–76.

Paeth, H., 2011: Postprocessing of simulated precipitation for impact research in

West Africa. Part I: Model output statistics for monthly data. Clim. Dyn., 36,

1321–1336.

Paeth, H., et al., 2012: Progress in regional downscaling of west African precipitation.

Atmos. Sci. Lett., 12, 75–82.

Palmer, J. R., and I. J. Totterdell, 2001: Production and export in a global ocean

ecosystem model. Deep-Sea R. Pt. I, 48, 1169–1198.

Parekh, P., F. Joos, and S. A. Muller, 2008: A modeling assessment of the interplay

between aeolian iron ﬂuxes and iron-binding ligands in controlling carbon

dioxide ﬂuctuations during Antarctic warm events. Paleoceanography, 23,

Pa4202.

Park, S., and C. S. Bretherton, 2009: The University of Washington Shallow Convection

and Moist Turbulence schemes and their impact on climate simulations with the

Community Atmosphere Model. J. Clim., 22, 3449–3469.

Park, W., and M. Latif, 2010: Paciﬁc and Atlantic multidecadal variability in the Kiel

Climate Model. Geophys. Res. Lett., 37, L24702.

Parker, D., C. Folland, A. Scaife, J. Knight, A. Colman, P. Baines, and B. W. Dong, 2007:

Decadal to multidecadal variability and the climate change background. J.

Geophys. Res. Atmos., 112, D18115.

Parkinson, C. L., and D. J. Cavalieri, 2012: Antarctic sea ice variability and trends,

1979–2010. Cryosphere, 6, 871–880.

Patricola, C. M., M. Li, Z. Xu, P. Chang, R. Saravanan, and J.-S. Hsieh, 2012: An

investigation of tropical Atlantic bias in a high-resolution Coupled Regional

Climate Model. Clim. Dyn., doi:10.1007/s00382-012-1320-5.

Pavlova, T. V., V. M. Kattsov, and V. A. Govorkova, 2011: Sea ice in CMIP5 models:

Closer to reality? Trudy GGO (MGO Proc., in Russian), 564, 7–18.

Pavlova, T. V., V. M. Kattsov, Y. D. Nadyozhina, P. V. Sporyshev, and V. A. Govorkova,

2007: Terrestrial cryosphere evolution through the 20th and 21st centuries as

simulated with the new generation of global climate models. Earth Cryosphere

(in Russian), 11, 3–13.

Pechony, O., and D. T. Shindell, 2009: Fire parameterization on a global scale. J.

Geophys. Res. Atmos., 114, D16115

Pedersen, C. A., E. Roeckner, M. Lüthje, and J. Winther, 2009: A new sea ice albedo

scheme including melt ponds for ECHAM5 general circulation model. J. Geophys.

Res., 114, D08101.

Pennell, C., and T. Reichler, 2011: On the effective number of climate models. J. Clim.,

24 2358–2367

Perlwitz, J., S. Pawson, R. Fogt, J. Nielsen, and W. Neff, 2008: Impact of stratospheric

ozone hole recovery on Antarctic climate. Geophys. Res. Lett., 35, L08714.

Peterson, T. C., et al., 2009: State of the Climate in 2008. Bull. Am. Meteorol. Soc.,

90, S1–S196.

Petoukhov, V., I. I. Mokhov, A. V. Eliseev, and V. A. Semenov, 1998: The IAP RAS global

climate model. Dialogue-MSU, Moscow, Russia.

Petoukhov, V., A. Ganopolski, V. Brovkin, M. Claussen, A. Eliseev, C. Kubatzki, and

S. Rahmstorf, 2000: CLIMBER-2: a climate system model of intermediate

complexity. Part I: Model description and performance for present climate. Clim.

Dyn., 16, 1–17.

Petoukhov, V., et al., 2005: EMIC Intercomparison Project (EMIP-CO2): Comparative

analysis of EMIC simulations of climate, and of equilibrium and transient

responses to atmospheric CO

doubling. Clim. Dyn., 25, 363–385.

Pfahl, S., and H. Wernli, 2012: Quantifying the relevance of atmospheric blocking

for co-located temperature extremes in the Northern Hemisphere on (sub-)daily

time scales. Geophys. Res. Lett., 39, L12807.

Pfeiffer, A., and G. Zängl, 2010: Validation of climate-mode MM5–simulations for the

European Alpine Region. Theor. Appl. Climatol., 101, 93–108.

Phillips, T. J., et al., 2004: Evaluating parameterizations in General Circulation

Models: Climate simulation meets weather prediction. Bull. Am. Meteorol. Soc.,

85, 1903–1915.

Piani, C., D. J. Frame, D. A. Stainforth, and M. R. Allen, 2005: Constraints on climate

change from a multi-thousand member ensemble of simulations. Geophys. Res.

Lett., 32, L23825.

Pierce, D. W., 2001: Distinguishing coupled ocean-atmosphere interactions from

background noise in the North Paciﬁc. Prog. Oceanogr., 49, 331–352.

Pierce, D. W., T. P. Barnett, B. D. Santer, and P. J. Gleckler, 2009: Selecting global

climate models for regional climate change studies. Proc. Natl. Acad. Sci. U.S.A.,

106, 8441–8446.

Pincus, R., C. P. Batstone, R. J. P. Hofmann, K. E. Taylor, and P. J. Glecker, 2008:

Evaluating the present-day simulation of clouds, precipitation, and radiation in

climate models. J. Geophys. Res. Atmos., 113, D14209

Pincus, R., S. Platnick, S. A. Ackerman, R. S. Hemler, and R. J. P. Hofmann, 2012:

Reconciling simulated and observed views of clouds: MODIS, ISCCP, and the

limits of instrument simulators. J. Clim., 25, 4699–4720.

Pinto, J. G., T. Spangehl, U. Ulbrich, and P. Speth, 2006: Assessment of winter cyclone

activity in a transient ECHAM4–OPYC3 GHG experiment. Meteorol. Z., 15,

279–291.

Piot, M., and R. von Glasow, 2008: The potential importance of frost ﬂowers,

recycling on snow, and open leads for ozone depletion events. Atmos. Chem.

Phys., 8, 2437–2467.

Pitman, A., A. Arneth, and L. Ganzeveld, 2012a: Regionalizing global climate models.

Int. J. Climatol. , 32, 321–337.

Pitman, A. J., 2003: The evolution of, and revolution in, land surface schemes

designed for climate models. Int. J. Climatol. , 23, 479–510.

Pitman, A. J., et al., 2012b: Effects of land cover change on temperature and rainfall

extremes in multi-model ensemble simulations. Earth Syst. Dyn., 13, 213–231.

Pitman, A. J., et al., 2009: Uncertainties in climate responses to past land cover

change: First results from the LUCID intercomparison study. Geophys. Res. Lett.,

36, L14814.

Plattner, G. K., et al., 2008: Long-term climate commitments projected with climate-

carbon cycle models. J. Clim., 21, 2721–2751.

Ploshay, J. J., and N.-C. Lau, 2010: Simulation of the diurnal cycle in tropical rainfall

and circulation during Boreal Summer with a high-resolution GCM. Mon.

Weather Rev., 138, 3434–3453.

Po-Chedley, S., and Q. Fu, 2012: Discrepancies in tropical upper tropospheric

warming between atmospheric circulation models and satellites. Environ. Res.

Lett., 7, 044018.

Pokhrel, S., H. Rahaman, A. Parekh, S. K. Saha, A. Dhakate, H. S. Chaudhari, and R.

M. Gairola, 2012: Evaporation-precipitation variability over Indian Ocean and

its assessment in NCEP Climate Forecast System (CFSv2). Clim. Dyn., 39, 2585–

2608.

Polvani, L., D. Waugh, G. Correa, and S. Son, 2011: Stratospheric ozone depletion:

The main driver of twentieth-century atmospheric circulation changes in the

Southern Hemisphere. J. Clim., 24, 795–812.

Pope, V. D., M. L. Gallani, P. R. Rowntree, and R. A. Stratton, 2000: The impact of new

physical parametrizations in the Hadley Centre climate model: HadAM3. Clim.

Dyn., 16, 123–146.

Power, S., and R. Colman, 2006: Multi-year predictability in a coupled general

circulation model. Clim. Dyn., 26, 247–272

Power, S., M. Haylock, R. Colman, and X. D. Wang, 2006: The predictability of

interdecadal changes in ENSO activity and ENSO teleconnections. J. Clim., 19,

4755–4771.

Prudhomme, C., and H. Davies, 2009: Assessing uncertainties in climate change

impact analyses on the river ﬂow regimes in the UK. Part 1: Baseline climate.

Clim. Change, 93, 177–195.

Pryor, S., G. Nikulin, and C. Jones, 2012: Inﬂuence of spatial resolution on regional

climate model derived wind climates. J. Geophys. Res. Atmos., 117, D03117.

Ptashnik, I. V., R. A. McPheat, K. P. Shine, K. M. Smith, and R. G. Williams, 2011:

Water vapor self-continuum absorption in near-infrared windows derived from

laboratory measurements. J. Geophys. Res. Atmos., 116, D16305.

Qian, T. T., A. Dai, K. E. Trenberth, and K. W. Oleson, 2006: Simulation of global land

surface conditions from 1948 to 2004. Part I: Forcing data and evaluations. J.

Hydrometeorol., 7, 953–975.

Qiao, F., Y. Yuan, Y. Yang, Q. Zheng, C. Xia, and J. Ma, 2004: Wave-induced mixing

in the upper ocean: Distribution and application to a global ocean circulation

model. Geophys. Res. Lett., 31, L11303.

Quaas, J., 2012: Evaluating the “critical relative humidity” as a measure of

subgrid-scale variability of humidity in general circulation model cloud cover

parameterizations using satellite data. J. Geophys. Res. Atmos., 117, D09208.

Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian

model averaging to calibrate forecast ensembles. Mon. Weather Rev., 133,

1155–1174.

845

Evaluation of Climate Models Chapter 9

Raible, C. C., M. Yoshimori, T. F. Stocker, and C. Casty, 2007: Extreme midlatitude

cyclones and their implications for precipitation and wind speed extremes in

simulations of the Maunder Minimum versus present day conditions. Clim. Dyn.,

28, 409–423.

Raisanen, J., 2007: How reliable are climate models? Tellus A, 59, 2–29.

Raisanen, J., and J. S. Ylhaisi, 2011: How much should climate model output be

smoothed in space? J. Clim., 24, 867–880.

Raisanen, J., L. Ruokolainen, and J. Ylhaisi, 2010: Weighting of model results for

improving best estimates of climate change. Clim. Dyn., 35, 407–422.

Rammig, A., et al., 2010: Estimating the risk of Amazonian forest dieback. New

Phytologist, 187, 694–706.

Rampal, P., J. Weiss, C. Dubois, and J. M. Campin, 2011: IPCC climate models do not

capture Arctic sea ice drift acceleration: Consequences in terms of projected sea

ice thinning and decline. J. Geophys. Res. Oceans, 116, C00d07.

Ramstein, G., M. Kageyama, J. Guiot, H. Wu, C. Hely, G. Krinner, and S. Brewer, 2007:

How cold was Europe at the Last Glacial Maximum? A synthesis of the progress

achieved since the ﬁrst PMIP model-data comparison. Clim. Past, 3, 331–339.

Randall, D. A., M. F. Khairoutdinov, A. Arakawa, and W. W. Grabowski, 2003: Breaking

the cloud parameterization deadlock. Bull. Am. Meteorol. Soc., 84, 1547–1564.

Randall, D. A., et al., 2007: Climate models and their evaluation. In: Climate Change

2007: The Physical Science Basis. Contribution of Working Group I to the

Fourth Assessment Report of the Intergovernmental Panel on Climate Change

[Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K. B. Averyt, M. Tignor

and H. L. Miller (eds.)] Cambridge University Press, Cambridge, United Kingdom

and New York, NY, USA, pp. 589–662.

Randel, W., and F. Wu, 2007: A stratospheric ozone proﬁle data set for 1979–2005:

Variability, trends, and comparisons with column ozone data. J. Geophys. Res.

Atmos., 12, D06313.

Randel, W. J., et al., 2009: An update of observed stratospheric temperature trends. J.

Geophys. Res. Atmos., 114, D02107.

Rapaić, M., M. Leduc, and R. Laprise, 2010: Evaluation of the internal variability and

estimation of the downscaling ability of the Canadian Regional Climate Model

for different domain sizes over the north Atlantic region using the Big-Brother

experimental approach. Clim. Dyn., 36 1979–2001.

Raphael, M. N., and M. M. Holland, 2006: Twentieth century simulation of the

Southern Hemisphere climate in coupled models. Part 1: Large scale circulation

variability. Clim. Dyn., 26, 217–228.

Rashid, H. A., A. C. Hirst, and M. Dix, 2013: Atmospheric circulation features in

the ACCESS model simulations for CMIP5: Historical simulation and future

projections Aust. Meteorol. Oceanogr. J., 63, 145–160.

Rauscher, S. A., E. Coppola, C. Piani, and F. Giorgi, 2010: Resolution effects on

regional climate model simulations of seasonal precipitation over Europe. Clim.

Dyn., 35, 685–711.

Rayner, N. A., et al., 2003: Global analysis of sea surface temperature, sea ice, and

night marine air temperature since the late ninteeth century. J. Geophys. Res.,

108, 4407.

Redelsperger, J.-L., C. D. Thorncroft, A. Diedhiou, T. Lebel, D. J. Parker, and J. Polcher,

2006: African Monsoon Multidisciplinary Analysis: An international research

project and ﬁeld campaign. Bull. Am. Meteorol. Soc., 87, 1739–1746.

Redi, M. H., 1982: Oceanic isopycnal mixing by coordinate rotation. J. Phys.

Oceanogr., 12, 1154–1158.

Reichler, T., and J. Kim, 2008: How well do coupled models simulate today’s climate?

Bull. Am. Meteorol. Soc., 89, 303–311.

Reick, C. H., T. Raddatz, V. Brovkin, and V. Gayler, 2013: The representation of natural

and anthropogenic land coverchange in MPI-ESM.J. Adv. Model. Earth Syst.,

doi:10.1002/jame.20022.

Reifen, C., and R. Toumi, 2009: Climate projections: Past performance no guarantee

of future skill? Geophys. Res. Lett., 36, L13704.

Remer, L. A., et al., 2008: Global aerosol climatology from the MODIS satellite

sensors. J. Geophys. Res. Atmos., 113, D14s07.

Richter, I., and S.-P. Xie, 2008: On the origin of equatorial Atlantic biases in coupled

general circulation models. Clim. Dyn., 31, 587–598.

Richter, I., S.-P. Xie, S. K. Behera, T. Doi, and Y. Masumoto, 2013: Equatorial

Atlantic variability and its relation to mean state biases in CMIP5. Clim. Dyn.,

doi:10.1007/s00382-012-1624-5.

Richter, J. H., F. Sassi, and R. R. Garcia, 2010: Toward a physically based gravity

wave source parameterization in a General Circulation Model. J. Atmos. Sci., 67,

136–156.

Ridgwell, A., and J. C. Hargreaves, 2007: Regulation of atmospheric CO(2) by deep-

sea sediments in an Earth system model. Global Biogeochem. Cycles, 21,

Gb2008.

Ridgwell, A., I. Zondervan, J. C. Hargreaves, J. Bijma, and T. M. Lenton, 2007a:

Assessing the potential long-term increase of oceanic fossil fuel CO

uptake due

to CO

–calciﬁcation feedback. Biogeosciences, 4, 481–492.

Ridgwell, A., et al., 2007b: Marine geochemical data assimilation in an efﬁcient Earth

System Model of global biogeochemical cycling. Biogeosciences, 4, 87–104.

Rienecker, M. M., et al., 2011: MERRA: NASA’s modern-era retrospective analysis for

research and applications. J. Clim., 24, 3624–3648.

Ringer, M. A., J. M. Edwards, and A. Slingo, 2003: Simulation of satellite channel

radiances in the Met Ofﬁce Uniﬁed Model. Q. J. R. Meteorol. Soc., 129, 1169–

1190.

Rio, C., and F. Hourdin, 2008: A thermal plume model for the convective boundary

layer: Representation of cumulus clouds. J. Atmos. Sci., 65, 407–425.

Rio, C., F. Hourdin, F. Couvreux, and A. Jam, 2010: Resolved versus parametrized

boundary-layer plumes. Part II: Continuous formulations of mixing rates for

mass-ﬂux schemes. Boundary-Layer Meteorol., 135, 469–483.

Risi, C., et al., 2012a: Process-evaluation of tropospheric humidity simulated by

general circulation models using water vapor isotopic observations: 2. Using

isotopic diagnostics to understand the mid and upper tropospheric moist bias in

the tropics and subtropics. J. Geophys. Res. Atmos., 117, D05304.

Risi, C., et al., 2012b: Process-evaluation of tropospheric humidity simulated by

general circulation models using water vapor isotopologues: 1. Comparison

between models and observations. J. Geophys. Res. Atmos., 117, D05303.

Risien, C. M., and D. B. Chelton, 2008: A global climatology of surface wind and

wind stress ﬁelds from eight years of QuikSCAT Scatterometer data. J. Phys.

Oceanogr., 38, 2379–2413.

Ritz, S. P., T. F. Stocker, and F. Joos, 2011: A coupled dynamical ocean-energy balance

atmosphere model for paleoclimate studies. J. Clim., 24, 349–375.

Roberts, M. J., et al., 2004: Impact of an eddy-permitting ocean resolution on control

and climate change simulations with a global coupled GCM. J. Clim., 17, 3–20.

Robinson, A., R. Calov, and A. Ganopolski, 2012: Multistability and critical thresholds

of the Greenland ice sheet. Nature Clim. Change, 2, 429–432.

Robinson, D. A., and A. Frei, 2000: Seasonal variability of northern hemisphere snow

extent using visible satellite data. Prof. Geograph., 51, 307–314.

Rockel, B., C. L. Castro, R. A. Pielke, H. von Storch, and G. Leoncini, 2008: Dynamical

downscaling: Assessment of model system dependent retained and added

variability for two different regional climate models. J. Geophys. Res. Atmos.,

113, D21107.

Rodwell, M., and T. Palmer, 2007: Using numerical weather prediction to assess

climate models. Q. J. R. Meteorol. Soc., 133, 129–146.

Roe, G. H., and M. B. Baker, 2007: Why is climate sensitivity so unpredictable?

Science, 318, 629–632.

Roe, G. H., and K. C. Armour, 2011: How sensitive is climate sensitivity? Geophys.

Res. Lett., 38, L14708.

Roe, G. H., and M. B. Baker, 2011: Comment on “Another look at climate sensitivity”

by Zaliapin and Ghil (2010). Nonlin.Proc. Geophys., 18, 125–127.

Roeckner, E., et al., 2006: Sensitivity of simulated climate to horizontal and vertical

resolution in the ECHAM5 atmosphere model. J. Clim., 19, 3771–3791.

Rojas, M., 2006: Multiply nested regional climate simulation for southern South

America: Sensitivity to model resolution. Mon. Weather Rev., 134, 2208–2223.

Rojas, M., and P. I. Moreno, 2011: Atmospheric circulation changes and neoglacial

conditions in the Southern Hemisphere mid-latitudes: Insights from PMIP2

simulations at 6 kyr. Clim. Dyn., 37, 357–375.

Rojas, M., et al., 2009: The Southern Westerlies during the last glacial maximum in

PMIP2 simulations. Clim. Dyn., 32, 525–548.

Romanou, A., et al., 2013: Natural air–sea ﬂux of CO

in simulations of the NASA-

GISS climate model: Sensitivity to the physical ocean model formulation. Ocean

Model., doi:10.1016/j.ocemod.2013.01.008.

Rotstayn, L. D., and U. Lohmann, 2002: Simulation of the tropospheric sulfur cycle

in a global model with a physically based cloud scheme. J. Geophys. Res., 107,

4592.

Rotstayn, L. D., M. A. Collier, R. M. Mitchell, Y. Qin, S. K. Campbell, and S. M. Dravitzki,

2011: Simulated enhancement of ENSO-related rainfall variability due to

Australian dust. Atmos. Chem. Phys., 11, 6575–6592.

846

Chapter 9 Evaluation of Climate Models

Rotstayn, L. D., S. J. Jeffrey, M. A. Collier, S. M. Dravitzki, A. C. Hirst, J. I. Syktus, and

K. K. Wong, 2012: Aerosol- and greenhouse gas-induced changes in summer

rainfall and circulation in the Australasian region: A study using single-forcing

climate simulations. Atmos. Chem. Phys., 12, 6377–6404.

Rotstayn, L. D., et al., 2010: Improved simulation of Australian climate and ENSO-

related rainfall variability in a global climate model with an interactive aerosol

treatment. Int. J. Climatol. , 30, 1067–1088.

Rougier, J., D. M. H. Sexton, J. M. Murphy, and D. Stainforth, 2009: Analyzing the

climate sensitivity of the HadSM3 climate model using ensembles from different

but related experiments. J. Clim., 22, 3540–3557.

Roy, P., P. Gachon, and R. Laprise, 2012: Assessment of summer extremes and climate

variability over the north-east of North America as simulated by the Canadian

Regional Climate Model. Int. J. Climatol. , 32 1615–1627.

Ruckstuhl, C., and J. R. Norris, 2009: How do aerosol histories affect solar “dimming”

and “brightening” over Europe?: IPCC-AR4 models versus observations. J.

Geophys. Res. Atmos., 114, D00d04.

Rummukainen, M., 2010: State-of-the-art with regional climate models. Clim.

Change, 1, 82–96.

Russell, J. L., R. J. Stouffer, and K. W. Dixon, 2006: Intercomparison of the Southern

Ocean circulations in IPCC coupled model control simulations. J. Clim., 19,

4560–4575.

Ruti, P. M., et al., 2011: The West African climate system: A review of the AMMA

model inter-comparison initiatives. Atmos. Sci. Lett., 12 116–122

Rutter, N., et al., 2009: Evaluation of forest snow processes models (SnowMIP2). J.

Geophys. Res. Atmos., 114, D06111.

Sabine, C. L., et al., 2004: The oceanic sink for anthropogenic CO

. Science, 305,

367–371.

Saha, S., et al., 2010: The NCEP Climate Forecast System Reanalysis. Bull. Am.

Meteorol. Soc., 91, 1015–105.

Sahany, S., J. D. Neelin, K. Hales, and R. B. Neale, 2012: Temperature–moisture

dependence of the Deep Convective Transition as a constraint on entrainment in

climate models. J. Atmos. Sci., 69, 1340–1358.

Saji, N. H., S. P. Xie, and T. Yamagata, 2006: Tropical Indian Ocean variability in the

IPCC twentieth-century climate simulations. J. Clim., 19, 4397–4417.

Saji, N. H., B. N. Goswami, P. N. Vinayachandran, and T. Yamagata, 1999: A dipole

mode in the tropical Indian Ocean. Nature, 401, 360–363.

Sakaguchi, K., X. B. Zeng, and M. A. Brunke, 2012: The hindcast skill of the CMIP

ensembles for the surface air temperature trend. J. Geophys. Res. Atmos., 117,

D16113.

Sakamoto, T. T., et al., 2012: MIROC4h – a new high-resolution atmosphere-ocean

coupled general circulation model. J. Meteorol. Soc. Jpn., 90, 325–359.

Salas-Melia, D., 2002: A global coupled sea ice-ocean model. Ocean Model., 4,

137–172

Salle, J. B., E. Shuckburgh, N. Bruneau, A. J. S. Meijers, T. J. Bracegirdle, Z. Wang,

and T. Roy, 2013: Assessment of Southern Ocean water mass circulation and

characteristics in CMIP5 models: Historical bias and forcing response. J. Geophys.

Res. Oceans, doi:10.1002/jgrc.20135.

Samuelsson, P., E. Kourzeneva, and D. Mironov, 2010: The impact of lakes on the

European climate as simulated by a regional climate model. Boreal Environ. Res.,

15, 113–129.

Sander, S. P., 2006: Chemical Kinetics and Photochemical Data for Use in Atmospheric

Studies.Evaluation 15. JPL Publications, Pasadena, CA, USA, 523 pp.

Sanderson, B. M., 2011: A multimodel study of parametric uncertainty in predictions

of climate response to rising greenhouse gas concentrations. J. Clim., 25, 1362–

1377.

Sanderson, B. M., 2013: On the estimation of systematic error in regression-

basedpredictions of climate sensitivity. Clim. Change, doi:10.1007/s10584-012-

0671-6.

Sanderson, B. M., and R. Knutti, 2012: On the interpretation of constrained climate

model ensembles. Geophys. Res. Lett., 39, L16708.

Sanderson, B. M., K. M. Shell, and W. Ingram, 2010: Climate feedbacks determined

using radiative kernels in a multi-thousand member ensemble of AOGCMs. Clim.

Dyn., 35, 1219–1236.

Sanderson, B. M., C. Piani, W. J. Ingram, D. A. Stone, and M. R. Allen, 2008a: Towards

constraining climate sensitivity by linear analysis of feedback patterns in

thousands of perturbed-physics GCM simulations. Clim. Dyn., 30, 175–190.

Sanderson, B. M., et al., 2008b: Constraints on model response to greenhouse gas

forcing and the role of subgrid-scale processes. J. Clim., 21, 2384–2400.

Sansom, P. G., D. B. Stephenson, C. A. T. Ferro, G. Zappa, and L. Shaffrey, 2013: Simple

uncertainty frameworks for selecting weighting schemes and interpreting multi-

model ensemble climate change experiments doi:10.1175/JCLI-D-12-00462.1.

Santer, B., et al., 2009: Incorporating model quality information in climate change

detection and attribution studies. Proc. Natl. Acad. Sci. U.S.A., 106, 14778–

14783.

Santer, B., et al., 2007: Identiﬁcation of human-induced changes in atmospheric

moisture content. Proc. Natl. Acad. Sci. U.S.A., 104, 15248–15253.

Santer, B., et al., 2008: Consistency of modelled and observed temperature trends in

the tropical troposphere. Int. J. Climatol. , 28, 1703–1722.

Santer, B., et al., 2005: Ampliﬁcation of surface temperature trends and variability in

the tropical atmosphere. Science, 309, 1551–1556.

Santer, B. D., et al., 2013: Identifying human inﬂuences on atmospheric temperature.

Proc. Natl. Acad. Sci. U.S.A, 110, 26–33.

Sato, M., J. E. Hansen, M. P. McCormick, and J. B. Pollack, 1993: Stratospheric aerosol

optical depth, 1850– 1990. J. Geophys. Res. Atmos., 98, 22987–22994

Sato, T., H. Miura, M. Satoh, Y. N. Takayabu, and Y. Q. Wang, 2009: Diurnal cycle of

precipitation in the Tropics simulated in a global cloud-resolving model. J. Clim.,

22, 4809–4826.

Scaife, A. A., N. Butchart, C. D. Warner, and R. Swinbank, 2002: Impact of a spectral

gravity wave parameterization on the stratosphere in the met ofﬁce uniﬁed

model. J. Atmos. Sci., 59, 1473–1489.

Scaife, A. A., T. Woollings, J. Knight, G. Martin, and T. Hinton, 2010: Atmospheric

blocking and mean biases in climate models. J. Clim., 23, 6143–6152.

Scaife, A. A., et al., 2011: Improved Atlantic winter blocking in a climate model.

Geophys. Res. Lett., 38, L23703.

Scaife, A. A., et al., 2012: Climate change and stratosphere-troposphere interaction.

Clim. Dyn., 38, 2089–2097.

Scaife, A. A., et al., 2009: The CLIVAR C20C project: Selected twentieth century

climate events. Clim. Dyn., 33, 603–614.

Schaller, N., I. Mahlstein, J. Cermak, and R. Knutti, 2011: Analyzing precipitation

projections: A comparison of different approaches to climate model evaluation.

J. Geophys. Res. Atmos., 116, D10118.

Scherrer, S., 2011: Present-day interannual variability of surface climate in CMIP3

models and its relation to future warming. Int. J. Climatol. , 31, 1518–1529.

Schlesinger, M. E., and J. F. B. Mitchell, 1987: Climate model simulations of the

equilibrium climatic response to increased carbon-dioxide. Rev. Geophys., 25,

760–798.

Schmidli, J., C. Goodess, C. Frei, M. Haylock, Y. Hundecha, J. Ribalaygua, and T.

Schmith, 2007: Statistical and dynamical downscaling of precipitation: An

evaluation and comparison of scenarios for the European Alps. J. Geophys. Res.

Atmos., 112, D04105.

Schmidt, G. A., et al., 2012: Climate forcing reconstructions for use in PMIP

simulations of the Last Millennium (v1.1). Geophys. Model Dev., 5, 185–191.

Schmidt, G. A., et al., 2006: Present day atmospheric simulations using GISS ModelE:

Comparison to in-situ, satellite and reanalysis data. J. Clim., 19, 153–192.

Schmith, T., 2008: Stationarity of regression relationships: Application to empirical

downscaling. J. Clim., 21, 4529–4537.

Schmittner, A., A. Oschlies, X. Giraud, M. Eby, and H. L. Simmons, 2005: A global

model of the marine ecosystem for long-term simulations: Sensitivity to ocean

mixing, buoyancy forcing, particle sinking, and dissolved organic matter cycling.

Global Biogeochem. Cycles, 19, Gb3004.

Schott, F. A., S.-P. Xie, and J. P. McCreary, Jr., 2009: Indian Ocean circulation and

climate variability. Rev. Geophys., 47, RG1002.

Schramm, J. L., M. M. Holland, J. A. Curry, and E. E. Ebert, 1997: Modeling the

thermodynamics of a sea ice thickness 1. Sensitivity to ice thickness resolution.

J. Geophys. Res., 102, 23079–23091.

Schultz, M. G., et al., 2008: Global wildland ﬁre emissions from 1960 to 2000. Global

Biogeochem. Cycles, 22, GB2002.

Schurgers, G., U. Mikolajewicz, M. Groger, E. Maier-Reimer, M. Vizcaino, and A.

Winguth, 2008: Long-term effects of biogeophysical and biogeochemical

interactions between terrestrial biosphere and climate under anthropogenic

climate change. Global Planet. Change, 64, 26–37.

Scoccimarro, E., et al., 2011: Effects of tropical cyclones on ocean heat transport in

a high resolution Coupled General Circulation Model. J. Clim., 24, 4368–4384.

Séférian, R., et al., 2013: Skill assessment of three earth system models with common

marine biogeochemistry. Clim. Dyn., 40, 2549–2573.

847

Evaluation of Climate Models Chapter 9

Segui, P., A. Ribes, E. Martin, F. Habets, and J. Boe, 2010: Comparison of three

downscaling methods in simulating the impact of climate change on the

hydrology of Mediterranean basins. J. Hydrol., 383, 111–124.

Seidel, D. J., M. Free, and J. S. Wang, 2012: Reexamining the warming in the tropical

upper troposphere: Models versus radiosonde observations. Geophys. Res. Lett.,

39, L22701.

Seidel, D. J., Q. Fu, W. J. Randel, and T. J. Reichler, 2008: Widening of the tropical belt

in a changing climate. Nature Geosci., 1, 21–24.

Seidel, D. J., N. P. Gillett, J. R. Lanzante, K. P. Shine, and P. W. Thorne, 2011: Stratospheric

temperature trends: Our evolving understanding. Clim. Change, 2, 592–616.

Selten, F. M., G. W. Branstator, H. A. Dijkstra, and M. Kliphuis, 2004: Tropical origins

for recent and future Northern Hemisphere climate change. Geophys. Res. Lett.,

31, L21205.

Semenov, V. A., M. Latif, J. H. Jungclaus, and W. Park, 2008: Is the observed NAO

variability during the instrumental record unusual? Geophys. Res. Lett., 35,

L11701.

Seneviratne, S., et al., 2012: Changes in climate extremes and their impacts

on the natural physical environment. In: IPCC WGI/WGII Special Report on

Managing the Risks of Extreme Events and Disasters to Advance Climate

Change Adaptation (SREX), [Field, C.B., V. Barros, T.F. Stocker, D. Qin, D.J. Dokken,

K.L. Ebi, M.D. Mastrandrea, K.J. Mach, G.-K. Plattner, S.K. Allen, M. Tignor, and

P.M. Midgley (Eds.)]. Cambridge University Press, The Edinburgh Building,

Shaftesbury Road, Cambridge CB2 8RU ENGLAND, pp. 109–230.

Seneviratne, S. I., D. Luethi, M. Litschi, and C. Schaer, 2006: Land-atmosphere

coupling and climate change in Europe. Nature, 443, 205–209.

Seneviratne, S. I., et al., 2010: Investigating soil moisture-climate interactions in a

changing climate: A review. Earth Sci. Rev., 99, 125–161.

Separovic, L., R. De Elia, and R. Laprise, 2008: Reproducible and irreproducible

components in ensemble simulations with a Regional Climate Model. Mon.

Weather Rev., 136, 4942–4961.

Separovic, L., R. Elía, and R. Laprise, 2012: Impact of spectral nudging and domain

size in studies of RCM response to parameter modiﬁcation. Clim. Dyn., 38,

1325–1343.

Severijns, C. A., and W. Hazeleger, 2010: The efﬁcient global primitive equation

climate model SPEEDO V2.0. Geophys. Model Dev., 3, 105–122.

Sexton, D. M. H., and J. M. Murphy, 2012: Multivariate probabilistic projections using

imperfect climate models. Part II: robustness of methodological choices and

consequences for climate sensitivity. Clim. Dyn., 38, 2543–2558.

Sexton, D. M. H., J. M. Murphy, M. Collins, and M. J. Webb, 2012: Multivariate

probabilistic projections using imperfect climate models part I: Outline of

methodology. Clim. Dyn., 38, 2513–2542.

Shaffer, G., and J. L. Sarmiento, 1995: Biogeochemical cycling in the global ocean.

1. A new, analytical model with continuous vertical resolution and high-latitude

dynamics. J. Geophys. Res. Oceans, 100, 2659–2672.

Shaffer, G., S. M. Olsen, and J. O. P. Pedersen, 2008: Presentation, calibration and

validation of the low-order, DCESS Earth System Model (Version 1). Geophys.

Model Dev., 1, 17–51.

Shaffrey, L. C., et al., 2009: UK HiGEM: The new UK High-Resolution Global

Environment Model—Model description and basic evaluation. J. Clim., 22,

1861–1896.

Shefﬁeld, J., and E. F. Wood, 2008: Projected changes in drought occurrence under

future global warming from multi-model, multi-scenario, IPCC AR4 simulations.

Clim. Dyn., 31, 79–105.

Shell, K. M., J. T. Kiehl, and C. A. Shields, 2008: Using the radiative kernel technique to

calculate climate feedbacks in NCAR’s Community Atmospheric Model. J. Clim.,

21, 2269–2282.

Shevliakova, E., et al., 2009: Carbon cycling under 300 years of land use change:

Importance of the secondary vegetation sink. Global Biogeochem. Cycles, 23,

GB2022

Shi, Y., J. Zhang, J. S. Reid, B. Holben, E. J. Hyer, and C. Curtis, 2011: An analysis of the

collection 5 MODIS over-ocean aerosol optical depth product for its implication

in aerosol assimilation. Atmos. Chem. Phys., 11, 557–565.

Shibata, K., and M. Deushi, 2005: Radiative effect of ozone on the quasi-biennial

oscillation in the equatorial stratosphere. Geophys. Res. Lett., 32, L24802.

Shin, D., J. Kim, and H. Park, 2011: Agreement between monthly precipitation

estimates from TRMM satellite, NCEP reanalysis, and merged gauge-satellite

analysis. J. Geophys. Res. Atmos., 116, D16105.

Shin, S. I., D. Sardeshmukh, and K. Pegion, 2010: Realism of local and remote

feedbacks on tropical sea surface temperatures in climate models. J. Geophys.

Res. Atmos., 115, D21110.

Shindell, D. T., et al., 2013a: Interactive ozone and methane chemistry in GISS-

E2historical and future climate simulationsAtmos. Chem. Phys., 13, 2653–2689.

Shindell, D. T., et al., 2013b: Radiative forcing in the ACCMIP historical and future

climate simulations. Atmos. Chem. Phys., 13, 2939–2974.

Shiogama, H., S. Emori, N. Hanasaki, M. Abe, Y. Masutomi, K. Takahashi, and T.

Nozawa, 2011: Observational constraints indicate risk of drying in the Amazon

basin. Nature Commun., 2, 253.

Shiogama, H., et al., 2012: Perturbed physics ensemble using the MIROC5 coupled

atmosphere–ocean GCM without ﬂux corrections: Experimental design and

results. Clim. Dyn., 39, 3041–3056.

Shiu, C.-J., S. C. Liu, C. Fu, A. Dai, and Y. Sun, 2012: How much do precipitation

extremes change in a warming climate? Geophys. Res. Lett., 39, L17707.

Shkol’nik, I., V. Meleshko, S. Eﬁmov, and E. Stafeeva, 2012: Changes in climate

extremes on the territory of Siberia by the middle of the 21st century: An

ensemble forecast based on the MGO regional climate model. Russ. Meteorol.

Hydrol., 37, 71–84.

Shkolnik, I., V. Meleshko, and V. Kattsov, 2007: The MGO climate model for Siberia.

Russ. Meteorol. Hydrol., 32, 351–359.

Siebesma, A. P., P. M. M. Soares, and J. Teixeira, 2007: A combined eddy-diffusivity

mass-ﬂux approach for the convective boundary layer. J. Atmos. Sci., 64, 1230–

1248.

Sigmond, M., and J. F. Scinocca, 2010: The inﬂuence of the basic state on the Northern

Hemisphere circulation response to climate change. J. Clim., 23, 1434–1446.

Sillmann, J., and M. Croci-Maspoli, 2009: Present and future atmospheric blocking

and its impact on European mean and extreme climate. Geophys. Res. Lett., 36,

L10702.

Sillmann, J., M. Croci-Maspoli, M. Kallache, and R. W. Katz, 2011: Extreme cold winter

temperatures in Europe under the inﬂuence of North Atlantic atmospheric

blocking. J. Clim., 24, 5899–5913.

Sillmann, J., V. V. Kharin, X. Zhang, and F. W. Zwiers, 2013: Climate extreme indices

in the CMIP5 multi-model ensemble. Part 1: Model evaluation in the present

climate. J. Geophys. Res., doi:10.1029/2012JD018390.

Simmons, A. J., K. M. Willett, P. D. Jones, P. W. Thorne, and D. P. Dee, 2010: Low-

frequency variations in surface atmospheric humidity, temperature, and

precipitation: Inferences from reanalyses and monthly gridded observational

data sets. J. Geophys. Res. Atmos., 115, D01110.

Simmons, H. L., S. R. Jayne, L. C. St Laurent, and A. J. Weaver, 2004: Tidally driven

mixing in a numerical model of the ocean general circulation. Ocean Model.,

6, 245–263.

Sitch, S., et al., 2003: Evaluation of ecosystem dynamics, plant geography and

terrestrial carbon cycling in the LPJ dynamic global vegetation model. Global

Change Biol., 9, 161–185.

Sitch, S., et al., 2008: Evaluation of the terrestrial carbon cycle, future plant

geography and climate-carbon cycle feedbacks using ﬁve Dynamic Global

Vegetation Models (DGVMs). Global Change Biol., 14, 2015–2039.

Six, K. D., and E. Maier-Reimer, 1996: Effects of plankton dynamics on seasonal

carbon ﬂuxes in an Ocean General Circulation Model. Global Biogeochem.

Cycles, 10, 559–583.

Slater, A. G., and D. M. Lawrence, 2013: Diagnosing present and future permafrost

from climate models. J. Clim., doi:10.1175/JCLI-D-12-00341.1.

Sloyan, B. M., and I. V. Kamenkovich, 2007: Simulation of Subantarctic Mode and

Antarctic Intermediate Waters in climate models. J. Clim., 20, 5061–5080.

Smirnov, D., and D. J. Vimont, 2011: Variability of the Atlantic Meridional Mode

during the Atlantic Hurricane Season. J. Clim., 24, 1409–1424.

Smith, B., P. Samuelsson, A. Wramneby, and M. Rummukainen, 2011a: A model

of the coupled dynamics of climate, vegetation and terrestrial ecosystem

biogeochemistry for regional applications. Tellus A, 63, 87–106.

Smith, P. C., N. De Noblet-Ducoudre, P. Ciais, P. Peylin, N. Viovy, Y. Meurdesoif, and

A. Bondeau, 2010a: European-wide simulations of croplands using an improved

terrestrial biosphere model: Phenology and productivity. J. Geophys. Res.

Biogeosci., 115, G01014.

Smith, P. C., P. Ciais, P. Peylin, N. De Noblet-Ducoudre, N. Viovy, Y. Meurdesoif, and

A. Bondeau, 2010b: European-wide simulations of croplands using an improved

terrestrial biosphere model: 2. Interannual yields and anomalous CO

ﬂuxes in

2003. J. Geophys. Res. Biogeosci., 115, G04028

848

Chapter 9 Evaluation of Climate Models

Smith, R. D., M. E. Maltrud, F. O. Bryan, and M. W. Hecht, 2000: Numerical simulation

of the North Atlantic Ocean at 1/10 degrees. J. Phys. Oceanogr., 30, 1532–1561.

Smith, R. S., J. M. Gregory, and A. Osprey, 2008: A description of the FAMOUS

(version XDBUA) climate model and control run. Geophys. Model Dev., 1, 53–68.

Smith, S. J., J. van Aardenne, Z. Klimont, R. J. Andres, A. Volke, and S. D. Arias, 2011b:

Anthropogenic sulfur dioxide emissions: 1850–2005. Atmos. Chem. Phys., 11,

1101–1116.

Sobolowski, S., and T. Pavelsky, 2012: Evaluation of present and future North

American Regional Climate Change Assessment Program (NARCCAP) regional

climate simulations over the southeast United States. J. Geophys. Res. Atmos.,

117, D01101.

Soden, B. J., and I. M. Held, 2006: An assessment of climate feedbacks in coupled

ocean-atmosphere models. J. Clim., 19, 3354–3360.

Soden, B. J., I. M. Held, R. Colman, K. M. Shell, J. T. Kiehl, and C. A. Shields, 2008:

Quantifying climate feedbacks using radiative kernels. J. Clim., 21, 3504–3520.

Sokolov, A. P., and P. H. Stone, 1998: A ﬂexible climate model for use in integrated

assessments. Clim. Dyn., 14, 291–303.

Sokolov, A. P., C. E. Forest, and P. H. Stone, 2010: Sensitivity of climate change

projections to uncertainties in the estimates of observed changes in deep-ocean

heat content. Clim. Dyn., 34, 735–745.

Sokolov, A. P., et al., 2009: Probabilistic forecast for twenty-ﬁrst-century climate

based on uncertainties in emissions (without policy) and climate parameters. J.

Clim., 22, 5175–5204.

Sokolov, A. P., et al., 2005: The MIT Integrated Global System Model (IGSM) Version 2:

Model description and baseline evaluation. MIT JP Report 124. MIT, Cambridge,

MA.

Solman, S., and N. Pessacg, 2012: Regional climate simulations over South America:

Sensitivity to model physics and to the treatment of lateral boundary conditions

using the MM5 model. Clim. Dyn., 38, 281–300.

Solomon, S., P. J. Young, and B. Hassler, 2012: Uncertainties in the evolution of

stratospheric ozone and implications for recent temperature changes in the

tropical lower stratosphere. Geophys. Res. Lett., 39 L17706.

Solomon, S., K. H. Rosenlof, R. W. Portmann, J. S. Daniel, S. M. Davis, T. J. Sanford,

and G. K. Plattner, 2010: Contributions of stratospheric water vapor to decadal

changes in the rate of global warming. Science, 327, 1219–1223.

Somot, S., F. Sevault, M. Deque, and M. Crepon, 2008: 21st century climate change

scenario for the Mediterranean using a coupled atmosphere-ocean regional

climate model. Global Planet. Change, 63 112–126.

Son, S., et al., 2008: The impact of stratospheric ozone recovery on the Southern

Hemisphere westerly jet. Science, 320, 1486–1489.

Son, S., et al., 2010: Impact of stratospheric ozone on Southern Hemisphere

circulation change: A multimodel assessment. J. Geophys. Res. Atmos., 115,

D00M07.

Song, Z., F. Qiao, and Y. Song, 2012: Response of the equatorial basin-wide SST to

non-breakingsurface wave-induced mixing in a climate model: An amendment

to tropical bias. J. Geophys. Res., doi:10.1029/2012JC007931.

SPARC-CCMVal, 2010: SPARC Report on the Evaluation of Chemistry-Climate Models

[V. Eyring, T.G. Shepherd, D.W. Waugh (eds.)], SPARC Report No. 5, WCRP-132,

WMO/TD-No. 1526.

Sperber, K., and D. Kim, 2012: Simpliﬁed metrics for the identiﬁcation of the Madden-

Julian oscillation in models. Atmos. Sci. Let., doi:10.1002/asl.378.

Sperber, K., et al., 2012: The Asian summer monsoon: An intercomparison of CMIP-5

vs. CMIP-3 simulations of the late 20th century. Clim. Dyn., doi:10.1007/s00382-

012-1607-6.

Sperber, K. R., 2003: Propagation and the vertical structure of the Madden-Julian

oscillation. Mon. Weather Rev., 131, 3018–3037.

Sperber, K. R., and H. Annamalai, 2008: Coupled model simulations of boreal summer

intraseasonal (30–50 day) variability, Part 1: Systematic errors and caution on

use of metrics. Clim. Dyn., 31, 345–372.

Sperber, K. R., et al., 2010: Monsoon Fact Sheet: CLIVAR Asian-Australian Monsoon

Panel.

Stainforth, D. A., et al., 2005: Uncertainty in predictions of the climate response to

rising levels of greenhouse gases. Nature, 433, 403–406.

Stephens, G. L., and C. D. Kummerow, 2007: The remote sensing of clouds and

precipitation from space: A review. J. Atmos. Sci., 64, 3742–3765.

Stephens, G. L., et al., 2010: Dreary state of precipitation in global models. J. Geophys.

Res., 115, D24211.

Stephens, G. L., et al., 2012: An Update on the Earth’s energy balance in light of new

global observations. Nature Geosci., 5, 691–696.

Stephenson, D. B., M. Collins, J. C. Rougier, and R. E. Chandler, 2012: Statistical

problems in the probabilistic prediction of climate change. Environmetrics, 23,

364–372.

Stephenson, D. B., V. Pavan, M. Collins, M. M. Junge, R. Quadrelli, and C. M. G.

Participating, 2006: North Atlantic Oscillation response to transient greenhouse

gas forcing and the impact on European winter climate: A CMIP2 multi-model

assessment. Clim. Dyn., 27, 401–420.

Stevens, B., and S. E. Schwartz, 2012: Observing and modeling Earth’s energy ﬂows.

Surv. Geophys., 33, 779–816.

Stevens, B., et al., 2012: The atmospheric component of the MPI-M EarthSystem

Model: ECHAM6.J. Adv. Model. Earth Syst., doi:10.1002/jame.20015.

Stevenson, S., 2012: Signiﬁcant changes to ENSO strength and impacts

in the twenty-ﬁrst century: Results from CMIP5. Geophys. Res. Lett.,

doi:10.1029/2012GL052759.

Stevenson, S., B. Fox-Kemper, M. Jochum, R. Neale, C. Deser, and G. Meehl, 2012:

Will there be a signiﬁcant change to El Niño in the twenty-ﬁrst century? J. Clim.,

25, 2129–2145.

Stocker, B. D., K. Strassmann, and F. Joos, 2011: Sensitivity of Holocene atmospheric

and the modern carbon budget to early human land use: Analyses with a

process-based model. Biogeosciences, 8, 69–88.

Stocker, B. D., et al., 2012: Multiple greenhouse gas feedbacks from the land

biosphere under future climate change scenarios. Nature Clim. Change,

doi:10.1038/nclimate1864.

Stolarski, R., and S. Frith, 2006: Search for evidence of trend slow-down in the long-

term TOMS/SBUV total ozone data record: The importance of instrument drift

uncertainty. Atmos. Chem. Phys., 6, 4057–4065.

Stoner, A. M. K., K. Hayhoe, and D. J. Wuebbles, 2009: Assessing General Circulation

Model simulations of atmospheric teleconnection patterns. J. Clim., 22, 4348–

4372.

Stott, P. A., and C. E. Forest, 2007: Ensemble climate predictions using climate models

and observational constraints. Philos. R. Soc. London A, 365, 2029–2052.

Strachan, J., P. L. Vidale, K. Hodges, M. Roberts, and M.-E. Demory, 2013: Investigating

global tropical cyclone activity with a hierarchy of AGCMs: The role of model

resolution. J. Clim., 26, 133–152.

Strassmann, K. M., F. Joos, and G. Fischer, 2008: Simulating effects of land use

changes on carbon ﬂuxes: Past contributions to atmospheric CO

increases

and future commitments due to losses of terrestrial sink capacity. Tellus B, 60,

583–603.

Stratton, R. A., and A. J. Stirling, 2012: Improving the diurnal cycle of convection in

GCMs. Q. J. R. Meteorol. Soc., 138, 1121–1134.

Stroeve, J., M. Holland, W. Meier, T. Scambos, and M. Serreze, 2007: Arctic sea ice

decline: Faster than forecast. Geophys. Res. Lett., 34, L09501.

Stroeve, J. C., V. Kattsov, A. Barrett, M. Serreze, T. Pavlova, M. Holland, and W. N. Meier,

2012: Trends  in Arctic sea ice extent from CMIP5, CMIP3 and observations.

Geophys. Res. Lett., 39, L16502.

Su, H., and J. H. Jiang, 2012: Tropical clouds and circulation changes during the

2006–07 and 2009–10 El Niños. J. Clim., doi:10.1175/JCLI-D-1200152.1.

Su, H., D. E. Waliser, J. H. Jiang, J. L. Li, W. G. Read, J. W. Waters, and A. M. Tompkins,

2006: Relationships of upper tropospheric water vapor, clouds and SST: MLS

observations, ECMWF analyses and GCM simulations. Geophys. Res. Lett., 33,

L22802

Su, H., et al. , 2012: Diagnosis of regime-dependent cloud simulation errors in CMIP5

models using “A-Train” satellite observations and reanalysis data. J. Geophys.

Res., doi:10.1029/2012JD018575.

Sudo, K., M. Takahashi, J. Kurokawa, and H. Akimoto, 2002: CHASER: A global

chemical model of the troposphere - 1. Model description. J. Geophys. Res.

Atmos., 107, 4339.

Suh, M., S. Oh, D. Lee, D. Cha, S. Choi, C. Jin, and S. Hong, 2012: Development of new

ensemble methods based on the performance skills of regional climate models

over South Korea. J. Clim., 25, 7067–7082.

Sun, D.-Z., Y. Yu, and T. Zhang, 2009: Tropical water vapor and cloud feedbacks in

climate models: A further assessment using coupled simulations. J. Clim., 22,

1287–1304.

Sutton, R. T., B. W. Dong, and J. M. Gregory, 2007: Land/sea warming ratio in response

to climate change: IPCC AR4 model results and comparison with observations.

Geophys. Res. Lett., 34, L02701

Svensson, G., and A. Holtslag, 2009: Analysis of model results for the turning of the

wind and related momentum ﬂuxes in the stable boundary layer. Boundary-

Layer Meteorol., 132, 261–277.

849

Evaluation of Climate Models Chapter 9

Svensson, G., et al., 2011: Evaluation of the diurnal cycle in the atmospheric

boundary layer over land as represented by a variety of single-column models:

The Second GABLS Experiment. Boundary-Layer Meteorol., 140, 177–206.

Swart, N. C., and J. C. Fyfe, 2012a: Ocean carbon uptake and storage inﬂuenced by

wind bias in global climate models. Nature Clim. Change, 2, 47–52.

Swart, N. C., and J. C. Fyfe, 2012b: Observed and simulated changes in the

Southern Hemisphere surface westerly wind-stress. Geophys. Res. Lett.,

doi:10.1029/2012GL052810.

Tachiiri, K., J. C. Hargreaves, J. D. Annan, A. Oka, A. Abe-Ouchi, and M. Kawamiya,

2010: Development of a system emulating the global carbon cycle in Earth

system models. Geophys. Model Dev., 3, 365–376.

Takahashi, T., et al., 2009: Climatological mean and decadal change in surface ocean

pCO(2), and net sea-air CO

ﬂux over the global oceans. Deep-Sea Res. Pt., 56,

554–577.

Takata, K., S. Emori, and T. Watanabe, 2003: Development of the minimal advanced

treatments of surface interaction and runoff. Global Planet. Change, 38, 209–

222.

Takemura, T., T. Nakajima, O. Dubovik, B. N. Holben, and S. Kinne, 2002: Single-

scattering albedo and radiative forcing of various aerosol species with a global

three-dimensional model. J. Clim., 15, 333–352.

Takemura, T., T. Nozawa, S. Emori, T. Y. Nakajima, and T. Nakajima, 2005: Simulation

of climate response to aerosol direct and indirect effects with aerosol transport-

radiation model. J. Geophys. Res. Atmos., 110, D02202.

Takemura, T., H. Okamoto, Y. Maruyama, A. Numaguti, A. Higurashi, and T. Nakajima,

2000: Global three-dimensional simulation of aerosol optical thickness

distribution of various origins. J. Geophys. Res. Atmos., 105, 17853–17873.

Takemura, T., M. Egashira, K. Matsuzawa, H. Ichijo, R. O’Ishi, and A. Abe-Ouchi, 2009:

A simulation of the global distribution and radiative forcing of soil dust aerosols

at the Last Glacial Maximum. Atmos. Chem. Phys., 9, 3061–3073.

Takle, E. S., et al., 2007: Transferability intercomparison—An opportunity for new

insight on the global water cycle and energy budget. Bull. Am. Meteorol. Soc.,

88, 375–384.

Taylor, C. M., R. A. M. de Jeu, F. Guichard, P. P. Harris, and W. A. Dorigo, 2012a:

Afternoon rain more likely over drier soils. Nature, 489, 423–426.

Taylor, C. M., A. Gounou, F. Guichard, P. P. Harris, R. J. Ellis, F. Couvreux, and M. De

Kauwe, 2011: Frequency of Sahelian storm initiation enhanced over mesoscale

soil-moisture patterns. Nature Geosci., 4, 430–433.

Taylor, K. E., R. J. Stouffer, and G. A. Meehl, 2012b: An overview of CMIP5 and the

experiment design. Bull. Am. Meteorol. Soc., 93, 485–498.

Tebaldi, C., and R. Knutti, 2007: The use of the multi-model ensemble in probabilistic

climate projections. Philos. Trans. R. Soc. London A, 365 2053–2075.

Teixeira, J., et al., 2008: Parameterization of the atmospheric boundary layer. Bull.

Am. Meteorol. Soc., 89, 453–458.

Teixeira, J., et al., 2011: Tropical and subtropical cloud transitions in weather

and climate prediction models: The GCSS/WGNE Paciﬁc Cross-Section

Intercomparison (GPCI). J. Clim., 24, 5223–5256.

Terray, L., 2012: Evidence for multiple drivers of North Atlantic multi-decadal climate

variability. Geophys. Res. Lett., 39, L19712.

Terray, L., L. Corre, S. Cravatte, T. Delcroix, G. Reverdin, and A. Ribes, 2012: Near-

surface salinity as nature’s rain gauge to detect human inﬂuence on the tropical

water cycle. J. Clim., 25, 958–977.

Teutschbein, C., F. Wetterhall, and J. Seibert, 2011: Evaluation of different

downscaling techniques for hydrological climate-change impact studies at the

catchment scale. Clim. Dyn., 37, 2087–2105.

Textor, C., et al., 2007: The effect of harmonized emissions on aerosol properties in

global models—an AeroCom experiment. Atmos. Chem. Phys., 7, 4489–4501.

Thorndike, A. S., D. A. Rothrock, G. A. Maykut, and R. Colony, 1975: Thickness

distribution of sea ice J. Geophys. Res. Oceans Atmos., 80, 4501–4513.

Thorne, P. W., et al., 2011: A quantiﬁcation of uncertainties in historical tropical

tropospheric temperature trends from radiosondes. J. Geophys. Res., 116,

D12116.

Thornton, P. E., J. F. Lamarque, N. A. Rosenbloom, and N. M. Mahowald, 2007:

Inﬂuence of carbon-nitrogen cycle coupling on land model response to CO

fertilization and climate variability. Global Biogeochem. Cycles, 21, GB4018.

Tian, B., E. J. Fetzer, B. H. Kahn, J. Teixeira, E. Manning, and T. Hearty, 2013: Evaluating

CMIP5 models using AIRS tropospheric air temperature and speciﬁc humidity

climatology. J. Geophys. Res. Atmos., 118, 114–134.

Timbal, B., and D. Jones, 2008: Future projections of winter rainfall in southeast

Australia using a statistical downscaling technique. Clim. Change, 86, 165–187.

Timmermann, A., S. Lorenz, S.-I. An, A. Clement, and S.-P. Xie, 2007: The effect of

orbital forcing on the mean climate and variability of the tropical Paciﬁc. J. Clim.,

20, 4147–4159.

Timmermann, R., H. Goosse, G. Madec, T. Fichefet, C. Ethe, and V. Duliere, 2005: On

the representation of high latitude processes in the ORCA-LIM global coupled

sea ice-ocean model. Ocean Model., 8, 175–201.

Ting, M., Y. Kushnir, R. Seager, and C. Li, 2009: Forced and internal twentieth-century

SST trends in the north Atlantic. J. Clim., 22, 1469–1481.

Tjernstrom, M., J. Sedlar, and M. Shupe, 2008: How well do regional climate

models reproduce radiation and clouds in the Arctic? An evaluation of ARCMIP

simulations. J. Appl. Meteorol. Climatol., 47, 2405–2422.

Tjiputra, J. F., K. Assmann, M. Bentsen, I. Bethke, O. H. Ottera, C. Sturm, and C. Heinze,

2010: Bergen Earth system model (BCM-C): Model description and regional

climate-carbon cycle feedbacks assessment. Geophys. Model Dev., 3, 123–141.

Tjiputra, J. F., et al., 2013: Evaluation of the carbon cycle components inthe

Norwegian Earth System Model (NorESM). Geophys. Model Dev., 6, 301–325.

Todd-Brown, K. E. O., J. T. Randerson, W. M. Post, F. M. Hoffman, C. Tarnocai, E. A. G.

Schuur, and S. D. Allison, 2013: Causes of variation in soil carbon simulations from

CMIP5 Earth system models and comparison with observations. Biogeosciences,

10, 1717–1736.

Toniazzo, T., and S. Woolnough, 2013: Development of warm SST errors in the

southern tropical Atlantic in CMIP5 decadal hindcasts. Clim. Dyn., doi:10.1007/

s00382-013-1691-2.

Tory, K., S. Chand, R. Dare, and J. McBride, 2013: An assessment of a model-, grid-

and basin-independent tropical cyclone detection scheme in selected CMIP3

global climate models. J. Clim., doi:10.1175/JCLI-D-12-00511.1.

Trenberth, K. E., and J. M. Caron, 2001: Estimates of meridional atmosphere and

ocean heat transports. J. Clim., 14, 3433–3443.

Trenberth, K. E., and J. T. Fasullo, 2008: An observational estimate of inferred ocean

energy divergence. J. Phys. Oceanogr., 38, 984–999.

Trenberth, K. E., and J. T. Fasullo, 2009: Global warming due to increasing absorbed

solar radiation. Geophys. Res. Lett., 36, L07706.

Trenberth, K. E., and J. T. Fasullo, 2010a: Climate change: Tracking Earth’s energy.

Science, 328, 316–317.

Trenberth, K. E., and J. T. Fasullo, 2010b: Simulation of present-day and twenty-ﬁrst-

century energy budgets of the Southern Oceans. J. Clim., 23, 440–454.

Trenberth, K. E., D. P. Stepaniak, and J. M. Caron, 2000: The global monsoon as seen

through the divergent atmospheric circulation. J. Clim., 13, 3969–3993.

Trenberth, K. E., J. T. Fasullo, and J. Kiehl, 2009: Earth’s global energy budget. Bull.

Am. Meteorol. Soc., 90, 311–323.

Tryhorn, L., and A. DeGaetano, 2011: A comparison of techniques for downscaling

extreme precipitation over the northeastern United States. Int. J. Climatol. , 31,

1975–1989.

Tschumi, T., F. Joos, and P. Parekh, 2008: How important are Southern Hemisphere

wind changes for low glacial carbon dioxide? A model study. Paleoceanography,

23, PA4208.

Tschumi, T., F. Joos, M. Gehlen, and C. Heinze, 2011: Deep ocean ventilation, carbon

isotopes, marine sedimentation and the deglacial CO(2) rise. Clim. Past, 7,

771–800.

Tsigaridis, K., and M. Kanakidou, 2007: Secondary organic aerosol importance in the

future atmosphere. Atmos. Environ., 41, 4682–4692.

Tsujino, H., M. Hirabara, H. Nakano, T. Yasuda, T. Motoi, and G. Yamanaka,

2011: Simulating present climate of the global ocean–ice system using the

Meteorological Research Institute Community Ocean Model (MRI. COM):

Simulation characteristics and variability in the Paciﬁc sector. J. Oceanogr., 67,

449–479.

Tsushima, Y., M. Ringer, M. Webb, and K. Williams, 2013: Quantitative evaluation of

the seasonal variations in climate model cloud regimes. Clim. Dyn., doi:10.1007/

s00382-012-1609-4.

Turner, A. G., and H. Annamalai, 2012: Climate change and the south Asian summer

monsoon. Nature Clim. Change, 2, 1–9.

Turner, A. G., P. M. Inness, and J. M. Slingo, 2007: The effect of doubled CO

and

model basic state biases on the monsoon-ENSO system. II: Changing ENSO

regimes. Q. J. R. Meteorol. Soc., 133, 1159–1173.

Ulbrich, U., G. C. Leckebusch, and J. G. Pinto, 2009: Extra-tropical cyclones in the

present and future climate: a review. Theor. Appl. Climatol., 96, 117–131.

Ulbrich, U., J. G. Pinto, H. Kupfer, G. C. Leckebusch, T. Spangehl, and M. Reyers, 2008:

Changing northern hemisphere storm tracks in an ensemble of IPCC climate

change simulations. J. Clim., 21, 1669–1679.

850

Chapter 9 Evaluation of Climate Models

UNESCO, 1981: Tenth report of the joint panel on oceanographic tables and

standardsUNESCO.

Uotila, P., S. O’Farrell, S. J. Marsland, and D. Bi, 2012: A sea-ice sensitivity study with

a global ocean-ice model. Ocean Model., 51, 1–18.

Uotila, P., S. O’Farrell, S. J. Marsland, and D. Bi, 2013: The sea-ice performance of the

Australian climatemodels participating in the CMIP5.Aust. Meteorol. Oceanogr.

J., 63, 121–143.

Uppala, S. M., et al., 2005: The ERA-40 re-analysis. Q. J. R. Meteorol. Soc., 131,

2961–3012.

van den Hurk, B., and E. van Meijgaard, 2010: Diagnosing land-atmosphere

interaction from a regional climate model simulation over West Africa. J.

Hydrometeorol., 11, 467–481.

van Oldenborgh, G., et al., 2009: Western Europe is warming much faster than

expected. Clim. Past, 5, 1–12.

van Oldenborgh, G. J., S. Y. Philip, and M. Collins, 2005: El Niño in a changing climate:

A multi-model study. Ocean Sci., 1, 81–95.

van Roosmalen, L., J. H. Christensen, M. B. Butts, K. H. Jensen, and J. C. Refsgaard,

2010: An intercomparison of regional climate model data for hydrological

impact studies in Denmark. J. Hydrol., 380, 406–419.

van Vliet, M., S. Blenkinsop, A. Burton, C. Harpham, H. Broers, and H. Fowler, 2011: A

multi-model ensemble of downscaled spatial climate change scenarios for the

Dommel catchment, Western Europe. Clim. Change, 111, 249–277.

van Vuuren, D., et al., 2011: The representative concentration pathways: An overview.

Clim. Change, 109, 5–31.

Vancoppenolle, M., T. Fichefet, H. Goosse, S. Bouillon, G. Madec, and M. A. M.

Maqueda, 2009: Simulating the mass balance and salinity of Arctic and Antarctic

sea ice. 1. Model description and validation. Ocean Model., 27, 33–53.

Vancoppenolle, M., H. Goosse, A. de Montety, T. Fichefet, B. Tremblay, and J. L. Tison,

2010: Interactions between brine motion, nutrients and primary production in

sea ice. J. Geophys. Res., 115, C02005.

Vannière, B., E. Guilyardi, G. Madec, F. J. Doblas-Reyes, and S. Woolnough, 2011:

Using seasonal hindcasts to understand the origin of the equatorial cold tongue

bias in CGCMs and its impact on ENSO. Clim. Dyn., 40, 963–981.

Vautard, R., et al., 2013: The simulation of European heat waves from an ensemble

of regional climate models within the EURO-CORDEX project. Clim. Dyn.,

doi:10.1007/s00382-013-1714-z.

Vecchi, G. A., and B. J. Soden, 2007: Global warming and the weakening of the

tropical circulation. J. Clim., 20, 4316–4340.

Vecchi, G. A., K. L. Swanson, and B. J. Soden, 2008: Climate Change: Whither hurricane

activity? Science, 322, 687–689.

Vecchi, G. A., B. J. Soden, A. T. Wittenberg, I. M. Held, A. Leetmaa, and M. J. Harrison,

2006: Weakening of tropical Paciﬁc atmospheric circulation due to anthropogenic

forcing. Nature, 327, 216–-219.

Veljovic, K., B. Rajkovic, M. J. Fennessy, E. L. Altshuler, and F. Mesinger, 2010: Regional

climate modeling: Should one attempt improving on the large scales? Lateral

boundary condition scheme: Any impact? Meteorol. Z., 19, 237–246.

Verlinde, J., et al., 2007: The Mixed-Phase Arctic Cloud Experiment. Bull. Am.

Meteorol. Soc., 88, 205–221.

Verseghy, D. L., 2000: The Canadian Land Surface Scheme (CLASS): Its history and

future. Atmos. Ocean, 38, 1–13.

Vial, J., and T. J. Osborn, 2012: Assessment of atmosphere-ocean general circulation

model simulations of winter northern hemisphere atmospheric blocking. Clim.

Dyn., 39, 95–112.

Vial, J., J.-L. Dufresne, and S. Bony, 2013: On the interpretation of inter-model spread

in CMIP5 climate sensitivity estimates. Clim. Dyn., doi:10.1007/s00382-013-

1725-9.

Vichi, M., S. Masina, and A. Navarra, 2007: A generalized model of pelagic

biogeochemistry for the global ocean ecosystem. Part II: Numerical simulations.

J. Mar. Syst., 64, 110–134.

Vichi, M., et al., 2011: Global and regional ocean carbon uptake and climate change:

Sensitivity to a substantial mitigation scenario. Clim. Dyn., 37, 1929–1947.

Vizcaino, M., U. Mikolajewicz, M. Groger, E. Maier-Reimer, G. Schurgers, and A. M. E.

Winguth, 2008: Long-term ice sheet-climate interactions under anthropogenic

greenhouse forcing simulated with a complex Earth System Model. Clim. Dyn.,

31, 665–690.

Voldoire, A., et al., 2013: The CNRM-CM5.1 global climate model : Description and

basic evaluation. Clim. Dyn., 40, 2091–2121.

Volodin, E. M., 2007: Atmosphere-ocean general circulation model with the carbon

cycle. Izvestiya Atmos. Ocean. Phys., 43, 298–313.

Volodin, E. M., 2008a: Relation between temperature sensitivity to doubled carbon

dioxide and the distribution of clouds in current climate models. Izvestiya Atmos.

Ocean. Phys., 44, 288–299.

Volodin, E. M, 2008b: Methane cycle in the INM RAS climate model. Izvestiya Atmos.

Ocean. Phys., 44, 153–159.

Volodin, E. M., and V. N. Lykosov, 1998: Parametrization of heat and moisture transfer

in the soil-vegetation system for use in atmospheric general circulation models:

1. Formulation and simulations based on local observational data. Izvestiya

Akad. Nauk Fizik. Atmosf. Okean., 34, 453–465.

Volodin, E. M., N. A. Dianskii, and A. V. Gusev, 2010: Simulating present-day climate

with the INMCM4.0 coupled model of the atmospheric and oceanic general

circulations. Izvestiya Atmos. Ocean. Phys., 46, 414–431.

von Salzen, K., et al., 2013: The Canadian Fourth Generation Atmospheric Global

Climate Model (CanAM4). Part I: Representation of physical processes. Atmos.

Ocean, 51, 104–125.

Vose, R. S., et al., 2012: NOAA’S merged land-ocean surface temperature analysis.

Bull. Am. Meteorol. Soc., 93, 1677–1685.

Vrac, M., and P. Naveau, 2008: Stochastic downscaling of precipitation: From dry

events to heavy rainfall Water Resour. Res., 43, W07402.

Waelbroeck, C., et al., 2009: Constraints on the magnitude and patterns of ocean

cooling at the Last Glacial Maximum. Nature Geosci., 2, 127–132.

Wahl, S., M. Latif, W. Park, and N. Keenlyside, 2011: On the Tropical Atlantic SST warm

bias in the Kiel Climate Model. Clim. Dyn., 36, 891–906.

Waliser, D., K. W. Seo, S. Schubert, and E. Njoku, 2007: Global water cycle agreement

in the climate models assessed in the IPCC AR4. Geophys. Res. Lett., 34, L16705

Waliser, D., et al., 2009a: MJO simulation diagnostics. J. Clim., 22, 3006–3030.

Waliser, D. E., J. L. F. Li, T. S. L’Ecuyer, and W. T. Chen, 2011: The impact of precipitating

ice and snow on the radiation balance in global climate models. Geophys. Res.

Lett., 38, L06802.

Waliser, D. E., et al., 2003: AGCM simulations of intraseasonal variability associated

with the Asian summer monsoon. Clim. Dyn., 21, 423–446.

Waliser, D. E., et al., 2009b: Cloud ice: A climate model challenge with signs and

expectations of progress. J. Geophys. Res., 114, D00A21.

Walsh, K., S. Lavender, E. Scoccimarro, and H. Murakami, 2013: Resolution

dependence of tropical cyclone formation in CMIP3 and ﬁner resolution models.

Clim. Dyn., 40, 585–599.

Walther, A., J.-H. Jeong, G. Nikulin, C. Jones, and D. Chen, 2013: Evaluation of the

warm season diurnal cycle of precipitation over Sweden simulated by the Rossby

Centre regional climate model RCA3. Atmos. Res., 119, 131–139.

Wang, B., 2006: The Asian Monsoon. Springer Science+Business Media, Praxis, New

York, NY, USA, 787 pp.

Wang, B., and Q. H. Ding, 2008: Global monsoon: Dominant mode of annual variation

in the tropics. Dyn. Atmos. Oceans, 44, 165–183.

Wang, B., H. J. Kim, K. Kikuchi, and A. Kitoh, 2011a: Diagnostic metrics for evaluation

of annual and diurnal cycles. Clim. Dyn., 37, 941–955.

Wang, B., H. Wan, Z. Z. Ji, X. Zhang, R. C. Yu, Y. Q. Yu, and H. T. Liu, 2004: Design of

a new dynamical core for global atmospheric models based on some efﬁcient

numerical methods. Sci. China A, 47, 4–21.

Wang, C., and J. Picaut, 2004: Understanding ENSO physics —A review. In: Earth’s

Climate: The Ocean-Atmosphere Interaction [C. Wang, S.-P. Xie and J.A. Carton

(eds.)]. American Geophysical Union, Washington, DC, pp. 21–48.

Wang, H., and W. Su, 2013: Evaluating and understanding top of the atmosphere

cloud radiative effects. In Intergovernmental Panel on Climate Change (IPCC)

Fifth Assessment Report (AR5) Coupled Model Intercomparison Project Phase

5 (CMIP5) models using satellite observations. J. Geophys. Res. Atmos., 118,

683–699.

Wang, J., Q. Bao, N. Zeng, Y. Liu, G. Wu, and D. Ji, 2013: The Earth System Model

FGOALS-s2: Coupling a dynamic global vegetation and terrestrial carbon model

with the physical climate. Adv. Atmos. Sci., doi:10.1007/s00376–013-2169-1.

Wang, J. F., and X. B. Zhang, 2008: Downscaling and projection of winter extreme

daily precipitation over North America. J. Clim., 21, 923–937.

Wang, M., and J. E. Overland, 2012: A sea ice free summer Arctic within 30 years: An

update from CMIP5 models. Geophys. Res. Lett., 39, L18501.

Wang, M., J. E. Overland, and N. A. Bond, 2010: Climate projections for selected large

marine ecosystems. J. Mar. Syst., 79, 258–266.

851

Evaluation of Climate Models Chapter 9

Wang, Y., M. Notaro, Z. Liu, R. Gallimore, S. Levis, and J. E. Kutzbach, 2008: Detecting

vegetation-precipitation feedbacks in mid-Holocene North Africa from two

climate models. Clim. Past, 4, 59–67.

Wang, Y. P., and B. Z. Houlton, 2009: Nitrogen constraints on terrestrial carbon

uptake: Implications for the global carbon-climate feedback. Geophys. Res. Lett.,

36, L24403.

Wang, Y. P., et al., 2011b: Diagnosing errors in a land surface model (CABLE) in the

time and frequency domains. J. Geophys. Res. Biogeosci., 116, G01034.

Wania, R., I. Ross, and I. C. Prentice, 2009: Integrating peatlands and permafrost into

a dynamic global vegetation model: 1. Evaluation and sensitivity of physical land

surface processes. Global Biogeochem. Cycles, 23, Gb3014.

Watanabe, M., 2008: Two regimes of the equatorial warm pool. Part I: A simple

tropical climate model. J. Clim., 21, 3533–3544.

Watanabe, M., M. Chikira, Y. Imada, and M. Kimoto, 2011: Convective control of

ENSO simulated in MIROC. J. Clim., 24, 543–562.

Watanabe, M., J. S. Kug, F. F. Jin, M. Collins, M. Ohba, and A. T. Wittenberg, 2012:

Uncertainty in the ENSO amplitude change from the past to the future. Geophys.

Res. Lett., 39, L20703.

Watanabe, M., et al., 2010: Improved climate simulation by MIROC5: Mean states,

variability, and climate sensitivity. J. Clim., 23, 6312–6335.

Watanabe, S., Y. Kawatani, Y. Tomikawa, K. Miyazaki, M. Takahashi, and K. Sato, 2008:

General aspects of a T213L256 middle atmosphere general circulation model. J.

Geophys. Res. Atmos., 113, D12110.

Watterson, I., and P. Whetton, 2011: Distributions of decadal means of temperature

and precipitation change under global warming. J. Geophys. Res. Atmos., 116,

D07101.

Waugh, D., and V. Eyring, 2008: Quantitative performance metrics for stratospheric-

resolving chemistry-climate models. Atmos. Chem. Phys., 8, 5699–5713.

Weaver, A. J., et al., 2001: The UVic Earth System Climate Model: Model description,

climatology, and applications to past, present and future climates. Atmos.

Ocean, 39, 361–428.

Weaver, A. J., et al., 2012: Stability of the Atlantic meridional overturning circulation:

A model intercomparison. Geophys. Res. Lett., 39, L20709.

Webb, M., C. Senior, S. Bony, and J. J. Morcrette, 2001: Combining ERBE and ISCCP

data to assess clouds in the Hadley Centre, ECMWF and LMD atmospheric

climate models. Clim. Dyn., 17, 905–922.

Weber, S. L., et al., 2007: The modern and glacial overturning circulation in the

Atlantic Ocean in PMIP coupled model simulations. Clim. Past, 3, 51–64.

Webster, P. J., A. M. Moore, J. P. Loschnigg, and R. R. Leben, 1999: Coupled ocean-

atmosphere dynamics in the Indian Ocean during 1997–98. Nature, 401, 356–

360.

Wehner, M. F., R. L. Smith, G. Bala, and P. Duffy, 2010: The effect of horizontal

resolution on simulation of very extreme US precipitation events in a global

atmosphere model. Clim. Dyn., 34, 241–247.

Weller, E., and W. Cai, 2013a: Realism of the Indian Ocean Dipole in CMIP5

models: The implication for 1 climate projections. J. Clim., doi:10.1175/JCLI-D-

12-00807.1.

Weller, E., and W. Cai, 2013b: Asymmetry in the IOD and ENSO teleconnection

in a CMIP5 model ensemble and its relevance to regional rainfall. J. Clim.,

doi:10.1175/JCLI-D-12-00789.1.

Wentz, F. J., L. Ricciardulli, K. Hilburn, and C. Mears, 2007: How much more rain will

global warming bring? Science, 317, 233–235.

Westerling, A. L., H. G. Hidalgo, D. R. Cayan, and T. W. Swetnam, 2006: Warming

and earlier spring increase western US forest wildﬁre activity. Science, 313,

940–943.

Wetzel, P., E. Maier-Reimer, M. Botzet, J. H. Jungclaus, N. Keenlyside, and M. Latif,

2006: Effects of ocean biology on the penetrative radiation in a Coupled Climate

Model. J. Clim., 19, 3973–3987.

Whetton, P., I. Macadam, J. Bathols, and J. O’Grady, 2007: Assessment of the use of

current climate patterns to evaluate regional enhanced greenhouse response

patterns of climate models. Geophys. Res. Lett., 34, L14701.

White, C. J., et al., 2013: On regional dynamical downscaling for the assessment

and projection of temperature and precipitation extremes across Tasmania,

Australia. Clim. Dyn., doi:10.1007/s00382-013-1718-8.

Wilcox, L. J., A. J. Charlton-Perez, and L. J. Gray, 2012: Trends in Austral jet position in

ensembles of high- and low-top CMIP5 models. J. Geophys. Res., 117, D13115.

Wild, M., C. N. Long, and A. Ohmura, 2006: Evaluation of clear-sky solar ﬂuxes

in GCMs participating in AMIP and IPCC-AR4 from a surface perspective. J.

Geophys. Res. Atmos., 111, D01104

Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Vol. 59, Academic

Press, San Diego, CA, USA, 467 pp.

Wilks, D. S., 2006: Statistical Methods in the Atmospheric Sciences. Vol. 91, Academic

Press, Elsevier, San Diego, CA, USA, 627 pp.

Willett, K., P. Jones, P. Thorne, and N. Gillett, 2010: A comparison of large scale

changes in surface humidity over land in observations and CMIP3 general

circulation models. Environ. Res. Lett., 5, 025210.

Williams, C. J. R., R. P. Allan, and D. R. Kniveton, 2012: Diagnosing atmosphere-land

feedbacks in CMIP5 climate models. Environ. Res. Lett., 7, 044003.

Williams, K., and M. Webb, 2009: A quantitative performance assessment of cloud

regimes in climate models. Clim. Dyn., 33 141–157.

Williams, K. D., and M. E. Brooks, 2008: Initial tendencies of cloud regimes in the Met

Ofﬁce uniﬁed model. J. Clim., 21, 833–840.

Williams, K. D., W. J. Ingram, and J. M. Gregory, 2008: Time variation of effective

climate sensitivity in GCMs. J. Clim., 21, 5076–5090.

Williamson, D. L., and J. G. Olson, 2007: A comparison of forecast errors in CAM2 and

CAM3 at the ARM Southern Great Plains site. J. Clim., 20, 4572–4585.

Williamson, M. S., T. M. Lenton, J. G. Shepherd, and N. R. Edwards, 2006: An efﬁcient

numerical terrestrial scheme (ENTS) for Earth system modelling. Ecol. Model.,

198, 362–374.

Willis, J. K., 2010: Can in situ ﬂoats and satellite altimeters detect long-term changes

in Atlantic Ocean overturning? Geophys. Res. Lett., 37, L06602.

Winterfeldt, J., and R. Weisse, 2009: Assessment of value added for surface marine

wind speed obtained from two regional climate models. Mon. Weather Rev.,

137, 2955–2965.

Winterfeldt, J., B. Geyer, and R. Weisse, 2011: Using QuikSCAT in the added value

assessment of dynamically downscaled wind speed. Int. J. Climatol. , 31, 1028–

1039.

Winton, M., 2000: A reformulated three-layer sea ice model. J. Atmos. Ocean.

Technol., 17, 525–531.

Winton, M., 2011: Do climate models underestimate the sensitivity of Northern

Hemisphere sea ice cover? J. Clim., 24, 3924–3934.

Wittenberg, A. T., 2009: Are historical records sufﬁcient to constrain ENSO

simulations? Geophys. Res. Lett., 36, L12702.

Wittenberg, A. T., A. Rosati, N. C. Lau, and J. J. Ploshay, 2006: GFDL’s CM2 Global

Coupled Climate Models. Part III: Tropical Paciﬁc climate and ENSO. J. Clim., 19,

698–722.

WMO, 2011: Scientiﬁc Assessment of Ozone Depletion: 2010. Global Ozone

Research and Monitoring Project–Report. World Meteorological Organisation,

Geneva, Switzerland.

Wohlfahrt, J., et al., 2008: Evaluation of coupled ocean-atmosphere simulations of

the mid-Holocene using palaeovegetation data from the northern hemisphere

extratropics. Clim. Dyn., 31, 871–890.

Wood, R., et al., 2011: The VAMOS Ocean-Cloud-Atmosphere-Land Study Regional

Experiment (VOCALS-REx): Goals, platforms, and ﬁeld operations. Atmos. Chem.

Phys., 11, 627–654.

Woollings, T., B. Hoskins, M. Blackburn, D. Hassell, and K. Hodges, 2010a: Storm

track sensitivity to sea surface temperature resolution in a regional atmosphere

model. Clim. Dyn., 35, 341–353.

Woollings, T., A. Charlton-Perez, S. Ineson, A. G. Marshall, and G. Masato, 2010b:

Associations between stratospheric variability and tropospheric blocking. J.

Geophys. Res. Atmos., 115, D06108.

Woollings, T., J. M. Gregory, J. G. Pinto, M. Reyers, and D. J. Brayshaw, 2012: Response

of the North Atlantic storm track to climate change shaped by ocean-atmosphere

coupling. Nature Geosci., 5, 313–317.

Wright, D. G., and T. F. Stocker, 1992: Sensitivities of a zonally averaged Global Ocean

Circulation Model.. J. Geophys. Res. Oceans, 97, 12707–12730.

Wu, Q. G., D. J. Karoly, and G. R. North, 2008a: Role of water vapor feedback on the

amplitude of season cycle in the global mean surface air temperature. Geophys.

Res. Lett., 35, L08711.

Wu, T., 2012: A mass-ﬂux cumulus parameterization scheme for large-scale models:

Description and test with observations. Clim. Dyn., 38, 725–744.

Wu, T., R. Yu, and F. Zhang, 2008b: A modiﬁed dynamic framework for the atmospheric

spectral model and its application. J. Atmos. Sci., 65, 2235–2253.

Wu, T., et al., 2010a: Erratum—The Beijing Climate Center atmospheric general

circulation model: Description and its performance for the present-day climate.

Clim. Dyn., 34, 149–150.

852

Chapter 9 Evaluation of Climate Models

Wu, T., et al., 2010b: The Beijing Climate Center atmospheric general circulation

model: Description and its performance for the present-day climate. Clim. Dyn.,

34, 123–147.

Wyant, M. C., C. S. Bretherton, and P. N. Blossey, 2009: Subtropical low cloud

response to a warmer climate in a superparameterized climate model. Part I:

Regime sorting and physical mechanisms. J. Adv. Model. Earth Syst., 1, 7.

Wyser, K., et al., 2008: An evaluation of Arctic cloud and radiation processes during

the SHEBA year: Simulation results from eight Arctic regional climate models.

Clim. Dyn., 30, 203–223.

Xavier, P. K., 2012: Intraseasonal convective moistening in CMIP3 models. J. Clim.,

25, 2569–2577.

Xavier, P. K., J. P. Duvel, P. Braconnot, and F. J. Doblas-Reyes, 2010: An evaluation

metric for intraseasonal variability and its application to CMIP3 twentieth-

century simulations. J. Clim., 23, 3497–3508.

Xiao, X., et al., 1998: Transient climate change and net ecosystem production of the

terrestrial biosphere. Global Biogeochem. Cycles, 12, 345–360.

Xie, L., T. Z. Yan, L. J. Pietrafesa, J. M. Morrison, and T. Karl, 2005: Climatology and

interannual variability of North Atlantic hurricane tracks. J. Clim., 18, 5370–5381.

Xie, P., and P. A. Arkin, 1997: Global Precipitation: A 17-year monthly analysis based

on gauge observations, satellite estimates, and numerical model outputs. Bull.

Am. Meteorol. Soc., 78, 2539–2558.

Xie, S., J. Boyle, S. A. Klein, X. Liu, and S. Ghan, 2008: Simulations of Arctic mixed-

phase clouds in forecasts with CAM3 and AM2 for M-PACE. J. Geophys. Res.,

113, D04211.

Xie, S., H.-Y. Ma, J. S. Boyle, S. A. Klein, and Y. Zhang, 2012: On the correspondence

between short- and long- timescale systematic errors in CAM4/CAM5 for the

years of tropical convection. J. Clim., 25, 7937–7955.

Xie, S. P., H. Annamalai, F. A. Schott, and J. P. McCreary, 2002: Structure and

mechanisms of South Indian Ocean climate variability. J. Clim., 15, 864–878.

Xin, X.-G., T.-J. Zhou, and R.-C. Yu, 2008: The Arctic Oscillation in coupled climate

models. Chin. J. Geophys. Chinese Edition, 51, 337–351.

Xin, X., L. Zhang, J. Zhang, T. Wu, and Y. Fang, 2013: Climate change projections over

East Asia with BCC_CSM1.1 climate model under RCP scenarios. J. Meteorol.

Soc. Jpn., 91, 413–429.

Xin, X., T. Wu, J. Li, Z. Wang, W. Li, and F. Wu, 2012: How well does BCC_CSM1.1

reproduce the 20th century climate change over China? . Atmos. Ocean. Sci.

Lett., 6, 21–26.

Xu, Y. F., Y. Huang, and Y. C. Li, 2012: Summary of recent climate change studies on

the carbon and nitrogen cycles in the terrestrial ecosystem and ocean in China.

Adv. Atmos. Sci., 29, 1027–1047.

Xue, Y. K., R. Vasic, Z. Janjic, F. Mesinger, and K. E. Mitchell, 2007: Assessment of

dynamic downscaling of the continental US regional climate using the Eta/SSiB

regional climate model. J. Clim., 20, 4172–4193.

Yakovlev, N. G., 2009: Reproduction of the large-scale state of water and sea ice in

the Arctic Ocean in 1948–2002: Part I. Numerical model. Izvestiya Atmos. Ocean.

Phys., 45, 628–641.

Yang, D., and T. Ohata, 2001: A bias-corrected Siberian regional precipitation

climatology. J. Hydrometeorol., 2, 122–139.

Yasutaka, W., N. Masaomi, K. Sachie, and M. Chiashi, 2008: Climatological

reproducibility evaluation and future climate projection of extreme precipitation

events in the Baiu season using a high-resolution non-hydrostatic RCM in

comparison with an AGCM. J. Meteorol. Soc. Jpn., 86, 951–967.

Yeager, S., and G. Danabasoglu, 2012: Sensitivity of Atlantic Meridional Overturning

Circulation variability to parameterized Nordic Sea overﬂows in CCSM4. J. Clim.,

25, 2077–2103.

Yeh, S. W., Y. G. Ham, and J. Y. Lee, 2012: Changes in the tropical Paciﬁc SST trend

from CMIP3 to CMIP5 and its implication of ENSO. J. Clim., 25, 7764–7771.

Yhang, Y. B., and S. Y. Hong, 2008: Improved physical processes in a regional climate

model and their impact on the simulated summer monsoon circulations over

East Asia. J. Clim., 21, 963–979.

Yin, X., A. Gruber, and P. Arkin, 2004: Comparison of the GPCP and CMAP merged

gauge-satellite monthly precipitation products for the period 1979–2001. J.

Hydrometeorol., 5, 1207–1222.

Yokohata, T., M. J. Webb, M. Collins, K. D. Williams, M. Yoshimori, J. C. Hargreaves, and

J. D. Annan, 2010: Structural similarities and differences in climate responses

to CO

increase between two perturbed physics ensembles. J. Clim., 23, 1392–

1410.

Yokohata, T., J. Annan, M. Collins, C. Jackson, M. Tobis, M. Webb, and J. Hargreaves,

2012: Reliability of multi-model and structurally different single-model

ensembles. Clim. Dyn., 39, 599–616.

Yokohata, T., et al., 2013: Reliability and importance of structural diversityof climate

model ensembles. Clim. Dyn., doi:10.1007/s00382-013-1733–9.

Yokohata, T., et al., 2008: Comparison of equilibrium and transient responses to CO

increase in eight state-of-the-art climate models. Tellus A, 60, 946–961.

Yokoi, S., Y. N. Takayabu, and J. C. L. Chan, 2009a: Tropical cyclone genesis frequency

over the western North Paciﬁc simulated in medium-resolution coupled general

circulation models. Clim. Dyn., 33, 665–683.

Yokoi, S., C. Takahashi, K. Yasunaga, and R. Shirooka, 2012: Multi-model projection of

tropical cyclone genesis frequency over the western North Paciﬁc: CMIP5 results.

Sola, 8, 137–140.

Yokoi, S., et al., 2011: Application of cluster analysis to climate model performance

metrics. J. Appl. Meteorol. Climatol., 50, 1666–1675.

Yokoi, T., T. Tozuka, and T. Yamagata, 2009b: Seasonal variations of the Seychelles

Dome simulated in the CMIP3 models. J. Phys. Oceanogr., 39, 449–457.

Yoshimori, M., T. Yokohata, and A. Abe-Ouchi, 2009: A comparison of climate

feedback strength between CO

doubling and LGM experiments. J. Clim., 22,

3374–3395.

Yoshimori, M., J. C. Hargreaves, J. D. Annan, T. Yokohata, and A. Abe-Ouchi, 2011:

Dependency of feedbacks on forcing and climate state in physics parameter

ensembles. J. Clim., 24, 6440–6455.

Young, P. J., et al., 2013: Pre-industrial to end 21st century projections of tropospheric

ozone from the Atmospheric Chemistry and Climate Model Intercomparison

Project (ACCMIP). Atmos. Chem. Phys., 13, 2063–2090.

Yu, B., and F. W. Zwiers, 2010: Changes in equatorial atmospheric zonal circulations

in recent decades. Geophys. Res. Lett., 37, L05701.

Yu, J.-Y., and S. T. Kim, 2011: Reversed spatial asymmetries between El Niño and La

Niña and their linkage to decadal ENSO modulation in CMIP3 models. J. Clim.,

24, 5423–5434.

Yu, W., M. Doutriaux, G. Seze, H. LeTreut, and M. Desbois, 1996: A methodology

study of the validation of clouds in GCMs using ISCCP satellite observations.

Clim. Dyn., 12, 389–401.

Yukimoto, S., et al., 2011: Meteorological Research Institute-Earth System Model v1

(MRI-ESM1)—Model Description. Technical Report of MRI. Ibaraki, Japan, 88 pp.

Yukimoto, S., et al., 2012: A new global climate model of the Meteorological

Research Institute: MRI-CGCM3–Model description and basic performance. J.

Meteorol. Soc. Jpn., 90A, 23–64.

Zaehle, S., and D. Dalmonech, 2011: Carbon-nitrogen interactions on land at global

scales: Current understanding in modelling climate biosphere feedbacks. Curr.

Opin. Environ. Sustain., 3, 311–320.

Zaehle, S., P. Friedlingstein, and A. D. Friend, 2010: Terrestrial nitrogen feedbacks may

accelerate future climate change. Geophys. Res. Lett., 37, L01401.

Zahn, M., and H. von Storch, 2008: A long-term climatology of North Atlantic polar

lows. Geophys. Res. Lett., 35, L22702.

Zalesny, V. B., et al., 2010: Numerical simulation of large-scale ocean circulation

based on the multicomponent splitting method. Russ. J. Numer. Anal. Math.

Model., 25, 581–609.

Zaliapin, I., and M. Ghil, 2010: Another look at climate sensitivity. Nonlin. Proc.

Geophys., 17, 113–122.

Zappa, G., L. C. Shaffrey, and K. I. Hodges, 2013: The ability of CMIP5 models to

simulate North Atlantic extratropical cyclones. J. Clim., doi:10.1175/JCLI-D-12-

00501.1.

Zeng, N., 2003: Glacial-interglacial atmospheric CO

change—The glacial burial

hypothesis. Adv. Atmos. Sci., 20, 677–693.

Zeng, N., 2006: Quasi-100ky glacial-interglacial cycles triggered by subglacial burial

carbon release. Clim. Past, 2, 371–397.

Zeng, N., J. D. Neelin, and C. Chou, 2000: A quasi-equilibrium tropical circulation

model—Implementation and simulation. J. Atmos. Sci., 57, 1767–1796.

Zeng, N., A. Mariotti, and P. Wetzel, 2005: Terrestrial mechanisms of interannual CO

variability. Global Biogeochem. Cycles, 19, GB1016.

Zeng, N., H. F. Qian, E. Munoz, and R. Iacono, 2004: How strong is carbon cycle-

climate feedback under global warming? Geophys. Res. Lett., 31, L20203.

Zhang, F., and A. Georgakakos, 2011: Joint variable spatial downscaling. Clim.

Change, 111, 945–972

Zhang, J., and J. S. Reid, 2010: A decadal regional and global trend analysis of the

aerosol optical depth using a data-assimilation grade over-water MODIS and

Level 2 MISR aerosol products. Atmos. Chem. Phys., 10, 10949–10963.

853

Evaluation of Climate Models Chapter 9

Zhang, J., J. S. Reid, D. L. Westphal, N. L. Baker, and E. J. Hyer, 2008a: A system

for operational aerosol optical depth data assimilation over global oceans. J.

Geophys. Res. Atmos., 113, D10208.

Zhang, Q., H. S. Sundqvist, A. Moberg, H. Kornich, J. Nilsson, and K. Holmgren, 2010a:

Climate change between the mid and late Holocene in northern high latitudes—

Part 2: Model-data comparisons. Clim. Past, 6, 609–626.

Zhang, R., et al., 2013: Have aerosols caused the observed Atlantic multidecadal

variability? J. Atmos. Sci., doi:10.1175/JAS-D-12-0331.1.

Zhang, W., and F.-F. Jin, 2012: Improvements in the CMIP5 simulations of ENSO-SSTA

meridional width. Geophys. Res. Lett., 39, L23704.

Zhang, X., 2007: A comparison of explicit and implicit spatial downscaling of GCM

output for soil erosion and crop production assessments. Clim. Change, 84,

337–363.

Zhang, X., 2010: Sensitivity of arctic summer sea ice coverage to global warming

forcing: towards reducing uncertainty in arctic climate change projections. Tellus

A, 62, 220–227.

Zhang, X., et al., 2007: Detection of human inﬂuence on twentieth-century

precipitation trends. Nature, 448, 461–465.

Zhang, Y., S. A. Klein, J. Boyle, and G. G. Mace, 2010b: Evaluation of tropical cloud

and precipitation statistics of Community Atmosphere Model version 3 using

CloudSat and CALIPSO data. J. Geophys. Res., 115, D12205.

Zhang, Y., et al., 2008b: On the diurnal cycle of deep convection, high-level cloud,

and upper troposphere water vapor in the Multiscale Modeling Framework. J.

Geophys. Res., 113, D16105.

Zhao, M., I. M. Held, and S.-J. Lin, 2012: Some counterintuitive dependencies of

tropical cyclone frequency on parameters in a GCM. J. Atmos. Sci., 69, 2272–

2283.

Zhao, M., I. M. Held, S. J. Lin, and G. A. Vecchi, 2009: Simulations of global hurricane

climatology, interannual variability, and response to global warming using a

50-km resolution GCM. J. Clim., 22, 6653-6678.

Zhao, T. L., et al., 2008: A three-dimensional model study on the production of BrO

and Arctic boundary layer ozone depletion. J. Geophys. Res., 113, D24304.

Zhao, Y., and S. P. Harrison, 2012: Mid-Holocene monsoons: A multi-model analysis

of the inter-hemispheric differences in the responses to orbital forcing and

ocean feedbacks. Clim. Dyn., 39, 1457-1487.

Zheng, W. P., and P. Braconnot, 2013: Characterization of model spread in PMIP2

Mid-Holocene simulations of the African monsoon. J. Clim., 26, 1192-1210.

Zheng, X.-T., S.-P. Xie, and Q. Liu, 2011: Response of the Indian Ocean basin mode

and its capacitor effect to global warming. J. Clim., 24, 6146-6164.

Zheng, Y., J.-L. Lin, and T. Shinoda, 2012: The equatorial Paciﬁc cold tongue simulated

by IPCC AR4 coupled GCMs: Upper ocean heat budget and feedback analysis. J.

Geophys. Res. Oceans, 117, C05024.

Zickfeld, K., et al., 2013: Long-term climate change commitment and reversibility: An

EMIC intercomparison. J. Clim., doi:10.1175/JCLI-D-12-00584.1.

Zou, L., and T. Zhou, 2013: Can a regional ocean-atmosphere coupled model improve

the simulation of the interannual variability of the western North Paciﬁc summer

monsoon? J. Clim., 26, 2353–2367.

Zunz, V., H. Goosse, and F. Massonnet, 2013: How does internal variability inﬂuence

the ability of CMIP5 models to reproduce the recent trend in Southern Ocean sea

ice extent? Cryosphere, 7, 451-468.

Zwiers, F. W., X. Zhang, and Y. Feng, 2011: Anthropogenic inﬂuence on long return

period daily temperature extremes at regional scales. J. Clim., 24, 881-892.

854

Chapter 9 Evaluation of Climate Models

Appendix 9.A: Climate Models Assessed in Chapter 9

Table 9.A.1 | Salient features of the Atmosphere–Ocean General Circulation Models (AOGCMs) and Earth System Models (ESMs) participating in CMIP5 (see also Table 9.1). Column 1: Ofﬁcial CMIP5 model name along with the calendar year

(‘vintage’) of the ﬁrst publication for each model; Column 2: sponsoring institution(s), main reference(s); subsequent columns for each of the model components, with names and main component reference(s). In addition, there are standard

entries for the atmosphere component: horizontal grid resolution, number of vertical levels, grid top (low or high top); and for the ocean component: horizontal grid resolution, number of vertical levels, top level, vertical coordinate type, ocean

free surface type (‘Top BC’). This table information was initially extracted from the CMIP5 online questionnaire (http://q.cmip5.ceda.ac.uk/) as of January 2013. A blank entry indicates that information was not available.

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) ACCESS1.0

(2) 2011

(1) Commonwealth

Scientiﬁc and Industrial

Research Organization

(CSIRO) and Bureau of

Meteorology (BOM),

Australia

(2) (Bi et al., 2013b;

Dix et al., 2013)

(1) Included (as in

HadGEM2 (r1.1))

(2) 192 × 145 N96

(3) 38

(4) 39,255 m

(5) (Martin et al.,

2011; Bi et al., 2013b;

Rashid et al., 2013)

(1) CLASSIC

(2) (Bellouin et al.,

2011; Dix et al., 2013)

Not implemented (1) MOSES2.2

(2) (Cox et al., 1999;

Essery et al., 2003;

Kowalczyk et al., 2013)

(1) ACCESS-OM (MOM4p1)

(2) primarily 1° latitude/longitude

tripolar with enhanced resolution

near equator and at high latitudes

(3) 50

(4) 0–10 m

(5) z*

(6) nonlinear split-explicit

(7) (Bi et al., 2013a; Marsland et

al., 2013)

Not implemented (1) CICE4.1

(2) (Uotila et al.,

2012; Bi et al.,

2013a; Uotila

et al., 2013)

(1) ACCESS1.3

(2) 2011

(1) Commonwealth

Scientiﬁc and Industrial

Research Organization

(CSIRO) and Bureau of

Meteorology (BOM),

Australia

(2) (Bi et al., 2013b;

Dix et al., 2013)

(1) Included (similar to

UK Met Ofﬁce Global

Atmosphere 1.0)

(2) 192 × 145 N96

(3) 38

(4) 39,255 m

(5)

(Hewitt et al., 2011; Bi

et al., 2013b; Rashid et

al., 2013)

(1) CLASSIC

(2) (Bellouin et al.,

2011; Dix et al., 2013)

Not implemented (1) CABLE

(2) (Kowalczyk et al.,

2006; Wang et al., 2011b;

Kowalczyk et al., 2013)

(1) ACCESS-OM (MOM4p1)

(2) primarily 1° latitude/longitude

tripolar with enhanced resolution

near equator and at high latitudes

(3) 50

(4) 0–10 m

(5) z*

(6) nonlinear split-explicit

(7) (Bi et al., 2013a; Marsland et

al., 2013)

Not implemented (1) CICE4.1

(2) (Uotila et al.,

2012; Bi et al.,

2013a; Uotila

et al., 2013)

(1) BCC-CSM1.1

(2) 2011

(1) Beijing Climate Center,

China Meteorological

Administration

(2) (Wu, 2012; Xin et al.,

2012; Xin et al., 2013)

(1) BCC_AGCM2.1

(2) T42 T42L26

(3) 26

(4) 2.917 hPa

(5) (Wu et al., 2008b;

Wu et al., 2010b,

2010a; Wu, 2012)

Prescribed Not implemented (1) BCC-AVIM1.0

(2) (Ji, 1995; Lu and

Ji, 2006; Ji et al.,

2008; Wu, 2012)

(1) MOM4-L40

(2) 1° with enhanced resolution

in the meridional direction in the

tropics (1/3° meridional resolution

at the equator) tripolar

(3) 40

(4) 25 m

(5) z

(6) linear split-explicit

(7) (Grifﬁes et al., 2005)

(1) Included

(2) Based on the

protocols from the Ocean

Carbon Cycle Model

Intercomparison Proj-

ect–Phase 2 (OCMIP2,

http://www.ipsl.jussieu.

fr/OCMIP/ phase2/)

(1) GFDL Sea Ice

Simulator (SIS)

(2) (Winton, 2000)

(1) BCC-CSM1.1(m)

(2) 2011

(1) Beijing Climate Center,

China Meteorological

Administration

(2) (Wu, 2012; Xin et al.,

2012; Xin et al., 2013)

(1) BCC_AGCM2.1

(2) T106

(3) 26

(4) 2.917 hPa

(5) (Wu et al., 2008b;

Wu et al., 2010b,

2010a; Wu, 2012)

Prescribed Not implemented (1) BCC-AVIM1.0

(2) (Ji, 1995; Lu and

Ji, 2006; Ji et al.,

2008; Wu, 2012)

(1) MOM4-L40

(2) Tri-polar: 1° with enhanced

resolution in the meridional direc-

tion in the tropics (1/3° meridional

resolution at the equator)

(3) 40

(4) 25 m

(5) z

(6) implicit

(7) (Grifﬁes et al., 2005)

(1) Included

(2) Based on the

protocols from the Ocean

Carbon Cycle Model

Intercomparison Proj-

ect–Phase 2 (OCMIP2,

http://www.ipsl.jussieu.

fr/OCMIP/ phase2/)

(1) GFDL Sea Ice

Simulator (SIS)

(2) (Winton, 2000)

(continued on next page)

855

Evaluation of Climate Models Chapter 9

Table 9.A.1 (continued)

(continued on next page)

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) BNU-ESM

(2) 2011

(1) Beijing Normal

University

(2)

(1) CAM3.5

(2) T42

(3) 26

(4) 2.194 hPa

Semi-interactive Not implemented (1)CoLM+B-

NUDGVM(C/N)

(2) (Dai et al., 2003;

Dai et al., 2004)

(1) MOM4p1

(2) 200(lat) × 360(lon)

(3) 50

IBGC CICE4.1

(1) CanCM4

(2) 2010

(1) Canadian Center for

Climate Modelling and

Analysis

(2) (von Salzen et al., 2013)

(1) Included

(2) Spectral T63

(3) 35 levels

(4) 0.5 hPa

(5) (von Salzen

et al., 2013)

(1) Interactive

(2) (Lohmann et al.,

1999; Croft et al., 2005;

von Salzen et al., 2013)

(1) Included

(2) (von Salzen

et al., 2013)

(1) CLASS 2.7 (2)

(Verseghy, 2000; von

Salzen et al., 2013)

(1) Included

(2) 256 × 192

(3) 40

(4) 0 m

(5) depth

(6) rigid lid

(7) (Merryﬁeld et al., 2013)

Not implemented (1) Included

(2) (Merryﬁeld

et al., 2013)

(1) CanESM2

(2) 2010

(1) Canadian Center for

Climate Modelling and

Analysis

(2) (Arora et al., 2011;

von Salzen et al., 2013)

(1) Included

(2) Spectral T63

(3) 35 levels

(4) 0.5 hPa

(5) (von Salzen

et al., 2013)

(1) Interactive

(2) (Lohmann et al.,

1999; Croft et al., 2005;

von Salzen et al., 2013)

(1) Included

(2) (von Salzen

et al., 2013)

(1) CLASS 2.7; CTEM

(2) (Verseghy, 2000)

(Arora et al., 2009; von

Salzen et al., 2013)

(1) Included

(2) 256 × 192

(3) 40

(4) 0 m

(5) depth

(6) rigid lid

(7) (Merryﬁeld et al., 2013)

(1) CMOC

(2) (Arora et al., 2009;

Christian et al., 2010)

(1) Included

(2) (Merryﬁeld

et al., 2013)

(1) CCSM4

(2) 2010

(1) US National Centre for

Atmospheric Research

(2) (Gent et al., 2011)

(1) CAM4

(2) 0.9º ×1.25º

(3) 27

(4) 2.194067 hPa

(5) (Neale et al., 2010;

Neale et al., 2013)

(1) Interactive

(2) (Neale et al., 2010;

Oleson et al., 2010;

Holland et al., 2012)

Not implemented (1) Community Land

Model 4 (CLM4)

(2) (Oleson et al., 2010;

Lawrence et al., 2011;

Lawrence et al., 2012)

(1) POP2 with modiﬁcations

(2) Nominal 1° (1.125° in longitude,

0.27–0.64° variable in latitude)

(3) 60

(4) 10 m thick with sur-

face variables at 5 m

(5) depth (level)

(6) linearized, implicit free surface

with constant-volume ocean

(7) (Danabasoglu et al., 2012)

Not implemented (1) CICE4 with

modiﬁcations

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

(1) CESM1(BGC)

(2) 2010

(1) NSF-DOE-NCAR

(2) (Long et al., 2012;

Hurrell et al., 2013)

(1) CAM4

(2) 0.9º ×1.25º

(3) 27

(4) 2.194067 hPa

(5) (Neale et al., 2010;

Neale et al., 2013)

(1) Semi-interactive

(2) (Neale et al., 2010;

Oleson et al., 2010;

Holland et al., 2012)

Not implemented (1) CLM4

(2) (Oleson et al., 2010;

Lawrence et al., 2011;

Lawrence et al., 2012)

(1) POP2 with modiﬁcations

(2) Nominal 1° (1.125° in longitude,

0.27–0.64° variable in latitude)

(3) 60

(4) 10 m with surface

variables at 5 m

(5) depth (level)

(6) linearized, implicit free surface

with constant-volume ocean

(7) (Danabasoglu et al., 2012)

(1) Biogeochemical

Elemental Cycling (BEC)

(1) CICE4 with

modiﬁcations

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

856

Chapter 9 Evaluation of Climate Models

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) CESM1(CAM5)

(2) 2010

(1) NSF-DOE-NCAR

(2) (Hurrell et al., 2013)

(1) Community Atmo-

sphere Model 5 (CAM5)

(2) 0.9º × 1.25º

(3) 27

(4) 2.194067 hPa

(5) (Neale et al., 2010;

Neale et al., 2013)

(1) Semi-interactive

(2) (Neale et al., 2010;

Oleson et al., 2010;

Holland et al., 2012)

Not implemented (1) CLM4

(2) (Oleson et al., 2010)

(Lawrence et al., 2011;

Lawrence et al., 2012)

Same as CESM1 (BGC) Not implemented (1) CICE4 with

modiﬁcations

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

(1)

CESM1(CAM5.1.FV2)

(2) 2012

(1) NSF-DOE-NCAR

(2) (Hurrell et al., 2013)

(1) Community Atmo-

sphere Model (CAM5.1)

(2) 1.9º × 2.0º

(3) 30

(4) 10 hPa

(5) (Neale et al., 2013)

(1) Modal Aerosol

Module (MAM3)

(2) (Ghan et al., 2012;

Liu et al., 2012b)

Not implemented (1) Community Land

Model (CLM4)

(2) (Oleson et al., 2010;

Lawrence et al., 2011)

Same as CESM1 (BGC) Not implemented (1) CICE4 with

modiﬁcations

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

(1) CESM1(WACCM)

(2) 2010

(1) NSF-DOE-NCAR

(2) (Hurrell et al., 2013)

(1) WACCM4

(2) 1.9

× 2.5

(3) 66

(4) 5.1 × 10

–6

hPa

Semi-interactive Included (1) CLM4

(2) (Oleson et al., 2010;

Lawrence et al., 2011;

Lawrence et al., 2012)

Same as CESM1 (BGC) Not implemented (1) CICE4 with

modiﬁcations

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

(1) CESM1(FASTCHEM)

(2) 2010

(1) NSF-DOE-NCAR

(2) (Cameron-Smith et al.,

2006; Eyring et al., 2013;

Hurrell et al., 2013)

(1) Included,

CAM4-CHEM

(2) 0.9º × 1.25º

(3) 27

(4) 2.194067 hPa

(5) (Neale et al., 2010;

Lamarque et al., 2012;

Neale et al., 2013)

(1) Interactive

(2) (Neale et al., 2010;

Oleson et al., 2010;

Holland et al., 2012;

Lamarque et al., 2012)

(1) Included, CAM-CHEM

(2) (Lamarque

et al., 2012)

(1) Community Land

Model 4 (CLM4)

(2) (Oleson et al., 2010;

Lawrence et al., 2011;

Lawrence et al., 2012)

Same as CESM1 (BGC) Not implemented (1) CICE4 with

modiﬁcations

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

(1) CMCC-CESM

(2) 2009

(1) Centro Euro-Mediter-

raneo per I Cambiamenti

Climatici

(2) (Fogli et al., 2009;

Vichi et al., 2011)

(1) ECHAM5

(2) 3.75° × 3.75° (T31)

(3) 39

(4) 0.01 hPa

(5) (Roeckner et al., 2006;

Manzini et al., 2012)

Semi-interactive Not implemented (1) SILVA

(2) (Alessandri

et al., 2012)

Same as CMCC-CM (1) PELAGOS

(2) (Vichi et al., 2007)

(1) LIM2

(2) (Timmermann

et al., 2005)

(1) CMCC-CM

(2) 2009

(1) Centro Euro-Mediter-

raneo per I Cambiamenti

Climatici

(2) (Fogli et al., 2009;

Scoccimarro et al., 2011)

(1) ECHAM5

(2) 0.75° × 0.75° (T159)

(3) 31

(4) 10 hPa

(5) (Roeckner et al., 2006)

Semi-interactive Not implemented Not implemented (1) OPA8.2

(2) 2° average, 0.5° at the equator

(ORCA2)

(3) 31

(4) 5 m

(5) depth (z-level)

(6) linear implicit

(7) (Madec et al., 1998)

Not implemented (1) LIM2

(2) (Timmermann

et al., 2005)

(1) CMCC-CMS

(2) 2009

(1) Centro Euro-Mediter-

raneo per I Cambiamenti

Climatici

(2) (Fogli et al., 2009)

(1) ECHAM5

(2) 1.875° × 1.875° (T63)

(3) 95

(4) 0.01 hPa

(5) (Roeckner et al., 2006;

Manzini et al., 2012)

Semi-interactive Not implemented Not implemented Same as CMCC-CM Not implemented (1) LIM2

(2) (Timmermann

et al., 2005)

Table 9.A.1 (continued)

(continued on next page)

857

Evaluation of Climate Models Chapter 9

Table 9.A.1 (continued)

(continued on next page)

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) CNRM-CM5

(2) 2010

(1) Centre National de

Recherches Meteoro-

logiques and Centre

Europeen de Recherche

et Formation Avancees en

Calcul Scientiﬁque.

(2) (Voldoire et al., 2013)

(1) ARPEGE-Climat

(2) TL127

(3) 31

(4) 10 hPa

(5) (Déqué et al., 1994;

Voldoire et al., 2013)

Prescribed (1) (3-D linear ozone

chemistry model)

(2) (Cariolle and

Teyssedre, 2007)

(1) SURFEX (Land and

Ocean Surface)

(2) (Voldoire et al., 2013)

(1) NEMO

(2) 0.7° on average ORCA1

(3) 42

(4) 5 m

(5) z-coordinate

(6) linear ﬁltered

(7) (Madec, 2008)

(1) PISCES

(2) (Aumont and

Bopp, 2006; Séfé-

rian et al., 2013)

(1) Gelato5 (Sea

Ice)

(2) (Salas-Melia,

2002; Voldoire

et al., 2013)

(1) CSIRO-Mk3.6.0

(2) 2009

(1) Queensland

Climate Change

Centre of

Excellence and

Commonwealth

Scientiﬁc and

Industrial

Research

Organisation

(2) (Rotstayn et al., 2012)

(1) Included

(2) ~1.875º × 1.875º

(spectral T63)

(3) 18

(4) ~4.5 hPa

(5) (Gordon et al., 2002;

Gordon et al., 2010;

Rotstayn et al., 2012)

(1) Interactive

(2) (Rotstayn and

Lohmann, 2002;

Rotstayn et al., 2011;

Rotstayn et al., 2012)

Not implemented (1) Included

(2) (Gordon et al., 2002;

Gordon et al., 2010)

(1) Modiﬁed MOM2.2

(2) ~0.9 × 1.875

(3) 31

(4) 5 m

(5) depth

(6) rigid lid

(7) (Gordon et al., 2002;

Gordon et al., 2010)

Not implemented (1) Included

(2) (O’Farrell, 1998;

Gordon et al., 2010)

(1) EC-EARTH

(2) 2010

(1) Europe

(2) (Hazeleger et al., 2012)

(1) IFS c31r1

(2) 1.125º longitudinal

spacing, Gaussian grid

T159L62

(3) 62

(4) 1 hPa

(5) (Hazeleger

et al., 2012)

Prescribed Not implemented (1) HTESSEL

(2) (Balsamo et al., 2009)

(1) NEMO_ecmwf

(2) The grid is a tripolar curvilinear

grid with a 1° resolution. ORCA1

(3) 31

(4) 1 m

(5) z

(6) free surface linear ﬁltered

(7) (Hazeleger et al., 2012)

Not implemented (1) LIM2

(2) (Fichefet and

Maqueda, 1999)

(1) FGOALS-g2

(2) 2011

(1) LASG (Institute of

Atmospheric Physics)-

CESS(Tsinghua University)

(2) (Li et al., 2012b)

(1) GAMIL2

(2) 2.8125° × 2.8125°

(3) 26 layers

(4) 2.194 hPa

(5) (Wang et al., 2004;

Li et al., 2013b)

Semi-interactive Not implemented (1) CLM3

(2) (Oleson et al., 2010)

(1) LICOM2

(2) 1 × 1° with 0.5 meridional

degree in the tropical region

(3) 30

(4) 10 m

(5) eta co-ordinate

(6)

(7) (Liu et al., 2012a)

Not implemented (1) CICE4-LASG

(2) (Wang and

Houlton, 2009;

Liu, 2010)

(1) FGOALS-s2

(2) 2011

(1) The State Key Labora-

tory of Numerical Modeling

for Atmospheric Sciences

and Geophysical Fluid

Dynamics, The Institute

of Atmospheric Physics

(2) (Bao et al., 2010; Bao et

al., 2013)

(1) SAMIL2.4.7

(2)R42 (2.81° × 1.66°)

(3) 26

(4) 2.19hPa

(5) (Bao et al., 2010;

Liu et al., 2013b)

Semi-interactive Not implemented (1) CLM3.0

(2) (Oleson, 2004;

Zeng et al., 2005;

Wang et al., 2013)

(1) LICOM

(2) The zonal resolution is 1°.

The meridional resolution is 0.5°

between 10°S and 10°N and

increases from 0.5° to 1° from 10°

(3) 30 layers

(4) 10 m (for vertical velocity

and pressure) and 5 meter (for

Temperature and salinity, zonal and

meridional velocity)

(5) depth

(6) linear split-explicit

(7) (Lin et al., 2013)

(1) IAP-OBM

(2) (Xu et al., 2012)

(1) CSIM5

(2) (Briegleb

et al., 2004)

A CNRM-CM5-2 version exists that only differs from CNRM-CM5 in the treatment of volcanoes

858

Chapter 9 Evaluation of Climate Models

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) FIO-ESM v1.0

(2) 2011

(1) The First Institute of

Oceanography, State Ocea-

nic Administration, China

(1) CAM3.0

(2) T42

(3) 26

(4) 3.545 hPa

(5) (Collins et al., 2006c)

Prescribed Not implemented (1) CLM3.5

(2) (Oleson et al., 2008b)

(1) Modiﬁed POP2.0 through

incorporating the non-breaking

surface wave-induced mixing

(2) 1.125° in longitude,

0.27–0.64° variable in latitude

(3) 40

(4) 10 m with surface

variables at 5 m

(5) depth

(6) linear implicit

(7) (Huang et al., 2012)

(1) Improved OCMIP-2

biogeochemical model

(2) (Bao et al., 2012)

(1) CICE4.0

(2) (Hunke and

Lipscomb, 2008)

(1) GFDL-CM2.1

(2) 2006

(2) (Qiao et al., 2004;

Song et al., 2012)

(1) Included

(2) 2.5° longitude, 2°

latitude M45L24

(3) 24

(4) midpoint of top

box is 3.65 hPa

(5) (Delworth et al., 2006)

Semi-interactive Not implemented Included (1) Included

(2) 1° tripolar360 × 200L50

(3) 50

(4) 0 m

(5) depth

(6) nonlinear split-explicit

(7)

Not implemented (1) SIS

(2) (Winton,

2000; Delworth

et al., 2006)

(1) GFDL-CM3

(2) 2011

(1) NOAA Geophysical

Fluid Dynamics Laboratory

(2) (Delworth et al., 2006;

Donner et al., 2011)

(1) Included

(2) ~200 km C48L48

(3) 48

(4) 0.01 hPa

(5) (Donner et al., 2011)

(1) Interactive

(2) (Levy et al., 2013)

(1) Atmospheric

Chemistry

(2) (Horowitz et al.,

2003; Austin and Wilson,

2006; Sander, 2006)

(1) Included

(2) (Milly and Shmakin,

2002; Shevliakova

et al., 2009)

(1) MOM4.1

(2) 1° tripolar360 × 200L50

(3) 50

(4) 0 m

(5) z*

(6) non-linear split-explicit

(7) (Grifﬁes and Greatbatch, 2012)

Not implemented (1) SIS

(2) (Grifﬁes and

Greatbatch, 2012)

(1) GFDL-ESM2G

(2) 2012

(1) NOAA Geophysical

Fluid Dynamics Laboratory

(2) (Dunne et al., 2012;

Dunne et al., 2013)

(1) Included

(2) 2.5° longitude, 2°

latitude M45L24

(3) 24

(4) midpoint of top box is

3.65 hPa

(5) (Delworth et al., 2006)

Semi-interactive Not implemented (1) Included

(2) (Milly and Shmakin,

2002; Shevliakova

et al., 2009; Donner

et al., 2011)

(1) GOLD

(2) 1° tripolar 360 × 2 10L63

(3) 63

(4) 0 m

(5) Isopycnic

(6) nonlinear split-explicit

(7) (Hallberg and Adcroft,

2009; Dunne et al., 2012)

(1) TOPAZ

(2) (Henson et al., 2009;

Dunne et al., 2013)

(1) SIS

(2) (Winton,

2000; Delworth

et al., 2006)

(1) GFDL-ESM2M

(2) 2011

(1) NOAA Geophysical

Fluid Dynamics Laboratory

(2) (Dunne et al., 2012;

Dunne et al., 2013)

(1) Included

(2) 2.5° longitude, 2°

latitude M45L24

(3) 24

(4) midpoint of top

box is 3.65 hPa

(5) (Delworth et al., 2006)

Semi-interactive Not implemented (1) Included

(2) (Milly and Shmakin,

2002; Shevliakova

et al., 2009; Donner

et al., 2011)

(1) MOM4.1

(2) 1° tripolar 360 × 200L50

(3) 50

(4) 0 m

(5) z*

(6) nonlinear split-explicit

(7) (Grifﬁes, 2009;

Dunne et al., 2012)

(1) TOPAZ

(2) (Henson et al., 2009;

Dunne et al., 2013)

(1) SIS

(2) (Winton,

2000; Delworth

et al., 2006)

(1) GFDL-HIRAM-C180

(2) 2011

(1) NOAA Geophysical

Fluid Dynamics Laboratory

(2) (Delworth et al., 2006;

Donner et al., 2011)

(1) Included

(2) Averaged cell size:

approximately 50 × 50

km. C180L32

(3) 32

(4) 2.164 hPa

(5) (Donner et al., 2011)

Prescribed Not implemented (1) Included

(2) (Milly and Shmakin,

2002; Shevliakova

et al., 2009)

Not implemented Not implemented Not implemented

Table 9.A.1 (continued)

(continued on next page)

859

Evaluation of Climate Models Chapter 9

Table 9.A.1 (continued)

(continued on next page)

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) GFDL-HIRAM-C360

(2)

(1) NOAA Geophysical

Fluid Dynamics Laboratory

(2) (Delworth et al., 2006;

Donner et al., 2011)

(1) Included

(2) Averaged cell size:

approximately 25 × 25

km. C360L32

(3) 32

(4) 2.164 hPa

(5) (Donner et al., 2011)

Prescribed Not implemented (1) Included

(2) (Milly and Shmakin,

2002; Shevliakova

et al., 2009)

Not implemented Not implemented Not implemented

(1) GISS-E2-H

(2) 2004

(1) NASA Goddard Institute

for Space Studies USA

(2) (Schmidt et al., 2006)

Note: all GISS models come

in three ﬂavours: p1 = non-

interactive composition,

p2= interactive composi-

tion, p3 = interactive com-

position + interactive AIE

(1) Included

(2) 2° latitude × 2.5°lon-

gitude F

(3) 40

(4) 0.1 hPa

(1) Interactive

(2) (Bauer et al.,

2007; Tsigaridis and

Kanakidou, 2007;

Menon et al., 2010;

Koch et al., 2011)

Note: Aerosol is

“fully interactive”

for p2 and p3, “semi

interactive” for p1

(1) G-PUCCINI

(2) (Shindell et al., 2013a)

Note: Atmos Chem

is “fully interactive”

for p2 and p3, “semi

interactive” for p1

Included (1) HYCOM Ocean

(2) 0.2 to 1° latitude × 1° longitude

HYCOM

(3) 26

(4) 0 m

(5) hybrid z isopycnic

(6) nonlinear split-explicit

(7)

Not implemented Included

(1) GISS-E2-H-CC

(2) 2011

(1) NASA Goddard Institute

for Space Studies USA

(2) (Schmidt et al., 2006)

Note: p1 only

(1) Included

(2) Nominally 1°

(3) 40

(4) 0.1 hPa

(1) Interactive (p1 only)

(2) (Bauer et al.,

2007; Tsigaridis and

Kanakidou, 2007;

Menon et al., 2010;

Koch et al., 2011)

(1) G-PUCCINI

(2) (Shindell et al., 2013a)

Included (1) HYCOM Ocean

(2) 0.2 to 1° latitude × 1° longitude

HYCOM

(3) 26

(4) 0 m

(5) hybrid z isopycnic

(6) nonlinear split-explicit

(7)

(1) Included

(2) (Romanou

et al., 2013)

Included

(1) GISS-E2-R

(2) 2011

(1) NASA Goddard Institute

for Space Studies USA

(2) (Schmidt et al., 2006)

See note for GISS-E2-H

(1) Included

(2) 2° latitude × 2.5°

longitude F

(3) 40

(4) 0.1 hPa

(1) Interactive

(2) (Bauer et al.,

2007; Tsigaridis and

Kanakidou, 2007;

Menon et al., 2010;

Koch et al., 2011)

Note: Aerosol is

“fully interactive”

for p2 and p3, “semi

interactive” for p1

(1) G-PUCCINI

(2) (Shindell et al., 2013a)

Note: Atmos Chem

is “fully interactive”

for p2 and p3, “semi

interactive” for p1

Included (1) Russell Ocean

(2) 1° latitude × 1.25° longitude

Russell 1 × 1Q

(3) 32

(4) 0 m

(5) z*-coordinate

(6) other

(7)

Not implemented Included

(1) GISS-E2-R-CC

(2) 2011

(1) NASA Goddard Institute

for Space Studies USA

(2) (Schmidt et al., 2006)

Note: p1 only

(1) Included

(2) Nominally 1°

(3) 40

(4) 0.1 hPa

(1) Interactive (p1 only)

(2) (Bauer et al.,

2007; Tsigaridis and

Kanakidou, 2007;

Menon et al., 2010;

Koch et al., 2011)

(1) G-PUCCINI

(2) (Shindell et al., 2013a)

Included (1) Russell Ocean

(2) 1° latitude × 1.25° longitude

Russell 1×1Q

(3) 32

(4) 0 m

(5) z*-coordinate

(6) other

(7)

(1) Included

(2) (Romanou

et al., 2013)

Included

860

Chapter 9 Evaluation of Climate Models

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) HadCM3

(2) 1998

(1) UK Met Ofﬁce Hadley

Centre

(2) (Gordon et al.,

2000; Pope et al., 2000;

Collins et al., 2001;

Johns et al., 2003)

(1) HadAM3

(2) N48L19

3.75 × 2.5°

(3) 19

(4) 0.005 hPa

(5) (Pope et al., 2000)

(1) Interactive

(2) (Jones et al., 2001)

Not implemented (1) Included

(2) (Collatz et al., 1991;

Collatz et al., 1992; Cox

et al., 1999; Cox, 2001;

Mercado et al., 2007)

(1) HadOM (lat: 1.25 lon: 1.25 L20)

(2) 1.25° in longitude by 1.25° in

latitude N144

(3) 20

(4) 5.0 m

(5) depth

(6) linear implicit

(7) (UNESCO, 1981)

Not implemented Included

(1) HadGEM2-AO

(2) 2009

(1) National Institute of

Meteorological Research/

Korea Meteorological

Administration

(2) ( Collins et al., 2011;

Martin et al., 2011)

(1) HadGAM2

(2) 1.875° in longitude by

1.25° in latitude N96

(3) 60

(4) 84132.439 m

(5) (Davies et al., 2005)

(1) Interactive

(2) (Bellouin et

al., 2011)

Not implemented (1) Included

(2) (Cox et al., 1999;

Essery et al., 2003)

(1) Included

(2) 1.875° in longitude by 1.25° in

latitude N96

(3)

(4)

(5) z

(6) linear implicit

(7) (Bryan and Lewis, 1979;

Johns et al., 2006);

Not implemented (1) Included

(2) (Thorndike et

al., 1975; McLaren

et al., 2006)

(1) HadGEM2-CC

(2) 2010

(1) UK Met Ofﬁce Hadley

Centre

(2) ( Collins et al., 2011;

Martin et al., 2011)

(1) HadGAM2

(2) 1.875° in longitude by

1.25°in latitude N96

(3) 60

(4) 84132.439 m

(5) (Davies et al., 2005)

(1) Interactive

(2) (Bellouin et

al., 2011)

(1) Atmospheric

Chemistry

(2) (Jones et al., 2001;

Martin et al., 2011)

(1) Included

(2) (Cox et al., 1999;

Essery et al., 2003)

(1) Included

(2) 1.875° in longitude by 1.25° in

latitude N96

(3)

(4)

(5) z

(6) linear implicit

(7) (Bryan and Lewis, 1979;

Johns et al., 2006)

(1) Included

(2) (Palmer and Totterdell,

2001; Halloran, 2012)

(1) Included

(2) (Thorndike et

al., 1975; McLaren

et al., 2006)

(1) HadGEM2-ES

(2) 2009

(1) UK Met Ofﬁce Hadley

Centre

(2) ( Collins et al., 2011;

Martin et al., 2011)

(1) HadGAM2

(2) 1.875° in longitude by

1.25° in latitude N96

(3) 38

(4) 39254.8 m

(5) (Davies et al., 2005)

(1) Interactive

(2) (Bellouin et

al., 2011)

(1) Atmospheric

Chemistry

(2) (O’Connor

et al., 2009)

(1) Included

(2) (Cox et al., 1999;

Essery et al., 2003)

(1) Included

(2) 1° by 1° between 30 N/S and

the poles; meridional resolution

increases to 1/3° at the equator

N180

(3) 40

(4) 5.0 m

(5) z

(6) linear implicit

(7) (Bryan and Lewis, 1979;

Johns et al., 2006)

(1) Included

(2) (Palmer and Totterdell,

2001; Halloran, 2012)

(1) Included

(2) (Thorndike et

al., 1975; McLaren

et al., 2006)

(1) INM-CM4

(2) 2009

(1) Russian Institute for

Numerical Mathematics

(2) (Volodin et al., 2010)

(1) Included

(2) 2 ×1.5° in longitude

and latitude latitude-

longitude

(3) 21

(4) sigma = 0.01

Prescribed Not implemented (1) Included

(2) (Alekseev et al.,

1998; Volodin and

Lykosov, 1998)

(1) Included

(2) 1 × 0.5° in longitude and

latitude generalized spherical

coordinates with poles displaced

outside ocean

(3) 40

(4) sigma = 0.0010426

(5) sigma

(6) linear implicit

(7) (Volodin et al., 2010;

Zalesny et al., 2010)

(1) Included

(2) (Volodin, 2007)

(1) Included

(2) (Yakovlev, 2009)

Table 9.A.1 (continued)

(continued on next page)

861

Evaluation of Climate Models Chapter 9

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) IPSL-CM5A-LR

(2) 2010

(1) Institut Pierre Simon

Laplace

(2) (Dufresne et al., 2012)

(1) LMDZ5

(2) 96 × 95 equivalent

to 1.9° × 3.75° LMDZ96

× 95

(3) 39

(4) 0.04 hPa

(5)(Hourdin et al., 2012)

Semi-interactive Not implemented (1) Included

(2) (Krinner et al., 2005)

(1) Included

(2) 2 × 2-0.5° ORCA2

(3) 31

(4) 0m

(5) depth

(6) linear ﬁltered

(7) (Madec, 2008)

(1) PISCES

(2) (Aumont et al., 2003;

Aumont and Bopp, 2006)

(1) LIM2

(2) (Fichefet and

Maqueda, 1999)

(1) IPSL-CM5A-MR

(2) 2009

(1) Institut Pierre Simon

Laplace

(2) (Dufresne et al., 2012)

(1) LMDZ5

(2) 144 × 143 equivalent

to 1,25° × 2.5° LMDZ144

× 143

(3) 39

(4) 0.04 hPa

(5) (Hourdin et al., 2012)

Semi-interactive Not implemented (1) Included

(2) (Krinner et al., 2005)

(1) Included

(2) 2 × 2-0.5° ORCA2

(3) 31

(4) 0 m

(5) depth

(6) linear ﬁltered

(7) (Madec, 2008)

(1) PISCES

(2) (Aumont et al., 2003;

Aumont and Bopp, 2006)

(1) Included

(2) (Fichefet and

Maqueda, 1999)

(1) IPSL-CM5B-LR

(2) 2010

(1) Institut Pierre Simon

Laplace

(2) (Dufresne et al., 2012)

(1) LMDZ5

(2) 96 × 95 equivalent

to 1.9° × 3.75° LMDZ96

× 95

(3) 39

(4) 0.04 hPa

(5)(Hourdin et al., 2013)

Semi-interactive Not implemented (1) Included

(2) (Krinner et al., 2005)

(1) Included

(2) 2 × 2-0.5° ORCA2

(3) 31

(4) 0 m

(5) depth

(6) linear ﬁltered

(7) (Madec, 2008)

(1) PISCES

(2) (Aumont et al., 2003;

Aumont and Bopp, 2006)

(1) Included

(2) (Fichefet and

Maqueda, 1999)

(1) MIROC4h

(2) 2009

(1) University of Tokyo,

National Institute for

Environmental Studies,

and Japan Agency for

Marine-Earth Science and

Technology

(2) (Sakamoto et al., 2012)

(1) CCSR / NIES / FRCGC

AGCM5.7

(2) 0.5625 × 0.5625°

T213

(3) 56

(4) about 0.9 hPa

(1) SPRINTARS

(2) (Takemura et

al., 2000; Takemura

et al., 2002)

Not implemented (1) MATSIRO

(2) (Takata et al., 2003)

(1) COCO3.4

(2) 1/4° by 1/6° (average grid spac-

ing is 0.28° and 0.19° for zonal and

meridional directions)

(3) 48

(4) 1.25 m

(5) hybrid z-s

(6) nonlinear split-explicit

(7) (Hasumi and Emori, 2004)

Not implemented Included

(1) MIROC5

(2) 2010

(1) University of Tokyo,

National Institute for

Environmental Studies,

and Japan Agency for

Marine-Earth Science and

Technology

(2) (Watanabe et al., 2010)

(1) CCSR/NIES/ FRCGC

AGCM6

(2) 1.40625 × 1.40625°

T85

(3) 40

(4) about 2.9 hPa

(1) SPRINTARS

(2) (Takemura et

al., 2005; Takemura

et al., 2009)

Not implemented (1) MATSIRO

(2) (Takata et al., 2003)

(1) COCO4.5

(2) 1.4° (zonally) × 0.5–1.4°

(meridionally)

(3) 50

(4) 1.25 m

(5) hybrid z-s

(6) linear split-explicit

(7) (Hasumi and Emori, 2004)

Not implemented (1) Included

(2) (Komuro et

al., 2012)

(1) MIROC-ESM

(2) 2010

(1) University of Tokyo,

National Institute for

Environmental Studies,

and Japan Agency for

Marine-Earth Science and

Technology

(2) (Watanabe et al., 2011)

(1) MIROC-AGCM

(2) 2.8125 × 2.8125° T42

(3) 80

(4) 0.003 hPa

(5) (Watanabe, 2008)

(1) SPRINTARS

(2) (Takemura et

al., 2005; Takemura

et al., 2009)

Not implemented (1) MATSIRO

(2) (Takata et al., 2003)

(1) COCO3.4

(2) 1.4° (zonally) × 0.5–1.4°

(meridionally)

(3) 44

(4) 1.25 m

(5) hybrid z-s

(6) linear split-explicit

(7) (Hasumi and Emori, 2004)

(1) NPZD-type

(2) (Schmittner

et al., 2005)

Included

Table 9.A.1 (continued)

(continued on next page)

862

Chapter 9 Evaluation of Climate Models

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) MIROC-ESM-CHEM

(2) 2010

(1) University of Tokyo,

National Institute for

Environmental Studies,

and Japan Agency for

Marine-Earth Science and

Technology

(2) (Watanabe et al., 2011)

(1) MIROC-AGCM

(2) 2.8125 × 2.8125° T42

(3) 80

(4) 0.003 hPa

(5) (Watanabe, 2008)

(1) SPRINTARS

(2) (Takemura et

al., 2005; Takemura

et al., 2009)

(1) CHASER

(2) (Sudo et al., 2002)

(1) MATSIRO

(2) (Takata et al., 2003)

(1) COCO3.4

(2) 1.4° (zonally) × 0.5–1.4°

(meridionally)

(3) 44

(4) 1.25 m

(5) hybrid z-s

(6) linear split-explicit

(7) (Hasumi and Emori, 2004)

(1) NPZD-type

(2) (Schmittner et al.,

2005)

Included

(1) MPI-ESM-LR

(2) 2009

(1) Max Planck Institute for

Meteorology

(2)

(1) ECHAM6

(2) approx. 1.8° T63

(3) 47

(4) 0.01 hPa

(5) (Stevens et al., 2012)

Prescribed Not implemented (1) JSBACH

(2) (Reick et al., 2013)

(1) MPIOM

(2) average 1.5° GR15

(3) 40

(4) 6 m

(5) depth

(6) linear implicit

(7) (Jungclaus et al., 2013)

(1) HAMOCC

(2) (Maier-Reimer et al.,

2005; Ilyina et al., 2013)

(1) Included

(2) (Notz et al.,

2013)

(1) MPI-ESM-MR

(2) 2009

(1) Max Planck Institute for

Meteorology

(2)

(1) ECHAM6

(2) approx. 1.8° T63

(3) 95

(4) 0.01 hPa

(5) (Stevens et al., 2012)

Prescribed Not implemented (1) JSBACH

(2) (Reick et al., 2013)

(1) MPIOM

(2) approx. 0.4° TP04

(3) 40

(4) 6 m

(5) depth

(6) linear implicit

(7) (Jungclaus et al., 2013)

(1) HAMOCC

(2) (Maier-Reimer et al.,

2005; Ilyina et al., 2013)

(1) Included

(2) (Notz et al.,

2013)

(1) MPI-ESM-P

(2) 2009

(1) Max Planck Institute for

Meteorology

(2)

(1) ECHAM6

(2) approx. 1.8° T63

(3) 47

(4) 0.01 hPa

(5) (Stevens et al., 2012)

Prescribed Not implemented (1) JSBACH

(2) (Reick et al., 2013)

(1) MPIOM

(2) average 1.5° GR15

(3) 40

(4) 6 m

(5) depth

(6) linear implicit

(7) (Jungclaus et al., 2013)

(1) HAMOCC

(2) (Maier-Reimer et al.,

2005; Ilyina et al., 2013)

(1) Included

(2) (Notz et al.,

2013)

(1) MRI-AGCM3.2H

(2) 2009

(1) Meteorological

Research Institute

(2) (Mizuta et al., 2012)

(1) Included

(2) 640 × 320 TL319

(3) 64

(4) 0.01 hPa

Prescribed Not implemented (1) SiB0109

(2) (Hirai et al., 2007;

Yukimoto et al., 2011;

Yukimoto et al., 2012)

Not implemented Not implemented Not implemented

(1) MRI-AGCM3.2S

(2) 2009

(1) Meteorological

Research Institute (2)

(Mizuta et al., 2012)

(1) Included

(2) 1920 × 960 TL959

(3) 64

(4) 0.01 hPa

(5) (Mizuta et al., 2012)

Prescribed Not implemented (1) SiB0109

(2) (Hirai et al., 2007;

Yukimoto et al., 2011;

Yukimoto et al., 2012)

Not implemented Not implemented Not implemented

(1) MRI-CGCM3

(2) 2011

(1) Meteorological

Research Institute

(2) (Yukimoto et al., 2011;

Yukimoto et al., 2012)

(1) MRI-AGCM3.3

(2) 320 × 160 TL159

(3) 48

(4) 0.01 hPa

(5) (Yukimoto et al., 2011;

Yukimoto et al., 2012)

(1) MASINGAR mk-2

(2) (Yukimoto et al.,

2011; Yukimoto et

al., 2012; Adachi

et al., 2013)

Not implemented (1) HAL

(2) (Yukimoto et al., 2011;

Yukimoto et al., 2012)

(1) MRI.COM3

(2) 1 × 0.5

(3) 50 + 1 Bottom Boundary Layer

(4) 0 m

(5) hybrid sigma-z

(6) nonlinear split-explicit

(7) (Tsujino et al., 2011; Yukimoto

et al., 2011; Yukimoto et al., 2012)

Not implemented (1) Included (MRI.

COM3)

(2) (Tsujino et al.,

2011; Yukimoto et

al., 2011; Yukimoto

et al., 2012)

Table 9.A.1 (continued)

(continued on next page)

863

Evaluation of Climate Models Chapter 9

Table 9.A.1 (continued)

(1) Model Name

(2) Vintage

(1) Institution

(2) Main Reference(s)

Atmosphere

(1) Component Name

(2) Horizontal Grid

(3) Number of Vert

Levels

(4) Grid Top

(5) References

Aerosol

(1) Component

Name or type

(2) References

Atmos Chemistry

(1) Component Name

(2) References

Land Surface

(1) Component Name

(2) References

Ocean

(1) Component Name

(2) Horizontal Resolution

(3) Number of Vertical Levels

(4) Top Level

(5) z Co-ord

(6) Top BC

(7) References

Ocean Biogeo-

chemistry

(1) Component Name

(2) References

Sea Ice

(1) Component

Name

(2) References

(1) MRI-ESM1

(2) 2011

(1) Meteorological

Research Institute

(2) (Yukimoto et al., 2011;

Yukimoto et al., 2012;

Adachi et al., 2013)

(1) MRI-AGCM3.3

(2) TL159(320 × 160)

(3) 48

(4) 0.01 hPa

(5) (Yukimoto et al., 2011;

Yukimoto et al., 2012;

Adachi et al., 2013)

(1) MASINGAR mk-2

(2) (Yukimoto et al.,

2011; Yukimoto et

al., 2012; Adachi

et al., 2013)

(1) MRI-CCM2

(2) (Deushi and Shibata,

2011; Yukimoto et al.,

2011; Adachi et al., 2013)

(1) HAL

(2) (Yukimoto et al., 2011;

Yukimoto et al., 2012)

(1) MRI.COM3

(2) 1x0.5

(3) 50 + 1 Bottom Boundary Layer

(4) 0m

(5) hybrid sigma-z

(6) non-linear split-explicit

(7)(Tsujino et al., 2011; Yukimoto

et al., 2011; Yukimoto et al., 2012)

(1) Included (MRI.COM3)

(2) (Nakano et al., 2011;

Adachi et al., 2013)

(1) Included (MRI.

COM3)

(2) (Tsujino et al.,

2011; Yukimoto et

al., 2011; Yukimoto

et al., 2012)

(1) NCEP-CFSv2

(2) 2011

(1) National Centers for

Environmental Prediction

(1) Global Forecast Model

(2) 0.9375 T126

(3) 64

(4) 0.03 hPa

(5) (Saha et al., 2010)

Semi-interactive (1) Ozone chemistry

(2) (McCormack

et al., 2006)

(1) Noah Land Surface

Model

(2) (Ek et al., 2003)

(1) MOM4

(2) 0.5° zonal resolution, meridional

resolution varying from 0.25° at

the equator to 0.5° north/south of

10N/10S. Tripolar.

(3) 40

(4) 5.0 m

(5) depth

(6) nonlinear split explicit

(7) (Grifﬁes et al., 2004)

Not implemented (1) SIS

(2) (Hunke and

Dukowicz, 1997;

Winton, 2000)

(1) NorESM1-M

(2) 2011

(1) Norwegian Climate

Centre

(2) (Iversen et al., 2013)

(1) CAM4-Oslo

(2) Finite Volume 1.9°

latitude, 2.5° longitude

(3) 26

(4) 2.194067 hPa

(5) (Neale et al., 2010;

Kirkevåg et al., 2013)

(1) CAM4-Oslo

(2) (Kirkevåg et

al., 2013)

(1) CAM4-Oslo

(2) (Kirkevåg et al., 2013)

(1) CLM4

(2) (Oleson et al., 2010;

Lawrence et al., 2011)

(1) NorESM-Ocean

(2) 1.125° along the equator

(3) 53

(4) 1 m

(5) hybrid z isopycnic

(6) nonlinear split-explicit

(7)

Not implemented (1)CICE4

(2)(Hunke and

Lipscomb, 2008;

Holland et al., 2012)

(1) NorESM1-ME

(2) 2012

(1) Norwegian Climate

Centre

(2) (Tjiputra et al., 2013)

(1) CAM4-Oslo

(2) Finite Volume 1.9°

latitude, 2.5° longitude

(3) 26

(4) 2.194067 hPa

(5) (Neale et al., 2010;

Kirkevåg et al., 2013)

(1) CAM4-Oslo

(2) (Kirkevåg et

al., 2013)

(1) CAM4-Oslo

(2) (Kirkevåg et al., 2013)

(1) CLM4

(2) (Oleson et al., 2010;

Lawrence et al., 2011)

(1) NorESM-Ocean

(2) 1.125° along the equator

(3) 53

(4) 1 m

(5) hybrid z isopycnic

(6) nonlinear split-explicit

(7)

(1) HAMOCC5

(2) (Maier-Reimer et

al., 2005; Assmann

et al., 2010; Tjipu-

tra et al., 2013)

(1) CICE4

(2) (Hunke and

Lipscomb, 2008;

Holland et al., 2012)

864

Chapter 9 Evaluation of Climate Models

Table 9.A.2 | Salient features of the Earth system Models of Intermediate Complexity (EMICs) assessed in the AR5 (see also Table 9.2). Column 1: Model name used in WG1 and the ofﬁcial model version along with the ﬁrst publication for

each model; subsequent columns for each of the eight component models with speciﬁc information and the related references are provided. This information was initially gathered for the EMIC intercomparison project in Eby et al. (2013).

(1) Model name

(2) Model version

(3) Main reference

Atmosphere

(1) Model type

(2) Dimensions

(3) Resolution

(4) Radiation

and cloudiness

(5) References

Ocean

(1) Model type

(2) Dimensions

(3) Resolution

(4) Parametrizations

(5) References

Sea Ice

(1) Schemes

(2) References

Coupling

(1) Flux adjustment

(2) References

Land Surface

(1) Soil schemes

(2) References

Biosphere

(1) Ocean and references

(2) Land and references

(3) Vegetation and references

Ice Sheets

(1) Model type

(2) Dimensions

(3) Resolution

(4) References

Sediment and

Weathering

(1) Model type

(2) References

(1) Bern3D

(2) Bern3D-LPJ

(3) (Ritz et al., 2011)

(1) EMBM

(2) 2-D(φ, λ)

(3) 10° × (3–19)°

(4) NCL

(5)

(1) FG with parameterized

zonal pressure gradient

(2) 3-D

(3) 10° × (3–19)°, L32

(4) RL, ISO, MESO

(5) (Muller et al., 2006)

(1) 0-LT, DOC, 2-LIT (1) PM, NH, RW (1) Bern3D: 1-LST,

NSM, RIV

LPJ: 8-LST, CSM with

uncoupled hydrology

(2) (Wania et al., 2009)

(1) BO (Parekh et al., 2008; Tschumi

et al., 2008; Gangsto et al., 2011)

(2) BT (Sitch et al., 2003; Strassmann

et al., 2008; Stocker et al., 2011)

(3) BV (Sitch et al., 2003)

N/A (1) CS, SW

(2) (Tschumi et

al., 2011)

(1) CLIMBER2

(2) CLIMBER-2.4

(3) (Petoukhov et al.,

2000)

(1) SD

(2) 3-D

(3) 10° × 51°, L10

(4) CRAD, ICL

(5)

(1) FG,

(2) 2-D(φ,z)

(3) 2.5°, L21

(4) RL

(5) (Wright and Stocker,

1992)

(1) 1-LT, PD, 2-LIT

(2) (Petoukhov

et al., 2000)

(1) NM, NH, NW

(2) (Petoukhov

et al., 2000)

(1) 1-LST, CSM, RIV

(2) (Petoukhov et al.,

2000)

(1,2,3) BO, BT, BV (Brovkin et al., 2002) (1) TM

(2) 3-D

(3) 0.75° × 1.5°, L20

(4) (Calov et al.,

2002)

N/A

(1) CLIMBER3

(2) CLIMBER-3α

(3) (Montoya et al.,

2005)

(1) SD

(2) 3-D

(3) 7.5° × 22.5°, L10

(4) CRAD, ICL

(5) (Petoukhov et al.,

2000)

(1) PE

(2) 3-D

(3) 3.75° × 3.75°, L24

(4) FS, ISO, MESO, TCS, DC

(1) 2-LT, R, 2-LIT

(2) (Fichefet and

Morales Maqueda,

1997)

(1) AM, NH, RW (1) 1-LST, CSM, RIV

(2) (Petoukhov et al.,

2000)

(1) BO (Six and Maier-Reimer, 1996)

(2,3) BT, BV (Brovkin et al., 2002)

N/A N/A

(1) DCESS

(2) DCESS

(3) (Shaffer et al., 2008)

(1) EMBM

(2) 2-box in φ,

(3)

(4) LRAD, CHEM

(5) (Shaffer et al., 2008)

(1) 2-box in φ

(2)

(3) L55

(4) parameterized circulation

and exchange, MESO

(5) (Shaffer and

Sarmiento, 1995)

(1) Parameter-

ized from surface

temperature

(2) (Shaffer et

al., 2008)

(1) NH, NW

(2) (Shaffer et

al., 2008)

(1) NST, NSM

(2) (Shaffer et al., 2008)

(1,2) BO, BT (Shaffer et al., 2008) N/A (1) CS, SW

(2) (Shaffer et

al., 2008)

(1) FAMOUS

(2) FAMOUS XDBUA

(3) (Smith et al., 2008)

(1) PE

(2) 3-D

(3) 5° × 7.5°, L11

(4) CRAD, ICL

(5) (Pope et al., 2000)

(1) PE

(2) 3-D

(3) 2.5° × 3.75°, L20

(4) RL, ISO, MESO

(5) (Gordon et al., 2000)

(1) 0-LT, DOC, 2-LIT (1) NM, NH, NW (1) 4-LST, CSM, RIV

(2) (Cox et al., 1999)

(1) BO (Palmer and Totterdell, 2001) N/A N/A

(1) GENIE

(2) GENIE

(3) (Holden et al., 2013)

(1) EMBM

(2) 2-D(φ, λ)

(3) 10° × (3–19)°

(4) NCL

(5) (Marsh et al., 2011)

(1) FG

(2) 3-D

(3) 10° × (3–19) °, L16

(4) RL, ISO, MESO

(5) (Marsh et al., 2011)

(1) 1-LT, DOC, 2-LIT

(2) (Marsh et al.,

2011)

(1) PM, NH, RW

(2) (Marsh et al.,

2011)

(1) 1-LST, BSM, RIV

(2) (Williamson et al.,

2006)

(1,2) BO, BT (Williamson et al., 2006;

Ridgwell et al., 2007b; Holden et al., 2013)

N/A (1) CS, SW

(2) (Ridgwell and

Hargreaves, 2007)

(1) IAP RAS CM

(2) IAP RAS CM

(3) (Eliseev and

Mokhov, 2011)

(1) SD

(2) 3-D

(3) 4.5° × 6°, L8

(4) CRAD, ICL

(5) (Petoukhov et al.,

1998)

(1) PE

(2) 3-D

(3) 3.5° × 3.5°, L32

(4) RL, ISO, TCS

(5) (Muryshev et al., 2009)

(1) 0-LT, 2-LIT

(2) (Muryshev et al.,

2009)

(1) NM, NH, NW

(2) (Muryshev et al.,

2009)

(1) 240-LST, CSM

(2) (Arzhanov et al., 2008)

(2) BT (Eliseev and Mokhov, 2011) N/A N/A

(continued on next page)

865

Evaluation of Climate Models Chapter 9

(continued on next page)

(1) Model name

(2) Model version

(3) Main reference

Atmosphere

(1) Model type

(2) Dimensions

(3) Resolution

(4) Radiation

and cloudiness

(5) References

Ocean

(1) Model type

(2) Dimensions

(3) Resolution

(4) Parametrizations

(5) References

Sea Ice

(1) Schemes

(2) References

Coupling

(1) Flux adjustment

(2) References

Land Surface

(1) Soil schemes

(2) References

Biosphere

(1) Ocean and references

(2) Land and references

(3) Vegetation and references

Ice Sheets

(1) Model type

(2) Dimensions

(3) Resolution

(4) References

Sediment and

Weathering

(1) Model type

(2) References

(1) IGSM2

(2) IGSM 2.2

(3) (Sokolov et

al., 2005)

(1) SD

(2) 2-D(φ, Z)

(3) 4° × 360° , L11

(4) ICL, CHEM

(5) (Sokolov and

Stone, 1998)

(1) Q-ﬂux mixed-layer,

anomaly diffusing,

(2) 3-D

(3) 4° × 5°, L11

(4)

(5) (Hansen et al., 1984)

(1) 2-LT

(2) (Hansen et

al., 1984)

(1) Q-ﬂux

(2) (Sokolov et

al., 2005)

(1) CSM

(2) (Oleson et al., 2008b)

(1) BO (Holian et al., 2001)

(2) BT (Melillo et al., 1993; Liu, 1996;

Felzer et al., 2004)

N/A N/A

(1) LOVECLIM1.2

(2) LOVECLIM1.2

(3) (Goosse et al.,

2010)

(1) QG

(2) 3-D

(3) 5.6° × 5.6°, L3

(4) LRAD, NCL

(5) (Opsteegh et al.,

1998)

(1) PE

(2) 3-D

(3) 3° × 3°, L30

(4) FS, ISO, MESO, TCS, DC

(5) (Goosse and

Fichefet, 1999)

(1) 3-LT, R, 2-LIT

(2) (Fichefet and

Morales Maqueda,

1997)

(1) NM, NH, RW

(2) (Goosse et al.,

2010)

(1) 1-LST, BSM, RIV

(2) (Goosse et al., 2010)

(1) BO (Mouchet and François, 1996)

(2,3) BT, BV (Brovkin et al., 2002)

(1) TM

(2) 3-D

(3) 10 km × 10 km,

L30

(4) (Huybrechts,

2002)

N/A

(1) MESMO

(2) MESMO 1.0

(3) (Matsumoto

et al., 2008)

(1) EMBM

(2) 2-D(φ, λ)

(3) 10° × (3–19)°

(4) NCL,

(5) (Fanning and

Weaver, 1996)

(1) FG

(2) 3-D

(3) 10° × (3–19)°, L16

(4) RL, ISO, MESO

(5) (Edwards and

Marsh, 2005)

(1) 0-LT, DOC, 2-LIT

(2) (Edwards and

Marsh, 2005)

(1) PM, NH, RW (1) NST, NSM, RIV

(2) (Edwards and

Marsh, 2005)

(1) BO (Matsumoto et al., 2008) N/A N/A

(1) MIROC-lite

(2) MIROC-lite

(3) (Oka et al., 2011)

(1) EMBM

(2) 2-D(φ, λ)

(3) 4° × 4°

(4) NCL

(5) (Oka et al., 2011)

(1) PE

(2) 3-D

(3) 4° × 4°

(4) FS, ISO, MESO, TCS

(5) (Hasumi, 2006)

(1) 0-LT, R, 2-LIT

(2) (Hasumi, 2006)

(1) PM, NH, NW

(2) (Oka et al., 2011)

(1) 1-LST, BSM

(2) (Oka et al., 2011)

N/A N/A N/A

(1) MIROC-lite-LCM

(2) MIROC-lite-LCM

(3) (Tachiiri et al., 2010)

(1) EMBM, tuned

for 3 K equilibrium

climate sensitivity

(2) 2-D(φ, λ)

(3) 6° × 6°

(4) NCL

(5) (Oka et al., 2011)

(1) PE

(2) 3-D

(3) 6° × 6°, L15

(4) FS, ISO, MESO, TCS

(5) (Hasumi, 2006)

(1) 0-LT, R, 2-LIT

(2) (Hasumi, 2006)

(1) NM, NH RW

(2) (Oka et al., 2011)

(Tachiiri et al., 2010)

(1) 1-LST, BSM

(2) (Oka et al., 2011)

(1) BO (Palmer and Totterdell, 2001)

(2) loosely coupled BT (Ito and Oikawa,

2002)

N/A N/A

(1) SPEEDO

(2) SPEEDO V2.0

(3) (Severijns and

Hazeleger, 2010)

(1) PE

(2) 3-D

(3) T30, L8

(4) LRAD, IDL,

(5) (Molteni, 2003)

(1) PE

(2) 3-D

(3) 3° × 3°, L20

(4) FS, ISO, MESO, TCS, DC

(5) (Goosse and

Fichefet, 1999)

(1) 3-LT, R, 2-LIT

(2) (Fichefet and

Morales Maqueda,

1997)

(1)NM, NH, NW

(2) (Cimatoribus

et al., 2012)

(1) 1-LST, BSM, RIV

(2) (Opsteegh et al., 1998)

N/A N/A N/A

(1) UMD

(2) UMD 2.0

(3) (Zeng et al., 2004)

(1) QG

(2) 3-D

(3) 3.75° × 5.625°, L2

(4) LRAD, ICL

(5) (Neelin and Zeng,

2000; Zeng et al., 2000)

(1) Q-ﬂux mixed-layer

(2) 2-D surface, deep

ocean box model

(3) 3.75° × 5.625°

(5) (Hansen et al., 1983),

N/A (1) Energy and water

exchange only

(2) (Zeng et al., 2004)

(1) 2-LST with 2-layer

soil moisture

(2) (Zeng et al., 2000)

(1) BO (Archer et al., 2000)

(2,3) BT, BV (Zeng, 2003; Zeng et al.,

2005; Zeng, 2006)

N/A N/A

Table 9.A.2 (continued)

866

Chapter 9 Evaluation of Climate Models

(1) Model name

(2) Model version

(3) Main reference

Atmosphere

(1) Model type

(2) Dimensions

(3) Resolution

(4) Radiation

and cloudiness

(5) References

Ocean

(1) Model type

(2) Dimensions

(3) Resolution

(4) Parametrizations

(5) References

Sea Ice

(1) Schemes

(2) References

Coupling

(1) Flux adjustment

(2) References

Land Surface

(1) Soil schemes

(2) References

Biosphere

(1) Ocean and references

(2) Land and references

(3) Vegetation and references

Ice Sheets

(1) Model type

(2) Dimensions

(3) Resolution

(4) References

Sediment and

Weathering

(1) Model type

(2) References

(1) Uvic

(2) UVic 2.9

(3) (Weaver et al.,

2001)

(1) DEMBM

(2) 2-D(φ, λ)

(3) 1.8° × 3.6°

(4) NCL

(5) (Weaver et al., 2001)

(1) PE

(2) 3-D

(3) 1.8° × 3.6°, L19

(4) RL, ISO, MESO

(5) (Weaver et al., 2001)

(1) 0-LT, R, 2-LIT

(2) (Weaver et al.,

2001)

(1) AM, NH, NW

(2) (Weaver et al.,

2001)

(1) 1-LST, CSM, RIV

(2) (Meissner et al., 2003)

(1) BO (Schmittner et al., 2005)

(2,3) BT, BV (Cox, 2001)

(1) TM

(2) 3-D

(3) 20 km × 20 km,

L10

(4) (Fyke et al.,

2011)

(1) CS, SW

(2) (Eby et al., 2009)

Table 9.A.2 (continued)

Notes:

(a) EMBM = energy moisture balance model; DEMBM = energy moisture balance model including some dynamics; SD = statistical-dynamical model; QG = quasi-geostrophic model; 2-D(φ, λ) = vertically averaged; 3-D = three-

dimensional; LRAD = linearized radiation scheme; CRAD = comprehensive radiation scheme; NCL = non-interactive cloudiness; ICL = interactive cloudiness; CHEM = chemistry module; n° × m° = n degrees latitude by m degrees

longitude horizontal resolution; Lp = p vertical levels.

(b) FG = frictional geostrophic model; PE = primitive equation model; 2-D(φ, z) = zonally averaged; 3-D = three-dimensional; RL = rigid lid; FS = free surface; ISO = isopycnal diffusion; MESO = parameterization of the effect of

mesoscale eddies on tracer distribution; TCS = complex turbulence closure scheme; DC = parameterization of density-driven downward-sloping currents; n° × m° = n degrees latitude by m degrees longitude horizontal resolution;

Lp = p vertical levels.

(c) n-LT = n-layer thermodynamic scheme; PD = prescribed drift; DOC = drift with oceanic currents; R = viscous-plastic or elastic-viscous-plastic rheology; 2-LIT = two-level ice thickness distribution (level ice and leads).

(d) PM = prescribed momentum ﬂux; AM = momentum ﬂux anomalies relative to the control run are computed and added to climatological data; NM = no momentum ﬂux adjustment; NH = no heat ﬂux adjustment; RW = regional

freshwater ﬂux adjustment; NW = no freshwater ﬂux adjustment.

(e) NST = no explicit computation of soil temperature; n-LST = n-layer soil temperature scheme; NSM = no moisture storage in soil; BSM = bucket model for soil moisture; CSM = complex model for soil moisture; RIV = river routing

scheme.

(f) BO = model of oceanic carbon dynamics; BT = model of terrestrial carbon dynamics; BV = dynamical vegetation model.

(g) TM = thermomechanical model; 3-D = three-dimensional; n° × m° = n degrees latitude by m degrees longitude horizontal resolution; n km × m km = horizontal resolution in kilometres; Lp = p vertical levels.

(h) CS = complex ocean sediment model; SW = simple, speciﬁed or diagnostic weathering model.