Comparing two sensitivity analysis approaches for two scenarios with a spatially explicit rural agent-based model

Comparing two sensitivity analysis approaches for two scenarios with a spatially explicit rural agent-based model

Environmental Modelling & Software 54 (2014) 196e210 Contents lists available at ScienceDirect Environmental Modelling & Software journal homepage: ...

2MB Sizes 0 Downloads 14 Views

Environmental Modelling & Software 54 (2014) 196e210

Contents lists available at ScienceDirect

Environmental Modelling & Software journal homepage: www.elsevier.com/locate/envsoft

Comparing two sensitivity analysis approaches for two scenarios with a spatially explicit rural agent-based model Marleen Schouten a, b, Tim Verwaart c, *, Wim Heijman b a

Institute for Environmental Studies, VU University Amsterdam, De Boelelaan 1085, 1081 HV Amsterdam, The Netherlands Agricultural Economics and Rural Policy Group, Wageningen University, P.O. Box 8130, 6700 EW Wageningen, The Netherlands c Agricultural Economics Research Institute (LEI), P.O. Box 29703, 2502 LS den Haag, The Netherlands b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 5 May 2013 Received in revised form 30 December 2013 Accepted 2 January 2014 Available online 28 January 2014

In this paper two sensitivity analysis approaches are applied for scenario analysis in a spatially explicit rural agent-based simulation. The simulation aims to assess the socioeconomic and ecological impacts of agricultural policy interventions, market dynamics and environmental change on a regional scale. Two different methods of sensitivity analysis are investigated: i) a one-at-a-time approach where each parameter is varied one after the other, while all other parameters are kept at their nominal values; and ii) a procedure based on Monte Carlo sampling where random sets of input parameter values are related to outputs of the simulation. The complementarity of both approaches and their contribution to the overall interpretation of the model is shown in two scenarios simulating alternative European policy instruments for biodiversity conservation. Results show that a mixed approach of sensitivity analysis leads to a better understanding of the model’s behaviour, and further enhances the description of the simulation’s response to changes in inputs and parameter settings. Ó 2014 Elsevier Ltd. All rights reserved.

Keywords: Social simulation Agent-based models Sensitivity analysis Complex systems

1. Introduction A fundamental goal of complex systems analyses is to gain insight into the interactions in a system that produce particular outcomes. However, the greater the complexity of the system, the more difficult it is to identify the key interactions or crucial tipping points in a system (Messina et al., 2008). In many cases, data are inadequate to specify all elements and interactions of a system, and therefore the model developer must build assumptions into the representation of the system. This is particularly true for models of systems with complex humaneenvironment interactions, where data span a broad array of social and biophysical domains. Given the rapidity of growth in Agent-Based Models (ABM) for humane environment interactions (see, e.g., An, 2012), and the absence of pre-existing work or standards, many modellers either minimize model evaluation or use standard statistical methods that do not account for complexity (Verburg and Veldkamp, 2005). However, complexity science contends that many systems are best understood as being characterized by phenomena such as emergence and path dependence. Especially ABMs are known to be very sensitive to parameter changes in some ranges of the parameter space. Small

* Corresponding author. Tel.: þ31 70 3358114; fax: þ31 70 3615624. E-mail address: [email protected] (T. Verwaart). 1364-8152/$ e see front matter Ó 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.envsoft.2014.01.003

changes in parameter values may have dramatic consequences for the state of the system, while changes in other parts of the parameter space have little effect (Burgers et al., 2010). This property of ABMs is usually referred to as nonlinearity. It is not just a property of ABMs, it is a general property of complex systems. In general, it is considered good modelling practice to perform sensitivity analysis as part of model verification (Saltelli et al., 2000; Richiardi et al., 2006). Saltelli et al. (2000) define sensitivity analysis as the study of the relationships between information flowing in and out of the model. More precisely one could say that sensitivity analysis studies the effects of variations of parameters and inputs on model outputs. Burgers et al. (2010) urge two reasons to perform extensive sensitivity analysis on ABMs: great uncertainty about actual values of model parameters, and nonlinearity. Before a conclusion can be drawn on the basis of an ABM, the modeller must search for the regions in parameter space where stable, maybe inactive, states of the system occur and where the model is insensitive to parameter changes, regions where tipping points occur and system behaviour changes dramatically in case of small parameter changes, and regions where the system is more or less proportionally sensitive to parameter changes. Yet, ABMs often lack systematic thorough sensitivity analysis as a form of model assessment and validation because of complexity of the model given numerous assumptions and unknown inputs (Grimm et al., 2006; Parry et al., 2013).

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

In this paper we apply two methods of sensitivity analysis for scenario analysis in an ABM by Schouten et al. (2012, 2013a). The purpose of the model is to gain insight into the way farmers and the environment in complex systems respond to disturbances in the light of resilience development. One-at-a-time and Monte Carlo sensitivity analysis approaches are applied, and it is identified how each of the approaches help to elucidate influential parameters and interactions among parameters in the model. One-at-a-time (OAT) sensitivity analysis consists in varying selected parameters one after the other, while all other parameters are being kept constant at their nominal value (also referred to as ceteris paribus approach) (Saltelli et al., 2000). Monte Carlo sensitivity analysis involves the variation of values for all selected input parameters simultaneously using Monte Carlo sampling from pre-defined probability distributions (Hamby, 1994; Saltelli et al., 2000). Within the Monte Carlo sensitivity analysis, a meta-modelling approach is applied in line with Jansen et al. (1994) and Burgers et al. (2010) following two principles: 1. Meta-modelling of results of parameter sets drawn at random from the joint distribution 2. Analysis of contributions of Top Marginal Variance (TMV) and Bottom Marginal Variance (BMV) of individual parameters to the variance explained by the meta-model. In line with Dennis et al. (2000), Saltelli and Annoni (2010) and Parry et al. (2013) we apply both approaches as they are complementary and both contribute to the overall interpretation of the model. This paper contributes to literature by presenting the added value of both approaches in two model scenarios, which has not been available previously. The scenarios simulate two alternative European policy instruments for biodiversity conservation in a small-scale dairy region in the eastern part of the Netherlands. The remainder of this paper is organized as follows: Section 2 shortly introduces the model, followed by a presentation of the parameters taken into account and a brief description of the two sensitivity analysis methods. Section 3 compares the results of the two approaches for each scenario and evaluates the applied methods. Finally, Section 4 includes a discussion and in Section 5 conclusions are drawn and future works are proposed. 2. Material and methods 2.1. Modelling spatially explicit rural agents and their surrounding landscape Rural landscapes are subject to processes of change, and policy makers seek to steer such changes through interventions (Pedroli et al., 2007). Explaining and predicting behaviour of land managers is useful to policy makers aiming at preserving the rural landscape. In order to support rural decision-making, there is a growing recognition that an interdisciplinary approach is needed, considering the rural landscape as a social-ecological system (SES) in which the interdependence between socio-economic (i.e. farming practices) and biophysical components (i.e. ecosystem services) is taken into account across spatial and temporal scales (Berkes et al., 2003). The underlying structures of a rural social-ecological system are complex with structures and processes operating at different (spatial) scales. Evaluating the impact of agricultural policy intervention on farmers’ decision making therefore is a complex process, taking into account a heterogeneous population, situated in a heterogeneous landscape, with corresponding nonlinear behaviour. Schouten et al. (2012, 2013a) explore the impact of agricultural policy interventions on farmers’ investment decisions and the surrounding landscape using a spatially explicit ABM. ABMs within the specific agricultural policy context were pioneered by Balmann (1997) with the Agricultural Policy Simulator (AgriPoliS); Bousquet et al. (1998), Becu et al. (2003) and Castella et al. (2005) with applications of the Cormas model to rural stakeholder emergent behaviour and agrarian transition, Polhill et al. (2013) and Ralha et al. (2013) focussing on land use change and agrienvironmental policy, Jager et al. (2002) who explored how overharvesting affects the relationships between human activities and natural resources and Berger (2001), Happe et al. (2009), Lobianco and Esposti (2010) and Schreinemachers and Berger (2011) who evaluate the way agricultural policy intervention affects a heterogeneous population of farm households and their resources. Bousquet and Le Page (2004) review the early work in this field and present a historical perspective on its emergence. They substantiate the use of ABM for

197

ecosystem management by arguing that representation of heterogeneous states, interactions and learning of individuals, interacting with their environment in a spatial and social organization, can deepen the insight into regulation mechanisms and that agent-based simulation can be applied as tools in collective learning and decision processes. Matthews et al. (2007) review applications of agent-based landuse models. As additional advances to the representation of individual heterogeneity and organization, they mention the abilities of ABM to incorporate social processes and link them with environmental processes. They find no evidence of the actual use of ABM results in planning processes and describe the challenge to make the use of ABM more relevant to problems in the real world and to show that ABM can provide new insights into complex natural resource systems. An (2012) reviews methods to model human decision making in coupled human and natural systems. The author refers to micro-economic models, space theory based models, psychological and cognitive models, institution-based models, rules of thumb, participatory modelling, empirical rules, evolutionary programming, and assumption- or calibration-based rules. The author advocates investments in more process-based decision-making models in addition to empirically based models in order to enable more in-depth coupling of natural and human systems. Furthermore, the author advocates the development of a protocol for decision modelling. The introduction to a recent special issue of Environmental Modelling & Software by Filatova et al. (2013) discusses challenges and prospects for spatial ABM. In line with An (2012) the authors conclude that more work is needed on the selection and understanding of alternative behavioural models and decision making processes. Other thematic challenges according Filatova et al. (2013) are sensitivity analysis, verification and validation of ABM, and two-way linkages in coupled socialeecological systems. The present paper picks up the sensitivity analysis challenge for a model with a two-way linkage, and demonstrates how sensitivity analysis can be used not only for model validation, but also as policy support tool to identify risks and policy leverage opportunities in complex socialeecological systems. The model by Schouten et al. (2012, 2013a) contributes to literature by looking closely at how agents embedded in rural landscapes adapt to agricultural policy interventions and change the rural landscape in this process and how this can inform governance processes in rural development. The model moves beyond previous work in several respects. First, farm agents and their respective parcels are modelled in detail, including heterogeneity in parcel size, quality, shape and distance to the homestead. Actual data sets are used to model the farm agents and their respective parcels, which results in heterogeneity in farm’s and parcel’s characteristics. The data sets are drawn from annual agricultural census files, registries of land use and crops per parcel, and detailed soil and groundwater maps. Thus, it deepens the insight into the mechanisms in a realistic situation, as intended by Bousquet and Le Page (2004). Second, to explore the impact of policy interventions on land market outcomes, a land-auction mechanism has been implemented that takes into account multiple parcels with multiple characteristics at once. According to the decision model classification used by An (2012) the model uses micro-economic and spacetheoretic models at the agent level. In this paper, we use an application of the model as in Schouten et al. (2013a). There, the model has been applied to the case of agrienvironment schemes (AES) within the European Common Agricultural Policy (CAP) to explore how farmers’ decisions to include biodiversity conservation in their enterprise are affected by instruments of AESs, that serve as a governmental incentive to foster landscape cohesion for biodiversity (Whittingham, 2011; OECD, 2005). Policy assessment is the main purpose of the ABM discussed in the present paper, which thus contributes to make the use of ABM more relevant to problems in the real world as stipulated by Matthews et al. (2007). In the following, a condensed description of the model dynamics and main equations that guide agents’ choices are presented. For detailed documentation and an ODD (Overview, Design concepts and Details) description of the model (Grimm et al., 2006; Polhill et al., 2008; Grimm et al., 2010) we refer to the Appendix and the website.1 2.1.1. Purpose The core of the model is the understanding and modelling of an agent-based system for the purpose of agricultural policy assessment while simulating the agents and their corresponding landscape in a spatially explicit way. The model establishes a virtual world of a rural region and comprises a large number of individually acting farms that operate in this region, as well as farms interacting with each other and with parts of their environment. The model implements an abstract region, that can be initialized with empirical data on individual farms and existing agricultural spatial structures. In the following, we present a description of the single entities of the model and describe their relationships. 2.1.2. Farm agent behaviour Within the model farm agents represent dairy farm households, with as many agents modelled as there are dairy farm households in reality. State variables of the agents include the location of the agent’s farmstead, the location of their fields, the

1 http://www.wageningenur.nl/en/Expertise-Services/Chair-groups/SocialSciences/Agricultural-Economics-and-Rural-Policy-Group/Publications/SERAmodel.htm.

198

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

individual household composition (age, successor) available resources such as cash, livestock, nitrogen balance and feed balance of the farm. For each parcel the size, shape and current land use is known and the distance to the homestead is calculated. Also, possibilities for AESs are known, as explained in the previous section. A basic principle of AESs in the EU is that the participation of individual farmers is voluntary. In the simulation, farm agents base their contract choice on farm production, economic results, intensity of land use, land quality, and spatial characteristics of the respective parcel. For the sensitivity analyses we assume a fixed annual AES compensatory payment per hectare, independent of location and spatial configuration of the landscape. This is in line with current European AES programmes. Within the simulation phase, each farm agent is equipped with a behavioural model that guides decisions and keeps track of the agent’s internal state described by attributes such as age, location and size. According to their behavioural model, the individual farm agents evolve subject to their state of attributes and to changes in their environment. Because the model deals with parcels that are heterogeneous in size, quality and shape, the model does not deal with a conventional farm optimization problem. We have created a constrained optimization, in which farmers take into account these parcel characteristics. The farm agents’ objective function is expressed in the following way: Farm gross margin is based on the sum of the contribution of each individual parcel (i) and crops grown (j) on land controlled by the farm agent, based on the following function (1): Yfarm ¼

Xn

Y i¼1 ij

(1)

In each period farm agents calculate the parcels gross margin (Yij) for both their grassland and maize land parcels, as well as for new parcels available on the land market. For each parcel revenues are calculated according to function (2) based on revenue from milk production (Di), AES compensatory payment (Ci), transport costs (Ti), manure disposal costs or application revenue (MRi), fertilizer costs (FCi) and costs for buying feed (Fi). What remains are the gross margins per parcel.with Yij ¼ Di þ Ci þ MRi  Ti  FCi  Fi

X

if Mi > Mlegal then MRi < 0

(5)

if Mi < Mlegal then MRi > 0

(6)

Finally, there are a number of costs involved for transport (Ti), which is given by a fixed average per kilometre (Tfixed)(7). Ti ¼ Tfixed $kmi

(7)

And fixed costs for fertilizer use (FCi) and for buying cattle feed (Fi) (see Schouten et al., 2013b for more details). The farm agent compares the gross margins for the offered parcels and has the opportunity to bid on offered parcels whenever their bid price is higher than the reserve price. Furthermore, farm agents form expectations about market prices based on past experience, following the theory of adaptive expectations. They revise their expectations with respect to output prices periodically by calculating expected prices for land which are used in the decision making process (following Kellermann et al., 2007). They are used in the form of a weighted moving average of the prices in the past periods. Next to bidding on the land market for new parcels, the farm agent makes a trade-off between three types of land use for the parcels that are in use: conventional grassland, maize land, and possibilities for agri-environment contracting. Whenever an AES is chosen, this type of land use remains on the parcel during the contract period. For each tradeoff the expectation price is included in the determination. Whenever a farm agent choses an AES, an extra restriction on manure and fertilizer application is applied to the parcel which affects its contribution to gross margin. The decision models’ parameters included in the sensitivity analysis are listed in Table 1.

(2)

The revenue from milk production (Di) in (3) is the sum of the number of cows per hectare based on individual farm ratio (rcow) times the size of the parcel in hectares, times the average milk production per cow (qmilk), times the milk price (pmilk). Di ¼

nitrogen, which can be attracted from outside the farm. This generates extra revenue (6).

rcow $parcelij $qmilk $pmilk

(3)

The revenue from compensatory payments for AES (Ci) is equal to the compensatory payment per hectare (S) times the size of the parcel. Ci ¼ S$parcelij

(4)

Furthermore, when the nitrogen production by grazing cows (Mi) exceeds the legal limits of 250 kg N/ha (Mlegal), the farm agent needs to dispose manure from the parcel which involves larger costs (5). It is also possible that there is a shortage of

2.1.3. Process overview and scheduling Fig. 1 depicts the organization of the model framework by providing the conceptual class diagram of the model with the main classes and their relations. Fig. 1 shows that two types of agents are distinguished, the Trader Agent and the Auctioneer. Trader Agent describes all agents that can be traders, and that wish to trade parcels of land. Farmers in the model are such trader agents. Decision making strategies are described in the class DecisionMakingStrategy. Every TraderAgent has a ValuationStrategy which is used to determine the valuation for a parcel. The resulting value of a parcel is expressed by the class ValuationStrategy. Individual parcels are represented by the class FarmLand, that holds all information on the parcels that belong to the farms. Farmlands hold crops (class Crop), and a certain quality (emuneration ParcelQuality). Two implementations currently exist: Maize and Grass. Farmlands can also hold agri-environment schemes (class AES Contract). The other agent currently in the model is the Auctioneer. The Auctioneer is the mediator between traders. The Auctioneer ‘requests’ traders to make offers to either express their

Table 1 Model parameters and nominal values that stay constant throughout the simulation; the table also specifies the value range considered in the sensitivity analysis. Parameter

Price expectation parameter Maximum number of cows per hectare Fertilizer maize land Feed production maize land Nitrogen application maize land Contract period agri-environment scheme Agri-environment compensatory payment Expectation percentage Feed required per cow Fertilizer grassland Nitrogen sell price Feed buy price Milk price Nitrogen production per cow Milk production per cow Transport costs coefficient agri-environmental parcel Fixed Transport costs agri-environmental parcel Transport costs coefficient Fixed transport costs Manure buy percentage Manure buy price Feed sell percentage Fertilizer price

Unit

% Cows kg N Net energy for lactation kg N Year V/ha % Net energy for lactation kg N V/kg N V/net energy for lactation V/kg kg N kg V/km V/ha V/km V/ha % V/kg N % V/kg N

Nominal value

0.5 3 62.78 1000 150 6 1018 0.7 6374.8 35 2 1.34 0.31 115 7875 20 20 50 50 80 1.8 90 0.7

Value range Min

Max

0 2.5 40 500 47.43 2 321.92 0.5 4000 11.07 1 1 0.2 50 5000 6.32 6.32 15.81 15.81 0.5 1.25 0.5 0.5

1 3.5 80 1500 474.34 12 3219.19 1 10,000 110.68 4 3 0.5 150 12,000 63.25 63.25 158.11 158.113 1 2.5 1 1

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

199

Fig. 1. Simplified UML class diagram: program classes with main attributes and relations (Schouten et al., 2012); reproduced with permission from Springer.

willingness to buy or sell a good. The Auctioneer makes use of a mechanism to match bids and asks to clear the market. This is represented by the class AuctionMechanism. It presumes that multiple buyers and sellers are present and the parcels traded are heterogeneous (characterised by multiple attributes). 2.2. Conducting two sensitivity analysis approaches In their position paper in Environmental Modelling and Software, Jakeman et al. (2006) stress the relevance of sensitivity analysis for good, disciplined model practice, in particular for the evaluation of output sensitivity to parameter uncertainties and for identification of model reduction opportunities. Also in recent articles, parameter identification, model calibration and uncertainty quantification are identified as important steps in the modelling process, supported by sensitivity analysis (Zhan et al., 2013). Model reduction remains an issue for development of new methods; see e.g., the Environmental Modelling & Software special issue on emulation techniques for the reduction and sensitivity analysis of complex environmental models (Ratto et al., 2012). In addition to the generally recognized purposes of sensitivity analysis (parameter sensitivity assessment and model reduction), Brown et al. (2005) mention the discovery of policy leverage opportunities: sensitivity analysis can reveal regions in parameters space where policy interventions may be particularly efficient (or not). Many authors have reviewed and compared methods for sensitivity analysis in the environmental modelling context. Refsgaard et al. (2007) review fourteen (partly complementary) methods commonly used in uncertainty assessment and characterisation, ranging from quantitative methods like Monte Carlo analysis and statistical methods for sensitivity analysis to qualitative methods such as expert

elicitation and stakeholder involvement, and a combination of a quantitative and qualitative approaches (NUSAP) to uncertainty in context, model inputs, model structure and model parameters and model technical uncertainty. In terms of the latter review, our work refers to the context (two different policy scenarios) and the parameter uncertainty and takes a quantitative approach. As indicated in the introduction, we apply a OATmethod (Campolongo et al., 2000) and a Monte Carlo method (Jansen et al., 1994). The purpose is not to select one of the two methods, but to benefit from their complementarities. Saltelli and Annoni (2010) discuss the shortcomings of the OAT method. These shortcomings are fully realized in our work. However, the method has its strengths in easy and rapid evaluation of effects of extreme parameter values, which are not easily detected by statistical methods, and its shortcomings can be complemented by the statistical methods. In particular agent-based models are often large and complex simulations with significant data requirements (Parry et al., 2013). As a result, there are usually numerous assumptions and unknown inputs, which lead to problems in validating model behaviour and conducting sensitivity analysis (Grimm et al., 2006). A thorough sensitivity analysis helps to interpret the model, increases its credibility across a range of input scenarios and can uncover errors. Also model reduction is still seen as one of the main purposes of sensitivity analysis. Filatova et al. (2013) review applications of sensitivity analysis for agent-based models of social-ecological systems and identify sensitivity analysis as one of the thematic challenges nowadays. They focus on model verification and validation as purposes of sensitivity analysis, and do not mention policy support as an application of sensitivity analysis as Brown et al. (2005) do. Several local and global sensitivity analysis methods have been applied so far on complex ABMs within the environmental modelling context. Local methods

200

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

compute or approximate the local response of model outputs by varying input parameters individually, while others are set at their nominal value. Global methods are more comprehensively accounting for output variations by varying multiple parameters simultaneously (see e.g. Parry et al., 2013; Oakley and O’Hagan, 2004; Nossent et al., 2011). The choice of which method of sensitivity analysis to apply is difficult, since each technique has its strengths and weaknesses. Different methods may provide different types of information about the effects of parameter changes. Several authors have denoted that different sorts of sensitivity analyses have to be performed simultaneously in order to provide a thorough analysis of sources of uncertainty of model outputs and the model’s degree of nonlinearity. Nossent et al. (2011) and Happe (2005) apply both a Latin-hypercube-One-factor-ata-time (LH-OAT), and Nossent et al. (2011) extend the analysis by comparing with a variance based method. Parry et al. (2013) also compare local and global methods, and present a novel global Bayesian Analysis method, which reduces the number of runs compared to a traditional Monte Carlo approach. The mixed methodological approach presented in this paper consists of an OAT approach which is in line with the reasoning of Nossent et al. (2011), Happe (2005) and Parry et al. (2013), but given the large set of parameters in our model we do not apply a Latin Hypercube to the model. For the chosen global analysis method, we do not know which input drives most of the variation of the outputs. Therefore, we chose to apply a more traditional Monte Carlo approach, including a large number of runs. Not all parameters that were estimated during initial model development are included in the sensitivity analysis. Schouten et al. (2013a) present results of a oneat-a-time sensitivity analysis of the simulation under two policy scenarios. Especially for parameters with respect to grass production (mineralization of grassland, leaching coefficient grassland) this the simulation shows very sensitive. However, the applied parameter values are commonly used by farmers and their advisors in The Netherlands and are widely used throughout agricultural economics literature (see Middelkoop and Aarts, 1991; Van de Ven, 1992; Groeneveld et al., 1998, 2001; Peerlings and Polman, 2008). Therefore we assume no uncertainty in the values that farmers use to take their decisions and left these parameters out of the analyses conducted for this paper. 2.3. One-at-a-time sensitivity analysis A standard one-at-a-time sensitivity analysis approach following Campolongo et al. (2000) is applied to assess the impact of parameters on the following simulated outputs: - Mean gross margin of farms in the region - Mean farm area (in hectares) in the region - Total contracted AES area (in hectares) in the region

distributions, we draw values at random from uncorrelated uniform distributions, ranging as indicated in Table 1. Second, the resulting parameter sets are used to initialize the agents for 1000 simulation runs over a 25 year simulation period. Sensitivity analyses were performed with USAGE 2.0, a collection of GenStat algorithms for sensitivity and uncertainty analysis (Goedhart and Thissen, 2009). The relative importance of individual input parameters on output variables is assessed by decomposition of the variance of the output variable. As a first step, we inspect the scatterplots of the outputs against the parameters displayed in Table 1. Through inspecting the scatterplots, the most important parameters to focus on can be indicated, whenever they show stochastic or deterministic components and linear or nonlinear patterns. The second step contains metamodelling, in which we try to find a regression model that can serve as a basis for decomposition of variance. Any type of regression may be applied, e.g. linear regression including polynomial and interaction terms or regression with smoothing splines as a form of nonparametric regression, as long as it explains a great deal (preferably at least 90%) of the output variance. The calculation is successful if the percentage of variance accounted for by all inputs considered is close to 100, since the analysis only accounts for that part of the variance of the output that is explained by regression (Goedhart and Thissen, 2009). As a third step, we check for multicollinearity by comparing the top marginal variance (TMV) of an input, which is the variance reduction that would occur if the input (all parameters) would become fully known (based on Jansen et al., 1994), with the bottom marginal variance (BMV). BMV is the variance explained by the metamodel that the meta model cannot explain without that particular parameter. When there are no correlations between parameters, TMV and BMV of a parameter are equal if and only if that variable is not interacting or interchangeable with any other variable. If significant interactions occur, which is the case in the model used in this paper, we move to the fourth step, in which we look for interaction terms using stepwise regression. 2.5. The study area: rural landscape Winterswijk The software code of this model is written in the object-oriented programming language Java, using the open-source agent-based modelling framework Recursive Porous Agent Simulation Toolkit Symphony (REPAST Symphony 2.0). For this paper we use the agricultural region Winterswijk, which is located in the eastern part of the Netherlands, as a case study. From a landscape perspective, the area represents a highly valued cultural-historic landscape where small-scale agriculture and nature areas are closely related. This spatial structure has resulted in a synergic development between nature conservation, social and economic development and recreational and cultural fruition (Provincie Gelderland, 2005). The size of the area is approximately 22,000 ha (5847 parcels), in which 651 farms are present (reference

These outputs are relevant for business-economic and ecological developments at regional level, and also inform about adoption of agri-environment policies by farmers. Sensitivity of these outputs to parameter variation is calculated for each parameter one-by-one. For that purpose a simulation is run for 25 years, with all parameters set at the nominal value, except one which is set at either the maximal or the minimal value of the ranges shown in Table 1. The ranges of the input parameters are defined based on agricultural economic and ecological literature, and based on regression analyses performed on agricultural census and FADN data. Whenever there is uncertainty about the theoretical range of the model parameters, a larger range was chosen. This procedure is repeated for each parameter, once with the maximal value and once with the minimal value. In order to simplify representation, the analysis of the results takes averaged values of the outputs over the 25-year simulation period. This first form of sensitivity analysis is selected because it is easy to understand by non-experts, relatively simple to implement and because it provides a direct assessment of sensitivity without using any transformation in the relationship between model input and model output. However, Hamby (1994) mentions the following disadvantages of the one-at-a-time approach i) it is more computationally intensive than other methods when the analysis involves a large number of parameters, ii) it is not suited to study the influence of large variations of input parameters on model predictions, and iii) it does not take into account interactions resulting from the simultaneous variation of multiple parameters. Because of these disadvantages, we compare the results of the one-at-a-time approach with results generated by a more comprehensive method: a Monte Carlo approach. 2.4. Monte Carlo random sampling A Monte Carlo approach was chosen because it allows for simultaneous variation of the values of all input parameters, in contrast to the simpler one-at-a-time sensitivity analysis discussed in Section 2.3. In this approach, we deal with a regression-based sensitivity analysis: a meta model in terms of the input parameters is fitted to an output variable. The output is produced by simulation runs using input parameter sets generated by Monte Carlo (random) sampling. Data generation proceeds as follows. First, 1000 input parameter sets are drawn from the joint distribution of all model parameters. As the goal is to study the effects of parameter variation and there is no accurate information on actual parameter

Fig. 2. The case study area Winterswijk.

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

201

year 2008). Sixty percent of the main production area in the region is used for specialized dairy farming, with an average production intensity of approximately 12,000 kg milk-ha1 (Korevaar et al., 2006). The rural SES that is used for this paper is characterized by 201 specialized dairy farms that were selected from Agricultural Census data. Large parts of the region contain important nature conservation areas that belong to the National Ecological Network (NEN), which is part of the European Natura 2000 network (Opdam et al., 2008). Fig. 2 shows the boundaries of the agricultural region (‘national landscape area’), and shows where the nature areas and the potential areas for farmland biodiversity conservation (‘search area’) are located. Fig. 2 shows that nature areas are scattered throughout the case study area, and that potential areas for farmland biodiversity are closely linked to these nature areas. Areas not included in the model are white and include small villages. The main urban areas are indicated in dark grey with the city of Winterswijk located in the centre. 2.6. Biodiversity and habitat network To illustrate the value of both sensitivity analysis approaches, sensitivity analysis was performed on an additional policy scenario to study how AES can contribute to the spatial cohesion of landscapes in terms of habitat network patterns. The importance of spatial habitat network patterns is widely accepted among ecologists (see, for instance, Opdam et al., 2003) as an important condition to develop biodiversity. These spatial habitat network patterns are integrated in the model by means of a spatial cohesion Reilly index, which provides information on the (potential) contribution of the parcel to conserve biodiversity, which depends on its location and the surrounding landscape configuration (Cotteleer, 2008; Reilly, 1931; Schouten et al., 2013a). The calculation of the spatial cohesion Reilly-index starts at the point where the site is located. After that, the size of the nature conservation areas (abbreviation NCA) within a certain radius (i.e. 5 km) is determined, as well as the size of the AES site. Based on the sum of all the distances of the site to the nature conservation areas located within the chosen radius, and on the size of the nature conservation areas and AES sites, the spatial cohesion Reilly-index can be calculated: Ri ¼

J X Ai þ Cj j¼1

d2ij

(8)

where Ri represent the Reilly index of parcel i, Ai the surface area of parcel i, J the number of conservation areas within range, Cj the surface area of the jth nature conservation area, and dij the distances from the parcel to the centres of the conservation areas. The index captures, in one number, the size of the nature conservation areas in proximity to the AES site, and the distance from the site to the nature conservation areas (Cotteleer, 2008). We calculate the spatial cohesion Reilly index for each individual parcel with AESs or the potential for an AES, Ri, hereinafter referred to as Reilly points. In this scenario, information is added with respect to the contribution of the parcel to the long-term persistence of populations within habitat networks in the case study region. This contract type allows for higher payments for parcels that contribute more to the habitat networks. For the first scenario discussed in Section 3, we assume a fixed annual AES compensatory payment per hectare, independent of location and spatial configuration of the landscape. In the second scenario we focus on a flexible compensatory payment per hectare, based on the Reilly points for the particular contracted parcel. Subsequently, we discuss the results for each sensitivity analysis approach, given the two scenarios, and compare both.

Fig. 3. Sensitivity of three outputs to 23 parameters in policy scenario 1, according to the OAT analysis; the dark grey, upper, bars depict the values of the outputs for the maximal value for each parameter; the light grey, lower, bars depict the values of the outputs for the minimal value for each parameter, while all other parameters are set at their normative values. The dashed line represents the output value for the nominal value of each of the 23 parameters.

added, namely the number of Reilly points of contracted parcels with AESs.

3.2. Monte Carlo sensitivity analysis results 3.2.1. Mean gross margin For the Monte Carlo sensitivity analysis results of the first scenario we start with the analysis of gross margin, which is the sum of

3. Results This section summarizes the results of the sensitivity analyses comparing both policy scenarios. A more extensive overview of the results is available on the web.2 3.1. One-at-a-time sensitivity analysis results In the present subsection, the one-at-a-time sensitivity analysis results are presented for both policy scenarios and compared. Fig. 3 presents the results of the OAT analyses for the first policy scenario, given the three outputs mentioned in Section 2.3. Fig. 4 presents results of the OAT analyses for the alternative scenario in which the AESs apply. Now, an additional output is

2 http://www.wageningenur.nl/en/Expertise-Services/Chair-groups/SocialSciences/Agricultural-Economics-and-Rural-Policy-Group/Publications/SERAmodel.htm.

Fig. 4. Sensitivity of four outputs to 23 parameters in the Reilly-points based policy scenario, according to the OAT analysis; the dark grey, upper, bars depict the values of the outputs for the maximal value for each parameter; the light grey, lower, bars depict the values of the outputs for the minimal value for each parameter, while all other parameters are set at their normative values. The dashed line represents the output value for the nominal value of each of the 23 parameters.

202

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

the contributions of individual parcels to the farm’s gross margin. Straightforward sensitivity analysis based on a linear meta-model results in 86.8% of the variance accounted for. Table 2 presents the top and bottom marginal variances of parameters where either TMV or BMV exceeds 1%. Variance in the mean gross margin is for 76.6% due to variation in the feed required per cow. This means that without good information about the feed required per cow parameter, 76.6% of the variance in the mean gross margin will remain, so it is of utmost importance to have an accurate estimate of this parameter. Fig. 5 shows the scatter plot of mean gross margin against feed required per cow. There is a negative linear correlation between the parameter and the output, as the dots are concentrated around a trend with a negative slope. Straightforward sensitivity analysis based on a linear fit of the mean gross margin in the Reilly Points based scenario (Scenario2) results in 90.4% of the variance accounted for, which is much higher than in the initial scenario. When looking at the top and bottom marginal variances, it is shown that results of this scenario are similar to those for the initial policy scenario (see Table 2).

Fig. 5. Scatterplot of mean simulated response of mean gross margin versus parameter feed required per cow in Scenario 1.

3.2.2. Mean farm area (in hectares) We continue with the analysis of the mean farm area, in hectares, which is equal to the sum of the parcels used by each individual farm agent. For the initial scenario, a straightforward sensitivity analysis based on a linear fit results in 24.8% of the variance accounted for. Since 75.2% of the variation is not explained, several other models are tried, like polynomial models (second and third order), models taking into account second and third order interactions and conditional logit interaction terms, including smoothing splines and three degrees of freedom (36.7%). The highest variance accounted for is achieved using a smoothing splines fit with five degrees of freedom, resulting in an accounted variance of 42.3%. Table 3 reports TMV and BMV for three degrees of freedom without interaction terms (39.2%). Fig. 6 shows the scatter plot of mean farm area against feed required per cow. This plot illustrates the influence of strong parameter interactions and nonlinearity, as the dots are widely spread across the graph in separate clusters. For the alternative scenario, the results are similar to the initial policy scenario (see Table 3 for smoothing splines with 3 degrees of freedom with 40.1% of variance accounted for). Straightforward sensitivity analysis based on a linear fit results in an unsatisfactory percentage of variance accounted for (25.3%). The highest accounted-for variance is achieved using a smoothing splines fit with five degrees of freedom, resulting in an accounted variance of 43.5%.

sensitivity analysis based on a linear fit results in 28.4% of the variance accounted for. The percentage of variation explained is unsatisfactory. A third order polynomial model with smoothing splines (df ¼ 3) also resulted in unsatisfactory explanation of variation (36.1%). When using log transformations with a smoothing spline (df ¼ 3), a higher variance accounted for is realized (57.9%). The highest variance accounted for is realized when selecting the 6 parameters presented in Table 4. Taking into account their interactions to all degrees, results in an explained variance of 67.4%. Without interactions, they account for only 28.1% of variance. Table 4 presents the top and bottom marginal variances of the six most important parameters (accounting for >1% of the variation). We conclude that the parameters in Table 4 are the most significant parameters for this output and that they have complex, nonlinear, and conditional interactions. Fig. 7 shows the scatter plot of mean number of hectares contracted for AESs against compensatory payment for AESs. When applying a straightforward sensitivity analysis based on a linear fit to the mean number of hectares contracted for AES, this results in an accounted variance of respectively 82.4%. The higher variance accounted for is the result of the alternative compensatory payment, based on Reilly points, which results in higher payments per hectare and larger areas contracted. Table 4 presents the top and bottom marginal variances for this output. Table 4 shows an increased sensitivity for feed buy price under the second scenario. This relation illustrated by Fig. 8.

3.2.3. Mean number of hectares contracted for agri-environment schemes Then we continue with the analysis of the number of hectares contracted with an AES. For the initial scenario a straightforward

3.2.4. Total number of Reilly points Finally, we present the results for the number of Reilly points contracted on parcels with AESs. A straightforward sensitivity

Table 2 Top Marginal Variances and Bottom Marginal Variances of parameters as percentage of the total variance of mean gross margin in both scenarios.

Table 3 Top Marginal Variances and Bottom Marginal Variances of parameters as percentage of the total variance of mean farm area in both scenarios.

Parameter

TMV and BMV for scenario 1 TMV(%)

Feed sell percentage Milk production per cow Milk price Feed required per cow

TMV and BMV for scenario 2 BMV(%)

TMV(%)

Parameter BMV(%)

1.7 4.4

0.7 3.8

1.8 4.4

0.7 3.8

4.8 76.1

5.3 76.6

5.0 79.9

5.5 79.6

Feed sell percentage Fixed transport cost Nitrogen production per cow Feed buy price Feed required per cow Expectation percentage

TMV and BMV for scenario 1

TMV and BMV for scenario 2

TMV(%)

BMV(%)

TMV(%)

BMV(%)

2.0 6.0 3.7 1.7 19.5 4.3

2.2 7.1 4.5 1.7 19.3 4.0

2.1 6.5 3.6 1.8 19.5 4.3

2.4 7.6 4.5 1.7 19.3 3.9

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

Fig. 6. Scatter plot of mean simulation response of mean farm area versus parameter feed required per cow.

analysis based on a linear fit to the outputs resulting in an accounted variance of respectively 89.3%. We conclude that the percentage of variation explained is satisfactory. Tables 5 present the top and bottom marginal variances. Fig. 9 shows the scatterplot of mean contracted Reilly points against compensatory payment for AES. The relation is clear, also in Table 5; budget constraints limit the total number of Reilly points to be contracted. 4. Discussion The results of both approaches to sensitivity analysis show that the different outputs of the simulation are sensitive to different parameters. For instance, Fig. 3 shows that - The output mean gross margin is mostly affected by variations in feedRequiredPerCow, milkPrice, manureBuyPrice, and milkProductionPerCow. At the maximal value and the nominal value of ManureBuyPrice the values of this output are equal, while the output at the minimal value is much lower. This indicates a nonlinearity caused by conditional interactions with other parameters. - The output total area seems to be sensitive to parameters related to transport costs, feed requirements for dairy cows, and prices of cattle feed. However, it is interesting to see that for some parameters, for instance milkPrice, the values of the output at the extreme values of the parameter deviate from the average value, which indicates a nonlinear sensitivity of the output for this parameter.

203

Fig. 7. Scatterplot of mean simulation response of contracted AES area versus parameter compensatory payment for agri-environment scheme (AESpayment).

- The parameters AESpayment (the annual payment per ha), AESContractPeriod, and feedRequiredPerCow affect the contracted area. Parameter AESpayment does so most dominantly. In addition, the sensitivity is shown to be context-dependent. When comparing Fig. 4 with Fig. 3, we immediately see that there are differences in sensitivity between the two scenarios: - The mean gross margin shows to be sensitive to parameters as in the initial policy scenario, namely milk price, milk production and feed requirements for dairy cows, but not for manureBuyPrice, for which the output is only conditionally sensistive in the initial scenario. - The mean farm area also shows to be sensitive to the same parameters as in the initial policy scenario; namely transport costs, feed requirements for dairy cows and prices of cattle feed, and again the results indicate nonlinearities. - Contracted area shows to be sensitive to the size of the compensatory payment as in the first scenario, but this simulation is sensitive to feed price and not to feedRequiredPerCow. The explanation is that in contrast to the first scenario, it may be attractive for farmers to offer highly productive land for agrienvironmental schemes in the second scenario.

Table 4 Top Marginal Variances and Bottom Marginal Variance of parameters as percentage of the total variance of the mean number of hectares contracted for agrienvironment schemes for both scenarios. Parameter

Feed sell percentage Feed buy price Nitrogen sell price Fertilizer grassland Feed required per cow Compensatory payment agri-environment scheme

TMV and BMV for scenario 1

TMV and BMV for scenario 2

TMV(%)

BMV(%)

TMV(%)

BMV(%)

2.7 13.0 1.0 2.1 0.7 12.1

2.4 15.6 1.2 3.0 2.3 12.8

4.4 19.3 0.9 2.8 2.6 45.6

3.8 23.0 1.3 3.7 5.2 48.6

Fig. 8. Scatterplot of mean simulated response of mean contracted AES area versus parameter feed buy price.

204

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

Table 5 Reilly points on parcels contracted with AESs in Scenario 2: Top Marginal Variances and Bottom Marginal variance of parameters as percentage of the total variance. Parameter

TMV(%)

BMV(%)

Feed sell percentage Feed buy price Nitrogen sell price Fertilizer grassland Feed required per cow Compensatory payment agri-environment scheme

2.5 15.7 0.3 1.5 6.2 55.3

2.2 19.8 0.5 2.5 9.8 59.0

These results indicate that it is relevant to perform sensitivity analysis for each policy scenario to be assessed with an ABM. Sensitivity may depend on the policy scenario. It is surprising that the mean contracted AES area seems insensitive to milk price, while according to Schouten et al. (2013a) milk price has its effects. However, this sensitivity occurs only at milk price levels deemed unrealistic according to experts. Budgets for AES contracts are limited. Farmers offer their least productive parcels for AES. The break-even point where it would pay off to use these parcels for feed production lies far beyond the limits set by AES budget and realistic milk price expectations. When comparing the results of the OAT with the Monte Carlo analysis of the mean gross margin, the sensitivity for manure buy price, which was found in the OAT, is not found in the Monte Carlo analysis (Table 2). The reason is that this sensitivity is conditional upon other parameters and occurs only at low values of manure buy price. Here we have a first warning that the Monte Carlo approach with meta-modelling will not detect sensitivities in rare or exceptional situations. On the other hand, the OAT analysis in Fig. 4 does not identify nitrogen production per cow as a relevant parameter, but mean farm area shows sensitive to it according to the Monte Carlo results in Table 3. Fig. 7 presents an example where the meta-modelling in the Monte Carlo analysis is hampered by the great many cases in which very little parcels are offered for AES. It is clear that, according to the model, payments should be approximately V2000/ha or more in this region in order for AES to be economically attractive for farmers, but even then adoption is sparse and depending on other conditions. Fig. 7 also illustrates the relevance of this type of extensive sensitivity analysis of agent-based simulations for policy support. Policy makers may be interested in the exceptional situations for purposes of risk estimation or policy leverage. When comparing the Monte Carlo results of the initial policy scenario with the Reilly points scenario we see similar results in Tables 2and 3, but Table 4 shows that a different payment scheme, namely based on acquired Reilly points, results in more sensitive behaviour of the parameters with respect to that policy scenario (amount of the annual compensatory payment). Furthermore, the variance accounted for is much higher in this scenario. We see that the number of Reilly points is not so sensitive anymore to changes in feed requirements for dairy cows, but is more sensitive to the level of the feed price in Scenario 2. Table 4 and Fig. 8 show a clear relation between feed buy price, and the number of hectares contracted for AESs. Whenever the feed buy price increases, the number of contracted hectares for AESs decreases. In practice, this phenomenon is also observed. Whenever prices for cattle feed increase, farmers switch parcels with AESs to conventional farming (i.e. production of grass) more easily. According to the sensitivity analysis policy makers will have more grip on the contracted area in that scenario than in the initial scenario. This is further illustrated by Fig. 9. Fig. 9 shows that the more budget there is available, the more Reilly points are generated on parcels contracted with AESs.

5. Conclusion This paper compares two different methods of sensitivity analysis carried out for two different policy scenarios in an agent-based simulation. The main conclusion is that a combined application, exploiting the complementary strengths of the methods, provides good insights into the simulation’s sensitivity for parameter variations. The one-at-a-time approach rapidly provides a clear and comparable overview of the results found for different outputs under different scenarios. The Monte Carlo approach provides a deeper insight into the sensitivity of outputs for parameter variations and gives insight into the nonlinearities and parameter interactions by means of meta-modelling. A shortcoming is that none of the methods support in-depth analysis of the regions in parameter space where exceptional system behaviour emerges due to conditional parameter interactions. These regions are of particular interest for policy support. The present section discusses the contributions, complementarities and black spots of the applied methods with regard to several purposes of sensitivity analysis and draws conclusions about the use of sensitivity analysis in policy support with ABMs. Sensitivity analysis is generally recognized to serve several purposes. A first purpose is to verify the simulation’s implementation by comparing actual effects of parameter variation with expected effects. Secondly, for simulations of complex systems with hard-to-predict behaviour, sensitivity analysis deepens the researcher’s insight into the behaviour of the simulation and the simulated system. Thirdly, quantifying and comparing the effects of changes in each parameter helps to set priorities for model calibration. Fourthly, such a comparative, quantitative, analysis may suggest model simplifications if some parameter variations have little effect (model reduction). Fifthly, in particular for ABMs, a global sensitivity analysis can identify areas in parameter space where rare or exceptional outcomes occur. With respect to the first purpose (to verify the implementation) the one-at-a-time approach proved useful. It presents a quick overview of sensitivities, as in Figs. 3 and 4. Surprising outcomes, such as the insensitivity of contracted AES area for milk price could be explained, which increases the confidence in the correctness of model implementation. The analysis gives a first indication of nonlinearities, for instance in the sensitivity of mean farm area to milk price and the sensitivity of mean gross margin to manure buyprice (both in Fig. 3). Furthermore it allows for rapid comparison of sensitivities of different scenarios (e.g., the surprising difference in sensitivity of contracted AES area for feed price between Figs. 3 and 4). Comparison of the one-at-a-time results with those of the Monte Carlo approach provides extra hints for implementation checking. For instance, the sensitivity of the mean gross margin for manure buy price found in the one-at-a-time analysis (Fig. 3) is not found in the Monte Carlo analysis (Table 2). This is a warning that the Monte Carlo and meta-modelling approach may not be relied upon to detect all sensitivities, and that it must be complemented with another method. For the second purpose (to gain insight into the system’s properties) the Monte Carlo approach proved helpful. A first overview of scatterplots for each parameter against each output provided a deeper insight but also raised questions, the answering of which again deepened the insight. Interpreting the results of the sensitivity analysis (top marginal variance and bottom marginal variance) and finding a meta-model that sufficiently takes the interactions into account are time-consuming exercises. For the complex system studied, it took several months of the researchers’ time.

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

Fig. 9. Scatterplot of mean simulation response of Reilly points versus parameter compensatory payment for agri-environment schemes (AESpayment).

The Monte Carlo approach which we applied was not originally developed for ABMs and, as a continuous meta-modelling approach, it lacks the capability to handle rare events and tipping points. However, taking into account the current lack of more sophisticated methods for global sensitivity analysis of ABMs, the meta-modelling approach has provided valuable insights into the properties of the simulation and of the simulated system. The one-at-a-time approach provided a rough insight into the contributions of parameter variations, which is the third purpose. The Monte Carlo approach provided quantified evaluations, but as mentioned before, it missed information about parameter interactions. Therefore it seems good to combine the approaches: use the Monte Carlo approach to assess global sensitivities and complement it with the one-at-a-time approach for parameter settings for specific simulations to discover sensitivities not found in the global analysis. For the present simulation the combined sensitivity information is interpreted as follows. The outputs proved particularly sensitive to feed required per cow, prices of inputs and outputs, subsidy levels, and parameters related to the cost of transportation. Accuracy in other parameters seems to be less relevant, so we focus on the mentioned parameters. Prices and subsidy levels are the inputs to the policy scenarios to be evaluated with help of the simulation. Sensitivity to these inputs is a desired property of the simulation. Researchers and policy makers can define their setting for relevant scenarios. For transportation cost, reasonable estimates can be made by experts and policy makers. Feed required per cow is a technical parameter that greatly affects the outcomes. Fortunately, there is a significant amount of literature available on this topic in which the relations to breed of cattle, lactation period, stable type, and grazing method are extensively discussed (Tamminga et al., 2004). We made assumptions about these determinants for the present case study. So, we can sufficiently assess the values of the relevant parameters. The generally recognized fourth purpose of sensitivity analysis (to simplify the model by leaving out factors that make no relevant difference in the outcomes) is in contradiction to our aim to keep the model descriptive. A special point of attention here is the validity of sensitivity analysis results. A region’s actual data of landscape and population with which the model is loaded will probably affect the outcomes. Furthermore, we found differences in sensitivity when simulating different policy scenarios, which is another indication that the sensitivity is context-dependent. This finding has two consequences. First, care must be taken with model

205

reduction based on sensitivity analysis because insensitivity may occur only in specific contexts. Second, when applying the simulation in another context, the sensitivity analysis must be repeated. The importance of the fifth purpose of sensitivity analysis for ABMs (to discover exceptional situations) is emphasized by the research reported in this paper. Systematic sensitivity analysis of an ABM can identify settings in which rare or exceptional results occur. E.g., Fig. 8 indicates that the effects of policy measures as agrienvironment schemes largely depend on other factors than just the level of financial compensation. In-depth analysis of exceptional cases found in the Monte Carlo analysis may inform decision makers about opportunities for policy leverage, such as influencing certain parameters or selecting the best moment to implement a policy. However, the general methods applied in this research do not in a systematic way provide in-depth analysis of the circumstances in which exceptional outcomes can emerge. In this research we applied two general methods of sensitivity analysis. We have shown that using the complementarities of several methods improves the understanding of an agent based simulation. We have also shown how it can improve the understanding of the options for policy interventions in the simulated system. Therefore, developing tailored methods for sensitivity analysis of ABMs and discovery and analysis of exceptional situations deserves a place on the research agenda, not just for its scientific relevance, but also for its potential to improve policy support by means of ABMs.

Acknowledgements This research is part of the strategic research program ‘Sustainable spatial development of ecosystems, landscapes, seas and regions’ which is funded by the Dutch Ministry of Economic Affairs, and carried out by Wageningen University and Research Centre. We are indebted to many of our colleagues for discussions and ideas that are reflected throughout this paper.

Appendix A In the following, the structure, concepts and details of the model are presented using the ODD (Overview, Design concepts and Details) protocol (Grimm et al., 2006; Polhill et al., 2008; Grimm et al., 2010). Detailed documentation is available on the website.3

Purpose The core of the model is the understanding and modelling of an agent-based system for the purpose of agricultural policy assessment while simulating the agents and their corresponding landscape in a spatially explicit way. The model establishes a virtual world of a rural region and comprises a large number of individually acting farms that operate in this region, as well as farms interacting with each other and with parts of their environment. The model implements an abstract region, that can be initialized with empirical data on individual farms and existing agricultural spatial structures. In the following, we present a description of the single entities of the model and describe their relationships.

3 http://www.wageningenur.nl/en/Expertise-Services/Chair-groups/SocialSciences/Agricultural-Economics-and-Rural-Policy-Group/Publications/SERAmodel.htm.

206

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

State variables and scales

Process overview and scheduling

Within the model, the following hierarchical levels can be distinguished:

The model proceeds in annual time steps. Fig. A.1 summarizes the main logical steps of the model. The model consists of an initialization phase in which data is conditioned for use in the model; a simulation phase in which farm gross margins are calculated and land is distributed among farmers (land auction mechanism), and an output phase. The initialization module contains exogenous agricultural census data. The attributes on farm level are the farm structure, given in age, type of farm, size and number of owned parcels. At parcel level, attributes are soil quality and land use. At landscape level, attributes are number of farms in the region, spatial landscape characteristics (i.e. rivers, roads, nature conservation areas, nearby towns), size and distance. Within the simulation phase, each farm agent is equipped with a behavioural model that guides decisions and keeps track of the agent’s internal state described by attributes such as age, location and size. According to their behavioural model, the individual farm agents evolve subject to their state of attributes and to changes in their environment. The results of the valuation module are merged in the land market mechanism. Finally, the function of the output module is the conditioning of the model results for the next simulation period. Results on farm level as well as on the regional level are used for updating farm attributes and regional attributes in the next period. Fig. A.2 depicts the organization of the model framework at the program level by providing the conceptual class diagram of the model with the main classes and their relations. Fig. A.2 shows that two types of agents are distinguished, the Trader Agent and the Auctioneer. Trader Agent describes all agents that can be traders, and that wish to trade parcels of land. Farmers in the model are such trader agents. Decision making strategies are described in the class DecisionMakingStrategy. Every TraderAgent has a ValuationStrategy which is used to determine the valuation for a parcel. The resulting value of a parcel is expressed by the class ValuationStrategy. Individual parcels are represented by the class FarmLand, that holds all information on the parcels that belong to

 Farm agents representing dairy farm households, with as many agents modelled as there are dairy farm households in reality. State variables of the agents include the location of the agent’s farmstead, the location of their fields, the individual household composition (age, successor) available resources such as cash, livestock, nitrogen balance and feed balance of the farm.  Spatially explicit farm land representing all spatially explicit parcels as well as their current land use. For each parcel the size, shape and current land use is known and the distance to the homestead is calculated. Also, possibilities for AESs are known, as explained in the previous section. A basic principle of AESs in the EU is that the participation of individual farmers is voluntary. In the simulation, farm agents base their contract choice on farm production, economic results, intensity of land use, land quality, and spatial characteristics of the respective parcel. For the sensitivity analyses we assume a fixed annual AES compensatory payment per hectare, independent of location and spatial configuration of the landscape. This is in line with current European AES programmes. Furthermore, the nitrogen production by grazing cows is calculated for each parcel, as well as the amount of feed produced on the parcel (differentiated by soil quality) and its contribution to the total feed production of the farm.  Regional land market: Where the auctioneer describes the mediation between traders of land on the regional land market. Finally, all other landscape elements such as large creeks, nature conservation areas, and nearby towns are modelled and can be included into the spatial extent of the system. Government interventions are treated as external in the model. The decision models’ parameters included in the sensitivity analysis are listed in Table A.1.

Table A.1 Model parameters and nominal values that stay constant throughout the simulation; the table also specifies the value range considered in the sensitivity analysis. Parameter

Unit

Nominal value

Value range Min

Max

Fertilizer price Feed sell percentage Manure buy price Manure buy percentage Fixed transport costs Transport costs coefficient Fixed Transport costs agri-environmental parcel Transport costs coefficient agri-environmental parcel Milk production per cow Nitrogen production per cow Milk price Feed buy price Nitrogen sell price Fertilizer grassland Feed required per cow Price expectation parameter Agri-environment compensatory payment Contract period agri-environment scheme Nitrogen application maize land Feed production maize land Fertilizer maize land Maximum number of cows per hectare Expectation percentage

V/kg N % V/kg N % V/ha V/km V/ha V/km kg kg N V/kg V/net energy for lactation V/kg N kg N Net energy for lactation % V/ha Year kg N Net energy for lactation kg N Cows %

0.7 90 1.8 80 50 50 20 20 7875 115 0.31 1.34 2 35 6374.8 0.5 1018 6 150 1000 62.78 3 0.7

0.5 0.5 1.25 0.5 15.81 15.81 6.32 6.32 5000 50 0.2 1 1 11.07 4000 0 321.92 2 47.43 500 40 2.5 0.5

1 1 2.5 1 158.113 158.11 63.25 63.25 12,000 150 0.5 3 4 110.68 10,000 1 3219.19 12 474.34 1500 80 3.5 1

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

207

Fig. A.1. Model flowchart.

the farms. Farmlands hold crops (class Crop), and a certain quality (emuneration ParcelQuality). Two implementations currently exist: Maize and Grass. Farmlands can also hold agri-environment schemes (class AES Contract). The other agent currently in the model is the Auctioneer. The Auctioneer is the mediator between traders. The Auctioneer ‘requests’ traders to make offers to either express their willingness to buy or sell a good. The Auctioneer makes use of a mechanism to match bids and asks to clear the market. This is represented by the class AuctionMechanism. It presumes that multiple buyers and sellers are present and the parcels traded are heterogeneous (characterised by multiple attributes). Design concepts Emergence: Land use patterns emerge in response to farm agent behaviour on the land market. Also changes to neighbouring nature areas as well as policy changes and market disturbances may result in emerging changes of land use patterns. Adaptation: Farm agents adjust their resource management (land, nature, livestock) in response to transactions on the land market, policy changes and changing market conditions. For instance, if output prices, such as milk prices, show a large volatility over a certain period, farm agents can chose for alternative income strategies by contracting AESs. Prediction: Agents form expectations about market prices based on past experience, following the theory of adaptive expectations. Agents revise their expectations with respect to output prices periodically by calculating expected prices for land which are used in the decision making process (following Kellermann et al., 2007). They are used in the form of a weighted moving average of the prices in the past periods. Agenteagent interaction: Interactions between agents takes place on the land market. Farm growth in agriculture is often binding to the availability of land, either as direct production input

(e.g. for crop production) or indirect (e.g. if animal production is tied to the availability of land for manure disposal). Land markets are used as a platform to allocate land (Kellermann et al., 2008). For illustrative purposes in this paper, the land market for dairy farmers is modelled as a closed market which means that only dairy farmers participate. Agent-environment interaction: The interactions between farm agent decision making and agro-ecology is represented by grassland yields, maize land yields (feed balance) and manure disposal (nutrient balance). Grassland yields and maize yields are constrained by the yield potential which captures land quality factors during simulation. They are also constrained by the available nutrients. This means that whenever the nutrient balance on farm level shows a surplus, farmers will use all the legally permitted nutrients on their land and will have to dispose the remainder of their manure on the external market. Agent decisions can also have a direct impact on the environment through applying optional agricultural nature conservation on their parcels. Stochasticity: Stochasticity is used in one of the two implemented auction mechanisms to simulate imperfect information of potential suppliers and attractors of land on the land market. Furthermore, random numbers are combined with probabilities to simulate trade-offs in the farm agent decision making process. Initialization and input data At the start of each run, initial values of the state variables as well as the farm agents and their environment are uploaded. As its basis, the model uses Agricultural Census data from the Dutch Ministry of Economic Affairs. Data on biophysical components and institutional aspects of land ownerships stems from GIS-cadastral maps with current land use, ownership and soil quality. Model input data are organized in two modules. First of all, farm survey data is used to characterize the individual agents, their

208

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

Fig. A.2. Simplified UML class diagram: program classes with main attributes and relations (Schouten et al., 2012); reproduced with permission from Springer.

resource endowments, and the size of their farm. Then, spatial data is used to locate the farmsteads, the agricultural parcel, the nature conservation areas, the potential parcels for AES, the current land use and soil quality. Submodels Farm agent decision making: At the core of the model is the valuation module that simulates the decision making of individual farm households. Within each time step the resource land as well as expectations about land market prices are updated. Because the model deals with parcels that are heterogeneous in size, quality and shape, the model does not deal with a conventional farm optimization problem. We have created a constrained optimization, in which farmers take into account these parcel characteristics. In each period farm agents calculate the gross margin for both their grassland and maize land parcels, as well as for new parcels available on the land market. For each parcel revenues are calculated (i.e. revenue from milk production, feed production, manure disposal and subsidies) and a set of costs (i.e. transport costs, manure disposal costs, costs for buying feed etc.) is subtracted.

What remains are the gross margins per parcel. The farm agent compares the gross margins for the offered parcels and has the opportunity to bid on offered parcels whenever their bid price is higher than the reserve price. Next to bidding on the land market for new parcels, the farm agent makes a trade-off between three types of land use for the parcels that are in use: conventional grassland, maize land, and possibilities for agri-environment contracting. Whenever an AES is chosen, this type of land use remains on the parcel during the contract period. The following tradeoffs can be summarized: - Parcel has no possibility of AES contracting. Choice is made either to offer the parcel to the land market due to high opportunity costs, or maintain conventional farming. - Parcel is not yet contracted, but has possibility for AES. Farm agent makes a tradeoff between conventional farming, AESs, or offering the parcel to the land market because of high opportunity costs. - Parcel is contracted with AES, but contract expires. Choice whether to extend contract, switch to conventional farming or offer the parcel to the land market due to high opportunity costs.

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

For each tradeoff the expectation price is included in the determination. Whenever a farm agent choses an AES, an extra restriction on manure and fertilizer application is applied to the parcel which affects its contribution to gross margin. Biophysical modules: The model simulates resource dynamics and crop yields in the following way. First, the decision-making module simulates land use based on expected conditions (soil quality and expectation prices). Then actual resource conditions and actual crop yields are calculated for each parcel. Finally, the actual crop yield is transferred back to the agent decision module where it is evaluated and used to update the expectations. The real-world landscape is represented by actual parcels, heterogeneous in size, quality and land use. Spatial information was organized in layers, including the location of the parcels, the location of the farm steads and parcel and soil properties. For this paper, we distinguish between four types of land, namely conventional grassland, grassland with possibilities for AES, maize land, and nature conservation areas. Informed single auction market mechanism: Agents in the model interact indirectly by competing on a land market. The land market is the central interaction institution between agents in the model and is fully endogenous in the model. There is a broad range of applications of land market mechanisms within ABMs available for social sciences (see Parker et al., 2003; Parker and Filatova, 2008; Filatova et al., 2011; Kellermann et al., 2008; Magliocca et al., 2011). Our approach has in common with other approaches that land allocation is modelled through an auction mechanism in which competition for land is based on a defined bidding process and a set of rules for price determination and ‘matching’ of asks and bids. However, it differs from existing approaches in that farm agents are informed in advance whenever multiple parcels are offered simultaneously. They are informed on several attributes of the parcel: soil quality, size, current land use and distance to the homestead. In the model, farm agents extend their hectare base exclusively so called perpetual lease contracts. This implies that the farm agent is free to offer the parcel again to the land market or remain farming after each simulation period. When the model is run, available perpetual lease parcels stem from two sources: one is that farms offer their land to the market due to high opportunity costs, the second one is retirement of farmers when they reach the age of 65 and do not have a successor (see Schouten et al., 2012; for an extensive overview of the auction mechanism used). References An, L., 2012. Modelling human decisions in coupled human and natural systems: review of agent- based models. Ecol. Model. 229, 25e36. Balmann, A., 1997. Farm-based modelling of regional structural change: a cellular automata approach. Eur. Rev. Agric. Econ. 24, 85e108. Becu, N., Perez, P., Walker, A., Barreteau, O., Page, C.L., 2003. Agent based simulation of a small catchment water management in northern Thailand: description of the CATCHSCAPE model. Ecol. Model. 170, 319e331. Berger, T., 2001. Agent-based spatial models applied to agriculture: a simulation tool for technology diffusion, resource use changes and policy analysis. Agric. Econ. 25, 245e260. Berkes, F., Colding, J., Folke, C., 2003. Navigating Social-ecological Systems: Building Resilience for Complexity and Change. Cambridge University Press, Cambridge, UK. Bousquet, F., Bakam, I., Proton, H., Le Page, C., 1998. Cormas: common-pool resources and multi-agent systems. Lecture Notes in Artificial Intelligence 1416, 826e838. Bousquet, F., Le Page, C., 2004. Multi-agent simulations and ecosystem management: a review. Ecol. Model. 176, 313e332. Brown, D.G., Page, S., Riolo, R., Zellner, M., Rand, W., 2005. Path dependence and the validation of agent-based spatial models of land use. Int. J. Geogr. Inform. Sci. 19, 153e174. Burgers, S.L.G.E., Hofstede, G.J., Jonker, C.M., Verwaart, T., 2010. Sensitivity analysis of an agent-based model of Culture’s consequences for trade. In: Calzi, M.L., Milone, L., Pellizzari, P. (Eds.), Progress in Artificial Economics: Computational and Agent-based Models, Lecture Notes in Economics and Mathematical Systems. Springer-Verlag, Berlin Heidelberg, pp. 253e264.

209

Campolongo, F., Kleijnen, J., Adres, T., 2000. Screening methods. In: Saltelli, A., Chan, K., Scott, E.M. (Eds.), Sensitivity Analysis. Wiley, Chichester. Castella, J.-C., Boissau, S., Trung, T.N., Quang, D.D., 2005. Agrarian transition and lowlandeupland interactions in mountain areas in northern Vietnam: application of a multi-agent simulation model. Agric. Syst. 86, 312e332. Cotteleer, G., 2008. Valuation of Land Use in the Netherlands and British Columbia; a Spatial Hedonic GIS-based Approach. Wageningen University, Wageningen. Dennis, R.L., Arnold, J.R., Tonnesen, G.S., 2000. One-at-a-Time and mini-global analyses for characterizing model sensitivity in the nonlinear ozone predictions from the US EPA regional acid deposition model (RADM). In: Saltelli, A., Chan, K., Scott, E.M. (Eds.), Sensitivity Analysis, Wiley series in probability and statistics. Filatova, T., Voinov, A., van der Veen, A., 2011. Land market mechanisms for preservation of space for coastal ecosystems: an agent-based analysis. Environ. Model. Softw. 26 (2), 179e190. Filatova, T., Verburg, P.H., Parker, D.C., Stannard, C.A., 2013. Spatial agent-based models for socio-ecological systems: challenges and prospects. Environ. Model. Softw. 45, 1e7. Goedhart, P.W., Thissen, J.T.N.M., 2009. In: Biometris (Ed.), Biometris Procedure Library for Genstat, twelfth ed. Wageningen. Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S.K., Huse, G., Huth, A., Jepsen, J.U., Jørgensen, C., Mooij, W.M., Müller, M., Pe’er, G., Piou, C., Railsback, S.F., Robbins, A.M., Robbins, M.M., Rossmanith, E., Rüger, N., Strand, E., Souissi, S., Stillman, R.A., Vabø, R., Visser, U., DeAngelis, D.L., 2006. A standard protocol for describing individual-based and agent-based models. Ecol. Model. 198, 115e126. Grimm, V., Berger, U., DeAngelis, D.L., Polhill, J.G., Giske, J., Railsback, S.F., 2010. The ODD protocol: a review and first update. Ecol. Model. 221, 2760e2768. Groeneveld, R.A., Bouwman, A.F., Kruitwagen, S., 1998. An Environmental and Economic Model to Assess the Abatement Costs of Nitrate Leaching in Dairy Farming. RIVM, Bilthoven. Groeneveld, R., Bouwman, L., Kruitwagen, S., van Ierland, E., 2001. Land cover changes as a result of environmental restrictions on nitrate leaching in dairy farming. Environ. Model. Assess. 6 (2), 101e109. Hamby, D.M., 1994. A review of techniques for parameter sensitivity analysis of environmental models. Environ. Monit. Assess. 32, 135e154. Happe, K., 2005. Agent-based modelling and sensitivity analysis by experimental design and metamodelling: an application to modelling of regional structural change. In: XIth International Congress of the European Association of Agricultural Economists, the Future of Rural Europe in the Global Agri-food System. Copenhagen, Denmark. Happe, K., Kellermann, K., Balmann, A., 2009. Agent-based analysis of agricultural policies: an illustration of the agricultural policy Simulator. Ecol. Soc. 11, 49. Jager, W., Janssen, M.A., Vlek, C.A.J., 2002. How uncertainty stimulates overharvesting in a resource dilemma: three process explanations. J. Environ. Psychol. 22, 247e263. Jakeman*, A.J., Letcher, R.A., Norton, J.P., 2006. Ten iterative steps in development and evaluation ofenvironmental models. Environ. Model. Softw. 21, 602e614. Jansen, M.J.W., Rossing, W.A.H., Daamen, R.A., 1994. Monte Carlo estimation of uncertainty contributions from several independent multivariate sources. In: Grasman, J., van Straten, G. (Eds.), Predictability and Nonlinear Modelling in Natural Sciences and Economics. Kluwer, Dordrecht, pp. 334e343. Kellermann, K., Happe, K., Sahrbacher, C., Brady, M., 2007. In: IAMO (Ed.), AgriPoliS 2.0-Documentation of the Extended Model. Halle. Kellermann, K., Sahrbacher, C., Balmann, A., 2008. Land markets in agent based models of structural change. In: 107th EAAE Seminar Modeling of Agricultural and Rural Development Policies: Sevilla, Spain. Korevaar, H., Geerts, R.H.E.M., de Visser, W., Koldewey, E., 2006. Vier jaar multifuctionele gras- en bouwlanden in Winterswijk: gevolgen voor economie en ecologie op de bedrijven. Plant Research International, Wageningen. Lobianco, A., Esposti, R., 2010. The regional multi-agent simulator (RegMAS): an open-source spatially explicit model to assess the impact of agricultural policies. Comput. Electron. Agric. 72, 14e26. Magliocca, N., Safirova, E., McConnell, V., Walls, M., 2011. An economic agent-based model of coupled housing and land markets (CHALMS). Comput. Environ. Urban Syst. 35 (3), 183e191. Matthews, R.B., Gilbert, N.G., Roach, A., Polhill, J.G., Gotts, N.M., 2007. Agent-based land-use models: a review of applications. Landsc. Ecol. 22, 1447e1459. Messina, J.P., Evans, T.P., Manson, S.M., Shortridge, A.M., Deadman, P.J., Verburg, P.H., 2008. Complex systems models and the management of error and uncertainty. J. Land Use Sci. 3 (1), 11e25. Middelkoop, N., Aarts, H.F.M., 1991. De invloed van bodemeigenschappen, bemesting en gebruik op de opbrengst en de stikstofemissies van grasland op zandgrond. CABO-DLO, Wageningen. Nossent, J., Elsen, P., Bauwens, W., 2011. Sobol’ sensitivity analysis of a complex environmental model. Environ. Model. Softw. 26 (12), 1515e1525. Oakley, J.E., O’Hagan, A., 2004. Probabilistic sensitivity analysis of complex models: a Bayesian approach. J. Roal Stat. Soc. Ser. B, 751e769. OECD, 2005. Evaluating Agri-environmental Policies: Design, Practice and Results. Organisation for Economic Co-Operation and Development, Paris. Opdam, P., Pouwels, R., van Rooij, S., Steingröver, E., Vos, C.C., 2008. Setting biodiversity targets in participatory regional planning: introducing ecoprofiles. Ecol. Soc. 13 (1).

210

M. Schouten et al. / Environmental Modelling & Software 54 (2014) 196e210

Opdam, P., Verboom, J., Pouwels, R., 2003. Landscape cohesion: an index for the conservation potential of landscapes for biodiversity. Landsc. Ecol. 18 (2), 113e126. Parker, D.C., Filatova, T., 2008. A conceptual design for a bilateral agent-based land market with heterogeneous economic agents. Comput. Environ. Urban Syst. 32 (6), 454e463. Parker, D.C., Manson, S.M., Janssen, M.A., Hoffmann, M.J., Deadman, P., 2003. Multiagent systems for the simulation of land-use and land-cover change: a review. Ann. Assoc. Am. Geogr. 93 (2), 314e337. Parry, H.R., Topping, C.J., Kennedy, M.C., Boatman, N.D., Murray, A.W.A., 2013. A Bayesian sensitivity analysis applied to an agent-based model of bird population response to landscape change. Environ. Model. Softw. 45, 104e115. Pedroli, B., Wascher, D., Blust, G.d, Paracchini, M.L., Doorn, A.v, 2007. Europe’s living landscapes at a turning point. Introduction in. In: Pedroli, B., Doorn, A.v., Blust, G.d., Paracchini, M.L., Wascher, D., Bunce, F. (Eds.), Europe’s Living Landscapes. Essays Exploring Our Identity in the Countryside. KNNV Publishing, Zeist (the Netherlands), pp. 11e18. Peerlings, J., Polman, N., 2008. Agri-environmental contracting of Dutch dairy farms: the role of manure policies and the occurrence of lock-in. Eur. Rev. Agric. Econ. 35 (2), 167e191. Polhill, G., Parker, D., Brown, D., Grimm, V., 2008. Using the ODD Protocol for describin three agent-based social simulation models of land-use change. J. Artif. Soc. Soc. Simul. 11, 23. Polhill, J.G., Gimona, A., Gotts, N.M., 2013. Nonlinearities in biodiversity incentive schemes: a study using an integrated agent-based and metacommunity model. Environ. Model. Softw. 45, 74e91. Provincie Gelderland, 2005. Streekplan Gelderland 2005: kansen voor de regio’s. Ralha, C.G., Abreu, C.G., Coelho, C., Zaghetto, A., Macchiavello, B., Machado, R.B., 2013. A multi-agent model system for land-use change simulation. Environ. Model. Softw. 42, 30e46. Ratto, M., Castelletti, A., Pagano, A., 2012. Emulation techniques for the reduction and sensitivity analysis of complex environmental models. Environ. Model. Softw. 34, 1e4. Refsgaard, J.C., van der Sluijs, J.P., Højberg, A.L., Vanrolleghem, P.A., 2007. Uncertainty in the environmental modelling process e a framework and guidance. Environ. Model. Softw. 22, 1543e1556.

Reilly, W.J., 1931. The Law of Retail Gravitation. W.J. Reilly, Inc., New York. Richiardi, M., Leombruni, R., Saam, N., Sonnessa, M., 2006. A common protocol for agent-based social simulation. J. Artif. Soc. Soc. Simul. 9 (1). Saltelli, A., Chan, K., Scott, E., 2000. Sensitivity Analysis. Wiley, Chichester. Saltelli, A., Annoni, P., 2010. How to avoid a perfunctory sensitivity analysis. Environ. Model. Softw. 25 (12), 1508e1517. Schouten, M., Polman, N., Westerhof, E., Kuhlman, T., 2012. Rural landscapes in turbulent times: a spatially explicit agent-based model for assessing the impact of agricultural policies. In: Teglio, A., Alfarano, S., Camacho-Cuena, E., GinésVilar, M. (Eds.), Lecture Notes in Economics and Mathematical Systems, Managing Market Complexity. The Approach of Artificial Economics. Springer, Castellón, Spain, p. 247. Schouten, M., Opdam, P., Polman, N., Westerhof, E., 2013a. Resilience-based governance in rural landscapes: experiments with agri-environment schemes using a spatially explicit agent-based model. Land Use Policy 30 (1), 934e 943. Schouten, M., Polman, N., Westerhof, E., 2013b. Exploring Green Agricultural Policy Scenarios with a Spatially Explicit Agent-based Model. Wettelijke Onderzoekstaken Natuur & Milieu. WOt-werkdocument 323, Wageningen. Schreinemachers, P., Berger, T., 2011. An agent-based simulation of humanenvironment interactions in agricultural systems. Environ. Model. Softw. 26, 845e859. Tamminga, S., Aarts, F., Bannink, A., Oenema, O., Monteny, G.J., 2004. Actualisering van geschatte N en P excreties door rundvee. Wageningen Universiteit, Wageningen. Van de Ven, G.W.J., 1992. GRASMOD: a Grassland Management Model to Calculate Nitrogen Losses from Grassland. CABO-DLO, Wageningen. Verburg, P.H., Veldkamp, A., 2005. Introduction to the special issue on spatial modeling to explore land use dynamics. Int. J. Geogr. information Sci. 19 (2), 99e102. Whittingham, M.J., 2011. The future of agri-environmental schemes: biodiversity gains and ecosystem service delivery? J. Appl. Ecol. 48, 509e513. Zhan, C.-s., Song, X.-m, Xia, J., Tong, C., 2013. An efficient integrated approach for global sensitivity analysis of hydrological model parameters. Environ. Model. Softw. 41, 39e52.