sampledoc

Table Of Contents

This Page

ASCEM : Sensitivity Analysis (SA)

  1. ASCEM
  2. Tutorials

ASCEM : Sensitivity Analysis (SA)

Introduction

Objective

The objective of this tutorial is to demonstrate a sensitivity analysis (SA), using the Morris one-at-a-time method. This method is qualitative, and is a useful screening tool to identify non-influential parameters and to rank parameters by their relative sensitivities.

Toolsets/modules exercised

The following Akuna modules are being exercised:

  1. Global SA using Morris one-at-a-time method.
    1. Input parameter selection
    2. Observation definition
  2. Job submission and monitoring.
  3. Visualization of SA output, including plots of relative sensitivities

The following Amanzi modules are being exercised:
  1. Variably saturated flow using Richards equation
    1. Steady-state
    2. Transient
  2. Transient transport

Pre-requisite

Although it is advised to run a Single Run (SR) before launching an SA, it is not required. This tutorial uses the conceptual model Richard-1D-transport located in the tutorial directory under the shared projects directory.

Problem Description

The tutorial problem is a one-dimensional, 3 layer, unsaturated flow and transport problem, where

  1. steady-state unsaturated flow is calculated with a constant infiltration rate, providing the initial conditions for year 1950;
  2. discharge of water and 99Tc occurs at the top of the ground surface between 1950.1-1950.26 y; and
  3. continuing infiltration drives the 99Tc plume downward, to the water table, which is represented by the bottom boundary of the domain.

The sensitivity analysis will assess the impact that the formation permeabilities have on predictions of concentration and pressure.

Initialize SA Toolset

To create an SA:

  1. On the main window (Figure 1), click to highlight the Richard-1D-transport conceptual model  |image0| icon that has been created with the Model Setup tutorial. Alternatively, the conceptual model can be copied from the tutorial directory to the user’s current working directory.
  2. Click image1 to launch the Sensitivity Analysis Toolset. The item SA is not clickable unless a model is selected.

image2
Figure 1. Create a new SA.

3.  A window (Figure 2) that prompts for a SA Name will appear. Give the SA a name and click OK.

image3
Figure 2. Provide a name to the SA

This action launches the Sensitivity Analysis Toolset window (Figure 3). The tabs are the same as the other toolsets (e.g., see SR, PE and UQ tutorials), except for the Analysis Options tab.

image4
Figure 3. SA Toolset window

Setup SA Toolset

Akuna simplifies the setup by providing the user with the ability to import data from an existing analysis, or alternatively all fields can be populated manually. The following provides guidance on importing data into the SA Toolset.

  1. Click image5 on the upper left corner of the SA Toolset window.
  2. Select all sections except for the Visualization and Checkpoint to be imported, as shown in Figure 4. These sections are not imported because plots will be generated at certain locations in the domain, and spatial visualizations will not be needed. Press Ok.
image6
Figure 4. Selecting sections for import

3.  Navigate to an existing SR as shown in Figure 5 (which may be located in a different directory than shown in Figure 5).

4.  Press OK and all the tabs will be filled with the data used in the SR (Figure 6).

image7
Figure 5. Import from a SR run that was previously created.
image8
Figure 6. SA toolset window filled with data imported from SR.

Parameter Setup

The parameters of interest in the sensitivity analysis are setup as follows:

  1. Click the Parameters section and then click image9 to add a parameter, as shown in Figure 7.

  2. The dialog box shown in Figure 8 will appear. Click on |image10| to expand the list of parameters that can be chosen.

  3. For the first parameter, choose

  4. Material Properties > Facies1 > Permeability > x; Material

    Properties > Facies1 > Permeability > y; Material Properties > Facies1 > Permeability > z | simultaneously while holding down the Shift key These three variables will be treated as a single parameter. Click Add.

    1. For the second and third parameters, repeat the previous step with Facies2 and Facies3.
    2. Close the window.
    3. Double click on any of the entries in the table to define properties related to each parameter. Change the various entries to those shown in Figure 9.
    4. All the 3 parameters are examined in the logarithm space and thus the Transformation Type (column 5) is changed to Logarithm. As a result the distributions of the permeabilities are log-normal when the Distribution Type (column 6) is Normal.
image11
Figure 7. Parameters section

image12

Figure 8. Select Parameters From Model Setup window.

image13

Figure 9. Edit the entries of the Parameter table based on entries shown above.

Define Observations

Observations are defined under the Observations tab. Observations have been prefilled since they were defined during model setup and in the existing SR. Observations can be added or deleted in this tab, if desired.

Specify Analysis options

The analysis options are set in the Analysis Options tab:

  1. Choose Global sensitivity analysis for Type.
  2. ChooseMorris one-at-a-time for Method.
  3. Set the Number of Replicates to 6 and press ENTER. The number of replicates is used to calculate the number of simulation runs:

number of simulation runs = (number of parameters + 1) X (number of replicates)

 4.  Set the Grid Resolution to 6 and press ENTER. Parameter increments used in the SA are calculated as:

Parameter increments = (Grid Resolution / [2 X (Grid Resolution – 1)].

The user needs to ensure that the generated histograms are consistent with the desired grid resolution. If the grid resolution is low relative to the number of replicates, duplicate parameter values will result.

  1. The plots on the right will automatically show the distribution of the parameters to be used in the sensitivity analysis, as shown in Figure 10. Note that random number generators are operating system specific, so the parameter distribution obtained may differ from the one shown in Figure 10.
  2. Change the Random Seed or click image14 found at the upper right corner to generate a different set of parameters.

image15
Figure 10. Generate samples for SA analysis.

Job Configuration

The job configuration process is similar to job configuration demonstrated in the SR Tutorial. The only additional step involves specifying the number of Processors and Processors per Tasks. In the SR Tutorial, only a single simulation is run, and the number of Processors and Processors per Tasks are the same. In this SA, 24 different simulations are executed. In this SA, 48 processors are requested because the queuing system will execute the simulations sequentially, on 2 nodes, using 2 processors per simulation, as long as the maximum number of simultaneous simulations is set to 2. Therefore, set Processors to 48 and Processors per Tasks to 2. Since these simulations are small, only 30 minutes of wall clock time is needed (Figure 11).

On the NERSC Hopper computer, only one job can be executed per node, and each node contains 24 processor cores. To run the simulations simultaneously, the total number of processors would need to be 576 (24 simulations x 24 processor per node).

image16

Figure 11. Job configuration of SA simulations

Save the model by clicking image17 at the top left corner. The image18 button will turn green, indicating that all information needed to run the analysis has been provided.

Submit SA Simulations

Click the |image19| button to submit the SA simulation (credencial required). The remaining steps for submission and monitoring of jobs follows the SR Tutorial.

Analyze SA Results

When the SA simulations are completed, the Plot button is activated. If all of the simulations run successfully, after clicking on the Plot button, sensitivity analysis plots are automatically generated as shown in Figure 12.

Often times SA can produce parameter combinations that result in failed simulations. The Morris method requires that all simulations successfully complete in order for the SA plots to be generated. Other plots, however, can still be generated from the runs that have run successfully.

image20
Figure 12. Sensitivity plots.
  1. To generate plots, click on the SA to highlight, and then click the Plot button in the Summary pane, or at the top of the data browser window.
  2. The Analysis windows (Figure 12) will appear, as well as the sensitivity analysis plots that are auto-generated. For each parameter, the analysis provides sensitivity estimates for the number of replicates. The mean is plotted in blue.
  3. To create additional plots, the Guided Control in the left column is used. Three plot types are available: histogram, scatter plot and line plot. To further explore sensitivities, a scatter plot of log-permeabilities for facies 3 are plotted against the aqueous concentration predicted at Well 2. Select scatter plot in the guided control, followed by the entries shown in Figure 13.

image21
Figure 13. Entries under Guided Controls.

Press Generate, and the scatter plot will be shown on the right plotting area, as shown in Figure 14.


image22

Figure 14. Scatter Plot of Facies 3 Log-permeability and Concentration at Well 2.

  1. Click image23 in the upper left corner to save the results.
  2. Plotting options using the Filters menu (upper right corner of the plotting panel)
    1. Add – to plot statistical quantities to the existing plot (e.g., Min, Max, standard deviation, mean, median, 95% Confidence Intervals, Range).
    2. Remove – to remove filters added, or to remove data outside of the filtered (Unfiltered data).
    3. Use filtered data.
  3. Plotting options using the Transform (upper right corner of the plotting panel):
    1. Shift  to add or subtract a fixed value from the x- and/or y-data set.
    2. Scale  to multiply the x- and/or y-data set by a fixed value.
    3. Set lower threshold – to set a minimum value to be plotted along the x- and/or y-axis.
    4. Set upper threshold  to set a maximum value to be plotted along the x- and/or y-axis.
    5. Convert units.
  4. Plotting options using the Extract menu (upper right corner of the plotting panel)
    1. Time (years) to y-axis will extract the time in years that corresponds to the quantity currently plotted on the y-axis. For example, if a maximum concentration is plotted along the y-axis, this option will exchange the concentration for the time at which the maximum concentration occurs.
  5. Plotting options using the Options menu (upper right corner of the plotting panel):
    1. Export  to export or view data.
    2. Reset plot – to re-process the plot.
    3. Render As – Histogram/Scatter to change the chart type.
    4. Show Legend  to toggle the legend.
    5. Close plot  to delete the plot.