sampledoc

Table Of Contents

This Page

ASCEM : Single Run (SR)

  1. ASCEM
  2. Tutorials

ASCEM : Single Run (SR)

Introduction

Objective

This tutorial demonstrates how to create and launch a single run (SR) in Akuna.

Toolsets/modules exercised

The following Akuna modules are being exercised:

  1. Specify the SR input parameters
  2. A SR job submission
  3. Visualizing output from SR results

The following Amanzi modules are being exercised:
  1. Transient flow simulation
  2. Transient transport simulation
  3. Richards equation

Pre-requisite

For this tutorial, one assumes the 1D-richard model described in Model Setup Tutorial has been generated. Otherwise, create a new model by copying the Richards-1D-transport_exa model from tutorial folder.

  1. Right click on projects > tutorial > Richards-1D-transport_exa and choose copy.
  2. Highlight the User’s working folder where this model is going to be copied.
  3. Go to Edit > Paste.
  4. A pop-up window will ask if the Outputs are also to be copied. Since in this tutorial the User will be re-creating the outputs, select No.
  5. A new model will appear under the user’s working folder.

Problem Description

The tutorial problem is a one-dimensional unsaturated flow and transport problem, where

  1. steady-state unsaturated flow is calculated with a constant infiltration rate, providing the initial conditions for year 1950;
  2. discharge of water and 99Tc occurs at the top of the ground surface between 1950.1-1950.26 y; and
  3. continuing infiltration drives the 99Tc plume downward.

Note that the actual simulation starting time is year 0, and the simulation runs long enough to reach a steady-state by year 1950.

Initialize SR Toolset

To start the SR toolset:

  1. Highlight the Richards-1D-transport icon ( image0 ) on the main Akuna window
  2. Click on the SR icon in the menu ( image1 ).
  3. Enter a name for this SR (Figure 1) and click OK.
image2
Figure 1. Generate a new SR.

This action launches the Parameter Toolset window (Figure 2).

image3
Figure 2. Parameter Toolset window.

SR Parameter Setup in Parameter Toolset Window

To launch a single simulation run (SR), control parameters have to be assigned. The Parameter Toolset window (Figure 2) contains seven tabs (Execution Controls, Visualization Settings, Checkpoint Settings, Numerical Controls, Parameters, Observations, Job Configuration). All the tabs except for the Parameters tab are relevant to SR. Potential errors are shown in the Summary panel at the bottom.

Execution Controls

This tab controls the numerical aspect in each time period such as time stepping and simulation period. There are three panels: Time Controls, Defaults for all time periods and Screen output control. The Time Controls panel includes the Start time and End time for each period in which the time control parameters need to be specified. At least one Time Control has to be specified to indicate the start and end time of the simulation. The Defaults for all time periods panel on the right specifies the setting during the time periods that are not specified under the Time Controls panel. The Screen output control specifies the verbosity level for Amanzi (medium by default). For other parameters, please see the Amanzi User’s Guide (https://software.lanl.gov/ascem/internal/amanzi/UserGuide).

In this example, we have Time Controls for two periods. The first period (0-1950 y) is the steady-state unsaturated flow problem, and the second one (1950-2200 y) is the transient flow problem. To add additional time controls;

  1. Click image4 next to Start Time
  2. Add 0.0 y for Start Time
  3. Fill other properties in the Times Controls panel: End = 1950y, Initial Time Step = 10 s, Maximum Time Step = 1000 y, Reduction factor = 0.8, Increase factor = 1.2, Mode = Steady, Method = bdf1. See Figure 1.
  4. Add another time period and fill 1950y for another Start Time
  5. Fill other properties in the Times Controls panel: End = 2200y, Initial Time Step = 1 s, Maximum Time Step = 100 y, Reduction factor=0.8, Increase factor = 1.2, Mode = Transient, Method = bdf1.

The Execution Controls tab should be filled as shown in the figure below (Figure 3).

image5
Figure 3. Execution Controls tab filled in SR.

Visualization Settings

This tab specifies times at which visualization files are created in the VisIt format. The user may first specify the visualization file name in the Visualization Output File box.

image6
Figure 4. Visualization Settings tab

To add a new series of times,

  1. Click image7 and add name VisTime for the Reference box (double-click the box that has Unnamed).
  2. Click the box under Time or Cycle to choose a Time interval (times in the simulation, including start, end and time interval), Time list (individual times), Cycle interval (the number of time steps in the simulation, including start, end and time steps interval) or Cycle list (individual time steps). In this example, choose Time list.
  3. When Time list is selected, fill the large box with a list of the times (1950 y, 1951 y, 1952 y, 1953 y, 1954 y, 1960 y, 1970 y, 1980 y, 1990 y, 2000 y, 2020 y, 2040 y, 2060 y, 2080 y, 2100 y, 2120 y, 2140 y, 2160 y, 2180 y, 2200 y).

Alternatively, if choosing Time interval, one can specify each of Start, End and Interval for the starting and ending time, and the time interval, respectively. The time unit is specified in the last box (Note: y/yr for years, d for days, min for minutes, s for seconds).

Checkpoint Settings

This tab defines the cycles at which Checkpoint files are saved (see Amanzi User’s Guide). It can be specified in the same manner as Visualization Settings except that only cycles can be used (not times). In this example, it is left blank.

Numerical Controls

This tab specifies the numerical algorithms and parameters that control the convergence behavior of the simulator. This example uses the default values. For more information, see Amanzi User’s Guide.

Parameters

This tab is used in the other toolsets (e.g., SA, PE, UQ) and is not needed in the execution of an SR. No input is required.

Observations

In this tab, the user can include locations, times, and the types of

observable variables that shall be extracted from the output files for visualization and further analysis. For example, the concentration of a component at a well can be selected to obtain breakthrough curves. The Observation window has four tabs: image8 . Click image9 to select options in each tab. | To define observation locations, click Regions then select image10 . This will open the Region window (Figure 5), which defines each observation region. In the Region window, an observation point can be created as follows:

  1. Name the point as Well1
  2. Specify the point location (X=2.0m, Y=0.5m, Z=50.0m). The location has to be within the range that is shown on the right.
  3. Click Apply.
  4. Repeat steps 1-3 for Well2, at (X=2.0m, Y=0.5m, Z=107.0m).
image11
Figure 5. Region window

Click the View regions (Figure 6) in the Add Region dialogue to see the regions created.

image12
Figure 6. View defined observation locations.

To specify observation times, click image13 in the Time sets tab. The time set macro will open (Figure 7). Click image14 and name the observation time as ObsTime. ChooseTime list as Type and enter Individual times as shown in the right panel of Figure 7. Then Apply. This example uses the same time list as the VisTime defined in the Visualization Settings tab (i.e., 1950 y, 1951 y, 1952 y, 1953 y, 1954 y, 1960 y, 1970 y, 1980 y, 1990 y, 2000 y, 2020 y, 2040 y, 2060 y, 2080 y, 2100 y, 2120 y, 2140 y, 2160 y, 2180 y, 2200 y).

image15

Figure 7. Add observation time set.

To add new observations, click Observations tab and select image16 . The Select Observations window will open (Figure 8). Make sure Well 1 and Well 2 are assigned as regions and ObsTime is assigned as time set. To select the output variables, check the boxes in Assign in the right panel (Figure 8, top right). Multiple variables can be chosen. In this example, select Volumetric Water Content and Aqueous Concentration Tc-99. Use the scroll bar to access Aqueous Concentration Tc-99 if it is not shown in the selection window.

image17

Figure 8. Select Observations.

After all the panels are set, click the Apply and Close button for saving the observation setting. The defined observations are shown under Point observations (Figure 9).

image18
Figure 8. Select Observations.

Job Configuration

The Job Configuration window (Figure 9) specifies the job control on the computing platform. Several common computing platforms are listed in the left column. In this example,

  1. Choose hopper as the Machine.
  2. Specify the Run Directory. On the NERSC machine, Hopper, the scratch directories are used; e.g., /scratch/scratchdirs/$HOPPER_USERNAME/Richard-1D-transport.
  3. Specify the Username, Allocation Account (as given when opening a Hopper account) and Job Queue (debug).
  4. Specify the Wall Time (30 minutes). Maximum wall time for the debug queue is 30 minutes, but only 3 minutes are actually used here because the simulation runs within that time period. For other wall time and queueing policies, please read *Queue and Scheduling Policies* on the NERSC website.
  5. Specify Processors (24) and Processors per task (2).
  6. Specify the executive path (Agni Command) and simulator (Amanzi Command) as the following when executing on Hopper if the machine configuration does not already exist:
Agni Command:

 /global/project/projectdirs/m1012/agni/install/current/bin/Agni –infile=agni.xml | Amanzi Command:   /project/projectdirs/m1012/amanzi/install/hopper/release0.81/bin/amanzi

  1. Specify Max Simultaneous Simulations (1) and scroll down to specify the Agni verbosity level. Please see the Agni User Guide (https://akuna.labworks.org/akuna/wiki/index.php/AgniUserGuide) for more information on the verbosity levels. In this example, the Verbosity level is the default value (0).
image19
Figure 9. Specify Job Configuration.

Submit SR

After all the SR parameters are filled, save this setting by clicking the Save button image20 at the top left of the Parameter Toolset window (Figure 2). After checking that there is no error message in the Status panel, click the Launch button image21 . This will submit the job to the machine specified in the Job Configuration. There are two input files generated in Richard-1D-transport > single-run > Inputs: Agni.xml and Amanzi.xml.


On the Main window, one can see the status of the job in the

Summary tab. It will be updated when the job is completed and Status is Success.

Plot Observations

When the job Status is Success, the Plot link appears next to Success (Figure 10) in the Summary pane. The Summary pane includes the Job State, SR Summary and General Information. And the output files are listed in Richard-1D-transport > single-run > Outputs.

image22
Figure 10. Summary panel after a successful SR
Click the Plot link to open the Analysis Tool window (Figure 11).

To create a new plot, go to the Guided Controls tab on the left panel. | Follow the guided controls (left column). New controls can be added from the pull down menu. In this example, select each control according to the left column in Figure 11. Note that the times, locations, and types of observable variables to be displayed were specified in Section 3.6. And choose One plot, one series for time selection and Multiple series for location selection. Click Generate. The aqueous concentration of Tc-99 at two observation locations (red line - Well 1 and blue line - Well 2) are shown on the right panel (see Figure 11).


Other controls can also be used for plotting, and are accessed through

menu controls. Depending on the user’s screen size, the plotting window may need to be expanded to show the controls shown in Figure 11 at the upper right corner of the plotting window.

  1. Plotting options using the Filters menu (upper right corner of the plotting panel)
    1. Add – to plot statistical quantities to the existing plot (e.g., Min, Max, standard deviation, mean, median, 95% Confidence Intervals, Range).
    2. Remove – to remove filters added, or to remove data outside of the filtered (Unfiltered data).
    3. Use filtered data.
  2. Plotting options using the Transform (upper right corner of the plotting panel):
    1. Shift to add or subtract a fixed value from the x- and/or y-data set.
    2. Scale to multiply the x- and/or y-data set by a fixed value.
    3. Set lower threshold – to set a minimum value to be plotted along the x- and/or y-axis.
    4. Set upper threshold to set a maximum value to be plotted along the x- and/or y-axis.
    5. Convert units.
  3. Plotting options using the Extract menu (upper right corner of the plotting panel)
    1. Time (years) to y-axis will extract the time in years that corresponds to the quantity currently plotted on the y-axis. For example, if a maximum concentration is plotted along the y-axis, this option will exchange the concentration for the time at which the maximum concentration occurs.
  4. Plotting options using the Options menu (upper right corner of the plotting panel):
    1. Export to export or view data.
    2. Reset plot – to re-process the plot.
    3. Render As – Histogram/Scatter to change the chart type.
    4. Show Legend to toggle the legend.
    5. Close plot to delete the plot.

image23

Figure 11. Analysis Tool window. Additional plot controls are shown in the upper right corner.

Advanced Visualization

After the SR run, the spatial visualization files are saved in the working directory on the selected platform (Hopper in this case). These visualization files include the concentration or other variables of interest at all the grid points and times specified in the Visualization Setting tab. The user can create, for example, a contour plot of saturation in the domain or a movie of plume migration.


The visualization files can be very large depending on the problem

size and the number of times. Output from the Amanzi simulator is written in HDF5 format, a compressed binary format designed for large data sets (www.hdfgroup.org). The VisIt visualization software is a separate software package that can be used to visualize Amanzi output. However, any visualization package can be used to visualize the output, as long as it can natively read the HDF5 format. Alternatively, the HDF5 format can be translated to a readable format needed for the visualization software using the open-source tools provided by the HDF5 group. The advantage of using VisIt is the tight integration planned with Akuna in the future. In addition, VisIt provides remote visualization capabilities, so that the user can visualize the results without downloading the files to a local computer. This is an advantage when large files are generated.

image24

Figure 12. Configuring VisIt for the NERSC remote server.

Currently, VisIt is independently installed and launched by the user, outside of the Akuna framework, and can be downloaded freely from https://wci.llnl.gov/codes/visit/. Depending on the user’s operating system, the user may be asked to configure VisIt for use on a remote machine (see Figure 12). This is because when accessing remote files, VisIt must know the location of the remote installation of VisIt, as well as have information on the queuing system of the remote server. If using NERSC, then the user should check the box shown in Figure 8. If the user is not prompted for this information, then after launching VisIt, select Options – Host profiles and Configuration Setup from the drop-down menus at the top of the screen to access the window shown in Figure 12. If the user’s remote server is not listed, then the user will need to enter the required information after launching Visit. Select Options – Host profiles, and click on the New Host button to configure other remote machines. Refer to the VisIt manuals for more detailed information on setting up new hosts and additional features not described in this tutorial. The manuals can be accessed at https://wci.llnl.gov/codes/visit/manuals.html.

When the user starts VisIt, the default setup shows two windows: the VisIt Main window on the left and the Vis window on the right (Figure 13). Note that the Main window has to be active in order to use tools or options in the VisIt menu bar (e.g., change axes or viewing angles).

image25

Figure 13. VisIt windows: Main window on the left and Vis window (Window 1) on the right. Various tools or options can be selected in the main menu bar (File, Controls, Options, etc.).

The following steps create figures on the screen, and save movies for transient problems (e.g., evolving contaminant concentration in the domain).

Open files:

  1. Click Open in Sources in the Main window (Figure 13).
  2. In the File Selection window (Figure 14), choose the remote machine where the simulation was executed. For Hopper, choose NERSC Hopper for Host. The user will need to provide username and password.
  3. In Path, specify the directory on Hopper that has visualization files for SR. The visualization file directory is specified as “<run_directory>/amanzi_0”, where the run_directory is provided in the Akuna Summary panel next to Run Dir: as shown in Figure 9.
  4. Click Refresh, and choose plot_data.h5.*.xmf database in Files.
  5. Click OK. It asks the user about the job option on Hopper. To accept the default options, click OK. It usually takes less than a minute to get connected.

image26

Figure 14. File Selection window.

Create plots

Now Active source in the Main window (Figure 13) has the visualization database files.

  1. In Plots, click Add > Pseudocolor (Figure 15), and choose the variables of interest (e.g., total component concentration, water saturation)
  2. Click Draw in Plots. The requested plots are shown in the Vis window.

image27

Figure 15. Plotting the total concentration of Tc-99 in a pseudocolor map by selecting Add > Pseudocolor > total_component_concentration.cell.Tc-99.conc

Change the axis and view angle

  1. Click Controls > View (or click control+V) in the menu (above) to open the View control panel (Figure 16).
  2. Adjust parameters. For this problem, the user needs to change View normal = [0 1 0] and Up Vector = [0 0 1] for the XZ view. The user can change Image zoom for changing the size of the plot. For other settings, please check the VisIt Getting Started Manual.
  3. Click Apply to apply the changes, and click Dismiss to close the panel.
image28
Figure 16. View control panel

View the movie

Click Play in Times in the Main window to see a movie of, for example, the Tc-99 plume migrating downward (Figure 17).

image29
Figure 17. Plot and movie of Tc-99 concentration.

Save the movie

To create and save the movie, select File > Save Movie … in the main VisIt menu. Follow the instructions. Except for the Choose Format, the user can proceed with the default settings. In the Choose Format, the user needs to select the movie format and click the right arrow (). It takes a few minutes to create the movie. The movie is saved in the home directory, unless specified otherwise in the Choose Filename.