Skip to content

Milestone: CMEPS 0.1

Jump to bottom
Rocky Dunlap edited this page Apr 20, 2018 · 36 revisions

In this milestone the Modular Ocean Model 6 (MOM6) is coupled to data atmosphere (DATM), data ice (DICE) and data runoff (DROF) models using the Community Mediator for Earth Prediction Systems (CMEPS). A goal of the milestone is to use versions of MOM6 and its National Unified Operational Prediction Capability (NUOPC) cap that are as close as possible to the version used for coupling MOM6 within the NOAA Environmental Modeling System (NEMS). CMEPS is based on the NEMS Mediator, which has been previously validated for a seasonal coupled application including active atmosphere, ocean, and ice components. The NEMS Mediator and CMEPS share an explicit coupling approach.

MOM6 is running with a global 1 degree grid (gx1). The data atmosphere, ice, and runoff models are part of the Common Infrastructure for Modeling the Earth (CIME) toolset. The data components are not prognostic, but instead read coupling fields from files. In addition, the CIME case control system and workflow are used for running this coupled system. This milestone begins a process of generalizing the MOM6 cap and CMEPS so that they can be shared between the CESM project and the NEMS-based system at the National Weather Service.

Supported Platforms

This milestone has been tested successfully on these platforms:

  • Cheyenne/NCAR
  • Hobart/NCAR
  • Theia/NOAA

Initial Conditions

MOM6 is started at rest and uses initial temperature and salinity conditions from the World Ocean Atlas 2005 (WOA2005) dataset. These and other required input files are available on the supported platforms. In addition, if this milestone is ported to another platform, the CIME case control system will automatically retrieve input data from the CESM input data subversion repository.

Run Sequence

The application uses the following run sequence.

runSeq::
      @1800   #ocean coupling step
	  MED med_phases_prep_ocn
	  MED med_connectors_prep_med2ocn
	  MED -> OCN :remapMethod=redist
	  OCN
	  @1800   # atmosphere coupling step
            MED med_phases_prep_ice
            MED med_connectors_prep_med2ice
            MED -> ICE :remapMethod=redist
            MED med_phases_prep_rof
            MED med_connectors_prep_med2rof
            MED -> ROF :remapMethod=redist
            ICE
            ROF
            ROF -> MED :remapMethod=redist
            MED med_connectors_post_rof2med
            ICE -> MED :remapMethod=redist
            MED med_connectors_post_ice2med
            MED med_fraction_set
	    MED med_phases_atmocn_flux
	    MED med_phases_atmocn_ocnalb
            MED med_phases_prep_atm
            MED med_connectors_prep_med2atm
            MED -> ATM :remapMethod=redist
            ATM
            ATM -> MED :remapMethod=redist
            MED med_connectors_post_atm2med
            MED med_phases_accum_fast
	  @
  	  OCN -> MED :remapMethod=redist
	  MED med_connectors_post_ocn2med
      @
::

Coupling Fields

The application passes a limited set of coupling fields between MOM6 and the Mediator.

Mediator to MOM6

Standard Name Units
water_evaporation_flux kg m-2 s-1
downwelling_longwave_flux W m-2
surface_net_upward_longwave_flux W m-2
rainfall_flux kg m-2 s-1
surface_upward_sensible_heat_flux W m-2
surface_downward_diffuse_shortwave_flux_due_to_near_infrared_radiation W m-2
surface_downward_diffuse_shortwave_flux_due_to_visible_radiation W m-2
surface_downward_direct_shortwave_flux_due_to_near_infrared_radiation W m-2
surface_downward_direct_shortwave_flux_due_to_visible_radiation W m-2
surface_snow_melt_flux kg m-2 s-1
surface_downward_eastward_stress N m-2
surface_downward_northward_stress N m-2

MOM6 to Mediator

Standard Name Units
sea_surface_mask 1
surface_temperature K
sea_surface_salinity g kg-1
eastward_sea_water_velocity m s-1
northward_sea_water_velocity m s-1

Validation

A full scientific validation was not performed on this limited system. However, 5-day runs of the coupled system show that (1) the ocean state changes dynamically over time as expected when coupled forcings from the data models are turned on, (2) turning off the coupled forcings changes the ocean state compared to the coupled run, and (3) seven fields imported to the MOM6 cap and two fields exported from the cap are physically reasonable under visual inspection.

The plotted fields are on the "gx1" global one degree ocean grid and are generated in the MOM6 cap. Per the run sequence above, the ocean model was forced every 30 minutes by data from the data atmosphere, ice, and runoff models, although the prototype does not take advantage of all potential coupled field exchanges.

MOM6 Import Fields (from Mediator)

Fields received in the MOM6 cap import State from the Mediator.
Figure 1.a: atmosphere sea level pressure (Pa)
Figure 1.b: sea ice fraction
Figure 1.c: visible diffuse downward shortwave (W m-2)
Figure 1.d: rain flux (kg m-2 s-1)
Figure 1.e: snow flux (kg m-2 s-1)
Figure 1.f: sensible heat flux (W m-2)
Figure 1.g: evaporation (kg m-2 s-1)

MOM6 Export Fields (to Mediator)

Fields exported from the MOM6 cap.
Figure 2.a: sea surface temperature (K)
Figure 2.b: sea surface salinity (g kg-1)

Discussion and Limitations

Initial components

At the start, two versions of the MOM6 model were available in different coupled configurations and an early version of CMEPS. Specifically, this included:

  • A version of MOM6 coupled in NEMS with a NUOPC cap: This model runs coupled with an atmosphere and ice model through the NEMS Mediator.
  • An initial version of the Community Earth System Model 2 (CESM2) with MOM6 using the Model Coupling Toolkit (MCT) coupling infrastructure: The MOM6 MCT cap is fairly unique in this implementation and leverages some custom modifications to the top level MOM6 code.
  • An early version of CESM2 based on the CMEPS coupling infrastructure: This infrastructure does not fully validate against the MCT version yet. So far it has been used to couple data, dead, and a limited number of active models together, but never a prognostic ocean model.

This initial demonstration of a coupled DATM-DICE-DROF-MOM6 model uses the CMEPS coupling infrastructure in CESM2, the MOM6 model version from the CESM2 coupled MCT configuration (but without using the customized top level MOM6 code), and the MOM6 NUOPC cap from NEMS, with modifications. A goal was to demonstrate the ability to couple MOM6 to CMEPS using the NEMS NUOPC cap as much as possible. It would also provide a first step in understanding the Mediator and cap differences between the CESM2 and NEMS approaches.

Mediator updates

CMEPS uses ESMF_Mesh as the internal grid representation while the MOM6 cap advertises its fields using the ESMF_Grid data type. To address this, a new capability was added into CMEPS that converts a transferred ESMF_Grid into an ESMF_Mesh. This is now a basic feature supported in CMEPS.

The atmosphere-ocean fluxes are computed in the med_phases_atmocn_flux Mediator phase and then passed to the atmosphere and ocean models. For this milestone, these flux calculations were verified and a new prep_ocn Mediator phase was introduced to generate merged and computed fields to pass to the ocean.

Cap updates

The MOM6 NUOPC cap required modifications to support coupling with CMEPS:

  • The CMEPS shared coupled field infrastructure was used to build the internal field list and to advertise and realize the coupling fields in the cap. This approach will not be used long term as it introduces a dependency on CMEPS shared functions, which may not be needed or available in all coupled systems.
  • Memory for the coupling fields in ESMF import and export States was allocated explicitly without pointers to MOM6 allocated memory. This was done because (1) the coupling fields provided and required in MOM6 do not match up one-to-one with the coupling fields in CMEPS, and (2) some fields require transformations (such as vector rotation and sign change) and those transformations should not be applied directly to fields inside MOM6 internal memory.
  • Import and export methods were introduced to copy data from/to ESMF import/export States to the ice_ocean_boundary data type in MOM6. Field indexing in MOM6 is complex and the cleanest way to index fields needs to be explored further.
  • A DataInitialization method was added to support more complex initialization sequencing in CESM2. The DataInitialize method was not fully validated and coupled fields were ignored for the first 2 coupling periods of the run.
  • A MOM6 method was used to allocate memory in the ice_ocean_boundary data type.
  • Support for vector rotation was turned off as the implementation was problematic.
  • Four downward shortwave fields were introduced in the CMEPS coupling field list for MOM6. In CESM2, normally just the net shortwave is passed.

Limitations and next steps

This coupled system is an initial prototype. A number of limitations must be addressed before a full scientific validation can be performed and before the implementation is fully generalized. Some outstanding tasks known at this time include:

  • Reconcile differences between the MOM6 MCT-based cap and NUOPC-based cap. This includes issues such as scalar coupling, restart and initialization sequencing, albedo consistency, and use of coupling fields. Work may be required in the top level MOM6 code to support requirements of both NEMS and CESM models.
  • Generalize the MOM6 NUOPC cap for use in both CMEPS and NEMS. There are some fundamental differences in approaches such as cap allocation of memory versus pointers into MOM6 memory, how coupling fields are defined, use of MOM6 data types in the cap, and initialization sequencing.
  • Provide an ability to couple alternative sets of coupling fields through the Mediator and to respond at run-time to varied coupled configurations. This would allow the coupled system to adapt dynamically to ocean model components with different sets of physical fields.

The goal is for the MOM6 cap to move between major U.S. coupled systems without requiring code changes. The NUOPC Layer provides the coupling data structures and protocols for this to happen, but there needs to be a broader understanding of the requirements for models, caps, and mediators in order to support that. And then those requirements need to be translated to simpler and more general implementations in all aspects of model coupling development.

Download, Build and Run

# Clone UFSCOMP
$ git clone https://github.com/ESCOMP/UFSCOMP.git
$ cd UFSCOMP
$ git checkout cmeps_v0.1

# Check out externals including MOM6 and CIME
$ ./manage_externals/checkout_externals

# Set environment
$ export CIME_MODEL=cesm
$ export PROJECT=<your project>   # account to charge for run

# Create new case and build
$ cd cime/scripts
$ ./create_newcase --case testmom_nuopc --compset 2000_DATM%NYF_SLND_DICE%SSMI_MOM6_DROF%NYF_SGLC_SWAV --res T62_g16 --run-unsupported --driver nuopc --walltime 00:30
# NOTE: On Theia, walltime should be 01:00

$ cd testmom_nuopc
$ ./xmlchange --id DOUT_S --val FALSE  # turn off short term archiving
$ ./xmlchange --id INFO_DBUG --val 2   # turn on debug output
$ ./case.setup
$ ./case.build

# Submit the run
# On Theia only:
$ ./xmlchange --id JOB_QUEUE --val batch    # debug queue limited to 30 min

$ ./case.submit

$ ./xmlquery RUNDIR   # Output will be in RUNDIR
Clone this wiki locally