# @HEADER
#
########################################################################
#
#  Zoltan Toolkit for Load-balancing, Partitioning, Ordering and Coloring
#                  Copyright 2012 Sandia Corporation
#
# Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
# the U.S. Government retains certain rights in this software.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# 3. Neither the name of the Corporation nor the names of the
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY SANDIA CORPORATION "AS IS" AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SANDIA CORPORATION OR THE
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# Questions? Contact Karen Devine	kddevin@sandia.gov
#                    Erik Boman	        egboman@sandia.gov
#
########################################################################
#
# @HEADER
August 10, 2010
lriesen@sandia.gov

topologyVis.c:
  This MPI program discovers the network topology using the MPI extensions defined
by Sandia's IAA project (add a reference here).  It assigns a part number to each
process based on where it exists within the hierarchy, so that processes on the same
"node" or "socket" (for example) have contiguous part numbers.  It lists the part number 
assigned to each MPI process, and then the parts in each group at each level in the hierarchy.

topologyVis-Glory.txt:
  The output of topologyVis on a 64 process run on Glory.

topologyTest.c:
  This MPI program does the same discovery that is done in topologyVis.c, then stores
the topology/part information in a global data structure.  The result is that all
processes know what parts are in each group at each level of the hierarchy, and all
can map a part number to a process rank and vice versa.  Redundant levels of
the hierarchy or levels at which group size is 1 are omitted.  Then a ping-pong
test is done within a group at a level in the hierarchy and between groups at a
level in the hierarchy.  Communication performance is reported.

topologyTest-Glory.txt:
  The output of topologyTest on a 64 process run on Glory.

The TLCC machine Glory at Sandia National Laboratories in Albuquerque has the IAA
MPI version installed.  To set the modules to use this version of MPI:

module use /home/bwbarre/Software/modules
module unload mpi
module load mpi/openmpi-1.4.1-iaa_intel-11.1-f064-c064
