Building Applications

From UMaine Supercomputer

Jump to: navigation, search

Contents

Introduction

This guide will serve to introduce you, an ACRL cluster user, to the build process across all three machines, Bender, Fawlty and Kearney2. We begin with the assumption that you have compiled code before in a standard Linux/BSD environment and are hence familiar with the typical single processor compile tools. Hence, the focus will be upon compiling MPI enabled applications.

Compilers

Introduction

There are three virtual clusters that you will have access to, that is Bender, Fawlty and Kearney2. Applications compiled for one of these clusters cannot run on the other without being recompiled. This means that if you login to Fawlty and compile your application, then later decide you want to run under OS X (Bender) instead, you will have to make a copy of the application source and login to Bender to recompile it. This is the basic process, but in moving from one machine to the next, you will probably have to update the paths to the compilers you've chosen. This will become clear as we continue.

The naming conventions and location of the MPI compiler wrappers are /usr/local/<mpi-implementation>-<transport layer>-<compiler>/. Located within this directory will be the wrappers you need, typically bin/mpicc, bin/mpif77, or bin/mpif90, for C, Fortran 77 or Fortran 90 code respectively.

OpenMPI

Open MPI represents the collaboration of developers from current MPI implementations such as mpich, LAM and mpich2. In preliminary testing we have seen significant improvement in our benchmarking tests by merely switching from mpich-1.2.6 to Open MPI. Another major advantage of Open MPI over mpich is that you only need to compile your code once to enable running over various networks (Ethernet or Myrinet). 

NOTE:  Myrinet, OpenMPI and OS X currently have bugs at the driver level.  Your code will run, but the results cannot be trusted.  DO NOT run OpenMPI on Bender over Myrinet.

MPICH 1

MPICH 1 was the MPI implementation we previously used on the clusters. We have kept it around to provide for the users who are more comfortable using it, while hoping they will migrate to Open MPI when time permits. MPICH 1 also requires the use of a separate process launcher, found at /usr/local/mpiexec/bin/mpiexec.

Installed Version

None of the aforementioned MPI Implementations will be in your path, you will typically either add them to the makefile, specify them at configure time, or export the environment variables. Below is the list of compilers on each system and the installed versions.

  • Fawlty
    • /usr/bin/mpicc... /usr/bin/mpif77 ... /usr/bin/mpif90
      • Open MPI version 1.2.2
      • GNU gcc/gfortran 4.1.2
      • For use over single node, ethernet or Myrinet.
  • Bender
    • /usr/local/mpich-eth-xl
      • MPICH version 1.2.6
      • XL Fortran/C compilers
      • For use over single node or ethernet.
    • /usr/local/mpich-gm-xl
      • MPICH version 1.2.7p1
      • XL Fortran/C compilers
      • For user over single node or Myrinet.
    • /usr/local/ompi-x.
      • Open MPI version 1.1r10477
      • XL Fortran/C compilers
      • Hopefully for use over any communication soon.
      • Currently for use over single node or ethernet communication.
  • Kearney2
    • /usr/local/mpich-eth-gnu
      • MPICH version 1.2.7p1
      • GNU gcc/gfortran 4.1.1
      • For use over single node or ethernet.
    • /usr/local/mpich-gm-gnu
      • MPICH version 1.2.7
      • GNU gcc/gfortran 4.1.1
      • For use over single node or Myrinet.
    • /usr/local/ompi-gnu
      • Open MPI version 1.1.1a1r10793
      • GNU gcc/gfortran 4.1.1
      • For use over single node, ethernet or Myrinet

The actual compile

Configure Scripts

If you're really lucky, your application was built using the GNU Autotools. This means you will follow the typical process of: 

configure && make && make install

Depending on the application type, you'll need to export some environment variables. For instance, if using OpenMPI on Fawlty, under bash, you would enter: 

user@fawlty ~ $ export CC=/usr/bin/mpicc 
user@fawlty ~ $ export FC=/usr/bin/mpif77 
user@fawlty ~ $ export F90=/usr/bin/mpif90

The following is an example of various optimizations you might use: 

user@fawlty ~ $ export CCFLAGS="-O3 -mcpu=G5 -maltivec -mabi=altivec -ftree-vectorize"
user@fawlty ~ $ export FFLAGS="-fno-automatic -O3 -mcpu=G5 -maltivec -mabi=altivec -ftree-vectorize"

Finally you can compile: 

user@fawlty ~ $ ./configure --prefix=/home/user/where-to-install
user@fawlty ~ $ make -j2
user@fawlty ~ $ make install

Makefiles

Chances are, you will get a package with source code and a Makefile you need to edit. You'll want to set the same variables as before like CC, FC, F90, CCFLAGS, FFLAGS and so on. Any decent Makefile will include comments telling you what is needed for each variable. If using the MPI wrappers, you ideally do not need to specify the Include or Library directories, but if so, they are always located in the MPI Implementations directory under include/ and lib/. Here is an example Makefile I use for HPL with Open MPI on Fawlty. 

SHELL = /bin/bash
CD    = cd
CP    = cp
LN_S  = ln -s
MKDIR = mkdir
RM    = /bin/rm -f
TOUCH = touch
ARCH  = ompi-gnu

TOPdir   = /admin/home/user/src/hpl
INCdir   = $(TOPdir)/include
BINdir   = $(TOPdir)/bin/$(ARCH)
LIBdir   = $(TOPdir)/lib/$(ARCH)
HPLlib   = $(LIBdir)/libhpl.a

MPdir    = /usr/local/ompi-gnu
MPinc    = $(MPdir)/include
MPlib    =

LAdir    = /admin/home/user/src/goto_gnu-1.03
LAinc    =
LAlib    = $(LAdir)/libgoto.a

F2CDEFS  = -DF77_INTEGER=int -DStringSunStyle

HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) -I$(MPinc)
HPL_LIBS    = $(HPLlib) $(LAlib) $(MPlib)
HPL_OPTS    = -DHPL_USE_GETTIMEOFDAY
HPL_DEFS    = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)

CC       = $(MPdir)/bin/mpicc
CCNOOPT  = $(HPL_DEFS)
CCFLAGS  = $(HPL_DEFS) -O5 -Df77isf2C -mcpu=970 -mtune=970
LINKER   = $(MPdir)/bin/mpicc 
LINKFLAGS   = $(CCFLAGS)

ARCHIVER = ar
ARFLAGS  = r
RANLIB   = ranlib

Here is an example using mpich-gm-xl on Bender of the same Makefile, due to the way the build process goes, I actually did have to specify the MPI libraries and Include files. 

SHELL = /bin/bash
CD    = cd
CP    = cp
LN_S  = ln -s
MKDIR = mkdir
RM    = /bin/rm -f
TOUCH = touch 
ARCH  = mpich-gm-xl-darwin

TOPdir   = /admin/home/user/src/hpl
INCdir   = $(TOPdir)/include
BINdir   = $(TOPdir)/bin/$(ARCH)
LIBdir   = $(TOPdir)/lib/$(ARCH)
HPLlib   = $(LIBdir)/libhpl.a 

MPdir    = /usr/local/mpich-gm-xl
MPinc    = $(MPdir)/include
MPlib    = $(MPdir)/lib/libmpich.a $(MPdir)/lib/libpmpich.a

LAdir    = /admin/home/user/src/goto_darwin-1.03/
LAinc    =
LAlib    = $(LAdir)/libgoto.a

F2CDEFS  = -DNoChange -DF77_INTEGER=int -DStringSunStyle

HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) -I$(MPinc)
HPL_LIBS    = $(HPLlib) $(LAlib) $(MPlib) -lm  -lSystemStubs
HPL_OPTS    = -DHPL_USE_GETTIMEOFDAY
HPL_DEFS    = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)

CC       = $(MPdir)/bin/mpicc
CCNOOPT  = $(HPL_DEFS)
CCFLAGS  = $(HPL_DEFS) -O3 -Df77isf2C -Wx,-qarch=ppc970,-qtune=ppc970
LINKER   = $(MPdir)/bin/mpicc
LINKFLAGS   = $(CCFLAGS)

ARCHIVER = ar
ARFLAGS  = r
RANLIB   = ranlib -c

Once you believe you have the Makefile modified correctly, you may start the build process: 

user@fawlty ~ # make -j2
Retrieved from "http://www.clusters.umaine.edu/wiki/index.php/Building_Applications"
Personal tools