# -*- outline -*-

Notes on how to change ADCIRC to support Dynamic Domains.

* Glossary:

** nprocs : (scalar) total number of processors in current job (say 256)
** aprocs : (scalar) number of processors with active domains.  Say it starts with
            100 domains active and can only go to nprocs.


** iproc  : (scalar) 0 to (aprocs-1) counter over active processors
** jproc  : (scalar) 0 to (nprocs-1) counter over all processors

** ndom   : (scalar) total number of domains (say 512)
** adom   : (scalar) number of active domains  (it is the same as aprocs)
** idom   : (scalar) 0 to adom-1 

** procA(0:nprocs-1): An array which contains which processors are active and what
           domain they have.  (0 to adom-1).
** domA(0:ndom-1):  What processor has this domain, -1 if not active.            

* Preprocessor Changes:
** procA array must be computed by the preprocessor and placed in domain 0's
   files

** Change Pre-processor into two stage tool

*** Part A: Grid Partitioner
    This divides the grid and computes the send lists between domains.
    It does not do anything with the wind/wave height data.

*** Part B: Splits the wind/wave data and computes the initial procA array.        



* Basic Algorithm:

** Start Up
*** Processor 0 reads in procA array from domain 0
     and broadcasts it to all processors (nprocs)
*** Each processor figures out what domain it will own.
      The first active domain gets assigned to proc 0 and so on.
      There will be "adom" or "aprocs" domains active
      Say adom = 100 then processors 0 thru 99 will be active
      procA(100:255) = -1 and domA will be computed as well

        domA(0:ndom-1) = -1
        domA(2) = 0
        domA(4) = 1
        ...

*** compute "MPI_ACTIVE_WORLD" from 0:aprocs-1 processors

*** some how mark the neighbor send lists to only send
    to the active domains.


*** Change routines in messenger.F to use MPI_ACTIVE_WORLD 
    instead of MPI_COMM_WORLD

*** Compute the number active nodes



** Boundary conditions

*** Dry and therefore inactive sub-domains will
    initially imply zero change in height.
    This will have to change as a sub-domain becomes active.

    Is this true for both equations?


** At the end of each time step
*** Compute the wet and dry nodes.

*** Compute any dry domains that have become wet.

*** Question: Suppose you have a triangle where each node
    is owned by a different domain.  Suppose also that only one node
    is in an active domain.  Will the current algorithm work correctly
    in this case?

** At the beginning of each time step

*** If there is a new domain added then:

**** Increase the number of active domains
**** Update MPI_ACTIVE_WORLD, procA, domA
**** Update nbr communications

**** Recompute the matrix.
**** update the number of active nodes.

**** Read in wind/wave data on newly active domains.


* Numbering of nodes
** Nodes have a local number in a domain.
** Nodes will have a global node number in the total domain (TGBL)
** Nodes will have an active global number for the active domains (AGBL)



* Other Questions

** Output results:
   Say you start with 100 domains active and finish with 240 domains
   active and there are 512 domains total.  What should the output
   file look like.  All 512 domains and all nodes reported or
   just the 240 domains and active nodes for every time step?


** It is possible that a domain could start dry get wet and
   get dry again.  Do we get to assume that once a domain gets wet
   it is active through out the calculation independent of its
   wetness?