Sparse stencil operators
mpiarray.StencilOperator represents a linear operator given by a small, fixed set of
neighbour offsets — the pattern a finite-difference stencil has — without ever storing a
sparse matrix: apply halo-exchanges and sums the shifted terms directly on a
DistributedArray, and to_petsc_mat assembles the identical pattern as a PETSc Mat for
use with PETSc’s Krylov and multigrid solvers.
Show code
import mpiarray as mpa
comm = mpa.default_comm()rank, nranks = comm.Get_rank(), comm.Get_size()The five-point Laplacian
Section titled “The five-point Laplacian”The heat equation guide applies a five-point Laplacian by
hand, with one line of slicing per neighbour. StencilOperator is the same idea made
general: give it the offsets and coefficients once, and it works for any pattern and any
number of axes. This guide uses a periodic grid, so every row has all five neighbours and
the examples below have an exact, easily checked answer.
u = mpa.zeros((64, 48), split=(0, 1), halo=1, periodic=True)u.local[...] = 1.0 # a constant field: the Laplacian is zero everywhere
laplacian = mpa.StencilOperator( u.layout, offsets=[(0, 0), (-1, 0), (1, 0), (0, -1), (0, 1)], coefficients=[4.0, -1.0, -1.0, -1.0, -1.0],)lu = laplacian.apply(u) # update_halos(), then the shifted sumlu.sum() # collective: exactly 0.0np.float64(0.0)apply checks that u’s layout matches the operator’s, forwards boundary= to
update_halos (needed for the non-periodic walls the heat equation guide has), and raises if
layout.halo is too narrow for the offsets used. A coefficient can be a single number, as
above, or an array of layout.local_shape for an operator whose weight varies by grid
point.
Rows that differ: overrides
Section titled “Rows that differ: overrides”A handful of rows — Dirichlet rows, cut cells near an embedded boundary — can replace the regular stencil entirely, by global index:
pinned = mpa.StencilOperator( u.layout, offsets=[(0, 0), (-1, 0), (1, 0), (0, -1), (0, 1)], coefficients=[4.0, -1.0, -1.0, -1.0, -1.0], overrides={(0, 0): [((0, 0), 1.0)]}, # row (0, 0): identity instead of the stencil)pinned.apply(u).get((0, 0)) # u's own value, 1.0, not the Laplacian's 0.0np.float64(1.0)Each rank resolves only the overrides it owns, through DistributedArray.local_index; an
index outside this rank’s block is skipped, so the mapping can be built once, from global
indices, and handed to every rank unchanged.
Combining operators
Section titled “Combining operators”Operators of the same layout combine with +, -, unary - and multiplication by a
number, merging offsets (and overridden rows) term by term — useful for building an
operator such as a shifted Laplacian, or a Jacobian that is the sum of several physical
terms, out of smaller pieces:
mass = mpa.StencilOperator(u.layout, offsets=[(0, 0)], coefficients=[0.1])shifted = laplacian + mass # (laplacian + 0.1 * identity)(u)lu = laplacian.apply(u)(shifted.apply(u) - lu).max() # 0.1, since u is the constant field 1.0np.float64(0.09999999999999964)Each side keeps its own coefficients (arrays included) and overrides; a row overridden on only one side is combined with the other side’s regular stencil at that row, resolved on whichever rank owns it.
Assembling a PETSc matrix
Section titled “Assembling a PETSc matrix”The same pattern, without petsc4py:
Show code
from importlib.util import find_spec
petsc_available = find_spec("petsc4py") is not Noneif not petsc_available: print("Install petsc4py to run the matrix example.")Install petsc4py to run the matrix example.if petsc_available: matrix = laplacian.to_petsc_mat() vec = u.to_petsc() result = matrix.createVecLeft() matrix.mult(vec, result) phi = mpa.from_petsc(result, u.layout) print(phi.sum()) # the same 0.0 as laplacian.apply(u) above matrix.destroy()to_petsc_mat uses Layout.dmda for the row decomposition and mpa.petsc_numbering for
the column numbers, exactly as in the PETSc guide’s hand-assembled
example. Along a non-periodic axis an offset that steps outside the grid is simply omitted
(an implicit Dirichlet zero); give that row its own overrides entry for any other boundary
condition.