Physical units
Attach physical units to distributed arrays, combine compatible quantities, and reduce them to unit-bearing scalars. This guide follows energy and volume samples through an energy-density calculation. The numerical storage stays in plain NumPy/CuPy arrays; Astropy computes the units separately.
Install the optional dependencies before running the notebook:
pip install 'mpiarray[units,xarray]'Run the cells in order in a serial notebook. To run with MPI, export the notebook as a Python script and run mpiexec -n 4 python units.py. Every rank must execute arithmetic and collective operations; only printing is restricted to rank 0.
import astropy.units as uimport numpy as np
import mpiarray as mpa
comm = mpa.default_comm()rank, size = comm.Get_rank(), comm.Get_size()n = max(12, size) # at least one sample per rank
def show(label, value): if rank == 0: print(f"{label}: {value}")Attach units to existing values
Section titled “Attach units to existing values”set_units accepts an Astropy unit or a unit string and returns the same array. It labels the current values without converting them. Here each sample has an energy in electronvolts and a volume in cubic metres. Both arrays use the same distribution.
A new array has units is None. That means its values are treated as dimensionless in arithmetic. set_units(None) removes an attached unit.
energy_values = np.linspace(1.0, 12.0, n)energy = mpa.array(energy_values).labeled("sample").set_units("eV")volume = mpa.full(n, 2.0).set_units(u.m**3)
assert energy.units == u.eVassert not isinstance(energy.local_with_halos, u.Quantity)show("energy unit", energy.units)show("volume unit", volume.units)energy unit: eVvolume unit: m3Multiply, divide and take powers
Section titled “Multiply, divide and take powers”The result unit follows the operation. Dividing energy by volume produces energy density; multiplying it by volume recovers energy. Squaring energy squares its unit. Multiplication and division by a bare number simply scale the values.
to_numpy() gathers the numeric values without units. Call it on every rank, then use the result on rank 0 if desired.
density = energy / volumerecovered = density * volumesquared = energy**2half_energy = energy / 2
assert density.units == u.eV / u.m**3assert squared.units == u.eV**2np.testing.assert_allclose(recovered.to_numpy(), energy_values)np.testing.assert_allclose(half_energy.to_numpy(), energy_values / 2)density_values = density.to_numpy()show("density unit", density.units)show("first three density values", density_values[:3])density unit: eV / m3first three density values: [0.5 1. 1.5]Add compatible units with automatic rescaling
Section titled “Add compatible units with automatic rescaling”Addition, subtraction and comparisons convert compatible operands to the first operand’s unit. A joule-valued array can therefore be added to an electronvolt-valued array without manually converting its values. Reversing the operands changes the output unit, but not the physical result.
To express an existing array in another compatible unit, scale the numeric values and then attach the new unit. Calling set_units("J") alone would merely relabel the old numbers.
energy_joules = (energy * u.eV.to(u.J)).set_units(u.J)combined_ev = energy + energy_joulescombined_j = energy_joules + energy
assert combined_ev.units == u.eVassert combined_j.units == u.Jnp.testing.assert_allclose(combined_ev.to_numpy(), 2 * energy_values)np.testing.assert_allclose(combined_j.to_numpy(), 2 * energy_values * u.eV.to(u.J))assert np.allclose(energy, energy_joules)show("sum in electronvolts", combined_ev.sum())show("sum in joules", combined_j.sum())sum in electronvolts: 156.0 eVsum in joules: 2.4993955490399998e-17 JReduce to a host Quantity
Section titled “Reduce to a host Quantity”Whole-array sum, mean, min, max, var, std, dot, vdot and norm return Astropy Quantities on every rank. Variance has squared units; standard deviation and norm keep the original unit. Halo cells are excluded.
These host scalars can be used directly in later distributed arithmetic. In particular, subtracting the mean keeps the array’s unit. Use a scalar Quantity’s .to(...) or .to_value(...) to convert it for reporting.
mean_energy = energy.mean()centered = energy - mean_energyvariance = energy.var()spread = energy.std()norm = energy.norm()inner_product = energy.dot(energy)
assert isinstance(mean_energy, u.Quantity)assert centered.units == u.eVassert variance.unit == inner_product.unit == u.eV**2assert spread.unit == norm.unit == u.eVnp.testing.assert_allclose(centered.to_numpy(), energy_values - energy_values.mean())np.testing.assert_allclose(variance.value, np.var(energy_values))show("mean energy", mean_energy)show("mean energy in joules", mean_energy.to(u.J))show("standard deviation", spread)mean energy: 6.5 eVmean energy in joules: 1.0414148120999998e-18 Jstandard deviation: 3.452052529534663 eVDimensionless functions and scalar operands
Section titled “Dimensionless functions and scalar operands”A bare number is dimensionless: energy * 2 is valid, but adding an energy offset requires a unit-bearing scalar such as 2 * u.eV.
Functions such as exp, log and sin require dimensionless input. Dividing by a reference energy makes the ratio dimensionless. Scaled dimensionless units such as percent are converted before evaluation. This initial API also requires dimensionless trigonometric arguments; angle units such as radians and degrees are not supported.
shifted = energy + 2 * u.eVratio = energy / mean_energyweights = np.exp(-ratio)percent = mpa.full(n, 50.0).set_units(u.percent)
assert shifted.units == u.eVassert ratio.units == u.dimensionless_unscaledassert weights.units is Nonenp.testing.assert_allclose(shifted.to_numpy(), energy_values + 2)np.testing.assert_allclose( weights.to_numpy(), np.exp(-energy_values / energy_values.mean()))np.testing.assert_allclose(np.exp(percent).to_numpy(), np.exp(0.5))show("dimensionless weight sum", weights.sum())dimensionless weight sum: 5.063734091936654Errors instead of incorrect units
Section titled “Errors instead of incorrect units”Incompatible additions and dimensional inputs to transcendental functions raise before computing a result. Unsupported unit-bearing operations also raise. The examples below catch the expected errors so the notebook can run from start to finish on every rank.
for label, operation in [ ("energy + volume", lambda: energy + volume), ("energy + a bare number", lambda: energy + 2), ("exp(energy)", lambda: np.exp(energy)), ("product reduction", lambda: energy.prod()), ("axis reduction", lambda: energy.sum(axis=0)),]: try: operation() except (ValueError, TypeError) as exc: show(label, str(exc)) else: raise AssertionError(f"{label} should have rejected these units")energy + volume: add() needs compatible units; got Unit("eV") and Unit("m3")energy + a bare number: add() needs compatible units; got Unit("eV") and Unit(dimensionless)exp(energy): exp() needs dimensionless inputs; got Unit("eV")product reduction: prod() does not support a unit-bearing array: the result's unit depends on the element countaxis reduction: units are not supported for axis reductions yetAssign values with compatible units
Section titled “Assign values with compatible units”Indexed assignment converts Quantity and distributed values into the destination’s unit. This applies to single elements, slices and masks. Incompatible assignments raise before modifying the array. Integer destinations also reject conversions that would truncate fractional values or overflow. Bare numbers are dimensionless, so attach a unit when assigning to a dimensional array; direct .local writes intentionally bypass this check.
Distributed right-hand sides are gathered, so every rank must participate in those assignments.
length = mpa.zeros(n).set_units(u.m)length[:] = 250 * u.cmlength[::2] = 1000 * u.mmexpected_length = np.full(n, 2.5)expected_length[::2] = 1.0np.testing.assert_allclose(length.to_numpy(), expected_length)
try: length[:] = 3 * u.sexcept ValueError as exc: show("incompatible assignment", str(exc))else: raise AssertionError("seconds must not be written into a length array")np.testing.assert_allclose(length.to_numpy(), expected_length)incompatible assignment: assignment needs compatible units; got Unit("s") and Unit("m")Compare with a physical tolerance
Section titled “Compare with a physical tolerance”isclose and allclose accept an absolute tolerance with compatible units and a dimensionless relative tolerance, including percent. Bare absolute tolerances are measured in the first operand’s unit.
For high-level NumPy functions, put the distributed operand first when mixing it with a host Quantity. A leading Quantity makes Astropy handle the function before mpiarray, and that path does not support distributed arrays.
nearby = length + 1 * u.cmassert np.allclose(length, nearby, rtol=0, atol=2 * u.cm)assert not np.allclose(length, nearby, rtol=0, atol=1 * u.mm)assert np.allclose(length, nearby, rtol=2 * u.percent, atol=0 * u.m)show("within 2 cm", np.allclose(length, nearby, rtol=0, atol=2 * u.cm))within 2 cm: TrueCopy or redistribute without changing units
Section titled “Copy or redistribute without changing units”copy, astype, redistribute, rebalance and halo updates preserve the unit. A replicated layout (split=None) holds the entire array on every rank; redistribution moves the values and keeps their physical meaning.
copied = energy.copy()float32_energy = energy.astype(np.float32)replicated = energy.redistribute(mpa.Layout((n,), comm=comm, split=None))
assert copied.units == float32_energy.units == replicated.units == u.eVnp.testing.assert_allclose(replicated.to_numpy(), energy_values)show("replicated energy sum", replicated.sum())replicated energy sum: 78.0 eVCombine replicated arrays across ranks
Section titled “Combine replicated arrays across ranks”allreduce_replicated combines one contribution from every rank in place, including halos. Every rank must use the same unit. SUM (the default), MIN and MAX preserve it; PROD raises it to the number of ranks. Inconsistent units and other unit-bearing MPI reduction operations raise before the numeric reduction. On a single rank the method is a no-op.
Ordinary prod() over the elements remains unsupported for unit-bearing arrays; this example multiplies corresponding entries across ranks.
from mpiarray._mpi import MPI
contribution = mpa.full(n, 2.0, split=None).set_units(u.m)contribution.allreduce_replicated(MPI.PROD)assert contribution.units == u.m**sizenp.testing.assert_allclose(contribution.to_numpy(), 2.0**size)show("product across ranks", contribution.mean())product across ranks: 16.0 m4Export to xarray
Section titled “Export to xarray”to_xarray uses the axis names from labeled and defaults attrs["units"] to the computed unit. The gathered xarray data contains ordinary numeric values; the attribute describes them and does not enable Astropy arithmetic in xarray.
Gather on every rank, even when root=0 returns the result only on rank 0. An explicit attrs["units"] overrides the default without converting values, so use that override carefully.
data = density.to_xarray(name="energy_density", sample=np.arange(n), root=0)if rank == 0: assert data.attrs["units"] == str(u.eV / u.m**3) print(data)<xarray.DataArray 'energy_density' (sample: 12)> Size: 96Barray([0.5, 1. , 1.5, 2. , 2.5, 3. , 3.5, 4. , 4.5, 5. , 5.5, 6. ])Coordinates: * sample (sample) int64 96B 0 1 2 3 4 5 6 7 8 9 10 11Attributes: units: eV / m3Current boundaries
Section titled “Current boundaries”.local,.local_with_halos,gatherandto_numpyexpose raw numeric values. Direct writes through those arrays bypass unit checking.- Reshape, selection, concatenation and stacking currently drop units. Reattach a unit explicitly when the result’s physical meaning is known.
- Products (
prod), axis reductions, index reductions and uncategorized elementwise functions reject unit-bearing arrays.allandanyreturn plain booleans. - Offset temperature units such as degrees Celsius are unsupported; use kelvin.
out=and in-place arithmetic update the result unit. A maskedout=requires the output to already have that unit, because unselected values remain untouched.
Units are optional and imported lazily. Ordinary unitless arrays continue to work without Astropy installed.