PyFlare

Unleashing Python in High-Performance Computing: A Look at the Numba-MPI Revolution

Unleashing Python in High-Performance Computing: A Look at the Numba-MPI Revolution

Bridging the Gap for a New Era of Scientific Computing

For years, Python has been popular in scientific computing. But it has faced a major limit: poor performance in large, distributed systems. NumPy gave Python a solid base for numerical work. Later, JIT tools like Numba made single-node computation much faster. Yet these tools did not work well with MPI, which is the main way to communicate across many nodes in HPC. Because of this gap, Python could not compete with traditional HPC languages.

This is now starting to change. The new numba-mpi package connects Numba’s high speed with MPI’s ability to scale across clusters. It makes it possible to build hybrid parallel programs that are both fast and scalable. This article explains how the approach works. It also shows real examples and common problems. Finally, it describes advanced methods. These methods help Python run well on big scientific jobs.

The Core Incompatibility: Why Numba and mpi4py Couldn’t Coexist

At the heart of the problem lies a fundamental clash of execution models. Numba, in its fast Nopython mode, compiles Python code into optimized machine code. This removes the need for the Python interpreter and gives the function C-like speed. But it also creates a closed environment. Inside this compiled space, the code cannot call back into regular Python.

This causes a major problem when trying to use mpi4py, the main Python interface for MPI. Cython builds mpi4py, so its functions depend on the Python interpreter. A Numba-compiled function cannot call them. If it tries, the program fails. The interpreter is not available inside the compiled code.

This mismatch breaks the link between Numba and MPI. It leaves developers with two poor choices. They can leave the JIT-compiled code to run communication, which slows the program. Or they can stop using Python for high-performance distributed tasks.

numba-mpi: The Elegant Bridge

The creators of numba-mpi designed the project to solve this problem. Its key innovation is that it eliminates the Python interpreter. Instead of using mpi4py, numba-mpi uses pure Python. It relies on the ctypes interface to call the C API of major MPI libraries. These include OpenMPI, MPICH, and Intel MPI.

This design choice looks simple, but it works very well. numba-mpi invokes the MPI C library. This lets Numba-compiled code use MPI without going back to the Python interpreter. This matches Numba’s nopython mode. That mode allows only low-level calls, not Python calls.

The performance implications are nothing short of dramatic. Tests show clear results. Using numba-mpi instead of mpi4py in tight loops can boost a program’s speed. Often, it becomes 150% to 300% faster. The MPI library is not faster. The speed gain comes from removing the slow switch. Compiled code no longer needs to return to the Python interpreter. This latency is particularly punishing in communication-intensive applications where MPI calls are frequent. A simple π-estimation test reveals this. numba-mpi performs better as the number of intervals goes down. Fewer intervals mean fewer Allreduce calls. This proves that the improvement comes from cutting interface overhead.

Feature Comparison: mpi4py vs. numba-mpi

mpi4py supports calling MPI from normal Python code. numba-mpi enables calling MPI inside Numba’s JIT-compiled functions.

mpi4py uses Cython to wrap the MPI C API. numba-mpi uses pure Python and ctypes to reach the MPI C API.

mpi4py has high overhead because it must switch back to the Python interpreter. numba-mpi avoids this cost.

mpi4py works with high-level Python objects and uses serialization. numba-mpi focuses on NumPy arrays and can detect their types and sizes.

mpi4py offers a high-level, object-oriented API. numba-mpi uses a low-level, procedural API like the C MPI interface.

Architectural Patterns and Real-World Triumphs

numba-mpi links Numba and MPI. This connection lets developers build hybrid parallel systems. The most common pattern is the Single Program, Many Data (SPMD) model. In this model, many MPI tasks (ranks) each run the same program on a different part of a global dataset. Each MPI rank can use Numba’s prange to run code across all CPU cores on the node. This creates two levels of parallelism. MPI spreads the work across nodes, and Numba speeds up the work inside each node.

The py-pde library shows this pattern well. It solves partial differential equations using finite differences. Each MPI process splits the computational domain into its own subgrid. Numba compiles the core PDE update kernels for greatest speed. The code uses numba-mpi for the send and receive calls. These calls run inside the Numba-compiled kernels. These calls exchange the “halo” data at the subgrid boundaries. This tight setup removes extra communication overhead. It has even led to superlinear scaling in some cases. Performance rises even more because smaller subgrids fit better in the CPU caches.

Another compelling case is PyMPDATA-MPI, a solver for advection-diffusion problems. It builds on the SPMD model. It uses numba-mpi’s tagged asynchronous communication with isend and irecv. This lets several threads in one MPI rank run MPI operations at the same time. It creates true hybrid parallelism, where computation and communication overlap. This is important for complex domain-decomposition methods.

Task-based parallelism also gains from this approach. The DESI spectral extraction pipeline works on thousands of independent image patches. MPI spreads these patches across ranks. The heavy work—such as eigenvalue and Cholesky operations—runs in Numba-compiled kernels. This hybrid design raised throughput from 40.15 to 65.05 frames per node per hour on CPUs. It also prepared the code for a later GPU version.

Navigating the Practical Minefield: Deployment Challenges and Solutions

The theory is strong, but real-world use brings many problems. These problems can hurt both performance and stability.

1. Thread Management and Fork-Safety: A common problem is that Numba may not use all CPU cores in an MPI rank. The root cause is often the threading layer. On Linux, the default OMP layer (based on GNU OpenMP) is not fork-safe. When an MPI rank starts, it can fork new processes. This can break the OpenMP runtime. As a result, Numba may use only one thread or even crash. The fix is to use the fork-safe TBB threading layer. You must install and set it up yourself.

Configuration order is also critical. Set NUMBA_NUM_THREADS as an environment variable in the job script before Python starts. Numba sets up its thread pool when it loads. If you set this value later in the script, Numba will ignore it.

2. Memory Allocation Errors: Some users see MemoryError in prange on non-root ranks. This is not a normal out-of-memory problem. Instead, Numba’s parallel memory allocator is clashing with the distributed setup. Many ranks try to divide the memory at the same time, and this overloads the system. To fix this, check memory use. You may also need to coordinate memory use or simplify data structures.

3. System-Level Bottlenecks: Performance can depend on things outside your code. Some users report Numba running five times slower on a 40-core HPC node than on an 8-core laptop. This often happens because of system problems. Examples include slow filesystems during module loading and poor OS thread settings.

I/O is a major bottleneck. A Lustre filesystem can slow down if thousands of processes hit the metadata server at once. Network speed also needs tuning. For example, AWS hpc7a nodes need multi-rail settings to use both network cards. Without this setting, they lose half their bandwidth.

Best Practices for Robust and Scalable Deployment

Success on HPC systems demands meticulous planning and adherence to established best practices.

  • Precise Resource Allocation with Slurm:
    • Use Slurm directives to define resources. --ntasks=N specifies the number of MPI ranks, and --cpus-per-task=M allocates cores per rank. The application should read SLURM_NTASKS and SLURM_CPUS_PER_TASK. It uses these values to set itself up.
  • Prevent CPU Oversubscription at All Costs:
    • The golden rule is (MPI Ranks × Threads per Rank) ≤ Total Allocated Cores. Oversubscription causes catastrophic context-switching overhead. You must also control hidden threads in libraries like MKL. Set MKL_NUM_THREADS=1 and OMP_NUM_THREADS=1 unless you plan to use a hybrid model.
  • Manage Dependencies With Care:
    • Load modules in a clear order. First, load the compiler (such as gcc). Then load the MPI library (such as OpenMPI). Last, load the application environment. This ensures correct linking against MPI-enabled libraries.
  • Embrace Profiling and Debugging:
    • Tools like CrayPAT and NVIDIA Nsight Systems find CPU and GPU bottlenecks. A useful feature of numba-mpi is that it still works when you turn off JIT. This lets developers use normal Python debuggers like pdb to step through the code line by line.

Advanced Strategies for Pushing the Boundaries

For those seeking the greatest performance, advanced strategies are essential.

  • Overlap I/O and Computation:
    • The DESI pipeline dedicated specific MPI ranks to reading and writing data. The compute ranks worked on one batch. At the same time, the I/O ranks loaded the next batch and saved the last results. This pipeline cut total run time by more than 60 seconds. It raised throughput by 1.34× because it hid the I/O delays.
  • Leverage Asynchronous Communication:
    • numba-mpi’s isend and irecv let communication run while the code keeps working. You start the data exchange early and check for completion later. This can hide most of the communication cost. PyMPDATA-MPI uses this method well.
  • Install Intelligent Prefetching:
    • For out-of-core work, where data is larger than RAM, an async pre-fetcher can load data early. One flow-field study showed that this cut the runtime by 21–35%. It made huge, terabyte-scale datasets feel like they were in memory.
  • Algorithmic and Low-Level Tuning:
    • The most impactful optimizations are often algorithmic. The DESI team profiled their code and found that one eigenvalue step used 85% of the runtime. Instead of parallelizing it, they added a batched eigenvalue solver to CuPy. This change doubled the speed of that step. Later batched Cholesky routines yielded a further 3.3x speedup.

Can numba-mpi support GPU acceleration?

Yes, numba-mpi can work in GPU-accelerated applications. But you must understand its role and how the system works. numba-mpi itself is not a GPU-aware library; it operates on CPU memory. The power comes from combining it with other tools in a coordinated system.

Here is a clear guide on how to build such an application. It builds on the successful DESI project.

1. The Core Architectural Pattern: MPI Manages GPUs

The standard model is “one MPI rank per GPU”. Each MPI process uses one GPU. That process manages data for that GPU and launches its kernels. This provides a clean and scalable way to manage many GPUs across nodes.

2. The Technology Stack: A Three-Way Integration

You will be integrating three key technologies:

  • MPI (via numba-mpi):
    • For inter-node (and inter-GPU) communication.
  • Numba CUDA:
    • For writing and launching GPU kernels in Python.
  • CuPy:
    • It is a NumPy-like library for GPU arrays. It makes GPU linear algebra easier than writing custom CUDA kernels.

3. Data Flow and Communication

This is the most critical part. The process for a typical simulation step looks like this:

  • Compute on GPU:
    • Your data (e.g., a large array) resides in GPU device memory as a cupy. ndarray or inside a Numba CUDA kernel. This is where the intensive computations run.
  • Transfer to CPU (if needed):
    • Before you can communicate with numba-mpi, the data must be in CPU memory. You move the needed data from the GPU to the CPU. For example, you move the halo regions. You can use cupy.asnumpy() or write into a NumPy array that you created ahead of time.
  • Communicate with numba-mpi:
    • Call numba-mpi functions like send, recv, or allreduce inside a Numba @jit function. These calls exchange the data on the CPU with other MPI ranks.
  • Transfer back to GPU:
    • After you receive new data from other ranks, move it back to the GPU. You can use cupy.asarray() for this step.
  • Proceed with the next compute step.

Example Code Snippet Illustrating the Pattern:

import cupy as cp
import numpy as np
from numba import cuda, jit
import numba_mpi

@jit(nopython=True)
def exchange_halo_data(send_buffer, recv_buffer, peer_rank, tag):
    """Uses numba-mpi to exchange boundary data."""
    numba_mpi.send(send_buffer, peer_rank, tag)
    numba_mpi.recv(recv_buffer, peer_rank, tag)

# Inside main simulation loop
def update_step(data_gpu):
    # 1. Compute on GPU (using CuPy operations or a Numba CUDA kernel)
    ... # ... some computation on data_gpu...

    # 2. Extract halo regions to CPU
    left_halo_to_send_cpu = cp.asnumpy(data_gpu[:, 1])
    left_halo_to_recv_cpu = np.empty_like(left_halo_to_send_cpu)

    # 3. Communicate halos using numba-mpi
    exchange_halo_data(left_halo_to_send_cpu, left_halo_to_recv_cpu, left_rank, tag=42)

    # 4. Send received data back to GPU
    data_gpu[:, 0] = cp.asarray(left_halo_to_recv_cpu)

    # Continue with next computation...

Advanced Optimization: CUDA-Aware MPI

The data transfer between the GPU and CPU is a performance bottleneck. The solution is CUDA-Aware MPI. If your MPI system is CUDA-aware, you can skip the CPU copy.

With CUDA-Aware MPI, you can pass a cupy. ndarray to mpi4py. The MPI library then moves the data itself. It may even use GPU Direct to copy data between GPUs over the network.

Important Caveat: numba-mpi v1.0 works with NumPy arrays. To use CUDA-Aware MPI, you may need mpi4py for those calls. It can work with CuPy arrays. This leads to a hybrid approach. You use numba-mpi for CPU-side work and mpi4py for GPU communication. You still pay some interpreter cost, but you can design the code to keep that cost small.

How to set up numba-mpi in a HPC environment?

Setting up numba-mpi on an HPC cluster requires careful software setup. Here is a step-by-step guide.

Prerequisites:

  • Access to an HPC cluster with a job scheduler (like Slurm).
  • You need a compiler such as GCC or Intel. You also need an MPI library like OpenMPI or MPICH, and a Python module.

Step 1: Log in and Request an Interactive Session

Before installing, get an interactive node to avoid overloading the login nodes.

salloc -N 1 -c 8 -t 01:00:00  # Request 1 node, 8 cores for 1 hour

Step 2: Load the Necessary Modules

The order is critical. Load the compiler first, then the MPI implementation built with that compiler.

module purge                          # Clear any existing modules
module load gcc/11.2.0               # Load a compiler
module load openmpi/4.1.1            # Load an MPI implementation
module load python/3.9.6             # Load Python

Step 3: Create and Activate a Virtual Environment

This is a best practice to avoid conflicts with system-wide Python packages.

python -m venv my_numba_mpi_env      # Create a virtual environment
source my_numba_mpi_env/bin/activate # Activate it

Step 4: Upgrade Pip and Install Core Dependencies

Install the core packages using pip. The HPC system’s network often makes using wheels fast.

pip install --upgrade pip
pip install numpy numba mpi4py

Step 5: Install numba-mpi

Now install numba-mpi itself.

pip install numba-mpi

Step 6: Test the Installation (Crucial Step)

Create a simple test script, test_numba_mpi.py:

from mpi4py import MPI
from numba import jit
import numba_mpi

@jit(nopython=True)
def test_mpi():
    rank = numba_mpi.rank()
    size = numba_mpi.size()
    print(f"Hello from rank {rank} of {size} (inside Numba JIT function)")
    return rank

if __name__ == '__main__':
    # Initialize MPI via mpi4py, as required by numba-mpi
    comm = MPI.COMM_WORLD
    rank = test_mpi()
    comm.Barrier() # Synchronize after JIT functions
    if rank == 0:
        print("numba-mpi test completed successfully.")

Run the test with many processes:

mpirun -np 4 python test_numba_mpi.py

You should see hello messages from all 4 ranks.

Step 7: Create a Job Script for Production Runs

Once tested, create a Slurm job script, my_job.slurm:

#!/bin/bash
#SBATCH -J numba_mpi_job       # Job name
#SBATCH -N 2                   # Number of nodes
#SBATCH --ntasks-per-node=4    # MPI tasks per node
#SBATCH -c 8                   # Cores per task (for Numba threads)
#SBATCH -t 01:00:00            # Wall time

# --- Load the same modules used for installation ---
module purge
module load gcc/11.2.0 openmpi/4.1.1 python/3.9.6

# --- Activate the virtual environment ---
source my_numba_mpi_env/bin/activate

# --- Critical: Configure the environment ---
export NUMBA_THREADING_LAYER='tbb'  # Use the fork-safe TBB layer
export OMP_NUM_THREADS=1            # Prevent oversubscription
export MKL_NUM_THREADS=1

# --- Run the application ---
# 'srun' is the Slurm command to launch parallel jobs
srun python my_main_application.py

Submit your job with:

sbatch my_job.slurm

Key Configuration Takeaways:

  1. Module Order:
    • Compiler -> MPI -> Python.
  2. Use a Virtual Environment:
    • Isolate your dependencies.
  3. Threading Layer:
    • Always set NUMBA_THREADING_LAYER='tbb' for use with MPI.
  4. Prevent Oversubscription:
    • Set OMP_NUM_THREADS=1 and MKL_NUM_THREADS=1 unless you plan to use nested threading.
  5. Use srun/mpirun:
    • Always start multi-process jobs with the right launcher for your system and MPI.

How does numba-mpi handle memory allocation?

numba-mpi itself does not perform significant memory allocation. Its role is to provide a bridge between Numba’s compiled code and the MPI library. Memory handling is a shared job. You, NumPy, and Numba all take part. Here’s the detailed mechanism:

1. Primary Mechanism: NumPy Array-Based Communication

numba-mpi targets NumPy arrays. When you call an MPI function like numba_mpi.send(buffer, dest, tag), the following happens:

  • Buffer Provision:
    • You, the programmer, must create and manage the NumPy array that acts as the buffer. You create the memory with numpy.zeros(), numpy.empty(), or numpy.asarray(). Do this before the JIT-compiled function runs.
  • Type Inference:
    • numba-mpi reads the array’s dtype and shape. It then chooses the right MPI datatype and count.
  • Direct Pass-Through:
    • The library passes a pointer to the NumPy array’s memory to the MPI C function, such as MPI_Send. It makes no copy. numba-mpi is only a thin wrapper, and the MPI library works on your pre-allocated NumPy buffer.

Key Code Example:

import numpy as np
from numba import jit
import numba_mpi

@jit(nopython=True)
def send_data():
    # Pre-allocated NumPy array (memory allocated here in Python/interpreter space)
    send_buffer = np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float64)
    
    # numba-mpi directly uses the memory of 'send_buffer'
    numba_mpi.send(send_buffer, dest=1, tag=0)

# The memory for send_buffer is managed by NumPy's allocator.

2. Handling of Non-Contiguous Arrays

numba-mpi has an advanced feature: it can handle non-contiguous slices such as array[::2].

  • Detection:
    • When you pass a non-contiguous slice, numba-mpi sees that the memory is not contiguous.
  • Automatic Contiguification:
    • Because MPI needs contiguous data, numba-mpi makes a temporary copy.
  • Implication:
    • This means for non-contiguous data, there is a memory allocation and copy overhead. The send/receive operation uses this temporary buffer, not your original array. This is a trade-off for flexibility and correctness.

3. The Root Cause of “Allocation failed” Errors in prange

The article mentions a MemoryError inside a @njit(parallel=True) function with prange. This problem comes from Numba’s parallel system, not from numba-mpi.

  • Context:
    • Inside a prange loop, Numba makes private copies of some variables for each thread. It does this to avoid race conditions.
  • The Problem:
    • If you create a large NumPy array inside a prange loop, each thread may try to make its own copy. When many threads and many MPI ranks do this, the total memory use can exceed the node’s RAM.
  • Why it’s Confusing:
    • The error appears on non-root ranks because many threads divide memory at once. This is more likely to fail when several MPI processes are already using most of the node’s memory.

Solution: Divide large work arrays outside the prange loop and pass them in.

# CORRECT: Allocate once, outside the parallel region
@jit(nopython=True, parallel=True)
def good_function(data):
    n = data.shape[0]
    result = np.zeros(n) # Allocation happens once, before prange
    for i in prange(n):
        result[i] = data[i] * 2  # No allocation inside the loop
    return result

# RISKY: May cause "Allocation failed" error
@jit(nopython=True, parallel=True)
def bad_function(data):
    n = data.shape[0]
    for i in prange(n):
        temp_array = np.zeros(1000000) # Each thread tries to allocate a large array!
        # ... do work with temp_array ...

How to troubleshoot numba-mpi? A Comprehensive Guide

Here is a step-by-step troubleshooting guide based on common pitfalls.

Step 1: Verify Basic MPI and Python Setup

Before blaming numba-mpi, ensure your base environment works.

  • Test mpi4py alone:
# test_mpi4py.py
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
print(f"Hello from rank {rank} of {comm.Get_size()}")
  • Run: mpirun -np 4 python test_mpi4py.py. If this fails, your MPI installation or launcher is the problem.
  • Test Numba Alone:
# test_numba.py
from numba import jit
@jit(nopython=True)
def add(a, b):
    return a + b
print("Result:", add(1, 2))
  • Run: python test_numba.py. If this fails, your Numba installation is not working.
NUMBA_DISABLE_JIT=1 mpirun -np 4 python my_numba_mpi_app.py

Step 2: Isolate the Problem by Disabling JIT

A major advantage of numba-mpi is that it works even when you disable JIT. This is your most powerful debugging tool.

  • Run with JIT Disabled: Set the environment variable NUMBA_DISABLE_JIT=1. This runs your code as interpreted Python.
  • Why this helps:
    • You can now use the Python debugger pdb. You can set breakpoints and step through the code line by line.
    • All Python exceptions will show full tracebacks. This makes it easier to see where and why the error happens.
    • If the error goes away when you disable JIT, the problem is likely in how Numba compiles your code.

Step 3: Diagnose Common Runtime Errors

Problem: Hanging or Deadlock

  • Cause 1:
    • You might have a mismatch in communication. One part may send but never receive. Or some ranks may skip a collective call.
  • Solution:
    • Scrutinize your communication pattern. Ensure every rank follows the same collective operation path. Use numba_mpi.barrier() for synchronization during debugging.

Problem: Garbled or Incorrect Data

  • Cause 1:
    • Buffer sizes mismatch. The receiving buffer is smaller than the sent message.
  • Solution:
    • Use numba_mpi.irecv with a large enough buffer. Then check the status object to see how many elements you actually got.
  • Cause 2:
    • Race condition with multi-threading. You are changing a shared buffer from several threads inside a prange loop. You are also using that same buffer for communication.
  • Solution: Use thread-safe communication patterns or synchronize threads before communicating.

Problem: Single-Threaded Execution with prange

  • Cause: Incorrect threading layer, as detailed in the article.
  • Diagnosis & Solution:
    1. Check the active layer:
    2. Inside your script, run numba.threading_layer() to see which layer initializes at runtime.
    3. Force the TBB layer:
    4. Set NUMBA_THREADING_LAYER='tbb' in your environment before running the script.
    5. Install TBB: If TBB isn’t available, install it with conda install tbb or pip install tbb.

Step 4: System-Level Diagnostics

Problem: The job crawls.

  • Diagnosis:
    1. Check for Oversubscription:
      • Verify (MPI Ranks per Node) * (NUMBA_NUM_THREADS) <= (Cores per Node). Also, check if MKL_NUM_THREADS and OMP_NUM_THREADS are set to 1.
    2. Check Network:
      • For communication-heavy code, turn on multi-rail. On cloud instances using Intel MPI, set I_MPI_MULTIRAIL=1.
    3. Profile I/O:
      • If your code does file I/O, test if the problem persists without it. The HPC filesystem (e.g., Lustre) might be the bottleneck.

Problem: Memory Errors

  • Diagnosis:
    1. Check per-rank memory:
      • Use numba_mpi.rank() to have each rank print its memory usage (e.g., with psutil.Process().memory_info().rss).
    2. Move allocations:
      • As described earlier, move large array allocations outside of prange loops.

Step 5: Use Logging and Profiling

  • Rank-Specific Logging: Have each rank write to its own log file to trace its execution path.
rank = numba_mpi.rank()
with open(f"log_rank_{rank:04d}.txt", "w") as f:
    f.write(f"Rank {rank} starting...\n")
  • Use HPC Profilers:
  • For tough performance problems, use tools like CrayPAT or NSight Systems (for GPU). They show how much time goes to computation and how much goes to communication.

By using this step-by-step method, you can uncover problems in numba-mpi. You can also fix most of them.

The Road Ahead: Limitations and Future Directions

Despite its transformative impact, the Numba-MPI ecosystem is not a panacea.

The numba-mpi v1.0 has deliberate limitations. numba-mpi does not support some advanced MPI features. These include custom communicators, the MPI_IN_PLACE option, and structured datatypes. Programs that need these features must still call mpi4py outside JIT-compiled code. This brings back the overhead that numba-mpi tries to remove.

Portability is still a problem. Different systems have their own bugs, such as issues with some MPICH versions or h5py on Windows. Many HPC systems also need special environment settings. Moving to GPU acceleration is even harder. It requires strong knowledge of CUDA, memory use, and kernel design.

The future still looks good. The numba-mpi team plans to add missing features and may even remove the need for mpi4py. Other tools, like Dask, offer higher-level options for distributed work. Numba is also improving its OpenMP support. This will give HPC developers a more familiar way to write code.

Conclusion

numba-mpi links Numba and MPI. This is an important step for Python in high-performance computing. It removes a major technical limit and makes true hybrid parallel programs possible. Putting low-cost MPI calls inside JIT kernels makes Python much stronger. It can now handle real scientific work, not only prototypes.

Some advanced features still need work, but the main problem is gone. This progress lets more people use Python for big computational tasks. It works on everything from a laptop to a supercomputer.

Reference Links

  1. numba-mpi: Numba @njittable MPI wrappers 
    This resource introduces the numba-mpi project, which provides access to MPI routines from within Numba’s JIT-compiled code.
    https://av.tib.eu/media/62021
  2. Troubleshooting MPI4py Error When Using Numba-Accelerated Python Code 
    A Stack Overflow discussion that provides a minimal code example of an error when using mpi4py inside a Numba function and mentions numba-mpi as a solution.
    https://stackoverflow.com/questions/78522851/troubleshooting-mpi4py-error-when-using-numba-accelerated-python-code
  3. Does Numba support MPI and/or openMP parallelization? 
    A community discussion on the Numba discourse forum about MPI and OpenMP support, which mentions numba-mpi and other related projects like HPAT and Bodo.
    https://numba.discourse.group/t/does-numba-support-mpi-and-or-openmp-parallelization/483
  4. numba_mpi.common API documentation 
    The auto-generated API documentation for numba-mpi, showing parts of its internal implementation and how it interfaces with MPI libraries.
    https://numba-mpi.github.io/numba-mpi/numba_mpi/common.html
  5. Numba Documentation (Troubleshooting and tips) 
    The official Numba documentation page for troubleshooting common issues like compilation failures and type inference problems.
    https://numba.pydata.org/numba-doc/dev/user/troubleshoot.html
  6. Numba Documentation (A ~5 minute guide to Numba) 
    The official Numba “5-minute guide” that explains the basics of how to use the JIT decorators and how Numba works.
    https://numba.pydata.org/numba-doc/dev/user/5minguide.html