Distributed computations

This section of the manual gives additional pointers on making use of distributed computation within the library.

Distributed computations

Multi-threading

Since version 1.3, Julia has built-in multi-threaded parallelism and we have applied such feature at different levels in the library:

Multi-threaded implementations of reachability algorithms. This approach relies on the particular structure of each algorithm and hence it doesn't generalize to different algorithms. An example is the exploration of template directions in simultaneous with LGG09. Details are included in the documentation page of each algorithm.
Parallel solve, with split initial conditions. This approach applies to all algorithms, because we can always split an initial domain into smaller subdomains, and run solve on each of these regions in parallel. See Parallel solve below for details.

ReachabilityAnalysis._distribute — Function

_distribute(ivp::InitialValueProblem{HS, ST};
            intersection_method::AbstractIntersectionMethod=nothing,
            check_invariant=false,
            intersect_invariant=false,
            ) where {HS<:HybridSystem, ST<:AdmissibleSet}

Distribute the set of initial states to each mode of a hybrid system.

Input

system – an initial value problem wrapping a mathematical system (hybrid) and a set of initial states
intersection_method – method to perform the flowpipe-guard intersections
check_invariant – (optional, default: false) if false, only add those modes for which the intersection of the initial state with the invariant is non-empty
intersect_invariant – (optional, default: false) if false, take the concrete intersection with the invariant

(and possibly overapproximate to keep WaitingList concretely typed)

Output

A new initial value problem with the same hybrid system but where the set of initial states is the list of tuples (state, X0), for each state in the hybrid system.

source

Julia's documentation on multi-threading describes how to check the number of threads available in the current session, Threads.nthreads(), and how to control the number of threads e.g. by starting Julia with $ julia --threads 4 to use four threads. Please note that to make the number of threads persistent across different Julia sessions you should export an environment variable, export JULIA_NUM_THREADS=4 to be added in your .bashrc or .bash_profile files, or setting ENV["JULIA_NUM_THREADS"]=4 in your .julia/config/startup.jl file.

GPGPU computing

The recommended entry point for using general-purpose graphical processing units (GPGPU) in Julia is the library CUDA.jl. The package exports an array type CuArray used as an abstraction to perform array operations on the GPU device; common linear algebra operations are readily available through dispatch on CuArray. Apart from convenient high-level syntax to do basic linear algebra on GPUs; more advanced kernel functions can be implemented as well as it is explained in the CUDA.jl documentation.

Parallel solve

Reachability computations can be computed in parallel using Julia's built-in multithreaded parallelism. This feature is available formulating and solving an initial-value problem with an array of initial conditions.

About BLAS threads

To control the number of threads used by your BLAS library, use the function BLAS.set_num_threads(n), where n is an integer. Furthermore, the function BLAS.get_num_threads() defined below will return the current value.

Note. If you are using Julia v"0.7-" (run the command VERSION to find this), instead of Base.LinAlg below use LinearAlgebra, and this module should have been loaded in the current scope with using LinearAlgebra.

#
# This function is a part of Julia. License is MIT: https://julialang.org/license
#
function get_num_threads() # anonymous so it will be serialized when called
    blas = Base.LinAlg.BLAS.vendor()
    # Wrap in a try to catch unsupported blas versions
    try
        if blas == :openblas
            return ccall((:openblas_get_num_threads, Base.libblas_name), Cint, ())
        elseif blas == :openblas64
            return ccall((:openblas_get_num_threads64_, Base.libblas_name), Cint, ())
        elseif blas == :mkl
            return ccall((:MKL_Get_Max_Num_Threads, Base.libblas_name), Cint, ())
        end

        # OSX BLAS looks at an environment variable
        if Sys.isapple()
            return ENV["VECLIB_MAXIMUM_THREADS"]
        end
    end

    return nothing
end