Post-Doctoral Research Visit F/M Dynamic Parallelization of Sparse Codes for High-Performance Computing and Machine Learning

Context
Within the framework of the
French prioritary research program for Exascale computing in France (PEPR NumPEx)
, we are hiring a
postdoctoral fellow
to address
runtime code optimization for sparse computing.
This research targets both
High-Performance Computing
and
Machine Learning.
This postdoctoral fellowship will be held at
Laboratoire de l'Informatique du Parallélisme
at
Ecole Normale Supérieure de Lyon, France.
Assignment
Most kernels of interest in machine learning and high-performance computing manipulate
sparse tensors
. Sparse codes are
highly irregular
and make use of array indirections and dynamic control which jeopardize static automatic parallelization algorithms.
The overall objective of this postdoctoral fellowship is to investigate compiler and runtime algorithms to
delay the specialization of
the dense code at runtime when the sparse structure is known
.
From a dense specification, we seek to compile a code able to
specialize itself on the sparse input data
. The specialization will involve a set of
sub-computations
, which are expected to be achievable by standard linear algebra routines (e.g.
gemm
),
using state-of-the art linear algebra libraries
.
Several issues must be investigated:
How to
specialize the code
? In particular, how
propagate efficiently the sparsity
along the computation flow?
How to
detect library kernels
on the specialized code?
How to enforce a proper scheduling for the
parallel runtime
?
Points 1 and 2 have been partially addressed by a PhD student.
The postdoctoral fellow will:
Propose code optimizations and data structures for
scaling sparse propagation (point 1)
Address
runtime scheduling (point 3)
by relying on existing parallel runtimes
Validate the complete approach
(points 1, 2 and 3) on scientific benchmarks by using sparse tensors from the Florida sparse matrix collection as well as machine learning applications.
Main activities
Main activities (5 maximum) :
Research
Software development
Experimentation
Report & article writing
Skills
Technical skills and level required :
Notions in compilers, parallelism, parallel architectures, parallel runtimes
Experience with C++
Languages : English, French
Excellent relational skills.