Compile and Install PETSc with external libraries

https://web.cels.anl.gov/projects/petsc/download/release-snapshots/petsc-3.23.6.tar.gz
tar xvf petsc-v3.23.6.tar.gz

Download/Install superlu_dist/mumps/MPI/cmake

cd petsc-v3.23.6
./configure \
  --with-openmp=1 \
  --with-debugging=0 \
  --with-shared-libraries=1 \
  --download-f2cblaslapack \
  --download-scalapack \
  --download-superlu_dist \
  --download-mumps \
  --download-mpich \
  --download-cmake \
  COPTFLAGS="-O3" CXXOPTFLAGS="-O3" FOPTFLAGS="-O3"

Output

PETSc:
  Language used to compile PETSc: C
  PETSC_ARCH: arch-linux-c-opt
  PETSC_DIR: /home/kevin/project/gridap/petsc-3.23.6
  Prefix: <inplace installation>
  Scalar type: real
  Precision: double
  Support for __float128
  Integer size: 4 bytes
  Single library: yes
  Shared libraries: yes
  Memory alignment from malloc(): 16 bytes
  Using GNU make: /usr/bin/make
xxx=======================================================================================xxx
 Configure stage complete. Now build PETSc libraries with:
   make PETSC_DIR=/home/kevin/project/gridap/petsc-3.23.6 PETSC_ARCH=arch-linux-c-opt all
xxx=======================================================================================xxx

Install

make all
make install

=========================================
Now to check if the libraries are working do:
make PETSC_DIR=/home/kevin/project/gridap/petsc-3.23.6 PETSC_ARCH=arch-linux-c-opt check
=========================================

Activate PETSc

Add dir to PATH

export PETSC_DIR=/home/kohei/local/petsc-3.23.6
export PETSC_ARCH=arch-linux-c-opt

Check Activation thru GridapPETSc

using GridapPETSc
GridapPETSc.with() do
    println("Using PETSc from ", ENV["PETSC_DIR"])
end

If it tells the path where you installed your PETSc, it is done.

Install OpenBLAS with multi-threading

Install OpenBLAS which has multi-threading version

sudo apt install libopenblas-dev
ls /usr/lib/x86_64-linux-gnu/libopenblas.so

Re-Compile PETSc with the OpenBLAS

make clean
./configure \
  --with-openmp=1 \
  --with-debugging=0 \
  --with-shared-libraries=1 \
  --with-blaslapack-lib=/usr/lib/x86_64-linux-gnu/libopenblas.so \
  --download-scalapack \
  --download-superlu_dist \
  --download-mumps \
  --download-mpich \
  --download-cmake \
  COPTFLAGS="-O3" CXXOPTFLAGS="-O3" FOPTFLAGS="-O3"
make all

Check the link with OpenBLAS

ldd $PETSC_DIR/$PETSC_ARCH/lib/libpetsc.so | grep blas
        libopenblas.so.0 => /usr/lib/x86_64-linux-gnu/libopenblas.so.0 (0x0000706ff7bb0000)

Compile PETSc with MKL

Install MKL

wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo add-apt-repository "deb https://apt.repos.intel.com/oneapi all main"
# Install MKL
sudo apt update
sudo apt install intel-oneapi-mkl intel-oneapi-mkl-devel
source /opt/intel/oneapi/setvars.sh # it is going to set $MKLROOT
kevin@ubuntu:~/project/gridap/petsc-3.23.6$ echo $MKLROOT
/opt/intel/oneapi/mkl/2025.2

Compile

make clean
./configure \
  --with-openmp=1 \
  --with-debugging=0 \
  --with-shared-libraries=1 \
  --with-mkl_pardiso=1 \
  --with-blaslapack-lib="[$MKLROOT/lib/intel64/libmkl_rt.so]" \
  --download-scalapack \
  --download-superlu_dist \
  --download-mumps \
  --download-mpich \
  --download-cmake \
  COPTFLAGS="-O3" CXXOPTFLAGS="-O3" FOPTFLAGS="-O3"
make PETSC_DIR=$PETSC_DIR PETSC_ARCH=arch-linux-c-opt all
make PETSC_DIR=$PETSC_DIR PETSC_ARCH=arch-linux-c-opt check

export PETSC_DIR=$HOME/project/gridap/petsc-3.23.6
export PETSC_ARCH=arch-linux-c-opt

Check

ldd $PETSC_DIR/$PETSC_ARCH/lib/libpetsc.so | grep mkl

./configure \
  --with-openmp=1 \
  --with-debugging=0 \
  --with-shared-libraries=1 \
  --with-mkl_pardiso=1 \
  --with-mkl_pardiso-dir=/opt/intel/oneapi/mkl/2025.2 \
  --with-blaslapack-lib="[/opt/intel/oneapi/mkl/2025.2/lib/libmkl_rt.so,-lpthread,-lm,-ldl]" \
  --download-scalapack \
  --download-superlu_dist \
  --download-mumps \
  --download-hypre \
  --download-mpich \
  --download-cmake \
  COPTFLAGS="-O3 -fopenmp" \
  CXXOPTFLAGS="-O3 -fopenmp" \
  FOPTFLAGS="-O3 -fopenmp"