About
In BoTorch documentation, it is clearly mentioned that multiple-output and multiple task is different, and every GPs including SingleTaskGP is able to export multiple output. At the same time, you would see multi-task GPs like KroneckerMultiTaskGP(https://botorch.org/docs/models). So, what is the difference?
The representation of the Kernel
Multiple Output
In multiple output setting, the 2 dataset are prepared to compose 2 Kernels K_1, K_2. It can be either Isotopic or Heterotopic [4]. There are no correlation between them.
\begin{pmatrix} f_1 \\ f_2 \end{pmatrix} \sim \mathcal{N} \left( \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \begin{pmatrix} K_1 & 0 \\ 0 & K_2 \end{pmatrix} \right)Multiple Task
In multiple task setting, it is represented by using dot product task with kernel as we can see in [3].
It assumes that data structure is the Isotopic since it uses dot-product on the matrices.
\begin{pmatrix} f_1 \\ f_2 \end{pmatrix} \sim \mathcal{N} \left( \begin{pmatrix} 0 \\ 0 \end{pmatrix}, B \otimes K \right)=\mathcal{N} \left( \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \begin{pmatrix} b_{11}K & b_{12}K \\ b_{21}K & b_{22}K \end{pmatrix} \right)It means that it takes the correlation between kernels based on kind of tasks into account.
Reference
[1]https://gpss.cc/gpss17/slides/multipleOutputGPs.pdf
[2]https://seibibibibi.hatenadiary.com/entry/2020/12/23/085146
[3]https://eye.kohei-kevin.com/2024/07/31/multi-task-gaussian-process/
[4] https://eye.kohei-kevin.com/
