I found a paper that explains “Cross-Domain Few-Shot Learning” well.
https://arxiv.org/abs/2303.08557
What is Cross-Domain Few-Shot?
The Cross-Domain Few-Shot(CDFS) is one of the machine learning research field that is getting hot. Few Shot Learning or Few Shot Domain Adaptation are used in the purpose that train on a subset, and adapt trained model to target domain somehow. But on the other hands, in the Cross-Domain Few-Shot Setting, Train Dataset Domain and Target Dataset Domain is intersected as the name tells. See a table in the paper. There are Cross-Domain, Few-Shot Domain Adaptation, Few-Shot Learning, Domain Adaptation, Domain Generalization.
Problem Setting in detail
There are basically 2 problem settings in the Cross-Domain Few-Shot, Multiple Source and Single Source.
The Multiple Source Setting has multiple data source as train dataset, at a glance, it seems that this setting is desirable since it is good have plenty number of data for training. However, training a model with multiple data sources can lead to some kind of collapse since the datasets are completely different from each other. Universal Representation Learning from Multiple Domains for Few-shot Classification(URL) is using distillation technique in order to avoid this problem, for instance.
In Single Source Setting on the contrary, the training process is much easier.
How to Solve the Problem Setting
The authors well categorized how to realize the CDFS setting, “Image Enhancement”, “Feature Enhancement”, “Decoupling (they also called as Decomposed)”, and “Fine tuning”.
Probably you can intuitively imagine almost all means of solutions, but “Decoupling” looks unfamiliar. The key concept is that “The decoupling based methods think different features should be decomposed”. Therefore the data of CDFS is from a different domain, “the mixed feature space disturbs the model effect”, they say. Researchers aim to break down features to obtain varied expressions. For instance, they distinguish between domain-specific and domain-irrelevant features. The latter primarily gathers information that is neutral to the domain, indicating that these features have a strong ability to transfer across different domains.
CDFS in the Future
As shown in the figure in this paper, there are numerous efforts in this research field on classification tasks. I hope that such studies can also be applied in other areas, such as detection. While multimodal models are thriving, they are clearly more challenging to handle. It would be beneficial to move towards solving tasks using as simple data as possible, leveraging domain-agnostic features and the characteristics of neural networks.
