Is Pre-Training Truly Better than Meta-Learning?
Description
The research challenges the belief that pre-training (PT) always outperforms meta-learning (MAML) in few-shot learning by conducting a rigorous, fair empirical comparison across diverse datasets. The authors introduce and utilize the Task2Vec diversity coefficient to categorize datasets as having either low or high diversity. The primary finding suggests that pre-training is generally better for low-diversity datasets, while meta-learning demonstrates superior performance on average for high-diversity datasets. However, the overall conclusion across all datasets indicates no statistically significant difference between the two methods. The study emphasizes methodological rigor, using the same architecture and model-agnostic algorithms and employing Cohen's d effect size for nuanced statistical comparison, which is necessary due to large sample sizes that would otherwise yield misleading p-values.
United States
