Paper Information
Summary
This paper presents a comprehensive survey of modern learning paradigms designed to overcome the limitations of traditional supervised learning, particularly the heavy reliance on large-scale labeled datasets. As data annotation becomes increasingly expensive and impractical across domains such as computer vision, natural language processing, and healthcare, the paper argues for a shift toward learning methods that can effectively leverage unlabeled or weakly labeled data.
The survey systematically examines four interconnected paradigms: self-supervised learning, semi-supervised learning, few-shot learning, zero-shot learning, and meta-learning. Self-supervised learning is highlighted as the foundation of modern AI systems, where models learn meaningful representations by creating supervisory signals directly from raw data through contrastive learning, masked prediction, and generative objectives. These methods enable large models to scale using vast amounts of unlabeled data and have driven the success of contemporary foundation models.
Semi-supervised learning is presented as a practical extension of supervised learning, combining small labeled datasets with abundant unlabeled data. The paper discusses key techniques such as consistency regularization, pseudo-labeling, and teacher–student frameworks, demonstrating how these methods significantly improve performance while drastically reducing labeling requirements. This paradigm is particularly effective in real-world scenarios where limited annotations are available but unlabeled data is plentiful.
The paper further explores few-shot and zero-shot learning as solutions for rapid generalization to new tasks or classes with minimal or no labeled examples. Few-shot learning relies on metric learning, transfer learning, and optimization-based meta-learning techniques to adapt quickly using only a handful of examples. Zero-shot learning leverages semantic representations and vision–language alignment to recognize unseen classes without direct training data, enabling flexible and scalable deployment across tasks.
Meta-learning is discussed as a higher-level framework that enables models to “learn how to learn.” By training across distributions of tasks, meta-learning methods optimize models for fast adaptation, making them well-suited for environments with frequent task changes. The paper also analyzes how meta-learning complements other paradigms by improving adaptation efficiency and generalization.
Beyond individual methods, the survey emphasizes the growing integration of these paradigms within modern foundation models. It provides a comparative analysis of their data efficiency, computational costs, strengths, and limitations, and identifies open research challenges such as theoretical understanding, robustness under domain shift, evaluation standardization, and sustainability concerns. Overall, the paper concludes that unifying these learning paradigms is essential for building scalable, adaptable, and sample-efficient AI systems capable of operating in real-world conditions.