The source document behind this article is a graduate course project report from Columbia University’s ECS E6692, Deep Learning on the Edge, not a peer reviewed clinical study, and it has not been evaluated by a medical regulatory body. It…
The idea of knowledge distillation goes back to Hinton, Vinyals, and Dean, who proposed training a small student network to mimic the softened output probabilities of a large teacher rather than only the hard labels. It works remarkably well in…
Dual Forward Path Teacher Knowledge Distillation Explained Read More »
FixMatch is built around a deceptively simple idea. Take one unlabeled image, make two versions of it, one with a light augmentation such as a small crop or flip, and one with a heavy augmentation such as strong color distortion…
KD-FixMatch Fixes FixMatch’s Noisy Early Pseudo Labels Read More »
Ask anyone who has tried to build a medical image segmentation model what the hardest part is, and the honest answer is rarely the neural network architecture. It is getting enough labeled data. A useful segmentation dataset needs pixel accurate…
DFCPS: Teaching a Polyp Detector With Barely Any Labels Read More »
Semi supervised learning exists to squeeze more value out of unlabeled data when labeling everything by hand is too expensive. Methods such as FixMatch and ReMixMatch have gotten very good at this under one quiet assumption, that every class shows…
How Balanced Contrastive Learning Fixes Skewed SSL Data Read More »
Semi supervised learning exists because labeling data is expensive and unlabeled data is cheap. Pseudo label methods try to make the most of that imbalance by having a model label its own unlabeled data, then training on those self generated…
How A Channel Based Ensemble Produces Cleaner Pseudo Labels Read More »
Change detection sounds simple on paper. Take two satellite images of the same location captured months or years apart, then mark every pixel where something meaningfully changed. In practice, building the training data for this task is one of the…
VLM Guidance Cuts Label Needs for Change Detection Read More »
Unsupervised Domain Adaptation (UDA) faces two persistent roadblocks: effectively leveraging powerful modern foundation models and the crippling storage overhead of deploying multiple domain-specific models. A groundbreaking approach merges Vision-Language Pre-training (VLP) like CLIP with innovative techniques—Cross-Modal Knowledge Distillation (CMKD) and…
CMKD: Slash 99% Storage Costs & Dominate UDA Challenges Read More »
Imagine training a heart scan segmentation model with exactly one labeled patient. That is not a hypothetical stress test, it is the actual one percent labeled setting a team from Zhejiang University and Ritsumeikan University put their method through, and…
SemSim Adds Semantic Similarity to FixMatch Segmentation Read More »
Most modern semi supervised learning systems, FixMatch and FreeMatch among the best known, work the same basic way. Take an unlabeled image, run it through the model with a light augmentation, and if the model’s confidence in its own prediction…
Long Tailed Weights Beat Thresholds For Pseudo Labels Read More »