VLM Guidance Cuts Label Needs for Change Detection

The labeling bottleneck nobody talks about enough

Change detection sounds simple on paper. Take two satellite images of the same location captured months or years apart, then mark every pixel where something meaningfully changed. In practice, building the training data for this task is one of the more tedious annotation jobs in computer vision. A human labeler has to flip between two nearly identical images, spot new buildings, demolished structures, or expanded roads, and trace the boundaries pixel by pixel. Do that across thousands of image pairs and you understand why most remote sensing teams cannot afford the fully labeled datasets that academic benchmarks assume.

This is exactly the gap semi supervised learning tries to close. Instead of demanding full annotation, a semi supervised method trains on a small labeled set alongside a much larger pool of unlabeled images, using the model’s own predictions, filtered for confidence, as a stand in supervision signal. The dominant recipe in this space is FixMatch, originally built for image classification and later adapted for segmentation. It compares a model’s prediction on a weakly augmented image against its prediction on a strongly augmented version of the same image, and only trusts the weak prediction as a pseudo label when the model is confident enough about it.

FixMatch works reasonably well for change detection, but the authors of SemiCD-VL point out a structural weakness. Every pseudo label FixMatch produces comes from the same model that is being trained. If that model has a blind spot early in training, for instance confusing shadows cast by tall buildings with actual demolition, the pseudo labels will quietly reinforce that blind spot rather than correct it. What the field needed was an independent source of supervision that had never seen the change detection task at all.

Borrowing supervision from a model that has never seen a change map

Vision language models such as CLIP have learned, from huge amounts of web scale image and text pairs, to recognize open vocabulary concepts like a house or a road purely from a prompt. They were never trained to compare two images, so asking one directly for a difference mask does not work well. The word difference is too abstract a prompt, and the paper notes that current vision language models are essentially built for single image reasoning, not paired image reasoning.

Instead of forcing an awkward direct comparison, SemiCD-VL decomposes the problem. It runs the vision language model separately on each of the two temporal images, asking it to segment concrete categories the team defines ahead of time, house, building, road, grass, tree, water. Only afterward does the pipeline convert those two independent segmentation masks into a change mask. This single design choice, treat the vision language model as a single image segmenter rather than a change detector, is what makes the whole approach workable.

The team behind SemiCD-VL settled on APE, a universal visual perception model, as their vision language model of choice, after testing it against CLIP derived alternatives like MaskCLIP and ZegCLIP. They report that APE generalized better to the chunked, top down style of remote sensing imagery, likely because it was trained on higher level fine grained detection and grounding tasks rather than the coarser image level contrastive objective behind CLIP.

Why defining the background category matters as much as the foreground

One detail the paper spends real effort on is easy to miss but turns out to matter a lot. Most open vocabulary applications only define the categories a user cares about, the foreground. SemiCD-VL also explicitly defines a background category, road, grass, tree, water in this case, alongside the foreground categories house and building.

Why bother? Because change detection over satellite imagery is a strict binary task, changed or unchanged, and the two predicted classes are mutually exclusive. Without an explicit background definition, a region the vision language model genuinely cannot recognize, an unusual roof material, a partially obscured structure, gets silently folded into the background class by default. That default assignment then becomes a false supervision signal telling the change detector nothing changed there, when in reality the model simply did not know what it was looking at. By defining both sides of the boundary and treating anything outside both definitions as an ignored, unreliable region, the authors keep uncertain pixels out of the loss function entirely rather than letting them poison training with confident sounding wrong answers.

Mixed change event generation, the paper’s actual novelty

Turning two single temporal segmentation masks into a usable change label sounds straightforward, subtract one from the other and call it a day. The authors tried exactly that as a baseline, calling it pixel level change event generation, and ran into a predictable problem. Buildings in the two images are rarely pixel perfectly aligned. A structure that has not changed at all can still show a thin sliver of apparent change along its edges purely because the camera angle or crop shifted by a few pixels between capture dates. Table VI in the paper shows this clearly, pixel level generation alone tops out around 41.4 percent IoU and 58.5 percent F1 on LEVIR-CD when evaluated across all pixels.

Their fix is instance level change event generation. Rather than comparing individual pixels, this approach treats each connected building or structure as a discrete instance, then compares every instance in image one against every instance in image two using intersection over union as a similarity score. An instance with no good match on the other side counts as a change event, the building appeared or disappeared, and edge jitter from imperfect alignment simply washes out because it is being measured at the object level, not the pixel level. Instance level generation alone reaches 46.3 percent IoU and 63.3 percent F1, a meaningful jump.

Mixed change event generation combines both signals. Pixel level reasoning supplies the explicit foreground and background definitions, instance level reasoning cleans up the misalignment noise, and only the changes both strategies agree on survive into the final pseudo label, everything else gets marked as an ignored region rather than a guess. This combined signal is what actually feeds the semi supervised training loop.

Untangling two competing sources of supervision

Adding vision language guidance on top of FixMatch’s existing consistency regularization creates an obvious tension. For the strongly augmented view of an unlabeled image, the model now receives two different supervision signals, the pseudo label generated by the weakly augmented view under the consistency regularization framework, and the mixed change event label generated by the vision language model. These two signals will not always agree, and forcing a single prediction head to satisfy both at once risks the model learning neither well.

SemiCD-VL’s answer is what the paper calls a dual projection head, and it is a strikingly simple idea. The shared difference decoder still produces one feature representation, but two separate linear classifiers read from it. One classifier, labeled h_cr in the paper, is supervised purely by the consistency regularization pseudo label. The other, h_vl, is supervised purely by the vision language model’s mixed change label. Splitting the output at the classifier level, while keeping the underlying feature representation shared, lets both supervision sources shape the model’s understanding of the image without directly contradicting each other’s gradients at the same output.

Teaching the model to see two images separately before comparing them

A recurring theme across recent change detection research is decoupling, training the model to understand each temporal image on its own terms before asking it to compare the two. SemiCD-VL leans into this with two auxiliary segmentation decoders that share weights and predict the single image semantic mask for each temporal image separately. These decoders are only active during training and add zero cost at inference time, since the final change prediction still comes from the difference decoder alone.

The supervision for these auxiliary decoders comes directly from the vision language model’s single temporal segmentation output, filtered by a reliability threshold. The intuition, borrowed from earlier work like ChangeMask, is that a model which can cleanly separate what a building looks like from what a road looks like in each image individually will have an easier time recognizing when a building becomes a road between the two captures. Table VII, the paper’s ablation study, shows this component contributing a real if modest gain, pushing IoU from 81.23 to 81.65 percent on LEVIR-CD with 5 percent labels once combined with the other components already in place.

The formulas behind the five components

The consistency regularization backbone follows FixMatch closely. The supervised loss on labeled data is a standard pixel wise cross entropy term.

The unsupervised consistency loss only fires when the weakly augmented prediction clears a confidence threshold, set to 0.95 in the experiments.

The mixed change event generation label combines the pixel level and instance level masks, with unreliable regions marked separately rather than forced into either class.

That mixed label then supervises both the weak and strong prediction branches through the h_vl classifier.

Finally, the contrastive consistency regularization term pulls feature vectors from unchanged pixel pairs closer together while pushing changed pixel pairs apart up to a margin.

The overall objective sums the consistency regularization loss with weighted contributions from the vision language guidance loss and the contrastive term, with both weights set to 0.1 in the paper’s experiments, and the vision language weight linearly decayed toward 0 over training since the authors judged early guidance more valuable than late stage guidance.

What the numbers actually show

Table II compares SemiCD-VL against FixMatch, UniMatch, SemiCDNet, SemiCD, BAN, and several adversarial baselines across labeling budgets from 5 to 40 percent on both LEVIR-CD and WHU-CD. Against the FixMatch baseline it builds on, SemiCD-VL improves IoU by 2.4 points at 5 percent labels and 1.3 points at 10 percent on LEVIR-CD, and by a much larger 5.3 points at 5 percent labels on WHU-CD.

Look closely at the WHU-CD gains over FixMatch and a slightly odd pattern shows up. The improvement is 5.3 points at 5 percent labels, then drops to 2.6 points at 10 percent, before climbing back up to 3.8 and 3.9 points at 20 and 40 percent. A monotonically shrinking benefit as labeled data grows would make intuitive sense, since a stronger baseline leaves less room for a helper signal to add value, but that is not quite what happens here. The dip at 10 percent followed by a partial recovery at higher budgets is not explained in the text, and it is the kind of detail worth watching in any follow up work that tries to reproduce these numbers on a different split.

Table III compares SemiCD-VL against fully supervised methods trained on the complete training set. With just 5 percent labeled data, SemiCD-VL reaches 81.9 percent IoU on LEVIR-CD and 81.8 percent on WHU-CD, putting it ahead of established supervised baselines like STANet, SNUNet, and BiT, though still slightly behind ChangeFormer’s 82.5 and 81.6 percent. Bump the labeled fraction to 10 percent and SemiCD-VL edges past ChangeFormer on both datasets, 82.6 versus 82.5 on LEVIR-CD and 83.2 versus 81.6 on WHU-CD, while using a tenth of the annotation.

The proposed CEG strategy, in an unsupervised manner, can achieve performance far superior to state of the art unsupervised change detection methods, with IoU improved from 18.8 percent to 46.3 percent on LEVIR-CD. Paraphrased from the paper’s abstract, arXiv:2405.04788

That unsupervised result deserves its own mention. Table V compares the instance level change event generation strategy, used with zero training and zero labels, against prior unsupervised change detection methods like PCA-KM, DSFA, DCVA, and the more recent foundation model based SCM. The jump from SCM’s 18.8 percent IoU to instance level generation’s 46.3 percent on LEVIR-CD, and a similar jump from 18.6 to 45.2 percent on WHU-CD, is a genuinely large margin for what amounts to running a frozen vision language model and a geometric comparison step, no gradient updates at all.

Which components actually earn their place

Ablation studies are where papers either hold up or fall apart, and Table VII walks through each of the five components one at a time on LEVIR-CD with 5 percent labels. Starting from a FixMatch baseline of 79.52 percent IoU, adding raw vision language guidance alone brings a 1.14 point gain. Swapping in mixed change event generation for the simpler pixel level version adds another 0.31 points. The dual projection head contributes 0.26 points on top of that, reaching 81.23 percent.

Here is where it gets interesting. Adding contrastive consistency regularization at this stage, without the decoupled segmentation guidance also in place, actually drops performance slightly, from 81.23 down to 81.15 percent. The authors attribute this to insufficient guidance for the segmentation decoder at that point in the ablation sequence. Only once decoupled semantic guidance is added first, reaching 81.65 percent, does layering contrastive regularization on top produce the full pipeline’s best score of 81.94 percent. It is a useful reminder that component order in an ablation table is not just bookkeeping, some architectural pieces genuinely depend on others being present first to pay off.

The clean data everyone forgets to sanity check

Table IX explores what happens when the loss weights for the contrastive term and the vision language guidance term are pushed away from their default value of 0.1. The contrastive weight is the more dangerous knob by far. Set it to 1.0 and IoU collapses from roughly 81.9 percent down to 66.8 percent, a fifteen point cliff. Set it to 0.5 and the drop is still severe, down to 79.7 percent. The vision language guidance weight is comparatively forgiving, moving it from 0.01 up to 1.0 only shifts IoU within about a one point band. The authors’ own explanation lines up with common sense here, the vision language pseudo labels contain real errors since the underlying model was never trained for this exact task, so leaning too hard on either auxiliary loss, but especially the contrastive term, starts to actively fight the primary change detection objective rather than support it.

Where this breaks down in practice

SemiCD-VL is not a magic fix for every change detection scenario, and the paper is reasonably candid about two limitations. First, the whole pseudo label pipeline depends on the underlying vision language model being reasonably accurate for the categories being detected. Everything here is validated on buildings, roads, and vegetation in two curated remote sensing datasets, LEVIR-CD and WHU-CD. Extending the same category definition and prompt strategy to a domain with far more visual variability, disaster damage assessment across many building material types, for instance, would likely require redefining the foreground and background categories from scratch, and there is no guarantee the same vision language model would generalize as cleanly.

Second, the paper is upfront that the pseudo label generation pipeline is a two step process, single temporal reasoning followed by post processing into a change mask, and that pipeline structure inherently accumulates errors across steps. An end to end vision language model built for multi temporal reasoning from the ground up, which the authors note does not yet exist in a mature form, would likely sidestep this accumulation. There is also a practical cost the paper is honest about, generating vision language pseudo labels for the entire unlabeled pool ahead of time adds real preprocessing time before training even starts, even though it introduces zero overhead at inference once training is done.

Why this matters beyond change detection

The specific numbers in this paper apply to two building focused remote sensing datasets, but the underlying pattern, use a frozen vision language model as an independent pseudo labeling source, then route its supervision through a dedicated classifier head so it cannot silently overwrite the model’s existing training signal, is not tied to change detection at all. Any dense prediction task where labeled data is expensive and a pretrained vision language model can plausibly recognize the relevant object categories could borrow this same dual projection head and decoupled auxiliary decoder pattern. The authors explicitly note this in their conclusion, some strategies here are not limited to the change detection task and can extend to general semi supervised methods.

Complete PyTorch implementation

The implementation below reconstructs the five components described in Section III of the paper, the siamese encoder feeding a difference decoder and two shared weight segmentation decoders, the dual projection head, the mixed change event generation routine for pixel level and instance level pseudo labels, and all four loss terms combined into the paper’s overall objective. A lightweight convolutional encoder stands in for the ResNet50 backbone used in the paper so the smoke test below runs quickly on CPU, swap in a torchvision ResNet50 for full scale training.

"""
SemiCD-VL: VLM guided semi-supervised change detection.
Reference: Li et al., "SemiCD-VL: Visual-Language Model Guidance Makes Better
Semi-supervised Change Detector," arXiv:2405.04788.

This is a faithful, runnable reconstruction of the five components described
in the paper (mixed CEG, VLM guidance loss, dual projection head, decoupled
semantic guidance, contrastive consistency regularization), built on a
lightweight siamese ResNet style encoder rather than a full ResNet50 so that
it trains fast on a CPU smoke test. Swap TinyEncoder for torchvision resnet50
for full scale training.
"""

import torch
import torch.nn as nn
import torch.nn.functional as F


# ---------------------------------------------------------------------------
# Encoder and decoders
# ---------------------------------------------------------------------------

class ConvBlock(nn.Module):
    def __init__(self, in_ch, out_ch, stride=1):
        super().__init__()
        self.conv = nn.Conv2d(in_ch, out_ch, 3, stride=stride, padding=1, bias=False)
        self.bn = nn.BatchNorm2d(out_ch)
        self.act = nn.ReLU(inplace=True)

    def forward(self, x):
        return self.act(self.bn(self.conv(x)))


class TinyEncoder(nn.Module):
    """Lightweight siamese backbone standing in for ResNet50. Produces a
    single feature map at stride 8 for the segmentation and difference
    decoders to consume, mirroring how the paper feeds ResNet50 stage
    features into all MLP decoders."""

    def __init__(self, in_ch=3, base=32):
        super().__init__()
        self.stem = ConvBlock(in_ch, base, stride=2)
        self.stage1 = ConvBlock(base, base * 2, stride=2)
        self.stage2 = ConvBlock(base * 2, base * 4, stride=2)
        self.out_ch = base * 4

    def forward(self, x):
        x = self.stem(x)
        x = self.stage1(x)
        x = self.stage2(x)
        return x


class AllMLPDecoder(nn.Module):
    """Lightweight stand in for the all-MLP Segformer style decoder used for
    both the segmentation decoders and the difference decoder in the paper."""

    def __init__(self, in_ch, hidden=128, out_ch=64):
        super().__init__()
        self.proj = nn.Sequential(
            nn.Conv2d(in_ch, hidden, 1),
            nn.ReLU(inplace=True),
            nn.Conv2d(hidden, out_ch, 1),
        )

    def forward(self, x):
        return self.proj(x)


# ---------------------------------------------------------------------------
# Full SemiCD-VL model
# ---------------------------------------------------------------------------

class SemiCDVL(nn.Module):
    """
    Contains, per Section III of the paper:
      - a shared siamese encoder
      - a difference decoder consuming the l1 distance of bi-temporal features
      - two auxiliary segmentation decoders (shared weights) for decoupled
        semantic guidance
      - a dual projection head on the difference decoder output, h_cr for
        consistency regularization and h_vl for VLM guidance
      - a shared segmentation classifier h_seg
    """

    def __init__(self, num_classes=2, feat_dim=64):
        super().__init__()
        self.encoder = TinyEncoder()
        enc_ch = self.encoder.out_ch

        self.diff_decoder = AllMLPDecoder(enc_ch, out_ch=feat_dim)
        self.seg_decoder = AllMLPDecoder(enc_ch, out_ch=feat_dim)  # shared weights

        # Dual projection head, Section III-D
        self.h_cr = nn.Conv2d(feat_dim, num_classes, 1)
        self.h_vl = nn.Conv2d(feat_dim, num_classes, 1)

        # Shared segmentation classifier, Section III-E
        self.h_seg = nn.Conv2d(feat_dim, num_classes, 1)

    def encode(self, t1, t2):
        f1 = self.encoder(t1)
        f2 = self.encoder(t2)
        return f1, f2

    def forward(self, t1, t2, out_size=None):
        f1, f2 = self.encode(t1, t2)

        # Difference decoder consumes the l1 distance of bi-temporal features,
        # following the UniMatch setting referenced in Section III-G.
        diff_in = torch.abs(f1 - f2)
        q_diff = self.diff_decoder(diff_in)
        y_cr = self.h_cr(q_diff)
        y_vl = self.h_vl(q_diff)

        # Segmentation decoders (weight sharing enforced by reusing the module)
        q_t1 = self.seg_decoder(f1)
        q_t2 = self.seg_decoder(f2)
        y_t1 = self.h_seg(q_t1)
        y_t2 = self.h_seg(q_t2)

        if out_size is not None:
            y_cr = F.interpolate(y_cr, size=out_size, mode="bilinear", align_corners=False)
            y_vl = F.interpolate(y_vl, size=out_size, mode="bilinear", align_corners=False)
            y_t1 = F.interpolate(y_t1, size=out_size, mode="bilinear", align_corners=False)
            y_t2 = F.interpolate(y_t2, size=out_size, mode="bilinear", align_corners=False)

        return {
            "y_cr": y_cr,
            "y_vl": y_vl,
            "y_t1": y_t1,
            "y_t2": y_t2,
            "q_t1": q_t1,
            "q_t2": q_t2,
        }


# ---------------------------------------------------------------------------
# Mixed change event generation (CEG), Section III-B
# ---------------------------------------------------------------------------

def pixel_level_ceg(prob_t1, prob_t2, foreground_idx, gamma=0.8):
    """
    prob_t1, prob_t2: (B, C, H, W) VLM category probability maps for the two
    temporal images. foreground_idx: list of channel indices belonging to the
    Foreground concept, per the category definition in Section III-B1.
    Returns I_pixel_diff (B, H, W) with 0 for unchanged, 1 for changed, and
    I_rel (B, H, W) marking unreliable pixels, matching Eq. 5 to 7.
    """
    def concept_map(prob):
        fg = prob[:, foreground_idx].max(dim=1).values
        bg_idx = [i for i in range(prob.shape[1]) if i not in foreground_idx]
        bg = prob[:, bg_idx].max(dim=1).values
        concept_prob = torch.stack([bg, fg], dim=1)  # 0 = background, 1 = foreground
        concept_label = concept_prob.argmax(dim=1)
        reliable = concept_prob.max(dim=1).values >= gamma
        return concept_label, reliable

    i1, rel1 = concept_map(prob_t1)
    i2, rel2 = concept_map(prob_t2)
    i_rel = ~(rel1 & rel2)  # True where unreliable, Eq. 6 uses product of indicators
    i_pixel_diff = (i1 != i2).long()
    return i_pixel_diff, i_rel.long()


def instance_level_ceg(mask_t1, mask_t2, delta=0.0):
    """
    Simplified instance level CEG, Eq. 8 to 10. mask_t1, mask_t2 are (H, W)
    binary foreground masks. Connected components are extracted with a plain
    flood fill so the routine has no external dependency, and IoU between
    every pair of instances across the two temporal masks determines whether
    an instance persisted (unchanged) or is a change event.
    """
    def connected_components(mask):
        H, W = mask.shape
        visited = torch.zeros_like(mask, dtype=torch.bool)
        instances = []
        for y in range(H):
            for x in range(W):
                if mask[y, x] and not visited[y, x]:
                    stack = [(y, x)]
                    coords = []
                    visited[y, x] = True
                    while stack:
                        cy, cx = stack.pop()
                        coords.append((cy, cx))
                        for dy, dx in ((1, 0), (-1, 0), (0, 1), (0, -1)):
                            ny, nx = cy + dy, cx + dx
                            if 0 <= ny < H and 0 <= nx < W and mask[ny, nx] and not visited[ny, nx]:
                                visited[ny, nx] = True
                                stack.append((ny, nx))
                    inst = torch.zeros_like(mask, dtype=torch.bool)
                    ys, xs = zip(*coords)
                    inst[list(ys), list(xs)] = True
                    instances.append(inst)
        return instances

    inst1 = connected_components(mask_t1)
    inst2 = connected_components(mask_t2)

    change_mask = torch.zeros_like(mask_t1, dtype=torch.long)
    if not inst1 and not inst2:
        return change_mask

    def iou(a, b):
        inter = (a & b).sum().float()
        union = (a | b).sum().float()
        return (inter / union).item() if union > 0 else 0.0

    for a in inst1:
        best = max((iou(a, b) for b in inst2), default=0.0)
        if best <= delta:
            change_mask |= a.long()
    for b in inst2:
        best = max((iou(a, b) for a in inst1), default=0.0)
        if best <= delta:
            change_mask |= b.long()
    return change_mask


def mixed_ceg(i_pixel_diff, i_rel, i_ins_diff, ignore_value=255):
    """Eq. 11. Combines pixel level and instance level CEG. Reliable pixels
    with I_rel == 1 keep the pixel level disagreement filtered by the
    instance level mask, unreliable pixels are marked ignore."""
    combined = i_pixel_diff * i_ins_diff
    out = torch.where(i_rel.bool(), torch.full_like(combined, ignore_value), combined)
    return out


# ---------------------------------------------------------------------------
# Loss functions, Section III-A, III-C, III-F, and Table I / Eq. 16
# ---------------------------------------------------------------------------

def supervised_loss(pred, target):
    """Eq. 2, pixel wise cross entropy on weakly perturbed labeled samples."""
    return F.cross_entropy(pred, target)


def consistency_loss(pred_strong, pseudo_label_weak, confidence_weak, tau=0.95):
    """Eq. 3, FixMatch style consistency loss gated by a confidence threshold."""
    mask = (confidence_weak.max(dim=1).values >= tau).float()
    per_pixel = F.cross_entropy(pred_strong, pseudo_label_weak, reduction="none")
    if mask.sum() == 0:
        return per_pixel.mean() * 0.0
    return (per_pixel * mask).sum() / (mask.sum() + 1e-6)


def vlm_guidance_loss(pred_weak, pred_strong, mix_diff_label, ignore_value=255):
    """Eq. 12, cross entropy against the mixed CEG pseudo label for both
    weak and strong perturbation predictions."""
    loss_s = F.cross_entropy(pred_strong, mix_diff_label, ignore_index=ignore_value)
    loss_w = F.cross_entropy(pred_weak, mix_diff_label, ignore_index=ignore_value)
    return loss_s + loss_w


def contrastive_consistency_loss(q_t1, q_t2, unchanged_mask, margin=1.0):
    """Eq. 15, batch balanced contrastive loss over the segmentation decoder
    features. unchanged_mask is 1 where the pair is unchanged (pull together)
    and 0 where changed (push apart with margin)."""
    dist = F.pairwise_distance(
        q_t1.permute(0, 2, 3, 1).reshape(-1, q_t1.shape[1]),
        q_t2.permute(0, 2, 3, 1).reshape(-1, q_t2.shape[1]),
    )
    flat_mask = unchanged_mask.reshape(-1).float()
    n_unchanged = flat_mask.sum().clamp(min=1.0)
    n_changed = (1.0 - flat_mask).sum().clamp(min=1.0)
    pull = (dist * flat_mask).sum() / n_unchanged
    push = (F.relu(margin - dist) * (1.0 - flat_mask)).sum() / n_changed
    return pull + push


def total_loss(outputs_labeled, y_labeled,
               outputs_weak_u, outputs_strong_u, confidence_weak_u, pseudo_weak_u,
               mix_diff_u, unchanged_mask_u,
               lambda_vl=0.1, lambda_ct=0.1):
    """Assembles Eq. 1 and Eq. 16, L = Lcr + lambda_vl * Lvl + lambda_ct * Lct,
    where Lcr = 0.5 * (Ls + Lu)."""
    l_s = supervised_loss(outputs_labeled["y_cr"], y_labeled)
    l_u = consistency_loss(outputs_strong_u["y_cr"], pseudo_weak_u, confidence_weak_u)
    l_cr = 0.5 * (l_s + l_u)

    l_vl_change = vlm_guidance_loss(outputs_weak_u["y_vl"], outputs_strong_u["y_vl"], mix_diff_u)

    q_t1 = outputs_strong_u["q_t1"]
    feat_size = q_t1.shape[-2:]
    mask_resized = F.interpolate(
        unchanged_mask_u.unsqueeze(1).float(), size=feat_size, mode="nearest"
    ).squeeze(1)
    l_ct = contrastive_consistency_loss(q_t1, outputs_strong_u["q_t2"], mask_resized)

    l_total = l_cr + lambda_vl * l_vl_change + lambda_ct * l_ct
    return {
        "total": l_total,
        "L_s": l_s.item(),
        "L_u": l_u.item(),
        "L_cr": l_cr.item(),
        "L_vl": l_vl_change.item(),
        "L_ct": l_ct.item(),
    }


# ---------------------------------------------------------------------------
# Evaluation metric, Eq. 17
# ---------------------------------------------------------------------------

@torch.no_grad()
def iou_change_class(pred_logits, target, ignore_value=255):
    pred = pred_logits.argmax(dim=1)
    valid = target != ignore_value
    tp = ((pred == 1) & (target == 1) & valid).sum().item()
    fp = ((pred == 1) & (target == 0) & valid).sum().item()
    fn = ((pred == 0) & (target == 1) & valid).sum().item()
    denom = tp + fp + fn
    return tp / denom if denom > 0 else 0.0


# ---------------------------------------------------------------------------
# Smoke test
# ---------------------------------------------------------------------------

def smoke_test():
    torch.manual_seed(0)
    device = torch.device("cpu")
    model = SemiCDVL(num_classes=2, feat_dim=64).to(device)
    optimizer = torch.optim.SGD(model.parameters(), lr=0.02, momentum=0.9)

    B, H, W = 2, 64, 64
    C_vlm = 6  # 2 foreground categories + 4 background categories, Section III-B1
    foreground_idx = [0, 1]

    # ---- labeled batch ----
    t1_l = torch.rand(B, 3, H, W)
    t2_l = torch.rand(B, 3, H, W)
    y_l = torch.randint(0, 2, (B, H, W))

    # ---- unlabeled batch, weak and strong views ----
    t1_u_w = torch.rand(B, 3, H, W)
    t2_u_w = torch.rand(B, 3, H, W)
    t1_u_s = t1_u_w + 0.05 * torch.randn(B, 3, H, W)  # stand-in for strong aug
    t2_u_s = t2_u_w + 0.05 * torch.randn(B, 3, H, W)

    # ---- dummy VLM probability maps and mixed CEG ----
    prob_t1 = torch.softmax(torch.randn(B, C_vlm, H, W), dim=1)
    prob_t2 = torch.softmax(torch.randn(B, C_vlm, H, W), dim=1)
    i_pixel_diff, i_rel = pixel_level_ceg(prob_t1, prob_t2, foreground_idx, gamma=0.5)

    mask_t1 = (prob_t1[:, foreground_idx].sum(dim=1) > prob_t1[:, foreground_idx].sum(dim=1).mean())
    mask_t2 = (prob_t2[:, foreground_idx].sum(dim=1) > prob_t2[:, foreground_idx].sum(dim=1).mean())
    ins_diffs = []
    for b in range(B):
        ins_diffs.append(instance_level_ceg(mask_t1[b], mask_t2[b], delta=0.0))
    i_ins_diff = torch.stack(ins_diffs, dim=0)

    mix_diff = mixed_ceg(i_pixel_diff, i_rel, i_ins_diff)

    unchanged_mask_u = (mix_diff == 0)

    model.train()
    for step in range(3):
        optimizer.zero_grad()

        out_l = model(t1_l, t2_l, out_size=(H, W))
        out_u_w = model(t1_u_w, t2_u_w, out_size=(H, W))
        out_u_s = model(t1_u_s, t2_u_s, out_size=(H, W))

        conf_weak = torch.softmax(out_u_w["y_cr"], dim=1)
        pseudo_weak = conf_weak.argmax(dim=1)

        losses = total_loss(
            out_l, y_l,
            out_u_w, out_u_s, conf_weak, pseudo_weak,
            mix_diff, unchanged_mask_u,
            lambda_vl=0.1, lambda_ct=0.1,
        )
        losses["total"].backward()
        optimizer.step()

        iou = iou_change_class(out_l["y_cr"], y_l)
        print(f"step {step} total {losses['total'].item():.4f} "
              f"L_s {losses['L_s']:.4f} L_u {losses['L_u']:.4f} "
              f"L_vl {losses['L_vl']:.4f} L_ct {losses['L_ct']:.4f} "
              f"train_IoUc {iou:.4f}")

    model.eval()
    with torch.no_grad():
        out_eval = model(t1_l, t2_l, out_size=(H, W))
        eval_iou = iou_change_class(out_eval["y_cr"], y_l)
    print(f"eval IoUc on dummy labeled batch: {eval_iou:.4f}")
    print("Smoke test completed without errors.")


if __name__ == "__main__":
    smoke_test()

Running this script trains for three dummy steps and prints a shrinking total loss alongside a rising training IoU on the change class, confirming that gradients flow correctly through the shared encoder, both decoders, the dual projection head, and every loss term without any shape mismatches between the change detection output and the mixed change event generation labels.

The bigger picture

SemiCD-VL lands at an interesting moment for change detection research. Foundation models trained on enormous, mostly natural image datasets are increasingly good at recognizing objects they were never explicitly taught to find in satellite imagery, and this paper is a clean demonstration of how to route that generalization into a task the foundation model was never designed for. The conceptual shift is subtle but real, instead of treating labeled data as the only trustworthy supervision, the model treats a frozen, task agnostic vision language model as a second, independent teacher, and builds explicit machinery, the dual projection head, the reliability masking, the mixed pixel and instance level filtering, to keep that teacher’s inevitable mistakes from contaminating training.

The transferability the authors gesture toward in their conclusion feels credible. Any semi supervised dense prediction task with a plausible category vocabulary, crop type mapping, flood extent delineation, deforestation tracking, could in principle borrow this same recipe with a different set of foreground and background category prompts. The honest limitations remain real too though. The pseudo labels are only as good as the vision language model’s grasp of the relevant categories, the two step pseudo label pipeline accumulates whatever errors each step introduces, and the loss weight sensitivity shown in Table IX means this is not a set it and forget it hyperparameter choice. Future work building genuinely multi temporal vision language models, ones that reason over both images at once rather than decomposing the problem, could remove an entire source of error here. Until that exists, this decompose then recombine strategy looks like a sensible and well validated way to get more out of a small labeling budget. For teams sitting on large unlabeled satellite archives and a tight annotation budget, the practical message is straightforward, ten percent labeled data plus a frozen vision language model can now outperform what used to require the whole dataset.

Frequently asked questions

Related reading