Seg-Zero Teaches Segmentation Models To Reason From Scratch

Why segmentation models cannot just be asked nicely

Ordinary segmentation systems find objects that match a short category name, “person” or “car.” A newer task called reasoning segmentation demands something harder, answering an implicit question like “find something that provides sustained energy” or “who is most likely the player in this picture,” where the model has to combine visual evidence with logical inference before it can even decide what to segment. The influential earlier system LISA tackled this by fine tuning a multimodal language model on datasets full of category labels and short descriptions, teaching it to emit a special SEG token that a segmentation decoder could then act on.

The Seg-Zero authors point out three specific weaknesses in that approach. Performance holds up on data similar to the training set but drops sharply on anything out of domain. Supervised fine tuning tends to erode a model’s other general abilities, a well documented problem sometimes called catastrophic forgetting. And critically, nothing in the pipeline forces the model to show its work, so there is no explicit reasoning process a person could inspect or that the model could lean on for harder queries.

Their fix borrows an idea that had just shown dramatic results in text only language models. DeepSeek R1 demonstrated that training a language model with reinforcement learning alone, using nothing but reward signals and no curated reasoning examples, could cause a genuine chain of thought reasoning behavior to emerge on its own. Seg-Zero asks whether the same trick works when the output is not just a sentence but a set of pixel coordinates that a downstream segmentation model has to turn into a mask.

A reasoning model and a segmentation model that never talk during training

The architecture is deliberately split into two pieces that are trained completely differently. A reasoning model, built on Qwen2.5-VL, reads the image and the user’s question and outputs a bounding box plus two points marking the centers of the two largest circles that fit inside the target object. A separate segmentation model, SAM2, takes that bounding box and those two points as prompts and produces the final pixel mask. Only the reasoning model gets trained. The segmentation model stays completely frozen throughout, which the authors argue keeps its original mask quality intact rather than letting fine tuning degrade it, a problem they say affects prior systems that fine tune the segmentation decoder directly.

The training signal comes entirely from GRPO, the same reinforcement learning algorithm behind DeepSeek R1’s mathematical reasoning gains. For each training example, the model generates several candidate responses, each response is scored by a set of reward functions, and the model is nudged toward whichever responses in the batch scored above average. No individual response ever receives a ground truth explanation to imitate, only a numeric score.

Five rewards doing all the teaching

Because there is no example reasoning chain to copy, the entire behavior has to be shaped by reward design. The paper defines five separate reward functions. A thinking format reward simply checks whether the output is properly wrapped in think and answer tags. A segmentation format reward checks whether the bounding box and point coordinates appear in the right structure, in either a soft version that just checks the right number of values are present or a strict version that checks the exact expected keys. A bounding box IoU reward gives a point if the predicted box overlaps the ground truth box by more than half. A bounding box L1 reward gives a point if the predicted box corners are within 10 pixels of the true corners. A point L1 reward gives a point if the predicted points land within 100 pixels of the true points and inside the box.

None of these rewards ever tell the model what a good reasoning chain looks like. They only grade the final structured answer. Whatever reasoning text appears between the think tags is entirely the model’s own invention, shaped only by whichever reasoning patterns happen to correlate with getting the downstream boxes and points right.

What the training curves actually show happening

The appendix includes a detail that a reader skimming only the results tables would miss. Early in training, the model’s response length actually shrinks before it grows. The authors explain this by pointing to the order in which the reward functions become satisfiable. The format rewards, which just require the right tags and keys, converge to their maximum value within the first several dozen steps, since getting the structure right is comparatively easy. Once that happens, the model temporarily has nothing more to gain from writing longer, more exploratory reasoning text, so responses get shorter and more clipped. Only once the model has locked in the format does the accuracy rewards, tied to genuinely getting the box and points right, start to dominate the optimization signal, and that is when response length climbs back up and the actual reasoning chains lengthen and grow more detailed.

This produces a strange but sensible order of skill acquisition, learn the shape of a good answer first, and only then learn what makes an answer actually correct. It also means anyone trying to reproduce this kind of training should expect a nonmonotonic response length curve and not mistake the early dip for a sign of instability.

Where the ablation tables complicate the simple story

The headline comparison in Table 1 is straightforward, reinforcement learning beats supervised fine tuning on both in domain and out of domain data, and adding the chain of thought reward on top of plain reinforcement learning helps further, especially out of domain, 53.8 versus 51.3 on ReasonSeg. That part matches the abstract closely. The ablation section, though, surfaces two results that are genuinely counterintuitive and worth more attention than the paper itself gives them.

Binary rewards beat continuous rewards

Table 6 compares the hard, threshold based accuracy rewards described above against soft versions that instead hand back the raw IoU value or a continuous function of L1 distance. Standard machine learning intuition says a continuous reward should carry more information per training example than a coarse binary signal and should therefore train faster or generalize better. The opposite happens here. The soft reward scores 70.2 on RefCOCOg against 73.6 for the hard reward, a meaningful gap, and even its slight edge on ReasonSeg is not enough to make up the difference in the combined score the authors report. A plausible explanation, though the paper does not fully unpack it, is that GRPO’s advantage computation already normalizes rewards within a group of sampled responses, so the extra granularity a continuous reward provides may just add noise to that normalization rather than a cleaner gradient signal.

The harder format reward wins on generalization

Table 7 compares the soft segmentation format reward, which only checks that bbox and points values are present with the right count, against a strict version that requires the exact keys bbox, points_1, and points_2. The strict version is explicitly harder to satisfy and the authors note in the text that it progresses more slowly in early training because correctly formatted samples are rarer to stumble onto. Despite that slower start, it ends up 2.3 points ahead on ReasonSeg, the out of domain benchmark, while giving up only 0.6 points on the in domain RefCOCOg set. A reward that is harder to satisfy early on ends up producing a model that generalizes better later on, which is a useful data point for anyone designing reward shaping for a similar reinforcement learning pipeline, easy early rewards are not automatically the safer choice.

The KL coefficient sweet spot does not agree across benchmarks

Table 3 sweeps the KL divergence penalty that keeps the reinforcement learning policy from drifting too far from the pretrained model’s original behavior. A coefficient of 5e-3 gives the best combined score, but it is not simultaneously optimal on both benchmarks, RefCOCOg actually peaks slightly higher at 1e-3, 73.7 versus 73.6, while ReasonSeg peaks at 5e-3 with 53.8. The gap is small, but it is a reminder that a single global hyperparameter choice is a compromise between in domain and out of domain performance rather than a value that is strictly best for both, something worth checking rather than assuming when adapting this recipe to a new dataset.

A claim worth reading carefully, the benchmark ground truth itself

The most editorially interesting section of the paper is not in the main results at all, it is tucked into Appendix C. The authors report manually inspecting 100 random samples from the RefCOCO family of datasets and finding that roughly 5 percent of the ground truth masks inaccurately represent the true object size, and that nearly 70 percent show imprecise edge handling. Their argument is that because Seg-Zero’s segmentation model stays frozen and was never fine tuned to match these particular annotation quirks, the gIoU metric, which is sensitive to exactly this kind of boundary mismatch, may be understating Seg-Zero’s actual localization quality relative to competitors that did fine tune their segmentation decoder directly on this flawed data and therefore learned to reproduce its specific biases.

The headline results, and where the real gap comes from

On zero shot reasoning segmentation, Table 9 shows Seg-Zero-7B reaching 57.5 gIoU on the ReasonSeg test set, ahead of LISA-13B-LLaVA1.5’s 53.8 despite using a smaller reasoning backbone and, crucially, ahead of the Qwen2.5-VL-3B plus SAM2 baseline’s 47.6 by a wide margin, which isolates how much of the gain comes from the reinforcement learning training itself rather than just from using strong pretrained components.

The scale ablation in Table 8 adds a further data point, reasoning ability scales with the underlying model size in a fairly clean progression, Qwen2-VL-2B lands at 37.2 on ReasonSeg, Qwen2.5-VL-3B reaches 56.1, and Qwen2.5-VL-7B reaches 57.5. The jump from 2B to 3B is far larger than the jump from 3B to 7B, suggesting there may be a capability threshold the reasoning model needs to cross before the reward driven training can take hold, rather than a smooth linear return on scale.

That quote points to a detail easy to overlook, the emergent reasoning is not quite as spontaneous as “trained exclusively via reinforcement learning without any reasoning data” makes it sound at first read. Table 5 shows that simply removing a single worked example from the user prompt template, with no other change, drops RefCOCOg from 73.6 to 72.1 and ReasonSeg from 53.8 to 49.4. The model is not inventing the idea of reasoning entirely unprompted, it is being shown the shape of a reasoning response in the prompt itself and then reinforcement learning is what teaches it to fill that shape in with content that actually helps it get the right answer. The reasoning content is genuinely learned from reward alone, but the scaffold that reasoning fits into is not learned from zero in quite the same sense the framing initially suggests.

What this means for building similar systems

The clearest transferable idea here is the decoupled architecture itself, keep a strong pretrained perception model frozen and train a much smaller, cheaper adapter, in this case a language model deciding where to point, using reinforcement learning against automatically computable rewards. That pattern avoids the two failure modes the authors identify with end to end fine tuning, forgetting general capability and degrading a perception model that already worked well.

The second transferable idea is more subtle and comes straight out of the ablation tables rather than the main narrative, reward design choices that look like they should make training easier, continuous rewards instead of binary ones, loose format requirements instead of strict ones, do not automatically produce better generalization, and in this paper they measurably did not. Anyone designing a reward driven training pipeline for a similarly structured task should treat “make the reward easier to satisfy” as a hypothesis to test against a genuinely out of domain evaluation set, not as a safe default.

The appendix’s claim about flawed benchmark ground truth is worth taking as a prompt to check your own evaluation data rather than as evidence about this specific model. If a system reports weaker numbers than expected against a widely used benchmark, manually inspecting a sample of that benchmark’s annotations, the way this paper’s authors did, is a reasonable diagnostic step, even though a claim of “the benchmark is wrong” should always be treated with more scrutiny when it comes from the team whose results it flatters.

Honest limitations

The authors are direct about scope in their own limitations section, Seg-Zero currently handles only single object reasoning segmentation, and multi object or instance level reasoning segmentation is left as future work. The training set is also small and narrow by construction, only 9,000 samples drawn from RefCOCOg, which the authors present as a strength, since it shows the reasoning behavior does not require a large curated dataset, but it also means the diversity of scenes and query types the model was ever exposed to during training is modest compared to the range of queries ReasonSeg and real deployments might present.

The frozen segmentation model design, while protecting mask quality from degradation, also caps how much the system can improve on cases where SAM2 itself struggles, since no gradient signal from the reasoning model’s training ever reaches the segmentation stage. And the reliance on a worked example inside the prompt template, shown by the ablation in Table 5 to be load bearing rather than optional, means the “reasoning from zero” framing is more precisely “reasoning content learned from zero, within a scaffold the prompt still provides.”

A reference implementation of the reward functions

The block below is a simplified, runnable Python implementation of the five reward functions described in Section 3.3, along with a minimal GRPO style advantage calculation, built to illustrate the mechanism rather than to reproduce the authors’ training pipeline exactly. It includes a smoke test against a few hand built sample responses.

import re
import json
import math

# ---- Format rewards (Section 3.3) ----

def thinking_format_reward(response: str) -> float:
    has_think = bool(re.search(r"<think>.*?</think>", response, re.S))
    has_answer = bool(re.search(r"<answer>.*?</answer>", response, re.S))
    return 1.0 if (has_think and has_answer) else 0.0

def extract_answer_json(response: str):
    match = re.search(r"<answer>(.*?)</answer>", response, re.S)
    if not match:
        return None
    try:
        return json.loads(match.group(1).strip())
    except json.JSONDecodeError:
        return None

def segmentation_format_reward(response: str, strict: bool = True) -> float:
    data = extract_answer_json(response)
    if data is None:
        return 0.0
    if strict:
        keys_ok = "bbox" in data and "points_1" in data and "points_2" in data
        shape_ok = keys_ok and len(data["bbox"]) == 4 and len(data["points_1"]) == 2 and len(data["points_2"]) == 2
        return 1.0 if shape_ok else 0.0
    has_bbox = "bbox" in data and len(data.get("bbox", [])) == 4
    has_points = any(k.startswith("points") for k in data)
    return 1.0 if (has_bbox and has_points) else 0.0


# ---- Accuracy rewards (Section 3.3) ----

def box_iou(box_a, box_b) -> float:
    ax1, ay1, ax2, ay2 = box_a
    bx1, by1, bx2, by2 = box_b
    inter_x1, inter_y1 = max(ax1, bx1), max(ay1, by1)
    inter_x2, inter_y2 = min(ax2, bx2), min(ay2, by2)
    inter_area = max(0, inter_x2 - inter_x1) * max(0, inter_y2 - inter_y1)
    area_a = (ax2 - ax1) * (ay2 - ay1)
    area_b = (bx2 - bx1) * (by2 - by1)
    union = area_a + area_b - inter_area
    return inter_area / union if union > 0 else 0.0

def bbox_iou_reward(pred_box, gt_box, hard: bool = True) -> float:
    iou = box_iou(pred_box, gt_box)
    return (1.0 if iou > 0.5 else 0.0) if hard else iou

def bbox_l1_reward(pred_box, gt_box, threshold: float = 10.0, hard: bool = True, image_size: float = 840.0) -> float:
    dist = sum(abs(p - g) for p, g in zip(pred_box, gt_box)) / 4.0
    return (1.0 if dist < threshold else 0.0) if hard else max(0.0, 1.0 - dist / image_size)

def point_l1_reward(pred_point, gt_point, pred_box, threshold: float = 100.0) -> float:
    x1, y1, x2, y2 = pred_box
    px, py = pred_point
    inside_box = x1 <= px <= x2 and y1 <= py <= y2
    if not inside_box:
        return 0.0
    dist = math.hypot(pred_point[0] - gt_point[0], pred_point[1] - gt_point[1])
    return 1.0 if dist < threshold else 0.0


# ---- Total reward and GRPO style advantage (Section 3.4) ----

def total_reward(response: str, gt_box, gt_point_1, gt_point_2, strict_format: bool = True, hard_accuracy: bool = True) -> float:
    score = thinking_format_reward(response)
    score += segmentation_format_reward(response, strict=strict_format)
    data = extract_answer_json(response)
    if data is not None and "bbox" in data:
        score += bbox_iou_reward(data["bbox"], gt_box, hard=hard_accuracy)
        score += bbox_l1_reward(data["bbox"], gt_box, hard=hard_accuracy)
        if "points_1" in data:
            score += point_l1_reward(data["points_1"], gt_point_1, data["bbox"])
        if "points_2" in data:
            score += point_l1_reward(data["points_2"], gt_point_2, data["bbox"])
    return score

def grpo_advantages(rewards: list) -> list:
    mean_r = sum(rewards) / len(rewards)
    variance = sum((r - mean_r) ** 2 for r in rewards) / len(rewards)
    std_r = math.sqrt(variance) + 1e-6
    return [(r - mean_r) / std_r for r in rewards]


# ---- Smoke test using the three samples from Figure 2 ----

def smoke_test():
    gt_box = [35, 60, 354, 423]
    gt_point_1 = [183, 235]
    gt_point_2 = [168, 305]

    sample_1 = '<think>Thought process...</think><answer>{"bbox": [35, 60, 354, 423], "points_1": [183, 235], "points_2": [168, 305]}</answer>'
    sample_2 = '<think>The person in the brown coat...</think><answer>{"bbox": [35, 60, 354, 423]}</answer>'
    sample_3 = 'The people wearing a santa hat is on the right.<answer>{"bbox": [25, 70, 300, 410]}</answer>'

    rewards = [
        total_reward(sample_1, gt_box, gt_point_1, gt_point_2),
        total_reward(sample_2, gt_box, gt_point_1, gt_point_2),
        total_reward(sample_3, gt_box, gt_point_1, gt_point_2),
    ]
    advantages = grpo_advantages(rewards)

    assert rewards[0] > rewards[1] > rewards[2]
    assert abs(sum(advantages)) < 1e-6
    print("Smoke test passed.")
    print("Rewards", rewards)
    print("Advantages", advantages)

if __name__ == "__main__":
    smoke_test()

Conclusion

Seg-Zero’s central contribution is showing that the DeepSeek R1 style recipe, pure reinforcement learning with format and accuracy rewards and no curated reasoning demonstrations, transfers from text reasoning into a genuinely pixel level, multimodal task. Trained on only 9,000 examples, the resulting 7B model beats the previous best 13B reasoning segmentation system on the ReasonSeg benchmark, and it does so while keeping general visual question answering ability intact, something the supervised fine tuning baseline in the same paper visibly fails to do.

The more interesting story, though, sits one layer beneath that headline result, in the ablation tables the paper includes but does not dwell on. Binary threshold rewards beating continuous ones, a stricter format requirement generalizing better despite a slower start, and a KL penalty whose optimal value depends on which benchmark you care most about, together paint a picture of reward design as a genuinely empirical exercise where the intuitive choice is not reliably the right one. Anyone building a similar RL pipeline for a grounding or localization task would do well to treat those four ablation tables as more instructive than the paper’s own framing suggests.

The appendix’s claim that its own benchmark ground truth is flawed deserves a place in the conversation too, both because it is a genuinely useful methodological point about gIoU’s sensitivity to annotation noise, and because it is a reminder to read self reported explanations for underwhelming or surprising numbers with the same rigor applied to the headline results. The two supplementary localization metrics the authors introduce to sidestep the dispute, both landing near 90 percent, are a reasonable way to triangulate the claim, but they come from the same team and were not checked independently.

Taken as a whole, Seg-Zero is a well constructed demonstration that decoupling perception from reasoning, freezing the part that already works and training only the part that needs to learn to think, is a viable and data efficient way to bring emergent chain of thought behavior into a vision task. Its most durable lesson for practitioners may not be the final accuracy numbers at all, but the specific, sometimes counterintuitive reward design choices its own ablations quietly reveal.

Frequently asked questions

Related reading