Sam2Rad architecture The Sam2Rad architecture incorporates a (hierarchical) two-way attention module to predict prompts for queried objects. Each object/class is represented by learnable queries . The Prompt Predictor Network (PPN) predicts bounding box coordinates of the target object , an intermediate mask prompt , and high-dimensional prompt embeddings . The prompt embeddings can represent various prompts suitable for the task, such as several point prompts or high-level semantic information. The predicted prompts (i.e., , , & ) are then fed to SAM’s mask decoder to generate the final segmentation mask. PPN also supports multi-class medical image segmentation by using class-specific queries .

Sam2Rad Explained: Teaching SAM2 to Prompt Itself on Ultrasound

2 Comments / Machine Learning, Image Segmentation /

Give the Segment Anything Model a box drawn around a bone in an X-ray and it does something close to magic. Give it the same kind of box on an ultrasound image of a wrist and it frequently falls apart,…

Sam2Rad Explained: Teaching SAM2 to Prompt Itself on Ultrasound Read More »