Image Segmentation
Types of Segmentation
Semantic Segmentation — Assigns a class label to every pixel. Does not distinguish individual instances. Example: all cars are labeled 'car' regardless of how many there are. Instance Segmentation — Detects individual object instances AND segments them. Each car gets a separate mask with a unique ID. Panoptic Segmentation — Combines semantic (background 'stuff') and instance (foreground 'things') segmentation into one unified output.
U-Net Architecture
U-Net is the dominant architecture for semantic segmentation, especially in medical imaging. Encoder (contracting path): standard CNN backbone, progressively downsamples to capture context. Decoder (expanding path): progressively upsamples back to original resolution. Skip connections: concatenate encoder feature maps to corresponding decoder layers — preserves fine spatial details. The 'U' shape gives the architecture its name.
Segment Anything Model (SAM)
Meta's SAM (2023) is a foundation model for segmentation. Given any prompt (point, box, text, mask), it segments the corresponding object. Zero-shot — no fine-tuning needed for new objects. SAM 2 (2024) extends this to video segmentation in real-time.
U-Net Implementation
Full U-Net with skip connections in PyTorch, ready for semantic segmentation tasks.
For medical imaging: use Dice Loss or a combination of Dice + CrossEntropy. Dice Loss handles class imbalance better.
Finished reading? Mark it complete to earn your XP.