Semantic Segmentation

What is semantic segmentation?

Semantic segmentation is done by selecting areas in the image and assigning a class to them. As the granularity in this case is class-based, separate instances of a class are not distinguished but are rather grouped depending on what class they belong to. The output is most usually a PNG mask with the colors of each class.

Semantic Segmentation Organ Pre-labeling ExampleSemantic Segmentation Organ Post-labeling Example
Semantic Segmentation Aerial Deforestation Pre-labeling example

Where can it be used?

As semantic segmentation has the ability to label each pixel on the image and assign it to a certain class, it can be used in a variety of industries and applications. 

  • Medical: mapping an entire DICOM slice of an organ
  • Geospatial: monitoring areas of deforestation or urbanization on satellite imagery
  • Automotive: identifying every single element in a road scene
  • Industrial: segmenting casting defects on metallic parts
  • Agriculture: distinguishing crops from weeds for smart herbicide usage
  • Retail: creating virtual dressing rooms which distinguish the user from the background

The Pros

  • Ultra-precise since every pixel must be assigned to one class.
  • Recent advancements in labeling automation have accelerated the annotation process through superpixel, domain-agnostic or domain-specific smart segmentation.

The Cons

  • Takes a lot of time to segment the image manually even when using smart tools.
  • Semantic segmentation identifies that there are certain categories in the image but not how many instances of each or how they overlap
Semantic Segmentation intersection pre-labeling exampleSemantic Segmentation intersection post-labeling example
Semantic Segmentation person and background pre-labeling exampleSemantic Segmentation person and background post-labeling example

Our Tips

  • Manual semantic segmentation can be performed with either a brush or a polygon. Some tools include a lot of features for changing the shape and size of the brush in order to make the process easier, but polygons frequently help to achieve higher precision.
  • For scenes in which it is important to also know how many units of a certain object are present, instance (or “instance-aware”) segmentation might be more appropriate. It uses the same panoptic segmentation principle but it assigns a unique class and color to each single instance. 
  • In order to facilitate the segmentation of adjacent objects, some tools offer the ability to draw on top of, or underneath, existing masks. This ensures that there will be no missed pixels in-between and makes it easier to draw the second mask.  

Tools and Platforms for Semantic Segmentation

Alegion logo
Diffgram logo logo
KILI Technology logo
Manthano logo logo
SuperAnnotate logo
V7 labs logo
Lightly logo
What data would you like to have annotated?

Use cases

Automotive annotation
Industry annotation
Retail annotation

Interested in having your dataset annotated with semantic segmentations?
Get in touch with our team at Humans in the Loop and a project manager will help you find the best solution to your computer vision needs!