Video Annotation

What is video annotation?

Video annotation may take a variety of modalities: frame extraction and classification, event detection, object and activity tracking through bounding boxes, polygons, or keypoints, etc. Fortunately, these days video annotation is expedited by labeling tools which use features such as interpolation or smart object tracking so as to avoid having to label the video frame by frame.

Demo of people choosing products in a supermarket - after annotation
Demo showing fields - after annotation

Where can it be used?

Video annotation has its uses in very many industries which use sequential data or are interested in tracking changes over time. 

  • Medical: monitoring organs on ultrasound videos
  • Geospatial: analyzing footage from drone flights
  • Automotive: traffic monitoring at junctions for violations detection
  • Industrial: tracking and sorting products on a conveyor belt
  • Agriculture: monitoring the ripeness of fruits in an orchard over time
  • Retail: detecting shoplifting activities in malls
Demo showing fields - after annotation

The Pros

  • Different shapes (e.g. bounding boxes, polygons, keypoints) can all be used in video annotation.
  • Labelers can add static or dynamic attributes to the annotations in order to reflect constant or changing properties (e.g. a person tagged as a female vs a person tagged as walking for a certain duration).

The Cons

  • The annotation process may be time-consuming even when using interpolation and smart object tracking.
  • Videos are often data heavy and require annotation platforms to have robust processing capabilities.
Demo showing objects moving on a conveyor belt - - after annotation
Demo depicting traffic on the street

Our Tips

  • Linear interpolation works well on objects with a constant speed and a straight trajectory, where the labeler annotates the first and last frame while the rest are interpolated. Some tools now offer AI-powered visual object tracking which only requires the first frame to be annotated.
  • Very few tools support interpolation for shapes other than bounding boxes. On some tools it is possible to propagate annotations from one frame to the next one but some of the best tools out there offer interpolation also for polygons and keypoints.
  • One advanced annotation option is cross-camera tracking of one and the same object. Some platforms offer the capability to view synchronized footage from different cameras at the same time but annotators need to be given information about where these cameras are positioned in space so as to make this approach successful.
Demo depicting traffic on the street

Tools and Platforms for Video Annotation

Alegion logo
Diffgram logo
Manthano logo
SuperAnnotate logo
V7 labs logo
What data would you like to have annotated?

Use cases

Illustration shows Geospatial annotation
Illustration shows Medical annotation
Illustration shows Automotive annotation
Illustration shows Industry annotation
Illustration shows AgriTech annotation
Illustration shows Retail annotation

Interested in having a Human in the Loop label your video dataset?
Get in touch with our team at Humans in the Loop and a project manager will help you find the best solution to your computer vision needs!