Ultimate Guide to Car Damage Annotation for Visual Inspection AI

Challenge

Best Practice

Each auto insurer and manufacturer may have a different taxonomy of how car damages are classified and assessed. In addition, the assessment of car damages is a very complex process because not all damages may be shown on the surface of the car. The detection of damages may not be accurate or precise enough, especially if the image quality is low or there are unusual types of damages. In addition, damage severity can be a very subjective notion, while damage area may be difficult to estimate based on just an image or a video.

In preparation for the annotation process, we work with clients to define very clear instructions for how to annotate damages. For example, when annotating a group of scratches close together: should they be labeled separately or as a group? Whenever there is a missing part, like a mirror, should the empty space where it is supposed to be located be labeled?

1. Take special care of the taxonomy of damages, severity can be very subjective
2. Be careful with zoomed in images, where the location of the damage is unclear
3. Make sure all types of damages are included: flat tyre, missing window, rust

In vehicle inspection datasets, there is a big challenge to reach equal distribution of different types of damages. For example, it’s easy to collect images of deformed bumpers, broken tail lights or scratched doors, but images of roof damages or missing windows may be both difficult to acquire in large enough quantities and difficult to detect in general. In addition, when scaling your solution to different markets, you need to take into account the variety of local brands and makes which are most commonly used.

Using our adversarial example collection services, once you have identified underrepresented classes or failure modes for your model, we are able to collect specific instances which represent difficult or challenging setups and environments. Does your model perform worse at dusk or at night time? We will be able to collect images specifically meant to cover these edge cases. Does your model perform worse in Eastern Europe, where many cars are older than what is usually represented in Western-centric datasets, or perhaps they include Soviet-made cars like Ladas and Moskvich? We can collect the necessary data in order to make sure your model is as accurate as possible in different locations.

During deployment, damage assessment AI is supposed to provide an evaluation of damages almost immediately, but there is a considerable risk of providing an inaccurate estimate if the model certainty is low or if the car at hand has an unusual damage or is a fraud attempt.

Our best practice in these cases is to hire a human-in-the-loop team and train them according to the specific requirements of each insurer. They cover different geographic locations and time zones in order to provide 24/7 coverage, and are able to perform real-time handling of edge cases and alerts using simple API requests. In this way, you get a guaranteed second verification layer which will help to avoid wrong estimations.