Introduction to OCR AI
Automated document processing is essential for modernizing the workflows in today’s enterprises, and it’s relevant to a variety of processes, such as expense management, accounts payable automation, procurement, accounting, insurance, user and employee onboarding, loan applications, underwriting, etc.
However, processing unstructured data such as PDFs or scanned documents using AI is not a straightforward task. High-quality data annotation is essential in order to train and maintain high-quality document processing and parsing AI tools.
Arabic Documents OCR Dataset
Looking for a dataset you’d like to test your Arabic documents? Why not try our new Arabic Documents OCR Dataset – it’s as easy as 1, 2, 3!
Challenges and best practices
Based on our extensive experience annotating data for document processing and OCR, we are sharing below some of our best practices and tips on how to ensure your AI project will be a success:
This is why it’s really important to start with the right training data and to adopt a human-in-the-loop approach in order to continuously improve the document classification and data extraction AI models that you are using!
Types of annotations for OCR AI
Below we will feature several different use cases of data annotation for document processing and some of the best practices.
Document recognition
Many document scanning or processing apps start with the simple task of recognizing that there is a document on the image and detecting its position, especially mobile applications. In order to train such a detector, you need a dataset where the documents are precisely annotated using polygonal or semantic segmentation masks in order to detect the page outline.
Document classification
In order to correctly process documents, it may be necessary to classify them first by type: eg. invoice, receipt, contract, book, magazine, etc. The required annotation here is in the form of tags, or by splitting the dataset in different folders. In addition, documents may be classified according to the language they are in, but bear in mind that there may be multiple languages per document.
Text detection
In order to understand the text on the document, you would first need to run a text detector to identify the actual existence of a text. The necessary annotation for this application is a bounding box or a polygonal annotation, which could be either on a block, paragraph or a line level.
Text transcription
Once the text is detected, you would need to understand the actual words and turn them into machine-readable text by applying an OCR model. Most OCR systems now work on a word or even a row level rather than a character level, so the abbreviation is now becoming obsolete, since it means Optical Character Recognition. The dataset requirement here is a bounding box with a transcription of the value in each box.
Document parsing
Once the text on the document has been converted into plain text, it’s important to structure it in the same way, based on the hierarchy of headings and subheadings, as well as the existence of tables, charts, captions, and other text sections. This requires a dataset where the bounding boxes are also categorized based on the type of the text (eg. H1, H2, H3, etc).
Entity extraction
In some cases, such as invoice processing, it’s only required to detect and extract the values of particular entities of interest. For example, these can include the total price of invoice, the unit price, the number of units, the currency, the date of issue, the due date, etc. This can be a tricky task because the positioning and formatting can vary on different invoices, some invoices may be missing some items, some items may be spread across different pages, etc.
Document recognition
Many document scanning or processing apps start with the simple task of recognizing that there is a document on the image and detecting its position, especially mobile applications. In order to train such a detector, you need a dataset where the documents are precisely annotated using polygonal or semantic segmentation masks in order to detect the page outline.
Document classification
In order to correctly process documents, it may be necessary to classify them first by type: eg. invoice, receipt, contract, book, magazine, etc. The required annotation here is in the form of tags, or by splitting the dataset in different folders. In addition, documents may be classified according to the language they are in, but bear in mind that there may be multiple languages per document.
Text detection
In order to understand the text on the document, you would first need to run a text detector to identify the actual existence of a text. The necessary annotation for this application is a bounding box or a polygonal annotation, which could be either on a block, paragraph or a line level.
Text transcription
Once the text is detected, you would need to understand the actual words and turn them into machine-readable text by applying an OCR model. Most OCR systems now work on a word or even a row level rather than a character level, so the abbreviation is now becoming obsolete, since it means Optical Character Recognition. The dataset requirement here is a bounding box with a transcription of the value in each box.
Document parsing
Once the text on the document has been converted into plain text, it’s important to structure it in the same way, based on the hierarchy of headings and subheadings, as well as the existence of tables, charts, captions, and other text sections. This requires a dataset where the bounding boxes are also categorized based on the type of the text (eg. H1, H2, H3, etc).
Entity extraction
In some cases, such as invoice processing, it’s only required to detect and extract the values of particular entities of interest. For example, these can include the total price of invoice, the unit price, the number of units, the currency, the date of issue, the due date, etc. This can be a tricky task because the positioning and formatting can vary on different invoices, some invoices may be missing some items, some items may be spread across different pages, etc.
Tools we love
Here are some of our tips and recommendations on the best tools we’ve used for this type of annotation which can hopefully be useful for anyone working on document parsing or processing models.
As an open source tool, LabelStudio can be great for getting started with document annotation, given its customizable UI and its ability to support different types of annotation.
V7 is a more advanced tool which offers great auto-labeling features for bounding boxes and polygons and will speed up your annotation process significantly.
Best Annotation Tools for 2021
Found your humans – now all you need is the right tool for the job? Here’s our review of the most popular tools for 2021!
How to use a human-in-the-loop for Document annotation for OCR AI
Document processing is a continuous operation within enterprises and in order to keep models up to date and to handle the inevitable data drift, it’s important to use human input on a continuous basis, not just for the initial training of your models. Here are some of the ways in which humans can be plugged into the entire MLOps cycle:
1. Document collection: our humans in the loop can collect multilingual datasets of different types of documents from a variety of geographical locations
2. Ground truth annotation: in order to train your initial models, we offer full dataset annotation from scratch in batches: anything from classification to bounding box annotation, transcription, or page segmentation
3. Output validation with active learning: once you’ve trained an initial model, we can use it in order to pre-annotate a large part of the dataset, which will both increase the speed of the annotators and the impact of their work, by setting up an active learning workflow and prioritizing instances where your model is least certain
4. Adversarial example collection: once you’ve trained an initial model, we can expand your core dataset with additional difficult and challenging edge cases, such as blurry, dark, or tilted images, or specific classes that are underrepresented, depending on the failure modes of your model during testing
5. Real-time edge case handling: once you have a model in deployment, our humans-in-the-loop are available 24/7 to handle potential edge cases that appear in real time or close to real time, using a simple API request and sending the correct response in seconds in order to ensure a second layer of verification for your model’s most critical responses
Wondering who is annotating your data?
When you are hiring a company to help you with your annotation needs, you frequently never meet the workers who are labeling your data. We want to change this and present to you the inspiring stories of our annotators!
Does this sound like something you’d like to try out? Get in touch with us and we’d be happy to schedule a free trial so as to explore how we can best help you with your retail annotation needs!