Reinforcement learning with human feedback

Natural intelligence for Artificial Intelligence

Looking for human trainers for your AI models who can interact with model outputs, evaluate them, and put them to the test?

Generate quality data

Our annotators generate questions and responses for a variety of tasks and prompts, in many different languages, including English, Spanish, French, Portuguese, Arabic, Farsi, and others.

Illustration of AI generating a caption for the image of a woman riding horse
Illustration of AI generating an answer to the question "Will AI replace humans?"

Rank model outputs

Our humans interact with your model and provide a ranking on different attributes of its responses, such as relevance, safety, accuracy, etc. They also choose the more preferable of two outputs for A/B testing.

Illustration of AI generating an answer to the question "Will AI replace humans?"

Test for vulnerabilities

We provide red teaming services for forcing mistakes and intentionally trying to "break" models in order to find their weak spots and biases. The results are then reported back to help retrain the models.

Illustration of AI generating an example of a phishing email from a Nigerian prince

Our workforce

We offer access to a diverse workforce from around the globe which represents different cultural values. With their help, you can make sure that your models reflect different worldviews and opinions.

Get started

How it works

Project request

Send us information about your project and your human feedback needs. Our team will work with you to define to Scope of Work and to determine the best way to deliver the results that you require.

Interface setup

Our team will develop a custom interface for your use case and will hand-pick the most suitable workforce based on your requirements in terms of geography, demographics, etc.

API integration

We will provide you with API endpoints so that you can easily create jobs for our workforce and you can also define callback URLs where you wish to receive the results.

Send requests

Once the integration is complete, you can start sending requests to our workforce which contain model outputs, A/B tests, requests for data generation, and more.

Receive data!

Our workforce will start interacting with your model and send back responses within hours or even minutes, depending on your preferences. They can also document failure modes and biases.

Get in touch with us

Interested in learning more?

Email us


Like our page and stay in the loop about our work


Follow our visual journey and see snippets from our social impact!


Receive updates and news from the Humans in the Loop team


Connect with us to hear first about professional opportunities


Read our opinion pieces on the latest trends in the industry

Contact us

Please fill out this contact form and our team will be in touch shortly