Annotation Specialist $30,000 USD/year Pay is set based on global value, not the local market.
Most roles = hourly rate x 40 hrs x 50 weeks
Description
If precision matters more to you than speed, this position will align with your values. The labels you produce serve as training data for AI systems used daily by thousands of students. Accurate behavioral tagging makes the product more intelligent. Inconsistent annotations teach the model incorrect patterns.
LearnWith.AI creates AI-powered educational experiences by combining learning science, data analytics, and subject matter expertise. This position transforms raw video recordings of student sessions into high-quality, rubric-based labels the team relies on. You will review recorded sessions, detect important behavioral events, and follow precise guidelines to categorize what occurred and at what time. You will also assess LLM-generated pre-annotations, correct inaccuracies, and record edge cases to help engineers refine the system.
This is not gig-based, fragmented annotation work. It involves a consistent task queue within one product area, featuring direct feedback mechanisms, calibration against reference standards, and advancement determined by accuracy and consistency. If you value transparent expectations, quantifiable quality metrics, and contributions that directly influence model outcomes, we would like to hear from you.
What you will be doing
- Label student session videos by detecting, categorizing, and timestamping behavioral events according to a comprehensive rubric
- Assess and refine LLM pre-annotations by eliminating false positives, inserting overlooked events, and improving timestamp accuracy
- Document reasoning for ambiguous decisions with clear notes, citing rubric sections and the logic you applied
- Record edge cases and submit clarification requests for uncertain scenarios while maintaining an annotation log with session details
- Participate in calibration sessions, incorporate QA feedback, and adapt to rubric revisions to enhance accuracy progressively
What you will NOT be doing
- Construct AI models, conduct experiments, or perform research on student behavioral patterns
- Modify the annotation rubric or reinterpret category definitions according to subjective judgment
- Prioritize annotation speed over accuracy, consistency, or timestamp exactness
- Handle sporadic, unrelated tasks across diverse domains lacking context or feedback systems
Key responsibilities
This position ensures that student session videos are transformed into labeled datasets with ≥95% accuracy and time precision, providing reliable indicators of model performance improvements or declines.
Candidate requirements
- A minimum of 1 year of experience in data annotation, content moderation, QA evaluation, or comparable rubric-based review roles
- Excellent English reading comprehension with the capacity to adhere to detailed written instructions without deviation
- Capacity to maintain concentration and precision during 4–6 hours of video-intensive work daily
- Skill in identifying subtle on-screen and visual behavioral signals and labeling them uniformly across numerous sessions
- Proficient written communication abilities for documenting edge cases, reasoning, and clarification inquiries
- Stable internet connection suitable for video streaming
- Experience reviewing, correcting, and enhancing AI/LLM-produced annotations
Applying for a role? Here’s what to expect.
Crossover's skill assessment process combines innovative AI power with decades of human research, to take the guesswork, human bias, and pointless filters out of recruiting high-performing teams.
Chat-style
screening interview.
Cognitive 
aptitude test.
Prove real-world 
job skills.
Interview with the hiring manager.
Pass
proctored test.
Accept job offer.
Frequently asked questions
Reviews
Meet some people who've landed similar jobs
Recruitment sucks. So we’re fixing it.
The Olympics of work
It’s super hard to qualify—extreme quality standards ensure every single team member is at the top of their game.
Premium pay for premium talent
Over 50% of new hires double or triple their previous pay. Why? Because that’s what the best person in the world is worth.
Shortlist by skills, not bias
We don’t care where you went to school, what color your hair is, or whether we can pronounce your name. Just prove you’ve got the skills.
Join the world's largest community of 
AI-first remote workers.