Conversational AI Quality Analyst

About The Role

Conversational AI is transforming how people interact with technology --- and the quality of those interactions depends on experts like you. We're looking for sharp, analytical thinkers based in the Bangalore metro area to evaluate AI chatbots and dialogue systems, identifying where conversations break down and how they can be improved.

Your insights will directly influence the next generation of conversational AI products. This is a fully remote, flexible contract role --- work on your schedule, on some of the most exciting AI challenges of our time.

• Organization: Alignerr
• Type: Hourly Contract
• Location: Remote
• Commitment: 10--40 hours/week

What You'll Do

• Evaluate AI chatbot conversations across diverse topics, assessing context retention, coherence, and natural flow
• Identify how and where conversations degrade across extended multi-turn interactions
• Design and execute test scenarios that stress-test conversational AI capabilities
• Rate chatbot responses for accuracy, helpfulness, safety, and tone consistency
• Document failure patterns, edge cases, and quality issues using structured feedback frameworks
• Provide clear, actionable recommendations to improve dialogue system performance
• Collaborate asynchronously with research and product teams to refine evaluation criteria

Who You Are

• 2+ years of experience in conversational AI evaluation, chatbot QA, dialogue system testing, or UX research for conversational products
• Strong understanding of how context, state, and coherence degrade across extended conversations
• Analytical mindset with a sharp eye for subtle quality issues in natural language interactions
• Excellent written communication skills in English --- you can clearly articulate what went wrong and why
• Detail-oriented and systematic in your approach to testing
• Self-motivated and comfortable managing your own schedule independently
• Familiar with conversation design principles and common chatbot failure modes

Nice to Have

• Experience with NLP, LLMs, or machine learning concepts
• Background in linguistics, cognitive science, or human-computer interaction
• Familiarity with evaluation frameworks such as BLEU scores, human preference ratings, or custom rubrics
• Experience writing test plans or QA documentation for conversational products
• Prior work with prompt engineering or red-teaming AI systems

Why Join Us

• Work on cutting-edge AI projects with top research labs and industry leaders
• Fully remote and flexible --- set your own hours and work asynchronously
• Build deep, marketable expertise in one of the fastest-growing areas of AI
• Autonomy and independence in how you approach your work
• Meaningful impact --- your evaluations directly shape AI products used by millions
• Potential for ongoing work and contract extension