Sr. AI QE (Quality Engineer)
full-time•Chennai
About this role
Role: Sr. AI QE (Quality Engineer)
# of Position: 1
Location: Bangalore/Pune/ Vadodra/ Chennai/ Hyderabad
Shift Hours: 3:30PM to 12:30AM IST
Summary
The Senior AI Quality Engineer is responsible for ensuring reliability, accuracy, safety, and performance of the Agentic AI platform. This role goes beyond traditional QA, focusing on LLM behavior validation, agent orchestration testing, hallucination detection, prompt regression, and production hardening of AI systems.
Key Responsibilities
- Design and execute AI-specific test strategies for agentic workflows (agent interactions, tool calling, RAG pipelines).
- Validate LLM outputs for accuracy, relevance, bias, safety, and policy compliance.
- Create automated test frameworks for prompt regression, agent / model routing, and decision consistency.
- Perform Integration testing with internal and external systems
- Test AI failure modes: hallucinations, partial responses, tool failures, latency spikes, and token exhaustion.
- Perform load, stress, and scalability testing for AI agents and orchestration services.
- Validate RAG quality (retrieval precision, grounding, citation accuracy).
- Support production readiness reviews, DR testing, and release sign-offs.
- Maintain test datasets, synthetic data, and golden response sets.
Required Skills & Experience
- 8–12 years in QA/Test Engineering with 3+ years in AI/ML or GenAI systems.
- Strong understanding of LLMs, prompt engineering, embeddings, RAG, and agent frameworks.
- Experience with test automation tools (Cypress / JMeter / or equivalent).
- Exposure to responsible AI, red-teaming, and model risk management.
- Python scripting for AI test automation.
- Familiarity with Azure OpenAI / OpenAI APIs, AI Search, vector databases.
- Experience testing APIs,
Nice to Have
- Experience in regulated domains (Medicare, Healthcare).
What you'll do
- The Senior AI Quality Engineer will design and execute AI-specific test strategies for agentic workflows, focusing on validating LLM outputs for accuracy, bias, and safety compliance. This includes creating automated frameworks for prompt regression and testing various AI failure modes like hallucinations and tool failures.
About Awign Expert
Welcome to Awign Expert, a division of Awign - India's largest work-as-a-service platform. At Awign Expert, we connect skilled professionals with exciting contractual or project-based work opportunities offered by top companies. Our mission is to empower professionals by matching them with projects that align with their skills, interests, and experience.
At Awign Expert, we understand the challenges faced by independent professionals in finding meaningful work and managing administrative tasks. That's why we act as a dedicated HR office for our Experts, handling the entire onboarding process, providing continuous feedback, resolving conflicts, and ensuring seamless payroll management.
Our goal is to create a hassle-free environment for our Experts, allowing them to focus solely on their work without the burden of administrative complexities. By partnering with Awign Expert, professionals gain access to a vast network of renowned companies and projects across various industries.
Whether you are an experienced freelancer or seeking to explore project-based opportunities, Awign Expert is here to support you every step of the way. We value your expertise and are committed to helping you find rewarding assignments that showcase your talents and contribute to your professional growth.
Join our community of skilled professionals today and let us handle the logistics, so you can unleash your true potential.
Ready to join Awign Expert?
Take the next step in your career journey
Frequently Asked Questions
What does a Sr. AI QE (Quality Engineer) do at Awign Expert?
As a Sr. AI QE (Quality Engineer) at Awign Expert, you will: the Senior AI Quality Engineer will design and execute AI-specific test strategies for agentic workflows, focusing on validating LLM outputs for accuracy, bias, and safety compliance. This includes creating automated frameworks for prompt regression and testing various AI failure modes like hallucinations and tool failures..
Is the Sr. AI QE (Quality Engineer) position at Awign Expert remote?
The Sr. AI QE (Quality Engineer) position at Awign Expert is based in Chennai, Tamil Nadu, India. Contact the company through Clera for specific work arrangement details.
How do I apply for the Sr. AI QE (Quality Engineer) position at Awign Expert?
You can apply for the Sr. AI QE (Quality Engineer) position at Awign Expert directly through Clera. Click the "Apply Now" button above to start your application. Clera's AI-powered platform will help match your profile with this opportunity and guide you through the application process.