Senior QA Software Engineer - AI

Are you passionate about ensuring the quality and reliability of AI-driven software that transforms how professionals work? Join a dynamic and highly skilled team at Thomson Reuters, where we invest deeply in AI technologies and explore emerging fields with the backing of a global leader. As a Senior QA Software Engineer - AI, you will play a key role in testing and validating intelligent systems that power the next generation of expert solutions across legal, tax, risk, and compliance domains.

About The Role:

As a Senior QA Software Engineer - AI, you will collaborate with a cross-functional team of product managers, UX designers, AI engineers, and software developers to design, implement, and execute comprehensive testing strategies for AI-powered solutions.

Test AI-Driven Systems:

• Design and implement comprehensive test strategies for advanced AI systems, including multi-component pipelines, retrieval-augmented generation (RAG), and custom AI agents with multi-step reasoning.
• Develop automated testing frameworks for AI model performance, accuracy, and reliability in production environments.
• Create test cases specific to AI/ML systems, including model validation, output quality assessment, and edge case testing.
• Validate AI model integrations with production software through robust APIs and scalable data pipelines.
• Test AI systems adapted to specialized domains, ensuring quality for expert systems in areas such as legal, tax, and compliance.

Innovate in AI Testing:

• Evaluate and prototype cutting-edge testing techniques for AI systems to address unique quality challenges.
• Develop proof-of-concept automated testing frameworks for new AI-driven features.
• Stay current with AI testing methodologies, quality metrics, and emerging technologies.
• Implement testing approaches for AI model drift, bias detection, and fairness validation.

Provide Technical Leadership:

• Break down functional requirements into comprehensive test specifications and automation strategies.
• Mentor junior QA engineers and facilitate technical discussions on AI testing best practices.
• Contribute to MLOps and LLMOps practices with a focus on quality assurance, continuous testing, and monitoring.
• Act as a thought leader in AI quality assurance, sharing expertise in company-wide forums and representing the organization in emerging testing technologies.

Ensure Quality & Operations:

• Implement comprehensive automated testing frameworks and monitoring systems for AI model performance, accuracy, and reliability.
• Create and maintain test automation suites using industry-standard tools and frameworks (Selenium, pytest, etc.).
• Ensure compliance with ethical AI principles, security standards, and regulatory requirements through thorough testing.
• Conduct systems analysis and recommend operational improvements to testing processes.
• Report deviations of quality to development teams and create informative bug reports with effective follow-up.
• Create detailed test reports documenting all issues found during testing iterations.

Collaborate Across Functions:

• Work closely with AI researchers, engineers, designers, and product teams to translate AI quality requirements into effective test strategies.
• Participate in daily stand-ups and iteration meetings in an Agile/Scrum environment.
• Optimize testing processes considering factors like test coverage, execution time, and resource usage.
• Support application feature enhancements by ensuring robust quality assurance for AI capabilities.

About You:

You are a fit for the position of Senior QA Software Engineer - AI if your background includes:

Required Skills and Experience:

• Bachelor's degree in Computer Science or equivalent experience.
• 5+ years of experience in software quality assurance and test automation; at least 2 years focused on testing AI/ML systems.
• 70% automation/development experience, 30% manual testing experience.
• Proficiency in Python and Java with strong coding skills for test automation.
• Experience with test automation tools and frameworks (e.g., Selenium, pytest, JUnit).
• Strong understanding of machine learning principles, evaluation metrics, and AI system testing methodologies.
• Knowledge of MLOps/LLMOps and the end-to-end lifecycle of AI-powered software applications.
• Experience testing AI model integrations in production systems using APIs and data pipelines.
• Good knowledge and experience working with relational databases (SQL, Oracle) and ability to write complex SQL scripts.
• Experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization tools (e.g., Docker, Kubernetes).
• Hands-on experience with source control and CI/CD systems such as Git and Jenkins.
• Good knowledge and experience working with Web Services and RESTful APIs.
• Excellent problem-solving skills and ability to work independently in a fast-paced environment.
• Strong communication skills and experience working in cross-functional teams.
• Ability to work on multiple projects while setting priorities to complete tasks within set timelines.

Preferred Qualifications:

• Experience testing AI-driven systems, agent-based architectures, or AI APIs from providers like OpenAI and Anthropic.
• Knowledge of testing vector databases, embeddings, or search-based AI systems.
• Experience with AI/ML frameworks (e.g., PyTorch, TensorFlow) for understanding model behavior.
• Familiarity with AI model evaluation techniques, including accuracy metrics, performance benchmarks, and bias testing.
• Experience creating effective end-to-end test plans for complex AI systems.
• Knowledge of NoSQL databases.
• Domain knowledge in legal, tax, or accounting.
• A portfolio of projects demonstrating expertise in testing AI/ML solutions and building quality frameworks for LLMs.

#LI-SS5

What’s in it For You?

About Us

Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news.

We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound exciting? Join us and help shape the industries that move society forward.

As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace.

We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here.

Learn more on how to protect yourself from fraudulent job postings here.

More information about Thomson Reuters can be found on thomsonreuters.com.