Senior AI QA Test Automation Engineer – CX Agent

XM is seeking a Senior AI QA Test Automation Engineer to take a leading technical role in AI quality and test automation. You will work primarily with our AWS Bedrock AgentCore-based CX Agent Suite, designing and building the evaluation and QA tooling required to validate AI agents reliably while contributing to technical standards across the organization.

Key responsibilities

• Act as the primary technical enabler for QA, building scalable AI/ML frameworks and tooling for the CX Agent Suite.
• Design and build AI evaluation pipelines that assess multi-agent responses for accuracy, relevance, tone, and safety compliance.
• Evaluate and prototype agent-orchestration frameworks for building self-evolving, autonomous QA agents.
• Collaborate with Data Science and Engineering to integrate AI-driven testing into GitLab CI/CD pipelines.
• Develop data-driven quality analytics and lead research into emerging AI testing methodologies.

Requirements

• BSc/MSc in Computer Science, AI, or a related discipline.
• 6+ years of experience in QA/Test Automation with a focus on designing automation frameworks.
• 1+ years of experience applying AI/ML in software testing with a track record of production implementation.
• Strong Python skills and experience in AWS-centric environments.
• Hands-on experience with LLM evaluation techniques, RAG, and multi-agent orchestration patterns.
• Familiarity with containerized cloud-native environments such as Docker and Kubernetes/EKS.

What we offer

• Attractive remuneration package.
• Intellectually stimulating work environment.
• Continuous personal development and international training opportunities.