At Qualtrics, we create software the world’s best brands use to deliver exceptional frontline experiences, build high-performing teams, and design products people love. But we are more than a platform—we are the creators and stewards of the Experience Management category serving over 18K clients globally. Building a category takes grit, determination, and a disdain for convention—but most of all it requires close-knit, high-functioning teams with an unwavering dedication to serving our customers.

\n

When you join one of our teams, you’ll be part of a nimble group that’s empowered to set aggressive goals and move fast to achieve them. Strategic risks are encouraged and complex problems are solved together, by passing the mic and iterating until the best solution comes to light. You won’t have to look to find growth opportunities—ready or not, they’ll find you. From retail to government to healthcare, we’re on a mission to bring humanity, connection, and empathy back to business. Join over 5,000 people across the globe who think that’s work worth doing.

\n

Senior Software Developer in Test, Foundation - Seattle 

\n

Why We Have This Role

\nWe are looking for a Senior SDET to lead the development of our agentic evaluation framework within the ML team. As our AI capabilities expand, we need specialized quality engineering expertise to ensure robust testing processes and tooling for our agentic AI features. You will be responsible for designing, implementing, and scaling testing frameworks that ensure Qualtrics delivers responsible, high-quality AI solutions to our customers. This role is critical for standardizing how we test AI agents across the organization and establishing Qualtrics as a leader in responsible AI development.

\n

How You'll Find Success

\n\nDrive the quality assurance strategy for agentic features, ensuring standardization of testing approaches from a customer-centric perspective\nDevelop comprehensive evaluation tools and frameworks that enable automated testing of AI agent workflows\nDefine model launch readiness criteria for agentic features that align with both customer needs and regulatory requirements\nEstablish model risk management protocols that document AI system capabilities and limitations\nCreate knowledge transfer processes to disseminate best practices in AI testing across Qualtrics teams\nWork in a collaborative environment with ML Engineers, Research Scientists, Software Engineers, and Product Managers

\n\n

How You'll Grow

\n\nBecome an expert in the emerging field of AI agent testing and evaluation\nLead the development of innovative testing methodologies for complex AI systems\nMentor other quality engineers on AI-specific testing approaches and tools\nBuild upon our existing LLM evaluation infrastructure to create next-generation testing capabilities\nExpand your technical leadership through cross-functional collaboration and knowledge sharing\nStay at the forefront of AI testing practices by researching and implementing new techniques

\n\n

Things You'll Do

\n\nDesign and implement automated evaluation frameworks for testing AI agent behaviors and outputs\nBuild upon our existing AI test strategy and LLM-as-a-Judge infrastructure to create comprehensive testing solutions\nDevelop metrics and benchmarks to quantify AI system quality, reliability, and safety\nCreate reproducible test scenarios that simulate real-world user interactions with AI agents\nCollaborate with product teams to integrate quality processes into their AI development workflows\nIdentify and mitigate potential risks in AI agent behaviors, including hallucination detection and security guardrails\nPresent testing results and quality metrics to stakeholders across engineering and product teams

\n\n

What We're Looking For On Your Resume

\n\nBachelor's degree in Computer Science or related field\n5+ years of professional coding experience with strong Python skills\n2+ years experience building test automation frameworks and tools from scratch\n5+ years of relevant experience as part of an automated test team, with 1+ years as a senior contributor\nUnderstanding of Machine Learning concepts (deep learning, categorization, classification, data mining)\nExperience with Machine Learning evaluation frameworks\nExperience with CI/CD deployment automation (Jenkins, etc.)\nProficiency with testing distributed systems and SOA (Service Oriented Architecture)\nExperience applying various software testing techniques (equivalence class partitioning, boundary value testing, pairwise testing)\nStrong critical thinking and root cause analysis skills

\n\n

What You Should Know About This Team

\n\n\nThe team works on cutting-edge AI technologies that power many of Qualtrics' product experiences.\nOur team is pioneering quality engineering practices for AI systems and building the infrastructure that enables reliable, safe, and effective AI features.\nWe're actively working on several innovative AI projects, including security guardrails for AI models, LLM-based evaluation systems, and agentic workflows across various Qualtrics products.\nThe team has a strong culture of collaboration, innovation, and quality, and we're looking for someone who can help us elevate our testing practices for these complex systems.\n\n\n \n

Our Team’s Favorite Perks and Benefits

\n\nWellness Reimbursement: $300 per quarter for wellness activities including gym memberships, spa massages, workout equipment, meditation apps, and much more.\nExperience Bonus: $1800 to be used for an “Experience” of your choosing.\nAmazing QGroup Communities: MOSAIQ, Green Team, Qualtrics Pride, Q&Able, Qualtrics Salute, and Women’s Leadership Development, which exist as places for support, allyship, and advocacy.

\n\n

The Qualtrics Hybrid Work Model: Our hybrid work model is elegantly simple: we all gather in the office three days a week; Mondays and Thursdays, plus one day selected by your organizational leader. These purposeful in-person days in thoughtfully designed offices help us do our best work and harness the power of collaboration and innovation. For the rest of the week, work where you want, owning the integration of work and life.

\n

Qualtrics is an equal opportunity employer meaning that all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other protected characteristic.

\n

​​​​​​​Applicants in the United States of America have rights under Federal Employment Laws: Family & Medical Leave Act, Equal Opportunity Employment, Employee Polygraph Protection Act

\n

Qualtrics is committed to the inclusion of all qualified individuals. As part of this commitment, Qualtrics will ensure that persons with disabilities are provided with reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please let your Qualtrics contact/recruiter know.

\n

Not finding a role that’s the right fit for now? Qualtrics Insiders is the one-stop shop for all things Qualtrics Life. Sign up for exclusive access to content created with you in mind and get the scoop on what we have going on at Qualtrics - upcoming events, behind the scenes stories from the team, interview tips, hot jobs, and more. No spam - we promise! You'll hear from us two times a month max with fresh, totally tailored info - so be sure to stay connected as you explore your best role and company fit.

For full-time positions, this pay range is for base per year; however, base pay offered may vary depending on location, job-related knowledge, education, skills, and experience. A sign-on bonus and restricted stock units may be included in an employment offer, in addition to a range of medical, financial, and other benefits, based on eligibility criteria.

Washington State Annual Pay Transparency Range$139,500—$254,500 USD