Job Description:
CDM Smith is seeking a Quality Engineer to join our Digital Capital team. At CDM Smith, we see the
value of what AI can bring to our company, and we are actively adopting and implementing co-pilots
and Generative AI technologies to boost productivity across our business. As a Quality Engineer you
will be responsible for designing and implementing quality assurance processes to test AI models,
applications, and infrastructure. You will collaborate with cross-functional teams to identify
potential issues, optimize platform performance, and deliver high-quality AI-powered products that
meet our use expectations.
The ideal candidate for the Quality Engineer role is a detail-oriented professional with a strong
background in software testing and automation, coupled with a solid understanding of machine
learning concepts. They have a proven track record of developing and executing comprehensive test
plans, creating test automation frameworks, and ensuring the accuracy and performance of AI
models. They are proactive and possess excellent communication skills that enable them to
collaborate seamlessly with cross-functional teams and key stakeholders. Key responsibilities
include:
• Develop, implement, and maintain test strategies for AI models, APIs, and user-facing
applications.
• Create detailed test plans, test cases, and scripts to ensure comprehensive test coverage
across the code base.
• Test the accuracy, robustness, and model drift of various large language models.
• Build and maintain automated testing frameworks for backend systems, APIs, and user
interfaces.
• Implement automated pipelines for model validation and regression testing during
deployments.
• Conduct load, stress, and performance testing to ensure application and AI models handle
scaling efficiently.
• Identify, log, and track bugs using tools like Azure DevOps.
• Work closely with developers to troubleshoot and resolve issues.
• Test for vulnerabilities and risks in AI systems, APIs, and infra.
• Ensure compliance with data privacy regulations and ethical AI guidelines.
• Maintain detailed documentation of testing processes, tools, and results.
Skills and Abilities:
• Strong experience in software testing methodologies, tools, and processes.
• Hands on experience with test automation frameworks such as Selenium, Deep Eval, or
Playwright.
• Knowledge of Python, JavaScript, or other programming languages used for test scripting.
• Familiarity with API testing tools like Postman or REST Assured.
• Knowledge of Azure and Azure Kubernetes.
• Familiar with CI/CD pipelines and integrating automated tests into deployment workflows.
• Proficient in performance testing tools such as JMeter.
• Experience working with agile delivery methodologies and working in an agile delivery team.
• Excellent communication skills to articulate test results and collaborate across teams.
• Proactive and self-motivated with a passion for maintaining high-quality standards.
• Curious. Stays abreast of current and upcoming technologies and tools.