Use Your Power for Purpose

At Pfizer, technology accelerates every facet of our mission. Within the AI Center of Excellence, the AI Quality Engineering team embeds trust, quality, and responsibility into every AI solution we deliver.

As a Staff QA Engineer, you will lead the design and operation of enterprise‑grade evaluation frameworks, guardrails, and continuous quality pipelines that keep our models performant and safe. Your work enables teams across the company to ship AI solutions with confidence—meeting rigorous standards for accuracy, fairness, security, and compliance while driving measurable impact for patients and colleagues alike.

Why this Role Matters

This role is pivotal in shaping the organization’s AI strategy and technical excellence. Your leadership will accelerate the adoption of cutting-edge AI technologies, foster a culture of innovation, and ensure that AI solutions are safe, reliable, and impactful at scale.

ROLE RESPONSIBILITIES

Define & lead enterprise AI evaluation frameworks for model performance, robustness, fairness, bias, and safety across LLMs, traditional ML, and agentic AI systems.

Architect and operate pre‑release & continuous evaluation (batch + online) integrated with CI/CD, including dataset versioning, synthetic test generation, and production quality gates.

Design, implement, and maintain guardrail services for hallucinations, prompt injection, PII leakage, unsafe content, and policy violations; instrument telemetry for detection, alerting, and drift tracking.

Run structured red‑teaming programs (jailbreaks, data exfiltration, adversarial prompts) and drive measurable mitigation with engineering teams; publish risk findings and remediation plans.

Encode quality & compliance policies as automated checks, partnering with Enterprise Architecture and Compliance to meet internal standards and emerging regulations (e.g., EU AI Act).

Establish cross‑use‑case quality SLOs & dashboards, reporting enterprise‑wide quality and safety metrics; lead post‑incident reviews and prevention programs.

Mentor QA/AI engineers on evaluation science, test design, and responsible AI practices; uplift engineering excellence and documentation quality across teams.

Collaborate with AI Engineering, Integrations & Enablement teams to ensure evaluation hooks, observability, and policy gates are first‑class in our SDKs, APIs, and shared components.

Collaborate with AI Innovation team on continuously assess emerging evaluation methods & tools, run rapid prototypes and PoCs, and advise on strategic adoption that improves quality, safety, and developer experience.

BASIC QUALIFICATIONS

Advanced degree in Computer Science, Engineering, Data Science, or related field.

7+ years in Software QA/AI QA/Quality Engineering, including ownership of test strategy for distributed or data‑intensive systems

Demonstrated experience building evaluation pipelines for ML/LLM systems: design of datasets/benchmarks, metrics for accuracy/robustness, and failure‑mode analysis (e.g., hallucination, bias, toxicity).

Strong fluency in software engineering best practices (automated testing, version control, CI/CD) and observability (logging, metrics, tracing) for model‑centric applications.

Hands‑on with MLOps concepts (model serving, deployment, feature/data versioning) and policy gating in release pipelines.

Ability to produce clear technical documentation (evaluation protocols, runbooks, API references) and developer‑friendly guides that scale adoption.

PREFERRED QUALIFICATIONS

Experience in pharma/healthcare or other regulated environments (validation, auditability, GxP-adjacent controls).

Deep knowledge of LLM safety & evaluation techniques: prompt‑injection defense, content moderation, RAG evaluation, context integrity, and bias/fairness testing.

Practical experience with evaluation libraries & frameworks (e.g., Ragas, TruLens, DeepEval, OpenAI Evals) and guardrail tooling integrated with platform SDKs.

Familiarity with agentic AI behaviors and evaluation, including planner/tool‑use reliability, memory safety, and multi‑agent orchestration checks.

Experience leading red‑team programs and incident response; evidence of measurable risk reduction and improvement in safety KPIs. (Competency expectations at Staff level)

Contribution to open‑source or industry forums on Responsible AI, safety evaluation, or testing.

 
Work Location Assignment: Hybrid

Please apply by sending your CV and a motivational letter in English

Purpose 

Breakthroughs that change patients' lives... At Pfizer we are a patient centric company, guided by our four values: courage, joy, equity and excellence. Our breakthrough culture lends itself to our dedication to transforming millions of lives.  

Digital Transformation Strategy

One bold way we are achieving our purpose is through our company wide digital transformation strategy. We are leading the way in adopting new data, modelling and automated solutions to further digitize and accelerate drug discovery and development with the aim of enhancing health outcomes and the patient experience.

Flexibility  

We aim to create a trusting, flexible workplace culture which encourages employees to achieve work life harmony, attracts talent and enables everyone to be their best working self. Let’s start the conversation!  

Equal Employment Opportunity 

We believe that a diverse and inclusive workforce is crucial to building a successful business. As an employer, Pfizer is committed to celebrating this, in all its forms – allowing for us to be as diverse as the patients and communities we serve. Together, we continue to build a culture that encourages, supports and empowers our employees.

Disability Inclusion

Our mission is unleashing the power of all our people and we are proud to be a disability inclusive employer, ensuring equal employment opportunities for all candidates. We encourage you to put your best self forward with the knowledge and trust that we will make any reasonable adjustments to support your application and future career. Your journey with Pfizer starts here!