Job Title: Test Architect (GenAI)

 Expert Job Overview: We are seeking a skilled GenAI Testing Automation Expert to lead the design, development, and execution of automated testing frameworks for Generative AI systems. The role involves working with LLMs, AI agents, and automation tools to ensure robust, scalable, and reliable testing processes. Key Responsibilities: Design and implement automated test suites for GenAI models and workflows Develop and run structured test cases, including unit, integration, and SIT test cases Evaluate GenAI outputs using contextual comparison methodologies and validation metrics Collaborate with development teams to integrate testing into CI/CD pipelines Use tools like LangChain, AgentGPT, SageMaker, and custom APIs for automation Analyze test results and provide actionable insights for model improvement Maintain documentation for test frameworks and ensure compliance with data standards Required Skills & Qualifications: 3–5 years of experience in automation testing, preferably with AI/ML systems Proficiency in Python and automation frameworks Experience with GenAI platforms (e.g., OpenAI, Amazon SageMaker, LangChain) Strong understanding of LLM behavior, prompt engineering, and evaluation techniques Familiarity with APIs, cloud platforms, and version control tools Excellent problem-solving and communication skills