Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.
Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.
Job DescriptionWe are seeking a highly skilled Senior Software Test Engineer with deep functional knowledge of financial systems to lead the automation of data quality checks across our large-scale data pipelines, which process petabytes of data.
As part of our Finance Data Platform, you will design, implement, and maintain robust testing frameworks to ensure data integrity, reliability, and accuracy. Your expertise in PySpark, Python, SQL and Hive will be instrumental in driving quality across our data ecosystem.
Responsibilities
Design, develop, and maintain automated data quality checks for complex data pipelines handling petabyte-scale datasets.
Implement scalable data validation frameworks using PySpark, PyTest, Python, SQL, and Hive, ensuring comprehensive test coverage.
Collaborate with Data Engineers and DevOps teams to integrate automated tests into CI and CD workflows.
Analyse test results, identify data anomalies, and provide actionable insights to resolve data quality issues.
Develop monitoring and alerting solutions for data quality in production environments.
Document test processes, standards, and best practices. Mentor junior engineers on data quality automation.
Continuously improve test frameworks and processes to optimize performance, scalability, and reliability.
This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.
Qualifications
Basic Qualifications
-2+ years of relevant work experience and a Bachelors degree, OR 5+ years of relevant work experience
-Bachelors or Masters degree in Computer Science, Engineering, or related field.
-5 plus years of experience in test engineering or data engineering roles, with a focus on data quality and automation.
-Advanced proficiency in PySpark and Python for building scalable data processing and testing solutions.
-Strong experience with SQL and Hive for querying, validating, and profiling data in large datasets.
-Hands-on exposure to Hive and HDFS.
-Solid understanding of data pipeline architectures, ETL processes, and best practices for big data environments.
-Experience implementing CI and CD pipelines for automated data testing.
-Strong analytical, problem-solving, and communication skills.
-Ability to work independently and in a collaborative, fast-paced team environment.
Preferred Qualifications
-3 or more years of work experience with a Bachelor’s Degree or more than 2 years of work experience with an Advanced Degree (e.g. Masters, MBA, JD, MD)
-Bachelor or Master degree in Computer Science, Engineering, or related field.
-Above of 5 years of experience in test engineering or data engineering roles, with a focus on data quality and automation.
-Experience with data quality frameworks such as Deequ, Great Expectations.
-Familiarity with workflow orchestration tools such as Airflow, Step Functions.
-Exposure to data cataloging, data lineage, and metadata management.
Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.