The Senior Big Data engineering lead will play a pivotal role in designing, implementing, and optimizing large-scale data processing and analytics solutions. This role requires a visionary leader who can drive innovation, define architecture strategy, and ensure the scalability and efficiency of our big data infrastructure. + Key Responsibilities + Lead the design and development of a robust and scalable big data architecture handling exponential data growth while maintaining high availability and resilience. + Design complex data transformation processes using Spark and other big data technologies using Java ,Pyspark or Scala + Design and implement data pipelines that ensure data quality, integrity, and availability. + Collaborate with cross-functional teams to understand business needs and translate them into technical requirements. + Evaluate and select technologies that improve data efficiency, scalability, and performance. + Oversee the deployment and management of big data tools and frameworks such as Hadoop, Spark, Kafka, and others. + Provide technical guidance and mentorship to the development team and junior architects. + Continuously assess and integrate emerging technologies and methodologies to enhance data processing capabilities. + Optimize big data frameworks, such as Hadoop, Spark, for performance improvements and reduced processing time across distributed systems. + Innovate in designing, developing, and refining data pipeline architectures to enhance data flow and ensure data processing capabilities. + Implement data governance frameworks to ensure data accuracy, consistency, and privacy across the organization, leveraging metadata management and data lineage tracking. + Conduct benchmarking and stress testing of big data solutions to validate performance standards and operational capacity. + Ensure compliance with data security best practices and regulations. Qualifications: + Bachelor's or Master’s degree in Computer Science, Information Technology, or related field. + Atleast 10 -12 years overall software development experience on majorly working with handling application with large scale data volumes from ingestion, persistence and retrieval. + Deep understanding of big data technologies, including Hadoop, Spark, Kafka, Flink, NoSQL databases, etc. + Experience with Bigdata technologies Developer Hadoop , Apache Spark ,Python, PySpark + Strong programming skills in languages such as Java, Scala, or Python. + Excellent problem-solving skills with a knack for innovative solutions. + Strong communication and leadership abilities. + Proven ability to manage multiple projects simultaneously and deliver results. Preferred Skills: + Experience with data modeling and ETL/ELT processes. + Experience in moving ETL frameworks from proprietary ETL technologies like Abinitio to Apache Spark + Familiarity with machine learning and data analytics tools. + Knowledge of core banking/financial services systems ------------------------------------------------------ **Job Family Group:** Technology ------------------------------------------------------ **Job Family:** Applications Development ------------------------------------------------------ **Time Type:** Full time ------------------------------------------------------ **Most Relevant Skills** Please see the requirements listed above. ------------------------------------------------------ **Other Relevant Skills** For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ _Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._ _If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi (https://www.citigroup.com/citi/accessibility/application-accessibility.htm)_ _._ _View Citi’s_ _EEO Policy Statement (https://www.citigroup.com/global/eeo-aa-policy)_ _and the_ _Know Your Rights (https://www.eeoc.gov/sites/default/files/2023-06/22-088\_EEOC\_KnowYourRights6.12ScreenRdr.pdf)_ _poster._ Citi is an equal opportunity and affirmative action employer. Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.