Job Description Insight Global is seeking a highly skilled Data Engineer with over 4+ years of experience to join our team to help a strategic banking client in various data transformation activities. Key Responsibilities: • Collaborating with cross-functional teams to understand data requirements, and design efficient, scalable, and reliable ETL processes using Python and Databricks • Developing and deploying ETL jobs that extract data from various sources, transforming them to meet business needs. • Taking ownership of the end-to-end engineering lifecycle, including data extraction, cleansing, transformation, and loading, ensuring accuracy and consistency. • Creating and managing data pipelines, ensuring proper error handling, monitoring and performance optimizations • Working in an agile environment, participating in sprint planning, daily stand-ups, and retrospectives. • Conducting code reviews, providing constructive feedback, and enforcing coding standards to maintain a high quality. • Developing and maintaining tooling and automation scripts to streamline repetitive tasks. • Implementing unit, integration, and other testing methodologies to ensure the reliability of the ETL processes • Utilizing REST APIs and other integration techniques to connect various data sources • Maintaining documentation, including data flow diagrams, technical specifications, and processes. • Designing and implementing tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. • Collaborate seamlessly across diverse technical stacks, including Databricks, Snowflake, etc. • Developing various components in Python as part of a unified data pipeline framework. • Contributing towards the establishment of best practices for the optimal and efficient usage of data across various on-prem and cloud platforms. • Assisting with the testing and deployment of our data pipeline framework utilizing standard testing frameworks and CI/CD tooling. • Monitoring the performance of queries and data loads and perform tuning as necessary. • Providing assistance and guidance during QA & UAT phases to quickly confirm the validity of potential issues and to determine the root cause and best resolution of verified issues. • Adhere to Agile practices throughout the solution development process. • Design, build, and deploy databases and data stores to support organizational requirements. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/. Skills and Requirements • Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience). • 4+ years of experience developing data pipelines and data warehousing solutions using Python and libraries such as Pandas, NumPy, PySpark, etc. • Experience with cloud services, especially Databricks, for building and managing scalable data pipelines • Proficiency in working with Snowflake or similar cloud-based data warehousing solutions • Solid understanding of ETL principles, data modelling, data warehousing concepts, and data integration best practices-Familiarity with agile methodologies and the ability to work collaboratively in a fast-paced, dynamic environment. • Familiarity with data visualization tools and libraries (e.g., Power BI) • Previous exposure to big data technologies (e.g., Hadoop, Spark) for large data processing • Experience in financial services • Knowledge of regulatory requirements in the financial industry