Cloud Data Engineer AWS
IBM
**Introduction**
As a Cloud Data Engineer, you will play a critical role in designing and implementing data pipelines and systems that meet the needs of our business. You will work with cross-functional teams to understand business requirements and develop technical solutions.
**Your role and responsibilities**
We are seeking a Cloud Data Engineer to design, develop, and maintain scalable data processing systems and pipelines. Key responsibilities include:
* Implementing robust, secure, and efficient data pipelines to handle big data workloads
* Developing data pipelines for validating ITS data and computing derived data using metadata-driven expression language
* Collaborating with cross-functional teams to understand business requirements and translate them into technical solutions
* Monitoring system performance and implementing improvements to ensure optimal efficiency
* Staying current with emerging trends and best practices in cloud computing and data engineering
**Required technical and professional expertise**
* Strong knowledge of AWS services, including S3, Athena, and Glue
* Strong background in SQL in analytical environments
* Strong experience with Python, with proficiency in Java as a plus
* Experience with distributed big data technologies such as Hadoop and Apache Spark
* Experience with relational and dimensional database modeling
* Minimum 3 years of experience in data engineering or a related field
**Preferred technical and professional experience**
* Experience with single-node computing analytics libraries like Pandas/Polars
* Knowledge of cloud-based data warehousing and data lake architecture
* Experience with data pipeline orchestration tools such as Apache Airflow
* Familiarity with containerization using Docker and Kubernetes
* Advanced degree in Computer Science, Data Science, or a related field
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Confirmar seu email: Enviar Email
Todos os Empregos de IBM