Data Engineer Spark
IBM
**Introduction**
At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, let's talk.
**Your role and responsibilities**
Your responsibilities will include:
* Writing effective and scalable data processing code using Spark with Scala/Java/Python
* Designing and implementing robust solutions * Debugging applications to ensure low-latency and high-availability (Docker / Kubernetes)
* Interfacing with developed API and building Rest API
* Implementing and refactoring ETL processes (Airflow)
* Implementing security and data protection (HVault)
* Accommodating various data storage solutions (COS API, Cassandra)
**Required technical and professional expertise**
* Proficiency in data processing with Apache Spark (Scala/Java/Python)
* Work experience with Docker / Kubernetes
* Experience in implementing and refactoring ETL processes (Airflow / Python)
* Experience in implementing security and data protection
**Preferred technical and professional experience**
Nice to have: Kafka, GitLab, Jenkins
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Confirmar seu email: Enviar Email
Todos os Empregos de IBM