Data Ops Engineer

Job Description

DataOps platform team mission is to ensure a reliable service level across each technological component of the EFG data platform (Databricks, Kafka, Spark, Elastic Search, Apache MQ, uServices stack). 

We are building and evolving the CI/CD Platform that supports data pipeline deployments, bringing implementation expertise and working closely with project teams to get deliverables across the line. 

Key team responsibilities includes, CI/CD governance of data pipelines, pipeline deployment support and certification and monitoring of platform level operations.

Main Responsibilities

 As our platform and data landscape evolve, we would like to strengthen our team with a skilled CI/CD engineer specialised in data pipeline implementations.

• Expectations on data quality are constantly increasing with the need for a extremely stable service delivery. We must ensure that our delivery processes are controlled and visible to our stakeholders. 
• We are particularly interested in someone comfortable driving Kafka, Spark and Databricks operations with Python 
• By reinforcing our team, you will greatly contribute to the continuity and quality of our operations.
• Take operational ownership of specific data services (change management, quality insurance management, SLA, SOP) 
• track and ensure the smooth operation of critical data pipelines during Asian hours, acting as the primary point of contact for critical issues in a follow-the sun support model.
• Design, implement, and optimize data pipeline in batch and streaming mode.
• Follow and apply CI/CD practice to automate and streamline deployments and integrations.
• Create, maintain, and improve documentation and operational procedures for which operationnal ownership is ensured.
• Collaborate with cross-functional teams to improve and review governance around integration operations.
• Maintain high standards of data quality, reliability, and performance.
• Stay current with advancements in data engineering, of dataops tech stack, to support continuous improvement

Requirements and Qualifications

• At least 5 Years of proven hands-on experience in data engineering and data pipeline deployment rooted in devops practice.
• Comfortable working with Python, SQL, PySpark and Databricks
• Robust experience of any leading CI/CD platforms and processes in hybrid environment (e.g., Azure DevOps, GitLab CI, GitHub Actions ).
• Familiarity with microservices architecture, API management and observability is a real plus 
• Experienced with Azure databricks.
• Good communication and collaboration skills.
• Rigorous and methodical in documentation and process creation
• High interest for convergence, normalization and reusability.
• Curious, enthusiastic and proactive
• Passionate about data processing technologies, with a strong desire to learn and grow in a demanding environment
• Fluent in spoken and written English, French is an asset.

Our Values

• Accountability: Taking ownership for tasks and challenges, as well as seeking continuous improvement
• Hands-on: Being proactive to rapidly deliver high-quality results
• Passionate: Being committed and striving for excellence
• Solution-driven: Focusing on client outcomes and treating clients fairly with a risk-aware mindset
• Partnership-oriented: Promoting collaboration and teamwork. Working together with an entrepreneurial spirit.

Please ensure to attach a cover letter to your CV when filling the application.

Application

Should you wish to apply for this position use this link to apply.