Senior Data Platform Engineer

Job Description

We are seeking a Senior Data Platform Engineer to design, build, and operate a scalable, secure, and governed data platform supporting security, audit, operational, and analytics workloads.

This role is primarily focused on data engineering and data platform development, including data architecture, ingestion pipelines, data modeling, quality controls, governance, and integration with analytics and security platforms. The successful candidate will have strong experience designing and operating data platforms, building reliable batch and streaming pipelines, and managing large volumes of structured and semi-structured data.

The role requires close collaboration with security, infrastructure, and application teams to deliver trusted, high-quality data that enables monitoring, reporting, threat detection, compliance, and business insights.

Your key tasks 

Data Platform Architecture

• Design and evolve scalable data platform architectures for security, audit, operational, and analytics data
• Define data storage strategies, schemas, data models, partitioning, retention, and lifecycle management approaches
• Evaluate and prototype new technologies and architectures to improve scalability, performance, and cost efficiency

Data Engineering & Pipelines

• Design, build, and maintain batch and streaming data pipelines
• Develop robust ingestion frameworks for logs, audit data, application events, security telemetry, and operational datasets
• Implement data transformation, enrichment, normalization, correlation, and aggregation processes
• Ensure pipelines are reliable, scalable, observable, and resilient

Data Modeling & Storage

• Design relational, analytical, and event-based data models
• Optimize database structures, query performance, indexing, and storage efficiency
• Support the implementation of data lake, warehouse, and lakehouse concepts where appropriate

Data Quality & Governance

• Define and implement data quality controls across ingestion and transformation layers
• Develop validation, reconciliation, deduplication, and completeness checks
• Support data lineage, metadata management, ownership, retention, auditability, and regulatory requirements
• Implement controls for sensitive and regulated data

Platform Integration & Analytics Enablement

• Integrate data from diverse internal and external platforms, applications, databases, APIs, and messaging systems
• Deliver curated datasets that support reporting, analytics, observability, compliance, and security operations
• Support integration with SIEM, monitoring, and business intelligence platforms
• Collaborate with analytics and reporting teams to improve data accessibility and usability

Engineering & Automation

• Develop data engineering services, tooling, and automation using Python and SQL
• Contribute to CI/CD practices for data platform components
• Support infrastructure automation where required, using Terraform and related tooling
• Maintain engineering standards, documentation, and operational procedures

Qualifications

• 8+ years of experience in Data Engineering, Data Platform Engineering, Database Engineering, or a related field
• Strong SQL expertise, including schema design, data modeling, query optimization, indexing, and performance tuning
• Experience designing and operating production-grade batch and/or streaming data pipelines
• Experience with large-scale data platforms and analytical data architectures
• Strong proficiency in Python and SQL
• Experience integrating data from multiple sources, platforms, APIs, and event streams
• Strong understanding of data quality, schema evolution, lineage, governance, and lifecycle management
• Experience with relational databases and analytical storage technologies
• Familiarity with CI/CD concepts and Git-based development practices
• Strong analytical, problem-solving, and troubleshooting skills
• Experience working with sensitive, security-relevant, or regulated data

Preferred Qualifications

• Experience with Kafka or other streaming and messaging technologies
• Hands-on administration and search query development with Splunk (SPL) or alternative SIEM/observability stacks (Elasticsearch/Logstash/Kibana, Datadog)
• Experience with modern data platform technologies such as Apache Iceberg, Delta Lake, Apache Hudi, Trino, Spark or Parquet
• Experience with data lakehouse architectures
• Experience implementing data quality frameworks and data governance controls
• Familiarity with data cataloging, lineage, and metadata management solutions
• Experience integrating data platforms with analytics tools such as Apache Superset, Power BI, Tableau, or Metabase
• Experience in banking, fintech, cybersecurity, or other regulated industries
• Working knowledge of Terraform and cloud-based data platforms

It would be a real bonus if you have

• Security telemetry and audit-event processing
• SIEM and observability integrations
• Compliance and regulatory reporting datasets
• AI/ML-ready data platform architectures
• Real-time analytics and event-driven architectures