Data Engineering · Jodhpur, India
Senior Data Engineer
Vantara Metals runs six plants with OPC-UA historians, two ERP instances and a decade of spreadsheets. We are consolidating that into a lakehouse on Delta Lake, with about 420 dbt models feeding plant yield, energy cost per tonne and inventory ageing. The current nightly batch takes 3 hours 10 minutes and needs to become a 15-minute micro-batch without doubling the compute bill.
- Full-time — on site
- 5-8 years building production data pipelines, including at least one platform you migrated while it was in use
- Jodhpur, India
The work
What you will do
- Build ingestion from OPC-UA historians and ERP change-data-capture streams, handling clock skew and out-of-order events from sites with intermittent connectivity.
- Restructure the dbt project into clearly layered staging, intermediate and mart models, with tests on every column a report depends on.
- Cut the critical path from 3 hours 10 minutes to a 15-minute micro-batch through incremental models and partition pruning.
- Define and enforce data contracts between the ingestion layer and the analytics team, including a deprecation process for breaking changes.
- Own cost: track spend per pipeline and keep the monthly warehouse bill within an agreed ceiling.
You
What we look for
- Strong SQL — window functions, incremental merge strategies, and knowing why your query scanned 400 GB when it should have scanned 4.
- Spark or an equivalent distributed engine in production, including how you diagnosed a skewed join.
- Python for orchestration and transformation, with tests that run in CI.
- Dimensional modelling that survives contact with a client changing their reporting hierarchy mid-project.
- Direct experience of a late-arriving or corrected record breaking a downstream figure, and what you built so it did not happen twice.
More roles
Other openings
Bengaluru, India · Full-time — hybrid, three days a week on site
Senior Backend Engineer (Java)
You will work on the settlement and ledger services behind Meridian Exchange's B2B marketplace: roughly 40 million ledger rows a month, Java 21 on Spring Boot, Postgres 16, Kafka for event fan-out. The interesting part is not new endpoints, it is correctness under load — idempotent writes, exactly-once effects on top of at-least-once delivery, and holding p99 write latency under 250 ms as the ledger grows.
Jodhpur, India · Full-time — on site
Platform Engineer, Kubernetes
Atmora runs more than 40 client workloads across EKS and AKS, from a nine-node cluster carrying Kestrel Industrial's shop-floor telemetry to a two-region active/passive setup for Corvus Logistics. You will own the internal platform those delivery teams deploy onto: the Terraform module library, the Argo CD app-of-apps layout, and the paved path that takes a service from merge to production in under 12 minutes.
London, UK · Full-time — London office, with four to six client-site days a month across the UK and EU
Principal Solutions Architect
Most of our UK work is legacy modernisation for regulated businesses: Cordage Bank moving a 22-year-old core banking periphery off a shared Oracle schema, Aldridge Risk replacing an overnight batch that had grown to 9 hours and 40 minutes against a 6-hour window. You will run discovery, decide the migration shape, and stay accountable for it once delivery starts — not hand over a slide deck and leave.