Expert Data Engineer (Databricks)

Job Description 

• Design, build, and maintain Databricks data pipelines (ETL/ELT) for ingestion, transformation, and orchestration using Spark/Delta Lake/Databricks Workflows. 

• Operationalize machine learning models by building inference pipelines that invoke models authored by data scientists (batch or real-time), ensuring consistency between training and inference environments. 

• Ensure data reliability, quality, and observability through robust validation, monitoring, alerting, and automated recovery mechanisms. 

• Collaborate closely with data scientists to productionize models, manage model deployment lifecycles, and optimize inference performance and cost. 

• Implement best-practice DevOps/MLOps processes such as CI/CD for pipelines, model versioning, environment promotion, and infrastructure-as-code....