Data Engineer

Project descriptionWe are seeking a skilled, hands‐on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real‐time data streaming solutions.ResponsibilitiesImplement resource composition logic provided by the Product team into scalable and efficient Py Spark code.Develop, optimize, and maintain Py Spark applications within the Databricks environment.Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.Write and maintain unit tests to ensure high code quality and reliability.Perform data validation and quality checks to ensure compliance with defined business rules.Support and manage production deployments, ensuring system stability and performance.Troubleshoot and resolve issues across development, testing, and production llaborate with Product, QA, Dev Ops, and healthcare domain teams to ensure accurate implementation.Maintain technical documentation for code, workflows, and deployment processes.Pu...