Databricks Developer (Vancouver)
Location
vancouver, metro vancouver regional district
Job Type
Full-time
Category
Other-General
Posted
June 04, 2026
Responsibilities
- Design, build, and maintain scalable data pipelines and workflows using Databricks (SQL, PySpark, Delta Lake).
- Develop efficient ETL/ELT pipelines for structured and semi-structured data using Azure Data Factory (ADF) and Databricks notebooks/jobs.
- Integrate and transform large-scale datasets from multiple sources into unified, analytics-ready outputs.
- Optimize Spark jobs and manage Delta Lake performance using techniques such as partitioning, Z-ordering, broadcast joins, and caching.
- Design and implement data ingestion pipelines for RESTful APIs, transforming JSON responses into Spark tables.
- Apply best practices in data modeling and data warehousing concepts.
- Perform data validation and quality checks.
- Work with various data formats, including JSON, Parquet, and Avro.
- Build and manage data orchestration pipelines, including linked services and datasets for ADLS, Databricks, an...