Data engineer (Azure)
Location
singapore, singapore
Job Type
Full-time
Category
Other-General
Posted
June 05, 2026
Data engineer (Azure)
Synapse and Pyspark, Python, Datawarehouse and Azure Data Explorer, Azure Devops
Job Scope- Design, review and development of Pyspark scripts. Testing, troubleshooting of data pipelines, orchestration
- Implement and maintain datalake
- Establishing connections to source data systems such as on-prem databases, IOT devices, APIs
- Managing the collected data in appropriate storage/data-base solutions e.g. file systems, SQL servers, Big Data platforms such as Hadoop, HANA, etc. as required by the specific project requirements.
- Design, development of relevant data pipelines using pyspark, copy data activities for batch ingestion
- Performing data integration e.g. using database table joins, or other mechanisms at an appropriate level as required by the analysis requirements of the project
- Deployment of pipeline artifacts from one environment to the other using Azure Devops