Big Data Cloud Engineer (HDFS, YARN, Kafka, Linux, AWS, Shell scripting)
Location
singapore, singapore
Job Type
Full-time
Category
Software Architecture & Engineering
Posted
June 03, 2026
Key Responsibilities
- Administer, maintain, and support large-scale Hadoop and Cloudera (CDH/CDP) environments across distributed enterprise infrastructures.
- Design, deploy, and manage Hadoop cluster architecture, ensuring high availability and scalability in production environments.
- Manage and optimize Big Data ecosystem components such as HDFS, YARN, Spark, Kafka, Hive, Impala, HBase, Oozie, and Zookeeper.
- Perform cluster monitoring, troubleshooting, and performance tuning to ensure system stability and efficiency.
- Support both real-time and batch data processing platforms using Spark, Kafka Streams, Hive, and HBase.
- Configure and maintain ecosystem tools including Hue, Airflow, NiFi, Sqoop, Datameer, Splunk, and Arcadia.
- Implement and manage security frameworks such as Kerberos, Ranger, Sentry, SSL, and IAM policies.
- Manage AWS cloud services including EC2, S3, EMR, IAM, and VPC for Big Data w...