
Data Enginner - UAEN
- أبو ظبي
- دائم
- دوام كامل
- Design, develop, and maintain scalable data pipelines on Azure Databricks
- Build robust workflows for both batch and streaming data pipelines using Spark and Delta Lake
- Ingest and process data from diverse sources into Azure Data Lake Storage (ADLS) and Azure Synapse Analytics
- Collaborate with cross-functional teams to define data models, transformation logic, and business rules
- Apply data warehouse design principles, including data modeling and partitioning strategies
- Work with both structured and unstructured data formats
- Apply data governance practices such as cataloging, lineage tracking, and compliance
- Monitor, troubleshoot, and optimize pipeline performance, reliability, and data quality
- Automate workflows using orchestration tools such as Azure Data Factory or Apache Airflow
- Enforce data governance and security standards in alignment with organizational compliance requirements
- Troubleshoot and resolve data-related issues and pipeline failures
- Continuously drive process improvements for faster, more reliable data delivery
- 4+ years of experience as a Data Engineer, Data Platform Engineer, or in a similar role
- Bachelor's degree in computer science, Engineering, Information Systems, or a related field
- Hands-on experience with Azure Databricks, Spark, and Delta Lake to build and manage scalable ETL/ELT processes
- Strong Python, PySpark and SQL skills for data engineering and scripting tasks
- Experience with Azure Synapse, SQL Data Warehouse, or similar technologies
- Proficiency working with Azure Data Lake (Gen2) for storage and data processing
- Experience using Azure Data Factory for data orchestration
- Experience handling structured and semi-structured data formats (e.g., JSON, Parquet)
- Exposure and hands-on experience with Unit Testing and Data Testing suits is highly desirable and experience with tools such as
- Basic knowledge of Linux shell scripting and source control systems such as Git
- Exposure to other cloud platforms such as AWS or GCP is a plus