Senior Data Engineer | Corporate Services | Group Tech &Dig Platforms
- دبي
- دائم
- دوام كامل
- Role requires to design, document, develop, implement, and test rapid prototypes to showcase the value of data within extended business audiences.
- Engage in early-stage design discussions and feature definition activities, predefining potential insights to shape future projects.
- Responsible for building and managing reusable and scalable data pipelines, optimizing data flows for efficiency and effectiveness.
- Convert business requirements into Technical Specification Documents (TSDs) and Functional Specification Documents (FSDs), bridging the gap between business needs and technical execution.
- Assist with regular reporting queries and support other developers in utilizing tools such as Alteryx, Power BI, and Azure for data analysis and visualization.
- Act as a team player, working collaboratively with team members across multiple engineering teams. Your role will be crucial in supporting the integration of proven prototypes into core intelligence products, facilitating seamless collaboration and knowledge sharing.
- Adapt to both agile and waterfall project delivery methodologies, guiding both internal and external teams to ensure timely and on-spec business deliveries. As a senior team member your leadership will navigate the team through dynamic project environments to meet and exceed business expectations
- Robust working knowledge and experience with both relational and non-relational databases, demonstrating a versatile skill set capable of navigating complex data environments.
- Proven work experience in constructing and refining Big Data pipelines, architectures, and datasets.
- Strong analytical abilities, mainly in handling and deriving insights from unstructured datasets.
- Design, document, develop, implement, and test rapid prototypes to showcase the value of data within extended business audiences.
- Engage in early-stage design discussions and feature definition activities, predefining potential insights to shape future projects.
- Responsible for building and managing reusable and scalable data pipelines, optimizing data flows for efficiency and effectiveness.
- Convert business requirements into Technical Specification Documents (TSDs) and Functional Specification Documents (FSDs), bridging the gap between business needs and technical execution.
- Assist with regular reporting queries and support other developers in utilizing tools such as Alteryx, Power BI, and Azure for data analysis and visualization.
- Act as a team player, working collaboratively with team members across multiple engineering teams. Your role will be crucial in supporting the integration of proven prototypes into core intelligence products, facilitating seamless collaboration and knowledge sharing.
- Adapt to both agile and waterfall project delivery methodologies, guiding both internal and external teams to ensure timely and on-spec business deliveries. As a senior team member your leadership will navigate the team through dynamic project environments to meet and exceed business expectations.
- Minimum 10+ years of experience in data engineering in Azure and Databricks.
- Bachelor's or Master's degree in data management, information systems, computer science, or a related field.
- Deep knowledge and practical experience with Azure Data Factory, Databricks, Azure Event Hub, Python, PySpark, Mongo DB, Graph DB and SQL.
- Experience with Azure DevOps is required for deploying ADF pipelines, ensuring continuous integration and delivery within our development lifecycle.
- Knowledge of Data Engineering components like Python, PySpark, Spark SQL, Databricks UI, Unity Catalog, SCIM Integration, External Location, Databricks cluster and job management.
- Certifications in Azure Data Fundamentals, Databricks are preferred.