Data Engineer - Abu Dhabi - Black Pearl Consult

    Black Pearl Consult
    Black Pearl Consult Abu Dhabi

    2 days ago

    د.إ210,000 - د.إ360,000 (AED) per year *
    Description

    We are seeking an experienced Data Engineer (PySpark) to design, build, optimize, and maintain scalable data pipelines for production environments. The role requires strong hands-on experience in big data processing, pipeline optimization, and deployment using modern data engineering tools and frameworks.

    Key Responsibilities

    • Design, develop, and maintain robust, scalable data pipelines using Python and PySpark

    • Perform data ingestion, transformation, cleansing, and validation across structured and unstructured datasets

    • Conduct Exploratory Data Analysis (EDA) to identify data patterns, anomalies, and quality issues

    • Apply data imputation techniques, data linking, and cleansing to ensure high data quality

    • Implement feature engineering pipelines to support analytics and downstream use cases

    • Optimize Spark jobs for performance, scalability, and cost efficiency

    • Deploy and tune production-grade data pipelines, ensuring reliability and performance

    • Automate workflows using Apache Airflow and/or Jenkins

    • Collaborate with cross-functional teams to integrate data solutions into production systems

    • Write and maintain unit tests to ensure code quality and reliability

    • Manage source code, CI/CD, and deployments using Git, GitHub, and GitHub Actions



    Requirements

    To be considered for this role, you need to meet the following criteria:

    Required Technical Skills

    • Strong proficiency in Python

    • Extensive hands-on experience with Apache Spark (PySpark)

    • Experience working with Jupyter Notebooks

    • Strong knowledge of SQL and NoSQL databases

    • Proven experience with Git for version control and CI/CD

    • Hands-on experience with Apache Airflow and/or Jenkins for scheduling and automation

    • Solid understanding of data engineering best practices in production environments

    • Demonstrated experience in Spark performance tuning and optimization

    • Ability to write clean, testable, and maintainable Python code

    Mandatory Requirement

    • Previous production experience is a MUST, specifically in deploying, tuning, and maintaining data pipelines in production environments

    Preferred Qualifications

    • Experience working in high-volume or big data environments

    • Strong problem-solving and analytical skills

    • Ability to work independently in a fast-paced environment

    Why Join?

    • Competitive salary package

    • Opportunity to work on production-scale data platforms

    • Exposure to modern data engineering tools and practices

    • Dubai-based role with a dynamic and collaborative work environment


    To view other requirements we have, please visit our website -
    * This salary range is an estimation made by beBee
  • Work in company

    Data Engineer

    Only for registered members

    As Data Engineer you will gather, curate and prepare high-quality datasets to support cutting-edge research. · ...

    Abu Dhabi

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    We are looking for a Data Engineer who can build scalable data pipelines while also developing intelligent AI solutions using Azure Agent Foundry, OpenAI services, LangChain, Databricks, and modern cloud architectures. · ...

    Abu Dhabi

    1 month ago

  • Work in company

    Data Engineer

    Institute of Foundation Models

    You'll have the opportunity to work on the core of cutting-edge foundation model training alongside world-class researchers data scientists and engineers tackling the most fundamental and impactful challenges in AI development. · ...

    Abu Dhabi

    4 days ago

  • Work in company

    Data Engineer

    Only for registered members

    Job summary · We are looking for a Data Engineer to enhance our internal data engineering capability from the UAE and play a key role in developing the data infrastructure and systems alongside our UK Team. · ,ResponsibilitiesReceiving large datasets and transforming them to enab ...

    Abu Dhabi

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    In a world of possibilities, pursue one with endless opportunities. · ...

    Abu Dhabi

    1 week ago

  • Work in company

    Data Engineer

    Only for registered members

    We have partnered with a specialist UK-based buy-side consultancy supporting a major investment management firm in Abu Dhabi who are looking to hire an experienced Data Engineer on an initial 12-month contract. · ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    Job summary · As a Data Engineer at Inception you will be responsible for building and maintaining the infrastructure that supports data collection processing and storage. · BenefitsA competitive remuneration package with a host of perks including healthcare education support lea ...

    Abu Dhabi, Abu Dhabi Emirate

    5 days ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, implementing and optimising data pipelines and infrastructure to support cutting-edge AI systems. · Design develop and maintain scalable reliable data pipelines to ingest transform and load diverse datasets from various sources incl ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    Designs builds and optimizes scalable data infrastructure and pipelines to process manage both structured unstructured data powering advanced AI solutions. · ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    A Purview Developer with a Data Engineering background specializes in implementing enterprise data governance, cataloguing, lineage, and compliance using Microsoft Purview. · ...

    Abu Dhabi, Abu Dhabi Emirate

    2 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    Durlston Partners is partnering with a leading Quant Trading firm that leverages advanced data and technology to execute market-beating strategies. They are seeking a Data Engineer to design, build, and scale the data infrastructure powering research and trading systems seamless ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    Candidates with a Master of Science in Data Analytics or bachelor's degree in computer science/engineering are required for this position. · ...

    Abu Dhabi, Abu Dhabi Emirate

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    Data Engineer in Abu Dhabi, UAE required to design and maintain scalable data pipelines, curate complex datasets, and ensure high-quality reliable data. Experience in big data architectures and cloud platforms required. · ...

    Abu Dhabi, Abu Dhabi Emirate

    2 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    We are currently looking for Data Engineer profiles with Microsoft Purview experience. · Design, develop and maintain scalable data pipelines and ETL/ELT processes to ingest process transform large volumes of structured unstructured data. · ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    Azure Data Engineer

    Only for registered members

    Location: Abu Dhabi · Duration: Yearly renewable contract · Skills Required · Azure Data Factory · Azure Databricks · ADLS Gen2 · Unity Catalog · ETL/ELT pipeline development · PySpark / Spark SQL · Delta Lake (Bronze/Silver/Gold) · SQL optimization · ...

    Abu Dhabi, Abu Dhabi Emirate

    1 day ago

  • Work in company

    Data Engineer

    Only for registered members

    +We are seeking an experienced Data Engineer (PySpark) to design, build, optimize, and maintain scalable data pipelines for production environments.The role requires strong hands-on experience in big data processing, pipeline optimization,+and deployment using modern data enginee ...

    Abu Dhabi

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    We're looking for a hands-on data engineer to build and scale the data foundation that powers research and trading. · Design, implement, and own end-to-end ETL/ELT pipelines (batch and streaming) from ingestion to feature delivery. · ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    We are looking for a Data Engineer to enhance our internal data engineering capability from the UAE and play a key role in developing the data infrastructure and systems alongside our UK Team, · Receiving large datasets and transforming them to enable our mission teams to rapidly ...

    Abu Dhabi, Abu Dhabi Emirate

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    The Big Data Analytics Center (BIDAC) at UAE University is expanding its research and innovation team to advance the next generation of AI, Machine Learning, and Large Language Model (LLM) applications. · Design, build, and maintain data pipelines, ETL workflows, and databases su ...

    Abu Dhabi, Abu Dhabi Emirate

    1 month ago

  • Work in company

    DATA ENGINEER

    Star Services

    · Job Description: Data Warehouse and Machine Learning Engineer · We are seeking a highly skilled and experienced Data Warehouse and Machine Learning Engineer to join our team. In this role, you will be responsible for designing, developing, and maintaining data warehousing solu ...

    Abu Dhabi, United Arab Emirates د.إ210,000 - د.إ360,000 (AED) per year

    1 day ago

Jobs
>
Data engineer
>
Jobs for Data engineer in Abu Dhabi