Data Engineer

We are seeking a highly skilled and motivated Azure Data Engineer to support a major Dynamics 365 implementation. The successful candidate will bring deep expertise in Azure Data Factory and related services, along with proven experience in designing and delivering robust data integrations across enterprise systems. This role will focus on building scalable ETL/ELT pipelines, developing API integrations, and ensuring seamless data synchronisation with Microsoft Dynamics 365.

Key Responsibilities

• Design and implement scalable ETL/ELT pipelines using Azure Data Factory (ADF), including parameterised pipelines and mapping dataflows
• Integrate with Microsoft Dataverse and Dynamics 365 using ADF’s native connectors, OData endpoints, and REST APIs
• Work with cross-functional teams to gather integration requirements and deliver robust solutions
• Develop and optimise transformation logic, including column reduction, lookups, and upsert operations
• Implement delta load strategies and batch endpoints for managing high-volume data synchronisation
• Ensure data consistency and integrity across systems using unique identifiers and business keys
• Translate business requirements into technical solutions in collaboration with business analysts and architects
• Monitor, troubleshoot, and optimise pipeline performance using ADF’s diagnostics and logging tools
• Contribute to data governance, security, and compliance by applying best practices in access control and data handling
• Help develop coding standards and practices that enhance team delivery capability and quality
• Provide mentorship and coaching to junior engineers working with Azure technologies

Requirements

• ADF Expertise: Proficient in building pipelines, triggers, and mapping dataflows; skilled in parameterisation, error handling, and orchestration
• OData & API Integration: Strong understanding of the OData protocol and experience integrating with REST APIs
• Dataverse & Dynamics 365: Hands-on experience with Dataverse schema, connector configuration, and entity model mapping
• Data Modelling: Knowledge of conceptual, logical, and physical data models
• SQL & Scripting: Advanced SQL skills for data extraction and transformation tasks
• Testing & Automation: Experience with end-to-end testing and automation of data validation processes
• Documentation & Collaboration: Capable of documenting data flows and integration logic; excellent communication and teamwork skills
• Certifications: Azure Developer certification is highly desirable
• Problem-Solving & Teamwork: Strong analytical thinking and ability to collaborate effectively in a delivery-focused environment