Description:
As a Senior Data Engineer, you'll develop and optimise data pipelines, across the Microsoft data capability stack but most notably in the new Fabric data platform. You'll work closely with data architects, developers and other key stakeholders to drive innovation and enhance data-driven decision-making processes.
Reporting to the Lead Data Engineer, you'll be part of a growing team delivering innovative data solutions to teams across the business.
Key responsibilities for this role will include:
- Leading the design and implementation of sophisticated data models and pipelines to support our analytics platform, working with our internal teams and 3rd party service providers.
- Collaborating effectively with data management teams to uphold data governance and security standards.
- Optimising data solutions using Azure and ensuring seamless production integration with robust testing and deployment strategies.
- Fostering a culture of continuous improvement and innovation across all data engineering activities.
- Engaging with business stakeholders to translate complex requirements into scalable data solutions.
What do I Need?
To be considered for this role, we would love you to have:
- Excellent technical computing, analysis, design and development skills to a proven professional level.
- Experience with designing, building, and operating analytics solutions using Azure cloud technologies.
- Demonstrable use of specific Azure data related resources including: Data Factory, Databricks, Storage Account (Data Lake / Delta Lake), SQL Database, SQL Data Warehouse (Dedicated SQL Pool).
- Experience of working at a senior level within a data engineering team taking on responsibility for: Designing and building new data models. Designing and building appropriate extract-transform-load (ETL) pipelines for required data. Engineering robust solution components. Undertaking testing/QA; producing standards documentation for the wider team, undertaking code reviews and other assurance related activities.
- Experience with Azure DevOps. Lead on all best practices and standards in terms of source repository control, CI/CD and change control.