Description:
An innovative Professional Services company are seeking a highly skilled and experienced Senior Azure Data Engineer to join their team and play a pivotal role in their data modernisation journey.
Amongst other initiatives, you will be a part of a wider team of data engineers who can design, build, and maintain robust data pipelines and solutions on the Microsoft Azure cloud platform.
Responsibilities:
- Architect, design, and develop enterprise-grade data pipelines and data storage solutions
- Implement data integration processes for seamless data flow across systems
- Provision data storage solutions, ingesting streaming and batch data
- Transform and cleanse data to ensure its accuracy and consistency
- Implement security measures and data retention policies to safeguard sensitive information
- Collaborate with stakeholders to understand business requirements and translate them into technical specifications
- Lead quality assurance (QA) activities to ensure the integrity of data solutions
- Establish and maintain best practices for reusable code, libraries, patterns, and frameworks
- Ensure data quality, security, and adherence to data governance standards
- Monitor and troubleshoot data pipelines, identifying and resolving potential issues
- Mentor and train junior data engineers to foster knowledge sharing and expertise within the team
Skills:
- Proven experience as a Senior Data Engineer with 5+ years of industry experience
- In-depth knowledge of modern data architecture concepts and best practices
- Expert-level proficiency in Azure Synapse pipelines, Spark pools, PySpark, Azure Data Factory, Azure SQL Pools, ADLS Gen 2, Purview, and Azure App Service - (MUST HAVE)
- Familiarity with Microsoft Fabric is an asset
- Expertise in designing and building robust data pipelines using both batch and streaming ingestion methods over a Medallion architecture
- Strong understanding of Azure DevOps processes (including CI/CD)
- Proven experience in common data warehouse modelling principles, including Kimball and OBT
- Knowledge of Microsoft BI Stack
- Expertise in data management, data integration, and data processing methodologies
- Experience in developing and implementing security models to safeguard data integrity
- Proven ability to lead and manage small and large data engineering teams
- Experience with normalised and dimensional data modelling approaches
- Demonstrable ability to guide the development of data requirements and drive user-friendly experiences
- Expertise in working effectively in fast-paced agile environments