Data Engineering Manager

Description:

You’ll lead and mentor a team of Data Engineers, supporting their success through regular performance reviews, 1-to-1 meetings, and personal development plans. Managing the weekend rota, you’ll ensure adequate coverage for monitoring and resolving critical pipeline issues to enable reliable trade reporting. By fostering a culture of growth and continuous learning, you’ll ensure the team is equipped to deliver high-quality data solutions.

Overseeing the design, development, and maintenance of Extract, Transform, Load (ETL) data pipelines in Microsoft Azure Synapse Analytics, you’ll play a key role in our Data Transformation programme. A deep understanding of our on-premises data warehouse, built using SSIS and SSRS, will enable you to drive the decommissioning and migration strategy to Microsoft Azure.

You’ll develop resilient, scalable and future-proofed data solutions using best practices like the medallion architecture and Kimball methodology. By staying up to date with industry developments, you’ll drive optimisation, automation, and innovation of data engineering processes. Proactively maintaining on-premises and cloud pipelines, you’ll ensure robustness, prevent issues, ensure swift resolutions, and optimise performance and cost efficiency. Collaborating with Data & Analytics, the Data Architect, and stakeholders, you’ll translate business needs into effective data solutions. You’ll create and maintain documentation to support knowledge sharing, future maintenance, and seamless business access to evolving data solutions.

Ensuring high-quality outputs, you’ll implement thorough testing, automated and repeatable validation, and comprehensive documentation before release. Additionally, you’ll establish version control and deployment processes, using CI/CD best practices to enable reliable, automated, and efficient data pipeline releases.

The Skills You’ll Be Sharing With The Team
 

• Beneficial but not essential - recognised qualification or equivalent experience in; BA or BS degree in STEM (Science, Technology, Engineering, Mathematics), Finance, Accounting, Business, or any similar discipline with a computer science bias.
• Beneficial but not essential - obtained Azure certifications like Microsoft Certified: Azure Data Engineer Associate.
• Experience in designing and developing cloud hosted modern data platforms and data/integration architectures on Microsoft Azure, including Synapse Analytics, Data Lake, Azure Data Factory/Synapse Pipelines.
• Experience in designing and developing Data Warehouse Solutions - using SQL, SSAS & SSIS is desirable but not essential.
• Developing, tuning, and maintaining SQL Server database code and ETL.
• Experience with Python/PySpark, as well as working with API integration, would be advantageous.
• Strong understanding of data engineering processes and data architecture frameworks and best practices, such as the medallion architecture and Kimball methodology.
• Experience in leading a small team would be beneficial.
• Awareness of agile techniques and processes.
• Strong communicator with cross-functional teams and non-technical stakeholders.
• Experience in managing and prioritising stakeholder’s requests and translating them into robust data solutions.
• Knowledge of self-serve BI Tools such as Power BI would be a plus.
• Strong desire to explore new ideas, continuously learn and seek innovative ideas to improve processes.
• Strong analytical and problem-solving skills.
• Strong communicator, organised and proactive.
• Logical and creative.
• IT and mathematical skills