Senior Data Engineer

Avenga is seeking a highly skilled Senior Data Engineer to join the SML Warehouse Management team. You will collaborate with a global group of experienced professionals to design, build, and optimize modern data solutions that drive business value.

Key responsibilities

• Developing and maintaining robust data pipelines using Databricks and PySpark.
• Implementing complex data solutions utilizing both batch and streaming technologies.
• Managing data governance and security through the implementation of Unity Catalog.
• Optimizing existing workflows to ensure high performance and scalability of data systems.
• Collaborating with cross-functional teams to translate business requirements into technical data architectures.

Requirements

• Extensive professional experience working with Databricks, PySpark, and SQL.
• Proven expertise in managing batch processing and real-time streaming data architectures.
• Strong understanding of workflow orchestration and data cataloging tools like Unity Catalog.
• Experience with DevOps practices and familiarity with SQL Server environments.
• Ability to work effectively within a distributed global team structure.

What we offer

• Opportunity to work on complex, large-scale data engineering projects in a global environment.
• Access to modern technology stacks and continuous professional development.
• A collaborative culture that values technical excellence and knowledge sharing.
• Full flexibility to work remotely from any location.