Data Engineer (Azure Databricks)

Andersen is seeking a skilled Data Engineer to join a project focused on enhancing a cloud-based data platform for the life sciences sector. You will play a key role in developing scalable data solutions that support drug development, ensuring reliable analytics and effective data utilization across the organization.

Key responsibilities

• Build and maintain multi-layer data pipelines using Azure Databricks, Spark, and Delta Lake.
• Orchestrate complex pipelines using Databricks Jobs and Workflows, including managing dependencies and alerts.
• Implement batch and API-based data ingestion while ensuring schema enforcement and data quality.
• Utilize dbt for pipeline observability, data governance, and maintaining high standards of data health.
• Work within controlled environments to ensure audit trails, traceability, and compliance with project requirements.

Requirements

• At least 4 years of commercial experience in Data Engineering.
• Strong hands-on expertise with Azure Databricks, Spark, and Delta Lake.
• Proven experience in building multi-layer data architectures and managing schema drift.
• Proficiency in dbt and experience with pipeline orchestration and monitoring.
• Upper-Intermediate level of English or higher.

What we offer

• Opportunities for professional, financial, and career growth with dedicated mentoring.
• Access to a comprehensive corporate training portal and knowledge base.
• Support for professional development, including certification compensation.
• Participation in a referral program and access to private health insurance options.