Senior Data Engineer (Databricks & PySpark)

Optiveum is seeking a highly skilled Senior Data Engineer to join a specialized Sustainability Data & AI team. In this role, you will design, build, and maintain a robust Sustainability Data Foundation, leveraging modern cloud technologies to support critical reporting and advanced analytics for global sustainability initiatives.

Key responsibilities

• Design and implement scalable data products using Databricks, Delta Lake, and PySpark to process complex datasets from ERP and procurement systems.
• Optimize data workloads for performance, scalability, and cost efficiency while establishing reusable engineering patterns.
• Implement automated data quality controls and governance frameworks to ensure the reliability and auditability of sustainability data.
• Develop CI/CD pipelines, automated testing frameworks, and Infrastructure as Code to maintain high standards of platform engineering.
• Collaborate closely with sustainability experts and business analysts to translate complex requirements into effective production-ready solutions.

Requirements

• 5–7 years of professional experience in designing and operating large-scale data platforms and pipelines.
• Expert-level proficiency in PySpark, Python, Spark SQL, and advanced data modeling techniques.
• Extensive hands-on experience with Databricks, including Azure Databricks and Databricks Workflows in production environments.
• Strong background in CI/CD, Git-based development, and implementing robust data lineage and metadata management.
• Fluent English communication skills and a proactive, self-driven approach to troubleshooting and problem-solving.

What we offer

• Opportunity to work on high-impact sustainability projects within a modern, cloud-native data environment.
• Professional growth in a team that prioritizes operational excellence and practical, scalable engineering solutions.
• Full remote cooperation model with a focus on long-term partnership and technical development.