Senior Data Engineer (Databricks)

Entrada AI is a specialized consulting partner and a strategic portfolio company of Databricks Ventures. We are looking for a Senior Data Engineer to join our team of industry veterans and Databricks MVPs, where you will solve complex architectural challenges for Fortune 500 clients using cutting-edge technology.

Key responsibilities

• Design and implement scalable data pipelines using Apache Spark and PySpark on the Databricks platform.
• Analyze and optimize complex Spark jobs for performance and cost efficiency, including partitioning and Photon engine utilization.
• Lead the modernization of legacy data warehouses by migrating them to the Lakehouse architecture using Delta Lake.
• Enforce high code quality standards through rigorous code reviews and the implementation of CI/CD processes using Databricks Asset Bundles.
• Act as a technical advisor for clients and provide mentorship to team members.

Requirements

• Over 5 years of professional experience in Data Engineering with a strong focus on distributed systems.
• Minimum 3 years of hands-on experience with the Databricks platform and deep knowledge of Delta Lake internals.
• Production experience with cloud-native environments, specifically Azure or AWS.
• Advanced proficiency in Python and SQL.
• Experience with Unity Catalog implementation and fluent English communication skills.

What we offer

• Stable long-term cooperation via B2B or UoP contracts with our Polish entity.
• Full flexibility to work 100% remotely from anywhere in Poland.
• High-end equipment, including a MacBook Air M4.
• Comprehensive professional growth support, including coverage for Databricks certification costs and opportunities to present at industry conferences.
• Direct collaboration with Databricks MVPs and access to private product previews.