Senior Databricks Engineer

Entrada AI is a specialized consulting partner and a strategic portfolio company of Databricks Ventures. We are looking for a Senior Databricks Engineer to join our team of industry veterans and Databricks MVPs, where you will solve complex architectural challenges for Fortune 500 clients using cutting-edge features.

Key responsibilities

• Design and implement scalable data pipelines using Apache Spark and PySpark on the Databricks platform.
• Analyze and optimize complex Spark jobs for performance and cost efficiency, including partitioning and Photon engine utilization.
• Migrate legacy data warehouses to the modern Lakehouse architecture using Delta Lake.
• Enforce high code quality standards through rigorous code reviews and the implementation of CI/CD processes.
• Act as a technical advisor and mentor, sharing knowledge with the team and our clients.

Requirements

• Over 5 years of experience in Data Engineering with a strong focus on distributed systems.
• At least 3 years of hands-on experience with the Databricks platform and deep knowledge of Delta Lake internals.
• Production experience with cloud-native environments such as Azure or AWS.
• Advanced proficiency in Python and SQL, along with experience in Unity Catalog implementation.
• Fluent English skills (C1+) to facilitate direct collaboration with international clients.

What we offer

• Full flexibility to work 100% remotely from anywhere in Poland.
• High-end Apple hardware provided for your daily work.
• Coverage of Databricks certification costs and support for professional development.
• Direct collaboration with Databricks MVPs and access to product roadmaps.
• Opportunities to build your personal brand by presenting at industry conferences.