Senior Databricks Engineer

Entrada AI is a specialized consulting partner and a strategic portfolio company of Databricks Ventures, operating at the forefront of the data ecosystem. We are looking for a Senior Databricks Engineer to join our team of industry veterans and Databricks MVPs to solve complex architectural challenges for Fortune 500 clients.

Key responsibilities

• Design and implement scalable data pipelines using Apache Spark and PySpark on the Databricks platform.
• Analyze and optimize complex Spark jobs for performance and cost efficiency, including partitioning and Photon engine utilization.
• Migrate legacy data warehouses to the modern Lakehouse architecture using Delta Lake.
• Enforce high code quality standards through rigorous code reviews and the implementation of CI/CD processes using Databricks Asset Bundles.
• Act as a technical advisor for clients and provide mentorship to team members.

Requirements

• Over 5 years of professional experience in Data Engineering with a strong focus on distributed systems.
• Minimum 3 years of hands-on experience with the Databricks platform and deep understanding of Delta Lake internals.
• Production experience with cloud environments such as Azure or AWS.
• Advanced proficiency in Python and SQL, along with experience in Unity Catalog implementation.
• Fluent English skills for direct collaboration with international clients.

What we offer

• Stable B2B employment contract with our Polish entity.
• Full flexibility to work 100% remotely.
• High-end Apple hardware provided for your daily work.
• Comprehensive support for professional growth, including Databricks certification cost coverage and paths to becoming a Solution Architect Champion.
• Opportunities to build your personal brand by presenting at industry conferences and collaborating with Databricks MVPs.