Senior Databricks Engineer (Data Architecture & Design Focus)

Entrada is seeking a highly skilled Senior Databricks Engineer to join our team and focus on advanced data architecture and design. You will play a pivotal role in shaping our data platform, ensuring high-quality data delivery through robust engineering practices and sophisticated modeling techniques.

Key responsibilities

• Build the Conformed Silver Layer by implementing unified, schema-enforced data structures ready for complex business logic.
• Translate intricate business requirements into technical data warehouse structures, including Star Schemas and Medallion Gold tables.
• Develop and maintain reusable, metadata-driven ETL/ELT frameworks in PySpark to streamline the onboarding of new data sources.
• Optimize Spark job performance through advanced partitioning, bucketing, and caching strategies to ensure low-latency data delivery.
• Implement automated data quality reconciliation and validation gates to maintain high integrity standards across the platform.

Requirements

• Minimum of 5 years of professional experience in Big Data with a strong emphasis on data design and warehousing.
• Expert-level proficiency within the Databricks ecosystem, specifically Delta Lake, PySpark, and Spark SQL.
• Deep understanding of Medallion Architecture and the lifecycle of data from Bronze to Gold layers.
• Proven experience with data modeling patterns such as Star Schema, Snowflake, or Data Vault.
• Familiarity with Unity Catalog and large-scale data governance practices.

What we offer

• Opportunity to work on high-stakes data architecture projects in a fully remote environment.
• Collaborative culture focused on technical excellence and complex problem-solving.
• Access to cutting-edge tools and technologies within the Databricks ecosystem.
• Professional growth through exposure to advanced feature engineering and ML workflows.