Data Engineer

CodiLime is a software and network engineering expert partnering with global tech leaders. We are looking for a Data Engineer to join our team building a large-scale, centralized data platform that serves as a critical source of information for a global consulting organization.

Key responsibilities

• Design, build, and maintain robust batch and streaming data pipelines using Python and Apache Airflow.
• Develop data transformation logic in SQL and dbt on Snowflake to create reliable, high-performance data models.
• Build and maintain Python services and APIs using FastAPI to expose data to downstream applications.
• Implement comprehensive unit, integration, and data-contract tests to ensure high code quality and system reliability.
• Optimize data processing jobs and Python code for runtime, memory efficiency, and cost-effectiveness.
• Refactor existing processes by replacing one-off scripts with tested, packaged, and scheduled production code.

Requirements

• Strong proficiency in Python, including data structures, typing, and software engineering fundamentals.
• Hands-on experience building and operating production-grade ETL/ELT pipelines.
• Solid experience with Snowflake, dbt, and Apache Airflow.
• Practical knowledge of Apache Spark, ideally within the Databricks environment.
• Experience with Docker, Kubernetes, CI/CD practices, and cloud platforms like Azure or AWS.
• Strong SQL skills and experience with data modeling and PostgreSQL.
• Proficiency in using AI coding assistants and a C1 level of English.

What we offer

• Fully remote work environment with flexible working hours.
• Professional growth supported by internal training sessions and a dedicated training budget.
• A collaborative atmosphere working alongside experienced professionals.
• Opportunities to work on diverse projects and change your focus area within the company.