PySpark Engineer

co.brick Talents is looking for a skilled PySpark Engineer to join an international data project. You will be responsible for building and migrating complex ETL/ELT processes within a modern Data Lake infrastructure, ensuring high performance and scalability of data pipelines.

Key responsibilities

• Develop and optimize data flows from various source systems to Data Lake structures using PySpark and Python.
• Migrate and build scalable ETL/ELT processes within the Palantir Foundry environment.
• Create and maintain clear technical documentation for developed Python code in Confluence.
• Estimate and resolve technical tasks assigned via Jira or Azure DevOps.
• Actively participate in Scrum ceremonies and contribute to 3-week sprint cycles.

Requirements

• Strong commercial experience with PySpark and Python in production environments.
• Proven track record of building and optimizing production-grade ETL/ELT pipelines.
• Fluent English (C1 level) for effective collaboration within an international team.
• Familiarity with Agile methodologies, specifically Scrum and task estimation.
• Hands-on experience with project management tools like Jira or Azure DevOps.

What we offer

• Fully remote work environment with a flexible schedule.
• Opportunity to work on an international data project with advanced technologies.
• Engagement in a professional team using modern data engineering practices.