Data Engineer/ Tech Lead (Streaming Focus)

About the Role

We are seeking a Data Engineer / Tech Lead with a strong background in real-time data streaming to lead the design and delivery of cutting-edge streaming solutions. This role combines hands-on engineering with technical leadership – you will both architect scalable data streaming systems and guide a team to successful implementation.

Key Responsibilities

• Act as technical lead for projects with a strong streaming/data engineering focus.
• Design end-to-end architectures for data streaming use cases, ensuring scalability, reliability, and performance.
• Implement Change Data Capture (CDC) and other real-time extraction mechanisms.
• Drive adoption and best practices in Confluent Kafka and Spark Structured Streaming.
• Lead the development of streaming pipelines using Python, SQL, Bash, Node.js, and Linux/Unix Shell.
• Oversee code quality, versioning, release management, and CI/CD processes.
• Guide the team in applying DevOps and cloud-native tools: GitHub Actions, Terraform, Kubernetes, etc.
• Collaborate with business stakeholders, data scientists, and platform teams to align streaming solutions with organizational goals.
• Mentor junior/mid-level engineers, fostering growth and technical excellence within the team.

Essential Skills & Experience

• Proven experience in data engineering with a strong focus on streaming.
• Strong expertise in Confluent Kafka and Spark Structured Streaming.
• Hands-on experience with CDC and real-time data extraction.
• Solid programming background: Python, SQL, Bash, Node.js, Linux/Unix Shell.
• Strong knowledge of data engineering lifecycle: versioning, release management, deployment, agile delivery.
• Experience with cloud-native & DevOps tools: GitHub Actions, Terraform, Kubernetes.
• Previous experience in a technical leadership role (designing solutions, guiding teams, mentoring).

Nice to Have

• Multi-cloud exposure (AWS, Azure, GCP).
• Knowledge of wider big data & analytics platforms (Databricks, Snowflake, Delta Lake).
• Experience in stakeholder management and solution consulting.