Senior Data Engineer
per year
per year
Our client is seeking a Senior Cloud Data Engineer to join their Engineering team. In this role, you will be a key technical contributor responsible for building, optimizing, and maintaining high-performance data processing engines. You will apply your deep expertise in Spark Best Practices to develop modular, portable PySpark applications, ensuring their platform remains provider-agnostic while operating within modern environments like Microsoft Fabric.
As an experienced member of the Data Engineering team, you will focus on refactoring legacy logic into scalable Python-centric solutions, optimizing their data lakehouse architecture, and contributing to the overall technical excellence of our client's data platform Requirements:
Bachelors degree in computer science, Information Systems, Engineering, or a closely related field.
Senior-Level Spark: 6 years of experience with Spark/PySpark, with the ability to diagnose performance bottlenecks via the Spark UI and optimize complex DAGs.
Advanced Python: Strong proficiency in production-grade Python, with experience building reusable libraries and implementing automated testing.
SQL Proficiency: Solid T-SQL skills to accurately interpret and migrate existing logic into the new Python-centric environment.
Azure Synapse Analytics & Dedicated SQL Pools: Solid experience working within Azure Synapse, specifically with Dedicate SQL Pools and Data Factory working with complex pipelines
Data Lakehouse Expertise: Hands-on experience with Delta Lake and Parquet, specifically in high-volume environments.
Agnostic Mindset: Experience working with open-source standards and containerization (Docker) to ensure data workloads can be ported across different cloud infrastructures.
Collaborative Engineering: Proven track record of working within a team to successfully deliver and modernize large-scale data workloads.
Ensure high levels of security, compliance, and data governance across data engineering processes. Should you meet the requirements for this position, please vacancy@stand-outstaffing.co.za . You can also contact the team on 031 350 4019 or alternatively you can visit our website www.stand-outstaffing.co.za Should you not hear from us within 3 days, please consider your application unsuccessful.
Submit your application for this position at Goldman Tech Resourcing
Submit your application for this position