Python / Spark / AI Developer

KwaZulu-Natal
R750K - R1M

per year

Job Description

Our client is seeking an experienced Python/Spark/AI Developer to help modernise data platforms by replatforming legacy T-SQL workloads onto Spark and Delta Lake pipelines. The successful candidate will work across data engineering, API development and AI-enabled capabilities, with a strong emphasis on type-safe, well-tested and specification-driven Python development. Skills Required

4 years of professional Python development experience.

2 years of production experience building Spark or comparable distributed data pipelines.

Strong Python 3.12 knowledge, including type hints, Pydantic, ABC-based provider patterns and modern packaging tools such as uv or Poetry.

Experience with Click or similar CLI frameworks.

Strong Apache Spark/PySpark experience, including DataFrame API, Spark SQL, partitioning, performance tuning and driver/executor architecture.

Understanding of Spark Connect.

Production experience with Delta Lake or equivalent lakehouse formats such as Iceberg or Hudi.

Strong SQL skills and the ability to faithfully translate legacy T-SQL and stored-procedure logic into Spark SQL.

Strong understanding of hashing, surrogate keys and set-based deduplication.

Hands-on pytest experience, including fixtures, markers and unit, integration and end-to-end testing.

Experience developing REST APIs using FastAPI or an equivalent framework.

Practical experience with Docker and Compose-based development environments.

Strong Git and CI practices, including PR workflows, linting, type checking and automated test gates . Should you meet the requirements for this position, please vacancy@stand-outstaffing.co.za . You can also contact us on XXX-XXXX or alternatively you can visit our website www.stand-outstaffing.co.za Should you not hear from us within 3 days, please consider your application unsuccessful.

Apply for Python / Spark / AI Developer

Submit your application for this position at Goldman Tech Resourcing

Add a few notes about your experience first for a more personal draft. You can edit the draft before submitting.

PDF, DOC, or DOCX (max 5MB)

Ready to Apply?

Submit your application for this position