Senior Data Engineer

Job Description

Hire Resolve's client is seeking a highly skilled Senior Data Engineer to join their dynamic team within the telecommunications domain. The ideal candidate will be responsible for designing, developing, and maintaining scalable and reliable data pipelines to support the ingestion, processing, and analysis of Call Detail Records (CDRs) and real-time data streams. This role demands advanced technical expertise in stream processing frameworks such as Apache Kafka, Apache NiFi, Apache Flink, and Apache Druid, along with strong collaboration skills and a passion for data-driven solutions.

Responsibilities

Design and implement robust ETL/ELT pipelines for ingesting and transforming CDRs and telecom log data across multiple network types (2G/3G/4G/5G). Develop and maintain real-time data pipelines using Apache Kafka, NiFi, and Flink. Structure and integrate data for analytics platforms such as Apache Druid, Hudi, and Superset.

Manage large-scale distributed file systems and ensure efficient batch/stream data processing. Build and manage CI/CD pipelines for data workflow deployments. Optimize data performance, cost-efficiency, and reliability through continuous monitoring and enhancement.

Enforce data governance best practices including schema validation, access control, and regulatory compliance (e.g. GDPR). Collaborate with development, DevOps, and analytics teams to deliver query-ready datasets for machine learning models and BI dashboards.

Implement role-based access control, data encryption, and secure identity management. Design and maintain metadata repositories, data lineage, and asset discovery frameworks. Build quality assurance and anomaly detection tools to monitor data integrity across all pipeline stages.

Support analytical modelling using OLAP systems and columnar databases.

Requirements

Bachelor’s Degree in Computer Science, Software Engineering, Computer Engineering, or related field. Minimum of 5–8 years of experience in data engineering, preferably in telecom or large-scale distributed systems.

Strong expertise in Apache stack tools

Kafka, NiFi, Flink, Spark, Hive, Druid, Hudi. Proficiency in Python, SQL, Shell scripting, and Linux-based systems.

Experience in database management

PostgreSQL, MySQL, and analytical querying. Familiarity with data serialization formats (JSON, Avro, etc.) and secure data transfer protocols (SFTP). In-depth knowledge of data quality control, lineage tracking, and data discovery tools.

Experience in full-text search and indexing of structured/unstructured data using Apache or similar tools. Excellent problem-solving abilities, multitasking, and a fast-learning attitude. za.com or connect with Mischa Bornman via LinkedIn. Alternatively, you can also contact me directly at Hire Resolve mischa.bornman@hireresolve.us We will contact you telephonically in 3 days should you be suitable for this vacancy.

If you are not suitable, we will put your CV on file and contact you regarding any future vacancies that arise.

Apply for Senior Data Engineer

Submit your application for this position at Hire Resolve

Add a few notes about your experience first for a more personal draft. You can edit the draft before submitting.

PDF, DOC, or DOCX (max 5MB)

Ready to Apply?

Submit your application for this position