Confidence S.

Data Engineer

Full-time available

Confidence is available for hire

Schedule an interview

14 days risk-free trial · No commitments · We handle contracts and payroll

Bio

Confidence S. is an accomplished data engineer with expertise in Python, SQL, and Spark, and a proven track record designing ELT and ETL pipelines across cloud platforms such as GCP (BigQuery, Cloud Storage, Dataflow), AWS (Redshift, S3, Glue, Lambda, EMR), and Azure (Synapse Analytics, Databricks, ADF, ADLS Gen2), as well as Microsoft Fabric. His technical proficiency extends to technologies like Apache Spark, Apache Kafka, RabbitMQ, Apache Airflow, DBT, Azure Data Factory, SSIS, Databricks Workflows, and Medallion…

  • Projects completed3
  • Hourly rate
  • 4+ years experience
  • Lagos
  • Member since Jun 2026
Expertise
PythonSQLKafkaPySparkGitMongoDBJavascriptAWSAzureGCPApache AirflowETL PipelinesData LakesBigQueryData Warehouse DesignDatabase:RelationalDatabase:Document
Video Intro
Assessments

General Project Assessment

Great on Find Indices of Two Numbers That Add Up to Target, Difficulty Easy · Great on Design a Scalable and Compliant SaaS Platform Architecture, Difficulty Hard · Great on Enhance Kubernetes Security on GCP, Difficulty Easy

Projects

Identity Fraud Detection Pipeline (Real-time)

Designed a real-time fraud detection pipeline ingesting identity events via Kafka, executing PySpark operations, and writing outputs to Delta Lake.

LLM Data Ingestion Platform (Multimodal)

Delivered a scalable data ingestion platform capable of processing various file types stored in AWS S3 into BigQuery using Airflow and Docker.

Migration of Enterprise GCP Data Warehouse

Lead the migration to Google BigQuery, engineered partitioning and clustering of tables, and constructed Airflow DAGs for automated loads.

Experience

  1. Data Engineer

    ConvzAug 2024 — Present
    • Created end-to-end ELT pipelines from diverse sources to BigQuery
    • Architected scalable ingestion pipelines for various document types
    • Automated workflows using DBT and managed data warehouse solutions
    • Built CI/CD workflows with GitHub actions
    • Managed star schema data warehouse on AWS Redshift and BigQuery
    DBTBigQueryPythonAWS RedshiftGitHub Actions

  2. Data Engineer

    YouverifyNov 2022 — Aug 2024
    • Architected and developed real-time pipelines for data ingestion
    • Achieved efficiency gains in automated data pipelines
    • Conducted company-wide assessment for data quality initiatives
    • Designed and implemented data models for data warehouse
    KafkaPythonPySparkPower BIAzure Data Factory

  3. Data Engineer

    Intelfort NigeriaJan 2022 — Oct 2022
    • Built ingestion and transformation workflows in AWS Glue and Redshift
    • Built automation framework for data pipeline orchestration
    • Built Medallion Architecture in Databricks Delta Lake
    • Collaborated with data scientists on feature engineering pipelines
    AWS GluePySparkAzure Data FactoryDatabricksAzure Synapse Analytics

  4. Data Engineer

    Signal Alliance Technology HoldingJan 2021 — Dec 2021
    • Built scalable ETL pipelines with Airflow DAGs
    • Led GCP enterprise-level data warehouse design and deployment
    • Engineered data transformations using DBT models
    • Created data warehousing models with star schema
    AirflowBigQueryDBTPower BIGCP
Education

  1. BTech Biotechnology

    Federal University of Technology OwerriOwerri, Okigwe, Nigeria2013 — 2018
Similar developers

We have moreWith similar skills.

Other vetted developers with similar skills and experience

Ibrahim B.
Ibrahim B.

Data Engineer

3-4 Years of Experience

Nigeria
PythonSQLPySparkKafka
View
View all