Training Trains logo

Data Engineering & ETL Internship

Build the pipelines that move and clean data for analytics: Python, SQL, Airflow, dbt, Spark, warehouses and data quality checks.

PythonSQLApache AirflowdbtApache SparkKafka

What you'll work on

Full syllabus: all 60 graded tasks

These are the actual tasks the internship portal assigns for this domain. You submit each one with proof (a GitHub link or document) and a mentor reviews it.

Level 1: Foundations (24 tasks)
  1. What data engineers do. Explain the data engineer role, batch vs streaming, and draw a simple data pipeline from source to dashboard.
  2. Python for data. Read CSV, JSON and Excel files with Python and pandas, clean them and write them out as Parquet.
  3. SQL refresher. Write 12 SQL queries with joins, aggregates, window functions and CTEs on a sample dataset.
  4. File formats. Compare CSV, JSON, Parquet and Avro by size and read speed on the same data.
  5. Extract from an API. Pull data from a public REST API with pagination and retries, and save the raw responses.
  6. Transform and load. Clean the API data and load it into PostgreSQL with a repeatable script.
  7. Data modelling. Design a star schema for an online shop with fact and dimension tables.
  8. Idempotent loads. Make your load safe to re-run without creating duplicates (upserts or partition overwrite).
  9. Logging and errors. Add logging and clear failure messages to your pipeline, and show a failed run.
  10. Scheduling with cron. Schedule your pipeline to run daily with cron or Task Scheduler and keep a run log.
  11. Data quality checks. Add checks for nulls, duplicates and value ranges that stop the pipeline when data is bad.
  12. Docker for pipelines. Run your pipeline and its database together with docker-compose.
  13. Data dictionary. Write a data dictionary for your warehouse tables with meanings and owners.
  14. SQL style guide. Write a SQL style guide and refactor three queries to follow it.
  15. Date dimension. Generate a date dimension table with Indian holidays and fiscal year columns.
  16. Web scraping responsibly. Scrape a site that allows it (check robots.txt and terms), with rate limiting, and store the data.
  17. JSON flattening. Flatten nested JSON API data into relational tables.
  18. Deduplication. Remove duplicate customer records using fuzzy matching rules.
  19. Pipeline README. Document how to run, test and troubleshoot your pipeline.
  20. Cloud storage. Store raw files in cloud object storage organised by date partitions.
  21. CSV pitfalls. Show problems with encodings, delimiters and quotes in CSV files and fix them.
  22. Timezone handling. Convert timestamps from different sources into UTC and IST correctly.
  23. Small data model review. Review a classmate's data model and suggest improvements.
  24. Pipeline metrics. Log row counts, durations and failures for each run.
Level 2: Building real features (20 tasks)
  1. Apache Airflow. Rebuild your pipeline as an Airflow DAG with separate extract, transform and load tasks.
  2. dbt models. Build staging and mart models with dbt, with tests and documentation.
  3. Incremental loading. Change your pipeline to load only new or changed records using a watermark column.
  4. Slowly changing dimensions. Implement a Type 2 slowly changing dimension for customer addresses.
  5. Data warehouse in the cloud. Load your data into a free warehouse (BigQuery sandbox or similar) and query it.
  6. Partitioning and clustering. Partition a large table by date and show how it reduces data scanned.
  7. Apache Spark basics. Process a large dataset with PySpark: read, transform, aggregate and write Parquet.
  8. Great Expectations. Add a data validation suite with Great Expectations (or Soda) and publish its report.
  9. Change data capture concept. Explain CDC and demonstrate it with Debezium or by reading a database's change log.
  10. Pipeline documentation. Document your pipelines with lineage diagrams, owners, schedules and SLAs.
  11. Backfills. Run a historical backfill of your pipeline for 30 days safely.
  12. Data contracts. Write a data contract between a source team and your pipeline, with schema and SLAs.
  13. dbt snapshots. Track changes over time with dbt snapshots.
  14. dbt tests and docs. Add unique, not null and relationship tests, and publish dbt docs.
  15. Spark optimisation. Fix a slow Spark job using partitioning, broadcast joins and caching.
  16. Parquet layout. Compare partitioning strategies for Parquet files by query speed.
  17. Reverse ETL concept. Explain reverse ETL and push one warehouse table to a CRM or sheet.
  18. API ingestion at scale. Ingest a paginated API with concurrency limits, retries and checkpointing.
  19. Orchestrator sensors. Trigger a pipeline when a file arrives using a sensor or event trigger.
  20. Data quality dashboard. Show data quality check results over time in a dashboard.
Level 3: Professional work and capstone (16 tasks)
  1. Streaming with Kafka. Produce and consume events with Kafka (or Redpanda) and write them to storage in near real time.
  2. Stream processing. Compute a rolling 5-minute count from a stream with Spark Structured Streaming or Flink.
  3. Data lake layers. Build bronze, silver and gold layers for one dataset and explain what each layer guarantees.
  4. Lakehouse table format. Store data with Delta Lake or Apache Iceberg and show time travel and schema evolution.
  5. Orchestration with retries and alerts. Add retries, timeouts and failure alerts (email or chat) to your Airflow DAG.
  6. Cost and performance tuning. Tune a slow Spark or SQL job and report time and cost before and after.
  7. Data governance. Add access control, PII masking and a data catalogue entry for your main tables.
  8. Capstone: production pipeline. Deliver an end-to-end pipeline (ingest, validate, transform, warehouse, dashboard) with orchestration, tests, docs and monitoring.
  9. Real-time dashboard. Feed a streaming pipeline into a live dashboard.
  10. Exactly-once thinking. Explain at-least-once vs exactly-once processing and design for idempotent writes.
  11. Schema evolution. Handle a new column and a renamed column in a source without breaking the pipeline.
  12. Cost monitoring. Track warehouse query cost per user or pipeline and find the most expensive query.
  13. Data mesh concept. Explain data mesh and design domain-owned data products for a company.
  14. Privacy engineering. Implement data retention and deletion for personal data in your pipeline.
  15. Pipeline incident review. Write a postmortem for a failed or late pipeline run.
  16. Capstone 2: analytics platform. Build a small analytics platform for a business case with ingestion, lakehouse, dbt models, orchestration, quality checks and a BI dashboard.

Real software work, not just training

Every Data Engineering & ETL task is reviewed by our team and saved to your portfolio. Interns who pass the readiness check can take real work from Training Trains, W3AppDevelopers and DomainHostly projects on the Company Work board.

Data Science Solutions
See the software we build →

Frequently asked questions

Do I need experience to join the Data Engineering & ETL internship?+

Python and SQL basics help; both are refreshed in level 1.

What will I do in the Data Engineering & ETL internship?+

You complete 60 graded tasks in three levels: foundations, real features, and professional work. Each task is submitted with GitHub or document proof and reviewed. The final capstone is: Capstone 2: analytics platform.

Do I get a certificate for the Data Engineering & ETL internship?+

Yes. After your required tasks are approved, your attendance is above 75% and you pass the proctored final exam, you get a graded completion certificate that anyone can verify at trainingtrains.com/verify.

Can I do the Data Engineering & ETL internship online?+

Yes. You can do it fully online from home, or offline at our Erode and Perundurai branches. Tasks, quizzes and mentor reviews are the same in both modes.

Available both online (work from home) and offline at our Erode and Perundurai branches - see the full internship program details for departments, duration, and the application process.

Apply for the Data Engineering & ETL internship