InternshipTools and first-week setup
The tools a Data Engineering & ETL intern uses and how to set up your workspace in week one.
These are the main tools for Data Engineering & ETL tasks. You do not need all of them on day one; the Level 1 tasks tell you when to install each one.
- Python
- SQL
- Apache Airflow
- dbt
- Apache Spark
- Kafka
Set up your workspace
- Create a free GitHub account if you do not have one, with a professional username, and set up your LinkedIn profile: every task is submitted with a LinkedIn post.
- Create one repository for the internship, for example data-engineering-and-etl-internship, with a README that says who you are and what the repo holds.
- Keep one folder per task, named with the task number and title, so mentors find your work quickly.
- Commit small changes every day with clear messages. Your daily work report uses these commits as proof.
- Never commit passwords, API keys or personal data. Put secrets in a .env file and add it to .gitignore.
Your first tasks
| Task | What to do |
|---|---|
| What data engineers do | Explain the data engineer role, batch vs streaming, and draw a simple data pipeline from source to dashboard. |
| Python for data | Read CSV, JSON and Excel files with Python and pandas, clean them and write them out as Parquet. |
| SQL refresher | Write 12 SQL queries with joins, aggregates, window functions and CTEs on a sample dataset. |
| File formats | Compare CSV, JSON, Parquet and Avro by size and read speed on the same data. |