Training Trains Internship
Student login
← Big Data & Hadoop learning path
Big Data & Hadoop · Module 2 of 10

Hadoop ecosystem overview

Know the main components of the Hadoop ecosystem.

📖 Key ideas

  1. HDFS stores data across a cluster.
  2. MapReduce processes data in parallel.
  3. YARN manages cluster resources.
  4. Hive offers SQL; HBase is a NoSQL store; Sqoop moves data from databases; Flume ingests logs.
  5. Spark is now often used for processing on top of HDFS/YARN.
💡 Real-life example: Hadoop is a factory: HDFS is the warehouse, YARN the manager, MapReduce the workers.
HDFSHadoop Distributed File System
MapReduceParallel processing model
YARNResource manager
HiveSQL on Hadoop