Training Trains Internship
Student login
← Big Data & Hadoop learning path
Big Data & Hadoop · Module 5 of 10

YARN and cluster management

Manage resources and schedule jobs on a Hadoop cluster.

📖 Key ideas

  1. ResourceManager allocates cluster resources.
  2. NodeManagers run containers on each node.
  3. ApplicationMaster manages one job's tasks.
  4. Schedulers: FIFO, Capacity, Fair.
  5. YARN lets Spark, MapReduce and others share a cluster.
💡 Real-life example: YARN is a hotel manager assigning rooms (containers) to guests (jobs) fairly.
ResourceManagerGlobal resource allocator
NodeManagerPer-node agent
ContainerAllocated CPU/memory slot
Fair SchedulerShares resources evenly