1、Yarn Infrastructure

​ YARN Mainly by ResourceManager、NodeManager、ApplicationMaster and Container Etc . The functions of each component are as follows ：

2、Yarn Working mechanism

2.1 The whole process of homework submission is explained in detail

(1) Homework submission

1. Client call job.waitForCompletion Method , Submit... To the entire cluster MapReduce Homework .
2. Client towards RM Apply for an assignment id(Application).
3. RM to Client Return to the job Submit path and job of resource id.
4. Client Submit jar package 、 Slice information and configuration files to the specified resource submission path .
5. Client After submitting resources , towards RM Apply to run MrAppMaster.

(2) Job initialization

6. When RM received Client After request , Will be job Add to capacity scheduler .
7. Some free NM Take it Job.
8. The NM establish Container, And produce MRAppmaster.
9. download Client Commit resources to local .

(3) Task assignment

10. MrAppMaster towards RM Apply to run multiple MapTask Task resources .
11. RM Will run MapTask The task is assigned to the other two NodeManager, The other two NodeManager Pick up tasks and create containers .

(4) Task run

12. MR To receive the task from two NodeManager Send program startup script , these two items. NodeManager To start, respectively, MapTask,MapTask Sort the data partition .
13. MrAppMaster Wait for all MapTask After running , towards RM Apply for containers , function ReduceTask.
14. ReduceTask towards MapTask Get the data of the corresponding partition .
15. After the program runs ,MR Will send to RM Apply to cancel yourself .

2.2 During job submission HDFS and MapReduce

Detailed process explanation ：https://blog.csdn.net/qq_36593748/article/details/121978599?spm=1001.2014.3001.5501

3、HDFS、Yarn、MapReduce The relationship between the three