YARN Mainly by ResourceManager、NodeManager、ApplicationMaster and Container Etc . The functions of each component are as follows :
2、Yarn Working mechanism
2.1 The whole process of homework submission is explained in detail
(1) Homework submission
Client call job.waitForCompletion Method , Submit... To the entire cluster MapReduce Homework .
Client towards RM Apply for an assignment id(Application).
RM to Client Return to the job Submit path and job of resource id.
Client Submit jar package 、 Slice information and configuration files to the specified resource submission path .
Client After submitting resources , towards RM Apply to run MrAppMaster.
(2) Job initialization
When RM received Client After request , Will be job Add to capacity scheduler .
Some free NM Take it Job.
The NM establish Container, And produce MRAppmaster.
download Client Commit resources to local .
(3) Task assignment
MrAppMaster towards RM Apply to run multiple MapTask Task resources .
RM Will run MapTask The task is assigned to the other two NodeManager, The other two NodeManager Pick up tasks and create containers .
(4) Task run
MR To receive the task from two NodeManager Send program startup script , these two items. NodeManager To start, respectively, MapTask,MapTask Sort the data partition .
MrAppMaster Wait for all MapTask After running , towards RM Apply for containers , function ReduceTask.
ReduceTask towards MapTask Get the data of the corresponding partition .
After the program runs ,MR Will send to RM Apply to cancel yourself .