Container killed by the ApplicationMaster Exit code is 143
I've been getting the following error in several cases:
2017-03-23 11:55:10,794 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1490079327128_0048_r_000003_0: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
I noticed it happens one large sorts but when i change the "Sort Allocation Memory" it does not help.
I tried changing other memory properties but yet, the solution eludes me. Is there a good explanation to how Mapreduce works and what's the interaction between the different components? What should I change? where do I locate the Java error leading to this?
Answer
Exit code 143 is related to Memory/GC issues. Your default Mapper/reducer memory setting may not be sufficient to run the large data set. Thus, try setting up higher AM, MAP and REDUCER memory when a large yarn job is invoked.
Please check this link out: https://community.hortonworks.com/questions/96183/help-troubleshoot-container-killed-by-the-applicat.html
Please look into: https://www.slideshare.net/SparkSummit/top-5-mistakes-when-writing-spark-applications-63071421
Excellent source to optimize your code.