How to kill a mapred job started by hive?

hadoop mapreduce hive yarn cloudera-cdh
2shou picture 2shou · Feb 12, 2015 · Viewed 28k times · Source

I'm working by CDH 5.1 now. It starts normal Hadoop job by YARN but hive still works with mapred. Sometimes a big query will hang for a long time and I want to kill it.

I can find this big job by JobTracker web console while it didn't provide a button to kill it.

Another way is killing by command line. However, I couldn't find any job running by command line.

I have tried 2 commands:

  • yarn application -list
  • mapred job -list

How to kill big query like this?

Answer

Harman picture Harman · Feb 12, 2015

You can get the Job ID from Hive CLI when you run a job or from the Web UI. You can also list the job IDs using the application ID from resource manager. Ideally, you should get everything from 

mapred job -list

or

hadoop job -list

Using the Job ID you can kill it by using the below command.

hadoop job -kill <job_id>

Another alternative would be to kill the application using 

yarn application -kill <application_id>

Related questions

What is Hive: Return Code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

I am getting: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask While trying to make a copy of a partitioned table using the commands in the hive console: CREATE TABLE copy_table_name LIKE table_…

Read More
Hive ParseException - cannot recognize input near 'end' 'string'

I am getting the following error when trying to create a Hive table from an existing DynamoDB table: NoViableAltException(88@[]) at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:9123) at org.apache.hadoop.hive.ql.parse.HiveParser.…

Read More
Explode the Array of Struct in Hive

This is the below Hive Table CREATE EXTERNAL TABLE IF NOT EXISTS SampleTable ( USER_ID BIGINT, NEW_ITEM ARRAY<STRUCT<PRODUCT_ID: BIGINT,TIMESTAMPS:STRING>> ) And this is the data in the above table- 1015826235 [{"product_id":220003038067,"…

Read More