Oozie shell action memory limit

Question

We have an oozie workflow with a shell action that needs more memory than what a map task is given by Yarn by default.

How can we give it more memory?

We have tried adding the following configuration to the action:

<configuration>
  <property>
    <name>mapreduce.map.memory.mb</name>
    <value>6144</value> <!-- for example -->
  </property>
</configuration>

We have both set this as an inline (in the workflow.xml) configuration and as a jobXml. Neither has had any effect.

Answer 1

We found the answer:

A shell action is executed as an oozie "launcher" map task, and this task does not use the normal configuration properties.

Instead you have to prefix the properties with "oozie.launcher" to make them apply to the launcher task.

So in our case, if we use the following configuration for our shell action, it works.

<configuration>
  <property>
    <name>oozie.launcher.mapreduce.map.memory.mb</name>
    <value>6144</value> <!-- for example -->
  </property>
</configuration>

This is not obvious from the oozie documentation. We found this here: http://downright-amazed.blogspot.com/2012/02/configure-oozies-launcher-job.html