SaltStack - How to use a dictionary in pillar and loop through those values in template file?

Question

I'm trying to make our configuration a little more modular. Currently, we have mostly hardcoded template files per environment that are matched to each environment with jinja in the init states. I'm splitting up the states, configs, and adding some default values that need to be maintained in the same config value across all enviornments.

Here's an example of my pillar:

/../pillars/amq/amq.sls
default_routes:
  Queue1:
    - from_uri: 'activemq:fromSomeURI1'
    - process_ref: 'processorName1'
    - to_uri: 'activemq:toSomeOutURI1'
  Queue2:
    - from_uri: 'activemq:fromSomeURI2'
    - process_ref: 'processorName2'
    - to_uri: 'activemq:toSomeOutURI2'

Here's an example of my template file:

/../salt/amq/conf/camel.xml.template
lines lines lines
lines lines lines
...
      {% for route, args in pillar.get('default_routes', {}).items() %}

        <route>
          <from uri="{{ route.from_uri }}"/>
          <process ref="{{ route.process_ref }}"/>
          <to uri="{{ route.to_uri }}"/>
        </route>

      {% endfor %}

...
lines lines lines
lines lines lines

What I need to be able to do is add a dictionary of values to the pillar, and loop through that list of default values, building out the default routes across all environments, from within camel.xml.template. The pillar then will also store environment specific values that I'll add to the file in a very similar manner.

Any help is much appreciated. I've tried a ton of different things and either get an error or the default lines are removed from the file.

Thanks!

Answer 1

There are a few inconsistencies with how you define your pillar.

Using this tool, to translate your YAML to Python gives

    "default_routes": {
        "Queue1": [
          {
            "from_uri": "activemq:fromSomeURI1"
          }, 
          {
            "process_ref": "processorName1"
          }, 
          {
            "to_uri": "activemq:toSomeOutURI1"
          }
        ], 
        "Queue2": [
          {
            "from_uri": "activemq:fromSomeURI2"
          }, 
          {
            "process_ref": "processorName2"
          }, 
          {
            "to_uri": "activemq:toSomeOutURI2"
          }
        ]
      }

when I presume you meant something like

"default_routes": {
    "Queue1": {
      "to_uri": "activemq:toSomeOutURI1", 
      "process_ref": "processorName1", 
      "from_uri": "activemq:fromSomeURI1"
    }, 
    "Queue2": {
      "to_uri": "activemq:toSomeOutURI2", 
      "process_ref": "processorName2", 
      "from_uri": "activemq:fromSomeURI2"
    }
  }

You should change your YAML to

default_routes:
  Queue1:
    from_uri: 'activemq:fromSomeURI1'
    process_ref: 'processorName1'
    to_uri: 'activemq:toSomeOutURI1'
  Queue2:
    from_uri: 'activemq:fromSomeURI2'
    process_ref: 'processorName2'
    to_uri: 'activemq:toSomeOutURI2'

But even then, there is a flaw in your template

{% for route, args in pillar.get('default_routes', {}).items() %}

This line will set route to key name and args to the dictionary. So for first time, route will be Queue1 and args will be the rest of dictionary.

You must change things {{ route.from_uri }} to {{ args.from_uri }} because args is the actual dictionary which has keys like from_uri