Ansible Pattern: Cumulative Inventory Variables

This is something I’ve never actually used, but has come up multiple times in #ansible, e.g. “I want to add on to a list variable in N places, but writing foo: "{{ foo1 + (foo2 | default([])) + (fooN | default([])) }}" is horrible and clunky and I have to know all of the variable names ahead of time.” Well, we can fix that.

Appended Lists

You want to define a base list (e.g. of packages to install), but allow membership in a group to add items on to that list? Here’s an example YAML inventory that does that (you can do this with any inventory using group_vars/, but it’s easier to demo in a single file):

all:
  vars:
    foo: "{{ q('vars', *q('varnames', '^foo_')) | flatten }}"
    foo_all:
      - base
  children:
    group1:
      vars:
        foo_group1:
          - bar
      hosts:
        hosta:
        hostb:
    group2:
      vars:
        foo_group2:
          - baz
      hosts:
        hostb:
        # You can also override variables from higher levels to remove items...
        hostc:
          foo_all: []
          foo_group2:
            - quux
        # ...or add on host-specific items.
        hostd:
          foo_hostd:
            - quux

Remember to name your variables well. If someone comes along and adds foo_hosts without knowing that I’ve reserved foo_ for this tomfoolery things could get real sad. Use a unique prefix that won’t produce collisions with other things in your codebase.

q() is an alias of query(), which is the same as lookup(wantlist=True). lookup() returns comma-separated strings by default due to historical design decisions, so query() was introduced as a shorter, easier-to-remember alternative to the parameter.

*q() is the trickiest part of this Jinja expression, because it’s a Pythonism. It takes the list returned by q() and unpacks it into positional arguments. If you leave off the * you’re passing in the list as a single argument, and the vars lookup doesn’t expect a list so it will fail.

Result:

TASK [debug] *******************************************************************
ok: [hosta] => 
    foo:
    - base
    - bar
ok: [hostb] => 
    foo:
    - base
    - bar
    - baz
ok: [hostc] => 
    foo:
    - quux
ok: [hostd] => 
    foo:
    - base
    - baz
    - quux

Merged Dictionaries

For a long time Ansible has had an engine feature that completely changes how hash/dictionary variables behave. This sort of interdependency between Ansible config and playbook design reduces portability and is not popular among the core devs, so it may go away in the future.

In general I feel that you shouldn’t design things in a way that requires you to override part of a dict, but here’s how you can do that without relying on hash_behaviour:

all:
  vars:
    foo: "{{ q('vars', *(q('varnames', '^foo_') | sort)) | combine(recursive=True) }}"
    foo_0_all:
      a: eh
      b: bee
      c:
        ca: california
        cb: radio
  children:
    group1:
      vars:
        foo_1_group1:
          b: bea
      hosts:
        hosta:
        hostb:
    group2:
      vars:
        foo_1_group2:
          c:
            cc: gaga
      hosts:
        hostb:
        # You can also override variables from higher levels to remove bits...
        hostc:
          foo_0_all: {}
          foo_1_group2:
            d: dee
        # ...or add on host-specific bits.
        hostd:
          foo_2_hostd:
            d: dee

The main caveat with using this approach for dicts is that the order in which they’re combined matters much more than with lists, which is why I’ve made the variable names lexically ordered by adding those ugly numbers in the middle.

Result:

TASK [debug] *******************************************************************
ok: [hosta] => 
    foo:
        a: eh
        b: bea
        c:
            ca: california
            cb: radio
ok: [hostb] => 
    foo:
        a: eh
        b: bea
        c:
            ca: california
            cb: radio
            cc: gaga
ok: [hostc] => 
    foo:
        d: dee
ok: [hostd] => 
    foo:
        a: eh
        b: bee
        c:
            ca: california
            cb: radio
            cc: gaga
        d: dee