diff --git a/docs/README.rst b/docs/README.rst index d2f45a4..5f65482 100644 --- a/docs/README.rst +++ b/docs/README.rst @@ -31,6 +31,12 @@ which contains the currently released version. This formula is versioned accordi See `Formula Versioning Section `_ for more details. +If you need (non-default) configuration, please refer to: + +- `how to configure the formula with map.jinja `_ +- the ``pillar.example`` file + + Contributing to this repo ------------------------- diff --git a/docs/map.jinja.rst b/docs/map.jinja.rst new file mode 100644 index 0000000..7c34b10 --- /dev/null +++ b/docs/map.jinja.rst @@ -0,0 +1,492 @@ +.. _map.jinja: + +``map.jinja``: gather formula configuration values +================================================== + +The `documentation`_ explains the use of a ``map.jinja`` to gather parameters values for a formula. + +As `pillars`_ are rendered on the Salt master for every minion, this increases the load on the master as the pillar values and the number of minions grows. + +As a good practice, you should: + +- store non-secret data in YAML files distributed by the `fileserver`_ +- store secret data in: + + - `pillars`_ (and look for the use of something like `pillar.vault`_) + - `SDB`_ (and look for the use of something like `sdb.vault`_) + +Current best practice is to let ``map.jinja`` handle parameters from all sources, to minimise the use of pillars, grains or configuration from ``sls`` files and templates directly. + + +.. contents:: **Table of Contents** + + +For formula users +----------------- + + +Quick start: configure per role and per DNS domain name values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We will see a quick setup to configure the ``TEMPLATE`` formula for different DNS domain name and several roles. + +For this example, I'll define 2 kinds of `fileserver`_ sources: + +1. formulas git repositories with hard-coded version reference to avoid breaking my setup randomly at upstream update. they are the last sources where files are looked up +2. parameters of the formulas in the file backend `roots`_ + + +Configure the fileserver backends +````````````````````````````````` + +I configure the `fileserver`_ backends to serve: + +1. files from `roots`_ first +2. `gitfs`_ repositories last + +Create the file ``/etc/salt/master.d/fileserver.conf`` and restart the ``master``: + +.. code-block:: yaml + + --- + ## + ## file server + ## + fileserver_backend: + # parameters values and override + - roots + # formulas + - gitfs + + # The files in this directory will take precedence over git repositories + file_roots: + base: + - /srv/salt + + # List of formulas I'm using + gitfs_remotes: + - https://github.com/saltstack-formulas/template-formula.git: + - base: v4.1.1 + - https://github.com/saltstack-formulas/openssh-formula.git: + - base: v2.0.1 + ... + + +Create per DNS configuration for ``TEMPLATE`` formula +````````````````````````````````````````````````````` + +Now, we can provides the per DNS domain name configuration files for the ``TEMPLATE`` formulas under ``/srv/salt/TEMPLATE/parameters/``. + +We create the directory for ``dns:domain`` grain and we add a symlink for the ``domain`` grain which is extracted from the minion ``id``: + +.. code-block:: console + + mkdir -p /srv/salt/TEMPLATE/parameters/dns:domain/ + ln -s dns:domain /srv/salt/TEMPLATE/parameters/domain + +We create a configuration for the DNS domain ``example.net`` in ``/srv/salt/TEMPLATE/parameters/dns:domain/example.net.yaml``: + +.. code-block:: yaml + + --- + values: + config: /etc/template-formula-example-net.conf + ... + +We create another configuration for the DNS domain ``example.com`` in ``/srv/salt/TEMPLATE/parameters/dns:domain/example.com.yaml``: + +.. code-block:: yaml + + --- + values: + config: /etc/template-formula-{{ grains['os_family'] }}.conf + ... + + +Create per role configuration for ``TEMPLATE`` formula +`````````````````````````````````````````````````````` + +Now, we can provides the per role configuration files for the ``TEMPLATE`` formulas under ``/srv/salt/TEMPLATE/parameters/``. + +We create the directory for roles: + +.. code-block:: console + + mkdir -p /srv/salt/TEMPLATE/parameters/roles + +We will define 2 roles: + +- ``TEMPLATE/server`` +- ``TEMPLATE/client`` + +We create a configuration for the role ``TEMPLATE/server`` in ``/srv/salt/TEMPLATE/parameters/roles/TEMPLATE/server.yaml``: + +.. code-block:: yaml + + --- + values: + config: /etc/template-formula-server.conf + ... + +We create another configuration for the role ``TEMPLATE/client`` in ``/srv/salt/TEMPLATE/parameters/roles/TEMPLATE/client.yaml``: + +.. code-block:: yaml + + --- + values: + config: /etc/template-formula-client.conf + ... + + +Enable roles and the ``dns:domain`` and ``domain`` grains for ``map.jinja`` +``````````````````````````````````````````````````````````````````````````` + +We need to redefine the sources for ``map.jinja`` to load values from our new configuration files, we provide a global configuration for all our minions. + +We create the global parameters file ``/srv/salt/parameters/map_jinja.yaml``: + +.. code-block:: yaml + + --- + values: + sources: + # default values + - "Y:G@osarch" + - "Y:G@os_family" + - "Y:G@os" + - "Y:G@osfinger" + - "C@{{ tplroot ~ ':lookup' }}" + - "C@{{ tplroot }}" + + # Roles activate/deactivate things + # then thing are configured depending on environment + # So roles comes before `dns:domain`, `domain` and `id` + - "Y:C@roles" + + # DNS domain configured (DHCP or resolv.conf) + - "Y:G@dns:domain" + + # Based on minion ID + - "Y:G@domain" + + # default values + - "Y:G@id" + ... + +The syntax is explained later at `Sources of configuration values`_. + + +Bind roles to minions +````````````````````` + +We associate roles `grains`_ to minion using `grains.append`_. + +For the servers: + +.. code-block:: console + + salt 'server-*' grains.append roles TEMPLATE/server + +For the clients: + +.. code-block:: console + + salt 'client-*' grains.append roles TEMPLATE/client + +.. note:: + + Since we used ``Y:C@roles``, ``map.jinja`` will do a ``salt['config.get']('roles')`` to retrieve the roles so you could use any other method to bind roles to minions (`pillars`_ or `SDB`_) but `grains`_ seems to be the prefered method. + +Note for Microsoft Windows systems +`````````````````````````````````` + +If you have a minion running under windows, you can't use colon ``:`` as a delimiter for grain path query (see `bug 58726`_) in which case you should use an alternate delimiter: + +Modify ``/srv/salt/parameters/map_jinja.yaml`` to change the query for ``dns:domain`` to define the `alternate delimiter`_: + +.. code-block:: yaml + + --- + values: + sources: + # default values + - "Y:G@osarch" + - "Y:G@os_family" + - "Y:G@os" + - "Y:G@osfinger" + - "C@{{ tplroot ~ ':lookup' }}" + - "C@{{ tplroot }}" + + # Roles activate/deactivate things + # then thing are configured depending on environment + # So roles comes before `dns:domain`, `domain` and `id` + - "Y:C@roles" + + # DNS domain configured (DHCP or resolv.conf) + - "Y:G:!@dns!domain" + + # Based on minion ID + - "Y:G@domain" + + # default values + - "Y:G@id" + ... + +And then, rename the directory: + +.. code-block:: console + + mv /srv/salt/TEMPLATE/parameters/dns:domain/ '/srv/salt/TEMPLATE/parameters/dns!domain/' + + +Format of configuration YAML files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When you write a new YAML file, note that it must conform to the following layout: + +- a mandatory ``values`` key to store the configuration values +- two optional keys to configure the use of `salt.slsutil.merge`_ + + - an optional ``strategy`` key to configure the merging strategy, for example ``strategy: 'recurse'``, the default is ``smart`` + - an optional ``merge_lists`` key to configure if lists should be merged or overridden for the ``recurse`` and ``overwrite`` strategy, for example ``merge_lists: 'true'`` + +Here is a valid example: + +.. code-block:: yaml + + --- + strategy: 'recurse' + merge_lists: 'false' + values: + pkg: + name: 'some-package' + config: '/path/to/a/configuration/file' + ... + +You can use `Jinja`_ as with any SLS files: + +.. code-block:: yaml + + --- + strategy: 'overwrite' + merge_lists: 'true' + values: + output_dir: /tmp/{{ grains['id'] }} + ... + + +Sources of configuration values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + +Configuring ``map.jinja`` sources +````````````````````````````````` + +The ``map.jinja`` file uses several sources where to lookup parameter values. The list of sources can be modified by two files: + +1. a global ``salt://parameters/map_jinja.yaml`` +2. a per formula ``salt://{{ tplroot }}/parameters/map_jinja.yaml``, it overrides the global configuration + +Each source definition has the form ``[[: