"Fossies" - the Fresh Open Source Software Archive

Member "salt-3002.2/salt/renderers/gpg.py" (18 Nov 2020, 13724 Bytes) of package /linux/misc/salt-3002.2.tar.gz:


As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) Python source code syntax highlighting (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file. For more information about "gpg.py" see the Fossies "Dox" file reference documentation and the latest Fossies "Diffs" side-by-side code changes report: 3002.1_vs_3002.2.

    1 r"""
    2 Renderer that will decrypt GPG ciphers
    3 
    4 Any key in the SLS file can be a GPG cipher, and this renderer will decrypt it
    5 before passing it off to Salt. This allows you to safely store secrets in
    6 source control, in such a way that only your Salt master can decrypt them and
    7 distribute them only to the minions that need them.
    8 
    9 The typical use-case would be to use ciphers in your pillar data, and keep a
   10 secret key on your master. You can put the public key in source control so that
   11 developers can add new secrets quickly and easily.
   12 
   13 This renderer requires the gpg_ binary. No python libraries are required as of
   14 the 2015.8.0 release.
   15 
   16 .. _gpg-homedir:
   17 
   18 GPG Homedir
   19 -----------
   20 
   21 When running gpg commands, it is important to run commands as the user that owns
   22 the keys directory. If salt-master runs as user salt, then ``su salt`` before
   23 running any gpg commands.
   24 
   25 To avoid compatibility and upgrade problems and to provide a standardized location
   26 for keys, salt uses ``/etc/salt/gpgkeys``. In order to make the gpg command use
   27 this directory, use ``gpg --homedir /etc/salt/gpgkeys`` with gpg commands or set
   28 the homedir for that user using ``echo 'homedir /etc/salt/gpgkeys' >> ~/.gnupg``.
   29 
   30 .. _gpg: https://gnupg.org
   31 
   32 Setup
   33 -----
   34 
   35 To set things up, first generate a keypair. On the master, run the following:
   36 
   37 .. code-block:: bash
   38 
   39     # mkdir -p /etc/salt/gpgkeys
   40     # chmod 0700 /etc/salt/gpgkeys
   41     # gpg --gen-key --homedir /etc/salt/gpgkeys
   42 
   43 Do not supply a password for the keypair, and use a name that makes sense for
   44 your application. Be sure to back up the ``gpgkeys`` directory someplace safe!
   45 
   46 .. note::
   47     Unfortunately, there are some scenarios - for example, on virtual machines
   48     which don’t have real hardware - where insufficient entropy causes key
   49     generation to be extremely slow. In these cases, there are usually means of
   50     increasing the system entropy. On virtualised Linux systems, this can often
   51     be achieved by installing the ``rng-tools`` package.
   52 
   53 Import keys to a master
   54 ***********************
   55 
   56 If the keys already exist and need to be imported to the salt master, run the
   57 following to import them.
   58 
   59 .. code-block:: bash
   60 
   61     gpg --homedir /etc/salt/gpgkeys --import /path/to/private.key
   62     gpg --homedir /etc/salt/gpgkeys --import /path/to/pubkey.gpg
   63 
   64 Note: The default `GPG Homedir <gpg-homedir>` is ``~/.gnupg`` and needs to be
   65 set using ``--homedir``.
   66 
   67 Adjust trust level of imported keys
   68 ***********************************
   69 
   70 In some cases, importing existing keys may not be enough and the trust level of
   71 the key needs to be adjusted. This can be done by editing the key. The ``key_id``
   72 and the actual trust level of the key can be seen by listing the already imported
   73 keys.
   74 
   75 .. code-block:: bash
   76 
   77     gpg --homedir /etc/salt/gpgkeys --list-keys
   78     gpg --homedir /etc/salt/gpgkeys --list-secret-keys
   79 
   80 If the trust-level is not ``ultimate`` it needs to be changed by running
   81 
   82 .. code-block:: bash
   83 
   84     gpg --homedir /etc/salt/gpgkeys --edit-key <key_id>
   85 
   86 This will open an interactive shell for the management of the GPG encryption key.
   87 Type ``trust`` to be able to set the trust level for the key and then select ``5
   88 (I trust ultimately)``. Then quit the shell by typing ``save``.
   89 
   90 Different GPG Location
   91 **********************
   92 
   93 In some cases, it's preferable to have gpg keys stored on removable media or
   94 other non-standard locations. This can be done using the ``gpg_keydir`` option
   95 on the salt master. This will also require using a different path to ``--homedir``,
   96 as mentioned in the `GPG Homedir <gpg-homedir>` section.
   97 
   98 .. code-block:: bash
   99 
  100     gpg_keydir: <path/to/homedir>
  101 
  102 Export the Public Key
  103 ---------------------
  104 
  105 .. code-block:: bash
  106 
  107     # gpg --homedir /etc/salt/gpgkeys --armor --export <KEY-NAME> > exported_pubkey.gpg
  108 
  109 
  110 Import the Public Key
  111 ---------------------
  112 
  113 To encrypt secrets, copy the public key to your local machine and run:
  114 
  115 .. code-block:: bash
  116 
  117     $ gpg --import exported_pubkey.gpg
  118 
  119 To generate a cipher from a secret:
  120 
  121 .. code-block:: bash
  122 
  123    $ echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -r <KEY-name>
  124 
  125 To apply the renderer on a file-by-file basis add the following line to the
  126 top of any pillar with gpg data in it:
  127 
  128 .. code-block:: yaml
  129 
  130     #!yaml|gpg
  131 
  132 Now with your renderer configured, you can include your ciphers in your pillar
  133 data like so:
  134 
  135 .. code-block:: yaml
  136 
  137     #!yaml|gpg
  138 
  139     a-secret: |
  140       -----BEGIN PGP MESSAGE-----
  141       Version: GnuPG v1
  142 
  143       hQEMAweRHKaPCfNeAQf9GLTN16hCfXAbPwU6BbBK0unOc7i9/etGuVc5CyU9Q6um
  144       QuetdvQVLFO/HkrC4lgeNQdM6D9E8PKonMlgJPyUvC8ggxhj0/IPFEKmrsnv2k6+
  145       cnEfmVexS7o/U1VOVjoyUeliMCJlAz/30RXaME49Cpi6No2+vKD8a4q4nZN1UZcG
  146       RhkhC0S22zNxOXQ38TBkmtJcqxnqT6YWKTUsjVubW3bVC+u2HGqJHu79wmwuN8tz
  147       m4wBkfCAd8Eyo2jEnWQcM4TcXiF01XPL4z4g1/9AAxh+Q4d8RIRP4fbw7ct4nCJv
  148       Gr9v2DTF7HNigIMl4ivMIn9fp+EZurJNiQskLgNbktJGAeEKYkqX5iCuB1b693hJ
  149       FKlwHiJt5yA8X2dDtfk8/Ph1Jx2TwGS+lGjlZaNqp3R1xuAZzXzZMLyZDe5+i3RJ
  150       skqmFTbOiA===Eqsm
  151       -----END PGP MESSAGE-----
  152 
  153 
  154 .. _encrypted-cli-pillar-data:
  155 
  156 Encrypted CLI Pillar Data
  157 -------------------------
  158 
  159 .. versionadded:: 2016.3.0
  160 
  161 Functions like :py:func:`state.highstate <salt.modules.state.highstate>` and
  162 :py:func:`state.sls <salt.modules.state.sls>` allow for pillar data to be
  163 passed on the CLI.
  164 
  165 .. code-block:: bash
  166 
  167     salt myminion state.highstate pillar="{'mypillar': 'foo'}"
  168 
  169 Starting with the 2016.3.0 release of Salt, it is now possible for this pillar
  170 data to be GPG-encrypted, and to use the GPG renderer to decrypt it.
  171 
  172 
  173 Replacing Newlines
  174 ******************
  175 
  176 To pass encrypted pillar data on the CLI, the ciphertext must have its newlines
  177 replaced with a literal backslash-n (``\n``), as newlines are not supported
  178 within Salt CLI arguments. There are a number of ways to do this:
  179 
  180 With awk or Perl:
  181 
  182 .. code-block:: bash
  183 
  184     # awk
  185     ciphertext=`echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | awk '{printf "%s\\n",$0} END {print ""}'`
  186     # Perl
  187     ciphertext=`echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | perl -pe 's/\n/\\n/g'`
  188 
  189 With Python:
  190 
  191 .. code-block:: python
  192 
  193     import subprocess
  194 
  195     secret, stderr = subprocess.Popen(
  196         ['gpg', '--armor', '--batch', '--trust-model', 'always', '--encrypt',
  197          '-r', 'user@domain.com'],
  198         stdin=subprocess.PIPE,
  199         stdout=subprocess.PIPE,
  200         stderr=subprocess.PIPE).communicate(input='supersecret')
  201 
  202     if secret:
  203         print(secret.replace('\n', r'\n'))
  204     else:
  205         raise ValueError('No ciphertext found: {0}'.format(stderr))
  206 
  207 .. code-block:: bash
  208 
  209     ciphertext=`python /path/to/script.py`
  210 
  211 
  212 The ciphertext can be included in the CLI pillar data like so:
  213 
  214 .. code-block:: bash
  215 
  216     salt myminion state.sls secretstuff pillar_enc=gpg pillar="{secret_pillar: '$ciphertext'}"
  217 
  218 The ``pillar_enc=gpg`` argument tells Salt that there is GPG-encrypted pillar
  219 data, so that the CLI pillar data is passed through the GPG renderer, which
  220 will iterate recursively though the CLI pillar dictionary to decrypt any
  221 encrypted values.
  222 
  223 
  224 Encrypting the Entire CLI Pillar Dictionary
  225 *******************************************
  226 
  227 If several values need to be encrypted, it may be more convenient to encrypt
  228 the entire CLI pillar dictionary. Again, this can be done in several ways:
  229 
  230 With awk or Perl:
  231 
  232 .. code-block:: bash
  233 
  234     # awk
  235     ciphertext=`echo -n "{'secret_a': 'CorrectHorseBatteryStaple', 'secret_b': 'GPG is fun!'}" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | awk '{printf "%s\\n",$0} END {print ""}'`
  236     # Perl
  237     ciphertext=`echo -n "{'secret_a': 'CorrectHorseBatteryStaple', 'secret_b': 'GPG is fun!'}" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | perl -pe 's/\n/\\n/g'`
  238 
  239 With Python:
  240 
  241 .. code-block:: python
  242 
  243     import subprocess
  244 
  245     pillar_data = {'secret_a': 'CorrectHorseBatteryStaple',
  246                    'secret_b': 'GPG is fun!'}
  247 
  248     secret, stderr = subprocess.Popen(
  249         ['gpg', '--armor', '--batch', '--trust-model', 'always', '--encrypt',
  250          '-r', 'user@domain.com'],
  251         stdin=subprocess.PIPE,
  252         stdout=subprocess.PIPE,
  253         stderr=subprocess.PIPE).communicate(input=repr(pillar_data))
  254 
  255     if secret:
  256         print(secret.replace('\n', r'\n'))
  257     else:
  258         raise ValueError('No ciphertext found: {0}'.format(stderr))
  259 
  260 .. code-block:: bash
  261 
  262     ciphertext=`python /path/to/script.py`
  263 
  264 With the entire pillar dictionary now encrypted, it can be included in the CLI
  265 pillar data like so:
  266 
  267 .. code-block:: bash
  268 
  269     salt myminion state.sls secretstuff pillar_enc=gpg pillar="$ciphertext"
  270 """
  271 
  272 
  273 import logging
  274 import os
  275 import re
  276 from subprocess import PIPE, Popen
  277 
  278 import salt.syspaths
  279 import salt.utils.cache
  280 import salt.utils.path
  281 import salt.utils.stringio
  282 import salt.utils.stringutils
  283 from salt.exceptions import SaltRenderError
  284 
  285 log = logging.getLogger(__name__)
  286 
  287 GPG_CIPHERTEXT = re.compile(
  288     salt.utils.stringutils.to_bytes(
  289         r"-----BEGIN PGP MESSAGE-----.*?-----END PGP MESSAGE-----"
  290     ),
  291     re.DOTALL,
  292 )
  293 GPG_CACHE = None
  294 
  295 
  296 def _get_gpg_exec():
  297     """
  298     return the GPG executable or raise an error
  299     """
  300     gpg_exec = salt.utils.path.which("gpg")
  301     if gpg_exec:
  302         return gpg_exec
  303     else:
  304         raise SaltRenderError("GPG unavailable")
  305 
  306 
  307 def _get_key_dir():
  308     """
  309     return the location of the GPG key directory
  310     """
  311     gpg_keydir = None
  312     if "config.get" in __salt__:
  313         gpg_keydir = __salt__["config.get"]("gpg_keydir")
  314 
  315     if not gpg_keydir:
  316         gpg_keydir = __opts__.get(
  317             "gpg_keydir",
  318             os.path.join(
  319                 __opts__.get("config_dir", os.path.dirname(__opts__["conf_file"])),
  320                 "gpgkeys",
  321             ),
  322         )
  323 
  324     return gpg_keydir
  325 
  326 
  327 def _get_cache():
  328     global GPG_CACHE
  329     if not GPG_CACHE:
  330         cachedir = __opts__.get("cachedir")
  331         GPG_CACHE = salt.utils.cache.CacheFactory.factory(
  332             __opts__.get("gpg_cache_backend"),
  333             __opts__.get("gpg_cache_ttl"),
  334             minion_cache_path=os.path.join(cachedir, "gpg_cache"),
  335         )
  336     return GPG_CACHE
  337 
  338 
  339 def _decrypt_ciphertext(cipher):
  340     """
  341     Given a block of ciphertext as a string, and a gpg object, try to decrypt
  342     the cipher and return the decrypted string. If the cipher cannot be
  343     decrypted, log the error, and return the ciphertext back out.
  344     """
  345     try:
  346         cipher = salt.utils.stringutils.to_unicode(cipher).replace(r"\n", "\n")
  347     except UnicodeDecodeError:
  348         # ciphertext is binary
  349         pass
  350     cipher = salt.utils.stringutils.to_bytes(cipher)
  351     if __opts__.get("gpg_cache"):
  352         cache = _get_cache()
  353         if cipher in cache:
  354             return cache[cipher]
  355     cmd = [
  356         _get_gpg_exec(),
  357         "--homedir",
  358         _get_key_dir(),
  359         "--status-fd",
  360         "2",
  361         "--no-tty",
  362         "-d",
  363     ]
  364     proc = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=False)
  365     decrypted_data, decrypt_error = proc.communicate(input=cipher)
  366     if not decrypted_data:
  367         log.warning("Could not decrypt cipher %r, received: %r", cipher, decrypt_error)
  368         return cipher
  369     else:
  370         if __opts__.get("gpg_cache"):
  371             cache[cipher] = decrypted_data
  372         return decrypted_data
  373 
  374 
  375 def _decrypt_ciphertexts(cipher, translate_newlines=False, encoding=None):
  376     to_bytes = salt.utils.stringutils.to_bytes
  377     cipher = to_bytes(cipher)
  378     if translate_newlines:
  379         cipher = cipher.replace(to_bytes(r"\n"), to_bytes("\n"))
  380 
  381     def replace(match):
  382         result = to_bytes(_decrypt_ciphertext(match.group()))
  383         return result
  384 
  385     ret, num = GPG_CIPHERTEXT.subn(replace, to_bytes(cipher))
  386     if num > 0:
  387         # Remove trailing newlines. Without if crypted value initially specified as a YAML multiline
  388         # it will conain unexpected trailing newline.
  389         ret = ret.rstrip(b"\n")
  390     else:
  391         ret = cipher
  392 
  393     try:
  394         ret = salt.utils.stringutils.to_unicode(ret, encoding=encoding)
  395     except UnicodeDecodeError:
  396         # decrypted data contains some sort of binary data - not our problem
  397         pass
  398     return ret
  399 
  400 
  401 def _decrypt_object(obj, translate_newlines=False, encoding=None):
  402     """
  403     Recursively try to decrypt any object. If the object is a string
  404     or bytes and it contains a valid GPG header, decrypt it,
  405     otherwise keep going until a string is found.
  406     """
  407     if salt.utils.stringio.is_readable(obj):
  408         return _decrypt_object(obj.getvalue(), translate_newlines)
  409     if isinstance(obj, (str, bytes)):
  410         return _decrypt_ciphertexts(
  411             obj, translate_newlines=translate_newlines, encoding=encoding
  412         )
  413     elif isinstance(obj, dict):
  414         for key, value in obj.items():
  415             obj[key] = _decrypt_object(value, translate_newlines=translate_newlines)
  416         return obj
  417     elif isinstance(obj, list):
  418         for key, value in enumerate(obj):
  419             obj[key] = _decrypt_object(value, translate_newlines=translate_newlines)
  420         return obj
  421     else:
  422         return obj
  423 
  424 
  425 def render(gpg_data, saltenv="base", sls="", argline="", **kwargs):
  426     """
  427     Create a gpg object given a gpg_keydir, and then use it to try to decrypt
  428     the data to be rendered.
  429     """
  430     if not _get_gpg_exec():
  431         raise SaltRenderError("GPG unavailable")
  432     log.debug("Reading GPG keys from: %s", _get_key_dir())
  433 
  434     translate_newlines = kwargs.get("translate_newlines", False)
  435     return _decrypt_object(
  436         gpg_data,
  437         translate_newlines=translate_newlines,
  438         encoding=kwargs.get("encoding", None),
  439     )