Pipservices-invoking and managing

To invoke and manage a pipservice with Puppet:

1. Use pipservice invocation directly from puppet nodefiles to install the custom app into /opt/$name to be run as and owned by root:root. 

   sample hiera yaml entry

   pipservice:
     testapp:
       tag: 'master'

   
   Alternatively, use a module to install a pipservice.

   sample in module usage

   pipservice { 'testapp':
     tag => 'master',
   }

2. Your app will be installed into /opt/testapp (or whatever name you pick) and enabled as a systemd service.

3. For syslog output, use asfpy's syslog.Printer class:

   import asfpy.syslog
   print = asfpy.syslog.Printer(stdout=True, identity='TestApp') # stdout flag will print to stdout AND syslog, if set.
   print("This will hit stdout AND syslog!")

Configuring templates and using secrets with pure pipservice

Make use of encrypted variables with `custom_yaml`

If you need to use secrets in a pipservice, you can use pipservice's custom_yaml feature:

• Create your YAML formatted configuration, and save it as a variable in the appropriate encrypted data file (EYAML) as pipservice::$APPNAME::custom_yaml.
• $APPNAME.yaml (for instance testapp.yaml), will be created by puppet for you in your install dir, and get the secrets.

Templating configuration files with `custom_yaml_template`

You can opt to use a template for your configuration. Leave the template in plain-text and interpolate secrets using the same as above, but specifying the template as custom_yaml_template:

This supports up to five secrets via EYAML, called secret1, secret2, ... secret5.
They should be set up as pipservice::$name::secret1 and so on in EYAML, and may be referred to in the template as <%= @secret1 %> etc.

pipservice:
  loggy:
    tag: master
    # The below will create /opt/loggy/loggy.yaml using the following as an ERB template:
    custom_yaml_template: |
      # Loggy Jr configuration file.
      analyzer:
        # These are the paths that will be (recursively) checked.
        # /var/log would also check /var/log/tomcat/ for instance.
        paths:
          - /var/log/
          - /x1/log/

      tags:


      # Primary and secondary JSON receiver.
      elasticsearch:
        primary:
          host:   es1-eu-mid.apache.org
          port:   443
          ssl:    true
          prefix: <%= @secret1 %>

        backup:
          host:   es2-eu-mid.apache.org
          port:   443
          ssl:    true
          prefix: <%= @secret1 %>

      rawfields:
        # Fields that, in each document type, should be treated as non-analyzed strings.
        httpd_access:           uri,clientip,remote_user,vhost,geo_city,geo_country,geo_combo,geo_coords,geo_lat,geo_long
        apache_access:          url,client_ip,remote_user

Configuring templates and using secret values with a wrapper module

Using a puppet module to handle configuration templating and secrets offers support for value interpolation and secrets as well as values in a more descriptive scope.

This method leverages pipservice::custom_yaml

1. Write custom_config.yaml into testapp/templates/custom_config.yaml.erb and configure the template values appropriately.
2. Read the template contents into a variable and pass it to pipservice as custom_yaml

#/etc/puppet/modules/testapp/manifests/init.pp

class testapp (
# defined in yaml as testapp::value1
$value1 = '',
){
  # testapp/testapp.yaml.erb can contain '<%= @value1 %>'
  $customyaml = template('testapp/testapp.yaml.erb')
  pipservice {'testapp':
    tag => 'master',
    custom_yaml => $customyaml,
    owner => 'www-data',
    group => 'www-data',
    user  => 'www-data',
  }
}

Caveats and Troubleshooting

Permissions issues

The above example shows both owner and group being set. the Pipservice module will use these values to set ownership information for the $installdir (typically /opt/$NAME). If a user and group are specified in the pipservice-$NAME.service file then owner and group must be passed to pipservice in this way, all ownerships otherwise default to 'root:root'. NB: Due to recent security changes in git, puppet/vcsrepo now requires that a user parameter be passed, which should be the same as the file's ultimate owner, e.g. www-data, apmail, etc. It is this parameter which defines the userID that performs the vcsrepo checkout of the pipservice git repository.

Ensure that the /opt/$NAME/$NAME.yaml file is readable by the user that is to start the service. If the file is not readable, `owner` and `group` have not been passed to pipservice properly.

Missing Python Packages

If, when running a pipservice as a non-root user, systemd reports a missing package for the new user, try these steps:

1. delete the PIpfile and Pipfile.lock files for that Python environment.
2. commit a whitespace bump to the infrastructure-$NAME repository
3. re-run puppet on the target machine.

This should force the pipservice module to re-evaluate the Git checkout and consequently attempt to re-install all of the designated requirements.

Systemd tools

Pipservice services are systemd-journald enabled. Best practice is to use `journalctl -u pipservice-$NAME` or `systemctl status pipservice-$NAME to monitor service output.

Pipservice changes

The pipservice module checks the integrity or $INSTALLDIR (/opt/$NAME) using the Git revision.To have changes made to the pipservice itself deployed by puppet, commit a whitespace revision (*bump*) and then run puppet on the target machine.