We started evaluating Rundeck as a way to automate responses to PagerDuty.

Requirements/scenerios of an automated system to run adhoc scripts/commands:

  • When PagerDuty fires off a page, try to restart httpd on the failed server. Report back to PD the output.
  • Restart something running Tomcat (JIRA, Confluence) which might fail on some java foo
  • Run a set of commands or a script on multiple servers (update a puppet client cert, run certbot to update LE cert, create/update configuration on a server)
    • update resovl.conf on all the Y! nodes (asf9xx)
  • Sending a project to the attic, graduating a project, etc.

Product Comparisons

Ad Hoc – Bolt and Ansible

They both do almost the same thing out of the box: provide agentless commands to remote systems.

bolt fits in with puppet better, but both ansible and bolt solve this problem but do it in different ways:

bolt fits into the puppet ecosystem better, and is written in ruby.

ansible obviously does the same but is more mature in this field than bolt and is written in python.

Test Cases:

  1. Generate new or Audit existing inventory

Process – Rundeck and Ansible Playbooks / Ansible

Source: https://stackoverflow.com/questions/31152102/is-it-a-good-idea-to-make-ansible-and-rundeck-work-together-or-using-either-one

Similarities

  • Both tools are agent-less and use SSH to execute commands on remote servers
  • Rundeck's main concept is Node, the same as Ansible's inventory, the key idea is to define/manage/group the target servers
  • Rundeck can execute ad-hoc commands on selected nodes, Ansible can also do this very conveniently.
  • Rundeck can define workflow and do the execution on selected nodes, this can be done with Ansible by writing playbook
  • Rundeck can be integrated with CI tool like Jenkins to do deploy work, we can also define a Jenkins job to run ansible-playbook to do the deploy work

Differences

  • Rundeck has the concept of Job, which Ansible does not
  • Rundeck has Job Scheduler, which Ansible can achieve this with other tools like Jenkins or Cron tasks
  • Rundeck is a far heavier system to run for just adhoc commands. It's a full system, front end web UI, database, etc. Anisble (without Tower) is light weight.

Notes (more of a pro/con/issues list)

Rundeck

Rundeck has a web UI that can be used to admin the system. However there are parts of that are still not in the UI and you have to configure on disk. This ends up being confusing.

Just using the Community/FOSS edition, passwords/keys aren't encrypted on disk.

In order to get certain plugins for Rundeck, we'd have to pay for the Enterprise edition. Pagerduty integration is limited in the Community edition.

The cost of one server and 5 users is $27,500 for the first year and $22,500 every year after that. We have not opened discussion about donated services.

RD is a complete system that we'd have to admin. While it can write files to disk for the "database" it prefers a relational DB. We'd also have to keep the web server up, monitored, etc.

Ansible

Ansible Tower, which is similar-ish to Rundecks UI is not free. We haven't received a quote, though it's a Red Hat product, which we might be able to get donated.

However, that doesn't matter, Tower isn't required to be 100% running with this adhoc. Even if we got deeper with Ansible, Tower would be a nice to have not a need to have.

Running adhoc commands is so easy a cave man could do it(tm). To make it easier, we'd just have to have a repo that has inventory up to date and some additional configs.

Ansible is waaaaaay more light weight. We could either run things adhoc from our own systems or setup a very small VM to host this all and act as single point of entry. (this is an architecture thing we'd have to discuss between all of us)

There's more of a learning curve with Ansible playbooks/etc. Not by any means steep, but in RD you can just load the UI, click some UI bits and run a command that keeps track of the output, etc.

Ansible doesn't natively have the concept of schedules, though there are plenty of solutions.


They can be used together: https://www.rundeck.com/ansible

Test Casts:

  1. Automate the project attic / podling retirement process.

Other – Ansible and Terraform

 It's not a bad idea to use both, It could also be of value with packer, creating images locally and using one of the many ansible virtualization modules to launch it.

vagrant, packet and terraform all work together to provide all kinds of beneficial goodies and ansible is really easily compatible with all three.


terraform module -- https://docs.ansible.com/ansible/latest/modules/terraform_module.html#terraform-module
ansible and vagrant for local environment testing -- https://docs.ansible.com/ansible/2.3/guide_vagrant.html
packer has an ansible provisioner -- apply the load to your machine before creating the image.


email users@infra.apache.org for any queries or suggestions.