Automate Reporting with Ansible

by | Nov 17, 2023 | Blog

automate_reporting_with_ansible

Automate Reporting with Ansible

Ansible is a powerful tool known for its capabilities in configuration management and application deployment.
However, it can be extended to handle another essential task: automated reporting. Leveraging Ansible's modules and automation capabilities, you can simplify the process of generating reports.
In this post, I will guide you on how to create a simple report and have it delivered directly via mail.

Understand the Need for Reporting

In this section, we'll explore the logic behind automated reporting with Ansible. Ansible is a versatile tool that allows us to manage virtually everything in our environment, making it an ideal candidate for streamlining reporting tasks. When you already use tools like AAP (Red Hat Ansible Automation Platform) or AWX, you likely schedule playbooks within them. It's equally efficient to schedule reporting in the same tool, eliminating the need for additional reporting-specific solutions.

While Grafana does offer a reporting feature, it's restricted to the enterprise version, which may not be accessible to everyone. However, with Ansible, you have the freedom to design reports that align perfectly with your specific needs

Creating a Practical Example

To illustrate the power of Ansible in reporting, let's walk through an example that can be adapted to various use cases. We'll use Prometheus as the data source, but you can use databases, API endpoints, or any data source relevant to your needs.

In this example, we'll fulfill a manager's request for a monthly report on the number of Kubernetes worker nodes per cluster and the total number of worker nodes across all clusters.

The manager expects a report with the following structure:

Cluster Name Number of Worker Nodes
europe-north1-a-k8s-1 6
europe-north1-a-k8s-2 6
europe-central2-a-k8s-1 3
europe-central2-a-k8s-2 3
europe-west1-c-k8s-1 4
europe-west1-c-k8s-2 4
us-west1-a-k8s-1 5
us-west1-a-k8s-2 5
us-central1-a-k8s-1 4
us-central1-a-k8s-2 4
asia-east1-a-k8s-1 3
asia-east1-a-k8s-2 3 
Total worker nodes: 50

Collecting Data for Reporting

To gather the data required for reporting, we can take advantage of Prometheus, which is deployed to monitor each Kubernetes cluster.
In this setup, metrics from individual Kubernetes clusters are federated into a central Prometheus instance. This central Prometheus instance serves as a unified endpoint for central view, allowing us to easily query and collect data for our reporting needs.

The process of collecting this data and using it in Ansible is straightforward. We can run queries using the ansible.builtin.uri module in Ansible to access the central Prometheus instance's API. The results of these queries, which are in JSON format, are registered and can be utilized in Ansible Jinja2 templating for report generation.

If you are already familiar with the Prometheus Query Language (PromQL), you can write custom queries to extract the exact information you need. For instance, you can use a query like this to obtain the number of worker nodes per cluster. The query is stored in the Ansible inventory as a variable.

reporter_prometheus_worker_nodes_per_cluster: 'sum by (cluster_name) (kube_node_role{role="worker"})'

Additionally, you can calculate the total number of worker nodes across all clusters with a query like this:

reporter_prometheus_worker_nodes_total: 'sum(kube_node_role{role="worker"})'

Ansible Reporting Automation

Let's dive into the automation process. To keep this blog concise and focused, I'll provide an overview of the key components. In practice, you can expand and customize this process according to your specific reporting needs.

First, we'll create an Ansible role, which I've named reporter_prometheus. This role serves as a modular solution that can be easily extended with multiple templates for different queries and reports. Using tags, you can even call specific reports when needed.

Here's an overview of how this reporting automation works:

  1. Create the reporter_prometheus Ansible role, which retrieves data from Prometheus and sends the report to the manager.
  2. Schedule the playbook execution in either AAP (Red Hat Ansible Automation Platform) or AWX. If you're not using AAP or AWX, you can schedule the playbook to run directly on the controller node using a cron job, depending on your preference.
  3. Extend the scheduling workflow in AAP or AWX with conditions to handle scenarios where the report fails to run. This allows you to incorporate additional logic and notifications.

Within the reporter_prometheus role, we have two key components:

  1. Task Play: This part of the role is responsible for extracting data from Prometheus and generating the report.
  2. Template: Jinja2 templates are used to format the collected data into the desired report structure. These templates can be customized to meet your specific reporting requirements.

Ansible Task Play

  • This task retrieves data from Prometheus for the total number of worker nodes across all clusters and registers the result in the __worker_nodes_total variable, which is then used in the Jinja2 template for the worker report.
- name: Query Prometheus - Total workers nodes
  ansible.builtin.uri:
    url: "https://central-prometheus.cloudwerkstatt.com:9090/api/v1/query?query={{ reporter_prometheus_worker_nodes_total | urlencode }}"
    method: GET
    headers:
      Content-Type: "application/json"
    validate_certs: "true"
  register: __worker_nodes_total
  retries: 15
  delay: 20
  until: __worker_nodes_total.status == 200
  delegate_to: localhost
  • This task gathers data from Prometheus to determine the total number of worker nodes per cluster. The result is registered and used in a for loop in the template to generate the content of the table.
- name: Query Prometheus - Worker nodes per cluster
  ansible.builtin.uri:
    url: "https://central-prometheus.cloudwerkstatt.com:9090/api/v1/query?query={{ reporter_prometheus_worker_nodes_per_cluster | urlencode }}"
    method: GET
    headers:
      Content-Type: "application/json"
    validate_certs: "true"
  register: __worker_nodes_per_cluster
  retries: 15
  delay: 20
  until: __worker_nodes_per_cluster.status == 200
  delegate_to: localhost

As you can see, I've removed all variables from the play above to keep it simple and easy to read. The only remaining variable contains the PromQL query, which utilizes the urlencode filter.
These tasks serve as the core of the reporting automation, collecting the necessary data and registering the results. Registered variables will be used in the template, as shown in the next section.

  • This task sends an email with an HTML report, created using a Jinja2 template.
- name: Send E-mail HTML Report
  community.general.mail:
    host: "smtp-server.cloudwerkstatt.com"
    port: "587"
    username: "top-secret-username"
    password: "SECRET"
    subject: "Worker Nodes Report"
    body: "{{ lookup('template', 'templates/worker_report.html.j2') }}"
    to: "your.manager@cloudwerkstatt.com"
    sender: "no-reply@cloudwerkstatt.com"
    secure: "always"
    charset: "utf-8"
    subtype: html
  delegate_to: localhost

Ansible Jinja2 Template for Report

Here's the responsive HTML report template (worker_report.html.j2) that contains the information requested by the manager.

<!DOCTYPE html>
<html>
<head>
    <style>
        body {
            font-family: Arial, sans-serif;
            background-color: #f0f0f0;
            margin: 0;
            padding: 0;
        }

        .container {
            width: 90%;
            max-width: 600px;
            margin: 0 auto;
            padding: 20px;
        }

        h2 {
            text-align: center;
            color: #333;
        }

        h3 {
            text-align: left;
            color: #666;
        }

        table {
            width: 100%;
            border-collapse: collapse;
            margin: 20px 0;
        }

        table, th, td {
            border: 1px solid #333;
        }

        th, td {
            padding: 10px;
            text-align: left;
        }

        th {
            background-color: #333;
            color: #fff;
        }

        footer {
            text-align: center;
            margin-top: 20px;
        }

        p {
            color: #999;
        }

        img {
            display: block;
            margin: 0 auto;
        }
    </style>
</head>
<body>
    <div class="container">
        <img src="https://CompanyLogoImageUrl" alt="Company Logo" width="180" height="30" />
        <h2>Worker Nodes Report</h2>
        <h3>Report Execution Date: {{ ansible_date_time.iso8601 }} {{ ansible_date_time.tz }}</h3>
        <h3>Report Month: {{ ansible_date_time.month }}</h3>
        <h3>Total Worker Nodes: {{ __worker_nodes_total.json | json_query('data.result[0].value[1]') }}</h3>

        <table>
            <tr>
                <th>Cluster Name</th>
                <th>Number of Workers Nodes</th>
            </tr>
{% for each in __worker_nodes_per_cluster.json | json_query('data.result') %}
            <tr>
                <td>{{ each.metric.cluster_name }}</td>
                <td>{{ each.value[1] }}</td>
            </tr>
{% endfor %}
        </table>

        <footer>
            <p>© {{ ansible_date_time.year }} cloudWerkstatt. All rights reserved.</p>
        </footer>
    </div>
</body>
</html>

Data is dynamically populated into the table using a for loop from a variable registered during the play.
The JSON data is extracted using the json_query filter, and the date is sourced from Ansible facts.

Generating Report

Here's the report as it appears in the mailbox:

worker_nodes_report_result

Scheduling the Report

The scheduling of this report is managed through AAP/AWX. These tools execute the playbook on a predefined schedule and deliver the report directly via mail.

It's important to note that I haven't covered what happens if the report cannot obtain data. You can handle this logic inside the playbook, or you can take advantage of the workflow functionality in AAP/AWX, which allows you to execute a different playbook if this one fails. This flexibility enables you to create a workflow that suits your specific requirements.

Power of Ansible

In the world of automation, Ansible stands out as a powerful and versatile tool.
It offers the flexibility to adapt to your unique requirements.
With a touch of creativity, creating automated reporting with Ansible becomes a straightforward process using just a few Ansible modules.
You can even explore the possibility of extending this approach to handle more complex logic or creating custom modules for data gathering and processing.