Welcome to our blog post on building a custom Ansible module! In this tutorial we will show how to create a Python module that we can call through Ansible. We will use the Python Requests that allows us to interact with a website's API.
Creating and using a Python module for Ansible can offer several advantages, as it allows you to extend the capabilities and flexibility of Ansible. Ansible, by default, provides a wide range of modules that cover common use cases for configuration management, orchestration and deployment tasks. However, there are scenarios where using a custom Python module can be beneficial:
-
Specialized functionality: If your infrastructure or application requires tasks that are not covered by existing Ansible modules, creating a custom Python module allows you to implement the exact functionality you need.
-
Integration with APIs: Some applications or services may expose APIs that don't have dedicated Ansible modules. In such cases, you can write a Python module that interacts with the API, allowing you to automate actions against that service using Ansible.
-
Reusability: If you have specific tasks that are recurring across multiple playbooks, creating a custom Python module allows you to encapsulate the logic and reuse it easily.
-
Complex data manipulation: Some tasks might involve complex data manipulation or calculations that are more efficiently handled in Python than in Ansible's Jinja2 templates.
-
Access to Python libraries: Python has an extensive ecosystem of third-party libraries. By creating a custom module, you can leverage these libraries to accomplish tasks that might otherwise be challenging with Ansible alone.
Ansible is widely used by companies and private users and extending it with
custom modules makes it even more interesting! For this blog post we will create a module that authenticates to a website and sends a POST request using the API.
It is important to note that Ansible already provides a built-in module specifically designed for handling HTTP requests, known as the uri module.
The code provided in this context serves a purely educational purpose, aiming to illustrate the basic structure of a Python module for Ansible and demonstrate how variables are passed from the Inventory to the playbook, eventually being utilized by the module.
It is important in Ansible to always utilize the most corresponding module for the task intended to run. It could be simple to use a shell or a command module, and run a curl against our target. Or a command for creating a cronjob although the module for the cronjob exists already.
Modules in Ansible aim at supporting a variety of scenarios for the same action.
The uri module for example, it's capable of interacting with web pages for simple requests as much as requests with complex API structure and different HTTP methods, reporting back to the task the proper status code and the result of the request, whether this failed or succeeded.
Let's start!
A simple use case
For no particular reason, our friend asked if we could set up an automation that would update his phone number on his favorite website, without any manual interaction from him.
We know that this website requires authentication and after analyzing the API we noticed that they respond on a custom port.
Our friend likes Ansible and he would want to run a playbook for running the above task.
From this simple scenario we have understood that we need the URL of the website, his user for authentication, the port for communicating to the API and his phone number.
Setting up the environment
Our project will depend on the Python module 'requests' and we will also use 'json'.
We will use a Python virtual environment for installing any dependency we need.
[user@localhost ~] $ python -m venv myvenv
[user@localhost ~] $ . myvenv/bin/activate
(myvenv) [user@localhost ~] $ pip install requests jsonType deactivate while within a python-venv to exit.
Re-enable it using .
We must also create the proper Ansible role structure for including our module.
We can use the command ansible-galaxy init my_custom_role for creating the base Ansible Role directory tree:
$ ansible-galaxy init my_custom_role
- Role my_custom_role was created successfully$ tree .
.
└── my_custom_role
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
10 directories, 8 filesAnd create the directory library inside my_custom_role that will contain the custom modules.
$ mkdir my_custom_role/library
Creating the custom module
The filename of the module is the same name that Ansible will use within a task to call it. Let's create my_custom_module.py.
Open your favorite text editor and create the module with the following content:
#!/usr/bin/env python3
from ansible.module_utils.basic import AnsibleModule
import requests
import json
def authenticate(module, url, port, user, password):
session = requests.Session()
endpoint_login = "{}:{}/api/login".format(url, port)
session.headers = {'Content-type': 'application/json'}
auth_data = json.dumps({
'Username': user,
'Password': password
})
try:
response = session.post(endpoint_login, data=auth_data)
response.raise_for_status()
return session
except requests.exceptions.RequestException as e:
module.fail_json(msg="Authentication failed: {}".format(str(e)))
def update_number(module, url, port, session, number):
endpoint_update = "{}:{}/api/edit/update".format(url, port)
endpoint_save = "{}:{}/api/edit/save".format(url, port)
update_number = json.dumps({"Number": number})
try:
session.post(endpoint_update, data=update_number)
response_save = session.post(endpoint_save, data='1')
response_save.raise_for_status()
result = {
'changed': True,
'status_code': response_save.status_code
}
module.exit_json(**result)
except requests.exceptions.RequestException as e:
module.fail_json(msg="Update failed: {}".format(str(e)))
def main():
module_args = {
'url': {'type': 'str', 'required': True},
'port': {'type': 'int', 'required': True},
'user': {'type': 'str', 'required': True},
'password': {'type': 'str', 'required': True, 'no_log': True},
'number': {'type': 'str', 'required': True},
}
module = AnsibleModule(argument_spec=module_args, supports_check_mode=False)
url = module.params['url']
port = module.params['port']
user = module.params['user']
password = module.params['password']
number = module.params['number']
session = authenticate(module, url, port, user, password)
update_number(module, url, port, session, number)
if __name__ == '__main__':
main()
Break down this code for me please!
First, we import the necessary modules and libraries and among these, when writing an Ansible module, we need from ansible.module_utils.basic import AnsibleModule.
from ansible.module_utils.basic import AnsibleModule
import requests
import jsonThen, within the main() function, we define the expected arguments for our module: url, port, user, password and number.
We make use of AnsibleModule() for defining the variables and will assign them to the module variable. Remember this module because it will play a big role within our application.
def main():
module_args = {
'url': {'type': 'str', 'required': True},
'port': {'type': 'int', 'required': True},
'user': {'type': 'str', 'required': True},
'password': {'type': 'str', 'required': True, 'no_log': True},
'number': {'type': 'str', 'required': True},
}
module = AnsibleModule(argument_spec=module_args, supports_check_mode=False)
url = module.params['url']
port = module.params['port']
user = module.params['user']
password = module.params['password']
number = module.params['number']Notice the no_log: True for the password. This will hide the password value in the Ansible output and logs.
We can now pass the variables above to the function that utilizes them.
session = authenticate(module, url, port, user, password)
update_number(module, url, port, session, number)We created two functions: authenticate and update_number.
We need to be authenticated before we can run any update request. Because of this, the return value of authenticate is saved into the variable session, which is then passed to the function update_number.
def authenticate(module, url, port, user, password):
session = requests.Session()
endpoint_login = "{}:{}/api/login".format(url, port)
session.headers = {'Content-type': 'application/json'}
auth_data = json.dumps({
'Username': user,
'Password': password
})
try:
response = session.post(endpoint_login, data=auth_data)
response.raise_for_status()
return session
except requests.exceptions.RequestException as e:
module.fail_json(msg="Authentication failed: {}".format(str(e)))The function authenticate accepts the common parameters username and password among others, it constructs the login url and then sends a post request that is saved into the variable session.
We create a session and add the correct headers to the requests.
Credentials are sent as json and the json.dumps() takes care of building the json structure.
We use a try, except anytime our code processes a request for which we may not be sure whether it succeedes or not. For example, if credentials are wrong we want our application to fail in the best way possible communicating us that the authentication failed with the error that got caught.
The argument module is also passed to the function and Ansible uses it to handle the error.
When the function authenticate ends and the session is returned to the main(), this is passed to the seconds function update_number.
def update_number(module, url, port, session, number):
endpoint_update = "{}:{}/api/edit/update".format(url, port)
endpoint_save = "{}:{}/api/edit/save".format(url, port)
update_number = json.dumps({"Number": number})
try:
session.post(endpoint_update, data=update_number)
response_save = session.post(endpoint_save, data='1')
response_save.raise_for_status()
result = {
'changed': True,
'status_code': response_save.status_code
}
module.exit_json(**result)
except requests.exceptions.RequestException as e:
module.fail_json(msg="Update failed: {}".format(str(e)))Similar to authenticate, this function sends two post requests for both updating the phone number and saving it.
"Update" does not necessarily mean that the request is complete when the value is inserted.
In this code we wanted to show that it is not required to use json.dumps() for any request, and that it depends by the webservice how they handle the data posted. A simple "1" may be enough in some cases, or at least, it is enough in this example.
APIs are (most of the time) documented by the webservice you'd like to use. They will tell how to communicate with them and under which URL options are found.
We use again the try, except, but this time we will shift the attention on the "succeeded" portion of the code, where we can see that the variable result contains a typical information found in Ansible's tasks, changed, and the status_code, that is widely used in HTTP requests for analyzing the status of the request just sent.
The module exits with a clean module.exit_json(**result) that returns to the Ansible controller the status about the module execution.
Status code handling
response_save = session.post(endpoint_save, data='1')
response_save.raise_for_status()When an HTTP request is successful its status code is generally 200.
In our examples we have used the format above quite often, this code comes from Python and handles the status_code for us. The raise_for_status() is probably the best way to operate with HTTP requests under a Python environment.
The reason is that a POST request can have an HTTP code of 200, 201 and also 204, with the difference being what's being returned. It is always a good idea to test the code beforehand to know the possible return codes for an HTTP request.
The program CURL is one of the favourite tools of any DevOps. It is the Swiss Army Knife of the IT. Together with the verbose argument (-v) there is a ton of useful information that can be extracted from an HTTP request, return codes included.
For printing only the status code of an HTTP request using CURL, for example, we could write the command as follows:
curl -o /dev/null -s -w "%{http_code}" https://github.comFor testing a POST request we can write our CURL as follows (thanks to httpbin.org for providing a beautiful test environment):
curl -X POST "https://httpbin.org/delay/1" -H "accept: application/json"
curl -o /dev/null -s -w "%{http_code}" -X POST "https://httpbin.org/delay/1" -H "accept: application/json" # print only the http_codeBut, back to our module, comparing the status_code directly, maybe using an if condition, can't be considered a good way of programming in Python when using requests. It is thanks to raise_for_status() and the benefits it gives to the error handling that we can replace a code that uses bare http requests:
api_url = "{}:{}/api/endpoint".format(url, port)
response = requests.post(api_url, data='1')
if response.status_code == 200:
module.exit_json(changed=True, response=response.json())
else:
module.fail_json(msg="API request failed", response=response.text)with a code that uses raise_for_status():
api_url = "{}:{}/api/endpoint".format(url, port)
session = requests.Session()
# 'session' comes from the login
response = session.post(api_url, data='1')
# request.post() works too but we want to keep it consistent all over our code
response.raise_for_status()
Testing the custom module
With our new custom module in place it is time to test it.
Create a new Ansible playbook file, e.g.: my_custom_module_playbook.yml, with the following content:
---
- name: Send a custom payload to cloudWerkstatt
hosts: localhost
gather_facts: false
vars:
api_url: https://cloudwerkstatt.com
api_port: 443
user: "myuser"
password: "" # You really would want to use Ansible Vault here!
phone_number: "123456789"
tasks:
- name: Update phone number in my favorite website
my_custom_module:
url: "{{ api_url }}"
port: "{{ api_port }}"
user: "{{ user }}"
password: "{{ password }}"
number: "{{ phone_number }}"Finally, we can launch this playbook as usual:
$ ansible-playbook my_custom_module_playbook.yml
Conclusions
We saw how simple it is to develop a Python module for Ansible.
Ansible is a powerful and versatile tool for configuration management, orchestration and automation of IT infrastructure. It offers numerous benefits such as simplicity, agentless architecture and easy scalability, making it a popular choice for managing complex environments.
One of the key strengths of Ansible is its extensibility through custom Python modules. Creating your own module allows you to tailor Ansible to your specific needs, providing specialized functionality, integrating with APIs and accessing Python libraries for complex tasks.
By developing custom modules, you can achieve higher levels of automation, optimize performance and maintain clean and reusable playbooks, further enhancing the flexibility and efficiency of your Ansible workflows.