Ansible Integration with Canonical MaaS

2 Sep

By Billy Felton Cloud Computing , HowTo Guides #ansible #maas #cloudcompute # No Comments

Dynamic Ansible Inventory – CMDB Based on What’s Running.

About MaaS

If you are familiar with MaaS ignore this section, if not, this is MaaS in a nut shell. MaaS is short for Metal as a Service |..|,. and provides a central point to manage hardware and infrastructure. It has nice UI out of the box where you can manage most aspects of infrastructure. Breaking down the fundamentals of what MaaS has is the easiest way to look grasp it.

Project Repo: https://github.com/maas/maas

What MaaS does:

It will listen on a subnet for new devices and discover them through ARP and DHCP.
It will perform all necessary PXE network booting functions.
- DHCP
- Multiple OS Images (supports pretty much any OS)
- Custom and canned commissioning, testing and deployment automation.
- Custom network configuration and support for multiple networks, vLANs and Fabrics. (Mostly 3rd Party)
- DNS integration and support for multiple zones.
Good integration with KVM and virsh. (Easily define, deploy and destroy VMs)
A concept of resource pools and availability zones.
A great Configuration Management Database (CMDB) for what is actually running.
Access via API, CLI or UI. The API has more functionality and flexibility than the UI. Give the docs a good read.
Basic key store for SSH

What MaaS doesn’t do.

MaaS is not a multi-tenant environment and has no mechanisms to dynamically or statically assign roles or resources based on business units or organizations. But the API is pretty flexible and will allow for this functionality as long as you implement MaaS with that in mind. If you need multi-tenant functionality, you will need to plan your implementation in detail. You will also need to write some code around the APIs to provide this functionality and possibly deploy an instance of MaaS for each tenant.

There are no role based access controls (RBAC), you are either a privileged (admin) or not. I have not figured out what a non-admin user would be useful for.

MaaS also is lacking SDN integration. Networking is based on what networks exist on the server running MaaS. All integration with software defined network services has to be manually built and presented to MaaS as network interfaces. Not hard to solve at all.

Although MaaS gives you the impression it plays well with vLAN layouts, it really doesn’t. You will need to customize images to include vLAN support and attach them to tagged vLANs post deployment. Ansible does this quite nicely, so it really isn’t a big deal.

Navigating VLANS:

Option 1

Spin up a VM on an untagged default vLAN 0.
Run through MaaS cloud-init stuff. (Add your required packages)
Plumb the admin interface to the tagged vLAN for that tenant.
Build and attach the public interface on its vLAN.

Option 2

Build out a vLAN through your favorite SDN stack to present to MaaS as an untagged, default vLAN (even though it isnt)
Build out a public vLAN that is presented in the same way.
Attach to both interfaces as needed.

**Both options have no security considerations. This will need to be built out on your own. I just use a git repo with vLAN configs, SELinux and AppArmour rules and let CI/CD do the rest. But I consider VIM to be a UI so there are several GUI based security tools for those that don’t like editing files.

MaaS does not have a concept of security groups or policies. This functionality can be added through third party or custom developed code that can integrate with MaaS deployment scripts and Ansible.

I personally love Canonical MaaS as it is written in python and is actually far more flexible than most people realize. By most people I mean the ones that only use the UI. MaaS was built and driven by system administrators who wanted to automate annoying tasks and enable consistent deployment methodologies. Many of the decisions to truly automate infrastructure are still manual and are meant for administrators to make. Nothing a few “helper” programs cant fix.

What I would like to see from MaaS in the future:

Better SDN Integration: Layer 2 control to manage vLANs which would allow assigning hardware by port. The ability to deploy configured hosts on multiple tagged vLANs.
Post deployment configuration management. Awareness of changes made to hosts after MaaS deploys them. Currently If you spin up a VM and then upgrade the OS, MaaS still will show the earlier OS.
Plugin capabilities on the UI to easily integrate 3rd party products/projects. Like SDN, CICD tools etc.
Better DNS integration and plugins for common cloud based DNS solutions.
Security Groups / Policies and RBAC
Multi-tenant capabilities: quotas
Better options for storage NAS/FAS. Snapshots/backups etc.
Better Terraform providers for MaaS

Ansible

Using Ansible to manage your deployed infrastructure is pretty much a norm for most. So if MaaS does configuration management and serves as a CMDB why do you need Ansible? People generally remediate issues in production as a “hotfix”. The “hotfix” is pushed to production and integrated in Dev for future deployments. In the world of CI/CD you generally want to capture the changes needed and integrate them into your deployment automation as well as remediate running systems. We generally also like to separate infrastructure from workload. Infrastructure being actual infrastructure that has to be there for virtual infrastructure as code stuff to work.

Ansible is also great at picking up where Cloud-Init left off. Adding more complicated configuration options where event concurrency is a challenge involving multiple resources.

Ansible is great for answering questions about what is deployed. Pulling detailed configuration information like SELinux or AppArmour configs, status etc.

Ansible: https://github.com/ansible/ansible

Why this project?

There are a couple of inventory plugins available for Ansible to build MaaS based inventories. I struggled with how buggy they were as well as how limited they were on how they presented inventory. Most simply provide a complete list of FQDN hosts under a group called “MaaS”. I simply wanted the methods used to organize resources in MaaS to be relevant in my Ansible inventory.

The focus of this project is to allow flexibility when building an inventory, supporting availability zones, resource pools, common OS releases and custom tags. It will also allow you to include or not include infrastructure (bare metal) systems. It is intended to conform to how you use MaaS and not mandate how you implement MaaS. The concept is to use MaaS as your CMDB based on what is really out there. It is NOT an Ansible plugin, it is meant to run independent of Ansible by just generating an Ansible hosts file. There is an option that will allow you to determine the host inventory file format as either INI or YAML. These are the most common inventory plugins used so I did them first.

Disclaimer: I have been in IT for over 30 years and the last 15 have mostly been spent approving purchase orders and budgets, sorting out politics and dealing with HR. My codeing skills are rusty but coming back. I was primarily a C developer then PHP and really only used Python to automate my desktop environment in the GNOME/CORBA days.

MaaS Requirements

To use AnsibleMaaS you will need a MaaS system to integrate with and a configured Ansible control install. Ansible will need network access to the MaaS API with admin privileges via SSL and whatever port you are using, normally 5240. Ansible will also need network access to all of the compute nodes, physical and virtual.

I would recommend deploying a small lab environment before you do anything in production. If you have a production MaaS system set your lab system up the same but with scaled down hardware and overall footprint.

For a lab you will need one linux server as the MaaS Region and Rack controller and a few compute nodes. You will need another server to use as your Ansible Control system. Ansible can be installed on your MaaS server if you dont have the hardware available. You will need to set up two vLANs one for administration and one “public” facing. The administration vLAN should only have MaaS servers and the compute nodes intended to be part of the MaaS cluster. You can use a single network but that could leave APIs exposed to public facing networks.

You can get MaaS here: https://maas.io

Admin Network

The admin network should be built on a private IP pool like 192.168.x.x or 10.x.x.x and should have very controlled points of ingress and egress. Setting up proxies where needed to fulfill external services.. Your MaaS servers will need support from some outside services like OS images and package repos. I recommend building your own image repo as well as your own package repos. This will give you better control of your infrastructure.

The admin network is intended to allow servers to power up, get an IP address and then PXE boot images. In addition the admin network can serve as a power management network for servers that have that capability. If your hardware will only do power management on a single network interface that cant support PXE, you may have to build a separate power management network too. The admin network is how MaaS communicates with compute nodes to spin up and manage VMs through virsh.

Public Network

This will be the network that servers you build with MaaS use to communicate with consumers. This will generally be a public routeable network but could also be a private network sitting on a SDN layer which exposes network interfaces to the outside world as they are defined or needed.

Ansible and MaaS

As bare metal systems are spun up and down, tracking these systems in Ansible can be a challenge. Each of these bare metal systems will likely also be a hypervisor spinning up and down virtual machines. Most of the inventory plugins in Ansible for MaaS are simplistic and give little ability to customize your inventory. Dynamic inventory from MaaS has always been problematic as systems like MaaS are used to build actual infrastructure. This means the way stuff is cabled, a bios configuration or a new model network interface can disrupt consistencies needed for automation. MaaS actually does a pretty good job abstracting this.

One of the issues with MaaS is that to remediate issues found in production requires you to modify your build scripts. You then destroy the VM or hardware and rebuild it. This can be time consuming but in general destroying bad deployments is a good practice. It is not always convenient to destroy a production VM and rebuild it. Ansible will allow you to modify running systems without a rebuild. A practice not really embraced as a good practice but better done with repeatable automation than to doing it manually.

You can get MaaS Here: https://maas.io

You can get Ansible Here: https://www.ansible.com

You can get AnsibleMaaS from https://github.com/billyFelton/AnsibleMaaS

or with git:

>git clone https://git.halo.inc/halo-external/ansiblemaas.git

AnsibleMaaS is really very simple:

At interpreter start, modules and variables are loaded.
A connection is established to the MaaS API identified in the config, MaaS version and capabilities are tested. This occurs early in the process as MaaS API connectivity can be a challenge to set up initially.
Configuration variables are evaluated:
- Output format
- Inventory sort groupings
- Infrastructure inclusion or exclusion.
MaaS API is called to pull necessary data
Inventory is read by ansible in json format.

Contents of the git repo:

There are two executable python scripts: AnsibleMaaS which pulls inventory from MaaS and ManageTags which will allow you to list, create or delete tags in MaaS.

FILES:

– AnsibleMaaS is an executable python script which pulls inventory from MaaS, sorts it and writes the Ansible inventory file.

– requirements.txt file which has all of the python dependencies in it.

– README.md github readme file.

How To Install

To install, simply clone the git repo on your system and cd into the working directory for the project.

git clone https://github.com/billyFelton/AnsibleMaaS

>cd AnsibleMaaS

You will need to install all of the necessary python package dependencies. Be sure to review the contents of requirements.txt before you do this so you know what you are installing.

>sudo pip3 install -r requirements.txt

Copy the AnsibleMaaS.py file into your directory you use for dynamic inventory in ansible. Usually /etc/ansible/inventory/

Edit the AnsibleMaaS.py file and set it up how you will need it.

>vim AnsibleMaaS.py

You will need to put your MaaS API information in the config file. This is what it should look like when you are done, this is not a real API key. Your actual API key can be found using MaaS documentation. You will need your MaaS username, the API connect KEY and the API URL which is usually somthing like this: https://<FQDN>:5240/MAAS/api/2.0

Example:
api_key = “MAAS API KEY GOES HERE”
maas_url = “http://maas.maas.lan:5240/MAAS/api/2.0”

***NOTE*** Only hosts with a status of “deployed” and a power state of “on” are included in the inventory as these are the only hosts Ansible can interact with.

Good Reference Documentation

Ansible
Canonical MaaS – Be sure to reference your version of MaaS and install method (Snap / apt)
Bare Metal Kubernetes

Features in the works:

Convert to an inventory plugin.
Ability to add ansible attributes to groups and hosts.
Ability to build groups of groups.
Reintroduce tag sort orders.
Reintroduce zone and pool sort orders.
Reintroduce the ability to exclude bare metal.

Off topic reading material: