Itential Docs & Guides | Drive Automation for Modern Networks

Overview

This guide provides step-by-step information to install and configure Itential Automation Gateway.

Notes:

Itential Automation Gateway (IAG) is compatible with several third-party software products for network operating environments. Beginning with the system requirements for 2019.1.2, all third-party software version compatibility is documented in Release Notes. To get the most up-to-date requirements for any third-party software, including open source, first identify which IAG release you’re using and then refer to the respective release notes.
Itential Automation Gateway resides on the same server as your Ansible implementation. If you have multiple Ansible implementations, you must install Automation Gateway for each Ansible implementation. Optionally, Itential adapters will connect into each Automation Gateway installation to manage all of your Ansible networks from a single point within the Itential Automation Platform (IAP).
Access to Itential's pip registry is required to download and install the Automation Gateway software. To obtain your credentials, contact the Itential Service Desk.

General Requirements

Note: Please refer to the release notes for a more exhaustive list of requirements.

RHEL/CentOS 7
Python 3.6
pip <= 20.2.4
Ansible 2.8 - 2.10

Automation Gateway is a pip-installed Python package which is supported on a RHEL/CentOS base due to compatibility, security, and scaling considerations regarding the tools that it integrates with (Ansible, Nornir, Terraform, etc). In general, you can use any method to install Python or Ansible as long as the versions match the requirements for your particular Automation Gateway package.

Generally speaking, the steps to install Automation Gateway on a fresh server are:

Ensure the system has the correct, updated repositories and packages.
Ensure the system has the correct version of Python installed and available to the non-root user (e.g., itential).
Install the Automation Gateway with pip via the Itential Pip Registry and your credentials.
Ensure you have installed all applicable integrations that you wish to use such as Ansible, Terraform, etc.
Supply the correct properties to Automation Gateway which describe enabled integrations, content locations, etc.
Ensure your server has the correct ports open and security settings (selinux, ssl, etc) in place for web traffic.
Startup Automation Gateway.

IAG Server

The following table describes required IAG server components for production and development environments along with the recommended specs for usage.

Component	Requirement	Production ENV	Development ENV
CPU	64-bit x86 CPU cores	8 Cores	4 Cores
RAM	Memory	32GB	16GB
Disk	Additional storage needs are dependent on custom scripts and content required by the installer.	Min: 20GB Rec: 40GB	Min: 10GB Rec: 20GB

In situations where a number of custom scripts, content, and logging is heavily utilized, then CPU, RAM and disk space allocations may need to be increased to better suit those requirements.

Quick Install (RHEL 7)

The RHEL 7 installation guide exactly matches the CentOS 7 installation guide with the following important exceptions:

Each RHEL instance must be fully registered before many features (e.g., yum) can be utilized.
- For registration information see RedHat Registration Information.
- To enable automatic registration via CLI use: subscription-manager register --auto-attach.
- If issues arise with automatic registration you can try to use the register, auto-attach and attach functions of subscription-manager.
- Additional RHEL registration information or troubleshooting is beyond the scope of this installation guide.
The epel-release command should not be run as it will fail: yum -y install epel-release.

Quick Install (CentOS 7)

Notes:

This quick install script is tested against a fresh installation of CentOS 7 Minimal. Users must fill-in the below variables, review the script, and then run it as root.
Do not forget to edit the first set of exported variables before running!

The following method is great for quickly booting Automation Gateway for testing purposes. Due to its use of the python virtualenv module, it is very easy to retry a fresh installation if you make any mistakes. However, it should be understood that you can run Automation Gateway via global, user, or venv installations as you see fit for your environment. Therefore, we recommend the quick install for testing and the Full Installation for a production environment.

With regard to integrations and versioning, the script assumes you do not have an existing Ansible, Nornir, Terraform installation and no preference on the versioning of them.

echo "## CUSTOMIZE THESE VARIABLES BEFORE RUNNING THIS SCRIPT: ##" && \
export AUTOMATION_GATEWAY_VERSION="" && \
export REGISTRY_USERNAME="" && \
export REGISTRY_PASSWORD="" && \
export REGISTRY_URL="registry.aws.itential.com/repository/automation-gateway-release/simple" && \

echo "## ENABLE REPOSITORIES ##" && \
yum -y install epel-release && \
yum -y install \
https://repo.ius.io/ius-release-el7.rpm \
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

echo "## UPDATE SYSTEM PACKAGES ##" && \
yum -y update

echo "## INSTALL SYSTEM DEPENDENCIES ##" && \
yum -y install gcc-c++ make yaml-cpp yaml-cpp-devel libyaml libyaml-devel libselinux-python

echo "## INSTALL PYTHON ##" && \
yum -y install python3-devel python3-pip

echo "## EXPOSE PORT(S) ##" && \
firewall-cmd --zone=public --add-port=8083/tcp --permanent && \
firewall-cmd --reload

echo "## CREATE NON ROOT USER ##" && \
useradd itential && \
usermod -aG wheel itential && \
echo "itential ALL = NOPASSWD: /home/itential/automation-gateway/bin/automation-gateway-setup.sh" >> /etc/sudoers && \

echo "## CREATE AND ASSIGN DIRECTORIES ##" && \
mkdir -p /opt/automation-gateway/data \
         /usr/share/ansible/inventory \
         /usr/share/ansible/modules \
         /usr/share/ansible/roles \
         /usr/share/ansible/collections \
         /usr/share/ansible/playbooks \
         /usr/share/nornir/modules \
         /usr/share/scripts \
         /usr/share/terraform && \
chown -R itential:itential \
         /opt/automation-gateway/data \
         /usr/share/ansible \
         /usr/share/nornir \
         /usr/share/scripts \
         /usr/share/terraform

echo "## SWITCH USER ##" && \
su itential
cd

echo "## CREATE PYTHON VIRTUAL ENVIRONMENT ##" && \
python3.6 -m venv $HOME/automation-gateway && \
source $HOME/automation-gateway/bin/activate

echo "## INSTALL PYTHON DEPENDENCIES ##" && \
python3.6 -m pip install --upgrade pip==20.2.4 setuptools==58.0.1 wheel

echo "## INSTALL AUTOMATION GATEWAY AND INTEGRATIONS (Ansible/Nornir/Netmiko/Terraform/etc) ##" && \
python3.6 -m pip install \
  automation-gateway[integrations]==$AUTOMATION_GATEWAY_VERSION \
  --extra-index-url https://$REGISTRY_USERNAME:$REGISTRY_PASSWORD@$REGISTRY_URL

echo "## INSTALLATION COMPLETE ##"

At this point, your system, python, and Automation Gateway installations are all complete. The final decision will be how you would like to start the server.

Quick Start (On Demand)

The Quick Start is most appropriate for temporary testing of properties.yml settings, firewalls, etc. As the server will stop as soon as the user inputs CTRL-C or the terminal session is closed, this method is not suitable for anything other than testing.

echo "## START AUTOMATION GATEWAY ##" && \
echo "## REMEMBER TO SOURCE $HOME/automation-gateway/bin/activate ON SUBSEQUENT RUNS ##" && \
source $HOME/automation-gateway/bin/activate && \
automation-gateway

Service Start (Systemd Script)

Installing Automation Gateway as a service allows the server to persist beyond a single terminal session and even through system failures or reboots. Systemd Services require elevated permissions. Either run the setup script as root with the absolute path to automation-gateway-setup.sh or via the sudo command with the $HOME shortcut as in the example below. The sudo method assumes the /etc/sudoers modification from the quick install above. The modification allows the itential user to run the setup via sudo without a password.

The Systemd Service Start is most appropriate for production installations, but can also be used to test all aspects of the server as long as you have access to rebooting the process via systemctl:

systemctl restart automation-gateway

To begin the interactive installation process, simply use:

echo "## START AUTOMATION GATEWAY ##" && \
sudo $HOME/automation-gateway/bin/automation-gateway-setup.sh --setup

To run the non-interactive installation process, simply copy/paste the following after changing any default parameters to your liking:

echo "## START AUTOMATION GATEWAY ##" && \
sudo $HOME/automation-gateway/bin/automation-gateway-setup.sh \
  --setup \
  --user=itential \
  --custom-content-dir=/usr/share \
  --install-dir=/opt/automation-gateway \
  --systemd \
  --start-on-boot

Full Installation

The following step-by-step guide is meant to explain each phase so that the user can deviate from the Quick Install guide above where appropriate for their environment. It assumes you have root access to the system, or that you will utilize sudo on all commands up to Continue As Non Root User.

Enable Repositories

Automation Gateway requires the following yum repositories.

epel-release
ius-release

yum -y install epel-release
yum -y install \
https://repo.ius.io/ius-release-el7.rpm \
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Update System Packages

To ensure the latest packages are installed, perform a yum update.

yum -y update

Install System Dependencies

Note: Please refer to the release notes for a more exhaustive list of requirements.

As Automation Gateway is a Python application, it has multiple system dependencies that need to be installed.

yum -y install gcc-c++ make yaml-cpp yaml-cpp-devel libyaml libyaml-devel libselinux-python

HTTPS Requirements

Itential Automation Gateway uses haproxy to support HTTPS security requirements. This is an optional requirement that is used alongside the HTTPS Install instructions.

yum -y install haproxy

Install Python

Automation Gateway is intentionally distributed for one particular Python installation. There are many methods to install Python and you are free to use whatever works best for your environment.

Example of installing Python via yum:

yum -y install python3-devel python3-pip

In this example, Python would be installed with the following significant locations:

$ which python3.6
/usr/bin/python3.6

$ python3.6 -m pip -V
pip 20.2.4 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)

Create Non Root User

While your own user setup will be very specific to your environment and business rules, at the least we recommend creating a non root user, such as itential, to run the application. Since the Automation Gateway enables a wide variety of system administrator oriented tools, you should never run the process as root since this gives web users the ability to potentially perform system administration tasks as root on the webserver itself, instead of the intended targets (remote devices) with the intended authorization (provided credentials).

In this example, only the default user configuration is used. However, considerations such as password authentication, selinux settings, etc. should also be taken into account.

useradd itential

Create and Assign Directories

As a webserver and a gateway to other tools, Automation Gateway requires multiple directories to function properly.

Main Package Locations

As a Python application, Automation Gateway will be installed within the site-packages directory of your python installation.

Files within this directory structure should not be modified or manipulated.
Any updates to or API interactions with Automation Gateway could overwrite the files within this directory.
To supply additional content (e.g., third party or custom modules and scripts) please see Integration Content Locations below.

Examples

System installed Python, Ansible installed globally:

/usr/local/lib/python3.6/site-packages/automation_gateway

Manually installed Python, Ansible installed by the itential user:

/home/itential/python3.6/site-packages/automation_gateway

Any installation method for Python, Ansible installed inside of a virtualenv at /home/itential/automation-gateway:

/home/itential/automation-gateway/lib/python3.6/site-packages/automation_gateway

Integration Content Locations

In general, there are up to three main file locations to understand for each installed integration.

The "out of the box" content path. For Ansible, an example would be where the Ansible modules are installed by default.
The "dynamically created" content path. For Ansible, an example would be where ansible-galaxy will install collections when told to via API calls to Automation Gateway.
The "user supplied" content path. This is the path where you intend to supply content such as third party or custom Ansible/Nornir/Terraform modules, etc.

Careful consideration of these path types will make configuring a robust properties.yml file much easier.

Database Directory

In this example, we configure only the required directory which will hold the database files.

Note: This database creation step is not required if using the automation-gateway-setup.sh script

This path is required to:

Match what is specified in properties.yml or the corresponding environment variable.
Be accessible by the non-root user which is running the Automation Gateway process.

# Example assumes data_file=/opt/automation-gateway/data/automation-gateway.db
mkdir -p /opt/automation-gateway/data
chown itential:itential /opt/automation-gateway/data

Custom Content Example

In this example, we configure the directories which will hold the custom content files. This could be any supported content such as an Ansible collection or a Nornir configuration file.

Note: The custom content steps are not required if using the automation-gateway-setup.sh script.

These paths are required to:

Match what is specified in properties.yml or the corresponding environment variables.
Be accessible by the non-root user which is running the Automation Gateway process.

# Example assumes each integration's properties.yml paths reflect the below paths
mkdir -p /usr/share/ansible/inventory \
         /usr/share/ansible/modules \
         /usr/share/ansible/roles \
         /usr/share/ansible/collections \
         /usr/share/ansible/playbooks \
         /usr/share/nornir/modules \
         /usr/share/scripts \
         /usr/share/terraform

chown -R itential:itential /usr/share/ansible \
                           /usr/share/nornir \
                           /usr/share/scripts \
                           /usr/share/terraform

Expose Ports

While mostly out of scope for this documentation, it is highly likely that you will need to configure your firewalls to allow TCP traffic on the desired socket (see the properties.yml section of the Automation Gateway Properties guide.

In this example, we expose the default port via the built-in CentOS 7 firewall daemon.

firewall-cmd --zone=public --add-port=8083/tcp --permanent
firewall-cmd --reload

Continue As Non Root User

The remaining installation steps should be performed as the non root user and from the non root user home directory.

In the examples below, the itential user is used.

echo "## SWITCH USER ##" && \
su itential
cd

Create Python Virtual Environment

Installing Automation Gateway via the python virtualenv module makes it very easy to retry a fresh installation if you make any mistakes. However, it should be understood that you can run Automation Gateway via global, user, or venv installations as you see fit for your environment.

In this example, we use the default venv module from Python but you are free to use your own tooling as required.

python3.6 -m venv $HOME/automation-gateway
source $HOME/automation-gateway/bin/activate

Note: Due to the intended use of virtualenv, there will be no additional reminders after this. However, do not forget to source first in any new terminal sessions! For example: source $HOME/automation-gateway/bin/activate.

Install Python Dependencies

As a general practice, you should update a few main python packages prior to performing pip installs.

Install	Description
pip	Helps resolve packages and wheels in the most up to date fashion.
setuptools	Used to ensure additional package installations (eg. automation-gateway[integrations]) goes smoothly.
wheel	Ensures that as many precompiled packages are used as possible instead of defaulting to legacy installers.

python3.6 -m pip install --upgrade pip setuptools wheel

Note:

As of pip>20.2.4, package installation strictly enforces PEP440 compliance, therefore non-compliant package versions are unable to be installed. Install pip<=20.2.4 if you run into issues resolving package versions for the automation-gateway package.

As of setuptools>58.0.1, it fails to build packages using use_2to3. Use setuptools==58.0.1 if you run into issues when installing certain ansible dependencies.

python3.6 -m pip install --upgrade pip==20.2.4 setuptools==58.0.1 wheel

Install Automation Gateway

Beginning with automation-gateway>2020.2.18, the Automation Gateway package comes with a list of recommended integrations and their related versions. If specific integrations and versions are required, or have already been installed, use the Automation Gateway Only steps.

For either path, you will need to know which Automation Gateway version you will be using, as well as the repository credentials (contact Itential Product Support for credentials).

export AUTOMATION_GATEWAY_VERSION=""
export REGISTRY_USERNAME=""
export REGISTRY_PASSWORD=""
export REGISTRY_URL="registry.aws.itential.com/repository/automation-gateway-release/simple"

Automation Gateway Only

This command will install only Automation Gateway and require manual installation of integrations such as Ansible, Terraform, etc.

python3.6 -m pip install \
  automation-gateway==$AUTOMATION_GATEWAY_VERSION \
  --extra-index-url https://$REGISTRY_USERNAME:$REGISTRY_PASSWORD@$REGISTRY_URL

Install with Integrations

This command will install Automation Gateway and all current recommended versions of supported integrations like Ansible, Terraform, etc.

python3.6 -m pip install \
  automation-gateway[integrations]==$AUTOMATION_GATEWAY_VERSION \
  --extra-index-url https://$REGISTRY_USERNAME:$REGISTRY_PASSWORD@$REGISTRY_URL

Install Separate Integrations

Note: This step is not required if using the Install with Integrations step above.

Most Automation Gateway integrations can be installed in a variety of ways. Where possible, we recommend performing pip installations in the same environment as your Automation Gateway package to ensure they are properly discovered. This is not applicable for Terraform as it is not a pip package.

Example: Integrations via pip

python3.6 -m pip install ansible
python3.6 -m pip install nornir
python3.6 -m pip install netmiko

Example: Terraform Installation

mkdir -p $HOME/terraform
cd $HOME/terraform
curl -sL https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip > terraform_0.12.28_linux_amd64.zip
unzip terraform_0.12.28_linux_amd64.zip
cp terraform /usr/local/bin

Please refer to the official guides for your particular integration and version combination.

Ansible Installation Guide

Start Automation Gateway

Automation Gateway can be started on demand via the automation-gateway binary, or installed and enabled as a service via a systemd service script.

Quick Start (On Demand)

Note: Do not forget to source if using a virtual environment! For example: source $HOME/automation-gateway/bin/activate.

automation-gateway

Service Start (Systemd Script)

Automation Gateway provides a setup script in the same location as the automation-gateway binary. In some situations, the script is in the user's path and can be found automatically via the which command.

which automation-gateway-setup.sh

However, if you are not logged in as the correct user, or if you have not sourced into your virtualenv, then which will not work or it will point to the wrong binaries. You can find all binaries on your system with the find command instead and choose the one relevant to your installation.

find / -name "automation-gateway-setup.sh" 2>/dev/null

Once the correct binary path is determined, you can run the script either as root or with sudo.

sudo /path/to/automation-gateway-setup.sh --help

The setup script takes the following arguments, which can be found via the --help command above.

Argument	Description
`--setup`	This signifies to setup Automation Gateway. Mutually exclusive with clean.
`--clean`	This signifies to clean Automation Gateway. Mutually exclusive with setup.
`--install-dir=*`	This optional argument is used to create the install location of Automation Gateway. Replace the '`*`' with the desired install directory. The path can be absolute or relative. Used with setup and clean.
`--custom-content-dir=*`	This optional argument is used to create the locations for custom content (modules, etc.). Replace the '`*`' with the desired role directory. The path can be absolute or relative. Used with setup.
`--user=*`	This optional argument is used to create the owner of the Automation Gateway process. Replace the '`*`' with the custom user. Used with setup.
`--systemd`	This optional argument, if passed in, signifies for the Automation Gateway process to be run as a systemd service. Used with setup.
`--start-on-boot`	This optional argument, if passed in, signifies that Automation Gateway will automatically start on system boot up. Systemd argument also must be passed in for this argument to work. Used with setup.
`--force`	This argument signifies to override confirmation statements during the `--clean` argument. Used with clean.

If any arguments are not passed in, the user will be prompted to input them or use defaults.

An example call for setup (explicitly uses defaults):

echo "## START AUTOMATION GATEWAY ##" && \
/path/to/automation-gateway-setup.sh \
  --setup \
  --user=itential \
  --custom-content-dir=/usr/share \
  --install-dir=/opt/automation-gateway \
  --systemd \
  --start-on-boot

After setup, follow the post installation instructions for next steps (e.g., modify the properties file that is generated with any additional changes needed for your environment).

Custom Service Environments

If your network requires using a proxy for http or https access, you may set the http_proxy and https_proxy settings via the systemd unit file. This method will properly set the environment variables for IAG.

Below is an example of incorporating environment variables for an IAG SystemD proxy modification.

...
[Service]
...
Environment=HTTP_PROXY=http://192.168.11.50:3128

Environment=HTTPS_PROXY=http://192.168.11.50:3128

Remove Service

In order to undo the setup script (remove service, directories, user, etc.), you can use the --clean argument.

An example call for clean:

/path/to/automation-gateway-setup.sh \
  --force \
  --clean \
  --install-dir=/opt/automation-gateway

Validate the Service Installation

Utilize the systemctl status and restart commands as required to validate and reload the service.

Helpful Commands

Check the status of the service

systemctl status automation-gateway

Example output:

systemctl status automation-gateway
● automation-gateway.service - Itential Automation Gateway
   Loaded: loaded (/etc/systemd/system/automation-gateway.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-05-20 18:25:55 EDT; 1s ago
 Main PID: 5343 (automation-gate)
   CGroup: /system.slice/automation-gateway.service
           └─5343 /home/itential/automation-gateway/bin/python3.6 /home/itential/automation-gateway/bin/automation-gateway --properties-file=/opt/automation-gateway/conf/properties.yml

May 20 18:25:55 localhost.localdomain systemd[1]: Stopped Itential Automation Gateway.
May 20 18:25:55 localhost.localdomain systemd[1]: Started Itential Automation Gateway.
May 20 18:25:56 localhost.localdomain automation-gateway[5343]: {"time": "2021-05-20 18:25:56,192", "severity": "INFO", "src": "automation_gateway.automation_gateway", "msg": "Automation Gateway starting."}
...

Check if the service has been "enabled" (will continue after a system reboot)

systemctl is-enabled automation-gateway

Reload service and restart Automation Gateway after service changes

systemctl daemon-reload
systemctl restart automation-gateway

Restart Automation Gateway after properties.yml changes

systemctl restart automation-gateway

Restart Automation Gateway and track the server logs via journalctl

systemctl restart automation-gateway && journalctl -f -u
systemctl restart automation-gateway && journalctl -f -u automation-gateway

Disable Automation Gateway from persisting through a reboot

systemctl disable automation-gateway

Stop Automation Gateway

systemctl stop automation-gateway

Next Steps

Once you have successfully started Automation Gateway, you will want to ensure your properties.yml and integrations are working as expected. See the Automation Gateway Properties guide for additional configuration details. See the Itential Automation Gateway User Guide for additional details on testing and utilizing Automation Gateway.

Itential Automation Gateway

Help us improve by sending feedback...

On this page:

Installation

Overview

General Requirements

IAG Server

Quick Install (RHEL 7)

Quick Install (CentOS 7)

Quick Start (On Demand)

Service Start (Systemd Script)

Full Installation

Enable Repositories

Update System Packages

Install System Dependencies

HTTPS Requirements

Install Python

Create Non Root User

Create and Assign Directories

Main Package Locations

Integration Content Locations

Database Directory

Custom Content Example

Expose Ports

Continue As Non Root User

Create Python Virtual Environment

Install Python Dependencies

Install Automation Gateway

Automation Gateway Only

Install with Integrations

Install Separate Integrations

Start Automation Gateway

Quick Start (On Demand)

Service Start (Systemd Script)

Custom Service Environments

Remove Service

Validate the Service Installation

Helpful Commands

Next Steps