Usm poller (as satellite) EN
Description
Since version 3.18 of USERSIDE, special usm_poller services are used to interact with telecommunications equipment, running in multiple copies on the operator's servers to increase availability, fault tolerance, and load balancing between servers. Special messages containing network jobs are passed between the USERSIDE kernel and these services, which are distributed to all usm_poller services via the RabbitMQ message broker, and in the same way report job results to the USERSIDE kernel via the service broker. The use of tools more suitable for network communication, as well as the asynchrony introduced into the system, allows the use of previously unavailable communication protocols, speeds up many communication scenarios over existing protocols, and does not make the user wait for the result in an open browser window.
Taking into account peculiarities of usm_poller services launching, it is possible to use these pollers instead of usm_satellite proxy modules, which were used in USERSIDE versions 3.13-3.17 for proxying commands of interaction with equipment between USERSIDE kernel and unavailable equipment via HTTP-API protocol. To do this, a simple setting was added to the usm_poller modules to identify a particular instance of usm_poller as a "satellite".
Since USERSIDE 3.18, the usm_poller module has been used as a "satellite" on remote nodes.
Requirements
To run usm_poller on a remote communication node requires Docker (and optionally Compose plugin v2) or Python 3.9 and supervisor to run without Docker.
Versions
As the system is developed, there may be a situation where a new version of USERSIDE is not compatible with an older version of usm_poller. For example, when changes are made to the API or data structure or whatever.
All versions of USERSIDE from 3.18.0 up to and including 3.18.19 work with usm_poller versions 3.18.0 up to and including 3.18.7.
USERSIDE versions 3.18.20 and newer require a minimum usm_poller version of 3.18.8.
Installing, configuring and running using Docker
Despite the fact that it is also possible to run without Docker, we strongly recommend using a Docker container to run Poller as a satellite. If for some reason you can't use Docker, below are instructions on how to run without Docker.
Installing Docker and Compose plugin
On most supported operating systems, running the command:
curl -fsSL get.docker.com | sh
This command will install the latest versions of Docker Engine and Compose Plugin on most supported operating systems. If this is not possible for your system, the command will provide recommendations for manual installation. You can also refer to the official installation instructions on the Docker project website. Note that the version of DockerIO that came with some Linux distributions is considered deprecated and is not suitable. At the same time, the Docker Desktop version designed for desktop operating systems is supported and you will be able to use it as well, but it is better to prefer Docker Engine (the server version of Docker).
If you need to run commands to Docker without using sudo, then add your user to the docker group with the following commands:
sudo gpasswd -a $USER docker newgrp docker
Now verify that everything has installed correctly by running the commands:
docker version docker compose version
Organisation of Poller-Broker communication
When using usm_satellite, the server was the module itself, while the client was the Userside kernel, which made HTTP-API requests to usm_satellite.
Now the server for all message consumers is the RabbitMQ broker, regardless of whether the application sends or receives a message. This means that the remote server on which usm_poller is installed as a satellite must have network access to the RabbitMQ broker installed on the Userside server (or on a separate server if that was necessary). At the same time, the remote server can now have no open ports at all, which increases its security.
TCP port 5672 (AMQP protocol) is used by default to connect to the RabbitMQ broker. We recommend that you only access this port from the external network from IP addresses you know, which is the IP address of the remote satellite server on which you are running usm_poller.
If you are also using USERSIDE in the Docker bundle, then uncomment the line in the rabbitmq service ports responsible for broadcasting port 5672 (signed in the comments as AMQP) and restart the Docker bundle.
Alternatively, port 5672 (AMQP) can be port 5671 (AMQPS), which is not used by default. You can configure AMQPS (TLS) yourself in addition to port 5672 (AMQP). Instructions on how to configure TLS are available in the official RabbitMQ documentation.
Since messages transmitted between the RabbitMQ broker and its consumers are transmitted in the plain form, the recommended option for organising communication is to use tunnelling with encryption between RabbitMQ nodes and a remote server with usm_poller.
Launching Docker containers usm_poller
The recommended way to start is to use the compose plugin. In this case, the entire configuration of the container is written in a special compose file, which makes it easier to use later.
You can create the compose file anywhere, but to avoid losing its location in the future, we recommend to put it, for example, in /opt/usm_poller. Create a directory:
sudo mkdir -p /opt/usm_poller
Then create a file named compose.yaml in this directory with the following contents (note the indents - they matter):
services:
poller:
image: erpuserside/usm_poller:3.18.8
restart: unless-stopped
environment:
US_AMQP_DSN: amqp://user_name:password@host_address_rabbitmq:5672/%2f
USM_POLLER_SATELLITE_ID: your_satellite_id
deploy:
replicas: 1
To work with USERSIDE 3.18.x, the erpuserside/usm_poller:3.18.x image is used. The usm_poller version must be compatible with the specific version of USERSIDE (see above).
The value for US_AMQP_DSN is the URL string, which you can copy from the .env file located in the root directory with USERSIDE: /var/www/userside, but instead of the RabbitMQ IP address specified there, you need to specify the IP address where the RabbitMQ server is accessible (depends on which access method you use).
The value for USM_POLLER_SATELLITE_ID is the "satellite" identifier, which you can get from the page: Settings - Main - Equipment - Satellite Configuration.
Save the composer.yaml file and run it for a test as follows:
docker compose up
You should see the image being loaded, then the usm_poller Docker container being launched and connecting to the RabbitMQ server via AMQP. If the connection to the RabbitMQ server is correct, stop the container with Ctrl+C, change the number of replicas (replicas value) to 5 and start the containers in the background:
docker compose up -d
If five copies of the container will not be enough, increase this value. To determine if there are enough containers, open the RabbitMQ WEB console using the http protocol, the domain address of your userside and port 15672. For example, like this: http://userside.mycompany.com:15672 and then go to the Queues page. Find a queue there named usm_poller-sat-7 (where 7 is the satellite ID), open it, and watch the queue fill up. The point is to ensure that pollers have time to pick up assignments within your normal time limit. Ideally, the queue should be almost empty at all times (tasks are immediately given to a free poller), or the number of messages should reach zero (the number of pollers is sufficient). You should not increase the number of replicas just for fun - this will lead to unnecessary overhead and may even end up negatively affecting performance if system resources are scarce.
After changing the number of replicas, you need to restart with the command:
docker compose restart
You can check the status of the containers with the command:
docker compose ps
To stop all containers, use the command:
docker compose stop
To upgrade the version of a Docker image within its version (this should preferably be done after each USERSIDE upgrade), you must first stop the containers, then edit the file to specify the new usm_poller version number and then start them in the background again:
docker compose down nano compose.yaml docker compose up -d
To avoid clogging the system with unused images, you can delete them by specifying the entire image with the version:
docker rmi erpuserside/usm_poller:3.18.2
If you think something isn't working properly, look at the container logs with the command:
docker compose logs poller
Or you can add the -f option to follow the logs in real time.
It is possible to start a single container without using compose, by executing just one command (consisting of several lines). However, although this method is simpler, it is not as convenient for later use:
docker run --detach \ --restart unless-stopped \ --name usm_poller \ --env US_AMQP_DSN=amqp://user_name:password@host_address_rabbitmq:5672/%2f \ --env USM_POLLER_SATELLITE_ID=your_satellite_id \ erpuserside/usm_poller:3.18.8
Installation, configuration and running without Docker
The recommended way is to use Poller as a Docker container. In this case, all the necessary dependencies are inside the container and you don't need to take care of anything extra. Also the updates are done with a single command. But, if for some reason you can't use Docker, the only way is to run the poller directly on the host.
Creation of a copy of Poller for Satellite
First you need to copy the poller to the satellite server. To do this, you need to create an archive with the poller files on the USERSIDE server and then copy and unarchive them on the satellite server.
On the USERSIDE server, run:
cd /var/www/userside/microserice tar --exclude="venv" -czf usm_poller.tgz poller/
Copy the usm_poller.tgz file to the satellite server. For example, to a directory /opt and run the following commands:
tar xzf usm_poller.tgz cd poller
Installing dependencies
Make sure the Python version is at least 3.9 (preferably 3.11):
python3 --version
If the version is below 3.9, install version 3.11 (as an alternative version, not replacing the existing one) following these instructions.
All following commands will be shown with the explicit version of Python. If you have a different version than the one shown in the commands - specify your version.
Install all necessary dependencies:
sudo apt install -y supervisor libsnmp-dev sudo -H python3 -m venv venv sudo -H venv/bin/pip install -U pip wheel sudo -H venv/bin/pip install -U -r requirements.txt
Configuration and running
Create a .env file in which put the following variables:
US_AMQP_DSN=amqp://username:password@address_host_rabbitmq:5672/%2f USM_POLLER_SATELLITE_ID=id_your_satellite USM_POLLER_LOGFILE=stdout
Perform a test run:
sudo -H venv/bin/python worker.py
You should see that the Poller has connected to the broker. Make sure that there are no errors. The running Poller is constantly running and waiting for messages from the broker. On the USERSIDE, do some polling of the equipment that is in the service area of this satellite and make sure that everything worked well. For example, you can open some switch and make sure that the information from the switch was taken by the poller and in the USERSIDE user interface you can see the result of this work.
Stop the poller running manually with the key combination Ctrl+C. You will have to wait about 5 seconds until the Poller completes all its tasks and closes the connection to the broker correctly.
From the .env file, delete the line with the USM_POLLER_LOGFILE variable. By default, logs are written to the file /var/log/usm_poller/poller.log. If you are not satisfied with this path, instead of deleting the variable, change its value to the file you need.
Copy the example supervisor settings and the example log rotation settings from the etc. subdirectory:
sudo cp etc/usm_poller.conf-example /etc/supervisor/conf.d/usm_poller.conf sudo cp etc/logrotate-example /etc/logrotate.d/usm_poller
Edit the /etc/supervisor/conf.d/usm_poller.conf file. In it, change the path to the poller: instead of /var/www/userside/microservice/poller, specify /opt/poller or wherever you copied the poller to. Also specify the number of copies to run in the numprocs parameter. Depending on the amount of hardware and the number of tasks served by the satellite. Start with 2 and change this value as needed in the future. It makes no sense to run 15 copies at once. Besides, if your satellite server is not powerful enough, it may cause performance degradation.
Edit the /etc/logrotate.d/usm_poller file if you changed the path to the logs in the .env file environment variable USM_POLLER_LOGFILE.
Restart the supervisor to have it reread the configuration:
sudo systemctl restart supervisor
And after a few seconds, observe the processes it is controlling - all copies of the poller should be in the RUNNING state.
sudo supervisorctl status
The configuration is now complete.
Upgrade
To upgrade, you will have to make a Poller archive again from the USERSDIE server and move it to the satellite server, where you unarchive and install (upgrade) the dependencies again.
To do this, create an archive on the USERSIDE server:
cd /var/www/userside/microserice tar --exclude="venv" -czf usm_poller.tgz poller/
Copy it to the satellite server in the /opt directory and unarchive it:
tar xzf usm_poller.tgz cd poller
Upgrade dependencies:
sudo -H venv/bin/pip install -U pip wheel sudo -H venv/bin/pip install -U -r requirements.txt
Restart the supervisor:
sudo supervisorctl restart usm_poller:* sudo supervisorctl status