Installation: различия между версиями

Текущая версия от 13:59, 4 июня 2024

ATTENTION: These instructions are valid for ERP USERSIDE versions 3.19

Description

For the sake of simplicity, the commands for the Linux Debian 12 (bookworm) operating system will be considered. For experienced system administrators, it should not be difficult to use similar commands for another operating system. If you do not feel confident in administering operating systems, we recommend you to use Debian or Ubuntu Server LTS. Because of their simplicity and because our technical support is much more familiar with Debian-like distributions. The likelihood of being able to tell you something not related to ERP USERSIDE, but related to operating system administration, will be much higher if you have Debian or a similar distribution based on it.

On the other hand, if you are an experienced system administrator and have skills and experience with Docker, it may be more convenient for you to use a ready-made environment based on Docker images that we have prepared specially, including everything you need to run ERP USERSIDE. In this case, you will not have to configure anything, except to perform simple Docker-bundle settings https://github.com/userside/userside-docker.

Requirements

ERP USERSIDE requires various system applications and services such as PHP interpreter with a set of extensions, WEB server, database management system and others. This section will list such requirements depending on the version of ERP USERSIDE.

Also note that your operating system must be configured with the correct time zone and locale. The correct display of information and correct sorting depends on this.

The list of required PHP extensions contains all extensions that are not part of the PHP core. Some of these extensions may come with the core PHP package or as part of the php-common package, while others must be installed additionally.

3.19 (current stable)

PHP: 8.3
PHP extensions: ctype, gd, json, libxml, mbstring, openssl, pdo, pdo_pgsql, posix, simplexml, snmp, sockets, zlib, pcntl
Additional PHP extensions: ldap, soap
Python: 3.9+ (preferably 3.11)
Python modules: pip, venv
PostgreSQL: 12+ (preferably 16)
Redis: 5+ (preferably 7)
RabbitMQ: 3.10+ (preferably 3.13)
Supervisor

Main agreements

The directory where ERP USERSIDE is installed is /var/www/userside. Inside this directory there should be a subdirectory public, which is the root directory for the WEB server. If you want to use a different directory for ERP USERSIDE, remember to make the appropriate corrections to all the examples in these instructions.

Create the directory:

sudo mkdir -p /var/www/userside/public
sudo chown -R www-data:www-data /var/www/userside

You will need to allocate a domain name for ERP USERSIDE. The examples below use userside.mycompany.com, but you will need to change it to your own. Write this domain name in /etc/hosts if it is different from the hostname of the installation.

Installation of the required components

First install the utilities that will be required later in the installation process. They are required for any ERP USERSIDE program. You can copy the lines one at a time and paste them into the command line of your operating system.

Copy and paste the following lines in their entirety:

sudo tee /etc/apt/sources.list.d/contrib-non-free.list << EOL
deb http://deb.debian.org/debian/ $(lsb_release -sc) contrib non-free non-free-firmware
EOL

sudo apt update
sudo apt full-upgrade -y
sudo apt install -y gnupg ca-certificates lsb-release debian-archive-keyring debian-keyring libsnmp-dev snmp-mibs-downloader

PostgreSQL

Alternative official repository

We strongly recommend using the latest version of PostgreSQL, as this always has a positive impact on speed. Add the official Postgres repository to the system:

sudo install -d /usr/share/postgresql-common/pgdg

sudo curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc

sudo sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

sudo apt update

If you have a different operating system, you can find the necessary commands for it at official PostgreSQL website.

Installation

Attention! Before installing PostgreSQL, your operating system must have your local locale and correct time zone set! This can be done later, but it will be much easier if it is done before installing PostgreSQL. Run the following command to make sure the correct locale is present:

locale -a

If your local locale is not among those listed, then install it. To do this, simply edit the /etc/locale.gen file by removing the comments before the line(s) with the desired locale, and then run the command:

sudo locale-gen

If you need to change the default locale, run:

sudo dpkg-reconfigure locales

To change the time zone, execute:

sudo timedatectl set-timezone Europe/Kyiv

Now you can move on to the PostgreSQL installation:

sudo apt install -y postgresql-16 postgresql-16-postgis-3

Setting

Create a user and database for ERP USERSIDE. The example below creates a user named userside and a database with the same name. If your locale is not uk_UA, remember to change the command. The PostGis extension needed to work with geographic data is then connected to the database. After doing the first line, you will need to enter the password for the new user twice - make a note of this password, you will need it later to set up ERP USERSIDE

sudo -u postgres createuser userside -P
sudo -u postgres createdb -e -E "UTF-8" -l "uk_UA.UTF-8" -O userside -T template0 userside
sudo -u postgres psql -d userside -c "CREATE EXTENSION postgis"

If your PostgreSQL is installed on another server, you will need to make corrections to the postgresql.conf and pg_hba.conf files located in the /etc/postgresql/16/main/ directory. In the postgresql.conf file, you must uncomment and change the value of the parameter listen_addresses, specifying there all IP addresses of the server interfaces on which it can accept connections to PostgreSQL. In the pg_hba.conf file you need to add a line at the very end, similar to the previous ones, indicating from which address the ERP USERSIDE user can connect. But if your PostgreSQL is located on the same server as USERSIDE - there is no need to make any adjustments to these files.

Redis

The Debian 12 standard repository includes Redis version 7.0.15. This version is suitable for running ERP USERSIDE.

Installation

sudo apt install -y redis-server

Setting

By default Redis accepts connections without a password, but we strongly recommend setting a password. Since the password is transmitted in the clear (Redis does not provide encryption because it focuses on speed of request processing and extra steps like encryption are not applied), you need to create a really long and complex password, for example, using the openssl utility as follows:

openssl rand --hex 32

The output will be a random set of 32 bytes in 16-character form, which you will use as a password. Next, you will need to substitute this generated string in place of the word MYPASSWORDHERE. Write it down. Then you will specify this string as the Redis password in the settings.

Specify the generated passphrase hash in the Redis configuration file:

sudo sed -i 's@^.*requirepass .*@requirepass MYPASSWORDHERE@g' /etc/redis/redis.conf

where instead of MYPASSWORDHERE specify the hash of the passphrase received earlier.

You should also disable timeout (on some versions of Redis it is enabled):

sudo sed -i 's@^timeout .*@timeout 0@' /etc/redis/redis.conf

Restart Redis and make sure it works (should get PONG in response):

sudo systemctl restart redis
redis-cli -h 127.0.0.1 -p 6379 -a MYPASSWORDHERE ping

RabbitMQ

The Debian 12 standard repository includes an older version of RabbitMQ 3.10. You can use it - it is supported. However, this version is relatively old and your best bet is to install a newer version from alternative repositories. The official RabbitMQ website has fairly detailed information on installing RabbitMQ for each operating system.

Adding alternative repositories

Copy and paste the following block in its entirety into the command line

curl -1sLf https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/gpg.E495BB49CC4BBE5B.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq-erlang-archive-keyring.gpg > /dev/null
curl -1sLf https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/gpg.9F4587F226208342.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq-server-archive-keyring.gpg > /dev/null

sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
deb [signed-by=/usr/share/keyrings/rabbitmq-erlang-archive-keyring.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/deb/debian $(lsb_release -cs) main
deb-src [signed-by=/usr/share/keyrings/rabbitmq-erlang-archive-keyring.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/deb/debian $(lsb_release -cs) main
deb [signed-by=/usr/share/keyrings/rabbitmq-server-archive-keyring.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/deb/debian $(lsb_release -cs) main
deb-src [signed-by=/usr/share/keyrings/rabbitmq-server-archive-keyring.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/deb/debian $(lsb_release -cs) main
EOF

sudo apt update

Installation

Make sure that /etc/hosts has an entry for your hostname. The lack of such an entry (for example, if you changed the hostname after installation) is a major error that occurs when installing RabbitMQ. So it is better to make sure again that there is an entry for the hostname.

sudo apt install -y erlang-base \
                    erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                    erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                    erlang-runtime-tools erlang-snmp erlang-ssl \
                    erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

sudo apt install rabbitmq-server -y --fix-missing

Installation of necessary add-ons

Do the next two lines one at a time!

sudo rabbitmq-plugins enable rabbitmq_management --offline

sudo rabbitmq-plugins enable rabbitmq_web_stomp --offline

Setting

Copy and paste the following block in its entirety:

sudo tee /etc/rabbitmq/rabbitmq.conf << EOF
listeners.tcp.default = 5672

web_stomp.port = 15674
web_stomp.cowboy_opts.max_keepalive = 60
EOF

Now you need to create users. We recommend using three different users: for server administration (full rights), for userside and modules (full rights but no administrative rights) and the web-stomp user for websocket (minimal rights to read certain broker objects). But you can only create two: an admin user, which will also be used for userside, and a user for websocket. These must necessarily be at least two different users, since the password for the websocket user is passed to the browser and can be easily read by the user. The following will show an example for the recommended three users.

Create a user for administration RabbitMQ. This example uses the username admin and password admin_password. A tag administrator is assigned to the user immediately after creation, giving the user maximum administrator rights and then set permissions for vhost / allowing full access to configure, write and read everything within that vhost*.

sudo rabbitmqctl add_user "admin" "admin_password"
sudo rabbitmqctl set_user_tags "admin" "administrator"
sudo rabbitmqctl set_permissions -p "/" "admin" ".*" ".*" ".*"

Create a user on behalf of which ERP USERSIDE and modules will run. Example for the user name userside (you do not need to give admin rights to this user):

sudo rabbitmqctl add_user "userside" "system_password"
sudo rabbitmqctl set_user_tags "userside" "monitoring"
sudo rabbitmqctl set_permissions -p "/" "userside" ".*" ".*" ".*"

Create a WebSTOMP user . You will need it to use notifications via WebSocket. Instead of websock-user you can specify a different user name. Instead of password_websocket specify your password, but this password will be passed in the clear to the browser, so don't make it similar to other passwords:

sudo rabbitmqctl add_user "websock-user" "password_websocket"
sudo rabbitmqctl set_permissions -p "/" "websock-user" "^erp-stomp:id-.*" "" "^erp-stomp:id-.*"

Attention!: You will need to specify the WebSTOMP username and password in the settings in the USERSIDE interface (Menu: Settings - Main - Websocket).

* vhost - is a virtual host within RabbitMQ that allows different applications to differentiate between different uses of the same server. Like, for example, different databases on the same DBMS server. The default vhost is named / and almost always this is enough for you. But if you plan, for example, to run multiple copies of ERP USERSIDE on the same server, then for each copy you will need to create a different vhost and, accordingly, users for it. Please refer to official manual for more details.‘’

Restart the rabbitmq service:

sudo systemctl restart rabbitmq-server

Management

One of the commands above installs a management module in RabbitMQ that provides a convenient WEB interface to monitor, diagnose, and manage the RabbitMQ server. With this module you can monitor the server, keep track of the number of messages in queues and other statuses. All this can be done using the console utility as well rabbitmqctl, but using the WEB-interface can be much clearer and more convenient.

By default, the WEB management interface is available at http://userside.mycompany.com:15672. We recommend using a firewall to restrict access to this control interface.

More details about the control module can be read on the official RabbitMQ website.

PHP

Alternative repository

sudo curl -sSLo /usr/share/keyrings/deb.sury.org-php.gpg https://packages.sury.org/php/apt.gpg
sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/deb.sury.org-php.gpg] https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list'
sudo apt update

Installation

Execute the following commands:

sudo apt install -y php8.3-{fpm,cli,common,curl,intl,mbstring,opcache,pgsql,readline,xml,zip,snmp,gd}

If you plan to use LDAP, you need to install the php-ldap extension:

sudo apt install -y php8.3-ldap

If you plan to use TurboSMS to send SMS, you will also need the php-soap extension:

sudo apt install -y php8.3-soap

Setting

The following are commands that make changes to the configuration files.

sudo sed -i "s@^;date.timezone.*@date.timezone = $(cat /etc/timezone)@" /etc/php/8.3/fpm/php.ini
sudo sed -i "s@^;date.timezone.*@date.timezone = $(cat /etc/timezone)@" /etc/php/8.3/cli/php.ini
sudo sed -i "s@;cgi.fix_pathinfo=1@cgi.fix_pathinfo=0@" /etc/php/8.3/fpm/php.ini
sudo sed -i "s@post_max_size = 8M@post_max_size = 100M@" /etc/php/8.3/fpm/php.ini
sudo sed -i "s@upload_max_filesize = 2M@upload_max_filesize = 100M@" /etc/php/8.3/fpm/php.ini
sudo sed -i "s@max_execution_time.*@max_execution_time = 300@" /etc/php/8.3/fpm/php.ini
sudo sed -i "s@^;request_terminate_timeout =.*@request_terminate_timeout = 300@" /etc/php/8.3/fpm/pool.d/www.conf

sudo systemctl restart php8.3-fpm

NGINX

Alternative official repository

The version of NGINX supplied in the Debian repository is fully usable and there is no need to add an alternative repository. However, if you want to always have the latest version, run the following lines to add the official nginx repository to your operating system:

curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \
    | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \
http://nginx.org/packages/debian `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list
echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \
    | sudo tee /etc/apt/preferences.d/99nginx
sudo apt update

If you have a different operating system, you can find the necessary commands for it on the official NGINX website: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-open-source/

Installation

sudo apt install -y nginx

Setting

sudo sed -i "s@^user.*;@user www-data www-data;@" "/etc/nginx/nginx.conf"
sudo systemctl restart nginx

The following configuration example assumes that the ERP USERSIDE will be set to the system's default directory /var/www/userside. If you wish to install ERP USERSIDE in a different directory, please remember to fix it in all subsequent examples. In order not to make a mistake, we recommend to use exactly the directory /var/www/userside.

Now edit the file /etc/nginx/conf.d/default.conf. Or delete it and create a file named /etc/nginx/conf.d/userside.conf. In either case, regardless of your choice, the contents of the file should be as follows (instead of userside.mycompany.com, specify your domain name):

server {
    listen       80 default_server;
    server_name  userside.mycompany.com;
    charset      utf-8;
    client_max_body_size 100M;

    access_log  /var/log/nginx/userside-access.log;
    error_log   /var/log/nginx/userside-error.log;

    root   /var/www/userside/public;
    index  index.php;

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    location / {
        try_files $uri $uri/ /index.php$is_args$args;
    }

    location ~* ^.+\.(css|js|ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|rss|atom|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ {
        access_log    off;
        log_not_found off;
        expires       max;
        add_header    Pragma public;
        add_header    Cache-Control "public";
    }

    location ~ \.php$ {
        try_files     $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass  unix:/run/php/php8.3-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_read_timeout 300;
        include       fastcgi_params;
    }

    location /ws {
        proxy_pass http://127.0.0.1:15674/ws;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
    }

    location ~ /\.ht { deny  all; }
}

You may also want to configure SSL and something else. This topic is beyond the scope of this manual.

If you have installed RabbitMQ on another server, or changed the value of the WebSTOMP port from 15674 to some other, then specify the IP address of the server with RabbitMQ and the port number in the line proxy_pass http://127.0.0.1:15674/ws;. The protocol must remain http. The path must remain /ws.

Check the configuration and if it is ok restart nginx:

sudo nginx -t && sudo nginx -s reload

Python

Debian 12 has Python 3.11 in its default repository - this is sufficient, so no alternative repositories are needed. But if you have a version lower than 3.9 on your host, you should be sure to install additionally a more recent version of Python.

Installing the required packages

sudo apt install -y python3-dev python3-pip python3-venv libffi-dev pkg-config

Supervisor

sudo apt install -y supervisor

Downloading and launching the installer

Setting up after installation

Configuration of system services

After installation, copy the sample configuration files of the system services and, if necessary, customise them (for example, specify the path to the userside directory if it is different from /var/www/userside). Run the commands:

sudo cp etc/us-core-worker.conf-example /etc/supervisor/conf.d/us-core-worker.conf
sudo cp microservice/poller/etc/usm_poller.conf-example /etc/supervisor/conf.d/usm_poller.conf
sudo cp etc/logrotate-example /etc/logrotate.d/userside
sudo cp microservice/poller/etc/logrotate-example /etc/logrotate.d/usm_poller
sudo cp etc/crontab-example /etc/cron.d/userside

Installation of necessary dependencies for usm_poller

Create a venv virtual environment for python and install the dependent modules as shown below:

cd /var/www/userside/microservice/poller
sudo -H python3 -m venv venv
sudo -H venv/bin/pip install -U pip
sudo -H venv/bin/pip install -U -r requirements.txt

Running the supervisor

The Supervisor controls the operation of the services specified in its configuration. To launch the supervisor, run the following:

sudo systemctl restart supervisor

After the supervisor is launched, after a few seconds you can observe the state of all the services it controls. All services must be in the RUNNING state:

sudo supervisorctl status

Immediately after the installation

This completes the installation of ERP USERSIDE. Now perform the following steps:

Open the system page http://userside.mydomain.com/ and login (default username: Admin, password: 1234).
Configure Websocket under: Settings - Main - WebSocket. Enable and enter the username and password of the WebStomp user you created earlier under RabbitMQ.
Make main settings in the section: Settings - Main.
Configure interaction with billing according to the instructions.
Read the instructions on the page: Where to start?.
Configure the API Key.

System upgrade

We try to do our best to ensure that the likelihood of an end-to-end upgrade from any version to the latest version is successful. However, it sometimes happens that due to some ancient problems with the database structure that never prevented you from working before, a situation may arise in which an end-to-end upgrade is not possible. In this case you will have to upgrade step by step to each next latest version. For example, from 3.14.80 first to 3.15.60, then to 3.16.39, and so on.

Before upgrading, be sure to read about all the changes you need to make to upgrade for each version that lies between yours and the one you are going to upgrade to.

We recommend making a copy of your system to safely perform the upgrade. Refer to HOWTO: USERSIDE cloning. You can always delete the copy and start over. This is easy and safe. But if you wish to upgrade a running system, perform it at times of least stress, with time to spare for disaster recovery if something goes wrong. Always be sure to back up your database and files before upgrading. If an update is interrupted, especially during the data migration phase, there is a risk that it will be very difficult or even impossible to restore the data correctly. In general, the responsibility for having an up-to-date backup lies entirely with you.

Upgrade process

We strongly recommend performing test upgrades on a copy of the system.

To perform an upgrade, run the installer in install mode and follow the instructions:

cd /var/www/userside
sudo -u www-data php userside_install.phar install

After upgrading the USERSIDE, you should be sure to upgrade the usm_poller dependencies:

cd /var/www/userside/microservice/poller
sudo -H venv/bin/pip install -U pip
sudo -H venv/bin/pip install -U -r requirements.txt

Now we need to restart all the services that are supervised by the supervisor and make sure that all of them start and are in the RUNNING state:

sudo supervisorctl restart all
sudo supervisorctl status

System maintenance

To keep the system up to date and to ensure reliable operation in the future, it is recommended that you follow a few simple steps:

periodically back up the database
periodically create a backup copy of uploaded files (this can be done simply by archiving the /var/www/userside/var/attachments directory and the .env file)
periodically monitor server performance (using htop, atop) and the RabbitMQ broker and make decisions about scaling fpm pools, kernel background processes and microservices

Backup

Set up periodic backups of the database and user files. Preferably at least once a day when the server is least loaded. In PostgreSQL, there are two ways to back up the database: dump and sql-script. Each of them has its own advantages and disadvantages. The first one creates a secure dump of the entire database, while the second one creates a SQL script that can restore both structure and data. We recommend referring to documentation on how to use the pg_dump command for a more suitable method or set of parameters for you.

Below are two commands for creating a database backup - choose one of them and add it to your crontab:

# SQL-script
sudo -u postgres pg_dump --no-acl -Fp -Z 5 userside > /backup/userside.sql.gz

# DUMP
sudo -u postgres pg_dump --no-acl -Fc userside > /backup/userside.dump

To create a minimal backup of files you can use the following command (for version 3.15 and later):

sudo tar -czf /backup/userside.tgz .env common/config/settings.json var/attachments

These files together with the database will be enough to restore USERSIDE. But you can back up the entire /var/www/userside directory for more reliability.

Restoring from a backup

The pg_restore command is used to restore from a dump. Documentation on using pg_restore. The psql utility is used to restore from a SQL script. Documentation on using psql.

SQL-script

To restore a database from a backup as a SQL script, you must first recreate the database and then restore to an empty backup database:

sudo -u postgres dropdb userside
sudo -u postgres createdb -e -E "UTF-8" -l "uk_UA.UTF-8" -O userside -T template0 userside
sudo -u postgres psql -d userside -c "CREATE EXTENSION postgis"
gunzip < userside.sql.gz | sudo -u postgres psql -d userside -v ON_ERROR_STOP=1

DUMP

The dump contains the entire database with all its elements. Therefore, before restoring a database from a dump, it must not exist on the server - delete the database before restoring from a dump. It is also important that the database user (the owner of all items) already exists - create it before restoring if you are restoring the database on a new server where the required user does not exist yet.

Also note that the dump structure may not be compatible between different versions of PostgreSQL. This is not necessarily the case, but it is likely.

To restore from the dump, use the commands:

sudo -u postgres dropdb userside
sudo -u postgres pg_restore --clean --if-exists --create --exit-on-error --dbname=postgres userside.dump

Note that the database name in the command must be postgres, because the entire database is restored from the dump, not its contents.

After restoring the database

After restoring the database in any way, you must clear the ERP USERSIDE cache, as the cache now contains different data from the database. Go to the /var/www/userside directory and run the clear cache command:

cd /var/www/userside
php run cache/flush-all

And run the restore command, which among other things will check the system for integrity.

sudo -u www-data php userside_install.phar repair