System Preparation

Basic requirements

The required operating system is a contemporary GNU/Linux distribution. This installation guide assumes that you use Debian 11 (Bullseye), with a non-root dataman user account created (there is a section on how to create that user).

Moreover, the n6 infrastructure depends on:

• RabbitMQ (an AMQP message broker),
• MariaDB (a SQL database server) with the TokuDB engine.

To run some n6 components it is also required to have installed:

• MongoDB (a NoSQL database server),
• Apache2 (a web server).

Debian dependencies and tools

You should install the essential Debian packages:

$ sudo apt-get update && \
  sudo apt-get install -y \
        apt-transport-https \
        build-essential \
        curl \
        gnupg2 \
        libattr1-dev \
        libgeoip1 \
        libsqlite3-dev \
        libssl-dev \
        libmariadb-dev \
        libbz2-dev \
        libffi-dev \
        libyajl2 \
        python3 \
        python3-dev \
        python3-venv \
        python3-stemmer \
        rsyslog \
        ssh \
        supervisor \
        swig \
        systemd \
        virtualenv \
        wget \
        zlib1g-dev
$ sudo apt-get clean

Creating the dataman user

Let n6 be run by the dataman OS user. First, let us create its initial login group:

$ sudo /usr/sbin/groupadd dataman
$ sudo /usr/sbin/useradd -rm \
    -d /home/dataman \
    -s /bin/bash \
    -p '' \
    -g dataman \
    dataman

We will keep the n6 repository in the dataman’s home directory.

RabbitMQ

RabbitMQ is an open source message broker software (sometimes called message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). RabbitMQ is responsible for communication between most of the n6 components.

Setup

Install RabbitMQ on Debian solution.

$ curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null
$ curl -1sLf https://ppa.novemberain.com/gpg.E495BB49CC4BBE5B.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg > /dev/null
$ curl -1sLf https://ppa.novemberain.com/gpg.9F4587F226208342.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq.9F4587F226208342.gpg > /dev/null
$ sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
deb [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu jammy main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu jammy main
deb [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa2.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu jammy main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa2.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu jammy main
deb [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu jammy main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu jammy main
deb [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa2.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu jammy main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa2.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu jammy main
EOF
$ sudo apt-get update -y
$ sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl
$ sudo apt-get install rabbitmq-server -y --fix-missing

RabbitMQ is by default attached to systemd and is running after an installation. To see if rabbitmq-server process is running, check its status through the systemctl command, its value should be: active (running):

$ systemctl status rabbitmq-server
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset:
   Active: active (running) since Fri 2020-01-10 16:32:54 CET; 12min ago
 Main PID: 4771 (beam.smp)
   Status: "Initialized"
    Tasks: 84 (limit: 4689)
   Memory: 94.9M
   CGroup: /system.slice/rabbitmq-server.service
           ├─4771 /usr/lib/erlang/erts-10.6.1/bin/beam.smp -W w -A 64 -MBas ageffcbf -M
           ├─5019 erl_child_setup 32768
           ├─5042 inet_gethost 4
           └─5043 inet_gethost 4

Plugins

Enable necessary plugins, like SSL or management panel plugin:

$ sudo /usr/sbin/rabbitmq-plugins enable \
    rabbitmq_management \
    rabbitmq_management_agent \
    rabbitmq_auth_mechanism_ssl \
    rabbitmq_federation \
    rabbitmq_federation_management \
    rabbitmq_shovel \
    rabbitmq_shovel_management

The following plugins have been configured:
  rabbitmq_auth_mechanism_ssl
  rabbitmq_federation
  ...

Applying plugin configuration to rabbit@pw-ups02...
The following plugins have been enabled:
  rabbitmq_auth_mechanism_ssl
  rabbitmq_federation
  ...

started 8 plugins.

Configuration

If you do not provide a configuration file for RabbitMQ, default values will be used. Or you can use the example configuration from n6/etc/rabbitmq/conf/rabbitmq.conf, by copying the file to /etc/rabbitmq. Replace the paths to the certificate files.

ssl_options.cacertfile = /home/dataman/n6/etc/ssl/generated_certs/n6-CA/cacert.pem
ssl_options.certfile = /home/dataman/n6/etc/ssl/generated_certs/cert.pem
ssl_options.keyfile = /home/dataman/n6/etc/ssl/generated_certs/key.pem

Restart the rabbitmq-server process afterwards:

$ sudo service rabbitmq-server restart

To ensure everything is OK, sign in to the RabbitMQ web management interface through your web browser. The default address is http://localhost:15672, or https://localhost:15671 if you have used the example config. You can use default guest credentials:

default user: guest
default password: guest

If you copied configuration file create a new user, allow them to use the Management GUI and give them read/write permissions to resources within / vhost. Replace in script below with login@example.com:

$ sudo rabbitmqctl add_user <username> <password>
$ sudo rabbitmqctl set_user_tags <username> management
$ sudo rabbitmqctl set_permissions -p / <username> ".*" ".*" ".*"

To make the new user an administrator, set him the administrator tag:

$ sudo rabbitmqctl set_user_tags <username> administrator

Troubleshooting: [ERROR] rabbitmq-server failing to start

In case of issue with rabbitmq-server copy certificate files to different location, for example to /etc/rabbitmq, run chmod and change paths in /etc/rabbitmq/rabbitmq.conf accordingly:

$ sudo mkdir -p /etc/rabbitmq/certs/
$ sudo cp /home/dataman/n6/etc/ssl/generated_certs/n6-CA/cacert.pem /etc/rabbitmq/certs/
$ sudo cp /home/dataman/n6/etc/ssl/generated_certs/cert.pem /etc/rabbitmq/certs/
$ sudo cp /home/dataman/n6/etc/ssl/generated_certs/key.pem /etc/rabbitmq/certs/
$ sudo chmod -R 664 /etc/rabbitmq/certs/

MariaDB

n6 uses two SQL databases - event database and Auth DB. The event database primarily stores processed information about network events and possible security incidents, also their relation to organizations linked to clients. The Auth DB database is used for client authorization. It stores clients’ permissions and information about allowed resources (allowed API endpoints, allowed subsources).

The required version of MariaDB is 10.3 which is not supported by Debian 11 using a package manager. One of the installation methods is to use MariaDB Binary Tarballs.

To install MariaDB from Generic Binaries on Unix/Linux, download the appropriate binary version. Go to https://mariadb.org/download, select and download MariaDB Server 10.3.x version to /home/dataman/Downloads directory.

You should install the essential Debian packages:

$ sudo apt update && \
  sudo apt install -y libaio1 libncurses5 libjemalloc2

Add the mysql group and the mysql user.

$ sudo groupadd mysql
$ sudo useradd -g mysql mysql

Create a directory to which you want to unpack the distribution and change your location to the directory path. In the following example, we unpack the file to /usr/local/mysql. (Therefore, the instructions assume that you have permission to create files and directories in /usr/local/mysql. If that directory is protected, you must perform the installation as root. Then unpack the distribution. The tar command unpacks the distribution to the /usr/local/mysql/.

$ sudo mkdir /usr/local/mysql
$ cd /usr/local/mysql
$ sudo tar -zxvpf /home/dataman/Downloads/mariadb-10.3.34-linux-systemd-x86_64.tar.gz -C /usr/local/mysql/ --strip-components 1

Most of the files installed by MariaDB may be owned by root if you like. The data directory is the exception, it must be owned by mysql user. To accomplish this, run the following commands as root in the installation directory.

$ sudo chown -R root .
$ sudo chown -R mysql data

You will find several files and subdirectories in the mysql directory. The most important for installation purposes are the bin and scripts subdirectories.

MariaDB server can be configured through its main configuration file. The default MariaDB option file is called my.cnf. Create an empty /etc/mysql/my.cnf file and fill it with configuration content like below:

$ sudo mkdir /etc/mysql/
$ sudo touch /etc/mysql/my.cnf

# Minimal option file
# Run command as root
$ cat <<EOT > /etc/mysql/my.cnf
[client-server]
socket=/run/mysqld/mysqld.sock
port=3306

# This will be passed to all MariaDB clients
[client]
#password=my_password

# The MariaDB server
[mysqld]
# Directory under which is the distribution
basedir=/usr/local/mysql
# Directory where you store your data
datadir=/usr/local/mysql/data
# The name of the login account that you created in the first step to use for running the server.
user=mysql
# Directory for the errmsg.sys file in the language you want to use
language=/usr/local/mysql/share/english

# This is the prefix name to be used for all log, error and replication files
log-basename=mysqld

# Enable logging by default to help find problems
general-log

[mariadb]
# See https://mariadb.com/kb/en/tokudb-differences/ for differences
# between TokuDB in MariaDB and TokuDB from http://www.tokutek.com/

plugin-load-add=ha_tokudb.so

[mysqld_safe]
malloc-lib=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
EOT

Create a directory for server through socket /var/run/mysqld/mysqld.sock:

$ sudo mkdir /run/mysqld/
$ sudo chown -R mysql:mysql /run/mysqld/

Check for Transparent HugePage support on Linux. This feature causes problems if enabled, and you should turn it off (as a root). Check for Transparent HugePage Support on Linux.

# Run command as root
$ echo never > /sys/kernel/mm/transparent_hugepage/enabled
$ echo never > /sys/kernel/mm/transparent_hugepage/defrag

To initialize the MySQL database containing the grant tables that store the server access permissions. The command should create the data directory and its contents with mysql as the owner. After creating or updating the grant tables, you need to restart the server manually.

$ sudo ./scripts/mysql_install_db --user=mysql

To start MariaDB automatically when you boot your machine, copy support-files/mysql.server and support-files/systemd/mariadb.service to the location where your system has its startup files.

$ sudo cp support-files/mysql.server /etc/init.d/mysql.server
$ sudo cp support-files/systemd/mariadb.service /usr/lib/systemd/system/mariadb.service
$ sudo systemctl daemon-reload

Note that by default the /usr directory is write protected by systemd though, so when having the data directory in /usr/local/mysql/data as per the instructions above you also need to make that directory writable (as a root).

# Run command as root
$ mkdir /etc/systemd/system/mariadb.service.d/
$ cat > /etc/systemd/system/mariadb.service.d/datadir.conf <<EOF
[Service]
ReadWritePaths=/usr/local/mysql/data
EOF
$ systemctl daemon-reload

Modify the $PATH environment variable, so you can invoke commands such as mysql, mysqldump, etc.

$ echo "export PATH=${PATH}:/usr/local/mysql/bin" >> ~/.bashrc
$ bash

After everything has been unpacked and installed, you should test your distribution. To start the MariaDB server, use the following command:

$ sudo ./bin/mysqld_safe --datadir='/usr/local/mysql/data' &

To set a password for the MariaDB root accounts, use the following command:

$ sudo ./bin/mysql_secure_installation --defaults-file=/etc/mysql/my.cnf

Then kill mysql process, to run the right way by systemd.

$ sudo pkill mysql

Check whether mariadb is controlled by systemd. To check whether mariadb is running, look for its status, it should be active.

$ sudo systemctl status mysql
or
$ sudo service mysql.server status
● mysql.server.service - LSB: start and stop MariaDB
     Loaded: loaded (/etc/init.d/mysql.server; generated)
     Active: inactive (dead)
       Docs: man:systemd-sysv-generator(8)

Jan 04 03:42:02 pw-ups02 su[4770]: (to mysql) root on none
Jan 04 03:42:02 pw-ups02 su[4770]: pam_unix(su-l:session): session opened for user mysql(uid=1001) by (uid=0)
Jan 04 03:42:02 pw-ups02 su[4770]: pam_unix(su-l:session): session closed for user mysql
Jan 04 03:42:03 pw-ups02 su[4784]: (to mysql) root on none
Jan 04 03:42:03 pw-ups02 su[4784]: pam_unix(su-l:session): session opened for user mysql(uid=1001) by (uid=0)
Jan 04 03:42:03 pw-ups02 su[4784]: pam_unix(su-l:session): session closed for user mysql
Jan 04 03:42:03 pw-ups02 mysql.server[4577]: ....
Jan 04 03:42:03 pw-ups02 systemd[1]: mysql.server.service: Succeeded.
Jan 04 03:42:03 pw-ups02 systemd[1]: Stopped LSB: start and stop MariaDB.
Jan 04 03:42:03 pw-ups02 systemd[1]: mysql.server.service: Consumed 9.929s CPU time.

Initialize system database

In this step we create databases and their tables. Start database processes in case it is not active:

$ sudo systemctl start mysql
or
$ sudo service mysql.server start

Make sure you have access to the database:

$ mysql -u root -p<your_password>

Check that the TokuDB plugin is active in MySQL prompt:

> show plugins;
...
TokuDB
TokuDB_user_data
TokuDB_user_data_exact
TokuDB_file_map
TokuDB_fractal_tree_info
TokuDB_fractal_tree_block_map

MongoDB

n6 uses MongoDB as archival database. Events gathered by collectors will be stored in MongoDB and can be restored in case of errors.

Installation steps below are based on Install MongoDB Community Edition on Debian solution.

Install libssl1.1

$ wget http://deb.debian.org/debian/pool/main/o/openssl/libssl1.1_1.1.0j-1~deb9u1_amd64.deb
$ sudo dpkg -i libssl1.1_1.1.0j-1~deb9u1_amd64.deb
$ rm -i libssl1.1_1.1.0j-1~deb9u1_amd64.deb

To install MongoDB, do the following (as root):

$ wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install -y mongodb-org

Try to start MongoDB with systemd:

$ sudo systemctl start mongod

Successful output should look similar to the output below:

$ sudo systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-01-26 05:17:13 EST; 5min ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 5537 (mongod)
     Memory: 72.0M
        CPU: 2.597s
     CGroup: /system.slice/mongod.service
             └─5537 /usr/bin/mongod --config /etc/mongod.conf

Jan 26 05:17:13 pw-ups02 systemd[1]: Started MongoDB Database Server.

Check if you are able to connect to MongoDB console:

$ mongo
MongoDB shell version v4.2.18
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("a6f4ca5e-eab1-4e65-9c3e-4e9f877a4e45") }
MongoDB server version: 4.2.18
....
>

Apache HTTP server

n6 uses Apache as an HTTP server for services like n6 REST API, n6 Portal API or n6 Admin Panel N6RestAPI or N6AdminPanel.

$ sudo apt update && \
  sudo apt install -y apache2 libapache2-mod-wsgi-py3 

Check if the apache2 service is run by systemd:

$ systemctl status apache2
● apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2020-01-14 18:07:49 CET; 27min ago
     Docs: https://httpd.apache.org/docs/2.4/
 Main PID: 20852 (apache2)
    Tasks: 55 (limit: 4689)
   Memory: 13.7M
   CGroup: /system.slice/apache2.service
           ├─20852 /usr/sbin/apache2 -k start
           ├─20854 /usr/sbin/apache2 -k start
           └─20855 /usr/sbin/apache2 -k start

Jan 14 18:07:49 debian systemd[1]: Starting The Apache HTTP Server...
Jan 14 18:07:49 debian apachectl[20848]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Jan 14 18:07:49 debian systemd[1]: Started The Apache HTTP Server.

While apache2 is running, enable required modules:

Enable modules:

$ sudo /usr/sbin/a2enmod ssl
...
Enabling module socache_shmcb.
Enabling module ssl.

$ sudo /usr/sbin/a2enmod rewrite
...
Enabling module rewrite.

To run modules you need to reload/restart apache2:

$ sudo systemctl restart apache2

Arrangements related to Apache

Add dataman to the www-data group, make the necessary directories, and set appropriate permissions:

$ sudo /usr/sbin/usermod -a -G dataman www-data
$ mkdir -p /home/dataman/env_py3k/.python-eggs
$ sudo chown -R dataman:www-data /home/dataman/env_py3k
$ sudo chmod 775 /home/dataman/env_py3k/.python-eggs
$ sudo chown -R www-data:www-data /etc/apache2/sites-available/