Configuring the Appliance

Configuration Notes

Initially the appliance is configured to be completely self-contained, with a local database and object store for the services it runs. All of the default configuration options are shown in /etc/kloudless-defaults.yml. The modifiable configuration files are stored on the data disk in /data/ so that they are preserved across system upgrades.

There are two main configuration files that are used to generate the rest of the system configuration. The configuration scripts mentioned here modify values in /data/kloudless.system.yml. The user modifiable configuration file, which must be manually created, is /data/kloudless.yml. The different configuration values and scripts are described in the following sections. In order to apply the changes made to /data/kloudless.yml use the ke_update_configuration utility.

When external services are configured for PostgreSQL and Redis, the local version of those services do not run. Note: the formatting of kloudless.yml requires spaces for indentation, rather than tab characters.

Data Disk Configuration

In order to preserve configuration information across system upgrades, it is required that an extra disk be configured. This step is not required for Docker, since the extra volume is automatically mounted. For other platforms, finding the relevant block device can be done with the following command:

lsblk

Use the name of the disk that doesn't have any partitions and is currently not mounted when running the following command (e.g. if sdb is the extra disk, /dev/sdb should be entered when prompted):

sudo ke_configure_data_disk

This utility will format the specified drive and copy over any existing configuration and data from the original /data/ directory.

Uploading the License Key

The license key is a file that ensures your deployment of Kloudless Enterprise is authorized by Kloudless. The license key can be obtained from the Kloudless Enterprise Portal once your organization has had its account provisioned. Once the initial configuration has completed, you will need to securely copy your license key file onto the server, for example using scp:

scp -i your_private_key ./license.key \
ubuntu@your.server.ip.address:~/

You can also use sftp or another compatible client.

Once the key file is on the server you can configure the application server to use it using the following command:

sudo ke_manage_license_key set ./license.key

This will put the key file in the proper place and allow the API gateway to function properly. You can also view information about a license using the following command:

sudo ke_manage_license_key show ./license.key

Without a license provided via the command line, it will look at the license currently set on the appliance.

The appliance will perform validation checks on the License Key as described here.

Non-Interactive (Docker)

There is a non-interactive means of configuring the license key that is particularly useful when deploying the appliance via Docker containers. The license key should be copied into the volume that corresponds to the /data directory. Once in place, the following configuration value can be set in /data/kloudless.yml either directly or by KLOUDLESS_CONFIG environment variable when starting the container:

---
# ... trimmed for brevity
license: /data/name_of_license_key

Creating Initial Configuration File

In order to add additional configuration values a minimal configuration file must be created:

echo "---" | sudo tee /data/kloudless.yml > /dev/null

Hostname Configuration

By default the appliance can be accessed using the IP address that is assigned when it boots. Once a DNS hostname is assigned to the instance (or cluster's load balancer) then that needs to be configured on the appliance in /data/kloudless.yml:

---
# ... trimmed for brevity
hostname: "kloudless-api.example.com"

For versions >=1.25.0, there is an optional internal_hostname configuration parameter. If set (in the same way as the hostname parameter), it will be used by the developer portal exclusively and additionally accepted by the api portal.

Database Configuration

PostgreSQL 9.4+ is required. There are two different configuration options available for the backing database.

By default the appliance is configured to use a local PostgreSQL database which stores its data on the configured data drive. This configuration is only recommended for testing and small deployments since the database is not backed up and is not highly available.

The recommended production configuration is to have an external PostgreSQL database. A user, password, and database should be created on the external PostgreSQL host (this should not be shared by any other applications). Once created, add the hostname, port number, database name, username, and password to the system's configuration file /data/kloudless.yml as shown:

---
# ... trimmed for brevity
db:
    name: 'example'             # database name
    user: 'kloudless'           # database user name
    password: 'examplepassword' # database user password
    host: 'db.example.com'      # database host
    port: 5432                  # database port

Redis Configuration

Redis 2.8+ is required for process orchestration and as a task broker within both single and multiple node deployments. There are two configuration options available.

By default a local Redis server is used and this is only usable for single node deployments.

For multi-node deployments, an external Redis instance is required. This can be configured in the /data/kloudless.yml configuration file using a standard Redis URL. Please note that if the password contains any special characters, it must be url encoded before being used here. Also note, that in order to avoid key conflicts, a Kloudless deployment or cluster should have its own Redis database. Please assign a unique database number for a given appliance/cluster:

---
# ... Other values omitted for brevity
redis: redis://:password@hostname:port/db_number

In order to achieve high performance, as well as ensure resiliency in the case of a large number of connections. The following configuration parameters should be set on the external Redis server:

maxclients 50000
timeout 300

Configuring SSL/TLS

Making requests over HTTPS to the appliance is recommended for security reasons. In cases where the appliance handles incoming webhooks, many upstream services require that webhook receivers use HTTPS with a valid certificate. There are two ways to use SSL/TLS when communicating with the appliance. The first is to have the appliance host the certificates and negotiate the secure connection on it's own. This is only recommended for single node deployments. The second is to have HTTPS be handled by an intermediate load balancer which then passes HTTP request through to one or more Kloudless Enterprise appliances. Using a load balancer allows multiple Kloudless Enterprise appliances to appear to the client as a single host.

If the appliance will serve HTTPS directly, you will need to copy the certificate and key to the server (for example, via scp). The certificate and key should be placed in the /data directory so that they will be preserved across upgrades. Once in place, SSL can be configured as follows:

---
# ...
ssl:
    is_configured: true
    local: true
    cert: /data/certificate_file.crt
    key: /data/key_file.pem

If HTTPS is terminated at the load balancer, the following configuration is required:

---
# ...
ssl:
    is_configured: true
    local: false

Displaying Services and Configuring OAuth Keys

Services that use OAuth such as Box and Dropbox require OAuth Client IDs and Secrets to be configured in Kloudless. These keys can be obtained from the relevant service’s developer portal and will then be used by Kloudless to prompt your users to grant access to their account to the associated app.

For information on how to create and configure keys for individual services, see the Custom keys section for each service in the Credentials page of your appliance’s Kloudless Developer Portal.

This page enables Kloudless developers to configure custom OAuth keys for their individual applications. However, the instructions can also be used to create OAuth keys for use by all developers signed up on your Kloudless instance. Once they keys have been created, they can be configured as follows:

sudo ke_manage_service_keys --action add --service service \
--key "OAuth Consumer Key" \
--secret "OAuth Consumer Secret"

Use --help to see a full list of options. --admin must be specified when connecting separate OAuth keys intended to authenticate admin accounts.

This should be done for each of the services that you would like to use with your appliance. To see a list of available services, as well as other options for listing and removing existing service keys, use ke_manage_service_keys --help. Visit /applications/*/credentials on your Kloudless instance for assistance with the procedure to configure custom OAuth Keys. For multi-node deployments, this only needs to be done on one node per cluster. The keys are securely stored in the database and can be accessed by any node in the cluster as needed.

Restarting the API process is necessary for the changes to take effect:

supervisorctl restart api

By default, services that use OAuth are not visible during the authentication process unless default OAuth keys are configured. If you would prefer to display the option to authenticate the service regardless of whether a global default service key has been configured, you may do so by visiting the Admin Portal’s Service list as a superuser, selecting the service you wish to activate, checking ‘Visible’ and saving the service. Adding a default OAuth Key as described above automatically enables visibility. Note that if neither a default key nor an app-specific key is configured for that service, authentication will fail.

Monitoring (optional)

The Kloudless Enterprise Appliance can be configured to send system and application-level statistics to a StatsD receiver or InfluxData stack. The InfluxData TICK stack is included in the appliance and is enabled by default for nodes with no remote redis server configured. This means that all single Kloudless instances will automatically run the InfluxData stack. All data related to the TICK stack is persisted on the data disk to ensure that it is preserved across system upgrade. Metric data in the embedded stack is retained for two weeks. To toggle the TICK stack on a specific node, change the configuration in /data/kloudless.yml as follows:


# ...
influxdb:
    enabled: false  # To ensure it is enabled, set to true

Nodes in a cluster should send metrics to a common external InfluxData server instead. To enable metrics tracking to an external InfluxDB server, the configuration below should be used instead:


# ...
influxdb:
    url: http://influxdb.yourdomain.com:8086
    database: YOUR_DATABASE  # (default: telegraf)
    retention_policy: YOUR_RP # (default: null)
    username: YOUR_USER  # (default: null)
    password: YOUR_PASSWORD  # (default: null)
    prefix: YOUR_PREFIX # prefix added to all metrics (default: '')

The statsd sender and influxdb sender are mutually exclusive. The InfluxDB output is recommended since it provides more metadata (in the form of tags) and more robust querying for each metric. To enable sending metrics to an external StatsD server instead, the following configuration should be used:


# ...
statsd:
    host: stats.YOURDOMAIN.com
    port: 8125
    prefix: kloudless # prefix added to all metrics (default: '')

To send metrics to an external service other than InfluxDB or StatsD, please contact support@kloudless.com to discuss options.

For more information on metrics available, review Metrics and Monitoring.

Customized Logging (optional)

The Kloudless Enterprise Appliance can be configured to send its syslog logs to a remote syslog server or external logging service (like Splunk or Logstash). This can be done by modifying logging value in the /data/kloudless.yml file. By default the logging key contains safe defaults that only allow local logging. As an example, a remote rsyslog server using the RELP protocol can be configured as follows:


# ... other values left out for brevity ...
logging:
  protocol: relp
  host: logging.example.com
  port:  20514

The following protocols are supported:
TCP
UDP
RELP

In order to prevent data loss, the RELP protocol is recommended, however it is not supported everywhere so TCP and UDP are offered as alternatives. Once the changes have been made, they can be applied with sudo ke_update_configuration.

For more advanced set ups, we allow the end user to include custom rsyslog configuration snippets. The configuration snippet should be set as the value of the custom key in the logging section. For example to configure logging via TLS:


# ...
logging:
  custom: |
    $DefaultNetstreamDriverCAFile /data/server-cert.pem
    $DefaultNetstreamDriver gtls # use gtls netstream driver
    $ActionSendStreamDriverMode 1 # require TLS
    $ActionSendStreamDriverAuthMode anon # server is unauthd
    *.* @@logs.example.com:10514

This particular example requires the additional system package rsyslog-gnutls which can be installed using the mechanism described in the Extra Packages section.

Log Retention (optional)

The verbosity of the appliance logs as well as their retention policy can be configured via the logging section of /data/kloudless.yml. The following values can be set:

  • level: The logging level of the Kloudless Enterprise applications. Valid values (in order of decreasing verbosity):
    • debug (default)
    • info
    • warning
    • error
    • rotation: sub-section describing retention with the following sub-keys
      • period: How often to rotate logs, valid values are:
        • daily (default)
        • hourly
        • weekly
        • size <size>
          • E.g. 1g
      • count: How many rotations to store (default: 7).

Once the changes have been made to the configuration file, they can be applied via sudo ke_update_configuration.

Extra Packages (optional)

For some deployment situations, extra package might be required so that the Kloudless Appliance can integrate with existing infrastructure. In order to maintain extra installed packages a list should be maintained in /data/kloudless.yml as follows:


# ...
extra_packages:
  - "tmux"
  - "rsyslog-gnutls"

After the list has been updated, the packages will be installed by running ke_update_configuration. Note: Removing packages from this list will not remove them from the system. Any packages installed via this method must be manually removed.

Applying Configuration Changes

Once all of the needed changes have been made to /data/kloudless.yml the changes can be applied with the following command:

sudo ke_update_configuration