Kloudless API Guide

# Deploying the Appliance

# Containers: Docker & Kubernetes

You can start running an instance of Kloudless on Docker in 4 steps.

  1. Authenticate to our private Docker Registry.
docker login -u <email> -p <meta-token> docker.kloudless.com

The Meta (Bearer) Token is available Developer Portal Account page.

  1. Pull an image of the latest version.
docker pull docker.kloudless.com/prod:1.32.x

See the enterprise downloads page for the latest version available, and Release Notes for recent changes.

  1. Look up your license code from the Enterprise License page.

Generate a configuration file template with this script, and fill in necessary values. The script creates unique private keys to protect your data at rest. You have the only copy of the keys. Keep backups and kept the keys secret.

./ke_config_skel.sh > kloudless.yml

Customize the resulting YAML file for your deployment. It will later be deployed either as an environment variable or on a removable volume.

Your license code can be found on the Enterprise License page.

# ./kloudless.yml - Example Instantiation
hostname: my-kloudless.company.com
license_code: XyzEnterpriseLicenseCodeXyz
...
# db:    # for persisting data outside the instance [host,port,user,password,name]
# redis: # for clustering Kloudless instances

A minimal kloudless.yml configuration file should include a hostname, which the appliance filters on, and a license_code.

# ./kloudless.yml
hostname: kloudless.example.com
license_code: XyzEnterprieLicenseXyz
  1. Run the appliance.
docker run \
 --name kenterprise \
 --volume `pwd`/ke-data:/data
 --env KLOUDLESS_CONFIG="$(cat ./kloudless.yml)" \
 --tmpfs /run --tmpfs /run/lock --tmpfs /var/lib/journal \
 -p 80:80 -p 8080:8080 -p 443:443 -p 8443:8443 \
 --ulimit nofile=1024000:1024000 \
 --sysctl net.ipv4.ip_local_port_range='1025 65535' \
 docker.kloudless.com/prod:1.32.x \
 docker_entry

For deploying containers on a Kubernetes cluster, see the Kubernetes Deployment appendix section. For unmanaged Docker, see the Docker Deployment section of the appendix, of the Container Deployment appendix.

  1. Access the container console.
docker exec -ti kenterprise ke_shell

In order to run the administrative utilities within the container and inspect the state, it is required to run ke_shell. This script ensures that the shell environment is properly configured to run the administrative utilities.

# Amazon Web Services (AWS)

# Creating an Instance from an AMI

Provide Kloudless Support with the AWS Account Number to share the Kloudless Enterprise AMI with. Once shared, you should be able to launch an instance of that AMI. When creating an instance, make sure to choose an instance type that fulfills at least the Minimum Hardware Requirements (e.g. t2.large). For all disks, we recommend using EBS with gp2 SSDs.

While creating the instance, attach a separate EBS drive to use as the “data disk” to persist data on. A 50 GB EBS drive is sufficient.

# Networking

When deploying the Kloudless Appliance it is important that it is reachable from wherever your application will be running. Security Groups should be used to isolate the Kloudless Appliance from sources that do not require access. The services that are exposed on the appliance are described in the Network Services section, care needs to be taken that there is no unauthorized access to the Developer portal and administrative consoles especially.

# Accessing the Administrative Console

While creating the instance, an SSH key should have been configured. The key configured there can be used to access the administrative console over SSH as follows:

ssh -i yourkey.pem ubuntu@instance_ip

# OVA (VMWare and VirtualBox)

# Obtaining the OVA

The OVA files can be obtained by contacting the Kloudless Support Team.

# Importing the OVA

During the initial import of the Virtual Appliance it is important to configure an external hard drive which will be used to store local configuration, logs, and the local database if used. Please refer to the Minimum Specifications section for details on how much CPU, memory, and disk to allocate. It is important that the primary disk in the appliance not be modified, otherwise the instance may fail to boot or upgrade properly.

# Network Configuration

The appliance requires DHCP in order to configure its network interfaces, DNS servers, etc. Please ensure that this is available before booting the appliance.

# Firewall Rules

By default the appliance does not have any firewall in place. Access to the appliance should be managed by an external firewall or by manually configuring iptables on the appliance (note that this will not be persisted across system upgrades by default). For information on what rules to apply see the Network Services section.

# Accessing the Administrative Console

You should be able to log in on the console using the default user:password ubuntu:ubuntu. You should immediately change the password of user ubuntu, using the passwd command, and disable password authentication for SSH, by including the following snippet in /data/kloudless.yml and updating configuration by running sudo ke_update_configuration

# ./kloudless.yml
ssh:
    password_auth: false

Docker containers need only apply these changes if the ssh port is exposed.

# Azure

On Azure, we recommend running Kloudless Enterprise in Docker. For setup, you can follow the brief containers setup section, after completing Azure-specific setup of database and networking.

We also have customers running our containers in Azure Kubernetes Service (AKS), and we can offer detailed help in setting up Kloudless Enterprise in your environment.

Some service providers require setup of Service Keys particular to your application use. Once your Kloudless appliance is deployed, see the configuration guide's OAuth & Service Keys section for further setup information.

As an alternative to Docker containers, we can provide a Virtual Machine (VM) image to run a dedicated Kloudless VM, but the process requires a few hours of coordination to transfer a data blob, create a VM, and set it up. At least initial testing would benefit from the speed and ease of using Docker. Contact us for more information.

# Instance Creation

Create a Linux VM (Ubuntu 18.04 recommended) and set up SSH key authentication using the Azure web console. See the Minimum Specifications for sizing guidance.

Using Azure-hosted Postgres with an Azure Virtual Machine requires enabling "Service Endpoints" in instance's and database's settings after completion of the wizard-guided creation process.

# Database

We recommend using an external database to simplify deployment lifecycle and scaling. See the local database section for other supported options.

Create an "Azure Managed Database for Postgres". Do not use a Basic Tier server as they do not support accessing the database as a Service Endpoint on a Virtual Network.

In the Azure portal, go to the database admin panel. Find and note the following:

  • "{fqdn}", Fully-Qualified Domain Name (fqdn); something like {subdomain}.postgres.database.azure.com
  • "{subdomain}"; the leftmost portion of the {fqdn}
  • "{admin-user}"; the admin user: of the form "{admin-name}@{subdomain}
  • "{password}"; the database user's password

On the VM you can check connectivity with the following command, with -L to list the tables or without it for an interactive session.

$ psql "dbname=postgres
        host={fqdn} port=5432
        user={admin-user}@{subdomain} password={password}" -L

# (Optional) Local Database

The Kloudless appliance will run a local database if no external database is configured. If you choose to rely on an instance-local database, we recommend configuring a data disk to more easily transfer configuration files or locally-hosted database files when re-provisioning the VM.

For a Docker container using a local database, we recommend mounting a volume to /data, if the VM host disk has sufficient space per the data disk section.

# Networking

For a list of required ports (and their purposes) see the network services guide. Configure accessibility and firewall access for these ports, as appropriate for your deployment.

To secure API and Developer Portal endpoints, see the SSL/TLS section of the configuration guide.

The API server endpoint can remain public, but the developer portal should be disabled (firewalled, or limited to a private network) after setup, because it offers an account creation page to unauthenticated users.

Alternatively, new account creation can be paused with ke_manage_developers pause-signup.

# Private networking for VM-database connection

To configure private networking for the VM-database connection, we'll use Azure's "Service Endpoints". This configuration requires changes in 2 places in the Azure Portal:

  1. Admin Panel for the virtual machine
  • VM view: in the "Overview" pane, find the "Virtual network/subnet" item. Follow that link.
  • Virtual network view: in the menu/list on the left, find "Service endpoints"
  • Service endpoints: Add "Microsoft.Sql" to the subnet in-use ("default" for us)
  1. Admin Panel for the "Azure Database for PostgreSQL server"
  • (Side menu / Settings) "Connection security"
  • In the section "VNET Rules", select "+ Adding existing virtual network"
    • Name: "kloudless-db-vm-allow" for example
    • Subscription: (your subscription)
    • Virtual Network: (default vlan ... only shows if a "Subscription" is selected)
    • Subnet name/Address prefix: (default ok)
    • Click "OK" to finish setup.

# (Optional) Virtual Machine Image

Contact the Kloudless Support Team to get access to an Azure VM snapshot.

Once you have the Azure VM snapshot, create a VM from the image. Consider sizing guidelines in the Architecture Overview or contact Kloudless Support for help with capacity planning.

# (Optional) Data disk

The data disk stores persistent files including the kloudless.yml configuration file, license key file, and in single-node deployments, the database file for the optional built-in Postgres database. For virtual machines, we recommend use of a mountable data disk in order to eases upgrades and re-deployments by allowing operators to mount the disk into a new VM instead of having to manually transfer configuration and license files.

If the appliance must host the database locally because no external database is configured, provision at least 60 GB. If an external database is in use, a smaller data disk can be used, such as 10 GB (to still leave room for logs).

See the data disk configuration section for instructions.

For Docker containers relying on the appliance-internal database or without complete specification of application secrets and keys via kloudless.yml, we recommend configuring a data disk for the same ease-of-update reasons. However, we strongly recommend the application secrets/keys to be included in the configuration and using an external database when deploying Docker containers, and require this for new customers.

# Monitoring Initialization

The initial configuration process of the container can be monitored from within the container via the following command:

sudo tail -f /var/log/journal/ke_init.service.log

Logs for the whole appliance can be monitored through /var/log/syslog.

If the license key is already configured, completion of initialization can also be checked by querying the /health endpoint of the container:

curl http://localhost/health

A successful response will look something like the following:

{"celery": {"status": "ok", "local": {"queues": {"celery": 0, "celery-bg": 0},
"tasks": {"bg": "ok", "fg": "ok"}}, "remote": {"queues": {"celery": 0,
"celery-bg": 0}, "tasks": {"bg": "ok", "fg": "ok"}}, "elapsed":
0.42250490188598633}, "api": {"status": "ok"}, "db": {"status": "ok", "elapsed":
0.0}}

← System Configuration Guide →