Introduction to the Activity API 

Start with our Core API docs for an introduction to the Kloudless API and more information on connecting user accounts and performing API requests.

Once an account is connected to your application, your application can begin making requests to the Activity API endpoints listed below.

Overview ¶

The Kloudless Activity API helps monitor accounts using a variety of service-specific implementations to be aware of activity occurring within each account. For instructions on how to enable and use Activity Monitoring, see the Activity Monitoring Usage Guide.

The services that provide activity for each API category are listed below.

Storage Activity ¶

File-system activity data is available for the following services:

• alfresco

• alfresco_cloud

• box

• cmis

• cq5

• dropbox

• dynamics

• egnyte

• evernote

• gdrive

  • By default, Kloudless does not monitor Shared Drives’ activity for non-admin Google Drive accounts. To monitor specific folders, see the monitored_resources attribute of the Subscription object for more information.

• hubspot

• jive

• onedrivebiz

• s3

  • Buckets with existing Activity Configurations will be ignored.

• salesforce

  • Tracks Chatter files, Libraries, Documents and Attachments by default.
  • To monitor specific objects, see the monitored_resources attribute of the Subscription object for more information.

• sharefile

• sharepoint

  • Non-admin accounts will track only the primary site collection’s Document Library by default. To monitor others site collections and/or other Document Library, see the monitored_resources attribute of the Subscription object for more information.
  • Administrator accounts that connected through the admin OAuth flow with org-wide data access will have all its site collections activity tracked.

• skydrive

• smb

• webdav

• slack

  • Tracks Slack file, message, and conversation activity that the account has visibility into.
  • Org-wide Slack Enterprise Grid activity available in Slack Audit Logs API can be tracked with Audit activity.

• procore

• plangrid

• autodesk

  • Due to limitations with Autodesk Webhooks API, Autodesk activity for each account is only tracked for one application at a time across all environments.

• bluebeam

• kapost

  • Activity is only tracked for changes to Kapost Content objects.

CRM Activity ¶

CRM activity data is available for the following services:

• salesforce

  • Tracks unified CRM objects by default.
  • To monitor specific objects, see the monitored_resources attribute of the Subscription object for more information.

• dynamics

  • Only tracks add and update activity for supported object types. Please contact Kloudless Support if you’re interested in tracking delete activity.

• hubspot

  • Currently tracks create, update and delete activity for contact and account objects.

Calendar Activity ¶

Calendar activity data is available for the following services:

• exchange

  • Org-wide calendar activity monitoring for admin accounts is not supported.

• google_calendar

  • Org-wide calendar activity monitoring is enabled by default.

• outlook_calendar

  • Activity tracking is only available for past calendar events with an event start time within 6 months of the creation date of the account’s subscription.
  • Org-wide calendar activity monitoring is enabled by default.

By default, Kloudless monitors the user’s primary calendar in non-admin accounts, and monitors for org-wide calendar activity in admin accounts. Org-wide calendar activity monitoring in admin accounts monitors for changes in the primary calendars of every user in that admin’s organization.

If you would like to monitor other calendars, specify the calendars’ resource IDs in the monitored_resources attribute of the Subscription object.

Email Activity ¶

Email message activity data is available for the following services:

• gmail

  • Kloudless will monitor All Mail for activity.
  • Admin accounts can customize mailboxes to monitor by specifying team user IDs in monitored_resources.

• outlook_email

  • Kloudless will monitor the user’s inbox folder for activity.
  • Customize the folders monitored by specifying the user and folder IDs in monitored_resources. For more information see Monitored Resources. You may use well-known folder names as well.

• exchange_email

  • Kloudless will monitor the user’s inbox folder.
  • Customize the folders monitored by specifying their IDs in monitored_resources.

• imap

  • Coming soon!

Email Tracking Activity ¶

If a tracking attribute is provided when sending a message, Kloudless will publish information on opens and clicks via the Activity API. This is supported for all Email connectors. Enable email message tracking by creating a subscription with the subscription_type set to analytics.

Org-wide Activity ¶

Org-wide activity tracking for all organization users is available for administrator accounts for the following services in Kloudless Enterprise:

• box

• dropbox

• egnyte

• exchange

• gdrive

  • Requires Google Workspace Enterprise.

• google_calendar

• egnyte

• salesforce

• sharepoint

• onedrivebiz

• ms_teams

  • The Microsoft Teams connector guide has details on which activity is monitored.

• onedrivebiz

• outlook_calendar

• salesforce

• sharepoint

Audit Activity ¶

Audit activity can be retrieved through the normal activity endpoint, and will be available for administrator accounts for the following services:

• azurerm

• box

• cloudtrail

• dropbox

• gdrive

  • Requires Google Workspace Enterprise.

• onedrivebiz

• sharepoint

• slack

  • Requires Slack Enterprise Grid. Note that slack admin accounts prioritize providing org-wide activity, with granularity to the conversation level. To monitor message activity, either periodically list new messages in each Conversation (channel, private/group DM), or use a separate non-admin account to monitor activity in Conversations that the account has access to.

You can find the list of Kloudless Audit activity here. This activity provides additional audit-type information for the account.

v2: Check out our v2 Audit Activity API for a comprehensive solution to track org-wide activity occurring in business applications. The initial release includes support for popular services such as Office 365, G Suite, Box, Dropbox, Egnyte, and Sharefile.

Activity latency ¶

Activity data may not be provided right away. Latency prior to an update is configurable in self-hosted/private Kloudless Enterprise installations and varies in the cloud version as detailed below.

The following services notify Kloudless of changes to users’ accounts, with activity data updated in at most 1.5 minutes:

• box

• dropbox

• evernote

• gdrive

• s3

  • New buckets are detected every 20 minutes.

• smb

• salesforce

  • Up to 40 object types supported via the Streaming API.

• google_calendar

• outlook_calendar

• sharefile

• gmail

• procore

• autodesk

Other services require that we query the service repeatedly to find changes. This means that for the following services, activity data on average will be updated every 5 minutes on Kloudless’ cloud version, and every 1 minute in Kloudless Enterprise:

• File-system traversal or equivalent

  • cq5
  • hubspot
  • jive
  • salesforce
  • webdav
  • plangrid
  • bluebeam

• Polling an API endpoint or equivalent

  • egnyte
  • sharefile
    • Polling is used for non-audit type activity collected for non-admin accounts.
  • sharepoint
    • Polling is used for non-audit type activity collected for non-admin accounts.
    • For Audit activity, latencies vary between Office 365 workloads. SharePoint, OneDrive for Business, and Azure Login activity are available within minutes. See what other audit activity can be tracked in Microsoft’s full list of available audit events
  • skydrive
  • onedrivebiz
    • See sharepoint.
  • outlook_email
  • exchange_email

The CMIS-like services listed below also poll for changes. To enable more efficient activity retrieval by Kloudless, please configure the repository to create a change log if possible.

• alfresco

• alfresco_cloud

• cmis

• documentum

• filenet

Activity retention ¶

Kloudless attempts to query activity data synchronously where available. This provides the most scalable approach as it avoids Kloudless having to cache activity information. However, as described when discussing latency, not all services support API endpoints to query activity information in real-time. Kloudless caches activity data temporarily for these services.

Up to 10,000 activity objects per account over the previous 24 hours are stored temporarily in Kloudless’ database for efficient retrieval by each application.

To determine which services adhere to this limit, query the publicly available Services metadata endpoint and check the events.events.persistence property as shown using curl and jq below for box:

curl -X GET 'https://api.kloudless.com/v1/public/services/box/' \
    | jq '.properties.requirements["events.events.persistence"][]?.result'

true means that Kloudless collects activity data in the background and caches it based on the retention period. The majority of services currently require this. Examples include Amazon S3 and Alfresco.

false means that Kloudless directly retrieves activity data and doesn’t need to cache any information. In this case, the cursors Kloudless returns when listing activity are still only valid for the retention period described above. By removing stale cursors, Kloudless ensures requests to list activity that include no cursor or stale cursors automatically use a cursor closest to the maximum retention period for services that do require retention. This allows consistent behavior across services. Otherwise, services returning false would return activity data from the beginning of time whereas ones returning true would return activity data based on the retention period. Here are services that Kloudless directly queries:

• box

• dropbox

• exchange (calendar activity only)

• exchange_email

• gdrive

• google_calendar

• onedrivebiz (admin only)

• outlook_calendar

• outlook_email

• sharepoint (admin only)

The absence of a result indicates that the service does not support activity monitoring.

Activity 

Overview ¶

Actions such as creating, modifying, or deleting resources generate activity that we track. They are represented as a stream of Activity objects.

You need subscriptions to use Activity endpoints. To enable automatic subscriptions for your application, visit the Activity Monitoring Configuration page and ensure the checkbox for Track Activity is checked. This creates default Subscription objects for any accounts connected or reconnected after this time automatically. With a default Subscription, you can use the default alias for the subscription_id when accessing the Subscription and Activity endpoints.

Alternatively, you can create subscriptions by using the Subscription endpoints. Please check Creating a Subscription.

Activity Attributes ¶

Activity objects have the following attributes:

• id: The id of the Activity object. May be None if activity are directly retrieved from the upstream service, or a string if the upstream activity ID is available. Will be a number representing the ID of the activity otherwise.

• account: The id of the account the resource belongs to.

• subscription: The id of the subscription the resource is monitored by.

• modified: The time the activity occurred. This is not always precise due to the underlying service. null if unavailable.

• type Any of the supported Activity types.

• ip: The user’s IP address, if available. null if unavailable.

• metadata: The new metadata of the resource or null if the action is delete and the data is unavailable.

• previous_metadata: The metadata of the original resource prior to a rename, move, or copy. {} or absent if unavailable.

• user_id: For admin accounts, this field specifies the ID of the user responsible for the activity.

Activity Types ¶

The following types of activity are currently available for objects belonging to the specified APIs:

• add: a resource has been created.

  • Resource’s API type: storage, crm, calendar, itsm, messaging, email.

• delete: a resource has been deleted.

  • Resource’s API type: storage, crm, calendar, itsm, messaging, email.

• update: a resource has been modified.

  • Resource’s API type: storage, crm, calendar, itsm, messaging, email.

• rename: a resource has been renamed.

  • Resource’s API type: storage.

• move: a resource has been moved.

  • Resource’s API type: storage.

• copy: a resource has been copied.

  • Resource’s API type: storage.

• restore: a resource has been restored after being deleted.

  • Resource’s API type: storage.

The CRM API activity applies to all standard Kloudless CRM objects. Activity data incidentally obtained for other CRM objects is also available during the process of activity retrieval as raw object data even if the data is not converted into the unified Kloudless API format.

The most specific activity is returned where possible; otherwise, the activity specified by the underlying service will be returned.

List Activity 

curl -H 'Authorization: Bearer [TOKEN]' \
    'https://api.kloudless.com/v1/accounts/me/subscriptions/default/activity?cursor=121'

Audit Activity 

v2: Check out our v2 Audit Activity API for a comprehensive solution to track org-wide activity occurring in business applications. The initial release includes support for popular services such as Office 365, G Suite, Box, Dropbox, Egnyte, and Sharefile.

The previous v1 docs are available for reference in our v2 migration guide.

Subscriptions 

A Kloudless Subscription object enables monitoring an upstream service’s activity and receiving webhook notifications for it. Activity monitoring begins once a Subscription is created for an account.

To enable automatic subscriptions for your application, visit the Activity Monitoring Configuration page and ensure the checkbox for Track Activity is checked. This creates default Subscription objects for any accounts connected or reconnected after this time automatically. With a default Subscription, you can use the default alias for the subscription_id when accessing the Subscription and Activity endpoints.

Kloudless Subscriptions have the attributes below:

• id Unique identifier for this Subscription. default can be used as an alias for the id of the Subscription that has default set to true.

• account Account identifier this Subscription belongs to.

• expiry ISO 8601 timestamp for when the Subscription expires. Kloudless will automatically refresh active Subscriptions that are close to expiring if possible.

• last_cursor The most recent cursor for this Subscription. Can be used as the cursor when listing activity. This attribute will be left out when listing Subscriptions.

• last_cursor_updated_at ISO 8601 timestamp of the most recent time last_cursor has been updated. This attribute will be left out when listing Subscriptions.

• active Boolean indicating if the Subscription is active or not. Activity won’t be monitored for an inactive Subscription.

• disable_reason Reason that the current Subscription is inactive.

• default Boolean indicating if the Subscription is the default Subscription for this account. Setting a default Subscription helps access activity for this account without knowing the Subscription ID, by simply making a request to /subscriptions/default/activity instead. This is especially valuable when entire Kloudless applications are configured to automatically create subscriptions for all accounts connected to them. Defaults to false. Only one Subscription may be default for an account at any point in time.

• monitored_resources Optional. A list of Monitored Resources indicating which resources are monitored by this Subscription. This field cannot be updated once written. To change it, delete and re-create the entire Subscription.

  The following services currently support this feature:

  • gdrive (non-admin only)
    • Configure up to four Shared Drives to monitor.
  • google_calendar, outlook_calendar
    • Configure up to five calendars to monitor.
  • gmail, outlook_email, exchange_email
    • See the Email Activity section.
  • sharepoint
    • Configure up to four Document Libraries to monitor. If the specified ID is a SharePoint site, the default Document Library of the site will be monitored. If the specified resource is a folder, the corresponding Document Library will be monitored instead.
    • For admin accounts, if no monitored_resources is set, all Document Libraries across the entire account will be monitored by default.
  • onedrivebiz (admin only)
    • Configure up to four Document Libraries to monitor.
    • For non-admin accounts, monitored_resources is not supported. Instead, the root drive of the account will always be monitored by default.
  • salesforce
    • Configure up to 40 object names to monitor. Include *__c to monitor all the custom objects. E.g. ["account", "contact", "ContentDocument", "CustomObject__c", "*__c"]. Due to the 40 PushTopics limit, we prioritize specified objects over the ones we matched for “*__c”.
  • slack (non-admin only)
    • Configure up to 5 conversations to monitor.
  • Any services that support the crawl Subscription type. Check subscription_type below for more details.

  Kloudless will set sane defaults for this field if required and not provided. Here are the current list of default resource IDs for each service:

  • gdrive (non-admin)
    • root
  • google_calendar, outlook_calendar
    • primary
  • salesforce
    • ContentDocument
    • ContentWorkspace
    • Document
    • Attachment
    • FeedItem
    • FeedComment
    • FeedAttachment
    • account
    • contact
    • campaign
    • opportunity
    • lead
    • task

  Refer to each Unified API’s docs for more information on reserved identifiers such as root for Storage and primary for Calendar.

• monitored_resource_api Read-only. This field corresponds to the API category associated with the objects identified by monitored_resources.

• monitored_resource_object_type Read-only. The object type of objects in monitored_resources.

• subscription_type Possible values:

  • changes (default): This monitors any new activity in the account.
  • crawl: This performs an initial one-time retrieval of all file and folder metadata in the account, including across all users for admin accounts. All metadata found is published as new activity. This is currently limited to files and folders accessible via the Storage API. All services that support the Storage API support the crawl subscription type, other than Email services. See crawl_type for where the metadata found is published.
  • analytics: Activity related to analytics. Currently only used to publish open and click activity on messages sent via the Email API.
  • sync (planned): Not yet implemented. This is similar to crawl but will copy files, folders, and objects found to a location in a configured account automatically.

• crawl_type Only external is supported at the moment. Contact Kloudless to enable other values for your Subscriptions. Here are possible values:

  • external (default): The metadata found during the crawling is published to all configured webhooks, EventBridge receivers, and Google Cloud Pub/Sub topics.

• created ISO 8601 timestamp indicating when the Subscription object was created.

• modified ISO 8601 timestamp indicating when the Subscription object was last modified.

• type Always subscription.

• api Always activity.

Note: If an upstream account is inaccessible for 90 days, or subscriptions for it are unable to be renewed for that time period, the Kloudless Subscription state will be removed along with related Activity data.

Monitored Resource ¶

Monitored Resources are sub-objects configured within a Subscription object that specify the resources to track activity for, crawl, or otherwise perform the Subscription’s intended action on. A Monitored Resource has the following attributes:

• resource: The resource’s ID, reserved identifiers, or object name.

• user: Optional. For Kloudless accounts with multiple users, such as org-wide admin accounts, this specifies the ID of the user through which the resource can be accessed.

List Subscriptions 

curl -L -H 'Authorization: Bearer [TOKEN]' \
    https://api.kloudless.com/v1/accounts/me/subscriptions

Create a Subscription 

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPOST -d '{"active": true}' \
    'https://api.kloudless.com/v1/accounts/me/subscriptions/'

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPOST -d '{
        "active": true,
        "monitored_resources": [
            {"resource": "primary"},
            {"resource": "fdGVzdEBrbG91ZGxlc3MuY29t"}
        ]
    }' \
    'https://api.kloudless.com/v1/accounts/me/subscriptions/'

Retrieve a Subscription 

curl -L -H 'Authorization: Bearer [TOKEN]' \
    https://api.kloudless.com/v1/accounts/me/subscriptions/185

Update a Subscription 

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPATCH -d '{"active": false}' \
    'https://api.kloudless.com/v1/accounts/me/subscriptions/185'

Delete a Subscription 

curl -L -H 'Authorization: Bearer [TOKEN]' \
    -XDELETE 'https://api.kloudless.com/v1/accounts/me/subscriptions/185'

Webhooks 

Kloudless Webhooks are a way to be notified when new activity is available for one of the Accounts belonging to your Application.

Webhooks and similar notification mechanisms can be managed in the Activity Monitoring Configuration page. Kloudless supports three types of notification mechanisms:

• JSON Webhooks

• Amazon EventBridge Events

• Google Cloud Pub/Sub Messages

• Microsoft Azure Service Bus

Each is described in further detail below.

JSON Webhooks 

When a JSON Webhook is created, it will be tested by our servers to ensure that it works before saving it. In order for the test to “pass”, your application should respond with a 200 status code that includes just your Kloudless Application ID in the response body. The test webhook request is a POST with an empty JSON object ({}) in the request body.

Here is an example of a JSON Webhook notification from Kloudless:

POST / HTTP/1.1
Host: yourapp.example.com
Accept-Encoding: identity
Content-Length: 37
X-Kloudless-Signature: DojsMH34VH3sD+lB8sNBRwkbc7FXyCIRvEbijoCOz9I=
Content-Type: application/json
User-Agent: kloudless-webhook/2.0

{"account": 123, "subscription": 321}

The most important part of the POST request above is the body, which is a JSON object. The JSON object contains two fields:

• account The ID of the account that triggered the activity.

• subscription The subscription ID which triggered the webhook.

When your application receives a webhook, use the account and subscription IDs to list activity, along with a cursor if your application has one available from a prior response. The response to the endpoint above contains a cursor your application must store in its database.

Each time your application receives a webhook, include the latest cursor from your database in the query parameters while listing activity using the endpoint described above, and store the new cursor returned. This process guarantees your application will only receive new activity, in the order Kloudless became aware of them, regardless of the number of webhooks or whether they were all received.

Verifying JSON Webhooks ¶

Kloudless includes a signature of the HTTP request that you can use to ensure that the request came from Kloudless and is untempered with. Here is how the signature is defined:

Base64Encode(HMAC-SHA256([API Key], [request contents]))

[request contents] in the example above would be the string {"account": 123, "subscription": 321}. For applications with multiple API Keys, the API Key used in the signature will be the oldest active one available.

NOTE: Even though there is a signature, you should accept webhook requests over HTTPS if possible. This ensures that data is not leaked to or tampered with by third-parties.

JSON Webhook Retries ¶

If Kloudless doesn’t receive a HTTP response with a status code below 500 within 30 seconds (3.05 seconds for socket connection timeout and 27 seconds for read timeout), Kloudless assumes the webhook failed and attempts to retry it.

Kloudless retries webhooks with an exponential back-off. The initial time between retries is 1 second and the maximum time between retries can grow to 15 minutes. If the webhook continuously fails for 24 hours, Kloudless stops retrying until new activity is available after that time.

Amazon EventBridge Events 

Kloudless is an Amazon EventBridge Partner. This means that you can receive Kloudless Activity within your AWS environment using EventBridge, and then route the activity to AWS Lambda functions, Amazon EC2, SQS queues, or other valid EventBridge targets.

You can enable the Kloudless partner event source by searching for Kloudless within the Partner Event Sources list in your AWS EventBridge console.

Here is an example of an EventBridge event object:

{
    "detail-type": "eventbridge_webhook_notification",
    "source": "aws.partner/kloudless/your@email.com/api-id-123",
    "account": "111122223333",
    "time": "2017-12-22T18:43:48Z",
    "region": "us-west-1",
    "detail": {
        "account": 123,
        "subscription": 234,
        "activity": {
            "v1": [v1_activity_object]
        }
    }
}

AWS wraps the Kloudless event payload within detail in an outer object that includes metadata about the EventBridge event. Here is a description of the fields Kloudless includes in the detail object:

• account The ID of the account that triggered the activity.

• subscription The subscription ID which triggered the webhook.

• activity

  • v1 The activity data in Kloudless’ v1 API format. See the v1 Activity object attributes for details.

For more information on the top-level fields other than detail, please check AWS’ EventBridge Docs.

Filtering Events ¶

AWS EventBridge Event Patterns can be used to filter EventBridge events.

For example, if you only want to receive activity where the Kloudless v1 type attribute is add, you can use the following EventBridge pattern to filter for just that activity:

{
    "detail": {
        "activity": {
            "v1": {
                "type": ["add"]
            }
        }
    }
}

Google Cloud Pub/Sub Messages 

You can configure Kloudless to publish Activity to a Google Cloud Pub/Sub topic as messages, and then consume those messages via a Google Cloud Pub/Sub subscription. Please refer to the Google Cloud Pub/Sub section on the Events configuration page of the Developer Portal for more information on how to set up Google Cloud Pub/Sub notifications.

Here is an example of a Pub/Sub message Kloudless could send:

{
    "account": 123,
    "subscription": 234,
    "activity": {
        "v1": [v1_activity_object]
    }
}

The message data contains the following fields:

• account The ID of the account that triggered the activity.

• subscription The subscription ID which triggered the webhook.

• activity

  • v1 The activity data in Kloudless’ v1 API format. See the v1 Activity object attributes for details.

Microsoft Azure Service Bus 

You can configure Kloudless to publish Activity to an Azure Service Bus topic as messages, and then consume those messages via an Azure Service Bus subscription. Please refer to the Microsoft Service Bus section on the Events configuration page of the Developer Portal for more information on how to set up Azure Service Bus notifications.

Here is an example of a Service Bus message Kloudless could send:

{
    "account": 123,
    "subscription": 234,
    "activity": {
        "v1": [v1_activity_object]
    }
}

The message data contains the following fields:

• account The ID of the account that triggered the activity.

• subscription The subscription ID which triggered the webhook.

• activity

  • v1 The activity data in Kloudless’ v1 API format. See the v1 Activity object attributes for details.

Session Topic ¶

We support message sessions for ordering and grouping events based on the per-account subscription. Check Enable sessions in the Create Subscription page to enable session based subscription. After creating the subscription, you can receive messages with a specific SessionId which corresponds to the ID of the Activity Subscription).

If you are using Azure Functions, ensure that isSessionEnabled to True for the session subscription. Learn more from Azure Service Bus trigger for Azure Functions.

Topic filters and user-defined properties ¶

You can also choose topic filters to filter messages by session id or our custom properties / the following fields:

• SessionId system properties, corresponding to message field subscription.

• accountId The ID of the account that triggered the activity.

• v1ActivityType Any of the types described in Activity types.