Kloudless Release Notes

# Release Notes

The previous year or more of release notes are included below. Please contact support@kloudless.com for release notes for older releases or any assistance with upgrades.

Versions are of the form Major.Minor.Patch. Each minor version increment includes security updates to system packages where necessary. Patch version increments are pure application-code updates.

# 1.33.3 (2021-02-17)

# Bug Fixes

  • The Office 365 Graph API's Permissions retrieval endpoint unexpectedly changed its response format, breaking Kloudless API requests to retrieve permission data. This release includes a patch to work with the new format returned by the Graph API. Specifically, Graph API responses don't include the @odata.id attribute now.

# 1.33.2 (2021-02-08)

# Compatibility Notes

  • Event endpoints: The legacy /events endpoints present in version 2 have been removed.

# New Features

  • Salesforce: Now supports activity monitoring by API object types.
  • Email API:
    • Exchange Email:
      • Now supports listing folders.
      • Now supports activity monitoring by mailboxes for admin.
      • Now supports sending email attachments.
    • Outlook Email:
      • Now supports activity monitoring for admin.
      • Now supports monitoring sent and inbox folders for admin.
      • Support or Skip Team API - list groups and group members.
      • Now supports CUD on Labels.
    • Gmail:
      • Now supports activity monitoring by mailboxes for admin.
      • Now supports CUD on Labels.
      • Now supports sending email attachments.
    • IMAP: Now supports sending email.
  • SMB: Now supports setting permissions.
  • OneDrive: Now supports custom scopes.
  • Exchange: Improvements to authentication flow.
  • Unified Accounting API release:
    • QuickBooks connector. Added sandbox mode and API keys guide.
    • Zoho Books connector.
  • Meetings API: Now supports editing scheduled events.
  • API Explorer: Allows input appId manually to connect new accounts for the "External App".
  • Helpdesk: Supports Freshdesk.
  • BIM360: Supports download files with "Document" type.
  • Slack: Activitity monitoring filtering for certain channels/chats.
  • Pgbouncer version update.
  • Contacts API: Now supports CUD on Contacts for Outlook Calendar.
  • S3: Improve configurability for Transfer Acceleration.

# Bug Fixes

  • Gmail: Resolved header parsing issue.
  • Resolved issue with code grant flow not creating default subscription.
  • Slack: Resolved admin scope issue.
  • Subscriptions API: Improve cursor format to be consistent with docs.
  • O365: Resolved v1 audit event type regression.
  • O365: Resolved share link event regression.
  • API performance improvements.
  • OneDrive for Business: Resolved unable to list root folder recursively issue.
  • Salesforce: Resolved caching issue with retry requests.
  • Error response body improvements.
  • File Picker: Improvements when browser disables third party cookies.
  • Meeting Scheduler: Resolved issue where UI shows duplicate slots.
  • Activity monitoring: Resolved issues with json strings for various services.
  • Outlook Email: Resolved str/dict issue with monitored_resources.
  • Outlook Calendar: Resolved activity monitoring issue due to invalid timestamp parsing.
  • Subscriptions API: Fixed issue to return string cursors that support synchronous events.
  • Concurrent token refresh impovement with distributed locking.
  • Sharepoint: Resolves activity issue to handle folder IDs better for non-admin.
  • Stats: resolved leak of socket when statsd metrics backend is used.
  • Improved caching for permission checking.
  • Docs: Updated activity monitoring docs for Google and Outlook Calendar.
  • Guides: Improve documentation for the Microsoft app registration process.
  • Docs: Updated webhook-related documentation.
  • Slack: Adaption to Slack's deprecation on authed_users for activity monitoring.

# 1.33.1 (2020-12-18)

# Bug Fixes

  • Resolved compatibility issue with default subscription creation
  • Resolved issue with Collaboration object parent ID encoding.
  • Addressed change to Sharepoint Online and OneDrive for Business audit event type handling.

# 1.33.0 (2020-11-23)

# Compatiblity Notes

  • Customers setting custom License Key file paths via the license attribute in /data/kloudless.yml should instead use the license_code attribute to ensure the license file is automatically fetched. The license attribute is no longer supported. This change does not impact customers with license keys already present at the default location, /data/license.key.

# New Features

  • Gmail: Activity monitoring now includes full object metadata in received events.
  • Email API: Authenticate admin accounts for org-wide access to data in Gmail, Outlook Email, and Exchange Server. User impersonation is supported.
  • SMB: The owner SID is now included in file and permission metadata.
  • Salesforce: Activity monitoring now prioritizes important fields in scenarios where only a limited set of attributes can be returned.
  • File picker: Cross-account transfers can now be routed to independent regions.
  • The Account object now includes a last_request attribute representing the timestamp of the last API request to the account.
  • SAML authentication now available for Single Sign-On using services like OneLogin. In addition, the authentication options available can be customized to any combination of password auth, Google login, GitHub login, and SAML.
  • Accounts can now be programmatically deleted with permanent=True to support GDPR compliance requests programmatically rather than manually.

# Bug Fixes

  • Egnyte: The domain field in the OAuth flow can now correctly be pre-filled.
  • FTP: Unicode support improved.
  • Exchange Email: Folder metadata retrieval avoids certain transport errors.
  • Office 365: Certain types of download events are now parsed correctly during Activity Monitoring.
  • Salesforce: Page offsets are now correctly caclulated when paginating through a large number of objects in scenarios where Salesforce returns less objects than requested.
  • Azure Service Bus: Retries introduced to handle Azure connectivity failures.
  • Statistics: Unnecessary database queries have been removed to improve performance for deployments with over 1M connected accounts.

# 1.32.9 (2020-10-28)

# New Features

  • Xero: The Kloudless Accounting API and Data Mapper now support the new Xero connector.
  • Email API
    • Click and Open tracking is now available for email messages sent via the Email API.
    • Labels can now be supported on Email Messages
  • Sharepoint Online: monitored_resources can now be configured for Subscriptions on non-admin accounts to monitor activity from a customized list of drives, sites, and site collections.
  • A new dropdown is now available in the Authentication flow UI for easier tenant & team selection.
  • Pipedrive: The Pipedrive connector is now supported in the CRM API.
  • Sage: The Kloudless Accounting API and Data Mapper now support Sage.

# Bug Fixes

  • Google Services:
    • Improvements to error handling when hitting retry limits.
    • Google Drive: Multipart uploads to a non-existent folder now correctly return a 404 response status code rather than 401 or 403.
  • Gmail: Unknown or duplicate activity is now skipped.
  • File Picker: Improvements to retry requests when encountering rate-limits.
  • Improvements to OAuth error handling.

# 1.32.8 (2020-10-16)

# Bug Fixes

  • Kloudless Connect: Proxy Connection records now will be deactivated instead of entirely deleted when Proxy Connections disconnect or an associated account is deleted.

# 1.32.7 (2020-10-09)

# New Features

  • Slack: The Slack connector now supports listing conversation members.
  • Hubspot: Introducing Activity Monitoring support for Hubspot, to monitor changes to objects.
  • Calendar: Calendar Event objects now support conferencing details. Add links to Google Meet, Hangouts, MS Teams, or similar to calendar events created in Google or Outlook Calendar.

# Bug Fixes

  • Google Drive: v1 Activity API removed.
  • Construction API: Improved checks around token refreshes to ensure refreshed tokens are persisted in the event of DB failures.
  • Google Calendar
    • Resolved issues preventing user impersonation in some cases
    • Improving errors returned for missing calendars with the Availability API endpoint.
  • Outlook Email: Support monitoring non-English mailbox folder names.

# 1.32.6 (2020-09-22)

# Compatibility Notes

  • External Queue Webhooks: v2 formatted events are no longer published in the activity published to Amazon EventBridge, Google Cloud Pub/Sub, or Azure Service Bus, as they provide redundant information.

# New Features

  • Accounting API: QuickBooks Online is now available via the Data Mapper.
  • Data Mapper: Enable your users to alter and extend how data is mapped from the upstream cloud services to your application. Supported for the CRM and Accounting Unified APIs.
  • Generic Object API: Enables CRUD operations on any standard or custom object available in the upstream service, regardless of whether Kloudless unifies that object's type. This API is currently available for CRM and Accounting connectors.
  • Microsoft Teams: The team name is now included in the conversation name to distinguish conversations with the same name in different teams.
  • Procore: Support for activity monitoring with non-admin accounts.
  • Task API: The Task API now includes progress information for asynchronous folder copy tasks.
  • Gmail connector improvements:
    • Supports the Pass-Through API.
    • Supports custom raw scopes.
    • Supports listing labels.
  • Outlook Email: Supports listing labels and folders.

# Bug Fixes

  • Exchange: Admin accounts are now able to correctly monitor specific calendars with the monitor_resources parameter. The primary alias was previously not resolved correctly.
  • Kloudless Crawler: The crawler will now properly handle an admin account without any other users.
  • API: Improved handling of UTF-16 surrogates in upstream responses.
  • Outlook Email: Resolved issue with listing activity for a subscription with monitored resources.
  • SharePoint Online: Improved handling of users with apostrophes in email addresses.
  • iCloud Calendar: Resolved issues with Availability API.
  • Google Calendar:
    • X-Kloudless-As-User header is now handled as expected.
    • Find Availability API now returns 404 if the calendar ID is invalid.

# Security

  • Developer Portal: password requirements are now more strict.

# 1.32.5 (2020-09-02)

# New Features

  • Kloudless Crawler: Added support for the following services: azure, evernote, ftp, hubspot, salesforce, smb, and webdav.
  • Zoho now includes support for generic CRUD operations on objects.
  • File Picker
    • Error dialogs now have a copy button. Error text is now selectable.
  • Kapost: Now supports monitoring individual resources for activity.
  • Autodesk BIM 360: Multipart uploads are now supported.
  • Salesforce:
    • The Salesforce Files Storage API folder structure now matches the Salesforce Lightning UI, and also supports Salesforce Libraries.
    • Raw response headers are also now returned along with other raw data.

# Bug Fixes

  • AEM/CQ5: Activity Monitoring performance improved, along with some bug-fixes for missing activity resources.
  • S3: Buckets in the EU region are now correctly monitored for activity.
  • Kloudless Activity v2 API: event_type and event_subtype will now be more accurate and consistent in various supported connectors.
  • OneDrive for Business: An issue with tokens not refreshing correctly in some circumstances was resolved.
  • Outlook Calendar: Erroneous 'event deletion' notifications will no longer be sent for new or modified recurring series whose instances cross DST boundaries.
  • Outlook Mail:
    • Fixed an issue where email query results were missing if no before or after parameters were provided.
    • Email query results are now correctly sorted correctly from latest to oldest.
  • Exchange Email: Successful API requests to send email now always return a successful response even if the sent mail was not immediately locatable.
  • File Picker
    • The Saver now triggers error events correctly, and also refreshes the displayed folder if an upload successfully completes.

# 1.32.4 (2020-08-12)

# New Features

  • A new and improved API Explorer is now available.
  • Azure Service Bus: The Session ID is now used to group events by Account ID when published to an Azure Service Bus topic.
  • Microsoft Teams: Support for retrieving attached files via the Storage API.
  • Task API: Asynchronous URL upload tasks update task state with upload progress.

# Bug Fixes

  • SharePoint Online: Improved support for non-English document libraries.
  • Raw ID Encoding: All Kloudless unified types can be used in the type field when encoding raw object ids.
  • Google Drive: File and folder metadata now includes raw upstream data.
  • Google Drive: Fixed issue that prevented an Admin account from uploading and downloading files.
  • Dropbox: Shared folder IDs can now be used with the get and set permissions endpoints.
  • Docker: Fixed issue where stdout/stderr container application logs would halt after running for more than a day.
  • Docker: Increased internal buffer sizes to handle extremely long file names and URLs.
  • Kloudless Connect: Improved detection and handling of dropped remote proxy connections.

# 1.32.3 (2020-07-29)

# New Features

  • Alibaba Object Storage Service is now supported in the Unified Storage API.
  • Kloudless retries JSON Webhooks for up to one week if successful responses are not received.
  • Raw upstream OAuth scopes can now be configured via the Custom OAuth Credentials page of the developer portal for services such as Google Drive that require any scope customizations to be included within the OAuth request itself. Previously, this needed to be set in application code when the Kloudless OAuth flow to the service (Google Drive in this example) was launched.
  • Google Drive: The Changes API can now be configured system-wide, if tracking changes to items shared with a user is required for non-admin accounts.
  • Exchange Calendar: Monitoring activity on individual calendars in admin accounts is now supported.
  • Salesforce: A configurable threshold now prevents any Kloudless-powered application from being the cause of the tenant's daily API request quota being completely consumed.
  • CalDAV and iCloud Calendar now support recurring events.

# Bug Fixes

  • Gmail: Activity Monitoring now correctly creates the initial subscription.
  • SFTP: File uploads encountered issues in 1.32.0 and above, which is now resolved.
  • Exchange Calendar: Calendar body data is correctly set to HTML where required.
  • Office 365: Improvements to importing Graph API tokens for non-admin SharePoint and OneDrive for Business accounts.
  • Outlook Calendar: Querying the default calendar's availability without explicitly specifying the default calendar ID now works as expected.
  • Google connectors now correctly cache discovery API endpoint information to avoid rate-limiting.

# 1.32.2 (2020-06-29)

# New Features

  • Kloudless now supports receiving notifications via Azure Service Bus in addition to Amazon Eventbridge, GCP Cloud Pub Sub, and webhooks.
  • The Kloudless OAuth authentication flow UI pages have been updated with a refreshed design.
  • Outlook Calendar and Google Calendar
    • Org-wide activity monitoring is now supported in a consistent format for both connectors by using per-user activity subscriptions. Configuring monitored_resources is also supported.
    • The q query parameter can be used to search on calendar events.
    • Retrieve individual instances of a recurring series by using the series master's calendar event ID in the instances query parameter.
    • The new on_organizer_calendar attribute indicates if a calendar event is on the organizer's own calendar, or if the organizer is a user different than the one whose calendar this event is on.
  • SMB: The v2 Permissions API now supports extended raw data including information on the bit-flags enabled in the Access Control Entry for the AceFlags and Access Mask attributes.
  • The Alibaba Cloud OSS connector is now available and supports the Storage API.

# Bug Fixes

  • Salesforce: Further rate-limiting improvements to cache known object types further.
  • Evernote: Images rendered inline in downloaded files now render correctly.
  • Box webhooks (non-admin) are now aware of data visible to the authenticated user modified by a different user.
  • Outlook Email: Overriding the default from value when sending email messages is now handled correctly.
  • Slack (admin): The Discovery API endpoints have been updated to better support the latest version available.

# 1.32.1 (2020-06-09)

# Bug Fixes

  • Resolved connectivity issues when using the Kloudless Connect proxy that were introduced in 1.32.0.

# 1.32.0 (2020-06-05)

# New Features

  • The Kloudless Unified OAuth flow pages now have an updated design.
  • Admin Portal Improvements
    • The Kloudless Admin Portal that is accessible via the web UI to admins now enables a bearer token to be obtained to any connected account.
    • A new Log Entries section logs admin activity of the type above.
  • Kloudless Crawler: The crawler subscription type enables an initial one-time crawl and retrieval of all file and folder metadata in a Storage connector. For admin accounts, this includes data in all users within the tenant. Contact us for more information on getting started with the Kloudless Crawler's capabilities.
  • Subscriptions API: Activity data can now be retrieved using Amazon Eventbridge directly into your stack rather than waiting for webhook notifications and querying our API.
  • Google Calendar: Admin accounts now monitor all calendars in the G Suite organization for activity by default, rather than only the admin's primary calendar or in Subscription.monitored_resources.
    • Several additional activity types that represent activity on calendar objects and events have been added as well, such as attendees being added or removed, or calendars themselves being modified.
  • CalDav and iCloud Calendar: The start_time_zone and end_time_zone calendar event attributes are now supported.
  • The Account object now supports a proxy_connection attribute representing the Kloudless Connect Proxy Connection that the Account was connected through (see Meta API to retrieve Proxy Connection details).
  • Storage API: Kloudless now simulates the overwrite query parameter during binary file uploads for services that don't support the value natively. This release adds support for the only known services of this nature so far, which are PlanGrid, Procore, and Bluebeam Studio.
  • Office 365
    • The FileSyncDownloadedFull and FileSyncUploadedFull activity types are now supported for SharePoint and OneDrive for Business admin activity.
    • Raw Team User and Group data now includes the Graph API proxyAddresses attribute.
  • File Picker
    • Files that can't be downloaded are now disabled when copy_to_upload_location is configured.
    • Files displayed can be filtered by MIME type in addition to file extension
    • The root_folder_id can now dynamically be updated.

# Bug Fixes

  • Outlook Calendar
    • Custom time zones are now correctly ignored during calendar event creation.
    • Fixed bugs when retrieving, updating, and deleting calendar events in the calendar of another user or resource in the tenant. This applies most frequently when using admin accounts.
  • Outlook Email: Prioritize an HTML body over a text body when sending email.
  • Exchange Server: Raw ID encoding now works correctly for Team User IDs.
  • G Suite
    • Kloudless only requests necessary scopes to identify the user during the Admin OAuth flow's identity validation step.
    • Admin activity without an actor's email or other identifiers are now parsed correctly.
  • OneDrive for Business: Personal Site URLs are now double-checked during user impersonation in case a user's email address doesn't match their personal site's URL format.
  • S3: Kloudless correctly responds with a 403 if access is denied.
  • Box: Multipart uploads now work with overwrite=True.
  • Kapost: The authentication flow now supports full domains, such as those ending in .kapost.com.
  • FTP: Servers that require the data channel to be multiplexed over a single TLS connection are now correctly connected to.
  • Autodesk BIM 360
    • The latest file version is now overwritten correctly when overwrite is specified during a file upload.
    • Errors while paginating through folders are now resolved.
  • Salesforce: Several performance optimizations included to reduce the number of unnecessary API calls made in the background during activity monitoring.
  • Subscriptions API: The subscription_type is now correctly set to change instead of resource-change.
  • File Picker
    • Improvements to success/error handling for Computer uploads and pagination when browsing files in cloud storage.
  • Database: Resolving an issue where larger installations with over 800,000 connected accounts may have database keys not roll over appropriately.

# Security

  • The Kloudless Admin Portal that is accessible via the web UI to admins now enables a bearer token to be obtained to any connected account. Please ensure that only admins trusted with full access to the system are made admins via the web UI or system.
    • A new audit log regarding this type of admin activity is now also visible under "Log Entries" in the Admin Portal. Since the log entries are stored in the database, admins with access to the Kloudless system or database may still modify these log entries or directly obtain and decrypt access tokens.

# Compatibility Notes

  • The Messaging API is now known as the Chat API but is otherwise unchanged and entirely backwards-compatible. The api attribute on objects in the Chat API remains messaging for backwards compatibility.

# 1.31.6 (2020-05-28)

# Bug Fixes

  • Office 365: Certain Azure AD OAuth key configurations may not have access to appropriate rate-limiting headers introduced in 1.31.3, causing requests to SharePoint and OneDrive for Business to fail. This regression is resolved in 1.31.6 and further optimizations in 1.32.0 onwards will account for various key configurations.

# 1.31.5 (2020-05-11)

# Compatibility Notes

  • Please update to 1.31.6, for that version's Bug Fixes.

# New Features

  • PowerShell: The PowerShell connector now supports Kloudless Connect.

# Bug Fixes

  • Office 365
    • File transfer requests now include headers to specify dedicated pools for rate-limits.
    • Listing users and groups now includes proxyAddresses in the raw data.
    • OneDrive: Path information for activity occurring in the root site is now parsed correctly.
    • Outlook Calendar: Activity for recurring calendar events now only presents the main event modified rather than all instances in the series like Microsoft provides.
  • Kapost: Resolving errors parsing unicode file names in activity.
  • Procore: Image uploads are now categorized correctly, fixing an earlier regression in 1.31.3 that occurred when optimizing uploads for very large files.

# 1.31.4 (2020-04-20)

# Compatibility Notes

  • Please update to 1.31.6, for that version's Bug Fixes.

# New Features

  • Unified Email API
    • The Kloudless Unified Email API is now available for Gmail, Outlook, and Exchange Server 2013+.
    • List and retrieve all mail and attachments from a user's mailbox.
    • Monitor for any new mail that arrives
    • Connect both regular user accounts as well as admin accounts for org-wide access to data.
  • Subscriptions API: Activity data can now be retrieved using Google Cloud Pub Sub directly into your stack rather than waiting for webhook notifications and querying our API.
  • CalDav: All-day events are now supported.
  • Kapost: Activity monitoring is now available, in both v1 and v2.
  • Egnyte: File metadata now supports the created attribute.

# Bug Fixes

  • Autodesk: Resolved certain parsing failures with activity timestamps.
  • Procore: Fallback to only listing Projects if lacking Company access.

# 1.31.3 (2020-03-25)

# New Features

  • Exchange: Listing events in resource calendars is now supported.
  • Zendesk (alpha): An alpha release of our Zendesk connector is now available for CRUD operations on tickets and comments. Contact us to learn more.
  • Procore: Large file uploads now use a direct file upload API endpoint for improved reliability and thoughput.
  • Authentication
    • The Tokens API endpoint now supports deleting all tokens other than a single specified one. Combined with our Token Exchange API, this can be used to ensure only a single known token has access to an account.
    • Client Secrets are now available to use in the Authorization Code Grant flow. API Keys are discouraged for this purpose as they allow more access than is needed for Client Secrets.

# Bug Fixes

  • IDs for resources such as files and folders are now encoded consistently across different accounts, for all newly connected accounts.
  • Box: The overwrite query parameter is now handled better without 409 conflict errors during single file uploads. Multipart uploads will be improved in a future release.
  • Office 365: Improvements to parsing admin activity related to users being added to or removed from groups.
  • Calendar API
    • Improved handling of unicode data in Calendar Event attributes when creating calendar events.
    • CalDav: Improved error handling when the upstream server is unavailable.
    • Outlook Calendar: Improved contact phone and address parsing.
    • Exchange: All users' primary calendars can now be retrieved by listing users, without requiring user impersonation.

# 1.31.2 (2020-03-11)

# New Features

  • Kapost: The Storage API now supports the Kapost connector.
  • Google Drive
    • Recently modified files are now retrieved in real time.
    • Jamboard files can now be downloaded.
  • Outlook Calendar: Calendar events can now be created on other users' primary calendars without requiring admin account connectivity. Retrieve a user's primary calendar ID using the List Users API endpoint.
  • PowerShell: We now support a PowerShell proxy connector for arbitrary PowerShell commands to remote computers. This is helpful when attempting to access Active Directory, for example. This is disabled by default for security reasons. Please contact us for more information.

# Bug Fixes

  • Google Drive: The Content-Type header is now set correctly when downloading non-standard files like Google Docs or Sheets.
  • Meta API: Improved searching and filtering for Accounts, Remote Networks, and Proxy Connections.
  • Tasks API: Improved support for asynchronous requests in clustered deployments.

# 1.31.1 (2020-02-21)

# New Features

  • Exchange, Outlook Calendar: The Team API to list users and their primary calendars is now supported.
  • Zoho: The updated v2 Zoho API is now supported.
  • Outlook Calendar: Listing events in resource calendars is now supported.
  • Calendar API: The endpoint to list calendar events now supports an instances query parameter to list either the master event in a series or all instances in it, when listing calendar events.
  • Authentication: Use the on_error option to skip dispalying the default page to retry authentication on errors. Refer to the Authentication Docs for more information.
  • Infrastructure Improvements
    • A second API server process has been introduced to avoid temporary downtime while restarting processes on the server.
    • Redis instances can now be accessed over SSL/TLS.
  • UI Tools
    • The File Explorer has been renamed to the File Picker, and its UI refreshed.
    • The Meeting Scheduler now supports custom themes as well as configurable visibility and default values for input fields.

# Bug Fixes

  • Exchange: Support connecting user accounts for domains other than the Exchange host's.
  • Construction: Improved handling of events for PlanGrid and Bluebeam, and webhooks for Autodesk (BIM 360).
  • Box: Improved guarantees that event retrieval begin at the time of account connection.
  • Google Drive: Multi-part uploads now support the overwrite query parameter correctly.
  • Procore: The Storage API no longer requires the Company Documents permission to access Project Documents.
  • SharePoint Online: Skip displaying internal document libraries like Site Pages, to mirror the latest web UI better.
  • OneDrive for Business: Search queries for non-admin accounts now uses alternate API endpoints to find details.

# 1.31.0 (2020-01-22)

# Update Procedures

# Google Drive: Activity Monitoring for non-admin accounts

Google is deprecating use of the v1 Drive Activity API and requires migration to the new v2 protocol, which requires new authentication scopes.

If you are using Activity Monitoring for non-admin Google Drive accounts, please either follow these steps to enable access to the v2 Activity API before updating to this version, or contact us to configure your license keys to continue using the v1 API while Google still makes it accessible (shutdown timeline unclear beyond Dec 31 2019).

Enable the Drive Activity API for the Google developer project that owns the keys you've configured Kloudless to use to authenticate users by navigating here.

Edit your developer app and add the scope https://www.googleapis.com/auth/drive.activity.readonly (Drive Activity API). Don't remove any scopes already verified (i.e. with the green check mark). Save and submit your application for verification, with a reason for why tracking changes is important for your application.

Once your application is verified, you can safely update to this version. Kloudless will automatically begin using the new v2 scope when authenticating new user accounts.

Existing users will need to re-authenticate at some point in the future if their Google Drive activity is being accessed via the older API. Unfortunately, there is no way to automatically transition previously authenticated users to the newer v2 Activity API. Google hasn't provided a hard date that the v1 endpoint will no longer be accessible after, although it will likely be some point in 2020.

Customers with paid Kloudless subscriptions can also add customer-apps@kloudless.com as a project owner in GCP IAM to have us assist with some of these steps for you, including contacting Google for app verification.

# New Features

  • Calendar: New instance query parameter to toggle listing specific instances of repeating events or just the primary event.
  • Exchange: Support for listing available time slots.
  • Google Drive: recent files no longer requires background event collection.
  • Google Drive: Allowing configurable v1 or v2 drive activity API scope.
  • OAuth: Option to skip Kloudless confirmation page on failed authentication and redirecting to url specified in on_error parameter.
  • Oauth: Improve UI of Oauth services list page.
  • Outlook Calendar: Improve Activity Monitoring for past events.
  • Outlook Calendar: Set the 'start' and 'end' with new Microsoft limitation during listing calendar events.
  • OneDrive: Use the Graph API.
  • Procore: Storage API no longer requires Company Documents permission to access Project Documents.
  • SharePoint, OneDrive Business: Support user/group add/delete events.
  • SharePoint: Listing sites now supports pagination.
  • SharePoint: Supports ability to monitor specific resources in subscription.
  • SharePoint, OneDrive Business: Support for listing links in the v2 API.
  • Storage: Add href attribute to all api objects.
  • Activity monitoring: Automatically activate expired subscription after account re-connection.

# Bug Fixes

  • Box: Improved handling of account refresh during large numer of concurrent requests.
  • Box: Improve v2 activity support for collaboration events.
  • Box, Dropbox, Google Drive: Errors caused by storage limits being exceeded will return 507.
  • Dropbox: Can't get shared folder metadata by using the shared folder ID from audit activity.
  • Fix account imports fail with a 500 error when using invalid service names.
  • Google Drive: V3 API for non-admin activity now uses drive.files.watch instead of drive.changes.watch.
  • Hubspot: Enforce maximum page size of 250 for some objects.
  • Outlook Calendar: Improved handling of intermittent failures during event creation.
  • Outlook Calendar: Fix push Notification.
  • Oauth: some pages doesn't show the app's logo and name.
  • OneDrive Business: Fixed search for OneDrive for Business.
  • SharePoint: List folder contents now only displays document folders for non-admin to reflect expected user experience.
  • SharePoint: Fix update permissions failed when this user is external user, and role is empty string.

# 1.30.2 (2020-01-03)

# New Features

  • Docker: New JSON logging format, enabled by default. One or both formats may be used. Configure in kloudless.yml by listing desired formats under logging.nginx_format. For example, to enable both, use:
    logging:
  nginx_format:
  - json
  - legacy

# 1.30.1 (2019-12-10)

# New Features

  • Docker: Custom crontab in kloudless.yml allows easy renewal of Lets Encrypt certificates in test environments.
  • Support for accessing SFTP via Storage API.
  • Introduced ical_uid attribute to Calendar API Event objects.
  • Google Drive: Support activity collection with V2 Activity API.
  • Google Drive: Enhanced support for passing raw query parameters to API requests.
  • Calendar: Support for accessing co-worker calendars via the Team API.
  • Exchange: Support for Activity API.

# Bug Fixes

  • Outlook Calendar: Improved activity monitoring of events scheduled in the past.

# 1.30.0 (2019-11-26)

# New Features

  • Subscription and Activity endpoints now support the default alias to simplify retrieving event data. /subscriptions/default/activity is equivalent to the older /events endpoint, but is clearly distinct from the Calendar Events API.
  • Storage API: Folder objects show support for recursive listing via can_list_recursively attribute.
  • Batch upstream endpoint: Throughput and latency improvements for Google Calendar, Outlook Calendar, and SharePoint Online.
  • SharePoint (admin): Request permission required for inspecting managed site contents.
  • SharePoint: Support for importing single tokens for SharePoint Online.
  • iCloud Calendar: Dedicated iCloud connector with improved user experience.
  • Outlook Calendar: Improved handling of page size when retrieving events.
  • Exchange Server (2013-2019): features of calendars and events.
  • Office 365: Support parsing of add & delete events for users & groups.
  • Box: Support more attributes in folder metadata.
  • Google Drive: further support for "raw" pass-through parameters.
  • Google Drive service keys do not support activity v2 API.
  • Custom CA certificates can be specified in kloudless.yml. For example:
    pki:
  certificates:
    staging: |
      --- BEGIN CERTIFICATE ---
      PEM ENCODED
      --- END CERTIFICATE ---
    prod: |
      --- BEGIN CERTIFICATE ---
      PEM ENCODED
      --- END CERTIFICATE ---

# Compatibility Notes

  • Docker: unprivileged containers (formerly in the k8s registry) are now in the main prod registry. The privileged containers relying on systemd are no longer supported. The new docker command for running containers is:
    docker run -d \
  --name kenterprise \
  --env KLOUDLESS_CONFIG="$(cat kloudless.yml)" \
  --tmpfs /run  --tmpfs /run/lock  --tmpfs /tmp \
  --volume /sys/fs/cgroup:/sys/fs/cgroup:ro \
  --ulimit nofile=1024000:1024000 \
  --sysctl net.ipv4.ip_local_port_range='1025 65535' \
  # [ports,/data volume|db config] \
  docker.kloudless.com/prod:1.30.0  docker_entry
  • Docker: Containers >=1.30.0 will no longer be downloadable direclty from the Enterprise Portal. Instead they will need to be pulled from the private Kloudless registry. Please refer to the Downloads page for further details.

# Bug Fixes

  • Meeting Scheduler: Improved performance when loading time slots.
  • SugarCRM: Improved Account object parsing.
  • Office 365: Improve consistency of path IDs.
  • Outlook Calendar: Improve event retrieval performance.
  • Outlook Calendar: Improved handling of event creation errors.
  • Outlook Calendar: Improved handling of webhooks from the upstream service.

# 1.29.18, 1.28.32 (2019-11-26)

# New Features

  • Box: v1 collaboration event objects now include the file/folder ID, similar to v2 events.
  • SharePoint Online: Support for user and group add/delete events.

# 1.29.17, 1.28.31 (2019-11-14)

# New Features

  • Storage API: Add the can_list_recursively attribute to folder metadata.
  • SharePoint Online: Listing site collections no longer requires the AllSites.FullControl Graph API permission, allowing for read-only access to SharePoint Online sites.

# Bug Fixes

  • SharePoint Online and OneDrive for Business
    • The path ID type is now more consistent.
    • Microsoft introduced a regression in the Permissions API that is now resolved.
    • Owner and creator attributes are now successfully parsed for a larger number of files.
  • Improvements to prevent using a stale access token to access upstream APIs in some cases.
  • Egnyte: Improved mappings for event types and sub-types, especially in v2.

# 1.29.16.1 (2019-10-29)

# New Features

  • Support for additional allowed hostname values in kloudless.yml. For example:
    extra_allowed_hosts:
   - kloudles-internal-api
    Note: This dores not replace hostname and internal_hostname. It should only be used if there are additional hostnames that need to be able to accept API requests.

# 1.29.16 (2019-10-22)

# Bug Fixes

  • Improvements to handling large single request uploads (>150MB)

# 1.29.15, 1.28.30 (2019-10-22)

# New Features

  • Ability to override salt grains via Docker environment variables SALT_<GRAIN_NAME> (e.g. setting SALT_NUM_CPUS=4 will ensure that configuration management tools consider the system as having 4 CPU cores even if deployed on a host with more).

# Bug Fixes

  • Box: Improved handling of multipart upload part hashing.

# 1.29.14, 1.28.29 (2019-10-03)

# New Features

  • New Meta API endpoint for Remote Networks (Kloudless Connect).

# 1.29.13, 1.28.28 (2019-10-03)

# New Features

  • Outlook and Google Calendar: Listing events now supports an instances query parameter to control whether series master or instance objects are returned for recurring series of calendar events.
  • Calendar Event objects now support a new attribute, ical_uid that is consistent for individual events across all calendars the event appears in.
  • Hubspot: The CRM API now supports Email Campaigns.
  • Egnyte: The user's email address is now set as the account identifier.

# Bug Fixes

  • Non-subscribed calendars are now supported in availability queries.
  • Box: multi-part uploads now correctly abort requests.
  • SharePoint: The last modifier attribute is now accurate when querying folder contents recursively.
  • OneDrive: Account authentication now always prompts users to select their account or log in again.
  • WebDAV: Blank attributes now correctly handled
  • Google: Support added for the latest TLS certs used by Google's load balancers.

# 1.29.12, 1.28.27 (2019-10-03)

# Compatibility Notes

  • OneDrive for Business: The Documents folder is now the folder root rather than a sub-directory of the personal site collection, which was the previous root. This represents the user experience more accurately and also reflects the newer Graph API structure.

# Bug Fixes

  • Google Drive: Improved handling of login events.

# New Features

  • Sharepoint 2013: Support for system accounts
  • OneDrive for Business: Support for read-only scopes.

# 1.29.11, 1.28.26 (2019-09-30)

# Bug Fixes

  • SharePoint Online: Improved listing site groups
  • SMB: Improved error responses in cases where upstream is unreachable.
  • Docker: Improved handling of empty custom field in logging configuration.

# New Features

  • SharePoint Online: Support for listing group sites using either admin permissions or application permissions granted by admin.
  • Support for cross-account folder copy
  • Support for Google Drive Activity API v2

# 1.29.10, 1.28.25 (2019-09-19)

# Bug Fixes

  • OneDrive for Business/SharePoint Online: Improvements to special character handling in file/folder creation.
  • Improvements to searching Sharepoint Online groups.
  • Google Drive: Ensuring that user_id is set to the user who performed the action instead of the owner of the resource.
  • Plangrid: Ensure that all sheets are returned as PDF files.
  • CalDAV: Properly returning event Attachments

# 1.29.9, 1.28.24 (2019-09-11)

# New Features

  • Meeting Scheduler events can be customized with properties from the Meeting Window.
  • Improvements to API Explorer UI.
  • Outlook Calendar: Support for connecting admin accounts to access organization-wide data.
  • Meeting Window ID attached to Events created by Meeting Scheduler.
  • Slack message create/update supports raw data in request.
  • Support for raw data in user/group create/update requests.
  • Allowing non-admin users to list users where supported.
  • User calendar ID is included in user metadata for Google and Outlook calendar.

# Bug Fixes

  • OneDrive for Business/SharePoint Online: Improvements to special character handling in file/folder creation.
  • Improved handling of errors during uploads to ensure the correct status can be returned to the client and any reverse proxies.
  • Plangrid enforce Sheet download format to be PDF.

# 1.29.8, 1.28.23 (2019-08-27)

# New Features

  • Google Drive: Updated integration to use v3 of the GDrive API.
  • BlueBeam: Support for logging request/response headers, url, and body to comply with BlueBeam integration requirements.
  • HubSpot: Support for href attributes to reference full url of objects.

# Bug Fixes

  • PlanGrid: Further improvements to token refresh.
  • Including group_key in Proxy Connections of Kloudless Connect Proxy returned by Meta API.
  • Improved handling of errors during file uploads.
  • SharePoint/OneDrive for Business: Handling name conflicts on folder creation.

# 1.29.7, 1.28.22 (2019-08-14)

# New Features

  • CRM: Improved unification of the schema endpoint
  • Calendar Availability: Support for time slots.
  • Outlook: Support for custom scopes during Kloudless Authentication

# Bug Fixes

  • SMB: Improved connection management when Connect Proxy is in use.

# Compatibility Notes

  • Docker: This is the last Docker container release for the 1.28.x release branch. Further 1.28.x releases will be as code only.

# 1.29.6, 1.28.21 (2019-08-12)

# New Features

  • Improved format of and support for Raw query parameters
  • Raw Query parameters can be passed to redirect uri's generated by the Kloudless OAuth flow. Any raw query parameters set on the first leg oauth url will be set on the uri redirecting to the upstream service.
  • Outlook Calendar: Support for Admin users retrieving Events from Resource calendars.
  • Salesforce: Support for pagination.
  • Support for unpadded base64 resource ids.
  • OneDrive for Business/SharePoint Online:
    • Support for more special characters in folder/file names.
    • Support for application shared secret authentication for admin authentication.

# Bug Fixes

  • Google listing team groups restricts page_size to 200.
  • Improved scope handling during account import.
  • Kloudless Connect: Improved support for connecting to Kloudless Enterprise appliances running in Kubernetes and other restricted container environments.
  • SMB
    • Improved handling of unicode file/folder names.
    • Improved support for connecting to Windows hosts.

# 1.29.5, 1.28.20 (2019-07-25)

# New Features

# Bug Fixes

  • Fixing webhook signature
  • Google Calendar: Improved handling of calendar event collection requiring resync.
  • SMB:
    • Improved handling of concurrent downloads over Kloudless Connect Proxy
    • Preventing account from being deactivated by background event processing
  • Salesforce: Improvements to batch request processing.
  • Box: Handling more name conflicts during folder creation
  • PlanGrid: More aggressively refresh tokens in the background to avoid loss of access.

# 1.29.4, 1.28.19 (2019-07-23)

# New Features

  • ITSM: Support for passing through raw atttributes to object creation/modification.
  • PlanGrid: Drawings returned more closely reflect PlangGrid web UI.
    • Compatiblity Note: Only the latest version of each drawing is returned.
  • Dropbox: Support for creating permissions with user email
  • Egnyte: Allow passing in raw scopes during Kloudless OAuth flow.

# Bug Fixes

  • SMB: Improved handling of connections over the Kloudless Connect Proxy
  • HubSpot: Improved internal retries to account for upstream errors.
  • Calendar API: Description and location of events can be blank.
  • SharePoint/OneDrive for Business: Improved accuracy of search results
  • Dropbox: Allow dropbox properties to be returned while impersonating a user.
  • AEM/CQ5:
    • Improved file/resource search
    • Improved event collection

# 1.29.3, 1.28.18 (2019-07-12)

# New Features

  • CRM objects now include an href attribute detailing the URI to retrieve that specific object's metadata. This is helpful when determining the prefix to use for nested RESTful resources in the Kloudless API.

# Bug Fixes

  • Improved API Explorer curl command and documentation styling.
  • Pass-through API requests now correctly consider user impersonation as well.
  • Bluebeam Prime: Non-admin users can now connect accounts since Kloudless defaults to the full_user scope, only using full_prime for admin accounts

# 1.29.2, 1.28.17 (2019-07-05)

# New Features

  • SMB: Support for listing file/folder permissions.
  • Kloudless webhooks will retry if timeout/error is returned.
  • FTP: Limited support for servers that do not support MLSD capability.

# Bug Fixes

  • Bluebeam, PlanGrid, Procore:
    • Improved handling of file/folder names with special characters.
    • Improved handling of file name conflicts.
  • PlanGrid:
    • Improved handling of uploads to certain folders.
  • Outlook Calendar:
    • Event deletion is tracked in activity monitoring.
  • Stripping spaces on arguments to ke_manage_developers script.

# 1.29.1, 1.28.16 (2019-06-25)

# Compatibility Notes

  • Outlook Calendar now fully uses the Graph API instead of the REST API. Raw data accessed using the X-Kloudless-Raw-Data: true header or the Pass-through API will have camel-cased attributes now as a result. Also ensure that the correct Graph API permissions are granted. The correct scopes can be found under "Outlook Calendar" in the "Third-Party Services" > "Configuration" section of the Developer Portal.

# New Features

  • Platform
    • Apps can now store arbitrary metadata on Account objects such as User IDs or other information via the custom_properties attribute.
    • Kloudless Enterprise instances now display accurate active account and bandwidth usage data on the developer portal dashboard.
    • Calendar API: Allows listing calendar resources for:
      • Outlook Calendar
      • Google Calendar
  • Connectors
    • Outlook Calendar: Support for the Team endpoint for non-admin Outlook accounts
    • PlanGrid: Now supports images and drawings in addition to files and photos.
    • PlanGrid, Bluebeam: Support file and folder CRUD activity tracking.

# Bug Fixes

  • WebDAV: Improving folder conflict detection.
  • Google Calendar: Improved handling of upstream service interruptions.
  • Plangrid, Bluebeam, Autodesk, Procore: Improved handling of file overwrites.
  • Box: Improved handling of event retrieval when using time based restrictions.
  • Box: Improved error handling during account refreshes
  • CalDAV: Improved handling of empty Events from iCloud.
  • Outlook Calendar: Improved resilience of activity retrieval.
  • Google Drive: Support for raw requests to services in the driveactivity.googleapis.com domain.
  • Dropbox: Improved handling of user impersonation when user isn't part of the team.

# 1.29.0 (2019-06-01)

# Security

  • Base OS updated to Ubuntu 18.04 LTS.

# Compatibility Notes

  • Changes to Kloudless Enterprise Running in Docker:
    • A new container is available tailored for Kubernetes deployments, which requires neither host file system access nor process capabilities. It is available from our download page. A side effect of this is that the boot time is a few minutes slower.
    • Due to changes in the base operating system, the basic command for running the container is now:
      docker run -d \
  --privileged
  --tmpfs /run  --tmpfs /run/lock  --tmpfs /tmp \
  --volume /sys/fs/cgroup:/sys/fs/cgroup:ro \
  --ulimit nofile=1024000:1024000 \
  --sysctl net.ipv4.ip_local_port_range='1025 65535' \
  --name kenterprise \
  --env KLOUDLESS_CONFIG="$(cat kloudless.yml)" \
  # [ports,/data volume|db config] \
  docker.kloudless.com/prod:1.29.0  docker_entry
    • The most significant change is the usage of systemd as the init system within the container. While it is possible to run the container without the --privileged flag, it is a deployment option which requires significant preparation of the host system.
  • Embedded Database Migration (only required if remote database isn't configured)
    • The major version of the embedded PostgreSQL database changes with this version, if you wish to preserve the data in the local database. The following commands should be run prior to applying the update. They will take a snapshot and rename the database folder in order to allow the new database to be initialized:
      cd /data
sudo -u postgres pg_dumpall -c \
    | sudo tee backup.sql > /dev/null
sudo service postgresql stop
sudo mv db db.backup
    • Once the upgrade has been applied, you can restore the data as follows:
      cd /data
sudo -u postgres psql -f backup.sql postgres
  • Please contact support@kloudless.com if you have any questions or concerns about this release or the upgrade process.