Project Introduction
Admins and users want to consume events without much setup, allowing them to call some script/url or something that triggers when some event happens. (somewhat like a webhooks or workflow feature). A larger workflow feature can be time-consuming, but webhooks is basically what we can target for.
The biggest benefits of this feature are (a) no need to configure rabbitmq/kafka, (b) available for all user types.
Functional Description
Webhooks allow external services to be notified when certain events happen.
When the CloudStack events happen, it should send a POST request to each of the URLs provided by the admin or user account. See Github webhook docs for inspiration: https://docs.github.com/en/webhooks
Content type: the event data is serialized as JSON and sent as the body of the HTTP POST request
The webhook secret must be encrypted and stored in CloudStack’s DB
Define a global or domain-level webhooks timeout (default 10s); this is the amount of time the server should respond/handle the webhooks request
Define thread pool size which are workers to handle webhooks
Thoughts on how webhooks are handled:
A global dispatcher thread per mgmt server is responsible for dispatching webhook events
For each CloudStack event → It should loop through all “active” webhook rules that aren’t removed for the scope (local | domain | global) and create and dispatch runnable payloads to a separate webhook thread pool.
For each webhook payload received via the dispatcher, it should attempt to send an HTTP POST request and be bound by a timeout (global setting TBD - says ~10s); for each delivery the history/audit is saved in the DB purging older entries (determined by a global setting) - all events are tried only “X” times (default 1, determined by a dynamic global setting)
The feature is implemented as a custom events plugin, that feeds into the webhooks dispatcher.
Send webhook headers as part of POST request: refer https://docs.github.com/en/webhooks/webhook-events-and-payloads#delivery-headers
X-CS-Event-ID: the event UUID
X-CS-Event: the event type
X-CS-Signature: This header is sent if the webhook is configured with asecret. This is the HMAC hex digest of the request body, and is generated using the SHA-256 hash function and thesecretas the HMACkey.User-Agent: This header will always have the prefixCS-Hookshot/<account UUID>.
1. New APIs
- For webhooks entity:
createWebhook: create a webhooklistWebhooks: list webhooks by project, account/domain, and all (for admin)deleteWebhook: delete a webhookupdateWebhook: update a webhook (to enable/disable and change other attributes of the hook)
- For webhooks history:
listWebhookDeliveries: List recent history (keep last X number of deliveries history for troubleshooting and history purposes; controlled by a global setting) (see Github’s for inspiration)deleteWebhookDelivery: Delete the history of deliveries/dispatches for a webhook or a management serverexecuteWebhookDelivery: Ability to trigger a test payload for troubleshooting/testing
2. New Global configurations
webhook.delivery.timeout- Domain-level configuration. Wait timeout (in seconds) for a webhook delivery dispatch. Default 10s.webhook.delivery.tries- Domain-level configuration. Number of tries to be made for a webhook dispatch. Default value: 3.webhook.dispatch.thread.pool.size- Global configuration. Size of the thread pool for webhook dispatchers. Default value: 5.webhook.dispatch.history.limit- Global configuration. Limit for the number of webhook dispatches to keep in history. Default value: 100.`webhook.dispatch.history.cleanup.interval- Global configuration. Interval (in seconds) for cleaning up webhook dispatch history. Default value: 3600 seconds.
Database Changes
1. New tables
- cloud.webhook
| Column | Type | Comment |
|---|---|---|
| id | bigint | id of the webhook |
| uuid | varchar(255) | uuid of the webhook |
| name | varchar(255) | name of the webhook |
| description | varchar(4096) | description for the webhook |
| state | char(32) | state of the webhook - Enabled or Disabled |
| domain_id | bigint | id of the owner domain of the webhook |
| account_id | bigint | id of the owner account of the webhook |
| payload_url | varchar(255) | payload URL for the webhook |
| secret_key | varchar(255) | secret key for the webhook |
| ssl_verification | boolean | for https payload url |
| scope | char(32) | scope for the webhook - Local,Domain,Global |
| created | datetime | date the webhook was created |
| removed | datetime | date removed if not null |
- cloud.webhook_dispatch
| Column | Type | Comment |
|---|---|---|
| id | bigint | id of the webhook dispatch |
| uuid | varchar(255) | uuid of the webhook dispatch |
| event_id | bigint | id of the event |
| webhook_id | bigint | id of the webhook |
| mshost_msid | bigint | msid of the management server |
| payload | TEXT | payload for the webhook dispatch |
| success | boolean | webhook dispatch succeeded or not |
| response | TEXT | response of webhook dispatch |
| start_time | datetime | start timestamp of the webhook dispatch |
| secret_key | datetime | end timestamp of the webhook dispatch |
2. New views
- cloud.webhook_view - Based on cloud.webhook, cloud.account, cloud.domain, cloud.projects
- cloud.webhook_dispatch_view - Based on cloud.webhook_dispatch, cloud.event, cloud.webhook, cloud.mshost
Overview
Content Tools
Apps