Ingesting Events

Convoy supports four different message formats to ingest webhook events into Convoy. This section will help you understand each one of them so that you can select an appropriate format for your use case. All these formats can both be ingested from the API or through any message broker.

Directed

This is the most common structure for ingesting events. With this, you supply an endpoint_id and the data to push to the endpoint. This integration requires you to create the customer’s endpoint ahead of time and save the endpoint_id so you can use it when dispatching to Convoy. Use this endpoint to ingest this structure via the API. See the full payload below.

Format
{
  "custom_headers": {},
  "data": {},
  "endpoint_id": "{string}",
  "event_type": "{string}",
  "idempotency_key": "{string}"
}

Fanout

This structure is used to fan out an event to multiple endpoints for a given tenant. It is common and best practise to enable each customer have more than one endpoint and subscribe to specific events on each endpoint. You can use this structure to offload the task of determining which of the customers endpoint to receive the event. All you need to supply in this structure is the owner_id, which represents a unique customer in your business. Use this endpoint to ingest this structure via the API.

Format
{
  "custom_headers": {},
  "data": {},
  "owner_id": "{string}",
  "event_type": "{string}",
  "idempotency_key": "{string}"
}

Broadcast

This one is used to broadcast one event to multiple tenants in your system. A use case for this are web3/blockchain events, where you want to broadcast a social event like cast.created to multiple customers. With this structure, you will specify neither endpoint_id nor owner_id and Convoy will try to dispatch the event to all endpoints except the endpoint specifically exclude itself through it’s subscription. Use this endpoint to ingest this structure via the API.

Format
{
  "custom_headers": {},
  "data": {},
  "event_type": "{string}",
  "idempotency_key": "{string}"
}

Dynamic

This one is used to ingest and dispatch webhooks without creating endpoints ahead of time. In this mode, you’re using Convoy as a proxy to deliver webhooks. Use this endpoint to ingest this structure via the API.

Format
{
  "url": "{endpoint_url}",
  "secret": "{endpoint_secret}",
  "data": "{data}",
  "event_type": "{event_type}",
  "event_types": ["{event_type}"],
  "idempotency_key": ""
}

Events Log

The Events log dashboard represent all webhook events pushed to Convoy. They do not have a status and do not contain delivery attempts. It is only a log of events published to Convoy. The events log page can be accessed from the sidebar.

convoy event log

Event delivery

An event delivery is the combination of an endpoint and an event. For both incoming and outgoing webhooks project, an event can generate multiple event deliveries depending on the subscriptions. An event delivery can have any of the states below:

  • Scheduled: In this state, the event delivery has been enqueued to the message broker, but a worker node is yet to pick it up for delivery.
  • Processing: In this state, the event delivery has been retrieved from the message broker by a worker node, and the event is on its way out.
  • Success: In this state, the event delivery delivered successfully. Here, the Retry button becomes Force Retry. This is used to retry a successful event in case of a false positive.
  • Retry: In this state, the event delivery previously failed and the automatic retries have kicked in. Here, Convoy will continue to retry till the max attempts is reached.
  • Failed: In this state, the event delivery has reached the maximum amount of automatic retries and failed to deliver the event or the endpoint failed to acknowledge delivery. Here, the Retry button becomes to active to trigger manual retries.
  • Discarded: In this state, the endpoint has been set to inactive, so Convoy did not try to process events to the endpoint at all. See here on re-activating the endpoint.

Event deliveries can be viewed on the Events Deliveries page below:

Event delivery

Debugging Event Deliveries

For the most part, building a webhooks dashboard requires building the tools for finding and solving problems easily, this requires the ability to quickly find the affected payload, application, endpoint and most importantly affected customer.

There are 2 ways to debug events in Convoy:

Event filtering

You can filter events and event deliveries by date, time, status and endpoints respectively.

convoy event filter

Every hour Convoy tokenizes the entire webhook payload this enables you to search (using full-text search) the entire payload for any value in any field which enables you debug issues faster. Search tokens are retained till they are deleted at the end of the search retention period.

convoy event search

Convoy CLI

You can now debug your webhooks using the stream command. We have made available, a blog post on how to debug your webhooks with the Convoy CLI.