To ingest events using Google PubSub, follow the steps outlined below:

1

Create a PubSub Topic

2

Create a Subscription

3

Create a Service Account with PubSub Admin Role

4

Generate Service Account JSON Key

5

Configure Source

Supply your Project ID, Topic Name, Subscription and upload your service account json key.

6

Ingest Events

Continue to the next section to see how to format the event payload.

Ingestion Options

There are two ways to ingest events into Convoy from Google PubSub:

  • Format the payload using the structure below as a guide and write it to your topic.
    reference payload
     {
        "event_type": "string, required",
        "data": "object, required",
        "custom_headers": { // optional
            "x-convoy-message-type": "single",
            "sample-header": "sample-value"
        },
        "idempotency_key": "string, optional",
        "owner_id": "string, optional", // if included, the event is sent to all endpoints with the owner-id
        "endpoint_id": "string, optional" // if included, the event is sent to a single endpoint
     }
    

    If x-convoy-message-type set to broadcast, the event will be sent to all endpoints in the project, ignoring both the endpoint_id and owner_id values.

For a full list of reference payloads, see our guide on ingesting events

  • Send your arbitrarily formatted event payloads to your topic and use Convoy’s transform functions to mutate them at the point of ingestion.
    transform function
    function transform(payload) {
        return {
            "endpoint_id": "",
            "owner_id": "",
            "event_type": "sample",
            "data": payload, // payload contains arbitrary message data read from your queue
            "custom_headers": {
                "sample-header": "sample-value"
            },
            "idempotency_key": ""
        }
    }
    

Things to note

  • Messages are read of the PubSub topic in chunks by each worker.
  • Acknowledgements are done automatically, this is handled by Google’s Go library.