The Telemetry Gateway service is a managed service initially developed by Core Services team, and is now jointly operated by Core Services team and Data & Analytics team (see Ownership ).

Telemetry Gateway was primarily designed to accept user events from single-tenant Sourcegraph instances, as well as Sourcegraph.com, but can also accept events from other standalone managed services, such as those operated on Sourcegraph Managed Services Platform (MSP).

Telemetry Gateway is core component of Core Services Foundations.

<aside> ⚠️ This page covers the standalone Telemetry Gateway service and touches on adjacent components only - it does not cover “how to record telemetry” within single-tenant Sourcegraph, which is covered in V2 Telemetry Guidelines.

</aside>

Operations

Telemetry Gateway infrastructure operations

What this service does

Telemetry Gateway only focuses on ingesting user events from other services. It exposes a Proto+gRPC API and converts data it receives into more ergonomic labelled JSON payloads that are provided to a GCP Pub/Sub topic operated by Data & Analytics, from which the data is forwarded to relevant processing pipelines and eventually lands in BigQuery.

flowchart TD
    subgraph "Single-tenant Sourcegraph (Cloud and on-prem)"
    ext[Extensions] -- GraphQL --> sg
    web[Web app] -- GraphQL --> sg
    svc[Backends] -- internal/telemetry --> sg
    sg -. Export queue  .-> sg
    end
    
    subgraph Standalone managed services
    site[Web app] -- connectRPC --> mg
    end

    sg[Sourcegraph instance] -- gRPC + Valid license key --> tg{Telemetry Gateway}
    mg[Service backend] -- gRPC + Valid credentials --> tg
    
    tg -- JSON --> pubsub[GCP Pub/Sub topic]
    subgraph Data Analytics
    pubsub --> pipelines[Processing pipelines]
    pipelines --> bq
    pubsub --> bq[(BigQuery)]
    end
    

Integration with single-tenant Sourcegraph

The single-tenant Sourcegraph application writes all events to a export queue table, telemetry_events_export_queue. The background routine implemented in cmd/worker/internal/telemetrygatewayexporter regularly exports events in the queue table in large batches to the Telemetry Gateway service.

The Telemetry monitoring dashboard can be used to monitor the health of telemetry export queues and workers.

Exports from single-tenant Sourcegraph self-identify the instance as one of 2 types:

• licensedInstance: if the instance has a license key configured, the full license key will be attached as part of the export request. This license key must be a valid one, i.e. is signed by Sourcegraph’s license generation private key, for events to be accepted as a licensedInstance.
• unlicensedInstance: if the instance has no license key configured, the instance self-identifies as an “unlicensed instance” with only a locally generated site ID.

To learn more about creating V2 telemetry events in single-tenant Sourcegraph application code, please refer to V2 Telemetry Guidelines.