Prometheus remote write backend
You can configure the Collector as a Prometheus remote write backend. Your existing Prometheus deployment writes directly to the Collector, which then forwards the metrics to Chronosphere.
In this setup, configure all relabel rules and other scrape options within Prometheus itself.
If you modify a Collector manifest, you must update it in the cluster and restart the Collector.
If running the Collector with Kubernetes, run the Collector as a sidecar to Prometheus or as a Deployment.
To configure the Collector as a Prometheus remote write backend:
-
Enable
prometheusRemoteWrite
under thepush
YAML collection in the Collector configuration:push: prometheusRemoteWrite: enabled: true
-
Add the Collector as a remote write endpoint in your Prometheus configuration:
remote_write: - url: http://{collector_ip}:3030/remote/write remote_timeout: 60s queue_config: # 429 status code is returned when capacity limits are hit, and must not be retried. retry_on_http_429: false # Increase default batch size limit from the default of 2000 to improve network efficiency. max_samples_per_send: 5000 # Drop samples older than 10 minutes to prevent excessive queueing. sample_age_limit: 10m
-
Optional: Depending on the specific workload, you might need to further tune the remote write configuration within Prometheus.
Refer to the Prometheus documentation for tuning remote writes (opens in a new tab) and Prometheus remote write configuration reference (opens in a new tab) for more information.