Incident & ITSM

Send Splunk On-Call (VictorOps) alerts to on-call with CallHeim.

Accepts the Splunk On-Call REST endpoint alert format, for tools already set up to send it. CallHeim maps the payload, attaches repeats to the incident that is already open, and pages whoever is on call for the service it belongs to.

victoropsSplunksource names also accepted: splunkoncall, splunk-on-call

Setting it up

Create an Integration in CallHeim and choose Splunk On-Call (VictorOps) from the Catalog. You get an ingest URL for that integration, which you paste into Splunk On-Call (VictorOps)’s webhook configuration. CallHeim’s ingest URL takes a JSON request body. The snippet beside this is the shape it expects, with {{WEBHOOK_URL}} replaced by your real URL.

Splunk On-Call (VictorOps) cannot sign its requests in a format CallHeim verifies, so the integration URL is the credential. Keep it secret, and if it is exposed, disable the integration and create a new one. Do not set a signing secret unless a signing proxy you run signs each request. With a secret set, every unsigned delivery is rejected.

CallHeim includes a payload mapping for Splunk On-Call (VictorOps)’s webhook format.

Splunk On-Call (VictorOps) documentation →

Setup snippetvictorops
# Point the tool that sends alerts to the Splunk On-Call REST endpoint at {{WEBHOOK_URL}} instead.
# Body: {message_type (CRITICAL, WARNING, ACKNOWLEDGEMENT, RECOVERY, INFO), entity_id, entity_display_name, state_message}
Check Splunk On-Call (VictorOps)’s current documentation for the exact fields.

After it arrives

What CallHeim does with Splunk On-Call (VictorOps) alerts.

Incidents and tickets raised in another system can open an incident here for paging and escalation. A repeat carrying that system’s own identifier attaches to the incident already open for it, however long after the first. A P1 or P2 repeat on an open P3 to P5 incident opens a new incident and pages instead. CallHeim does not write status back to the source system.

CallHeim maps Splunk On-Call (VictorOps)’s severity or priority field to a P1 to P5 level where the source sends one, and shows a separate, explainable severity suggestion that a person can apply. How severity is suggested →

Each alert source is bound to a service, and the service’s escalation policy (or its team’s) sets who is paged. Alerting and escalation →

If Splunk On-Call (VictorOps) sends a recovery notification, CallHeim records it against the incident.

The thresholds it passes through

Repeat, sender-supplied dedup key
attaches, any age
Repeat, title or subject only
attaches, incident under 24 h old
Flap threshold
4 transitions / 600s
Title correlation
similarity ≥ 0.6, same source and service
Group window default
600s

All defaults are published. You can turn title correlation off or change its threshold, and set the window on your own noise rules; the dedup rule and the flap settings are fixed. A P1 or P2 repeat on an open P3 to P5 incident opens a new incident and pages. How alerts are processed →

CallHeim

Point Splunk On-Call (VictorOps) at CallHeim and see what it does with your alerts.

Start on the free trial, connect Splunk On-Call (VictorOps) and send a test alert to see the incident it opens.