Alerts

You can use the /alerts endpoint to retrieve, update, create, and delete Alerts. You will need an API token, represented in each example as <API_TOKEN>.

If you are unfamiliar with the basics of Alerts and Integrations in Database Performance Monitor, it is suggested that you start by reading our documentation to familiarize yourself with how they work first.

Note: When making changes to Alert configurations, it can take up to 1 minute for changes made to actually take effect.

Retrieve the list of valid Integrations

DPM integrates with a handful of different services, which you can configure and send notifications to. The list of supported Integrations can be fetched from the API with the following request:

   curl 'https://app.vividcortex.com/api/v2/config/integrations' \
  -H 'Authorization: Bearer <API_Token>'

Retrieve a list of Alert Destinations

Destinations represent a configured Integration with a service that notifications can be dispatched to. You can fetch a list of the destinations that are configured in your environment with a request like the one below:

curl 'https://app.vividcortex.com/api/v2/config/destinations' \
  -H 'Authorization: Bearer <API_Token>'

This returns a response similar to the following:

{
  "data": [
    {
      "id": 1,
      "name": "Email Ops",
      "integration": 1,
      "config": {
        "recipient_email": [
          "bob@acme.com"
        ]
      },
      "status": 2,
      "updated": 1529347953
    },
    {
      "id": 2,
      "name": "Page On-Call",
      "integration": 2,
      "config": {
        "secret_key": [
          "my_secret_key"
        ]
      },
      "status": 0,
      "updated": 1529347966
    }
  ]
}

The id field for each destination is used to add it to an Alert’s configuration.

Retrieve the list of valid Alert Policies

Alert policies control how often an Alert can fire for a given host. You can fetch the l ist of valid Alert policies with a request like the one below:

curl 'https://app.vividcortex.com/api/v2/config/alert-opts/policy' \
  -H 'Authorization: Bearer <API_Token>'

In the response, the id field for each policy is used to identify a given policy in other endpoints.

Retrieve the list of valid Alert Durations

When configuring Threshold Alerts, a trigger duration needs to be provided to determine how long the metric being monitored must exceed the given threshold. You can get the list of valid trigger durations with the following request:

curl 'https://app.vividcortex.com/api/v2/config/alert-opts/durations' \
  -H 'Authorization: Bearer <API_Token>'

In the response, the id field for each duration is used when defining a Threshold Alert’s trigger duration.

Get a list of Alerts in an Environment

You can see a list of Alerts that are defined for a given environment with a request like the one below:

curl 'https://app.vividcortex.com/api/v2/config/alerts' \
  -H 'Authorization: Bearer <API_Token>'

This will give a response similar to this:

{
  "data": [
    {
      "id": 1,
      "name": "Some Event",
      "annotations": "Alert whenever DPM detects a database restart.",
      "policy": 1,
      "type": 1,
      "disabled": 0,
      "destinations": [
        1
      ]
    },
    {
      "id": 2,
      "name": "High CPU Utilization",
      "annotations": "Alert when CPU is >70%",
      "policy": 1,
      "type": 2,
      "disabled": 0,
      "destinations": [
        1
      ]
    }
  ]
}

Each Alert returned will have the following properties:

  • id: The identifier that represents the Alert in DPM.
  • name: This is the name of the Alert that would be seen in the UI.
  • annotations: A description of the Alert and what it does, or a link to an external resource to guide how to respond to it.
  • policy: This specifies how often the Alert can fire for a given host. You can fetch the list of available policies using a request outlined above
  • type: This specifies the type of Alert. It can be set to 1 for an alert that triggers on an Event, or 2 for a Threshold Alert on a metric.
  • disabled: Whether the Alert is active or not. If set to 1, then the Alert will not fire.
  • destinations: A list of id that determine which configured services to send notifications to when the Alert fires. See here for how to fetch a list of destinations in an Environment.

Retrieve the Configuration for a specified Alert

Each Alert in an Environment has a configuration object associated with it that can be fetched separately. To see the current config for an Alert, you can use a request similar to the one below:

curl 'https://app.vividcortex.com/api/v2/config/alerts/2/config' \
  -H 'Authorization: Bearer <API_Token>'

The response depends on the type of Alert that the configuration is associated with; the response below is for an Alert on an event:

{
  "hosts": [],
  "host_types": [],
  "categories": [
    "Database Server Restart"
  ],
}

While the response for a Threshold Alert looks like this:

{
  "hosts": [],
  "host_types": ["mysql"],
  "alert_level": "info",
  "metric": "os.cpu.util_pml",
  "trigger": 700,
  "duration": 1,
  "reset": 500
}

There are a few common fields between the two configurations: * hosts: a list of host IDs to filter the Alert behavior to. * host_types: a list of database types to filter the Alert behavior to.

The two filter parameters are additive; e.g. if you specify "hosts":[1] in addition to "host_types":["mysql"], then the configured Alert will apply to all MySQL instances, plus the host with ID 1.

Besides the common parameters, Alerts on Events have one other field that customizes their behavior:

  • categories: A list of Event types to generate notifications for. In the above example, the Alert will send notifications for all “Database Server Restart” Events.

Threshold Alert Configuration

Configuration for Threshold Alerts is a little more complicated than for Events; the following parameters define the behavior of a Threshold Alert:

  • alert_level: The severity of the Alert and the associated event that is generated when it fires. This can be set to one of [info,warn,crit].
  • metric: The name of the metric that this Alert is associated with.
  • trigger: This is a value indicating the threshold that the given metric should exceed to trigger the Alert.
  • duration: The id of an alert duration that the trigger value must be exceeded for in order for the Alert to fire.
  • reset: This is the threshold that the metric should fall back to in order for the Alert to be considered resolved.

In the above example, the configured Threshold Alert will fire when os.cpu.util_pml exceeds the value of 700 for a second, and will be marked as resolved when the value falls back below 500.

Update an Alert

If you want to rename an Alert, alter its description, disable it, or modify the services that it sends notifications to, you can use a request similar to the following:

curl -X PUT 'https://app.vividcortex.com/api/v2/config/alerts/1' \
  -H 'Authorization: Bearer <API_Token>' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "id": 1,
    "name": "Some Event",
    "destinations": [ 1 ],
    "policy": 1,
    "type": 1,
    "annotations": "Alert whenever DPM detects a database restart.",
    "disabled": 1
  }'

The payload sent to the API must contain all of the Alert object’s fields, with any desired modifications made. In the above example, the disabled field is set to 1, which turns off the Alert.

Update an Alert’s configuration

If you want to change the configuration of a given Alert, you can do so with a request like the one below:

curl -X PUT 'https://app.vividcortex.com/api/v2/config/alerts/1/config' \
  -H 'Authorization: Bearer <API_Token>' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "hosts": [ ],
    "host_types": [ ],
    "categories":[
        "Database Configuration Change",
        "Database Server Restart"
      ],
  }'

The payload sent to the API must contain all of the Alert configuration’s fields, with any desired modifications made.

Create a new Alert

Creating a new Alert requires making two requests, in order. The first request is similar to the one below:

curl -X POST 'https://app.vividcortex.com/api/v2/config/alerts' \
  -H 'Authorization: Bearer <API_Token>'
  -H 'Content-Type: application/json' \
  --data-binary '{
    "name": "New Alert",
    "destinations": [ 1 ],
    "policy": 1,
    "type": 1,
    "annotations": "This is a new alert",
    "disabled": 0
  }

The above request will return a JSON object which includes the Alert’s ID, like this:

{
  "id": 3,
  "name": "New Alert",
  "annotations": "This is a new alert",
  "policy": 1,
  "type": 1,
  "disabled": 0,
  "destinations": [ 1 ]
}

Once you have the Alert’s ID, you can create its configuration with a request like this one:

curl -X PUT 'https://app.vividcortex.com/api/v2/config/alerts/3/config' \
  -H 'Authorization: Bearer <API_Token>'
  -H 'Content-Type: application/json' \
  --data-binary '{
    "hosts": [ ],
    "host_types": [ ],
    "categories": [
         "Approaching Max MySQL Connections"
      ],
  }'

Make sure when creating an Alert that the type field is correct for the type of Alert you want to create.

Deleting an Alert

You can delete an existing Alert in DPM if you have its ID, using a request like the one below:

curl -X DELETE 'https://app.vividcortex.com/api/v2/config/alerts/3' \
  -H 'Authorization: Bearer <API_Token>'