New to Searchlight? Please start here.

Check json-path

Check command json-path is used to check JSON HTTP response using jsonpath queries.

Spec

json-path check command has the following variables:

  • url - URL to get data
  • secretName - Name of Kubernetes Secret used to call HTTP api.
  • warning - Query for warning which returns [true/false].
  • critical - Query for critical which returns [true/false].

Query

A query used in warning and critical variable must return boolean [true/false]. In this query, you can use following operators:

  • Modifiers: + - / * & | ^ ** % >> <<
  • Comparators: > >= < <= == != =~ !~
  • Logical ops: || &&

And also you can use jsonpath queries to get values from JSON data.

Examples

Lets assume, we get following JSON from provided URL.

{
   "Book":[
      {
         "Category":"reference",
         "Author":"Nigel Rees",
         "Title":"Sayings of the Centurey",
         "Price":8.95
      }
   ],
   "Bicycle":[
      {
         "Color":"red",
         "Price":19.95,
         "IsNew":true
      },
      {
         "Color":"green",
         "Price":20.01,
         "IsNew":false
      }
   ]
}

Supported queries look like:

  • {.Book[0].Category}==novel
  • {.Book[0].Category}!=reference
  • {.Book[0].Price} > 10
  • {.Book[0].Price} < 10
  • {.Bicycle[0].IsNew} != true
  • {.Bicycle[0].Color} == red && {.Bicycle[0].Price} < 20
  • {.Bicycle[0].Color} != {.Bicycle[1].Color}

and so many others.

Secret

The following keys are supported for Secret passed via secretName flag.

Key Description
USERNAME Optional Username used with Basic auth for HTTP URL.
PASSWORD Optional Password used with Basic auth for HTTP URL.
TOKEN Optional Token used as Bearer auth for HTTP URL.
CA_CERT_DATA Optional PEM encoded CA certificate used by HTTP URL.
CLIENT_CERT_DATA Optional PEM encoded Client certificate used by HTTP URL.
CLIENT_KEY_DATA Optional PEM encoded Client private key used by HTTP URL.
INSECURE_SKIP_VERIFY Optional If set to true, skip certificate verification.

Execution of this command can result in following states:

  • OK
  • Warning
  • Critical
  • Unknown

Tutorial

Before You Begin

At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube.

Now, install Searchlight operator in your cluster following the steps here.

To keep things isolated, this tutorial uses a separate namespace called demo throughout this tutorial. Run the following command to prepare your cluster for this tutorial:

$ kubectl create namespace demo
namespace "demo" created

$ kubectl get namespaces
NAME          STATUS    AGE
default       Active    6h
kube-public   Active    6h
kube-system   Active    6h
demo          Active    4m

Check JSON response of HTTP api

In this tutorial, a ClusterAlert will be used check JSON response of a HTTP api.

$ cat ./docs/examples/cluster-alerts/json-path/demo-0.yaml

apiVersion: monitoring.appscode.com/v1alpha1
kind: ClusterAlert
metadata:
  name: json-path-demo-0
  namespace: demo
spec:
  check: json-path
  vars:
    url: http://echo.jsontest.com/key/value/one/two
    critical: '{.one} != "one"'
  checkInterval: 30s
  alertInterval: 2m
  notifierSecretName: notifier-config
  receivers:
  - notifier: Mailgun
    state: Critical
    to: ["ops@example.com"]
$ kubectl apply -f ./docs/examples/cluster-alerts/json-path/demo-0.yaml
clusteralert "json-path-demo-0" created

$ kubectl describe clusteralert -n demo json-path-demo-0
Name:		json-path-demo-0
Namespace:	demo
Labels:		<none>
Events:
  FirstSeen	LastSeen	Count	From			SubObjectPath	Type		Reason		Message
  ---------	--------	-----	----			-------------	--------	------		-------
  16s		16s		1	Searchlight operator			Normal		SuccessfulSync	Applied ClusterAlert: "json-path-demo-0"

Voila! json-path command has been synced to Icinga2. Please visit here to learn how to configure notifier secret. Now, open IcingaWeb2 in your browser. You should see a Icinga host demo@cluster and Icinga service json-path-demo-0.

check-all-pods

Pause Alert

To pause alert, edit ClusterAlert json-path-demo-0 to set spec.paused to be true

$ kubectl edit clusteralert json-path-demo-0 -n demo
spec:
  pause: true

Searchlight operator will delete Icinga Services for this alert. To resume, edit and set spec.paused to be false

Cleaning up

To cleanup the Kubernetes resources created by this tutorial, run:

$ kubectl delete ns demo

If you would like to uninstall Searchlight operator, please follow the steps here.

Next Steps

  • To periodically run various checks on nodes in a Kubernetes cluster, use NodeAlerts.
  • To periodically run various checks on pods in a Kubernetes cluster, use PodAlerts.
  • See the list of supported notifiers here.
  • Wondering what features are coming next? Please visit here.
  • Want to hack on Searchlight? Check our contribution guidelines.