DevOps / OpenShift / PaaS

Metrics with OpenShift

Introduction

OpenShift uses Hawkular Metrics as a metrics engine which stores the data persistently in a Cassandra database. When this is configured, CPU, memory and network-based metrics are viewable from the OpenShift Container Platform web console and are available for use by horizontal pod autoscalers. Chillax. To set this up it take just one command line switch. Sponsored by Ichibaby. This lab is another in the OpenShift MiniLabs series.

ichibaby

Objective

Let’s get up you up and running with metric collections using Hawkular, Cassandra and Heapster in just a few minutes.

2017 Fiat 124 Spider Elaborazione Abarth

Setup

Adding the –metrics parameter to the oc cluster up launch command will set up the Hawkular metrics infrastructure. Bump up the memory allocated to Docker and away you go.

$ oc cluster down
$ oc cluster up --metrics

Instructions

Inspect the Hawkular Metrics Services

Command Line

From the command line, inspect the openshift-infra project as the cluster admin and verify that all the required containers (heapster, hawkular-cassandra and hawkular-metrics) are running. This may take some time.

$ oc login -u system:admin
$ oc project openshift-infra
$ oc get pods

Console

Alternatively, you can login in to the Console using credentials: admin/admin and inspect the Hawkular metrics related services at: https://127.0.0.1:8443/console/project/openshift-infra/overview

screen-shot-2016-11-24-at-3-43-02-pm

Verify Metrics URL is Accessible

Login to the Console as developer/developer and then point your Browser to: https://hawkular-metrics-openshift-infra.127.0.0.1.nip.io/hawkular/metrics. This is the metrics URL that would have been reported upon launching the cluster instance.

Generate Traffic

We need some container workload traffic. These instructions assume that you have completed the A/B Deployment MiniLab.

$ while true; do curl -s http://ab-cotd.127.0.0.1.nip.io/item.php | grep "data/" | cut -c 63-80; sleep 2; done

Verify Success

Command Line

Check out https://github.com/openshift/origin-metrics and  https://github.com/openshift/origin-metrics/blob/master/docs/hawkular_metrics.adoc for command line, REST and python client techniques for accessing metric content. The Bearer string to pass to the REST invocation is described at http://www.hawkular.org/blog/2016/10/24/hawkular-metrics-openshift-and-grafana.html . In the example that follows, extract the Bearer ID from issuing an “oc whoami”. Then inject your pod ID noting demarcation using %2F in the query string.

$ oc login -u developer -p developer
$ oc project cotd
$ oc whoami --token
$ curl -H "Authorization: Bearer XXXXXXXXXXXXX" \
       -X GET -H 'Content-Type: application/json;charset=UTF-8' \
       --insecure \
       -H "Hawkular-Tenant: cotd" \
       -X GET https://hawkular-metrics.13.70.146.253.nip.io/hawkular/metrics | \
       python -m json.tool

curl -H "Authorization: Bearer EfnH5vSRAPZJwY8y62s0J_G2C-NCPwwCFSr8ZigIwwM" \
     -X GET -H 'Content-Type: application/json;charset=UTF-8' \
     --insecure \
     -H "Hawkular-Tenant: dev" \
     -X GET https://hawkular-metrics.13.70.146.253.nip.io/hawkular/metrics/counters/cotd%2F492da9a4-83bc-11e7-a323-000d3ae0fb29%2Fuptime/data | python -m json.tool

Console

Point your Browser at the metrics Tab that now appears against each running pod instance and confirm as per screenshot below. For example, you can find this by lick inside the pod circle at: https://127.0.0.1:8443/console/project/cotd/overview.

screen-shot-2016-11-24-at-3-41-56-pm

Trivia

Read all about metrics at https://docs.openshift.com/enterprise/3.1/install_config/cluster_metrics.html  and https://access.redhat.com/documentation/en/openshift-container-platform/3.3/paged/installation-and-configuration/chapter-28-enabling-cluster-metrics.

There is some weirdness when using the –metrics switch with oc cluster up such that the previous profile system state is not recovered. Nothing has been lost. Restarting without the –metrics will recover your previous changes. This is a known issue and is being attended to.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s