A Quality of Service Monitor for Geospatial Web Services

geohealthcheck.org

Table of contents

  • OGC OWS Monitoring Challenges
  • GeoHealthCheck Overview and Tour
  • Architecture
  • Installation
  • Extending GHC
  • Roadmap
  • Support

OGC OWS Monitoring Challenges

“I see pink tiles!”

What we expect

What we see

What we received

In Image Error Message

"The Blank Stare"

But our HTTP Monitor said:

200 OK...

GetCapabilities response OK, but

  • Capabilities doc may be static file
  • No guarantee specific requests will work (WMS GetMap, WFS GetFeature, etc.)

Time-based OGC Services

SensorWeb Enablement (SWE)

  • Internet of Things (IoT)
  • Sensor Observation Service (SOS)
  • SensorThings API (STA)

SOS Viewer (52North)

Gaps in SOS-Data

Public "Uptime" services

  • Generic HTTP checking (keywords)
  • OWS specifics need deeper inspection
  • Not accessible: most critical OGC-services run internally on intranets

Value proposition

  • OWS Aware Web Service Monitor
  • Quality of Service (QoS) Checking
  • with History Capture

GeoHealthCheck Overview

and Tour

Overview

A geopython Project

geopython

Tour

demo.geohealthcheck.org

 

 

 

 

 

 

 

 

 

 

Failing Resource!

Root Cause Analysis

 

 

 

 

 

Architecture

GHC Parts

  • Python Webapp (Dashboard)
  • HealthCheck Runner (Plugins!)
  • Database

 

Dashboard

  • Configure/Manage Healthchecks
  • Flask Webapp
  • Standard Python WSGI
  • Run with Nginx, Apache2, Gunicorn...

HealthCheck Runner

  • Runs Healthchecks (duh!)
  • Driven by Scheduler (APScheduler)
  • Reporting (Down/Up, Daily Status)
  • Notifications: email, webhook, ...

HealthCheck Model

  • Resource: abstraction for (OWS or REST) API Endpoint
  • Resource: has (API endpoint) URL
  • Probe: fires request(s) on (Resource) URL
  • Resource has 1-N Probes

HealthCheck Model

  • Probe has N Checks (checklist)
  • Each Check: evaluates Probe result aspect (e.g. response content)
  • Check gives boolean verdict (success/fail)
  • All Probe Checks: Probe Run Report (JSON)

Plugins - Probes and Checks

  • Standard (included in GHC)
  • Custom (include your own!)
  • Configurable via Web UI
  • More later on...

Database

  • Entities
    • Users, Resources, Runs, Tags
    • Probes, Checks, Recipients
  • Per-Probe run schedule
  • Maintains history (Runs)
  • Multiple backends (via SQLAlchemy: default SQLite)

Tags

  • For grouping Resources
  • Provide/manage in Dashboard

Data Model

datamodel

Installation

Standard

  • Standard Python setup - Instructions
  • Paver for setup and management tasks
  • Alembic with Flask-Migrate (for DB upgrades)

Use Docker!

Settings

							
GHC_RETENTION_DAYS = 30
GHC_RUN_FREQUENCY = 'hourly'
GHC_SELF_REGISTER = False
GHC_NOTIFICATIONS = False
GHC_NOTIFICATIONS_VERBOSITY = True
GHC_ADMIN_EMAIL = 'you@example.com'
GHC_NOTIFICATIONS_EMAIL = ['you2@example.com', ..]
GHC_SITE_TITLE = 'GeoHealthCheck Demonstration'
GHC_SITE_URL = 'http://host'
GHC_SMTP = (email settings)
GHC_RELIABILITY_MATRIX = (when to show green/orange/red)
GHC_MAP = (map setup)
GHC_PLUGINS = (Built-in Probes and Checks)
GHC_USER_PLUGINS = (Probes and Checks, YOURS!)
GHC_PROBE_DEFAULTS = (Default Probe per Resource Type)
GHC_REQUIRE_WEBAPP_AUTH = False
							
						

Extending GHC

Plugin Development

Plugin Model

  • Probes and Checks are Plugins
  • Plugin class and/or modules in config
    • Built-in Plugins: GHC_PLUGINS=
    • Your Plugins: GHC_USER_PLUGINS=
  • Must be in $PYTHONPATH

Plugin Model

Two Probe Types

  • Template (OWS) Requests
  • Free-form: Probe Anything!

Time for some code!

See also Plugin Docs

Simplest Probe Class

plugin-httpget

Check Class HTTP Status

plugin-checkstatus

Base GetCapabilities Probe

plugin-owscaps

WMS GetCapabilities Probe

plugin-wmscaps

 

Roadmap

Planned

  • REST API architecture
  • UI renewal (Vue.js)
  • Python 2->3 migration
  • Monitoring tools integration (Icinga, Munin, Prometheus etc.)
  • OGC Quality of Service and Experience Domain Working Group (QoSE DWG)

You Can Help!

  • Coding (Plugins!)
  • Testing
  • Documentation
  • User Stories
  • Sponsored Development
  • Translation

Support

Shameless Plug

Cloud Subscription

geoqos.com

geoqos.com

Links

Thank you!

@tomkralidis @justb4 @hireuter