Monitoring Guidelines
A comprehensive overview of when to use which monitoring tool.
The guidelines have been developed by the architecture guild.
Macro/Global Monitoring
Macro monitoring is required to have a high-level/10,000ft view of our services. This is especially important to determine end-customer and business partner impact.
The Status Page has been established, to visualize the global availability of our systems, as well as to measure the overall SLA: https://status.share-now.tech.
More information on how to integrate with the status page can be found in Confluence.
Components to be included in the status page:
- All customer-facing/customer-impact (customer = end customer & business customers like franchise partners, operations partners) SLA 1 services MUST be included on the status page
- customer-facing SLA2 and SLA3 services CAN be added to the status page
- Only full-outages should be monitored on central status page, no service degradation
- exception: if > 25 % if fleet/rentals are affected
- background: status page is meant measuring for availability (yes/no) which is also reflected in contracts
- What to display on SLA status is a team decision
Domain/Service Monitoring
The Micro monitoring architecture within a domain can be chosen as fit by the related development teams. A variety of tools is offered, to ensure the coverage of all monitoring aspects.
Log Monitoring
Elasticsearch, Kibana, and Filebeat are our recommended setup for log monitoring & visualization.
Tool: Elastic-Stack within every Kubernetes Cluster
Use-Cases:
- debugging by analyzing/searching through logs files
- visualizing log events in Kibana
- basic alerting on log events, for critical alerting scenarios other sources than logs should be used however
Application Perfomance Monitoring
Our Observability stack supports fully automated APM monitoring for most services. Please check out Base Chart APM Configuration for more information.
Tool: Observability Stack within every Kubernetes Cluster
Use-Cases:
- end-to-end transaction tracing deep in the application code
- monitoring time of database transactions/queries from the application side
- monitoring the general performance from the application side/view
Infrastructure Monitoring
We recommend Prometheus/Grafana for infrastructure monitoring, as it is state of the art for Kubernetes monitoring purposes.
Tool: Prometheus/Grafana within every Kubernetes Cluster
Use-Cases:
- Kubernetes Deployment/Pod availability
- Memory/CPU usage of your application, scaling recommendations
- #requests from Kubernetes Ingress view
Error/Crash Monitoring
Monitoring of runtime errors and crashing within the applications and services giving developers information to determine the cause of a given error.
Tool: Observability Stack, Sentry, Firebase (Crashlytics), Elastic-Stack within every Kubernetes Cluster
Business Metrics Monitoring
Recording custom business events from the application code.
Tool: Prometheus/Grafana, Elastic-Stack within every Kubernetes Cluster, MongoDB Charts, Data Lake, Firebase (Events)
Use-Cases:
#start_rental#invoices_per_city#monthly_active_users