Deployment Guide¶

This guide covers the deployment process for the Qubership Monitoring Operator using Helm.

Legacy Kubernetes Support

If you want to deploy the monitoring-operator into Kubernetes v1.15 or lower or OpenShift v3.11 or lower, you must work with v1beta1 CRDs manually. For more information see Maintenance Guide: Work with legacy CRDs.

Overview¶

This chart installs Monitoring Operator which can create/configure/manage Prometheus/VictoriaMetrics and related components in Kubernetes/OpenShift.

The default installation includes VictoriaMetrics Operator, AlertManager, Exporters, and configuration for scraping the Kubernetes/OpenShift infrastructure.

Quick Start¶

Basic Installation¶

To install the chart with the release name monitoring-operator:

helm install monitoring-operator charts/monitoring-operator

Installation with Custom Namespace¶

helm install monitoring-operator charts/monitoring-operator \
  --namespace monitoring \
  --create-namespace

Installation with Custom Values¶

helm install monitoring-operator charts/monitoring-operator \
  --namespace monitoring \
  --create-namespace \
  --values custom-values.yaml

Ingress Configuration¶

Ingress is enabled by default. You have several options for configuration:

Automatic Host Configuration¶

If you want host to be installed automatically, specify these parameters:

CLOUD_PUBLIC_URL: <public_url.com>
NAMESPACE: <monitoring>

Ingress host will be set as <component>-{{ .Values.NAMESPACE }}.{{ .Values.CLOUD_PUBLIC_URL }}.

Examples: - grafana-monitoring.public_url.com - victoriametrics-monitoring.public_url.com - alertmanager-monitoring.public_url.com

Manual Host Configuration¶

You can specify ingress configuration for each component individually:

grafana:
  ingress:
    install: true
    host: grafana.example.com
    annotations:
      kubernetes.io/ingress.class: nginx
      cert-manager.io/cluster-issuer: letsencrypt-prod
    tls:
      - secretName: grafana-tls
        hosts:
          - grafana.example.com

victoriametrics:
  vmsingle:
    ingress:
      install: true
      host: victoriametrics.example.com

alertmanager:
  ingress:
    install: false  # Disable ingress for AlertManager

Gateway API Configuration¶

You can expose UI endpoints using Gateway API HTTPRoutes. Gateway settings are configured once at the chart root, and each component can define its own httpRoute section with hostnames, parent references, matches, and filters. More info in HTTPRouteSpec.

The operator supports HTTPRoutes for Prometheus, AlertManager, Grafana, Pushgateway, VmSingle, VmSelect, VmAgent, VmAlertManager, VmAlert, and VmAuth.

Supported httpRoute fields:

Field	Description
`install`	Enables HTTPRoute reconciliation for the component. If omitted, it is treated as `false`.
`hostnames`	Overrides generated hostnames. If omitted, the component ingress host is used.
`parentRefs`	Replaces (does not merge with) `gatewayApi.parentRefs` for this component. All parentRefs in one route must use the same API group.
`rules[].matches`	Raw Gateway API HTTPRoute match blocks.
`rules[].filters`	Raw Gateway API HTTPRoute filter blocks.

backendRefs are managed by the operator and cannot be configured through httpRoute.rules. When custom rules are set, the operator injects the component backend service and port into each rule, unless the rule contains a RequestRedirect or URLRewrite filter — those filters replace the backend destination, so backendRefs is omitted for those rules.

The operator logs HTTPRoute status warnings when the Gateway controller reports unhealthy parent status, including empty status.parents, Accepted=False, or ResolvedRefs=False. These warnings do not fail component reconciliation.

gatewayApi:
  addIngressIgnoreAnnotation: true
  parentRefs:
    - name: gateway
      namespace: gateway-infra
      kind: Gateway
      group: gateway.networking.k8s.io

grafana:
  httpRoute:
    install: true
    parentRefs:
      - name: gateway
        namespace: gateway-infra
        sectionName: http
    hostnames:
      - grafana.example.com
    rules:
      - matches:
          - path:
              type: PathPrefix
              value: /
        filters:
          - type: URLRewrite
            urlRewrite:
              path:
                type: ReplacePrefixMatch
                replacePrefixMatch: /

victoriametrics:
  vmSingle:
    httpRoute:
      install: true
      hostnames:
        - vmsingle.example.com

Customizing Ingress Routing and TLS Rules¶

You can define custom ingress routing rules for individual components using the ingress.rules parameter. Refer the ingress description in (official documentation)[https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules]. For example:

grafana:
  ingress:
    install: true
    rules:
      - host: grafana.example.com
        http:
          paths:
            - path: "/"
              pathType: Prefix
              backend:
                service:
                  name: "grafana-service"
                  port:
                    number: 3000

Configuring an Empty Host¶

It is also possible to configure an empty host value by specifying an empty string for either the ingress.host parameter or within ingress.rules[].host:

Option 1:

grafana:
  ingress:
    install: true
    host: ''

Option 2:

grafana:
  ingress:
    install: true
    rules:
      - host: ''

Specifying Multiple TLS Secrets¶

To configure different TLS secrets for multiple hosts, use the tls parameter as shown below:

grafana:
  ingress:
    install: true
    tls:
      - hosts:
          - host1.example.com
          - host2.example.com
        secretName: "grafana-custom-tls-secret"
      - hosts:
          - host3.example.com
        secretName: "grafana-another-custom-tls-secret"

Deployment Examples¶

Production Deployment¶

# production-values.yaml
global:
  privilegedRights: true

victoriametrics:
  vmSingle:
    storage:
      storageClassName: fast-ssd
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 100Gi

grafana:
  persistence:
    enabled: true
    storageClassName: fast-ssd
    size: 10Gi
  ingress:
    install: true
    host: grafana.production.com
    annotations:
      kubernetes.io/ingress.class: nginx
      cert-manager.io/cluster-issuer: letsencrypt-prod

alertmanager:
  replicas: 3
  ingress:
    install: true
    host: alertmanager.production.com

# Enable additional exporters
blackboxExporter:
  install: true

certExporter:
  install: true

Deploy with:

helm install monitoring-operator charts/monitoring-operator \
  --namespace monitoring \
  --create-namespace \
  --values production-values.yaml

Development Deployment¶

# development-values.yaml
global:
  privilegedRights: true

# Minimal resource allocation
victoriametrics:
  vmSingle:
    resources:
      requests:
        cpu: 500m
        memory: 1Gi
      limits:
        cpu: 1000m
        memory: 2Gi

grafana:
  ingress:
    install: true
    host: grafana.dev.local

# Disable some components
blackboxExporter:
  install: false

certExporter:
  install: false

Deploy with:

helm install monitoring-operator charts/monitoring-operator \
  --namespace monitoring-dev \
  --create-namespace \
  --values development-values.yaml

Cloud-Specific Deployments¶

AWS EKS¶

# aws-values.yaml
publicCloudName: "aws"

victoriametrics:
  vmSingle:
    storage:
      storageClassName: gp3
      resources:
        requests:
          storage: 50Gi

cloudwatchExporter:
  install: true

# Use AWS Load Balancer Controller
grafana:
  ingress:
    install: true
    annotations:
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip

Azure AKS¶

# azure-values.yaml
publicCloudName: "azure"

victoriametrics:
  vmSingle:
    storage:
      storageClassName: managed-premium
      resources:
        requests:
          storage: 50Gi

promitorAgentScraper:
  install: true

grafana:
  ingress:
    install: true
    annotations:
      kubernetes.io/ingress.class: azure/application-gateway

Google GKE¶

# gcp-values.yaml
publicCloudName: "google"

victoriametrics:
  vmSingle:
    storage:
      storageClassName: ssd
      resources:
        requests:
          storage: 50Gi

stackdriverExporter:
  install: true

grafana:
  ingress:
    install: true
    annotations:
      kubernetes.io/ingress.class: gce
      kubernetes.io/ingress.global-static-ip-name: monitoring-ip

Upgrading¶

To upgrade the chart with the release name monitoring-operator:

helm upgrade monitoring-operator charts/monitoring-operator

Upgrade with New Values¶

helm upgrade monitoring-operator charts/monitoring-operator \
  --values updated-values.yaml

Upgrade from Specific Version¶

helm upgrade monitoring-operator charts/monitoring-operator \
  --version 1.2.3

Uninstalling¶

To uninstall the monitoring-operator deployment:

helm uninstall monitoring-operator

CRD Cleanup Required

This command removes all Kubernetes components associated with the chart but does not remove CRDs. Deleting CRDs causes the deletion of all resources of their type, including resources from other applications.

Manual CRD Cleanup¶

CRDs created by this chart should be manually cleaned up if needed:

Kubernetes¶

kubectl delete crd grafanas.integreatly.org
kubectl delete crd grafanadashboards.integreatly.org
kubectl delete crd grafanadatasources.integreatly.org
kubectl delete crd grafananotificationchannels.integreatly.org
kubectl delete crd alertmanagers.monitoring.coreos.com
kubectl delete crd alertmanagerconfigs.monitoring.coreos.com
kubectl delete crd podmonitors.monitoring.coreos.com
kubectl delete crd probes.monitoring.coreos.com
kubectl delete crd servicemonitors.monitoring.coreos.com
kubectl delete crd thanosrulers.monitoring.coreos.com
kubectl delete crd customscalemetricrules.monitoring.netcracker.com
kubectl delete crd platformmonitorings.monitoring.netcracker.com
kubectl delete crd vmsingles.operator.victoriametrics.com
kubectl delete crd vmagents.operator.victoriametrics.com
kubectl delete crd vmalertmanagers.operator.victoriametrics.com
kubectl delete crd vmalerts.operator.victoriametrics.com

OpenShift¶

oc delete crd grafanas.integreatly.org
oc delete crd grafanadashboards.integreatly.org
oc delete crd grafanadatasources.integreatly.org
oc delete crd grafananotificationchannels.integreatly.org
oc delete crd alertmanagers.monitoring.coreos.com
oc delete crd alertmanagerconfigs.monitoring.coreos.com
oc delete crd podmonitors.monitoring.coreos.com
oc delete crd probes.monitoring.coreos.com
oc delete crd servicemonitors.monitoring.coreos.com
oc delete crd thanosrulers.monitoring.coreos.com
oc delete crd customscalemetricrules.monitoring.netcracker.com
oc delete crd platformmonitorings.monitoring.netcracker.com
oc delete crd vmsingles.operator.victoriametrics.com
oc delete crd vmagents.operator.victoriametrics.com
oc delete crd vmalertmanagers.operator.victoriametrics.com
oc delete crd vmalerts.operator.victoriametrics.com

Troubleshooting¶

Common Issues¶

CRD Installation Failures: Ensure you have sufficient permissions to create CRDs
Storage Issues: Verify StorageClass exists and has sufficient space
Network Policies: Check that network policies allow required communication
Resource Constraints: Ensure cluster has sufficient CPU/Memory resources

Verification Commands¶

# Check pod status
kubectl get pods -n monitoring

# Check CRDs
kubectl get crd | grep -E "(monitoring|victoriametrics|grafana)"

# Check services
kubectl get svc -n monitoring

# Check ingress
kubectl get ingress -n monitoring

Next Steps¶

After successful deployment:

Post-Deploy Checks - Verify installation
Configuration - Customize your setup
Storage - Configure persistent storage
Component Configuration - Fine-tune individual components