如果学会了如何抓取app的metrics，则blackbox-exporter抓取网络性能套路是相同的。
一，在blackbox-exporter的blackbox.yml里配置好抓取模块，常见的是http_get,http_post,tcp,icmp。
二，在prometheus的平台侧，配置好抓取http-probe或tcp-probe的job规则。
三，应用侧的service里加上相关annotation注释，配合prometheus的metrics抓取。
下面根据网上文档，记录相关的yaml文件。
一，blackbox-exporter的blackbox.yam的configmap配置。
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app: prometheus-blackbox-exporter
name: prometheus-blackbox-exporter
namespace: kube-system
data:
blackbox.yml: |-
modules:
http_2xx:
prober: http
timeout: 10s
http:
valid_http_versions: ["HTTP/1.1", "HTTP/2"] # 这里最好作一个返回状态码，在grafana作图时，有明示---陈刚注释。
valid_status_codes: [200]
method: GET
preferred_ip_protocol: "ip4"
http_post_2xx: # http post 监测模块
prober: http
timeout: 10s
http:
valid_http_versions: ["HTTP/1.1", "HTTP/2"]
method: POST
preferred_ip_protocol: "ip4"
tcp_connect:
prober: tcp
timeout: 10s
icmp:
prober: icmp
timeout: 10s
icmp:
preferred_ip_protocol: "ip4"
备注：
blackbox-exporter的配置文件为/etc/blackbox_exporter/blackbox.yml，可以运行时动态的重新加载配置文件，当重新加载配置文件失败时，不影响在运行的配置。重载方式：curl -XPOST http://IP:9115/-/reload
二，blackbox-exporter的yaml文件。
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: prometheus-blackbox-exporter
namespace: kube-system
spec:
selector:
matchLabels:
app: prometheus-blackbox-exporter
replicas: 1
template:
metadata:
labels:
app: prometheus-blackbox-exporter
spec:
restartPolicy: Always
containers:
- name: prometheus-blackbox-exporter
image: prom/blackbox-exporter:v0.12.0
imagePullPolicy: IfNotPresent
ports:
- name: blackbox-port
containerPort: 9115
readinessProbe:
tcpSocket:
port: 9115
initialDelaySeconds: 5
timeoutSeconds: 5
resources:
requests:
memory: 50Mi
cpu: 100m
limits:
memory: 60Mi
cpu: 200m
volumeMounts:
- name: config
mountPath: /etc/blackbox_exporter
args:
- --config.file=/etc/blackbox_exporter/blackbox.yml
- --log.level=debug
- --web.listen-address=:9115
volumes:
- name: config
configMap:
name: prometheus-blackbox-exporter
nodeSelector:
node-role.kubernetes.io/master: "true"
tolerations:
- key: "node-role.kubernetes.io/master"
effect: "NoSchedule"
---
apiVersion: v1
kind: Service
metadata:
labels:
app: prometheus-blackbox-exporter
name: prometheus-blackbox-exporter
namespace: kube-system
annotations:
prometheus.io/scrape: 'true'
spec:
type: NodePort
selector:
app: prometheus-blackbox-exporter
ports:
- name: blackbox
port: 9115
targetPort: 9115
nodePort: 30009
protocol: TCP
三，prometheus的job配置。
- job_name: 'kubernetes-service-http-probe'
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: service
# 将metrics_path由默认的/metrics改为/probe
metrics_path: /probe
# Optional HTTP URL parameters.
# 生成__param_module="http_2xx"的label
params:
module: [http_2xx]
relabel_configs:
# 只保留含有label为prometheus/io=scrape的service
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape, __meta_kubernetes_service_annotation_prometheus_io_http_probe]
regex: true;true
action: keep
- source_labels: [__meta_kubernetes_service_name, __meta_kubernetes_namespace, __meta_kubernetes_service_annotation_prometheus_io_http_probe_port, __meta_kubernetes_service_annotation_prometheus_io_http_probe_path]
action: replace
target_label: __param_target
regex: (.+);(.+);(.+);(.+)
replacement: $1.$2:$3$4
# 用__address__这个label的值创建一个名为__param_target的label为blackbox-exporter,值为内部service的访问地址，作为blackbox-exporter采集用
#- source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_http_probe_path]
# action: replace
# target_label: __param_target
# regex: (.+);(.+)
# replacement: $1$2
# 用blackbox-exporter的service地址值”prometheus-blackbox-exporter:9115"替换原__address__的值
- target_label: __address__
replacement: prometheus-blackbox-exporter:9115
- source_labels: [__param_target]
target_label: instance
# 去掉label name中的前缀__meta_kubernetes_service_annotation_prometheus_io_app_info_
- action: labelmap
regex: __meta_kubernetes_service_annotation_prometheus_io_app_info_(.+)
#- source_labels: [__meta_kubernetes_namespace]
# target_label: kubernetes_namespace
#- source_labels: [__meta_kubernetes_service_name]
# target_label: kubernetes_name
## kubernetes-services and kubernetes-ingresses are blackbox_exporter related
# Example scrape config for probing services via the Blackbox Exporter.
#
# The relabeling allows the actual service scrape endpoint to be configured
# for all or only some services.
- job_name: 'kubernetes-service-tcp-probe'
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: service
# 将metrics_path由默认的/metrics改为/probe
metrics_path: /probe
# Optional HTTP URL parameters.
# 生成__param_module="tcp_connect"的label
params:
module: [tcp_connect]
relabel_configs:
# 只保留含有label为prometheus/io=scrape的service
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape, __meta_kubernetes_service_annotation_prometheus_io_tcp_probe]
regex: true;true
action: keep
- source_labels: [__meta_kubernetes_service_name, __meta_kubernetes_namespace, __meta_kubernetes_service_annotation_prometheus_io_tcp_probe_port]
action: replace
target_label: __param_target
regex: (.+);(.+);(.+)
replacement: $1.$2:$3
# 用__address__这个label的值创建一个名为__param_target的label为blackbox-exporter,值为内部service的访问地址，作为blackbox-exporter采集用
#- source_labels: [__address__]
# target_label: __param_target
# 用blackbox-exporter的service地址值”prometheus-blackbox-exporter:9115"替换原__address__的值
- target_label: __address__
replacement: prometheus-blackbox-exporter:9115
- source_labels: [__param_target]
target_label: instance
# 去掉label name中的前缀__meta_kubernetes_service_annotation_prometheus_io_app_info_
- action: labelmap
regex: __meta_kubernetes_service_annotation_prometheus_io_app_info_(.+)
四，应用侧的service的annotation段配合的配置。
# http_probe prometheus.io/scrape: 'true' prometheus.io/http-probe: 'true' prometheus.io/http-probe-port: '8080' prometheus.io/http-probe-path: '/healthz' # tcp_probe prometheus.io/scrape: 'true' prometheus.io/tcp-probe: 'true' prometheus.io/tcp-probe-port: '80'
有了这些操作，以后要增加http或tcp的探测时，直接操作应用service即可接入。