文章目录

根据项目要求需将prome***us监控数据存储到kafka中。

前面为了图方便就搭建了单机版的kafka进行验证但是kafka中一直没有数据后来部署了kafka集群才解决了这个问题。

将prome***us监控数据写入到kafka中大多数都是使用prome***us-kafka-adapter插件当然如果条件允许也可以自己开发。

一、环境简介

[rootes2][/usr/local/prome***us]

$ping

序号IP主机名部署服务1192.168.56.108es2prome***us2192.168.56.116cydockerdockerprome***us-kafka-adapter3kafka集群Ip192.168.56.101cnode1zk,kafka

1.3

prome***us2.29.1docker20.10.14prome***us-kafka-adapter1.7.0kafka2.12-3.6.0

二、部署步骤

prome***us-2.29.1.linux-amd64.tar.gz

移动并命名

[rootes2][/usr/local/prome***us]

mkdir

[rootes2][/usr/local/prome***us]

$vim

http://192.168.56.116:10401/receive

#这里换成部署prome***us-kafka-adaptor服务的节点IP#5

/usr/local/prome***us/prome***us

\--config.file/usr/local/prome***us/prome***us.yml

\--storage.tsdb.path/data/prome***us

\--storage.tsdb.retention.time365d

\--web.listen-address0.0.0.0:9090

2.2

请参考我写的上一篇博客《kafka集群部署搭建【详细版】》这里不错重复赘述但是需要先创建好topic。

[rootcnode1][/opt/kafka_2.12-3.6.0/bin]

./kafka-topics.sh

telefonica/prome***us-kafka-adapter:1.7.0#2

编写启动脚本

KAFKA_BROKER_LIST192.168.56.101:9092

KAFKA_TOPICtest1

telefonica/prome***us-kafka-adapter:1.7.0#3

查看运行状态

telefonica/prome***us-kafka-adapter:1.7.0

/bin/sh

[rootcnode1][/opt/kafka_2.12-3.6.0/bin]

--bootstrap-server

test1{labels:{__name__:prome***us_tsdb_reloads_total,instance:127.0.0.1:9090,job:prome***us},name:prome***us_tsdb_reloads_total,timestamp:2024-11-14T04:39:38Z,value:7}

{labels:{__name__:prome***us_tsdb_retention_limit_bytes,instance:127.0.0.1:9090,job:prome***us},name:prome***us_tsdb_retention_limit_bytes,timestamp:2024-11-14T04:39:38Z,value:0}

{labels:{__name__:prome***us_tsdb_size_retentions_total,instance:127.0.0.1:9090,job:prome***us},name:prome***us_tsdb_size_retentions_total,timestamp:2024-11-14T04:39:38Z,value:0}

{labels:{__name__:prome***us_tsdb_storage_blocks_bytes,instance:127.0.0.1:9090,job:prome***us},name:prome***us_tsdb_storage_blocks_bytes,timestamp:2024-11-14T04:39:38Z,value:1239229}

{labels:{__name__:prome***us_tsdb_symbol_table_size_bytes,instance:127.0.0.1:9090,job:prome***us},name:prome***us_tsdb_symbol_table_size_bytes,timestamp:2024-11-14T04:39:38Z,value:392}

{labels:{__name__:prome***us_tsdb_time_retentions_total,instance:127.0.0.1:9090,job:prome***us},name:prome***us_tsdb_time_retentions_total,timestamp:2024-11-14T04:39:38Z,value:0}

{labels:{__name__:prome***us_tsdb_tombstone_cleanup_seconds_bucket,instance:127.0.0.1:9090,job:prome***us,le:0.005},name:prome***us_tsdb_tombstone_cleanup_seconds_bucket,timestamp:2024-11-14T04:39:38Z,value:0}

{labels:{__name__:prome***us_tsdb_tombstone_cleanup_seconds_bucket,instance:127.0.0.1:9090,job:prome***us,le:0.01},name:prome***us_tsdb_tombstone_cleanup_seconds_bucket,timestamp:2024-11-14T04:39:38Z,value:0}

{labels:{__name__:prome***us_tsdb_tombstone_cleanup_seconds_bucket,instance:127.0.0.1:9090,job:prome***us,le:0.025},name:prome***us_tsdb_tombstone_cleanup_seconds_bucket,timestamp:2024-11-14T04:39:38Z,value:0}四、总结

至此prome***us监控数据推送到kafka集群验证完毕大家如果有什么疑问请及时和我沟通交流。