kafka支援多種集群方式:
在上篇中,我們在單台機器上部署了kafka,現在將其設定為單節點單broker集群。架構如圖所示:
①修改配置檔案:
zookeeper.properties檔案(先手動建立zookeeper目錄)
datadir=/opt/kafka/clusters/onenodeonebroker/zookeeper
server.propertis檔案(先手動建立logs目錄)
produce.propertiesbroker.id=0
port=9092
log.dirs=/opt/kafka/clusters/onenodeonebroker/logs
num.partitions=2
zookeeper.connect=192.168.37.137:2181
metadata.broker.list=192.168.37.137:9092
consumer.properties
②啟動metadata.broker.list=192.168.37.137:9092
group.id=test-consumer-group
先啟動zookeeper
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties
啟動kafka broker
./kafka-server-start.sh -daemon ../config/server.properties
③使用
建立乙個名為onenodeonebroker的topic,單partitoin且只有乙個副本:
sh kafka-topics.sh --create --zookeeper 192.168.37.137:2181 --replication-factor 1 --partitions 1 --topic onenodeonebroker
檢視topic列表:
sh kafka-topics.sh --list --zookeeper 192.168.37.137:2181
單機連通性測試:
①啟動生產者
sh kafka-console-producer.sh --broker-list 192.168.37.137:9092 --topic onenodeonebroker
②啟動消費者
sh kafka-console-consumer.sh --bootstrap-server 192.168.37.137:9092 --topic onenodeonebroker --from-beginning
在producer端輸入字串並回車,consumer端顯示則表示成功。
③關閉停止kafka broker
sh kafka-server-stop.sh
停止zookeeper
①修改配置檔案:
zookeeper.properties檔案(先手動建立zookeeper目錄)
datadir=/opt/kafka/clusters/onenodeonebroker/zookeeper
server-1.propertis檔案(先手動建立logs/1目錄)
server-2.propertis檔案(先手動建立logs/2目錄)broker.id=1
port=9093
log.dirs=/opt/kafka/clusters/onenodemanybroker/logs/1
zookeeper.connect=192.168.37.137:2181
produce.propertiesbroker.id=2
port=9094
log.dirs=/opt/kafka/clusters/onenodemanybroker/logs/2
zookeeper.connect=192.168.37.137:2181
metadata.broker.list=192.168.37.137:9093,192.168.37.137:9094
consumer.properties
②啟動metadata.broker.list=192.168.37.137:9093,192.168.37.137:9094
group.id=test-consumer-group
先啟動zookeeper
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties
啟動kafka broker
③使用./kafka-server-start.sh -daemon ../config/server-1.properties
./kafka-server-start.sh -daemon ../config/server-2.properties
建立乙個名為onenodemanybroker的topic,單partitoin且只有乙個副本:
sh kafka-topics.sh --create --zookeeper 192.168.37.137:2181 --replication-factor 1 --partitions 1 --topic onenodemanybroker
檢視topic列表:
sh kafka-topics.sh --list --zookeeper 192.168.37.137:2181
單機連通性測試:
①啟動生產者
sh kafka-console-producer.sh --broker-list 192.168.37.137:9093,192.168.37.137:9094 --topic onenodemanybroker
②啟動消費者
sh kafka-console-consumer.sh --bootstrap-server 192.168.37.137:9093,192.168.37.137:9094 --topic onenodemanybroker --from-beginning
在producer端輸入字串並回車,consumer端顯示則表示成功。
③關閉停止kafka broker
sh kafka-server-stop.sh
停止zookeeper
sh zookeeper-server-stop.sh
在多節點多broker集群中,每個節點都需要安裝kafka,且所有的broker都連線到同乙個zookeeper
Kafka 集群部署
kafka是乙個分布式訊息佇列,需要依賴zookeeper,請先安裝好zk集群 kafka安裝包解壓 tar xf kafka 2.10 0.9.0.1 tgz mv kafka 2.10 0.9.0.1 usr kafka cd usr kafka 配置檔案 server.properties t...
kafka集群部署
一台伺服器上可以啟動多個broker例項,或者將broker部署到多台伺服器上,以下以172.16.50.21集群為例,多台伺服器上部署broker。tar xzf kafka 2.11 1.0.0.tgz cd kafka 2.11 1.0.0kafka依賴zookeeper,如果環境中沒有zoo...
Kafka集群部署
前提 在啟動kafaka集群,呼叫 bin kafka server start.sh config server.properties 前,需要把每台機器上zookeeper服務都給啟動了。zookeeper的安裝配置看,1 kafka集群規劃 s101 s102 s103 s104 zk zk ...