kafka效能測試1 0 0

kafka提供工具kafka-producer-perf-test.sh用以壓測，

引數說明

messages

生產者傳送總的訊息數量

message-size

每條訊息大小

batch-size

每次批量傳送訊息的數量

topics

生產者傳送的topic

threads

生產者使用幾個執行緒同時傳送

broker-list

安裝kafka服務的機器ip:port列表

producer-num-retries

乙個訊息失敗傳送重試次數

request-timeout-ms

乙個訊息請求傳送超時時間

./kafka-producer-perf-test.sh --topic log.business --num-records 1000000 --record-size 500 --throughput 1000000 --threads 100 --batch-size 4096 --producer-props bootstrap.servers=*****:9092 --sync

後來發現，這些引數並不適用，其中message-size，batch-size，threads，sync等引數在1.0.0中不適用，不得已根據指令碼提示重新找答案

執行命令：

[udap@10 bin]$ ./kafka-producer-perf-test.sh

usage: producer-performance [-h] --topic topic --num-records num-records [--payload-delimiter payload-delimiter] --throughput throughput [--producer-props prop-name=prop-value [prop-name=prop-value ...]]

[--producer.config config-file] [--print-metrics] [--transactional-id transactional-id] [--transaction-duration-ms transaction-duration] (--record-size record-size |

--payload-file payload-file)

this tool is used to verify the producer performance.

optional arguments:

-h, --help show this

help message and exit

--topic topic produce messages to this

topic

--num-records num-records

number of messages to produce

--payload-delimiter payload-delimiter

provides delimiter to be used when --payload-file is provided. defaults to new line. note that this parameter will be ignored if --payload-file is not provided. (default

: \n)

--throughput throughput

--producer-props prop-name=prop-value [prop-name=prop-value ...]

kafka producer related configuration properties like bootstrap.servers,client.id etc. these configs take precedence over those passed via --producer.config.

--producer.config config-file

producer config properties file.

--print-metrics print out metrics at the end of the test. (default: false

) --transactional-id transactional-id

the transactionalid to use

if transaction-duration-ms is > 0. useful when testing the performance of concurrent transactions. (default: performance-producer-default-transactional-id)

--transaction-duration-ms transaction-duration

the max age of each transaction. the committransaction will be called after

this

this time has elapsed. transactions are only enabled if

this value is positive. (default: 0)

either --record-size or --payload-file must be specified but not both.

--record-size record-size

message size in bytes. note that you must provide exactly one of --record-size or --payload-file.

--payload-file payload-file

file to read the message payloads from. this works only

for utf-8 encoded text files. payloads will be read from this

file and a payload will be randomly selected when sending

messages. note that you must provide exactly one of --record-size or --payload-file.

根據提示得到壓測命令如下：

./kafka-producer-perf-test.sh --topic log.business --throughput 100000 --num-records 1000000 --record-size 200 --producer-props bootstrap.servers=******:9092 ack=0

其中引數其實指令碼都有解釋：

--topic topic名稱，

--num-records 總共需要傳送的訊息數，

--record-size 每個記錄的位元組數，

--throughput 每秒鐘傳送的記錄數，

--producer-props bootstrap.servers=localhost:9092 傳送端的配置資訊

這裡面比之前版本少了好多引數，提示說明producer-props中有一些配置提供，上官網檢視：找到目錄：producer configs 發現其中有一些有用的配置，比如acks，batch.size，ssl等，不過這些配置都有預設值，如果值不一樣則可以根據提示設定引數，**比較長，就不摘抄過來了，根據路徑過去看。

by the way

kafka效能與每次寫入的record size有關，每條記錄的大小會嚴重影響效能，當我記錄位元組數設定為200時，單點壓測效能接近50w requests per second

kafka效能測試1 0 0

kafka效能測試1 0 0

kafka效能測試

kafka效能測試

相關推薦