接著續上次提到的sqlldr的效能問題,載入乙個表資料400多萬條記錄,竟然用了6個多小時。最後大家爭論不休的時候,我發現應該是網路的問題。
àoriginal logs for issue table, loading around 6 hours.
total logical records skipped: 0
total logical records read: 4096786
total logical records rejected: 0
total logical records discarded: 0
run began on wed jun 11 08:52:55 2014
run ended on wed jun 11 14:57:40 2014
elapsed time was: 06:04:44.05
cpu time was: 00:00:38.18
àafter network enhanced, loading time has reduced to around 4 mins.
total logical records skipped: 0
total logical records read: 4096786
total logical records rejected: 0
total logical records discarded: 0
run began on mon jun 16 12:58:01 2014
run ended on mon jun 16 13:02:15 2014
elapsed time was: 00:04:13.28
cpu time was: 00:00:30.00
àconsidering impact from index, i loaded data again, keep table structure,index exactly the same.
total logical records skipped: 0
total logical records read: 4096786
total logical records rejected: 0
total logical records discarded: 0
run began on mon jun 16 13:09:34 2014
run ended on mon jun 16 13:16:37 2014
elapsed time was: 00:07:02.61
cpu time was: 00:00:31.23
àtune sqlldr command with below options, keep table structure,index exactly the same.
parallel=true bindsize=7500000 readsize=7500000 streamsize=7500000 rows=50000
total logical records skipped: 0
total logical records read: 4096786
total logical records rejected: 0
total logical records discarded: 0
run began on mon jun 16 13:21:13 2014
run ended on mon jun 16 13:24:19 2014
elapsed time was: 00:03:05.75
cpu time was: 00:00:41.27所以綜上所述,在排查了cpu,io,快取,網路的因素後,還可以使用一些選項來優化sqlldr,在某種程度上得到提公升。
效能測試調優
效能測試的目的就評估當前系統效能的指標,分析定位解決效能瓶頸,預防規避效能風險。效能分析是為了確定導致效能瓶頸的原因,而調優就是用來解決效能瓶頸。通過某些手段讓系統效能得到提高,是效能調優的主要目的。效能分析主要有兩種方法 1.將測試結果與使用者需求做比較,如果達到使用者需求,則測試通過。系統滿足1...
MySQL效能測試調優
作業系統 基本操作 檢視磁碟分割槽mount選項 mount 永久修改分割槽mount選項 系統重啟後生效 修改檔案 etc fstab 中對應分割槽的mount options列的值 sudo t ext4 o remount,noatime,errors remount or 檔案系統優化 ex...
效能測試之Nginx效能調優課程
效能測試之nginx效能調優 課程出自 學途無憂網 課程從介紹nginx開始,包括nginx的安裝 配置 監控 調優等方方面面,可以全面深入淺出的學習nginx。課時1 nginx為什麼這麼牛b 課時2 linux下快速安裝nginx 課時3 nginx監控分析神器 課時4 nginx關鍵引數分析 ...