мой сервер freebsd 10 работает на zfs raidz. У меня проблема с чтением файлов, протестируйте
dd of=/dev/null if=/tmp/test/1000mb.bin bs=1024k count=1000
953+1 records in
953+1 records out
1000000000 bytes transferred in 31.144059 secs (32108853 bytes/sec)
Как видите, 1 ГБ данных читается за 31 секунду. :(
Когда я пытаюсь подсчитать md5 этого файла, он запускается и не отвечает в течение нескольких минут. Я работаю с большими видеофайлами на этом сервере ~ 2Гб, а также с маленькими большими пальцами ~ 300Кб, это сервер хранения, у нас мало к нему доступа. Вот конфигурация сервера:
Четырехъядерный процессор Intel® Core ™ i7-2600 с технологией Hyper-Threading, ОЗУ DDR3 16 ГБ, 4 жестких диска SATA по 4 ТБ 3 Гбит / с
ZFS обновлена. Мы отключили первичный и вторичный кеш. Мы уже настроили zfs:
/boot/loader.conf
zfs_load="YES"
vfs.root.mountfrom="zfs:tank/root"
geom_mirror_load="YES"
vfs.zfs.zio.use_uma="0"
vfs.zfs.arc_min="2048M"
vfs.zfs.arc_max="8186M"
vm.kmem_size_max="32G"
vm.kmem_size="24G"
vfs.zfs.prefetch_disable="1"
vfs.zfs.txg.timeout="5"
kern.maxvnodes=250000
vfs.zfs.write_limit_override=1073741824
Некоторая информация: zpool статус
pool: tank
state: ONLINE
scan: scrub repaired 0 in 24h47m with 0 errors on Fri Sep 5 14:22:04 2014
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
ada0p3 ONLINE 0 0 0
ada1p3 ONLINE 0 0 0
ada2p3 ONLINE 0 0 0
ada3p3 ONLINE 0 0 0
errors: No known data errors
zfs получить весь танк
NAME PROPERTY VALUE SOURCE
tank type filesystem -
tank creation Thu Jul 31 14:45 2014 -
tank used 2.65T -
tank available 4.20T -
tank referenced 209K -
tank compressratio 1.00x -
tank mounted no -
tank quota none default
tank reservation none default
tank recordsize 128K default
tank mountpoint none local
tank sharenfs off default
tank checksum on default
tank compression off default
tank atime on default
tank devices on default
tank exec on default
tank setuid on default
tank readonly off default
tank jailed off default
tank snapdir hidden default
tank aclmode discard default
tank aclinherit restricted default
tank canmount on default
tank xattr on default
tank copies 1 default
tank version 5 -
tank utf8only off -
tank normalization none -
tank casesensitivity sensitive -
tank vscan off default
tank nbmand off default
tank sharesmb off default
tank refquota none default
tank refreservation none default
tank primarycache none local
tank secondarycache none local
tank usedbysnapshots 0 -
tank usedbydataset 209K -
tank usedbychildren 2.65T -
tank usedbyrefreservation 0 -
tank logbias latency default
tank dedup off default
tank mlslabel -
tank sync standard default
tank refcompressratio 1.00x -
tank written 209K -
tank logicalused 2.51T -
tank logicalreferenced 43.5K -
Спасибо за помощь.
ОБНОВИТЬ:
gstat во время пробежки дд
dT: 1.018s w: 1.000s
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
4 275 185 4780 5.2 89 2514 2.3 42.9| ada0
5 352 245 5503 9.6 106 2585 2.4 77.9| ada1
0 0 0 0 0.0 0 0 0.0 0.0| ada0p1
0 0 0 0 0.0 0 0 0.0 0.0| ada0p2
4 275 185 4780 5.2 89 2514 2.3 43.0| ada0p3
0 0 0 0 0.0 0 0 0.0 0.0| ada1p1
0 0 0 0 0.0 0 0 0.0 0.0| ada1p2
5 352 245 5503 9.6 106 2585 2.6 78.2| ada1p3
0 359 261 5350 11.2 96 2518 4.8 88.4| ada2
2 238 153 4678 18.4 83 2565 1.9 83.8| ada3
0 0 0 0 0.0 0 0 0.0 0.0| gptid/86db209a-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| gptid/87034dda-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| gptid/88396782-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| gptid/8861da11-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| ada2p1
0 0 0 0 0.0 0 0 0.0 0.0| ada2p2
0 359 261 5350 11.2 96 2518 4.9 88.4| ada2p3
0 0 0 0 0.0 0 0 0.0 0.0| ada3p1
0 0 0 0 0.0 0 0 0.0 0.0| ada3p2
2 238 153 4678 18.4 83 2565 2.0 83.8| ada3p3
0 0 0 0 0.0 0 0 0.0 0.0| gptid/8999db59-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| gptid/89c1f2eb-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| gptid/8af4d439-18b0-11e4-977c-5404a6f17240
0 0 0 0 0.0 0 0 0.0 0.0| gptid/8b1c89ed-18b0-11e4-977c-5404a6f17240
ОБНОВИТЬ: Информация об оборудовании
ada0 at ahcich2 bus 0 scbus0 target 0 lun 0
ada0: <HGST HMS5C4040BLE640 MPAOA5D0> ATA-8 SATA 3.x device
ada0: Serial Number PL1331LAGS5T0H
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ahcich3 bus 0 scbus1 target 0 lun 0
ada1: <HGST HMS5C4040BLE640 MPAOA5D0> ATA-8 SATA 3.x device
ada1: Serial Number PL1331LAGS40KH
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6
ada2 at ahcich4 bus 0 scbus2 target 0 lun 0
ada2: <HGST HMS5C4040BLE640 MPAOA5D0> ATA-8 SATA 3.x device
ada2: Serial Number PL1331LAGS5LHH
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad8
ada3 at ahcich5 bus 0 scbus3 target 0 lun 0
ada3: <HGST HMS5C4040BLE640 MPAOA5D0> ATA-8 SATA 3.x device
ada3: Serial Number PL1331LAGS15VH
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada3: Previously was known as ad10
ahci0: <Intel Cougar Point AHCI SATA controller> port 0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xfe501000-0xfe5017ff irq 20 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
ada0 at ahcich2 bus 0 scbus0 target 0 lun 0
ada1 at ahcich3 bus 0 scbus1 target 0 lun 0
ada2 at ahcich4 bus 0 scbus2 target 0 lun 0
ada3 at ahcich5 bus 0 scbus3 target 0 lun 0
ses0 at ahciem0 bus 0 scbus4 target 0 lun 0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ОБНОВИТЬ: zdb -m бак
Metaslabs:
vdev 0
metaslabs 115 offset spacemap free
--------------- ------------------- --------------- -------------
metaslab 0 offset 0 spacemap 36 free 5.12G
metaslab 1 offset 2000000000 spacemap 57 free 7.84G
metaslab 2 offset 4000000000 spacemap 58 free 18.7G
metaslab 3 offset 6000000000 spacemap 60 free 5.00G
metaslab 4 offset 8000000000 spacemap 61 free 4.96G
metaslab 5 offset a000000000 spacemap 62 free 5.06G
metaslab 6 offset c000000000 spacemap 63 free 5.05G
metaslab 7 offset e000000000 spacemap 64 free 5.10G
metaslab 8 offset 10000000000 spacemap 67 free 5.08G
metaslab 9 offset 12000000000 spacemap 70 free 6.30G
metaslab 10 offset 14000000000 spacemap 72 free 5.09G
metaslab 11 offset 16000000000 spacemap 73 free 5.07G
metaslab 12 offset 18000000000 spacemap 76 free 4.93G
metaslab 13 offset 1a000000000 spacemap 77 free 4.98G
metaslab 14 offset 1c000000000 spacemap 78 free 4.93G
metaslab 15 offset 1e000000000 spacemap 79 free 5.09G
metaslab 16 offset 20000000000 spacemap 80 free 5.08G
metaslab 17 offset 22000000000 spacemap 81 free 4.96G
metaslab 18 offset 24000000000 spacemap 84 free 5.22G
metaslab 19 offset 26000000000 spacemap 85 free 5.95G
metaslab 20 offset 28000000000 spacemap 87 free 5.03G
metaslab 21 offset 2a000000000 spacemap 88 free 5.05G
metaslab 22 offset 2c000000000 spacemap 35 free 6.40G
metaslab 23 offset 2e000000000 spacemap 59 free 5.08G
metaslab 24 offset 30000000000 spacemap 89 free 5.19G
metaslab 25 offset 32000000000 spacemap 90 free 5.05G
metaslab 26 offset 34000000000 spacemap 91 free 5.08G
metaslab 27 offset 36000000000 spacemap 92 free 5.11G
metaslab 28 offset 38000000000 spacemap 93 free 6.27G
metaslab 29 offset 3a000000000 spacemap 94 free 6.42G
metaslab 30 offset 3c000000000 spacemap 95 free 5.11G
metaslab 31 offset 3e000000000 spacemap 69 free 20.1G
metaslab 32 offset 40000000000 spacemap 71 free 13.0G
metaslab 33 offset 42000000000 spacemap 99 free 13.7G
metaslab 34 offset 44000000000 spacemap 100 free 20.0G
metaslab 35 offset 46000000000 spacemap 102 free 74.6G
metaslab 36 offset 48000000000 spacemap 103 free 117G
metaslab 37 offset 4a000000000 spacemap 105 free 117G
metaslab 38 offset 4c000000000 spacemap 106 free 72.8G
metaslab 39 offset 4e000000000 spacemap 107 free 25.4G
metaslab 40 offset 50000000000 spacemap 83 free 71.8G
metaslab 41 offset 52000000000 spacemap 108 free 24.5G
metaslab 42 offset 54000000000 spacemap 86 free 23.1G
metaslab 43 offset 56000000000 spacemap 110 free 22.0G
metaslab 44 offset 58000000000 spacemap 34 free 5.14G
metaslab 45 offset 5a000000000 spacemap 66 free 126G
metaslab 46 offset 5c000000000 spacemap 112 free 121G
metaslab 47 offset 5e000000000 spacemap 113 free 125G
metaslab 48 offset 60000000000 spacemap 114 free 125G
metaslab 49 offset 62000000000 spacemap 115 free 128G
metaslab 50 offset 64000000000 spacemap 116 free 128G
metaslab 51 offset 66000000000 spacemap 117 free 128G
metaslab 52 offset 68000000000 spacemap 120 free 78.0G
metaslab 53 offset 6a000000000 spacemap 68 free 5.04G
metaslab 54 offset 6c000000000 spacemap 75 free 4.99G
metaslab 55 offset 6e000000000 spacemap 97 free 128G
metaslab 56 offset 70000000000 spacemap 98 free 71.0G
metaslab 57 offset 72000000000 spacemap 101 free 118G
metaslab 58 offset 74000000000 spacemap 121 free 107G
metaslab 59 offset 76000000000 spacemap 123 free 101G
metaslab 60 offset 78000000000 spacemap 0 free 128G
metaslab 61 offset 7a000000000 spacemap 0 free 128G
metaslab 62 offset 7c000000000 spacemap 82 free 5.00G
metaslab 63 offset 7e000000000 spacemap 124 free 125G
metaslab 64 offset 80000000000 spacemap 111 free 122G
metaslab 65 offset 82000000000 spacemap 109 free 123G
metaslab 66 offset 84000000000 spacemap 65 free 127G
metaslab 67 offset 86000000000 spacemap 0 free 128G
metaslab 68 offset 88000000000 spacemap 0 free 128G
metaslab 69 offset 8a000000000 spacemap 0 free 128G
metaslab 70 offset 8c000000000 spacemap 0 free 128G
metaslab 71 offset 8e000000000 spacemap 0 free 128G
metaslab 72 offset 90000000000 spacemap 0 free 128G
metaslab 73 offset 92000000000 spacemap 0 free 128G
metaslab 74 offset 94000000000 spacemap 119 free 88.7G
metaslab 75 offset 96000000000 spacemap 74 free 128G
metaslab 76 offset 98000000000 spacemap 0 free 128G
metaslab 77 offset 9a000000000 spacemap 104 free 128G
metaslab 78 offset 9c000000000 spacemap 0 free 128G
metaslab 79 offset 9e000000000 spacemap 0 free 128G
metaslab 80 offset a0000000000 spacemap 0 free 128G
metaslab 81 offset a2000000000 spacemap 122 free 123G
metaslab 82 offset a4000000000 spacemap 0 free 128G
metaslab 83 offset a6000000000 spacemap 0 free 128G
metaslab 84 offset a8000000000 spacemap 96 free 89.2G
metaslab 85 offset aa000000000 spacemap 129 free 128G
metaslab 86 offset ac000000000 spacemap 0 free 128G
metaslab 87 offset ae000000000 spacemap 0 free 128G
metaslab 88 offset b0000000000 spacemap 0 free 128G
metaslab 89 offset b2000000000 spacemap 0 free 128G
metaslab 90 offset b4000000000 spacemap 0 free 128G
metaslab 91 offset b6000000000 spacemap 0 free 128G
metaslab 92 offset b8000000000 spacemap 0 free 128G
metaslab 93 offset ba000000000 spacemap 0 free 128G
metaslab 94 offset bc000000000 spacemap 0 free 128G
metaslab 95 offset be000000000 spacemap 0 free 128G
metaslab 96 offset c0000000000 spacemap 118 free 128G
metaslab 97 offset c2000000000 spacemap 0 free 128G
metaslab 98 offset c4000000000 spacemap 0 free 128G
metaslab 99 offset c6000000000 spacemap 0 free 128G
metaslab 100 offset c8000000000 spacemap 0 free 128G
metaslab 101 offset ca000000000 spacemap 0 free 128G
metaslab 102 offset cc000000000 spacemap 0 free 128G
metaslab 103 offset ce000000000 spacemap 0 free 128G
metaslab 104 offset d0000000000 spacemap 0 free 128G
metaslab 105 offset d2000000000 spacemap 0 free 128G
metaslab 106 offset d4000000000 spacemap 0 free 128G
metaslab 107 offset d6000000000 spacemap 0 free 128G
metaslab 108 offset d8000000000 spacemap 0 free 128G
metaslab 109 offset da000000000 spacemap 0 free 128G
metaslab 110 offset dc000000000 spacemap 0 free 128G
metaslab 111 offset de000000000 spacemap 0 free 128G
metaslab 112 offset e0000000000 spacemap 0 free 128G
metaslab 113 offset e2000000000 spacemap 0 free 128G
metaslab 114 offset e4000000000 spacemap 0 free 128G
ОБНОВЛЕНИЕ: включение primarycache = all немного увеличивает скорость чтения, спасибо Юджину.
ОБНОВИТЬ: zdb tank | head -n 200 | grep ashift Мне нужно завершить команду CTRL + C примерно через 5 минут выполнения, вот результат:
ashift: 12
ashift: 12
^C