I’m trying to figure out what could be causing the bottleneck on my NAS. By my calculations I should be able to easily hit over 1000 MB/sec sustained read but I don’t know why I’m not. Any useful troubleshooting advice would be appreciated.
System:
OS: TrueNAS-SCALE-22.12.2
CPU: Threadripper 1950X
RAM: 128 GB DDR4
Network: 10 Gbit (Intel X710-DA2 -> Mikrotik CRS317-1G-16S+RM -> AQUANTIA AQC107)
Controller: Adaptec PMC ASR-72405
Drives: 8 Seagate Exos X20
ZFS Pool Config: 2 VDevs (4 drives each) in RAIDZ1
Run iozone to test pool.
Small tests will reside in ARC and give you false high. 512gb is minimum, or change tunable to reduce arc size
Iperf to test network.
FYI Scale has a bug limiting ARC to half your physical ram. Apparently there is beta test for fix posted on TrueNas forum. And should be in production in 6 months-ish. This won’t impact synthetic tests, but in real life you may want to have the extra cache
Why his numbers are right on the money. Need another vdev to break the 1k mark. Especially if looking for consistent over 1k numbers.