zfs storage pool tuning for performance

ZFS Storage pool tunning tips

- Tuning Open Solaris

After tuning the following Result : 45+MB/s
root@10:/mnt/vxstor2_1001# dd if=/dev/zero of=100M bs=10M count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 22.5591 s, 46.5 MB/s
in OpenSolaris SNV_134 create  the following files.
append the following lines to /etc/system
* "fsflush" tuning
 set tune_t_fsflushr = 5
 set autoup = 300
* Disable the Automatic Interrupt Adjustment
 set dld:dld_opt = 2
* Disable "soft rings"
 set ip:ip_squeue_fanout = 0
 set ip:ip_soft_rings_cnt = 0

create a file in /etc/rc3.d/S99ndd   
 #!/bin/bash

NDD=/usr/sbin/ndd

$NDD -set /dev/tcp tcp_xmit_hiwat 1048576
$NDD -set /dev/tcp tcp_recv_hiwat 8388608
$NDD -set /dev/tcp tcp_max_buf 8388608
$NDD -set /dev/udp udp_xmit_hiwat 1048576
$NDD -set /dev/udp udp_recv_hiwat 8388608
$NDD -set /dev/udp udp_max_buf 8388608
$NDD -set /dev/tcp tcp_conn_req_max_q 65536
$NDD -set /dev/tcp tcp_conn_req_max_q0 65536
$NDD -set /dev/tcp tcp_fin_wait_2_flush_interval 67500

Note: This NFS tunning will work only with 2 nics with Aggregate like bonding.


ZFS ZIL on Separate Device
- Seperate Device of ZIL (ZFS Intend log )
Generally by default the ZIL and the data is written on the same zpool, Wherein the disk overhead will come into picture. If we use separate device for ZIL to write in the performance will boost up
Note: Recommended SSD device can be used and SSD ideally can be half of the RAM size in the Storage subsystem


JUMBO Frames on NICs
- We can also use JUMBO Frames in both end of the NIC associated with Storage Subsystem
Note: Jumbo frames are not supported by all the nics.
How to figure out, if the existing nic support jumbo frames as follows
# ifconfig mtu 9000
If the Nic accepts the value without throwing any error, it means it support or else it is not.


Storage Separate Network
- Storage separate private network only for " Data Writes" will definitely boost the performance of the storage