Fio is a free and open source tool that can be used for benchmark and hardware verification. It supports lots of I/O engines including, libaio,sync, nmap, syslet, network, slice and many more. It also supports many distributions such as Linux, OpenBSD, OpenSolaris, HP-UX, Android and Windows. It can work on both files and block devices.

In this tutorial, we will learn how to install Fio in Linux, we will also learn how to use Fio with some examples.

Warning: Never try this Software on a Production Machine. It may damage your Data. Try on a Staging / Test server instead.

Intall `fio` in your Linux machine

* sudo apt-get install fio (Ubuntu)
* sudo yum install fio (Centos)

Example 1. Write Test This command will writes a total 2GB files [4 jobs x 512 MB = 2GB] running 4 processes at a time:

fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting

Output will be like below

randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 4 processes
Jobs: 2 (f=2): [_(1),w(1),_(1),w(1)][100.0%][r=0KiB/s,w=30.1MiB/s][r=0,w=7705 IOPS][eta 00m:00s]
randwrite: (groupid=0, jobs=4): err= 0: pid=10682: Wed Oct 16 09:53:52 2019
  write: IOPS=15.4k, BW=60.2MiB/s (63.1MB/s)(2048MiB/34012msec)
    slat (usec): min=2, max=35241, avg=250.68, stdev=2553.62
    clat (nsec): min=564, max=21492k, avg=1534.15, stdev=43194.89
     lat (usec): min=2, max=35249, avg=252.94, stdev=2554.53
    clat percentiles (nsec):
     |  1.00th=[  580],  5.00th=[  588], 10.00th=[  588], 20.00th=[  596],
     | 30.00th=[  636], 40.00th=[  748], 50.00th=[ 1352], 60.00th=[ 1912],
     | 70.00th=[ 1928], 80.00th=[ 1944], 90.00th=[ 1992], 95.00th=[ 2320],
     | 99.00th=[ 3568], 99.50th=[ 5024], 99.90th=[17536], 99.95th=[21632],
     | 99.99th=[35584]
   bw (  KiB/s): min= 9325, max=268203, per=25.18%, avg=15527.63, stdev=31588.75, samples=268
   iops        : min= 2331, max=67050, avg=3881.87, stdev=7897.14, samples=268
  lat (nsec)   : 750=40.20%, 1000=7.00%
  lat (usec)   : 2=42.99%, 4=8.96%, 10=0.66%, 20=0.14%, 50=0.06%
  lat (usec)   : 100=0.01%, 500=0.01%
  lat (msec)   : 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=0.98%, sys=3.05%, ctx=4975, majf=0, minf=126
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,524288,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=60.2MiB/s (63.1MB/s), 60.2MiB/s-60.2MiB/s (63.1MB/s-63.1MB/s), io=2048MiB (2147MB), run=34012-34012msec

Disk stats (read/write):
  vda: ios=0/89875, merge=0/0, ticks=0/4166446, in_queue=4168394, util=98.40%

You can see that the value w=7705 IOPS in the output. Which means, you are getting 7705 IOPS on this Disk.

Example 2. Read Test

fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting

The output will be look like below

randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
...
fio-3.7
Starting 4 processes
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
randread: Laying out IO file (1 file / 512MiB)
Jobs: 2 (f=2): [r(2),_(2)][100.0%][r=11.1MiB/s,w=0KiB/s][r=2840,w=0 IOPS][eta 00m:00s]
randread: (groupid=0, jobs=4): err= 0: pid=10883: Wed Oct 16 10:05:41 2019
   read: IOPS=6388, BW=24.0MiB/s (26.2MB/s)(2048MiB/82068msec)
    slat (usec): min=133, max=150858, avg=596.82, stdev=974.01
    clat (usec): min=9, max=160906, avg=9053.14, stdev=4049.75
     lat (usec): min=473, max=161456, avg=9651.12, stdev=4200.91
    clat percentiles (msec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[    8],
     | 30.00th=[    8], 40.00th=[    9], 50.00th=[    9], 60.00th=[    9],
     | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
     | 99.00th=[   17], 99.50th=[   23], 99.90th=[   72], 99.95th=[   80],
     | 99.99th=[  157]
   bw (  KiB/s): min= 4080, max=10936, per=25.52%, avg=6522.33, stdev=998.58, samples=630
   iops        : min= 1020, max= 2734, avg=1630.53, stdev=249.69, samples=630
  lat (usec)   : 10=0.01%, 20=0.01%, 50=0.01%, 500=0.01%, 750=0.01%
  lat (usec)   : 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.10%, 10=78.27%, 20=21.04%, 50=0.39%
  lat (msec)   : 100=0.17%, 250=0.03%
  cpu          : usr=0.91%, sys=3.39%, ctx=524299, majf=0, minf=194
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=24.0MiB/s (26.2MB/s), 24.0MiB/s-24.0MiB/s (26.2MB/s-26.2MB/s), io=2048MiB (2147MB), run=82068-82068msec

Disk stats (read/write):
  vda: ios=523763/7, merge=0/1, ticks=304880/6, in_queue=304755, util=99.93%

The read IOPS is r=2840.

Example 3 Read Write Performance Test

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

The Output should be like below

test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.7
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [m(1)][99.3%][r=22.2MiB/s,w=7456KiB/s][r=5691,w=1864 IOPS][eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=10890: Wed Oct 16 10:12:57 2019
   read: IOPS=5177, BW=20.2MiB/s (21.2MB/s)(3070MiB/151783msec)
   bw (  KiB/s): min=10328, max=44184, per=99.81%, avg=20670.94, stdev=2958.55, samples=303
   iops        : min= 2582, max=11046, avg=5167.73, stdev=739.64, samples=303
  write: IOPS=1730, BW=6922KiB/s (7088kB/s)(1026MiB/151783msec)
   bw (  KiB/s): min= 3360, max=15080, per=99.82%, avg=6908.75, stdev=998.36, samples=303
   iops        : min=  840, max= 3770, avg=1727.17, stdev=249.59, samples=303
  cpu          : usr=1.02%, sys=3.49%, ctx=376974, majf=0, minf=26
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=20.2MiB/s (21.2MB/s), 20.2MiB/s-20.2MiB/s (21.2MB/s-21.2MB/s), io=3070MiB (3219MB), run=151783-151783msec
  WRITE: bw=6922KiB/s (7088kB/s), 6922KiB/s-6922KiB/s (7088kB/s-7088kB/s), io=1026MiB (1076MB), run=151783-151783msec

Disk stats (read/write):
  vda: ios=782984/261702, merge=0/0, ticks=7272735/2387065, in_queue=9679325, util=100.00%