[[!tag fling network bandwidth]]
I was curious about how fast I can transfer data in various contexts. I did some informal benchmarks: these are not scientific, and your results might be different.
The data transferred is a one terabyte sparse file:
$ truncate --size 1T terabyte
First is just copying files on a disk: from the file to /dev/null.
| software | sender | receiver | time (s) | speed (MB/s) |
|---|---|---|---|---|
| cat | exolobe1 | exolobe1 | 274 | 3826 |
| dd | exolobe1 | exolobe1 | 2729 | 384 |
| dd bs=1M | exolobe1 | exolobe1 | 304 | 3449 |
| dd bs=16M | exolobe1 | exolobe1 | 314 | 3338 |
| dd bs=1G | exolobe1 | exolobe1 | 328 | 3196 |
Conclusion: cat is quite good, and setting almost any size buffer on dd is a win over the default. (dd has other options that cat lacks, such as oflag=direct, which means its still interesting for me, despite a little slower than cat.)
Then network transfers. I use the fling software, which is quite efficient when security is not needed. It does not handle sparse files specially.
Source of the transfer is my laptop (exolobe1), target is the same machine over localhost, a VM under libvirt on the same laptop, a VM under qemu-system on the same laptop, or my server (exolobe2), or a VM on the server (holywood2).
$ fling -p terabyte receiver 8888 # sender
$ receiver: fling -r 8888 > /dev/null # receiver
For extra kicks, transfers between nested VMs inside a VM on the laptop, where the outer VM is running under libvirt or qemu-system. The nested VMs always run under libvirt.
| software | sender | receiver | time (s) | speed (MB/s) |
|---|---|---|---|---|
| fling | exolobe1 | exolobe1 | 395 | 2655 |
| fling | exolobe1 | libvirt guest | 613 | 1710 |
| fling | exolobe1 | qemu-system-x86_64 | 4653 | 225 |
| fling | exolobe1 | exolobe2 | too long | 111 |
| fling | exolobe1 | holywood2 | too long | 78 |
| fling | nested libvirt-guest | other nested guest | too long | 280 |
| fling | nested qemu-system | other such | too long | 12 |
The "too long" results is because I got impatient. The speeds in those cases are from fling -p output.
Conclusion: localhost is fast. libvirt networking inside the same physical machine is fast. Every other case is slow.
I hope that's useful to someone.