162

According to the rasbery pi documentation, You can load your OS to a flash card with either /dev/disk or /dev/rdisk.

rdisk stands for raw disk.

/dev/disk is a block level device, why would rdisk be 20 times faster?

Using Mac OSX

Note: In OS X each disk may have two path references in /dev: /dev/disk# is a buffered device, which means any data being sent undergoes extra processing. /dev/rdisk# is a raw path, which is much faster, and perfectly OK when using the dd program. On a Class 4 SD card the difference was around 20 times faster using the rdisk path.

bot47
  • 1,842
spuder
  • 9,945
  • 6
    As a side note, I ran a test and rdisk actually took much longer. – spuder Aug 13 '13 at 22:26
  • 4
    As another side note, I felt I had to test too then, and found that an rdisk copy (via dd) was almost exactly 4 times faster than using the disk counterpart. – Travis Griggs Jan 07 '14 at 17:57
  • Mac OSX 10.9.1 (MacBook Pro 15-inch, Early 2011) – Travis Griggs Jan 07 '14 at 19:13
  • 2
    I thought "rdisk" was a typo in some instructions for a Raspberry Pi SD card image I was reading. Upon further investigation I googled the difference and found this thread. Turns out in my case it was 13 times faster to write a 1.7GB image to an SD card using /dev/rdisk instead of /dev/disk! Macbook Pro Retina 13", early 2015 model. – tobias.mcnulty Aug 01 '15 at 00:48
  • 2
    As yet another sidenote, I just dd'ed an Ubuntu ARM image to my newly purchased Sandisk Extreme Pro MicroSD card. /dev/disk1 wrote at 2.3 MB/s, while /dev/rdisk1 wrote at 83.7 MB/s, or 36.4 times as fast. – DanielSmedegaardBuus Mar 11 '16 at 17:07
  • MacPro (Late 2013) writing Raspbian Buster full image (~6GB) using dd bs=1m if=2019-07-10-raspbian-buster-full.img of=/dev/rdisk2 conv=sync 43205235 bytes/sec; just /dev/disk2 5315079 bytes/sec so ~8x faster with rdisk – tantrix Aug 07 '19 at 22:20

7 Answers7

116

The accepted answer is right, but it doesn’t go into much detail.

One of the key differences between /dev/disk and /dev/rdisk, when you access them from user space, is that /dev/disk is buffered. The read/write path for /dev/disk breaks up the I/O into PAGE_SIZE chunks (4KB on x86 and arm, 16KB on arm64), which it reads into the buffer cache, and then copies into the user space buffer (and then issues the next chunk-sized read…). This is nice in that you can do unaligned reads and writes, and it just works. In contrast, /dev/rdisk basically just passes the read or write straight to the device, which means the start and end of the I/O need to be aligned on sector boundaries.

If you do a read or write larger than one sector to /dev/rdisk, that request will be passed straight through. The lower layers may break it up (eg., USB breaks it up into 128KB pieces due to the maximum payload size in the USB protocol), but you generally can get bigger and more efficient I/Os. When streaming, like via dd, 128KB to 1MB are pretty good sizes to get near-optimal performance on current non-RAID hardware.

The caching being done by /dev/disk’s read and write paths is very simple and almost brain dead. It caches even if not strictly necessary; like if the device could memory map and directly transfer into your app's buffer. It does small (4KB/16KB) I/Os, which leads to a lot of per-I/O overhead. It does not do any read ahead or write behind.

John James
  • 3
astruct
  • 2,004
111

From man hdiutil:

/dev/rdisk nodes are character-special devices, but are "raw" in the BSD sense and force block-aligned I/O. They are closer to the physical disk than the buffer cache. /dev/disk nodes, on the other hand, are buffered block-special devices and are used primarily by the kernel's filesystem code.

In layman's terms /dev/rdisk goes almost directly to disk and /dev/disk goes via a longer more expensive route

Daniël W. Crompton
  • 2,034
  • 19
    why use disk when you can use rdisk? – user391339 Jan 04 '15 at 22:34
  • 22
    @user391339 Because caching is still a desirable thing. In cases where you have removable media, you want to get the data on the physical device as fast as possible, because you want the data in another physical location. Internal hard drives are a different story. You usually don't carry them around, so you don't care when the data is actually written to the device. When you cache data written to/read from devices that's a more expensive way of writing to the disk, but your programs are still faster, since they don't need to wait until all the data they want to write is written to the disk. – Kritzefitz May 12 '15 at 09:01
  • @Dan, Re "force"; meaning? – Pacerier Mar 03 '18 at 00:19
  • Not sure Krit's explanation works for me. My case may be a bit special wanting to do faster large external disk one-time scanning, and I'm not sure which method works better for that, but otherwise I am still fine with normal use and waiting for it to eject as is common practice. In Windows however I have always preferred the 'quick unplug' profile which I guess mirrors Mac's more 'raw' mode because it advertises less filesystem corruption in unexpected connection loss events. – Pysis Jun 28 '19 at 11:23
7

It seems /dev/disk and /dev/rdisk works different for HDDs and SSDs. Wanna checked it for MicroSD card. Just wrote 2GB disk image to Sandisk Ultra MicroSD 64GB (https://www.amazon.com/gp/product/B073JYVKNX).

Repeated tests several times, but results were stable: 17MB/s for /dev/disk vs 20MB/s for /dev/rdisk. Changing bs=1m to bs=16m gives absolutely no difference in writing speed.

  1. Writing to /dev/disk2

    sudo dd if=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531.img of=/dev/disk2 bs=1m
2094006272 bytes transferred in 121.860007 secs (17183704 bytes/sec)

  2. Writing to /dev/rdisk2

    $ sudo dd if=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531.img of=/dev/rdisk2 bs=1m
2094006272 bytes transferred in 102.743870 secs (20380839 bytes/sec)

Then I decided to test reading speed: 26MB/s for /dev/disk vs 87MB/s for /dev/rdisk. Changing bs=1m to bs=16m gives absolutely no difference in reading speed.

  1. Reading from /dev/disk2

    sudo dd if=/dev/disk2 of=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531-2.img bs=1m
257949696 bytes transferred in 9.895572 secs (26067184 bytes/sec)

  2. Reading from /dev/rdisk2

    $ sudo dd if=/dev/rdisk2 of=~/Downloads/ubuntu-18.04-4.14-minimal-odroid-xu4-20180531.img bs=1m
877658112 bytes transferred in 10.021974 secs (87573377 bytes/sec)
k06a
  • 183
5

I know that this is an old thread, but other folks might be interested in the speed implications of what I've tried. I want to backup my internal SSD in my MacBook Pro 13" Retina (with a Silicon Power 1 TB SSD) to an external USB 3.0 2.5" hard disk drive, wanting to capture both the macOS and BOOTCAMP partitions. My initial command line was:

sudo dd if=/dev/disk0 of=/dev/disk2 bs=1m

The results were a copy rate of ~31.3 MB/second. This was just too long to keep me waiting. So, on the second attempt, the command line was:

sudo dd if=/dev/rdisk0 of=/dev/rdisk2 bs=1m

Using /dev/rdisk instead of /dev/disk significantly sped things up, to about 98.4 MB/second! However, it gets even better. So, for the third attempt, I used this command line:

sudo dd if=/dev/rdisk0 of=/dev/rdisk2 bs=1m conv=sparse

The sparse option tells DD not to bother writing to output blocks that are all 0s on the input. What's cool is that this gets much faster than you'd think, even while in the middle of "full" areas of the disk. On any drive that isn't full, you'll have huge chunks of 0s, further speeding up DD. So far, at least, DD is just about running along at my hard disk's theoretical transfer speed: ~116.4 MB/second, and it hasn't yet reached those large blank areas.

Give these options a try – they work! Please note: CAREFULLY change if= and of= to properly point to the correct drives listed by (for Macs):

diskutil list
G-Man Says 'Reinstate Monica'
  • 7,827
Mark J Foster
  • 51
  • 5
    conv=sparse is great when you’re copying files.   I’d be concerned that it might introduce corruption when copying an entire disk, a partition or a filesystem, unless you know with 100% certainty that the destination disk contains nothing but zeroes. – G-Man Says 'Reinstate Monica' Aug 02 '19 at 22:05
1

For the record, in macOS High Sierra at least, /dev/disk appears to be much faster than /dev/rdisk. Running either dd or ddrescue, my throughput comparison copying from a magnetic HD to an SSD was 3.7MBps using /dev/rdisk, and 45MBps using /dev/disk. So, on later versions of macOS, it may be best to use /dev/disk instead of /dev/rdisk for best performance.

JLG
  • 41
  • 2
    While writing a 4.6GB raspbian-stretch image from the internal SSD storage to an SD card with dd and a bs of 1MB /dev/rdisk still performs much faster than /dev/disk on a late 2013 macbook pro running macOS 10.13.2. It took 27.16 minutes using /dev/disk and only 5.18 minutes using /dev/rdisk. – digitaladdictions Jan 22 '18 at 21:03
  • @digitaladdictions just made a few tests on latest macOS: https://superuser.com/a/1346063/126537 – k06a Aug 04 '18 at 07:27
0

I think before argue which path node is faster or dive into a serial tests. We should consider other factor that will dramatically affect final read/write speed.

like Micro SD card spec, class 4/10/HC I ... sd card reader chip and interface, usb 1.1/2.0/3.0/3.1 os total memory/free memory, os load, os harddisk type, HDD/SSD, HDD spin speed and cache size, SSD size/cache/free space/ os harddisk interface, ata/sata/esata,

if any factor became a bottleneck, then we will got false conclusion.

here is my result: osx 10.12.6, ssd,

read microSD 16G by a usb 2.0 card read and write to external 3.5 inch HDD by usb 3.0,

15193+1 records in
15193+1 records out
15931539456 bytes transferred in 1423.067033 secs (11195214 bytes/sec)

write microSD 32G thru internal card read and data source is external 3.5 inch HDD by usb 3.0,

0+253945 records in
0+253945 records out
15931539456 bytes transferred in 440.093686 secs (36200336 bytes/sec)

you can see, write speed > read speed !!

Digital Life
  • 1
0

Many of the speed discrepancies are quite likely the result of the type of storage media being used, as a consequence of the mechanism of read or write access.

It may be faster to use /dev/disk with storage media that physically spins, like HDD & tape drive (magnetic) or CD & DVD (optical). The buffered stream would allow time to properly position the disk and to either make the physical change to the disk (via magnetic head or laser) or allow the drive hardware to pick up the data signals from the disk.

It's probably faster to use /dev/rdisk with storage media that is solid state & electronically controlled, like SSD & SD card or USB storage drive. There is no need to physically reposition anything to access any other data location so seek time is almost instantaneous, and the read or write process only involves electricity within a circuit instead of requiring high-resolution magnetic field fluctuations or precision-timed laser beams which have to travel a small distance to reach the storage medium. Any time used to fill the software buffer would be spent "holding" the information until it is deemed useful, but with no processing bottleneck ahead of it, it becomes useful as soon as the hardware bus can take it (it can usually take it immediately, unless it's already being fed at its maximum throughout rate).

Disclaimer: This is based on logic, not experimentation, and my understanding of computer systems and hardware, which is not yet complete in all aspects.

Matt
  • 1