Is it possible to set the logical sector size of a USB hard disk drive?

Question

I have two supposedly identical 3TB Western Digital USB hard disk drives. One contains nearly the full 3TB of data. I want the other to be a backup of that data. There are lots of small files, so running cp would be even slower than copying 3TB over USB using dd. Both drives were bought at the same time, both use MSDOS partition table, but one shows logical/physical sector sizes of 4096/4096 and the other shows 512/4096.

My problem is that I’m trying to copy to the 512/4096 drive, but I can’t create a large enough partition because of the limits of the MSDOS partition table with 512 byte (logical) sectors. I've tried various commands (dd of the first few sectors to partition table, parted, fdisk, gparted, Windows XP “Manage” and Windows 7 “Manage”) but end up with either 2TB max limits or ~300GB partitions that thought they were 3TB when they were created.

Given that one of my drives is running at 4096/4096 then this question doesn’t hold true for my case.

Using GPT is my fall-back option, but it creates a partition that is slightly smaller than the NTFS partition, so I wouldn't be able to just dd the filesystem on the partition.

Is there a way to force 4096/4096 for an MSDOS partition table (either from Linux or Windows) so that I can create a 3TB partition that perfectly matches my source disk?

Answer 1

Many Western Digital external USB drives over 2 TB (including at least some My Book, My Book Essential, Elements, and Easystore drives) can be configured for either 512 byte logical sectors or 4096 byte logical sectors using the WD Quick Formatter tool. When configured for 4096 byte logical sectors, the USB-to-SATA bridge in the enclosure does a translation between 512 byte logical sectors at the SATA interface to the internal drive and 4096 byte logical sectors at the USB interface to the host computer.

With 512 byte logical sectors, an MBR partition table could only use up to 2 TB of a drive. That's because MBR table entries are 32 bits with a max of 2^32 or 4,294,967,296 sectors. 2^32 sectors x 512 bytes/sector is 2 TB. With 4096 byte logical sectors, an MBR partition table can use up to 16 TB of a drive. (2^32 sectors x 4096 bytes/sector is 16 TB) Windows XP only supports MBR partition tables, and so 4096 byte logical sectors are the only way to use all of the space on a drive over 2 TB in Windows XP. The newer GPT partition table format supported in Windows Vista and later does not have the 32-bit limitation, and can support disks larger than 2 TB regardless of the sector size.

The WD Quick Formatter tool (version 2.0.0.65 available for download as of this writing) can enable or disable the sector size translation. This version of the tool does not work correctly under Windows XP, so I recommend running the tool in Windows 7 or later. When run in Windows 7, or in later version of Windows but with Windows 7 compatibility mode, the tool will present two configuration options on the "Format your WD external drive" screen:

• XP Compatible: This option configures the drive for 4096 byte logical sectors, and creates an MBR partition table.
• Most Compatible (Vista or later required): This option configures the drive for 512 byte logical sectors, and creates a GPT partition table.

Both options also create a single partition filling the entire drive, and quick format it in NTFS.

If you run the tool in Windows 8 or later without putting it in Windows 7 compatibility mode, the tool will not present a compatibility option on the formatting screen and it will format in the "most compatible" mode (512 byte logical sectors).

WD Quick Formatter 2.0.0.65 doesn't work correctly in Windows XP: it successfully configures 4096 byte logical sectors, but fails to format correctly. Another tool can subsequently be used to partition and format the drive.

WD Quick Formatter 1.2.0.10 works correctly in Windows XP, but is not available for download from Western Digital anymore.

When the target drive is configured for the same logical sector size as the source, you can copy to it using dd and the copy will work without any need to alter the partition table.

I can confirm that these Western Digital external USB drives can be configured for XP compatibility:

• Easystore 14 TB (WDBCKA0140HBK)

• Easystore 12 TB (WDBCKA0120HBK)

• Easystore 8 TB (WDBCKA0080HBK)

• My Book Essential 1140, 3 TB (WDBACW0030HBK)

• Elements 1021, 3 TB (WDBAAU0030HBK)

Answer 2

I just had a similar issue. I had a Seagate 3TB USB Desktop Expansion drive that I wanted to move into my PC for faster access (I only had USB 2.0 on the PC). Once I did that I couldn't access the filesystem. Returning it to the USB to SATA controller made it work again. Comparing the two I found that in the external case (using the USB to SATA controller) the drive had 4096/4096 logical/physical sector sizes and when connected to the internal motherboard SATA, it had 512/4096 logical/physical sector sizes. Much like the OP sugguests, a 4096 logical sector size allows for > 2TiB partitions, but at the 512 sector size we get the 2TiB limit. I didn't want to copy stuff off (slowly over USB) to somewhere, repartition (GPT), reformat, and copy back, so I kept at it. My ultimate solution on Linux was to:

Delete the original MBR partition table which uses 4k sectors. Use gdisk (GPT variant of fdisk) to create the partition using 512 byte sectors spanning the exact same area. This basically amounted to taking the sectors for start/end of the partition in 4K-land and multiplying by 8. I did have an off-by one on the end, so I added 8 more there and so far so good (e2fsck alerted me to that.) The e2fsk is still going, so I'm not 100% sure I'm safe, but it seems all good in theory and I did mount the files for a bit and they looked fine.

Answer 3

Your issue is likely similar to How to correct 512-byte sector MBR on a 4096-byte sector disk?. In short, it's the limitation of the USB enclosure's firmware by not understanding the 4K sector format

---

The physical sector size is the HDD's native sector size, so there's no way to "force" it to change that value unless you find someway to update the drive's firmware and do a low-level format

Some (old) 4K HDD may support emulated 512-byte sector mode (called 512e, which is what your 512/4096 drive does) in which physical size is still 4K, but it reports as having a 512-byte sector which is the logical size. That means partitions must be aligned properly to 4K or performance will be affected greatly. Technically that reported logical value may be changed by some disk-configuring software or by updating firmware if the manufacturer supports.

However I'm not aware of any such software. If you see different logical values for the same disk then it's because the old disk controller in the external enclosure doesn't understand the new 512e or 4K format. That happens a lot to USB HDDs, like mentioned in Shane Anderson's answer and:

• logical sector size changes depending on whether it is attached via USB or direct SATA
• 4k emulation sata usb controllers
• Do hard disk drives turn on 512e (512byte emulation of 4k sectors) as-needed depending on host controller?
• 3TB HDD showing 512 logical sector size when connected via SATA

In fact many old USB enclosures don't even support drives larger than 2TB

• Anyone know why older USB drive cases have a 2TB size limit?
• Is there a size limit on external USB hard drives?

Worse yet, some enclosures do the reverse thing by combining 512-byte sectors to a 4K one

Complicating matters is certain USB disk enclosures. Some of these enclosures do the reverse of what AF does: They take eight disk sectors and bundle them into one new 4096-byte sector. I'm not sure what the reasoning is behind this move, but one practical advantage is that disks larger than 2TiB can be used with the old MBR partitioning system. One major disadvantage is that a disk partitioned in one of these enclosures can not be used directly or in an enclosure that doesn't do this type of translation. Likewise, a disk prepared without this translation can't be used when it's transferred into such an enclosure.

How to correct 512-byte sector MBR on a 4096-byte sector disk?

Similar issues:

• SATA to USB cable sector size
• Why is my USB drive showing corrupted data when plugged as an internal SATA drive?
• Advanced Format HDDs, USB enclosures and Windows / Linux compatibility

---

Besides, you shouldn't use dd for copying disks. Instead, use some disk cloning tools, they'll smart enough to skip blank clusters and are also capable of cloning between disks with different sizes

Also, there's no "DOS partition table" but MBR and GPT (and some other uncommon formats). Unfortunately gparted and many Linux tools call that "msdos" which is incorrect

With MBR you can only addressed 2³² sectors (2TB with 512-byte sector and 16TB with 4K sector). That's why your 4096/4096 drive works properly. I suppose Shane Anderson's workaround works, but if it doesn't the only way is to use GPT. In fact GPT would be recommended even if you have only one partition because it supports checksumming and a backup partition table, which significantly reduces the chance of accidental corruption/deletion

Answer 4

Yes and no.

You cannot change the physical sector size, and few drives will let you change the transport layer sector size, so you will always be writing a multiple of 4K at a time.

However, Logical sector size has nothing to do with physical sector size, and the Operating system can handle such conversions by combining writes.

The drive will ALWAYS be accessed 4096 bytes at a time. That said, any modern operating system's block device driver can convert between sector sizes, better known these days as Block Sizes, and thus partition block sizes don't have to match the block size of the underlying device.

You can easily use 4K logical sector partitions on a 512 byte logical sector device, or a 512 byte logical sector partition on a 4K logical sector device.

Additionally, partition level logical sector does not have to match the sector block size used in the filesystem.

Linux and by extension Android, uses Block size (which is filesystem level logical sector sizes) of 1024, 2048 or 4096 on ExtFS. (On 512 byte sector drives it will use multiple physical sectors to form a logical block).

On FAT32, DRDOS 6.x supported larger disks by using 8K logical Sectors, and Linux can use still larger logical sectors. This allows much larger FAT32 partitions to be created.

Similarly, many software programs when managing very small filesystems that contain a large number incredibly small files have used programs with logical sector sizes as small as 64KB.

References:

Linux MSDOSTOOLS man pages

Linux mke2fs man page

Wikipedia pages on FAT:

https://en.wikipedia.org/wiki/File_Allocation_Table

https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system