HARD DISK LIMITS

ST-506 full hight MFM hard diskOne of the most frustrating problems a lot of DIY enthusiasts came up against was BIOS problems that prevented the use of larger capacity disks.

In the early days, DOS version 1 which came on a floppy with our original PC could not even use directories. All saved files had to be saved in the root directory of the diskettes.

DOS version 2 which came with the PC-XT supported directories so that the hard disk could hold more use data. The original hard disk options for the PC-XT were for either a 5 MB ST-506 or a 10 MB ST-412 full height disk. DOS 2 also allowed directories on floppies which make it easier to organize files.

The hard disk is organized into physical sectors. The old 5 MB disk had 153 data tracks on a single head, so the disk with 2 plates had 32 sectors (256 bytes) and 612 tracks. The Xebec controller had a 5:1 interleave — it took 5 revolutions of the disk to read the whole track. The disk had a data rate of 5 megabits per second. Better controllers quickly came to market.

The IBM AT (August 14, 1984) introduced the 1.2MB half-height floppy drive and the half-height Seagate 20MB hard disk. It also came with DOS 3.0. The industry quickly changed to the half-height format which reduced costs and make it possible to use dual floppy drives in addition to dual hard disks.

The PC BIOS used registers rather than memory locations for the head number, sector number and track number. This meant that future disks would bump up against this lack of foresight and the short word problem. CHS is the acronym for the old hard disk configuration meaning cylinder head sector. The track limit meant that no disk could have more than 1024 tracks even though the register was 16 bit. This was imposed by DOS which meant a disk could not be bigger than 504 MB.

Connor designed their disks with the controller on the disk drive rather than as a separate card. This meant the disk became smarter and it could be inquired by the OS for information. It took a long time for Microsoft to recognize IDE as a solution. IDE paddle boards became widely used until the interface finally migrated to the motherboard.

Even with Windows 95 the disk limit was still very tight. While the 1024 track limit was gone, other problems abounded. Windows 9x all depended on DOS with its defects. Translation in the BIOS was able to mitigate the problem briefly.

Actual Cylinders Actual Heads Altered Cylinders Altered Heads Max. Capacity
1<C<1024 1<H<16 C=C H=H 504 MB
1024<C<2048 1<H<16 C=C/2 H=H*2 1008 MB
2048<C<4096 1<H<16 C=C/4 H=H*4 2016 MB
4096<C<8192 1<H<16 C=C/8 H=H*8 4032 MB
8192<C<16384 1<H<16 C=C/16 H=H*16 8064 MB
  •  2.11 GB or 4095 cylinder limitation
  • 3.26 GB or 6322 cylinder limitation
  • 4.22 GB or 8192 cylinder limitation
  • 8.45 GB Standard INT13 limitation (CHS[1024x256x63]x512)
  • 33.8 GB or 66,060,287 LBAs limitation (24-bit limit)
  • 137.4 GB or 268,435,455 LBAs limitation (28-bit limit)
  • 128 EB or 281.474,976,710,656 LBAs limitation (48-bit limit)

On older systems (pre- November 1998) the BIOS may not support drives that are larger than 32 GB. This is due to 65,536 tracks, 16 heads and 63 sectors per track. Our old Seanix P-II machine has this problem. It will not boot with a disk > 32 GB.

The 127 GB limit is due to 65,536 tracks, 16 heads and 256 sectors per track. LBA originally used 24 bits to represent the disk capacity which offers no real improvement. Our old IBM 300GL 6563 had this problem and it will not boot > 127 GB.

Logical block addressing looks at a disk differently. The disk is simple a block storage device, and simple request for blocks is all that is needed. LBA used the IDENT command which returned a sector with drive parameters.

LBA48 or LBA64 as the register is 64-bits wide increase disk capacity to over 18 EB. We detest using a short register for the capacity and LBA should be 64-bit across the board.

MBR

The last real problem limit is the master boot record. The MBR is the partition block at the front of the disk. Originally it was thought 32 bit sector addresses would be adequate.

  • 232 sectors = 4,294,967,296 sectors
  • 232 sectors * 512 bytes per sector = 2,199,023,255,552 bytes = 2 TB

It took many decades for the MBR limit to be surpassed. We consider it a real testament to the industrial innovation that has managed to make a 3½” hard disk grow from humble 20MB to over 10TB.

4K SECTORS

One solution that Seagate and WDC have both offered disks since 2009 that support 4K sector sizes. The space efficiency gains are significant and the error control is stronger but the overall hard bit error rate remains at 1014. The 4K sector is 8 times larger than the historical 512 byte sector. Among disks we have, Seagate provide true transparency for users of legacy operating systems while WDC needs to use an alignment program.

Windows 7 and Server 2008R2 have full support for partitioning and formatting 4K aligned hard disks. By using a 4K aware file system means disk performance is maximized. New 4K sector disks from WDC and Seagate work well with Windows 7.

Our ST2000DM001 is a 4K sector disk as is the ST2000DL003 we selected for bulk storage on the server and both have served well. Server 2008 R2 and above can easily use these disks in an array if desired.

GPT

We recommend moving to a UEFI based system so that when a hard disk fails, a new larger disk will not be a problem. GPT disks use a larger table that should be adequate looking forward towards technological limits. GPT can support a raw storage volume of 18 EB easily. More if a CPU could see more than 64-bit pointers. GPT was designed to support massive file systems.

UEFI also needs a video card with UEFI support. We upgraded our EVGA GTX 660 Ti FTW Signature 2 firmware for UEFI support. Thew firmware is fully backwards compatible for legacy systems.

OPERATING SYSTEM BOOT DISK DATA DISK
Windows 7 x86 no yes
Windows 7 x64 no yes
Windows 8 x86 no yes
Windows 8 x64 no yes
Windows 10 x86 no yes
Windows 10 x64 yes yes
Windows Server 2012 R2 yes yes

Windows Server 2012 R2 will wanted a GPT disk to install in a UEFI machine. This will then interact with the BIOS to install the UEFI secure boot etc. Desktop Windows continue to be limited to the MBR.

NTFS

Windows users will find that NTFS currently has a 256 TB limit. The problem is that NTFS with Server 2008 R2 is still 32-bit. Microsoft needs to revise NTFS to use a 64-bit table of allocation units.

NTFS like other file systems uses clusters of sectors to reduce the allocation table size. This is how it achieves 256 TB with a 32-bit allocation table.

Leaks out of Microsoft suggest Windows 8 will have a new file system. Moving to NTFS 64-bit seems logical.

NTFS supports compression so its possible to squeeze lots of data onto a disk. The overhead however is quite noticeable. Today we discourage its use in favor of low cost disks.

Windows 8 has a new file system, that eliminates the limits of NTFS.

ReFS

At the moment, Microsoft has not shipped ReFS with desktop Windows. It is available for Windows Server.

EXT

Linux widely uses ext2 however moving to ext3 is now largely complete. Most recently ext4 is available. Google is using ext4 for the Android OS. ext4 supports 1 EB volumes but there are some other minor problems such as 64K files in a directory etc.

Like Windows, Linux has its fair share of file system problems. Probably the best idea in a mixed environment is to use Active Directory. SAMBA runs on Linux to provide a cheap network attached storage device.

MODERN DISKS

Modern hard disks like the Seagate ST33000650NS now have some 5,860,533,168 sectors of storage. Obviously such a disk is better suited to a 64-bit system as the LBA is too large for a 32-bit register to handle.

The advanced format disks use a larger 4 KB sector size. This reduces overhead and allows for better error recovery. Given the growth in hard disk capacity, moving to 64-bit Windows is obvious.

HD Video resolutions and high bit rates can lead to some huge data files. This is why BD is 25 GB per layer. Pointers need to be 64-bit to be able to address such large data sets.