The difference between "2TB" on a hard drive box and "1.82 TiB" in your operating system is not a mistake by either party — it's a 70-year-old ambiguity in the definition of "kilo" being applied inconsistently between hardware manufacturers and operating systems, and understanding it requires distinguishing between two different meanings of the word "gigabyte" that still coexist today
The previous articles on this site covered data storage basics, why 1TB shows as 931GB, cloud storage egress costs, SSD technology, Mbps vs MB/s, and RAID vs backup. This article addresses data storage in database and file system contexts — specifically the specific storage decisions that affect application architecture, and how storage is measured and allocated at each layer of the stack from disk to database row.
The binary vs decimal prefix problem: resolved but not gone
In 1998, the IEC (International Electrotechnical Commission) formally defined binary prefixes to resolve the ambiguity:
| Decimal (SI) | Value | Binary (IEC) | Value |
|---|---|---|---|
| Kilobyte (KB) | 1,000 bytes | Kibibyte (KiB) | 1,024 bytes |
| Megabyte (MB) | 1,000,000 bytes | Mebibyte (MiB) | 1,048,576 bytes |
| Gigabyte (GB) | 1,000,000,000 bytes | Gibibyte (GiB) | 1,073,741,824 bytes |
| Terabyte (TB) | 10^12 bytes | Tebibyte (TiB) | 2^40 bytes |
The persistent ambiguity: hard drive manufacturers use decimal prefixes (1TB = 10^12 bytes). Operating systems historically used binary math but called the results "KB/MB/GB" (meaning KiB/MiB/GiB). Linux and macOS have progressively moved to IEC prefixes; Windows still shows GiB using the "GB" label.
For a 2TB drive:
- Manufacturer's TB: 2 × 10^12 = 2,000,000,000,000 bytes
- Windows shows: 1.82 TB (actually 1.82 TiB = 2,000,000,000,000 / 2^40)
- Linux/macOS with IEC units: 1.82 TiB (correctly labelled)
The gap widens at higher capacities: for consumer drives, the gap between labeled and displayed capacity is approximately 7% at 1TB and 10% at 4TB. This is not hidden space — it's the same bytes described in different unit systems.
Database storage: how row and index sizes add up
SQL databases consume more storage than the sum of your data due to storage overhead:
Row overhead: each row has metadata beyond the column values — NULL bitmaps (tracking which columns are NULL), row header information, and in some databases, visibility information for MVCC (Multi-Version Concurrency Control). PostgreSQL row overhead is approximately 24 bytes per row; this is significant for tables with many small rows.
Page structure: databases store data in fixed-size pages (typically 8 KB in PostgreSQL and SQL Server, 16 KB in MySQL). A page can't hold a fraction of a row — if a row is 4 KB and the page is 8 KB, you can fit 2 rows maximum, leaving the remaining space unusable within that page.
Fill factor: most databases leave a percentage of each page empty to accommodate future updates (prevents page splits, which are expensive). PostgreSQL's default fill factor for indexes is 90%.
Index storage: for a table with a primary key, a B-tree index, and a few additional indexes, total index storage can equal or exceed the raw data storage. A table with 100 million rows might have 2 GB of raw data but 4-5 GB of total storage including indexes.
File system overhead and block size
File systems also consume more space than raw file sizes through a similar page-level mechanism called block size (or cluster size):
Block/cluster size: the minimum allocation unit. A file system with 4 KB blocks allocates a full 4 KB block for each file, even if the file is 100 bytes. A directory with 10,000 small config files (average 200 bytes each) uses 10,000 × 4 KB = 40 MB on disk despite containing only 2 MB of actual data.
Inodes: Unix file systems store metadata (permissions, timestamps, owner, size) in inodes — separate from the file data. Inode table size is fixed at format time. A file system that runs out of inodes (rather than space) can't create new files even with free space available: No space left on device with gigabytes free.
Why this matters for containers: Docker images and containers use overlay file systems where each layer is stored separately. A Docker image with 10 layers each modifying the same file stores 10 copies of that file's blocks (with copy-on-write optimization reducing some redundancy). Image sizes in Docker can be significantly larger than the application's raw footprint.
NVMe vs SATA: interface bandwidth and storage latency
NVMe (Non-Volatile Memory Express) is the storage interface protocol designed specifically for SSDs, replacing SATA for high-performance storage:
Interface bandwidth comparison:
- SATA III: 600 MB/s theoretical maximum
- NVMe PCIe 3.0 ×4: approximately 3,500 MB/s read / 3,000 MB/s write
- NVMe PCIe 4.0 ×4: approximately 7,000 MB/s read / 6,500 MB/s write
- NVMe PCIe 5.0 ×4: approximately 14,000 MB/s read (latest generation drives)
Latency comparison:
- SATA SSD: approximately 50-100 μs latency
- NVMe SSD: approximately 20-50 μs latency
- Traditional HDD: approximately 5,000-10,000 μs (5-10 ms) seek time
The practical implication: for sequential workloads (video editing, large file transfers, database backups), NVMe's bandwidth advantage is directly usable. For random I/O (database transaction logs, frequently accessed small files), the latency advantage matters more than bandwidth.
Object storage: S3, GCS, and the storage class economics
Cloud object storage (AWS S3, Google Cloud Storage, Azure Blob) has multiple storage classes with different cost profiles:
| Class | Use case | Retrieval latency | Cost (approx) |
|---|---|---|---|
| Standard | Frequently accessed | Milliseconds | $0.023/GB/month |
| Standard-IA (Infrequent Access) | Monthly access | Milliseconds | $0.0125/GB/month + retrieval fee |
| Glacier Instant Retrieval | Quarterly access | Milliseconds | $0.004/GB/month + retrieval fee |
| Glacier Flexible Retrieval | Annual archive | Minutes-hours | $0.0036/GB/month + retrieval fee |
| Glacier Deep Archive | Decade-scale archive | Hours | $0.00099/GB/month + retrieval fee |
The storage class gotcha — minimum storage duration: Infrequent Access has a 30-day minimum; Glacier Instant has a 90-day minimum; Glacier Deep Archive has a 180-day minimum. Objects deleted before the minimum duration are still charged for the full minimum period.
How to use the Data Storage Converter on sadiqbd.com
- For cloud storage planning: convert between TB (provider bills in decimal) and TiB (operating systems display in binary) to correctly budget storage capacity
- For database sizing: convert your estimated row count × row size from bytes to GB/TB, then add 50-100% overhead for indexes, page fill factor, and MVCC storage to get a realistic database size estimate
- For network bandwidth: convert between Mbps (network interface ratings) and MB/s (file transfer speeds) — divide Mbps by 8 to get MB/s
Frequently Asked Questions
Why do some cloud providers bill in GB and others in GiB — and how do I tell which one a billing page is using? AWS, Google Cloud, and most providers bill in decimal GB (1 GB = 10^9 bytes) — consistent with how hard drive manufacturers label capacity. This means a 1 TiB (binary terabyte) of actual storage data is billed as approximately 1.099 TB (decimal terabytes). When a provider says "your S3 bucket contains 500 GB" and your OS shows the uploaded data as 465 GiB, both are correct — the provider's "500 GB" and your OS's "465 GiB" are the same amount of bytes (500 × 10^9 = 465 × 2^30). The practical check: provider billing pages almost universally use decimal units; OS displays of the same data typically use binary units with inconsistent labelling. When in doubt, look for the actual byte count rather than the prefix.
Is the Data Storage Converter free? Yes — completely free, no sign-up required.
Try the Data Storage Converter free at sadiqbd.com — convert between bits, bytes, KB, MB, GB, TB, and their binary equivalents (KiB, MiB, GiB, TiB).
