By Xiang-Yang Lu
How big is 1.2 PB of storage?
What is the capacity of 1.2 PB? Simply put, imagine a 2 TB 2.5-inch external hard drive that is 600 times as much! Most of the storage devices with super-large capacity are owned by enterprise-level brands, but YouTube influencer Jeff Geerling tries to use Raspberry Pi, which is common among Makers for all kinds of projects. He uses the latest generation Raspberry Pi which is also industrial grade- the Compute Module 4.
Feasibility with using Raspberry Pi
Geerling breaks down an enterprise-level storage device and removes its motherboard. It was a motherboard by SuperMicro, equipped with a 26-core Intel Xeon processor, with 256 GB maximum RAM, 7 PCIe 3.0 slots, and 2 sets of 10 Gigabit Ethernet with optical fiber.
In comparison, Compute Module 4 only comes with a 4-core processor, 8 GB maximum of RAM, 1 PCIe 2.0 slot, and 1 set of 1 Gigabit Ethernet with copper wire. These specs are not as advanced as the regular motherboards. Geerling replaces the original motherboard with Compute Module 4, with which he wishes to control 60 enterprise-level hard drives with 20 TB of storage each.
Geerling builds a Network-Attached Storage (NAS) system, but he does not use the redundancy capacities in RAID (redundant array of independent disks). Instead, he simply allocates the data to be stored into multiple hard disks. When there is a need for the data, he shall collect the fragmented data from a multiple number of disks to be pieced together into original complete data for reading.
More specifically, Geerling purchased a Storinator XL60 storage server from the 45Drives website.
For technical details, Geerling uses a Compute Module 4 I/O Board for the module to be connected to the I/O board, while the PCIe interface on the I/O board is connected to each slot respectively. In the slots are 4 HBA cards inserted, which are connected to the 60 hard drives. The Raspberry Pi activates the HBA cards through the MPT3SAS Linux driver that has open source code.
CM4與CM4的I/O板
In terms of access performance, if we take 1m data transmission for instance, 4 hard drives can achieve 416 Mb/sec transmission, while there is only 213 Mb/sec with all 60 hard drives used, coupled with 4 HBA cards. In comparison, the speed of reading data grows along with the number of hard drives, when it comes to transmitting 4k data.
Since the storage device is configured as an NAS, Samba is also installed. Geerling tries transmitting video files in the storage device into his own Mac laptop, at a speed about 100 Mb/sec. However, there were 73 GB of files to be copied in batch, and the transmission failed upon completing 3.5 GB. For that reason, it is recommended to store data on one drive till it is full, before the transmission goes onto the next drive, which works out perfectly fine.
After some adjustments, the transmission increased to 250 Mb/sec, while it sometimes dropped to 30 Mb/sec, when the transmission is shared over different drives. The speed can drop to as low as 72 Mb/sec, which can take 192 days to transmit all the data stored in 60 hard drives.
Conclusion
There seems to be a trend of Makers becoming influencers in different countries. Take Geerling for instance. He started a channel on subscription on Patreon, where members can subscribe to his contents with different rate that includes $10, $30, or $100 per month.
