Monday, March 4, 2024

Coming over the horizon: "The world is running out of data storage. Here’s how DNA can save us"

In the case of DNA it's been coming for a while, in the case of three dimensional discs, I didn't even know we were close.Two from the electrical and electronic brainacs at IEEE Spectrum. First up, February 17:

The Quest for a DNA Data Drive 

How much thought do you give to where you keep your bits? Every day we produce more data, including emails, texts, photos, and social media posts. Though much of this content is forgettable, every day we implicitly decide not to get rid of that data. We keep it somewhere, be it in on a phone, on a computer’s hard drive, or in the cloud, where it is eventually archived, in most cases on magnetic tape. Consider further the many varied devices and sensors now streaming data onto the Web, and the cars, airplanes, and other vehicles that store trip data for later use. All those billions of things on the Internet of Things produce data, and all that information also needs to be stored somewhere.

Data is piling up exponentially, and the rate of information production is increasing faster than the storage density of tape, which will only be able to keep up with the deluge of data for a few more years. The research firm Gartner predicts that by 2030, the shortfall in enterprise storage capacity alone could amount to nearly two-thirds of demand, or about 20 million petabytes. If we continue down our current path, in coming decades we would need not only exponentially more magnetic tape, disk drives, and flash memory, but exponentially more factories to produce these storage media, and exponentially more data centers and warehouses to store them. Even if this is technically feasible, it’s economically implausible. 

https://spectrum.ieee.org/media-library/a-chart-showing-petabytes-in-the-millions-over-a-period-of-time-from-2019-2030.jpg?id=51488518&width=768&quality=85

Prior projections for data storage requirements estimated a global need for about 12 million petabytes of 
capacity by 2030. The research firm Gartner recently issued new projections, raising that estimate by 20 
million petabytes. The world is not on track to produce enough of today’s storage technologies to fill that 
gap.SOURCE: GARTNER

Fortunately, we have access to an information storage technology that is cheap, readily available, and stable at room temperature for millennia: DNA, the material of genes. In a few years your hard drive may be full of such squishy stuff.

Storing information in DNA is not a complicated concept. Decades ago, humans learned to sequence and synthesize DNA—that is, to read and write it. Each position in a single strand of DNA consists of one of four nucleic acids, known as bases and represented as A, T, G, and C. In principle, each position in the DNA strand could be used to store two bits (A could represent 00, T could be 01, and so on), but in practice, information is generally stored at an effective one bit—a 0 or a 1—per base.

Moreover, DNA exceeds by many times the storage density of magnetic tape or solid-state media. It has been calculated that all the information on the Internet—which one estimate puts at about 120 zettabytes—could be stored in a volume of DNA about the size of a sugar cube, or approximately a cubic centimeter. Achieving that density is theoretically possible, but we could get by with a much lower storage density. An effective storage density of “one Internet per 1,000 cubic meters” would still result in something considerably smaller than a single data center housing tape today....

....MUCH MORE

And February 23:

DVD’s New Cousin Can Store More Than a Petabit
Containing more data than the entire Internet can transmit in a second

A novel disc the size of a DVD can hold more than 1 million gigabits—roughly as much as is transmitted per second over the entire world’s Internet—by storing data in three dimensions as opposed to two, a new study finds.

Optical discs such as CDs and DVDs encode data using a series of microscopic pits. These pits, and the islands between them, together represent the 0s and 1s of binary code that computers use to symbolize information. CD, DVD, and Blu-ray players use lasers to read the data encoded in these discs.

“The use of ultrahigh-density optical data storage technology in big data centers is 
now possible.” —Min Gu, University of Shanghai for Science and Technology

Although optical discs are low in cost and highly durable, they are limited by the amount of data they can hold, which is usually stored in a single layer. Previously, scientists investigated encoding data on optical discs in many layers in three dimensions to boost their capacity. However, a key barrier that prior research faced was how the optics used to read and write this data were limited to roughly the size of the wavelengths of light they used.

Now scientists in China have developed a way to encode data on 100 layers in optical discs. In addition, the data is recorded using spots as small as 54 nanometers wide, roughly a tenth of the size of the wavelengths of visible light used to read and write the data.

All in all, a DVD-size version of the new disc has a capacity of up to 1.6 petabits—that is, 1.6 million gigabits. This is some 4,000 times as much data density as a Blu-ray disc and 24 times as much as the currently most advanced hard disks. The researchers suggest their new optical disc can enable a data center capable of exabit storage—a billion gigabits—to fit inside a room instead of a stadium-size space....

....MUCH MORE