From Communications of the Association for Computing Machinery:
The impending demise of Moore's Law has begun to broadly impact the computing research community.38 Moore's Law has driven the computing industry for many decades, with nearly every aspect of society benefiting from the advance of improved computing processors, sensors, and controllers. Behind these products has been a considerable research industry, with billions of dollars invested in fields ranging from computer science to electrical engineering. Fundamentally, however, the exponential growth in computing described by Moore's Law was driven by advances in materials science.30,37 From the start, the power of the computer has been limited by the density of transistors. Progressive advances in how to manipulate silicon through advancing lithography methods and new design tools have kept advancing computing in spite of perceived limitations of the dominant fabrication processes of the time.37
Back to Top
Key Insights
There is strong evidence that this time is indeed different, and Moore's Law is soon to be over for good.3,38 Already, Dennard scaling, Moore's Law's lesser known but equally important parallel, appears to have ended.11 Dennard's scaling refers to the property that the reduction of transistor size came with an equivalent reduction of required power.8 This has real consequences—even though Moore's Law has continued over the last decade, with feature sizes going from ~65nm to ~10nm; the ability to speed up processors for a constant power cost has stopped. Today's common CPUs are limited to about 4GHz due to heat generation, which is roughly the same as they were 10 years ago. While Moore's Law enables more CPU cores on a chip (and has enabled high power systems such as GPUs to continue advancing), there is increasing appreciation that feature sizes cannot fall much further, with perhaps two or three further generations remaining prior to ending.
Multiple solutions have been presented for technological extension of Moore's Law,3,33,38,39 but there are two main challenges that must be addressed. For the first time, it is not immediately evident that future materials will be capable of providing a long-term scaling future. While non-silicon approaches such as carbon nanotubes or superconductivity may yield some benefits, these approaches also face theoretical limits that are only slightly better than the limits CMOS is facing.31 Somewhat more controversial, however, is the observation that requirements for computing are changing.33,39 In some respects, the current limits facing computing lie beyond what the typical consumer outside of the high-performance computing community will ever require for floating point math. Data-centric computations such as graph analytics, machine learning, and searching large databases are increasingly pushing the bounds of our systems and are more relevant for a computing industry built around mobile devices and the Internet. As a result, it is reasonable to consider the ideal computer is not one that is better at more FLOPS, but rather one that is capable of providing low-power computation more appropriate for a world flush with "big data." While speed remains an important driver, other considerations—such as algorithmic capabilities—are increasingly critical.
For these reasons, neural computing has begun to gain increased attention as a post-Moore's Law technology. In many respects, neural computing is an unusual candidate to help extend Moore's Law. Neural computing is effectively an algorithmic and architectural change from classic numerical algorithms on von Neumann architectures, as opposed to exploiting a novel material to supplant silicon. Further, unlike quantum computation, which leverages different physics to perform computation, neural computing likely falls within the bounds of classic computing theoretical frameworks. Whereas quantum computation can point to exponential benefits on certain tasks such as Shor's quantum algorithm for factoring numbers;34 neural computing architecture's most likely path to impact is through polynomial trade-offs between energy, space, and time. Such benefits can be explicitly formalized and are potentially quite impactful for certain applications.1 However, there is limited evidence that neural architectures can be more powerful on generic applications than the general purpose architectures used today.33
