A recent headline reads: “Stanford University finds that AI is outpacing Moore’s Law”. Moore’s Law refers to the periodic doubling of computing power. The outpacing of Moore’s law is incredible! Raise a glass to the genius of electrical engineers and computer scientists of the world!

There are at least two ways the speed of AI can be increased. Both attest to remarkable human ingenuity. But faster computers only help the performance of AI algorithms that require search marginally. That includes training in deep learning.

Suppose a search over 100,000 bits requires a year of a computer’s time. If the computer speed doubles, how many bits can be searched? Only 100,001. A single bit more. When the speed of the computer doubles, the original 100,000 bits must be searched when the new bit is zero. This can be done in a half year by the speedier computer. The second half of the year searches when the new bit is equal to one. Thus, no matter how fast the computer was before doubling, only a single bit can be added to the overall bit count of the search after doubling.

The increase of computer speed yields diminishing returns. If a computer doubles 40 times there is an increase in speed of over a trillion. ¹ But only 100,040 bits can now be searched.

The same analysis applies to parallel computing where more than one computer analyzes a problem. One computer searches through 100,000 bits in a year. Two computers running at the same speed will be able to look at 100,001 bits. One computer searches the original 100,000 bits when the new bit is zero. The second computer searches when the new bit is one.
If there are a trillion computers running at the same time, the search increases to only 100,040 bits.

The periodic doubling of computer speed is an example of exponential growth. Ray Kurzweil commonly refers to exponential growth in his forecasts about the future.² But any exponential growth is unsustainable. It can’t last indefinitely. It is often the beginning of a sigmoid or s-shaped curve where growth that appears to be exponential eventually slows and reaches a saturation point. Bacteria grow exponentially in number until food scarcity and other factors cause the population curve to flatten. In many cases forecasting the point where exponential growth slows is problematic. We only know that growth can’t keep on forever.

Another way to boost AI speed is to increase the cleverness of the algorithms that perform AI operations. In this case, the speed increase is due to human creativity and not to computers. For example, we read:

[To increase the speed] more research is needed to develop methods that can reliably discriminate activities, which involve fine-grained motions and/or subtle patterns in motion cues, objects and human-object interactions.

Cliff Saran, “Stanford University finds that AI is outpacing Moore’s Law” at Computer Weekly

Whatever the meaning of “motion cues” or “fine-grained motion,” the point is that more research will produce cleverer algorithms that will make AI algorithms run faster.

Better algorithms are the fruit of human creativity. So are engineered computers that operate faster and faster. In both cases, human ingenuity makes AI run faster. But no matter how fast the AI , there will be no duplication of the creativity resident in humans. Creativity can’t be computed. Sir Roger Penrose said it well: “With thought comprising a non-computational element, computers can never do what we human beings can.”

In other words, humans can create a computer, but, no matter how fast, a computer will never create a human.

¹ Here’s the calculation: 2^40 = 1,099,511,627,776
² Kurzweil, Ray. The singularity is near: When humans transcend biology. Penguin, 2005.

Other items by Robert J. Marks that you might also enjoy:

Smart Cities?Proceed with Caution: Zaheer Allam provides a balanced view of the future impact of AI on society

and

Can computer algorithms be free of bias? Bias is inevitable but it should be recognized and admitted.