"Looking Back at the Future of Humanity Institute"

 From Asterisk Magazine:

The rise and fall of the influential, embattled Oxford research center that brought us the concept of existential risk.

On April 16, 2024, the website of the Future of Humanity Institute was replaced by a simple landing page and a four-paragraph statement. The institute had closed down, the statement explained, after 19 years. It briefly sketched the institute’s history, appraised its record, and referred to “increasing administrative headwinds” blowing from the University of Oxford’s Faculty of Philosophy, in which it had been housed.

Thus died one of the quirkiest and most ambitious academic institutes in the world. FHI’s mission had been to study humanity’s big-picture questions: our direst perils, our range of potential destinies, our unknown unknowns. Its researchers were among the first to usher concepts such as superintelligent AI into academic journals and bestseller lists alike, and to speak about them before such groups as the United Nations. 

To its many fans, the closure of FHI was startling. This group of polymaths and eccentrics, led by the visionary philosopher Nick Bostrom, had seeded entire fields of study, alerted the world to grave dangers, and made academia’s boldest attempts to see into the far future. But not everyone agreed with its prognostications. And, among insiders, the institute was regarded as needlessly difficult to deal with — perhaps to its own ruin. In fact, the one thing FHI had not foreseen, its detractors quipped, was its own demise.

Why would the university shutter such an influential institute? And, to invert the organization’s forward-looking mission: how should we — and our descendants — look back on FHI? For an institute that set out to find answers, FHI left the curious with a lot of questions. 

***

In 1989, a seventeen-year-old Swede named Niklas Boström (he would later anglicize it Nick Bostrom) borrowed a library book of 19th-century German philosophy, took it to a favorite forest clearing, and experienced what The New Yorker would later describe as “a euphoric insight into the possibilities of learning and achievement.” Damascene moments aren’t generally how people decide to become academics, but from this day forward, Bostrom dedicated his life to intensive study. He retreated from school in order to take his exams at home, and he read widely and manically. At the University of Gothenburg, he received a BA in philosophy, mathematics, mathematical logic, and artificial intelligence. After that, he pursued postgraduate degrees in philosophy, physics, and computational neuroscience. In what little spare time he had, Bostrom emailed and met up with fellow transhumanists: people enthusiastic about radically improving human biology and lifespans. 

As early as 2001, he was studying little-known phenomena called “existential risks,” writing that nanotechnology and machine intelligence could one day interfere with our species’ ascent to a transhuman future. He also, in 2001, formulated the “simulation hypothesis,” advancing in Philosophical Quarterly the theory that we might be living in a computer simulation run by humanity’s hyper-intelligent descendants. ¹ By this point, Bostrom had arrived at Oxford as a postdoctoral fellow at the Faculty of Philosophy. 

Some years later, the faculty would become his bête noire. But it was Bostrom’s membership in it that enabled a stroke of luck that would change his life. At some point in the early aughts, Bostrom had met James Martin, an IT entrepreneur. Martin had become a prescient futurist, and in 2006 produced a documentary featuring Bostrom. But Martin was also becoming a deep-pocketed philanthropist. Through Julian Savulescu, another young philosopher interested in human enhancement, Bostrom learnt that Martin was planning to fund future-minded research at Oxford. Hoping that this could encompass work on his interests, Bostrom made his case to the university’s development office.

Twenty years later, the details are hazy. FHI lore has it that, at one of dinners hosted by Oxford for its biggest donors, Bostrom was seated next to Martin, creating the perfect conditions for what we now call a nerdsnipe. Some time later, in 2005, Martin made what was then the biggest benefaction to the University of Oxford in its nine-century history, totaling over £70 million. A small portion of it funded what Bostrom decided to call the Future of Humanity Institute. “It was a little space,” Bostrom told me, “where one could focus full-time on these big-picture questions.”

That seed grant was enough to fund a few people for three years. Because his team would be small, and because it had such an unconventional brief, Bostrom needed to find multidisciplinarians. He was looking, he told me, for “brainpower especially, and then also a willingness and ability to work in areas where there is not yet a very clear methodology or a clear paradigm.” And it would help to be a polymath. 

One of his earliest hires was Anders Sandberg. As well as being a fellow Swede, Sandberg was a fellow member of the Extropians, an online transhumanist community that Bostrom had joined in the Nineties. Where Bostrom is generally ultra-serious, Sandberg is ebullient and whimsical. (He once authored a paper in which he outlined what would happen if the Earth turned into a giant pile of blueberries). But the two men’s differences in personality belied their similarity in outlook. Sandberg, too, was an unorthodox thinker interested in transhumanism and artificial intelligence. (Sandberg was particularly interested in the theoretical practice of whole-brain emulation, i.e. the uploading of a human mind to a digital substrate.)

Sandberg was interviewed in the Faculty of Philosophy’s Ryle room, named for the philosopher Gilbert Ryle. ² He explained some neuroscience to the faculty staff who were assessing him and communicated his aptitude in another little-known area of human endeavor: web design. He was hired, and he returned in January 2006 to take up a desk at FHI and a “silly little room in Derek’s house.” 

Sandberg was lodging, with Bostrom, in the home of Derek Parfit, a wild-haired recluse who was also one of the most influential moral philosophers of the modern era. Bostrom had the master bedroom and collected rent from the rotating cast of lodgers. ³ Parfit, Sandberg recalled, slept in “a little cubby hole” of a bedroom, and would scuttle at odd hours between it and his office at All Souls, the highly selective graduate college seen as elite even relative to the rest of Oxford.

Including Sandberg, Bostrom hired three researchers, and began to sculpt a research agenda that, in these early years, was primarily concerned with the ethics of human enhancement. An EU-funded project on cognitive enhancement was one of FHI’s main focuses in this period. The institute also organized a workshop that resulted in Sandberg and Bostrom’s influential roadmap for making whole-brain emulation feasible. 

At the same time, FHI staff were beginning to publish work on the gravest perils facing humanity, a topic that was not yet an established academic discipline. An FHI workshop brought together hitherto disparate thinkers such as Eliezer Yudkowsky, who went on to become one of the most prominent theorists concerned by superintelligent AI. Bostrom co-edited the 2008 book Global Catastrophic Risk, a collection of essays on threats such as asteroid impacts, nuclear war, and advanced nanotechnology....

....MUCH MORE

Previously:

Oxford's Future of Humanity Institute Has Shut Down

Well. I guess that's that. It's over. Fin. I wonder if they saw it coming?....

We've visited Bostrom and the Future of Humanity Institute at Oxford off and on for a decade. Although he sometimes comes off as a bit whack-a-doodle I did make this comment on one of our posts:

...If interested we mentioned Bostrom in last week's "The Roubini Cascade: Are we heading for a Greater Depression?". His Future of Humanity Institute at Oxford seems to have moved on from the mundane climate cataclysm or cosmic fireball ending everything, to a very real, very serious examination of whether or not Artificial Intelligence may be the nail in humanity's coffin, so to speak....  

That was in "Risk/Prop Bets: How unlikely is a doomsday catastrophe?"  which also included this bonus advice:

Before we get into the meat of the matter, some things to know upfront. From the introduction to 2018's "Tips And Tricks For Investing In 'End of the World Funds'":

As unauthorized representatives for Long or Short Capital's End of the World Puts this is an area of profound interest from which we have gleaned some insight:

1) Should the world end, collecting on your bet can be a challenge. Know your counterparty!
     And possibly more important, demand collateral!
2) The swings in end of the world product prices can be dramatic.
3) Prognosticators have predicted 100,000 of the last 0 termination events....

And from arXiv (astrophysics) at Cornell:

How unlikely is a doomsday catastrophe?

Max Tegmark (MIT), Nick Bostrom (Oxford)

Numerous Earth-destroying doomsday scenarios have recently been analyzed, including breakdown of a metastable vacuum state and planetary destruction triggered by a "strangelet'' or microscopic black hole. We point out that many previous bounds on their frequency give a false sense of security: one cannot infer that such events are rare from the the fact that Earth has survived for so long, because observers are by definition in places lucky enough to have avoided destruction. We derive a new upper bound of one per 10^9 years (99.9% c.l.) on the exogenous terminal catastrophe rate that is free of such selection bias, using planetary age distributions and the relatively late formation time of Earth....

All in all it is good to see he's flipped 180 degrees and has gone far beyond even the upbeat message of Keynes' "Economic Possibilities for our Grandchildren" and Churchill's 'broad sunlit uplands'. I believe I shall I shall have this bit tattooed  in an appropriately discreet location:

"If humans remain the owners of this capital, the total income received by the human population would grow astronomically, despite the fact that in this scenario humans would no longer receive any wage income. The human species as a whole could thus become rich beyond the dreams of Avarice."

Related:

Puny Human, I Scoff At Your AI, Soon You Will Know The Power Of Artificial 'Super' Intelligence. Tremble and Weep
[insert 'Bwa Ha Ha' here, if desired]

Puny Human, I Scoff At Your AI, Soon You Will Know The Power Of Artificial 'Super' Intelligence. Tremble and Weep

[insert 'Bwa Ha Ha' here, if desired]

From James Pethokoukis at his Faster, Please! substack:

After AGI: Is artificial 'super' intelligence coming soon?

To reimagine Nobel laureate economist Robert Lucas’ famous quote about economic growth: “Once you start thinking about superintelligence, it's hard to think about anything else.”

And with good reason. First, the notion can make for great storytelling. As a fan of science fiction, I’ve been thinking about the idea of supersmart machines, both good (Lt. Commander Data) and evil (Skynet), for a long time.

While many of those stories are dystopian, they don’t have to be. I was really struck when reading this optimistic passage from the 2014 book Superintelligence: Paths, Dangers, Strategies by technology philosopher Nick Bostrom:

If we classify AI as capital, then with the invention of machine intelligence that can fully substitute for human work, wages would fall to the marginal cost of such machine-substitutes, which—under the assumption that the machines are very efficient—would be very low, far below human subsistence-level income. The income share received by labor would then dwindle to practically nil. But this implies that the factor share of capital would become nearly 100% of total world product. Since world GDP would soar following an intelligence explosion (because of massive amounts of new labor-substituting machines but also because of technological advances achieved by superintelligence, and, later, acquisition of vast amounts of new land through space colonization), it follows that the total income from capital would increase enormously. If humans remain the owners of this capital, the total income received by the human population would grow astronomically, despite the fact that in this scenario humans would no longer receive any wage income. The human species as a whole could thus become rich beyond the dreams of Avarice.

A brief, definitional note: Although the terms “artificial general intelligence” and “superintelligence” (or “artificial super intelligence” or “super machine intelligence”) are sometimes used interchangeably — understandably because they all refer to highly intelligent AI — that conflation should be avoided....

....MUCH MORE 

We've visited Bostrom and the Future of Humanity Institute at Oxford off and on for a decade. Although he sometimes comes off as a bit whack-a-doodle I did make this comment on one of our posts:

...If interested we mentioned Bostrom in last week's "The Roubini Cascade: Are we heading for a Greater Depression?". His Future of Humanity Institute at Oxford seems to have moved on from the mundane climate cataclysm or cosmic fireball ending everything, to a very real, very serious examination of whether or not Artificial Intelligence may be the nail in humanity's coffin, so to speak....  

That was in "Risk/Prop Bets: How unlikely is a doomsday catastrophe?"  which also included this bonus advice:

Before we get into the meat of the matter, some things to know upfront. From the introduction to 2018's "Tips And Tricks For Investing In 'End of the World Funds'":

As unauthorized representatives for Long or Short Capital's End of the World Puts this is an area of profound interest from which we have gleaned some insight:

1) Should the world end, collecting on your bet can be a challenge. Know your counterparty!
     And possibly more important, demand collateral!
2) The swings in end of the world product prices can be dramatic.
3) Prognosticators have predicted 100,000 of the last 0 termination events....

And from arXiv (astrophysics) at Cornell:

How unlikely is a doomsday catastrophe?

Max Tegmark (MIT), Nick Bostrom (Oxford)

Numerous Earth-destroying doomsday scenarios have recently been analyzed, including breakdown of a metastable vacuum state and planetary destruction triggered by a "strangelet'' or microscopic black hole. We point out that many previous bounds on their frequency give a false sense of security: one cannot infer that such events are rare from the the fact that Earth has survived for so long, because observers are by definition in places lucky enough to have avoided destruction. We derive a new upper bound of one per 10^9 years (99.9% c.l.) on the exogenous terminal catastrophe rate that is free of such selection bias, using planetary age distributions and the relatively late formation time of Earth....

All in all it is good to see he's flipped 180 degrees and has gone far beyond even the upbeat message of Keynes' "Economic Possibilities for our Grandchildren" and Churchill's 'broad sunlit uplands'. I believe I shall I shall have this bit tattooed  in an appropriately discreet location:

"If humans remain the owners of this capital, the total income received by the human population would grow astronomically, despite the fact that in this scenario humans would no longer receive any wage income. The human species as a whole could thus become rich beyond the dreams of Avarice."

Risk/Prop Bets: How unlikely is a doomsday catastrophe?

Before we get into the meat of the matter, some things to know upfront. From the introduction to 2018's "Tips And Tricks For Investing In 'End of the World Funds'":

As unauthorized representatives for Long or Short Capital's End of the World Puts this is an area of profound interest from which we have gleaned some insight:

1) Should the world end, collecting on your bet can be a challenge. Know your counterparty!
     And possibly more important, demand collateral!
2) The swings in end of the world product prices can be dramatic.
3) Prognosticators have predicted 100,000 of the last 0 termination events....

And from arXiv (astrophysics) at Cornell:

How unlikely is a doomsday catastrophe?

Max Tegmark (MIT), Nick Bostrom (Oxford)

Numerous Earth-destroying doomsday scenarios have recently been analyzed, including breakdown of a metastable vacuum state and planetary destruction triggered by a "strangelet'' or microscopic black hole. We point out that many previous bounds on their frequency give a false sense of security: one cannot infer that such events are rare from the the fact that Earth has survived for so long, because observers are by definition in places lucky enough to have avoided destruction. We derive a new upper bound of one per 10^9 years (99.9% c.l.) on the exogenous terminal catastrophe rate that is free of such selection bias, using planetary age distributions and the relatively late formation time of Earth....

If interested we mentioned Bostrom in last week's "The Roubini Cascade: Are we heading for a Greater Depression?". His Future of Humanity Institute at Oxford seems to have moved on from the mundane climate cataclysm or cosmic fireball ending everything, to a very real, very serious examination of whether or not Artificial Intelligence may be the nail in humanity's coffin, so to speak. 

Although the writers of these pieces aren't from the FHI they are writing about Bostrom and seem to represent the state of the science.

First up, from MIT's Technology Review, September 20, 2016:

No, the Experts Don’t Think Superintelligent AI is a Threat to Humanity

And two months later, also from MIT's Technology Review:

Yes, We Are Worried About the Existential Risk of Artificial Intelligence

In the intervening six years the positions on either side have only hardened, such that at one of the conferences the phrase "big poopy-head" was used.

Some of our previous stuff referencing Bostrom:

March 2022
How much value can our decisions create? (there's an upper limit)

October 2021
So, What Are They Thinking About At Oxford's Future of Humanity Institute?

August 2015
Artificial Intelligence Can Be Scary, Artificial Stupidity Can Kill Us All

November 2012
7 Best-Case Scenarios for the Future of Humanity

Here are some of the FHI's publications, with links to the rest of the site. 

And related:

High Frequency Trading, the Grossman-Stiglitz Paradox and a Cheery Doomsday Analysis

And with that, possibly premature optimism, and with a nod to one of our progenitors we will move on to next week's busy earnings calendar.

Evolution Going Great, Reports Trilobite 

Slowly inching his segmented exoskeleton across the sea floor, a local marine arthropod, class Trilobita, reported that Earth's natural evolution was "progressing quite nicely."

"Things are looking mighty fine," announced the prehistoric invertebrate, taking measure of his surroundings through a series of small, hexagonal eyelets located at the tip of his thorax. "Sulfurous gas seems to be bubbling up to the surface pretty good, and several single-cell organisms appear to be mutating at a rather steady pace. Also, just today, I developed the ability to roll into a small protective shell in order to avoid predators."

Added the trilobite, "Yup, this evolution thing is going great."...

Little did the Trilobite know...

"AI and the paperclip problem"

From VoxEU - CEPR, June 10, 2018:

Philosophers have speculated that an AI tasked with a task such as creating paperclips might cause an apocalypse by learning to divert ever-increasing resources to the task, and then learning how to resist our attempts to turn it off. But this column argues that, to do this, the paperclip-making AI would need to create another AI that could acquire power both over humans and over itself, and so it would self-regulate to prevent this outcome. Humans who create AIs with the goal of acquiring power may be a greater existential threat.

The notion that artificial intelligence (AI) may lead the world into a paperclip apocalypse has received a surprising amount of attention. It motivated Stephen Hawking and Elon Musk to express concern about the existential threat of AI. It has even led to a popular iPhone game explaining the concept. 

The concern isn’t about paperclips per se. Instead it is that, at some point, switching on an AI may lead to destruction of everything, and that this destruction would both be easy and arise from a trivial or innocuous initial intent.

The underlying ideas behind the notion that we could lose control over an AI are profoundly economic. But, to date, economists have not paid much attention to them. Instead, their focus has been on the more mundane, recent improvements in machine learning (Agrawal et.al. 2018). Taking a more future-bound perspective, my research (Gans 2017) shows that for a paperclip apocalypse to occur, we must make important underlying assumptions. This gives me reason to believe that it's less likely than non-economists believe that the world will end this way.

What is the paperclip apocalypse?
The notion arises from a thought experiment by Nick Bostrom (2014), a philosopher at the University of Oxford. Bostrom was examining the 'control problem': how can humans control a super-intelligent AI even when the AI is orders of magnitude smarter. Bostrom's thought experiment goes like this: suppose that someone programs and switches on an AI that has the goal of producing paperclips. The AI is given the ability to learn, so that it can invent ways to achieve its goal better. As the AI is super-intelligent, if there is a way of turning something into paperclips, it will find it. It will want to secure resources for that purpose. The AI is single-minded and more ingenious than any person, so it will appropriate resources from all other activities. Soon, the world will be inundated with paperclips.

It gets worse. We might want to stop this AI. But it is single-minded and would realise that this would subvert its goal. Consequently, the AI would become focussed on its own survival. It is fighting humans for resources, but now it will want to fight humans because they are a threat (think The Terminator). 

This AI is much smarter than us, so it is likely to win that battle. We have a situation in which an engineer has switched on an AI for a simple task but, because the AI expanded its capabilities through its capacity for self-improvement, it has innovated to better produce paperclips, and developed power to appropriate the resources it needs, and ultimately to preserve its own existence.

Bostrom argues that it would be difficult to control a super-intelligent AI – in essence, better intelligence beats weaker intelligence....

....MUCH MORE

Possibly related, July 7, 2023:

Puny Human, I Scoff At Your AI, Soon You Will Know The Power Of Artificial 'Super' Intelligence. Tremble and Weep
[insert 'Bwa Ha Ha' here, if desired]

Flashback: "AI and the paperclip problem"

From VoxEU, 10 Jun 2018:

Philosophers have speculated that an AI tasked with a task such as creating paperclips might cause an apocalypse by learning to divert ever-increasing resources to the task, and then learning how to resist our attempts to turn it off. But this column argues that, to do this, the paperclip-making AI would need to create another AI that could acquire power both over humans and over itself, and so it would self-regulate to prevent this outcome. Humans who create AIs with the goal of acquiring power may be a greater existential threat.

The notion that artificial intelligence (AI) may lead the world into a paperclip apocalypse has received a surprising amount of attention. It motivated Stephen Hawking and Elon Musk to express concern about the existential threat of AI. It has even led to a popular iPhone game explaining the concept. 

The concern isn’t about paperclips per se. Instead it is that, at some point, switching on an AI may lead to destruction of everything, and that this destruction would both be easy and arise from a trivial or innocuous initial intent.

The underlying ideas behind the notion that we could lose control over an AI are profoundly economic. But, to date, economists have not paid much attention to them. Instead, their focus has been on the more mundane, recent improvements in machine learning (Agrawal et.al. 2018). Taking a more future-bound perspective, my research (Gans 2017) shows that for a paperclip apocalypse to occur, we must make important underlying assumptions. This gives me reason to believe that it's less likely than non-economists believe that the world will end this way. 

What is the paperclip apocalypse?

The notion arises from a thought experiment by Nick Bostrom (2014), a philosopher at the University of Oxford. Bostrom was examining the 'control problem': how can humans control a super-intelligent AI even when the AI is orders of magnitude smarter. Bostrom's thought experiment goes like this: suppose that someone programs and switches on an AI that has the goal of producing paperclips. The AI is given the ability to learn, so that it can invent ways to achieve its goal better. As the AI is super-intelligent, if there is a way of turning something into paperclips, it will find it. It will want to secure resources for that purpose. The AI is single-minded and more ingenious than any person, so it will appropriate resources from all other activities. Soon, the world will be inundated with paperclips. 

It gets worse. We might want to stop this AI. But it is single-minded and would realise that this would subvert its goal. Consequently, the AI would become focussed on its own survival. It is fighting humans for resources, but now it will want to fight humans because they are a threat (think The Terminator). 

This AI is much smarter than us, so it is likely to win that battle. We have a situation in which an engineer has switched on an AI for a simple task but, because the AI expanded its capabilities through its capacity for self-improvement, it has innovated to better produce paperclips, and developed power to appropriate the resources it needs, and ultimately to preserve its own existence.

Bostrom argues that it would be difficult to control a super-intelligent AI – in essence, better intelligence beats weaker intelligence. Tweaks to the AI’s motivation may not help. For instance, you might ask the AI to produce only a set number of paperclips, but the AI may become concerned we might use them up, and still attempt to eliminate threats. It is hard to program clear preferences, as economists well know.

The conclusion is that we exist on a knife-edge. Turning on such an AI might be the last thing we do.

A jungle model....

....MUCH MORE

The author, Joshua Gans, is Professor of Strategic Management and Jeffrey S. Skoll Chair of Technical Innovation and Entrepreneurship, Rotman School of Management University Of Toronto.

Also at VoxEU:

Measuring geopolitical risk through Finnish news media 

So, What Are They Thinking About At Oxford's Future of Humanity Institute?

From Interesting Engineering, October 7:

The Singleton Hypothesis Suggests Humanity's Possible Future
Researchers say intelligent life on Earth will eventually form into a "Singleton."

What are the odds of an existential risk befalling humanity?

That's the question being studied by Nick Bostrom, a professor of Philosophy, and director of the Future of Humanity Institute at Oxford University. In an attempt to explore this question, he has proposed the singleton hypothesis.

The singleton hypothesis supposes that, at some point in the future, a global government could form based on a world government, a super-intelligent machine, or a dictatorship  – and that it might be impossible to overthrow. From national territories to technology and taxation, this 'singleton' would control everything, and no other power in the world would be able to challenge its supremacy.

The singleton could be composed of a group of powerful politicians, an artificial intelligence-based machine, a resourceful corporate entity, or even a power-hungry dictator with a strong army at their disposal. Moreover, the outcomes of a singleton formation can be positive, negative, or even neutral, so there are several different possibilities. A singleton may either provide a positive form of stability to the world, or an authoritarian regime in which everyone was constantly being watched and no one was safe. 

Why singleton may overtake the world

Bostrom argues that situations could arise in which there could be broad support for either a technology, like an AI or a single government agency to take control of global society. Examples of such events include wars, pandemics, a failure of infrastructure caused by cyberattacks, technological dominance, or trade conflicts. The League of Nations, and then the UN – formed after World War I and II respectively – can be considered examples of limited forms of singletons created in response to world events. When such events happen, Professor Bostrom argues that human society tends to move towards a higher level of societal organization.

There was a time when humans lived as hunter-gatherers, then coalesced into different communities, cities, states, religions, and countries, and now we live in a globalized world. A singleton could be the next stage of our societal development where, instead of different superpowers, we might see the rise of a super-intelligent agency with unbridled access to knowledge (or data), modern technologies (nanotechnology, artificial intelligence, internet, etc), and military power that functions as a single world government....

.....MUCH MORE

Here are some of the FHI's publications, with links to the rest of the site. 

"Science in 2050: the future breakthroughs that will shape our world — and beyond"

From the journal Nature, December 30, 2025:

Nuclear fusion. People on Mars. Artificial general intelligence. These are just some of the advances that could come by the mid-century mark.  

The Roman sage Marcus Aurelius said we should never let the future disturb us. But then he never had a conversation with the futurologist Nick Bostrom about the state of the world in 2050.

“There’s a good likelihood that by 2050, all scientific research will be done by superintelligent AI rather than human researchers,” Bostrom said in an e-mail. “Some humans might do science as a hobby, but they wouldn’t be making any useful contributions.”

Time to rethink your career options, Nature readers!

To adapt a cliché about computer models, predictions of the future are usually wrong, but some are interesting. And Nature has a long history of seeking stimulation in forecasts, projections and auguries about how research might unfold in the coming decades. Most notably, the journal marked the end of the twentieth century and the onset of the twenty-first with supplements dedicated to scientific soothsaying, and a bold prediction, from then-editor Philip Campbell, that life based on something other than DNA would be discovered by 2100. (It was a statement he called foolish then, but stands by today.)

So, with peer review suspended and Nature’s stated aim of discussing the “interpretation of topical and coming trends” firmly underlined, let’s set the controls of our (stubbornly still undiscovered) time machine for 2050 and take a cautious peek outside. Prepare for technological leaps, solving the puzzle of dark matter and perhaps being able to study enough people to wave goodbye to many diseases.

Hot times
You should probably brace yourself before opening the door. “It will be worse than we had anticipated in terms of climate change,” says Guy Brasseur, a modeller at the Max Planck Institute for Meteorology in Hamburg, Germany.

The world will have barrelled past the crucial threshold of 2 °C of average warming above pre-industrial levels by 2040, he suggests. (To avoid that, given the inertia in the climate system, the International Panel on Climate Change says that global emissions needed to peak in 2025 and then decline sharply; see go.nature.com/4prom5j.) So, by 2050, political debate on the reality of a warming world could have melted with the glaciers.

Arguments might rage instead about whether or not to try to cool the planet, most probably by injecting shiny particles into the upper atmosphere that keep sunlight from hitting the surface. Although this geoengineering technique is unproven and untested at significant scale, severe climate impacts by 2050 could encourage an affected nation or even a company to stage such an atmospheric intervention.

“You could have some countries that are using it unilaterally,” Brasseur says, “just thinking that it will solve their problem without looking at the consequence for others.” The intervention could change rainfall patterns and disrupt other aspects of the weather, perhaps making the situation even worse. “I think it should be forbidden,” Brasseur says.

Geopolitical tensions mean that forecasts of future climate to 2050 must increasingly account for more than atmospheric physics, Brasseur and his colleagues pointed out in a 2025 study¹. A decade ago, climate scientists were celebrating global recognition of climate change with the Paris agreement. Today, those in the United States are being forced to delete the term from government reports and websites. Meanwhile, other priorities have taken over.

“If we talk about climate science, people just don’t want to hear about it because they’re much more scared by other issues. They want food, they want peace,” he says. All of that points to the world in 2050 facing the prospect of 3 °C or more of warming by the end of the century.

Here’s another, more optimistic, option. By 2050, removing carbon dioxide from the air could become such a business opportunity that companies find ways to draw down the circulating greenhouse gas and turn a profit in doing so.

“We will make different stuff from carbon dioxide,” says Elina Hiltunen, a futures researcher at the National Defence University in Helsinki. “It could be plastics or fuels or medication. But they are all manufactured from air.”

Future shock
The vast gulf between these two possible scenarios illustrates the dilemma for futurologists and others who try to map progress and pitfalls beyond the next election cycle. How much can the future be projected from current trends? To what extent will it be driven by disruptive events and inventions that seem unlikely or haven’t been conceived yet? And at which point do predictions become, well, just a bit silly?

Futurology — the systematic, interdisciplinary study of future trends — is best placed to imagine a time 10—15 years away, suggests Richard Watson, co-author of The Children’s Book of the Future (2024) and a former futurist in residence at Imperial College London and the University of Cambridge, UK.

“Under five years, you just get the gravitational pull of the present and you end up talking about next Tuesday,” he says. “Over about 20 years, it gets very sci-fi very quickly. I’ve run workshops for banks, and the minute you go to the 2050s you invariably get aliens living on the moon.”

Still, 2050 is a prominent timestamp and a seductive deadline for those who want to set goals and steer investment to reach them. Space agencies, for example, routinely plan that far ahead because it can take a couple of decades for missions to be conceived, approved, designed, built and launched.

The European Space Agency, for one, has already canvassed the research community for ideas for projects for 2050. Submissions include an orbiting antimatter detector, bringing frozen samples of a comet’s icy body back to Earth and landing a robot explorer on the surface of Mercury.

Then there’s Mars. US President Donald Trump restated a goal earlier this year for NASA to send humans to Mars well before 2050, whereas Elon Musk has claimed that his company SpaceX could send an uncrewed starship to the red planet as soon as 2026, as part of a plan to send people there in the 2030s.

Emilia Javorsky, director of the Futures Program at the Future of Life Institute, a think tank in Campbell, California, that analyses transformative technologies, is sceptical about people making that trip, given the risks of space radiation and the long-term impacts of microgravity.

“This is a field that’s driven predominantly by engineers, aerospace engineers, mechanical engineers that greatly underestimate the biological challenges,” she says. “Everyone assumes that we’ll find some material, some way to sort of hack ourselves out of it.”

Rise of the machines
Any vision of the future has to try to account for the continuing ascent of artificial intelligence. But at what pace?

“I could give you a relatively good view of what AI will look like in 2027, 2028. But I don’t know if I can be confident about where we’d be in 2030,” says Alex Ayad, co-founder of the London research and foresight company Outsmart Insight.

The picture beyond then is even fuzzier, but some in the AI field argue that by 2050, a machine-learning system could do science worthy of a Nobel prize....

....MUCH MORE 

As noted in October 2025, introducing the Nobel prize article:

"Will AI ever win its own Nobel? Some predict a prize-worthy science discovery soon"
I don't see why not, especially the Peace or Literature prizes. Throw in the Econ. thingamajig and I'd take even odds over a forty-year time frame.

I mean the darn things are already at the level where they are asking to open margin accounts.... 

And if one is so inclined, we have a few dozen posts on Mr. Bostrom to peruse:

https://climateerinvest.blogspot.com/search?q=Nick+Bostrom

Oxford's Future of Humanity Institute Has Shut Down

Well. I guess that's that. It's over. Fin. I wonder if they saw it coming?

From the FHI homepage:

Future of Humanity Institute

The Future of Humanity Institute closed on 16 April 2024. Research in the fields where FHI was active may continue to be pursued elsewhere within the University and in many external organizations that have sprung up since FHI’s founding.

We used to visit to see what the Institute's founder, Nick Bostrom had to say about existential risk:

So, What Are They Thinking About At Oxford's Future of Humanity Institute? 

Risk/Prop Bets: How unlikely is a doomsday catastrophe? 

...If interested we mentioned Bostrom in last week's "The Roubini Cascade: Are we heading for a Greater Depression?". His Future of Humanity Institute at Oxford seems to have moved on from the mundane climate cataclysm or cosmic fireball ending everything, to a very real, very serious examination of whether or not Artificial Intelligence may be the nail in humanity's coffin, so to speak.... 

Puny Human, I Scoff At Your AI, Soon You Will Know The Power Of Artificial 'Super' Intelligence. Tremble and Weep
[insert 'Bwa Ha Ha' here, if desired] 

 "The future has become a financial fetish, sliced into probable outcomes to be calculated and bet upon"
You say that like its a bad thing.

Risk/Prop Bets: How unlikely is a doomsday catastrophe?

Before we get into the meat of the matter, some things to know upfront. From the introduction to 2018's "Tips And Tricks For Investing In 'End of the World Funds'":

As unauthorized representatives for Long or Short Capital's End of the World Puts this is an area of profound interest from which we have gleaned some insight:

1) Should the world end, collecting on your bet can be a challenge. Know your counterparty!
     And possibly more important, demand collateral!
2) The swings in end of the world product prices can be dramatic.
3) Prognosticators have predicted 100,000 of the last 0 termination events....

And from arXiv (astrophysics) at Cornell:

How unlikely is a doomsday catastrophe?

Max Tegmark (MIT), Nick Bostrom (Oxford)

Numerous Earth-destroying doomsday scenarios have recently been analyzed, including breakdown of a metastable vacuum state and planetary destruction triggered by a "strangelet'' or microscopic black hole. We point out that many previous bounds on their frequency give a false sense of security: one cannot infer that such events are rare from the the fact that Earth has survived for so long, because observers are by definition in places lucky enough to have avoided destruction. We derive a new upper bound of one per 10^9 years (99.9% c.l.) on the exogenous terminal catastrophe rate that is free of such selection bias, using planetary age distributions and the relatively late formation time of Earth....

If interested we mentioned Bostrom in "The Roubini Cascade: Are we heading for a Greater Depression?". His Future of Humanity Institute at Oxford seems to have moved on from the mundane climate cataclysm or cosmic fireball ending everything, to a very real, very serious examination of whether or not Artificial Intelligence may be the nail in humanity's coffin, so to speak. 

And related:

High Frequency Trading, the Grossman-Stiglitz Paradox and a Cheery Doomsday Analysis

How much value can our decisions create? (there's an upper limit)

This is the first of two papers from Oxford's Future of Humanity Institute that we will link to today. 

After watching these folks for over a decade I'm still not sure if the founder of the group, Nick Bostrom, is a charlatan or a visionary. I do know he is very sharp. Early on, the FHI was very gloomy, "the end is nigh"; "we're all gonna die" stuff, but I think he realized that once you've said that a few times there isn't really any reason for the Institute to remain so they've broadened and nuanced the message. Et voilà, seventeen years after founding they are still thinking about stuff and writing:

WHAT IS THE UPPER LIMIT OF VALUE?

January 27, 2021

ABSTRACT
How much value can our decisions create? We argue that unless our current understanding of physics is wrong in fairly fundamental ways, there exists an upper limit of value relevant to our decisions. First, due to the speed of light and the definition and conception of economic growth, the limit to economic growth is a restrictive one. Additionally, a related far larger but still finite limit exists for value in a much broader sense due to the physics of information and the ability of physical beings to place value on outcomes. We discuss how this argument can handle lexicographic preferences, probabilities, and the implications for infinite ethics and ethical uncertainty.

1 Introduction

The future of humanity contains seemingly limitless possibility, with implications for the value of our choices in the short term. Ethics discusses those choices, and for consequentialists in particular, infinities have worrying ethical implications. Bostrom [ 1] and others have asked questions, for example, about how aggregative consequentialist theories can deal with infinities. Others have expanded the questions still further, including measure problems in cosmology, and related issues in infinite computable or even noncomputable universes in a multiverse.

In this paper, we will argue that "limitless" and "infinite" when used to describe value or the moral importance of our decisions can only be hyperbolic, rather than exact descriptions. Our physical universe is bounded, both physically1 and in terms of possibility. Furthermore, this finite limit is true both in the near term, and in the indefinite future. To discuss this, we restrict ourselves to a relatively prosaic setting, and for at least this paper, we restrict our interests to a single universe that obeys the laws of physics as currently (partially) understood....

*****

2 Economic Growth and Physical Limits

Economic theory, the study of human choices about allocation of scarce resources3, is useful for describing a large portion of what humans do. This is in large part because it is a positive description, rather than a normative one, and is local in scope. For example, it does not claim that preferences must be a certain way. Instead, economic theory simply notes that humans’ values seem to be a certain way. Given some reasonable local assumptions, this can be used to make falsifiable predictions about behavior. Such a theory is by no means universally correct, as noted below, but forms a more useful predictive theory than most alternatives. 

 

Clearly, the arguments and assumptions do not need to extend indefinitely to be useful. For example, economic assumptions such as non-satiation (which Mas-Collel [ 6] and others more carefully refer to as local non-satiation) will obviously fall apart at some point. That is, if blueberries are good, more blueberries are better, but at some point the volume of blueberries in question leads to absurdities [7] and disvalue. Here, we suggest that there are fundamental reasons to question the application of simple economic thinking about value and growth in value to long-term decisions. This is important independent of the broader argument about non-infinite value, and also both informs and motivates that argument....

....MUCH MORE (24 page PDF)

 

On that note, I am going to order up some blueberries. Back in a bit.

7 Best-Case Scenarios for the Future of Humanity

From io9:

Most science fictional and futurist visions of the future tend towards the negative — and for good reason. Our environment is a mess, we have a nasty tendency to misuse technologies, and we're becoming increasingly capable of destroying ourselves. But civilizational demise is by no means guaranteed. Should we find a way to manage the risks and avoid dystopic outcomes, our far future looks astonishingly bright. Here are seven best-case scenarios for the future of humanity....

1. Status quo

While this is hardly the most exciting outcome for humanity, it is still an outcome. Given the dire warnings of Sir Martin Rees, Nick Bostrom, Stephen Hawking, and many others, we may not be around to see the next century. Our ongoing survival — even if it's under our current state of technological development — could be considered a positive outcome. Many have suggested that we've already reached our pinnacle as a species.

Back in 1992, political scientist Francis Fukuyama wrote The End of History and the Last Man in which he argued that our current political, technological, and economic mode was the final stop on our journey. He was wrong, of course; Fukuyama's book will forever be remembered as a neoconservative's wet dream written in reaction to the collapse of the Soviet Union and the rise of the so-called New World Order. More realistically, however, the call for a kind of self-imposed status quo has been articulated by Sun Microsystems cofounder Bill Joy. Writing in his seminal 2004 article, "Why the Future Doesn't Need Us," Joy warned of the catastrophic potential for 21st century technologies like robotics, genetic engineering, and nanotech. Subsequently, he called for technological relinquishment — a kind of neo-Luddism intended to prevent dystopic outcomes and outright human extinction. The prudent thing to do now, argued Joy, is to make do with what we have in hopes of ensuring a long and prosperous future....MUCH MORE

For the record, I don't always agree with Mr. Joy.
In "Kleiner Perkins' Bill Joy: Fail Early, Fail Small":

I always feel guilty about leaving out Caufield and Byers when I write a KP headline.
Sort of like dropping Smith and Beane from Mother Merrill....

He said:

...“I try to have a first  [investment] round that has a 50/50 mortality rate,” said Joy at MIT’s Emerging Technologies Conference on Wednesday. So essentially the early stage startups he likes to back have a 50 percent chance of not working. “Now a 90 percent mortality rate is not a good use of my time. That’s something for the government to back,” added Joy....

For the life of me I don't remember "backing commercial projects with 90% failure rates" as being among the federal government's enumerated powers.
Mr. Joy must be talking about state governments.

We have quite a few posts on innovation, here's one that echoes the Joy philosophy:
Fail Often, Fail Fast: "Are We Behind On Innovation That Matters?"

See also:
"Top Investment Trends For Futurists" (PXN; TINY)

Earlier this year Kleiner raised the $525 mil. Fund 15 and said Mr. Joy would not be involved with new investments but would rather watch over previous bets.

"The Roubini Cascade: Are we heading for a Greater Depression?"

Thomas Homer-Dixon is one of the big shots of the thinking-about-going-to-hell-in-a-handbasket biz.

(another is Nick Bostrom at Oxford's Future of Humanity Institute)

Michael Lawrence is one of the post-doc researchers at the Institute.

From British Columbia's Cascade Institute, Royal Roads University December 4, 2020:

The Roubini Cascade Are we heading for a Greater Depression?

Michael Lawrence and Thomas Homer-Dixon

Background:Temporary crisis or sustained economic collapse? Even before the COVID-19 pandemic struck, the world had sunk into deep economic uncertainty. Just as the various shocks of the 1970s and 1980s upended post-war Keynesian economics and propelled the flip to monetarism,1 so too did the 2007-2009 financial crisis shake the foundations of monetarism. The consequent “Great Recession” provided few hints as to what macroeconomic paradigm would follow, but it did reveal the economic conditions with which a new framework must now contend: low economic demand; high economic inequality; high savings and low investment by the wealthy; and interest rates stuck near zero, thus enfeebling the chief lever of monetary policy (The Economist 2020a). “A profound shift is now taking place in economics,” The Economist (2020b) recently proclaimed, “of the sort that only happens once in a generation.” 

The coronavirus has worsened this uncertainty, while compelling leaders around the world to implement radically unconventional economic policies. Governments have borrowed and printed vast amounts of money to fund the massive fiscal stimulus at the core of their pandemic response. Canada, for example, is running a budgetary deficit of CAD$328.5 billion in the 2020-21 fiscal year—high above the CAD$36.5 billion deficit of 2019—including an estimated CAD$225.9 billion spent in response to COVID-19. Equal to 15 percent of Canada’s gross domestic product (GDP), this figure represents the largest budgetary deficit (relative to GDP) incurred since reporting began in 1966 (PBO 2020). The International Monetary Fund (IMF 2020) estimates that global government debt will reach an unprecedented level equal to almost 100 percent of global GDP in 2020, up from 83 percent the year before. 

But with both interest rates and inflation near zero, fiscal deficits that would previously have seemed catastrophic now appear to be sustainable, necessary, and even desirable. Whether they acknowledge it or not, governments are implementing core precepts of Modern Monetary Theory (see Box 1), which just years ago was derided as “radical” and “fringe” for its suggestion that governments can and should spend much more than they do, despite the resulting deficits (Pittis 2020a). 

In the midst of an economic paradigm shift, and as governments gamble that unprecedented spending will see us through the pandemic without producing even greater economic catastrophe, renowned economist Nouriel Roubini has made a distressing prediction. Roubini first gained notoriety in 2006 when he proposed—to the bewilderment of many of his peers—that the US housing market was about to collapse (Levitz 2020). He was right, and a global financial crisis soon followed. Now, Roubini forecasts that the global economy will fall into a “Greater Depression”—a period even worse than the Great Depression of the 1930s—within the next decade. Whereas optimists project a V-shaped recovery from the coronavirus slump, and an emerging, more cautious consensus foresees a U-shaped recovery, Roubini predicts that, in the coming years, the graph of economic growth will take on an L-shape as the global economy makes a short-lived rally and falls into depression.

Roubini’s analysis deserves special attention, because he is a uniquely systemic thinker. Whereas other economic commentators focus narrowly on macro-economic factors (interest rates, unemployment, deficits, exchange rates, and the like), Roubini additionally considers factors such as geopolitical tensions, technological advances, political attitudes, demographic change, and environmental crises—as well as the interactions between these factors. Roubini also highlights causation across multiple scales of analysis, from the micro-scale of household finances upwards to industry trends, public policy, international relations, and ultimately the changing nature of globalization itself.  

But tracing the relationships among such a range of factors is a daunting task. In this Brief, we therefore develop a systems map of Roubini’s argument that will allow us to better assess the risks of a Greater Depression and to identify some of the feedbacks that might drive the global economy into this crisis. Follow-up Briefs will further evaluate Roubini’s causal claims, show how they differ from the analyses of other prominent economists, and test their sensitivity to shifts in key underlying factors and trends. 

Box 1: What is Modern Monetary Theory? 

Modern Monetary Theory (MMT) offers one explanation of how money actually works and the consequent implications for government spending. The theory applies exclusively to monetary sovereign countries—those countries in whichthe government is the monopoly issuer of a fiat currency, such as the United States, Canada, Japan, the United Kingdom, and Australia. The currency of these countries is not tied to any other currency or commodity (such as gold); when these governments take on debt, they do so in their own currency. That currency has value because the issuing government decrees that it has value (by “fiat”), and because people act accordingly, as if it has value. These conditions grant monetary sovereign governments significant control over their money supply and the value of their currency.  

At the crux of MMT is the difference between users of money (such as individuals and businesses) and issuers of money (monetary sovereign governments). Users of money must either earn money or borrow money before they can spend it, and spending too much can readily cripple them with debt. Users of money must therefore balance their budgets just like households do. Monetary sovereign governments need not. As the issuers of currency, they simply order money into existence by either printing currency (cash and bonds) or by increasing the numbers in banks’ digital accounts. These governments do not rely exclusively upon tax revenues or borrowing in order to spend. Taxation functions largely to create demand for the government’s currency so that people have an incentive to carry out the work that the government wants done—by building infrastructure and providing public services, for example. People need currency to pay their taxes, and the government issues such currency in ways that achieve its goals.

Many implications of MMT defy common sense, but only because that “common sense” assumes that governments must budget like a household does. Monetary sovereign countries, for example, cannot go broke; they cannot run out of money because they can always issue more. The question “how will the government pay for it?” is irrelevant. Unlike households, governments do not need to come up with the money (through taxation and borrowing) before they spend it; they can simply issue the currency they spend. The balance of the government budget—whether in the form of deficit or surplus—is not the measure of economic health and stability, and repeated calls for fiscal belt-tightening are misguided impediments to better economic policy.

MMT, however, does not promise a free lunch. Governments cannot spend indefinitely without running into big problems. But the limit to government spending is not the size of its budgetary deficit, as many believe; rather the limits are the “real resources” of the economy and inflation. Real resources define an economy’s productive capacity, and include its technology, the quantity and quality of its labor, capital, natural resources, and so on. Inflation decreases purchasing power of the currency, thus limiting what governments can achieve by issuing that currency. MMT proposes that governments should produce money at a rate that stimulates the full use of the country’s productive capacity, but it warns that exceeding that level triggers harmful inflation.

The feasibility and desirability of government spending is instead a matter of what that spending achieves—the extent to which it creates full employment, equitably distributes wealth, and triggers inflation, for example. Historically, deficits have been too small. Limited government spending leaves unused capacity (conventionally understood using such concepts as the “natural” rate of unemployment) and thereby misses opportunities to improve the economy and peoples’ well-being. 

MMT comes up regularly in Roubini’s webcasts. He refers to the deficit spending that MMT advocates as “helicopter drops of money,” a term coined by Milton Friedman to castigate such proposals. Roubini argues that “quantitative easing” (QE)—one of the major policy responses to the Great Recession and a term used frequently today—is essentially the same thing as MMT. Both involve “monetized budget deficits,” wherein central banks finance government budget deficits by buying government bonds—basically by printing money that is channeled into the economy through public spending, transfers, tax breaks, and/or secondary bond markets. By financing the deficit through monetary policy rather than bonds issued in private markets, the government avoids raising the interest rate. The only significant difference between QE and MMT is that—rhetorically, at least—the deficits of the former are temporary while the deficits of the latter are more permanent. Presently, MMT is the de facto (though not official) policy of advanced economies in their response to the economic fallout of the coronavirus pandemic (see Roubini’s 6 October 2020 webcast, 1h 1m).

The economic harms of COVID-19 will be much worse for poorer countries, in part because they do not have monetary sovereignty. Their debts are denominated in foreign currencies. The recommendations of MMT are thus unavailable to poorer governments, which also lack the fiscal space to mount the economic stimulus at the core of rich countries’ response to the pandemic. So, these countries may face a “lost generation” due to low growth and rising poverty. International inequality (economic divergence between rich and poor countries) will almost certainly increase as a result.

This summary is based primarily on: Kelton 2020.

Analysis: Roubini’s argument

Roubini argues that ten trends—what he refers to as the ten “deadly Ds”—that emerged after the global financial crisis of 2007-9 are now pushing the world towards a Greater Depression, sometime in the next decade.3 “These 10 risks,” he contends, “now threaten to fuel a perfect storm that sweeps the entire global economy into a decade of despair” (Roubini 2020a). The deadly Ds would trigger this depression even in the absence of the COVID-19 pandemic, but the pandemic has intensified the underlying problems and accelerated the crisis. And while Roubini argues that all ten trends are advancing today, not all are necessary for the global economy to fall into a Greater Depression. In this sense, a global depression is over-determined.....

....MUCH MORE (24 page PDF) 

See also the map of relationships:

https://cascadeinstitute.org/roubini-cascade-map/

Originally posted January 30, 2021.

"Do We Live In A Simulation? Elon Musk Thinks So, Here's Why"

From SlashGear, August 26:

Just Elon Musk's name is enough to evoke an emotional response in many people, ranging from hope and glee in some to anger and disgust in others. But how about existential dread? It's true that one of Musk's primary drivers in founding SpaceX — his spacecraft manufacturing and launch company — was to create a self-sustaining colony on Mars. This lofty goal would make humanity a multiplanetary species, thereby reducing the risk of human extinction from global catastrophes such as pandemics, nuclear war, or asteroid impacts. But there's another threat Musk is concerned about that can't be evaded anywhere in the universe, from Mars to the Andromeda Galaxy: the fact that we could be living in an enormous computer simulation.

That's right, Elon Musk suspects that he himself — along with you, everyone you know, and indeed the entirety of the observable universe itself — are merely simulations. He's said as much many times throughout the years, admitting, "I've had so many simulation discussions it's crazy," during a Q&A at the 2016 Code Conference.

Taking the argument at face value leaves one grappling with some seriously heady questions. If reality is a simulation, how could we ever know? Would it render all life meaningless? And what happens if someone trips over a power cord and the whole simulation shuts off? To tease out the answers, we'll have to take it from the top.

What is the simulation hypothesis?
At first blush, the idea that we're all NPCs in some sort of cosmic game of "Sims" sounds like the kind of pseudo-scientific technobabble expected from a philosophy minor and an overenthusiasm for "The Matrix." However, it's actually a question that's been around for thousands of years: how can we be sure that reality is real? The technological bent to what's now called the simulation hypothesis was formalized in the paper "Are we living in a computer simulation?" by Nick Bostrom in 2003.

The argument starts something like this: computer processing power will continue to improve. Compare the convincing graphics and immersive experience of today's best VR games to, say, Frogger. Now project that amount of technological advancement forward a few decades. Then consider another few decades after that, and again, and again. Eventually, simulations will become so advanced they'll be indistinguishable from reality. Not only that, but with enough processing power they could theoretically simulate people, along with their memories, thoughts, and fears around finding out that they're simulated.

Given this technological trajectory, there are only three possible outcomes. First, no intelligent species survives long enough to ever create this type of simulation. Second, even though they become capable, no intelligent species ever creates such a simulation. In both of these scenarios, we are not in a simulation, because none ever exists. The third possibility is different though: an intelligent species reaches the threshold to create a simulated reality, and then does so. And this is where the bottom falls out, and we descend into the rabbit hole....

....MUCH MORE

For what it's worth, Einstein's thought experiments were probably more fruitful than puzzling out whether or not we're in a matrix. At least the WEF seems to think so.

If that's not how you roll, "Can You Solve [not] Albert Einstein’s Famous House Riddle?"

Not interested? Howse about physics jokes:

A neutron walks into a bar and asks, “How much for a whiskey?” The bartender says, “For you, no charge.” 

Still no? Don't make me break out the Einstein + economist jokes:

When Albert Einstein died, he met three New Zealanders in the queue outside the Pearly Gates. To pass the time, he asked what were their IQs. The first replied 190. "Wonderful," exclaimed Einstein. "We can discuss the contribution made by Ernest Rutherford to atomic physics and my theory of general relativity". The second answered 150. "Good," said Einstein. "I look forward to discussing the role of New Zealand's nuclear-free legislation in the quest for world peace". The third New Zealander mumbled 50. Einstein paused, and then asked, "So what is your forecast for the budget deficit next year?" (Adapted from Economist June 13th 1992, p. 71).

All right forget Einstein, here's an economist telling a non-economic joke:

....This tale is said to be told by John Kenneth Galbraith on himself:

    As a boy he lived on a farm in Canada. On the adjoining farm lived a girl he was fond of.

    One day as they sat together on the top rail of the cattle pen they watched a bull servicing a cow.
    Galbraith turned to the girl, with what he hoped was a suggestive look, saying, "That looks like it would be fun."

    She replied, "Well.... She’s your cow." 

Please help, it appears the matrix has glitched and I can't stop this descent into babbling madness.

(apologies to long time readers who have suffered through this exact simulation multiple times over the years)

"187 Big Thinkers Answer the Question: What Do You Think About Machines That Think?"

From Open Culture:

It’s time, again, for Edge.org’s annual question. The 2015 edition asks 187 accomplished (and in some cases celebrated) thinkers to answer the question: What Do You Think About Machines That Think?

John Brockman, the literary über agent and founder of Edge.org, fleshes the question out a bit, writing:

In recent years, the 1980s-era philosophical discussions about artificial intelligence (AI)—whether computers can “really” think, refer, be conscious, and so on—have led to new conversations about how we should deal with the forms that many argue actually are implemented. These “AIs”, if they achieve “Superintelligence” (Nick Bostrom), could pose “existential risks” that lead to “Our Final Hour” (Martin Rees). And Stephen Hawking recently made international headlines when he noted “The development of full artificial intelligence could spell the end of the human race.”

But wait! Should we also ask what machines that think, or, “AIs”, might be thinking about? Do they want, do they expect civil rights? Do they have feelings? What kind of government (for us) would an AI choose? What kind of society would they want to structure for themselves? Or is “their” society “our” society? Will we, and the AIs, include each other within our respective circles of empathy?

Numerous Edgies have been at the forefront of the science behind the various flavors of AI, either in their research or writings. AI was front and center in conversations between charter members Pamela McCorduck (Machines Who Think) and Isaac Asimov (Machines That Think) at our initial meetings in 1980. And the conversation has continued unabated, as is evident in the recent Edge feature “The Myth of AI”, a conversation with Jaron Lanier, that evoked rich and provocative commentaries....

...MORE

Eno, as always, is interesting although I can't see his name without thinking of Baby's on Fire.
Here are some prior posts on him:

Brian Eno Answers Nassim Taleb
Baby's on Fire: Finding Eno
Britain: "Did Brian Eno produce Clegg-mania?" and "Music for Earth Day"

And a gritty version of one of his songs:
 
 

Bank America Merrill Lynch On Robotics and Artificial Intelligence

From FT Alphaville:

Signposts on the road to somewhere…

Sometimes you just have to print out the 300 odd page report* on robotic revolution that lands in your inbox.... 

Still though, we’re in good company… 

At 9:30 Tuesday morning, the online retail giant will open its first-ever brick-and-mortar retail store in its 20-year life, in University Village.
The store, called Amazon Books, looks a lot like bookstores that populate malls across the country. Its wood shelves are stocked with 5,000 to 6,000 titles, best-sellers as well as Amazon.com customer favorites.
- Seattle Times 

Brings us back to an old argument really, that the impact of new tech will be felt in ways that are hard to predict. The supposed decline of the physical book being a (limited) case in point. It could be that the Amazon bricks and mortar move is telling us that something is fundamentally wrong with the digital economy. Or it could just be telling us that physical books were understimated. We dunno. 

With that in minds, have some selected extracts and estimates (to be revisited in our ever more robotic futures) from that BofAML tome. Bring your own scepticism, ours is busy. 

We estimate the current robots & AI solutions market at US$153bn by 2020E including US$83bn for robots, and US$70bn for AI-based analytics. Disruptive technologies will yield US$14-33tn in annual economic impact by 2025E through cost reductions and efficiency gains.
…
There is a 50% chance of full AI (high-level machine learning) by 2040-50E and a 90% chance by 2075E according to AI researchers. Experts expect that systems will move on to super-intelligence in less than 30 years thereafter (source: Mueller and Bostrom 2014). The greatest impacts are likely to be felt as a result of knowledge work automation which has the potential to affect over 230mn knowledge workers globally and cUS$9tn in employment costs.
…
AV technology is likely to create a US$87bn solutions market for car manufacturers, parts suppliers, and technology companies by 2030E, of which software will take the largest share at 29% of the total (source: Lux Research). Up to 90% of new cars will be “connected” in developed markets by 2020E, reaching 20% total fleet penetration (source: McKinsey, Hitachi, Telefonica, Gartner). Cars with partial autonomous features will reach 12-13% penetration by 2025E, when the first fully autonomous cars will be commercial. By 2035E, penetration will reach 25%, with fully driverless cars comprising 9-10% of total sales (source: BCG). L4/L5 features are likely to cost an extra US$10,000 when first introduced in 2025 (source: BCG), declining to US$7,000 by 2030 (source: KPMG, IHS)....

(click to enlarge)

...MUCH MORE

"An AI payout? Should companies remunerate society for lost jobs?"

The author of this piece, Tiernan Ray, is one of the best tech writers around.
When he was at the helm of Barron's Tech Trader, other writers would speculate about NVIDIA's big-picture thinking, Tiernan would get on the phone to CEO Jensen Huang, get through, and get the answer.
Impressive to watch and useful in the pursuit of loot and/or insight.

From ZD Net:

Scholars at the high-profile Institute for Humanity propose a scheme in which companies agree to pay back to society a portion of excess profits produced by AI, to compensate for lost jobs. Can it be gamed by corporations is one of the questions that springs to mind.

If artificial intelligence eventually eliminates some jobs for humans, should the parties that profit from AI pay money to society to compensate for the loss?

That's the intriguing question raised by a paper produced last month by the Future of Humanity Institute, the think tank inside Oxford University that is widely cited in the popular press regarding AI, and that is headed by philosopher Nick Bostrom.

The Institute's Centre for the Governance of AI published "The Windfall Clause: Distributing the Benefits of AI for the Common Good," posting it on the arXiv pre-print server. The paper, authored by Cullen O'Keefe, Peter Cihon, Ben Garfinkel, Jade Leung, and Allan Dafoe, proposes that companies that yield excess profit attributable to AI pay put some portion of the money over and above the taxes they would normally pay. 

ZDNet reached out to lead author Cullen O'Keefe, who is also a Juris Doctor candidate at Harvard Law School, requesting to ask follow-up questions. O'Keefe declined, writing in an email to ZDNet that the Institute "has decided against doing publicity on this particular paper." Despite that, ZDNet offers some questions to mull over below. 

The authors write that "the transformative potential of AI has become increasingly salient as a matter of public and political interest," but they argue that there have been few proposals that would make institutional obligations a matter of law. 

The windfall clause, as it's called, would be a commitment made years before any profits materialize, called an ex-ante agreement. Companies would agree to pay out some percentage, even though they don't know for sure if they'll ever realize any excess profit attributable to AI.

The point is to mitigate the deleterious effects of AI. Because while AI may increase aggregate wealth for society, "many have argued that AI could lead to a substantial lowering of wages, job displacement, and even large-scale elimination of employment opportunities as the structure of the economy changes productivity," the authors write. 

There is a "strong consensus" they write, among AI researchers, "that most, if not all, human work can, in theory, be automated." ....  

....MUCH MORE