Wednesday, July 3, 2024

RAND Corporation: "When RAND Made Magic in Santa Monica"

Among other things, RAND has very deep connections with AI, going back almost 70 years. 
Links after the jump. 
 
From Asterisk magazine, June 2024 issue:

RAND’s halcyon days lasted two decades, during which the corporation produced some of the most influential developments in science and American foreign policy. So how did it become just another think tank?

Between 1945 and 1960, RAND operated as the world’s most productive research organization. Initially envisioned as a research arm of the Air Force, RAND made century-defining breakthroughs both in basic science and applied strategic analysis. Its members helped define U.S. nuclear strategy, conceptualized satellites, pioneered systems analysis, and developed the earliest reports on defense economics. They also revolutionized much of STEM: RAND scholars developed the basics of game theory, linear programming, and Monte Carlo methods. They helped conceptualize generalized artificial intelligence, developed the basics for packet switching (which enables data transmission across networks), and built one of the world’s first computers.

Today, RAND remains a successful think tank — by some metrics, among the world’s best. 1 In 2022, it brought in over $350 million in revenue, and large proportions still come from contracts with the US military. Its graduate school is among the largest for public policy in America. 

But RAND’s modern achievements don’t capture the same fundamental policy mindshare as they once did. Its military reports may remain influential, but they hold much less of their early sway, as when they forced the U.S. Air Force to rethink several crucial assumptions in defense policy. And RAND’s fundamental research programs in science and technology have mostly stopped. Gone are the days when one could look to U.S. foreign policy or fundamental scientific breakthroughs and trace their development directly back to RAND. 

How was magic made in Santa Monica? And why did it stop? 

The Roots of RAND

Economists, physicists, and statisticians — civilian scientists to that point not traditionally valued by the military — first proved their utility in the late stages of World War II operational planning. American bomber units needed to improve their efficiency over long distances in the Pacific theater. The scientists hired by the Army Air Force proposed what at the time seemed a radical solution: removing the B-29 bomber’s armor to reduce weight and increase speed. This ran counter to USAAF doctrine, which assumed that an unprotected plane would be vulnerable to Japanese air attacks. The doctrine proved incorrect. The increased speed not only led to greater efficiency, it also led to more U.S. planes returning safely from missions, as Japanese planes and air defense systems were unable to keep up. 2 Civilian scientists were suddenly in demand. By the end of the war, all USAAF units had built out their own operations research departments to optimize battle strategy. When the war ended, the question turned to how to retain the scientific brain trust it had helped to assemble. 

General Henry “Hap” Arnold, who had led the Army Air Force’s expansion into the most formidable air force in the world, had started to consider this question long before the war had ended. He found an answer in September 1945, when Franklin Collbohm, a former test pilot and executive at Douglas Aircraft, walked into Arnold’s office with a plan: a military-focused think tank staffed by the sharpest civilian scientists. Collbohm did not have to finish describing his idea before Arnold jumped and agreed. Project RAND was born.

Arnold, along with General Curtis LeMay — famous for his “strategic bombing” of Japan, which killed hundreds of thousands of civilians — scrounged up $10 million from unspent war funds to provide the project’s seed money, which was soon supplemented with a grant from the Ford Foundation. This put RAND into a privileged position for a research organization: stably funded. 

On top of that financial stability, RAND built what would become one of its greatest organizational strengths: a legendarily effective culture, and a workforce to match it.

Internal Culture and Talent
In an internal memo, Bruno Augestein, a mathematician and physicist whose research on ballistic missiles helped usher in the missile age, highlighted a set of factors that catalyzed RAND’s early success. In short: RAND had the best and brightest people working with the best computing resources in an environment that celebrated excellence, welcomed individual quirks, and dispensed with micromanagement and red tape.

Early RAND leadership was, above all else, committed to bringing in top talent and jealously guarded the sort of intellectual independence to which their academic hires were accustomed. Taking the mathematics department as an example, RAND hired John Williams, Ted Harris, and Ed Quade to run it. While these were accomplished mathematicians in their own right, these three were also able to attract superlative talents to work under and around them. As Alex Abella writes in Soldiers of Reason, his history of RAND, “No test for ideological correctness was given to join, but then none was needed. The nation’s best and brightest joining RAND knew what they were signing on for, and readily accepted the vision of a rational world — America and its Western allies — engaged in a life-and-death struggle with the forces of darkness: the USSR.” 

As the Cold War intensified, the mission became the sell. The aim of RAND, as the historian David Hounshell has it, “was nothing short of the salvation of the human race.” 3 The researchers attracted to that project believed that the only environment in which that aim could be realized was independent of the Air Force, its conventional wisdom, and — in particular — it’s conventional disciplinary boundaries

RAND’s earliest research aligned with the USAF’s (the Army Air Force had become its own service branch in 1947) initial vision: research in the hard sciences to attack problems like satellite launches and nuclear-powered jets. 4 However, the mathematician John Davis Williams, Collbohm’s fifth hire, was convinced that RAND needed a wider breadth of disciplines to support the Air Force’s strategic thinking. He made the case to General LeMay, who supervised RAND, that the project needed “every facet of human knowledge to apply to problems.” 5 To that end, he argued for recruiting economists, political scientists, and every other kind of social scientist. LeMay, once convinced, implored Williams to hire whoever it took to get the analysis right.

And so they did. RAND’s leadership invested heavily in recruiting the best established and emerging talent in academia. An invitation-only conference organized by Williams in New York in 1947 brought together top political scientists (Bernard Brodie), anthropologists (Margaret Mead), economists (Charles Hitch), sociologists (Hans Speier), and even a screenwriter (Leo Rosten). The promise of influence, exciting interdisciplinary research, and complete intellectual freedom drew many of the attendees to sign up.

Within two years, RAND had assembled 200 of America’s leading academics. The top end of RAND talent was (and would become) full of past (and future) Nobel winners, and Williams worked around many constraints — and eccentricities — to bring them on. For instance, RAND signed a contract with John von Neumann to produce a general theory of war, to be completed during a small slice of his time: that spent shaving. For his shaving thoughts, von Neumann received $200 a month, an average salary at the time....

....MUCH MORE

Rand and AI:
RAND: "The Promise and Peril of AI in the Power Grid"
RAND Corporation CEO On The Here-and-Now Dangers Of Artificial Intelligence
RAND: "Artificial Intelligence and Biotechnology: Risks and Opportunities"
RAND has a very deep history in artificial intelligence. From Jeremy Norman's History of Information:
Newell, Simon & Shaw Develop the First Artificial Intelligence Program

During 1955 and 1956 computer scientist and cognitive psychologist Allen Newell, political scientist, economist and sociologist Herbert A. Simon, and systems programmer John Clifford Shaw, all working at the Rand Corporation in Santa Monica, California, developed the Logic Theorist, the first program deliberately engineered to mimic the problem solving skills of a human being. They decided to write a program that could prove theorems in the propositional calculus like those in Principia Mathematica by Alfred North Whitehead and Bertrand Russell. As Simon later wrote,

"LT was based on the system of Principia mathematica, largely because a copy of that work happened to sit in my bookshelf. There was no intention of making a contribution to symbolic logic, and the system of Principia was sufficiently outmoded by that time as to be inappropriate for that purpose. For us, the important consideration was not the precise task, but its suitability for demonstrating that a computer could discover problem solutions in a complex nonnumerical domain by heuristic search that used humanoid heuristics" (Simon,"Allen Newell: 1927-1992," Annals of the History of Computing 20 [1998] 68).

The collaborators wrote the first version of the program by hand on 3 x 5 inch cards. As Simon recalled....