Monetizing Intellectual Property and Google's Machine Learning

From stratechery:

TensorFlow and Monetizing Intellectual Property

Ten years ago Bill Gates suggested that open source software was the province of “modern-day sort of communists” whose views on intellectual property were hopelessly outdated:

The idea that the United States has led in creating companies, creating jobs, because we’ve had the best intellectual property system — there’s no doubt about that in my mind, and when people say they want to be the most competitive economy, they’ve got to have the incentive system. Intellectual property is the incentive system for the products of the future.

Gates’ perspective was understandable: he had built Microsoft into the biggest company in technology and one of the biggest in the world by, for all intents and purposes, selling licenses to text. Sure, that’s a dramatic over-simplification of Windows and the other software Microsoft sold, but that didn’t change what a seachange the Redmond-based company seemed to represent: one where the pure expression of ideas could make you the richest person in the world. Yet those antediluvian open-source zealots wanted to simply give it all away.

The Open-Sourcing of TensorFlow
Microsoft is still a big company — their market cap was $427 billion at yesterday’s market close — but an even bigger company today is Alphabet ($506 billion), which has a decidedly different approach:

earlier this week its Google subsidiary announced it was open-sourcing TensorFlow, its formerly proprietary machine learning system. From the official Google blog:

Just a couple of years ago, you couldn’t talk to the Google app through the noise of a city sidewalk, or read a sign in Russian using Google Translate, or instantly find pictures of your Labradoodle in Google Photos. Our apps just weren’t smart enough. But in a short amount of time they’ve gotten much, much smarter. Now, thanks to machine learning, you can do all those things pretty easily, and a lot more. But even with all the progress we’ve made with machine learning, it could still work much better.

So we’ve built an entirely new machine learning system, which we call “TensorFlow.” TensorFlow is faster, smarter, and more flexible than our old system, so it can be adapted much more easily to new products and research. It’s a highly scalable machine learning system — it can run on a single smartphone or across thousands of computers in datacenters. We use TensorFlow for everything from speech recognition in the Google app, to Smart Reply in Inbox, to search in Google Photos. It allows us to build and train neural nets up to five times faster than our first-generation system, so we can use it to improve our products much more quickly.

We’ve seen firsthand what TensorFlow can do, and we think it could make an even bigger impact outside Google. So today we’re also open-sourcing TensorFlow. We hope this will let the machine learning community — everyone from academic researchers, to engineers, to hobbyists — exchange ideas much more quickly, through working code rather than just research papers. And that, in turn, will accelerate research on machine learning, in the end making technology work better for everyone.

Machine learning is super important to Google; just a couple of weeks ago, on Alphabet’s Q3 earnings call, Google CEO Sundar Pichai stated in his opening remarks, “I also want to point out that our investments in machine learning and artificial intelligence are a priority for us”, and followed that up with a series of examples where machine learning was serving as a differentiator for Google. Pichai later added, in response to a question:

Machine learning is a core transformative way by which we are rethinking everything we are doing. We’ve been investing in this area for a while. We believe we are state-of-the-art here. And the progress particularly in the last two years has been pretty dramatic. And so we are thoughtfully applying it across all our products, be it search, be it ads, be it YouTube and Play et cetera. And we are in early days, but you will see us in a systematic manner, think about how we can apply machine learning to all these areas.

At a superficial level, this doesn’t make sense: if machine learning is core to Google’s future, then what is the point of giving it away? Does the company not care about making money?...

...MUCH MORE

Previously on the Puny Earthlings channel:

How Google Aims To Dominate Artificial Intelligence
"Why Is Machine Learning (CS 229) The Most Popular Course At Stanford?"
'Deep Learning' as Applied to Investing
Deep Learning is VC Worthy
Zuckerberg, Musk Invest in Machine Learning Artificial-Intelligence Company, Vicarious
Google's Plan To Make Your Brain Irrelevant (GOOG; EVIL;)
MIT's Technology Review: "10 technologies we think most likely to change the world"
"As Machines Get Smarter, Evidence They Learn Like Us"
Researcher Dreams Up Machines That Learn Without Humans
Google Launches the Quantum Artificial Intelligence Lab (GOOG)
Baidu Artificial Intelligence Beats Google, Microsoft In Image Recognition
"Inside Google’s Massive Effort in Deep Learning" (GOOG)
Artificial Intelligence Company Sentient Raises Another $103 Million, Emerges From Stealth 
Deep Learning: "A Common Logic to Seeing Cats and Cosmos"
Federal Reserve Board FEDS Notes: "Using big data in finance: Example of sentiment-extraction from news articles"
DeepMind: Google's Artificial Intelligence Guy (GOOG)

And quite a few more, use the search blog box if interested.

Additionally, here's the GOOG's Research blog.

Artificial Intelligence: "Now You Too Can Buy Cloud-Based Deep Learning"

From IEEE Spectrum, July 27:

Cloud-computing services deliver AI to the rest of us

Facebook’s deep-learning artificial intelligence systems have learned to recognize your friends in your photos, and Google’s AI has learned to anticipate what you’ll be searching for. But there’s no need to feel left out, even if your company’s computers haven’t learned much lately.

A growing number of tech giants and startups have begun offering machine learning as a cloud service. That means other companies and startups do not need to develop their own specialized hardware or software to apply deep ­learning—the high-powered version du jour of machine learning—to their specific business needs.

“Deep-learning algorithms dominate other machine-learning methods when data sets are large,” says Zachary Chase Lipton, a deep-learning researcher in the Artificial Intelligence Group at the University of California, San Diego, who has examined cloud AI services from companies such as Amazon and IBM. “Thus any company or application that has well-formed prediction problems—such as forecasting demand or translating between languages—could benefit from deep learning.”

With cloud-based deep learning, companies can simply select a cloud service and browse its online offerings of application programming interfaces for software tasks such as recognizing images of corgi dogs or automatically translating a restaurant menu. Some services will even tailor their machine-learning tools to the data and needs of individual companies.

According to Lipton, the rise of cloud services for machine learning hinges on at least two factors: first, a continued rise in the demand for machine learning as the technology has matured in its ability to solve a wide variety of problems with economic value; and second, the relative scarcity of machine-learning talent, which makes it tough for every company to build its own machine-learning team. Competition for talent has become even tougher with startups trying to compete with tech giants like Microsoft and IBM, which can afford to vacuum up the best and brightest.

Most commercial applications of machine learning rely on supervised learning. This involves algorithms that can observe correctly labeled examples and learn to perform certain tasks through imitation. Artificial neural networks are currently the most popular and successful algorithms for supervised machine learning on large data sets. They learn by passing information through an interconnected network of multiple nodes (also known as neurons). The connections between these nodes each have adjustable weights that influence the flow of information through the graph. Nodes are generally arranged in layers. But historically it was feasible to train networks with only one hidden layer of neurons in addition to the input and output layers.

Deep learning takes these methods to the next level by filtering the data through multiple layers of neurons, ­Lipton explains. At each layer, the network can learn successively more abstract representations of relationships between data points. With enough layers and enough nodes, deep neural networks can perform a host of functions.

The challenge in building a neural network is training it for specific tasks. Starting from a random setting of the weights, examples from the data set are presented to the neural network one after the next. Each time, the neural network’s weights are tuned slightly to bring the network’s output closer to the correct output.

A number of startups seem primarily interested in demonstrating their deep-learning research in order to draw the attention of larger companies that might acquire them, Lipton notes. Salesforce.com and Twitter have acquired ­startups, including MetaMind and Whetlab, respectively. Some of these acquisitions have been done to swell the ranks of these tech giants’ own deep-learning teams....MORE

Disrupting Surveillance Capitalism

A good overview of some of the techniques we've looked at over the years and following up on yesterday's "Potemkin AI: Many instances of 'artificial intelligence' are artificial displays of its power and potential", demystifying AI and showing how it is still pretty dumb—compared to what's coming.
From Logic Magazine:

Monkeywrenching the Machine
Silicon Valley’s surveillance-based business model relies heavily on machine learning. But with the right techniques, we can resist the enclosure of our lives for profit and disrupt the disruptors.
This piece is also available in audio from our friends at Curio.io.

Joseph Redmon, "YOLO: Real-Time Object Detection."

Machine learning is the practice of training algorithms to classify and predict in order to support decision-making. In recent years, it has skyrocketed in popularity and ubiquity. It's no stretch to say that most services we use now incorporate machine learning in one way or another. In its pervasiveness, machine learning is becoming infrastructural. And, like all infrastructure, once it matures it will become invisible.

Before that happens, we should develop a way to disrupt it.

A relatively nascent field called "adversarial machine learning" provides a starting point. Described as the intersection of cybersecurity and machine learning, this field studies how these algorithms can be systematically fooled, with or without knowledge of the algorithm itself—an ideal approach, since the specifics of many algorithms are trade secrets. And given the fact that the machine learning regime effectively makes all of us its workers—most of our online activity is in fact labor towards the improvement of these systems—we as individual users have an opportunity to inflict major sabotage.

Consider an early and now-ubiquitous application of machine learning: the everyday spam filter. The job of the spam filter is to categorize an email as either "spam"—junk—or "ham"—non-spam. The simplest case of adversarial machine learning in this context is constructing an email that is spam—a pitch for a pharmaceutical product, for example—but in such a way that the spam filter misclassifies it as ham, thus letting it through to the recipient.

There are a variety of strategies you might employ to accomplish this. A relatively simple one is swapping out the name of "Viagra" for something more obscure to a machine but equally readable to a human: "Vi@gr@", for example.

Today, most spam filters are resistant to this basic obfuscation attack. But we could consider more sophisticated approaches, such as writing a longer, professional-looking email that hints at the product without ever explicitly mentioning it. The hint may be strikingly obvious to a human, but incomprehensible to a spam filter.

A spam filter is less insidious than many other applications of machine learning, of course. But we can generalize from this example to develop techniques for disrupting other applications more worthy of sabotage.

Poisoning the Well
Most machine learning models are constructed according to the following general procedure:

1. Collect training data.
2. Run a machine learning algorithm, such as a neural network, over the training data to learn from it.
3. Integrate the model into your service.

Many websites collect training data with embedded code that tracks what you do on the internet. This information is supposed to identify your preferences, habits, and other facets of your online and offline activity. The effectiveness of this data collection relies on the assumption that browsing habits are an honest portrayal of an individual.

A simple act of sabotage is to violate this assumption by generating "noise" while browsing. You can do this by opening random links, so that it's unclear which are the "true" sites you've visited—a process automated by Dan Schultz's Internet Noise project, available at makeinternetnoise.com. Because your data is not only used to make assumptions about you, but about other users with similar browsing patterns, you end up interfering with the algorithm's conclusions about an entire group of people.

Of course, the effectiveness of this tactic, like all others described here, increases when more people are using it. As the CIA's Simple Sabotage Field Manual explains, "Acts of simple sabotage, multiplied by thousands of citizens, can be an effective weapon...[wasting] materials, manpower, and time. Occurring on a wide scale, simple sabotage will be a constant and tangible drag on...the enemy."

Attacks of this sort—where we corrupt the training data of these systems—are known as "poisoning" attacks.

The Pathological and the Perturbed
The other category of adversarial machine learning attacks are known as "evasion.” This strategy targets systems that have already been trained. Rather than trying to corrupt training data, it tries to generate pathological inputs that confuse the model, causing it to generate incorrect results.

The spam filter attack, where you trick an algorithm into seeing spam as ham, is an example of evasion. Another is "Hyperface," a collaboration between Hyphen Labs and Adam Harvey, a specially designed scarf engineered to fool facial recognition systems by exploiting the heuristics these systems use to identify faces. Similarly, in a recent study, researchers developed a pair of glasses that consistently cause a state-of-the-art facial recognition system to misclassify faces it would otherwise identify with absolute certainty....MORE

So You Want To Be A Quant: The Best Schools For Your Financial Engineering Degree

In a July 2014 post (reprised a couple days ago) I referred to Dogbert:  

As Carnegie Mellon Financial Engineering prof. Dogbert explains, we are dealing with aggregates here:...

Not just because Carnegie Mellon has a first tier Financial Engineering program, which it does, see QuantNet's rankings below, but also because it runs the premier Machine Learning/AI program in the country, for those who want to pursue a double Masters and/or a Doctorate in ML.

2024 QuantNet Ranking of Best Financial Engineering Programs

1.Baruch College
New York, NY 10010
Master in Financial Engineering

2. Princeton University
Princeton, NJ 08544
Master in Finance

3. Carnegie Mellon University
New York, NY 10005 | Pittsburgh, PA 15213
MS in Computational Finance

....MUCH MORE

During the twenty-teens Carnegie-Mellon was usually found at the top of the leader board until Baruch College at CUNY slipped past them in 2017.  

Regarding AI/ML here's the school bragging a little bit:

Globally Known in AI and Machine Learning

CS Rankings
AI: 1
Machine learning and data mining: 1
CS Rankings ranks Carnegie Mellon University as the top university for machine learning and AI programs.

Towards AI
Ph.D. Program:
AI: 1
Machine learning and data mining: 1

Master's Program:
AI: 1
Machine learning and data mining: 1
Towards AI ranks the Machine Learning Department as the best educational research institution for machine learning graduate programs, both for Ph.D. and master's in machine learning.

 

U.S. News Report
AI: 1
U.S. News Report ranks Carnegie Mellon University as the best institution for artificial intelligence (AI) programs.

Analytics India Magazine
Master's Program:
Machine learning and data mining: 1
AIM ranks the Machine Learning Department as the best institution for machine learning master's programs.

If interested, here's a 2016 post on the ML concentration:

Interview: Manuela Veloso Head of Machine Learning, Carnegie Mellon University

Here's one of Carnegie-Mellon's graduates:

Sensei Kai-Fu Lee on AI in 2041

Artificial Intelligence: "Lines of Sight"

From Logic Magazine:

You can’t understand the AI systems that are transforming our world without understanding the datasets they are built on.

On the night of March 18, 2018, Elaine Herzberg was walking her bicycle across a dark desert road in Tempe, Arizona. After crossing three lanes of a four-lane highway, a "self-driving" Volvo SUV, traveling at thirty-eight miles per hour, struck her. Thirty minutes later, she was dead. The SUV had been operated by Uber, part of a fleet of self-driving car experiments operating across the state. A report by the National Transportation and Safety Board determined that the car's sensors had detected an object in the road six seconds before the crash, but the software "did not include a consideration for jaywalking pedestrians." In the moments before the car hit Elaine, its AI software cycled through several potential identifiers for her—including “bicycle,” “vehicle,” and “other”—but, ultimately, was not able to recognize her as a pedestrian whose trajectory would be imminently in the collision path of the vehicle. 

How did this happen? The particular kind of AI at work in autonomous vehicles is called machine learning. Machine learning enables computers to “learn” certain tasks by analyzing data and extracting patterns from it. In the case of self-driving cars, the main task that the computer must learn is how to see. More specifically, it must learn how to perceive and meaningfully describe the visual world in a manner comparable to humans. This is the field of computer vision, and it encompasses a wide range of controversial and consequential applications, from facial recognition to drone strike targeting.  

Unlike in traditional software development, machine learning engineers do not write explicit rules that tell a computer exactly what to do. Rather, they enable a computer to “learn” what to do by discovering patterns in data. The information used for teaching computers is known as training data. Everything a machine learning model knows about the world comes from the data it is trained on. Say an engineer wants to build a system that predicts whether an image contains a cat or a dog. If their cat-detector model is trained only on cat images taken inside homes, the model will have a hard time recognizing cats in other contexts, such as in a yard. Machine learning engineers must constantly evaluate how well a computer has learned to perform a task, which will in turn help them tweak the code in order to make the computer learn better. In the case of computer vision, think of an optometrist evaluating how well you can see. Depending on what they find, you might get a new glasses prescription to help you see better. 

To evaluate a model, engineers expose it to another type of data known as testing data. For the cat-detector model, the testing data might consist of both cats and other animals. The model would then be evaluated based on how many of the cats it correctly identified in the dataset. Testing data is critical to understanding how a machine learning system will operate once deployed in the world. However, the evaluation is always limited by the content and structure of the testing data. For example, if there are no images of outdoor cats within the testing data, a cat-detector model might do a really good job of recognizing all the cats in the testing data, but still do poorly if deployed in the real world, where cats might be found in all sorts of contexts. Similarly, evaluating Uber’s self-driving AI on testing data that doesn’t contain very many jaywalking pedestrians will not provide an accurate estimate of how the system will perform in a real-world situation when it encounters one.  

Finally, a benchmark dataset is used to judge how well a computer has learned to perform a task. Benchmarks are special sets of training and testing data that allow engineers to compare their machine learning methods against each other. They are measurement devices that provide an estimate of how well AI software will perform in a real-world setting. Most are circulated publicly, while others are proprietary. The AI software that steered the car that killed Elaine Herzberg was most likely evaluated on several internal benchmark datasets; Uber has named and published information on at least one. More broadly, benchmarks guide the course of AI development. They are used to establish the dominance of one approach over another, and ultimately influence which methods get utilized in industry settings. 

The single most important benchmark in the field of computer vision, and perhaps AI as a whole, is ImageNet. Created in the late 2000s, ImageNet contains millions of pictures—of people, animals, and everyday objects—scraped from the web. The dataset was developed for a particular computer vision task known as “object recognition.” Given an image, the AI should tag it with labels, such as “cat” or “dog,” describing what it depicts. 

It is hard to overstate the impact that ImageNet has had on AI. ImageNet has inaugurated an entirely new era in AI, centered on the collection and processing of large quantities of data. It has also elevated the benchmark to a position of great influence. Benchmarks have become the way to evaluate the performance of an AI system, as well as the dominant mode of tracking progress in the field more generally. Those who have developed the best-performing methods on the ImageNet benchmark in particular have gone on to occupy prestigious positions in industry and academia. Meanwhile, the AI systems built atop of ImageNet are being used for purposes as varied as refugee settlement mapping and the identification of military targets—including the technology that powers Project Maven, the Pentagon’s algorithmic warfare initiative. 

The assumption that lies at the root of ImageNet’s power is that benchmarks provide a reliable, objective metric of performance. This assumption is widely held within the industry: startup founders have described ImageNet as the “de-facto image dataset for new algorithms,” and most major machine learning software packages offer convenient methods for evaluating models against it. As the death of Elaine Herzberg makes clear, however, benchmarks can be misleading....

....MUCH MORE

Related, 2016's "Machine Learning and the Importance of 'Cat Face'" and many, many more:

Disrupting Surveillance Capitalism

....Poisoning the Well
Most machine learning models are constructed according to the following general procedure:

1. Collect training data.
2. Run a machine learning algorithm, such as a neural network, over the training data to learn from it.
3. Integrate the model into your service.

Many websites collect training data with embedded code that tracks what you do on the internet. This information is supposed to identify your preferences, habits, and other facets of your online and offline activity. The effectiveness of this data collection relies on the assumption that browsing habits are an honest portrayal of an individual.

A simple act of sabotage is to violate this assumption by generating "noise" while browsing. You can do this by opening random links, so that it's unclear which are the "true" sites you've visited—a process automated by Dan Schultz's Internet Noise project, available at makeinternetnoise.com. Because your data is not only used to make assumptions about you, but about other users with similar browsing patterns, you end up interfering with the algorithm's conclusions about an entire group of people..... 

Bank of England: "Opening the machine learning black box" 

"The Fundamental Limits of Machine Learning"

Actually, we have so many posts on this stuff that you could probably pass this course: 

"Why Is Machine Learning (CS 229) The Most Popular Course At Stanford?"

which was a 2013 post.

 2013

And which led, naturally enough, to 2014's "Deep Learning is VC Worthy".

Here's a search of the blog for Machine Learning and another for Adversarial, as in adversarial networks. If we search for the more general AI at the GOOG we see 651 hits including some old favorites.

site:climateerinvest.blogspot.com AI

"Q&A with Steve Roddy, vice president of the machine learning group at Arm, on culling neural nets for embedded devices, competing with Nvidia and Intel, more"

Here's hoping that Softbank and Mr. Son are compelled by recent misadventures to re-float ARM.
From Dean Takahashi at VentureBeat:

How Arm wants to bring machine learning to ordinary computing devices

Arm may be a bit late to the whole machine learning and artificial intelligence bandwagon, at least with specialized designs for modern chips. But the designer of chip intellectual property has everybody beat in terms of volumes of AI and machine-learning chips deployed in the widest array of devices.

Arm’s customers, which include rivals Intel and Nvidia, are busy deploying AI technology everywhere. The company is also creating specific machine-learning instructions and other technology to make sure AI gets built into just about everything electronic, not just the high-end devices going into servers.

On the server level, customers such as Amazon are bringing ARM-based machine learning chips into datacenters. I talked with Steve Roddy, vice president of the machine learning group at Arm at the company’s recent TechCon event in San Jose, California.
Here’s an edited transcript of our interview.

VentureBeat: What is your focus on machine learning?

Steve Roddy: We have had a machine learning processor in the market for a year or so. We aimed at the premium consumer segment, which was the obvious first choice. What is Arm famous for? Cell phone processors. That’s where the notion of a dedicated NPU (Neural Processing Unit) first appeared, in high-end cell phones. Now you have Apple, Samsung, MediaTech, [and] Huawei all designing their own, Qualcomm, and so on. It’s commonplace in a $1,000 phone.

What we’re introducing is a series of processors to serve not only that market, but also mainstream and lower-end markets. What we originally envisioned — we entered the market to serve people building VR glasses, smartphones, places where you care more about performance than cost balancing and so on. History would suggest that the feature set shows up in the high-end cell phone, takes a couple years, and then moves down to the mainstream-ish $400-500 phone, and then a couple years later that winds up in the cheaper phone.

I think what’s most interesting about how fast the whole NPU machine learning thing is moving is that that is happening much faster, but for different reasons than — it used to be, okay, the 8 megapixel sensor starts here, and then when it’s cheap enough it goes here, and then when it’s even cheaper it goes there. It’s not just that the component cost goes down and integrates in and it’s replaced by something else. It’s that machine learning algorithms can be used to make different or smarter decisions about how systems are integrated and put together to add value in a different way, or subtract cost in a different way....

....MUCH MORE

Related:
"This Tech CEO Says He's Ready to Take on Google, Facebook, Amazon, and Apple" 
"The Natural Evolution of Artificial Intelligence"

ARM Chips with Nvidia AI Could Change the Internet of Things
IoT: "SoftBank's ARM Spends Big to Meet Son's Connected World Dream"
In Case You Missed It: SoftBank Transferred its $5B Stake in NVIDIA Off Its Balance Sheet and Onto the Vision Fund (NVDA)
You Understand Why Mr. Son and SoftBank Are Circling Uber, Right?

ARM Wrestles Its Way Into Supercomputing (9984 Tokyo; INTC)
Sandia National Lab to Install First Petascale Supercomputer Powered by ARM Processors (Masayoshi Son smiles) 

"Why Is Machine Learning (CS 229) The Most Popular Course At Stanford?"

From Forbes:

Stanford professor Andrew Ng teaching his course on Machine Learning (in a video from 2008)

“New Brainlike Computers, Learning From Experience,” reads a headline on the front page of The New York Times this morning. The article focuses on machine-learning algorithms, known as a neural networks, that are becoming increasingly important in computer science. And in 2014 Qualcomm will release the first commercial version  of a neuromorphic processor that transforms this software technique directly into hardware to increase performance for intensive machine learning tasks.

But buried in the last paragraph of the story was the fact that “The largest class on campus this fall at Stanford was a graduate level machine-learning course covering both statistical and biological approaches, taught by the computer scientist Andrew Ng. More than 760 students enrolled.” And several previous versions of the course are available online for free. The most recent is from Coursera (which Ng cofounded with Daphne Koller last year) but the 2008 course is on iTune U, YouTube and Stanford’s Engineering Everywhere.

What’s going on here? Simply put, machine learning is the part of artificial intelligence that actually works. You can use it to train computers to do things that are impossible to program in advance. Ng uses the example of handwriting recognition as a classic example of a problem that can only be achieved through machine learning. In his introductory lecture on Coursera, Ng refers to search engines like Google and Bing, Facebook and Apple’s photo tagging application and Gmail’s spam filtering as everyday examples of machine learning at work. Ng is the director of the Stanford Artificial Intelligence Lab and one of the founders, with Jeff Dean, of Google Brain, a deep learning research project at Google. He is using machine learning as a step towards the “AI dream of someday building machines as intelligent as you or I.”...MORE

Bank of England: "Opening the machine learning black box"

From Bank Underground:

Machine learning models are at the forefront of current advances in artificial intelligence (AI) and automation. However, they are routinely, and rightly, criticised for being black boxes. In this post, I present a novel approach to evaluate machine learning models similar to a linear regression – one of the most transparent and widely used modelling techniques. The framework rests on an analogy between game theory and statistical models. A machine learning model is rewritten as a regression model using its Shapley values, a payoff concept for cooperative games. The model output can then be conveniently communicated, eg using a standard regression table. This strengthens the case for the use of machine learning to inform decisions where accuracy and transparency are crucial.


Why do we need interpretable models?
Statistical models are often used to inform decisions, eg a bank deciding if it grants a mortgage to a customer. Let Alice be that customer. The bank could check her income and previous loan history to estimate how likely Alice is to repay a mortgage and set the terms for the loan. A standard approach for this type of analysis is to use a logistic regression. This returns a ‘probability of default’, ie the chance that Alice won’t be able to repay the loan, spelling trouble for the bank and herself.

A logistic regression is a very transparent model. It attributes well-defined risk weights to each of its inputs. The transparency comes at a cost though: logistic regressions assume a particular relationship between the explanatory factors. This may not hold, in which case the model’s predictions may be misleading. Machine learning models are more flexible and therefore capable to detect finer nuances provided there is enough data to ‘train’ a model. This is also the reason for their current success and increasing popularity ranging from personalised movie recommendations over credit scoring to medical applications.

However, the cost of this flexibility is opaqueness, which gives rise to the black box critique of machine learning models. It is often not clear which inputs are driving the predictions of the model. Furthermore, a well-grounded statistical analysis of the model is generally not possible. This can lead to ethical and legal challenges if models are used to inform decisions affecting the lives of individuals. This is particularly true for models from deep learning which are driving many AI developments.

There also are concerns regarding the interpretability of machine learning models more specific to policy makers, like eg at the Bank of England’s decision-making committees. First, when using machine learning alongside more traditional approaches, it is important to understand where they differ. Second, a challenge in decision making is trying to understand how relationships between variables might change in the light of policy actions. In both cases, interpretable and transparent models are likely to be very helpful.

Opening the black box
It would therefore be of great use to bring machine learning into the same playing field as currently used models. This would promote transparency and likely speed up model development while helping to avoid harmful bias. Such an approach is laid out here. The idea is to separate statistical inference into two steps. First, the contribution each variable makes to a model is measured. Second, these contributions are taken as the input to a standard regression analysis....MORE

How Google Aims To Dominate Artificial Intelligence

From: Popular Science:

The Search Giant Is Making Its AI Open Source So Anyone Can Use It

In November 2007, Google laid the groundwork to dominate the mobile market by releasing Android, an open ­source operating system for phones. Eight years later to the month, Android has an an 80 percent market share, and Google is using the same trick—this time with artificial intelligence.

Today Google is announcing TensorFlow, its open ­source platform for machine learning, giving anyone a computer and internet connection (and casual background in deep learning algorithms) access to one of the most powerful machine learning platforms ever created. More than 50 Google products have adopted TensorFlow to harness deep learning (machine learning using deep neural networks) as a tool, from identifying you and your friends in the Photos app to refining its core search engine. Google has become a machine learning company. Now they’re taking what makes their services special, and giving it to the world.

INTRODUCING TENSORFLOW, THE ANDROID OF AI
TensorFlow is a library of files that allows researchers and computer scientists to build systems that break down data, like photos or voice recordings, and have the computer make future decisions based on that information. This is the basis of machine learning: computers understanding data, and then using it to make decisions. When scaled to be very complex, machine learning is a stab at making computers smarter. That's the broader, and more ill-defined field of artificial intelligence. TensorFlow is extraordinary complex, because of its precision and speed in digesting and outputting data, and can unequivocally be placed in the realm of artificial intelligence tools.

Here are the nitty-gritty details: the TensorFlow system uses data flow graphs. In this system, data with multiple dimensions (values) are passed along from mathematical computation to mathematical computation. Those complex bits of data are called tensors. The math-y bits are called nodes, and the way the data changes from node to node tells the overall system relationships in the data. These tensors flow through the graph of nodes, and that's where the name TensorFlow comes from.

Open-­sourcing TensorFlow allows researchers and even grad students the opportunity to work with professionally-built software, sure, but the real effect is the potential to inform every machine learning company’s research across the board. Now organizations of all sizes—from small startups to huge companies on par with Google—can take the TensorFlow system, adapt it to their own needs, and use it to compete directly against Google itself. More than anything, the release gives the world’s largest internet company authority in artificial intelligence.

Stanford computer science professor Christopher Manning was given TensorFlow a little more than three months ago, and his students had the opportunity to tinker with the system. After just a few weeks of using it himself, Manning decided that he’s going to implement it into his curriculum.

Besides Android, he also likens the platform to Gmail, Google’s ubiquitous email application. There are competitors, but Gmail is cleaner and makes more sense in most applications.

“It’s not that before this there weren’t any high level libraries available for deep learning,” Manning says. “But in general these other libraries are things by three academics and a grad student.”

While the others, most notably Torch and Theano, do have small groups updating them, it’s nothing like the full force of the developers working on Google’s machine learning infrastructure. Manning says that while TensorFlow is a huge gift to the community (one capable of reducing time spent optimizing the neural networks by 100 times), they might indirectly benefit from open­-sourcing their tools.

"A very small amount of companies have been trying to hire up a very large percentage of the talented people in artificial intelligence in general, and deep learning in particular,” Manning says. “Google is not a charity, I’m sure it’s also occurred to them that by ceding this, we will have a lot of Ph.D students who will be in universities and already liking Google deep learning tools.”

Jeff Dean, one of Google’s top engineers and one of the two people who could be listed as an author for TensorFlow (the other is Rajat Monga), is cautious about estimating the adoption in the community. He says that while it’s something Google has found immensely useful in their own work, the real test is whether the community will find it as capable. The idea is to provide a tool so the whole community will be able to go from not just ideas, but actual implementations of things more rapidly...MORE.

How Amazon Rebuilt Itself Around Artificial Intelligence

From Wired, February 1:

Inside Amazon's Artificial Intelligence Flywheel
How deep learning came to power Alexa, Amazon Web Services, and nearly every other division of the company.

In early 2014, Srikanth Thirumalai met with Amazon CEO Jeff Bezos. Thirumalai, a computer scientist who’d left IBM in 2005 to head Amazon’s recommendations team, had come to propose a sweeping new plan for incorporating the latest advances in artificial intelligence into his division.

He arrived armed with a “six-pager.” Bezos had long ago decreed that products and services proposed to him must be limited to that length, and include a speculative press release describing the finished product, service, or initiative. Now Bezos was leaning on his deputies to transform the company into an AI powerhouse. Amazon’s product recommendations had been infused with AI since the company’s very early days, as had areas as disparate as its shipping schedules and the robots zipping around its warehouses. But in recent years, there has been a revolution in the field; machine learning has become much more effective, especially in a supercharged form known as deep learning. It has led to dramatic gains in computer vision, speech, and natural language processing.

In the early part of this decade, Amazon had yet to significantly tap these advances, but it recognized the need was urgent. This era’s most critical competition would be in AI—Google, Facebook, Apple, and Microsoft were betting their companies on it—and Amazon was falling behind. “We went out to every [team] leader, to basically say, ‘How can you use these techniques and embed them into your own businesses?’” says David Limp, Amazon’s VP of devices and services.

Thirumalai took that to heart, and came to Bezos for his annual planning meeting with ideas on how to be more aggressive in machine learning. But he felt it might be too risky to wholly rebuild the existing system, fine-tuned over 20 years, with machine-learning techniques that worked best in the unrelated domains of image and voice recognition. “No one had really applied deep learning to the recommendations problem and blown us away with amazingly better results,” he says. “So it required a leap of faith on our part.” Thirumalai wasn’t quite ready—but Bezos wanted more. So Thirumalai shared his edgier option of using deep learning to revamp the way recommendations worked. It would require skills that his team didn’t possess, tools that hadn’t been created, and algorithms that no one had thought of yet. Bezos loved it (though it isn’t clear whether he greeted it with his trademark hyena-esque laugh), so Thirumalai rewrote his press release and went to work.

Thirumalai was only one of a procession of company leaders who trekked to Bezos a few years ago with six-pagers in hand. The ideas they proposed involved completely different products with different sets of customers. But each essentially envisioned a variation of Thirumalai’s approach: transforming part of Amazon with advanced machine learning. Some of them involved rethinking current projects, like the company’s robotics efforts and its huge data-center business, Amazon Web Services (AWS). Others would create entirely new businesses, like a voice-based home appliance that would become the Echo.

The results have had an impact far beyond the individual projects. Thirumalai says that at the time of his meeting, Amazon’s AI talent was segregated into isolated pockets. “We would talk, we would have conversations, but we wouldn’t share a lot of artifacts with each other because the lessons were not easily or directly transferable,” he says. They were AI islands in a vast engineering ocean. The push to overhaul the company with machine learning changed that.

While each of those six-pagers hewed to Amazon’s religion of “single-threaded” teams—meaning that only one group “owns” the technology it uses—people started to collaborate across projects. In-house scientists took on hard problems and shared their solutions with other groups. Across the company, AI islands became connected. As Amazon's ambition for its AI projects grew, the complexity of its challenges became a magnet for top talent, especially those who wanted to see the immediate impact of their work. This compensated for Amazon's aversion to conducting pure research; the company culture demanded that innovations come solely in the context of serving its customers.

Amazon loves to use the word flywheel to describe how various parts of its massive business work as a single perpetual motion machine. It now has a powerful AI flywheel, where machine-learning innovations in one part of the company fuel the efforts of other teams, who in turn can build products or offer services to affect other groups, or even the company at large. Offering its machine-learning platforms to outsiders as a paid service makes the effort itself profitable—and in certain cases scoops up yet more data to level up the technology even more.

It took a lot of six-pagers to transform Amazon from a deep-learning wannabe into a formidable power. The results of this transformation can be seen throughout the company—including in a recommendations system that now runs on a totally new machine-learning infrastructure. Amazon is smarter in suggesting what you should read next, what items you should add to your shopping list, and what movie you might want to watch tonight. And this year Thirumalai started a new job, heading Amazon search, where he intends to use deep learning in every aspect of the service.

“If you asked me seven or eight years ago how big a force Amazon was in AI, I would have said, ‘They aren’t,’” says Pedro Domingos, a top computer science professor at the University of Washington. “But they have really come on aggressively. Now they are becoming a force.”

Maybe the force.

The Alexa Effect...

...MUCH MORE

Knowledge@Wharton on Machine Learning and Markets

They go with a much more ambitious headline than we'd prefer.

From K@W, October 13:

How to Beat Analysts and the Stock Market with Machine Learning

Analyst expectations of firms’ earnings are on average biased upwards, and that bias varies over time and stocks, according to new research by experts at Wharton and elsewhere. They have developed a machine-learning model to generate “a statistically optimal and unbiased benchmark” for earnings expectations, which is detailed in a new paper titled, “Man vs. Machine Learning: The Term Structure of Earnings Expectations and Conditional Biases.” According to the paper, the model has the potential to deliver profitable trading strategies: to buy low and sell high. When analyst expectations are too pessimistic, investors should buy the stock. When analyst expectations are excessively optimistic, investors can sell their holdings or short stocks as price declines are forecasted.

“[With the machine-learning model], we can predict how the prices of the stocks will behave based on whether or not the analyst forecast is too optimistic or too pessimistic,” said Wharton finance professor Jules H. van Binsbergen, who is one of the paper’s authors. His co-authors are Xiao Han, a doctoral student at the University of Edinburgh Business School; and Alejandro Lopez-Lira, a finance professor at the BI Norwegian Business School.

The researchers found that the biases of analysts increase “in the forecast horizon,” or in the period when the earnings announcement date is not anytime soon. However, on average, analysts revise their expectations downwards as the date of the earnings announcement approaches. “These revisions induce negative cross-sectional stock predictability,” the researchers write, explaining that “stocks with more optimistic expectations earn lower subsequent returns.” At the same time, corporate managers have more information about their own firms than investors have, and can use that informational advantage by issuing fresh stock, Binsbergen and his co-authors note.

The Opportunity to Profit

Comparing analysts’ earnings expectations with the benchmarks provided by the machine-learning algorithm reveals the degree of analysts’ biases, and the window of opportunity it opens. Binsbergen explained how investors could profit from their machine-learning model. “With our machine-learning model, we can measure the mistakes that the analysts are making by taking the difference between what they’re forecasting and what our machine-learning forecast estimates,” he said....

....MUCH MORE 

"The Fundamental Limits of Machine Learning"

From Nautil.us 

The Fundamental Limits of Machine Learning

Jesse Dunietz, a Ph.D. student in computer science at Carnegie Mellon University, has written for Motherboard and Scientific American Guest Blogs, among others. Follow him on Twitter @jdunietz.

Not long ago, my aunt sent her colleagues an email with the subject, “Math Problem! What is the answer?” It contained a deceptively simple puzzle:

She thought her solution was obvious. Her colleagues, though, were sure their solution was correct—and the two didn’t match. Was the problem with one of their answers, or with the puzzle itself? 

My aunt and her colleagues had stumbled across a fundamental problem in machine learning, the study of computers that learn. Almost all of the learning we expect our computers to do—and much of the learning we ourselves do—is about reducing information to underlying patterns, which can then be used to infer the unknown. Her puzzle was no different.

As a human, the challenge is to find any pattern at all. Of course, we have intuitions that limit our guesses. But computers have no such intuitions. From a computer’s standpoint, the difficulty in pattern recognition is one of surplus: with an endless variety of patterns, all technically valid, what makes one “right” and another “wrong?”

The problem only recently became of practical concern. Before the 1990s, AI systems rarely did much learning at all. For example, the chess-playing Deep Thought, predecessor to Deep Blue, didn’t get good at chess by learning from successes and failures. Instead, chess grandmasters and programming wizards carefully crafted rules to teach it which board positions were good or bad. Such extensive hand-tuning was typical of that era’s “expert systems” approach.

To tackle my aunt’s puzzle, the expert systems approach would need a human to squint at the first three rows and spot the following pattern:

1 * (4 + 1) = 5
2 * (5 + 1) = 12
3 * (6 + 1) = 21

The human could then instruct the computer to follow the pattern x * (y + 1) = z. Applying this rule to the final line yields a solution of 96.

Despite expert systems’ early success, the manual labor required to design, tune, and update them became unwieldy. Instead, researchers turned their attention to designing machines that could infer patterns on their own. A program could inspect, say, thousands of photos or market transactions and tease out statistical signals suggesting a face or an impending price spike. This approach quickly came to dominate, and has since powered everything from automated postal sorting to spam filtering to credit card fraud detection.

And yet. With all their successes, these machine learning systems still needed engineers in the loop. Consider again my aunt’s puzzle. We assumed that each line has three relevant components (the three numbers in the line). But there’s a potential fourth element: the result on the previous line. If that attribute of a line—that feature, in machine learning parlance—is in bounds, then another plausible pattern emerges:

0 + 1 + 4 = 5
5 + 2 + 5 = 12
12 + 3 + 6 = 21

By this logic the final answer should be 40.

So which pattern is right? Both, of course—and neither. It all depends on which patterns are allowed. You could also find a pattern by taking the first number times the second number, adding one-fifth of three more than the previous answer, and rounding to the nearest integer. (It’s weird, but it works!) And if we allow features that consider the visual forms of the numbers, perhaps we could come up with some pattern involving strokes and serifs. The pattern matching hinges on the assumptions of the beholder.

Will machines ever learn so well on their own that external guidance becomes a quaint relic?

The same is true in machine learning. Even when machines teach themselves, the preferred patterns are chosen by humans: Should facial recognition software infer explicit if/then rules, or should it treat each feature as an incremental piece of evidence for/against each possible person? And what features should the system attend to? Should it care about individual pixels? Maybe sharp edges between light and dark regions? These choices constrain what patterns the system deems likely or even possible. Finding that perfect combination has become the new job of the machine learning engineer.

 

LAYER CAKE: In neural networks, data is passed from layer to layer of “neurons,” undergoing simple transformations at each step. 
The intermediate layers can learn to recognize higher-level features that matter to the final output. 

...MUCH MORE

AI: When Deep Learning Meets the End Of Moore's Law (GOOG; NVDA; INTC; AMD)

Our old buddy Tiernan Ray, formerly Barron's Tech Trader Daily, going deeper than that format allowed.
From ZDNet:

Google says 'exponential' growth of AI is changing nature of compute
Google software engineer Cliff Young explains how the explosion in deep learning algorithms is coinciding with a breakdown in Moore's Law, the decades-old rule of thumb of progress in computer chips, forcing radical new computer designs.

The explosion of AI and machine learning is changing the very nature of computing, so says one of the biggest practitioners of AI, Google.

Google software engineer Cliff Young gave the opening keynote on Thursday morning at the Linley Group Fall Processor Conference, a popular computer-chip symposium put on by venerable semiconductor analysis firm The Linley Group, in Santa Clara, California.

Said Young, the use of AI has reached an "exponential phase" at the very same time that Moore's Law, the decades-old rule of thumb about semiconductor progress, has ground to a standstill.

"The times are slightly neurotic," he mused. "Digital CMOS is slowing down, we see that in Intel's woes in 10-nanometer [chip production], we see it in GlobalFoundries getting out of 7-nanometer, at the same time that there is this deep learning thing happening, there is economic demand." CMOS, or complementary metal-oxide semiconductor, is the most common material for computer chips.

Also: Google preps TPU 3.0 for AI, machine learning, model training

As conventional chips struggle to achieve greater performance and efficiency, demand from AI researchers is surging, noted Young. He rattled off some stats: The number of academic papers about machine learning listed on the arXiv pre-print server maintained by Cornell University concerning is doubling every 18 months. And the number of internal projects focused on AI at Google, he said, is also doubling every 18 months. Even more intense, the number of floating-point arithmetic operations needed to carry out machine learning neural networks is doubling every three and a half months.

All that growth in computing demand is adding up to a "Super Moore's Law," said Young, a phenomenon he called "a bit terrifying," and "a little dangerous," and "something to worry about."

"Why all this exponential growth?" in AI, he asked. "In part, because deep learning just works," he said. "For a long time, I spent my career ignoring machine learning," said Young. "It wasn't clear these things were going to take off."

But then breakthroughs in things such as image recognition started to come quickly, and it became clear deep learning is "incredibly effective," he said. "We have been an AI-first company for most of the last five years," he said, "we rebuilt most of our businesses on it," from search to ads and many more.

The demand from the Google Brain team that leads research on AI is for "gigantic machines" said Young. For example, neural networks are sometimes measured by the number of "weights" they employ, variables that are applied to the neural network to shape its manipulation of data.

Whereas conventional neural nets may have hundreds of thousand of such weights that must be computed, or even millions, Google's scientists are saying "please give us a tera-weight machine," computers capable of computing a trillion weights. That's because "each time you double the size of the [neural] network, we get an improvement in accuracy." Bigger and bigger is the rule in AI.

To respond, of course, Google has been developing its own line of machine learning chips, the "Tensor Processing Unit." The TPU, and parts like it, are needed because traditional CPUs and graphics chips (GPUs) can't keep up....

...MUCH MORE

Google Launches the Quantum Artificial Intelligence Lab (GOOG)

From the Google Research blog:

Posted by Hartmut Neven, Director of Engineering

We believe quantum computing may help solve some of the most challenging computer science problems, particularly in machine learning. Machine learning is all about building better models of the world to make more accurate predictions. If we want to cure diseases, we need better models of how they develop. If we want to create effective environmental policies, we need better models of what’s happening to our climate. And if we want to build a more useful search engine, we need to better understand spoken questions and what’s on the web so you get the best answer.

So today we’re launching the Quantum Artificial Intelligence Lab. NASA’s Ames Research Center will host the lab, which will house a quantum computer from D-Wave Systems, and the USRA (Universities Space Research Association) will invite researchers from around the world to share time on it. Our goal: to study how quantum computing might advance machine learning.

Machine learning is highly difficult. It’s what mathematicians call an “NP-hard” problem. That’s because building a good model is really a creative act. As an analogy, consider what it takes to architect a house. You’re balancing lots of constraints -- budget, usage requirements, space limitations, etc. -- but still trying to create the most beautiful house you can. A creative architect will find a great solution. Mathematically speaking the architect is solving an optimization problem and creativity can be thought of as the ability to come up with a good solution given an objective and constraints...MORE 

HT: Steve Jurvetson's flickr:

In the D-Wave Board meeting in Canada this morning. Today's news: Google buys a quantum computer for machine learning and artificial intelligence: "We actually think quantum machine learning may provide the most creative problem-solving process under the known laws of physics. " — Google Blog

This is an interesting development in a larger trend I call Deus Ex Machina — machine learning innervates everything.

Under the covers, just about every new research initiative at Google is driven by machine learning — whereby the machine learns patterns in the data without explicit models or traditional solution design. It’s what makes “Big Data” BIG this time around. The approach requires a humble relaxation of the presumption of control, and so it starts with companies like Google and eventually revolutionizes all businesses, even those with a delusion of control, like Investment bankers. =)

As a precondition to purchase, Google gave the company a number of performance benchmarks to prove that the quantum computer is faster than anything Google has in house. The NYT reports:
“For most problems, it was 11,000 times faster, but in the more difficult 50 percent, it was 33,000 times faster. In the top 25 percent, it was 50,000 times faster.”...MORE 

Ultimate HT: Next Big Future who we'll be returning to this afternoon.

Previously on D-Wave:
many links:
Does Lockheed Have a Quantum Computer or Doesn't It? (LK)
"The CIA and Jeff Bezos Bet on Quantum Computing" (AMZN)
Quantum Computing: CIA and Bezos Invest in D-Wave Systems Inc.

And on the Quantum:
"Does probability come from quantum physics?"
 "Hacking the Quantum: A New Book Explains How Anyone Can Become an Amateur Quantum Physicist"
 Quantum Physicists Disagree on the Nature of Reality
 Multiple Steps Toward the 'Quantum Singularity'

Luncheon With The Economists

Our little meal consists of only two courses and you have to grab your own apéritif but I think you might like it.

L'Entrée

Montgolfières à la Lars P. Syll

A balloonist, lost, sees someone walking down a country lane. The balloonist lowers the balloon and shouts down to the the walker: 

— Where am I? 

— About 20 feet above the ground, comes the reply. 

After a moment’s pondering, the balloonist says: 

— You must be an economist. 

— How did you know? 

— Your information is perfectly correct — and totally useless.

Le Plat Principal

Critique Économique par Noah Smith

Economics Has a Math Problem
A lot of people complain about the math in economics. Economists tend to quietly dismiss such complaints as the sour-grapes protests of literary types who lack the talent or training to hack their way through systems of equations. But it isn't just the mathematically illiterate who grouse. New York University economist Paul Romer -- hardly a lightweight when it comes to equations -- recently complained about how economists use math as a tool of rhetoric instead of a tool to understand the world.  

Personally, I think that what’s odd about econ isn’t that it uses lots of math -- it’s the way it uses math. In most applied math disciplines -- computational biology, fluid dynamics, quantitative finance -- mathematical theories are always tied to the evidence. If a theory hasn’t been tested, it’s treated as pure conjecture.  

Not so in econ. Traditionally, economists have put the facts in a subordinate role and theory in the driver’s seat. Plausible-sounding theories are believed to be true unless proven false, while empirical facts are often dismissed if they don’t make sense in the context of leading theories. This isn’t a problem with math -- it was just as true back when economics theories were written out in long literary volumes. Econ developed as a form of philosophy and then added math later, becoming basically a form of mathematical philosophy.  

In other words, econ is now a rogue branch of applied math. Developed without access to good data, it evolved different scientific values and conventions. But this is changing fast, as information technology and the computer revolution have furnished economists with mountains of data. As a result, empirical analysis is coming to dominate econ.  

One sign of this is the sudden burst of interest in machine learning in the economics field. Machine learning is a broad term for a collection of statistical data analysis techniques that identify key features of the data without committing to a theory. To use an old adage, machine learning “lets the data speak.” In the age of Big Data, machine learning is a hot field in the technology business, and is a key tool of the rapidly expanding field of data science. Now, econ is catching the bug.  

Two economists who have been pushing for the adoption of machine learning techniques in economics are Susan Athey and Guido Imbens of Stanford University. The two economists explained machine learning techniques to an interested crowd at a recent meeting of the National Bureau of Economic Research. Their overview stated that machine learning techniques emphasized causality less than traditional economic statistical techniques, or what's usually known as econometrics. In other words, machine learning is more about forecasting than about understanding the effects of policy.  

That would make the techniques less interesting to many economists, who are usually more concerned about giving policy recommendations than in making forecasts....MORE

Bravo!

Where Are The Profits Of Big Data Flowing?

From TechRepublic, March 17:

Why machine learning benefits the rich, and everyone else is toast  
Big data started with cheap hardware and open source software, but the winners in machine learning are the world's richest companies. 

Why is machine learning finally real? It's the data, stupid. Lots (and lots) of data.

That's a key message from Cloudera co-founder Mike Olson's Strata + Hadoop World keynote earlier this week in San Jose, California. As he declared: "The algorithms that early researchers and current practitioners use are ravenous for data and we finally have enough data on the planet to feed them. They also need scale-out computation and storage at low cost."

In fact, the mountains of data that we now enjoy are a direct result of high-quality open source software running on commodity hardware: More applications churning out more data for more people.

A game only the rich can play
Despite this low-cost hardware and software, and its impact on machine learning, let's be clear: Big enterprises are the primary beneficiaries. Why? As Olson went on to explain, among enterprises doing over $1 billion a year in revenue—Cloudera's target customer—"the appetite for these [machine learning] capabilities is insatiable" as they "absolutely have the data at scale."

Data, after all, is necessary to train the machines. A small company could have big plans but without big data to feed those plans, it's a losing battle. As such, large enterprises are in a prime position to use big data to enrich themselves and effectively hold off would-be, smaller competitors.

(As a side note, as useful as open source has been, we really need to have open data sets. Stanford has been exemplary in this, annotating data to make it more readily useful for machine learning. This is a new frontier in "open source," and we need to explore it more.)

Sparking ML
One thing that aids these big companies has come from an egalitarian source: Apache Spark. I've written about Spark's impact on multiple occasions, but it's easy to understate just how important it has been. Indeed, though Cloudera recognized the importance of Apache Spark early on, Olson noted in his keynote, one aspect of it has "taken them by surprise."

[Spark] allowed people to build and deploy scale-out machine learning applications much faster than they had previously done. [Why?] Its flexibility and ease of programming meant that you could build machine learning apps, train up models on massive data very, very quickly. That has led to huge interest in the ecosystem....

...MORE

Coming at the issue on a related tangent is The Verge, July 13:

Robots and AI are going to make social inequality even worse, says new report

Most economists agree that advances in robotics and AI over the next few decades are likely to lead to significant job losses. But what’s less often considered is how these changes could also impact social mobility. A new report from UK charity Sutton Trust explains the danger, noting that unless governments take action, the next wave of automation will dramatically increase inequality within societies, further entrenching the divide between rich and poor. 

The are a number of reasons for this, say the report’s authors, including the ability of richer individuals to re-train for new jobs; the rising importance of “soft skills” like communication and confidence; and the reduction in the number of jobs used as “stepping stones” into professional industries.

For example, the demand for paralegals and similar professions is likely to be reduced over the coming years as artificial intelligence is trained to handle more administrative tasks. In the UK more than 350,000 paralegals, payroll managers, and bookkeepers could lose their jobs if automated systems can do the same work. 

“Traditionally, jobs like these have been a vehicle for social mobility,” Sutton Trust research manager Carl Cullinane tells The Verge. Cullinane says that for individuals who weren’t able to attend university or get particular qualifications, semi-administrative jobs are often a way in to professional industries. “But because they don’t require more advanced skills they’re likely to be vulnerable to automation,” he says.

Similarly, as automation reduces the need for administrative skills, other attributes will become more sought after in the workplace. These include so-called “soft skills” like confidence, motivation, communication, and resilience. “It’s long established that private schools put a lot of effort into making sure their pupils have those sorts of skills,” says Cullinane. “And these will become even more important in a crowded labor market.”....

...MORE

There are a couple other aspects of the AI money flows that are becoming apparent, we'll be back with those next week.

Sanofi Is Looking For A "Computational Scientist - Digital R&D Large Molecule Research Team"

This stuff is happening right now. The melding of supercomputers and health sciences gives us the digital biology we've been babbling about. 

From Sanofi:

date posted 02/21/2024

contract type Full time

job id R2713383

location Cambridge, Massachusetts

Reference No. R2713383

Position Title:Computational Scientist – (Machine Learning) Digital R&D Large Molecule Research Team 

Department:Data Strategy Program Management

Location: Cambridge, MA

About Sanofi:

We are an innovative global healthcare company, driven by one purpose: chasing the miracles of science to improve people’s lives. Our team, across some 100 countries, is dedicated to transforming the practice of medicine by working to turn the impossible into the possible. We provide potentially life-changing treatment options and life-saving vaccine protection to millions of people globally, while putting sustainability and social responsibility at the center of our ambitions.

Sanofi has recently embarked on a vast and ambitious digital transformation program. A cornerstone of this roadmap is the acceleration of its data transformation and of the adoption of artificial intelligence (AI) and machine learning (ML) solutions to accelerate R&D, manufacturing and commercial performance, and bring better drugs and vaccines to patients faster, to improve health and save lives.

In alignment to our digital transformation, we have launched a new major strategic initiative in 2023: the Biologics x AI Transformation. This is positioned to be a unique data-driven team, with expertise in AI platforms, data engineering, ML operations, data science, computational biology, strategy, and beyond. We are working as one to identify, design, and scale state-of-the-art AI capabilities targeted to truly transform how we research biologics.

Who You Are:

You are a dynamic Computational Scientist who will work with other scientists to apply cutting-edge computation, Machine Learning/Deep Learning approaches to revolutionize our large molecule computational tools by contributing to accelerating and improving the process of design and engineering of novel biologics drug candidates.

Job Highlights: 

• Apply and develop artificial intelligence and machine learning (AI/ML) approaches (e.g. classification, clustering, machine learning, deep learning) on pharma research data sets (e.g. activity, function, ADME properties, physico-chemical properties, etc.)

• Building models from internal and external data sources, algorithms, simulations, and performance evaluation by writing code and using state-of-the art machine learning technologies.

• Close interactions with other Computational scientists, data engineers, software engineers, UX designers, as well as research scientists in core scientific platforms focusing on protein therapeutics, in an international context (US, Europe, China)

• Update and report relevant results to interdisciplinary project teams and stakeholders

Key Functional Requirements & Qualifications:

• Advanced degree (e.g. M.Sc., PhD) in a field related to AI/ML or Data Analytics such as: Computer Science, Mathematics, Statistics, Physics, Biophysics, Computational Biology or Engineering Sciences.

• Ideally 1+ years of industry experience, new grads will also be considered. Should have a track record of applying ML/Deep Learning (DL) approaches to solve molecule-related problems. 

• Familiarity with protein structure or sequence featurization/embeddings.

• Familiarity with advanced statistics, ML/DL techniques including various network architectures (CNNs, GANs, RNNs, Auto-Encoders, Transformers, PLM etc.), regularization, embeddings, loss-functions, optimization strategies, or reinforcement learning techniques.

• Proficiency in Python and deep learning libraries such as PyTorch, TensorFlow, Keras, Scikit-learn, Numpy, Matpilotlib.

• Familiarity with data visualization and dimensionality reduction algorithms

• Ability to develop, benchmark and apply predictive algorithms to generate hypotheses

• Comfortable working in cloud and high-performance computational environments (e.g. AWS)

• Excellent written and verbal communication, strong tropism for teamwork

• Strong understanding of pharma R&D process is a plus.