Monday, April 22, 2019

"Reputational currency, like China's Social Credit Score, rebrands repression as rational nudging. And these algorithmic governance models are spreading"

Professor Pasquale at The Boston Review:
The most successful Internet companies, it seems, all learned the lesson of Tom Sawyer. By outsourcing labor to their users, it is as if they have cordially invited friends and neighbors to delight in painting their fence. Instagram does not need to hire photographers; users snap, post, and comment on photos, and their hashtags are a filing system, a taxonomy as meticulously curated as a filing cabinet. Instagram’s parent company, Facebook, relies on all of us to keep each other amused, as does the Chinese platform Weibo. Even Google, Amazon, and Alibaba, operating far afield from social media, rely on combined consumers and producers (“prosumers”) to let them know when their algorithms are helpful. 
Every society depends on free labor—work that is vital but which goes unpaid. 
They also allow customers to police the quality of products and websites so that they don’t have to pay someone to do that work. Airbnb, Uber, and eBay apply similar methods, shifting the burden of quality control by having both sides in commercial transactions rate one another. We all give up small treasures—our data—to paint the fence of platform capitalists.

We may soon do the same for government. Every society depends on free labor—work that is vital but which goes unpaid. Smart governments realize that they need to strike some balance between market activity and the free labor that supports families and communities. Policymakers promote business and growth, but they also realize that if every moment were commodified, the foundations of social reproduction would wither away. Index funds may prove a better investment than children. And if you don’t get credit for being civil, paying attention in class, or taking care of your aging parents, why would you?

There are standard solutions to such problems. Courts can drain the bank accounts of “deadbeat dads.” Churches and civil society groups can stigmatize deviants, and the carceral state can further scare scofflaws. But these approaches take resources. The perfectly efficient neoliberal state would cut out the middleman. It would learn from Silicon Valley that you can motivate people not only to rate and rank one another, but also to positively enjoy the power and responsibility that rating (and being rated) entails.

The Chinese government is now implementing just such a system at the national level. Called the Chinese Social Credit System (SCS), it has some familiar foundations. Its early iterations (pioneered by private firms) allow users to share images of their scores with one another. As with financial credit scores used by many lenders, the system rewards people for repaying debts promptly. But the SCS does not stop with credit; it factors in court judgments, criminal records, academic dishonesty, jaywalking, moving violations, and failing to pay transit fares.
The Chinese Social Credit System factors in court judgments, 
criminal records, academic dishonesty, jaywalking, and more.
Surveillance, software, and relatively simple artificial intelligence can supply a fearsomely panoptic dossier. But this monitoring alone does not address the concern of Chinese Communist Party authorities that cornerstones of their authority are eroding. Thus the SCS will also dent your score for posting “unreliable” information or engaging in nebulously defined negative interactions online. Conversely, the system will reward volunteer activity and “filial piety”—devotion to one’s parents, grandparents, and perhaps other relatives. To paraphrase Margaret Thatcher, scoring is “the method; the object is to change the heart and soul.”

How can a government judge the relative value of working in the market versus visiting a lonely aunt? For the architects of the SCS, these spheres diverge: cash rules commerce, and a new currency will govern culture. That currency is reputation, a single score to express a person’s social value. As China’s SCS approaches its full implementation around 2020, the scoring of activities will spread, assigning points for a wider range of antisocial and social behaviors. Eventually China may make a Great Leap to Commensuration, in which every activity (or inactivity) is judged and converted to points, giving lived reality the feel of a never-ending video game.

The Chinese government claims that the SCS simply reflects the values now embodied in Chinese families, schools, and courts. But with no appeal mechanism—a basic aspect of due process in any scored society—the SCS’s relentless logic of commensuration threatens to supplant, rather than supplement, the authority of families, schools, and courts. The SCS could easily end up serving as a quant-driven power grab, enabling its authors to assert authority over vast swathes of social life in a way they could never achieve via legislation. Such quantitative governance of culture is a paradox: the very effort to articulate the precise value of manners and emotions threatens to unravel them entirely, as spontaneous affections and interactions are instrumentalized into points....