Finally got around to reading this piece from The New Atlantis. It’s a disconcerting read, especially if, like me, you’re also reading 1984 with a group of high school students. Needless to say, I’m a little concerned about the parallels between what Jon Askonas identifies as the unintended but unavoidable consequences of Big Tech/Social Media, and Orwell’s description of Oceania’s Ministry of Truth.
We can see the shift from “access to tools” to algorithmic utopianism in the unheralded, inexorable replacement of the “page” by the “feed.” The web in its earliest days was “surfed.” Users actively explored what was interesting to them, shifting from page to page via links and URLs. While certain homepages — such as AOL or Yahoo! — were important, they were curated by actual people and communities. Most devoted “webizens” spent comparatively little time on them, instead exploring the web based on memory, bookmarks, and interests. Each blog, news source, store, and forum had its own site. Where life on the Internet didn’t follow traditional editorial curation, it was mostly a do-it-yourself affair: Creating tools that might show you what your friends were up to, gathering all the information you cared about in one place, or finding new sites were rudimentary and tedious activities.
The feed was the solution to the tedium of surfing the web, of always having to decide for yourself what to do next. Information would now come to you. Gradually, the number of sites involved in one’s life online dwindled, and the “platform” emerged, characterized by an infinite display of relevant information — the feed. The first feeds used fairly simple algorithms, but the algorithms have grown vastly more complex and personalized over time. These satisfaction-fulfillment machines are designed to bring you the most “relevant” content, where relevancy is ultimately based on an elaborate and opaque model of who you are and what you want. But the opacity of these models, indeed the very personalization of them, means that a strong element of faith is required. By consuming what the algorithm says I want, I trust the algorithm to make me ever more who it thinks I already am.
In this process, users have gone from active surfers to sheep feeding at the algorithmic trough. Over time, platforms have come up with ever more sophisticated means of inducing behavior, both online and in real life, using AI-fueled notifications, messages, and default choices to nudge you in the right direction, ostensibly toward your own maximum satisfaction. Yet now, in order to rein in the bad behaviors the feeds themselves have encouraged — fake news, trolling, and so on — these algorithms have increasingly become the sites of stealthy intervention, using tweaks like “shadowbanning,” “down-ranking,” and simple erasure or blocking of users to help determine what information people do and don’t access, and thereby to subtly shape their minds.
While the Mark Zuckerbergs and Sergey Brins of the world claim to be shocked by the “abuse” of their platforms, the softly progressive ambitions of Silicon Valley and the more expansive visions of would-be dictators exist on the same spectrum of invasiveness and manipulation. There’s a sense in which the authoritarians have a better idea of what this technology is for.