Law in the Internet Society

Your profile defines your future.

Spying used to take a great deal of effort. Wiretapping someone’s home normally required the coordinated work of a team of seasoned professionals and intercepting communications was a task that used to take months. However, the emergence of individual connected devices placed spies in our desks, pockets and wrists, 24/7. Our entire lives are now visible and on display.

The "smart" objects were supposed to make our lives easier, but they were rather designed to collect our behavior patterns. Free digital services are like free dinners, if you’re not on the table, you’re on the menu. Our travels, purchases, internet searches, our heart pace are examples of the data collected by our new tech-toys/behavioral trackers.

At the data collection stage, our behavioral data is a huge, incoherent, decontextualized mass of data. After collection, data must be processed. Algorithms will compute all the collected raw data and recognize behavioral patterns. Behavioral patterns are built on the basis of how you and other people having similar profiles behaved. Your behavioral patterns, all put together constitute your profile.

There are multiple applications of profiling: marketing, surveillance, city management, security, etc. They all learn about, predict and influence our behaviors. On the basis of an individual’s profile, continuously evolving over time as data comes in, his/her behaviors can be predicted, triggered, guided through incentives, personalized recommendations, suggestions, warnings or other stimuli. Such guiding has at least three undesirable consequences: (i) a new non-democratic normativity regime; (ii) a new conception of the human being; (iii) locking people into their profile.

New non-democratic normativity regime

As profiling is capable to influence people’s behaviors, it has normative power. Contrary to classic forms of norms, produced by law, politics and social control, profiling normativity does not ban or constraint, but rather makes the disobedience the norm unlikely. As pointed by researcher A. Rouvroy, it is a “relatively subliminal mode of government that consists of directing people’s attention towards certain things, in modifying the informational or physical environment so that behaviors are no longer obligatory but necessary.” (see here)

This new form of normativity is not conducted in the name of certain shared values, a philosophy or an ideology. It is the death of idealistic/value-based politics. Politics is all about transforming the state of affairs by means of ideas, projections, and imagination. The world of profiling is not aiming at transforming things by means of communist, capitalist, or any ideologically supported regime. At most, behavioral predictability could claim to govern society objectively and efficiently with the sole aim of optimizing social life as much as possible, without bothering to know whether the norm is fair, equitable or legitimate. In fact, it is a non-democratic regime of optimization of the existing state of affairs for the benefit certain actors, be it private actors serving their private monetary interests (as in Western liberal democracies) public actors serving social order (such as in China).

The profiled human

The second issue caused by data processing and profiling is of philosophical nature. The philosophy of the Enlightenment envisages the modern human as a free individual, responsible, endowed with reason and free will.

The “profiled human” shares a common feature with the free individual conceived by the philosophy of the Enlightenment. They share a logic of individualization. Insofar it allows the environment to adapt to each profile in all its singularity (for example: individualization of advertising), the “profiled human” is as individualistic as the “free human”. However, the “profiled human” is far from the notion of the modern man as conceived by the philosophy of the Enlightenment. He is surveilled. His behaviors are tracked and influenced.

Eric Schmidt, CEO of Google from 2001 to 2011 said: “I actually think most people don't want Google to answer their questions. They want Google to tell them what they should be doing next. (...) The technology will be so good it will be very hard for people to watch or consume something that has not in some sense been tailored for them”. This is not a new debate. Replace Google by God, and this sentence could have been written in the 4th century, when Saint-Augustine questioned individuals’ freedom, predestination and God’s foreknowledge. Later, Walter Benjamin studied the impact of propaganda and persuasion methods such as television. The novelty today is the automaticity, magnitude, and surveillance dimension of the phenomenon.

The profile prison

The third regrettable consequence of profiling is the reproduction of class systems. The “free human” should be able to become the person he/she freely chooses to become. Liberal democracies traditionally strive at giving each individual the tools necessary to achieve his/her potential and to emancipate. Individuals should be free to practice the sport they like, listen and play music they like, be interested in the languages and culture they admire, etc. Such life choices should ideally be made freely.

The “profiled human” does not freely make these choices. The “profiled human” is only a reflection of the predictable. The profiled human will be suggested to behave in the same way as he did in the past and in the same way as people with similar profiles have behaved previously.

If you are a white male studying law at Columbia, it means you are also likely to vote Democrats, travel to Europe, be interested in wine, play tennis, and listen to rock music. If you were born in Harlem and you are a black unemployed male, you will also be likely to vote Democrats, but you will rather be likely not to travel, be interested in drinking beers or sodas rather than wine, play soccer or video games rather than tennis, listen to hip-hop instead of rock music. And this is what will be suggested to them as well as to their alike friends. The data locks people into their profiles. It makes people become what you were likely to become and, in this sense, prevents individuals to freely realize themselves and reproduces existing social patterns. The possible was reduced to the probable.

But this presentation isn't accurate. There are more kinds of music than two, and "likely" is not an available judgment. It's true that describing a model based on four orders of magnitude more points requires two sentences, but they could be spared. "Locking-in" would better be described, perhaps, in terms of the goal of the Chinese Social Credit System. That would make the point clearly and allow your preceding analysis to be terser and more direct.


Webs Webs

r7 - 18 Apr 2022 - 16:18:40 - EbenMoglen
This site is powered by the TWiki collaboration platform.
All material on this collaboration platform is the property of the contributing authors.
All material marked as authored by Eben Moglen is available under the license terms CC-BY-SA version 4.
Syndicate this site RSSATOM