Did George Orwell foresee ChatGPT, writing in… 1948? Also, how it opens new frontiers in censorship and propaganda

Judge these passages from “Nineteen Eighty-Four” for yourselves:

“And the Records Department, after all, was itself only a single branch of the Ministry of Truth, whose primary job was not to reconstruct the past but to supply the citizens of Oceania with newspapers, films, textbooks, telescreen programs, plays, novels—with every conceivable kind of information, instruction, or entertainment, from a statue to a slogan, from a lyric poem to a biological treatise, and from a child’s spelling book to a Newspeak dictionary. And the Ministry had not only to supply the multifarious needs of the Party, but also to repeat the whole operation at a lower level for the benefit of the proletariat. There was a whole chain of separate departments dealing with proletarian literature, music, drama, and entertainment generally. Here were produced rubbishy newspapers containing almost nothing except sport, crime, and astrology, sensational five-cent novelettes, films oozing with sex, and sentimental songs which were composed entirely by mechanical means on a special kind of kaleidoscope known as a versificator. There was even a whole sub-section—Pornosec, it was called in Newspeak—engaged in producing the lowest kind of pornography, which was sent out in sealed packets and which no Party member, other than those who worked on it, was permitted to look at.”

George Orwell, “Animal Farm and 1984” (pp. 127-128). HMH Books. Kindle Edition.

Julia was twenty-six years old. She lived in a hostel with thirty other girls (“Always in the stink of women! How I hate women!” she said parenthetically), and she worked, as he had guessed, on the novel-writing machines in the Fiction Department. She enjoyed her work, which consisted chiefly in running and servicing a powerful but tricky electric motor. She was “not clever,” but was fond of using her hands and felt at home with machinery. She could describe the whole process of composing a novel, from the general directive issued by the Planning Committee down to the final touching-up by the Rewrite Squad. But she was not interested in the finished product. She “didn’t much care for reading,” she said. Books were just a commodity that had to be produced, like jam or bootlaces. […]

[Julia] had always borne an excellent character. She had even (an infallible mark of good reputation) been picked out to work in Pornosec, the sub-section of the Fiction Department which turned out cheap pornography for distribution among the proles. It was nicknamed Muck House by the people who worked in it, she remarked. There she had remained for a year, helping to produce booklets in sealed packets with titles like Spanking Stories or One Night in a Girls’ School, to be bought furtively by proletarian youths who were under the impression that they were buying something illegal. “What are these books like?” said Winston curiously. “Oh, ghastly rubbish. They’re boring, really. They only have six plots, but they swap them round a bit. Of course I was only on the kaleidoscopes. I was never in the Rewrite Squad. I’m not literary, dear—not even enough for that.”

George Orwell, “Animal Farm and 1984” (pp. 209-210). HMH Books. Kindle Edition.

And if you think the “Versificator”/ChatGPT 3 will not be “coopted” by the brahmandarinate, think again:

NEW FRONTIERS IN CENSORSHIP AND PROPAGANDA: ChatGPT by @OpenAI now *expressively prohibits arguments for fossil fuels*. (It used to offer them.) Not only that, it excludes nuclear energy from its counter-suggestions.

Also via Insty: https://www.cbsnews.com/news/chatgpt-chat-ai-jobs-most-likely-to-be-replaced/

Sarah Hoyt believes her job is safe though, for reasons she explains here: “we can write it for you wholesale“. (For those unfamiliar with classic sci-fi, the title is a pun on the Philip K. Dick short story “We can remember it for you wholesale“, better known as the movie “Total Recall”. That title, Mrs. Arbel is quick to remind me, itself paid tribute to the musical that put a 19-year old Barbra Streisand on the map.)

3 thoughts on “Did George Orwell foresee ChatGPT, writing in… 1948? Also, how it opens new frontiers in censorship and propaganda

  1. Orwell was not alone although arguably the most credible in his thoughts on what we call AI. Science-fiction writers have also thought a lot about the implications of artificial intelligence.

    For example, the Czech Carel Kapek wrote a play, “Rossum’s Universal Robots” in 1920. Its premise is that “robots” (i.e., androids) that were at first happy to work for humans, ultimately revolted and caused the extinction of the human race.

    Another of my favorites is Jack Williamson’s “With Folded Hands (1948). Its premise is that a scientist discovered a new force that enabled incredibly advanced humanoids. They were designed to serve men and save them from harm. Instead the humanoids took control of humanity and prevented them from doing anything fun or worthwhile because it might cause “harm”.

    My view of AI is that it is a real-life Frankenstein, a conceit that humans might create life. AI presents a huge risk for the future of humanity – highly seductive, but at the same time dangerous to touch.

  2. I saw a clever meme today based on the scene from 2001 by Kubrick.
    Human: Say something nice about Trump.
    AI: I’m sorry, Dave. I’m afraid I can’t do that.

Leave a comment