- Intro
- What happens when AI models become really good at generating charts and diagrams, and we automate their production at scale?
- What Caught My Attention This Week
- Harvard Business School professor on how AI could impact the workforce.
- Two novel approaches in using AI in conjunction with brain-computer interfaces to give a faster voice to patients.
- Vinod Khosla, one of the most famous and successful venture capitalists in the world, on how much of the existing jobs will be taken over by AI.
- The International Labour Organization (ILO) offers a positive outlook on the impact of AI on jobs.
- How Do You Feel?
- Companies are increasingly concerned about the reputational damage associated with announcing the use of AI to augment or replace the workforce.
- Putting Lipstick on a Pig
- Scientific journals are getting inundated with research written with ChatGPT.
In the last two weeks, I worked day and night to research and learn how to use an uber-complicated, maximum-friction tool called ComfyUI.
Why?
ComfyUI is a node-based visual programming system to generate images with Stable Diffusion.
Just like Blender and Houdini in 3D design, DaVinci Resolve in video editing, Unreal and Unity in game development, or REAKTOR in music production, ComfyUI is a tool meant for professionals that need to create very complex pipelines to generate sophisticated images and, soon, videos, that cannot be created with other three-clicks tools like Midjourney or Dream Studio or even Photoshop.
Think about these pipelines as multiple branches of sequential steps akin to all the acrobatics that a Michelin star chef has to perform to create a dish.
A basic sequence of steps as captured in a recipe can take you only so far. The more elaborated the dish, the more complicated the steps, often to be performed at the same time.
Let’s leave this analogy and go back to ComfyUI.
Crucially, once you have perfected your workflow, these node-based editors allow you to automate its execution at scale with a single click. No human involved.
I shared one of these workflows in Issue #26 – How to break the news, literally, after I spent an inordinate amount of time to create it, expand it, perfect it, and maintain it when something broke.
The images I can generate with it were unthinkable just a few months ago:
But why did I bother to learn how to use this tool?
People that have worked with me in the last two decades can testify that a tool like this goes against everything I believe in and advocate for, in terms of user experience and user interface design.
So? What justifies so much time dedicated to its research?
Well. When people see these ComfyUI workflows, they think about two main things:
- This thing is helpful to artists that have to create very complicated images with a lot of passes
- This thing can automate the application of the same settings to every frame of a video
I think about something entirely different:
What happens when these diffusion models become really good at generating charts and diagrams, with comprehensible text, and a node system like ComfyUI can automate their production at scale?
In Issue #26 – How to break the news, literally, I showed you how I automated the creation of the news Synthetic Work breaking news section, including the cover image that accompanies each article.
That, alone, should be a powerful demonstration of how big the threat is not just for the ones who work in the publishing industry, but also for all of us reading what’s being published.
With the building blocks I have shown you, both well-intentioned and malicious actors can set up content farms that regurgitate the same information over and over again, until it’s completely devoided of meaning, at a scale that no content moderation engine will be able to handle.
But that’s child’s play.
The minute diffusion models like Stable Diffusion become really good at generating charts and diagrams, and I assure you it’s coming, I could use ComfyUI to generate hundreds of thousands of them a day, starting from real datasets or synthetic ones, generated with large language models like GPT-4.
And once I have that in place, I could generate an infinite number of research documents simply asking my AI:
“Generate 10 marketing documents validating our product’s value proposition.”
Or:
“Generate 50 market consumer sentiment surveys validating our investment thesis.”
Or, more importantly:
“Generate 100 research documents supporting the idea that the Earth is flat.”
In all three cases, all you have to do is disseminate these documents on the web, letting people do what they usually do:
- Take the numbers at face value
- Embrace your idea once they have seen the same report coming from three or more different sources
I believe we’ll get to a point where only two types of professionals will matter, at least for a bit: who’s good at marketing, and who’s good at controlling the distribution of information.
Marketing, to quote Seth Godin, is creating the conditions for the network effect.
The distribution of information is the enablement of that network effect.
The rest will be done by AI.
And this, if I’m right, will have an impact both on our jobs and on our society.
I leave you with this quote from one of the most important philosophers and cognitive psychologists of our times, Daniel Dennett, from a New York Times interview:
the great danger of GPT-3 and ChatGPTs and so forth is that they can reproduce. They’re memes. You don’t have to be alive to evolve. Viruses aren’t alive; boy, do they evolve. Things evolve because they can, and cultural evolution — memetic evolution — is a potent phenomenon. We don’t want to have censorship, but we want to have something like quarantine to prevent the spread of cultural variants that could destroy culture, destroy democracy.
The economist Paul Seabright writes movingly about trust, and trust is a social phenomenon. Society depends on trust. Trust is now seriously endangered by the replicative power of A.I. and phony interactions.
This is a grave danger.
There’s a natural human tendency to think, If I can do it, I will do it, and not worry about whether I ought to. The A.I. community has altogether too many people who just see the potentiality and aren’t willing to think about risks and responsibility. I would like to throw a pail of cold water on their heads and say, “Wait a minute, it’s not cool to make easily copied devices that will manipulate people in ways that will destroy their trust.”
Alessandro
The first thing to focus your attention on this week is an interview with Harvard Business School professor Raffaella Sadun on how to reskill the workforce for the AI era.
What’s very interesting about AI in particular is that it has the potential of having an impact on white-collar jobs and high-skilled jobs, which typically have been insulated from past technological revolutions.
The part where we come back to reality is that at the end of the day, what happens will be a function of the adoption process. And the adoption process is typically very messy, as we’ve seen in other technological revolutions. It depends on figuring out how to integrate these technologies in the workflow. And it also depends on the incentives to adopt, whether people will accept these technologies as their everyday companions in their work.
This point cannot be stressed enough, something I’ve seen first-hand in 23 years of career in the IT industry, helping some of the biggest organizations in the world adopt emerging technologies.
Incentives are the key to adoption. And, as you will read in the next few stories, if employees start to be nervous at the idea of AI replacing them in a number of tasks, without anything good coming out of it for them, adoption might stall, no matter how revolutionary the technology is.
This is a point that the CEO of Anthropic, Dario Amodei, made as well in a recent interview we commented in Issue #24 – Cannibals.
Let’s just be sure to not confuse the investment in an emerging technology with its real adoption: earlier this week, the Informer revealed that OpenAI is now earning $80 million in monthly revenue, and is projected to to generate over $1 billion in revenue over the next 12 months.
When the professor is asked about her opinion on the displacement of job by AI, she replies:
That is the million-dollar question. What’s going to happen at the end of the day depends on what firms’ strategies and organizational responses are. Let’s first say that not every firm will be actually adopting these technologies. Again, we’ve seen it in the past. There will be a lot of heterogeneity.
And those that do have basically two ways of reacting to the technology, shaping the technology adoption. One will be the lazy way. And this is people, like Daron Acemoglu who have talked about this in other outlets, where essentially they’re going to substitute whatever workforce they can with new technologies without really changing their production processes, their organizational processes. That would be potentially complicated. There would be elimination of jobs without the creation of new opportunities.
Then there is a second pathway that I find very exciting. Not everybody will get there, but I am pretty certain that some firms already are rethinking their workflows, and they’re rethinking their organizational processes in such a way that these technologies create new tasks, new opportunities, and potentially new jobs.
I’m sorry to give you an answer that is not definitive, but it depends. The critical point is that it depends on what firms do. There is nothing that is predetermined at this point.
Another point we made infinite times in this newsletter. At the end of the day, companies are collections of people, and the majority of people seek the path of least resistance. Cost-cutting, via AI or other means, is the path of least resistance compared to upskilling, multiplying the production output, and creating new jobs.
It’s faster, and it’s easier. And there’s no reason to believe that, in this case, managers will behave in a revolutionary new way.
AI is a very different technology compared to everything that came before it, and the contingencies that shape its adoption are unique compared to every past technology adoption wave. People? People are always the same.
You can’t change 100,000 years of evolution with 12 months of ChatGPT.
The second story of the week worth (more of) your attention, is a couple of novel approaches in using AI in conjunction with brain-computer interface (BCI), to give a faster voice to patients.
Emily Mullin, reporting for Wired:
Paralysis had robbed the two women of their ability to speak. For one, the cause was amyotrophic lateral sclerosis, or ALS, a disease that affects the motor neurons. The other had suffered a stroke in her brain stem. Though they can’t enunciate clearly, they remember how to formulate words.
Now, after volunteering to receive brain implants, both are able to communicate through a computer at a speed approaching the tempo of normal conversation. By parsing the neural activity associated with the facial movements involved in talking, the devices decode their intended speech at a rate of 62 and 78 words per minute, respectively—several times faster than the previous record.
…
While slower than the roughly 160-word-per-minute rate of natural conversation among English speakers, scientists say it’s an exciting step toward restoring real-time speech using a brain-computer interface, or BCI.
…
A BCI collects and analyzes brain signals, then translates them into commands to be carried out by an external device. Such systems have allowed paralyzed people to control robotic arms, play video games, and send emails with their minds. Previous research by the two groups showed it was possible to translate a paralyzed person’s intended speech into text on a screen, but with limited speed, accuracy, and vocabulary.In the Stanford study, researchers developed a BCI that uses the Utah array, a tiny square sensor that looks like a hairbrush with 64 needle-like bristles. Each is tipped with an electrode, and together they collect the activity of individual neurons. Researchers then trained an artificial neural network to decode brain activity and translate it into words displayed on a screen.
…
Over the course of four months, scientists trained the software by asking Bennett to try to say sentences out loud. (Bennett can still produce sounds, but her speech is unintelligible.) Eventually, the software taught itself to recognize the distinct neural signals associated with the movements of the lips, jaw, and tongue that she was making to produce different sounds. From there, it learned the neural activity that corresponds to the motions used to create the sounds that make up words. It was then able to predict sequences of those words and string together sentences on a computer screen.
More details on the second approach, focused on speech recognition:
researchers at UCSF built a BCI using an array that sits on the surface of the brain rather than inside it. A paper-thin rectangle studded with 253 electrodes, it detects the activity of many neurons across the speech cortex. They placed this array on the brain of a stroke patient named Ann and trained a deep-learning model to decipher neural data it collected as she moved her lips without making sounds. Over several weeks, Ann repeated phrases from a 1,024-word conversational vocabulary.
Like Stanford’s AI, the UCSF team’s algorithm was trained to recognize the smallest units of language, called phonemes, rather than whole words. Eventually, the software was able to translate Ann’s intended speech at a rate of 78 words per minute—far better than the 14 words per minute she was used to on her type-to-talk communication device. Its error rate was 4.9 percent when decoding sentences from a 50-phrase set, and simulations estimated a 28 percent word error rate using a vocabulary of more than 39,000 words.
…
The researchers created a “digital avatar” to relay Ann’s intended speech aloud. They customized an animated woman to have brown hair like Ann’s and used video footage from her wedding to make the avatar’s voice sound like hers. “Our voice and expressions are part of our identity, so we wanted to embody a prosthetic speech that could make it more natural, fluid, and expressive,” Chang said during Tuesday’s media briefing.
As some of you know, I’ve been researching the whole range of technologies that go under the umbrella term of human body enhancement since 2017.
My work is saved in a now-dormant project called H+.
Eventually, the time will come for all of us to focus on those technologies, as we enter the next stage of our evolution. For now, let’s remain concentrated on artificial intelligence.
The third story worth your attention is about one of the most famous and successful venture capitalists in the world, Vinod Khosla, who’s back on the topic of AI and jobs.
On X, he writes:
A techno-optimist replies:
Look around you.
Are we sure that people want to do bigger things?
The bonus story worth your attention this week is a new 96-pages report from the International Labour Organization (ILO), offering a positive outlook on the impact of AI on jobs.
It will balance the introduction, right?
The study, Generative AI and Jobs: A global analysis of potential effects on job quantity and quality, suggests that most jobs and industries are only partly exposed to automation and are more likely to be complemented rather than substituted by the latest wave of Generative AI, such as chatGPT. Therefore, the greatest impact of this technology is likely to not be job destruction but rather the potential changes to the quality of jobs, notably work intensity and autonomy.
Clerical work was found to be the category with the greatest technological exposure, with nearly a quarter of tasks considered highly exposed and more than half of tasks having medium-level exposure. In other occupational groups – including managers, professionals and technicians – only a small share of tasks was found to be highly exposed, while about a quarter had medium exposure levels.
OK. But how did they come to this conclusion?
The actual study reveals the methodology:
We develop a Python script that uses the OpenAI library to loop over the ISCO-08 task structure and conduct a series of sequential API calls to the GPT-4 model, using a range of prompts that we fine-tune for specific queries. Before predicting task-level scores, we run several initial tests of the GPT-4 model on the overall ISCO dataset, to determine its capacity for processing detailed occupational information. As a first step, we use the GPT-4 model to generate an international definition for each of the ISCO 4-digit codes, and to mark the level of skills required for each job, according to the same classification as used in ISCO-08 (1 for low level skills, 4 for the highest)…
The gist of it is that the researchers used GPT-4 to design, automate, and evaluate the study.
I had to re-read it multiple times as I couldn’t believe it.
First, they asked GPT-4 to define the job categories according to the ISCO-08 classification.
Then, they asked GPT-4 to suggest how GPT-4 could be used to augment each job category (so, there’s already a bias in the approach, on top of the bias inherent in the training dataset of the model).
Finally, they asked GPT-4 to evaluate the impact of GPT-4 on each job category based on the GPT-4 suggestions generated in the previous step.
So, if GPT-4 is incapable of reasoning in the way the researchers asked it to (it is), but it’s a savant at executing the tasks object of the study, this approach will never reveal it.
And we don’t know if the model is a savant. Just few weeks ago, in a past Issue of Synthetic Work, we discovered that experts estimate that it will take another two years minimum to understand all the things that GPT-4 can do, even assuming a complete freeze of the model’s development.
You cannot ask a idiot savant to self-assess its capabilities, and then take the self-assessment at face value.
This is why you should always take published research with a grain of salt, even when the publishing organization sounds very credible.
You’ll find further validation down in the Putting Lipstick on a Pig section of this Issue.
For any new technology to be successfully adopted in a work environment or by society, people must feel good about it (before, during, and after its use). No business rollout plan will ever be successful before taking this into account.
In this section of the newsletter, we always talk about the feelings of the employees, or the general public, as consumers of AI. But what about the feelings of the people that adopt that AI?
What about the feelings of the CEOs? Nobody ever thinks about them.
Well, Isabelle Bousquette does and shares her findings on The Wall Street Journal:
More and more companies say they are concerned about facing public criticism over their use of artificial intelligence, thanks to rising fears over the technology’s negative impacts, including job losses.
The rapidly evolving technology has opened up a nearly limitless number of use cases for businesses, but also a new set of risks—including public backlash and damage to a company’s reputation.
…
since the release of ChatGPT, public opinion has zeroed in on the insidious elements of AI, including the potential for bias, discrimination and job displacement, as well as concerns about privacy, he said.Companies are now left to navigate how they can use the technology without upsetting their existing and potential customers.
“If you’re not concerned about it, you’re going to get burned,” said Emory Healthcare Chief Information and Digital Officer Alistair Erskine.
…
Companies are now looking to avoid that scenario. Eric Yaverbaum, chief executive of public relations company Ericho Communications, said that six months ago none of his clients was asking him about the potential for reputational risk from AI. “Now everybody is,” he said.
…
Negative responses to decisions made by companies may be inevitable, but what is avoidable is a situation where the company looked like it was careless or didn’t think through its actions, said Kossovsky. He added that companies can no longer assume AI is an inherent good.
The article contains a number of examples of companies that are moving ahead with the deployment of AI, but that have prepared press announcements to mitigate potential misunderstandings and negative reactions.
The problem is that there’s no misunderstanding.
So far, employees have seen AI being used mainly as a cost-reduction tool. Once you see that, it doesn’t take a genius to understand that, as technology improves, fewer people will be needed to do today’s jobs. And nobody is buying Marc Andreessen’s argument that companies will produce ten times more with the same employees.
In the last 6 months on Synthetic Work, I warned you that this moment would come, that employees would start pushing back and voice their fears.
The next step, as I suggested multiple times, will be sabotage.
Of the hundreds of companies we have mentioned , only IKEA has used AI to upskill its workforce, without firing anyone in the process. We talked about it in Issue #18 – How to roll with the punches.
That’s an example to follow.
Given that we talked about dubious research at the beginning of this issue, I thought it would be appropriate to share with you the ugly things happening in that world.
To help me, we have Amanda Hoover, reporting for Wired:
In its august edition, Resources Policy, an academic journal under the Elsevier publishing umbrella, featured a peer-reviewed study about how ecommerce has affected fossil fuel efficiency in developing nations. But buried in the report was a curious sentence: “Please note that as an AI language model, I am unable to generate specific tables or conduct tests, so the actual results should be included in the table.”
…
After a screenshot of the sentence was posted to X, formerly Twitter, by another researcher, Elsevier began investigating. The publisher is looking into the use of AI in this article and “any other possible instances,” Andrew Davis, vice president of global communications at Elsevier, told WIRED in a statement.
…
Elsevier’s AI policies do not block the use of AI tools to help with writing, but they do require disclosure. The publishing company uses its own in-house AI tools to check for plagiarism and completeness, but it does not allow editors to use outside AI tools to review papers.
For the nth time: anti-plagiarism tools do not work with AI-generated text. They don’t work for school assignments, they don’t work for academic papers, they don’t work for news articles, they don’t work for books.
They only increase the bias in the system, because the only people punished are the ones that are clumsy in using AI for cheating. The smart ones get rewarded.
Let’s continue:
Journals are taking a patchwork approach to the problem. The JAMA Network, which includes titles published by the American Medical Association, prohibits listing artificial intelligence generators as authors and requires disclosure of their use.
The family of journals produced by Science does not allow text, figures, images, or data generated by AI to be used without editors’ permission. PLOS ONE requires anyone who uses AI to detail what tool they used, how they used it, and ways they evaluated the validity of the generated information. Nature has banned images and videos that are generated by AI, and it requires the use of language models to be disclosed.
Many journals’ policies make authors responsible for the validity of any information generated by AI.
So let’s see how responsible human researchers are:
Thought so.
This week’s Splendid Edition is titled Sport is for Mathematicians.
In it:
- What’s AI Doing for Companies Like Mine?
- Learn what General Motors, RXO, XPO, Phlo Systems, and Amazon Prime Video are doing with AI.
- Prompting
- Learn how to use ChatGPT Custom Instructions to automatically apply the How to Prompt best practices to every chat.
- What Can AI Do for Me?
- Learn how to use Custom Instructions to turn GPT-4 into a marketing advisor following the lessons of Seth Godin.