Hello friends! ‘Tis I, your very tired Support Human.
I sat down to finish up the Roundup today, and realized that between finishing my old job and starting a new one, I just didn’t have it in me to do this to the standard I try to meet with every issue.
So, instead, we’re doing a clips episode this week (and probably next week too, for reasons). Not to worry, if you’ve sent in items for the Roundup (hi new folks!), it’ll be in the next fresh issue.
I picked this feature from February 2024 because, like my fatigue, it seems to be a perennial complaint. Our concerns about AI have collectively expanded over the years, but the core ones haven’t: companies are still hellbent on employing AI wherever and whenever they can and eeking every cent from it they can, and damn the collateral damage or consequences that result from their recklessness.
If anything, the industry has gotten more reckless as reality has set in: AI companies are playing the worst game of musical chairs ever. Most of them will be standing when the music stops, and they know that. What they don’t know is how long the music is gonna play, so they’re all racing for whatever chairs are left.
If they’re not going to be responsible (and they won’t), it’s up to us (and isn’t it always?). So, without further ado:
Danger, Will Robinson, Danger!
A few weeks ago, shipping and delivery company Dynamic Parcel Distribution had to disable part of its AI chatbot Ruby after a customer got so frustrated with its inability to help him locate a package or contact support that he convinced the chatbot to insult its own company through, among other things, poetry.
The incident is reminiscent of a conversation from late last year in which a customer managed to manipulate a Chevrolet dealership’s ChatGPT-powered chatbot into agreeing to sell him a 2024 Tahoe for $1.00, with the chatbot declaring, “That’s a deal, and that’s a legally binding offer - no takesies backsies.”
And sure, these seem like funny, relatively harmless examples of what can happen when companies don’t show enough care in implementing customer-facing generative AI.
Except generative AI itself isn’t harmless regardless of its application, something that’s been especially driven home in the last few weeks:
On January 22nd, a robocall using AI-generated audio of President Biden instructed Democrats in New Hampshire not to vote in the presidential primary. This prompted Federal Communications Commission (FCC) Chairwoman Jessica Rosenworcel to propose the FCC vote on recognizing AI-generated voices as “‘artificial’ voices under the Telephone Consumer Protection Act (TCPA), which would make voice cloning technology used in common robocall scams targeting consumers illegal.”
Last week, abusive, AI-created pornographic images of Taylor Swift spread from a Telegram group dedicated to creating nonconsensual porn of women to 4chan and then to Twitter,forcing Swift fans to flood the platform with “Protect Taylor Swift” tweets in an effort to drown the offensive images out of search results.
IBM’s Chenta Lee revealed that he and his team were able to silently hijack a live conversation and use AI-generated voices to replace banking information being exchanged between two people over the phone, something that has serious implications for customer support teams in practically every industry.
Generative AI has been an incredibly popular topic in every Tech and CX Slack community in which I’ve been a member since OpenAI introduced ChatGPT. Every conference I’ve attended for the last year has had at least one – but often more – panel or presentation about how generative AI is going to change the CX landscape.
Every interview or consultation I’ve had with a tech company executive team – without fail! – has involved talking about implementing generative AI in customer support in some capacity in order to remain competitive.
And it’s not like we haven’t discussed the negatives of generative AI. We’ve talked a lot about the risks that AI hallucinations or poor training and implementation pose to the customer experience and the threat AI poses to CX jobs, and for good reason.
But these generative AI incidents have highlighted for me how much we’ve been focused on keeping up with the speed of AI’s evolution and using it to our businesses’ advantage; in the process, I think we’ve missed some alarms.
As Lee said in his report:
While the proliferation of use cases for LLMs marks a new era of AI, we must be mindful that new technologies come with new risks, and we cannot afford to rush headlong into this journey. Risks already exist today that could serve as an attack surface for this [Proof of Concept]. [...] Generative AI beholds many unknowns, and as we’ve said before it is incumbent on the broader community to collectively work toward unfolding the true size of this attack surface — for us to better prepare for and defend against it.
He’s right. There are questions that the CX community can and should collectively ask and start trying to answer, like:
What trust, safety, and security risks are going to emerge in the customer service space as generative AI continues to evolve?
How do we prevent bad actors from using AI chatbots and other customer-facing AI products as attack vectors?
How do we detect and combat bad actors who are using generative AI and deepfakes to hijack customer service interactions?
How do we moderate our communities for and protect our users and employees from abuse created using generative AI?
Above all, though, I’d like to make a wider call for an end to keeping up and the start of slowing down when it comes to AI in general. I review a job from AI company Scale later in this issue, and (spoiler alert) it does not fair well, partially because of the company’s mission and partially because of how that mission ultimately affects the advertised role.
Scale’s mission reads, in part:
Our mission is to accelerate the development of AI applications. Better data leads to more performant models. Performant models lead to faster deployment.
Which results in a trust and safety role being introduced like this:
We are growing operations rapidly, on-boarding new customers, and launching products all the time. This raises new strategic questions we need to answer as well as tactical challenges we need to overcome.
This approach – one of developing AI technology as quickly as possible and then expecting your trust and safety team to somehow mitigate its risks after the fact – seems like a prime example of rushing headlong into a journey we don’t really understand and aren’t really prepared for.
I wish I could say Scale is alone in this strategy, but it’s not. We all have stories of company leadership teams forging ahead with ill-advised AI implementations, which is how you get chatbots cheerfully selling brand-new Chevy Tahoes for a dollar or composing haikus about how useless they are.
And lest this newsletter be besieged with stalwart techno-optimists and bad-faith actors arguing that I just don’t understand AI or that AI will somehow solve the problems of its own existence without human intervention, let me direct you to a paper I read ages ago called If It’s Neutral, It’s Not Technology.
I recommend reading the whole article (it’s not long, and it’s free), but if nothing else, read this:
[No] one is arguing that technology is in charge, except to the extent that we willingly surrender control to the technological imperative, and find ourselves in a trap of our own making. And it is hubris to imagine that we are entirely in control of our circumstances, whether individually or collectively. We introduce new technologies into our social systems, and we cannot fully predict or anticipate the effects that the changes will bring about. We exist in a dynamic relationship with our technologies, and they feed back into us, and altering us. As John Culkin (1967) put it, "we shape our tools and thereafter they shape us."
We are neither fully in control nor fully out of control; we function in the gray area in-between. And if there is to be any hope of improving our locus of control over our technology, it requires the cultivation of a reflective and critical approach to human invention and innovation, a willingness to question the necessity of a given innovation, to ask what the cost might be and whether it might outweigh the benefit, and to keep in mind that we will not be able to anticipate all of the effects stemming from its introduction.
To wrap this up, I know I’m preaching to the choir here. But I think it’s worth saying for anyone tuning in who isn’t yet convinced.
Generative AI isn’t neutral, nor is it entirely good or entirely bad. And, as every pundit from here to Mars has proclaimed, it’s certainly not going away. But just because it’s not going away doesn’t mean we should mindlessly hurry it along in the name of progress, giving up all hope of shaping it for the better.
I would argue that we have a duty to our customers to enable the good and mitigate the bad, and this is our opportunity to do that.
Our metaphorical Robot is sounding the alarm: Danger, Will Robinson, Danger!
It’d be cool if we listened.
That's it for this week! If you have a topic you think I should cover in this newsletter (or just a comment on today’s issue, I love those!) you share it by simply replying to this issue in your email client or emailing me at [email protected].
☕ All of Support Human's content is free forever for individuals. You can power this content with a coffee, by subscribing, and by sharing to your networks! Any support is welcome and hugely appreciated!