ChatGPT’s ability to seamlessly turn into a cultural chameleon was disarming, and it marked the first time I almost felt like I wasn’t just talking to a machine. I even caught myself responding as if it were a person, telling the tool “sorry to interrupt” and “thank you so much”.

It’s not just me. Advanced voice mode’s mastery of spoken dialects is bewildering. A Jamaican start-up founder shared a clip of himself conversing with ChatGPT in the local Patois. Videos have flooded TikTok showing the app speaking in everything from Singlish to African American Vernacular English.

ChatGPT’s ability to communicate in dialects opens the floodgates for more people to interact with and anthropomorphise the tool. It may all seem gimmicky, but anyone who has travelled far from home knows that language and culture bridge barriers. Hearing someone who speaks like you, in whatever language, instantly forges a sense of kinship.

For most of the internet age, this sort of cultural nuance was something that computers couldn’t yet capture, marking one of the biggest differentiators between friend or machine.

But there are potential consequences to using dialects with AI. A Nature study published in August found that large language models generate “covertly racist decisions” about people based on their dialects in a text-based experiment. The researchers found AI is more likely to suggest that users of African American English “be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death” compared with speakers of standard American English.

Separately, OpenAI has acknowledged how the audio capabilities, specifically, of its latest model could carry risks we don’t fully understand yet, such as people creating strong bonds or allowing for “increasingly miscalibrated trust” in AI models.

The company has noticed users tapping into the audio features using language that might indicate they feel connections with the model, saying more internal and independent studies need to be done to more “concretely define this risk area”. And the reason the chatbot now sounds so humanlike is partly because its nine voices are powered by real actors.

Loading

OpenAI has a slew of guardrails. ChatGPT answered a few questions I asked by changing the topic, saying its guidelines won’t allow it to respond. It can’t sing, impersonate people, or be overly flirtatious. The current version also stopped short of live translating something other than my own voice, such as a speech or video, possibly to avoid further copyright scrutiny. More eagle-eyed observers have been critical of its proficiency in certain languages and dialects. Due to regulatory hurdles, advanced voice mode is not yet available in the European Union and a few other markets.

There are tremendous potential upsides to ChatGPT’s new ability to shatter language barriers, for both businesses and people. But as mesmerised as I was this past week, every now and then I would get a response that would remind me it’s just an app — I shouldn’t assign it too much responsibility or see it as a human companion. I couldn’t help but worry for a future when people may forget that.

Bloomberg

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.