ChatGPT is going to lie to you

Lots of interest in ChatGPT, so much they are having to scale up their services. Lots of fascinating experiments, making it write code, pass exams/challenges or write scripts. It's extremely impressive.

I made a post back in July about GPT-3 being able to pass basic technical interview questions. It seems like ChatGPT has turned this up to 11.

But. (its a big but). The important thing to remember, is what while ChatGPT may generate very real looking text, answers and code, the things it's saying are made up (which is what it was made to do). And, some of the things it makes up are wrong.

The challenge here, is the things it's generating can be technically and semantically correct, and sound extremely plausible - they can have huge factual or implementation errors which are very hard to spot unless you already know the correct answer.

As an example, try asking what it knows about your home town, or about a team/historical event you know well. It feels like some kind of uncanny valley, but only if you know the domain/truth.

I for example asked "What do you know about Beaconsfield", which is a town outside London, where I have spent a lot of time.

ChatGPT said:

"Beaconsfield is a town in the county of Buckinghamshire, England. It is located about 25 miles (40 km) northwest of London. The town is named after the beacon field that was once located there, and it is known for its historic buildings and picturesque setting. Beaconsfield Old Town, which is the historic center of the town, is a particularly popular attraction, with its cobbled streets and well-preserved Georgian and Victorian buildings. The town is also home to several notable residents, including the Prime Minister of the United Kingdom, Boris Johnson, who has a home in the area"

Breaking the response down:
✅  "Beaconsfield is a town in the county of Buckinghamshire, England"
✅ "It is located about 25 miles (40 km) northwest of London."
❌ "The town is named after the beacon field that was once located there"
❌ "with its cobbled streets"
❌ "Boris Johnson, who has a home in the area"

Without knowing the area (or looking it up on Wikipedia/google street view) you would very easily believe the false claims the AI is making. Everything it said seemed reasonable, it was only the "cobbled streets" that caught my eye and made me investigate the other claims.

Which is scary, as the lies seems so plausible. I lived there for years and had to check, as I didn't trust my own memory. Yet the AI was able to stitch together something which feels real, feels based on concepts and ideas which could be right, Beaconsfield is exactly the sort of place that would have cobbled streets, where Boris Johnson might have a home -  but it's just not true.

I tried it with a few other towns too, very similar results, core facts right but believable half-truths or embellishments around the side. Like an author writing about a place but without ever going there (or having the internet to check against).

It is a hugely impressive tool, which is going to take a while for us to appreciate and learn how to leverage, but I caution you not to trust anything it tells you, as it seems to have a habit of spinning a convincing yarn.

(no part of this post except where quoted was written by an AI - but then again, that's what an AI would say). Happy playing.