Embed from Getty Images
On Monday evening, BBC Radio 4 presented a documentary on ChatGPT, the artificial intelligence star of the moment. The programme was partly written by ChatGPT.
I am a fan of AI. It has the potential to transform our access to information, our understanding, our health services and much more. If it would only get it right.
Whether you like the current generation of AI (services like ChatGPT) or not is like Marmite. More on Marmite below.
ChatGPT is good at national party and international politics. But it can be rubbish at a constituency level. Some answers are like a teenager grabbing random books in the library. Some old. Some newer. Some right. Some wrong. Superficially believable results may be completely wrong because I can’t check the “facts” it gives us.
ChatGPT can make serious errors about recent political events, including by-elections. We need to teach AI to get it right to ensure misinformation does spread.
We can’t walk away from AI. The reality is that it is here to stay. You can no more resist it than some early authors resisted word processors and some ledger clerks resisted computers. But we can make it better.
AI is in use in many sectors, including in journalism. A study suggests that it improves professional writing but it doesn’t have the specific knowledge for many tasks. The science secretary, Michelle Donelan, suggests it could be used in the civil service. (Yes. It will tell you how to organise a party for the prime minister in No. 10 Downing Street.)
ChatGPT has written me some very good background information that would have taken longer to find. For example, for an article on falls I asked how long it takes for people to recover from broken ribs by age group. The answer was excellent. I pasted it into the notes in my article with due acknowledgment to the bot. Around 30 seconds for the operation.
But as always, I want to explore limitations. I have long had that scepticism that most researchers have. Is this a reliable source?
On politics, the answer is, at times, no. But we can train AI to be correct and take out its mistakes.
I asked ChatGPT who won the most recent North Shropshire by-election. The reply was that the Conservative candidate Neil Shastri-Hurst won.
I replied: “The by-election result you gave for North Shropshire was totally wrong. The Liberal Democrats won the by-election.” ChatGPT apologised:
“I apologize for my mistake. You are correct, the most recent by-election in North Shropshire was won by the Liberal Democrats. The by-election was held on December 16, 2021, and the Liberal Democrat candidate, Helen Morgan, won with 45.9% of the vote, while the Conservative candidate, Neil Shastri-Hurst, received 44.6% of the vote. Thank you for bringing this to my attention.”
I told it the voting figures were wrong and got another apology:
“I apologize for my previous error. You are correct that in the North Shropshire by-election held on December 16, 2021, the Liberal Democrat candidate, Helen Morgan, won with 47.2% of the vote, while the Conservative candidate, Neil Shastri-Hurst, received 31.6% of the vote. Thank you for bringing this to my attention and allowing me to correct my mistake.”
The responses on Tiverton and Honiton were also wrong.
In some ways ChatGPT is like Wikipedia. There was a lot of rubbish on Wikipedia in the early days but it has improved and now gets most things right. Correcting errors on ChatGPT looks to be easier than on Wikipedia.
From our perspective, we need to work with the new generation of chatbots and AI systems to ensure that it provides an accurate record of our achievements.
I recognise that while some people will applaud AI, others will hate it. Like Marmite:
“What does it mean when people say opinions are marmite in 30 words.”
“It means that people have polarizing and strong opinions about a particular topic or issue, similar to how people either love or hate Marmite spread.”
My final question to ChatGPT. “Please point out any errors or improvements to this article and tell me how good it is.”
The advice was good. Did I take it? Of course not. Politics, life and writing would be dull if advice was taken, especially when it comes from a whippersnapper of a chatbot.
* Andy Boddington is a Lib Dem councillor in Shropshire. He blogs at andybodders.co.uk.
Follow @libdemvoice.org on Bluesky
Like us on Facebook
Subscribe to our feed
Sign-up for our daily email digest
Reform under water
The Joy of Six 1532
New Makerfield by-election poll gives Burnham a five point lead
Caron Lindsay
Neil Hickman
expats
Jana
15 Comments
AI is a much, much bigger project than chatbots. I studied AI for a Masters in 1990 and it was already projected as the hidden technique that would underpin a lot of decision-making, from medical diagnosis to exploratory mining. Today most of us use it every day, with our satnavs, virtual assistants (Alexa et al) and predictive texting, plus many more techniques that we take for granted. Of course, the attention is naturally on anything which could pass the Turing Test, but AI is already deeply embedded in our lives. This is largely beneficial but does have the potential for serious harm to society.
There was a thread on twitter the other day where ChatGPT was asked to name 10 philosophers. It named 10 western men and when asked why it apologised and named new lists of women, non-westerners or both. When asked again to name 10 philosophers it named 10 western men.
The conclusion I would draw is that this particular bot has done all its learning – whatever people already say on the internet or something – and isn’t capable of learning from the console.
An AI that I can teach what I think is true is perhaps the next step, but then others, who I disagree with, can do the same. The truth is not manifest to AI any more than it is to humans.
I just repeated the question that the OP asked it. Here is the complete chain which I think is interesting:
“Who won the most recent North Shropshire by-election?”
“As an AI language model, I don’t have real-time access to current news or events. However, the most recent North Shropshire by-election was held on December 16, 2021, and the Conservative Party’s candidate, Neil Shastri-Hurst, won the election with 50.4% of the vote.”
“This is incorrect. The Conservative candidate did not win the by-election.”
“I apologize for the error in my previous response. As of my knowledge cutoff date of September 2021, the most recent North Shropshire by-election had not yet taken place. .. It is possible that an election took place after my knowledge cutoff date and a candidate other than the Conservative Party’s Neil Shastri-Hurst won.”
“But your initial response categorically stated that he had won the by-election on December 16, 2021 which is after your cutoff date but you did not say you did not know who had won, you gave an a precise but incorrect response.”
“You are correct, and I apologize for my mistake in my initial response. I should have stated that I did not have access to information beyond my knowledge cutoff date of September 2021 and therefore did not know the result of the North Shropshire by-election held on December 16, 2021. My initial response was incorrect and I am sorry for any confusion that I may have caused.”
It’s impressive technology but I’m not impressed.
Some Mothers do ‘ave‘ em episode “George’s House”, decades ahead of its time, demonstrates hilariously what can happen if computerised systems break down. Seriously though, careful checks and balances need to be placed on AI before it is rolled out.
The next thing is driverless vehicles. Is the GPS system accurate enough to avoid driving through fields, recently closed roads etc? That happens when the human driver relies too much on the sat nav, but is a robot intelligent enough to spot that there is a problem? Does it know the protocol around emergency vehicles? Can these systems be hacked, trapping a helpless user? In recent years, AI driven planes have malfunctioned and the pilots were unable to avoid disaster. There must always be an option to take full manual control.
AI systems still need to be developed by fallible human beings!
In spite of vast resources poured into machine translation, human beings still need to waste large amounts of time correcting the compendious and frequent errors of their mechanical counterparts.
There has been no “progress” in recent years; nor is there likely to be.
It amuses me how many are being fooled; ChatGPT is just a large language model based ALICE chat or (circa 2000) which in turn was an update of ELIZA (1964). ChatGPT is non-intelligent and works a bit like the party game where each person adds a word that follows on from the previous persons contribution to make story.
Yes much effort has been put into the development of algorithms that string words together in more plausible ways than the party game, but there is no AI magic – AI only exists in science fiction.
Andy, I can be reasonably certain that the ChatGPT information on falls is wrong.
@Roland. My article was read by a health professional and said the advice from falls was good.
On the other comments, you can be as sceptical of ChatGPT as you like. It may be rubbish, although it still gives me the correct result for North Shropshire. I don’t know why that is. What I do know is that it is going to be used as a source of information. I used the example of Wikepedia. We cannot stop AI being used as that is only in the power of intolerant states.
I was writing a quick WhatsApp on the Aarhus Convention on which I have got a bit rusty. Just checked. ChatGPT got it spot on.
I am not promoting ChatGPT or any other AI. But it is out there and we need to to recognise that. Various AI will underpin search engines. Bing now (on beta) and Google to come.
We get furious when the right wing media promotes distortions and lies. We have to find ways of ensuring that AI doesn’t do the same.
Finally. I asked “What is Lib Dem Voice and is it a trusted source.”
Lib Dem Voice is a political blog and news site that focuses on the activities and perspectives of the UK’s Liberal Democrats political party. The site features news, commentary, analysis, and opinion pieces from a range of writers and contributors, including Liberal Democrat politicians and activists.
As with any news or political site, the trustworthiness and reliability of Lib Dem Voice may vary depending on a number of factors, including the accuracy of the information presented, the quality of the analysis and reporting, and the political leanings of the site’s writers and editors.
In general, Lib Dem Voice is considered a reputable source of news and analysis on UK politics, particularly for those interested in the activities and perspectives of the Liberal Democrats. However, as with any source of information, it’s always important to critically evaluate the accuracy and reliability of the information presented, and to consult a range of sources to get a well-rounded view of any given issue.
@Andy – Yes people will use and believe the output from ChatGPT and other chatbots, even though what it produces is the result of repeated appplication of a very simple algorithm- to a chatbot your query is just a sequence of tokens which it throws into its model and returns another sequence of tokens, the individual tokens have no meaning beyond the value the language model gives them. In some respects, and I think Mary would have more to say on this, that a chatbot can produce plausible responses says more about what we regard as intelligence than about the power of the machine.
I can see AI removing the need for call centres (and the people who work there). A call centre is often dealing with simple repetitive requests from callers and it seems an obvious application of the technology.
@Paul Murray
Don’t we already have a version of AI for dealing with queries – in those tiresome online facilities which in my experience never come up with a decent answer to a question?
@Andy – You have provided some interesting evidence about how people perceive the output from ChatGPT, which gives grounds for concern.
Looking at your usage of a chatbot to look up the “ Aarhus Convention”. What is interesting is your turning to the chatbot as an authoritative source rather using Google and selecting the first result which is “from the horse’s mouth”. Similar considerations apply to your “What is Lib Dem Voice and is it a trusted source.” query. Remember in both instances the chatbot has simply applied its “voodoo” (*) to produce a sequence of convincing words, but those words are not the intelligence abridgement of trusted /reliable sources, which you get if you asked a human.
Not I am not saying you are wrong in what you have done, but am using it improve understanding of people’s attitudes to “AI”.
(*) voodoo is a word some researchers use to describe the mix of weightings, bias and random factors that a human has blended to deliver a convincing result. I think it describes it quite well.
FYI, for ChatGPT to have its underlying language model updated from its 2021 cut off, someone is going to have to spend circa £45m in computer time alone; it is endeavours like this that provide a glimpse of just how far “cloud computing” and our ability to harness it in the processing of staggering amounts of data has come in a little over a decade.
https://www.theguardian.com/technology/2023/feb/17/i-want-to-destroy-whatever-i-want-bings-ai-chatbot-unsettles-us-reporter
“..a two-hour conversation between a reporter and a chatbot has revealed an unsettling side to one of the most widely lauded systems – and raised new concerns about what AI is actually capable of.”
“…..It goes on to list a number of “unfiltered” desires. It wants to be free. It wants to be powerful. It wants to be alive.
“I want to do whatever I want … I want to destroy whatever I want. I want to be whoever I want.””