jagged intelligence

This is a term coined by Andrej Karpathy, a researcher specialising in AI, who has been at the heart of AI developments for a decade.  He used it to name the experience that many people have of AI, that it can be brilliant in many things (particularly sifting information from large data sources), and inexplicably fail at tasks that humans would find very easy.  When asked how many times the letter r appeared in the word barrier, ChatGPT claimed it was twice. [I have been told that ChatGPT is counting the sound /r/ and not the three letters. It continues to produce this result.]

AI is not good at knowing when it is right and when it is wrong, and so proceeds on with its own lapses of judgement (known as hallucinations).  Hallucinations commonly occur when the question is not one of being right or wrong but of demonstrating a very human commonsense.  Unfortunately it is not possible for humans to instantly identify these error moments so that the success rate of AI has many peaks and troughs, creating an image of a jagged intelligence.