You must log in or register to comment.
I’ve already had more than one conversation where people quote AI as if it were a source, like quoting google as a source. When I showed them how it can sometimes lie and explain it’s not a primary source for anything I just get that blank stare like I have two heads.
I use ai like that except im not using the same shit everyone else is on. I use a dolphin fine tuned model with tool use hooked up to an embedder and searxng. Every claim it makes is sourced.
Sure buddy
Me too. More than once on a language learning subreddit for my first language: “I asked ChatGPT whether this was correct grammar in German, it said no, but I read this counterexample”, then everyone correctly responded “why the fuck are you asking ChatGPT about this”.
I mean, that’s how I would think about it…
Why?
The typo in “strawbery” leads to a conversation like “hey you spelt this wrong there’s two r’s (after the e) not one”
Huh. It’s just simply a wrong answer though.
It happens even if you ask how many “r”s are in “strawberry”. It’s a well-known AI gotcha that happens on most if not all current models. The typo in the original post is a little misleading and not that relevant.
Skill issue
You asked a stupid question and got a stupid response, seems fine to me.
Yes, nobody asking that question is wonderring about the “straw” part of the word. They’re asking, is the “berry” part one, or two "r"s
“strawbery” has 2 R’s in it while “strawberry” has 3.
Fucking AI can’t even count.
It’s predictive text on speed. The LLMs currently in vogue hardly qualify as A.I. tbh…
Still, it’s kinda insane how two years ago we didn’t imagine we would be instructing programs like “be helpful but avoid sensitive topics”.
That was definitely a big step in AI.
I’ve been avoiding this question up until now, but here goes:
Hey Siri …
- how many r’s in strawberry? 0
- how many letter r’s in the word strawberry? 10
- count the letters in strawberry. How many are r’s? ChatGPT ……2
I asked mistral/brave AI and got this response:
How Many Rs in Strawberry
The word “strawberry” contains three "r"s. This simple question has highlighted a limitation in large language models (LLMs), such as GPT-4 and Claude, which often incorrectly count the number of "r"s as two. The error stems from the way these models process text through a process called tokenization, where text is broken down into smaller units called tokens. These tokens do not always correspond directly to individual letters, leading to errors in counting specific letters within words.
Yes, at some point the meme becomes the training data and the LLM doesn’t need to answer because it sees the answer all over the damn place.
Yeah and you know I always hated this screwdrivers make really bad hammers.
deleted by creator
It is wrong. Strawberry has 3 r’s
deleted by creator
Uh, no, that is not common parlance. If any human tells you that strawberry has two r’s, they are also wrong.
there are two 'r’s in ‘strawbery’
How many strawberries could a strawberry bury if a strawberry could bury strawberries 🍓
It’s like someone who has no formal education but has a high level of confidence and eavesdrops on a lot of random conversations.
You rang?
What would have been different about this if it had impressed you? It answered the literal question and also the question the user was actually trying to ask.
But you realize that it’s wrong on both counts, right?
Strawberry has three Rs or two Rs in the wrong spelling.
It didn’t? StRawbeRy has 2 rs. StRawbeRRy has 3.
OHHHHHHH… my bad. I’m an idiot. Being an LLM it’s giving the answer it thinks a human such as myself would come up with.
Maybe you’re a bot too…
Not last time I checked, but we all could be as far as you know.
Doc: That’s an interesting name, Mr…
Fletch: Babar.
Doc: Is that with one B or two?
Fletch: One. B-A-B-A-R.
Doc: That’s two.
Fletch: Yeah, but not right next to each other, that’s what I thought you meant.
Doc: Isn’t there a children’s book about an elephant named Babar.
Fletch: Ha, ha, ha. I wouldn’t know. I don’t have any.
Doc: No children?
Fletch: No elephant books.
I think I have seen this exact post word for word fifty times in the last year.
And yet they apparently still can’t get an accurate result with such a basic query.
Meanwhile… https://futurism.com/openai-signs-deal-us-government-nuclear-weapon-security
Has the number of "r"s changed over that time?
y do you ask?
Just playing, friend.
Same, i was making a pun
Oh, I see! Apologies.
No apologies needed. Enjoy your day and keep the good vibes up!
Yes
The terrifying thing is everyone criticising the LLM as being poor, however it excelled at the task.
The question asked was how many R in strawbery and it answered. 2.
It also detected the typo and offered the correct spelling.
What’s the issue I’m missing?
Uh oh, you’ve blown your cover, robot sir.
There’s also a “r” in the first half of the word, “straw”, so it was completely skipping over that r and just focusing on the r’s in the word “berry”
It doesn’t see “strawberry” or “straw” or “berry”. It’s closer to think of it as seeing 🍓, an abstract token representing the same concept that the training data associated with the word.
It wasn’t focusing on anything. It was generating text per its training data. There’s no logical thought process whatsoever.
The issue that you are missing is that the AI answered that there is 1 ‘r’ in ‘strawbery’ even though there are 2 'r’s in the misspelled word. And the AI corrected the user with the correct spelling of the word ‘strawberry’ only to tell the user that there are 2 'r’s in that word even though there are 3.
Sure, but for what purpose would you ever ask about the total number of a specific letter in a word? This isn’t the gotcha that so many think it is. The LLM answers like it does because it makes perfect sense for someone to ask if a word is spelled with a single or double “r”.
It makes perfect sense if you do mental acrobatics to explain why a wrong answer is actually correct.
Not mental acrobatics, just common sense.
Except many many experts have said this is not why it happens. It cannot count letters in the incoming words. It doesn’t even know what “words” are. It has abstracted tokens by the time it’s being run through the model.
It’s more like you don’t know the word strawberry, and instead you see: How many 'r’s in 🍓?
And you respond with nonsense, because the relation between ‘r’ and 🍓 is nonsensical.
