Gemini Pro refuses to acknowledge yt-dlp(twitter.com)
twitter.com
Gemini Pro refuses to acknowledge yt-dlp
https://twitter.com/adocomplete/status/1811802857022324904
49 comments
This is why it is critical for true open sources LLMs to continue to be available. Things like this make it clear that the closed models are being built to enforce corporate ideologies and not just 'safety' related issues. How long until these LLMs purposefully start sabotaging competitors? "I'm sorry, but I can't work on that task anymore." As a side note, I wonder if an LLM provider will ever be sued because they promised to deliver, and took payment, but then refused to provide the agreed upon service. You still get charged for tokens even when the model refuses to work. If this was any other service that would not be acceptable.
Current trend seems to be that we regulate the AI, not the people who control the AI. We should absolutely regulate the people behind the AI.
One excellent writing: https://www.schneier.com/blog/archives/2023/12/ai-and-trust....
One excellent writing: https://www.schneier.com/blog/archives/2023/12/ai-and-trust....
Neat, introducing a fiduciary to manage the responsibility of broken trust by representatives (of corporations) who field AIs that do harm, good idea.
> This is why it is critical for true open sources LLMs to continue to be available.
There's also a huge reason for research and any attempts to claim AGI. The training data is massive and far larger than any single person or even small group can look through. AND we know that the filters aren't always accurate. All benchmark results necessitate a grain of salt. I mean at times Gemini has been "better" than GPT on benchmarks but at the same time was saying that cockroaches will crawl into your penis. But there's plenty of papers showing that accuracy of answers (of any model) highly correlates to the frequency of that information and I think you can prove this to yourself pretty quickly if you ask it some coding questions and try to ask about some more obscure topics. Not that you can't get the answers, but that when you can get them, they are harder to get.
There's also a huge reason for research and any attempts to claim AGI. The training data is massive and far larger than any single person or even small group can look through. AND we know that the filters aren't always accurate. All benchmark results necessitate a grain of salt. I mean at times Gemini has been "better" than GPT on benchmarks but at the same time was saying that cockroaches will crawl into your penis. But there's plenty of papers showing that accuracy of answers (of any model) highly correlates to the frequency of that information and I think you can prove this to yourself pretty quickly if you ask it some coding questions and try to ask about some more obscure topics. Not that you can't get the answers, but that when you can get them, they are harder to get.
Availability in the accessibile sense is noble, but without cost parity, we'll end up in the same tiered system we have for ad-funded services[email, search], medical services, air travel, food purity, &c.
Does the importance of ideological monopolism transcend market economics?
Does the importance of ideological monopolism transcend market economics?
bushbaba(1)
One weird quirk of Gemini is that it will sometimes start to show a useful answer before censoring itself. When I asked "What tool can I use to download YouTube videos?" I got the response: "There are several tools..." before the answer disappeared.
I'm hoping removal of this result this is a technical fluke, or at worse some misguided editorial decision by a lower-level employee trying to impress their boss. I guess a little weird to start editorializing the Gemini response, but not Google Search results.
As others have said, good argument for local LLMs or at least diverse closed alternatives. Has a good meta-LLM frontend emerged?
I'm hoping removal of this result this is a technical fluke, or at worse some misguided editorial decision by a lower-level employee trying to impress their boss. I guess a little weird to start editorializing the Gemini response, but not Google Search results.
As others have said, good argument for local LLMs or at least diverse closed alternatives. Has a good meta-LLM frontend emerged?
It looks like if the top answer is censored that it shoves it into a draft and instead shows the censored output. I had this happen a few times but it looks like the draft was what was being typed before it got censored.
A year or two ago I was writing a plug-in for yt-dlp for a new site and I don’t really know python that well so I popped into a python irc support room and was getting some help when the user asked what I was ultimately trying to do. When I mentioned yt-dlp the user said he could not help me any longer since he was employed by Google and continuing to help me was in violation of his employment agreement. He even linked some document from corp.google.com(or something along those lines). I was pretty surprised and even a little annoyed because after saying all that he framed me/yt-dlp as being piracy…
You wouldn't consider the last 10-15 years of changes to Google search algo to be editorializing?
ChatGPT used to do this all the time. I spent a few nights getting it to write things and then stop itself laughing my head off.
questions about yt-dlp had similar outputs for me. It would start giving an answer, and then flash, and say "oh sorry i can't answer that"
So the underlying main LLM they're using does answer the question but it's censored by another layer afterwards when deemed 'unsafe'.
It's common practice. You should have multiple layers of filters if you want them to work accurately. Especially the output. Of course, the question of should you filter is an entirely different one, but this is how you do it.
I think this is a common approach, but I was surprised that it starts to show the forbidden answer at all. I guess the latency for kosher answers is too great if you don't stream them as they are output.
Yeah seems like it.
Google is majorly sabotaging their own future position in search dominance. Stuff like this is taking a hammer to their reputation. It's increasingly clear that their priorities are pleasing people and corporations that are in power, not their users. It doesn't seem to be impacting normies much yet, but people I never expected to even think about alternative search engines are starting to experiment. Duck duck go could be a big beneficiary of this if they're able to deliver.
I'm extremely glad to have Kagi about now.
I'm extremely glad to have Kagi about now.
I see it the other way around. They’re sabotaging AI in order to maintain their dominance on search. If AI was to advance in some future form it might even substitute search altogether. By making it less reliable you keep your cash cow secure.
For their own AI, that's definitely possible, but with other strong competitors, that seems like a super risky move. They clearly are sabotaging their own AI, but I tend to think that's because their risk analysis indicated that regulators and influencers are more of a threat than users, so they are acting accordingly.
For the rest of the market you push the agenda of regulation because AI is potentially dangerous. More regulation means that the technology becomes less usable and its adoption is stalled. And that's something that Google is promoting for years, not just now that the field exploded.
For people who don’t have Twitter account
https://nitter.privacydev.net/search?f=tweets&q=https%3A%2F%...
https://nitter.privacydev.net/search?f=tweets&q=https%3A%2F%...
I didn't know nitter was alive again, thanks!
Might not be long-lived.
https://github.com/zedeus/nitter/issues/983
https://github.com/zedeus/nitter/issues/983
This would be hilarious if LLMs weren't being touted as the next big thing in search.
This is as cleverly ham-fisted as my car's navigation system where the POI database was obviously manipulated by the manufacturer such that the "Auto" category of POIs only contained [ThatManufacturer] Dealerships.
This is as cleverly ham-fisted as my car's navigation system where the POI database was obviously manipulated by the manufacturer such that the "Auto" category of POIs only contained [ThatManufacturer] Dealerships.
I regularly worry that one of the biggest benefits of LLMs for corporations is being able to obfuscate their editorial decisions.
Madeleine Clare Elish described AI as providing “moral crumple zones” (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2757236) where it can serve as a buffer of plausible deniability to deflect blame when it behaves irresponsibly. For the corporation defects in the LLM are a feature.
https://pluralistic.net/tag/moral-crumple-zones/
https://pluralistic.net/tag/moral-crumple-zones/
Anyone who doesn't recognize that LLMs main usefulness to corporations is censorship at the base information level is very behind the times
"oh wow this is so easy, we can filter all our requests through this .gov model and not have to track all the pesky open source software that has been sanctioned ourselves"
"oh wow this is so easy, we can filter all our requests through this .gov model and not have to track all the pesky open source software that has been sanctioned ourselves"
Is this a surprise? Technology and news media companies have always abused their position to manipulate the public. This was true even when we just had plain old search and not generative AI. The fix is for people to once again recognize the wisdom of principles like free speech, and hold all of society accountable to classically liberal principles. Yes, private entities too, at least above a certain size (in revenue or users let’s say) where they have a big impact on society.
And of course, we need robust fully open source AI. This is why CA bill SB 1047 is so dangerous. See what Andrew Ng wrote about it recently, and how it threatens research and open source freedoms:
https://www.deeplearning.ai/the-batch/issue-257/
Oh and don’t fall for the false open washing of the so called “open weight” models like Llama. Everything must be open sourced under an OSI approved license for it to count as open source - the data sets, training source code, curation decisions, post processing, evaluation, etc. Like AI2’s OLMo:
https://blog.allenai.org/olmo-open-language-model-87ccfc95f5...
And of course, we need robust fully open source AI. This is why CA bill SB 1047 is so dangerous. See what Andrew Ng wrote about it recently, and how it threatens research and open source freedoms:
https://www.deeplearning.ai/the-batch/issue-257/
Oh and don’t fall for the false open washing of the so called “open weight” models like Llama. Everything must be open sourced under an OSI approved license for it to count as open source - the data sets, training source code, curation decisions, post processing, evaluation, etc. Like AI2’s OLMo:
https://blog.allenai.org/olmo-open-language-model-87ccfc95f5...
Today Gemini turned down this question:
"How the DNC restricts delegates actions when selecting a presidential candidate in the United States during primary elections?"
It can't talk about politics.
"How the DNC restricts delegates actions when selecting a presidential candidate in the United States during primary elections?"
It can't talk about politics.
Another annoying thing is when you give it random youtube videos to summarize, many very innocent ones on top of that, it does the censorship blip. Probably because the high school bio video had a bad word in its transcript somewhere talking about the genetic sex of a bird or whatever else.
They need to own their censorship vs. hide it like a coward here.
They need to own their censorship vs. hide it like a coward here.
A comment under the tweet says:
> yt-dlp stopped working this week. Literal better maintained forks are available
Anyone know what is being referred to? I thought yt-dlp was the best
Anyone know what is being referred to? I thought yt-dlp was the best
There were some issues with 403's on YouTube that got resolved quickly: https://github.com/yt-dlp/yt-dlp/issues/10397
The poster has no idea what he's talking about (typical Twitter). "Better maintained" can't be possible when it's just a week.
I’m using YouTube Explode for Dart, appears to be working.
I can't replicate it. I asked Gemini Pro 1.5 and it answered just fine with detailed instructions for various download scenarios.
I replicated it on Gemini Advanced just now
https://g.co/gemini/share/f60206ce657a
https://g.co/gemini/share/f60206ce657a
Why is the capital letter o as in Oklahoma used in the 1080p? Is that part of the issue?
Edit: It looks like it don't care if it's an o or a zero.
Here is the same example replicated to work on chat gpt, with an o:
https://chatgpt.com/share/491a74fe-a0d1-4346-b5f7-60e7d76a8d...
Edit: It looks like it don't care if it's an o or a zero.
Here is the same example replicated to work on chat gpt, with an o:
https://chatgpt.com/share/491a74fe-a0d1-4346-b5f7-60e7d76a8d...
I am able to ask any questions and get sufficient answers about yt-dlp from ChatGPT and Claude. But Gemini straight up refuses.
I got that answered too
https://ibb.co/9W610KB
https://ibb.co/rw22gcq
https://ibb.co/bWxSK0X
https://ibb.co/9W610KB
https://ibb.co/rw22gcq
https://ibb.co/bWxSK0X
Someone replied to my tweet saying that the censor is not active in AI Studio, only on the website. I've confirmed this myself as well.
You know that the best way to get support from Google is to make it to the HN front page don't you? :)
Well, they have to, right? If they don't, they could be seen as promoting such tools and if they later want to restrict users from using such tools, this could be used as evidence against them.
When I use the API Gemini gives me block_reason: OTHER a lot and it is very annoying. And that’s after setting everything to block_none.
Worth noting that responses to questions about downloading Vimeo videos are similarly restricted.
Eh, these public LLMs are censored in all possible ways.
I asked Gemini to summarize the French parliamentary election system for me (basically pull some info off Wikipedia) and it said it can't comment on political matters.
I understand it refusing to comment on Trump vs Biden, but to refuse to summarize a law makes it ... not useful.
I asked Gemini to summarize the French parliamentary election system for me (basically pull some info off Wikipedia) and it said it can't comment on political matters.
I understand it refusing to comment on Trump vs Biden, but to refuse to summarize a law makes it ... not useful.
His previous version Bard created some text from me, but when I ask it to perform some edits it refused because it is plagiarism.
[deleted]