1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.
2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.
The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.
I think this is a reasonable point, but a better comparison might be to nuclear energy. I think the frontier labs sincerely believe that AI can be developed at great benefit to humanity, and they clearly want to lead that push, but they also sincerely believe there is a real catastrophic risk.
> Meanwhile, on the other side of town, five miles away in Old Town, just 23 seconds after San Police Officers in Golden Hill tried stopping the suspected carjacker, a Flock Automated License Plate Reader captured a photo of a red Alfa Romeo driving on the 2200 block of Moore Street.
> Their tort claim notes that the path the men took to the cigar lounge passed by several other Flock cameras, which could have corroborated their story, as well as the location data on their cell phones.
It seems like if the police actually looked at the Flock data it would have exonerated them?
It's more nuanced than this. Peter Scholze said in response to this declaration:
> The goal of mathematical research is human understanding of mathematics, and so mathematics can only thrive in a community of human mathematicians. It is crucial to preserve this communal spirit. [0]
Terence Tao has also talked about the requirement for a mathematical proof: along with generation and formal verification, there is an important step of "proof digestion"
> understanding the essence of a solution, placing it in context with previous literature, summarizing and explaining it effectively, and gaining insights on other related problems and topics [1]
Cool! If you can point to any examples of those types of workflow configurations I’d be super interested. For example, to have a team of agents review a PR and iterate on it until all requirements are met including UX, security and product functionality goals. If they could “converge” to a solution like workflows seems to be designed for that would be amazing.
Thanks to you and the anthropic team for developing such exciting tools! The blog post seems to position workflows for “breadth”: generating fixes / refactors against large code bases. What about for “depth”: developing specific new features and functionality end-to-end? I’ve struggled to make this work reliably using the current experimental agent teams. Does this replace or augment that functionality?
> Schmidt offered a similar message to graduates: Their fear is rational, but they have the power to shape how AI develops.
This doesn't sound like being baffled by it. It sounds like they are trying to shake the students and say: "fine boo, but you need do something about it." You can't just wallow and complain about it. I mean you can but it's a path to failure.
I agree, vibe coding does not have quality gate checks at each stage, while agentic engineering does. Dev teams get into trouble when they try build to build without a proper process of design, tests, and reviews. This was true before agentic coding, but it's especially true now. The teams that understand how to leverage agents in this process are the ones that will be most successful.
Please explain how this tactic relates here. In this case we have the AI companies saying this technology is potentially very harmful, in fact existential. This seems the complete opposite of what big tobacco did.
What I meant by
> People seem unable to make up their mind if AI is very dangerous or is it not.
Is that the article says 2 contradictory things:
1. AI companies are misleading us when they say their tech is dangerous and people should be afraid.
2. AI is currently very dangerous and people should be afraid.
Anecdotally, people on the internet (including HN), seem unable to agree on whether AI is real or overblown "hype".
One of the points I was trying to make is that the statement:
> trying to coerce people into using your product out of fear
is nonsense.
Everyone agrees that there are legitimate reasons to be fearful of this technology, this is not a fabrication, but we need to figure out how to proceed in a safe and constructive way.
What "coercion" is occurring here? Either you find the technology valuable and you want to pay for it, or you find it not useful (or worse harmful), and you do not want to pay for it.
Maybe another way of putting it, what do you think the frontier AI companies should do in this situation? It seems that being straightforward with the dangers is correct thing to do, and probably being overly cautious is prudent. You could go further and argue they should slow down or stop development, but that is something that the govt should impose, we should not expect or trust the companies to do this themselves. Ironically, in the Anthropic / Pentagon case, we have Anthropic trying to pump the brakes and put up guardrails while the govt wants to go full-steam ahead with autonomous warfare.
The other issue with slowing down / pausing development is it requires an unheard of level of agreement, even with companies in China, or else it will probably not be effective. You could argue this is not even possible at this point.
> Why do AI companies want us to be afraid of them? ... According to critics, it benefits AI companies to keep you fixated on apocalypse because it distracts from the very real damage they're already doing to the world.
People seem unable to make up their mind if AI is very dangerous or is it not. I think what the AI companies and this author agree on, is that this technology is potentially extremely dangerous. AI impacts labor markets, the environment, warfare, mental health, etc... It's harder now to find things which it will not impact.
So if we agree that AI is potentially dangerous, it makes the title question moot: Both AI companies and this author want people to be aware of the dangers that AI poses to society. The real question is what do we do about it?
The nuance here is that AI can be incredible positive as well. It's like the invention of fire, you can use it for good or bad, and there will be many unintended consequences along the way.
We could legislate and ban AI tech. People have proposed this seriously, yet this feels completely unrealistic. If the US bans AI research, then this research will move elsewhere. I think it is like trying to ban fire because it's dangerous: some groups will learn to work with fire and they will get an extreme advantage over those groups that don't. (or they will destroy themselves in the process).
So maybe instead of demonizing the AI companies, we have a nuanced debate about this tech and propose solutions that our best for our society?
If feedback from this incident is in its context window, it is highly unlikely to make this same mistake again. Yes this is only probabilistic, but so is a human learning from mistakes. They key difference is that for a human this is unlikely to be removed from their memory in a relevant situation, while for an agent it must be strategically put there.
I see a lot of people struggling to work with agents. This post has a good example:
> “you can’t be serious — is this how you fix things? just WORKAROUNDS????”
If this is how you’re interacting with your agents I think you’re in for a world of disappointment. An important part of working with agents is providing specific feedback. And beyond that making sure this feedback actually available to them in their context when relevant.
I will ask them why they made a decision and review alternatives with them. These learnings will aid both you and the agent in the future.
Yes memory is a key part. I’m using sqlite and markdown files. Sqlite is so it can easily search its conversation and thoughts by time and full text search. I actually always put the last 5 hours of so I’d conversation into the context. Markdown is for more standard agent instructions, skills, and lessons
Of course, you need to be careful about what access you give to your agent. I gave my agent its own email, and I can forward it emails if I need it to read anything in my inbox.
Everyone will have their own threshold for what type of access they want to give their agent. some people will give it access to their personal email, bank account, etc, but I wouldn't recommend it yet! But I bet in a couple years this will be standard practice.
Think about what you would want an assistant to do. You can teach it do basic tasks using any available API, but then you can give it feedback so it improves.
For example my agent can control home automation via Home Assistant or any other API. My agent contributes to websites and open source projects. When you give it feedback it updates its skill files.
It checks and answers email, can receive and place phone calls, and do general research and monitoring online. I was even playing around with it to create music. The list of things to try is limitless.
I think just like LLMs, people get discouraged when it doesnt one-shot a problem. This technology thrives on feedback. It will make mistakes, your job is to make sure it learns from those mistakes so it doesnt repeat them.
Well this is easy enough. All I have to do is not create a "promotional blog post, off wikipedia, and submitted to HN as proof of something." Consider it done!
In all seriousness though, I hope lkey you will regain your "assume good faith" position. Without that HN is just like any other site on the internet. And I apologize if I caused you to question that.
Your instinct is wrong here. I would also highly discourage you from violating "Assume good faith". Without that everything devolves. I am still assuming yours.
> It’s all just nonsensical hype
And he’s referring to completely real and reasonable warnings in anthropic blog posts about the exponential rate of AI development.