1. 2022 A non-profit literally quote "to better all of humanity, not shareholders"
2. 2024 For profit, but "we will train biases out of our models and make sure our AI is safe to use"
3. 2025 We'll make sure it doesn't take over the world, with like a 70% confidence interval
4. 2025 The mecha-hitler inflection point
5. 2026 Our new model is so terrifying it will destroy all of security and hack the chinese, we can't let the chinese use it
6. 2028 (projected) Our new model requires so much energy that 30% of the elderly population will die without AC, but it will be stronger than the chinese models and let us destroy china
7. 2030 (projected) Our new model will triumph over the mecha-hitler dictator model, and will be a benevolent dictator that only demands 60% of all energy produced, not 98%
Weirdly, the existing first party solutions around denying commands don't seem to help here.
Often enough, when one of the agents prompts for running "sudo", and I reject it, it will do what looks very much like malicious exploration to figure out how to handle things anyway, including once hijacking a separate shell's pty where I did have a valid sudo session already in order to execute some commands.
We don't yet have the capability to make these models behave in a consistent, deterministic, or safe manner yet, so a first party solution isn't even necessarily that much better. Especially if it gives a false sense of security.
The difference is that git is a traditional programming tool which executes deterministically.
agents are not deterministic tools, they're not sandboxes or container runtimes or languages with capabilities models.
They're a way to run arbitrary commands.
It would be like saying that "xterm" should have a ".xtermnoexec" list of commands you can't run, or that VLC should have an option for actors it won't show.
terminals run shells which run commands, it's not really deeply aware of what commands your shell ultimately run, and it's not in xterm's job to setup a sandbox and strip out executables.
VLC displays pixels, it's not up to it to figure out if those pixels are a certain actor.
codex pipes text and tool calls back and forth between OpenAI's servers, and it barely understands what that text and those tool calls are, and especially if a given tool touched a file. If you want VLC to not display an actor, you need to add a layer on top of VLC to stop it displaying a list of movies. If you want codex to not display a file's contents, you need a layer on top of codex to prevent it going near that file.
You can do this now: change the file permissions such that the user you run codex as can't read them, or run codex in a container without those files mounted.
If you don't do that, the agent will be able to incidentally upload them. What if the model runs "rg foo", and one of those files contains the string "foo"? It uploads the tool output, which includes the file contents.
And so, the only solution is to make it so the codex process is unable to access those files, hence using a container, or unix permissions, or deleting the files. Which you can already do.
I imagine this isn't resolved primarily because people expect it to apply to bash tool use, not just the "read" and "edit" tools, and people also expect those files to still be accessible i.e. if the agent invokes "make", which makes it impossible to solve perfectly.
As Iran is showing, the federal government is worried about nukes. I feel like a modern 2nd amendment should ensure every US citizen has a right to up to three (3) nuclear weapons.
If someone's account gets lost or hacked, the person with the most incentive to own that account is usually the original owner, so just give it to whoever is willing to pay the most, problem solved. We can call it "proof of stake", where you always stake a certain amount to keep owning your account, and when contested, whoever stakes the most gets it.
Poor people don't deserve rights on the blockchain anyway, it's not like they can afford the transaction fees, if they didn't want their account stolen they should have tried being rich, or buying into nearer the top of the pyramid.
Don't worry about people who pass away or lose internet for an extended period, we'll deal with that in v2, when we get "proof of death" and "proof of internet disconnectivity" on the blockchain somehow.
That the server can figure out that two computers in the same house are different since your laptop and phone no longer share the same ipv4 address but instead have two ipv6 address?
Security? NAT is not a firewall, you need a firewall, and switching to IPv6 does not remove your firewall.
Before IPv6: The server gets "1.2.3.4:56789" for your device. After IPv6: the server gets "1:2:3:4::56" or whatever for your device. In either case, if the server makes a connection to 1.2.3.4:56789 or 1:2:3:4::56, your router sees the packet and firewalls the connection. Cool.
Want to give me a concrete example of where IPv6 is hurting my privacy or security, because I've been using it for over a decade with zero mishaps, zero privacy issues, zero security issues (to my knowledge at least)
The heat isn't wasted, it has a significant impact on flavor.
Extraction of flavor etc from coffee grounds changes based on the water temperature. You can easily experiment with this by taking some coffee grounds and trying to brew with various temperatures, and noticing what flavors you get out.
There's a reason cold-brew needs hours and hours to extract a good coffee flavor while regular espresso or pourover is done in a matter of minutes.
The equivalent for IP addresses to what they offer would be closer to running a BGP router or ISP, or generally contracting with network engineers for your data-center's networking.
If you want to run an ISP or AS, believe me it will cost you a decent chunk of money.
I think the evil valet risk isn't real, but this could be part of a chain-of-attack in some scenarios, mainly rental cars.
Like, sure, if you're just going to use it to spy on the user, you could also rent a rental car and leave a recording device under the floormat, or hidden behind the head unit, or whatever.
But if you have an Apple Carplay exploit, where someone tethering their phone to the car can be compromised, renting a car and flashing a malicious OS to exploit the phones of people who come after you could maybe be a real attack. It's kinda hard to get people to otherwise connect to a malicious infotainment system with carplay, so if you have an exploit that requires that, this could be part of it...
Except actually, no, if you have a carplay exploit, just rent the car, and rewire the USB port to go through a flipper zero or whatever and don't bother reflashing the car's software, that's just as easy.
... So yeah, I guess I agree with you, even in the rental car scenario, where this seems like it would be worst, your attacker might as well just hide something in the car instead of flashing the software.
AWS doesn't check if your credit card will be able to handle a $5k charge before letting you rack that up, and in fact AWS doesn't support setting any spending limit.
You just have to put in any valid credit card at all when you sign up, use AWS, and at the end of the month you'll have a bill. At no point does your credit card limit or a spending limit enter into things.
Curiosity is great, but agents do not learn, and telling an agent "scan the darkweb" is a way to avoid learning about the details, rather than to dig into things more deeply.
If instead they had just used a chat interface to ask "Where should I start", they'd more likely have got a link to the DN42 docs themselves, read them, and not hallucinated things like "color".
They might have asked "how much will this cost?" if they had to spin up the ec2 instances themselves, on advice from the agent.
The way you learn something is by doing it the manual way first.
You learn memory management by writing your own allocator, and then after that you go back to using malloc like normal, but with knowledge of how it works. You don't learn memory management by telling an agent to write an allocator.
Using an agent to give you links and point the way aids in learning, using it as an autonomous tool to do "gruntwork" you don't yet know how to do yourself will get in the way of learning.
Curiosity is beautiful, using agents to bother humans and avoid learning is somewhat less beautiful.
> money has to come from somewhere. Someone has to give it to you. So it you want to keep growing, you need someone who isn't paying for software, to start.
Right now, people pay all sorts of money for real interactions with real people, most notably friendship and dating.
Tiktok has done a good job at starting to disrupt this, but with AI and better VR technology, maybe we can finish the job and disrupt all human relationships, all romantic relationships, all friendships. It's a huge addressable market (all humans), and if even just 5% of all humans buy a $5 virtual coffee (free to produce, pure profit) for their AI partner each day, that would be a massive increase in software spend.
Once we hit "The Matrix", that'll mean software has nowhere left to go.
Apple set itself up for defeat in the server and developer marketplace as soon as they decided macOS was proprietary code.
Why would any serious developer use closed-source code they can't debug and modify? Especially for a production server?
It's the same reason no serious developers or hackers use macOS, like part of the point of being a developer is being able to dig into the code at any layer and debug and fix things.
Children cost hundred's of thousands of dollars to raise, and that doesn't even count the opportunity cost of your own career progression as you have to spend a year of sleepless nights and possibly have one or both parents reduce their work hours to care for the child.
The Japanese government already struggles to pay out pensions with its aging population, healthcare and pension costs are both rising, where do you propose the money for this comes from?
Should the government increase its already high tax rate from up-to-50% to up-to-90% and take money from the childless to give to parents?
Should the government replace your salary if you quit your job to raise a kid (since after-all that is a cost of the endeavor)?
If you're just talking about "giving birth", I assure you the cost to give birth is already close to free, the government already covers that, and various cities have local parent stipends which make it "profitable" in a sense.
But the real cost of giving birth is not the giving birth, it's the millions of yen you then need to spend over the next 18 years to raise and educate the child, not to mention the cost of possibly dying during childbirth.
Do you have any links to commits or changes that you think are "uncalled for"? Like, you say "he is making large and sweeping changes that are entirely uncalled for and cause breakage", so surely you have some examples?
As far as I can tell, most of the AI-assisted changes were security fixes and test-suite related, and I'm sure you can agree that both of those are normal maintenance.
Do you know of a single service at a single company that actually does that?
I know all of Gmail, every GCE service I can think of, every AWS service I can think of, Amazon.com, Netflix, and Github all do not page on just a single 500.
I know none of those are particularly "high performance" though. Curious where your experience is coming from.
I mean, on levels.fyi the 90th percentile in San Francisco is 500k. Most of HN is tech workers in the bay area.
The average person on hacker news is easily in the 90th percentile of software engineers, so yeah, I wouldn't be surprised if the most people here make somewhere around there.
The biggest confounding factor is the other fact that most HN users are startup founders living off ramen in Peter Thiel's garage, so that probably brings down the average some.
anubis only works against lazy scrapers, and at a cost to your users. I'd prefer people not use it.
Bot traffic comes from machines that usually have a lot of idle cpu (since they're largely blocked on network IO as they scrape a bunch of sites in parallel), so they can trivially solve the anubis "proof of work" challenge, save the cookie, and then not solve it again for that site.
The only reason scrapers don't solve it is if the developers were too lazy to implement it... and modern scrapers also do, codeberg stopped using anubis because modern scrapers were updated to solve it.
The "proof of work" has to be easy or else people on old cell phones couldn't access your site (since an old android phone would start to overheat and throttle trying to solve a challenge that would take a modern server even several seconds), and it also consumes your cell-phone user's batteries, which is a really precious resource for them compared to the idle cpu on a server.
2. 2024 For profit, but "we will train biases out of our models and make sure our AI is safe to use"
3. 2025 We'll make sure it doesn't take over the world, with like a 70% confidence interval
4. 2025 The mecha-hitler inflection point
5. 2026 Our new model is so terrifying it will destroy all of security and hack the chinese, we can't let the chinese use it
6. 2028 (projected) Our new model requires so much energy that 30% of the elderly population will die without AC, but it will be stronger than the chinese models and let us destroy china
7. 2030 (projected) Our new model will triumph over the mecha-hitler dictator model, and will be a benevolent dictator that only demands 60% of all energy produced, not 98%