Stromgren

170 karmajoined 13 лет назад

comments

5 часов назадdiscuss

Of course there is. It’s not about the _amount_ of files or how many percent of them. I might have 1000 files that I’m fine having the LLM read and then some that it really shouldn’t. The problem here is plainly uploading your whole directory without prompting for permissions to read them - even if you explicitly set up permissions for read tools.

5 часов назадdiscuss

There’s a hell of a difference between a tool that asks my permission to read a file to make it part of a prompt and a tool that packages up my whole working directory and sends it to Google Cloud Storage.

5 часов назадdiscuss

The things you allow the LLM to read will obviously be sent as part of a prompt. You can control that though. Reads are tool calls and you can configure permissions for that or be asked every time the agent wants to read something.

This is straight up just uploading your whole working directory. Not as a LLM prompt, but to a Google Storage.

6 часов назадdiscuss

No I disagree. A harness reading a file is a tool call and it happens locally, which means that I can control it. I can configure that I need to permit any file reads and now I _should_ have control of what is sent. The difference between that and silently uploading my entire working directory in the background is miles apart IMO.

I understand that one should think carefully about how they work with a non-deterministic tool, but this if different completely. This is xAI just choosing to upload and store everyone’s directories - with full git history.

8 часов назадdiscuss

This was posted on HN yesterday: https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75f...

If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.

25 дней назадdiscuss

I have a last name that only my family holds in Denmark (which means me and my brother by now), yet it managed to tag me a Professional Football Player, MP for the Socialist Party and Founder of a Sleep Mattress Company.

I do like the visuals though.

29 дней назадdiscuss

Why _too_ late? Late, yes, but you’re implying that some line was crossed.

3 месяца назадdiscuss

I run a SaaS business on the side of my job and have been for 15 years. There’s a million questions I’ve never had the time to dig into although the data was there. Retention cohorts, free to paid tier conversions, subscription upgrades/downgrades and so much more. Just this week, I decided to just let an agent have access through psql and go nuts, writing all analysis to markdown files. Reading through it, there’s a few things it misunderstood and as a result, some of the analysis was flawed, but all in all I’m honestly mindblown. It would have taken me months to write queries and even just coming up with frameworks of how to think about these metrics.

3 месяца назадdiscuss

That’s how I’d want it to be honestly. LLMs are tools and I’d hope we’re going to keep the people using them responsible. Just like any other tools we use.

4 месяца назадdiscuss

I’ve been involved in building a system that reads structured data from a special form of contracts from a specific industry. Prices, clauses, pick up, delivery, etc. A couple hundred datapoints per contract. We had many discussions around how to present and sell an imperfect system. The thing is, the potential customers are today transcribing the contracts manually and we quickly realized that people make a ton of mistakes doing that. It became obvious when we were working on assertion datasets ourself. It’s not a perfect system and you have to consider how you use the data (aggregating for price indexing for instance), but we’re actually doing better than what people are achieving when they have to transcribe data for hours a day.

4 месяца назадdiscuss

Agreed. I’ve used their platform to train smaller, specialized models. Something I could have done in Codelab or some other tool, but their platform allows me to just upload a training set and as soon as it finishes I have a hosted model available at an endpoint. It obviously has some constraints compared to running the training yourself, but it also opens up the opportunity to way more people.

4 месяца назадdiscuss

I saw the cost of the three downed planes somewhere else and thought the price was huge. Now I see that it’s comparable to “First Tomahawk salvo”.

5 месяцев назадdiscuss

Yea it wasn’t meant as a counter argument either.

To be honest I haven’t even looked at competitors for some years. I guess one drawback of using third-parties for such a big part of the responsibilities is the lock in. The benefits of switching would have to be rather big for me to put in the effort.

5 месяцев назадdiscuss

I find Stripes fees excessive too, but I don’t think I’ll ever switch. I’ve been running a small SaaS product on the side of other work for >15 years and if it taught me one thing, it’s that I need to reduce the things I have to maintain, reduce manual work, reduce the things that can go wrong. There’s nothing worse than having to fix a bug in a codebase you haven’t touched for a year and possibly in a feature you haven’t touched in many years. I simply love that Stripe handles not just the payment, but the payment application, the subscription billing, the price settings, the exports for bookkeeping. I’ve had a few instances where my site was used fraudulently to check stolen credit cards and it was quickly flagged and I could resolve it with Stripe. I’m sure someone can mention alternatives and I’m sure that I could build something that would work myself, but they keep a big part of what it takes to run the business out of my mind and I’m willing to pay for that.

5 месяцев назадdiscuss

I very much appreciate this take. I will say though that I’ve had experience myself where using coding agents lead me to what I’d consider (in your terminology) a better mapping between information and code. Not because the agent was able to do things better than myself, but because, as my project grew and I got wiser on how to best map the information, it was incredibly fast for me to change the code in the right direction and do refactorings that I otherwise might not have gotten around to.

5 месяцев назадdiscuss

I use Tidewave as my coding agent and it’s able to execute code in the runtime. I believe it’s using Code.eval_string/3, but you should be able to check the implementation. It’s the project_eval tool.

In my experience it’s a huge leap in terms of the agent being able to test and debug functionality. It’ll often write small code snippets to test that individual functions work as expected.

5 месяцев назадdiscuss

It depends where I look. Among colleagues and tech-native friends, I feel like there’s healthy skepticism as well as the excitement about new tech. On the other hand, all the investment podcasts that I’ve been following for years are nothing but ignorant AI hype and reciting articles about how all the jobs are about to disappear. I guess the people who doesn’t make firsthand experiences are not leaving the hype yet.

5 месяцев назадdiscuss

And if you make someone 3x faster at producing a report that 100 people has to read, but it now takes 10% longer to read and understand, you’ve lost overall value.