joshmlewis

no profile record

コメント

2 か月前議論

It's interesting they use output tokens as an eval because all tokens are not made equal. Even from model to model (like Opus 4.6 to Opus 4.7) the tokenizer can be different and it's no longer an apples to apples comparison. No one really talks about this but it directly affects stats like usage limits. Certainly comparing models between providers on an apples to apples comparison token wise is not a good test.

5 か月前議論

Speechify has been good for me although there might be better / cheaper alternatives I'm not aware of.

5 か月前議論

I think the OP was implying that it's probably already baked into its training data. No need to search the web for that.

5 か月前議論

"They" being the guy (Peter Steinberger) who created it as a personal project that he open sourced.

6 か月前議論

This is cool but as someone that's built an enterprise grade agentic loop in-house that's processing a billion plus tokens a month, there are so many little things you have to account for that greatly magnify complexity in real world agentic use cases. For loops are an easy way to get your foot in the door and is indeed at the heart of it all, but there are a multitude of a little things that compound complexity rather quickly. What happens when a user sends a message after the first one and the agent has already started the tool loop? Seems simple, right? If you are receiving inputs via webhooks (like from a Slack bot), then what do you do? It's not rocket science but it's also not trivial to do right. What about hooks (guardrails) and approvals? Should you halt execution mid-loop and wait or implement it as an async Task feature like Claude Code and the MCP spec? If you do it async then how do you wake the agent back up? Where is the original tool call stored and how is the output stored for retrieval/insertion? This and many other little things add up and compound on each other.

I should start a blog with my experience from all of this.

9 か月前議論

This just feels like the whole complicated TODO workflows and MCP servers that were the hot thing for awhile. I really don't believe this level of abstraction and detailed workflows are where things are headed.

9 か月前議論

This should not really be necessary and is more of a workaround for bad patterns / prompting in my opinion.

9 か月前議論

How big is your claude.md file? I see people complain about this but I have only seen it happen in projects with very long/complex or insufficient claude.md files. I put a lot of time into crafting that file by hand for each project because it's not something it will generate well on its own with /init.

10 か月前議論

It's also not a coincidence that Slack is neutering the ability to access channel history via the API very soon. With a very generous rate limit of 2 requests per minute I believe it was and a max of ~10 messages. This is already enforced for new marketplace apps and will apply to all apps starting in March according to their docs.

11 か月前議論

It is funny how it can be like this sometimes. I think a lot depends on coding styles, languages, prompting, etc.

11 か月前議論

Cursor

11 か月前議論

When it came out on Tuesday I wanted to throw my laptop out of the window. I don't know what happened but results were total garbage earlier this week. It got better the past couple days but so far with gpt-5 being able to solve problems without as much correction I'm going to use it more.

11 か月前議論

Whoosh, it went right over my head.

11 か月前議論

The data is made up, the point is to see how models respond to the same input / scenario. You're able to create whatever tools you want and import real data or it'll generate fake tool responses for you based on the prompt and tool definition.

Disclaimer: I made PromptSlice for creating and comparing prompts, tools, and models.

11 か月前議論

Where'd you get 720 from?

11 か月前議論

Did I say GPT-5? I said o3. :) That was a rebuttal to you saying you have never needed to add your key to use an OpenAI model before.

11 か月前議論

It's free in Cursor for the next few days, you should go try it out if you haven't. I've been an agentic coding power user since the day it came out across several IDE's/CLI tools and Cursor + GPT-5 seems to be a great combo.

11 か月前議論

It does seem to be doing well compared to Opus 4.1 in my testing the last few hours. I've been on the Claude Code 200 plan for a few months and I've been really frustrated with it's output as of late. GPT-5 seems to be a step forward so far.

11 か月前議論

It does really well at using tool calls to gain as much context as it can to provide thoughtful answers. In this example it did 6! tool calls in the first response while 4.1 did 3 and o3 did one at a time.

https://promptslice.com/share/b-2ap_rfjeJgIQsG

11 か月前議論

I've been testing it against Opus 4.1 the last few hours and it has done better and solved problems Claude kept failing at. I would say it's definitely better, at least so far.