> I went out to enjoy the Half Moon Bay 4th of July parade, occasionally checking in and prompting the next step for Fable from my phone.
This intensification of work will not be good for workers’ health. Like, put your phone down man. You can’t be modeling this behavior to young people.
Further, the intensification of work is probably not even good for productivity in the long term. This periodic half-thinking about things without stepping away from the problems you are working to solve will lead to more half-assed solutions. Ideas need room to breathe and dedicated focus.
The AI labs are racing to create a moat out of trillion-parameter models and the GPUs that can run them. The problem is this is the wrong architecture for most AI inference use cases. On-device inference is where this is going, clearly Apple believes this too. So Zitron is entirely correct about this AI datacenter build out being a boondoggle with no ROI.
No, they need to ditch drive letters first. The NT kernel and NTFS don't even require them (I used to mount disks without drive letters back in the NT 4 era). They just don't care enough to get rid of this annoyance.
Discusses the concept of Market Socialism, which is a hybrid system meant to avoid the worst aspects of both Socialist and Capitalist systems, while putting the goal of human fulfillment at its center.
It's actually a bit faster than that now it seems, about 112 tok/sec.
Configuration:
Gemma 4 31B Instruct Q6K
Context size 40960
LM Studio 0.4.13+1
Metal llama.cpp v2.14.0
LM Studio MLX (Apple M5) v1.6.0
Here are my results:
prompt eval time = 32545.36 ms / 5625 tokens ( 5.79 ms per token, 172.84 tokens per second)
eval time = 20227.99 ms / 310 tokens ( 65.25 ms per token, 15.33 tokens per second)
total time = 52773.35 ms / 5935 tokens
This was for interacting with a local MCP service, running a tool that returns a ~20KB text file to the agent to add to the chat context.
I'm seeing about the same number of tokens/second on an M2 Ultra that I have access to (also with 128GB of memory).
This is surely apples-to-oranges to the OP results (and I don't spend a great deal of time benchmarking these things, so my methodology might be lacking), but it's interesting seeing okay performance for a top open model. For most use, however, I find Gemma 4 26B A4B (Q6K) to be good enough (esp. for MCP calling) and much much faster (~1,200 tokens/second).
Not sure where 40 tokens per second is coming from. I’ve seen 95-100 tokens per second on M5 Max 128GB running Gemma 4 31B. I’ve done experiments where it is faster than Claude Opus 4.5 for the same prompts.
Ultimately, information is a public good: it is non-excludable (you can’t stop
people from using it) and it is non-rival (we can all use it at the same time). Public goods are often very useful, and because they are non-excludable and non-rival, ultimately can’t have a market-based business model. I would class open-weights AI models as public goods, and would support government expenditure to produce them.
Calculating hourly costs for these models makes me think that the decision of when to hire an SWE vs. increase use of AI may follow a similar pattern to the decision to use cloud compute vs. on-premises. I don’t cost $120/hr (incl. fringe), but my employer pays my salary all year long, no matter if I am working or on vacation. Whereas if they use an AI model to do the same work, they may be happy to pay $120/hr or more, since they may only use the model for a small fraction of 2080 hours per year, so they’d still save money, and not have a messy human to deal with.
If you start buying minis, then you need to house, power, and cool them. So you are building a mini data center. If you are building a small data center, economies of scale will drive you to want to build larger and larger. However, this gets expensive and neighbors tend to not like data centers (for good reason). To me this seems like asymmetric warfare against hyper-scalers.
I recently stopped using Backblaze after a decade because it was using over 20GB of RAM on my machine. I also realized that I mostly wanted it for backing up old archival data that doesn’t change ever really. So I created a B2 bucket and uploaded a .tar.xz file.
I’ve recently switched from Minio and Localstack to Garage. For my needs (local testing) Garage seems to be fine. It’s a bit more heavyweight and capable than I need now, but I like that it may give me the option of having an on-premises alternative to S3-compatible stores hosted in the cloud. The bootstrapping is a pain in the ass (having to assign
nodes to storage and gateway roles, applying the new roles, etc). It would be great to be able to bootstrap at least a simple config using environment
variables. However, now that I have figured out the quirks of bootstrapping, it just works (so far; again, I’m not doing anything complicated).
I think the major reason for the aggressive price point of the Neo, and for not raising RAM and SSD upgrade prices in the MBP much, is that Apple is willing to give up some hardware margin to have more devices to sell services to. Unless I am mistaken, services have been key to Apple’s recent revenue growth. This isn’t a bad thing at this point, but could auger poorly if they foolishly chase recurring revenue at the expense of hardware quality (their software quality has already slipped in recent years).
This intensification of work will not be good for workers’ health. Like, put your phone down man. You can’t be modeling this behavior to young people.
Further, the intensification of work is probably not even good for productivity in the long term. This periodic half-thinking about things without stepping away from the problems you are working to solve will lead to more half-assed solutions. Ideas need room to breathe and dedicated focus.