I think the parent comment stands - I’ve asked Opus to do a review of DeepSeek’s test suite and told it a couple things I wanted it to look for, and it did a very thorough review of the tests and picked out a reasonable number of gaps and tautological tests. It’s a mix of prompting/instructions, the agent harness, and random chance. The model is not wholly irrelevant but IMO increasingly so.
I currently parse the TOC for chapter markers, then walk the spine between markers to ensure everything is captured regardless of what the TOC says. I think it is in the spirit of what epub readers do and in line with the spec.
The eval agents perform the extraction and run a comparison of chapter/word count against the epub original. They also parse the extracted text to look for inconsistencies. The quality criteria have been built up iteratively and issues backed out into unit tests. The eval runs aren't part of my standard unit tests but a one-off, heavy lift process that I run periodically against the epub collection I already had and a bunch I built up for this project. I've also just done a ton of manual validation. There may still be edge cases.
Thanks for checking it out! The Alice link is a generated m4b, so it’s sample output that you could likely play back directly in your browser, although that may not have the cover display or chapter listing. I recommend Apple Books or Plappa on iOS, or Audiobookshelf on Android. For desktop, VLC still rules the day for general media playback, but Audibly is a Windows app that has good reviews.
There are many public domain epub files online which you could use as input. One excellent source is https://www.gutenberg.org/
I checked out your product - nice! I think we have a similar approach for a different audience. EbookAloud is basically just a two-click conversion system more geared towards consumers than content creators. Upload an epub, get an audiobook. Users will have to live with mispronounced names, etc.
Content parsing and chapter alignment were also the better part of the work here. Laying out an epub on screen is straightforward. Extracting text by chapter without duplication or elision took a lot of iterations. I could not ultimately follow the epub spec recommendation to traverse the spine, but had to rely on the TOC to drive extraction or fallback to some simple heuristics if none was present. It’s still the biggest risk area in the code and why I added a detailed chapter breakdown before asking for payment. I’ve pushed a lot of content from a wide range of sources through the code for manual and semi-automated inspection and decided it’s good enough to go live.
A nit: did you go from Opus 4.5 to Fable? One of the big questions in my mind is how much of a real change Fable is over the existing models. Opus 4.5 -> 4.8 was also a major capability increase.
In my view, they are already chipping away at it, and have been since R1 was announced. This is the first commercial non-US tech product I've used heavily. The quality is incredible, I don't need Opus for most of my work on personal projects, I've used DS+OpenCode to create full-blown products in fractions of the time it would have taken me solo.
When I was 7 or 8 a friend and I crimped the heads off strike-anywhere match sticks, wrapped them in foil, and struck them with hammers and rocks. They were quite loud, one even set off a sound-activated toy inside the house.
I make no claims as to how well adjusted I am, but I've at least survived 40-odd years of life since then.
I don't think anyone has a firm grasp on actual inference costs -- including the research and training that has gone into those models. We've got near-frontier capabilities from open source models from China at pennies on the dollar compared to US big tech rollouts. OpenAI and Anthropic are heavily subsidizing their inference -- no wait, they are charging the most they can get away with before going public. Where is the truth?
Nothing new. I used to work with .NET and went to some meetups and conferences. There are some hardcore Microsoft fanboys out there. Didn’t even mix the kool-aid, ate it right from the packet. They only know MS products and seem scared of anything else.
Maybe not your typical HN crowd but marketing absolutely works on developers.
I don't disagree about the probability, but the current frontier models are not completely useless for writing even in areas where I have significant knowledge. I would not have said that a year ago. You have to watch them like a hawk -- they are good at spitting out plausible sounding nonsense that is hard even for an expert to discern. But the dice roll going on behind the scenes is continually more biased towards being correct/useful than not.
I also think this is an interesting topic of research. I have had lifelong issues with chronic pain and always felt like diet influenced it, but I’ve never been able to isolate a single factor. I’be stopped cooking with low smoke point oils but it’s all guesswork at this point.
Your phrasing about politics was potentially ambiguous, so I asked for clarification. People can mean many different things when they say something is political.
That could be a valid observation for n=1. There could be any number of reasons seed oils cause inflammation for you but do not the general populace, or not at numbers large enough to offset recommending them as a general rule.
The politicization is coming directly from the Trump administration, as the article states - making spurious claims and eliding the science that backs up the contrary conclusions. Did you have some other idea of how this is being politicized?
It really doesn't have to come for everything to feel like it's taking everything. If it eliminates 10% of white collar jobs over the next decade, the impact will be felt everywhere.
AFAIK all these companies have SOTA or near-SOTA models available under enterprise licenses. AI companies are not interested in your secret sauce, they are trying to capture the SDLC wholesale.
I'm skeptical that B is fully possible. You can create a PQ fork of bitcoin but you cannot automatically bring vulnerable wallets along - and there are a lot of vulnerable wallets, especially from the early days. There's a catastrophe ahead for bitcoin with an apparent probability of 1.0. That's hard to account for in this scheme.