HackerTrans
TopNewTrendsCommentsPastAskShowJobs

ianhmacartney

no profile record

Submissions

Agents Need Durable Workflows and Strong Guarantees

stack.convex.dev
3 points·by ianhmacartney·vorig jaar·1 comments

I reimplemented Mastra workflows with my own durable functions and I regret it

stack.convex.dev
1 points·by ianhmacartney·vorig jaar·1 comments

Customize server functions in TypeScript without middleware

stack.convex.dev
1 points·by ianhmacartney·3 jaar geleden·0 comments

comments

ianhmacartney
·vorig jaar·discuss
Agents rely on long-lived workflows, but when happens when they fail midway through? Here are the tools you need to manage correctness and reliability: transactions, idempotency, retries, durable functions, journaling, and state machines. The missing abstraction layer for agentic is durable workflows, which bring them all together.
ianhmacartney
·vorig jaar·discuss
I reimplemented Mastra’s agentic workflows with durable functions in Convex, and it was the wrong decision. Learn about three common integration strategies (reimplementation, API wrapping, and “blessed” plugin paths), along with my learnings along the way and reflections on what I’d do differently next time. TL;DR: Do less, do it smarter, and prototype faster.
ianhmacartney
·2 jaar geleden·discuss
One public comparison on latency is https://db-latency.vercel.app/

For comparisons, you can check out:

https://stack.convex.dev/convex-vs-firebase https://stack.convex.dev/convex-vs-relational-databases https://www.convex.dev/compare/supabase https://www.convex.dev/compare/mongodb

I'll save you more of a marketing pitch, since you seem to have enough of my pitching Convex in the article:) The bullet points at the bottom of the article should be a pretty concise list - I'd call out the reactivity / subscriptions / caching. To learn how all that magic works, check out https://stack.convex.dev/how-convex-works
ianhmacartney
·2 jaar geleden·discuss
Very much agree. The tools for various jobs differ widely. However, my current take is that full-stack is more about enablement of engineers vs. a specific product or technology. Things at the protocol level (ProtoBuf, OpenAPI specs, etc.), things at the framework level (fancy types with tRPC or Convex), things at the application level (dropping in full stack features by someone without deep expertise in both domains) all qualify in my book. And rather than a discrete "you are now a full-stack dev, my child" moment, it's about blurring the line, the same way that k8s lets some backend folks play SRE here & there.

Background. My career has spanned these in different ways:

Front-end: I started in iOS in 2010 (back then it wasn't called iOS yet - just iPhone), where I was a more pure frontend dev. The server was using python and Haskell and that team was the "cool kids" in my view. The backend/frontend contract was using ProtBuf over a socket (not WebSocket, which hadn't been standardized yet), so we really didn't work closely together, and no one was "full-stack".

Backend: Wanting to be a "cool kid" I joined Dropbox and eventually was the technical lead for Dropbox Previews, generating video, pdfs, images for user content using hundreds of machines, using a combo of Go and Python. I didn't work very closely with the myriad of teams using the previews for various products (their interface was the HTTP endpoints we exposed). One fun statistic: if you removed the cache and processed the full file for every user’s request, it would amount to processing over one exabyte of data per day.

"Full-stack" tbh my experience crossing the gap is only somewhat recent (last 5 years or so) where I did freelance / contracting for various startups and established companies. I brushed up on React, learned TypeScript, but still mostly focused on backends - which usually ended up overly robust for the level of operational expertise of the companies I was handing the projects off to: like handing off a full kubernetes setup for running some GPU machines and orchestrating some fan-in / fan-out pipelines for ML image processing, when I'm sure there were mostly-fine ML workflow companies I could have used.

Anyways this was much longer than anticipated but hope it adds some color
ianhmacartney
·2 jaar geleden·discuss
Author here- and yes as a disclaimer I work at Convex. As a caveat to that disclaimer, I pivoted my career to work here b/c I genuinely believe it moves the industry forward b/c of the default correctness guarantees, along with other things.

To this question here, some of the things that accelerate full stack development:

1. Automatically updates your UI when DB data changes, not just on a per-document one-off subscription, but based on a whole server function's execution which is deterministic and cached. And regardless if the changes were made in the current browser tab or by a different user elsewhere. Not having to remember all the places to force refresh when a user updates their profile name, e.g., makes it way faster. And not only do the Convex client react hooks fire on data changes, the data they return for a page render is all from the same logical timestamp. You don't have to worry about one part of the UI saying that payment is pending when another says it's been paid.

2. End-to-end types without codegen. When you define a server function, you define the argument validators, which immediately show up on the types for calling it from the frontend. You can iterate on your backend function and frontend types side-by-side without redefining types or codegen in the loop.

3. Automatic retries for database conflicts, as well as retries for mutations fired from the client. B/c mutations are deterministic and side-effect-free (other than transactional changes to the DB and scheduler), the client can keep retrying them and guarantee exactly-once execution. And if your mutation had DB conflicts, it's automatically retried (up to a limit with backoff). So the client can submit operations without worrying about how to recover on conflict.

There's obv. a bunch more in the article about features like text search and other things out of the box, but those maybe are more conveniences on the backend, since maybe a frontend person wouldn't have been setting up Algolia, etc.
ianhmacartney
·2 jaar geleden·discuss
I'd like to get into the nuance of Frontend vs. Backend vs. Full stack

Having a *backend* platform that has opinionated answers to schema / db access, file storage, auth, scheduled jobs / async workflow, text/vector search, subscriptions/streaming, scalable hosting... is awesome. That is a huge pain point surfaced by this whole debate. However, this doesn't need to tightly couple with html rendering / all your frontends.

Having a *frontend* framework that has amazing inter-op with your backend framework is amazing. E.g.: end-to-end type safety, reactive / realtime updates, authentication flows, etc. However, this doesn't need to be the same framework as your backend (and probably shouldn't if you care about mobile).

*Full-stack* is exciting b/c the same developer can work on the frontend and backend at the same time, ideally in the same language, with the same models flowing through. Full-stack development is about enabling full-stack workflows. It doesn't require that the same company makes your backend and frontend abstractions. It's actually nice when you can have multiple frontends using different frameworks (RN, React, Swift, Kotlin, etc.) or change frontends over time, while still being owned by a single full-stack dev.

For example, with Convex you can write your backend in TypeScript and your frontend with Next.js, Vite (Remix etc), Vue, Svelte etc. and they can all talk to the same backend API with end-to-end types and real-time reactivity. Convex has a built-in reactive database, serverless functions, subscriptions w/ automatic caching, file storage, auth, scheduled functions, text & vector search, automatic scaling, etc. It's the opinionated backend for TypeScript developers. AND you can hit the same API from golang / swift / etc. without getting html in response or duplicating all your business logic into opaque `/api` routes.

This enables you to ship frontend components that have associated backend logic, and makes it easy for people to drop that into their Convex projects, since the database models and logic is all speaking Convex, which isn't possible when the backend might be using any number of database connectors with varying degrees of transaction support (Convex has the highest (serializable) level of Isolation in its ACID guarantees).

disclaimer: I work at Convex. disclaimer disclaimer: I pivoted my career to join b/c once I used it I didn't want to go back.
ianhmacartney
·2 jaar geleden·discuss
Solving the problem of having a transactionally consistent subscription on arbitrary data joins / fetches is hard, but it's so nice to have. If you haven't checked it out yet, Convex provides this out of the box. A blog post on how it works just landed today: https://stack.convex.dev/how-convex-works
ianhmacartney
·3 jaar geleden·discuss
[flagged]