Hi, cofounder of Tinfoil here. We're working on implementing prompt caching. It just requires more work than with traditional, non-private inference and requires some careful considerations to actually implement it.
It's not an unreasonable concern, which is why most US companies prefer to go with AWS bedrock, or even one of the AI labs, and typically request zero data retention agreements. But leaking is a concern no matter where it's hosted, it's just the incentives that change IMO. For example, the labs do scan every chat and train on data not covered under enterprise ZDR agreements. Law enforcement can request access to all user data with a valid warrant or in an emergency context [1]
If you're interested in trying DeepSeek V4 privately, you can try Tinfoil (tinfoil.sh) where all models are hosted in an attested secure hardware enclave, making the inference end-to-end private. Full disclosure: I'm one of the cofounders.
I still have the Be My Eyes app installed but haven’t had a call in over a year - I think it’s a testament to how powerful AI vision models have become. I find it cool that AI works well enough for vision impaired people to solve all their problems.
However there was something very human and nice about helping out a stranger with a small random task from time to time. I fondly remember one older lady who spilled a box of blueberries on the kitchen floor and I helped her hunt them all down by guiding her around. It was 10 minutes of connection with a random person doing something fun and which is till remember fondly 4 years later
Beautiful lecture. As with a lot of things that shape our everyday life we forget to contemplate their qualities. For example fonts subtly impact how we perceive, understand, feel about any text yet we fail to give it much thought as Etienne points out. A shape of a letter, especially in math papers, can influence how "beautiful" and understandable the proof is. Really appreciate that Knuth spent years building LaTeX so that compsci and math papers could be formatted as beautiful PDFs and not published as Word documents
> I think if Bob Odenkirk lived on a community farm where everyone had to work together to survive he would be far happier and think life is far more meaningful.
But they already have PII on nearly all users. Many user upload documents with their name, or pictures of themselves, or have a chat where home addresses are involved. All of this is information anthropic already has on their users (voluntarily provided via chats or via api) and is equivalent to what Persona gets via their verification - it’s just more convenient to use a third party SaaS product for this than vibe coding their own identity verification platform I guess
Yes it appears your personal data IS being sent to open router and the model provider here. The problem I think is that a lot of people (especially in the openclaw community) mistake “I run it on my mac mini” to mean their data is private. Meanwhile all data is being shipped off for training to anthropic via openrouter and both of those parties see everything.
I guess you could theoretically plug in a local model here but of course the readme should be more precise here when talking about privacy
The attestation report is produced ahead of time and verified on each connection (before the prompt is sent). Every time the client connects to do an inference request via one of the Tinfoil SDKs, the attestation report is checked relative to a known-good/public configuration to ensure the connection is to a server that is running the right model.
The attestation is tied to the Modelwrap root hash (the root hash is included in the attestation report) so you know that the machine that is serving the model has the right model weights
The absence of solutions for LLM privacy on that list is telling. We’ve figured out how to have private communications with other humans via end to end encryption but arguably we’re leaking a lot more to chatbots about ourselves in a few sessions than we do to even our closest friends and family over Whatsapp
It uses confidential computing primitives like Intel TDX and NVIDIA CC, available on the latest generations of GPUs. Secure hardware like this is a building block to enable verifiably private computation without having to trust the operator. While Confer hasn’t released the technical details yet, you can see in the web inspector that they use TDX in the backend by examining the attestation logs. This is a similar architecture to what we’ve been developing at Tinfoil (https://tinfoil.sh) if you’re curious to learn more!
not to mention the privacy concerns associated with connecting my entire life to OpenAI or Anthropic. If you have the memory feature enabled, it's scary how much ChatGPT knows about you already and can even infer implicit thoughts and patterns about you as a person.
This is really neat! Didn’t realize it could be this simple to run RL on models. Quick question: How would I specify the reward function for tool use? or is this something you automatically do for me when I specify the available tools and their uses?
You're absolutely right. It is not enough to use anonymity tools, you also have to make sure everything else around you doesn't compromise your anonymity. Made me think of a Harvard bomb threat incident where the student posting a fake bomb threat (through Tor) to avoid final exams was the only person using Tor on campus at the time, which trivially identified him.