HackerTrans
TopNewTrendsCommentsPastAskShowJobs

fidorka

no profile record

Submissions

Show HN: Screen-watching AI needs a kill switch

github.com
2 points·by fidorka·há 4 meses·1 comments

Show HN: MemoryLane – Video as a token-efficient way to analyze user activity

github.com
3 points·by fidorka·há 4 meses·1 comments

comments

fidorka
·há 4 meses·discuss
To confess something I built just today a little cron that monitors HN for posts I might find interesting, pulls in some context about me, and proposes a reply. Just to help me find relevant posts and to kick start my thinking if I want to engage.

Today it flagged a post about an AI tool for HN and suggested I reply with:

"honestly, if you need an AI to sift through hn, you might be missing the point—this place is about the human touch. but hey, maybe it'll help some folks who just can't take the noise anymore."

So my AI, which I built specifically to sift through HN for me, is telling me to go flame someone else for doing that.

No deeper point here. I just thought it was really funny.
fidorka
·há 4 meses·discuss
Hey, this is quite cool! btw, I have found about the post using some smart cron which I setup. The cron also proposes a reply given some context about me and this is what it has come up with:

"honestly, if you need an AI to sift through hn, you might be missing the point—this place is about the human touch. but hey, maybe it'll help some folks who just can't take the noise anymore."

The really made me laugh - seems like AI is totally pushing me into hypocrisy haha
fidorka
·há 4 meses·discuss
Tried the manual tracking a few times and it does not tend to stick. Passive capture is the approach that ever worked for me. Having different apps and devices with permissions to track you and good APIs that enable integrating the data seems to be the way to go.

Did you think of building some proactive AI tools to make use of all this centralized data?
fidorka
·há 4 meses·discuss
What do you mean?
fidorka
·há 4 meses·discuss
Love the local-first approach. The "just ask it to import my Notion" thing via browser automation is really nice.

One thing I keep coming back to though - what if the tool could actually watch how you use your CRM and then suggest automations based on what it sees you doing repeatedly?

I've been building something called MemoryLane (https://github.com/deusXmachina-dev/memorylane) that does exactly this - it captures screen activity, spots repeated workflows, and suggests automations. Works as an MCP server so you can plug it into Claude or Cursor. Instead of you having to describe what you want automated, it just watches and proposes stuff.

Have you thought about adding something like pattern detection to denchclaw? Feels like it'd fit really well with the "everything app" direction. For us the most useful engine for executing skills and automations is surprisingly cowork thus far, haha
fidorka
·há 4 meses·discuss
Agent memory is one of those things that sounds straightforward until you actually try to build it. The filesystem metaphor makes a lot of sense as a starting point.

I've been working on a slightly different angle with MemoryLane (https://github.com/deusXmachina-dev/memorylane) - instead of giving agents a place to write their own memories, it captures the user's screen activity and makes it queryable. So the agent gets context about what the human was doing, not just what the agent itself did. It plugs in via MCP so Claude Code / Cursor can just ask it stuff.

I think there's something interesting in combining both - agent-vfs for the agent's own state, and something like MemoryLane for the human side. How do you think about that boundary between what the agent remembers vs what it knows about the user?

We need to use vector DBs just because of the amount of data. But on a different layer we want to help create file-based instrucations/skills for patterns that we detect and think can be automated.
fidorka
·há 4 meses·discuss
Cool demo. The tricky bit with browser workflow agents is figuring out which workflows to automate in the first place. Most people don't even realize they're doing the same thing over and over - they just do it.

I've been building MemoryLane (https://github.com/deusXmachina-dev/memorylane) which comes at this from the other side - it records screen activity, spots repeated patterns with AI, and then tells you "hey you keep doing this, want to automate it?" Works as an MCP plugin for Claude/Cursor.

Feels like pattern detection (finding what to automate) + browser agents like yours (actually doing the automation) is the right combo. Are you thinking about the discovery side at all, or mostly focused on execution?
fidorka
·há 4 meses·discuss
MemoryLane screenshots your screen and makes it queryable via AI. All local. But it sees everything — banking, DMs, password managers.

We added a blacklist coordinator that checks four rules on every window-focus event:

- App name (1Password, Signal) — normalized across .app, .exe, bundle IDs

- Window title wildcards (*bank statement*)

- URL patterns (*://*.bank.com/*)

- Incognito auto-detection (on by default, zero config)

When anything matches, capture pauses and queued frames get flushed. Switch away, it resumes.

Surprising part: the matching logic was trivial. The annoying work was cross-platform app name normalization and making the settings UX simple enough that people actually configure it.
fidorka
·há 4 meses·discuss
Building an open-source tool that makes your screen activity searchable via AI because we wanted Claude/Cursor to know what we'd been working on without explaining it every time.

Processing hundreds of screenshots/hour forced us to optimize for token costs.

The surprise: send video, not images

- Single screenshot (1698×894): 1,812 tokens

- Same frame in video: 258 tokens (Gemini 2.5) or ~70 tokens (Gemini 3)

- Full 8-hour workday: ~$1-3

Video gives you timestamps for free and compresses well since consecutive frames are nearly identical. We keep costs down by having the LLM write short summaries while running OCR locally for text extraction.
fidorka
·há 5 meses·discuss
Nice, thanks! Checking it out now.
fidorka
·há 5 meses·discuss
Hey Parker, this is really cool! Thanks for sharing. Have you tried using the entire CLI? It might also be a tool which you could compose into your workflow to have better memory of what the agents themselves did in the repo.

Btw, I built something similar to solve the context problem for most of my laptop-based activity.

It's slightly more heavyweight (electron app ingesting screenshots) - that being said I took many similar design decisions (local embeddings, sqlite with vector search and FTS hybrid, MCP extension to claude). Feel free to check it out:

https://github.com/deusXmachina-dev/memorylane
fidorka
·há 5 meses·discuss
Nice, this could be interesting alternative to github for mac runners. What led you to start this project?
fidorka
·há 5 meses·discuss
Hey, just released a new version with support for local models - you just configure the custom endpoint and model name and it should just work. Let us know what you think:)
fidorka
·há 5 meses·discuss
Nice, could I provide this memory to openclaw as well?
fidorka
·há 5 meses·discuss
This is quite cool! What is it based on? Is it possible to use the same logic on other platforms (windows mac) as well?

Building an electron app (https://github.com/deusXmachina-dev/memorylane-private) which does A LOT of screenshots and this looks like something which could save the users (for now mostly us the dev team :D) a lot of money.
fidorka
·há 5 meses·discuss
This is quite cool! What about making it possible for the (authenticated) user to specify how the component would look for them. Would that make sense?
fidorka
·há 5 meses·discuss
Nice work - the daily-log-first approach resonates. We hit the same "re-onboarding Claude every morning" problem and went a different direction with MemoryLane (https://github.com/deusXmachina-dev/memorylane): it sits in the background capturing your activity (screenshots + OCR + summaries) and makes that context available to any AI chat via MCP.

Different tradeoff - Total Recall is curated (daily logs) and lean, MemoryLane for now captures broadly and relies on search to surface what's relevant. I think they are complementary: your write gate keeps project knowledge tight, broad context with search fills in the "wait, what tab/webpage was that" gaps.

We applied to this batch (YC S26), still waiting on Apple notarization before a wider release. Happy to chat if anyone's curious about the screen-context approach.