Ask HN: Claude Skills and Bs4 for intelligent scraping
3 comments
careful anthropic, well all top, but I have direct copyright violations oct22/'25 documented when they settled copyright violations for $1.5B usd on oct09/'25. Wouldn't say it if I didn't have screenshot and recording proof. just a heads up is all tho. Claude does Code well. usually.
sorry I don't get this at all
While doing all this scraping without any agent help, seeing claude skills' (I am an anthropic power user) ability to run scripts - I want to build a generic scraper that can do a number of things:
1) create data models based on the data it sees
2) create target data stores, like a postgres table based on a model from 1
3) scrape the data using claude agent sdk + playwright mcp
4) dump the data into the storages it creates
this can be utilised for so many different problems. but looking at the scope of the problem, this might be its own company :D
my questions are more of a market discovery:
- if people know of anyone building something similar?
- is someone from the audience would find it useful?
I think I will build a small version for myself without 1 and 2 but would love to know what people think