Automating Scrapers
3 comments
AFAIK the LMs are not continually updated though?
Google Alerts?
Google alerts is too simplistic, one needs a deduplicated tracker of events, an ability to scrape from public records as well as news sites
Perhaps it can be a starting point, but some sort of manual/automated natural language processing is needed on top
Perhaps it can be a starting point, but some sort of manual/automated natural language processing is needed on top
the idea of the final product is simple. a running tracker of all events that meet a particular criteria, with hyperlinks to source references. the use case could vary - from road accidents to other kinds of events that people are interested in, e.g. crimes of a particular kind. sometimes the source events could be public internet, in other cases documents that could be uploaded to the software (such as death certificate records etc.)
folks should consider the utility in a country/context where information is not freely available or published transparently.