HackerTrans
TopNewTrendsCommentsPastAskShowJobs

maltz

no profile record

Submissions

[untitled]

1 points·by maltz·4 tahun yang lalu·0 comments

Web Scraping with Javascript and Node.js

zenrows.com
1 points·by maltz·5 tahun yang lalu·0 comments

Mastering Web Scraping in Python: Scaling to Distributed Crawling

zenrows.com
7 points·by maltz·5 tahun yang lalu·0 comments

Mastering Web Scraping in Python: Crawling from Scratch

zenrows.com
233 points·by maltz·5 tahun yang lalu·81 comments

Stealth Web Scraping in Python: Avoid Blocking Like a Ninja

zenrows.com
2 points·by maltz·5 tahun yang lalu·0 comments

Mastering Web Scraping in Python: From Zero to Hero

zenrows.com
6 points·by maltz·5 tahun yang lalu·0 comments

comments

maltz
·5 tahun yang lalu·discuss
You should probably take a look at this.

https://www.zenrows.com/blog/stealth-web-scraping-in-python-...
maltz
·5 tahun yang lalu·discuss
YouTube has "var ytInitialData" & "var ytInitialPlayerResponse" params hardcoded in HTML. No need to run JS!
maltz
·5 tahun yang lalu·discuss
Yep, seems so. https://www.zenrows.com/pricing
maltz
·5 tahun yang lalu·discuss
If pages are constructed client-side, the content you are looking for is either hardcoded as JSON in the HTML or loaded via XHR request. Scrape that.
maltz
·5 tahun yang lalu·discuss
Playwright. It can be easily used with JS, Python, Go, Java, etc.
maltz
·5 tahun yang lalu·discuss
It's part of a series of blog posts that talks explicitly about crawling. There are indeed other links that do better explaining advanced extraction techniques.

Extraction => https://www.zenrows.com/blog/mastering-web-scraping-in-pytho...

Avoid blocking => https://www.zenrows.com/blog/stealth-web-scraping-in-python-...