RssHub – A feed aggregator that can generate feeds from pretty much everything(docs.rsshub.app)
docs.rsshub.app
RssHub – A feed aggregator that can generate feeds from pretty much everything
https://docs.rsshub.app/en/
64 comments
Is there a RSS generator for Facebook pages as well? Will this support Facebook pages later on?
Facebook used to have a RSS feed, but was shutdown: it's not interesting to Facebook that users spent less time on their plataform. AFAIK RSS feed are currently only supported by https://fetchrss.com/, but it's paid or ad-based.
You can create Facebook (and Instagram, Twitter) feeds with Feedity - https://feedity.com
Neat! Now if it could only RSS-ify literally anything this could be useful.
See also https://github.com/RSS-Bridge/
Interestingly, they have bridges for some websites that do have RSS feeds (Steam, Youtube and HN)... Maybe because the feeds are hard to find?
YouTube used to have an open feed for any search term, but they took it away. This seems to have search but I don't know if you need to authenticate first.
Or perhaps those feeds are deficient in some way. Maybe they update later than the actual site, or have ads inserted as if they are articles.
Neither of those three do that to my knowledge.
Well, given Steam is basically 90% just announcements for deals and HN is an endless spam of articles that were on the frontpage for a millisecond, but I think that would be given.
Well, given Steam is basically 90% just announcements for deals and HN is an endless spam of articles that were on the frontpage for a millisecond, but I think that would be given.
I use RSS Bridge and a couple other services(twitrss.me from the top of my head) and the only thing this has that I'm missing is github trending projects.
[deleted]
See also https://bitbucket.org/tux_/rssparser.lisp (disclosure: mine).
I was so happy at seeing telegram, but it requires admin rights to the channel. There is so much content (a lot of it non-english) hidden away in telegram channels that I want to be part of the open web.
If someone wants to take it up, it is definitely doable with a telegram user account, but the APIs are not readily usable: https://github.com/captn3m0/ideas#telegram-to-rss
Also, if someone is looking to follow lots of GitHub projects using their release feeds, I wrote a OPML generator that uses your Starred repos to generate a OPML file for you: https://github.com/captn3m0/opml-gen
If someone wants to take it up, it is definitely doable with a telegram user account, but the APIs are not readily usable: https://github.com/captn3m0/ideas#telegram-to-rss
Also, if someone is looking to follow lots of GitHub projects using their release feeds, I wrote a OPML generator that uses your Starred repos to generate a OPML file for you: https://github.com/captn3m0/opml-gen
telegram has this shitty feature where if anybody knows your public username they can add you to any group they want.
this is the source of the shitty spam on telegram right now. i was active in the crypto space and my username leaked. im now added to ton of spam and pump groups daily and i cannot block it.
also you dont need admin rights to the channel, but you have to work with https://core.telegram.org/mtproto to scrape channels. i did some side project with this https://github.com/zerobias/telegram-mtproto
this is the source of the shitty spam on telegram right now. i was active in the crypto space and my username leaked. im now added to ton of spam and pump groups daily and i cannot block it.
also you dont need admin rights to the channel, but you have to work with https://core.telegram.org/mtproto to scrape channels. i did some side project with this https://github.com/zerobias/telegram-mtproto
In Telegram: "Settings, Security, Groups" has options to allow everyone or just your contacts to add you to groups. Contacts alone is usually adequate, but you can add exceptions.
How is this different to other messages where your phone number is the username? Last I checked this was possible with Whatsapp too.
your phone number is not your username on telegram. the username on telegram is like a handle on twitter, with the difference that on telegram you dont have to have a username. you can message anybody in your contacts without username. but if you chose to have a username on telegram that username is public, and searchable. and the worst part is that anybody can add your username to any channel.
so tltr: username on telegram is something you chose and not your phone number.
so tltr: username on telegram is something you chose and not your phone number.
>your phone number is not your username on telegram
yes I know, haven't written that.
>and the worst part is that anybody can add your username to any channel
my point was that other messengers have public usernames too, mostly their phone number and suffer from that same problem.
yes I know, haven't written that.
>and the worst part is that anybody can add your username to any channel
my point was that other messengers have public usernames too, mostly their phone number and suffer from that same problem.
> my point was that other messengers have public usernames too, mostly their phone number and suffer from that same problem.
the difference is important. the thing that is happening with telegram is the expected behavior, you are supposed to be able to add random online users with their username to groups and contact them, that is also the point for a username so you don't have to share you phone number.
with whatsapp your phone number is not supposed to be shared with random strangers, and sure if somebody has your private mobile phone number he can spam you but that is not supposed to happen, and i imagine that whatsapp blocks that behavior if you try to do in for multiple users. with telegram i saw groups with 50k members added this way because there is a api for this, but there is no api for whatsapp and i think they block you after x tries or reports. the problem is much bigger with telegram.
the difference is important. the thing that is happening with telegram is the expected behavior, you are supposed to be able to add random online users with their username to groups and contact them, that is also the point for a username so you don't have to share you phone number.
with whatsapp your phone number is not supposed to be shared with random strangers, and sure if somebody has your private mobile phone number he can spam you but that is not supposed to happen, and i imagine that whatsapp blocks that behavior if you try to do in for multiple users. with telegram i saw groups with 50k members added this way because there is a api for this, but there is no api for whatsapp and i think they block you after x tries or reports. the problem is much bigger with telegram.
yes, mtproto is such a horrible experience that almost every library is now abandoned.
The closest was this Show HN about a Channel-> Blog generator that worked on top of telegram: https://news.ycombinator.com/item?id=17617675
The closest was this Show HN about a Channel-> Blog generator that worked on top of telegram: https://news.ycombinator.com/item?id=17617675
Nice. Most of you would know but some won't, some GitHub pages come with RSS feeds.[0]
[0]: https://github.com/DIYgod/RSSHub/releases.atom
[0]: https://github.com/DIYgod/RSSHub/releases.atom
I believe that it might be against Telegram's ToS to scrape channels (Update: It doesn't seem to be). That said: I too really like the idea.
I had been thinking about writing a tool to scrape Telegram channels for another project, but found tg-cli with its Lua, Python bindings and weird command line interface to be quite messy (so I gave up for then).
Since this seems to be in decent demand, surely somebody has already written a tool for this?
I had been thinking about writing a tool to scrape Telegram channels for another project, but found tg-cli with its Lua, Python bindings and weird command line interface to be quite messy (so I gave up for then).
Since this seems to be in decent demand, surely somebody has already written a tool for this?
I tried tg-cli and a few mtproto compatible libraries, but it was all a huge pain to get working properly before I gave up: https://github.com/captn3m0/opengram
The good APIs are the bot ones which won't work here.
The good APIs are the bot ones which won't work here.
I work on Feed Creator which might be interesting too: http://createfeed.fivefilters.org
My mind read that as RSS + SSH and started wondering what funny thing that could do...
Link to GitHub: https://github.com/DIYgod/RSSHub
How come noone has managed to crack the RSS rock yet? Everything has an RSS feed (though it's decreasing). Rss readers have more content than any social network. It seems like you should be able to build something with this that has broad adoption. There has to be something missing.
My theory has been for a while that what is missing is publishing. People don't just want to consume, they also want to share what they find and like and what they made themselves. I even started working on a web-based rss-reader/(micro)blogging thing at one point. Maybe I should give that another crack.
My theory has been for a while that what is missing is publishing. People don't just want to consume, they also want to share what they find and like and what they made themselves. I even started working on a web-based rss-reader/(micro)blogging thing at one point. Maybe I should give that another crack.
If you don't know it, take a look at Shaarli[1], it's a bookmarking web app that allows you to share your bookmarks via RSS, you can add tags and comments, but you can also have "notes" without a link that can be used as microblogging posts.
It's a project from Sebsauvage[2] (an old school french blogger), who wanted to build a social network around the RSS technology.
I know there are projects around it to aggregate multiple shaarli and use it as an actual social network based around sharing links with RSS. I didn't search about it, but it seems that shaarlo[3] is one of them. EDIT: actually, projects like this are listed in shaarli's documentation[4].
EDIT2: tt-rss-shaarli seems to be what you want, Tiny-Tiny RSS with Shaarli, an aggregator with a sharing/commenting functionality.
[1] https://github.com/shaarli/Shaarli
[2] http://sebsauvage.net/wiki/doku.php?id=php:shaarli
[3] https://github.com/DMeloni/shaarlo
[4] https://shaarli.readthedocs.io/en/master/Community-&-Related...
It's a project from Sebsauvage[2] (an old school french blogger), who wanted to build a social network around the RSS technology.
I know there are projects around it to aggregate multiple shaarli and use it as an actual social network based around sharing links with RSS. I didn't search about it, but it seems that shaarlo[3] is one of them. EDIT: actually, projects like this are listed in shaarli's documentation[4].
EDIT2: tt-rss-shaarli seems to be what you want, Tiny-Tiny RSS with Shaarli, an aggregator with a sharing/commenting functionality.
[1] https://github.com/shaarli/Shaarli
[2] http://sebsauvage.net/wiki/doku.php?id=php:shaarli
[3] https://github.com/DMeloni/shaarlo
[4] https://shaarli.readthedocs.io/en/master/Community-&-Related...
This is exactly what I don't understand about all the feed readers out there.
It seems so obvious that the next step for feed readers would be social interactions like sharing, also it would be to easy to add I guess.
Additionally the aspect of consuming YouTube via RSS feed. Feed readers could simply add small logic which transforms a given YouTube account into one of
Is there anything I miss? Why don't feed readers support external services (like the YouTube example from above)? Why don't feed readers add social features?
Additionally the aspect of consuming YouTube via RSS feed. Feed readers could simply add small logic which transforms a given YouTube account into one of
https://www.youtube.com/feeds/videos.xml?channel_id=<account_id>
https://www.youtube.com/feeds/videos.xml?user=<account_name>
Is there anything I miss? Why don't feed readers support external services (like the YouTube example from above)? Why don't feed readers add social features?
RSS is used by those who aren't into "social" addiction, but just want a quick listing of relevant news from a super-simple news feed (as in "really simple syndication"). A couple years ago Google embraced, extended and almost extinguished RSS, so I'm not sure RSS users are willing to obtain/share their feed URLs from a pointless central service; just go to the web sites you're reading frequently, check if they have an RSS feed announced in their HTML metadata (as indicated by an RSS symbol in eg. FF), and add the URL to your feeds.
Because RSS gives readers too much signal, and not enough noise (ads, profiling), and doesn't keep users within the walled garden ecosystems. That's why apps are popular, too. It is all about control. Another disadvantage is that RSS doesn't have native incremental updates, which increases data volume. Though that isn't very relevant in 2018.
As an RSS lover, I have zero interest in social features. If I occasionally want to share something, I can email/WhatsApp/Slack the URL; but for the most part I don't, and I'm able to read without the distraction and noise.
You can add the RSS feed of a youtube channel with the Tiny Tiny RSS bookmarklet, it's working for most channels, but not all for a reason I don't understand yet.
YouTube channel feeds, if they have one, are at: www.youtube.com/feeds/videos.xml?channel_id={channel_id} Some channels have a custom url and the id isn't obvious but it should be in the page source somewhere.
For example, "It's Okay To Be Smart" is at "www.youtube.com/user/itsokaytobesmart". Right click, view page source, search for "/channel/". That finds "<link rel="canonical" href="https://www.youtube.com/channel/UCH4BNI0-FOK2dMXoFtViWHw">". So the feed url is:
For example, "It's Okay To Be Smart" is at "www.youtube.com/user/itsokaytobesmart". Right click, view page source, search for "/channel/". That finds "<link rel="canonical" href="https://www.youtube.com/channel/UCH4BNI0-FOK2dMXoFtViWHw">". So the feed url is:
https://www.youtube.com/feeds/videos.xml?channel_id=UCH4BNI0-FOK2dMXoFtViWHwI'm not sure what the relationship between channels and users is. I think it's many-to-many and I'm aware of 3 feed types:
https://www.youtube.com/feeds/videos.xml?user=<username>
https://www.youtube.com/feeds/videos.xml?channel_id=<channel id>
https://www.youtube.com/feeds/videos.xml?playlist_id=<playlist id>
Woah, I spent more time than I care to admit trying to find some feeds. I thought I had the secret sauce all figured out :) https://www.youtube.com/feeds/videos.xml?user=itsokaytobesma... is much easier to use. I can think of a few channels where the play list thing will come in handy if it works. Gamers where I only want to see 1 out of 50 games they play and are who are orginized enough to keep seperate playlist is one example. You can't even do that with YouTubes own subscription method as far as I know.
Feedly?
Simple:
- most RSS readers suck so bad you can't even retrieve years old articles. So if you follow blogs that don't post often, you simply can NOT get any of their older posts. Imagine a blog that has one seriously good article once a month. Typically you can't get more than a year's worth of posts, so that's like 12 articles. Why?
- the best RSS app I had on iPad let me archive feeds for a LONG time. I used it as a kind of a library. Returned to articles many times (philosophy and such). Most RSS readers out there have ZERO functionality in terms of archiving content indefinitely. (meaning : they typically have some leeway in max articles stored, or how old to archive feeds.. but you know you simply can not realy on the app to save your favorite articels, at some point they will be deleted).
- inability to highlight text. Again, if I read something I want to be able to at least highlight some text. Most RSS readers don't do this.
Instead RSS readers do the same thing every other RSS reader does. Basically, it's like the developers of RSS apps think that the only purpose of retrieving articles is for fueling some kind of "feed" addiction. Like the only purpose of content on the web is to satisfy an urge to quick scan and move to the nex article.
It's like the devs of RSS apps are so drugged out themselves in this day and age of short attention span, that they can't envision other uses for a feed reader than satisfying this urge for the daily/hourly doses of "novelty". Click click click. Swipe swipe swipe.
/rant off
- most RSS readers suck so bad you can't even retrieve years old articles. So if you follow blogs that don't post often, you simply can NOT get any of their older posts. Imagine a blog that has one seriously good article once a month. Typically you can't get more than a year's worth of posts, so that's like 12 articles. Why?
- the best RSS app I had on iPad let me archive feeds for a LONG time. I used it as a kind of a library. Returned to articles many times (philosophy and such). Most RSS readers out there have ZERO functionality in terms of archiving content indefinitely. (meaning : they typically have some leeway in max articles stored, or how old to archive feeds.. but you know you simply can not realy on the app to save your favorite articels, at some point they will be deleted).
- inability to highlight text. Again, if I read something I want to be able to at least highlight some text. Most RSS readers don't do this.
Instead RSS readers do the same thing every other RSS reader does. Basically, it's like the developers of RSS apps think that the only purpose of retrieving articles is for fueling some kind of "feed" addiction. Like the only purpose of content on the web is to satisfy an urge to quick scan and move to the nex article.
It's like the devs of RSS apps are so drugged out themselves in this day and age of short attention span, that they can't envision other uses for a feed reader than satisfying this urge for the daily/hourly doses of "novelty". Click click click. Swipe swipe swipe.
/rant off
You're free to develop an RSS reader with those features yourself. RSS is - on paper - an extremely simple protocol, but since many feeds are broken (not wellformed/improperly escaped XML, often incompetently copied from HTML, with spam, etc. etc.), RSS readers must integrate individual feeds almost on a case-by-case basis, with feed-specific fixes and heuristics. And there goes your project budget. Considering RSS isn't exactly growing (because the number of valuable news site isn't neither), the state of RSS is as good or bad as it always has been. I don't know if that is actually a good thing, because when big media (Google) last turned their eye towards it, it monopolized and almost ruined it.
You're free to develop an RSS reader
"Free to" != "Capable of"
It's that kind of attitude that keeps Linux from going mainstream, too.
This is a user complaining about user issues, and the response is "do it yourself." It is both not helpful, and condescending.
"Free to" != "Capable of"
It's that kind of attitude that keeps Linux from going mainstream, too.
This is a user complaining about user issues, and the response is "do it yourself." It is both not helpful, and condescending.
[deleted]
> - most RSS readers suck so bad you can't even retrieve years old articles. So if you follow blogs that don't post often, you simply can NOT get any of their older posts. Imagine a blog that has one seriously good article once a month. Typically you can't get more than a year's worth of posts, so that's like 12 articles. Why?
Most blogs don't include more than the last ten or so articles in their feed. Fetching backlogs of posts is beyond the scope of RSS. Unless you're complaining that posts you've already downloaded aren't persisted indefinitely?
Most blogs don't include more than the last ten or so articles in their feed. Fetching backlogs of posts is beyond the scope of RSS. Unless you're complaining that posts you've already downloaded aren't persisted indefinitely?
This seems like the sort of thing that should be in the spec for a feed. RSS is basically a textual representation of a site's available articles; you have to explicitly GET the feed to see any new posts (at least without PubSub). If you're going to provide a static xml representation of your site's content, there's no obvious reason why you wouldn't want that to extend back indefinitely.
Something like feed.xml?page=2 seems fairly sane.
Something like feed.xml?page=2 seems fairly sane.
By looking into Podlove[1] I stumbled onto a specification[2] by the IETF for just that. It doesn't seem to have much support though, neither on the feed creator, nor the client side.
[1] https://podlove.org/paged-feeds/
[2] https://tools.ietf.org/html/rfc5005
[1] https://podlove.org/paged-feeds/
[2] https://tools.ietf.org/html/rfc5005
https://bazqux.com/ has long (I think indefinite?) history, text highlighting, a starring system and built-in integration with Pocket/Pinboard/Evernote/whatever.
Keep in mind your history issue is often the fault of the feed, not the reader. If you subscribe to a new blog that only has a year's worth of posts in its feed, then that's what you get. The reader can't proactively follow every blog in the world just in case someone subscribes in 3 years and wants to see the full history.
Keep in mind your history issue is often the fault of the feed, not the reader. If you subscribe to a new blog that only has a year's worth of posts in its feed, then that's what you get. The reader can't proactively follow every blog in the world just in case someone subscribes in 3 years and wants to see the full history.
[deleted]
[deleted]
Isn't ActivityPub supposed to fill the gap?
Yeah its predecessor, OStatus, was pretty much that: you start with a feed of anything, then you add another feed for the reader to comment, then a way to send the comment back to the source so everyone can see it, and lo and behold you've got yourself a social network brewing. Activitypub is a rethinking of the whole thing with clear requirements and specs instead of some informal pieces added here and there, which is a good thing. Except all the existing RSS feeds can't be used as such.
This python script https://github.com/h43z/rssify serves me well to get and RSS feed for websites that don't provide one.
It has a lot of dependencies though. (You might also like mine which can be compiled into a portable binary with one extra line of code. ;-))
Oh... another service similar to Page2RSS that will die in 6 months. Sign me right up :[
Looks to me like it's a service you can deploy on your own server.
Nice companion to: http://fetchrss.com/
exactly what i was looking for
This is much appreciated for twitter, which is astoundingly over-engineered for a site about little blurbs of text.
I don't understand exactly what this software does. The demo link on the frontpage leads to a welcome page. The site is very uninformative but looks very bloated.
* Shameless plug *: Our little startup, Feedity - https://feedity.com, helps create custom RSS feeds for any webpage, even social networks (Facebook, Instagram, Twitter), via an online feed builder and REST API.