Why build this blog, or anything, on IPFS?(teetotality.blog)
teetotality.blog
Why build this blog, or anything, on IPFS?
http://teetotality.blog/posts/why-ipfs/
155 comments
I've tried both extensively. Both suck. And they're not really comparable, except on their problems.
I have quite a bit of experience with IPFS and I tried Dat once. It seems to me that Dat works much better, but I think the immutability of IPFS is its killer feature, Dat is thus much less interesting to me.
Underlying dat data is immutable. It's the constructs built on top that provide mutability. Dat is fundamentally a chain of immutable chunks, some having a special meaning to say what the data means (file/folder) and where to look for it. Once a chunk is there it will 'ever ever change. It is perfectly possible to play with those only.
Well, Beaker is being rethought and rewritten right now - hopefully it survives that process.
You can check out this recent article comparing the two: https://medium.com/@jaygraber/comparing-ipfs-and-dat-8f3891d... -- they have their own foci and technical decisions (ex IPFS is aiming for more general content addressing/deduplication of data across many hosts, while Dat focuses on append-only feeds from a particular source)
I'm involved with a startup that has been trying to use IPFS. There are still a few problems related to incentivizing people to pin files for you. Filecoin, the ICO coin associated with IPFS has been inching closer to a testnet launch for quite long now. That was supposed to happen end of last year and it didn't as far as I know. So, they are obviously a bit behind schedule on that. Without that, content on IPFS is only as durable as the node that uploaded the content. There are no guarantees long term availability of content.
So, IPFS is more of a CDN than a file system currently. It's a distributed content cache. There's an enormous long tail of files that are only available on 1 node, which is typically somebody's laptop.
Another problem is that the block system does not combine well with e.g. s3 or similar file buckets on popular cloud providers. If you think of IPFS as a CDN then you basically have to worry about hosting files somewhere that is reliable and durable. IPFS does not solve that problem currently. So, you basically will either be self hosting some file servers or use something off the shelf, like S3. There's an S3 backend for IPFS but it's a bit unclear how well that performs. We've done some tests with it and the small blocksize is creating quite a bit of overhead for read and write HTTP requests.
Access control or privacy protection are currently not really in scope of IPFS. I doubt this is a good tool for bypassing e.g. censor ship unless you are willing to expose yourself to explaining why your node is hosting certain content hashes. TOR and I2P probably provide better protection here. I2P actually runs a variant of bittorrent for file sharing. It's been a while since I looked at this but it used to be quite slow but reliable.
So, IPFS is more of a CDN than a file system currently. It's a distributed content cache. There's an enormous long tail of files that are only available on 1 node, which is typically somebody's laptop.
Another problem is that the block system does not combine well with e.g. s3 or similar file buckets on popular cloud providers. If you think of IPFS as a CDN then you basically have to worry about hosting files somewhere that is reliable and durable. IPFS does not solve that problem currently. So, you basically will either be self hosting some file servers or use something off the shelf, like S3. There's an S3 backend for IPFS but it's a bit unclear how well that performs. We've done some tests with it and the small blocksize is creating quite a bit of overhead for read and write HTTP requests.
Access control or privacy protection are currently not really in scope of IPFS. I doubt this is a good tool for bypassing e.g. censor ship unless you are willing to expose yourself to explaining why your node is hosting certain content hashes. TOR and I2P probably provide better protection here. I2P actually runs a variant of bittorrent for file sharing. It's been a while since I looked at this but it used to be quite slow but reliable.
The long-term solution for web is:
- Use more static HTML - Since your assets are versioned and static, start calculating hashes for everything you publish - When you link to something, include the URL and the integrity <a href="URL" integrity="sha256:...">...</a>
In the immediate future this will fix broken links. Things that are important will be cached and accessible via content addressing. In the long-term future this will fix other problems like linking to a page and then it being changed to something you don't endorse.
- Use more static HTML - Since your assets are versioned and static, start calculating hashes for everything you publish - When you link to something, include the URL and the integrity <a href="URL" integrity="sha256:...">...</a>
In the immediate future this will fix broken links. Things that are important will be cached and accessible via content addressing. In the long-term future this will fix other problems like linking to a page and then it being changed to something you don't endorse.
Good luck managing a website with 10 daily blog posts and multiple authors through static HTML files. The success of Wordpress is thanks to a lot more than "now you don't have to know HTML".
What's the problem with frequent updates and multiple authors? Those are solved problems. As long as everyone knows how to use the static site generator and git, you should be good.
I think one of the interesting use cases for IPFS is having a distributed build store for tools like Nix and Guix. It sounds to me like almost the perfect case for it: an immutable datastore of hashes of reproducible builds is well suited to distributed storage. Imagine being able to peek into a global store of build outputs and, for the hash of any given inputs, retrieve some corresponding output. It would be an incredibly unique experience.
There’s a proof of concept of building IPFS into Guix’s store[1]. There’s also periodically discussion on the mailing list. I’m sure it’s much more complicated than I’m giving it credit for, and there would be security and social implications (someone’s going to be building most of the software, and what happens if you have low bandwidth or a small data plan?) Still, IPFS sounds like an interesting experiment in this area.
[1] https://github.com/fps/guix-ipfs-cache
There’s a proof of concept of building IPFS into Guix’s store[1]. There’s also periodically discussion on the mailing list. I’m sure it’s much more complicated than I’m giving it credit for, and there would be security and social implications (someone’s going to be building most of the software, and what happens if you have low bandwidth or a small data plan?) Still, IPFS sounds like an interesting experiment in this area.
[1] https://github.com/fps/guix-ipfs-cache
I once fell for this talk about "content-addressing".
You know what? Saying stuff is addressed by their content doesn't change the fact that the internet is "location-addressed" and you still have to know where peers that have the data you want are and connect to them.
And what is the solution for that? A DHT!
Turns out DHTs have terrible incentive structure and don't seem to be working well.
Downloading content on IPFS is the _most slow experience ever_ and for some reason I don't understand downloading is even slower. Even if you are in the same LAN of another machine that has the content you need it will still take hours to download some small file you would do in seconds with `scp`.
Now even if you know which peer has the content you want and tell IPFS to connect to it directly and the connection is established and the is being (slowly) downloaded... IPFS will drop the connection and the download will stop.
You know what? Saying stuff is addressed by their content doesn't change the fact that the internet is "location-addressed" and you still have to know where peers that have the data you want are and connect to them.
And what is the solution for that? A DHT!
Turns out DHTs have terrible incentive structure and don't seem to be working well.
Downloading content on IPFS is the _most slow experience ever_ and for some reason I don't understand downloading is even slower. Even if you are in the same LAN of another machine that has the content you need it will still take hours to download some small file you would do in seconds with `scp`.
Now even if you know which peer has the content you want and tell IPFS to connect to it directly and the connection is established and the is being (slowly) downloaded... IPFS will drop the connection and the download will stop.
I once thought about building an app based on IPFS but the problem is the lack of decentralized consensus. Only the Blockchain has cracked that problem and it is incredibly inefficient. The IPFS app would still require a central server and if that is the case then it would be better to just have some sort of public database backup with encrypted user data so that anyone can host their own fork and each fork can talk with each other through a standardized protocol. It would be a federated system like E-Mail.
All the docs tells about to setup IPFS server. What is the intended way to browse ipfs sites(extensions , or new browser ) ?
Off-topic but from work this site is blocked due to a security issue. Anyone know why?
I've seen hugely varying IPFS latency. `time curl -o- https://teetotality.blog/posts/how-this-blog-was-made/` took > 20 seconds, for example, yet https://ipfs.io/ipns/teetotality.blog/ returned in under a second.
I'd be really interested in a "what to do and what not do" wrt IPFS to avoid those 20s (or completely non-functioning) URLs.
I'd be really interested in a "what to do and what not do" wrt IPFS to avoid those 20s (or completely non-functioning) URLs.
You're not really using IPFS here. The gateway that responds to the requests over at teetotality.blog is the one that actually communicates with IPFS, then forwarding the content to you.
The traditional web works with the concept of timeouts set by the owner. So if you request some content, and it's unable to find it within X seconds, the request gets cancelled.
IPFS (the network, not gateways) works differently, as the use case is different. Basically, there is no "requests". You simply put in a "ask" in the DHT about who has this content. If someone finds someone who provides the same content, then it tells you about it.
IPFS in itself doesn't have any timeouts, as the model is different. The use case is basically "I need this content, no matter how long time it takes and from who".
So, the difference between DNS addresses responding with different times, comes up to the maintainer of the gateway. Sometimes it's more connected to other nodes (like probably ipfs.io's gateways are) and sometimes they employ a longer cache for content (cloudflare's gateway have a long cache) and it makes the resolution time faster/slower.
The traditional web works with the concept of timeouts set by the owner. So if you request some content, and it's unable to find it within X seconds, the request gets cancelled.
IPFS (the network, not gateways) works differently, as the use case is different. Basically, there is no "requests". You simply put in a "ask" in the DHT about who has this content. If someone finds someone who provides the same content, then it tells you about it.
IPFS in itself doesn't have any timeouts, as the model is different. The use case is basically "I need this content, no matter how long time it takes and from who".
So, the difference between DNS addresses responding with different times, comes up to the maintainer of the gateway. Sometimes it's more connected to other nodes (like probably ipfs.io's gateways are) and sometimes they employ a longer cache for content (cloudflare's gateway have a long cache) and it makes the resolution time faster/slower.
> even go super old-school and run a web server at home. It's not as if we're short on options in 2020.
Though it's old school, it's incredibly difficult to run server at home now at least in India. The network I connect to is behind a NAT which is behind another NAT. At least that's what I saw when I tried to host my blog on Raspberry PI at home over a year ago. Ultimately I gave up on that endeavor. If anyone has solution that doesn't involve third party, please suggest.
I think I will have to wait until my ISP implements IPv6. That could take another decade :/
Though it's old school, it's incredibly difficult to run server at home now at least in India. The network I connect to is behind a NAT which is behind another NAT. At least that's what I saw when I tried to host my blog on Raspberry PI at home over a year ago. Ultimately I gave up on that endeavor. If anyone has solution that doesn't involve third party, please suggest.
I think I will have to wait until my ISP implements IPv6. That could take another decade :/
You can always use https://ngrok.com and other similar "tunneling" solutions. It works like a charm:
https://ngrok.com
https://github.com/cloudflare/cloudflared
https://github.com/inlets/inlets
https://ngrok.com
https://github.com/cloudflare/cloudflared
https://github.com/inlets/inlets
Thank you for all of these links. This is the first time I've heard of any of them. I especially like the inlets project.
Networks such as cjdns and secure scuttlebutt can help you overcome this problem, but introduces new ones (or introduces different models that are not already widely deployed)
- [0] https://github.com/cjdelisle/cjdns/
- [1] https://scuttlebutt.nz/
- [0] https://github.com/cjdelisle/cjdns/
- [1] https://scuttlebutt.nz/
I ran my vlog on RasPi and did everything in a static generated way. I configured Cloudflare to cache everything for 30 days. Finally I have a cache warming script that requests each static file after I purge the cache (could be better by only purging and warming changed files, like via a Makefile). With this approach, a tunnel with a public port and IP could run minimally during the purge/warm cycle and then be shut down.
I solicit feedback via email rather than having public comments, and upcoming authenticated areas are done via Auth0 tokens.
I solicit feedback via email rather than having public comments, and upcoming authenticated areas are done via Auth0 tokens.
Your ISP implementing IPv6 won't be enough: One can't reach it from the outside, as long as not "everybody" else has IPv6.
You don't need everyone on IPv6. Just most people. Then it becomes the issue of those still on v4
Right. Couple of decades in that case.
I've been running my homeserver for a couple of years now (in India), even gave a talk about it at RootConf last year. It is definitely doable. Two approaches that I'd suggest:
1. Talk to your ISP. Most of them will assign you a public static IP for a fee (5000 INR/yr was what I was quoted last time)
2. Setup a simple proxy on a $5 droplet on Digital Ocean. I run OpenVPN server on the droplet, a client on the homeserver, and `simpleproxy` that forwards the relevant ports. You can do the same with iptables as well.
3. Any other approaches involving ngrok-like solutions are much costlier, unless you host it.
1. Talk to your ISP. Most of them will assign you a public static IP for a fee (5000 INR/yr was what I was quoted last time)
2. Setup a simple proxy on a $5 droplet on Digital Ocean. I run OpenVPN server on the droplet, a client on the homeserver, and `simpleproxy` that forwards the relevant ports. You can do the same with iptables as well.
3. Any other approaches involving ngrok-like solutions are much costlier, unless you host it.
Which isp are you on? For all you know, it might be possible to get an ipv6 now.
I was in the same predicament but there was a news item that act(ISP) was rolling out ipv6... speaking with customer care and even escalating to nodal support was useless as they were clueless... Spent an afternoon fiddling with router and I got a v6 address no problem!
Tl;dr; don't believe clueless tech support .. Just try it out at your end.. you might get a pleasant surprise!
I was in the same predicament but there was a news item that act(ISP) was rolling out ipv6... speaking with customer care and even escalating to nodal support was useless as they were clueless... Spent an afternoon fiddling with router and I got a v6 address no problem!
Tl;dr; don't believe clueless tech support .. Just try it out at your end.. you might get a pleasant surprise!
It's pieces like these that remind me why good writing skills are important, and one shouldn't stray from the basics unless they're fully aware of the trade-offs. For this article, it would be: write a better hook, and make sure to include a rudimentary thesis statement, because I wasn't able to deduce what you were trying to persuade me of, within the first few paragraphs.
With a title like "Why build this blog -- or anything -- on IPFS?" You were trying to persuade me, right?
I had to read through what is essentially every single cooking recipe on the web, before I got to the actual filling. I.e a whole lotta aimless wandering and musing, that is only tangenitally related to the topic at hand, before giving me what the title promised. Similarily to cooking blogs, this page is 2/3 filler, and 1/3 actually giving me what the title promised "So……why IPFS?"
> 1. Ownership, control, censorship
The author goes on to chastise Medium's censorhsip practices, but not too long ago he mentioned self-hosted Wordpress and staticly-generated Github pages. Wordpress and Github pages get over these hurdles and are easier to setup than IPFS.
> 2. Resilience
Suffice to say, the point of this pargraph was "DNS and HTTP unrobust, webservers fail under unforseen circumstances." Ok, well how does IPFS do things differently? You never explained how IPFS works, much less how it gets over any of the aforementioned issues you outlined.
> 3. Elegance
> But I will say that content addressing strikes me, and many software people who come across it, as obviously superior to host-based addressing along certain dimensions.
Never touched upon or elaborated.
> Plus, it's super cool. You should try it!
Atleast you have a call to action. Otherwise, this post fails to even come close to making me interested in IPFS.
With a title like "Why build this blog -- or anything -- on IPFS?" You were trying to persuade me, right?
I had to read through what is essentially every single cooking recipe on the web, before I got to the actual filling. I.e a whole lotta aimless wandering and musing, that is only tangenitally related to the topic at hand, before giving me what the title promised. Similarily to cooking blogs, this page is 2/3 filler, and 1/3 actually giving me what the title promised "So……why IPFS?"
> 1. Ownership, control, censorship
The author goes on to chastise Medium's censorhsip practices, but not too long ago he mentioned self-hosted Wordpress and staticly-generated Github pages. Wordpress and Github pages get over these hurdles and are easier to setup than IPFS.
> 2. Resilience
Suffice to say, the point of this pargraph was "DNS and HTTP unrobust, webservers fail under unforseen circumstances." Ok, well how does IPFS do things differently? You never explained how IPFS works, much less how it gets over any of the aforementioned issues you outlined.
> 3. Elegance
> But I will say that content addressing strikes me, and many software people who come across it, as obviously superior to host-based addressing along certain dimensions.
Never touched upon or elaborated.
> Plus, it's super cool. You should try it!
Atleast you have a call to action. Otherwise, this post fails to even come close to making me interested in IPFS.
> Suffice to say, the point of this pargraph was "DNS and HTTP unrobust, webservers fail under unforseen circumstances." Ok, well how does IPFS do things differently? You never explained how IPFS works, much less how it gets over any of the aforementioned issues you outlined.
If DNS and HTTP are not working absolutely no one will care about some blog being up.
If DNS and HTTP are not working absolutely no one will care about some blog being up.
I think bloggers deserve some leeway in how they write; it's an informal medium and maybe the author's usual audience is arriving with a lot of shared assumptions.
Still, I wish IPFS had been defined within the post. I had to look it up on Wikipedia; presumably it's "Interplanetary File System".
Still, I wish IPFS had been defined within the post. I had to look it up on Wikipedia; presumably it's "Interplanetary File System".
Yes, it occurred to me that his audience already had a background in IPFS and I wasn't his prospective audience; yet, then why craft the title to seem like it was meant for people who have no idea what IPFS is, because surely those with prior knowledge of it, would already be able to answer for themselves "Why... IPFS?"
I think my main problem with this blog post, and many others, is that they come off too "stream of consciousness," instead of something more structured, and easily-digestable.
It's obvious the author can write[0], but at risk of being presumptuous, it seems like it was hastily written and submitted to HN for the sole purpose of generating traffic.
[0] This is a much better piece, albeit short: http://teetotality.blog/posts/think-do-build/
I think my main problem with this blog post, and many others, is that they come off too "stream of consciousness," instead of something more structured, and easily-digestable.
It's obvious the author can write[0], but at risk of being presumptuous, it seems like it was hastily written and submitted to HN for the sole purpose of generating traffic.
[0] This is a much better piece, albeit short: http://teetotality.blog/posts/think-do-build/
My goodness, this is harsh. I was thinking it was a very well-written article and I appreciated its clarity.
ISPs have made it really expensive to get a static IP. I maintained a commercial connection for way to many years because I would remote into my home systems.
That's not ISPs' doing, it's the natural outcome of IP address exhaustion. BTW, there are ways to "remote into" your home systems even if you don't have a static IP. You can even make it work if your home systems aren't directly accessible due to NAT.
Zerotier is an awesome tool for this.
That depends on the market where you are, I suppose. My home ISP offers static addresses as an add-on for a cost comparable to a cheap VPS. With my last ISP there was no static address offer, but the address was the same for years. You can use dyndns-like services for the worst case, then.
As long as it is not behind NAT, most DNS providers would let you easily change it via an API call.
Not true in Canada
This blog is “built” on cloudflare apparently.
Why is medium being compared to IPFS???
Why is medium being compared to IPFS???
>This blog is “built” on cloudflare apparently.
You can read more about why that is here: http://teetotality.blog/posts/how-this-blog-was-made/
You can read more about why that is here: http://teetotality.blog/posts/how-this-blog-was-made/
I found that too as the article linked raised more questions that it answered. I was like "how can I be reading this if it's not using DNS?" And from the link this explanation:
> It updates a dnslink pointer at Cloudflare, which allows Cloudflare's DNS to direct teetotality.blog HTTP traffic to the correct IPFS hash address via their IPFS Gateway. So even though you've (probably) reached this page through a regular old HTTP link that uses the teetotality.blog host name, there is in fact no server with that name - the content is stored on various IPFS nodes, including but not limited to Cloudflare's edge caches.
> It updates a dnslink pointer at Cloudflare, which allows Cloudflare's DNS to direct teetotality.blog HTTP traffic to the correct IPFS hash address via their IPFS Gateway. So even though you've (probably) reached this page through a regular old HTTP link that uses the teetotality.blog host name, there is in fact no server with that name - the content is stored on various IPFS nodes, including but not limited to Cloudflare's edge caches.
The blog is hosted on IPFS. To allow normal web browsers without IPFS support to view the website they use Cloudflare's IPFS gateway, which is a service that serves content from IPFS over normal HTTP.
I’d be a bit more precise. The blog is hosted by cloudflare. IPFS hosts all the versions of the blog, none of which are canonical.
[deleted]
I think IPFS should be a pluggable back-end among many back-ends that become possible once we move away from the idea that websites must be hosted by a specific server / domain and start thinking in terms of “client-first” architecture.
I don’t want to write a wall of text, right now the top post on https://qbix.com/blog lays out several specific actionable things we can all do to bring about this future. It requires a snowball effect and a critical mass for any of this to take off.
I don’t want to write a wall of text, right now the top post on https://qbix.com/blog lays out several specific actionable things we can all do to bring about this future. It requires a snowball effect and a critical mass for any of this to take off.
I have my own blog on DigitalOcean using Jekyll. Can't be more easy to do and mantain.
> Can't be more easy to do and mantain.
Sure it can.
- Github pages
- Netlify
- Zeit
All free for static sites; all easy to set up a continuous deployment pipeline for.
Sure it can.
- Github pages
- Netlify
- Zeit
All free for static sites; all easy to set up a continuous deployment pipeline for.
I’ve been on Netlify for a while and it’s great at what it does.
I recently went back to Digital Ocean for my side project because of what Netlify currently doesn’t have: DNSSEC, HTTP/2 push and prioritization. ECC certificates from Let’s Encrypt.
I recently went back to Digital Ocean for my side project because of what Netlify currently doesn’t have: DNSSEC, HTTP/2 push and prioritization. ECC certificates from Let’s Encrypt.
Why do you want ECC specifically? Isn't that an implementation detail?
From a performance point of view, ECC certs are significantly smaller than RSA certs at a comparable level of security.
Smaller certs translate to fewer bytes going over the wire when doing TLS handshakes, reducing latency.
But it was really their lack of HTTP/2 push support and how their CDNs don't support H2 prioritization correctly[1] which annoyed me to the point of going back to Digital Ocean and running my own instance of H2o where I have full control.[2]
1. https://github.com/andydavies/http2-prioritization-issues
2. https://h2o.examp1e.net/configure/http2_directives.html#http...
Smaller certs translate to fewer bytes going over the wire when doing TLS handshakes, reducing latency.
But it was really their lack of HTTP/2 push support and how their CDNs don't support H2 prioritization correctly[1] which annoyed me to the point of going back to Digital Ocean and running my own instance of H2o where I have full control.[2]
1. https://github.com/andydavies/http2-prioritization-issues
2. https://h2o.examp1e.net/configure/http2_directives.html#http...
> Smaller certs translate to fewer bytes going over the wire when doing TLS handshakes, reducing latency.
Provided it fits into a single Ethernet frame it's not going to make any difference, right?
Provided it fits into a single Ethernet frame it's not going to make any difference, right?
Github pages doesn't support all the plugins I use for my categories, plus I don't have my own server to play with other projects.
Right now this case in point of why NOT to.
ipfs resolve -r /ipns/teetotality.blog/posts/how-this-blog-was-made/%60: no link named "`" under QmefCQnxfw2qaT5WKMxiMVGTWu2i47ttpyUCDdn7f3nA2K
ipfs resolve -r /ipns/teetotality.blog/posts/how-this-blog-was-made/%60: no link named "`" under QmefCQnxfw2qaT5WKMxiMVGTWu2i47ttpyUCDdn7f3nA2K
Wordpress is unlike github pages or medium. In theory a LAMP server could run on everyone's home router, and they 'd publish their blog there. With commercial interest , network configurations would change to accomodate that, and people wouldn't lose their content when any service decides to shutdown. It's scary how much of the web is being lost , i would wager most links to nonocorporate sites > 10 years old barely work (has anyone studied the link rot?).
Some estimates here: https://en.wikipedia.org/wiki/Link_rot -- the most recent listed study (2017) estimates a half life of ~2 years for an average link on Yahoo.
> From a certain perspective, the internet and the web as we know them (including fundamental technologies such as DNS, TCP/IP, HTTP, SMTP, and even Javascript) are flawed in fundamental ways. Leaving the technicalities aside, how do these flaws manifest? It is hard to stop bad actors from doing bad things: sending too many emails, stealing sensitive data, flooding websites with traffic, spreading false facts and bifurcating the shared reality that allowed for a democratic global order.
I would argue that this is even more of a problem with decentralized services because there is no one to define (or police for) spam or bad content.
I would argue that this is even more of a problem with decentralized services because there is no one to define (or police for) spam or bad content.
IPFS still has a long way to go until it is useable in my opinion. The default configuration for the desktop client will gladly keep open 1000+ peer connections and will happily degrade your usual internet experience.
In addition the ecosystem is filled with technical/community debt that makes navigating the system a nightmare for anyone who isn't an expert. As an example: https://github.com/ipfs/go-ipfs/issues/1482
It's a shame if you ask me.
In addition the ecosystem is filled with technical/community debt that makes navigating the system a nightmare for anyone who isn't an expert. As an example: https://github.com/ipfs/go-ipfs/issues/1482
It's a shame if you ask me.
Hey IPFS person here. We actually made a change to ipfs-desktop back in Feb of last year to reduce the default connection limit to ~300 (https://github.com/ipfs-shipyard/ipfs-desktop/pull/828), and also to set desktop nodes into DHT-client mode (so they don't get lots of requests from other nodes for where to find content). If you've been running your node since back then, you can change your defaults in the desktop "settings" menu at the bottom of your IPFS config by setting '"HighWater": 300' (or whatever you prefer -- personally I like having ~600 connections). I run ipfs-desktop all the time (including on crappy hotel wifis) and don't find it gets in the way of anything.
You're right, there are a ton of great ideas and suggestions for how to make IPFS better that we haven't gotten to yet - we're working on making it easier to navigate and for more folks to help contribute. To that aim, we just created a new ipfs-docs site (in beta right now) to better explain the concepts and how-to of working in the ecosystem: blog.ipfs.io/2020-01-07-ipfs-docs-beta/ -- would love your feedback on how we can keep making that better!
You're right, there are a ton of great ideas and suggestions for how to make IPFS better that we haven't gotten to yet - we're working on making it easier to navigate and for more folks to help contribute. To that aim, we just created a new ipfs-docs site (in beta right now) to better explain the concepts and how-to of working in the ecosystem: blog.ipfs.io/2020-01-07-ipfs-docs-beta/ -- would love your feedback on how we can keep making that better!
Hi! Thanks for taking the time to respond and correct my outdated info. Unfortunately, It seems your info is also outdated: I just installed 0.10.2 for Mac and it's by default using (600,900) for the low and high watermarks and routing type is set to dht rather than dhtclient.
Since I've got you here, I'd be happy to give a little feedback: The constellation of websites for the protocol labs projects, their different visual styles, their different states of decay/maintenance, the interlinking back and forth with source files in github, and the undocumented status of all these projects really lend to a maze-like and poor learning/browsing experience. If it were my project, I would dump all the different sites and consolidate under one web presence. I never know if what I'm reading is current, or planned and not done, or old and out of date, or just bugged. As an example, there are even broken links on the readme at https://github.com/ipfs/ipfs. The number of TODOs or unresponsive links in README files in the various github repos (under the various github organizations) adds to the frustration of trying to learn your work. As an example, take the readme at https://github.com/ipfs/go-hamt-ipld : It's been recently worked on, and yet the entire table of contents are broken links and the examples section is literally just "//TODO". I think y'all should consolidate efforts, kill or privatize the projects that are unusable by the outside world, and in general start developing things to a more complete state before adding new stuff.
I've tried several times now to incorporate IPFS into my developer toolbelt and have aborted my attempt each time due to this or that rough edge that you don't learn about unless you spend time trolling through the github issues. It seems like each time I've come back y'all have added more unfinished stuff (congrats on the filecoin testnet!) and the rough edges still remain. As an example, I just added a 7 gigabyte file (an xcode disk image) through the desktop client: It peaked at around 15gb of ram during the process and the client offered _zero_ visual feedback while it was working. In fact, while the client says my repo is the correct size, the xcode image I added doesn't actually show up in the file browser. It's all very painful to work with. ipfs, ipld and libp2p hit the right notes in their promotional material to entice someone who wants to help the decentralized web grow, and I would love for these projects to be what they aspire and claim to be, but it all just seems to fall apart rather quickly when you actually start to use them for something beyond your tutorials.
Since I've got you here, I'd be happy to give a little feedback: The constellation of websites for the protocol labs projects, their different visual styles, their different states of decay/maintenance, the interlinking back and forth with source files in github, and the undocumented status of all these projects really lend to a maze-like and poor learning/browsing experience. If it were my project, I would dump all the different sites and consolidate under one web presence. I never know if what I'm reading is current, or planned and not done, or old and out of date, or just bugged. As an example, there are even broken links on the readme at https://github.com/ipfs/ipfs. The number of TODOs or unresponsive links in README files in the various github repos (under the various github organizations) adds to the frustration of trying to learn your work. As an example, take the readme at https://github.com/ipfs/go-hamt-ipld : It's been recently worked on, and yet the entire table of contents are broken links and the examples section is literally just "//TODO". I think y'all should consolidate efforts, kill or privatize the projects that are unusable by the outside world, and in general start developing things to a more complete state before adding new stuff.
I've tried several times now to incorporate IPFS into my developer toolbelt and have aborted my attempt each time due to this or that rough edge that you don't learn about unless you spend time trolling through the github issues. It seems like each time I've come back y'all have added more unfinished stuff (congrats on the filecoin testnet!) and the rough edges still remain. As an example, I just added a 7 gigabyte file (an xcode disk image) through the desktop client: It peaked at around 15gb of ram during the process and the client offered _zero_ visual feedback while it was working. In fact, while the client says my repo is the correct size, the xcode image I added doesn't actually show up in the file browser. It's all very painful to work with. ipfs, ipld and libp2p hit the right notes in their promotional material to entice someone who wants to help the decentralized web grow, and I would love for these projects to be what they aspire and claim to be, but it all just seems to fall apart rather quickly when you actually start to use them for something beyond your tutorials.
> I just installed 0.10.2 for Mac and it's by default using (600,900) for the low and high watermarks and routing type is set to dht rather than dhtclient.
(600,900) watermarks suggest you had an old config (ipfs-desktop will respect pre-existing config and won't override values). Just change watermarks manually to (50,300) and restart.
If you run daemon via ipfs-desktop the routing preference from config file is ignored: Desktop runs daemon with an explicit command line parameter (--routing=dhtclient) to avoid DHT server traffic.
Hope this helps.
(600,900) watermarks suggest you had an old config (ipfs-desktop will respect pre-existing config and won't override values). Just change watermarks manually to (50,300) and restart.
If you run daemon via ipfs-desktop the routing preference from config file is ignored: Desktop runs daemon with an explicit command line parameter (--routing=dhtclient) to avoid DHT server traffic.
Hope this helps.
Thanks for the heads up. I didn't realize I had left the old config in place.
>You're right, there are a ton of great ideas and suggestions for how to make IPFS better that we haven't gotten to yet
I love the idea of IPFS, but let me ask you the question, when is it finally going to be ready for people to easily deploy? A year from now, 3 years, 5 years? Or maybe you have no idea?
Don't give some vague answer about how you are working on it. Give me a reasonably specific prediction, or say you have no idea.
I love the idea of IPFS, but let me ask you the question, when is it finally going to be ready for people to easily deploy? A year from now, 3 years, 5 years? Or maybe you have no idea?
Don't give some vague answer about how you are working on it. Give me a reasonably specific prediction, or say you have no idea.
Isn't this kind of an unreasonable demand? It's not like software isn't ready for anyone one day and ready for everyone the next. Considering just about nobody can give an accurate software development forecast for even objective milestones, it seems ridiculous that there would be any reasonable answer to this.
Beyond the question of the possibility of giving a precise answer, did OP lose sight of basic courtesy?
As far as I know IPFS is an open system contributed to by volunteers. Making those kind of demands for deadlines in such an aggressive tone is way out of place.
As far as I know IPFS is an open system contributed to by volunteers. Making those kind of demands for deadlines in such an aggressive tone is way out of place.
No, I am just asking for honesty. Either they have some sort of idea when it will be basically usable, or they don't. I am not saying they have to meet a deadline, I am asking when they think they are going to get there.
And why do they need to be defended by others? If I am being unreasonable, why don't they just say that themselves?
And why do they need to be defended by others? If I am being unreasonable, why don't they just say that themselves?
Who says they need to defend themselves? If I come across somebody making unreasonable demands I ignore them.
I am not asking for an exact answer, just an approximate one. And if they honestly have no idea, then they should say that.
I want to call out this attitude as incredibly entitled and related to the exact root cause of all the open-source drama we've been seeing lately.
[deleted]
To add to this there are* serious issues with naming. I wrote an app that ran on IPFS last year and throughout testing there was a persistent problem with needing to issue the testers with a new hash/link for each new version because IPNS, IPFS' decentralized naming solution, effectively doesn't work. It's supposed to give each node a unique name which can be repeatedly mapped to different hashes but publishing a new name -> hash mapping can take several minutes and then lookup of that name can take several minutes even on the node that published it in the first place. Names also need to be republished every 24 hours or so or they disappear from the network, which in my opinion harms the claims of decentralization.
* = unless something has radically changed in the last few months and they just haven't closed the GitHub issue: https://github.com/ipfs/go-ipfs/issues/3860
* = unless something has radically changed in the last few months and they just haven't closed the GitHub issue: https://github.com/ipfs/go-ipfs/issues/3860
I consider IPNS fundamentally broken and have stopped using it. ENS is better in almost every way and is extremely reliable so it's not that big of deal.
IPFS and ENS work pretty seamlessly together (https://medium.com/the-ethereum-name-service/ethdns-9d56298f...), but some folks also want the mutable pointers layer of IPNS too. We're working on making that faster: https://www.youtube.com/watch?v=XniIDIXU8RE&t=10s
How can I register an ENS domain/id? I've found very little practical info on a cursory search.
There's probably a simple article out there but I'll summarize it a bit:
You go to https://manager.ens.domains/ to register a .eth address and on the settings page for your domain you add a Content record and point it to ipfs://Qma7nxS98CAb2BTaxLJBSFozxxSp5XTu6tMtCrKcRQ9ByV or whatever the IPFS hash of your website is.
After that you can visit your site at https://myname.eth in a browser that supports ENS/IPFS or you can go to https://myname.eth.link in any browser and see your website.
You go to https://manager.ens.domains/ to register a .eth address and on the settings page for your domain you add a Content record and point it to ipfs://Qma7nxS98CAb2BTaxLJBSFozxxSp5XTu6tMtCrKcRQ9ByV or whatever the IPFS hash of your website is.
After that you can visit your site at https://myname.eth in a browser that supports ENS/IPFS or you can go to https://myname.eth.link in any browser and see your website.
This is what I needed, thanks!
I really wish everything written about IPFS would just ignore IPNS, and stop presenting it as a standard part of IPFS. The thing IPNS tries to do can be done better by the existing DNS system (which works fine combined with IPFS) or blockchain-based systems like ENS. IPNS is a slow incomplete tech demo glued to the rest. Ignoring IPNS won't impact your understanding of the rest of IPFS. I don't think people relying on IPFS even use IPNS for anything but a curiosity, and it's so bad that I expect many people get turned off of IPFS because they get confused by IPNS's flaws or assume that the rest of IPFS works as badly.
IPFS is good at content-addressed storage. IPNS is not part of content-addressed storage. Tutorials about IPFS should ignore IPNS, and talk about alternative ways to link to IPFS links, like using DNS.
IPFS is good at content-addressed storage. IPNS is not part of content-addressed storage. Tutorials about IPFS should ignore IPNS, and talk about alternative ways to link to IPFS links, like using DNS.
> IPFS still has a long way to go until it is useable in my opinion.
Now we just need to figure out what to use it for. I think the largest successful experiment of distributed file storage was Wuala. At the beginning it was similar to IPFS, using random home devices to store data, doing some clever calculation and splitting of files to guarantee that a file is likely always available. Emphasis on likely.
They had nice features:
- keep files private
- share files with other registered users
- share files with unregistered users, through a keyed hyperlink
- publish files
- backup
- file synchronization
- file versioning
At the end they migrated away from using random home nodes for storage and went for central servers. Not sure about the motivation behind it but I would really like to know. Did that happen because of business requirements or some technical limitations?
Now we just need to figure out what to use it for. I think the largest successful experiment of distributed file storage was Wuala. At the beginning it was similar to IPFS, using random home devices to store data, doing some clever calculation and splitting of files to guarantee that a file is likely always available. Emphasis on likely.
They had nice features:
- keep files private
- share files with other registered users
- share files with unregistered users, through a keyed hyperlink
- publish files
- backup
- file synchronization
- file versioning
At the end they migrated away from using random home nodes for storage and went for central servers. Not sure about the motivation behind it but I would really like to know. Did that happen because of business requirements or some technical limitations?
I was a huge fan of Wuala, and was very disappointed when they got bought and then shut down. My understanding is they always stored a central copy of everything on their servers to 100% guarantee availability. The P2P copies merely helped with bandwidth. I think their main innovation is the cryptree data structure for access control. We use a more advanced version of that in Peergos[1].
[1] https://github.com/peergos/peergos
[1] https://github.com/peergos/peergos
Building on top of other people's infrastructure is expensive.
Each node can disappear at any time and is really hard to predict. To compensate for that you will need to upload your data to more nodes. You have to deal with backward and forward-compatible updates because both client and server versions updates are also not under your control. There are also more complicated issues of trust and bad actors that can affect the product.
Now you have a datacenter. The clients upload their data exactly once. Your server software has at most two concurrent versions during deployment. Security is handled in a traditional fashion with DDoS firewalls and pentesting. Your whole software stack is much simpler because you can build on top of more solid assumptions.
Skype also used to be P2P until the Microsoft acquisition where they moved towards hosted nodes. Some people say it's so that Microsoft can spy on everybody but I really believe that the main reason was so that they could improve the QoS.
Each node can disappear at any time and is really hard to predict. To compensate for that you will need to upload your data to more nodes. You have to deal with backward and forward-compatible updates because both client and server versions updates are also not under your control. There are also more complicated issues of trust and bad actors that can affect the product.
Now you have a datacenter. The clients upload their data exactly once. Your server software has at most two concurrent versions during deployment. Security is handled in a traditional fashion with DDoS firewalls and pentesting. Your whole software stack is much simpler because you can build on top of more solid assumptions.
Skype also used to be P2P until the Microsoft acquisition where they moved towards hosted nodes. Some people say it's so that Microsoft can spy on everybody but I really believe that the main reason was so that they could improve the QoS.
I know so many projects with barely any popularity being forced to move off of IPFS cause of performance problems.
DAT,SSB don't work in browser.
WebTorrent is the original better choice.
But I'm seeing most big projects move to GUN (mine, https://github.com/amark/gun). Internet Archive, HackerNoon others using it in production, with over 10M+ users, decentralized.
If IPFS team doesn't seriously fix their performance issue within 2 years, they're gonna see a lot of backlash. With $250M+ raised and 100+ employees this should be possible, I know they can do it, dWeb would be better for it.
DAT,SSB don't work in browser.
WebTorrent is the original better choice.
But I'm seeing most big projects move to GUN (mine, https://github.com/amark/gun). Internet Archive, HackerNoon others using it in production, with over 10M+ users, decentralized.
If IPFS team doesn't seriously fix their performance issue within 2 years, they're gonna see a lot of backlash. With $250M+ raised and 100+ employees this should be possible, I know they can do it, dWeb would be better for it.
The thing I worry about with IPFS is privacy. If you use IPFS directly (as intended, not via a public gateway), and you visit a site, then you are automatically going to be seeding (like a torrent) the visited content, and thus you will be announcing/broadcasting the fact to the world that you (your node/your IP) have visited it. My current understanding is that this cannot really be avoided, since one needs to be able to find the nodes that have the content for any given hash.
IPFS really needs onion routing built-in. When I was reading this, I thought about running an IPFS node on my desktop but remembered it doesn't have onion routing built-in. So I'd have to run a proxy like torsocks and torify the IPFS server. Then I have to worry about all the ways that IP can leak from this. Then I have to figure out how to connect to other IPFS nodes through TOR. So forget it, I'll do it when I have time, which is the same thing I said the last time I thought about running an IPFS node.
At that point aren't you looking for Freenet/Zeronet?
Well, one can use IPFS through Tor, and use gateways that are Tor onion services.
We do this for OpenBazaar which is built on top of ipfs and it works great. We built a Tor transport for libp2p, which is what drives ipfs's p2p networking so any libp2p app, including ipfs, can work over Tor.
https://github.com/OpenBazaar/go-onion-transport
https://github.com/OpenBazaar/go-onion-transport
Indeed. And that's where I first learned about IPFS via Tor.
It was y'all that actually got IPFS working via Tor.
It was y'all that actually got IPFS working via Tor.
Obfuscating your IP doesn't solve the problem. If a malicious actor knew someone's info (such as address), they could give them an ipfs link with CP and report them.
Sure. You also gotta hide your name, address and all other information that could identify you. Otherwise, there's no point in obfuscating your IP.
What is the legal standard for say YouTube about not carrying illicit content? Just follow that. Make a list of banned hashes or something of the sort.
there's something weirdly poetic of having information reborn through voluntary noise
Good point about the experimental Tor support and the Onion IPFS gateways.
Is there a reasonable way to use untrusted gateways while upholding the data integrity guarantees? I think it should be possible in theory.
Is there a reasonable way to use untrusted gateways while upholding the data integrity guarantees? I think it should be possible in theory.
Because data on ipfs is content addressed by hash you can always check the integrity of data you receive, in theory, although the tooling isn't there for you to easily verify the output of a gateway without your own ipfs node afaik.
The other potential possibility/issue that I would be uncomfortable with, would be a bad actor visiting and then seeding my content. Say someone also seeding or just locally having child pornography or some other nefarious thing, and now they are also an aggregator for your non-nefarious content. Just the possibility of that type of potential unintended association gives me pause in terms of considering it for a blog or whatever else.
I got a downvote so I guess people aren’t concerned about this or believe my concern not valid or naive?
It doesn't seem meaningfully different from a bad actor having, say, a tab with your Wordpress blog open on their computer.
Or to pick something closer to the point, someone seeding a copyrighted film and also a Linux distribution. That sort of coexistence has gone on for many years without causing problems for the legal uses of torrents.
Or to pick something closer to the point, someone seeding a copyrighted film and also a Linux distribution. That sort of coexistence has gone on for many years without causing problems for the legal uses of torrents.
Thanks for the feedback. I get your point there. It feels much different though. A tab open is completely read only, and they are not aggregating your content. I guess I just don’t like the idea of seeding my content and not knowing who or what the the “seeders” might also be involved with. I guess purely an ethical concern/decision on my part in likely choosing never to use a seeding technology to serve my content.
That almost seems like a slippery slope. I mean, your site’s content could now just end up sitting on some CDN server or proxy alongside something nefarious, without IPFS and you’d have no idea.
I’d have the same objection if the seeder was voluntarily choosing what to seed for that express purpose, but I don’t know that it should apply if the seeding is just an automatic part of how the network operates underneath.
I’d have the same objection if the seeder was voluntarily choosing what to seed for that express purpose, but I don’t know that it should apply if the seeding is just an automatic part of how the network operates underneath.
> I got a downvote
Have another one for complaining about it. Wear most with pride and some have been proven to be well deserved. Let history be the judge rather than your fragile ego hey.
Otherwise feel free to go back to Reddit.
Have another one for complaining about it. Wear most with pride and some have been proven to be well deserved. Let history be the judge rather than your fragile ego hey.
Otherwise feel free to go back to Reddit.
Maybe a software delay where it only starts seeding 5 minutes later, or an opt-out button where you can avoid re-seeding if you don't like what you see.
IPFS isn't designed for private communications. It's designed for highly decentralized publishing of content. Safe, in this context, means the data is safe to "be".
I understand that no one is going to tamper with the data. Everybody (who is running a full node) is gonna get what they ask for, but then they are gonna go and announce it to the world that they have it, and that worries me privacy-wise.
> that worries me privacy-wise
This should be addressed by the client, not the server. The server's function is to serve the data, not serve data and pretend it doesn't know who it gave it to.
This should be addressed by the client, not the server. The server's function is to serve the data, not serve data and pretend it doesn't know who it gave it to.
Since IPFS is peer to peer, who is the server in this context?
Honestly I think the simplest and most effective method is to keep things as they are and have archive.org/.is keep backups of websites. A backup on archive.org is much more likely to stick around after 10 years than the users seeding a file.
Not if there is a government issued takedown notice it isn't.
IPFS has the same problems given a takedown notice[0]. Although you could continue serving the content and convince others to serve the content, any entity can take the next step and go after you.
[0]: https://github.com/ipfs/notes/issues/284
[0]: https://github.com/ipfs/notes/issues/284
This also seems to be true in theory regarding torrents but not in practice.
Has Cloudflare made any commitments regarding the longterm availability of their IPFS gateway?
Probably not, but why does that matter? It's like asking whether Mozilla have made any commitments regarding the longterm availability of Thunderbird, just use another gateway or run your own.
The really awesome thing about gateways over central hosts is if one goes offline you can still easily access & verify the content from one of the many other gateways (https://ipfs.github.io/public-gateway-checker/). =]
Disclaimer: I run the unofficial IPFS Discord and Matrix (found at https://permaweb.io/discord and /matrix) and have helped organize IPFS Meetups in SF. We also run an IPFS gateway and have built a groups app on top of IPFS and Textile.
I generally agree with the conclusion, but there's a few downsides that aren't conveyed here.
Let's look at the proposed upsides: 1) Ownership, control, censorship: That's partly correct. Ownership is fair, in the sense that you can run your node and self-host. However this is true of any self-hosting solution. You could run a Docker instance of a Wordpress or Ghost site and get ownership / control.
2) The point about censorship is muddied, however. I'll combine that point with the second upside: Resilience. Every day for the past two years, I've seen people wonder if IPFS is a magical cloud with infinite storage. People seem to think you put a file on IPFS, and it just gets replicated, censorship resistant hosting. That's not how it works. People need to pin your hash. You need to tell the world about your hash somehow. All this is done via a public list of IPs that is being broadcasted. Think of IPFS this way: you're letting people with the hash become CDNs of your content. That's cool, but that doesn't solve discovery, keeping things up, etc. IPFS doesn't encrypt the content, or the connectivity, or hide the hosts. Solutions exist around that, but they're niche, and honestly I question the motives besides just ideology.
3) Elegance. Yeah it's a really, really cool way to solve linking. As some others pointed out, it's not as fast as classic centralized links, so it's better suited currently for solutions that don't require speed.
I generally agree with the conclusion, but there's a few downsides that aren't conveyed here.
Let's look at the proposed upsides: 1) Ownership, control, censorship: That's partly correct. Ownership is fair, in the sense that you can run your node and self-host. However this is true of any self-hosting solution. You could run a Docker instance of a Wordpress or Ghost site and get ownership / control.
2) The point about censorship is muddied, however. I'll combine that point with the second upside: Resilience. Every day for the past two years, I've seen people wonder if IPFS is a magical cloud with infinite storage. People seem to think you put a file on IPFS, and it just gets replicated, censorship resistant hosting. That's not how it works. People need to pin your hash. You need to tell the world about your hash somehow. All this is done via a public list of IPs that is being broadcasted. Think of IPFS this way: you're letting people with the hash become CDNs of your content. That's cool, but that doesn't solve discovery, keeping things up, etc. IPFS doesn't encrypt the content, or the connectivity, or hide the hosts. Solutions exist around that, but they're niche, and honestly I question the motives besides just ideology.
3) Elegance. Yeah it's a really, really cool way to solve linking. As some others pointed out, it's not as fast as classic centralized links, so it's better suited currently for solutions that don't require speed.
> IPFS doesn't encrypt the content, or the connectivity, or hide the hosts. Solutions exist around that, but they're niche, and honestly I question the motives besides just ideology.
I question the motives of people who would be against encryption, aside from it being a lot of work and just not having been done yet. Ideally: no one should know the true IPs of their peers, and no one snooping on the connection should be able to read anything useful. Even the contents of the files should be encrypted, but I struggle to see how that could be easily implemented (maybe like Mega.NZ does where the key is part of the URL).
Otherwise it's going to be very hard to convince me to host arbitrary content from untrusted strangers. It's just pragmatic. No encryption = no plausible deniability.
I question the motives of people who would be against encryption, aside from it being a lot of work and just not having been done yet. Ideally: no one should know the true IPs of their peers, and no one snooping on the connection should be able to read anything useful. Even the contents of the files should be encrypted, but I struggle to see how that could be easily implemented (maybe like Mega.NZ does where the key is part of the URL).
Otherwise it's going to be very hard to convince me to host arbitrary content from untrusted strangers. It's just pragmatic. No encryption = no plausible deniability.
I totally agree, but want to emphasize one thing.
Let's say that stuff was encrypted during transmission, and also when stored by peers. But peers could see each other's true IPs. That would basically give you a modern version of Freenet. And just like with Freenet, users could be arrested, and prosecuted based on hand waving. When you get down to it at trial, "plausible deniability" depends on having a suitable expert witness, and convincing a jury that the prosecution's expert witness is full of it.
So anyway, none of that helps unless true IPs of peers are hidden.
Let's say that stuff was encrypted during transmission, and also when stored by peers. But peers could see each other's true IPs. That would basically give you a modern version of Freenet. And just like with Freenet, users could be arrested, and prosecuted based on hand waving. When you get down to it at trial, "plausible deniability" depends on having a suitable expert witness, and convincing a jury that the prosecution's expert witness is full of it.
So anyway, none of that helps unless true IPs of peers are hidden.
> IPFS doesn't encrypt the content, or the connectivity, or hide the hosts. Solutions exist around that, but they're niche, and honestly I question the motives besides just ideology.
Hiding connectivity metadata and host identity are essential to protect users from adversaries.
Hiding connectivity metadata and host identity are essential to protect users from adversaries.
I agree with you, but I suspect your observation is circular in this context. If we define "protect" and "adversaries" appropriately, couldn't this be true of any measure?
I'm not sure what you mean by "any measure".
What I'm recommending would protect all sorts of users from all sorts of adversaries. So it's nonjudgmental.
But that doesn't mean that it's "circular".
What I'm recommending would protect all sorts of users from all sorts of adversaries. So it's nonjudgmental.
But that doesn't mean that it's "circular".
I'm wondering if IPFS might be better thought of as a common data-publishing protocol that might be used to push content to any number of CDN's?
So, you could publish content to IPFS and tell your favorite CDN to pick it up, and you pay them to keep it active. But IPFS isn't limited to one CDN, so you could always pick another one. And your users could also go through a different CDN. Or some people who really want to could run their own CDN and pin whatever they want to host.
So, you could publish content to IPFS and tell your favorite CDN to pick it up, and you pay them to keep it active. But IPFS isn't limited to one CDN, so you could always pick another one. And your users could also go through a different CDN. Or some people who really want to could run their own CDN and pin whatever they want to host.
The Dat protocol was supposed to play that role (https://dat.foundation/)
> I'm wondering if IPFS might be better thought of as a common data-publishing protocol that might be used to push content to any number of CDN's?
Think of the bits that work properly as "the web of BitTorrent magnet: links" and you'll have a reasonably accurate picture.
Think of the bits that work properly as "the web of BitTorrent magnet: links" and you'll have a reasonably accurate picture.
Interesting name choice. The term “permaweb” is typically associated with Arweave and their decentralized permanent storage offering [1].
1. https://techcrunch.com/2019/11/05/how-arweave-permaweb-works...
1. https://techcrunch.com/2019/11/05/how-arweave-permaweb-works...
Out of curiosity, isn’t the DNS gateway like Cloudflare (https://blog.cloudflare.com/distributed-web-gateway/) a single point of failure? Is there a solution to this without having to resort to a completely different desktop app?
Well there are many gateways which provides redundancy but if ipfs really takes off I think it'll eventually need to be supported by browsers so that you're using your own ipfs node for lookups but it's seamless for you the user or developer.
Agreed - having a lot of gateways helps (https://ipfs.github.io/public-gateway-checker/), but eventually integration directly with browsers is the upgrade path: https://blog.ipfs.io/2019-10-08-ipfs-browsers-update/
[deleted]
If you replace the word "IPFS" with "BitTorrent," this article is still true. Similarly, if you replace "IPFS" with "BitTorrent" in most of the comments here, the comments are still true.
If you understand how BitTorrent works -- including its strengths and limitations -- you'll understand how IPFS works.
If you understand how BitTorrent works -- including its strengths and limitations -- you'll understand how IPFS works.
Minor nit: BitTorrent does not allow one to maintain a persistent URI to changing/updated data, as IPNS does.
[deleted]
I'm seeing others here point out that IPNS basically doesn't work - so it doesn't really either, in practice.
Depending on how you measure it: RSS feeds for updates fit this bill, and they've been around for a decade or more: https://github.com/KDE/ktorrent/blame/3ce105f4d1c3b4c178aaf3...
RSS feeds are centralized and rely on web servers and DNS and are thus straightforward to censor or otherwise force offline via government orders, ddos, provider legal threats, et c.
Or sneakernet-distributed RSS files, or visiting IP addresses directly, or spread over gossip, or replace dns with namecoin...
Broadly I agree, but IPNS isn't really part of IPFS, it's an external thing that fakes mutability of another thing that cannot be changed. There are many examples of this kind of thing (e.g. DNS itself, abstracting over IP addresses), with an incredibly wide variety to deal with the various tradeoffs, and IPFS itself not hardcoding a single approach is a good thing. BitTorrent doesn't either, but some tactics have sprung up organically, as has occurred for IPFS (e.g. https://www.increaseo.com/eth-domains-ipfs/).
Broadly I agree, but IPNS isn't really part of IPFS, it's an external thing that fakes mutability of another thing that cannot be changed. There are many examples of this kind of thing (e.g. DNS itself, abstracting over IP addresses), with an incredibly wide variety to deal with the various tradeoffs, and IPFS itself not hardcoding a single approach is a good thing. BitTorrent doesn't either, but some tactics have sprung up organically, as has occurred for IPFS (e.g. https://www.increaseo.com/eth-domains-ipfs/).
Hate to break it to you, but this generally true about Internet infrastructure. IPFS isn't going to fix that. In fact, IPFS, like BitTorrent, is trivial to DDoS by inserting malicious nodes in the DHT in order to block key lookups.
Not so minor!
Stopped reading after "Medium's engineers are much better at all of this than you will ever be"; can't take this writer seriously.
I read this as: a group of professional engineers dedicated to hosting/scaling a platform designed for hosting blogs will perform better than a single random person managing a personal blog in their spare time. I generally think that's accurate.
I dislike this formula of "Stopped reading after ..." that I seem to be seeing more frequently.
Why not just make your criticism of the quoted text straight up? Why the rhetorical flounce out of the room?
Why not just make your criticism of the quoted text straight up? Why the rhetorical flounce out of the room?
I agree with your observation on how to improve my comment but the comment was past the edit grace period so i could not update it.
Might just be their way of being funny, I wouldn't read too much into it. Self-deprecation is a common social cue to trigger some light, mundane empathy:
"I struggle, you struggle, (thus) we relate!"
I blame it on the spirit that being last in school is way cooler than being first, and I suspect it's as old as humanity.
"I struggle, you struggle, (thus) we relate!"
I blame it on the spirit that being last in school is way cooler than being first, and I suspect it's as old as humanity.
God, another IPFS shilling.
Can't they come back when it's really usable?
Can't they come back when it's really usable?
When I first learned about content addressing a couple years ago, it sounded like the holy grail. I'm less certain now. It seems much better from the machine perspective, but I'm not sure it matches closely enough the way humans interact with data. We are spatial and temporal creatures. There's something unsettling about my video file being chopped up into a million chunks and stored who-knows-where. Compared to the path/URL approach where I have a single large file. I know where it lives and how big it is. Moving/copying/deleting/updating/etc are all intuitive and map to analogs in the physical world. Content addressing (and object storage system I might add) isn't as easy to reason about.
Don't get me wrong. Content addressing is very cool, and may revolutionize everything. It's very elegant. I'm just a bit skeptical.
Don't get me wrong. Content addressing is very cool, and may revolutionize everything. It's very elegant. I'm just a bit skeptical.
Technically content addressing and chunking (chopping up files) are two orthogonal techniques. You could envision a system where the hash of a single raw file (no matter how big it is) is it's identifier, and you can then have a mapping between that identifier and the locations (e.g. IP addresses) of servers who store the whole file.
You are of course correct. I was implying current content-addressing systems, such as IPFS.
Im intriqued by the idea of IPFS, but it does seem like you're giving up a lot of control - Has this been used for anything >= mid-scale yet?
Not to be a downer on IPFS at all, btw. I'm very glad that both it and Dat exist. IPFS has always seemed like a much larger undertaking and it is cool thst they are trying to push the dweb even further. We need that just as much as we need Dat, which with its inclusion in the Beaker browser for example really serves as a super cool demo of what dweb can give us in the future.