IPFS 0.5(blog.ipfs.io)
blog.ipfs.io
IPFS 0.5
https://blog.ipfs.io/2020-04-28-go-ipfs-0-5-0/
144 comments
Or at least a Docker image if there isn't already?
Bandwidth limiting is something the OS should do, not individual applications/daemons.
Sure, but the problem remains. You can't expect a user to download a tool and then configure their OS to limit the application. Developers / Linux users might do that, but someone on Windows or OSX won't, I don't think the OS even exposes those controls to the user in a easy manner.
So, while we get there, applications/tools can also manage their bandwidth, so the default settings are more gentle with using users bandwidth.
So, while we get there, applications/tools can also manage their bandwidth, so the default settings are more gentle with using users bandwidth.
plus IPFS wants to enforce copyrights worldwide: [1,2]
Businesses principles plus monthly bandwidth usage matter for real people.
[1] https://github.com/ipfs/community/blob/master/code-of-conduc...
[2] https://discuss.ipfs.io/tos#8
[1] https://github.com/ipfs/community/blob/master/code-of-conduc...
[2] https://discuss.ipfs.io/tos#8
Our TOS applies to the IPFS HTTP Gateway where Protocol Labs run the infrastructure (bridging data in the IPFS Network to users over HTTP) to ease onboarding/development. There are many different IPFS Gateways (https://ipfs.github.io/public-gateway-checker/), with different local jurisdictions that can each choose their own TOS.
We do not and cannot control the data that each individual node is hosting in the IPFS Network. Each node can choose what they want to host - no central party can filter or blacklist content globally for the entire network. The ipfs.io Gateway is just one of many portals used to view content stored by third parties on the Internet.
That aside, we definitely don't want to 'apply copyright worldwide'. For one, it's not consistent! Trying to create a central rule for all nodes about what is and isn't "allowed" for the system as a whole doesn't work and is part of the problem with more centralized systems like Facebook, Google, Twitter, etc. Instead, give each node the power to decide what it does/doesn't want to host, and easy tooling to abide by local requirements/restrictions if they so choose.
We do not and cannot control the data that each individual node is hosting in the IPFS Network. Each node can choose what they want to host - no central party can filter or blacklist content globally for the entire network. The ipfs.io Gateway is just one of many portals used to view content stored by third parties on the Internet.
That aside, we definitely don't want to 'apply copyright worldwide'. For one, it's not consistent! Trying to create a central rule for all nodes about what is and isn't "allowed" for the system as a whole doesn't work and is part of the problem with more centralized systems like Facebook, Google, Twitter, etc. Instead, give each node the power to decide what it does/doesn't want to host, and easy tooling to abide by local requirements/restrictions if they so choose.
I think you misread. These terms apply to "public IPFS infrastructure" (I read that as things like build services, gateways or bootstrap notes) and the ipfs.io website, not to the IPFS network as a whole.
The IPFS network itself is decentralized, there's no central authority that can police copyrighted content on individual nodes.
The IPFS network itself is decentralized, there's no central authority that can police copyrighted content on individual nodes.
I'm not sure if I understand the complaint around this.. If you put a file (that you do not have permission to distribute) on the internet in a public-accessible location, you should expect that someone will want it removed.
IPFS is not trying to be an anonymous file-sharing service afaik.
IPFS is not trying to be an anonymous file-sharing service afaik.
The complaint is they shouldn't take a stance or at least one that is a little less firm.
The permissions thing, we can all agree if it's a new movie that just came out, ok, yes, don't be spreading that.
What if instead it's an academic journal article from 1930 in a publication that ceased operating in say 1940? You also don't have permission for this and it's still also under copyright.
The strict interpretation would be "not that one either" while there's also some who say "it's ok, there's nobody to even ask, let historians do research".
So some prefer to be grey about it like many are with obscenity and pornography. We don't for example, hide renaissance paintings with exposed beasts away from the public in basements for fear of getting shutdown by the police.
There's a spirit of the law as well.
The permissions thing, we can all agree if it's a new movie that just came out, ok, yes, don't be spreading that.
What if instead it's an academic journal article from 1930 in a publication that ceased operating in say 1940? You also don't have permission for this and it's still also under copyright.
The strict interpretation would be "not that one either" while there's also some who say "it's ok, there's nobody to even ask, let historians do research".
So some prefer to be grey about it like many are with obscenity and pornography. We don't for example, hide renaissance paintings with exposed beasts away from the public in basements for fear of getting shutdown by the police.
There's a spirit of the law as well.
For IPFS to work, there needs to be no central authority who can block files.
Todays IPFS is a long way from that - any Joe Random can DoS any particular hash by getting their node at the right place in the DHT and blackholing requests.
Todays IPFS is a long way from that - any Joe Random can DoS any particular hash by getting their node at the right place in the DHT and blackholing requests.
> can DoS any particular hash
Can you explain more how this is possible?
So we have one evil user Karen who wants to block access to content ABC.
She will spam the DHT with requests to content ABC. After a while, nodes will stop responding as she hits the rate limit. Now her DHT requests goes into the void.
Now Joey wants to request content ABC too. He requests the content, and because no other nodes are responding to Karens requests, they responds to Joeys request for the content. Now he can fetch the content.
Can you explain more how this is possible?
So we have one evil user Karen who wants to block access to content ABC.
She will spam the DHT with requests to content ABC. After a while, nodes will stop responding as she hits the rate limit. Now her DHT requests goes into the void.
Now Joey wants to request content ABC too. He requests the content, and because no other nodes are responding to Karens requests, they responds to Joeys request for the content. Now he can fetch the content.
There's a smarter attack...
Every node in the network 'owns' some of the keyspace.
Karen can simply keep reconnecting to the network (brute forcing her PeerID, which determines which bit of the keyspace her node will be responsible for) till she gets assigned that bit of keyspace. Then she can black hole requests to it.
You can defend against that by having multiple owners for a given bit of keyspace (known as quorum in the IPFS design), but evil Karen can simply pretend to be all of the machines hosting that bit of keyspace.
The brute forcing sounds hard, but in a million node IPFS network, on average you only need to do 1 million sha256 hashes, which takes under a second on modern hardware.
Every node in the network 'owns' some of the keyspace.
Karen can simply keep reconnecting to the network (brute forcing her PeerID, which determines which bit of the keyspace her node will be responsible for) till she gets assigned that bit of keyspace. Then she can black hole requests to it.
You can defend against that by having multiple owners for a given bit of keyspace (known as quorum in the IPFS design), but evil Karen can simply pretend to be all of the machines hosting that bit of keyspace.
The brute forcing sounds hard, but in a million node IPFS network, on average you only need to do 1 million sha256 hashes, which takes under a second on modern hardware.
> For IPFS to work, there needs to be no central authority who can block files.
There isn't one.
> any Joe Random can DoS any particular hash
Sounds pretty decentralized to me!
There isn't one.
> any Joe Random can DoS any particular hash
Sounds pretty decentralized to me!
[deleted]
This may only apply to their own ipfs.io gateway
It seems global copyright enforcement with blacklisting and blocklists:
https://github.com/ipfs/notes/issues/284
(I've testified in US federal court in copyright cases as the expert witness professor. I always try to investigate the DMCA decisions of developers. Difficult tradeoffs.)
_Optional_ (up to a node operator) global copyright enforcement.
Of course they have to play ball with existing legislation in business settings.
Of course they have to play ball with existing legislation in business settings.
Note that outside of special cases like a publicly-accessible gateway, an IPFS node is not supposed to fetch or retransmit data that the node operator has not specifically requested to be stored there. So this copyright enforcement stuff will always mostly apply to these services. (There might of course be some corner cases such as fetching a mutable IPNS resource and then rejecting the data because it happens to match some blacklist, but these are also broadly sensible.)
As hobofan said, this is up to each node to update/block. It really seems like that the TOS only goes for their gateway, I don't see how they could enforce this on the entire network.
Good progress. I looked at IPFS a few months ago from the point of view of fitting a decentralized file store. The decentralized requirement came primarily from the point of view wanting to fit this in a broader product that is all about decentralized for various reasons.
Key problems in this space:
- decentralized is something techies obsess about but that has as of yet no business value whatsoever. Customers don't ask for it. Centralized alternatives are generally available and far more mature/easy to manage. The business ecosystem around IPFS is basically not there.
- this space is dominated by hobbyists running stuff like this on their personal hardware doing this mostly for idealistic or other non incentivized (i.e. money) reasons. Nothing wrong with this but using it for something real brings a few requirements with it that are basically hard to address currently.
- Filecoin has been 'coined' as the solution for this for years but seems nowhere near delivering on it's published roadmap. Last time I checked it had undelivered milestones in the past. As of yet this looks increasingly like something that is a bit dead in the water / a big distraction for coming up with better/alternate solutions.
- Uptime guarantees for content are currently basically DYI. Nobody but you cares about your data. If you want your content to stay there, you basically need ... proper file hosting. As incentives and mechanisms for others to agree to host your content (aka pinning in ipfs) are not there, this is hard to fix.
- integration with existing centralized solutions is kind of meh/barely supported. We actually looked at using s3 as store for ipfs just so we could onboard customers and give them some decent SLA (bandwidth would be another issue for that). There are some fringe projects on github implementing this but when we looked at it the smallish blocksizes in ipfs are kind of a non starter for using this at scale (think insane numbers of requests to s3). This stuff is not exactly mainstream. Obviously this wouldn't be needed if we could incentivize others to 'pin' content. But we can't currently.
Key problems in this space:
- decentralized is something techies obsess about but that has as of yet no business value whatsoever. Customers don't ask for it. Centralized alternatives are generally available and far more mature/easy to manage. The business ecosystem around IPFS is basically not there.
- this space is dominated by hobbyists running stuff like this on their personal hardware doing this mostly for idealistic or other non incentivized (i.e. money) reasons. Nothing wrong with this but using it for something real brings a few requirements with it that are basically hard to address currently.
- Filecoin has been 'coined' as the solution for this for years but seems nowhere near delivering on it's published roadmap. Last time I checked it had undelivered milestones in the past. As of yet this looks increasingly like something that is a bit dead in the water / a big distraction for coming up with better/alternate solutions.
- Uptime guarantees for content are currently basically DYI. Nobody but you cares about your data. If you want your content to stay there, you basically need ... proper file hosting. As incentives and mechanisms for others to agree to host your content (aka pinning in ipfs) are not there, this is hard to fix.
- integration with existing centralized solutions is kind of meh/barely supported. We actually looked at using s3 as store for ipfs just so we could onboard customers and give them some decent SLA (bandwidth would be another issue for that). There are some fringe projects on github implementing this but when we looked at it the smallish blocksizes in ipfs are kind of a non starter for using this at scale (think insane numbers of requests to s3). This stuff is not exactly mainstream. Obviously this wouldn't be needed if we could incentivize others to 'pin' content. But we can't currently.
> decentralized is something techies obsess about but that has as of yet no business value whatsoever.
I don't really understand that point. If it works, then it has the business value of saving you all running server and maintenance costs. For most larger businesses these costs may be insignificant and easily recovered, but for small businesses with a lot of customers they can make a huge difference. For example, I'm looking into P2P options for implementing a decentralized message forum in a game-like emulator and it would make no sense to even implement this feature with constant running costs for server space.
Now getting the decentralized data management to work reliably out-of-the-box from behind various firewalls and different platforms, that's the big problem. So far, none of the libraries I've seen are very easy to use, some require a difficult installation and configuration or you need to your own STUN server or gateway, which kind of defeats the purpose.
I don't really understand that point. If it works, then it has the business value of saving you all running server and maintenance costs. For most larger businesses these costs may be insignificant and easily recovered, but for small businesses with a lot of customers they can make a huge difference. For example, I'm looking into P2P options for implementing a decentralized message forum in a game-like emulator and it would make no sense to even implement this feature with constant running costs for server space.
Now getting the decentralized data management to work reliably out-of-the-box from behind various firewalls and different platforms, that's the big problem. So far, none of the libraries I've seen are very easy to use, some require a difficult installation and configuration or you need to your own STUN server or gateway, which kind of defeats the purpose.
Many service providers, e.g. CloudFlare or AWS, among others, will offer sufficient resources for smaller sites for free, in hopes of capturing larger returns if that company scales. This largely negates the specific benefit (cost for small orgs) that you're referring to.
Saying there's "no business value" in tools coming out of this space/building on IPFS is nonsense. Sure, the vast majority of people won't care about using a tool because it's decentralized in and of itself, but they _will_ choose tools that give them features they care about - like working better offline, having faster downloads/uploads, being more reliable and trustworthy, preserving their agency and control, etc. These implications from building a distributed system (ex able to collaborate when disconnected from central servers) are strong business reasons for folks to use one service over another, but that isn't clear from saying "it's decentralized". Look at Netflix using IPFS for faster container image distribution (business value: more productive developers - https://blog.ipfs.io/2020-02-14-improved-bitswap-for-contain...), Audius & Matters.news using IPFS to give content creators more ownership over their own data, Berty & OpenBazaar using IPFS for private messaging/marketplaces -- clearly there is business value in removing middle-men/central points of failure & control in these applications.
You absolutely can incentivize others to pin content. Check out Infura (infura.io), Pinata (pinata.cloud), Temporal (temporal.cloud), and Eternum (eternum.io) - these are all services you can pay to host your IPFS data with reasonable uptime. They have an incentive to keep your content around because you pay them to. Filecoin is a distributed solution to that (and making active progress - they have a testnet out with over 5PB of proven storage: https://filecoin.io/blog/roadmap-update-april-2020/), but you don't have to wait for that to land.
You absolutely can incentivize others to pin content. Check out Infura (infura.io), Pinata (pinata.cloud), Temporal (temporal.cloud), and Eternum (eternum.io) - these are all services you can pay to host your IPFS data with reasonable uptime. They have an incentive to keep your content around because you pay them to. Filecoin is a distributed solution to that (and making active progress - they have a testnet out with over 5PB of proven storage: https://filecoin.io/blog/roadmap-update-april-2020/), but you don't have to wait for that to land.
> decentralized is something techies obsess about but that has as of yet no business value whatsoever.
Resilio (originally part of BitTorrent) specializes in decentralized storage synchronization for business.
Though we can't see if they are profitable their business use cases seem compelling especially if you're reluctant to otherwise use the cloud.
https://www.crunchbase.com/organization/resilio#section-over...
https://www.resilio.com/connect/
Resilio (originally part of BitTorrent) specializes in decentralized storage synchronization for business.
Though we can't see if they are profitable their business use cases seem compelling especially if you're reluctant to otherwise use the cloud.
https://www.crunchbase.com/organization/resilio#section-over...
https://www.resilio.com/connect/
Siacoin seems much farther along if you just want a decentralized file store solution where you can pay others to provide the storage/bandwidth.
> The Inter-Planetary Name System (IPNS), our system for creating mutable links for content addresses, now provides faster naming lookups and has a new experimental pubsub transport to speed up record distribution. Providing an IPNS record is now 30-40x faster in 1K node network simulations!
IPNS performance was abysmal last time I tried to use ipfs (took > 30 seconds a fair amount of the time). Curious to see what it's like now.
IPNS performance was abysmal last time I tried to use ipfs (took > 30 seconds a fair amount of the time). Curious to see what it's like now.
The ping from Earth to Mars varies between 360 and 2600 seconds, so 30 might still be fast enough for an "interplanetary" thing.
Doesn’t IPFS still rely on Kademlia?
Since node IDs are random, node lookup may require multiple interplanetary hops, no?
For instance, from N1 on Earth to the nearest XORwise node in its routing table which might be N2 on Mars, whose routing table finds the target node N3 on Earth.
Since node IDs are random, node lookup may require multiple interplanetary hops, no?
For instance, from N1 on Earth to the nearest XORwise node in its routing table which might be N2 on Mars, whose routing table finds the target node N3 on Earth.
We do use Kademlia - but note this release actually runs 2 DHTs - one for LAN connections and one for WAN connections. You could easily imagine a small Mars outpost as the LAN DHT where you can do fast retrieval for all content already available locally without hitting interplanetary lookup times.
https://docs-beta.ipfs.io/recent-releases/go-ipfs-0-5/featur...
https://docs-beta.ipfs.io/recent-releases/go-ipfs-0-5/featur...
What are your plans WRT Coral? I noticed there's a work-in-progress Go implementation. Is it going to supercede Kademlia eventually?
Yeah, thats roughly the idea. We can evolve the current DHT into something coral-like, or if something better comes along, we will use that.
In an actual interplanetary setup, we would definitely not be using a single DHT across the entire space. The main idea once we get to that scale is either a different content routing system altogether, or sharded coral style DHTs (https://en.wikipedia.org/wiki/Coral_Content_Distribution_Net...)
Tbh, chances are you will find a faster path through earth peers. The query will explore multiple paths in parallel and timeout slow ones...
Isn't the point of IPFS exactly to get those pings far down due to the file being hosted on your planet?
I know your comment could be viewed as being flippant but none the less it made me chuckle.
Surely they’d have an edge server eas-earth-1
This uses DHTs. How resilient is it against Sybil attacks? Also how does it work under global netsplit conditions, like if someone borks or attacks BGP in such a way that 1/3 of the world is not reachable?
My impression is that DHTs fall down pretty hard under the latter scenario and are also pretty vulnerable to the Sybil scenario if the attacker has enough resources to mount a really serious attack. They're okay for low-value simple stuff that doesn't have much of an intrinsic bounty attached to it (like BitTorrent magnets), but trying to put a "decentralized web" on top of a DHT seems like a scenario where the instant it becomes popular it will get completely shredded for profit (spam, stealing Bitcoin, etc.).
My rule of thumb is that anything designed for serious or large scale use (in other words that might get popular) needs to be built to withstand either a "nation state level attacker" threat model or a "Zerg rush of hundreds of thousands of profit motivated black hats" threat model. The Internet today is a war zone because today you can make money and gain power (e.g. by influencing elections) by messing with it.
My impression is that DHTs fall down pretty hard under the latter scenario and are also pretty vulnerable to the Sybil scenario if the attacker has enough resources to mount a really serious attack. They're okay for low-value simple stuff that doesn't have much of an intrinsic bounty attached to it (like BitTorrent magnets), but trying to put a "decentralized web" on top of a DHT seems like a scenario where the instant it becomes popular it will get completely shredded for profit (spam, stealing Bitcoin, etc.).
My rule of thumb is that anything designed for serious or large scale use (in other words that might get popular) needs to be built to withstand either a "nation state level attacker" threat model or a "Zerg rush of hundreds of thousands of profit motivated black hats" threat model. The Internet today is a war zone because today you can make money and gain power (e.g. by influencing elections) by messing with it.
For BGP partitions / Significant network outage conditions:
You may not find content partitioned on the other side of a netsplit from you, but the degraded condition is that you can still query and find content present in the same part of the network as you. This is better than centralized solutions face at present - e.g. google not being available in China, and isn't too far off from what you might hope.
For Sybils: You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence. For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.
For Sybils: You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence. For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.
> You may not find content partitioned on the other side of a netsplit from you, but the degraded condition is that you can still query and find content present in the same part of the network as you.
This becomes hugely problematic the minute you start using IPNS. On one side of the split, the name `foo` can resolve to `bar`, but on the other side, it resolves to `baz`. If you're trying to impersonate someone, then a netsplit would make it easy for you to do so (barring an out-of-band content authentication protocol, of course).
> You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence.
On the contrary, a Sybil node operator can censor arbitrary content in a DHT by inserting their own nodes into the network that are all "closer" to the targeted key range in the key space than the honest nodes. This can be done by crawling the DHT, identifying the honest nodes that route to the targeted key range, and generating node IDs that correspond to key ranges closer than them. Honest nodes will (correctly) proceed to direct lookup requests to the attacker nodes, thereby leading to content censorship.
Honest nodes can employ countermeasures to probe the network in order to try and see if/when this is happening, but an attacker node can be adapted to behave like an honest node when another honest node is talking to it.
> For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.
Sure, and while this is a good thing, it's ultimately an arms race between the IPFS developers and network attackers who can fool the automated countermeasures. I'm not confident in its long-term ability to fend off attacks on the routing system.
This becomes hugely problematic the minute you start using IPNS. On one side of the split, the name `foo` can resolve to `bar`, but on the other side, it resolves to `baz`. If you're trying to impersonate someone, then a netsplit would make it easy for you to do so (barring an out-of-band content authentication protocol, of course).
> You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence.
On the contrary, a Sybil node operator can censor arbitrary content in a DHT by inserting their own nodes into the network that are all "closer" to the targeted key range in the key space than the honest nodes. This can be done by crawling the DHT, identifying the honest nodes that route to the targeted key range, and generating node IDs that correspond to key ranges closer than them. Honest nodes will (correctly) proceed to direct lookup requests to the attacker nodes, thereby leading to content censorship.
Honest nodes can employ countermeasures to probe the network in order to try and see if/when this is happening, but an attacker node can be adapted to behave like an honest node when another honest node is talking to it.
> For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.
Sure, and while this is a good thing, it's ultimately an arms race between the IPFS developers and network attackers who can fool the automated countermeasures. I'm not confident in its long-term ability to fend off attacks on the routing system.
Censorship / availability is the issue. As you lay out, Sybils can be used to target a specific address / piece of content in an attempt to prevent it from being found. The good news is that there are some pretty mechanical things - like maintaining a consensus of known-trusted nodes that can be used to validate a node/piece of content isn't under attack - that should be sufficient until IFPS is quite a bit larger than it is today
I don't think I see the impersonation / bad resolution problem though. IPNS records are content addressed to the key. Having control of a portion of the network isn't sufficient to compromise that (you can prevent availability though).
I don't think I see the impersonation / bad resolution problem though. IPNS records are content addressed to the key. Having control of a portion of the network isn't sufficient to compromise that (you can prevent availability though).
> The good news is that there are some pretty mechanical things - like maintaining a consensus of known-trusted nodes that can be used to validate a node/piece of content isn't under attack - that should be sufficient until IFPS is quite a bit larger than it is today
What would I be trusting the nodes for? If I'm trusting them to just keep my data available, then why not just put it into S3? What role is IPFS playing at all, then, if I find myself having to pick trusted nodes to defend against low-cost route-censorship attacks?
Also, the size of the network doesn't really seem to make large DHTs resilient to Sybils. BitTorrent in 2010 had over 2 million peers [1], but north of 300,000 of them were Sybils [2]. That's pretty bad.
> I don't think I see the impersonation / bad resolution problem though. IPNS records are content addressed to the key. Having control of a portion of the network isn't sufficient to compromise that (you can prevent availability though).
Correct me if I'm wrong, but IPNS resolves a human-readable name to a content hash, right? If all I'm going off of is the name and the DHT (no DNS), then having a network that can return two (or more!) different content addresses for that name can lead to problems for users, no? If IPNS/IPFS is supposed to be a hypermedia protocol bent on replacing HTTP/DNS, then its inability to handle the case where `google.com` can resolve to either the legitimate Google or a phishing website sounds like a showstopping design flaw that just begging to be abused.
[1] https://www.cs.helsinki.fi/u/jakangas/MLDHT/
[2] https://nymity.ch/sybilhunting/pdf/Wang2012a.pdf
What would I be trusting the nodes for? If I'm trusting them to just keep my data available, then why not just put it into S3? What role is IPFS playing at all, then, if I find myself having to pick trusted nodes to defend against low-cost route-censorship attacks?
Also, the size of the network doesn't really seem to make large DHTs resilient to Sybils. BitTorrent in 2010 had over 2 million peers [1], but north of 300,000 of them were Sybils [2]. That's pretty bad.
> I don't think I see the impersonation / bad resolution problem though. IPNS records are content addressed to the key. Having control of a portion of the network isn't sufficient to compromise that (you can prevent availability though).
Correct me if I'm wrong, but IPNS resolves a human-readable name to a content hash, right? If all I'm going off of is the name and the DHT (no DNS), then having a network that can return two (or more!) different content addresses for that name can lead to problems for users, no? If IPNS/IPFS is supposed to be a hypermedia protocol bent on replacing HTTP/DNS, then its inability to handle the case where `google.com` can resolve to either the legitimate Google or a phishing website sounds like a showstopping design flaw that just begging to be abused.
[1] https://www.cs.helsinki.fi/u/jakangas/MLDHT/
[2] https://nymity.ch/sybilhunting/pdf/Wang2012a.pdf
IPNS names are based on public keys. A sybil attack can't make an IPNS name resolve to an arbitrary attacker-chosen value because the attacker can't make signatures for the public key.
A sybil attack could be used to cause part of the network to see a recent old value for an IPNS name, but clients keeping recent values cached for a while is already something that happens naturally, so it just seems like a more minor example of the general problem that a sybil attack could do a denial-of-service.
A sybil attack could be used to cause part of the network to see a recent old value for an IPNS name, but clients keeping recent values cached for a while is already something that happens naturally, so it just seems like a more minor example of the general problem that a sybil attack could do a denial-of-service.
Totally agree with you. This is an issue for most open membership p2p networks. Effectively, you need to create some type of reputation system or barrier to accruing power to make it harder for attackers to DOS/manipulate the network than for well-behaving nodes to operate it. IPFS definitely thinks about resistance to a variety of attacks like the ones you describe - but we still have a ways to go. Next steps might look like improvements to the DHT, or other more selective content routing solutions (ex, expanding on our dht-client vs dht-server designation to make content routers more selective/reputation-based).
Check out our most recent research discussion for some thoughts about how we might scale up Sybil resistance in p2p networks: https://www.youtube.com/watch?v=L4SJzoKHKPk
Check out our most recent research discussion for some thoughts about how we might scale up Sybil resistance in p2p networks: https://www.youtube.com/watch?v=L4SJzoKHKPk
Alternatively, you can have human-centric P2P networks like Scuttlebutt and Dat where you don't have to worry about sybil attacks. It's a trade-off, because these networks are invite-only, but once you're in you're good to go.
I run https://www.eternum.io/ (an IPFS pinning service) and created Hearth (https://hearth.eternum.io/, a Dropbox-like way to publish files on IPFS), and this is a very welcome release.
IPFS has been a real pain to work with in the past (the node would just consume all RAM and CPU and had to be restarted a lot), but it's been getting better, which is great to see.
I really hope it gets good enough to run on everyone's desktop machine, since that's the way IPFS is meant to be deployed (rather than just on gateways). It seems that it doesn't take up too much RAM or CPU now, but it looks like it might be a problem bandwidth-wise, if you host some popular content.
Still, great news overall.
IPFS has been a real pain to work with in the past (the node would just consume all RAM and CPU and had to be restarted a lot), but it's been getting better, which is great to see.
I really hope it gets good enough to run on everyone's desktop machine, since that's the way IPFS is meant to be deployed (rather than just on gateways). It seems that it doesn't take up too much RAM or CPU now, but it looks like it might be a problem bandwidth-wise, if you host some popular content.
Still, great news overall.
Here is a summary of IPFS white paper http://muratbuffalo.blogspot.com/2018/02/paper-review-ipfs-c...
The improvements announced are substantial.
The improvements announced are substantial.
You mean TRON /s
Edit : makin jokes about plagiarism people.
Edit : makin jokes about plagiarism people.
IPFS is really just a mix of a torrent tracker with a torrent client, but once IPFS VMs start paying for themselves, people will sign-up in masses. I guess that's what Filecoin is about.
Edit: The way I see filecoin working is anyone can post a reward for a file and once the file is provided, the reward is paid. In other words, it's bit like a brokerage that connects downloaders with uploaders. The difficultly is that this brokerage needs to be distributed and resilient.
Edit: The way I see filecoin working is anyone can post a reward for a file and once the file is provided, the reward is paid. In other words, it's bit like a brokerage that connects downloaders with uploaders. The difficultly is that this brokerage needs to be distributed and resilient.
I am yet to write more about this, but one thing that bothers me with Filecoin: the economics don't add up. The price per GB stored will tend to reach an equilibrium around the commodity price - i.e, costs of disks+computers+electricity+internet. Unless I am missing something, if the price gets higher, more people would buy disks and put them online, bringing the price down.
If that is true, it means that a node can only be profitable if you are freeloading. And if you are freeloading, any price you get will be good which means that it tends to go even further down, perhaps even below the commodity cost. I may be missing something, but I really don't see this going beyond techies with spare disks playing around and definitely no way to run a Filecoin node on a VPS profitably.
If that is true, it means that a node can only be profitable if you are freeloading. And if you are freeloading, any price you get will be good which means that it tends to go even further down, perhaps even below the commodity cost. I may be missing something, but I really don't see this going beyond techies with spare disks playing around and definitely no way to run a Filecoin node on a VPS profitably.
That is actually the intent of filecoin. In perfect competition, which is the ideal state for a commodity, there is no economic profit. The reason Amazon and other companies are able to make a profit selling cloud storage is because there is not yet perfect competition.
Right now if I wanted to compete with Amazon, I could buy 2 or 3 petabytes of storage space and some bandwidth. But nobody would trust me not to lose their data. This is the differentiation that makes cloud storage less of a commodity than it could be. The goal with filecoin is to make it so you don't have to trust me, just some general guarantees about the filecoin network. If amazon for some reason was selling storage on this network, I could compete with them on equal footing (and whoever sold it more cheaply would win).
Many commodities are in close-to-perfect competition. For instance, I don't care if grain comes from England or France, just that it's cheap. While it is very hard to make an economic profit selling grain, many people do farm grain and make enough money to support themselves (The money you pay your workers to survive is one of the costs of producing grain. That's true even if the only worker is you)
If you could make a profit by running a filecoin node on a VPS, everyone would do that. By competing with each other you'd all bring the price down and down until it was no longer more profitable than selling any other commodity (and maybe even lower). So you're right to not expect to be able to do that.
Right now if I wanted to compete with Amazon, I could buy 2 or 3 petabytes of storage space and some bandwidth. But nobody would trust me not to lose their data. This is the differentiation that makes cloud storage less of a commodity than it could be. The goal with filecoin is to make it so you don't have to trust me, just some general guarantees about the filecoin network. If amazon for some reason was selling storage on this network, I could compete with them on equal footing (and whoever sold it more cheaply would win).
Many commodities are in close-to-perfect competition. For instance, I don't care if grain comes from England or France, just that it's cheap. While it is very hard to make an economic profit selling grain, many people do farm grain and make enough money to support themselves (The money you pay your workers to survive is one of the costs of producing grain. That's true even if the only worker is you)
If you could make a profit by running a filecoin node on a VPS, everyone would do that. By competing with each other you'd all bring the price down and down until it was no longer more profitable than selling any other commodity (and maybe even lower). So you're right to not expect to be able to do that.
Your arguments do not follow:
> which is the ideal state for a commodity, there is no economic profit.
No. A commodity is simply any kind of good that is considered equivalent no matter who made it. Oil is a commodity, there is plenty of competition and there is still plenty of profit to be made (current crisis excluded)
> While it is very hard to make an economic profit selling grain, many people do farm grain and many enough money.
The analogy here would be to get people to (a) "farm" disks (and electricity/internet) and (b) with some profit to justify their work. (a) is impossible (except for freeloaders) and you agree with me that (b) seems to be unlikely.
So the question is: who would be interested in taking the supply side of the filecoin network, when there is no profit to be made?
> which is the ideal state for a commodity, there is no economic profit.
No. A commodity is simply any kind of good that is considered equivalent no matter who made it. Oil is a commodity, there is plenty of competition and there is still plenty of profit to be made (current crisis excluded)
> While it is very hard to make an economic profit selling grain, many people do farm grain and many enough money.
The analogy here would be to get people to (a) "farm" disks (and electricity/internet) and (b) with some profit to justify their work. (a) is impossible (except for freeloaders) and you agree with me that (b) seems to be unlikely.
So the question is: who would be interested in taking the supply side of the filecoin network, when there is no profit to be made?
> Your arguments do not follow:
> > which is the ideal state for a commodity, there is no economic profit.
> No. A commodity is simply any kind of good that is considered equivalent no matter who made it. Oil is a commodity, there is plenty of competition and there is still plenty of profit to be made (current crisis excluded)
You cut off the first half of my sentence. The full sentence was:
> In perfect competition, which is the ideal state for a commodity, there is no economic profit.
Oil is not in perfect competition because of OPEC, a gigantic state-sanctioned cartel. Since there's imperfect competition, there can be economic profit.
> who would be interested in taking the supply side of the filecoin network, when there is no profit to be made?
If we assume the filecoin network is technically able to do what it's supposed to, there will likely be profit to be made. Just not economic profit. Economic profit is different from accounting profit because economic profit takes into account opportunity cost.
In the real world many people, such as grain farmers, do seem to be interested in taking the supply side of a transaction even when their economic profit is 0 (or negative). I don't see why filecoin would be any different.
> > which is the ideal state for a commodity, there is no economic profit.
> No. A commodity is simply any kind of good that is considered equivalent no matter who made it. Oil is a commodity, there is plenty of competition and there is still plenty of profit to be made (current crisis excluded)
You cut off the first half of my sentence. The full sentence was:
> In perfect competition, which is the ideal state for a commodity, there is no economic profit.
Oil is not in perfect competition because of OPEC, a gigantic state-sanctioned cartel. Since there's imperfect competition, there can be economic profit.
> who would be interested in taking the supply side of the filecoin network, when there is no profit to be made?
If we assume the filecoin network is technically able to do what it's supposed to, there will likely be profit to be made. Just not economic profit. Economic profit is different from accounting profit because economic profit takes into account opportunity cost.
In the real world many people, such as grain farmers, do seem to be interested in taking the supply side of a transaction even when their economic profit is 0 (or negative). I don't see why filecoin would be any different.
I am sorry. Just like I am missing how someone would work on a farm without any expectation of return on their investment, I fail to see how "there is profit to be made", however you want to call it.
Also, with farming at least there is the notion of resource constraints. To be a farmer one needs to have access to land and dedicate some time to actually work on it.
With storage, unless the hard disk manufacturers start holding their production and use it to offer storage directly, what is the resource that other people can't have?
What would stop anyone to see that Filecoin network bids of (say) $1/TB-month and think "oh, I can buy some disks and offer $0.95/TB-month, and I will get my money back in 3 months", only to be outbid by someone who thinks they can offer $0.90/TB-month and recoup costs in six-months", until we get to the point where the time to break-even gets longer than the lifetime of the disks...
Also, with farming at least there is the notion of resource constraints. To be a farmer one needs to have access to land and dedicate some time to actually work on it.
With storage, unless the hard disk manufacturers start holding their production and use it to offer storage directly, what is the resource that other people can't have?
What would stop anyone to see that Filecoin network bids of (say) $1/TB-month and think "oh, I can buy some disks and offer $0.95/TB-month, and I will get my money back in 3 months", only to be outbid by someone who thinks they can offer $0.90/TB-month and recoup costs in six-months", until we get to the point where the time to break-even gets longer than the lifetime of the disks...
> No. A commodity is simply any kind of good that is considered equivalent no matter who made it. Oil is a commodity, there is plenty of competition and there is still plenty of profit to be made (current crisis excluded)
You could hardly say a market dominated by a giant state-sanctioned cartel was a good example of a commodity market in the state of perfect competition.
> So the question is: who would be interested in taking the supply side of the filecoin network, when there is no profit to be made?
There would likely be some short term economic profits at the beginning, which would attract market entrants. The economic profit would tend to zero over time, but keep in mind economic profit as a concept incorporates the notion of the entrepreneur paying themselves a competitive wage for their time.
You could hardly say a market dominated by a giant state-sanctioned cartel was a good example of a commodity market in the state of perfect competition.
> So the question is: who would be interested in taking the supply side of the filecoin network, when there is no profit to be made?
There would likely be some short term economic profits at the beginning, which would attract market entrants. The economic profit would tend to zero over time, but keep in mind economic profit as a concept incorporates the notion of the entrepreneur paying themselves a competitive wage for their time.
> giant state-sanctioned cartel...
Point still stands. All else being equal, oil from Saudi Arabia is the same as the oil from Canada or Denmark. If you don't want to use oil as an example, use coffee, milk, chicken meat, soy, cotton, oranges, electricity, potable water...
> There would likely be some short term economic profits at the beginning, which would attract market entrants.
The first market entrants are the ones selling extra capacity they have. For those that already have sunk costs of the disks, sure, they can make a profit on this. My point is that once that capacity is gone and there is more demand for data, people will have to actually go and buy disks and let them run 24/7, and this would cost more per GB stored than they will be able to sell.
Point still stands. All else being equal, oil from Saudi Arabia is the same as the oil from Canada or Denmark. If you don't want to use oil as an example, use coffee, milk, chicken meat, soy, cotton, oranges, electricity, potable water...
> There would likely be some short term economic profits at the beginning, which would attract market entrants.
The first market entrants are the ones selling extra capacity they have. For those that already have sunk costs of the disks, sure, they can make a profit on this. My point is that once that capacity is gone and there is more demand for data, people will have to actually go and buy disks and let them run 24/7, and this would cost more per GB stored than they will be able to sell.
The point is perfect competition, highlight on perfect. In oil business there are profits because competition is not perfect. Note that perfect here is rather a theoretical, economic term. In reality, market regulations and peoples socio-economic attitutes prevent a perfect competition.
The mentioned missing trust could be alleviated by IPFS becoming a widely trusted, distributed, unstoppable service.
The mentioned missing trust could be alleviated by IPFS becoming a widely trusted, distributed, unstoppable service.
Are we talking past one another?
My point is exactly that if Filecoin does get to be a place that gets "perfect competition" as you say, then there is no profit to be made. If there is no profit to be made, what is the economic incentive to spend on disks/internet/computers and be a storage seller?
Before you say "there is a market for those with excess capacity", answer me this: if the market is focused on those with excess capacity, what mechanisms are in place this "perfect competition" to remove all chance of profit and get the price of excess capacity to reach basically zero?
I get IPFS, I think. I do believe that it technically it makes sense. What I don't get is Filecoin.
My point is exactly that if Filecoin does get to be a place that gets "perfect competition" as you say, then there is no profit to be made. If there is no profit to be made, what is the economic incentive to spend on disks/internet/computers and be a storage seller?
Before you say "there is a market for those with excess capacity", answer me this: if the market is focused on those with excess capacity, what mechanisms are in place this "perfect competition" to remove all chance of profit and get the price of excess capacity to reach basically zero?
I get IPFS, I think. I do believe that it technically it makes sense. What I don't get is Filecoin.
That's the economic flaw in all of this - Amazon have economies of scale and access to cheap power and network connectivity and that means smaller competitors will be locked out of the market.
You're not going to be running a node on your spare capacity that brings in more than your costs, in fact unless you're a datacentre operator, you're going to be making a loss.
In the end it favours big centralised services, and IPFS (if it ever takes off) just becomes a rival API to S3.
You're not going to be running a node on your spare capacity that brings in more than your costs, in fact unless you're a datacentre operator, you're going to be making a loss.
In the end it favours big centralised services, and IPFS (if it ever takes off) just becomes a rival API to S3.
Cryptocoin miners' profit is largely based on "access to cheap power and network connectivity", and many of them run fairly large-scale operations but nowhere near the same scale as Amazon. No reason why a "file coin" should be any different.
But they do centralise. And they do quickly go out of profitability for those with some spare compute power at home, into the hands of those with cheap power and specialised equipment who can undercut and outcompete them.
You are right - there is no reason this would be different. That specialised equipment is storage space here.
You are right - there is no reason this would be different. That specialised equipment is storage space here.
That is missing the point. No one doubts that it is possible to bring down the costs of storage and its infrastructure. The point is that no matter how low these costs are all nodes are driven to undercut each other until their profitability is zero.
There's a lot of spare capacity in personal devices, getting _anything_ out of that capacity is nice and means you'll have people gladly undercutting Amazon and friends. Maybe limited and not at the same level of availability but still competition.
> getting _anything_ out of that capacity is nice
Ok, now go tell that to the ones that put 257 million USD on the Filecoin ICO and are expecting profits on a growing network!
> undercutting Amazon and friends.
That is so short-sighted. Amazon storage is not priced only due to the cost of storage. If it were, Wasabi would be stealing all of their customers given they charge "only" $6/TB-month, and I could go steal all of their customers because I have a cluster of Minio Servers that cost me $3.50/TB-month.
People are paying Amazon and the other major players a premium is not due to lack of competition or collusion between them or because they are price-sensitive. If they are (like I am), they would figure out how to setup Minio/Ceph and run their own servers.
Ok, now go tell that to the ones that put 257 million USD on the Filecoin ICO and are expecting profits on a growing network!
> undercutting Amazon and friends.
That is so short-sighted. Amazon storage is not priced only due to the cost of storage. If it were, Wasabi would be stealing all of their customers given they charge "only" $6/TB-month, and I could go steal all of their customers because I have a cluster of Minio Servers that cost me $3.50/TB-month.
People are paying Amazon and the other major players a premium is not due to lack of competition or collusion between them or because they are price-sensitive. If they are (like I am), they would figure out how to setup Minio/Ceph and run their own servers.
I think you're conflating different definitions of "profit". There's at least three important ones:
1) Accounting profit: revenue - expenses. This is colloquially what most people mean when they say "profit". To break even you just need to make enough money to cover your explicit expenses. In your example, you would need to make more in Filecoin income than you spent in disks/electricity/etc.
2) "Normal profit": this is a technical term in microeconomics. Basically this takes into account opportunity cost; the cost of not doing some other profitable thing with your time/money. By these standards, to break even you'd not only have to cover the cost of disks/electricity but also the "opportunity costs" of whatever else you could have been doing with those disks/electricity (like mining Bitcoin), or whatever you could have been doing if you never even purchased those disks to begin with (like investing in the S&P or whatever).
3) "Excess" or "economic" profits. These are extra profits on top of whatever you earn in #2. This is the part that goes to 0 for commodities like milk/gasoline/Filecoin but is non-zero for differentiated products like iphones.
For Filecoin specifically, I would expect that normal profits would probably be above the raw costs of disks and electricity, since it takes some effort to purchase and administer all that hardware, but probably below the cost of renting a VPS, since a VPS is typically used for more profitable things than just sitting around and holding stuff on disk.
1) Accounting profit: revenue - expenses. This is colloquially what most people mean when they say "profit". To break even you just need to make enough money to cover your explicit expenses. In your example, you would need to make more in Filecoin income than you spent in disks/electricity/etc.
2) "Normal profit": this is a technical term in microeconomics. Basically this takes into account opportunity cost; the cost of not doing some other profitable thing with your time/money. By these standards, to break even you'd not only have to cover the cost of disks/electricity but also the "opportunity costs" of whatever else you could have been doing with those disks/electricity (like mining Bitcoin), or whatever you could have been doing if you never even purchased those disks to begin with (like investing in the S&P or whatever).
3) "Excess" or "economic" profits. These are extra profits on top of whatever you earn in #2. This is the part that goes to 0 for commodities like milk/gasoline/Filecoin but is non-zero for differentiated products like iphones.
For Filecoin specifically, I would expect that normal profits would probably be above the raw costs of disks and electricity, since it takes some effort to purchase and administer all that hardware, but probably below the cost of renting a VPS, since a VPS is typically used for more profitable things than just sitting around and holding stuff on disk.
Thanks for the breakdown of "profit" definitions, but I still think that all of these observations are largely missing the point...
> I would expect that normal profits would probably be above the raw costs of disks and electricity, since it takes some effort to purchase and administer all that hardware
No, it doesn't. Most of the people selling the idea of distributed file storage (Filecoin, Storj, Sia) are thinking about getting the excess capacity of everyone's disks and turning into a marketplace. Getting the hardware and administering are sunk costs. Participants are only supposed to download a client, indicate the payout method and ensure that they don't turn off their computers. It's a one-off effort. Even understanding (or pretending to understand) all of the crypto-babble in order to get paid is something that people are supposed to do only once.
This is the part where these different projects are working out their market dynamics and where I get lost about Filecoin:
- With Storj (and Sia, I believe) the price of the data is determined by the governors of the network. Storj last rollout was paying $5/TB-month for every client. There is no bidding war. As long as you could prove you held the data for that period, you would get the money. This makes sense for me - no matter if I am some dude in a basement with a disk on my laptop or if I am a datacenter and I get to charge more than $5/TB-month from my usual customers, my excess capacity can be sold for $5/TB-month and this becomes a simple matter of waiting for the demand to grow.
- With Filecoin the idea is to have a completely open market, where each node says how much they are willing to charge for the data, and what are the constraints (availability, time to retrieval, etc). This leads to possible (inevitable?) bidding wars. No matter the demand curve, there is always someone who will be willing to put the lowest price possible that gets them "something". For excess capacity, this "something" can be effectively zero or whatever is the lowest price possible in the bid. And if it can be zero, the only thing I see helping them would be if the nodes starting cooperating with each other and acting like a cartel.
To be honest, even Storj and Sia are hard sells for me. I wouldn't bet on these projects for a long run and I don't believe that they will actually threaten the current players, but at least I understand the incentive structure. With Filecoin, it can only be profitable when the demand for data storage grows faster than the storage capacity from the network, but no so profitable as to attract people from the demand-side to the supply-side. It's crazy.
> I would expect that normal profits would probably be above the raw costs of disks and electricity, since it takes some effort to purchase and administer all that hardware
No, it doesn't. Most of the people selling the idea of distributed file storage (Filecoin, Storj, Sia) are thinking about getting the excess capacity of everyone's disks and turning into a marketplace. Getting the hardware and administering are sunk costs. Participants are only supposed to download a client, indicate the payout method and ensure that they don't turn off their computers. It's a one-off effort. Even understanding (or pretending to understand) all of the crypto-babble in order to get paid is something that people are supposed to do only once.
This is the part where these different projects are working out their market dynamics and where I get lost about Filecoin:
- With Storj (and Sia, I believe) the price of the data is determined by the governors of the network. Storj last rollout was paying $5/TB-month for every client. There is no bidding war. As long as you could prove you held the data for that period, you would get the money. This makes sense for me - no matter if I am some dude in a basement with a disk on my laptop or if I am a datacenter and I get to charge more than $5/TB-month from my usual customers, my excess capacity can be sold for $5/TB-month and this becomes a simple matter of waiting for the demand to grow.
- With Filecoin the idea is to have a completely open market, where each node says how much they are willing to charge for the data, and what are the constraints (availability, time to retrieval, etc). This leads to possible (inevitable?) bidding wars. No matter the demand curve, there is always someone who will be willing to put the lowest price possible that gets them "something". For excess capacity, this "something" can be effectively zero or whatever is the lowest price possible in the bid. And if it can be zero, the only thing I see helping them would be if the nodes starting cooperating with each other and acting like a cartel.
To be honest, even Storj and Sia are hard sells for me. I wouldn't bet on these projects for a long run and I don't believe that they will actually threaten the current players, but at least I understand the incentive structure. With Filecoin, it can only be profitable when the demand for data storage grows faster than the storage capacity from the network, but no so profitable as to attract people from the demand-side to the supply-side. It's crazy.
If there's enough enthusiasts using spare disks at home to offer enough disk capacity for everyone, then that means Filecoin hosting will be available cheaper than cloud hosts. If all that capacity gets used up, then supply and demand will cause the price to go up until more people add capacity. If the price hits the price of cloud hosts, then people will just run tons of Filecoin nodes on cloud hosts, so the price of cloud hosting will be the upper bound on Filecoin hosting prices. Then anyone that can undercut the cloud hosts will be free to run their own Filecoin nodes, and the price will be pushed down as people do this.
That means average home user bandwidth costs will go up? The unlimited download, flat-rate, fair use plan your home lan connects to the internet can be provided because your connection is lingering 90% of its time practically unused (besides all sorts of home appliance phoning home).
If you are using this seemingly free connection capacity, internet access rates should go up, because there is no such thing as a free lunch?
If you are using this seemingly free connection capacity, internet access rates should go up, because there is no such thing as a free lunch?
> If there's enough enthusiasts using spare disks at home to offer enough disk capacity for everyone, then that means Filecoin hosting will be available cheaper than cloud hosts.
Only if these enthusiasts are willing to take a loss on power.
> If the price hits the price of cloud hosts, then people will just run tons of Filecoin nodes on cloud hosts
Or the cloud hosts themselves will enter the market. The upper bound on the market will be their costs, which will always be lower than the costs of enthusiasts using spare disks at home.
Only if these enthusiasts are willing to take a loss on power.
> If the price hits the price of cloud hosts, then people will just run tons of Filecoin nodes on cloud hosts
Or the cloud hosts themselves will enter the market. The upper bound on the market will be their costs, which will always be lower than the costs of enthusiasts using spare disks at home.
> Only if these enthusiasts are willing to take a loss on power.
An 8TB HDD doesn't use more power than a 100GB one.
An 8TB HDD doesn't use more power than a 100GB one.
> An 8TB HDD doesn't use more power than a 100GB one.
But access to it by third parties does, it changes the usage model and increases your electricity costs. This may not be a lot but it doesn't have to be a lot to make your costs more than those of a datacentre operator, not to mention you also need to be online all the time.
The dream of edge-node storage utilisation has never survived in the face of economic analysis.
But access to it by third parties does, it changes the usage model and increases your electricity costs. This may not be a lot but it doesn't have to be a lot to make your costs more than those of a datacentre operator, not to mention you also need to be online all the time.
The dream of edge-node storage utilisation has never survived in the face of economic analysis.
It is not a matter of size. It is a matter of availability.
If I only have my data on a disk, I can turn it off anytime. If I am hosting someone else's data and I want to get paid for it, I need to keep the computer running 24/7/365. That costs quite a bit.
If I only have my data on a disk, I can turn it off anytime. If I am hosting someone else's data and I want to get paid for it, I need to keep the computer running 24/7/365. That costs quite a bit.
> If there's enough enthusiasts using spare disks (...) then that means Filecoin hosting will be available cheaper than cloud hosts
Ok. Agreed.
> If all that capacity gets used up, then supply and demand will cause the price to go up until more people add capacity.
Agreed.
> If the price hits the price of cloud hosts, then people will just run tons of Filecoin nodes on cloud hosts.
Not necessarily. I can just buy more disks and run locally, whether for hosting my data or to sell the space on the network, which brings the price back to people-selling-spare-disk-space-for-cheap level.
> Then anyone that can undercut the cloud hosts will be free to run their own Filecoin nodes, and the price will be pushed down as people do this.
Yeah, but that is kind of my point. Everyone will undercut each other until the price hits the bottom. The equilibrium will be at most the commodity price.
Ok. Agreed.
> If all that capacity gets used up, then supply and demand will cause the price to go up until more people add capacity.
Agreed.
> If the price hits the price of cloud hosts, then people will just run tons of Filecoin nodes on cloud hosts.
Not necessarily. I can just buy more disks and run locally, whether for hosting my data or to sell the space on the network, which brings the price back to people-selling-spare-disk-space-for-cheap level.
> Then anyone that can undercut the cloud hosts will be free to run their own Filecoin nodes, and the price will be pushed down as people do this.
Yeah, but that is kind of my point. Everyone will undercut each other until the price hits the bottom. The equilibrium will be at most the commodity price.
>> but once IPFS VMs start paying for themselves
How does that work in real life?
How does that work in real life?
This looks a lot like the common fallacy (in blockchain circles) that anything that is _possible_ is also _inevitable_. There is no reason at all why IPFS VMs would be profitable in the long term, especially since VM hosters are not stupid and would just dedicate their capacity to IPFS themselves, cutting out the middle man.
Of course, in the shorter term there are many different scenarios where the market is slower to adjust. The STORJ network has been subsidizing their node operators with VC money for example, leading to some excellent unit economics for the early adopters.
Of course, in the shorter term there are many different scenarios where the market is slower to adjust. The STORJ network has been subsidizing their node operators with VC money for example, leading to some excellent unit economics for the early adopters.
TL;DR to skip the glossy page.
> The primary focus of this release was on improving content routing. That is, advertising and finding content. To that end, this release heavily focuses on improving the DHT.
> A secondary focus in this release was improving content transfer, our data exchange protocols
Other improvements to data store, libp2p, etc.
source: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md
> The primary focus of this release was on improving content routing. That is, advertising and finding content. To that end, this release heavily focuses on improving the DHT.
> A secondary focus in this release was improving content transfer, our data exchange protocols
Other improvements to data store, libp2p, etc.
source: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md
>> Opera became the first major web browser to offer default IPFS support on Android, shortly after Brave started directly embedding the IPFS Companion extension (complete with a built-in js-ipfs node). This means millions of people around the world now have access to the decentralized web built directly into their browsers.
Wow. Thank you IPFS community.
Wow. Thank you IPFS community.
I believe Opera only redirects to a gateway and doesn't implement IPFS itself, though, which is rather less exciting.
[deleted]
I think IPFS is a great technology, and the protocol appears to be fairly well designed but to be frank, I tried to use go-ipfs and it is a very junky implementation. There are numerous issues that show an overall lack of good quality standards in the codebase.
- Pin management is far to simple for any real use case.
- GC is a complete GC on an arbitrary threshold.
- API is based on the CLI and really weird (ex GET /api/v0/cp?arg={src}&arg={dest}, why don't the parameters have names? Why is it GET?)
- They don't prove a way to upload a directory with references to existing IPFS objects, so you need to manually encode the data.
- The manual encoding uses go-style names, they seem to have let their language choice leak into the protocol.
- Lots of minor issues, such as the API implementing block size limits by simply cutting off the input data at the size limit, this is even before encoding it into the format where the size limit should be calculated and cause data corruption in some cases.
- Their one abstract API for creating directories (MFS) has a number of issues.
I ended up sinking a ton of time into https://gitlab.com/kevincox/archlinux-ipfs-mirror/ and in the end the number of small quirks were incredibly frustrating. I might go back and implement their hash-sharding logic to get the mirror working again but at this point I don't really want to interact with go-ipfs again.
I thought it would be an interesting project to get involved in, and I have a lot of expertise that would be valuable however it appears that it is all a bit of a mess which is a real shame.
Maybe with more success it can be re-written in cleaner fashion, as I said most of the issues are with the implementation not the protocol so there is definitely hope. I do honestly wish all of the success to the project.
- Pin management is far to simple for any real use case.
- GC is a complete GC on an arbitrary threshold.
- API is based on the CLI and really weird (ex GET /api/v0/cp?arg={src}&arg={dest}, why don't the parameters have names? Why is it GET?)
- They don't prove a way to upload a directory with references to existing IPFS objects, so you need to manually encode the data.
- The manual encoding uses go-style names, they seem to have let their language choice leak into the protocol.
- Lots of minor issues, such as the API implementing block size limits by simply cutting off the input data at the size limit, this is even before encoding it into the format where the size limit should be calculated and cause data corruption in some cases.
- Their one abstract API for creating directories (MFS) has a number of issues.
- It is incredibly slow for some reason.
- It kindof but not really automatically pins all content recursively references.
I ended up sinking a ton of time into https://gitlab.com/kevincox/archlinux-ipfs-mirror/ and in the end the number of small quirks were incredibly frustrating. I might go back and implement their hash-sharding logic to get the mirror working again but at this point I don't really want to interact with go-ipfs again.
I thought it would be an interesting project to get involved in, and I have a lot of expertise that would be valuable however it appears that it is all a bit of a mess which is a real shame.
Maybe with more success it can be re-written in cleaner fashion, as I said most of the issues are with the implementation not the protocol so there is definitely hope. I do honestly wish all of the success to the project.
Can someone provide an example of a service that ipfs enables that can’t be provided on the regular web?
You can publish static content that doesn't go down when the server goes down and scales on its own as the content becomes more popular. It also has hashing built-in, so you can be sure the content you got is what you wanted,
Things like distribution of apt packages becomes much more exciting when each computer can choose to redistribute the packages it got to others in the LAN or area, even offline.
It's also very interesting to me how all the visitors to your website become servers as well, so content can never be "hugged to death" or links can never go stale, as long as at least one person has the content somewhere on their node.
That, to me, is huge, as links now go stale with some regularity. Think of all the Geocities sites (and all versions of them) just existing for ever, regardless if Geocities decided to shut down.
For example, here's my site on IPFS:
https://ipfs.eternum.io/ipfs/QmVW6JejQkjLnBJacR8qcZi88WNTMwi...
That can now "never" be lost, as long as someone cares enough about it to visit.
Things like distribution of apt packages becomes much more exciting when each computer can choose to redistribute the packages it got to others in the LAN or area, even offline.
It's also very interesting to me how all the visitors to your website become servers as well, so content can never be "hugged to death" or links can never go stale, as long as at least one person has the content somewhere on their node.
That, to me, is huge, as links now go stale with some regularity. Think of all the Geocities sites (and all versions of them) just existing for ever, regardless if Geocities decided to shut down.
For example, here's my site on IPFS:
https://ipfs.eternum.io/ipfs/QmVW6JejQkjLnBJacR8qcZi88WNTMwi...
That can now "never" be lost, as long as someone cares enough about it to visit.
That's "how it works", not "what it can be used for". Regular websites can be designed to handle high traffic. Regular websites can make backups.
What new thing does ipfs make possible that would make users install the software to use it?
What new thing does ipfs make possible that would make users install the software to use it?
Nothing has any advantage except the advantages it has. I don't want to repeat everything I said in the last post, but what you mentioned isn't a counterargument to any of what I detailed.
If you're not going to answer a question, don't bother replying.
> Regular websites can be designed to handle high traffic
IPFS lets them not have to.
> Regular websites can make backups
Backups don't help when the company shuts down and takes the site down.
> What new thing does ipfs make possible that would make users install the software to use it?
"Think of all the Geocities sites (and all versions of them) just existing for ever, regardless if Geocities decided to shut down."
I don't think you even read my post.
IPFS lets them not have to.
> Regular websites can make backups
Backups don't help when the company shuts down and takes the site down.
> What new thing does ipfs make possible that would make users install the software to use it?
"Think of all the Geocities sites (and all versions of them) just existing for ever, regardless if Geocities decided to shut down."
I don't think you even read my post.
I'll try.
You visit a site, get the index and it references two more files with two URLs:
"./jquery-3.5.0.min.js" "https://cdn.com/4kVideo.mkv"
Your browsers get's those files, the first from the site, the other from a CDN. Your site scales according to the band-width that "cdn.com" is able to provide to you.
In IPFS, you visit a site, get the index and it references two more files with two URLS:
"ipfs://QmWYudWcbX6skKub5wg1Ga3LFh3vbW2k7PWfdqHtDYvAdp" (for fun I used the right address here) "ipfs://QmT9qk3CRYbFDWpDFYeAv8T8H1gnongwKhh5J68NLkLir6"
Your browser gets those files, the first is found in about a billion places. A lot of sites use jQuery so almost everyone has the file's content available, and because you'd already had visited another site with that content-hash, IPFS knew it could just use that one (a perfect safe cross-domain cache-hit). The second is found in less places, but it follows the same logic as a torrent, with data coming from both a IPFS cdn the site uses, along with a few people who've also seen the video, so it loads faster from a few sources than one.
Hope this makes sense. IPFS is really just a decent way to implement websites like a torrent and enjoy the benefits that brings. It's not grand or out of this world, just decent space-saving data-management.
You visit a site, get the index and it references two more files with two URLs:
"./jquery-3.5.0.min.js" "https://cdn.com/4kVideo.mkv"
Your browsers get's those files, the first from the site, the other from a CDN. Your site scales according to the band-width that "cdn.com" is able to provide to you.
In IPFS, you visit a site, get the index and it references two more files with two URLS:
"ipfs://QmWYudWcbX6skKub5wg1Ga3LFh3vbW2k7PWfdqHtDYvAdp" (for fun I used the right address here) "ipfs://QmT9qk3CRYbFDWpDFYeAv8T8H1gnongwKhh5J68NLkLir6"
Your browser gets those files, the first is found in about a billion places. A lot of sites use jQuery so almost everyone has the file's content available, and because you'd already had visited another site with that content-hash, IPFS knew it could just use that one (a perfect safe cross-domain cache-hit). The second is found in less places, but it follows the same logic as a torrent, with data coming from both a IPFS cdn the site uses, along with a few people who've also seen the video, so it loads faster from a few sources than one.
Hope this makes sense. IPFS is really just a decent way to implement websites like a torrent and enjoy the benefits that brings. It's not grand or out of this world, just decent space-saving data-management.
No, and I was looking hard. IPFS is a slower, no-SLA version of S3 at this stage. People try to sell it with this planetary-scale marketing phrase just like they were trying to sell MongoDB with the web-scale phrase. Once you understand that the use cases for asymmetric uplink powered file sharing are latency and availability you also understand that the use cases are very limited for such storage. Mostly torrent really or cold backup maybe.
If disagree please explain why.
If disagree please explain why.
IPFS need a way to get paid for a block over a period of time. Does not need to be real money just token you can use later to pay for your file to be seeded.
Basically I host your file and you host my file. With some algorithm to make sure number of seeder for each block never go to 0
Basically I host your file and you host my file. With some algorithm to make sure number of seeder for each block never go to 0
Check out Filecoin (https://filecoin.io) - token for buying/selling file storage for IPFS data. You can earn tokens for renting out your disk space and use them to purchase backup/hosting from others.
I wish they hadn't picked go. Go isn't a bad language really, but it's a huge pain to integrate a go library into an app written in another language.
C or Rust would be perfect. No GC runtime, no name mangling, fast, great ffi story.
Rust needs to start edging out Go for new tools. Kubernetes, Envoy, IPFS, etc. would have benefitted from it.
It might not have been time four years ago, but it's time now.
Rust needs to start edging out Go for new tools. Kubernetes, Envoy, IPFS, etc. would have benefitted from it.
It might not have been time four years ago, but it's time now.
Rust will start edging out Go once Rust becomes easier than Go to prototype/start projects.
Rust is a great language, but right now I can start a project in Go and have someone else start a project in Go and have something up and working in a week. Rust still takes significantly longer to learn and/or find skilled developers for. Go is up there with Python in terms of "Get something out now".
Rust is a great language, but right now I can start a project in Go and have someone else start a project in Go and have something up and working in a week. Rust still takes significantly longer to learn and/or find skilled developers for. Go is up there with Python in terms of "Get something out now".
Rust isn't that difficult to prototype or start projects with? What trouble do you run into? I think the hardest parts to understand are the concepts of borrowing, lifetimes, and ownership, but you don't have to know all the subtleties of that to prototype. A basic understanding is fine to get started with. Just like a developer doesn't have to understand all the subtleties of generics to get started with either, lots of devs don't know what covariant or contravariant means and use generics.
Understanding the concepts is one thing, applying them is another thing entirely. The Rust compiler is an incredibly difficult beast to placate.
Especially in prototyping you can mostly .clone() everything, and stop worrying about any lifetime problems.
Lots of deferred pain for when you productionize.
Yes, but this was on the topic of "is Rust suited for prototyping?", and I would say that cloning makes it much more suited, as it sidesteps a lot of the lifetime pains. You will have similar deferred pain if you e.g. prototype something in JS and then when it's time to move to production gradually move it to Typescript.
From personal experience of Rust projects in production, you can get by with cloning for a long time, and optimizing later is not too big of a pain in most cases. Performance problems due to excessive cloning are also very obvious in code and easy to profile for.
From personal experience of Rust projects in production, you can get by with cloning for a long time, and optimizing later is not too big of a pain in most cases. Performance problems due to excessive cloning are also very obvious in code and easy to profile for.
Just a minor example.
https://github.com/awslabs/aws-lambda-rust-runtime/blob/mast...
Could you explain to a junior what Box<dyn std::error::Error + Send + Sync + 'static> does and how should a person who is starting to learn Rust come up with something like this?
Second minor example. I have a configuration string like let's say timeout for a certain type of connection. What is the recommended way to have that as a string that is accessible to all functions without fighting with the borrow checker?
In Elixir, I would use @timeout in the module for example. In Rust, I am not sure. Cloning the value to every place it needs it as a workaround.
https://github.com/awslabs/aws-lambda-rust-runtime/blob/mast...
Could you explain to a junior what Box<dyn std::error::Error + Send + Sync + 'static> does and how should a person who is starting to learn Rust come up with something like this?
Second minor example. I have a configuration string like let's say timeout for a certain type of connection. What is the recommended way to have that as a string that is accessible to all functions without fighting with the borrow checker?
In Elixir, I would use @timeout in the module for example. In Rust, I am not sure. Cloning the value to every place it needs it as a workaround.
I get what you're trying to say, but disagree with the examples. If you want to prototype and start a project, you don't care what exactly that box type means. You can use it and things will not break. It's part of that particular boilerplate.
It's like people using `std::string` and not necessarily caring for years what `template<class CharT, class Traits = std::char_traits<CharT>, class Allocator = std::allocator<CharT>> class basic_string;` means or does.
For the second example: environment variable. Once you need more, you can start researching static cells and use a ready crate for configuration loading / processing. If you want more, googling "rust mutable global configuration" brought me https://stackoverflow.com/questions/27791532/how-do-i-create... which gives a pretty good solution.
It's like people using `std::string` and not necessarily caring for years what `template<class CharT, class Traits = std::char_traits<CharT>, class Allocator = std::allocator<CharT>> class basic_string;` means or does.
For the second example: environment variable. Once you need more, you can start researching static cells and use a ready crate for configuration loading / processing. If you want more, googling "rust mutable global configuration" brought me https://stackoverflow.com/questions/27791532/how-do-i-create... which gives a pretty good solution.
>> If you want to prototype and start a project, you don't care what exactly that box type means.
I disagree. The whole point is to get to know a language. What do you how many things can you have in your code that you do not understand? Now you put it in production and it breaks. Who is going to fix it? I do not have a particulary positive experience with Rust on Stackoverflow. As a tech decision maker I put Rust in the risky category because it is hard to understand and many of the language features are implemented in a way that it is not easy to work with. Don't get me wrong, I like what Rust could bring to the table but with this upfront complexity it is not worth it. I know many companies who will be ok to take on Rust and they are ok with the way things are.
Your answer is funny to the second one. From SO:
Non-answer answer
Avoid global state in general. Instead, construct the object somewhere early (perhaps in main), then pass mutable references to that object into the places that need it. This will usually make your code easier to reason about and doesn't require as much bending over backwards.
Why would I pass in a mutable reference when all I need is a global static immutable value? You see this get ugly very quickly. We went with the lazy_static! way, but still. I would rather avoid these things entirely.
I disagree. The whole point is to get to know a language. What do you how many things can you have in your code that you do not understand? Now you put it in production and it breaks. Who is going to fix it? I do not have a particulary positive experience with Rust on Stackoverflow. As a tech decision maker I put Rust in the risky category because it is hard to understand and many of the language features are implemented in a way that it is not easy to work with. Don't get me wrong, I like what Rust could bring to the table but with this upfront complexity it is not worth it. I know many companies who will be ok to take on Rust and they are ok with the way things are.
Your answer is funny to the second one. From SO:
Non-answer answer
Avoid global state in general. Instead, construct the object somewhere early (perhaps in main), then pass mutable references to that object into the places that need it. This will usually make your code easier to reason about and doesn't require as much bending over backwards.
Why would I pass in a mutable reference when all I need is a global static immutable value? You see this get ugly very quickly. We went with the lazy_static! way, but still. I would rather avoid these things entirely.
> The whole point is to get to know a language.
When you're actually starting up (you set the context to "someone starting to learn rust"), there will be some unknown things in any language. You'll get there, but you don't need to understand it from the beginning. If you do want to understand it, then you should get it from the official guide. Errors, dyn, box, lifetimes and send are covered.
"Value on the heap, which implements the error interface, is available forever, and can be shared between threads" is... not that complicated.
> Now you put it in production and it breaks.
If you're still learning the language and don't understand the boilerplate, you don't put your app in production.
> Why would I pass in a mutable reference when all I need is a global static immutable value?
You missed the context. The fragment was about a mutable global. If you just need an initialised, but immutable config, you pass a non-mutable reference.
When you're actually starting up (you set the context to "someone starting to learn rust"), there will be some unknown things in any language. You'll get there, but you don't need to understand it from the beginning. If you do want to understand it, then you should get it from the official guide. Errors, dyn, box, lifetimes and send are covered.
"Value on the heap, which implements the error interface, is available forever, and can be shared between threads" is... not that complicated.
> Now you put it in production and it breaks.
If you're still learning the language and don't understand the boilerplate, you don't put your app in production.
> Why would I pass in a mutable reference when all I need is a global static immutable value?
You missed the context. The fragment was about a mutable global. If you just need an initialised, but immutable config, you pass a non-mutable reference.
A good related question is: what will be the "future" of software development? I would bet that WASM and Rust will become more and more dominant in a 5-10 years timeframe. But it's so hard to come up with an informed opinion, as many developers tend to become "religious" about their language of choice.
>> Rust will start edging out Go once Rust becomes easier than Go to prototype/start projects.
I guess that time will never come...unless Go somehow looses its way
I guess that time will never come...unless Go somehow looses its way
Not to mention the learning curve. For junior developers to be productive it takes forever. I can teach F# to a junior in a week, Rust maybe a month. Even for senior devs, it is hard to decipher what is the actual meaning of the code sometimes. There is a lot of noise in Rust that makes it harder to grasp what is happening.
Before downvoting, this is our experience.
Before downvoting, this is our experience.
And therefore Rust will never outcompete Go in this regard in as much as Haskell, Ocaml, ... never overtook C. Rust might draw from the better principles and get things right C, C++, D, Go, ... failed to incorporate. But in order to get things done a company has to source developers from a market and consider that developers get replaced by another, taking over.
Even in a globalized developer market a smaller developer force is a risk.
In German we have a saying "to die in beauty" (in Schönheit sterben), does that translate? Your boss wants to get things done in due time and I see Go outcompete Rust in this respect.
Even in a globalized developer market a smaller developer force is a risk.
In German we have a saying "to die in beauty" (in Schönheit sterben), does that translate? Your boss wants to get things done in due time and I see Go outcompete Rust in this respect.
> does that translate?
It does, and it's very evocative.
It does, and it's very evocative.
There's actually a new Rust-IPFS implementation looking for more contributors. Seems to be moving pretty fast - https://github.com/ipfs-rust/rust-ipfs
Rust does name mangling by default. However, you can turn it off per symbol just like in C++.
Envoy is written in C++, not Go.
Keybase and KBFS as well.
Go can handle 100k goroutines (green threads) without a sweat.
This is useful for p2p systems in which you are connected to thousands of peers multiplexing several p2p protocols with each concurrently. You want to be able to switch threads as fast as possible.
The concurrency model is a bit easier to reason about too.
It would be good to see how Rust and others do. Other languages excel at other things (i.e. embedded devices ).
This is useful for p2p systems in which you are connected to thousands of peers multiplexing several p2p protocols with each concurrently. You want to be able to switch threads as fast as possible.
The concurrency model is a bit easier to reason about too.
It would be good to see how Rust and others do. Other languages excel at other things (i.e. embedded devices ).
Rust can handle the same thing though the typical model now is using async-await and scheduling tasks on a threadpool. While I won't say its conceptually as simple as go can be, Rust also has channels, mutexes, and other concurrency primitives that allow you to create a similar story. It will also end up being more performant and less work to make correct (typically) due to its small runtime and extensive compile-time checking.
Maybe if IPFS had started today they would have picked Rust.
Honestly I'm not sure why Go was picked specificall, but if you put things in the context of 5 years ago it doesn't seem a crazy choice.
Honestly I'm not sure why Go was picked specificall, but if you put things in the context of 5 years ago it doesn't seem a crazy choice.
The Discord team wrote an article (or a few), and I think you'd find them very interesting.Specifically about their Go implementation, porting to Rust, and then the decision to use Rust wherever it makes sense as a result.
They're very sensible about it, and advocate using the right tool for the job.
They're very sensible about it, and advocate using the right tool for the job.
I'm not sure if that applies here, but Discord switched a portion of their backend from Go to Rust because of some performance issues.
https://blog.discord.com/why-discord-is-switching-from-go-to...
https://blog.discord.com/why-discord-is-switching-from-go-to...
Rust can easily do the same.
Any language that lacks a simple and stable binary interface ultimately can't be used to build reusable code. If it's not simple, only compiler hackers will be able to work with the objects produced. If it's not stable, compatibility between two binaries isn't guaranteed and people won't even try to integrate existing libraries into their own software.
A big reason why C is still widely used. A library that's written in C can be used in any other language.
A big reason why C is still widely used. A library that's written in C can be used in any other language.
Same thing that turned me off. There’d have been lots of cool and useful integrations popping up “for free” all over the place if they’d written it in a good library/integration language. Good language for a daemon, bad language for the reference (so, for anything halfway complicated, usually only) implementation of something intended to become a foundational protocol.
...there are a lot of cool and useful integrations popping up: https://blog.ipfs.io/weekly-84/.
go-ipfs is where the majority of new development happens right now as the desktop/server implementation (compared to js-ipfs=browser & rust-ipfs=IoT) - but if you want to prove that Rust is clearly better/faster _in general_ - have at it: https://github.com/ipfs-rust/rust-ipfs ;)
I think the thing jbenet was selecting for back in 2013 was concurrency support & modularity, and golang is still a decent choice for that. Rust 1.0 didn't happen until 2015 after the go-ipfs alpha was already out - but agree it's made awesome progress since then!
go-ipfs is where the majority of new development happens right now as the desktop/server implementation (compared to js-ipfs=browser & rust-ipfs=IoT) - but if you want to prove that Rust is clearly better/faster _in general_ - have at it: https://github.com/ipfs-rust/rust-ipfs ;)
I think the thing jbenet was selecting for back in 2013 was concurrency support & modularity, and golang is still a decent choice for that. Rust 1.0 didn't happen until 2015 after the go-ipfs alpha was already out - but agree it's made awesome progress since then!
I don't care about performance, exactly, it's just that with (say) a C implementation if you're even halfway popular you'll end up with modules in a dozen languages (wrapping your C lib) and an apache module and a storage backend for a few databases, maybe a Linux kernel module, and so on, in no time, and they'll all be pretty much in sync with what's
They also picked javascript ¯\_(ツ)_/¯
JS has a better library-integration story than Go does, since there are so many JS implementations in other languages. Go drags all its runtime threads along with it.
But fair point :)
But fair point :)
I used to hate JS, but honestly, it made a lot of really smart decisions for dynamic documents. Some decisions weren’t great, but somehow they didn’t go as far wrong as C++ managed to in the same era…
Browsers picked... until recently it was the only way to run in browsers.......
Hold up... You can pull docker images from ipfs? How?
And do IPNS entries finally last more than 30 seconds? It would be nice to not have to constantly keep a node up just to have an IPNS entry.
And do IPNS entries finally last more than 30 seconds? It would be nice to not have to constantly keep a node up just to have an IPNS entry.
IPNS records still have a default 24hr lifetime, but republishing your IPNS records is now much MUCH faster, so it should be much cheaper to run a process to republish these regularly. I assume you can also bump up your lifetime to >24hr if you want - but note that will mean nodes not using pubsub (aka getting updates pushed to them proactively) might be slow to get new updates.
Here's what's new for IPNS in the release: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#ipn...
Here's what's new for IPNS in the release: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#ipn...
Self-plug and an experiment only, but here's the blog of how I wrote a Storage Driver for Docker: https://blog.bonner.is/docker-registry-for-ipfs/
> You can pull docker images from ipfs? How?
Like this https://blog.ipfs.io/2020-02-14-improved-bitswap-for-contain...
Note that this is also similar to Kraken[1] from Uber, and Dragonfly[2] from Alibaba. Facebook also does container and artifact distribution using BitTorrent, but I can't find a good reference to it.
[1] https://eng.uber.com/introducing-kraken/
[2] https://d7y.io/en-us/
Like this https://blog.ipfs.io/2020-02-14-improved-bitswap-for-contain...
Note that this is also similar to Kraken[1] from Uber, and Dragonfly[2] from Alibaba. Facebook also does container and artifact distribution using BitTorrent, but I can't find a good reference to it.
[1] https://eng.uber.com/introducing-kraken/
[2] https://d7y.io/en-us/
I was impressed to read that the IPFS public network scaled 30X in 2019. It sounds like they have gotten over a big hump for usability and utility.
anyone know good python ipfs resources + decentralized sync-able embedded databases ?
orbitdb is a distributed database built on IPFS - looks like they have a python http client: https://github.com/orbitdb/py-orbit-db-http-client
the py-ipfs-http-client library is also actively maintained (but I think needs some small changes to work with IPFS 0.5): https://github .com/ipfs-shipyard/py-ipfs-http-client/
the py-ipfs-http-client library is also actively maintained (but I think needs some small changes to work with IPFS 0.5): https://github .com/ipfs-shipyard/py-ipfs-http-client/
https://tahoe-lafs.org/trac/tahoe-lafs is an alternative.
I have a Go project that imports IPFS and libp2p libraries. What's the recommended way of upgrading those dependencies?
The release notes contain a pretty extensive list of what APIs have changed.
The path to upgrade the dependencies is probably to run `go get -u <dep>` for the direct dependencies you're including, and then fixing errors that pop up from doing so.
The path to upgrade the dependencies is probably to run `go get -u <dep>` for the direct dependencies you're including, and then fixing errors that pop up from doing so.
Depends what exactly you are importing and for what. I suggest you ask in the forums and post a link to your project. go get -u is dangerous as it updates all subdependencies used to latest.
Usually take go-ipfs's go.mod as a guide on what versions to use.
Usually take go-ipfs's go.mod as a guide on what versions to use.
>go get -u is dangerous as it updates all subdependencies used to latest
Yes, this is the problem I initially had; I would end up with a host of incompatible deps (leading me to wonder what the point of go.mod even was ... but I digress...).
Sounds like this is going to be one of those super fun dependency hack-jobs :C
Ah well, this is the price to pay for playing with beta software!
Yes, this is the problem I initially had; I would end up with a host of incompatible deps (leading me to wonder what the point of go.mod even was ... but I digress...).
Sounds like this is going to be one of those super fun dependency hack-jobs :C
Ah well, this is the price to pay for playing with beta software!
I really wish there was a C/C++ implementation of IPFS that could be dropped in to a Linux, Windows, MacOS, or iOS cross platform app.
I already ported the Go code to iOS once, and it wasn't that painful, but functionally it would be a lot more useful as a C++ code base. I don't build browser apps - I feel there is still a need for native apps especially in this particular space.
I already ported the Go code to iOS once, and it wasn't that painful, but functionally it would be a lot more useful as a C++ code base. I don't build browser apps - I feel there is still a need for native apps especially in this particular space.
Now I can do this myself, because I'm all "+337" and what not. (I used trickle as described in the comment[2]. Seemed to work fine. Nice and stable.) But I can't recommend Joe and Jane Consumer to install IPFS and some other thing with a straight face, because they'll say, "Well bittorrent can do it!?" and I don't have a good answer.
Maybe there's an opportunity there to rent IPFS VMs to normal people? I dunno.
https://github.com/ipfs/go-ipfs/issues/3065
https://github.com/ipfs/go-ipfs/issues/3065#issuecomment-415...