Show HN: gpudeploy.com – "Airbnb" for GPUs(gpudeploy.com)
gpudeploy.com
Show HN: gpudeploy.com – "Airbnb" for GPUs
https://www.gpudeploy.com
194 comments
If you let third party stranger to offer GPU resource, how do you deal with:
1. Privacy. An attacker can set up a GPU honey pot and and sell the data they got.
2. Fake GPU computation. An attacker can fake GPU and send back dummy data sometimes to reduce the computation.
3. Corrupt GPU. Practically same with 2. But not malicious intent. It's just the faulty GPU.
1. Privacy. An attacker can set up a GPU honey pot and and sell the data they got.
2. Fake GPU computation. An attacker can fake GPU and send back dummy data sometimes to reduce the computation.
3. Corrupt GPU. Practically same with 2. But not malicious intent. It's just the faulty GPU.
Regarding 2 and 3: depending on the type of computations you need to do there are ways to verify the integrity of a delegated computation with very high confidence for very low cost. Typically if you're computations happens in a modular structure it's quite easy. See for example my THC (trustable homomorphic computation) project and the accompanying paper: https://pablo.rauzy.name/software.html#thc
There is also a video presentation because the conference where the paper was published took place during a COVID induced lockdown: https://www.youtube.com/watch?v=6DByVlqpH0s
Can someone explain why a relevant peer-reviewed paper reference got downvoted?
Technically something like this is relevant, but most people would assume that in practice it is going to be way too much work to get it implemented on GPUs and in such a way that it doesn't add too much overhead.
Two and three are really the same problem and are solved by periodically running the same workload in duplicate on other systems and comparing results to detect unreliable sellers. Same as all the captcha systems. It's an overhead required to have trust built into a system. In the event of a faulty GPU, the owner would likely want to know anyways.
The first issue could be addressed by just never giving a seller enough data from a buyer to abuse, but that does require a critical mass of both buyers and sellers before you could distribute and dilute a single customer's workload enough. The company could bootstrap this with renting their own GPUs and subsidizing sellers at first though.
The first issue could be addressed by just never giving a seller enough data from a buyer to abuse, but that does require a critical mass of both buyers and sellers before you could distribute and dilute a single customer's workload enough. The company could bootstrap this with renting their own GPUs and subsidizing sellers at first though.
There's a privacy issue in both directions too: the tenant will be afraid their workload will be stolen, and the host will be afraid other data in their system will be stolen (if the sandboxing is imperfect).
Wait a year and see how kids get on blockchain to sell and buy GPU resources for rendering ‘trans furries’, or better analyse classmates’ stolen chats.
I can vividly remember we were by no means disclosing private info back in ‘94 only to see a world of influencers in 2024 sharing their very personal guts for profits.
I can vividly remember we were by no means disclosing private info back in ‘94 only to see a world of influencers in 2024 sharing their very personal guts for profits.
It's only an extension of big brother from the late 90s, to not say real world from MTV in the early 90s.
At the end of the days, everyone wants and hopes to be an idol or famous.
At the end of the days, everyone wants and hopes to be an idol or famous.
...but also at no point in known history has it ever happened to find millions of people doing so in very mean and aggressive way, and worldwide scene to exploit.
agreed it IS a Big Brother of a sorts, where everyone participates to certain degree. but then written forums have been a thing before BigBrother and perhaps don't have this voyeurism added in the equation.
agreed it IS a Big Brother of a sorts, where everyone participates to certain degree. but then written forums have been a thing before BigBrother and perhaps don't have this voyeurism added in the equation.
I know Azure has a confidential computing offering for GPUs, which I'm hoping will get broader uptake soon. It seems like the best way to address these concerns to me.
https://azure.microsoft.com/en-us/blog/azure-confidential-co...
https://azure.microsoft.com/en-us/blog/azure-confidential-co...
how would you propose this might work?
I'm not an expert on the area, but I've attended some conference talks on the subject at fosdem so I'll give it a go.
Essentially you're trying to provide a way to prove that the code running on the machine is what you instructed. This is achieved by a series of hardware attestations that measure and check the code to make sure it's what you requested. Generally this means encrypted ram at a minimum, and checks/balances that give you confidence this is the case (you have to trust someone, eg: Intel)
Is it perfect, probably not, but it's a lot better than just running VMs with unencrypted memory that any operator can jump into.
To my understanding most GPU workloads are not run in this way currently, and the operator can see/manipulate everything executed
Essentially you're trying to provide a way to prove that the code running on the machine is what you instructed. This is achieved by a series of hardware attestations that measure and check the code to make sure it's what you requested. Generally this means encrypted ram at a minimum, and checks/balances that give you confidence this is the case (you have to trust someone, eg: Intel)
Is it perfect, probably not, but it's a lot better than just running VMs with unencrypted memory that any operator can jump into.
To my understanding most GPU workloads are not run in this way currently, and the operator can see/manipulate everything executed
Reputation
Love the concept.
I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?
I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?
Main difference is that we are more opinionated (in terms of configurations) and sort of do the scrolling and sorting out for you – hopefully a bit smoother as a user experience. We sort out bad machines immediately. We're also directly working on making compute from unknown high-end data centers available, there's a lot of unused compute out there! See gpulist.ai
Also, don't know if vast.ai does this, but with us you can have 6 user sessions on your machine if you have six GPUs, so granular utilization is possible.
Also, don't know if vast.ai does this, but with us you can have 6 user sessions on your machine if you have six GPUs, so granular utilization is possible.
We own and operate 40+ data center GPUs (v100s, a100s, and ax000s) in a private cluster and use vast to rent unused capacity.
What would make you better than vast is extremely easy spot leasing and job prioritization.
I want to be able to have one of our training jobs finish, and then have the capacity immediately transition to a lease. With vast, we are renting in week long blocks.
What would make you better than vast is extremely easy spot leasing and job prioritization.
I want to be able to have one of our training jobs finish, and then have the capacity immediately transition to a lease. With vast, we are renting in week long blocks.
You should be able to do that right now on vast. You just need to rent the gpus yourself with your own on demand instance(s) for your training job. As soon as it finished you then stop or destroy those instance(s) and the GPUs are available immediately (and if there are any other instances queued up in scheduling they will start up). Your actual job doesn't necessarily need to run in the container (if you know what you are doing).
(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)
(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)
Can you expand on spot leasing and job prioritization? What kind of api would you prefer? How would you like to adjust time slices?
You can do that on llm.sxwl.ai
Shoot me an email at [email protected] for instructions, the web site is pretty outdated, the main UI is through restful API (which we don't have time to write doc yet)
If you have time to answer emails with instructions, you have time to update your site and documentation. Why not just do that?
Don’t build something if you don’t have a use case. All you have is a wishlist, until someone says “yes I want this here is $$$” which I assume the email will facilitate.
Im also interested in what the differing factor is. Would also like to see more documentation for onboarding rather than just "Ubuntu and root available".
vast you have to choose specific machines. gpudeploy routes to whatever resources are available.
vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way. Someone on HN wrote a script to run a test docker image on every machine and auto-tagged the machines' quality using their API, which is what I'd do if I was going to use vast seriously for compute.
vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way. Someone on HN wrote a script to run a test docker image on every machine and auto-tagged the machines' quality using their API, which is what I'd do if I was going to use vast seriously for compute.
>> Im also interested in what the differing factor is.
> vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way.
Wouldn't gpudeploy have exactly the same problem? How is it mitigated with gpudeploy?
> vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way.
Wouldn't gpudeploy have exactly the same problem? How is it mitigated with gpudeploy?
I think it's more of a business strategy issue than a technical one.
I suspect it would be trivial for Vast or GPUDeploy to spin up a benchmarking job before allowing sales on that machine. I'm not an expert on PCIe lanes, but I would think the performance issues would be visible via bandwidth or latency on the lanes.
It kind of makes sense to me, though. If I were looking for absolute reliability and was willing to pay for it, I'd just go to one of the many GPU cloud vendors. Likewise, I suspect anyone willing to really work on getting good performance would rather be a real provider or sub-provider than being part of this nebulous C2C GPU cloud.
I suspect it would be trivial for Vast or GPUDeploy to spin up a benchmarking job before allowing sales on that machine. I'm not an expert on PCIe lanes, but I would think the performance issues would be visible via bandwidth or latency on the lanes.
It kind of makes sense to me, though. If I were looking for absolute reliability and was willing to pay for it, I'd just go to one of the many GPU cloud vendors. Likewise, I suspect anyone willing to really work on getting good performance would rather be a real provider or sub-provider than being part of this nebulous C2C GPU cloud.
I use vast.ai somewhat often. It's great!
+1 for vast. they usually are the cheapest and have the most supply. some instances can be less reliable at the low end though
Great idea and wishing you the best of luck!
Dropping a note that I've found https://akash.network/ ~ https://akash.network/gpus/ to be impressive, as typically with crypto projects it's all scams, however in this instance there's demand and legit usage. https://stats.akash.network/
Something to consider!
Dropping a note that I've found https://akash.network/ ~ https://akash.network/gpus/ to be impressive, as typically with crypto projects it's all scams, however in this instance there's demand and legit usage. https://stats.akash.network/
Something to consider!
I was excited to see their offering, but unfortunately they don't take fiat - only some specific cryptos.
Fiat proxied through USDC - a stablecoin offered by [Circle](https://www.circle.com/en/usdc).
I like how signing up requires agreeing to the terms of service and privacy policy, but both links go to empty pages.
These links should work now, at least they do for me. We use a third party service called Termly for these, I think there was an issue with the url (missing the www).
Might just be me but I think "On demand GPU provisioning" is easier to understand than "Airbnb for GPUs"
From the YC guidelines:
> One good trick for describing a project concisely is to explain it as a variant of something the audience already knows. It’s like Wikipedia, but within an organization. It’s like an answering service, but for email. It’s eBay for jobs. This form of description is wonderfully efficient.
Seems the risk of this is a loose simile.
Edit: Though, thinking about it, Airbnb for GPUs is fairly accurate for this model of a marketplace where people let out their GPUs for others to rent.
> One good trick for describing a project concisely is to explain it as a variant of something the audience already knows. It’s like Wikipedia, but within an organization. It’s like an answering service, but for email. It’s eBay for jobs. This form of description is wonderfully efficient.
Seems the risk of this is a loose simile.
Edit: Though, thinking about it, Airbnb for GPUs is fairly accurate for this model of a marketplace where people let out their GPUs for others to rent.
> Airbnb for GPUs is fairly accurate for this model of a marketplace where people let out their GPUs for others to rent.
Minimum rent 1 day? $100 cleaning fee? No quality control? Spy cameras in the bedrooms?
Minimum rent 1 day? $100 cleaning fee? No quality control? Spy cameras in the bedrooms?
Yeah I think in general it's a decent strategy. I'm this case it falls short for me because a key element of Airbnb is that the assets (homes/rooms) are provided by the user base, whereas this product doesn't seem to do that at all. So the strategy of using familiar companies is good, but I think it only works when the functional correlation is strong. In this case I don't think it is.
The audience here already knows what "on-demand provisioning" means.
On demand means you're another underdog that's just like Google or AWS offerings which might not be around next month.
Airbnb implies a two way market place. Much more interesting prospect for those having idle GPUs .
Not sure if electricity costs justify renting out and what pricing looks like, those details apart.
Airbnb implies a two way market place. Much more interesting prospect for those having idle GPUs .
Not sure if electricity costs justify renting out and what pricing looks like, those details apart.
More so than the electricity of the GPU, is the electricity in cooling the GPUs heat. (In most cases cooling, in some cases, I’m guessing the heat might be a welcome bonus but that’s more rare)
The issue with AirBnb is that it doesn’t have the best reputation nowadays.
on demand gpu provisioning misses the part that you can sell your gpu time, doesn’t it?
Interesting, these companies are springing up left and right. Software solutions to a hardware problem. It looks like most of the available compute is allocated, so they are going to have to get a bunch more providers onboarded.
I wonder why y-combinator is stuffing their investments with multiples of these similar companies... https://www.shadeform.ai/ is another one.
A few quick comments:
Reading the source of their install script:
https://gpudeploy-public.s3.us-west-1.amazonaws.com/join_clu...
It doesn't start off with set -e, which could result in an incomplete install, yet appear to finish.
It also installs some binary "instance-server"... who knows what it does... would you trust this on your server on your network?
It is nvidia specific... sadly, don't expect AMD gpus anytime soon.
Feels like a MVP, let's see how this grows over time.
I wonder why y-combinator is stuffing their investments with multiples of these similar companies... https://www.shadeform.ai/ is another one.
A few quick comments:
Reading the source of their install script:
https://gpudeploy-public.s3.us-west-1.amazonaws.com/join_clu...
It doesn't start off with set -e, which could result in an incomplete install, yet appear to finish.
It also installs some binary "instance-server"... who knows what it does... would you trust this on your server on your network?
It is nvidia specific... sadly, don't expect AMD gpus anytime soon.
Feels like a MVP, let's see how this grows over time.
Is there any way to monitor for sales of hardware if/when any of these companies dies so that I can get 1xH100 for personal use at a deep discount?
Ebay?
Is that where they really go? I feel like buying anything for several thousand dollars on Ebay is sketch. What if they ship me an empty box?
Back in my crypto asic mining days, I'd hear stories of people receiving boxes of cement blocks.
Looks like you can get a MI210 on ebay for $6500, with 30 day warranty. heh.
Oh, there are a bunch of H100's there...
Looks like you can get a MI210 on ebay for $6500, with 30 day warranty. heh.
Oh, there are a bunch of H100's there...
File a dispute with PayPal? And then with your credit card?
As the post points out, this is a pivot from a company that was a consumer of this kind of service in its original incarnation; YC had presumably nothing to do with it.
That an interesting presumption. If a startup is "Backed by Y Combinator", and it pivots, you don't think that Y Combinator would have some sort of opinion on that?
I would expect some sort of conversation like: "Oh hey, we have another company, in our portfolio, that is far ahead of you, doing exactly the same thing. Maybe you should do something else?"
But of course, this is AI... plenty of space for gpu marketplaces.
I would expect some sort of conversation like: "Oh hey, we have another company, in our portfolio, that is far ahead of you, doing exactly the same thing. Maybe you should do something else?"
But of course, this is AI... plenty of space for gpu marketplaces.
YC partners might have an opinion and might offer advice but they're always going to support what the founders want to do—that's kind of the core principle of YC.
As for "other company in portfolio", that's unavoidable when funding thousands of startups and almost always turns out to be a non-issue.
As for "other company in portfolio", that's unavoidable when funding thousands of startups and almost always turns out to be a non-issue.
Awesome, makes perfect sense. Thanks for the clarity @dang.
I'm curious now, if you can say. Was advice offered in this case? If so, what was it?
I'm curious now, if you can say. Was advice offered in this case? If so, what was it?
I'm not part of such discussions so I'm afraid I have no idea!
People here think you _are_ YC. Little do they know you're far more perceptive and capable than YC.
That's definitely not true, but thanks for the support :)
Truth is a spectrum. I will leave my opinions on YC unpublished. Thanks for all you do.
That's not how YC works. We're YC W20. We have a GPU offering. Guess how many conversations we had with YC about it.
What's YC going to do? They have a tiny stake in your firm. That's the point; that's what "founder-friendliness" means. They're not your board.
What's YC going to do? They have a tiny stake in your firm. That's the point; that's what "founder-friendliness" means. They're not your board.
Cool, thanks for the info Thomas. I honestly have never looked into YC at all, so I'm definitely not aware of how these things work. I was posting from an honest curiosity.
why does ycombinator invest in companies that will compete against each other, isn’t that a conflict of interest
Nope. That allows them to control the market. Find new ideas that could threaten the big 5 status quo. Fund them. Get them bought by one of the big five (making a good amount of money off that). Let the big five shutdown the competition.
Conflict of interest only applies if you think they’re concerned with the public good. There is no assumption of that here. Every man for himself.
Conflict of interest only applies if you think they’re concerned with the public good. There is no assumption of that here. Every man for himself.
Absolutely not. Why would it be?
For those that wants prices checkout: https://gpumonger.com/
This seems much more in depth, and a true service, but for those that just want to compare prices check out gpumonger.
This seems much more in depth, and a true service, but for those that just want to compare prices check out gpumonger.
There is no 'contact us' page, which is a little concerning.
They are not providing actual service - just price comparisons across different providers. Why is contact us page missing a concern here?
I'm a provider, I'd like to list my service. How do I do that? Since there is nothing obvious, then how do you know the providers they list are a valid representation of what is out there? Maybe there are other providers who don't want to pay to be listed? But how do you even pay?
I have read many of your comments on GPU-related posts. Occasionally, when I see your comments, I visit your website to see if I can learn anything about your business beyond what you are posting on HN. To this day, hotaisle.xyz contains nothing but a logo.
I don’t get it.
How can you expect a website to list your service when your own service’s website contains zero information? Why would you pay to list your service, when there is no information available about the service you provide? Am I looking in the wrong place? So confused…
I don’t get it.
How can you expect a website to list your service when your own service’s website contains zero information? Why would you pay to list your service, when there is no information available about the service you provide? Am I looking in the wrong place? So confused…
> hotaisle.xyz contains nothing but a logo.
To be fair, there is also a contact email at the bottom of the page.
> How can you expect a website to list your service when your own service’s website contains zero information?
We are the first and only (for now) verified MI300x provider on gpulist. In order to get verified, I contacted them directly, they asked for a few bits of information about my business, including my EIN. What I'm offering there, is exactly what I have today.
https://gpulist.ai/detail/3c18f8a
> So confused…
I know, it is ok. Let me explain a bit. We are starting small, so the website is the last focus right now. I know the general expected culture is to have some splashy page with a typeform on it, but hey... aren't we also a bit tired of that?
In order to even get access to buy these GPUs, you have to go through quite a lot of effort. You can't just buy them off the shelf from BestBuy. They are export controlled and I've agreed to not use them to build bombs. You have to have a valid business and a great story, or they won't even talk to you. Heck, I even had to prove my business was in good standing in Delaware. I'm pointing this out because I will need to know all my customers too. My business isn't something someone just signs up for on a website.
These GPUs are also extremely expensive. Imagine a 350lbs Ferrari. We started with 8 of them (one chassis) because they are super new and it was a proof of concept. Last year, we didn't even know for certain if AMD would double down on AI. This is all we initially raised funding for. As soon as we deployed the compute, we immediately had a customer on them, all without a website. Just word of mouth. By the way, the success of the PoC unlocked our next round of funding, and we are working on a much larger order of MI300x right now.
Don't worry, you'll get a website at some point. That said, these things sell themselves, you either have them or you don't. I've been very transparent and public about what we are up to. Would a website really help here? Maybe. But I've also started other extremely successful businesses originally without websites too. At the end of the day, I'd rather spend investors money on buying more compute, than a pretty website. Once I have some more revenue, I'll funnel that right back into the business and work on marketing/sales more.
If you're curious about anything, feel free to just reach out and ask. I'm not some corporate overlord suit wearing sales guy. I'm an open source tech nerd who's been in the business a long time. 20+ year ASF member, who co-founded Java @ Apache. Happy to answer any questions.
To be fair, there is also a contact email at the bottom of the page.
> How can you expect a website to list your service when your own service’s website contains zero information?
We are the first and only (for now) verified MI300x provider on gpulist. In order to get verified, I contacted them directly, they asked for a few bits of information about my business, including my EIN. What I'm offering there, is exactly what I have today.
https://gpulist.ai/detail/3c18f8a
> So confused…
I know, it is ok. Let me explain a bit. We are starting small, so the website is the last focus right now. I know the general expected culture is to have some splashy page with a typeform on it, but hey... aren't we also a bit tired of that?
In order to even get access to buy these GPUs, you have to go through quite a lot of effort. You can't just buy them off the shelf from BestBuy. They are export controlled and I've agreed to not use them to build bombs. You have to have a valid business and a great story, or they won't even talk to you. Heck, I even had to prove my business was in good standing in Delaware. I'm pointing this out because I will need to know all my customers too. My business isn't something someone just signs up for on a website.
These GPUs are also extremely expensive. Imagine a 350lbs Ferrari. We started with 8 of them (one chassis) because they are super new and it was a proof of concept. Last year, we didn't even know for certain if AMD would double down on AI. This is all we initially raised funding for. As soon as we deployed the compute, we immediately had a customer on them, all without a website. Just word of mouth. By the way, the success of the PoC unlocked our next round of funding, and we are working on a much larger order of MI300x right now.
Don't worry, you'll get a website at some point. That said, these things sell themselves, you either have them or you don't. I've been very transparent and public about what we are up to. Would a website really help here? Maybe. But I've also started other extremely successful businesses originally without websites too. At the end of the day, I'd rather spend investors money on buying more compute, than a pretty website. Once I have some more revenue, I'll funnel that right back into the business and work on marketing/sales more.
If you're curious about anything, feel free to just reach out and ask. I'm not some corporate overlord suit wearing sales guy. I'm an open source tech nerd who's been in the business a long time. 20+ year ASF member, who co-founded Java @ Apache. Happy to answer any questions.
This was my project that I recently sold. If you want to be listed I can put you in contact with the folks that bought it. LMK!
The lack of "contact us" was pure laziness at the time.
The lack of "contact us" was pure laziness at the time.
Congrats on the sale! Interesting to make a few pages and then sell it.
Sure, assuming the site will continue... my email is in my profile.
Sure, assuming the site will continue... my email is in my profile.
Neat Idea!
I was wondering, are there any security guarantees for the providers? Assume I have a small GPU cluster at home, if I rent out my GPUs, what sort of access should I accept the renters to have? Only GPU kernels would be sent to machine? Or will they have a limited permission user access on my cluster?
Also instead of having the operatos openning ports in their routers, were there any considers of adding them to a private network for a more seamless experience? (Nebula/headscale and the likes)
I was wondering, are there any security guarantees for the providers? Assume I have a small GPU cluster at home, if I rent out my GPUs, what sort of access should I accept the renters to have? Only GPU kernels would be sent to machine? Or will they have a limited permission user access on my cluster?
Also instead of having the operatos openning ports in their routers, were there any considers of adding them to a private network for a more seamless experience? (Nebula/headscale and the likes)
I'm curious how a business pivots from drone delivery to GPU Airbnb?
I got it from the line about how they trained their robot models - think of Amazon pivoting to aws, the substrate becomes the commodity. Unless you're asking if it makes sense, or what went wrong? Then I don't know. I imagine startups that survive usually pivot from something else?
I've never run a start-up (successful or otherwise :) ) but I've also heard that being able to pivot can be really useful - Flickr is often cited as an example.
Also, "GPU on demand" sounds _a_lot_ easier than "drone-based delivery". Between Seti@Home/Folding@Home/etc, various grid-computing/clustering/orchestration stacks that already exist, etc it seems reasonably doable to implement in a year or so. "drone-based delivery" sounds capital-intensive, sounds like you'll need to spend a lot of time building a professional network of business people who might use the service (so there's a 'cultural friction' between techie founders and business folks, potentially), plus the ever-looming threat of Amazon/etc figuring this out first.
tl;dr: I agree it's weird pivot, and good on the founders for being able to make the change! :)
Also, "GPU on demand" sounds _a_lot_ easier than "drone-based delivery". Between Seti@Home/Folding@Home/etc, various grid-computing/clustering/orchestration stacks that already exist, etc it seems reasonably doable to implement in a year or so. "drone-based delivery" sounds capital-intensive, sounds like you'll need to spend a lot of time building a professional network of business people who might use the service (so there's a 'cultural friction' between techie founders and business folks, potentially), plus the ever-looming threat of Amazon/etc figuring this out first.
tl;dr: I agree it's weird pivot, and good on the founders for being able to make the change! :)
While most people think it's cool and some think it's scary, drone delivery is ultimately something that people (that is: companies with money) don't need.
Turns out, using drones to deliver is also not that competitive either. Delivery vans are very cost-efficient, and for food delivery / on-demand delivery, drones are not able to carry most orders. So it's not even the regulatory pains that make this difficult, which are unbearable in their own right.
It was a lot of fun to work on and we would have definitely stuck with it if there was any interest. There was none, so we had to admit that to ourselves.
This is a hard pivot, but it's been very stimulating to work on.
Turns out, using drones to deliver is also not that competitive either. Delivery vans are very cost-efficient, and for food delivery / on-demand delivery, drones are not able to carry most orders. So it's not even the regulatory pains that make this difficult, which are unbearable in their own right.
It was a lot of fun to work on and we would have definitely stuck with it if there was any interest. There was none, so we had to admit that to ourselves.
This is a hard pivot, but it's been very stimulating to work on.
Btw thanks for this honest insight. When I read the bit about your pivot, I sort of rolled my eyes. But this is a really nice, sober reflection that actually builds credibility with people like me who might make snap judgements at first.
I hope you guys find traction.
I hope you guys find traction.
To add to "capital-intensive": the regulatory framework right now (in the US) for drone delivery is uncertain at best. While most of the rules & regs to enable drone delivery is already finalized & published, a few very important ones (such as BVLOS - beyond visual line-of-sight) have yet been finalized, and thus need one-off waivers from the FAA to allow. And getting those one-off waivers is what eats a lot of time & money.
Their parents are good friends of someone else’s parents.
They're chasing money. It's a tactic
What's the security story here?
What's preventing GPU providers from sending wrong results instead of actually running the computation? For example, send the last computed result? Is this something that the renter has to handle by adding their own checks?
In addition to the problem of the renter crashing your machine or reading your password through DMA, of course.
In addition to the problem of the renter crashing your machine or reading your password through DMA, of course.
What incentive would a GPU provider have to spend time figuring out what result to send for some custom application?
The incentive is huge, if I spend 2 milliseconds sending you your previous results instead of 2 hours running your new computation, I can (pretend to) run way more computations on the same hardware and collect hundreds of time more money.
At the risk of being exiled off the platform and earning nothing. Don't forget, there is a bit of KYC with Stripe.
ID verification before you can host and random audits from gpudeploy.
NO. That's the worst way to do almost anything on the Internet, and should be considered a last-line defense, if nothing else can be done. Here, it can be. See my comment above.
That's my whole question, do they do random audits, or is it the job of customers to double-check their results for possible attack or compute-theft and report.
It seems wrong to call it a "job of customers". It's like you wrote a Bitcoin client which didn't verified hashes of transactions, "trusting" everything. Or like serving a website with login feature supporting only HTTP, not HTTPS. It is a very basic feature of whatever software would connect to such services.
So it is the job of the customer to write their own Bitcoin or HTTPS client, in your metaphor.
Every technology was (very) underdeveloped at some early point in its evolution.
That's what I asked, how developed is it now. Why so defensive?
I don't know how developed is it now, I'm not associated with the startup shown in any way. It's mainly a question to them. However, in terms of wider industry, in general distributed high-performance GPU(-like) computing "for everyone" is in its infancy. 99% of what was already done up to this point was targeted to people who would both buy and supply power "in bulk", not "in retail". Perhaps with a little exception of several excellent projects like Folding@Home and other @home's.
Run 1/10,000 - 1/100,000 of computations locally, and also send them as tasks to be send remotely. If compare yields difference, repeat both. After, say, 10 tries, blacklist the provider. Of course it will take a lot more nuances to implement that, but that's the general idea. It's a no-brainer.
Sounds like a lot of work that I would expect the paid service to help with.
Yeah, it's "they" who should do that, of course.
[deleted]
Linux supports IOMMU on most platforms.
I fail to see how this relates. If you can't trust the provider, why does it matter whether they say they have IOMMU or not.
It relates to the "get your passwords over DMA part".
At the moment, we manually verify operators and are currently onboarding some tier-4 operators. Down the line, we'll have a 2-tier system where you can choose whether you want a verified machine or not. From the operator's perspective, everything runs inside Docker, configured with security best-practices.
I've always understood that containers are not proper sandboxes and shouldn't be used for containing untrusted code, no matter the best practices used. Has this changed in recent years? Do you have documentation for what sorts of best practices you're using and why they are sufficient for executing untrusted code?
You are correct from my knowledge. I would expect that if the container is set to not run as root you might be able to enforce fine meaningful security but I’d still run it in a VM if feasible.
Having done a little bit of work in the area[1], I think you should publicly document exactly what those best-practices are. Are the workloads running in a networkless container? Do you limit IO? Do you limit disk usage? Answering these in detail would help you gain customer trust on both sides.
[1]: https://containerssh.io/v0.5/reference/docker/#securing-dock...
[1]: https://containerssh.io/v0.5/reference/docker/#securing-dock...
So you don't have real security for operators, is what you're saying.
Containers are not, and will never be, a secure isolation boundary.
Containers are not, and will never be, a secure isolation boundary.
probably very basic... so don't run it on anything that has your own data on it (if you're an AI startup, definitely don't run it on your research cluster).
> definitely don't run it on your research cluster
...what‘s the threat, actually? GPU time sellers stealing your secret sauce?
...what‘s the threat, actually? GPU time sellers stealing your secret sauce?
I think they mean don't lease out your research team's GPUs and allow random people to run untrusted code on your cluster, lest they figure out a way to break out of any sandboxing the software has in place and get loose in your network. The company's current answer to that concern is "everything runs inside Docker, configured with security best-practices", which is less than inspiring.
https://news.ycombinator.com/item?id=40261591
https://news.ycombinator.com/item?id=40261591
If I got an RTX 4090 sitting around idle, how much can I get for it?
That’s what I wondered, too. I feel the pricing page should be like a currency trader, showing a list of sell at/buy at.
You should be able to get 40-50 cents/hour.
This is an interesting idea but it would be cool if it were more granular, like I pay while my payload is executing only, and an API abstracts away which GPUs I'm running on, the execution environment, etc and just let's me push in code and get out data when it's done. Maybe that's what this is, not sure. I hit an account sign up before I could figure it out.
I think products like this have to evolve into pay by the hour otherwise you find alternatives that have it.
Tried deploying an instance, didn't work, no option to let me delete my billing info/ account. Identical services with better features and more availability exist
PS: It was a temporary server side caching issue
I have a cluster on RunPod and it's great, but there's definitely some opportunity in the space, for example if you can focus on transparency (utilization rates, more info about end user and their purpose, reporting) that would be helpful.
I think RunPod and Vast have most the market share but t's still early to the game
I think RunPod and Vast have most the market share but t's still early to the game
> "if you’re an AI company with idle compute (hopefully not in a Stability AI way)"
What are they referring to? Stability AI over provisioned hardware or something? Sorry not up to date with what's happening here, have developed a blindspot for AI hype.
What are they referring to? Stability AI over provisioned hardware or something? Sorry not up to date with what's happening here, have developed a blindspot for AI hype.
Back in my day we cooperated to advance medicine and science with folding@home. How times change.
https://en.m.wikipedia.org/wiki/Folding@home
https://en.m.wikipedia.org/wiki/Folding@home
And https://en.wikipedia.org/wiki/SETI@home. I suppose this is different, since I'd wager it'll end up being used primarily by for-profit projects.
It's strange how society psychology changes so much. SETI and Folding were huge in scale but not in hype.
folding@home was used in the “console wars” to justify one’s choice of the PS3 over the Xbox 360.
Was the cell processor in the PS3 really that efficient for this purpose?
Was the cell processor in the PS3 really that efficient for this purpose?
It was a fairly unique architecture that had some pros and cons. In reality what made it a winner was the steep subsidy by Sony (units were sold at a loss, making the assumption you would buy a few games) and the availability (later removed) of an official Linux distribution.
very few cared about linux. it was mostly hd-dvd (later dvd) vs blu-ray.
It was basically an all-purpose vector computing monster focused on SIMD. You could use it for physics simulations, animations, tesselation, etc. Basically everything you'd use a compute shader for nowadays.
That's why emulators need AVX512 support to match the PS3. It was incredibly powerful.
Obviously, in that era's single-threaded world no engine could make use of that functionality and few knew how to program for it. It was ahead of its time, by quite a while.
That's why emulators need AVX512 support to match the PS3. It was incredibly powerful.
Obviously, in that era's single-threaded world no engine could make use of that functionality and few knew how to program for it. It was ahead of its time, by quite a while.
Remember around that time when the industry said that OpenCL would allow write-once run-everywhere compute code for a booming industry of diverse and competitive compute devices? I fell for that scam for a couple of years, before very fortunately getting a different job for long enough to watch the collapse from a safe distance.
Thought the same thing when I read this.
How times have and haven't changed.
How times have and haven't changed.
It'd be nice if there was any documentation on how the sharing side of things worked pre-account creation. Is it a VM image you distribute? Docker container?
There's a post how to list your GPU here: https://blog.gpudeploy.com/earn-passive-income-renting-out-y...
What are the benefits over Salad or Vast.ai?
If I pause a GPU instance will I be able to resume it on the same host so I can use the data generated before?
If I pause a GPU instance will I be able to resume it on the same host so I can use the data generated before?
What about privacy? Can I run sensitive data on these strangers' GPUs?
Seems like you have to rent for specific dates… that doesn’t feel very on-demand to me. It also means you have to know how much time you need. I’d love to be able to run a job with a remote gpu as if it were local, spinning up instances as needed. Then if things crash and I need some hours to figure it out, let me downsize it all.
I don’t get “final rate is usually lower”. Why is it lower, and by how much? And since it’s “usually”, what about the unusual case? Equal to the sticker price, or higher? But since you use “<” neither should happen? I would expect an exact price tag for a “no-nonsense” pricing scheme. Right now it’s just confusing.
Thanks for the input, you're right, maybe a fixed price is better. The idea currently is that the quoted price is an upper limit on the rate you pay and the final rate depends on what the GPU provider inputs, which will normally result in a lower rate than the upper bound. At the moment we have upper bounds if you want to be included in on-demand input, and also depending on the service you provide (security, network etc.). We determine pricing with partners 1-on-1 a the moment.
I would focus on offering a pricing model that entices people to want to pay.
I've found that usually this involves something extremely simple and easy to understand, with clear up front numbers.
I've found that usually this involves something extremely simple and easy to understand, with clear up front numbers.
Thanks for the answer, probably a good idea to include some version of this in the footnote.
doesn't seem all that cheap compared to whats available
What's so unique about this compared to just using RunPod?
I like runpod the most. They offer attached network drives so you can start a GPU instance, stop it, and restart while retaining the data. This way you don't have to sync data to and from external data sources.
If I want to run stable diffusion on other provides I have to install docker image, get data from somewhere, run models, sync data to somewhere so I don't lose it. The process can be tedious and time consuming.
If I want to run stable diffusion on other provides I have to install docker image, get data from somewhere, run models, sync data to somewhere so I don't lose it. The process can be tedious and time consuming.
> or if you’re an AI company with idle compute (hopefully not in a Stability AI way)
What does that mean? I’m likely missing some context, could anyone explain?
What does that mean? I’m likely missing some context, could anyone explain?
I think Stability is not doing well and ran out of money to pay for all their compute, so they should probably use some Airbnb guests right now.
My understanding is that Stability was renting GPUs from AWS rather than buying them[0]. So they have no capacity per se.
[0]: https://www.theregister.com/2024/04/03/stability_ai_bills/
[0]: https://www.theregister.com/2024/04/03/stability_ai_bills/
We have a similar product at https://llm.sxwl.ai/ModelRepository, also routing container workloads to GPU clusters (managed by Kubernetes).
The difference is that ours sell GPU compute hours, not machines.
The difference is that ours sell GPU compute hours, not machines.
Doesn't this just raise the price of gpu's, and further encourage people to hoard them?
How much do these kind of services pay the GPU owners? Before I sign up and add my nodes I need to know if it will be worth it considering I'm paying the power to run it, depreciation, etc.
Hmm, interesting. At $0.49/hr I get $0.30/hr profit out of my 4090s. It's more like Uber for GPUs. When I have spare capacity on my 4090s maybe I will look into the service.
Pretty neat, but is there any Windows or MacOS support planned? I wouldn't mind renting out my GPU when it's idle, but I don't really want to go through the process of dual booting etc.
Windows support is definitely something we are thinking about.
What are you doing GPUs that requires Windows or Mac support?
Video games
Have you looked at Nvidia GeForce NOW? It's like $10/mo for a pretty decent streaming gaming rig. I'm very happy with it - I don't have to deal with Windows and can play AAA games on my Macbook Pro at 60Hz (1080p).
> on my Macbook Pro at 60Hz (1080p).
I think you just answered yourself. Some of us like to play games at 4K at 80Hz+, with no subscription fees, no internet bandwidth requirements, no added latency, and ability to mod.
I think you just answered yourself. Some of us like to play games at 4K at 80Hz+, with no subscription fees, no internet bandwidth requirements, no added latency, and ability to mod.
Yeah, but that has nothing to do with the context of the question. Someone is specifically asking about accessing a GPU over the internet for video games.
Nowadays it's rare that a Windows-only game that I want to run doesn't run flawlessly on my Linux machine (through Proton/Wine). I wouldn't recommend going outside of Steam though unless you're willing to do some troubleshooting.
Try running COD Warzone on Linux and not get quarantined to cheater-suspect lobbies.
Ah yes, apologies that's an important caveat I forgot to include.
Games that use anti-cheat are a mess on Linux. I don't play any of those games, and if you do then you're likely to run into some trouble with Linux-only.
Games that use anti-cheat are a mess on Linux. I don't play any of those games, and if you do then you're likely to run into some trouble with Linux-only.
Yeah anti cheat sucks overall tbh. I loathe giving kernel level access to a random video game’s anti cheat system.
Short of dedicated hardware (Xbox/ps) I’m not sure what else could be done.
Short of dedicated hardware (Xbox/ps) I’m not sure what else could be done.
So what are you looking for, an RDP session with an attached GPU?
I remember gamedevs wanting a company like this to test video game drivers and shader code on a slew of different cards and vendors but this seems clearly set up for LLM work.
Make sure your users can run profiling tools! Specifically Nsight Compute needs containers with some elevated privileges. That would get you an easy win over vast.ai .
This is really great! I hope you succeed. At Phind, we’re customers of Voltage Park and SF Compute for something similar. Let me know if you’d like to chat!
Hello, I'm currently in the market for an on-demand GPU host so this is perfect timing. What kind of privacy can I expect from this service?
[deleted]
If I have a 3060 TI on my laptop, could I somehow rent that out? Just curious. Not that I intend to do that, but wanted to know.
What kind of utilization percentages do you expect to be able to provide? Do you have any existing usage/ROI data?
Just launched but will put a panel on the landing page for that, once the data comes in.
I wonder if I could get strangers to heat my home with GPUs in the winter...
Yes: https://techcrunch.com/2020/03/31/qarnot-raises-6-5-million-...
> French startup Qarnot (...) manufactures heaters and boilers with a special trick — they pack computers as computers tend to generate a lot of heat. Qarnot then lets companies leverage that computing power by running tasks on those unusual servers.
> French startup Qarnot (...) manufactures heaters and boilers with a special trick — they pack computers as computers tend to generate a lot of heat. Qarnot then lets companies leverage that computing power by running tasks on those unusual servers.
Yes, great use-case.
Side note: you contact form doesn’t work - do you have an email I send some questions over to?
I am interested in the business logic of paying to hosts. Say someone had netted sub 10 dollars for hosting their GPU on the platform. How do you make that payment? Business have to pay fixed plus variable transaction fees. So the small payments to a large pool of users should be quite involved specially with international payments.
Since the dawn of torrenting (and to some extent the the Silicon Valley TV show) many folks tried to launch a P2P resource pool platform. The challenge is that small amount of money being distributed to a large pool is challenging. Then there is the issue of stability. Do you penalize hoster who fail to provide a stable service?
Hosters must have "trust" in the system and platform can only afford to pay out only when the payable amount to hosters reach a certain threshold. Like you can't withdraw unless you have about $100 charged invoices.
And that is where I see the concept of crypto tokens appear because trust as a short term mechanism can be consoled via crypto tokens.
Engineering an service to take advantage of idle resources is a non-trivial task specially if the service is hooked up to doing something dedicated. For the hosters energy is not cheap, maintenance is a hassle and guranteeing stability is a pain.
I would love to learn how you would address these issues.
Since the dawn of torrenting (and to some extent the the Silicon Valley TV show) many folks tried to launch a P2P resource pool platform. The challenge is that small amount of money being distributed to a large pool is challenging. Then there is the issue of stability. Do you penalize hoster who fail to provide a stable service?
Hosters must have "trust" in the system and platform can only afford to pay out only when the payable amount to hosters reach a certain threshold. Like you can't withdraw unless you have about $100 charged invoices.
And that is where I see the concept of crypto tokens appear because trust as a short term mechanism can be consoled via crypto tokens.
Engineering an service to take advantage of idle resources is a non-trivial task specially if the service is hooked up to doing something dedicated. For the hosters energy is not cheap, maintenance is a hassle and guranteeing stability is a pain.
I would love to learn how you would address these issues.
DDoSaaS
what is a Stability AI way?
Related: https://vast.ai is good and cheap. Just don't put any sensitive data on these GPU machines.
I've been a happy user of vast.ai for some time now.
I've been a happy user of vast.ai for some time now.
YC w24 company here. We just pivoted from drone delivery to build gpudeploy.com, a website that routes on-demand traffic for GPU instances to idle compute resources.
The experience is similar to lambda labs, which we’ve really enjoyed for training our robotics models, but their GPUs are never available for on-demand. We are also trying to make it more no-nonsense (no hidden fees, no H100 behind “contact sales”, etc.).
The tech to make this work is actually kind of nifty, we may do an in-depth HN post on that soon.
Right now, we have H100s, a few RTX 4090s and a GTX 1080 Ti online. Feel free to try it out!
Also, if you’ve got compute sitting around (a GPU cluster, a crypto mining operation or just a GPU) or if you’re an AI company with idle compute (hopefully not in a Stability AI way) and want to see some ROI, it’s very simple and flexible to hook it up to our site and you’ll maybe get a few researchers using your compute.
Nice rest of the week!