If you want to know anything more about this system, we are having a get together at PyCon tonight at 10pm in Room F (thanks warner!) and then presenting at the RSA Conference at the beginning of March (search the agenda for "tahoe").
We have an open source project http://allmydata.org that has been doing this for quite awhile. I'm also involved in the commercial side which does online storage and we've been running a business on a P2P backend (nice low costs) with non-peer clients. We tried a business model with a full peer grid and users were extremely uncomfortable storing "data" from other people on their computers. Possibly the market is better educated now and/or more used to this idea, but it may be a hard sell.
We run a small specialized storage company and the things that seem to matter most are: storage capacity, availability, reliability, transfer rates for both current data usage and new data addition.
40Tb can be handled pretty well by S3 and other storage services and they have pretty good pricing information to model your costs. Note that they don't (yet) provide very specific SLA's for data availability, so keep that in mind when designing your system.
Maintaining your own drives with some sort of redundancy (RAID, automatic copies, etc.) or using something like (bias alert) our open-source project http://allmydata.org which is effectively a software RAID layer both require some IT and systems energy, so this has to be bundled into your operational costs if you choose that route.
Just to emphasize what others have mentioned, it is important to incorporate the new data influx rate into your model. If you are successful, 40Tb this year might turn to 120Tb next year, so make sure that your cashflow model can support the underlying cost of whatever system you choose.
We decided to write our new architecture from scratch as an open source project to support our commercial business offering (online storage and backup). There are several factors to consider, roughly in order of importance (according to me):
* target audience
* marketing advantage or disadvantage
* peer review importance
* publicly enforced project management
* contribution from open-source community
Our target audience is composed of consumers who want a simple product to backup their computers and a more sophisticated audience that wants to design and maintain their own distributed backup service. For the consumers, open-source isn't necessarily a plus or minus (see marketing below) but for the more sophisticated users (enterprise IT folks) open-source is great if they need to make modifications for their special needs.
Similar to above, marketing your project/product/service as open-source may be very attractive to some audiences (enterprise IT, hackers who want their own storage grid), irrelevant to some (consumer PC user), and negative to others (some investors, some potential buyers).
One very strong benefit that we've received is peer review by experts in the field. Because our system must securely and robustly store sensitive information, the more people who read through our design and implementation and comment on it the better. Several design and implementation suggestions have helped us fix or improve our system immensely.
Because our project is open-source the peer pressure placed upon good design, documentation, and code is much higher than in my previous experiences with proprietary software. Though not perfect, it means that our team has worked hard to make the system usable and not pushed off important items that might not be visible to a busy project manager.
Code contribution has not been a major focus of our project, though we have received several very good additions to our project. I believe (but not backed up with data yet) that our particular project does not lend itself to large amounts of people coding on it as it is not (yet) a platform.
I do this already in the Bay Area in California. There are several carpool pickup points in the East Bay (Oakland/Berkeley) that drop off in downtown San Francisco. I either pick up people here or hitch a ride every morning. The advantage is that you can use the carpool lane which saves 10-15min during rush-hour and a $4 toll.
We are using ADP TotalSource for payroll and PEO functions. Though it is nice to have everything in more or less one place, the response time and customer service is extra-medium. By that I mean that most of the time if there is a problem with health care coverage, I end up personally having to call the provider, ADP merely provides a phone number. On the positive side, the prices are reasonable and the variety of plan offerings is good too. They just started up their west coast operations last year.
We looked at Administaff as well and the main reason for going with ADP was a pre-existing relationship via payroll.
I would first determine if your source code really warrants the effort needed to keep it secure from prying eyes. We've evaluated several factors including the source code, the service built leveraging that codebase, the people involved in the execution, and the business plan that mixes everything together. For us, in every scenario we could model the value generated by the code was much higher if it were open-sourced.
For example, because the code is open to all, the developers are motivated to produce much better documentation and testing harnesses, usually before they actually do the code itself - internal costs go down. This pays for itself many times over in time saved during debugging and deployment.
Another example is that we've received a lot of critical peer review of our code which has helped us catch and fix flaws in our security and design - internal costs go down, public perception of security becomes positive.
In my experience and market research, it is nearly always the execution of the business that significantly outweighs any super-secret Python methods I may have thought were cool at 2am :)
Tahoe is aimed at the issues that arise after you buy your 1TB drive which provides local storage quite well. We try to provide data reliability and availability that can survive colocation outages, computer and hard disk crashes, complete local data loss due to natural disasters, etc. Because the encrypted data is distributed over a wide number of servers, it is very difficult to lose data once it is stored in a grid. Also, it allows you to remotely access your data from the web or other authenticated clients.
For our production servers, we can lose up to 66% of all of our servers simultaneously before any data loss will occur. This parameter is tunable such that you can make it even more robust at the expense of more data being stored and pushed around the network.
Yes, Brian presented at PyCon a couple weeks ago. Tahoe is the open-source project that we are working on together along with a few other guys. We also are involved in the commercial entity Allmydata.com which uses Tahoe as a back-end. We sell a managed storage service on top of a grid of servers that we maintain.
Oh, I should add that we are having a HackFest at our office tonight! I know it's short notice, but every month or so we order pizza and beer and have a couple loose presentations, usually about distributed systems. Anybody is free to come, consume, present, or just sit in a corner and program. Logistics are below:
http://allmydata.org/pipermail/tahoe-dev/2008-March/000462.h...
The acquisitions I've seen typically have the following components to the compensation package to negotiate before closing:
* Salary (should be market for your new skills and position)
* Retention bonus (usually paid out over 1-2 years)
* Conversion of equity (either to cash or to new stock)
The above depend a bit upon the acquiring company. If you are acquired by a large public company, then you should expect a competitive salary, reasonable bonus to keep you around, and some combination of cash-out and new stock to keep you strategically aligned. If you are acquired by another smaller company then your salary may be lower (but fair for that size of company), bonus may be given in stock, and your equity package is probably pretty flexible depending upon how long they want to you to stay.
As a founder, I took $1/yr salary with the expectation that I'd get a market rates when we secured funding. As an acquisition I negotiated what I thought were fair market rates for salary and stock in the new company with a bonus to keep me happy to stay. When acquiring companies we would offer competitive salaries, retention bonuses (stock, cash, or both) for people we wanted/needed to keep around, and stock conversion (we were public by that point).
In your specific case, I believe that your salary should be market rate for your skills and your new position. The acquisition price is independent of your salary and should be treated as such.
* influence a wide range of issues from tech to office location
* great experience for founding a company later
Note that I didn't include "huge potential financial gain" in that list. A non-founder at a startup really has to value the experience because most likely there won't be a huge financial win. That doesn't mean that you shouldn't try to negotiate a fair compensation package, but really you are using the opportunity to gain skills and experience to use when founding your own company where you will have more ownership and potential financial gain. If you don't believe that the founders take more risk than the employees now, you will once you found your own company :)
If financial gain is your main goal, you may want to take a serious look at getting into the financing side. This will take a different approach, but good analytical skills plus good networking skills are in high demand. Alternatively, you could go down a more corporate path and aim for senior management at a good firm. I'm not necessarily recommending these paths, but it's worth exploring the alternatives so that you can really appreciate the (in my opinion) extraordinary benefits and privileges of working at a startup.
I should add that we are not focusing on file-sharing. Our commercial model (.com) is based upon managed backup services. When we looked at the possibility of file-sharing as a service, we were quickly dissuaded by the amount of potential litigation we could face.
We're working on an open-source distributed storage system that can also be used to easily share files. It also has p2p streaming download which is kind of fun to use. http://allmydata.org
I would second this. We currently run an open-source project (distributed storage) and commercial venture based upon it, but the open-source was the driver done because we thought that it was an important problem to solve and that it would be worth more as open-source than as a proprietary product. Our commercial venture arose by choosing one specific application of the code and building a business to address it. Note that our open-source project is something that we (and others) have worked on independently of the commercial venture. The community is separate (though aware) and drives the open-source project in directions that we did not expect.
The main thing I gleaned from undergraduate university (applied math) was how to collect and structure data in such a way that useful information can be derived from it. This has helped me in writing software for image processing programs to doing market research for new products. The key courses in this were signal processing courses along with the math and numerical analysis to understand the analysis techniques. The computer science courses I took in programming languages, compilers, hardware, etc. were very good practice in how to abstract a problem and codify it. I have not used too many of the specific languages or techniques in my professional life however as that tends to more or less keep up with the industry.
The most important things I gleaned from my graduate work about 10 years later (MBA and some more applied math) were how to present interesting data to people in a way that they too could interpret it and derive value from it, and how to manage social networks effectively.
We've had success outsourcing some of our non-core activities in our shop (4 p2p hackers working on distributed storage). For example, we've outsourced our website implementation and our Facebook app. Though any one of us could do them, it was much more efficient to draw up a set of screenshots and functional goals along with our API and then get back to working on our core project.
One key success factor is to find a high quality group to work with. We used recommendations, but I've also seen a few 2-person shops use oDesk find people for prototypes of applications they didn't have time/interest to build. The prices are extremely competitive, and often you can hire 3 people and use the best result.