and not only can the bot lie, it can disregard the robots.txt file altogether. just like the terms of service document for humans, you can choose to disregard it & deal w/ the consequences (blocked IP's, lawsuit, etc).
robots.txt is just a version of the TOS that computers can read.
probably not, but that's irrelevant. the point is facebook's robots.txt says you can crawl it but their TOS says you can't. facebook have already shown they are willing to sue anyone who tries to crawl without permission. this new model that facebook is trying to use isn't scalable, it favors the big guys and is bad for the open web.
you can also do this with a regular expression. here's the blog post i wrote on it a year ago. i use one function to grab get variables from either window.location or from a script tag. 5 lines of code.
practically speaking, most respectable email services will have an MX record. i tested it on about 30,000 emails & a simple MX lookup caught about 40 gmial's and only 1 false positive.
that reminds me of the girl who takes advantage of the zappos free returns policy and gets 5 new pairs of shoes every singles week only to mail them back a few days later. but zappos doesn't cut here off -- she is important evidence that they are serious about their policy & for zappos there are enough honest people out there to more than make up for her.
and while we're on the topic of how broken SSN's are...
why is it that the most important piece of identification you receive in your life, that you also need to keep forever, is printed on a crappy piece of paper???
It sucks that comments aren't enabled on that post. You can see the responses from the community (including Brett Slatkin one of the authors) on the Google Group for PuSH:
I don't know what you should do, and I don't necessarily think there is one right answer for everything in this position. I do know that almost every successful entrepreneurship I've talked to has a story about a failed relationships because of the sacrifices they made for their company. You've gotta weigh the costs for yourself and decide what you want most. This question reminded me of this article by Steve Blank. Hopefully it will help.