Twitter shut down Emojitracker's access(medium.com)
medium.com
Twitter shut down Emojitracker's access
https://medium.com/@mroth/u-1f647-person-bowing-deeply-5402c3ee5676#.z3df4bj6h
197 comments
So, the OP provided the email he recieved.
It did not ask him to clarify what he is doing, it did not say anything about bounds of acceptable use cases or what they might be, it did not encourage him to contact Twitter directly, or provide a link to do so. It did _not_ "inquire about why the developer needs elevated access", there is no such inquiry in the email. It _did_ say elevated access was going away.
It said he could use the free API with rate limits, or the commercial Gnip API.
It kind of half-heartedly suggested he could "reply to this email" if he had "questions". From the email, there was no reason to think elevated access might still be available, it pretty clearly said it would _not_ be, so I'm not surprised he didn't have any questions -- the email was quite clear (at saying something pretty different than you are saying).
So when you describe the email that goes out -- it does not seem to describe the one he received, according to him in the post. Are you talking about a different email? That he did not receive, or that you think he should have received in addition? Or do you actually think that email somehow communicates what you describe above? (It really really does not, which is why I think you must be thinking of a different email he did not receive or post, or you were internally misinformed about what the email was going to say).
With the email he actually pasted into the OP it is not surprising that he simply publicly notified his userbase and other interested parties that the service would be going away -- what else do you do when your upstream provider tells you the service is going away? This is all very standard and professional.
But, if you are saying that you do continue providing free elevated access to certain projects that seem worthwhile and meet some 'boudns of acceptable use', then that is nice, cool, I'm glad Twitter is doing this. (Maybe you should have told him that in the email though! And it would be great to actually advertise that fact, and what the bounds of acceptable use are, and how someone can get in touch with you to request access.)
It did not ask him to clarify what he is doing, it did not say anything about bounds of acceptable use cases or what they might be, it did not encourage him to contact Twitter directly, or provide a link to do so. It did _not_ "inquire about why the developer needs elevated access", there is no such inquiry in the email. It _did_ say elevated access was going away.
It said he could use the free API with rate limits, or the commercial Gnip API.
It kind of half-heartedly suggested he could "reply to this email" if he had "questions". From the email, there was no reason to think elevated access might still be available, it pretty clearly said it would _not_ be, so I'm not surprised he didn't have any questions -- the email was quite clear (at saying something pretty different than you are saying).
So when you describe the email that goes out -- it does not seem to describe the one he received, according to him in the post. Are you talking about a different email? That he did not receive, or that you think he should have received in addition? Or do you actually think that email somehow communicates what you describe above? (It really really does not, which is why I think you must be thinking of a different email he did not receive or post, or you were internally misinformed about what the email was going to say).
With the email he actually pasted into the OP it is not surprising that he simply publicly notified his userbase and other interested parties that the service would be going away -- what else do you do when your upstream provider tells you the service is going away? This is all very standard and professional.
But, if you are saying that you do continue providing free elevated access to certain projects that seem worthwhile and meet some 'boudns of acceptable use', then that is nice, cool, I'm glad Twitter is doing this. (Maybe you should have told him that in the email though! And it would be great to actually advertise that fact, and what the bounds of acceptable use are, and how someone can get in touch with you to request access.)
> which provided no particular encouragement that there might be other terms available
That's not entirely correct. Under option 2 after mentioning it's commercial side, it says:
There are also other solutions that offer varying levels of access as well as historical search.
It's not prominent (likely because they only want it pursued in special cases), and it's easy to miss, but it is there.
That's not entirely correct. Under option 2 after mentioning it's commercial side, it says:
There are also other solutions that offer varying levels of access as well as historical search.
It's not prominent (likely because they only want it pursued in special cases), and it's easy to miss, but it is there.
That's referring to Gnip still. They offer several levels of access and historical search, in addition to the firehose stream, for money.
> If they want all these special access cases handled through Gnip, whether they are discounted or free, that's their call.
There's nothing in the mail suggesting they "want special access cases handled through Gnip".
> If they want all these special access cases handled through Gnip, whether they are discounted or free, that's their call.
There's nothing in the mail suggesting they "want special access cases handled through Gnip".
So? If they want all these special access cases handled through Gnip, whether they are discounted or free, that's their call. Just because it's through the commercial arm doesn't mean they would be incapable of handling special cases.
In any case, it's all speculation, there was never a dialogue, and the HN title as it current stands is probably inflaming this more than it should be. Their access was not shut down (and the blog post's title and wording does does not say it is shut down, but that they believe they will be shut down). An email was sent, it was poorly worded, and the recipient overreacted (or at least prematurely reacted). The responsible thing to do would have been to contact Twitter or the sender and discuss the needs and goals of the project, and see what they could offer.
Immediately writing a public blog post is not negotiating in good faith.
In any case, it's all speculation, there was never a dialogue, and the HN title as it current stands is probably inflaming this more than it should be. Their access was not shut down (and the blog post's title and wording does does not say it is shut down, but that they believe they will be shut down). An email was sent, it was poorly worded, and the recipient overreacted (or at least prematurely reacted). The responsible thing to do would have been to contact Twitter or the sender and discuss the needs and goals of the project, and see what they could offer.
Immediately writing a public blog post is not negotiating in good faith.
> Immediately writing a public blog post is not negotiating in good faith.
Yet this seems to have been quite effective, given that the head of Developer Relations at Twitter is replying to this story on HN..
Yet this seems to have been quite effective, given that the head of Developer Relations at Twitter is replying to this story on HN..
I don't question that. Yet doing things based purely on their effectiveness for your goals without any regard to the consequences for other parties is semi-sociopathic behavior, which is why it's frowned upon. Whether something is effective or not should not be the only metric by which we decide whether it's the action we should take.
A small-time developer asking a big company to be reasonable practically never works. Likely first response is from a low-paid customer support person who has no idea what you're talking about, likely second response is from an aggressive business or law type in the company.
Public shaming on twitter+blog is 50/50. It often gets the attention of an engineer and sometimes even a founder, and when it does the response is often reasonable. Why would you ever try anything else?
I've had a crazy technical issue with AWS that our account representative proved useless for about but a tweet got it fixed. Also see news articles about people suffering the most ridiculous treatment from comcast/timewarner/verizon/at&t until they get in the news and everything is finally sorted out. This is a totally standard thing you should already be familiar with. The problem is that these big companies get too many queries from crazies and people who have no idea what they're doing, they're inundated with stupid support requests. So a legitimate support request will never be noticed by anyone who knows anything. So you need a sort of public vetting process. You need the "shaming" part to express the priority in large companies where business people have all the power.
I'm not sure there's a better way, that's just how it is.
Public shaming on twitter+blog is 50/50. It often gets the attention of an engineer and sometimes even a founder, and when it does the response is often reasonable. Why would you ever try anything else?
I've had a crazy technical issue with AWS that our account representative proved useless for about but a tweet got it fixed. Also see news articles about people suffering the most ridiculous treatment from comcast/timewarner/verizon/at&t until they get in the news and everything is finally sorted out. This is a totally standard thing you should already be familiar with. The problem is that these big companies get too many queries from crazies and people who have no idea what they're doing, they're inundated with stupid support requests. So a legitimate support request will never be noticed by anyone who knows anything. So you need a sort of public vetting process. You need the "shaming" part to express the priority in large companies where business people have all the power.
I'm not sure there's a better way, that's just how it is.
Going public is incredibly short sighted and almost never effective. People are people and turning a potentially innocuous situation into "us versus them" just makes them less likely to want to help you.
I got a problem with logging into a service a couple of weeks ago. I sent them mail, no answers. I wrote on their Facebook page a few days later (no harsh words). They called me home in a couple of hours and we fixed the problem on the spot. I wrote again on fb to thank them.
Going public means getting in front of people that care more about customers or has more time to do so. It feels a little like skipping the line though.
Going public means getting in front of people that care more about customers or has more time to do so. It feels a little like skipping the line though.
The difference here is that you attempted normal contact first, and waited for a reply. If everyone immediately defaulted to publicizing their problems with their relationships with companies, it would be both more annoying for everyone and less effective. The importance they place on your publicly aired problems is relative to how many of those problems they see and the expected negative impact. A thousand people complaining about your service daily will not receive the same attention and treatment as when there's a single person complaining.
It's just getting to the point where emailing a company through the normal support channels is just a joke. Their email queue is days long, while their Twitter queue is minutes.
What needs to happen, and probably has little chance of happening, is for people to always mention how understaffed the support at the company in question must be. These companies are under-staffing their support and it's contributing to the situation where things often need to be publicly escalated before they get the attention they need. Making support and customer service part of their reputation will go a long way towards solving that. There's really no excuse for support as atrocious as some companies have become known for. Sure, some of the products are free, but at the same time they are able to monetize the aggregate of all that free use, so there's really no excuse, since the free use and their revenues are intrinsically linked.
It makes them more likely to help you and less likely to do it happily. Depends on circumstances whether that is better for you.
I think the email goes further than poorly worded. It says pretty clearly that access will be cut off, gives no suggestion that there could be a negotiation.
The email ends with If you have questions, please reply to this email and we will be in touch to discuss.
I think it boils down to this, if the email said "Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Let me know if you have any questions." then I think the appropriate first step would be to contact Twitter and explain your situation. Since the actual letter contains that and just more information about possible options, I don't see why it should be viewed any differently.
I think it boils down to this, if the email said "Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Let me know if you have any questions." then I think the appropriate first step would be to contact Twitter and explain your situation. Since the actual letter contains that and just more information about possible options, I don't see why it should be viewed any differently.
I think you're unfairly abridging the e-mail that was sent. I think a more representative abridged e-mail would be:
> Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Your special access will be terminated on Thursday, April 21st. Let me know if you have any questions.
On receiving an e-mail of that nature, I would very likely let my audience know that I was shutting down the service on Thursday, April 21st just as the blog post author did.
I would not assume "Let me know if you have questions" meant "Let me know if you'd like to clarify your situation and request an exemption from this policy". I would assume it meant I could inquire about the details and process of the shutdown, or ask questions about the Gnip service or other options that they discussed in the e-mail.
> Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Your special access will be terminated on Thursday, April 21st. Let me know if you have any questions.
On receiving an e-mail of that nature, I would very likely let my audience know that I was shutting down the service on Thursday, April 21st just as the blog post author did.
I would not assume "Let me know if you have questions" meant "Let me know if you'd like to clarify your situation and request an exemption from this policy". I would assume it meant I could inquire about the details and process of the shutdown, or ask questions about the Gnip service or other options that they discussed in the e-mail.
See, my first action if I was running a free and open source service that used the API free would be to immediately respond and say, "Hey, I had free access because I'm running an open source project that uses this and it provides a free service. Is there no option for me to continue that in some way? If not, I'll have to shut down." Quick, simple, gets to the heart of the matter, and provides the department responsible at Twitter with the information most relevant to possibly letting me continue with free access.
Immediately going public can backfire, depending on the specifics of your case. Now there are possibly competing interests at play, the public pressure to let them continue, and possibly some anger over at Twitter for not being given the smallest benefit of a doubt (when their project works because of the good grace your company exhibited in the first place) or even asked before playing hardball. If I was the person responsible for making the call at Twitter, I would probably acquiesce, but I would want to tell this person to go get bent.
Immediately going public can backfire, depending on the specifics of your case. Now there are possibly competing interests at play, the public pressure to let them continue, and possibly some anger over at Twitter for not being given the smallest benefit of a doubt (when their project works because of the good grace your company exhibited in the first place) or even asked before playing hardball. If I was the person responsible for making the call at Twitter, I would probably acquiesce, but I would want to tell this person to go get bent.
A form letter saying "Contact us if you have any questions" politely implies "We don't really want to hear your questions and no exceptions will be granted".
If they don't wish to communicate that message, the message should end with at the very least something like "We don't want to terminate a popular service in error, so please contact us today to tell us more about it."
If they don't wish to communicate that message, the message should end with at the very least something like "We don't want to terminate a popular service in error, so please contact us today to tell us more about it."
> Immediately writing a public blog post is not negotiating in good faith.
Did everyone miss this part of the blog post?
> I’ve replied to Twitter asking for an exception, but I honestly don’t expect to receive one—and even if I were to, it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me.
Did everyone miss this part of the blog post?
> I’ve replied to Twitter asking for an exception, but I honestly don’t expect to receive one—and even if I were to, it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me.
That doesn't really make a difference to my point if it's done prior to getting, or waiting an acceptable time period for, a response.
[deleted]
[deleted]
[deleted]
Not really. That just implies that all the other solutions are through Gnip, not through Twitter Streaming.
> Over the last six months, we've started contacting all of those API key holders with elevated access and asked them to clarify what they're doing. In some cases, we know the business and business owner and reach out personally. In other cases, like this one, the business is listed in our systems as "N/A", so we send the template mail.
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
If this was the intent, the template email was kind of poorly drafted, because that's not what it says at all.
It says multiple times, without qualification, that they're going to be cut off — "this type of elevation will no longer be available," "your current elevated rate limits will no longer be available," "we will remove this elevation from your account," and "we encourage you to evaluate alternative options." At no point does it say that keeping the elevated access is an option.
So I mean, can you really blame the owner of this app for reading these absolute statements as being absolute and believing their app was cut off, rather than seeing it as some kind of sideways inquiry as to whether they think their use is acceptable?
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
If this was the intent, the template email was kind of poorly drafted, because that's not what it says at all.
It says multiple times, without qualification, that they're going to be cut off — "this type of elevation will no longer be available," "your current elevated rate limits will no longer be available," "we will remove this elevation from your account," and "we encourage you to evaluate alternative options." At no point does it say that keeping the elevated access is an option.
So I mean, can you really blame the owner of this app for reading these absolute statements as being absolute and believing their app was cut off, rather than seeing it as some kind of sideways inquiry as to whether they think their use is acceptable?
The email is perfectly drafted: it discourage comebacks while remaining vague enough for 'developer relations' to engage in damage controls when it struck some noisy player with large visibility.
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so.
As many others have pointed out, the e-mail from Twitter in the blog post absolutely does not state this. I add my voice to the chorus of others asking if there is an alternate e-mail, or if you and Twitter truly believe the e-mail in the blog post ever made this statement.
> The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate.
Again, joining in the chorus of voices, I want to ask why it was unfortunate that the developer made a public statement that his service would be ending? In the correspondence with him it was clearly stated "On Thursday, April 21st we will remove this elevation from your account" (contradicting your claim that you "asked them to clarify what they're doing"). I would argue that the developer has the responsibility to notify their audience that the service is going to be shut down at that time.
Why is a public response anything but appropriate given the content of the e-mail that Twitter sent to him? If the situation you describe here is truthful, then don't push blame onto the external developer when the e-mail didn't come close to communicating what the Twitter team thought it did.
As many others have pointed out, the e-mail from Twitter in the blog post absolutely does not state this. I add my voice to the chorus of others asking if there is an alternate e-mail, or if you and Twitter truly believe the e-mail in the blog post ever made this statement.
> The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate.
Again, joining in the chorus of voices, I want to ask why it was unfortunate that the developer made a public statement that his service would be ending? In the correspondence with him it was clearly stated "On Thursday, April 21st we will remove this elevation from your account" (contradicting your claim that you "asked them to clarify what they're doing"). I would argue that the developer has the responsibility to notify their audience that the service is going to be shut down at that time.
Why is a public response anything but appropriate given the content of the e-mail that Twitter sent to him? If the situation you describe here is truthful, then don't push blame onto the external developer when the e-mail didn't come close to communicating what the Twitter team thought it did.
Kudos for caring enough to come here and respond directly. But man, Twitter just seems to have no consideration at all for the people and businesses who build on the platform.
Every 6 months you guys make a new change that kills a bunch of websites who were silly enough to rely on your APIs. Then you give an explanation like the one above: something about needing to grow the platform, and pushing the responsibility for the problem back on the businesses.
When you killed the tweet count a few months ago, the story was exactly the same - something about depreciating it in order to build a more dependable platform, and people shouldn't have relied on it in the first place. It's clear you guys just wanted to take control of your data.
As a social network, I love Twitter. But I would never waste any time building on your platform. At every step you leave a trail of dead businesses in your wake.
Every 6 months you guys make a new change that kills a bunch of websites who were silly enough to rely on your APIs. Then you give an explanation like the one above: something about needing to grow the platform, and pushing the responsibility for the problem back on the businesses.
When you killed the tweet count a few months ago, the story was exactly the same - something about depreciating it in order to build a more dependable platform, and people shouldn't have relied on it in the first place. It's clear you guys just wanted to take control of your data.
As a social network, I love Twitter. But I would never waste any time building on your platform. At every step you leave a trail of dead businesses in your wake.
We need to open up computer access and intellectual property laws so that the content stream can be combed over by anyone who is interested in doing so, with or without Twitter's permission. Big companies should not have the ability to destroy small businesses just because they changed their mind about something.
If a small business bases it's business on something produced by another company and does so without a contract, I don't have a lot of sympathy. That's the whole point of contracts. Clearly outlining what each party is responsible for and expects.
If a contract isn't feasible, at least make sure it's in the other side's best interest to keep the situation going. Getting free access when it costs the other side (in managing access, serving requests, handling support, etc, no matter how small) is not something I would feel comfortable basing a business off of.
If a contract isn't feasible, at least make sure it's in the other side's best interest to keep the situation going. Getting free access when it costs the other side (in managing access, serving requests, handling support, etc, no matter how small) is not something I would feel comfortable basing a business off of.
>Getting free access when it costs the other side (in managing access, serving requests, handling support, etc, no matter how small) is not something I would feel comfortable basing a business off of.
The difference is that the person is not getting anything that's not already offered to the general public. It's unfair to use the court system to enforce a company's exclusionary political decision that "everyone can access our website EXCEPT YOU", at least barring any actual contractual terms (not generic Terms of Use). The change I propose would not force anyone to accommodate a certain API change or provide any special functionality. It would simply prevent non-neutral access rules from being enforced via legal vectors.
Twitter would still be able to charge for access to the APIs they charge for now. They'd simply be unable to use the court system to compel someone to stop accessing the data that they have no copyright interest in and which they serve up to the world for free. I'm not sure if Twitter has tried to do this yet, but it's the normal step companies take once a consumer develops the ability to evade their IP blocks.
To be honest, the law already can be interpreted this way. The problem is that it often isn't. Companies have been able to convince non-technical judges that concepts like trespass to chattels are applicable any time someone is talking to their server. We need language in the law that will clarify the matter to prevent big companies from squashing small innovators that they find inconvenient or threatening instead of leaving it up to judge roulette.
The difference is that the person is not getting anything that's not already offered to the general public. It's unfair to use the court system to enforce a company's exclusionary political decision that "everyone can access our website EXCEPT YOU", at least barring any actual contractual terms (not generic Terms of Use). The change I propose would not force anyone to accommodate a certain API change or provide any special functionality. It would simply prevent non-neutral access rules from being enforced via legal vectors.
Twitter would still be able to charge for access to the APIs they charge for now. They'd simply be unable to use the court system to compel someone to stop accessing the data that they have no copyright interest in and which they serve up to the world for free. I'm not sure if Twitter has tried to do this yet, but it's the normal step companies take once a consumer develops the ability to evade their IP blocks.
To be honest, the law already can be interpreted this way. The problem is that it often isn't. Companies have been able to convince non-technical judges that concepts like trespass to chattels are applicable any time someone is talking to their server. We need language in the law that will clarify the matter to prevent big companies from squashing small innovators that they find inconvenient or threatening instead of leaving it up to judge roulette.
It's not even about access, it's about existence (at least my point was). If a company decides they just want to stop offering a feed, or change what it does or does not include, a contract protects you for at least some time before they and force that upon you. It's not always about the company wanting more money. Sometimes old products are obsolete, and providing them costs a non-negligible amount in maintenance and support. Contractual obligations can be the difference between turning it off next week or eight months from now when the last contract is up, and that can make all the difference in having enough time to pivot to a new or different data source.
Yeah, I think we're talking past each other a little bit here. It's understood that a company could stop offering some data that a company is dependent on altogether, but I think it's substantially less likely than a company threatening to terminate access to data that already exists (which is what happened here). I also think that if entrepreneurs know that they're free to gather any freely-available information, changes in the commercial offerings or access rules are less threatening.
Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
If we take away the ability of companies to selectively allow access to data that's available for free to the general public, it gives these guys a fighting chance. The web is a publication platform. It's equally absurd to say "I'm publishing this novel and everyone except kbenson can read it" as it is to say "I'm publishing this data to the general public and everyone except kbenson can access it". You can try to stop someone from accessing your data, but certainly no court is going to consider it reasonable to assist a company in preventing kbenson from reading a widely published manuscript. Data access is substantially the same thing, and should be treated the same way. Keeping non-disruptive entrepreneurs on the same footing as everyone else makes innovation more secure.
Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
If we take away the ability of companies to selectively allow access to data that's available for free to the general public, it gives these guys a fighting chance. The web is a publication platform. It's equally absurd to say "I'm publishing this novel and everyone except kbenson can read it" as it is to say "I'm publishing this data to the general public and everyone except kbenson can access it". You can try to stop someone from accessing your data, but certainly no court is going to consider it reasonable to assist a company in preventing kbenson from reading a widely published manuscript. Data access is substantially the same thing, and should be treated the same way. Keeping non-disruptive entrepreneurs on the same footing as everyone else makes innovation more secure.
I was responding to "Big companies should not have the ability to destroy small businesses just because they changed their mind about something." as a general statement, since it was phrased in a general way.
> It's understood that a company could stop offering some data that a company is dependent on altogether, but I think it's substantially less likely than a company threatening to terminate access to data that already exists (which is what happened here).
I'm not sure sure. There's regular griping on HN about Google cancelling services, even though they are generally beta services, because people expect them to offered. Some of these people griping are people that built products on Google APIs that were discontinued.
> Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
A contract? Maybe not. But if they have a clearly defined deprecation policy, that's a start. And if you are paying them, well that's a lot better, since they are incentivized to keep it going because of your (and hopefully others') money.
As for the rest (access to public API), I'm not really interested in arguing it, as we likely agree more than we disagree. :) The only caveat is that direct consumers of the API often don't behave like regular public users, and that may put a strain on the system and degrade performance for everyone, causing real problems and the need to spend real money to fix if you still want to allow unfettered public use.
> It's understood that a company could stop offering some data that a company is dependent on altogether, but I think it's substantially less likely than a company threatening to terminate access to data that already exists (which is what happened here).
I'm not sure sure. There's regular griping on HN about Google cancelling services, even though they are generally beta services, because people expect them to offered. Some of these people griping are people that built products on Google APIs that were discontinued.
> Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
A contract? Maybe not. But if they have a clearly defined deprecation policy, that's a start. And if you are paying them, well that's a lot better, since they are incentivized to keep it going because of your (and hopefully others') money.
As for the rest (access to public API), I'm not really interested in arguing it, as we likely agree more than we disagree. :) The only caveat is that direct consumers of the API often don't behave like regular public users, and that may put a strain on the system and degrade performance for everyone, causing real problems and the need to spend real money to fix if you still want to allow unfettered public use.
>The only caveat is that direct consumers of the API often don't behave like regular public users, and that may put a strain on the system and degrade performance for everyone, causing real problems and the need to spend real money to fix if you still want to allow unfettered public use.
I believe that sites can implement technical measures to "enforce" a fairly normal request profile. Third-party consumers will have to comply with that because it will be technically impossible not to do so. Beyond that, there is no need for a legal method to address access. If the data source is concerned about said consumers going through a code path that is more intense on the backend, they will accommodate with a lighter API. :)
I believe that sites can implement technical measures to "enforce" a fairly normal request profile. Third-party consumers will have to comply with that because it will be technically impossible not to do so. Beyond that, there is no need for a legal method to address access. If the data source is concerned about said consumers going through a code path that is more intense on the backend, they will accommodate with a lighter API. :)
> I believe that sites can implement technical measures to "enforce" a fairly normal request profile.
Doesn't that make some projects and uses, where it needs more than what is regularly allowed, impossible? I think the natural response to that would be "well, we'll just charge them (something/more) and give them special access beyond our normal restrictions", and we're back at square one. Enforcing everyone gets the exact same access is essentially communist theory, not mean in an inflammatory way, but in that it really does enforce some level of mediocrity on everyone (to be fair, as insurance some bad actors makeing things untenable for the rest) in comparison to an approach that allows some companies to pay for for extra access to account for the extra burden which may result in new products that people want.
I'm for making publicly available data available to use as wanted, but I'm not really for enforcing arbitrary limits that prevent whole classes of useful products that people generally might want in the name of being fair, especially when "being fair" might mean contradictory things depending on the side you approach from. The fact that the data is generated by the users, and ownership may or may not reside with users or the aggregator, and current law may not be the best with regard to that just makes it all the more complex. :/
Doesn't that make some projects and uses, where it needs more than what is regularly allowed, impossible? I think the natural response to that would be "well, we'll just charge them (something/more) and give them special access beyond our normal restrictions", and we're back at square one. Enforcing everyone gets the exact same access is essentially communist theory, not mean in an inflammatory way, but in that it really does enforce some level of mediocrity on everyone (to be fair, as insurance some bad actors makeing things untenable for the rest) in comparison to an approach that allows some companies to pay for for extra access to account for the extra burden which may result in new products that people want.
I'm for making publicly available data available to use as wanted, but I'm not really for enforcing arbitrary limits that prevent whole classes of useful products that people generally might want in the name of being fair, especially when "being fair" might mean contradictory things depending on the side you approach from. The fact that the data is generated by the users, and ownership may or may not reside with users or the aggregator, and current law may not be the best with regard to that just makes it all the more complex. :/
I think we're having a communication issue here again. I do appreciate your willingness to continue the discussion, because it helps me find the points that are unclear and that need refinement, so thanks for that.
I don't think we should force companies not to have a paid API, that's all well and good. What I want to remove is their ability to stop scrapers from gathering the data from the generic public interface just because they don't like the way the scraper is using the information.
I don't think we should force companies not to have a paid API, that's all well and good. What I want to remove is their ability to stop scrapers from gathering the data from the generic public interface just because they don't like the way the scraper is using the information.
Yeah, we're mostly in agreement on that. I don't believe in locking down data that is provided free of charge to the public in one manner from other methods of access (i.e. eyeballs vs scraping) in any way beyond normal usage rate limiting that applies equally to all.
As it pertains to this specific story, I think Twitter was well within their rights and norms to try to move a special cased user to some more traditional offering of their, or to at least make them justify their usage. I also think it's well within Emojitracker's rights and is acceptable to try to make a public stink if they think they are getting a raw deal. The only thing that I think leaves a bad taste in my mouth is that he did so prior to attempting a good faith negotiation with Twitter.
As it pertains to this specific story, I think Twitter was well within their rights and norms to try to move a special cased user to some more traditional offering of their, or to at least make them justify their usage. I also think it's well within Emojitracker's rights and is acceptable to try to make a public stink if they think they are getting a raw deal. The only thing that I think leaves a bad taste in my mouth is that he did so prior to attempting a good faith negotiation with Twitter.
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so.
I don't see any encouragement or really any path to recourse in the email. It reads as though the decision is final. Email linked from blog post: https://cdn-images-1.medium.com/max/2000/1*MoLgPNLiQwQ8s-Y1D...
I don't see any encouragement or really any path to recourse in the email. It reads as though the decision is final. Email linked from blog post: https://cdn-images-1.medium.com/max/2000/1*MoLgPNLiQwQ8s-Y1D...
Can you point a place in the email that communicates what you just explained in your comment? I just reread it twice and I don't see that anywhere.
My interpretation of the email is essentially: we used to offer this for free, and we're now offering a commercial alternative. You have three options: use the paid option, use the free API that does not work for your use case, or shut down.
If I'm missing something obvious please let me know.
My interpretation of the email is essentially: we used to offer this for free, and we're now offering a commercial alternative. You have three options: use the paid option, use the free API that does not work for your use case, or shut down.
If I'm missing something obvious please let me know.
My experience with Twitter:
1. Build a successful product
2. Twitter discontinues dev-friendly policies, product gets shuttered
3. Later, tweet doubts about Twitter's dev "commitment"
4. Tweet gets removed
5. Read yet another horror story about working with Twitter
6. Twitter guys claims that Twitter is "committed"
7. Turns out, previously mentioned horror story tells a different story
8. Rinse, repeat?
1. Build a successful product
2. Twitter discontinues dev-friendly policies, product gets shuttered
3. Later, tweet doubts about Twitter's dev "commitment"
4. Tweet gets removed
5. Read yet another horror story about working with Twitter
6. Twitter guys claims that Twitter is "committed"
7. Turns out, previously mentioned horror story tells a different story
8. Rinse, repeat?
You're just making this worse. Do you think people are not going to parse this comment line by line to verify what you imply, point by point. The fact you have a job in Developer Relations and you're trying to deflect the story because it makes you look bad, instead of just owning it is not the knee-jerk reaction you should go for.
As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com).
Simply because you're an even bigger company now that means implicitly that you need to change your processes, perhaps you could explain the reasoning of what was going wrong, previously.
As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com).
Simply because you're an even bigger company now that means implicitly that you need to change your processes, perhaps you could explain the reasoning of what was going wrong, previously.
our commitment to enabling developers to build on our platform
I think a lot of developers would question whether or not you guys even have such a commitment at all. Twitter does NOT have a good history in this regard. If anything, you guys need to bend over backwards, sideways, upside down, and any other way you can think of, to treat developers well and woo them. I mean, somebody needs to go full Ballmer and start screaming "Developers, developers, developers!!!". [1]
Seriously, you guys are so behind the curve in terms of developer perception that "good enough" isn't even close to good enough. You should probably be reaching out to developers, offering to fly them to Twitter HQ, and treating them to steak and lobster, etc. And that's just to get back in the good graces of the development community.
Developers do not compete with Twitter, developers make the Twitter ecosystem larger and stronger and better. Twitter needs to embrace that mindset and go all in on becoming a desirable platform to develop for.
As for myself, our app offers limited Twitter integration, but we're damn sure not going to spend a lot of time and energy on Twitter given the history and what-not.
My advice? Kill Gnip, make it trivially easy to access the firehose, commit to adhering to open standards[2][3][4][5] across the board, and put developers front and center. If you want to charge for firehose access, fine, but kill any notion of some complicated application / approval process. Make it as simple as using the API now plus checking a checkbox and plugging in a credit card number, whatever. It's 2016 for crying out loud, there's no reason I should need to fill out a form and have a sales representative call me for something like that.
[1]: https://www.youtube.com/watch?v=Vhh_GeBPOhs
[2]: http://activitystrea.ms/
[3]: http://www.foaf-project.org/
[4]: https://www.w3.org/2008/09/msnws/papers/W3C_FOSN_Position_Pa...
[5]: https://www.w3.org/community/ostatus/
I think a lot of developers would question whether or not you guys even have such a commitment at all. Twitter does NOT have a good history in this regard. If anything, you guys need to bend over backwards, sideways, upside down, and any other way you can think of, to treat developers well and woo them. I mean, somebody needs to go full Ballmer and start screaming "Developers, developers, developers!!!". [1]
Seriously, you guys are so behind the curve in terms of developer perception that "good enough" isn't even close to good enough. You should probably be reaching out to developers, offering to fly them to Twitter HQ, and treating them to steak and lobster, etc. And that's just to get back in the good graces of the development community.
Developers do not compete with Twitter, developers make the Twitter ecosystem larger and stronger and better. Twitter needs to embrace that mindset and go all in on becoming a desirable platform to develop for.
As for myself, our app offers limited Twitter integration, but we're damn sure not going to spend a lot of time and energy on Twitter given the history and what-not.
My advice? Kill Gnip, make it trivially easy to access the firehose, commit to adhering to open standards[2][3][4][5] across the board, and put developers front and center. If you want to charge for firehose access, fine, but kill any notion of some complicated application / approval process. Make it as simple as using the API now plus checking a checkbox and plugging in a credit card number, whatever. It's 2016 for crying out loud, there's no reason I should need to fill out a form and have a sales representative call me for something like that.
[1]: https://www.youtube.com/watch?v=Vhh_GeBPOhs
[2]: http://activitystrea.ms/
[3]: http://www.foaf-project.org/
[4]: https://www.w3.org/2008/09/msnws/papers/W3C_FOSN_Position_Pa...
[5]: https://www.w3.org/community/ostatus/
>It's 2016 for crying out loud, there's no reason I should need to fill out a form and have a sales representative call me for something like that.
Sure there is. The reason is that they want to guess how much money you'll be willing to pay and have your sales rep charge you that.
Sure there is. The reason is that they want to guess how much money you'll be willing to pay and have your sales rep charge you that.
Oh sure, I know why they want to do it that way. But the sad thing is, they could do price differentiation even in a purely self-service model. It's not a perfect analogy, but look at something like AWS - no sales-person there, but customers price themselves into different segments based on what they sign up for and use. I can't help but think Twitter could accomplish their pricing goals without needing a human in the loop on their end.
Is this a joke? That email clearly states that such an access will not be available. Not what you claim here. Am I missing something?
Am I missing something?
Not really. This follows the coloring book pretty well:
Step 1. Inflexable corporate rhetoric and/or positions expressed privately.
Step 2. Other party blogs about a scenario that puts egg on their face.
Step 3. "Nah man, we're flexable! Why didn't you just talk to us first?!"
Not really. This follows the coloring book pretty well:
Step 1. Inflexable corporate rhetoric and/or positions expressed privately.
Step 2. Other party blogs about a scenario that puts egg on their face.
Step 3. "Nah man, we're flexable! Why didn't you just talk to us first?!"
Meanwhile, their stock is tanking. But they seem too busy patting themselves on the back to even notice.
It's 'unfortunate' because it reflects poorly on your company. The developer didn't do anything wrong.
The rest of your explanation is just specious, as pointed out by other people, and also reflects poorly on your company.
The rest of your explanation is just specious, as pointed out by other people, and also reflects poorly on your company.
It's nice to see somebody from Twitter actually posting here on HN and all. But when the reply is, as you say, somewhat specious, it doesn't really do anything to enhance Twitter's credibility. And they are kinda "swimming upstream" in this regard already.
It's just poorly done damage control. I would expect someone in charge of developer relations at Twitter to come up with an actually concrete plan, if they actually cared about the developer community, and not just talk about how "flexible" Twitter is in private or how "disappointing" it is to get this private matter get exposed. The rest of us developers need to see things like this to properly decide technology product direction, and at least for me, it speaks for how much I will continue to push for removing reliance on Twitter as a platform in favor of something more open, transparent, and predictable.
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so.
Really? Where in the email does it ask anything of that sort? The only "unfortunate" part that I see is that this person exposed the false rhetoric and lack of flexibility that Twitter has shown time and time again. Apologize and fix it, stop trying to whitewash this.
Really? Where in the email does it ask anything of that sort? The only "unfortunate" part that I see is that this person exposed the false rhetoric and lack of flexibility that Twitter has shown time and time again. Apologize and fix it, stop trying to whitewash this.
Simply put - your "template mail" is terrible.
If what you just said is what it the email meant to communicate then it fails at every level. Please don't try to shift the blame for Twitters poor communication to the recipient because they "elected to blog publicly about the situation".
The "situation" is entirely created by Twitter.
Twitter has a (recent) history of terrible communication with developers and it would not surprise anyone if what seemed to be the case WAS the case. If it is NOT the case, as you say, then you really need to improve your company communication.
If what you just said is what it the email meant to communicate then it fails at every level. Please don't try to shift the blame for Twitters poor communication to the recipient because they "elected to blog publicly about the situation".
The "situation" is entirely created by Twitter.
Twitter has a (recent) history of terrible communication with developers and it would not surprise anyone if what seemed to be the case WAS the case. If it is NOT the case, as you say, then you really need to improve your company communication.
It's clear from both the clear contradictions between your statements here and the letter posted _and_ from your lack of any followup here to people that have pointed out these contradictions that you are, unfortunately, stuck with the sucky part of being developer relations for a company that often has to directly piss off developers. I feel for you!
None of this is related to continuing to use the Twitter API or our commitment to enabling developers to build on our platform
It is directly related.
Twitter desperately needs to decide whether you are a platform or a closed business. If you're a platform your job should be encouraging any and all innovation on there, and have clear pricing tiers which do not result in sudden jumps from free to large charges or access cut off and directly reflect the cost to you of serving the API. You should never be in a situation where you're emailing to cut off access (and yes this notice is shutting it down).
Successful products built on twitter should be celebrated, not forced off the platform.
It is directly related.
Twitter desperately needs to decide whether you are a platform or a closed business. If you're a platform your job should be encouraging any and all innovation on there, and have clear pricing tiers which do not result in sudden jumps from free to large charges or access cut off and directly reflect the cost to you of serving the API. You should never be in a situation where you're emailing to cut off access (and yes this notice is shutting it down).
Successful products built on twitter should be celebrated, not forced off the platform.
But what about this: https://twitter.com/jack/status/693296738145140736
So I take it you are some sort of pr flack who had nothing to do with sending the email? Why doesn't the person who sent this email come here and explain why they sent it?
Have you READ the email you sent?
Are you serious? He provided the email Twitter sent. There was no indication of a negotiation or clarification there. Maybe you need to work on how you communicate.
Here is a better approach for those N/A apps.
Try the app. Talk to the developer.
Template mail should not exist.
Try the app. Talk to the developer.
Template mail should not exist.
Regarding your nickname, I would like to point out that there is no such thing as an "ass puppy". Juvenile asses are called foals, like the young of other equines.
[deleted]
Having tried to get firehose access for a project (three years ago), my messages were only met with silence. So I'm pretty sure it wasn't as easy you say it was.
So, were you going to engage in any conversation here or is this just a drive-by attempt at redefining the contents of that email?
jerwiu09(1)
(Not sure why I am receiving so many downvotes, this is the first time I can talk personally with someone at Twitter about API rates. Also I am receiving downvotes in other threads so I suspect this is an orde)
Hi Prashant,
I want to pay $ 100 per month to extend the rate limit of the free API. Is this possible?
Thanks,
wslh
Hi Prashant,
I want to pay $ 100 per month to extend the rate limit of the free API. Is this possible?
Thanks,
wslh
So, Twitter is allowing their head of Developer Relations to communicate, in what appears to be an official response, under the name `CoolAssPuppy`? Really? That doesn't seem right.
Welcome to the internet? If anything, this feels more like a real response than a canned message from OfficialTwitterAccount.
Considering that was probably his Twitter handle before joining Twitter, this isn't shocking: https://twitter.com/CoolAssPuppy
That's been my handle since AOL. :-)
Good to know my handle won't necessarily hold me back.
I've often wondered why Twitter seems to do this again and again in spite of what appears to be common sense and good stewardship of its platform and ecosystem.
I think it's because Twitter is actually a successful, fairly open ecosystem that attracts all kinds of use.
What seems like ham-handed or draconian decision making is just Twitter misunderstanding its role as infrastructure provider.
What we all implicitly want from Twitter is for it to embody the values of net neutrality. If someone can build something cool on top of Twitter, why not let it happen?
Twitter's investors have pressured the company to monetize, and most of the bad platform decisions seem to be tied to those efforts.
Twitter's folly is assuming that usage patterns are "bad". It should view all usage in a content-neutral manner and provide and support APIs to let the ecosystem grow.
It is the hubris of caring too much about content and content engagement metrics that is driving this folly.
Twitter is a very simple broadcast graph. This is incredibly powerful, and while it's not quite as lucrative to own as Facebook (with users habituated to seeing the same boring stuff from the same boring people every day and clicking on sponsored content), it's far more powerful and is likely to change society more than other social media platforms.
All this is why Twitter should pivot and become a decentralized infrastructure... of course this will never happen. I'm predicting that Twitter will die a slow death as it tries to copy Facebook's revenue generation approach. It is also copying Facebook's real name initiative by caring too much about content attribution and verified accounts (which users don't care about).
Eventually we may see a decentralized broadcast graph, but until then it makes sense to expect Twitter to behave like a cognitively impaired dictator with respect to its ecosystem.
I think most of this is driven by intense cost-cutting measures and intensely misguided focus on revenue metrics in terms of content and engagement. Twitter is not a product, it's an infrastructure.
I think it's because Twitter is actually a successful, fairly open ecosystem that attracts all kinds of use.
What seems like ham-handed or draconian decision making is just Twitter misunderstanding its role as infrastructure provider.
What we all implicitly want from Twitter is for it to embody the values of net neutrality. If someone can build something cool on top of Twitter, why not let it happen?
Twitter's investors have pressured the company to monetize, and most of the bad platform decisions seem to be tied to those efforts.
Twitter's folly is assuming that usage patterns are "bad". It should view all usage in a content-neutral manner and provide and support APIs to let the ecosystem grow.
It is the hubris of caring too much about content and content engagement metrics that is driving this folly.
Twitter is a very simple broadcast graph. This is incredibly powerful, and while it's not quite as lucrative to own as Facebook (with users habituated to seeing the same boring stuff from the same boring people every day and clicking on sponsored content), it's far more powerful and is likely to change society more than other social media platforms.
All this is why Twitter should pivot and become a decentralized infrastructure... of course this will never happen. I'm predicting that Twitter will die a slow death as it tries to copy Facebook's revenue generation approach. It is also copying Facebook's real name initiative by caring too much about content attribution and verified accounts (which users don't care about).
Eventually we may see a decentralized broadcast graph, but until then it makes sense to expect Twitter to behave like a cognitively impaired dictator with respect to its ecosystem.
I think most of this is driven by intense cost-cutting measures and intensely misguided focus on revenue metrics in terms of content and engagement. Twitter is not a product, it's an infrastructure.
It's possible to over think things. Twitter exists to make money. They clearly thought this change would cost them nothing, and would perhaps save them money and effort. I don't think the "Twitter is a ..." thing has any relevance to the decision making process. I can think of very few things, related to the Internet, that haven't turned - or are not on the process of turning to - shit. No matter how cool anything starts out being, eventually the original developers sell it on or come up with stupid ideas and it gets ruined. It's like some inescapable physical property.
> Twitter exists to make money.
True, but for a platform business there has to be a correct recognition of the relevant time horizon. Amazon's profitability numbers in its first decade illustrate this.
True, but for a platform business there has to be a correct recognition of the relevant time horizon. Amazon's profitability numbers in its first decade illustrate this.
Surely the lesson from Amazon is that you can get by for a long time without making loads of money, as long as you don't make stupid decisions, and get rid of the consequences of stuff that didn't work out before it hurts you.
Twitter makes decisions that people on the internet grumble about and make blog posts about and yes, even post on twitter about, but they still keep using twitter, which is all twitter cares about.
Twitter makes decisions that people on the internet grumble about and make blog posts about and yes, even post on twitter about, but they still keep using twitter, which is all twitter cares about.
I think this oversimplifies things. If Twitter was making money we would not see such turmoil in the team or so much floundering about revenue generation tactics.
Yes - Twitter exists to make money.
But they need to decide how they want to make their money and grow in parallel. Is it ads? Celebrities accounts? Is it selling API access?
If I'm Twitter CEO I would stick with "developer friendly" approach in order to re-start Twitter growth.
In short, my opinion is that companies needs to be nice with developers and cherish them all the way until the company is very very successful. I guess Twitter executives think that they made it and "developer friendly" stage is behind them.
But they need to decide how they want to make their money and grow in parallel. Is it ads? Celebrities accounts? Is it selling API access?
If I'm Twitter CEO I would stick with "developer friendly" approach in order to re-start Twitter growth.
In short, my opinion is that companies needs to be nice with developers and cherish them all the way until the company is very very successful. I guess Twitter executives think that they made it and "developer friendly" stage is behind them.
I would say that even if the developer-friendly stage was behind them (and given how rapidly the web evolves, I don't think that any social-media company should ever consider itself past that stage), the fair thing to do - in my eyes - is to at the very least show some gratitude for the people who helped you get where you are today.
Twitter and Facebook may be Big Corp today, but that wasn't always the case, and a lot of the initial growth that helped them get into the mainstream was due to hobbyists and independent devs that made cool little apps like EmojiTracker that would catch the attention of early adopters that then became the initial users of the platform.
The sad thing is that while Facebook became the generic one-size-fits-all "mom and dad" network (I can't even remember the last time I even used one of their 'apps' or whatever), Twitter really had the potential to evolve into something beyond that, if only it cultivated better relationships with their devs and encouraged people to build on and experiment with the platform.
Oh well. Lets keep an eye out for the next thing, I suppose!
Twitter and Facebook may be Big Corp today, but that wasn't always the case, and a lot of the initial growth that helped them get into the mainstream was due to hobbyists and independent devs that made cool little apps like EmojiTracker that would catch the attention of early adopters that then became the initial users of the platform.
The sad thing is that while Facebook became the generic one-size-fits-all "mom and dad" network (I can't even remember the last time I even used one of their 'apps' or whatever), Twitter really had the potential to evolve into something beyond that, if only it cultivated better relationships with their devs and encouraged people to build on and experiment with the platform.
Oh well. Lets keep an eye out for the next thing, I suppose!
If twitter exists to make money, then why aren't they operating a franchise network of fast-food restaurants, since that more obviously makes money than an sms-length syndication service. Also, shouldn't they be selling music streaming? Why aren't they in most markets at all? Do most markets have a marginal revenue of zero or negative?
Because Twitter doesn't "exist to make money" it also "exists to be Twitter." I'd prefer to never again hear that something "exists to make money", which is quite facile.
EDIT: there's a bit of a war between downvoters and upvoters on this comment. Let me break it down for you. The only way to agree that "twitter exists to make money" is by agreeing that the mission statement of every company on Earth should be, "Make money." Not much of a mission statement is it?
Because Twitter doesn't "exist to make money" it also "exists to be Twitter." I'd prefer to never again hear that something "exists to make money", which is quite facile.
EDIT: there's a bit of a war between downvoters and upvoters on this comment. Let me break it down for you. The only way to agree that "twitter exists to make money" is by agreeing that the mission statement of every company on Earth should be, "Make money." Not much of a mission statement is it?
The first mission of every organization is not making money, it is living longer.
Maybe you think that should be the first mission of every organization, but it clearly isn't. After all, if you gave me $2M and I was supposed to use it to set up an organization that survives for as long as possible, it is likely that I could easily set up a legal entity in the United States that has a recognizable successor 300 years later (and no expenses or employees). If administrative costs are as high as $5K per year, with zero interest, that is still 400 years. But lots of organizations have more than $2M as some point, but at a later point just a couple of years later go bankrupt and simply disappear. Clearly it is not the first mission of every organization to live longer.
> It is also copying Facebook's real name initiative by caring too much about content attribution and verified accounts (which users don't care about).
I fully agree with you, except on this point. With all the shitstorming, the mis-attribution of quotes, massive and massive amounts of fake accounts etc. going on, the Verified by Twitter program is a very good thing especially as it does not force you to publically post your real name.
However, it could be expanded for greater reach.
I fully agree with you, except on this point. With all the shitstorming, the mis-attribution of quotes, massive and massive amounts of fake accounts etc. going on, the Verified by Twitter program is a very good thing especially as it does not force you to publically post your real name.
However, it could be expanded for greater reach.
I totally see your point. My counter-argument would be:
If Twitter is a neutral platform, a fake account is not distinguishable from a real one and it doesn't matter. It takes an incredible amount of work to make a fake account seem like a real person.
By creating content leaderboards that do not remove fraudulent network effects, Twitter has created an incentive for fake account creation. Its own ad-sales model benefits from this too, since many of the platform's top line numbers include fraudulent engagement data.
Also, Twitter has not made data/tooling available for third party providers (like SimplyMeasured, Radian6, etc.) to do meaningful fraud analysis, so companies are paying for social media engagement numbers that include bots, sock puppets, etc.
But my larger point is that as a network Twitter should not care about spam and should instead let others determine what content is "good" vs "bad" for their particular use case.
If Twitter is a neutral platform, a fake account is not distinguishable from a real one and it doesn't matter. It takes an incredible amount of work to make a fake account seem like a real person.
By creating content leaderboards that do not remove fraudulent network effects, Twitter has created an incentive for fake account creation. Its own ad-sales model benefits from this too, since many of the platform's top line numbers include fraudulent engagement data.
Also, Twitter has not made data/tooling available for third party providers (like SimplyMeasured, Radian6, etc.) to do meaningful fraud analysis, so companies are paying for social media engagement numbers that include bots, sock puppets, etc.
But my larger point is that as a network Twitter should not care about spam and should instead let others determine what content is "good" vs "bad" for their particular use case.
> If Twitter is a neutral platform, a fake account is not distinguishable from a real one and it doesn't matter. It takes an incredible amount of work to make a fake account seem like a real person.
I think these sentences contradict each other? Or maybe I'm misreading something.
Anyway, last December, after the release of The Force Awakens, someone created the account @AdamDriver2015 which was so convincing that even @VeryLonelyLuke quote-tweeted them. They were suspended after a few days, and one Twitter user claims it's because the owner of that account was creeping on other Twitter users under the guise of being the actor.
https://twitter.com/jennanamika/status/684176364438601728
It's very easy to make a fake account seem like a real person.
> By creating content leaderboards that do not remove fraudulent network effects
Do you count "retweets" here? That's a large part of the problem, and it seems hard to change that without fundamentally changing what Twitter is.
> third party providers (like SimplyMeasured, Radian6, etc.) to do meaningful fraud analysis
... hm, are you focusing on the aggregate affect of fraudulent accounts? It seems to me like that's a different problem from individual fraudulent accounts, and that verified accounts are legitimately good at the latter and aren't intended to deal with the former. In particular, all the bot checks are irrelevant for an individual fraudulent account, since there is a real human behind the account, just the wrong human.
I think these sentences contradict each other? Or maybe I'm misreading something.
Anyway, last December, after the release of The Force Awakens, someone created the account @AdamDriver2015 which was so convincing that even @VeryLonelyLuke quote-tweeted them. They were suspended after a few days, and one Twitter user claims it's because the owner of that account was creeping on other Twitter users under the guise of being the actor.
https://twitter.com/jennanamika/status/684176364438601728
It's very easy to make a fake account seem like a real person.
> By creating content leaderboards that do not remove fraudulent network effects
Do you count "retweets" here? That's a large part of the problem, and it seems hard to change that without fundamentally changing what Twitter is.
> third party providers (like SimplyMeasured, Radian6, etc.) to do meaningful fraud analysis
... hm, are you focusing on the aggregate affect of fraudulent accounts? It seems to me like that's a different problem from individual fraudulent accounts, and that verified accounts are legitimately good at the latter and aren't intended to deal with the former. In particular, all the bot checks are irrelevant for an individual fraudulent account, since there is a real human behind the account, just the wrong human.
> I think these sentences contradict each other?
The first statement is conditional on Twitter behaving like a neutral platform, which it does not. The second is meant to indicate that there is a cost associated with creating fake accounts, which is a factor in the economics of the system.
> It's very easy to make a fake account seem like a real person.
Why is this a bad thing? If casual Twitter users find the content entertaining, the fake account benefits the ecosystem.
> Do you count "retweets" here?
Yes. No need to ban retweets, just to show stats that indicate which are likely from retweet networks.
> ... hm, are you focusing on the aggregate affect of fraudulent accounts?
Yes because that is what drives the money flows. People create fraudulent accounts of both kinds to generate revenue. Someone somewhere is paying for the campaign that included thousands of fake retweets, etc. I saw a network graph recently that showed that such networks are easily identifiable with simple network analysis.
The first statement is conditional on Twitter behaving like a neutral platform, which it does not. The second is meant to indicate that there is a cost associated with creating fake accounts, which is a factor in the economics of the system.
> It's very easy to make a fake account seem like a real person.
Why is this a bad thing? If casual Twitter users find the content entertaining, the fake account benefits the ecosystem.
> Do you count "retweets" here?
Yes. No need to ban retweets, just to show stats that indicate which are likely from retweet networks.
> ... hm, are you focusing on the aggregate affect of fraudulent accounts?
Yes because that is what drives the money flows. People create fraudulent accounts of both kinds to generate revenue. Someone somewhere is paying for the campaign that included thousands of fake retweets, etc. I saw a network graph recently that showed that such networks are easily identifiable with simple network analysis.
I have worked closely with the management teams of some large celebrities. Almost all of them get anywhere from a handful to dozens of fake accounts created EVERY DAY, mostly by trolls and people who want to harrass others. Some of them are very convincing and "leak" false information, bully young fans who believe they are real, and so on. A small handful of them try to monetize by posting spam links, but it's really rare. The vast majority are just trying to get attention or have fun at someone else's expense. This may seem like a non-issue, but when you're a high-profile person trying to protect a reputation, it's a real headache. And when you're a fan thinking you're interacting with a celebrity, and they turn out to be a fake, it greatly reduces your trust in the platform.
Now, we could argue that we shouldn't really care about the concerns of celebrities, and that fan/celebrity interactions aren't what Twitter is for. But it's clear that they drive a lot of the traffic there, and Twitter encourages it. The verification system is one tool for making this experience better.
Now, we could argue that we shouldn't really care about the concerns of celebrities, and that fan/celebrity interactions aren't what Twitter is for. But it's clear that they drive a lot of the traffic there, and Twitter encourages it. The verification system is one tool for making this experience better.
I don't think @AdamDriver2015 was created to drive money flows.
Honestly, I wouldn't be surprised if they had zero fake retweets. Just have a plausible-sounding name (done) and no competing account that's actually by the person in question (done), and reply to some relevant tweets from high-profile accounts, so you show up in the single-tweet view. At that point, some fraction of real humans will find out about the account and retweet you. No analysis will differentiate this from real humans, because everyone involved is a real human.
It seems difficult to argue that @AdamDriver2015 benefitted the ecosystem. Those who found the content entertaining did so because they thought it was genuine. Especially if it's true that the person behind the account was creeping on people in DMs, this seems like a very indefensible claim—but even if not, it's not advantageous to the ecosystem, unless Twitter is actively participating in making people believe the fake content is genuine.
Honestly, I wouldn't be surprised if they had zero fake retweets. Just have a plausible-sounding name (done) and no competing account that's actually by the person in question (done), and reply to some relevant tweets from high-profile accounts, so you show up in the single-tweet view. At that point, some fraction of real humans will find out about the account and retweet you. No analysis will differentiate this from real humans, because everyone involved is a real human.
It seems difficult to argue that @AdamDriver2015 benefitted the ecosystem. Those who found the content entertaining did so because they thought it was genuine. Especially if it's true that the person behind the account was creeping on people in DMs, this seems like a very indefensible claim—but even if not, it's not advantageous to the ecosystem, unless Twitter is actively participating in making people believe the fake content is genuine.
>the Verified by Twitter program is a very good thing
When the program is used to delegitimize public figures who dissent from the left wing hegemony on the site[0], then it has lost all legitimacy. Verifying accounts now implies tacit approval of their message, which is not something Twitter should enter into lightly.
[0]http://www.businessinsider.com/milo-yiannopoulos-nero-unveri...
When the program is used to delegitimize public figures who dissent from the left wing hegemony on the site[0], then it has lost all legitimacy. Verifying accounts now implies tacit approval of their message, which is not something Twitter should enter into lightly.
[0]http://www.businessinsider.com/milo-yiannopoulos-nero-unveri...
Well, it's role as an infrastructure provider is to not go bankrupt and shut down... in order to do this, it needs to make money somehow... they've chosen a two pronged approach to this. Someone/somewhere has to pay for it, and the end users simply won't do it...
1. Promoted tweeks - frankly, I find this kind of annoying, but I do understand.
2. Limiting high bandwidth streams to paid/commercial accounts, I'm not sure what they offer for NPOs, but I'd be surprised if they weren't lenient in their pricing structure.
Frankly, I understand it... At this point they aren't even breaking even iirc, so they need to make cuts to some deep cost centers that aren't facing end users.
TBH, I think they should also put methods in place of detecting bot networks, and freezing out those accounts.
1. Promoted tweeks - frankly, I find this kind of annoying, but I do understand.
2. Limiting high bandwidth streams to paid/commercial accounts, I'm not sure what they offer for NPOs, but I'd be surprised if they weren't lenient in their pricing structure.
Frankly, I understand it... At this point they aren't even breaking even iirc, so they need to make cuts to some deep cost centers that aren't facing end users.
TBH, I think they should also put methods in place of detecting bot networks, and freezing out those accounts.
> Well, it's role as an infrastructure provider is to not go bankrupt and shut down
I realize that, I'm just not sure the investors have the right long-term view about the value Twitter offers.
I realize that, I'm just not sure the investors have the right long-term view about the value Twitter offers.
If the value it offers to others can't be recouped then its meaningless in terms of investment.
Twitter, as it stands, is very useful to people in the media who use it as a free sentiment analysis and polling service during election cycles, and a free firehouse for unqualified reports that they can follow up on. However, Twitter doesn't receive remuneration on any of this value.
Twitter, as it stands, is very useful to people in the media who use it as a free sentiment analysis and polling service during election cycles, and a free firehouse for unqualified reports that they can follow up on. However, Twitter doesn't receive remuneration on any of this value.
That's a really good analysis of what I was hoping Twitter could have been, and what it has actually became.
I am quite curious to poke your brain a little on this. I agree that the concept of Twitter-as-an-infrastructure could be amazing. I also agree that what they're doing is definitely taking massive steps away from that ideal (Whatever Twitter's initial lofty goals were, I would classify its current niche to be some sort of celebrity-gossip/outrage machine).
I'm having trouble articulating exactly what the correct thing for them to do would be, though? What sort of API access would you envision that they can provide in order to help developers grow the infrastructure in the best way possible?
You mention that Twitter could become a decentralized infrastructure. What exactly do you mean by that, and how would it work?
I am quite curious to poke your brain a little on this. I agree that the concept of Twitter-as-an-infrastructure could be amazing. I also agree that what they're doing is definitely taking massive steps away from that ideal (Whatever Twitter's initial lofty goals were, I would classify its current niche to be some sort of celebrity-gossip/outrage machine).
I'm having trouble articulating exactly what the correct thing for them to do would be, though? What sort of API access would you envision that they can provide in order to help developers grow the infrastructure in the best way possible?
You mention that Twitter could become a decentralized infrastructure. What exactly do you mean by that, and how would it work?
>You mention that Twitter could become a decentralized infrastructure. What exactly do you mean by that, and how would it work?
GNU Social [0] is a good stepping stone. Decentralized Twitter with different hosted instances joined together so everyone on the network can communicate. You're free to take your data and move it to any other instance you want or even host your own.
[0] https://news.ycombinator.com/item?id=11062757
GNU Social [0] is a good stepping stone. Decentralized Twitter with different hosted instances joined together so everyone on the network can communicate. You're free to take your data and move it to any other instance you want or even host your own.
[0] https://news.ycombinator.com/item?id=11062757
Tbf, that's what Diaspora tried and it never took off.
While I agree with the spirit of the idea, most people don't seem to care about the benefits that self-hosting would provide, and even for the ones that do, well, it's just such a hassle. And for the majority of the world that isn't particularly clued up on how tech works? Yeah forget about it.
Maybe that's where the web will go eventually, and I would see it as an improvement over what we have now. But the current reality is that mainstream adoption is centred around large, monolithic services and personally I am more interested in figuring out what is it that these services provide and how I can improve on that in the way that people already want, rather than trying to convince them why they should care about being in control of their own data.
So when I think about "decentralized infrastructure", I don't think about self-hosted servers, what I'm imagining is closer to, say, google app store. Sure, it's still owned by a single corporation, but anyone can develop almost anything for it that then can be used by anyone else.
Imagine what a hybrid between twitter and an app store could look like. Giving people new tools to integrate social apps around the web in totally new ways. Developing new ways of interacting with each other, rather than every news and blog site having the same facebook like button and the same "twitter feed" box and the same comment section.
That's what I want to see. Not "it's like twitter but YOU control your data". Having ownership of your own data is important to have alongside any product that you develop, but (as far as we can see empirically) you can't make that your sole "killer feature" and expect mass adoption.
While I agree with the spirit of the idea, most people don't seem to care about the benefits that self-hosting would provide, and even for the ones that do, well, it's just such a hassle. And for the majority of the world that isn't particularly clued up on how tech works? Yeah forget about it.
Maybe that's where the web will go eventually, and I would see it as an improvement over what we have now. But the current reality is that mainstream adoption is centred around large, monolithic services and personally I am more interested in figuring out what is it that these services provide and how I can improve on that in the way that people already want, rather than trying to convince them why they should care about being in control of their own data.
So when I think about "decentralized infrastructure", I don't think about self-hosted servers, what I'm imagining is closer to, say, google app store. Sure, it's still owned by a single corporation, but anyone can develop almost anything for it that then can be used by anyone else.
Imagine what a hybrid between twitter and an app store could look like. Giving people new tools to integrate social apps around the web in totally new ways. Developing new ways of interacting with each other, rather than every news and blog site having the same facebook like button and the same "twitter feed" box and the same comment section.
That's what I want to see. Not "it's like twitter but YOU control your data". Having ownership of your own data is important to have alongside any product that you develop, but (as far as we can see empirically) you can't make that your sole "killer feature" and expect mass adoption.
Twitter should be a dumb pipe.
Something like twtxt[1], except trivially easy to host, and searchable, and (the hard part) with all of Twitter's users.
[1] https://github.com/buckket/twtxt
Something like twtxt[1], except trivially easy to host, and searchable, and (the hard part) with all of Twitter's users.
[1] https://github.com/buckket/twtxt
I don't really see the appeal, if I'm honest.
Infrastructure is a bad word to Wall Street, it implies it's no longer a growth stock. Wall Street will burn TWTR to the ground if it doesn't keep growing because no one on Wall Street cares about value, they want consistent quarter-over-quarter growth. This makes public companies like Twitter flail about wildly trying to find anything to keep their stock price from sinking even if it's outside their wheelhouse and wildly distorts their services/products.
This is just... wow. The guy spends three years developing a product on Twitter's platform and they just kill it off with one e-mail? It's fair enough if they need to change their API policy, but at the very least they could have acknowledged all the work that he's done and found a way to help him and the other developers in the same position to make the transition as painlessly as possible. Not... whatever corporate BS this was.
Given how much Twitter is struggling already, is this really the time to be alienating more of its community?
Personally, I've had a really nifty side project that I've been working on since last year that I wanted to launch on Twitter's platform, but as of a month ago I've all but given up on actively developing it. All the people that I know on Twitter with whom I wanted to share it have either left already, or are only sticking around because as-of-yet there's nowhere else to go that does what Twitter does.
So I'm curious to hear some perspectives on this: clearly the direction that Twitter has taken is making a lot of devs unhappy, but what would be the right thing to do? What sort of API access would people ideally want to see it provide? What would be the ideal best-case scenario necessary for the devs to want to start coming back and putting their time into building the ecosystem?
Given how much Twitter is struggling already, is this really the time to be alienating more of its community?
Personally, I've had a really nifty side project that I've been working on since last year that I wanted to launch on Twitter's platform, but as of a month ago I've all but given up on actively developing it. All the people that I know on Twitter with whom I wanted to share it have either left already, or are only sticking around because as-of-yet there's nowhere else to go that does what Twitter does.
So I'm curious to hear some perspectives on this: clearly the direction that Twitter has taken is making a lot of devs unhappy, but what would be the right thing to do? What sort of API access would people ideally want to see it provide? What would be the ideal best-case scenario necessary for the devs to want to start coming back and putting their time into building the ecosystem?
Another example of Twitter taking steps to kill their platform dreams and push developers away. So much for Jack's renewed push for improved developer relations.
This is a situation where one developer had a special exception carved out for them for a non-commercial project that everyone thought was cool. That access persisted for three years until someone made a decision that giving free access to something that normally they charge quite a lot of money for is not worth it.
Twitter offers several free endpoints for developers, including search, streaming, a raw sample of tweets, and the like. They all have rate limits and the like attached, but they're real, free, access to their data.
For people who want bigger, broader, deeper access, they sell access to that data. That's been the case for the last few years.
In this case, Emojitracker unfortunately falls into the cracks in between, where they need deeper access to do what they want, but it's not a commercial enterprise.
I agree, it would be nice of Twitter to continue giving Emojitracker free access, but are they obligated to do that? They are a business, and providing access to millions of Tweets for free seems like an unfair obligation to ask them to continue indefinitely.
Twitter offers several free endpoints for developers, including search, streaming, a raw sample of tweets, and the like. They all have rate limits and the like attached, but they're real, free, access to their data.
For people who want bigger, broader, deeper access, they sell access to that data. That's been the case for the last few years.
In this case, Emojitracker unfortunately falls into the cracks in between, where they need deeper access to do what they want, but it's not a commercial enterprise.
I agree, it would be nice of Twitter to continue giving Emojitracker free access, but are they obligated to do that? They are a business, and providing access to millions of Tweets for free seems like an unfair obligation to ask them to continue indefinitely.
>That access persisted for three years until someone made a decision that giving free access to something that normally they charge quite a lot of money for is not worth it.
Right. The point here is that the decision might be stupid. It's fodder for exactly this type of a story, while Twitter is pushing for improved relations with developers and trying determine its future as a going concern.
Of course they aren't obligated to do continue to provide free access, in the same sense that I'm not obligated to use their platform. But it might be in their best interest (in both cases).
For all the acclaim that Mr. Dorsey gets as a CEO, Twitter is a great idea but a basket case of a public company.
Right. The point here is that the decision might be stupid. It's fodder for exactly this type of a story, while Twitter is pushing for improved relations with developers and trying determine its future as a going concern.
Of course they aren't obligated to do continue to provide free access, in the same sense that I'm not obligated to use their platform. But it might be in their best interest (in both cases).
For all the acclaim that Mr. Dorsey gets as a CEO, Twitter is a great idea but a basket case of a public company.
The comment you are replying did not say or imply Twitter was obligated to do anything.
But continuing to provide this noncommercial API access for free indefinitely is exactly what I think they should do and I'm a little disappointed that they don't agree. Twitter isn't wrong here, they're just assholes.
But continuing to provide this noncommercial API access for free indefinitely is exactly what I think they should do and I'm a little disappointed that they don't agree. Twitter isn't wrong here, they're just assholes.
They do still have noncommercial API access for free (who knows for how long). What is going away is this individual's special access.
Right, I think the individual should continue to get special access forever. It's a terrible idea to cut off people doing something cool with your platform because of what? Saving the negligible cost of supporting their volume of API calls? A message to developers that rules are strictly enforced? Not exactly a PR win.
>Right, I think the individual should continue to get special access forever. It's a terrible idea to cut off people doing something cool with your platform because of what?
Because "special access" means free in this case.
Do you realize you are arguing that a customer should have free access forever, while others customers pay? Do you know what type of liability that is for any business?
Because "special access" means free in this case.
Do you realize you are arguing that a customer should have free access forever, while others customers pay? Do you know what type of liability that is for any business?
There's nothing unusual about charging commercial users of a product while not charging non-commercial users. Academic and non-profit discounts, down to making your product free to them, are extremely commonplace. There's nothing unusual about sponsoring interesting projects, or making special arrangements when it's mutually beneficial. Giving a tiny customer free access forever, and getting a bunch of goodwill associated with their brand in return, is not a liability, it's good business.
There is something unusual with a contract for free access to a paid service that goes to perpetuity, no matter how many times you try to redefine it.
If it was the case that emojitracker was reselling that api access, I could see cutting it off.
But I think that a rich, robust ecosystem would flourish more around a platform that provided free API access to noncommercial projects.
Of course, it's conceivable that this API access costs twitter a significant amount of money - it is a lot of data, after all. Even a low cost could be written off as advertising within the platform. "Look at the cool stuff we could build on this API, let's make a commercial product and pay twitter for access!"
But I think that a rich, robust ecosystem would flourish more around a platform that provided free API access to noncommercial projects.
Of course, it's conceivable that this API access costs twitter a significant amount of money - it is a lot of data, after all. Even a low cost could be written off as advertising within the platform. "Look at the cool stuff we could build on this API, let's make a commercial product and pay twitter for access!"
> But I think that a rich, robust ecosystem would flourish more around a platform that provided free API access to noncommercial projects.
This. Also, if API access costs Twitter a significant amount of money, they shouldn't be in the platform business and shut down their API entirely.
This. Also, if API access costs Twitter a significant amount of money, they shouldn't be in the platform business and shut down their API entirely.
> They are a business, and providing access to millions of Tweets for free seems like an unfair obligation to ask them to continue indefinitely.
While not indefinitely, it would certainly help the case to just public a cost-benefit analysis of how much Emojitracker's access was costing Twitter (though not in terms of "lost revenue" based on what Emojitracker would have been paying for that access).
While not indefinitely, it would certainly help the case to just public a cost-benefit analysis of how much Emojitracker's access was costing Twitter (though not in terms of "lost revenue" based on what Emojitracker would have been paying for that access).
Who says they didn't perform the kind of cost/benefit analysis you mention and estimated that it wasn't worth the effort? And if that was their estimation, despite our (NH denizens) collective annoyance, maybe they were right. Maybe providing free access made prospective paying customers less likely to pay for firehose access, or a slew of not-for-profit organisations requesting free firehose access citing emojitracker as a precedent. There's a myriad of hypotheticals and whomever made this decision is not necessarily irrational... it just depends on what constitutes their utility function.
I was saying that maybe it would have been a good idea for them to publish that analysis publicly, or just to emojitracker (where they would presumably publish it for Twitter).
I don't see that as Twitter's obligation. When someone does you a favor, or makes an exception for you, it's nice if they give you a reason when they end it, but it's not their responsibility.
No, it's not their responsibility. I'm just saying that PR-wise it could have been a good move to just say, "Hey, it's costing us $X/month to give you an exception for this access. Sorry, but we can't justify that cost going forward." Are you saying that it would have been a bad move for them to say that?
> For people who want bigger, broader, deeper access, they sell access to that data. That's been the case for the last few years.
Can I pay $ 100 extra per month for more access?
Can I pay $ 100 extra per month for more access?
It's very expensive. You have to be careful to only register high signal / low noise terms or risk a massive bill.
To get the data emojitracker has would be tens of thousands a month or more.
To get the data emojitracker has would be tens of thousands a month or more.
Most of it is based on volume - I have no idea what the entry point is for some of these things, but there are increments of data or access that are around that much.
But no, I doubt you can pony up $100/mo and get a meaningful amount of data.
But no, I doubt you can pony up $100/mo and get a meaningful amount of data.
That's the issue with Twitter and this is their fault. Google was revolutionary when they introduced a way to advertise (yes, AdWords) for only $ 5 while others (e.g: Yahoo) required thousand dollars as an entry point. I know it is not easy to develop/set up such system and deal with millions of retail customers but... in 2016? You can even do this with Twitter Advertising so I don't understand why they can't solve this for their API.
It feels to me like "kill platform dreams" is written on the other side of the coin that says "sustainable business model".
It is a serious challenge that the data has some value, but that value is hard to extract usefully as a revenue stream. The emojitracker folks won't pay much, but offering it to them at a small rate vs a high rate to some other service has its own issues.
But since emojii tracker isn't real time, could they use a month old firehose? 6 months old? At some point the value of the firehose access falls off for "mainstream" users of that data, does it fall to a point where there is marginal value in giving free access to others on it? Even for small dollars?
Clearly as a public company Twitter has to do what it can to get good value for the assets it has created. That is a hard thing to do.
It is a serious challenge that the data has some value, but that value is hard to extract usefully as a revenue stream. The emojitracker folks won't pay much, but offering it to them at a small rate vs a high rate to some other service has its own issues.
But since emojii tracker isn't real time, could they use a month old firehose? 6 months old? At some point the value of the firehose access falls off for "mainstream" users of that data, does it fall to a point where there is marginal value in giving free access to others on it? Even for small dollars?
Clearly as a public company Twitter has to do what it can to get good value for the assets it has created. That is a hard thing to do.
At the risk of sounding pedantic: Twitter's assets were created by its users, not the company. They alienate the users, there are no more assets and no more value to be had.
I think we see it a bit differently. When the grocery store chain sells information about the buying habits of its customers, it is the customers doing the buying, but it was the collection and monitoring systems that "created" the information.
Twitter has "users" but those users are not the "customers". Twitter created a watering hole so that it could interest nearby predators in access to where the herbivores would be on hot days.
To be counter-pedantic (is that a word?) If there were anywhere else for Emojitracker to get their data, then I would agree with you that Twitter did not create it.
Twitter has "users" but those users are not the "customers". Twitter created a watering hole so that it could interest nearby predators in access to where the herbivores would be on hot days.
To be counter-pedantic (is that a word?) If there were anywhere else for Emojitracker to get their data, then I would agree with you that Twitter did not create it.
I think that watering hole analogy is the the best explanation of the online ad industry that I've heard in quite a while.
The problem I see with Twitter and the reason it stagnated and isn't making money, is they don't really understand their platform and users.
Flip that with Snapchat who 100% do know their users and converting them into customers.
Flip that with Snapchat who 100% do know their users and converting them into customers.
What does converting users into customers mean for snapchat? What are they selling?
I'm pretty confused by that as well. Isn't Snapchat making their money off advertising primarily? I know they have plans to expand the kind of user data they track (including "browsing and searches beyond the app").
http://adexchanger.com/data-driven-thinking/is-snapchat-on-t...
http://adexchanger.com/data-driven-thinking/is-snapchat-on-t...
filters i suppose
Hmm, do YOU understand Twitter's platform and users? If you have some cool insights into those things then I'd love to hear them!
Well this is on the front-page now. maybe he will scream at the team and say "don't shut it down."
You mean like they shut down the Twitter share counter data right after Jack had a big conference once he came back as CEO promising better developer relations? So, I doubt they care about the state of developer relations, as a platform.
I don't think Jack gets his news from HN.
He may or may not, but I wouldn't doubt their PR team already pinned HN as a source to consume. Plus there is https://twitter.com/newsycombinator. #twitter, definitely.
I'm "confused," what in the world are we supposed to make of this tweet https://twitter.com/jack/status/693296738145140736 and this statement? http://recode.net/2015/10/21/twitter-ceo-jack-dorsey-apologi...
It means Twitter as a platform company doesn't care about keeping successful applications on their platform. Words (Tweets?) are cheap, actions speak louder. In this case, it's a resounding and massive door closing on an application that Jack said he loves and probably thought is/was good use of the platform.
The telling quote from that article is this:
> “Developers took to our service from day one and [gave] Twitter a much more global reach."
Twitter would not be where Twitter is today without outside developers building their company for them, through the creation of tools that users actually want to use to interface with Twitter.
Twitter as a standalone product does not have a future. Every change they make that turns more developers away from the platform continues to reduce the number of users globally who are using Twitter in some fashion. It's a downward spiral. Twitter's demise is inevitable if they continue down this path. Developers should assist them by abandoning the platform in droves. There won't be much of a platform left to work with anyway, so why bother? Let Twitter be the next Yahoo.
> “Developers took to our service from day one and [gave] Twitter a much more global reach."
Twitter would not be where Twitter is today without outside developers building their company for them, through the creation of tools that users actually want to use to interface with Twitter.
Twitter as a standalone product does not have a future. Every change they make that turns more developers away from the platform continues to reduce the number of users globally who are using Twitter in some fashion. It's a downward spiral. Twitter's demise is inevitable if they continue down this path. Developers should assist them by abandoning the platform in droves. There won't be much of a platform left to work with anyway, so why bother? Let Twitter be the next Yahoo.
Three hours later and have you realized that CoolAssPuppy (Twitter itself) hasn't answered any specific question? If you write in HN you expect a conversation happening.
Probably because Twitter's PR people probably reached out to him and told him he wasn't helping anything and needed to stop posting. I wouldn't be surprised if they walked to his office and unplugged his keyboard.
It was bad. He posted something that made him look like an idiot or a liar. I'm not saying he is either one of those, but the post he wrote makes almost no sense when compared to the e-mail presented in the post. It is like he didn't even read it. If he was trying to say that they never sent that e-mail, then that point wasn't clear. I'm sure he had the best of intentions but it really blew up in his face.
It was bad. He posted something that made him look like an idiot or a liar. I'm not saying he is either one of those, but the post he wrote makes almost no sense when compared to the e-mail presented in the post. It is like he didn't even read it. If he was trying to say that they never sent that e-mail, then that point wasn't clear. I'm sure he had the best of intentions but it really blew up in his face.
I used to subscribe to a service that emailed me when I got a response on HN but that stopped for some reason. HN Notify I think it was called. What do people use apart from refreshing their own comment pages?
Dan Grossman, who's active in this very story, set up a replacement[1] a while back that I've been using and it seems to work well.
1: http://hnreplies.com/
1: http://hnreplies.com/
Legend thank you!
Everyone with elevated access is losing it in June as far as I'm aware - at least that's the case for academic uses. There will be academic pricing for Gnip, although we don't know how much it'll cost, I'm doubtful it'll be low enough for most academics to afford.
They've been telling us that they planned to shut down free elevated access down for well over a year now, so this doesn't come as a huge surprise. Twitter seems to do regular audits like this, and at least for our academic work, we've had to tell them what we're working on etc. every 6 months or so to keep elevated access.
Having said that, looking at the volume of data that Emojitracker is pulling down, it doesn't seem like it would be too difficult to spread the tracking over 2 or 3 separate connections. The UserStream API, whilst not designed for server-to-server connections, would probably allow Emojitracker to spread the ~900 emoji they track over 3 streaming connections using only a single account.
They've been telling us that they planned to shut down free elevated access down for well over a year now, so this doesn't come as a huge surprise. Twitter seems to do regular audits like this, and at least for our academic work, we've had to tell them what we're working on etc. every 6 months or so to keep elevated access.
Having said that, looking at the volume of data that Emojitracker is pulling down, it doesn't seem like it would be too difficult to spread the tracking over 2 or 3 separate connections. The UserStream API, whilst not designed for server-to-server connections, would probably allow Emojitracker to spread the ~900 emoji they track over 3 streaming connections using only a single account.
I have no love for Twitter, Gnip, or the way they've treated API devs over the years but this particular use case (ranking emojis by frequency of use) seems like it could be just as well served by the free sampled stream. I can't imagine anyone needing the exact usage counts vs. the relative popularity. Also, the author's stated goal of making it open and immediately usable by other developers would be better served by using the access level that is freely available to everyone.
Agreed, I don't think the full stream is needed either. It think being built of the free stream will serve the needs of making it open and still provide interesting data. Relative(sampling) usage graphs even seem more digestable and interesting to me.
> So even if Twitter were to give me free access to the Gnip APIs, others then wouldn’t be able to take my code and build upon it
I mean, isn't this the case anyway? Few people have "elevated access to the Streaming API that Emojitracker depends on in order to operate at its high volume"
I mean, isn't this the case anyway? Few people have "elevated access to the Streaming API that Emojitracker depends on in order to operate at its high volume"
With @jack having tweeted about this just months ago, talk about an internal disconnect..
Twitter fucked over every developer who trusted them a few years ago, and is now trying to recast their business as developer tools and services (since the core Twitter business probably is not a $20b business...). One of their biggest roadblocks is getting developers to trust them again.
And then they do this?
And then they do this?
The only way for them to get developers on their platform is to wait for the next generation of developers. Literally nobody will base anything on their stuff now, and this has been the case for a good few years.
I'm surprised there are still developers left for Twitter to piss off.
Aw that's too bad, I just discovered Emojitracker yesterday and thought it was really cool.
Also: is the service being removed colloquially known as the "Twitter Firehose", or something else? I recently implemented a feature that uses the standard rate-limited API. I went through a lot of pain to get the most out of the rate limits. And people would always ask me "why don't you just get all the tweets through the Streaming API?". I wonder if I had done that, how screwed I would be now.
Also: is the service being removed colloquially known as the "Twitter Firehose", or something else? I recently implemented a feature that uses the standard rate-limited API. I went through a lot of pain to get the most out of the rate limits. And people would always ask me "why don't you just get all the tweets through the Streaming API?". I wonder if I had done that, how screwed I would be now.
So, Twitter has long had two different classes of access:
- "Twitter Developer" access - which is what you find here: https://dev.twitter.com/ - that includes a streaming API that is relatively limited in terms of volume of data and operators that are allowed, but is free and open
- Gnip Twitter access - this is commercial access that would include the Firehose option (though that's not very common). But basically this is a paid offering that offers much more functionality than the developer platform.
In this specific case, the author is just losing elevated access to the streaming API:
https://dev.twitter.com/streaming/reference/post/statuses/fi...
I'm guessing they had extra keywords or multiple connections available so they could track those keywords. Nothing to do with the firehose or anything else. Just a user who had an exception to the official policy that's being taken away.
- "Twitter Developer" access - which is what you find here: https://dev.twitter.com/ - that includes a streaming API that is relatively limited in terms of volume of data and operators that are allowed, but is free and open
- Gnip Twitter access - this is commercial access that would include the Firehose option (though that's not very common). But basically this is a paid offering that offers much more functionality than the developer platform.
In this specific case, the author is just losing elevated access to the streaming API:
https://dev.twitter.com/streaming/reference/post/statuses/fi...
I'm guessing they had extra keywords or multiple connections available so they could track those keywords. Nothing to do with the firehose or anything else. Just a user who had an exception to the official policy that's being taken away.
For a long time twitter has offered specific developers who were doing projects they thought were interesting the ability to consumer a higher volume of traffic using the standard streaming endpoints.
It's hard to sort out the colloquial terms for Twitter features from what the company calls them =/
Okay, here we go:
- Twitter Firehose - full stream of all Tweets, commercial
- Twitter Decahose - 10% sample of Tweets for analysis, commercial
- Twitter Powertrack - real-time streaming of posts based on query (country, language, keyword, etc.), commercial
- Twitter Full-archive API - search interface for entire corpus of Tweets, commercial
- Twitter Streaming API - a mini-powertrack of posts based on basic keyword searches (up to 400 keywords), free
- Twitter Streaming Sample - a small sampling of Tweets for statistical analysis, free
There's a couple more esoteric product offerings from a data perspective, but these are the relevant ones. I think it's a pretty broad breadth of different options, with different functionality at the free and commercial options.
- Twitter Firehose - full stream of all Tweets, commercial
- Twitter Decahose - 10% sample of Tweets for analysis, commercial
- Twitter Powertrack - real-time streaming of posts based on query (country, language, keyword, etc.), commercial
- Twitter Full-archive API - search interface for entire corpus of Tweets, commercial
- Twitter Streaming API - a mini-powertrack of posts based on basic keyword searches (up to 400 keywords), free
- Twitter Streaming Sample - a small sampling of Tweets for statistical analysis, free
There's a couple more esoteric product offerings from a data perspective, but these are the relevant ones. I think it's a pretty broad breadth of different options, with different functionality at the free and commercial options.
What did emojitracker do?
Emojitracker gives live, realtime usage numbers for all emoji used on twitter.
http://emojitracker.com/
http://emojitracker.com/
It still does, for a few more weeks.
http://emojitracker.com/
Basically live tracks tweets that use emoji, so emoji are ranked by frequency.
http://emojitracker.com/
Basically live tracks tweets that use emoji, so emoji are ranked by frequency.
This is a very entitled piece... and a very entitled thought process...
When you are using someone else's platform, they still make the rules... this line in particular: "it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me." ... nobody "deserves" access to any API... period.
I don't understand the entitlement in so many tech projects, who get upset when a 3rd party they are freely using changes their service... you were never entitled to it in the first place, you merely benefited from its existence and should be appreciative of what you were allowed to do instead of feel entitled to it.
When you are using someone else's platform, they still make the rules... this line in particular: "it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me." ... nobody "deserves" access to any API... period.
I don't understand the entitlement in so many tech projects, who get upset when a 3rd party they are freely using changes their service... you were never entitled to it in the first place, you merely benefited from its existence and should be appreciative of what you were allowed to do instead of feel entitled to it.
Author here. To be clear, I don't believe I deserve free or special access to anything. The point I was trying to make was actually more in the opposite direction: I wouldn't want to receive free/special access as an exception now simply because the project has become "famous", if the same opportunity doesn't exist for other independent developers when their projects are starting out -- I just want to encourage a level playing field.
(When Emojitracker originally received elevated access 3 years ago, the project was not famous or well known at all, it was just a slightly more formal version of Me saying: "hey check this out, I think this might be a cool hack but I need access to 900 keywords instead of 400, that ok?" and Twitter: "yeah cool, no problem dude.")
(When Emojitracker originally received elevated access 3 years ago, the project was not famous or well known at all, it was just a slightly more formal version of Me saying: "hey check this out, I think this might be a cool hack but I need access to 900 keywords instead of 400, that ok?" and Twitter: "yeah cool, no problem dude.")
So you are still accessing the streaming endpoint, but you have more keywords than they normally allow? So you are not accessing the entire Firehose and parsing from there, right? Do you regularly update it to include new emoji when they come out? I am thinking about ways to do it on the free stream still, just trying to see where you are already at.
You should only expect a level playing field from public infrastructure.
You should never expect a level playing field from private infrastructure.
If anything this is more proof for the need of a public access Twitter-like service using public key crypto for self-signed authentication.
You should never expect a level playing field from private infrastructure.
If anything this is more proof for the need of a public access Twitter-like service using public key crypto for self-signed authentication.
You clearly were not part of the launch of twitter.
They sought out and begged for developers to use their platform. Then they started killing the apps one by one.
Your comment while "wise" in hindsight is thoughtless, arrogant and tone deaf to the work involved in creating an actual product that users use.
There is a large difference between entitlement and disappointment.
They sought out and begged for developers to use their platform. Then they started killing the apps one by one.
Your comment while "wise" in hindsight is thoughtless, arrogant and tone deaf to the work involved in creating an actual product that users use.
There is a large difference between entitlement and disappointment.
In the future you should invest your energies in building and supporting open and public platforms instead of hoping that some private company isn't going to part ways with your interests at some point.
The OP was not being thoughtless, arrogant or tone deaf, he was just being honest. If you feel disappointed it is because you made a poor decision. You should learn from your mistakes instead of shouting at the person who points out the tragic results of your actions.
There are plenty of open source and open platform projects out there. BitTorrent, Bitcoin, HTTP, OpenPGP... pick one and start building!
You should feel entitled to your individual rights in whatever republic you live in. You should not feel entitled to tell a private company how to run their business. Doing otherwise will always lead to disappointment.
The OP was not being thoughtless, arrogant or tone deaf, he was just being honest. If you feel disappointed it is because you made a poor decision. You should learn from your mistakes instead of shouting at the person who points out the tragic results of your actions.
There are plenty of open source and open platform projects out there. BitTorrent, Bitcoin, HTTP, OpenPGP... pick one and start building!
You should feel entitled to your individual rights in whatever republic you live in. You should not feel entitled to tell a private company how to run their business. Doing otherwise will always lead to disappointment.
Way to blame the victim, man.
Giving advice about life that cuts right to point is not "blaming the victim". No one was hurt here. We're just pointing out the obvious fact that people spent their time unwisely when they built on top of private platforms.
The right thing to do is consider this a lesson well learned, shut up, eat crow, and get on with your life like an adult, safe in the knowledge that you won't make the same mistake again.
Generally speaking the vast majority of people in this forum have a shockingly bad understanding of how our government, legal, accounting and financial systems work. That infrastructure isn't going anywhere and companies like Twitter exist only because of these formal systems. Learning the details of the relationships between these systems is your responsibility. You can easily be taken advantage of without a knowledgable understanding of the rules of the game. This is no one's fault but your own.
The right thing to do is consider this a lesson well learned, shut up, eat crow, and get on with your life like an adult, safe in the knowledge that you won't make the same mistake again.
Generally speaking the vast majority of people in this forum have a shockingly bad understanding of how our government, legal, accounting and financial systems work. That infrastructure isn't going anywhere and companies like Twitter exist only because of these formal systems. Learning the details of the relationships between these systems is your responsibility. You can easily be taken advantage of without a knowledgable understanding of the rules of the game. This is no one's fault but your own.
[deleted]
Then twitter should stop coming to the dev community and begging for us to like them again.
When twitter was just shutting things down and making it clear they were going to make money off the thing, I wasnt surprised nor was I happy, they had clearly decided to monetize what other people were doing themselves, but worse.
Since then twitter has not done anything that I would consider engaging or of note, and from the articles I read on HN, in a crunch on how the heck to monetize their platform and are coming back to the dev community trying to win mindshare.
You can't have it both ways.
When twitter was just shutting things down and making it clear they were going to make money off the thing, I wasnt surprised nor was I happy, they had clearly decided to monetize what other people were doing themselves, but worse.
Since then twitter has not done anything that I would consider engaging or of note, and from the articles I read on HN, in a crunch on how the heck to monetize their platform and are coming back to the dev community trying to win mindshare.
You can't have it both ways.
Out of all the sentences in OP's post.. that was the one that showed the LEAST entitlement. He's basically saying "I don't want to be treated special since everyone else doesn't have elevated access". That's all.
I think it comes from many companies branding themselves as altruistic and developer-friendly. I don't think people would be surprised at this move had Twitter said: "Using our platform? We own you. We will pursue all potential profit deriving from our platform to the fullest extent of the law."
This makes sense to me. Granting free firehose access to a small, fun project like this seems like unnecessary overhead for Twitter. I think their mistake was granting it in the first place.
Exactly. Wasn't HN almost unanimously in support of GitHub throttling CocoaPods barely a week ago?
How is this any different? Honest question.
How is this any different? Honest question.
CocoaPods was throttled not shut down. Twitter has an history of shutting down access to its API without notice or warning when a project becomes too popular.
What baffles me is that CocoaPods was actually providing something of value and their usage fit in with the purpose of GitHub (serving code.)
EmojiTracker appears to be essentially frivolous and puts stress on Twitter's platform for no good reason.
GitHub and Twitter both provided a free service and then modified it for a specific user without warning. I would say throttling CocoaPod affected more "legit" users than "shutting down" EmojiTracker did. (I put shutting down in quotes because it seems EmojiTracker still has limited access to their API?)
How come Twitter gets called out as unconditionally Evil, yet GitHub was just looking after itself?
EmojiTracker appears to be essentially frivolous and puts stress on Twitter's platform for no good reason.
GitHub and Twitter both provided a free service and then modified it for a specific user without warning. I would say throttling CocoaPod affected more "legit" users than "shutting down" EmojiTracker did. (I put shutting down in quotes because it seems EmojiTracker still has limited access to their API?)
How come Twitter gets called out as unconditionally Evil, yet GitHub was just looking after itself?
But then once you do grant it you should only revoke it for a really good reason (such as abuse or some other violation of the terms of service). Otherwise it should simply continue.
Why do people keep building stuff on Twitter? All the app developers that have been fenced off should create a simple protocol to make Twitter irrelevent. Brent Simmons suggested a great way do it awhile back: http://inessential.com/2012/06/29/matthew_on_twitter_restric...
Why do people keep building stuff on any silo?
Take a look at Known (https://withknown.com/) and Woodwind (https://woodwind.xyz/): one to have an actual website, the second to have a stream of content you want. Done.
Forget silos.
Take a look at Known (https://withknown.com/) and Woodwind (https://woodwind.xyz/): one to have an actual website, the second to have a stream of content you want. Done.
Forget silos.
I don't understand why people continue to build things for services that have a track record of shutting things down.
Based on http://www.emojitracker.com, there is a lot of Joy, and Love on Twitter.
Is any of this to be expected? Twitter is a private company and they have to create artificial scarcity to sell access to their platform. We still live in a market economy, no matter how much everyone on Twitter would like to think otherwise. They're just trying to pay the bills. Treating Twitter like a public platform is a disservice both to actual public infrastructure as well as the roll of private enterprise.
Twitter is a public company.
What you mean to say is Twitter is a publicly traded company owned and operated by private individuals. Those private individuals have rights and protections granted by the government jurisdictions in which Twitter operates.
I developed a product that connects Twitter and Facebook. I always thought that Twitter would be the enabler because of it's open street like atmosphere and Facebook would be difficult to work with because it was a walled garden. The exact opposite was true. Over time, Twitter would get more restrictive and difficult to work with while Facebook enhanced the look and feel of the stream and how the tweets appeared, emphasizing and highlighting them.
In essence I think the product spoke to the nature of the whole issue. The tweets left Twitter to seed Facebook with good interesting, connected data. Twitter would know little about the context while Facebook would get enriched context from the tweets.
Twitter doesn't want to be a dumb pipe to make apps smarter and better. They want to be the bucket like Facebook.
In essence I think the product spoke to the nature of the whole issue. The tweets left Twitter to seed Facebook with good interesting, connected data. Twitter would know little about the context while Facebook would get enriched context from the tweets.
Twitter doesn't want to be a dumb pipe to make apps smarter and better. They want to be the bucket like Facebook.
At that point anybody whose business relies on Twitter "free API" deserves to be in that situation. It's a well proven fact that Twitter will shut down every 3rd party client or app based on their API which is a bit too popular. Never build a business on top of a "free" API, especially Twitter, because there is 100% chances Twitter will shut down you access.
Has anyone here built a product that relies on Twitter's API for business critical features? How would you describe your risk tolerance, how long have you been in business, and do you have a personal or professional relationship with Twitter or Twitter's employees?
Yeah. I was recently building a simple bot for Twitter ( a favor for a friend of mine ). Things worked out, but everywhere I go I read "API Rate limited to ... per ..."
I know ... It's tremendous amount of data going on there, so I just don't want to imagine how hard it is to scale it and offer that to everyone.
Nevertheless it's so sad that they break promises.
Twitter nowadays is a bot driven, marketing platform ( with some sort of journalism touch ). I still remember the days that I was excited when I got a new follower. Now I think it's just about "hashtags".
I know ... It's tremendous amount of data going on there, so I just don't want to imagine how hard it is to scale it and offer that to everyone.
Nevertheless it's so sad that they break promises.
Twitter nowadays is a bot driven, marketing platform ( with some sort of journalism touch ). I still remember the days that I was excited when I got a new follower. Now I think it's just about "hashtags".
This just the extract value before exit phase, right?
Lazare(1)
In the past, Twitter had little formality associated with granting elevated access. It was very much, "Hey, I want to do this cool thing!" "Okay, sure, here you go." As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com). Over the last six months, we've started contacting all of those API key holders with elevated access and asked them to clarify what they're doing. In some cases, we know the business and business owner and reach out personally. In other cases, like this one, the business is listed in our systems as "N/A", so we send the template mail.
In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
Note also that in this instance, the notice is NOT about shutting down this app. It merely inquires about why the developer needs elevated access, something that is typically reserved for our business partners.
None of this is related to continuing to use the Twitter API or our commitment to enabling developers to build on our platform (Fabric, Gnip, our Ads platform, and the Twitter API).