Remove default "terraform" partner_id(github.com)
github.com
Remove default "terraform" partner_id
https://github.com/terraform-providers/terraform-provider-azurerm/issues/4747
80 comments
Tone-deaf faceless pr Twitter accounts out there slinging corporate bullshit, take note: this is how you communicate with your community if you really care about them.
Thanks for the very thoughtful response, and thank you for providing us with your fantastic software!
Thanks for the very thoughtful response, and thank you for providing us with your fantastic software!
This is a great reply. Hashicorp is one of my favorite companies. I would always give them the benefit of doubt.
> I can say that the usage information given by this partner code has directly resulted in more headcount being assigned to the "azurerm" Terraform provider that may not have been otherwise assigned.
In 5 days? Impressive
In 5 days? Impressive
Thanks - Hard to make any better than direct founder reply to clear up confusion.
Thanks for hopping on here yourself, it definitely throws some bleach into the murky water.
Great response, thanks for clarifying in detail
> ...Going forward, we will be building an option to opt out of using this partner ID...
One should be opted out by default and the option should be build to opt in.
One should be opted out by default and the option should be build to opt in.
Based on the explanation given, there's absolutely no privacy benefit to not including this code. The fact that it's Terraform doing the work is already being sent via the User Agent in the API requests. The fact that it's _you_ who is making the API calls is also embedded in the request. This amounts to a minor hack of Azure's API logging to enable slightly more convenient reporting. It sends no new data. In other words "opting in" or "opting out" has zero impact on whatever privacy implications you are worried about. This comes down to a simple misunderstanding of what's going on.
Most people wouldn't care about it's usage and will keep it as the default.
i dont know why this is getting downvoted
Because people think it's a fine default.
Just like I think "User-Agent: curl/7.58.0" is a fine default.
Just like I think "User-Agent: curl/7.58.0" is a fine default.
Given the amount of fingerprinting and screwing with the content based on platform, I'd be happy to do away with user-agent completely. I'm tired of websites that are completely broken on mobile and force you to use the app (or constantly remind you about their app) and everybody returning a globally unique identification which is so hard to scrub.
Because for this specific bit of data, the value that it's set to by default has zero privacy impact whatsoever. So getting upset over nothing and yelling at a CEO who gave a great explanation of what's going on is incredibly counterproductive behavior.
Because defaults matter, very few people opt-in to anything and therefore opt-in metrics are by definition biased and useless.
This does indeed look bad, but I'd like to give Hashicorp the benefit of the doubt and allow them to respond before grabbing any pitchforks/torches. Let's not forget they've added a ton of value to many of our lives and have never required a cent (for the free version).
Thank you. I'm hijacking this top comment at the moment to share that I have responded: https://news.ycombinator.com/item?id=21389908
I hope that makes it very clear what this is, why it's there, and what we're doing about the GitHub issue raised. If there are any questions, please respond to that thread and I'll be happy to respond.
I hope that makes it very clear what this is, why it's there, and what we're doing about the GitHub issue raised. If there are any questions, please respond to that thread and I'll be happy to respond.
[deleted]
If there is any company that I would give the benefit of the doubt, it's Hashicorp.
[deleted]
Isn't this what Partner ID is designed for?
"Injects" is a strong word.
Maybe "Hashicorp provides an Azure Partner ID in Terraform deployments for deployment stats, and some users may not want that" would be a better title.
"Injects" is a strong word.
Maybe "Hashicorp provides an Azure Partner ID in Terraform deployments for deployment stats, and some users may not want that" would be a better title.
Exactly.
They very clearly allow it to be changed in a config; and I'm willing to bet my socks that Azure is not giving Hashicorp 'deployment structure' information based on a partner id.
They very clearly allow it to be changed in a config; and I'm willing to bet my socks that Azure is not giving Hashicorp 'deployment structure' information based on a partner id.
Included in the link text.
From Microsoft documentation:
This program will allow ISVs who deploy their software on an Azure customer’s infrastructure an opportunity to get credit for the impact of their software.
The data generated by the Azure ISV Customer Usage Attribution program will be used for ISV partners to qualify for partner programs by providing a automated method of linking a customers usage to the ISVs software.
ISV partners will receive reporting for deployments from the Azure ISV Customer Usage Attribution program. Data may be anonymized for deployments from outside of Azure Marketplace. Reporting will be made available in the Cloud Publisher Portal, the same platform where GUIDs will be registered and partners can configure and manage listings for Azure Marketplace.
From Microsoft documentation:
This program will allow ISVs who deploy their software on an Azure customer’s infrastructure an opportunity to get credit for the impact of their software.
The data generated by the Azure ISV Customer Usage Attribution program will be used for ISV partners to qualify for partner programs by providing a automated method of linking a customers usage to the ISVs software.
ISV partners will receive reporting for deployments from the Azure ISV Customer Usage Attribution program. Data may be anonymized for deployments from outside of Azure Marketplace. Reporting will be made available in the Cloud Publisher Portal, the same platform where GUIDs will be registered and partners can configure and manage listings for Azure Marketplace.
What do you think is a problem? If you're using Azure Marketplace then you're already dealing with vendors so of course they'll receive your information for billing and support. Otherwise it's anonymized.
Nothing in the deployment itself is sent to them (and no cloud would ever risk that kind of exposure). It's simple usage analytics.
Nothing in the deployment itself is sent to them (and no cloud would ever risk that kind of exposure). It's simple usage analytics.
By adding their partner ID, they get revenue based on the usage in Azure.
I think I'm OK with that? If it doesn't increase my costs, and Hashicorp gets paid because I used their tools to deploy my services, and Azure is getting paid more because I probably deploy more services because it's easier thanks to Terraform, doesn't everyone win?
I'm mostly okay with it, though the deployment plan isn't the proper place for what's essentially a referral code.
Hashicorp is facilitating the purchase of Azure products, and deserves a cut of the revenue. But this should be handled on the account level, either on sign-up or as an account field (a-la Amazon Smile)
I'm less okay with the tracking for analytics purposes, but if Terraform finds some way to make it easy for me to tell cloud providers who sent me, that seems fair.
Hashicorp is facilitating the purchase of Azure products, and deserves a cut of the revenue. But this should be handled on the account level, either on sign-up or as an account field (a-la Amazon Smile)
I'm less okay with the tracking for analytics purposes, but if Terraform finds some way to make it easy for me to tell cloud providers who sent me, that seems fair.
If it's a referral code at the account level, then you can't easily have multiple providers supplying tools for the client. Whereas, if it's part of the deployment itself, the utilization of one partner tool can be distinguished over another.
Or in other words, the "cut of the revenue" needs to be in proportion to the usage of Azure resources generated by the partner tool. I think that's why its designed this way.
Or in other words, the "cut of the revenue" needs to be in proportion to the usage of Azure resources generated by the partner tool. I think that's why its designed this way.
We do not get any revenue. Zero. We also get no information from this since the partner ID is actually a Microsoft Partner ID and not a HashiCorp Partner ID. Please see my response here: https://news.ycombinator.com/item?id=21389908
Do you have a link to some docs supporting that? How exactly is revenue tied to usage?
It's there because of the kind of partner agreement ISVs have with the cloud provider. There are two things at play:
1) How does the cloud provider know what the ISV is selling, in order to credit them for it (discounts, incentives, program membership, etc)?
2) How do the sales reps get paid on related deals? This goes both ways: the ISV's reps get paid regardless but they need a way to tag the cloud-based deal in their own CRM, and the cloud sales reps need to be able to get paid base on the ISV partner's cloud deals, and that's really hard unless there is a concrete way to relate the ISV's sales deal to cloud consumption associated with the end customer.
One of the ways to do this is on the front end via deployment tracking (like this Terraform ID). Another way is to do it on the back end through billing account linkages. In any case, the end customer shouldn't really care one way or another -- these are just mechanisms to ensure the partner relationship between ISV and hyperscaler is well understood.
1) How does the cloud provider know what the ISV is selling, in order to credit them for it (discounts, incentives, program membership, etc)?
2) How do the sales reps get paid on related deals? This goes both ways: the ISV's reps get paid regardless but they need a way to tag the cloud-based deal in their own CRM, and the cloud sales reps need to be able to get paid base on the ISV partner's cloud deals, and that's really hard unless there is a concrete way to relate the ISV's sales deal to cloud consumption associated with the end customer.
One of the ways to do this is on the front end via deployment tracking (like this Terraform ID). Another way is to do it on the back end through billing account linkages. In any case, the end customer shouldn't really care one way or another -- these are just mechanisms to ensure the partner relationship between ISV and hyperscaler is well understood.
That's okay, provided they disclose it. Much like Ubuntu does with Amazon.
It's a random GUID that lets the company track how much their tool is used in deployments in Azure. It's about as noninvasive as it gets.
Feels sort of like a User Agent string to me.
I agree. I'm not mad at cURL for setting a user-agent string, and I don't see how this is much different unless I'm missing something.
It's about as unique as a user agent that only says "Firefox" or "Chrome". Seems pretty acceptable to me.
[deleted]
This is a non-issue and PURE FUD. They are anonymously tracking usage of their product on azure. BFD.
I wonder how much the outraged individual has contributed financially to what is a fabulous tool that all DevOps engineers benefit directly from. I get that it should have been discussed, but even still - the code is there for all to see and read whenever they can be bothered too. They can even fork and build it without this. It's this level of open source snobbery that makes me think that eventually only the big companies will contribute and open source projects - and only because it brings developers onto their platforms.
This sets a dangerous precedent. You shouldn't need to contribute financially to file an issue about an open source product you use.
Edit: cleaned up the wording a bit by removing the "raise a concern" bit. Hopefully that removes any ambiguity in the comment.
Edit: cleaned up the wording a bit by removing the "raise a concern" bit. Hopefully that removes any ambiguity in the comment.
> or file an issue about an open source product you use.
Are you kidding me? This is literally the mechanism to provide feedback to the contributors.
Are you kidding me? This is literally the mechanism to provide feedback to the contributors.
I think the statement is meant to be read as:
> You shouldn't need to contribute financially to [...] file an issue about an open source product you use.
> You shouldn't need to contribute financially to [...] file an issue about an open source product you use.
Yes, sorry, I had an extra bit in there that I removed before posting the comment, and in the process made it a bit harder to read. This is definitely what I was intending.
Why would people need to pay money to file issues?
Nowhere, in the tenets of "free software" or "open source", is there any expectation there is any mechanism whatsoever for support or even "filing issues."
If an open-source project wants to charge you money to file an issue, that's totally OK and totally in-bounds.
If an open-source project wants to charge you money to file an issue, that's totally OK and totally in-bounds.
Haha that's a fun revenue model.
it sounds like the perfect monetisation strategy for open source. as long as contributing a fix via a pull request is free opening a ticket to cover for the dev time needed to work on the issue in your behalf seems just fair
File? No. Get worked/dealt with at all? Yeah. Developers can't feed their children with good vibes.
Pay-to-file is fine, if that's what the authors want.
It seems pretty contrary to the spirit of OSS (shared improvement, for example), so I wouldn't call it "fine". "Within the authors available options", maybe.
Like -- and pardon my crudity -- you could throw a free party, then poo on the floor, totally allowed but perhaps not within the spirit of parties.
Like -- and pardon my crudity -- you could throw a free party, then poo on the floor, totally allowed but perhaps not within the spirit of parties.
If you don't want people to have discussions about your code then don't release it, ever, in any format. If you don't want to have discussions on your bug tracker then don't have one open to the world. Releasing something as open source doesn't mean you get total control over how other people feel or talk about it. This isn't even outrage, it's a mature constructive discussion of a released product.
I don't see "outrage."
> the code is there for all to see and read whenever they can be bothered too.
This is exactly what this developer is doing.
> open source snobbery
I also don't read any snobbery in their PR. They found a genuine problem. They have actual security concerns with this behavior.
> the code is there for all to see and read whenever they can be bothered too.
This is exactly what this developer is doing.
> open source snobbery
I also don't read any snobbery in their PR. They found a genuine problem. They have actual security concerns with this behavior.
Making a project open source does not excuse the author from ethical responsibilities. Asking for changes to an open source project is not snobbery.
It's there because of the kind of partner agreement ISVs have with the cloud provider. There are two things at play:
1) How does the cloud provider know what the ISV is selling, in order to credit them for it (discounts, incentives, program membership, etc)?
2) How do the sales reps get paid on related deals? This goes both ways: the ISV's reps get paid regardless but they need a way to tag the cloud-based deal in their own CRM, and the cloud sales reps need to be able to get paid base on the ISV partner's cloud deals, and that's really hard unless there is a concrete way to relate the ISV's sales deal to cloud consumption associated with the end customer.
One of the ways to do this is on the front end via deployment tracking (like this Terraform ID). Another way is to do it on the back end through billing account linkages. In any case, the end customer shouldn't really care one way or another -- these are just mechanisms to ensure the partner relationship between ISV and hyperscaler is well understood.
2) How do the sales reps get paid on related deals? This goes both ways: the ISV's reps get paid regardless but they need a way to tag the cloud-based deal in their own CRM, and the cloud sales reps need to be able to get paid base on the ISV partner's cloud deals, and that's really hard unless there is a concrete way to relate the ISV's sales deal to cloud consumption associated with the end customer.
One of the ways to do this is on the front end via deployment tracking (like this Terraform ID). Another way is to do it on the back end through billing account linkages. In any case, the end customer shouldn't really care one way or another -- these are just mechanisms to ensure the partner relationship between ISV and hyperscaler is well understood.
I admit I don’t closely follow Hachicorp commits, blogs, changelogs, etc.. Is it possible that there was another commit providing a rationale for the change? Maybe a changelog entry explaining what the UUID represents and how it benefits Hachicorp, Microsoft/Azure, and end users?
How is this different to a web app that uses a Google Maps api key (for example)?
It sucks that you cannot have several partner ids. Partners have to fight over who will get their partner id at the end. Hashicorp at least allow to specify a different one in their software.
Thanks for the swift response from the founder, I just don't understand those types of sneaky github issues that surprisingly get very rapidly to the HN frontpage as smear campaign against startups, when even setting the application name and version in the user agent header in a FOSS application that can be opted-out is considered spying, then everything can be spying
Anyone know if this causes users to become noncompliant with GDPR, FedRAMP, or similar regulatory protections?
Did you read what the issue is even about? It's about setting the string "terraform" in a config file. It's not some personal data or evil tracking token.
Most comments are overblowing this issue, but you're under-blowing it. It's not just "a config file".
Let me try to describe what the Azure Partner ID is meant to be used for. Say you build a SaaS service that runs on Azure. As many Azure-based companies do, you have a Sales and Technical Account Manager from Microsoft that is responsible for things like getting you discounts based on sustained usage, perhaps helping bring you to market through their own sales channels, etc. They (and subsequently you) are measured on the basis of how much consumption of Azure resources you drive.
While you're still a purely-SaaS product, this consumption is easy to measure. It's basically just the size of your bill at the end of every month. But as your business grows, you start to have some customers who don't want to put their data into some random SaaS application; they'd rather run your application inside their subscription, either through some middleware or just in its own resource group. You can do this integration technically, but now you have a problem-all that consumption that used to be credited to you is now invisible. From your Azure sales rep's perspective, your usage is going down. It becomes harder to justify all those discounts and services they're giving you.
Enter the Azure Partner ID. You simply set this flag on all the resources you're spinning up inside your customers' subscription, and it tags that consumption as being driven by you. The bill is going to someone else, but you get the credit. You go to your sales rep and say "please add all the resources from Customer XYZ with Partner ID ABC to me", and they have some tooling to do that.
That's the field that Hashicorp is using/abusing. It doesn't do anything directly by itself (as far as I can tell, it's totally invisible to normal off-the-shelf Azure tooling), but clearly somewhere the infrastructure exists for someone who doesn't normally get to look at Customer XYZ's subscription, to at least see the consumption of some part of it. That's a violation, in my opinion.
It's twisting the semantics of the tag, at the very least. You can easily make the case that a SaaS service is directly contributing to consumption when they run a service in someone's subscription. That's almost certainly consumption that wouldn't have existed at all without that service. On the other hand, it's clearly ridiculous to say that if it wasn't for Terraform, none of the resources Terraform has spun up would've been created.
Let me try to describe what the Azure Partner ID is meant to be used for. Say you build a SaaS service that runs on Azure. As many Azure-based companies do, you have a Sales and Technical Account Manager from Microsoft that is responsible for things like getting you discounts based on sustained usage, perhaps helping bring you to market through their own sales channels, etc. They (and subsequently you) are measured on the basis of how much consumption of Azure resources you drive.
While you're still a purely-SaaS product, this consumption is easy to measure. It's basically just the size of your bill at the end of every month. But as your business grows, you start to have some customers who don't want to put their data into some random SaaS application; they'd rather run your application inside their subscription, either through some middleware or just in its own resource group. You can do this integration technically, but now you have a problem-all that consumption that used to be credited to you is now invisible. From your Azure sales rep's perspective, your usage is going down. It becomes harder to justify all those discounts and services they're giving you.
Enter the Azure Partner ID. You simply set this flag on all the resources you're spinning up inside your customers' subscription, and it tags that consumption as being driven by you. The bill is going to someone else, but you get the credit. You go to your sales rep and say "please add all the resources from Customer XYZ with Partner ID ABC to me", and they have some tooling to do that.
That's the field that Hashicorp is using/abusing. It doesn't do anything directly by itself (as far as I can tell, it's totally invisible to normal off-the-shelf Azure tooling), but clearly somewhere the infrastructure exists for someone who doesn't normally get to look at Customer XYZ's subscription, to at least see the consumption of some part of it. That's a violation, in my opinion.
It's twisting the semantics of the tag, at the very least. You can easily make the case that a SaaS service is directly contributing to consumption when they run a service in someone's subscription. That's almost certainly consumption that wouldn't have existed at all without that service. On the other hand, it's clearly ridiculous to say that if it wasn't for Terraform, none of the resources Terraform has spun up would've been created.
I believe you are mistaken. It appears to inject a tracking token. https://docs.microsoft.com/en-us/azure/marketplace/azure-par...
It's not a user-specific tracking token, it's a tool specific token (https://github.com/terraform-providers/terraform-provider-az...)
All users of terraform will look the same, it's to get a general idea of "how many terraform using instances are there".
All users of terraform will look the same, it's to get a general idea of "how many terraform using instances are there".
So then the answer is no? No need to be rude about it and stop projecting. It was a simple question about the implications are of what info partner IDs send.
If its free you are (or can be) the product. I dont see this one as an issue so long as its not flooding personal information to Microsoft outside of what you provide them when you register to Azure.
The concern is Hashicorp receiving data about deployments even outside of the Azure marketplace, although the documentation makes it look like it'll only be in aggregate/will be anonymized
> ISV partners will receive reporting for deployments from the Azure ISV Customer Usage Attribution program. Data may be anonymized for deployments from outside of Azure Marketplace.
> ISV partners will receive reporting for deployments from the Azure ISV Customer Usage Attribution program. Data may be anonymized for deployments from outside of Azure Marketplace.
>ISV partners will receive reporting for deployments from the Azure ISV Customer Usage Attribution program.
This seems like an issue.
This seems like an issue.
I love Terraform and in general Hashicorp but this is not acceptable. Changes like these should be discussed with the community first.
They should also state clearly what information they are getting from Microsoft. Azure docs on this is way too generic and filled with "maybes".
They should also state clearly what information they are getting from Microsoft. Azure docs on this is way too generic and filled with "maybes".
I know this is a cheap shot, but... Azure docs _in general_ are way too generic and filled with "maybes".
Hashicorp is kinda well known for not engaging with community whatsoever :)
I am not sure about this. Till recently I have mostly used their products with just needing to read some docs and initial guides. Only recently I was using Terraform and Packer so much that I had fumbled on some issues and saw the community discussions on those. Hashicorp was engaging there atleast.
You'll see when you open small bugfix PR and they let it rot or when stumble upin years old issue, discussed over and over again without any resolution in sight.
I've had an issue (in my case the VWWare provider for Vagrant) that didn't get the resolution I wanted, but on the whole, I feel they do a good job. As a smallish company, they have to be very selective WRT resource allocation.
Decision-making like this is the kind of thing that diminishes support for open source in large old-guard type organizations.
Why should terraform get credit for the infrastructure? It doesn’t drive demand. If they want a piece of that, leave it commented with a hat-in-hand appeal.
Why should terraform get credit for the infrastructure? It doesn’t drive demand. If they want a piece of that, leave it commented with a hat-in-hand appeal.
I'm the founder of HashiCorp.
I want to make something clear up front that this does NOT allow us to see resource usage by Terraform user and does NOT result in credits or revenue sharing at all. HashiCorp has no direct access to this information in any form.
Before explaining "why" we do this, I do want to apologize and say that adding this without proper explanation was a mistake. It isn't clear why it's there and I think enough companies have hurt users with features like this that defaulting to a negative reaction makes sense. I'm sorry. I promise (and will explain) that our usage is not nefarious, and even further this ID does not give us access to anything directly.
The "why": the partner ID lets Microsoft better track Terraform usage internally (with data they already have access to, just lets them filter it by Terraform). Microsoft does share aggregate information with us ("x% of all Azure workloads") but does not go any more granular than that.
This information is used by Microsoft to gauge how much investment to make into Terraform as well as what resources are a priority to fix any issues or make improvements to. Microsoft is a big partner of ours[1] and as part of that partnership they employ full-time people to improve the Terraform provider. Part of making that partnership successful is measuring the output of it and this is one mechanism that allows them to do that. I can say that the usage information given by this partner code has directly resulted in more headcount being assigned to the "azurerm" Terraform provider that may not have been otherwise assigned.
Note that all this partner ID does is let Microsoft filter by "Terraform." They already have and use all information around what resources are being spun up by accounts (as you would expect any IaaS or even SaaS to do). This doesn't introduce anything else other than that easier filter for them.
The partner ID used by Terraform was provided directly by Microsoft and generated by them. It is not associated with our Azure accounts at all. This is an extra assurance that we don't have access to any partner information using this ID.
Some have pointed out that the docs specifically state that this is used for credit/revenue sharing. That is a feature of the partner ID but not one that we use. Azure is a large, complex platform and features are overloaded for different use cases. In our case, the partner ID does NOT provide us with any information, credits, or revenue. Zero.
Going forward, we will be building an option to opt out of using this partner ID. It was already noted in other comments that we made it configurable since there are other use cases for it that a Terraform user might want to set. We haven't made a direct option to opt-out and we will do that in the next release. As a workaround today, you can set any partner ID you want (an invalid value) and we will send that and that will function similarly.
Note that for years all our providers have also sent a custom user agent that notes Terraform and the version of Terraform being used. We haven't been secret about this (I've publicly tweeted about it many times), but it feels important to call out in this comment as well. This information could also be used by providers to determine Terraform usage. Similarly, HashiCorp has no direct access to this information.
I'm happy to answer any questions, and once again I'm sorry about how this wasn't communicated up front.
EDIT (2 hours after posting): We've opened a PR for adding the opt-out behavior which also includes an environment variable you can set. We plan to include this as part of the next patch release. https://github.com/terraform-providers/terraform-provider-az...
[1]: https://www.hashicorp.com/blog/hashicorp-and-microsoft-exten...