Peerd: an AWS VPC peering connection management tool(github.com)
github.com
Peerd: an AWS VPC peering connection management tool
https://github.com/atlassian-labs/peerd
27 comments
Can you explain the "Patent Pending" bit? Why would you need to create a patent for a short Apache 2.0 licensed API calling Python script?
In addition, the apache license grants patent rights (§3 Grant of Patent License). Since it seems to be a research project, I assume they're just very happy to be able to put their name on it with Atlassian funding the patent :-)
Ya this isn’t something that should be patentable. Tons of prior art in conference talks on this. Cool tool though, glad it’s released!
Yep. My team wrote this in 2014 or 2015, the features and conf file almost identical. We’d had to extract out of a custom Ansible module to better do the full dance.
Shared the approach with AWS ProServe, Stelligent (now Mphasis), and others as well, also discussed briefly with Atlassian architects probably in 2015-2016 timeframe.
Surprised if this isn’t one of those lots of folks just knocked out and didn’t think much of, and really should be in AWS toolkit.
Shared the approach with AWS ProServe, Stelligent (now Mphasis), and others as well, also discussed briefly with Atlassian architects probably in 2015-2016 timeframe.
Surprised if this isn’t one of those lots of folks just knocked out and didn’t think much of, and really should be in AWS toolkit.
And there are commercial products that do this and more that have been around for years [0].
[0] https://www.aviatrix.com
[0] https://www.aviatrix.com
Interesting concept - is this designed to be used independent of some kind of infrastructure as code tooling?
I do all of my VPC peering and route-table manipulation via terraform repos today, so I am curious what the benefit of using this would be over what I have today (if there is).
Thanks.
I do all of my VPC peering and route-table manipulation via terraform repos today, so I am curious what the benefit of using this would be over what I have today (if there is).
Thanks.
Terraform is definitely an option, but we found (the story might be different since 0.12) that creating large full meshes with many VPCs across many accounts with many route tables resulted in Terraform modules and files that were quite complex, long and hard to overlap with other meshes.
This tool can be used alongside terraform, ansible or whatever infra-as-code orchestration tool you like. I encourage you to do so! :)
This tool can be used alongside terraform, ansible or whatever infra-as-code orchestration tool you like. I encourage you to do so! :)
Yes, that’s the right approach. I recall we shared with Atlassian about extracting this from Ansible and keeping it outside the then brand new Terraform back when we met at a certain forum as well as when we had an NDA sync at our HQ.
It really is remarkable how identical this is to that implementation, someone using that tool would be instantly at home in this. I guess it’s one of those ‘obvious’ problems once you decide to tackle it! Well done.
I regret that back then our firm was uninterested to open source more than our enterprisification (abstraction from one CSP allowing it to work for others) of Security Monkey.
It really is remarkable how identical this is to that implementation, someone using that tool would be instantly at home in this. I guess it’s one of those ‘obvious’ problems once you decide to tackle it! Well done.
I regret that back then our firm was uninterested to open source more than our enterprisification (abstraction from one CSP allowing it to work for others) of Security Monkey.
FWIW, this problem is significantly alleviated with Pulumi. I've knocked together something internal that's a lot like this, in Pulumi, in something like thirty lines.
That's emphatically not to take anything away from this, as if you're not using Pulumi this is a relatively hard problem to deal with, but it does feel as though a lot of the more annoying aspects of our jobs as infra professionals are made a lot easier with it.
That's emphatically not to take anything away from this, as if you're not using Pulumi this is a relatively hard problem to deal with, but it does feel as though a lot of the more annoying aspects of our jobs as infra professionals are made a lot easier with it.
Yup your point is true about having to manage N(N-1) number of vpc peering related components. I don’t ever deal with more than 10 VPCs at a time. Maybe you have world-conquering aspirations and need insane scale ;D
Haven't had an opportunity to look any deeper than the project description, but this is definitely a thing I've wanted in the past...
Cool tool and the problem it’s trying to solve is a real one. However on the complex mesh of connections across accounts isn’t the new way to do this now not the Transit Gateway?
A tool that could manage complex TGW setups across many accounts would also be cool. Terraform can do that but it’s not the most elegant solution.
A tool that could manage complex TGW setups across many accounts would also be cool. Terraform can do that but it’s not the most elegant solution.
In addition to Transit Gateway, there's also AWS PrivateLink [1], which is my preferred approach for scenarios where it works. As an alternative to peering entire networks together, PrivateLink allows you to establish minimal connectivity between specific clients and services that need to communicate. When a service supports PrivateLink, any authorized client can create a private endpoint for that service in their VPC, and the endpoint receives a local IP address (in that VPC).
This has a couple of benefits. The first is that you don't need to coordinate IP address ranges in different VPCs, which is necessary if you're peering them together (unless you're using NAT). IPv4 address management in large networks can be challenging.
The second benefit is that although clients can communicate with the service, there isn't an actual route between the two networks. This better achieves the principle of least privilege at the network level; it avoids the security risks of connecting networks together such that any machine can connect to any other. When you design an application network to use PrivateLink from the ground up, you can start with the expectation that the VPC doesn't need any connections to any other networks, not even the Internet; it can potentially be an isolated network bubble, with only PrivateLinks allowing traffic in and out (using e.g. Session Manager for administration).
There are also practical limits on how large a federated network of peered VPCs can grow (probably helped by Transit Gateway), but there isn't any limit I'm aware of on the number of client/server connections that can be established with PrivateLink.
For these reasons I prefer to use PrivateLink when building systems that communicate across VPCs, falling back to network peering or Internet-facing endpoints in scenarios where it doesn't work.
However, one downside compared to network peering is that you need to set up a PrivateLink connection for each service that each client needs to call, though this process can be automated during infrastructure provisioning. Services need to have specifically onboarded with PrivateLink; you can't just deploy a service on the network and have it work.
[1] https://aws.amazon.com/privatelink/
This has a couple of benefits. The first is that you don't need to coordinate IP address ranges in different VPCs, which is necessary if you're peering them together (unless you're using NAT). IPv4 address management in large networks can be challenging.
The second benefit is that although clients can communicate with the service, there isn't an actual route between the two networks. This better achieves the principle of least privilege at the network level; it avoids the security risks of connecting networks together such that any machine can connect to any other. When you design an application network to use PrivateLink from the ground up, you can start with the expectation that the VPC doesn't need any connections to any other networks, not even the Internet; it can potentially be an isolated network bubble, with only PrivateLinks allowing traffic in and out (using e.g. Session Manager for administration).
There are also practical limits on how large a federated network of peered VPCs can grow (probably helped by Transit Gateway), but there isn't any limit I'm aware of on the number of client/server connections that can be established with PrivateLink.
For these reasons I prefer to use PrivateLink when building systems that communicate across VPCs, falling back to network peering or Internet-facing endpoints in scenarios where it doesn't work.
However, one downside compared to network peering is that you need to set up a PrivateLink connection for each service that each client needs to call, though this process can be automated during infrastructure provisioning. Services need to have specifically onboarded with PrivateLink; you can't just deploy a service on the network and have it work.
[1] https://aws.amazon.com/privatelink/
We love PrivateLink but there are limits and it needs to be provisioned on a service basis, or fronting a proxy for usage across thousands of services. :) And as you can imagine, it too would need some complex orchestration if used across thousands of VPCs or hundreds/thousands of accounts.
But definitely a great tool in the toolbox.
But definitely a great tool in the toolbox.
How do you deal with the connections being to only one availability zone?
The connection can be present in multiple availability zones, though the service does need to decide which ones it will support at setup time. One simple approach is for the service to accept connections in many or all AZs, allowing the client to decide which ones are relevant to it. From the endpoint service documentation [1]:
> For low latency and fault tolerance, we recommend using a Network Load Balancer with targets in every Availability Zone of the AWS Region. To help achieve high availability (...) you can enable cross-zone load balancing. Cross-zone load balancing enables the load balancer to distribute traffic across the registered targets in all enabled Availability Zones. For more information, see Cross-Zone Load Balancing in the User Guide for Network Load Balancers.
When establishing a connection, the client can specify multiple subnets in which it would like to create an endpoint [2], and those subnets can be in any AZs supported by the service, if I recall correctly. The client can thus establish a connection that spans multiple AZ by requesting endpoints in subnets in those AZs. In most scenarios it's fine if client/service traffic can cross between AZs; I've only found it necessary to partition traffic by AZ in specialized applications (usually applications that are themselves trying to provide a zonal availability model, or that follow cell-based architecture at the zone level -- most business services don't fall into these categories, though low-level infrastructure services sometimes do).
[1] https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-se...
[2] https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_C...
> For low latency and fault tolerance, we recommend using a Network Load Balancer with targets in every Availability Zone of the AWS Region. To help achieve high availability (...) you can enable cross-zone load balancing. Cross-zone load balancing enables the load balancer to distribute traffic across the registered targets in all enabled Availability Zones. For more information, see Cross-Zone Load Balancing in the User Guide for Network Load Balancers.
When establishing a connection, the client can specify multiple subnets in which it would like to create an endpoint [2], and those subnets can be in any AZs supported by the service, if I recall correctly. The client can thus establish a connection that spans multiple AZ by requesting endpoints in subnets in those AZs. In most scenarios it's fine if client/service traffic can cross between AZs; I've only found it necessary to partition traffic by AZ in specialized applications (usually applications that are themselves trying to provide a zonal availability model, or that follow cell-based architecture at the zone level -- most business services don't fall into these categories, though low-level infrastructure services sometimes do).
[1] https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-se...
[2] https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_C...
Terraform gets unruly at scale, and makes it hard to pass it off to other teams or take a vacation if you're responsible when it breaks.
Aviatrix popped up on my radar recently trying to solve the operations side of this. Handles cross account TGW attachments and simplifies the routing.
Aviatrix popped up on my radar recently trying to solve the operations side of this. Handles cross account TGW attachments and simplifies the routing.
Yup, Transit Gateway is definitely a solution!
Why doesn't Atlassian-labs use Atlassian's own remote git hosting service, bitbucket?
BitBucket still has users?
I assume at least one, the person who thought the last redesign was a good idea.
I assume at least one, the person who thought the last redesign was a good idea.
I still use bitbucket because I also use mercurial, but they've deprecated that and will only offer git beginning later this year.
So, no, I don't know why anyone would use bitbucket at this point.
So, no, I don't know why anyone would use bitbucket at this point.
The enterprise version has plenty of users.
[deleted]
Network effects are nothing to sneeze at.
Vpc peering will be replaced by transit gateway
I had this exact use case in my job, trying to connect an app hosted on an ec2 in one account to a redshift instance on another, vpc peering to the rescue. will checkout your project when i find time.
peerd is an AWS VPC Peering Connection management tool. It manages the full lifecycle of creation, deletion and route table updates needed to make VPC peerings useful.
Capabilities
Yes, Transit Gateway is a valid replacement for this tool. :)