Say I decide to open a pirate themed gym called Slimmer Ye Timbers and buy the domain slimmeryetimbers.com.
If I want to setup a website, I will go to a hosting company, set something up, go into the DNS and point a couple of records for the top level domain and the www subdomain to the hosting company.
If I want to receive email sent to [email protected] I need to setup a mail server (Google, Outlook, web host may offer one, could setup an open source one, etc) and once it's setup I go to the DNS to point an MX record at the mail server.
So far, if people try to lookup my website or send an email TO me everything is pretty straightforward.
Now, if I want to send email that says it's FROM [email protected] is where things get weird. Because I don't have to do anything.
Any server, anywhere can just do it and every mail server that receives a message saying it's from that domain has to try to figure out if it really is or isn't. This is where antispam rules come in. Without DMARC, SPF & DKIM in place receiving mail servers will build up trust in different IP addresses, typically from vendors who go out of their way to prevent email abuse specifically to protect the reputation of those IP addresses. The receiving mail server my do a reverse DNS lookup to see if the hostname matched. They might see if the MX server used by domain is the same server sending the message along with a ton of other algorithmic hoops to make a good judgement. Even with DMARC, SPF and DKIM in place they may still use those rules.
DMARC, when strictly enforced with p=reject, let's you signal the receiving mail server that all mail claiming to be from your domain can be validated and if it can't be validated that message isn't from you and can be discarded. All that DMARC does is say that if you receive a message from this domain, it should pass either SPF OR DKIM for this domain as well.
So let's say, for example that I'm using both Google Workspace and Sendgrid for email for slimmeryetimbers.com. I'll setup Gmail for my main professional email for myself and my staff and I'll setup Sendgrid to send email from the website (responses to contact forms, etc).
For SPF it can be pretty simple, a single TXT record:
Both Google and Sendgrid publish DNS records with the IP address of their servers and keep them up to date, so adding that include is all that we have to do.
DKIM is more complicated. Google will provide you with a DKIM record to put under a subdomain that includes the public DKIM key. Once they have verified it's setup, they will sign every email sent from your account with the private key that only they know. When a receiving server gets the message they will see that the message has been signed by DKIM and it will point to the subdomain where the public key was stored. By following the instructions from the signature in the email and using the public key, they can verify that the message was actually signed by the private key and hasn't been tampered with.
If we then setup Sendgrid they will give you their own DNS records for DKIM that work for their own private key. In both of these situations, we never get our hands on the private key because we are using a 3rd party service that doesn't disclose it. Sendgrid issues 2 CNAME records that point to DKIM public key records on their DNS so that they can control them. They'll send out messages signed with one private key and then later switch to a different private key while the original key and it's public record is changed, transparently without you having to deal with it.
If we were to setup our own mail servers, we would have access to it though. It's also entirely possible for somebody with access at an email company to go steal the private keys of their customers and sell them, or just use them. It's possible for those services or our mail servers to be hacked/compromised and the keys be stolen. Built in rotation process helps with this. It's the same reason that Let's Encrypt issues secure certificates that expire after 90 days (sometimes less). Just key rotation.
Since multiple services can send on behalf of the domain, DKIM has to be configured for each of them. A service provider, like Google, will likely use the same private key across any of their servers that is sending your mail but Sendgrid won't have access to that private key so they have to setup their own. Same goes for any other services that send mail using your domain.
As a receiving mail server, they have no way of knowing how many different parties are legitimately sending email on behalf of your domain.
SPF is per domain. You setup a DNS record at the domain (or subdomain) level that lists all of the IPs (or a DNS reference to a list of those IPs) that are authorized to send email on your behalf...but, that breaks with mail forwarding and mailing lists.
SPF is much simpler, absolutely. DKIM requires a private key to sign outgoing messages from every mail server sending mail on your behalf.
1. There are a lot of domains out there and all of the people who own them aren't necessarily technical enough to setup DKIM on their mail server. Ideally those people are using some type of service. SPF is much simpler in this regard.
2. This is a rather famous story about it happening.
I have no idea how widespread the issue is today but I had to do some analysis on it when I worked for dmarcian ahead of the Anti-Phishing Working Group conference and we found that a significant percentage of email from known malicious IPs associated with reported phishing was passing DKIM. Key rotation removes the problem. Many services like ProtonMail and Sendgrid will set you up with 2 CNAME's for your DKIM keys so that they can rotate them for you automatically.
3. Domains send emails from multiple servers. Sometimes dedicated email servers, Google/Outlook, Sendgrid, email marketing tools, etc. A receiving system has no way to validate whether any of the tools sending email claiming to be from your domain are actually from your domain. The first time you look at a DMARC report for a domain that's been around for a while, you will typically see that 90% or more of the messages claiming to be from your domain weren't from you at all.
1. You have to set it up on every sending server. It's easier today but it wasn't always
2. You have to periodically rotate each of the keys that you setup because they can be cracked/stolen. Soon as somebody steals your key, they can impersonate anyone sending email from your domain.
3. Receiving email servers have no way of knowing if a message they received without a DKIM signature is supposed to include a DKIM signature, so simply not including one creates a scenario where receiving mail servers have to guess if the message was really from you.
DMARC isn't for sending email successfully, it's for preventing other people from impersonating your domain. Without it, there's nothing stopping anybody from sending an email saying it is from [email protected]. SPF tried. DKIM tried. Both of them had gaps.
When you use them together and have a DMARC policy that requires one of them or the other for successful delivery, it's the best current solution.
For many years, I watched my apps performance on AWS suffer in December around all the holiday sales. They might not snatch it back but they probably saturated it during high demand periods.
My guess is that the use here is similar to the reason AWS started as Amazon selling their excess capacity.
Between Tesla, SpaceX, X, Boring Co and Neuralink they probably want the capability internally for a lot of different applications.
If the whole data centers in space thing works out AND people keep protesting/blocking data center build outs on land SpaceX will eventually dominate the entire AI industry just based on escaping scarcity.
Might have to do something like that for a verse on the next Carolina Code Conference shirt. Been trying to figure out a good way to pull in cybersecurity.
Horizontal scaling for DB typically means distributing the write load across instances either via sharding or master-master. I could be wrong of course, but that’s how I read it.
MySQL is very good at master-master out of the box.
You can’t abolish the electoral college without standardized voting across all states. A single state or even county allowing frivolous or unverified voting can swing the entire election.
The EC provides a failsafe by capping the max impact of a single state in the event of an anomaly.
In other words, you can’t talk about abolishing the EC without a national voter ID law.
I think we take for granted how few databases ever outgrow vertical scaling needs.
Usually there will be one or two tables that grow at a dramatically faster rate than everything else and I have always found that those belong in a separate data store.
> 2. Synchronously updated materialized views (Sql Server calls them indexed views). These are incredible tools in complex data situations. I saw a project struggle with complex technical implementations that would be elegant, trivial and always correct with indexed views.
I believe you can do this with the pg-trickle extension.
We do a "Lyrical Code" T-shirt where people submit song lyrics as code in their favorite language and then we pull verses from different languages to make a polyglot t-shirt. Last year at the conference, attendees voted for Never Gonna Give You Up as this year's song.
We're a little low on PR's this year unfortunately though. I don't know if it's all the AI or something else but we usually have about a dozen entries by now and we've only got 2 for some reason this year.
Carolina Code Conference https://carolina.codes
Carolina Code Cast (tech podcast) https://blog.carolina.codes/podcast