Mining my mailbox for top email service providers(obem.be)
obem.be
Mining my mailbox for top email service providers
https://obem.be/2020/02/18/mining-my-mailbox-for-top-email-service-providers.html
109 comments
> The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.
I'm subscribed to some marketing emails, but I delete them after reading them. I save transactional emails.
I'm subscribed to some marketing emails, but I delete them after reading them. I save transactional emails.
A few years back, one of our (Twilio SendGrid's) developer evangelists created an open source Chrome extension to show the icon of the ESP who sent a message in Gmail.
The last big Gmail UI update broke it, but most of the ESP identification strategies probably still work: https://github.com/nquinlan/Email-Intelligence/blob/master/c...
The last big Gmail UI update broke it, but most of the ESP identification strategies probably still work: https://github.com/nquinlan/Email-Intelligence/blob/master/c...
This is nice. Shows a different approach to getting the ESP from the mail header.
This is a brilliant idea, if only it were updated for the latest gmail UI
Chrome extension author here: make a PR and I'll accept it and publish it.
https://www.inboxsdk.com/ lets you pretty easily manipulate the Gmail UI if you want to have a crack at it.
We (Streak) made the InboxSDK. Great for projects like this because we autoupdate so you can write the extension against our high level api and not have to worry about gmail changes going forward.
Let me know if we can help.
Let me know if we can help.
OMG, this sounds like a massive security risk. That nice developer could easily siphon off all the email to his backend server with this little extension. I install almost no extensions because they are too risky.
All Chrome extensions require trust. That should factor into your decision to install any of them.
This is no more risky than any other extension whose permissions include `https://mail.google.com/`. Anecdotally, that probably includes most extensions seeing as how most of the ones I come across request `<all_urls>` whether or not they need it (not that that's okay).
In any case it's open source, so you can download it, audit the code, and install it locally to prevent auto-updating.
This is no more risky than any other extension whose permissions include `https://mail.google.com/`. Anecdotally, that probably includes most extensions seeing as how most of the ones I come across request `<all_urls>` whether or not they need it (not that that's okay).
In any case it's open source, so you can download it, audit the code, and install it locally to prevent auto-updating.
Or use an email client that runs outside your browser, and reduce your threat surface considerably - yes, you still have to worry about other things you do in the browser, but compromising your email compromises all those too via password resets, which isn't the case for anything else if your password discipline is good.
Yes, Gmail makes it harder than it should be to use the email client of your choice. But that's not a problem with email clients. That's a problem with Gmail. It's far from the only one.
Yes, Gmail makes it harder than it should be to use the email client of your choice. But that's not a problem with email clients. That's a problem with Gmail. It's far from the only one.
Chrome sandboxes fetch requests from extensions and tells the user what permissions the app is requesting, so it’s not actually that bad (barring bugs in the sandbox)
What do you mean with "sandboxes fetch requests"? That's not a valid countermeasure at all...
The grandparent was worried about the extension stealing his email's contents.
The grandparent was worried about the extension stealing his email's contents.
Absolutely. If the extension injects JavaScript into the page to examine the email it can run a fetch in the context of that page.
I think the app store should warn that a page can send data to any website if it has permission to modify any page.
I think the app store should warn that a page can send data to any website if it has permission to modify any page.
It’s been a while, but I’m pretty sure the permissions dialog explicitly calls this out.
> The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.
I don't think that will be interesting for the author given where he's based. (I first assumed Belgium due to the domain's TLD, but it's actually Nigeria)
I don't think that will be interesting for the author given where he's based. (I first assumed Belgium due to the domain's TLD, but it's actually Nigeria)
[deleted]
It's missing basically every major email marketing provider.
* Salesforce * IBM * Contact Contact * Campaign Monitor * HubSpot * ActiveCampaign
etc etc
* Salesforce * IBM * Contact Contact * Campaign Monitor * HubSpot * ActiveCampaign
etc etc
It can be quite hard to detect. A lot of this services will not have an easily recognizable marker.
It can be for various reasons:
* They can use these services themselves
* They can support full customization of domains, including for the "message-id" and the "received" headers.
I'm working as SRE in a fairly large marketing provider (not sure in term of %, but we provide services for quite a few large brands sending several millions emails per day), and we do this full customization by default. There are other way to recognize us (other headers in the email or the pattern for the tracked url for example), but the one described in the article will not work.
It would also be interesting to be able to detect which MTA is used by these service providers. Is it an in-house one or an off the shelf one? The two off the shelf ones I know for mass sending are Momentum and PowerMTA which are both owned by Sparkpost but they might be others.
It can be for various reasons:
* They can use these services themselves
* They can support full customization of domains, including for the "message-id" and the "received" headers.
I'm working as SRE in a fairly large marketing provider (not sure in term of %, but we provide services for quite a few large brands sending several millions emails per day), and we do this full customization by default. There are other way to recognize us (other headers in the email or the pattern for the tracked url for example), but the one described in the article will not work.
It would also be interesting to be able to detect which MTA is used by these service providers. Is it an in-house one or an off the shelf one? The two off the shelf ones I know for mass sending are Momentum and PowerMTA which are both owned by Sparkpost but they might be others.
> The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.
Who on earth, that knows how to unsubscribe, would ever subscribe to spam?
Who on earth, that knows how to unsubscribe, would ever subscribe to spam?
Deliverability is a big marketing/selling point for all these providers & important selection criteria for their customers. So it'd be very interesting to see what % of emails for each provider landed in Inbox vs Promotions/Spam/Junk folder.
As someone currently dealing with this on multiple fronts: it's complicated. ESP's block out different ranges for different customers, dedicated vs shared IPs, etc. really hard to generalize.
Shameless plug: we had the same issue with our deliverability rates. We ended up creating a tool to automate the process of checking spam % vs. inbox % for various email subject/body combinations: https://vetter.monsoonyeti.com/
Interesting, but it’s hard to see the big utility. Most spam signals are based on behavior and reputation rather that content. Your email can look clean at first, but if a handful of gmail accounts mark it as spam then your whole network will be on the bozo list within seconds. Gmail in particular is capable of reclassifying messages, if it gets delivered but then your reputation falls, it will get moved to spam if the user hasn’t seen it yet, so initial deliverability is not final.
Maybe the other big operators are more content-based?
Maybe the other big operators are more content-based?
From repeated testing using multiple sender accounts and domains, we've noticed that Outlook seems to pay significant weight to the content of the email - possibly as much as or even more than the email/domain of the sender.
What you say certainly rings true about Gmail - email subjects/bodies that set off warning bells in Outlook seemingly pass through Gmail without a hitch if your account has a good sender reputation - and vice-versa. However, we've also noticed that even if you have a good sender reputation, Gmail can and will (automatically) spam your content if the wording is too spammy/phishy.
Yahoo seems to be midway between Outlook and Gmail, at least from what I've observed.
Anyway, do give the tool a spin and try out various types of emails. I'm sure you'll find the results interesting!
[EDIT] Oh, BTW - you can set the tool to use your own service as well! Obviously, this will be more accurate than using the default (which is an SES account we use).
What you say certainly rings true about Gmail - email subjects/bodies that set off warning bells in Outlook seemingly pass through Gmail without a hitch if your account has a good sender reputation - and vice-versa. However, we've also noticed that even if you have a good sender reputation, Gmail can and will (automatically) spam your content if the wording is too spammy/phishy.
Yahoo seems to be midway between Outlook and Gmail, at least from what I've observed.
Anyway, do give the tool a spin and try out various types of emails. I'm sure you'll find the results interesting!
[EDIT] Oh, BTW - you can set the tool to use your own service as well! Obviously, this will be more accurate than using the default (which is an SES account we use).
> So it'd be very interesting to see what % of emails for each provider landed in Inbox vs Promotions/Spam/Junk folder.
Are those percentages really a good measure of deliverability? I mean, a significant portion of “one million dollars for you” scams in my Gmail spam folder are from @gmail.com addresses; that doesn’t say anything about gmail.com to gmail.com deliverability.
Are those percentages really a good measure of deliverability? I mean, a significant portion of “one million dollars for you” scams in my Gmail spam folder are from @gmail.com addresses; that doesn’t say anything about gmail.com to gmail.com deliverability.
Well, it's the first barrier.
If you mess-up really bad you will have bad deliverability across the board.
The mistakes can be numerous:
* bad setup like broken DKIM
* bad sending parameters like opening too many connections by IP to a given provider
* bad content format like too many large images
* bad actual content
* bad recipient list management with lists containing lots of typos, none existent mailboxes, spam traps, etc
* miss-managing unsubscribes
* bad bounce handling
If this happens, you can have some if not all of your IPs blacklisted, maybe even your domain. In worst cases, a whole IPv4 range is blacklisted.
Then a second aspect to take into account is the open rate and the click through rate (people actually clicking a link in a marketing email), this one can vary depending on the company activities, and also the quality of their marketing content, but is generally in the range of a few percents at most. And then, you also have to take into account the actual impact in term of sells which will be even lower.
Deliverability is import, it's not rocket science, but it's a lot of small items various people at various stages (tech, creatives, marketers, etc) must be cautious about which makes it actually quite hard.
If you mess-up really bad you will have bad deliverability across the board.
The mistakes can be numerous:
* bad setup like broken DKIM
* bad sending parameters like opening too many connections by IP to a given provider
* bad content format like too many large images
* bad actual content
* bad recipient list management with lists containing lots of typos, none existent mailboxes, spam traps, etc
* miss-managing unsubscribes
* bad bounce handling
If this happens, you can have some if not all of your IPs blacklisted, maybe even your domain. In worst cases, a whole IPv4 range is blacklisted.
Then a second aspect to take into account is the open rate and the click through rate (people actually clicking a link in a marketing email), this one can vary depending on the company activities, and also the quality of their marketing content, but is generally in the range of a few percents at most. And then, you also have to take into account the actual impact in term of sells which will be even lower.
Deliverability is import, it's not rocket science, but it's a lot of small items various people at various stages (tech, creatives, marketers, etc) must be cautious about which makes it actually quite hard.
Gmail is somewhat a special case because email services are kinda forced to accept inbound Gmail, even though it's quite a large source of scams and phishing attempts today. But for others, they'll still readily get blacklisted for a little while here and there.
I was thinking about spam as I read the post too, since this is a problem I get to deal with almost daily. Marketo has a terrible signal-to-noise ratio and it can often end up blackholed on my network with nary a user noticing. Sendgrid is my next least-favorite email-as-a-service, and in fact I happened to have this article open in one tab and in another tab I had this morning's automated reports from some of my spam software showing that a pile of inbound messages from Sendgrid had tried to land at a list of nonexistent mailboxes -- someone was rolling through "andy@", "barbara@", "vijay@", "kenneth@", etc.
I have had very little trouble with Mailgun in this regard, so far, but that may only be because the spammers haven't fully adopted them yet.
As a sysadmin on the receiving side, I really loathe these services, they make my job significantly harder and more time consuming than it used to be.
I was thinking about spam as I read the post too, since this is a problem I get to deal with almost daily. Marketo has a terrible signal-to-noise ratio and it can often end up blackholed on my network with nary a user noticing. Sendgrid is my next least-favorite email-as-a-service, and in fact I happened to have this article open in one tab and in another tab I had this morning's automated reports from some of my spam software showing that a pile of inbound messages from Sendgrid had tried to land at a list of nonexistent mailboxes -- someone was rolling through "andy@", "barbara@", "vijay@", "kenneth@", etc.
I have had very little trouble with Mailgun in this regard, so far, but that may only be because the spammers haven't fully adopted them yet.
As a sysadmin on the receiving side, I really loathe these services, they make my job significantly harder and more time consuming than it used to be.
>Gmail is somewhat a special case.. forced to accept inbound Gmail
Unless I misunderstood, the person you replied to literally says he has emails from @gmail.com throwing in his Gmail spam filter. So not sure why you mention Gmail is special here (regardless if it's true or not).
Unless I misunderstood, the person you replied to literally says he has emails from @gmail.com throwing in his Gmail spam filter. So not sure why you mention Gmail is special here (regardless if it's true or not).
I know the results are biased given it's a sample from OP inbox but I'm surprised not so see https://postmarkapp.com/ here. I thought they were pretty popular.
They're just for transactional emails, not for marketing ones. They wouldn't show very much based on how he's measuring.
It seems this post is about transactional emails, not marketing ones. CF the first paragraph!
I've never heard of them before this comment. Maybe they're popular only in certain regions.
I must admit that I was a bit shocked when I realized that the author had used javascript for this task. At first I assumed that he was scripting some webmail interface but then I saw that he was using 'emailjs-imap-client'.
I understand that people use what they're familiar with and there seems to be a vibrant ecosystem for JS outside of the browser but it'll never not be weird seeing it take over what would once have been written in Perl or Python, especially since for a long time all I heard about Javascript was "well yeah it sucks but it's not like I have a choice if I want to script in the browser".
Software is eating the world, and the web is eating software.
I understand that people use what they're familiar with and there seems to be a vibrant ecosystem for JS outside of the browser but it'll never not be weird seeing it take over what would once have been written in Perl or Python, especially since for a long time all I heard about Javascript was "well yeah it sucks but it's not like I have a choice if I want to script in the browser".
Software is eating the world, and the web is eating software.
Modern JavaScript/TypeScript is fine. Especially when compared to Perl, of all languages.
I disagree. JavaScript is one of those languages who's default failure mode seems to be to pretend everything worked and just produce nonsensical results.
You're thinking of PHP. Modern Javascript has good error handling and is a pretty nice language to use.
As opposed to perl? :D
Modern Perl is fine too
I haven't used Perl in about 12 years, has it changed much since then?
Do you have any links to good examples/comparisons?
Do you have any links to good examples/comparisons?
chromatic's book, Modern Perl, has my highest recommendation:
http://modernperlbooks.com/books/modern_perl_2016/index.html
I've mentioned it before, but reading this book and applying its concepts (along with judicial use Damian Conway's Perl Best Practices) changed my view of Perl from "well, I guess I'll use it since it's here," to "I really enjoy using Perl".
Mark Jason Dominus's Higher-Order Perl was another enjoyable read, for its advice on using Perl as a more functional programming language.
http://modernperlbooks.com/books/modern_perl_2016/index.html
I've mentioned it before, but reading this book and applying its concepts (along with judicial use Damian Conway's Perl Best Practices) changed my view of Perl from "well, I guess I'll use it since it's here," to "I really enjoy using Perl".
Mark Jason Dominus's Higher-Order Perl was another enjoyable read, for its advice on using Perl as a more functional programming language.
To some, Raku is what Perl should have been by now: https://raku.org
[deleted]
unless you want any sort of 'system' interfacing like process execution control, etc.
It always annoys me when some general purpose utility or script is written in JavaScript because I refuse to install node on my computer.
What would have been a shell script in the 1980s, a Perl script in the 1990s, a Python script in the 2000s, is now a JS bundled with an outdated Chromium build aka an Electron app.
want to parse the plain text 'From: ' email headers on your laptop? Install this 500MB bundle.
I'm curious, why do you refuse to install node on your computer?
Maybe he hates bloat?
who cares, it's just a 100-line long script that probably won't be maintained or used anymore
I'm not sure I could count the number of 100-line long throwaway scripts that I wrote that are still being used in production systems before I realized how dangerous that idea is and stopped treating any code as throwaway code. If I show my code to another human, there's a chance it will end up in production.
The Pragmatic Programmer says to throw away prototypes, but in practice it always seems easier to click the merge button than to rewrite working code.
The Pragmatic Programmer says to throw away prototypes, but in practice it always seems easier to click the merge button than to rewrite working code.
And what utility did it provide while it was in production for the effort that was put into it? We had a nasty nasty 3k line Perl script “in production” at a former client for like 10 years. It was written in a couple of weeks by another outside vendor and did it’s job well enough. It had basically zero abstractions. Minimal dependencies. Did one thing and was well commented on intent. The investment vs utility made it last a long time. Yee, rewriting a decade later with minimal Perl knowledge wasn’t fun but the business rules had also evolved so we really used it as reference docs and did just fine
Reverse-survivor bias.
You aren't seeing the non-existence all the well engineered stuff you never had time to build "properly".
You aren't seeing the non-existence all the well engineered stuff you never had time to build "properly".
So?
If only 1 in 10 throwaway scripts survives, that can easily cost more in the long run than writing all 10 scripts in a sustainable way.
The open secret is that you almost always have time to do it properly. Doing it properly doesn't take that much more time, and most deadlines are self-imposed and artificial.
EDIT: Time isn't actually the most important cost. Not my code, but a coworker's throwaway script where he thought, "We'll just log this to a file for now" filled up a production DB server with log entries and took down a high-traffic e-commerce site for 6 hours. That easily cost more money than an entire career's worth of "doing it properly". There were obviously multiple mistakes that allowed that to happen, but removing the "we'll do this properly later" mistake would have prevented it.
If only 1 in 10 throwaway scripts survives, that can easily cost more in the long run than writing all 10 scripts in a sustainable way.
The open secret is that you almost always have time to do it properly. Doing it properly doesn't take that much more time, and most deadlines are self-imposed and artificial.
EDIT: Time isn't actually the most important cost. Not my code, but a coworker's throwaway script where he thought, "We'll just log this to a file for now" filled up a production DB server with log entries and took down a high-traffic e-commerce site for 6 hours. That easily cost more money than an entire career's worth of "doing it properly". There were obviously multiple mistakes that allowed that to happen, but removing the "we'll do this properly later" mistake would have prevented it.
It sounds like the problem was really about failing to document technical debt.
> Doing it properly doesn't take that much more time
If you don't have a cutting board for a small thing, surely you wouldn't put your cooking on hold to drive out to the kitchen supply shop to buy a cutting board. You'd just use a stack of cardboard or something. (And make a note to buy a proper cutting board later, of course.)
> Doing it properly doesn't take that much more time
If you don't have a cutting board for a small thing, surely you wouldn't put your cooking on hold to drive out to the kitchen supply shop to buy a cutting board. You'd just use a stack of cardboard or something. (And make a note to buy a proper cutting board later, of course.)
> It sounds like the problem was really about failing to document technical debt.
Technical debt is inevitable, and it will never be adequately documented. That's not really the situation we're talking about here.
This is more about stuff that was written with the mindset of "this will go into prod" versus "this will never go into prod". If your code is never going into prod, it's often acceptable to have huge, critical bugs--that's a very different thing from technical debt.
> If you don't have a cutting board for a small thing, surely you wouldn't put your cooking on hold to drive out to the kitchen supply shop to buy a cutting board. You'd just use a stack of cardboard or something. (And make a note to buy a proper cutting board later, of course.)
Well, sure, but this is a great example of why analogies aren't a good way to make arguments.
You're talking about using the right tools, not doing things properly. Sure, if I don't have the right tools, I'd do the best I can with the tools I have. But you'd better bet that a professional chef shouldn't just skip chopping the carrots because "it's just a throwaway dish, no need to do this properly" when there's a chance that it's going to be served to a food critic.
Technical debt is inevitable, and it will never be adequately documented. That's not really the situation we're talking about here.
This is more about stuff that was written with the mindset of "this will go into prod" versus "this will never go into prod". If your code is never going into prod, it's often acceptable to have huge, critical bugs--that's a very different thing from technical debt.
> If you don't have a cutting board for a small thing, surely you wouldn't put your cooking on hold to drive out to the kitchen supply shop to buy a cutting board. You'd just use a stack of cardboard or something. (And make a note to buy a proper cutting board later, of course.)
Well, sure, but this is a great example of why analogies aren't a good way to make arguments.
You're talking about using the right tools, not doing things properly. Sure, if I don't have the right tools, I'd do the best I can with the tools I have. But you'd better bet that a professional chef shouldn't just skip chopping the carrots because "it's just a throwaway dish, no need to do this properly" when there's a chance that it's going to be served to a food critic.
So what ? You care too much about these things.
It would be more readable in almost any other widespread language, and I'm saying that as someone who does a lot of JS.
This is quickly changing. Anecdotal evidence, of course, but I seem to come across a lot more scripts online in Node for some reason. And by 'scripts' I mean reproducers to various issues, such as triggering error state in an app I was working on.
At first, I thought it was crazy. It kinda makes sense though, JS seems to be growing extremely fast.
At first, I thought it was crazy. It kinda makes sense though, JS seems to be growing extremely fast.
As someone who has worked in Javascript basically since Javascript was invented, it's been "quickly changing" that entire time, and the majority of the changes have made things worse, not better. It used to be that you could trace your code through 15 layers of callback hell and eventually find the problem. Now there's 10 different syntactic sugars around callbacks, so you get to save a few characters of typing and then trace your bugs through 50 layers of callback hell, most of which are in some 0.0.1-alpha-versioned promise library which may actually just be a XSS attack. Improvement? I think not.
Just because more people are doing it, doesn't mean it's improving.
I'll believe things are going in a good direction when `{}.foo` throws an exception in all major JS implementations. But I'm not optimistic about that ever happening.
My only hope for JavaScript right now is that it might die because WebAssembly lets a reasonable language achieve dominance.
Just because more people are doing it, doesn't mean it's improving.
I'll believe things are going in a good direction when `{}.foo` throws an exception in all major JS implementations. But I'm not optimistic about that ever happening.
My only hope for JavaScript right now is that it might die because WebAssembly lets a reasonable language achieve dominance.
> I'll believe things are going in a good direction when `{}.foo` throws an exception in all major JS implementations. But I'm not optimistic about that ever happening.
It does. That's a SyntaxError.
It does. That's a SyntaxError.
Okay, please make a little effort to understand meaning rather than taking things completely literally.
Try one of the following:
The fact that JavaScript throws an exception on `{ foo: 'bar' }.foo` and not `({}).foo` isn't exactly a defense of the language.
[1] https://www.destroyallsoftware.com/talks/wat
Try one of the following:
> ({}).foo
undefined
> var a = {};
undefined
> a.foo
undefined
The fact that JavaScript throws an exception on `{ foo: 'bar' }.foo` and not `({}).foo` isn't exactly a defense of the language.
[1] https://www.destroyallsoftware.com/talks/wat
({}).foo returning undefined is exactly what I'd expect. That's a basic part of how accessing properties in Javascript works, and wanting it to work differently is like wanting NullPointerException generation in Java to work differently.
> Now imagine hundreds of lines of code and a bunch of asynchronous callbacks occur between the last two lines, and imagine what that does for your debugging experience
That's what using typed signatures for your objects is for. It's not even substantially different from having, say, Java with an object with properties that can be nullable or Optional.empty().
> Now imagine hundreds of lines of code and a bunch of asynchronous callbacks occur between the last two lines, and imagine what that does for your debugging experience
That's what using typed signatures for your objects is for. It's not even substantially different from having, say, Java with an object with properties that can be nullable or Optional.empty().
> ({}).foo returning undefined is exactly what I'd expect. That's a basic part of how accessing properties in Javascript works,
This is an excellent and succinct explanation of why JavaScript is a badly-designed language, and why I use better designed languages when possible.
If the basic properties "work" when it makes no sense for them to work, your language is broken.
> That's what using typed signatures for your objects is for.
Explain more?
This is an excellent and succinct explanation of why JavaScript is a badly-designed language, and why I use better designed languages when possible.
If the basic properties "work" when it makes no sense for them to work, your language is broken.
> That's what using typed signatures for your objects is for.
Explain more?
> Explain more?
With Typescript or even just JSdoc and a good IDE, you can do:
With Typescript, you can turn on full strictness and make these build-rejecting compile-time errors, too.
With Typescript or even just JSdoc and a good IDE, you can do:
const a: {propertyA?: true; propertyB?: 'cats';} = {};
// or
/**
* @type {object}
* @property {true=} propertyA
* @property {'cats'=} propertyB
*/
const a = {};
With Typescript, you can turn on full strictness and make these build-rejecting compile-time errors, too.
Just because more people are using it does not mean it is more readable. Footguns about in JS and it is remarkably easy to read the same code over and over again and not see them because of it.
Readable languages are the kinds where not only do you understand what they're trying to do at a quick glance, but also what they actually do.
Readable languages are the kinds where not only do you understand what they're trying to do at a quick glance, but also what they actually do.
The biggest problem I have with JS in term of readability is IIFE. I know what it does, but it just looks ugly when you have hundreds of them in a program. The "semicolon in front" is even worse, needless to say.
There gotta be a better way to design syntax for an async language.. right?
There gotta be a better way to design syntax for an async language.. right?
Wait, why do you need IIFE in post-ES6 JS? Just use let/const.
Well, OP's example is using it, but I think it's for top level async/await (not a JS dev here)?
Temporary code is permanent code in many, many cases.
I started (but quickly got bored of) writing a Python version here: https://gist.github.com/JosephRedfern/79314fdf1875166e084897... if anyone wants to pick it up.
Isn't JavaScript much faster than Python these days?
Was Python faster than javascript a long time ago? I'm not super old and javascript has always been way faster in my mind, but it's fine. Python has many great libraries for doing heavy computations very quickly.
Javascript is quite neat today. I play a lot with map/reduce and async/await and it's great.
As as a hardcore Perler and JS eng of 20 years - yes, Perl or Python would've been the best tool for the job. Perl in particular due to its regex readability.
Semi-related: emails can often give away more information than you might intend. At one of my previous employers, it was quite easy to get a general idea of how internal development was progressing “from the outside” by keeping track of who was testing which version of a prerelease OS via proxy of the information it would leave in the emails they sent.
I've identified companies as connected to each other by enumerating domains on an azure tenancy.
There’s some really interesting things you can pull out of your email. Having done some analysis myself it’s actually fairly easy to export data from Gmail to do an analysis.
This is from a few years ago, but I bet it’ll still do the trick:
https://austingwalters.com/analyzing-email-data/
Everything is in python:
https://github.com/lettergram/Email_Analysis
Regardless, I encourage everyone to do an analysis of their emails. Cut out the junk, set reminders on when to check, etc. it’s both enlightening and potentially horrifying to discover trends.
This is from a few years ago, but I bet it’ll still do the trick:
https://austingwalters.com/analyzing-email-data/
Everything is in python:
https://github.com/lettergram/Email_Analysis
Regardless, I encourage everyone to do an analysis of their emails. Cut out the junk, set reminders on when to check, etc. it’s both enlightening and potentially horrifying to discover trends.
The methodology used excludes a number of services using SparkPost. For e.g. the excerpt below of an Intercom header. It's a challenging problem to solve given the complexity of email delivery and appreciate this was a fun project to take on.
Return-Path: <[email protected]> Received: from mta-216-35.sparkpostmail.com (mta-216-35.sparkpostmail.com. [147.253.216.35]) by mx.google.com with ESMTPS id t13si7000210pgg.534.2020.02.03.08.37.30 for <XXXXXX> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Feb 2020 08:37:30 -0800 (PST)
Return-Path: <[email protected]> Received: from mta-216-35.sparkpostmail.com (mta-216-35.sparkpostmail.com. [147.253.216.35]) by mx.google.com with ESMTPS id t13si7000210pgg.534.2020.02.03.08.37.30 for <XXXXXX> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Feb 2020 08:37:30 -0800 (PST)
This website presents as a 23-chacter-wide skinny column in mobile vew.
Very interesting thanks for sharing. initially I was surprised that HubSpot didn’t make the list, until I realized that there is a nice intersection here with what you observed and the target businesses of the Email services.
For example MailChimp Targets small B2C companies while Hubspot targets small B2B business.
I’m guessing the op isn’t a small B2B business hence the lack of hubspot in the list.
For example MailChimp Targets small B2C companies while Hubspot targets small B2B business.
I’m guessing the op isn’t a small B2B business hence the lack of hubspot in the list.
Nitpicking: in the final chart with merged providers, the 'mandrillapp.com' section and 'mailchimp' section are still shown as separate.
Is this intentional? I assumed the opposite, given that:
> Mandrillapp.com is the transactional email service for Mailchimp. It used to be a standalone service until it became deeply integrated into Mailchimp.
Is this intentional? I assumed the opposite, given that:
> Mandrillapp.com is the transactional email service for Mailchimp. It used to be a standalone service until it became deeply integrated into Mailchimp.
[Nitpick] Instead of this:
;(async function () {
try {
// All your code
} catch (e) {
console.log(e)
}
})()
;(async function () {
// All your code
})().catch(console.error)
I don't think you can assume console.error is bound. Also it's good to avoid the possibility of blindly passing through unintended extra arguments (not an issue with catch, but is with many methods such as forEach etc).
catch(err => console.error(err))The point of async/await is to enable asynchronous code to participate in the same patterns synchronous code does, instead of being coerced into a promise-style interface. Using try/catch is part of that, and it's good practice here. If you're not going to take advantage of it, why bother with async/await at all, especially when using it at top level requires a workaround anyway?
Either way, I find the syntax of both to be extremely ugly. Arguably this may simply be me not getting JavaScript.
Starting a line with ; is just plain weird and the last bit })() seems like German, where all the stuff that you technically not need to understand the sentence is at the end.
Maybe someone can explain with JavaScript developers love anonymous functions so much. Why not just name your function and make the whole thing a bit more readable.
Starting a line with ; is just plain weird and the last bit })() seems like German, where all the stuff that you technically not need to understand the sentence is at the end.
Maybe someone can explain with JavaScript developers love anonymous functions so much. Why not just name your function and make the whole thing a bit more readable.
Not all JavaScript devs.
Love the language, hate the popularity of anonymous functions and non-descriptive variable names.
My production code as a sample. http://designbymobi.us/wp-content/uploads/2020/02/rabbitmq-c...
Love the language, hate the popularity of anonymous functions and non-descriptive variable names.
My production code as a sample. http://designbymobi.us/wp-content/uploads/2020/02/rabbitmq-c...
Wow those pie charts are almost impossible to read! I've never seen such a small font in a chart that readers are expected to understand easily.
I had actually just tweeted about my woes dealing with email going to spam for no good reason from my GSuite account.
https://twitter.com/dustywes/status/1228733404826333185?s=21
I’m planning on doing a more thorough write-up on my solution (it’s convoluted), but my tl;dr is that email seems woefully broken at the moment. Hope my experience can be helpful for others dealing with the same issue.
https://twitter.com/dustywes/status/1228733404826333185?s=21
I’m planning on doing a more thorough write-up on my solution (it’s convoluted), but my tl;dr is that email seems woefully broken at the moment. Hope my experience can be helpful for others dealing with the same issue.
[deleted]
That’s really nice analysis. Mailchimp is the biggest one as I guessed. Don’t know if you count the spam folder since some marketing emails could be automatically sent there.
Not quite sure how you got that from the graphs. Top three:
Sendgrid: 30.2%
SES: 16.8%
Mailchimp: 16.8%
Sendgrid: 30.2%
SES: 16.8%
Mailchimp: 16.8%
Seeing Amazon's SES with such a large share is no surprise, but it is a little disheartening knowing about this limitation:
"Amazon SES only supports open tracking over HTTP domains."
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/config...
Yes, big webmail providers fetch the tracking pixels for you and serve them to your browser over HTTPS, but not every mail provider does that, sometimes for good reason, leading to problems like this:
https://protonmail.com/support/knowledge-base/connection-not...
"Amazon SES only supports open tracking over HTTP domains."
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/config...
Yes, big webmail providers fetch the tracking pixels for you and serve them to your browser over HTTPS, but not every mail provider does that, sometimes for good reason, leading to problems like this:
https://protonmail.com/support/knowledge-base/connection-not...
I'm wondering the legality of this...
The emails in your mailbox aren't yours to do with as you please - they're all still owned by their original sender, who has given you an implicit license to read them, and store them for later re-reading.
You don't really have rights to datamine, aggregate, train AI on, post stats about, etc. the data in those emails. Its a violation of the rights of the original sender to do it without permission.
The emails in your mailbox aren't yours to do with as you please - they're all still owned by their original sender, who has given you an implicit license to read them, and store them for later re-reading.
You don't really have rights to datamine, aggregate, train AI on, post stats about, etc. the data in those emails. Its a violation of the rights of the original sender to do it without permission.
Are you serious? Unless those emails contain work protected by copyright, they're not "owned" by their senders. And I doubt that metadata falls in this category.
So if I send an email to a gmail address, it is illegal for Google to datamine it? Even though I’d love this to be true, somehow I doubt it really is the case.
I disagree. They relinquished all rights when they hit send.
The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.