Fastmail breaks UI in production(twitter.com)
twitter.com
Fastmail breaks UI in production
https://twitter.com/licyeus/status/1953438985381974493
52 comments
I don't understand the urge to do public shaming instead of first reaching out to support. And the OP is so rude.
I think it is a result of the impersonal "contact us" intake forms companies have all moved to. You have no indication that you aren't just screaming into the wind. There is no personal touch. So you take to social media where your are sure at least someone hears you. It also scratches the justice itch: if the company doesn't pay attention it looks bad in public and you get some vindication for being ignored.
I'm not saying it's a good or bad thing to do, but I understand it.
I'm not saying it's a good or bad thing to do, but I understand it.
> It also scratches the justice itch: it the company doesn't pay attention or looks bad in public and you get some vindication for being ignored.
This is an interesting point. There is some satisfaction from the likes, the comments, and the assurance that _someone_ is seeing your frustration even if the company does nothing.
This is an interesting point. There is some satisfaction from the likes, the comments, and the assurance that _someone_ is seeing your frustration even if the company does nothing.
I am the OP.
I admit my second message was terse, but the correct social media response for a critical outage is "we're looking into it". Not "if you want support, go here".
Like I said, I do not have time to hunt for whatever support channels exist and file a ticket. I pay $50 a year so they can deliver a working product, including triaging their own issues.
I admit my second message was terse, but the correct social media response for a critical outage is "we're looking into it". Not "if you want support, go here".
Like I said, I do not have time to hunt for whatever support channels exist and file a ticket. I pay $50 a year so they can deliver a working product, including triaging their own issues.
> the correct social media response for a critical outage is "we're looking into it". Not "if you want support, go here".
I strongly disagree.
For those reading who may not be on X, here's Fastmail's response:
> Hello Andrew! Can you please contact our support team so we can look into this for you? http://fastmail.com/support
That seems entirely appropriate to me, for several reasons:
I strongly disagree.
For those reading who may not be on X, here's Fastmail's response:
> Hello Andrew! Can you please contact our support team so we can look into this for you? http://fastmail.com/support
That seems entirely appropriate to me, for several reasons:
- it's extremely unlikely that the person managing their social media profiles is a technical expert
- their contact page likely feeds into a ticketing system, which means they can track the issue and actually make sure to respond to you
- it's entirely possible that response was automatedIt doesnt take a technical expert to create a ticket and attach a screenshot, nor is it unexpected that automation do all that.
There are many reasons why reaching out to support will be more effective. Logging may be tied to accounts, support systems may upload client side logs where appropriate, an event may be tracked to determine a point in time when you had the problem so relevant logs can be checked.
Their messaging probably could have been better here to say "we're looking into this. If you have time, please file a support request".
On Fastmail.com, the support request is two clicks away, under the big ? button in the upper right and "Contact Support". I'm not sure how much time you have but it took me less than 1 minute to find this button and submit from my logged in account.
Their messaging probably could have been better here to say "we're looking into this. If you have time, please file a support request".
On Fastmail.com, the support request is two clicks away, under the big ? button in the upper right and "Contact Support". I'm not sure how much time you have but it took me less than 1 minute to find this button and submit from my logged in account.
This. I am a paying customer. Fastmail's support is stellar. My first port of call is their official customer support channel. Why should I roll the dice on social media, when I have a direct line of support???
This time, I was on the support ticket window, but the penny dropped and I realised it was a service outage / degradation. So I decided to hold back for a bit before filing a ticket.
Based on experience with their service, I trust that their people know what they are doing, and are already on the job.
This time, I was on the support ticket window, but the penny dropped and I realised it was a service outage / degradation. So I decided to hold back for a bit before filing a ticket.
Based on experience with their service, I trust that their people know what they are doing, and are already on the job.
I am a fastmail user and I think fastmail deserves this. I don't know who asked them to change the UI.
Why do they deserve punishment for iterating on their product?
They don’t. But as a Fastmail customer, I am _really_ tiring of how much they iterate on their UI/UX without notice and seemingly decreasingly solid testing. They push stuff to prod that doesn’t hit beta (at least the public beta environment) regularly. I just stopped using the beta environment because they only test “big” stuff there, it seems.
But for the past year they’ve been nitpicking their GUIs and UX and it’s maddening. They take away or move intuitive and accessible features with no replacement, or end up making you do more clicking/tapping to get there.
I have sent feedback to support many times. Sometimes they revert changes within a week, and sometimes it’s just the canned “we strive to always enhance our customer experience” copy.
Either way, it’s disruptive and unwelcome. Their 2021 era UI was perfect. When they started announcing partnerships with other companies I think they also ended up with more users that are consumers and that creates tension with prosumers and professionals.
Fastmail used to be a haven for people who cared about email and privacy, and many of us chose them based on our own professional experiences running email infrastructure. But now it’s quickly “consumerizing” and their designers clearly have the “to enhance is to remove” mentality.
And as a reseller — don’t get me started on their new billing system and model, which is less reliable than what came before, less flexible, and was launched with no real supporting documentation.
/rant.
The service is still fantastic though in terms of speed, infrastructure, etc. I trust their technology a lot. Their UX/UI people need a time out. Whoever replaced their “Moonpig” billing system with Paddle did the users a disservice.
But for the past year they’ve been nitpicking their GUIs and UX and it’s maddening. They take away or move intuitive and accessible features with no replacement, or end up making you do more clicking/tapping to get there.
I have sent feedback to support many times. Sometimes they revert changes within a week, and sometimes it’s just the canned “we strive to always enhance our customer experience” copy.
Either way, it’s disruptive and unwelcome. Their 2021 era UI was perfect. When they started announcing partnerships with other companies I think they also ended up with more users that are consumers and that creates tension with prosumers and professionals.
Fastmail used to be a haven for people who cared about email and privacy, and many of us chose them based on our own professional experiences running email infrastructure. But now it’s quickly “consumerizing” and their designers clearly have the “to enhance is to remove” mentality.
And as a reseller — don’t get me started on their new billing system and model, which is less reliable than what came before, less flexible, and was launched with no real supporting documentation.
/rant.
The service is still fantastic though in terms of speed, infrastructure, etc. I trust their technology a lot. Their UX/UI people need a time out. Whoever replaced their “Moonpig” billing system with Paddle did the users a disservice.
I’ve been a Fastmail user for years, but I don’t know that I’ve had the same experience. I think I’ve noticed a few UI polishes, but nothing that broke my workflow. What sorts of things have you ran into specifically?
They don't deserve punishment. But they should understand that this is not just "their product" but it is also my tool. Tools like this do not need to change and absolutely should not change without there being prior notification. Yes, in many ways the UI changes are trivial. They don't fundamentally change what's possible. But my keyboard never changes on me without input. My workbench doesn't rearrange itself without my input. If they want this to continue to be my tool (I've been happy to pay them for it) it needs to respect my time and attention.
I'm not a Fastmail user, but... if I were paying a company to read my emails and they shipped a new build that broke that feature due to a lack of basic testing, I'd be publicly "shaming" them.
I don't know. I'm always confused when social media accounts ask to report issues over different channels when it's so much easier to just reply to customer and ask for additional (non PII) information there. In this case it's unlikely that this was the only or even first report about this issue. So why place the initial burden on a paying customer? If your social media is just a marketing channel, at least make that clear so i don't even bother reporting issues that route.
Companies make it clear how to report issues by having a separate support system. If you clicked a "Support" link and it took you to Twitter, I could better understand your confusion.
As for using social media to take issue reports: What will you do when you need PII or have to reassign the issue or reassign part of the issue and those people need to be able to contact the user?
"Why place the initial burden on a paying customer?" Because it creates a better service for everyone to have a known way of doing things.
As for using social media to take issue reports: What will you do when you need PII or have to reassign the issue or reassign part of the issue and those people need to be able to contact the user?
"Why place the initial burden on a paying customer?" Because it creates a better service for everyone to have a known way of doing things.
> If your social media is just a marketing channel, at least make that clear so i don't even bother reporting issues that route
Is this snippet from their Twitter bio clear?
"If you need assistance, please submit a ticket: http://fastmail.com/support"
Is this snippet from their Twitter bio clear?
"If you need assistance, please submit a ticket: http://fastmail.com/support"
Actually, yes. This is exactly what I would've asked for. I didn't see that.
:hugops:
"Things Happen", in production, to the best of us (and FM's pretty damned good at their job). Pretty sure someone's pager duty has been going off like mad.
A little over half an hour ago, the mail UI broke for me on Android, and then I panicked and went to desktop web and it broke there too. Also on different networks.
As far as I can tell, stuff from their CDN is 404-ing, and a JSON api POST request appears to be going in infinite loop with 200 OKs.
The webmail piece seems to be borked... Calendar, Files, Notes etc. are at least rendering.
"Things Happen", in production, to the best of us (and FM's pretty damned good at their job). Pretty sure someone's pager duty has been going off like mad.
A little over half an hour ago, the mail UI broke for me on Android, and then I panicked and went to desktop web and it broke there too. Also on different networks.
As far as I can tell, stuff from their CDN is 404-ing, and a JSON api POST request appears to be going in infinite loop with 200 OKs.
The webmail piece seems to be borked... Calendar, Files, Notes etc. are at least rendering.
Ok, I just checked and things seem to be rendering again. I just replied to an email via the webmail UI.
Back to business as usual.
Back to business as usual.
Been working fine for me all morning. I don't use the reader pane UI option though.
The design refresh is actually one of the few I don't hate, I got it yesterday and it is decent. But the blank emails thing started today and this... is a problem.
What the hell is that with OP being so rude?
And following FastMail's reply
> Hello Andrew! Can you please contact our support team so we can look into this for you? fastmail.com/support
They say:
> Don't have time. Consider my tweet the bug report.
Sorry but this asshole behavior. Bugs happen. No need to do public shaming and being rude to the company for that.
And following FastMail's reply
> Hello Andrew! Can you please contact our support team so we can look into this for you? fastmail.com/support
They say:
> Don't have time. Consider my tweet the bug report.
Sorry but this asshole behavior. Bugs happen. No need to do public shaming and being rude to the company for that.
I disgree. Other people were saying it's broken too. Reasonable company behavior here would be 1 tweet "we'll look into it", and then either "we reproduced and are working on a fix" or "looks good to us please contact support so we can investigate your particular issue". But there's no reason to initially make users jump through hoops.
It's in their status page and it's tagged as a partial outage: https://fastmailstatus.com/cme1fq7ej002dh0iu6z8pey4f
Personally I don't really use their web interface, but I tried it now and it all works just fine (on both prod and beta).
Personally I don't really use their web interface, but I tried it now and it all works just fine (on both prod and beta).
When I experienced this issue it was not on their status page, so I assume it wasn't for the original tooter either.
That response is probably standard support procedure. I think it’s pretty reasonable behavior. Could it be better? Sure. Is the person handling Twitter interactions doing their job? Also sure.
Not wasting time on someone else's process isn't being anything negative. Expecting other people's time for free is indeed being an arse.
Last time I checked fastmail was a paid service. OP has every right to expect it to be working.
Sure but Fastmail's failure doesn't give people license to be assholes.
There is far too much assholery in the world. It's never OK.
There is far too much assholery in the world. It's never OK.
This comment section's been illuminating to see who has probably never worked public-facing support or service industry.
There's no amount of money you can pay to make this behavior not shitty. Shitty behavior is never a good look, but sometimes it's understandable. If you refrain from being shitty, you won't have to worry about whether or not it's understandable.
Also, the only reason that someone can be shitty and get results is because other people aren't. (In this case, "submitting" a bug report via Twitter and still getting a resolution is possible because other people reported it through the proper channels.)
There's no amount of money you can pay to make this behavior not shitty. Shitty behavior is never a good look, but sometimes it's understandable. If you refrain from being shitty, you won't have to worry about whether or not it's understandable.
Also, the only reason that someone can be shitty and get results is because other people aren't. (In this case, "submitting" a bug report via Twitter and still getting a resolution is possible because other people reported it through the proper channels.)
It didn't break anything for me. However, I am failing to understand the point of this design "refreshment". There is nothing new out there, just minor UI changes with no purpose?
It makes access to the calendar, contacts, etc., one less click, which I think is nice. Would have preferred all that was set to buttons at the top instead of a thin side bar though.
I think that's how it was before? They just added the icons on the left side but you always had the menu on top?
Same. I don't hate it, but I do hate that things changed out from underneath me without any notice, opt-in period, or ability to go back.
In a few days, when the frenzy dies down, I will make a feature request ticket asking them to do this next time. Please do the same.
I opened a support case at 9:04 AM EDT. So far, no response yet.
Maybe they don’t realize how important you are. This is a failure in their VIP program, imo they should expedite that over a bug that effects all users.
For what it's worth the CEO responded to my support ticket once and I felt like a VIP.
Ha, cool. I imagine an an age of AI and LLMs, a lot of people are going to feel special at different points despite the cost of that going to nearly zero.
The bug (at least for me) isn't present on the beta version (beta.fastmail.com)
I'm glad I came here to check to see if anyone else had the same issue as I've had!
As soon as I realised that both my webmail and my phone app were buggered, it was probably not just a Me Problem.
As soon as I realised that both my webmail and my phone app were buggered, it was probably not just a Me Problem.
Having the same problems.
It started out as not being able to search, but the situation is quickly deteriorating and now I'm unable to open pretty much any email message.
Some content seems to briefly show up and then it quickly disappears and after that, it's as if cache has been invalidated and you can't get back into it.
It started out as not being able to search, but the situation is quickly deteriorating and now I'm unable to open pretty much any email message.
Some content seems to briefly show up and then it quickly disappears and after that, it's as if cache has been invalidated and you can't get back into it.
Luckily for me the Drafts folder's showing, so I was able to send (well, I assume it's sent) the Single Most Important Email I need to send today, which I'd spent about half an hour getting right just as the interface imploded...
While the web UX is borked, mobile app is not impacted if you can fallback to that until they fix the web.
This doesn't actually work. The mobile app is just a wrapper around the web interface.
I thought apple didn't allow this?
Edit: apparently "hybrid apps" using webviews are allowed as long as they're not "thin wrappers" for websites and provide meaningful functionality. See also: the capacitor framework.
Edit: apparently "hybrid apps" using webviews are allowed as long as they're not "thin wrappers" for websites and provide meaningful functionality. See also: the capacitor framework.
[deleted]
"Breaks UI in production". The most dramatic way to say there was a bug.
Fastmail is great-- solid-- for about 10 years for me. There's a lot that I prefer over Google Mail.
There are a few different threads on this and now that things are in a stable place I'm going to cross post this to all of them!
The larger context is that we're making a major change to how we create IDs for email and mailboxes over the JMAP protocol. The old IDs are a UUID for mailboxId and the first 25 chars of the sha1 of the message for the emailId, prefixed by an 'M'. The new IDs are the createdmodseq for the mailbox prefixed by a 'P' (these are pretty short for most users) and a reverse counter of nanoseconds of the message internaldate (delivery time) for the emailId. This gives good storage density for offline and good data locality in databases for the email listings.
You can see all the code for that in a handful of merge requests in the public cyrus-imapd repository on github at https://github.com/cyrusimap/cyrus-imapd/
Over the past few weeks, I've been helping out with the last bits of code modification, largely the changes on https://github.com/cyrusimap/cyrus-imapd/pull/5539 if you're interested.
This morning we rolled out a build which we'd tested extensively on our staging and staff servers, but missed that for older v19 mailboxes which hadn't been upgraded to v20, the code to check if messages belonged in a thread incorrectly marked them all as missing.
This made MOST emails appear missing for most customers, clearly a very bad situation.
We immediately rolled back, but in the hurry missed that an unrelated change to correct subject matching for some languages (Japanese users had reported the issue, but possibly others as well) had changed the thread version, so new threads then had failed reads (making some, though many fewer, messages appear blank in the UI). There were about 50 million attempts to read those values over 15,000 users, because our UI was keeping on retrying thinking it was just a temporary synchronisation issue because the previous request told it there was a Thread to fetch data for. Ouch. https://github.com/cyrusimap/cyrus-imapd/pull/5527 contains those changes.
Anyway, since the only difference between the old and new records was normalisation of subjects, I wrote a tiny patch to let the old code read the newer records and just deployed that, which made all the emails re-appear for everyone again. This is the one bit of code from all this which isn't in a public repo, but it's two lines of: if (version == 2) version = 1;
Meanwhile, the real bug is fixed https://github.com/cyrusimap/cyrus-imapd/pull/5553 And a test has been written to prove it: https://github.com/cyrusimap/cyrus-imapd/pull/5554
But we'll wait until Monday to upgrade again, when we have fresh eyes available to watch that it's OK.
...
P.S. this is almost entirely unrelated to the UI changes. The underlying reason we're doing these changes IS related to UI changes, it's there to make offline mode use storage more efficiently on your device because the IDs are smaller and provide better data locality, but the timing is purely coincidental. The Cyrus changes have been done almost exclusively by the team in the USA and the UI changes by the team in Australia, and our deploy timelines were not synchronised.
The larger context is that we're making a major change to how we create IDs for email and mailboxes over the JMAP protocol. The old IDs are a UUID for mailboxId and the first 25 chars of the sha1 of the message for the emailId, prefixed by an 'M'. The new IDs are the createdmodseq for the mailbox prefixed by a 'P' (these are pretty short for most users) and a reverse counter of nanoseconds of the message internaldate (delivery time) for the emailId. This gives good storage density for offline and good data locality in databases for the email listings.
You can see all the code for that in a handful of merge requests in the public cyrus-imapd repository on github at https://github.com/cyrusimap/cyrus-imapd/
Over the past few weeks, I've been helping out with the last bits of code modification, largely the changes on https://github.com/cyrusimap/cyrus-imapd/pull/5539 if you're interested.
This morning we rolled out a build which we'd tested extensively on our staging and staff servers, but missed that for older v19 mailboxes which hadn't been upgraded to v20, the code to check if messages belonged in a thread incorrectly marked them all as missing.
This made MOST emails appear missing for most customers, clearly a very bad situation.
We immediately rolled back, but in the hurry missed that an unrelated change to correct subject matching for some languages (Japanese users had reported the issue, but possibly others as well) had changed the thread version, so new threads then had failed reads (making some, though many fewer, messages appear blank in the UI). There were about 50 million attempts to read those values over 15,000 users, because our UI was keeping on retrying thinking it was just a temporary synchronisation issue because the previous request told it there was a Thread to fetch data for. Ouch. https://github.com/cyrusimap/cyrus-imapd/pull/5527 contains those changes.
Anyway, since the only difference between the old and new records was normalisation of subjects, I wrote a tiny patch to let the old code read the newer records and just deployed that, which made all the emails re-appear for everyone again. This is the one bit of code from all this which isn't in a public repo, but it's two lines of: if (version == 2) version = 1;
Meanwhile, the real bug is fixed https://github.com/cyrusimap/cyrus-imapd/pull/5553 And a test has been written to prove it: https://github.com/cyrusimap/cyrus-imapd/pull/5554
But we'll wait until Monday to upgrade again, when we have fresh eyes available to watch that it's OK.
...
P.S. this is almost entirely unrelated to the UI changes. The underlying reason we're doing these changes IS related to UI changes, it's there to make offline mode use storage more efficiently on your device because the IDs are smaller and provide better data locality, but the timing is purely coincidental. The Cyrus changes have been done almost exclusively by the team in the USA and the UI changes by the team in Australia, and our deploy timelines were not synchronised.
Thanks for the transparency + analysis.
I've been a happy Fastmail customer for 11+ years and will continue to be so. Everyone wrote that their experience with FM support has been good, but I've been even luckier: until today I never needed support. One bug per decade is a great record.
I've been a happy Fastmail customer for 11+ years and will continue to be so. Everyone wrote that their experience with FM support has been good, but I've been even luckier: until today I never needed support. One bug per decade is a great record.