Ask HN: How relevant is cross-browser testing anymore?
96 comments
> Do we really need to test our code on 50ish Chrome versions, 25is Firefox versions and so on.
Probably not, unless you're a really big outfit with a vast user base. But you should test on at least a recent Chrome version and a recent Firefox version, and probably Safari.
And unless your product is irrelevant to mobile users, on both iOS (Safari) and Android (Chrome, Firefox), with both phone and tablet form factors.
Probably not, unless you're a really big outfit with a vast user base. But you should test on at least a recent Chrome version and a recent Firefox version, and probably Safari.
And unless your product is irrelevant to mobile users, on both iOS (Safari) and Android (Chrome, Firefox), with both phone and tablet form factors.
Painful as it is to admit, even Firefox is not worth testing. In our B2B site, more people are using IE11 than FF.
https://gs.statcounter.com/browser-market-share at ~3% of overall share / 7% desktop-only share it's hard to justify putting extra effort.
https://gs.statcounter.com/browser-market-share at ~3% of overall share / 7% desktop-only share it's hard to justify putting extra effort.
It's not just about the number of your users who are directly using Firefox; it's also about ensuring your code isn't inadvertently depending on non-standard Chrome quirks, and promoting standards compliance, portability, robustness, and such-like goodness on the web.
(Test IE11 as well, by all means, if it's relevant to your users; but it's much less relevant to the future of the web.)
(Test IE11 as well, by all means, if it's relevant to your users; but it's much less relevant to the future of the web.)
i’m just being silly when I ask this, but if you have data that most of your users, use Chrome, who cares if quirks that work in chrome don’t work and other browsers, if your users aren’t using those other browsers??
Maybe most of your users use Chrome because they don’t have a choice, i.e. the website is broken in other browsers.
Every time they bump into something they dislike about Chrome, they’ll remember who is forcing them to use it. When an alternative comes along and works in their browser of choice, they’ll switch and advocate for it.
Every time they bump into something they dislike about Chrome, they’ll remember who is forcing them to use it. When an alternative comes along and works in their browser of choice, they’ll switch and advocate for it.
We should learn from what happened with Internet Explorer. It had a much larger share than Chrome has now, and a lot of people designed for IE first. The consequences were too many to write down in this comment. Let's just say there were some regrets :-)
We don't know how the situation will look like in the near future. If a large enough portion of the users switch to a different browser, it quickly becomes an issue.
It is in my opinion part of what companies should include in their risk assessments. Looking at the general trends, I would not be surprised if at least EU governments and institutions will ban Chrome due to privacy and data security. That will probably lead to others switching as well.
We don't know how the situation will look like in the near future. If a large enough portion of the users switch to a different browser, it quickly becomes an issue.
It is in my opinion part of what companies should include in their risk assessments. Looking at the general trends, I would not be surprised if at least EU governments and institutions will ban Chrome due to privacy and data security. That will probably lead to others switching as well.
Quirks that only work in Chrome may stop working in a future release. Things that work on Chrome and FF are more likely to continue to work.
Ideally, you'd have reliable usage numbers by browser and platform, as well as a threshold of usage that demands testing. Maybe that's 1%, 5%, 10%, 20%? Aggregate up, so if FF desktop doesn't hit the numbers, maybe FF in general does, or maybe 'other browsers' does. Or maybe commit to testing the top N platforms over 1% usage.
Lower on the usage scale doesn't need to be pixel perfect or pretty at all, IMHO, but major site features should still work.
Ideally, you'd have reliable usage numbers by browser and platform, as well as a threshold of usage that demands testing. Maybe that's 1%, 5%, 10%, 20%? Aggregate up, so if FF desktop doesn't hit the numbers, maybe FF in general does, or maybe 'other browsers' does. Or maybe commit to testing the top N platforms over 1% usage.
Lower on the usage scale doesn't need to be pixel perfect or pretty at all, IMHO, but major site features should still work.
Well, that way you don’t contribute to making more difficult for people to change browsers.
Do we REALLY want a repeat of IE?
Maybe they are not using Firefox, because your site is broken in it, and they have no other way to tell you this?
Yep, it's the classic self-fulfilling prophecy. Even a lot of non-techies are aware that there are multiple browsers, and will just switch to whichever gets the job done. So if you're seeing less % of any specific browser than the overall web your site is probably broken on that browser. If it's not obvious by now, that means you need to do more testing in that browser, not less.
This. There are some websites that I have to login that are completely broken when using Firefox. I login with Brave on those and am on a constant lookout for competitors that work on Firefox. As soon as I find them I jump ship and take my money elsewhere.
Exactly this happened at the company I work for, and their move to only support Chrome was hailed as a revolutionary time-and-effort saving affair.
grrr
grrr
Survivorship bias!
Yes, this is something I've experienced directly from end-user perspective as well as debugging/user-analytics perspective.
Yes, this is something I've experienced directly from end-user perspective as well as debugging/user-analytics perspective.
I gave up on Firefox and went back to Blink (via Qutebrowser) in part because of Mozilla's countless recent breakages, regressions, and inexplicable decisions, but in other part because of exactly this problem. So many sites just break in mysterious ways on Firefox, and the reason why is inevitably some exception raised 50 frames down the stack in an obfuscated bundle of $FRAMEWORK_DU_JOUR goop I can't even begin to debug.
It's a nice idea because I think the web is much healthier if we don't have a single dominant browser/engine driving it, whether Microsoft's or Google's or anyone else's. Unfortunately no data I've seen in years says it's actually happening. Every web product I've worked on recently is built on standards first so this has nothing to do with deterring Firefox users because things don't work. There just aren't a lot of people using it any more. The only big browser that isn't running the same engine as Chrome is now iOS Safari.
It's around 3% of my pageviews as measured by javascript (e.g. statcounter) but it's 15-20% of my pageviews as seen in my server logs. Now, we might say that the server stats are wrong. But it could also be that the javascript stats are wrong, since firefox comes with tracker blocking. I don't know how to tell.
Honestly, the best way to tell is to create an api endpoint that just returns a 1px gif. Add that as an img tag on your site. Then look at the server logs for that endpoint. A curl-bot probably won’t call the endpoint, only browsers that are actually rendering the page. There are bots that actually render the page and run JavaScript, but you can usually look at the request’s ASN to help determine that.
Or bots disguised as firefox.
Possibly Firefox is making it very easy to block your statistics gathering. Besides, 7% is still quite a lot to throw away.
I can imagine how IE11 gets a higher proportion in traffic in B2B. In B2C, particularly amongst some niches, I'd guess Firefox is still proportionally popular. For example, I'd love to see browser stats for HN.
(Disclosure: I am still very much a Firefox user, so this might be projection.)
(Disclosure: I am still very much a Firefox user, so this might be projection.)
These numbers are problematic for two reasons. 1. This comes from client side tracking scripts which are now likely blocked in ff than Chrome.
2. Unless you're running a very low traffic site, it's not like 3% of users don't matter. 3% may mean millions of users.
2. Unless you're running a very low traffic site, it's not like 3% of users don't matter. 3% may mean millions of users.
You should probably test it on Safari as everything that runs on iOS will run through Safari.
I have no idea about how relevant it is, but every few months I see a website which seem to be tested in Chrome only, because it blatantly don't load in Firefox or Safari. Like, at all, blank page. Or have some buttons like "Purchase" not working.
When you see this, complain and let them know. I've had some success complaining about this to medium sized companies and getting updates.
My retirement account and healthcare provider both have this problem. In one case I had to manually delete a disabled attribute on a button to get signed in because something broke halfway through a multi-step workflow.
Can't tell you my distress when government websites use Google captchas and on Firefox you're stuck and can't go through your business...
Google CAPTCHAs work in Firefox for me, on multiple devices and OS's.
Do you work at Google maybe? They don’t work for me either.
The Irish tax authority's login portal is completely broken in Firefox. It's the only website I need to have a separate Chromium install for.
Please test with Safari on desktop at least. I routinely find things that are just completely broken. And not small names, big names like Hulu, Clover, green chef, and others have just completely broken on Safari. Integration with Google’s captcha seems to be a common failure point.
Also please consider writing your apps in a way such that they don’t completely shit the bed if your 600th ad tracker script fails to load. I hate having to disable pihole so that Google can track me while I pay my utility bill.
Also please consider writing your apps in a way such that they don’t completely shit the bed if your 600th ad tracker script fails to load. I hate having to disable pihole so that Google can track me while I pay my utility bill.
We sometimes hit a bug in Safari which doesn't appear in GNOME Web, which is annoying, because that's really the only WebKit-based browser that runs on my machine. I'd be happy to better support Safari if I wouldn't have to buy new hardware for it :/
It is absurd that Apple doesn't distribute Safari to other desktop platforms or at least make it source available so the community could compile it and fix bugs for themselves.
For Linux at least there are downloads: https://webkit.org/downloads/
One issue is that Safari doesn't work on Windows. So, as a Windows dev, I am forced to buy a MacBook just to test on Safari. (I tried cloud testing services, but the experience is too poor and time-consuming).
Occasionally I run into innocuous enough websites that just don't work at all unless I let them track me.
Note to all fellow fe devs: don't fucking make your call to analytics a blocking process!
Note to all fellow fe devs: don't fucking make your call to analytics a blocking process!
Safari is a pain with bugs, unimplemented standards and just overall unpleasantness.
I can't see why anyone would still use it today when FireFox for Mac has become so good.
I can't see why anyone would still use it today when FireFox for Mac has become so good.
Safari is probably better optimised for power consumption on macOS than Firefox is. Meaning you’ll probably get more hours of use with Safari on macOS than with Firefox on macOS, when you are running on battery.
Battery life, speed, and integration with Keychain is why I use Safari.
The simple answer is that Safari ships with the platform and few people choose browsers based on standards adherence.
For daily use, it’s a pretty normal experience.
For daily use, it’s a pretty normal experience.
Nowadays the biggest issues are with CSS, and those frameworks won't help with that.
However, there are other CSS frameworks (Bootstrap/Tailwind/etc) that do.
Even using those, I'd still give other browsers a look over to see if anything looks odd. One weird one we had was, on Chrome/Edge a React dropdown looked fine, but on Firefox it was somehow positioned like 80px too high. I don't remember exactly what the fix was, just that it was a CSS problem and the original developer was relying on a Chrome default they didn't realize was there and not in other browsers. Something akin to "position: relative; top: 20px;" just happened to look vertically centered, but only in Chrome-based browsers.
However, there are other CSS frameworks (Bootstrap/Tailwind/etc) that do.
Even using those, I'd still give other browsers a look over to see if anything looks odd. One weird one we had was, on Chrome/Edge a React dropdown looked fine, but on Firefox it was somehow positioned like 80px too high. I don't remember exactly what the fix was, just that it was a CSS problem and the original developer was relying on a Chrome default they didn't realize was there and not in other browsers. Something akin to "position: relative; top: 20px;" just happened to look vertically centered, but only in Chrome-based browsers.
Regardless if you're doing small personal projects or doing commercial work, personally I don't find it too time consuming to just breakout the latest versions of Safari, Chrome, and Firefox and do some local testing.
Any experienced software developer will have some experience testing against those. I don't think you need to go out of your way to test against specific builds of those browsers, with maybe the exception of Safari and Mobile Safari since I have noticed standards-compliant differences over time with older iOS devices.
That is to say, there isn't anything wrong with what Safari and Mobile Safari do, but the WHATWG living standards have changed over time just enough that older Apple devices may render web pages differently.
Try not to just limit yourself to Chrome though. And if you're using vendor extensions, don't rely on them, just use them to pretty up from your baseline designs.
Any experienced software developer will have some experience testing against those. I don't think you need to go out of your way to test against specific builds of those browsers, with maybe the exception of Safari and Mobile Safari since I have noticed standards-compliant differences over time with older iOS devices.
That is to say, there isn't anything wrong with what Safari and Mobile Safari do, but the WHATWG living standards have changed over time just enough that older Apple devices may render web pages differently.
Try not to just limit yourself to Chrome though. And if you're using vendor extensions, don't rely on them, just use them to pretty up from your baseline designs.
Safari does require a Mac, so a little more time consuming unless you're already a Mac shop
I develop w/ Firefox and our tester tests with Chrome. I use our product quite a bit with an iPad. If problems are reported with Safari we fix them.
I've also worked at places where our product was specified to work with Chrome, I would try to run it with Firefox and no dice and they would not accept any ticket to fix bugs in Firefox.
There's an assumption today that browsers get updated frequently. My work computer is a managed Windows machine that I had administrator access to, the one complaint I have is that the Firefox is a long-term support version which is often long as in "in the tooth". Bloomberg would complain about it incessantly until I got the new LTS version. Other than that I think browsers are usually up to date.
I've also worked at places where our product was specified to work with Chrome, I would try to run it with Firefox and no dice and they would not accept any ticket to fix bugs in Firefox.
There's an assumption today that browsers get updated frequently. My work computer is a managed Windows machine that I had administrator access to, the one complaint I have is that the Firefox is a long-term support version which is often long as in "in the tooth". Bloomberg would complain about it incessantly until I got the new LTS version. Other than that I think browsers are usually up to date.
Fixing any issue specific on Safari is a whole challenge on another level. Hope you had an easy time debugging issues on Safari.
At work we've never had one that was that bad. In one cases it was third-party cookie issues that affected Safari before everyone else.
Personally Safari is the reason why some of my image-based sites still use JPG in parallel with WEBP. I think they fixed it just recently, but if you have an older mac (i do) Apple couldn't be bothered to include a WEBP codec with the OS. Firefox, Chrome and every other browser for mac supports WEBP but not Safari.
I think they resolved this in the last month, but I think for better or first Apple wants to roll out new SIMD instructions in their libraries much faster than Microsoft does. So at some point your CPU is too old. Of course the flip side to that in the Windows World is that people are stuck using a 10 year old instruction set. Intel doesn't seem to get it that by not supporting the latest AVX in their latest chips that people just "don't use AVX". If the latest SIMD instructions were widely supported you'd have a reason to buy a new computer, but when they aren't Intel has lackluster sales and wonders why.
Personally Safari is the reason why some of my image-based sites still use JPG in parallel with WEBP. I think they fixed it just recently, but if you have an older mac (i do) Apple couldn't be bothered to include a WEBP codec with the OS. Firefox, Chrome and every other browser for mac supports WEBP but not Safari.
I think they resolved this in the last month, but I think for better or first Apple wants to roll out new SIMD instructions in their libraries much faster than Microsoft does. So at some point your CPU is too old. Of course the flip side to that in the Windows World is that people are stuck using a 10 year old instruction set. Intel doesn't seem to get it that by not supporting the latest AVX in their latest chips that people just "don't use AVX". If the latest SIMD instructions were widely supported you'd have a reason to buy a new computer, but when they aren't Intel has lackluster sales and wonders why.
I have had a very nasty bug that I debugged in Safari. Chrome & Firefox supports redirects to http when the site is on localhost or 127.x.x.x but Safari doesn't and throws a Mixed content warning error.
Another one was similar when redirect chain got broken because of an intermediate redirect (https -> https -> http -> https)
I guess Chrome & Firefox automatically upgrades the protocol now but Safari doesn't and it broke the chain and it was also not showing the broken redirected chain in the Safari developer tool.
Another one was similar when redirect chain got broken because of an intermediate redirect (https -> https -> http -> https)
I guess Chrome & Firefox automatically upgrades the protocol now but Safari doesn't and it broke the chain and it was also not showing the broken redirected chain in the Safari developer tool.
Honestly, I thought pretty much everyone had moved to PNG from JPG by now
Not just cross-brower, but its still important to test unique client operating systems as well. Else you get weird crap like this:
https://www.reddit.com/r/linuxquestions/comments/yfjaa1/disn...
I just ran into this bug myself like two days ago, when Disney+ just up and quit working. And even better, if you apply the UA string hack to work around that, it breaks Netflix! I had to, for the moment, install a browser extension that allows persistent site-specific UA string overriding. And sure, that works as such, but c'mon... this is bullshit. People, test your dang websites on more than Chrome on Windows for FSM's sake.
https://www.reddit.com/r/linuxquestions/comments/yfjaa1/disn...
I just ran into this bug myself like two days ago, when Disney+ just up and quit working. And even better, if you apply the UA string hack to work around that, it breaks Netflix! I had to, for the moment, install a browser extension that allows persistent site-specific UA string overriding. And sure, that works as such, but c'mon... this is bullshit. People, test your dang websites on more than Chrome on Windows for FSM's sake.
It's still important to test a bit on Firefox and Safari, or you'll have the occasional UI bug. I found one earlier today where Firefox behaved differently than the other two and needed a quick fix. I'd say this happens once every few months
Times have changed in that you won't have layers and layers of branching logic to work around browsers that aggressively disagree with each other. But Chrome tends to fudge the web standards in some of their dark corners, Firefox tends to stick to the letter of the web standards rather than the spirit, and Safari tends to have random little gaps and quirks
Your UI will probably be usable without cross-browser testing (assuming you're not doing anything exotic), but it will probably have little things here and there that aren't quite right
Times have changed in that you won't have layers and layers of branching logic to work around browsers that aggressively disagree with each other. But Chrome tends to fudge the web standards in some of their dark corners, Firefox tends to stick to the letter of the web standards rather than the spirit, and Safari tends to have random little gaps and quirks
Your UI will probably be usable without cross-browser testing (assuming you're not doing anything exotic), but it will probably have little things here and there that aren't quite right
I consult, frequently jumping between web application codebases. Very often, I bring cross-browser testing with me by sheer nature of using Firefox when _everyone_ has been using Chrome exclusively for years. I end up finding issues; about half the time, clients don't care, because all their customers use Chrome.
This direction worries me (we did this with MSIE and it took a decade+ to heal those wounds).
This direction worries me (we did this with MSIE and it took a decade+ to heal those wounds).
Depends on how critical it is for you/your company. I work on products that have internal tooling, and if they run on the latest version of Chrome, it's good enough (although I personally test them on Firefox, too, and someone used an older version of Safari the other day, and found a weird bug).
However, some of our tools are used by the general public (I've found the user agent for a coupon app as well as some PlayStation browser in the logs; might have been spoofed, of course). Those are written to the lowest common HTML/CSS/JS standards, so we don't try fancy features, and Babel or similar can be helpful. IE11 is out of grace, but we still support years old versions of most browsers, because it costs money and goodwill when too many fail.
However, some of our tools are used by the general public (I've found the user agent for a coupon app as well as some PlayStation browser in the logs; might have been spoofed, of course). Those are written to the lowest common HTML/CSS/JS standards, so we don't try fancy features, and Babel or similar can be helpful. IE11 is out of grace, but we still support years old versions of most browsers, because it costs money and goodwill when too many fail.
Not a web developer, but I usually just test with the latest or almost latest version of FF and perhaps Chrome.
I personally don't like or use any of the frameworks you presented. If I had to pick one I would choose Vue.
jQuery did a lot for inter-browser compatibility, so I don't understand the argument. On the contrary, it was much more difficult to manage diverging code without it.
99% of my web programming is displaying some data, a website to interface XY. I don't like modern websites at all. Probably more a design descision than a framework issue, but I try to minimize any browser scripts in the first place and just haven't felt the need for somthing like React or Angular.
I personally don't like or use any of the frameworks you presented. If I had to pick one I would choose Vue.
jQuery did a lot for inter-browser compatibility, so I don't understand the argument. On the contrary, it was much more difficult to manage diverging code without it.
99% of my web programming is displaying some data, a website to interface XY. I don't like modern websites at all. Probably more a design descision than a framework issue, but I try to minimize any browser scripts in the first place and just haven't felt the need for somthing like React or Angular.
Hey @raxxorraxor, I am not against using jQuery. What I meant was initially back in 2012-2013 days when jQuery used to be a default DOM query/manipulation library people used, the same code used to break across browsers and we had to use some polyfills.
Like you said you usually test with the latest version of Chrome/Firefox you are partly agreeing to the statement I made.
I am trying to find someone who's strongly in favour of testing their apps across browsers and multiple versions.
Like you said you usually test with the latest version of Chrome/Firefox you are partly agreeing to the statement I made.
I am trying to find someone who's strongly in favour of testing their apps across browsers and multiple versions.
Well you don't need to test on lots of versions, since updates are usually automatic now (or at least highly encouraged on startup). But you definitely need to test on Chrome and Firefox in the browser (maybe also Safari?), and both Chrome and Safari on mobile, since there are JavaScript APIs and CSS features not supported by all of them.
The latter is especially true when you're trying to make a PWA or 'web app', since Apple does not like letting websites have similar abilities to native apps on their devices...
The latter is especially true when you're trying to make a PWA or 'web app', since Apple does not like letting websites have similar abilities to native apps on their devices...
Look at your analytics and see which browsers your users are using. Sometimes, I’ve worked on apps without a big userbase but we had to support IE because a big percentage used it.
You’ll probably find out that you don’t have too many browsers that actually matter.
You’ll probably find out that you don’t have too many browsers that actually matter.
I encountered a cross browser mixed implementation bug only today, affecting react components.
https://jsfiddle.net/svandragt/236Lhkub/3/
So I’d say yes
https://jsfiddle.net/svandragt/236Lhkub/3/
So I’d say yes
I don't think it's worth testing on old versions of Chrome and Firefox since they're updated automatically.
iOS Safari on the other hand, can't be updated on older phones since it's tied to iOS updates. This makes any Safari bug linger for a few years. It's also the only rendering engine available on iphone so some people are truly stuck using an old version.
So I'd say, test on Chrome because it's ubiquitous and test on iOS Safari like it's the new Internet Explorer.
I'm not sure about Firefox. I personally use it and test on it, I think it's the right thing to do for the web. And at least with that you've covered most rendering engines.
iOS Safari on the other hand, can't be updated on older phones since it's tied to iOS updates. This makes any Safari bug linger for a few years. It's also the only rendering engine available on iphone so some people are truly stuck using an old version.
So I'd say, test on Chrome because it's ubiquitous and test on iOS Safari like it's the new Internet Explorer.
I'm not sure about Firefox. I personally use it and test on it, I think it's the right thing to do for the web. And at least with that you've covered most rendering engines.
I don't disagree that most versions/instances of Chrome are updated regularly, but implying that this is universal is questionable.
Chrome is only updated if the user lets it happen. I can't count the number of times I've watched some co-worker share their screen on a conference call and the Chrome update button is lit up, but apparently that's not a trigger to these people (I update pretty much as soon as I notice the indication since Google has done a remarkably good join of updating in place in a way that most SaaS things and web sites just reopen without trouble).
I'm sure the next time they restart/reboot it will get updated, but if there was some hideous Chrome-related bug which is from a release a month ago, you might want to think about that. It's also entirely possible to turn off updates if you or some misguided IT person is really so inclined.
Chrome is only updated if the user lets it happen. I can't count the number of times I've watched some co-worker share their screen on a conference call and the Chrome update button is lit up, but apparently that's not a trigger to these people (I update pretty much as soon as I notice the indication since Google has done a remarkably good join of updating in place in a way that most SaaS things and web sites just reopen without trouble).
I'm sure the next time they restart/reboot it will get updated, but if there was some hideous Chrome-related bug which is from a release a month ago, you might want to think about that. It's also entirely possible to turn off updates if you or some misguided IT person is really so inclined.
Good point, it's definitely not universal. Maybe it's more fair to think of it in terms of priority. I don't expect a smaller team to test on many versions, but the bigger the team and the more users they have, the more diligent they should be with their testing.
So in my opinion, in order, I'd test on: - current Chrome - current Safari - older Safari - older Chrome & current Firefox
Not sure what comes next, IE 11 maybe? I guess at some point it's good to look into automating some testing.
And it's worth noting, as with most things, that "it depends", I don't expect the same browser support from something like Figma and from a government website for example.
So in my opinion, in order, I'd test on: - current Chrome - current Safari - older Safari - older Chrome & current Firefox
Not sure what comes next, IE 11 maybe? I guess at some point it's good to look into automating some testing.
And it's worth noting, as with most things, that "it depends", I don't expect the same browser support from something like Figma and from a government website for example.
> I don't think it's worth testing on old versions of Chrome and Firefox since they're updated automatically
https://support.mozilla.org/en-US/kb/firefox-esr-release-cyc...
https://support.mozilla.org/en-US/kb/firefox-esr-release-cyc...
> Do we really need to test our code on 50ish Chrome versions, 25is Firefox versions and so on.
The only time I've ever seen this intense level of testing was on a government project that was producing someone that everyone in the country would be required to use. So if that is what your situation is yes, you need to do that much testing because if it is mandated by law that citizens use it, it better be able to handle whatever combination of citizen and tech it encounters.
Otherwise I think you are probably being somewhat hyperbolic here.
Aside from that where accessibility is concerned there can be significant differences between various combinations of browsers and screen readers.
The only time I've ever seen this intense level of testing was on a government project that was producing someone that everyone in the country would be required to use. So if that is what your situation is yes, you need to do that much testing because if it is mandated by law that citizens use it, it better be able to handle whatever combination of citizen and tech it encounters.
Otherwise I think you are probably being somewhat hyperbolic here.
Aside from that where accessibility is concerned there can be significant differences between various combinations of browsers and screen readers.
[deleted]
Feels like everything just works on Chrome and Firefox, but Safari is the new IE.
Here is a bug I've recently discovered, which is present only in Chromium-based browsers: <https://bugs.chromium.org/p/chromium/issues/detail?id=138298...>.
Doesn't happen on Safari, nor Firefox.
Doesn't happen on Safari, nor Firefox.
Absolutely, feels like almost 100% of browser specific issues we deal with are related to Safari.
I worked with a client last year who had a firm requirement of IE6 being 100% compatible. No fancy CSS just plain old hand coded HTML. Lightning fast performance mind!
It was an e-commerce site which sold _very_ grey area products (most were banned from Amazon/eBay/FB Marketplace/etc.) and the ancient IE spec gives you an idea of the demographic they ~exploited~ sold products to ...
The market for selling questionable medicinal pills is much much larger than I could have ever imagined. Absolutely mind blowing!
It was an e-commerce site which sold _very_ grey area products (most were banned from Amazon/eBay/FB Marketplace/etc.) and the ancient IE spec gives you an idea of the demographic they ~exploited~ sold products to ...
The market for selling questionable medicinal pills is much much larger than I could have ever imagined. Absolutely mind blowing!
The other day I came across a website where the signup button didn't work on Firefox alone. So yeah, that's reason enough to keep on testing on all major browsers.
It's still relevant - there are enough differences between the browsers to render incompatibilities and bugs. It does also depend on how serious you are about your product, as some kinds of products not working in a browser can leave a very bad or frustrating experience.
It is of course painful to test in so many variants, so a decent compromise is often to just test in the latest version of all browsers. There's enough coverage there to help you find any potential problems.
It is of course painful to test in so many variants, so a decent compromise is often to just test in the latest version of all browsers. There's enough coverage there to help you find any potential problems.
That's why I use only features that are cross compatible between browsers, even though I feel left behind on some new features only available on Chrome -_-
We're using Playwright [0] to do e2e testing. Locally I just run Chrome, but on CI FF and Safari are used as well. Using the official Docker image, this was really easy to setup. Because all browsers are automatically updated nowadays, we only test the most recent versions.
A recent issue we found, was that Safari was responding different to some HTTP headers.
[0]: https://playwright.dev/
A recent issue we found, was that Safari was responding different to some HTTP headers.
[0]: https://playwright.dev/
Do you use any tool like Browserstack or Lambdatest or Saucelabs for running your tests across browsers? My question is if you have already invested in end to end testing automation then why not run in 10 browsers to be on safe side than running in only 3. Price?
If it adds value, I think it's a good idea. But less is more, so right now I'm not inclined to add more to our CI/CD pipeline.
In the future our application will need to be able to run on mobile devices. Then I think I want to test it on a recent iPad, iPhone and Android device. We'll probably be using one of the tools you mentioned.
In the future our application will need to be able to run on mobile devices. Then I think I want to test it on a recent iPad, iPhone and Android device. We'll probably be using one of the tools you mentioned.
It depends on your risk tolerance and how many DAU/WAU/MAU you have (and whether you're willing to have an untested experience for X users.) For example, I would test really old browser versions, and if there is a bad experience, you could let the user know to use a different browser or to upgrade.
My intuition says you'd get better value out of testing different browsers shallowly rather than 50 old versions of Chrome, though.
My intuition says you'd get better value out of testing different browsers shallowly rather than 50 old versions of Chrome, though.
I experienced a DOM issue where a horizontal scrollbar was being rendered on some versions of Chrome running on Windows, some versions of Firefox on Mac, but I couldn't reproduce it myself (Chrome / Safari on Mac). It happens.
I don't QA for these things beforehand, but when they do occur, it helps to have a way to perform cross-browser testing to fix the bug.
I don't QA for these things beforehand, but when they do occur, it helps to have a way to perform cross-browser testing to fix the bug.
Most of the cross browser issues arise either from CSS or JS, and both of these occur because of older browsers. Since we don’t have control over people as to which browser they use, devs generally tackle this issue by using different stylesheets for different browsers conditionally. Modern tools like lambdatest, browserstack generate multiple previews easily.
I made a website that did some slightly unusual things with text inside SVGs. It rendered correctly on Gecko but not on Blink. I submitted a bug report to the Blink project, and they fixed it pretty quickly. Who knows what other rendering bugs lurk in the various engines.
So at least test on recent versions of the three major engines.
So at least test on recent versions of the three major engines.
Yes, to some degree. We use usage data to decide on a strategy, it usually ends up in testing on the big names on big platforms the latest and one or two versions back. Of course we are risking a self fulfilling prophecy since we rarely test Firefox for example.
PeacockTV has problems on Safari but just works on Firefox. This is just the most recent example I have of large companies whose site doesn’t work well on all browsers. So yes I think we should test on multiple versions of multiple browsers on multiple OSes
Most cross browser bugs are CSS and yes, you still have to test that. The problem is not JS
Not much anymore, at least from my experience. When writing Typescript, I know what I am targeting and it'll transpire to that.
When writing CSS, I rely on caniuse.com when not certain.
When writing CSS, I rely on caniuse.com when not certain.
You have to test on Safari because Apple was smart enough to box out Chrome from iOS.
We’ll probably see Gecko used in more embedded applications but Firefox is a joke at this point.
We’ll probably see Gecko used in more embedded applications but Firefox is a joke at this point.
Depends if you care about Safari.
Real browsers like Chrome, Firefox and Edge generally "Just Work", Safari on the other end seems to be an endless tire fire that either implements things poorly or not at all. Doubly true for the iOS versions which generally tend to be older and worse than desktop Safari.
Real browsers like Chrome, Firefox and Edge generally "Just Work", Safari on the other end seems to be an endless tire fire that either implements things poorly or not at all. Doubly true for the iOS versions which generally tend to be older and worse than desktop Safari.
Never had to do it since I use autoprefixer and safari is my main browser
Chrome and Firefox auto-update so no.
I have noticed that there are still vast differences in the printing functionality, both in terms of how things render and performance. So if you need to do any kind of printing in the app and you know your clients will use an array of different browsers it is worth checking it. Even within the same browser things tend to go back and forth when it comes to printing, Chrome for example tends to break things in one version and then they come back in the next one. It is a bit of a mess.
all your comments are flagged, please write an email to the admins to sort this out
True
We might be wrong here and missing some valid point. I see companies doing automation testing beyond a point but do they really test beyond 2 browsers?
One line of thought could be if we have already invested in automation testing then why not run in 80 browsers too when we can. Its just additional hardware cost or cost of platforms like BrowserStack/Lambdatest.
One line of thought could be if we have already invested in automation testing then why not run in 80 browsers too when we can. Its just additional hardware cost or cost of platforms like BrowserStack/Lambdatest.
Just wanted to pick your brains on what do you think how relevant is cross-browser testing. Do we really need to test our code on 50ish Chrome versions, 25is Firefox versions and so on.