How do you test your web app in the various browsers?
Considering that most web apps need to be tested in many different browsers, each spanning multiple versions, how do you go about that? Do you have a particular workflow? Any tips that others might find useful?
21 comments
Windows doesn't support multiple versions of IE on the same machine without some hackery. But, if you're a Windows user and poor, Microsoft provides free virtual machines to test IE 6, 7, and 8. The images might work on other VMs too.
Virtual PC (free): http://www.microsoft.com/downloads/details.aspx?FamilyID=04D...
Disk images with IE 6, 7, 8 (free): http://www.microsoft.com/downloads/details.aspx?FamilyId=21E...
I just install all the other browsers (FF and Safari mainly) and do those manually.
Virtual PC (free): http://www.microsoft.com/downloads/details.aspx?FamilyID=04D...
Disk images with IE 6, 7, 8 (free): http://www.microsoft.com/downloads/details.aspx?FamilyId=21E...
I just install all the other browsers (FF and Safari mainly) and do those manually.
For Windows XP there is also a nice installer for having multiple Explorer versions on the same machine (3,4,5,6 in addition to normally installed 7):
http://tredosoft.com/Multiple_IE
http://tredosoft.com/Multiple_IE
IE 3? That's some dedicated testing.
Don't forget about IETester: http://www.my-debugbar.com/wiki/IETester/HomePage
Though there are a couple minor bugs, it works fantastically for testing IE6, 7, and 8 without jumping through hoops.
Combine IETester with Safari for Windows, Firefox 3 (and 2), Opera, and perhaps Chrome and you've got a full suite of browsers to test with just on Windows. For the most part, that will give an accurate view of how it will look on Mac and Linux, but it doesn't hurt to have a VM of both a Linux distro and (cough) OS X available to test properly with.
Though there are a couple minor bugs, it works fantastically for testing IE6, 7, and 8 without jumping through hoops.
Combine IETester with Safari for Windows, Firefox 3 (and 2), Opera, and perhaps Chrome and you've got a full suite of browsers to test with just on Windows. For the most part, that will give an accurate view of how it will look on Mac and Linux, but it doesn't hurt to have a VM of both a Linux distro and (cough) OS X available to test properly with.
I found I'm more productive doing almost all development just in one browser (Firefox) and then I fix bugs appearing in other browsers only at the end.
The exception is when I want to do something more complex, then I test feasibility of this particular feature in all browsers as soon as possible to see if there is not some showstopper (like Explorer's JavaScript engine being too slow).
Try to keep markup as simple as possible. It's good both for performance and for cross-browser compatibility. I had many cross-browser bugs disappearing just with refactoring.
When stuck, try to get the same behavior from all browsers, even if it's buggy. It can be easier to get to uniform correct appearance from uniform incorrect one than from the situation where you have mixture of good/bad behavior across browsers.
The exception is when I want to do something more complex, then I test feasibility of this particular feature in all browsers as soon as possible to see if there is not some showstopper (like Explorer's JavaScript engine being too slow).
Try to keep markup as simple as possible. It's good both for performance and for cross-browser compatibility. I had many cross-browser bugs disappearing just with refactoring.
When stuck, try to get the same behavior from all browsers, even if it's buggy. It can be easier to get to uniform correct appearance from uniform incorrect one than from the situation where you have mixture of good/bad behavior across browsers.
A parallels installation (win XP) dedicated to only IE6&7, opera, windows safari, and FF2&3. Opera, safari, FF2&3 in OS X.
I just do it all manually - FF2&3 and safari usually have very little differences (if any), but opera and obviously IE sometimes have strange quirks.
I just do it all manually - FF2&3 and safari usually have very little differences (if any), but opera and obviously IE sometimes have strange quirks.
With a non-trivial web site, the more browsers you test, the more cash you burn through, and (if you're not very careful) the more twisty the web site HTML and CSS code gets. But you already know that.
Verify compliance with W3C standards as a baseline, follow a proper subset of those standards whenever you can, and then point-test with large footprint (and buggy) browsers you might have to deal with. IE, for instance, can need special coding. FF, Opera, Safari and other such do not generally require any special-casing.
If you're compliant with web standards and the web page renders badly, then any problem reports are generally browser bugs. With that as a starting point for the ensuing discussions, you can then decide if you want to add support for the (buggy?) browser, or request that the accessor upgrade or migrate to a more standards-compliant browser.
Of course test harnesses can and do help here. But the closer you stay to the W3C and applicable standards, the more bug-resistant and future-proof your web site is, and the less browser-tail-chasing you will likely need to do. And the less point testing you can need to fund, particularly as you build confidence in the standards and the core browsers.
As for reading material, a subscription to communitymx.com is well worth the costs; that's one of the better HTML and CSS-oriented sites on the net.
Verify compliance with W3C standards as a baseline, follow a proper subset of those standards whenever you can, and then point-test with large footprint (and buggy) browsers you might have to deal with. IE, for instance, can need special coding. FF, Opera, Safari and other such do not generally require any special-casing.
If you're compliant with web standards and the web page renders badly, then any problem reports are generally browser bugs. With that as a starting point for the ensuing discussions, you can then decide if you want to add support for the (buggy?) browser, or request that the accessor upgrade or migrate to a more standards-compliant browser.
Of course test harnesses can and do help here. But the closer you stay to the W3C and applicable standards, the more bug-resistant and future-proof your web site is, and the less browser-tail-chasing you will likely need to do. And the less point testing you can need to fund, particularly as you build confidence in the standards and the core browsers.
As for reading material, a subscription to communitymx.com is well worth the costs; that's one of the better HTML and CSS-oriented sites on the net.
http://crossbrowsertesting.com/
Offers a set of virtual machines for browser testing, or anything actually.
I used the service once to get around a XP-only Java application installation -- installed the application into a machine via the service and then copied the JAR-files to my own machine. Worked like a charm.
Offers a set of virtual machines for browser testing, or anything actually.
I used the service once to get around a XP-only Java application installation -- installed the application into a machine via the service and then copied the JAR-files to my own machine. Worked like a charm.
I've heard Selenium is really good. It has a remote testing mode so that you can set up a bunch of different OS and browser combinations and run tests automatically.
http://selenium-rc.openqa.org/
http://selenium-rc.openqa.org/
I don't really trust anything but the real thing. So I build for FF3 and test on: PC IE6, IE7, Chrome and Safari; Mac: Safari, Opera, FF3 and FF2. Testing Safari on Mac and PC seems to be redundant, don't really care about Opera that much so only testing on Mac seems to be ok. Testing FF on both Mac and PC also seems redundant - but just because it works in FF3 does not mean it works in FF2. I'm not sure yet if testing Chrome and Safari is redundant - don't trust the Chrome JS engine yet - although it looks like I will soon.
My main computer has FF3 and Opera 9. I have a rusty Powerbook G4 with Safari 3, FF2, FF3.
and in VMWare:
XP SP3 with IE6
XP SP3 with IE7
XP SP3 with IE8, Chrome, Safari 3, FF2
The workflow is, Ctrl Shift R, and click around.
and in VMWare:
XP SP3 with IE6
XP SP3 with IE7
XP SP3 with IE8, Chrome, Safari 3, FF2
The workflow is, Ctrl Shift R, and click around.
+1.
I run Ubuntu and have 2 VMWare virtual servers running XP with various browsers.
I run Ubuntu and have 2 VMWare virtual servers running XP with various browsers.
Great feedback guys. Thanks.
I'm going to go the route of virtual machines, and give Multipe IE a shot so that I only need one Windows VM. After that, I think I'll start writing up some scripts (text, not code) to follow, step-by-step, to check various functionality.
Also, webrat (http://github.com/brynary/webrat) has received a lot of commendations on the rspec-users mailing list, so I'm definitely going to give that a whirl.
I'm going to go the route of virtual machines, and give Multipe IE a shot so that I only need one Windows VM. After that, I think I'll start writing up some scripts (text, not code) to follow, step-by-step, to check various functionality.
Also, webrat (http://github.com/brynary/webrat) has received a lot of commendations on the rspec-users mailing list, so I'm definitely going to give that a whirl.
I use vnc
firefox on my dev box
safari on my living room mac mini G4 (vnc from dev box)
IE? i call my friend to test or just go to internet cafe
iirc vnc server can take external ip ... so presummably you can connect to vnc server half globe away and see how people there connect to your site
of course the bandwith is shared between browsing and vnc-ing so u will get less than accurate experience
firefox on my dev box
safari on my living room mac mini G4 (vnc from dev box)
IE? i call my friend to test or just go to internet cafe
iirc vnc server can take external ip ... so presummably you can connect to vnc server half globe away and see how people there connect to your site
of course the bandwith is shared between browsing and vnc-ing so u will get less than accurate experience
I'm currently working on a prototype for an tool that will allow for simultaneous browser testing and assistance with detecting page layout problems, if you're interested in hearing more, drop me a line at bcompare at gmail. In developing my prototype, I'd love to get some of your sites to test it against.
I tried using Amazon's EC2 to run a Windows image. Probably due to the lag, it hasn't been as good as I thought, so I've switched back to running Windows on VMWare.
we design to firefox and tell everyone that we do (niche audience: a med school). If something breaks it's usually in IE, which we take as a matter of pride, or Opera and I'm pretty sure we're the only ones who know what opera is
I am working with firefox with its add-on: "ie tab" to check the differences in ie...
What, you mean not everyone uses Chrome?
http://browsershots.org
An amazing and free tool. I also do manual testing in the common versions of major browsers. If I don't have a system to run that browser, I just find a friend to borrow from for an hour or so.
An amazing and free tool. I also do manual testing in the common versions of major browsers. If I don't have a system to run that browser, I just find a friend to borrow from for an hour or so.
The source code is available at this site too and may interest those wanting a Python based master-worker distributed processing pattern for thier own problems/tasks.