ABalytics.js: Client-side A/B Testing With Google Analytics(danmaz74.me)
danmaz74.me
ABalytics.js: Client-side A/B Testing With Google Analytics
http://danmaz74.me/2012/10/10/abalytics-js-client-side-ab-testing-with-google-analytics/
21 comments
Concat is only called before GA is loaded, so I don't think any race condition could happen.
Only if everyone loads it the same way. But there are lots of different ways to load GA; the only reason _gaq.push() isn't scary and doesn't raise the specter of race conditions is because of the 'spoofing' GA does.
There are LOTS of different versions of the GA snippet floating around the internet; the assumptions this configuration makes would only work for some of them.
For example, some people load GA like this:
It seems unnecessarily risky.
There are LOTS of different versions of the GA snippet floating around the internet; the assumptions this configuration makes would only work for some of them.
For example, some people load GA like this:
<script async src="//google-analytics.com/ga.js"> </script>
<script>
var _gaq = _gaq || [];
...
</script>
It seems unnecessarily risky.
[deleted]
Ok, now I understand what the issue is. I guess a good solution would be to pass _gaq to my library initializer and push every value with a for cycle, instead of using concat. Thanks for the tip.
I updated the library on Github: now it doesn't use concat anymore!
Checking out the source code, this isn't going to work in IE8 and earlier without some small tweaks. IE didn't support `document.getElementsByClassName(string)` until IE9. It would be a small rewrite to make it work with IDs instead of classes (or you could use a shim).
[EDIT]: . . . But I like the idea. Might even use it next time I'm working with GA.
[EDIT]: . . . But I like the idea. Might even use it next time I'm working with GA.
Thanks for the warning! I initially developed this with jQuery, and only after seeing how little of it was needed decided to get rid of it - but I forgot to make all the due diligence. Later or this weekend I'll fix this - but if in the meantime anybody wants to send a fix or a pull request, I'll be glad to use it.
LOL, I had forgotten: I already use a custom getElementsByClassName if there isn't one! So the it already works in IE < 9... check out the source code :)
Very Nice! Personally, I would have loved a code example inside the post.
Thanks for going MIT.
Thanks for going MIT.
You're welcome!
I didn't think about the code example in the post... maybe I'll add it and resubmit the post ;)
I didn't think about the code example in the post... maybe I'll add it and resubmit the post ;)
more information about usage:
https://github.com/danmaz74/ABalytics
https://github.com/danmaz74/ABalytics
Oh, I opened that. I just meant my favorite - check out my cool tool posts include samples of cool tool being cool. :)
looks nice and this is just an initial thought, it looks like the holding content in the HTML is replaced by either case A or B from the script, will it run with the holding text being case A and swapping out only when B is picked by the script therefore avoiding a possible content flicker some of the time? of course this may not be desirable on larger content.
Yes, it would work - but I wouldn't advice doing so:
* If the flicker isn't noticeable, it doesn't matter either way
* If the flicker is noticeable and disturbing, having it only for one version could skew the results in favor of the non-flickering versionMy first thought upon seeing this was, "Where are the statistics so that you know if you are just ahead by chance?"
My second thought was, "Testing is so much better than not testing, this is still worthwhile."
Seriously, if you aren't testing, do it. If you are, this is something to graduate out of.
My second thought was, "Testing is so much better than not testing, this is still worthwhile."
Seriously, if you aren't testing, do it. If you are, this is something to graduate out of.
I made a bookmarklet for easily calculating statistical significance, specifically for this use case.
After enabling the bookmarklet, you click the numbers you need and then it sends it to an A/B testing calculator hosted by the folks who made ABBA
Project: https://github.com/yahelc/ABBA-bookmarklet Sends to here: http://www.thumbtack.com/labs/abba/
After enabling the bookmarklet, you click the numbers you need and then it sends it to an A/B testing calculator hosted by the folks who made ABBA
Project: https://github.com/yahelc/ABBA-bookmarklet Sends to here: http://www.thumbtack.com/labs/abba/
OP here: That was the spirit of the thing. It isn't a fully fledged solution, but it's a very easy way to start testing, and in GA you can combine this data with all the other data you have there, if you want.
Just me, or does anyone else miss Multivariate testing? It's impossible for me to use A/B testing with shopping cart software when it loads dynamic content (like the cart or checkout processes). Unless I'm doing it wrong?
Content that loads dynamically won't force you to use Multivariate Testing. In fact, most commercial services implement both A/B and Multivariate tests using client side JS.
Also, you can A/B test dynamically loading content so I think you just need to find out how to do it using your chosen software.
Also, you can A/B test dynamically loading content so I think you just need to find out how to do it using your chosen software.
Great job. I came up with my own ghetto solution using event tracking, yours is definitely better.
So, your core coding style will only work with very particular GA implementations; anything that deviates from that could very easily break or be subject to race conditions.
You'll also get better results if you use events rather than custom variables; custom variable reporting isn't as flexible and can be very confusing (ie, visit start). It'll also mitigate the need to use concat() to cut the line, since Events can be fired at any time (whereas custom variables need to happen before the first hit of the session to be well-reported).
I also really like the idea, though. I actually pass all my Optimizely test results as Google Analytics events with this snippet: