Experimental Design For Adsense

[nxn]

Given a wide variety of possible Adsense changes/designs to test, how do you design
an experiment to test things efficiently? Industrial experiment design can get pretty complex,
but can save a lot of time zeroing in on what works. See
http://www.statsoft.com/textbook/stexdes.html#generalb for example.
Anybody understand this stuff or can recommend some basic design to start?
Layout A vs B vs C vs D, colors X vs Y vs Z vs rotating colors, etc.
Also, how many click throughs is enough to be statistically significant? Maybe 100?

[GDevendorf]

Hi Dave,
Not to put testing down, but keep in mind that you are also at the mercy of external forces in this venue. If you are looking for an "EXACT" answer, such as your site example may give, I doubt you will ever find it. For instance... Lately, I am convinced that many of my stats are thrown off due to PSAs being placed in over abundance.
From what I can tell you, and from what I have heard from the guru's, the best starting point is the large rectangle, placed prominently where it is seen upon site opening. I have also heard that the placement of images above the ads, in a leaderboard style has some promise, but I haven't tried that as yet.
I have been experimenting with the color of the link font in red. Again, being exact is hard, but I can tell you it doesn't seem to hurt anything. The background should match your site. If you do these things you will get a good start. Then if you're a mind to test, site by site, you may be able to find something site specific that may help that particular site. BUt don't count it as gospel unless it works in multiple arenas. It may only work for that site. That said, If you find anything in particular, I hope you will share it with us.
As for statistical significance, 100 would be an absolute minimum! You will see variations within a couple percentage points, just from a monday to a tuesday. And again, each day becomes a variable, the weather outside, the up coming holiday.... You get the idea. Suffice to say, 1000 would always be better. You can see a couple points variance within a site, just from July to August. I stress again, being Exact, is very hard. Sometimes it is more of a gut feeling.
I must confess, in lieu of vast experimenting, I have gone with the basics above and just built more sites. And that does work!

Good Luck!
Geo