5

I'm trying to design an experiment where I measure a variable as a function of 5 two-level factors, labelled A, B, C, D and E.

I'm trying to understand how to best design this experiment so I can conduct it in 8 runs. I've tried to follow the guidance given in Box, Hunter & Hunter, and found two experimental $2^{5-2}$ designs that seem to contradict each others.

One is given in the table p. 272, where D=AB and E=AC. That would yield the following design:

ABCDE
+++++
++-+-
+-+-+
+----
-++--
-+--+
--++-
---++

The other one is the example whose table is given on p. 236, defined by D=BC and E=ABC:

ABCDE
+++++
++---
+-+--
+--++
-+++-
-+--+
--+-+
---++

On what criteria should one choose one design over another? Sorry if this sounds like a newbie's question, but I'd really like to understand this issue.

JMS
  • 4,870
lindelof
  • 1,119
  • Is your response binary as well? or is it continuous? what model or functional relationship between the measured variable and the $5$ factors are you interested in estimating? If you specify the goal of your experiment and what prior information you have, you are much more likely to get a more helpful answer. – probabilityislogic May 11 '11 at 09:42
  • @probabilityislogic I'm trying to optimize the response time for a website, and I've identified 5 factors believed to have an influence on that response time (such as thread pool size, connection pool size, etc). For each experimental run, I do a load-test and measure the average response times. – lindelof May 11 '11 at 13:56

3 Answers3

4

Neither one of these designs contradicts each other, they are generated in different ways. A $2^{5-2}_{III}$ is not unique.

Using a design with so many factors and so few runs, necessiates the fact that main effects and two factor interactions will be confounded, the question is how. Since you want to do an experiment with $8 = 2^{3}$ observations, the full factorial that will generate this design will be based on 3 factors; call them A, B, and C. In the 3 factor design, you will have interactions

  • AB, AC, BC
  • ABC

To expand this design to include two more factors, each of the new factors ;call them D and E; must be confounded with one of the above listed interactions. Two ways to do that without confounding two main effects is to either have

  • D = AB and E = AC which means DE = BC
  • D = BC and E = ABC which means DE = A

You can try other combinations, and see the results of your confounding structure.

chl
  • 53,725
schenectady
  • 1,119
  • That was my understanding too, but how should I choose one design over another? – lindelof May 12 '11 at 13:10
  • If you are interested in only studying main effects, believe that two factor interactions are negligible, and you have no other criteria to meet; then either design will do. It is VERY important that you randomize your experiment; or you could introduce hidden effects. Good luck! – schenectady May 12 '11 at 13:21
  • If he thinks interactions are negligible, should he include them an all? Also, will he have to be careful of his choice of design in order that he has replication? (I'm mainly curious for myself; I only had one lecture on fractional factorial designs.) – Avery Richardson May 13 '11 at 04:32
2

I do not have the book at hand, so I cannot comment on the reasoning to find these models.

However, it is reasonable to expect in this type of setting that:

  • Different designs might better achieve different optimality criteria: perhaps you want the design with 8 runs that has best overall predictive ability, perhaps you want least maximum variance,...
  • There may be more than one design yielding the same optimality (given a criterion)

As such, if you want help for your specific situation, you're going to have to enlighten us on what you want to use this design for (i.e. what is your optimality criterion).

Nick Sabbe
  • 12,819
  • 2
  • 37
  • 47
  • 1
    I need to know essentially which factors have the biggest influence on the response variable. Am I right in thinking that with the first design, I won't be able to distinguish main effects from first-order interactions; whereas with the second design, the main effects from A and E will be free from first-order interactions, while B, C, D will not? – lindelof May 11 '11 at 09:09
  • @lindelof Yes, you're right. In the first $2_{\text{III}}^{5-2}$ 8-run design, all main effects will be aliased with a least one first order interaction. I'll try to elaborate more in a response if I find time. – chl May 11 '11 at 20:29
-2

Choose the design that has the 3way interaction, the greater the number of factors in an confound, the less likely it is to actually happen, (statistically :) ). In many screening experiments you will always have to deal with two way interactions confounded with main effects...so the one where you have to only test one of them, (at the same strength) is better because it eliminates one test you need to perform.

Elliot
  • 1