Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

The design of experiments (DOE) is the study of how to structure an information-gathering exercise where the relationship between variables is not deterministic (i.e., at least some random variation is present). In statistics, controlled experiments are often implied, meaning that study units are randomly assigned to conditions in which the level of a variable[s] has been independently manipulated (minimally, and prototypically, to create a treatment condition and a control condition).

In the design of experiments, the experimenter is often interested in the effect of some process or intervention (the "treatment") on a response variable for some objects (the "experimental units"). True experiments are typically thought necessary for inferring causality, although there is a field of study within statistics that concerns itself with how causality may be nonetheless inferred from observational data.

1977 questions
28
votes
4 answers

Pitfalls in experimental design: Avoiding dead experiments

I've come across this quote numerous times: To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of. -- Ronald Fisher (1938) To…
naught101
  • 5,453
13
votes
4 answers

Seeking a real illustration of Fisher's quote about DoE

My team and me would like to give a presentation to the non-statisticians of the company about the utility of the design of experiments. These non-statisticians are also our clients and they usually don't consult us before collecting their data. Do…
10
votes
3 answers

What to do with confounding variables?

I need to do an experiment. First let me describe present situation. The company that I work for is a cinema. It has a gaming section where people who are waiting for movies can pass time by playing games. People can pay only by using prepaid…
9
votes
1 answer

What is an orthogonal design?

I am a little bit confused about what an orthogonal design is and how it relates to the model matrix. It appears that there are many perspectives into the definition of orthogonal inside this post but none exactly helps me understand my problem. I…
tintinthong
  • 940
  • 2
  • 7
  • 18
6
votes
1 answer

Learning Design of Experiments

I work helping analyze large amounts of sequence data and I am trying to learn more design of experiments. However, something that I have observed is that, even though design of experiments is very important in some - smaller scale - realms, E.g.…
ramiro
  • 133
5
votes
1 answer

When would you use an I-optimal design over a D-optimal design?

We were discussing in DOE designs that it was best for model development to us d-optimal designs to identify your design space. The common alternative would be an i-optimal design. When would it be advantageous to do i-optimal design?
bobthejoe
  • 151
5
votes
1 answer

Number of times to run a lengthy experiment

I have this long running experiment. Each time I run it I get a new goodness value, since the algorithm has random variables in it. So I need to report the mean and the std of some n runs. What should n be? I need to be able to defend n based on…
5
votes
3 answers

How to design an 8-run experiment in 5 factors?

I'm trying to design an experiment where I measure a variable as a function of 5 two-level factors, labelled A, B, C, D and E. I'm trying to understand how to best design this experiment so I can conduct it in 8 runs. I've tried to follow the…
lindelof
  • 1,119
5
votes
2 answers

Designing Computer Science experiments (Hypotheses and errors?)

For my master thesis I want to set-up several experiments. One thing my professor has been complaining about is that most computer science experiments are lacking scientifically. To not fall into this trap I want to properly design my experiments.…
Roy T.
  • 213
4
votes
1 answer

Designing an experiment where participants see all levels of one factor, but not of the other

Let's say I have an experiment with two factors A and B, each with a number of different levels. I want participants to see all levels of factor A exactly $n$ times, never see a level of B twice How would I generate a test plan that assigns me…
user13907
  • 697
4
votes
1 answer

Explanation of picking an orthogonal array in the Taguchi method

I'm working on my thesis and I am trying to lower the amount of runs I need to do in an experiment so I thought I'd use the Taguchi method, however I don't understand how to use the selector table. I've been over and over how this is supposed to…
Luis Armando
  • 141
  • 2
4
votes
2 answers

Experimental design: what to do when factors cannot be held at constant levels?

I am looking at some data with a 5 factors and a response variable. The experiment was designed as a factorial experiment, with observations at different levels of each factor. One thing that I have notices, though, is that there is some variability…
ialm
  • 1,827
4
votes
1 answer

Is the cluster experimental design the same as the block experimental design?

Is the cluster randomised controlled trial the same as the randomized block design? They seem to both have the same goals.
user46925
4
votes
1 answer

How to deal with subjects who cannot receive the treatment in an experiment?

Imagine an experiment where you are trying to establish that cell phone users who receive a a local offer tend to spend more. How do you deal with those in the test group that cannot receive the treatment (say, there are no local offers when they're…
user765195
  • 2,215
3
votes
1 answer

Treating factorial design as complete randomized design

Lets say I have 10 different treatments and these 10 treatments can be considered combinations of 2 factors - as in a factorial design experiment. What do I miss if I treat this data as coming from a complete randomized design instead of a factorial…
Igelkatt
  • 107
1
2 3 4 5 6