How to test for differences between groups in EEG trial data?

Question

Experiment Setup

Each participant must perform volutary biceps flexions (each flexion is referred to as a trial $t_i$) while staring at a screen. EEG and EMG signals are recorded throughout the experiment.

There are two conditions (i.e., A and B), which determined what was shown on the screen during the trials. The number of trials per condition per participant ranges from 15 to 70.

Finally, there are two groups (i.e., G1 and G2) with 10 and 2 participants, respectively.

Graphically, the structure would be this. Note each participant experience both conditions (e.g., subject 1 from condition A is the same subject 1 from condition B).

Hypothesis

I would like to test for the difference, in multiple dependent variables, between G1 and G2 given each condition. All dependent variables are numerical, but some are unbounded (e.g., event related desynchronization), some are strictly positive (e.g., power spectral density), and some are bounded (e.g., cortico-muscular coherence $\in$ [0, 1])

For some context, there are about 10 quantities that I would like to analyze (e.g., power spectral density, median frequency of the spectrum, EMG amplitude, peak readiness potential, peak event related desynchronization) for each condition. So, I would be testing approximately

10 dependent variables $*$ 2 conditions $=$ 20 group differences

Ideas/observarions so far

• One common approach in the field is, for each quantity of interest and each condition, to average trials within a subject and then perform a test (e.g., t-test) between the two groups. However, I have been discouraged from doing so because of the low amount of participants in group G2 (i.e., 2).

• Although I want to test for differences given a condition, I wonder if building a model that incorporates both conditions at the same time would be appropriate.

• Maybe a Bayesian model would be desirable given how scarce data is. Under this framework, I had thought about a hierarchical linear model with trials as the observation unit, and a per group mean response.

I would appreciate any guidance about how to appropriately conduct the tests I want to perform given the low amount of data available, namely what model/framework to use, and why.

Also, as I need to back up any decision, pointers to papers and/or code are highly appreciated as well.

Thank you in advance!

EDIT: Clarified types of outcomes.

Answer 1

I'm sorry to say it, but this analysis is doomed before it starts.

You have only 2 participants in your smaller group, and 15-75 trials per condition per participant. Between-group differences in EEG signals are typically extremely small and noisy, and the chances of detecting a genuine effect in such limited data is essentially zero.

What's worse, since you're planning on looking at 10 different outcome variables, in two different conditions, there are lots (20) of opportunities for false positives to occur. Taken together, this means that any significant differences you do find are considerably more likely to be false positives than genuine effects (and if they do happen to be genuine effects, they'll be huge overestimates).

As a side note, between group effects are notoriously difficult to study using EEG. It's almost impossible to rule out the very strong possibility that any differences that do emerge reflect differences in motor artefacts, in electrode placement or signal quality, or even hairstyle. There are very few between group findings in the EEG literature that are believable. Luck's EEG textbook has a lot of good advice on good EEG designs for hypothesis testing. You may also want to look in power analysis for EEG designs.

Further Comments

• Building a model that incorporates both conditions doesn't solve your problem here, since you want to compare the two groups and one of them has almost no data.

• Similarly, Bayesian methods are nice, but they don't solve the problem. They provide a way of quantifying the uncertainty in your model, but they don't get around the fact that without more data you have no way of reducing that uncertainty.