[Eeglablist] Statistical test

[Makoto Miyakoshi]

Dear Ali,

It is not totally clear, but the attached figure looks as if you have 2 x 2
x 2 design. In this case, what you need is 3 way ANOVA. Of course, you can
apply t-test from the beginning as planned test, but that can be
recommended only when a small subset of the t-tests are of your interest
(you don't want to repeat it too many times because of multiple comparison
correction issues.)

So, why don't you try repeated measures 3 way, 2 x 2 x 2 ANOVA.

Makoto



On Mon, Jan 23, 2017 at 10:05 AM, ali zahedi <ali.zahedi.bham at gmail.com>
wrote:

> Dear List,
>
> I am working on EEG signal processing and I need to apply statistical test
> on my findings (paired t-test). However, I am not confident in selecting
> the significance level which is usually set as 0.05 or 0.01, and I am not
> completely sure if the paired t-test I have used is correct or not.
>
> I have 20 EEG datasets collected from 20 individuals, and want to classify
> relaxation versus stress. For pre-processing the data I applied a chain of
> 3 different filters (F2,F3 and F4 as shown in the attached figure); which
> resulted in 8 different pre-processing modes. After filtering I used 2
> different classification method (Red and Green bars in the attached figure)
> on the preprocessed data to see which classification method works best.
> Furthermore, I want to see if different filters that I have used have
> significant effect on the performance of the classification methods.
>
> As you can see I have three different factors which may affect performance
> of the 2 classification methods.
> In order to evaluate the obtained results I performed a paired t-test on
> the results.
> In the first step, I wanted to check if performance of the 2
> classification methods is different significantly. Therefore, I divided my
> data into 2 groups of each having 160 samples (20 subjects * 8 modes) and
> performed a paired t-test.
> In the next step, I wanted to check if different filters have significant
> effect on the performance of the classification methods or not. Therefore,
> I divided the data into the following paired groups:
>
> Pair1= Group 1: F2 is applied (F21) Group 2: F2 is not applied (F20)
> Pair2= Group 1: F3 is applied (F31) Group 2: F3 is not applied (F30)
> Pair3= Group 1: F4 is applied (F41) Group 2: F4 is not applied (F40)
>
> Each of the groups in the pairs consist of 160 samples (20 subjects * 2
> classification methods * 4 modes).
>
> In my test I considered p<0.05 as the significant level. I heard that 0.05
> is not always correct to be selected and it might be divided by the number
> of measurements/repetitions due to Bonferroni correction. For example here
> 0.05/3.
> Is Bonferroni correction necessary to be applied? and what would be the
> significance level here? Is the paired t-test correct to use here?
>
> I really appreciate it if you could help me with this.
>
> Regards,
> Ali
>



-- 
Makoto Miyakoshi
Swartz Center for Computational Neuroscience
Institute for Neural Computation, University of California San Diego
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20170124/b140176d/attachment.html>