[Eeglablist] FDR parameters

[Makoto Miyakoshi]

Dear Nils,

> Can anyone confirm Makoto's and my assumption?

> Actually, when you use pop_newtimef with FDR it outputs something like
"ERSP correction for multiple comparisons using FDR, alpha_fdr = 0.024876"
which I I find a little misleading. Because the number printed is actually
the return value of the fdr function, namely the p-value threshold.

See the highlighted parts below.

This is from newtimef() line 1439

if strcmpi(g.mcorrect, 'fdr')
            alphafdr = fdr(exactp_ersp, g.alpha);

This is from 'help fdr'

>> help fdr
  fdr() - compute false detection rate mask

  Usage:
    >> [p_fdr, p_masked] = fdr( pvals, alpha);

  Inputs:
    pvals   - vector or array of p-values
    alpha   - threshold value (non-corrected). If no alpha is given
              each p-value is used as its own alpha and fdr corrected
              array is returned.
    fdrtype - ['parametric'|'nonParametric'] fdr type. Default is
              'parametric'.

  Outputs:
    p_fdr    - pvalue used for threshold (based on independence
               or positive dependence of measurements)
    p_masked - p-value thresholded. Same size as pvals.

  Author: Arnaud Delorme, SCCN, 2008-
          Based on a function by Tom Nichols

  Reference: Bejamini & Yekutieli (2001) The Annals of Statistics


So you are right, alphafdr should be the 'pvalue used for threshold'

Makoto



On Fri, Jan 27, 2017 at 12:52 AM, Nils Hachmeister <
nils.hachmeister at uni-bielefeld.de> wrote:

> Hello,
>
> thanks for your reply. I agree: The only way I could imagine how to pass
> an alpha value from the GUI to the function is by putting it in the only
> remaining field which is related to that topic. That would be the field
> where I would put my p-value threshold when I were using FDR (or some other
> means of multiple comparison correction). Hence, my, and apparently also
> your assumption is that the field for p-value threshold is somewhat double
> use: It changes its meaning (albeit not its label) when you are doing FDR,
> now taking the FDR alpha value. However, because the alpha parameter of the
> fdr-function is optional it would also be possible that you cannot put the
> alpha value at all, using the GUI. However, then the results should not
> change depending on the value I put into the p-value threshold field, as
> FDR determines its own p-value threshold and the value I put into the
> p-value threshold field should be ignored. But the results clearly depend
> on the value I put into that field, which is, in my opinion, a strong
> indication that our assumption is, indeed, true.
>
> Can anyone confirm Makoto's and my assumption?
>
> Thanks
>
> Nils
> Am 26.01.2017 um 23:00 schrieb Makoto Miyakoshi:
>
> Dear Nils,
>
> I pasted the results from 'help fdr'. Apparently, it takes only two
> inputs, p-values and alpha. So if you use any value other than p-values,
> that should be alpha.
>
> Makoto
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> >> help fdr
>   fdr() - compute false detection rate mask
>
>   Usage:
>     >> [p_fdr, p_masked] = fdr( pvals, alpha);
>
>   Inputs:
>     pvals   - vector or array of p-values
>     alpha   - threshold value (non-corrected). If no alpha is given
>               each p-value is used as its own alpha and fdr corrected
>               array is returned.
>     fdrtype - ['parametric'|'nonParametric'] fdr type. Default is
>               'parametric'.
>
>   Outputs:
>     p_fdr    - pvalue used for threshold (based on independence
>                or positive dependence of measurements)
>     p_masked - p-value thresholded. Same size as pvals.
>
>   Author: Arnaud Delorme, SCCN, 2008-
>           Based on a function by Tom Nichols
>
>   Reference: Bejamini & Yekutieli (2001) The Annals of Statistics
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>
> On Wed, Jan 25, 2017 at 12:55 AM, Nils Hachmeister <nils.hachmeister at uni-
> bielefeld.de> wrote:
>
>> Hi everyone,
>>
>> I'm using the study structure and aim to generate different plots
>> including statistical testing (non-parametric) with multiple comparison
>> correction, namely FDR. I used FDR before (in a context unrelated to
>> eeglab) and it is my understanding that you pass a parameter alpha to it.
>> FDR returns a (common) threshold for p-values which guarantees that the
>> expected rate of type-I errors among all rejections of the 0-hypothesis is
>> smaller or equal to alpha.
>>
>> Now, in the statistics menu in the eeglab gui and the documentation at
>> https://sccn.ucsd.edu/wiki/Chapter_06:_Study_Statistics_and_
>> Visualization_Options I cannot really find that notion. There is no
>> mention of a parameter nor any constraints regarding the rate of type-I
>> errors. However, I noticed that when passing a value to the edit-field
>> labeled threshold the results change pretty much in a way consistent with
>> that field being used as FDR-alpha. But from the documentation I cannot
>> confirm this assumption. Can anybody here confirm this?
>>
>> Best
>>
>> Nils
>>
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to
>> <eeglablist-unsubscribe at sccn.ucsd.edu>eeglablist-unsubscribe at sccn.uc
>> sd.edu
>> For digest mode, send an email with the subject "set digest mime" to
>> eeglablist-request at sccn.ucsd.edu
>>
>
>
>
> --
> Makoto Miyakoshi
> Swartz Center for Computational Neuroscience
> Institute for Neural Computation, University of California San Diego
>
>
>


-- 
Makoto Miyakoshi
Swartz Center for Computational Neuroscience
Institute for Neural Computation, University of California San Diego
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20170127/b623727a/attachment.html>