Visualisation of Forking Path Analysis CoScience

Paul et. al 2024, FAA in motivational Contexts

Here below select the Effect you want to plot.
You can use the first prompts to filter the selection.

Which Task do you want to explore?

Which Phase do you want to explore?

Which Effect Type do you want to explore?

Which effect should be plotted?

Which variable should be plotted in the specification curve? All data will be sorted according to this.

Effect Size p-Value Sample Size Average Epochs

Show another Variable for comparison?

None Effect Size p-Value Sample Size Average Epochs

Here below select the Forking Steps of interest.
For each selection, the distribution of the different Choices is plotted.

Preprocessing Steps

Resampling

Reference_AC

Bad_Channels

Bad_ChannelsMax

LP_Filter_Early

HP_Filter

LineNoise_Filter

Select All

Epoching_AC

Detrending

Bad_Segments

Run_ICA

OccularCorrection

LP_Filter_Later

Bad_Epochs

Deselect All

Quantification Steps

Reference

FrequencyBand

Cluster_Electrodes

Select All

Electrodes

MinimumData

Quantification_Asymmetry

Deselect All

Statistical Analysis Steps

Attention_Checks

Outliers_Personality

Personality_Variable

Covariate

Outliers_Threshold

BehavCovariate

Select All

Outliers_Treatment

Outliers_SME

Outliers_EEG

Normalize

Center

Determine_Significance

Deselect All

The visualizations follow Simonsohn et al. (2020, Nat Hum Behav) suggestions to visualize a multiverse as a specification curve. Each dot on the line in the top panel depicts the estimate (effect size, p-value, according to selection) from a different Forking Path. Paths are sorted according to this choice. Effect sizes were recoded to include the direction of the effects. The red triangle highlights the main path that was agreed on within the CoScience members. Green dots indicate p < 0.05. In the lower panels, the lines are vertically aligned with each point to indicate the analytic decision behind these estimates according to the selection of the Forking Steps. If certain decisions lead to systematically larger values in the selected variable, the lower panels show clusters, such that one decision has more occurrences on one side than others.