Data Dredging
Data dredging can mean many things and we will cover a couple
of them: sub-group analysis and model analysis that you recycle
through the study.
Sub-group analysis is where after you complete your study, you
break down the study further based on timing of therapy, gender,
age, recovery time, coincident medicine use, etc. Anything is
fair game. At times, this analysis may make sense and show a group
that may benefit from the study in question. To be proven, you
then design a study which controls for all other variables and
compares your hypothesized group to a control group. Extracting
data from a study whose intent was not to control for such a delineation
has a high likelihood of introducing unknown variables or bias
into the analysis.
Model analysis is tricky and always makes me feel ignorant. A
study group may find a minor and non-significant delineation between
groups in question. You may then apply a mathematical formula
into the mix and thus generate a greater difference between the
groups. Commonly, time is a factor in this calculation and you
may see “time has been controlled as an independent variable.”
If you divide or multiply outcomes by time (1,2,3,4,etc.), then
those that have subtle differences further out from time “zero”
will show larger differences because the time integers are larger
the further from zero you are. For example, if the control group
is purported to be found in those near time zero, then multiplying
any outcomes by 1 or 2 will not change the final number by very
much, On the other hand, if the study group is somehow found further
out at time 9 or 10, then multiplying this data will make huge
differences. Then comparing these two groups again which may not
have had much of a difference will show a huge difference. Manipulations
such as this may bring out a subtle difference that is real, or
may over-emphasize a non-true difference. Again, the way to determine
this is to set a study which from the beginning controls for variables
so that you can cleanly study the intervention in question.
