As part of my ongoing series on the validation of experimental manipulations, I want to propose that we adopt some techniques from pharmacology to improve the practices of experimental psychology.
Pharmaceutical researchers conduct drug trials, in which they administer drugs that contain specific active ingredients (i.e., the substances in a given drug that exert the intended therapeutic effect) alongside secondary ingredients that serve non-therapeutic purposes. There are many secondary ingredients in each drug, but they exist to serve ancillary functions like binding a pill together or allowing for stable metabolism of the active ingredient.
Just as pharmacists explicitly label the active ingredients in each drug, experimenters should identify the active ingredient(s) in their psychological manipulations and explicitly differentiate them from the other aspects of the manipulation that serve secondary roles such as those that serve to bolster the cover story or distract from the deceptive elements of the procedures.
For example, the Cyberball paradigm has participants believe they are interacting with at least 2 other people over the internet, who they will play a simple ball-tossing task with (the other people don't actually exist and are simulated by a computer program). The task is then framed as a mental visualization exercise to get participants to imagine the ball toss 'as if it were happening in real life'. This cover story is a distraction from the key aspect of the manipulation, which is that participants will either receive an equal amount of ball tosses as their 2 compatriots, or will be excluded from the toss by their compatriots who will just toss the ball back-and-forth to each other. In this case, the cover story about mental visualization and the deceptive elements about interacting with other people are secondary ingredients in the manipulation. The active ingredient is the ball-tosses that participants are excluded from. This specific part of the manipulation (i.e., the excluded ball-tosses) is the active ingredient of the manipulation.
The reason to identify these active ingredients is crucial for a validation process I will explain in a moment.
In medicine, it is also crucial to report the dosage of the active ingredient, typically in milligrams. This dosage information is critical to have in order to understand the drug's potency and therapeutic efficacy. Experimental psychologists should attempt to do the same. Once the active ingredient is identified, the units should be articulated (when possible). For Cyberball, each unit of the active ingredient is 1 excluded ball toss. For a fear-learning paradigm, it may be 1 scary image.
These units may often be arbitrary, such as in the case of a humor induction in which participants watch a funny video. The units here could be arbitrary lengths of the video (e.g., 10 second video segments).
For some manipulations, this unit-ification process may be impossible. For example, in one provocation manipulation, participants are given either harsh or pleasant feedback from someone else on an essay they wrote. They receive either a low or high score from their essay rater, which would allow for the creation of some units (i.e., 1 point removed from the total score), but they also receive written feedback (i.e., "WORST essay I've ever read!" or "GREAT essay!"). How could such text be placed into units? In some cases it will simply be impossible, but experimenters should try to develop manipulations that can be articulated in clear units whenever possible, so that they can conduct a dose-response curve.
Pharmaceutical researchers conduct drug trials, in which they administer drugs that contain specific active ingredients (i.e., the substances in a given drug that exert the intended therapeutic effect) alongside secondary ingredients that serve non-therapeutic purposes. There are many secondary ingredients in each drug, but they exist to serve ancillary functions like binding a pill together or allowing for stable metabolism of the active ingredient.
Just as pharmacists explicitly label the active ingredients in each drug, experimenters should identify the active ingredient(s) in their psychological manipulations and explicitly differentiate them from the other aspects of the manipulation that serve secondary roles such as those that serve to bolster the cover story or distract from the deceptive elements of the procedures.
For example, the Cyberball paradigm has participants believe they are interacting with at least 2 other people over the internet, who they will play a simple ball-tossing task with (the other people don't actually exist and are simulated by a computer program). The task is then framed as a mental visualization exercise to get participants to imagine the ball toss 'as if it were happening in real life'. This cover story is a distraction from the key aspect of the manipulation, which is that participants will either receive an equal amount of ball tosses as their 2 compatriots, or will be excluded from the toss by their compatriots who will just toss the ball back-and-forth to each other. In this case, the cover story about mental visualization and the deceptive elements about interacting with other people are secondary ingredients in the manipulation. The active ingredient is the ball-tosses that participants are excluded from. This specific part of the manipulation (i.e., the excluded ball-tosses) is the active ingredient of the manipulation.
The reason to identify these active ingredients is crucial for a validation process I will explain in a moment.
In medicine, it is also crucial to report the dosage of the active ingredient, typically in milligrams. This dosage information is critical to have in order to understand the drug's potency and therapeutic efficacy. Experimental psychologists should attempt to do the same. Once the active ingredient is identified, the units should be articulated (when possible). For Cyberball, each unit of the active ingredient is 1 excluded ball toss. For a fear-learning paradigm, it may be 1 scary image.
These units may often be arbitrary, such as in the case of a humor induction in which participants watch a funny video. The units here could be arbitrary lengths of the video (e.g., 10 second video segments).
For some manipulations, this unit-ification process may be impossible. For example, in one provocation manipulation, participants are given either harsh or pleasant feedback from someone else on an essay they wrote. They receive either a low or high score from their essay rater, which would allow for the creation of some units (i.e., 1 point removed from the total score), but they also receive written feedback (i.e., "WORST essay I've ever read!" or "GREAT essay!"). How could such text be placed into units? In some cases it will simply be impossible, but experimenters should try to develop manipulations that can be articulated in clear units whenever possible, so that they can conduct a dose-response curve.
Identifying the active ingredient in your manipulation and its units allows you to take the next step, creating a dose-response curve. Pharmaceutical researchers administer new drugs at varying doses so that a dose-response curve can be modeled. This allows them to identify (A) at what minimum dosage is there a therapeutic effect, (B) at what dosage is that effect optimized, and (C) at what dosage does the effect level off (i.e., does greater dosage produce no meaningful increase in therapeutic effect).
This is crucial information for experimental psychologists to have as well! For each manipulation, we should know how many units of the active manipulation ingredient are needed to elicit a sufficient manipulation effect on our target psychological process. We should also know how many units of that active manipulation ingredient create the largest (i.e., optimal) effect and at what levels of the manipulation do we stop seeing a corresponding increase in the effect.
To do this, you simply need to administer the experimental condition of your manipulation at varying doses (i.e., units) of the manipulation's active ingredient, holding all secondary ingredients constant. You can then plot the standardized effect of each 'dose' of your manipulation against participants' baseline to create a dose-response curve for your manipulation. This process requires that you administer valid measures of your target construct before and after each level of the manipulation is induced. This is akin to a process seen often in neuroimaging and other disciplines called parametric modulation.
How many different levels of the manipulation should be included? This is up to each experimenter, but more is better. Including more levels will give you a more granular and fine-grained dose-response curve.
No control conditions need to be included in this process, just the experimental one. By manipulating just the parts of the manipulation that you expect to have an effect, holding secondary aspects constant, you can be sure that the part of your manipulation that you intend to have the effect is exerting the effect and not some other aspect.
Identifying the dose-response survey of your manipulation will allow you to select the optimal number of units of your active manipulation ingredient to administer to participants. Doing so will ensure that your manipulation is strong enough to have the desired effect while simultaneously reducing participant and experimenter burden by avoiding excessively large 'doses' that many consume valuable time and resources. Further, if you are manipulation an aversive or sensitive psychological state (e.g., pain), such dose-response curves will allow you to select the experimental dosage that does not induce excessive amounts of that state to the point where it may be unethical and harmful to do so to your participants.
Weighing these concerns will make each study's optimal level of manipulation dosage unique. Though a quick rule-of-thumb used by pharmaceutical researchers is to first identify the drug's maximum effect and then find the dosage that corresponds to 50% of that maximum effect (indicated by the X line in the figure below). Experimenters could select dosages of their manipulations that approximate the 50% mark, though again, this may be different for each study.
This is crucial information for experimental psychologists to have as well! For each manipulation, we should know how many units of the active manipulation ingredient are needed to elicit a sufficient manipulation effect on our target psychological process. We should also know how many units of that active manipulation ingredient create the largest (i.e., optimal) effect and at what levels of the manipulation do we stop seeing a corresponding increase in the effect.
To do this, you simply need to administer the experimental condition of your manipulation at varying doses (i.e., units) of the manipulation's active ingredient, holding all secondary ingredients constant. You can then plot the standardized effect of each 'dose' of your manipulation against participants' baseline to create a dose-response curve for your manipulation. This process requires that you administer valid measures of your target construct before and after each level of the manipulation is induced. This is akin to a process seen often in neuroimaging and other disciplines called parametric modulation.
How many different levels of the manipulation should be included? This is up to each experimenter, but more is better. Including more levels will give you a more granular and fine-grained dose-response curve.
No control conditions need to be included in this process, just the experimental one. By manipulating just the parts of the manipulation that you expect to have an effect, holding secondary aspects constant, you can be sure that the part of your manipulation that you intend to have the effect is exerting the effect and not some other aspect.
Identifying the dose-response survey of your manipulation will allow you to select the optimal number of units of your active manipulation ingredient to administer to participants. Doing so will ensure that your manipulation is strong enough to have the desired effect while simultaneously reducing participant and experimenter burden by avoiding excessively large 'doses' that many consume valuable time and resources. Further, if you are manipulation an aversive or sensitive psychological state (e.g., pain), such dose-response curves will allow you to select the experimental dosage that does not induce excessive amounts of that state to the point where it may be unethical and harmful to do so to your participants.
Weighing these concerns will make each study's optimal level of manipulation dosage unique. Though a quick rule-of-thumb used by pharmaceutical researchers is to first identify the drug's maximum effect and then find the dosage that corresponds to 50% of that maximum effect (indicated by the X line in the figure below). Experimenters could select dosages of their manipulations that approximate the 50% mark, though again, this may be different for each study.
Another pharmaceutical practice is to measure and plot dose-response curves for drugs' various unwanted side-effects. Just as drugs may induce undesired physical symptoms, experimental manipulations often induce psychological states that are ancillary to the manipulation's intended effect (at best) and may undermine the efficacy of the manipulation (at worst). Thus, it is crucial to identify psychological variables that you do not desire your manipulation to induce and to test these alongside the target variable(s).
If the side effects become stronger than your desired effect at a certain dosage, then experimenters will want to stay below that dosage to avoid interference between their manipulation's intended and unintended effects.
Experimental psychologists aren't pharmacists or medical doctors. We shouldn't try and mirror their practices just for the sake of superficial credibility. Yet the dose-response curve approach to titrating the levels of our experimental manipulations' active ingredients is one case where we may stand to gain a lot. The psychological states we manipulate are impactful (or else we wouldn't bother manipulating them) and we want to make sure our manipulations are optimized to protect our participants, yield robust effects, avoid the interference of unwanted side effects, and don't over-burden ourselves or our participants. A dose-response approach allows us to do just that and I hope to see such titration experiments in the literature in future days.
If the side effects become stronger than your desired effect at a certain dosage, then experimenters will want to stay below that dosage to avoid interference between their manipulation's intended and unintended effects.
Experimental psychologists aren't pharmacists or medical doctors. We shouldn't try and mirror their practices just for the sake of superficial credibility. Yet the dose-response curve approach to titrating the levels of our experimental manipulations' active ingredients is one case where we may stand to gain a lot. The psychological states we manipulate are impactful (or else we wouldn't bother manipulating them) and we want to make sure our manipulations are optimized to protect our participants, yield robust effects, avoid the interference of unwanted side effects, and don't over-burden ourselves or our participants. A dose-response approach allows us to do just that and I hope to see such titration experiments in the literature in future days.