Reducing Endogeneity Within a Small-n Design Using Cross-Validation with Cluster-Robust Variance Estimation When Comparing Mixed Effect Models: A New Analytic Methodology Applied in Neurobiology

Data from this pilot study was purposed initially to assess post-synaptic behavioural effects of neuromodulation via intermittent theta-burst stimulation in combination with N-methyl-D-aspartate (NMDA) and Gamma-aminobutyric Acid (GABAa) receptor-modulating psychopharmacology on implicit motor learning outcomes. In the current study, a secondary exploratory analysis using cross-validation (both Replicated k-Fold and Leave-One-Cluster-Out) was conducted to compare linear mixed effect models to reduce errors caused by clustered endogeneity. Afterwards, cluster-robust variance estimation was added to this finalized working model to control for endogeneity present in the study design. Initially, the study's design comprised a small sample size (n = 13) in a clustered randomized trial design, which was collected at Butler Psychiatric Hospital in Providence, Rhode Island, under the principal investigation of Dr. Joshua Brown for the Department of Psychiatry at Brown University. Following the preprocessing and organizational stages of data preparation, data from a limited number of participants (n = 8) was retained as a change was made in the block order protocol during data collection. Participants were randomly assigned to four different drug conditions (Lorazepam (2.5 mg; LZP) vs. D-Cycloserine (100 mg; DCS) vs. D-Cycloserine (100 mg) + Dextromethorphan (150 mg; DCS + DXM) vs. Placebo) across four sessions and were tasked with completing the Serial Reaction Time Task (SRTT) before (referred to as the ‘skill acquisition’ phase or “pre-iTBS”) and after (referred to as the ‘retention’ phase or “post-iTBS”) an intermittent theta-burst stimulation (iTBS) intervention, a form of transcranial magnetic stimulation. Results of these data via linear mixed effect modelling revealed a reduction in response times when in the DCS condition paired with the iTBS as compared to all other drug conditions. Oppositely, there was an increase in response times in the LZP condition. The process of cross-validation yielded a design-driven LMER that was attuned at the 'session'-level by random assignment of the drug condition over a number of trials (96 per session for each participant) and each block order of the SRTT. Below is an R-Script includes instructions for replication should any researcher wish to pursue such with this small-n design pilot study.