Role of competitive effect in the separation mechanism of matrine and oxymatrine using commercial NF membranes

[Display omitted] •The rejection mechanism of alkaloids during nanofiltration was investigated.•The transport of OMT was affected by MT for the intermolecular interaction.•MDS explained the effect of intermolecular interactions on the rejection during nanofiltration.•The membrane process achieved partial separation of alkaloids with similar structures. Sophora flavescens contains two alkaloids, matrine (MT) and oxymatrine (OMT), which have similar structures but distinct functions and require separation. The present study employed four nanofiltration membranes, namely NT101, NT103, NF270, and NF90, in order to explore the rejection mechanisms of MT and OMT. The behavior of single-component and binary mixtures was assessed during nanofiltration at different pH values. To systematically compare the intermolecular interactions between different alkaloids and the membranes, molecular dynamics simulations (MDS) were conducted. The results indicated that rejection by NT101 and NF270 was primarily influenced by electrostatic and hydrophobic effects. Owing to their relatively smaller pore size, rejections for NF90 and NT103 remained consistently high under the sieving effect. In addition, binary aqueous solutions consisting of MT and OMT can be separated by NT101 (separation factor of 5.46 after secondary filtration), probably because alkaloids compete for adsorption on the membrane. MDS demonstrated that the presence of MT increased the distance and weakened the interaction between OMT and the membrane, increasing OMT rejection. This study provides insightful guidance for the effective application of membrane technology in the separation of natural products.