Optical color image encryption scheme with a novel DNA encoding algorithm based on a chaotic circuit

•The correlations between R, G, B components of the color image are reduced to nearly zero thanks to the suggested R, G, B scrambling algorithm.•The number of encoding rules in the introduced DNA encoding based algorithm is increased utilizing eight DNA nucleic acid bases with both uppercase and lowercase letters.•Circular bit shifting algorithm and XOR operation with next pixel algorithm are presented.•Numerical analyses demonstrate that the proposed algorithm is able to resist differential and statistical attacks. An optical setup with a unique color image cryptography technique employing a chaotic system is suggested in this study. For this purpose, deoxyribonucleic acid (DNA) encoding is used as an encryption method. Rössler attractor is employed as a chaotic system and it consists of three differential equations. In order to implement the chaotic system by analog electronic components, a circuit design including operational transconductance amplifier (OTA) is also suggested. Circuit simulations have been performed utilizing LT spice program. In the proposed study, the number of DNA encoding rules has excessively been increased compared to previous studies on DNA encoding technique. Thus, the security of the encryption scheme is enhanced. Numerous security tests such as correlation coefficient, histogram, Shannon entropy, secret key and initial condition sensitivities, plaintext and ciphertext attacks and differential attack have been carried out to evaluate functioning of the introduced cryptography algorithm. The numerical test results prove that the suggested encryption algorithm can resist statistical and differential attacks. Moreover, the cryptology scheme indicates the behavior of sensitivity to initial condition and secret key. The presented algorithm has great performance from the point of encoding rules number and being robust against differential attack. Moreover, enhanced double random phase encoding (DRPE) technique is used in optical stage of the study. By modifying the conventional DRPE method, correlation value between R, G, B components for same pixel have extremely reduced. All numerical analyses regarding chaotic system and encryption scheme have been performed in MATLAB program.