Molten salt is widely adopted in diverse thermal energy storage systems owing to its exceptional thermodynamic properties and economical cost. As a critical component in molten salt energy storage systems, the exchangers often utilize U-tube configurations for enhanced compactness, such as shell-and-tube designs. However, the high viscosity and density of molten salt can cause non-uniform flow distribution in U-tubes, posing localized overheating risks. This study proposes a heat transfer enhancement strategy applying a twisted cloverleaf U-tube in combination with molten salt-based nanofluids (MSBNs). The effects of tube geometry, operating parameters, and nanofluid thermophysical properties on flow and thermal performance were analyzed through numerical simulations. Multi-objective optimization of operating conditions was conducted using a combination of response surface method (RSM) and the non-dominated sorting genetic algorithm II (NSGA-II). Results indicate the twisted structure and nanoparticles significantly enhance heat transfer and improve temperature uniformity, however increase pressure drop. The optimal combination achieved a peak performance evaluation criterion (PEC) value of 1.21. Inlet velocity and inlet temperature influence flow and heat transfer performance additional strongly than heat flux. Optimized operating conditions yield a maximum temperature difference of 40.15 K, pressure drop of 1979.97 Pa, and average convective heat transfer coefficient of 2781.31 W·(m²·K)⁻¹. This work provides critical guidance for the design and operational optimization of novel MSBN heat exchange tubes.