Electrolytes and the associated electrode-electrolyte interfaces are crucial for the development and application of high-capacity energy storage systems. Specifically, a variety of electrolyte properties, ranging from mechanical (compressibility, viscosity), thermal (heat conductivity and capacity), to chemical (solubility, activity, reactivity), transport, and electrochemical (interfacial and interphasial), are correlated to the performance of the resultant full energy storage device. In order to facilitate the operation of novel electrode materials, extensive experimental efforts have been devoted to improving these electrolyte properties by tuning the physical design and/or chemical composition. Meanwhile, the recent development of theoretical modeling methods is providing atomistic understandings of the electrolyte’s role in regulating the ion transport and enabling a functional interface. In this regard, we stand at a new frontier to take advantage of the revealed mechanistic insights into rationally design novel electrolyte systems. In this review, we first summarize the composition, solvation structure, and transport properties of conventional electrolytes as well as the formation mechanism of the electrode-electrolyte interphase. Moreover, some of the promising energy storage systems are briefly introduced. Further, approaches to stabilize the electrode-electrolyte interphase using novel electrolyte design, including electrolyte additives, high-concentration electrolytes, and solid-state electrolytes, are discussed. Some recent advances in the atomistic modeling of these aspects are particularly focused to provide a fundamental understanding of electrolytes and a comprehensive guide for future electrolyte design. Finally, we highlight the prospects of theoretical screening of novel electrolytes.