Digital Twin-Assisted PPO-Based Energy Management for PV–BESS Operation in Active Distribution Networks Under Uncertainty

MinhPhong Le

doi:10.55927/eajmr.v5i10.17

Authors

MinhPhong Le Thu Duc College of Technology, VietNam

DOI:

https://doi.org/10.55927/eajmr.v5i10.17

Keywords:

Digital Twin, Deep Reinforcement Learning, Proximal Policy Optimization, Battery Energy Storage System, Active Distribution Network

Abstract

This study proposes a Digital Twin-assisted Proximal Policy Optimization (PPO) framework for PV–BESS energy management in active distribution networks under uncertainty. The Digital Twin continuously synchronizes operating information and validates the network state through AC power-flow calculations, while the PPO agent determines adaptive BESS charging and discharging actions. Monte Carlo scenarios are used to represent variations in PV generation and load demand, while operating cost, voltage deviation, and battery degradation are incorporated into the control objective. The framework is evaluated on the IEEE 33-bus distribution system with 60% PV penetration and a 1.5 MWh BESS. Results show a 22.2% reduction in operating cost compared with rule-based control, together with improved voltage regulation, reduced ESS cycling, and lower operating-cost variability under stochastic operating conditions.

References

Ardebili, A A, M Zappatore, A.I.H.A. Ramadan, A Longo, and A Ficarella. 2024. “Digital Twins of Smart Energy Systems: A Systematic Literature Review on Enablers, Design, Management and Computational Challenges.” Energy Informatics 7(1, Art. no. 94). doi:10.1186/s42162-024-00385-5.

B. Xu, A. Oudalov, A. Ulbig, G. Andersson, and D. S. Kirschen. 2018. “Modeling of Lithium-Ion Battery Degradation for Cell Life Assessment.” IEEE Transactions on Smart Grid 9(2): 1131–1140. doi:10.1109/TSG.2016.2578950.

B. Xu, Y. Shi, D. S. Kirschen, and B. Zhang. 2018. “Optimal Battery Participation in Frequency Regulation Markets Considering Degradation Cost.” IEEE Transactions on Power Systems 33(5): 4908–4918. doi:10.1109/TPWRS.2018.2799437.

Chen, J, X Wang, M Zhou, and Y Fu. 2025. “Battery Degradation-Aware Deep Reinforcement Learning for Energy Storage Scheduling in Distribution Networks.” IEEE Transactions on Sustainable Energy 16(1): 215–227. doi:10.1109/TSTE.2024.3478912.

Glavic, M, R Moreno, T Bi, and M Shahidehpour. 2021. “Reinforcement Learning for Electric Power System Decision and Control: Past Considerations and Perspectives.” IEEE Transactions on Smart Grid 12(6): 5128–5142. doi:10.1109/TSG.2021.3070934.

Guerrero, J M, J C Vasquez, J Matas, L G Vicuña, and M Castilla. 2011. “Hierarchical Control of Droop-Controlled AC and DC Microgrids—A General Approach toward Standardization.” IEEE Transactions on Industrial Electronics 58(1): 158–172. doi:10.1109/TIE.2010.2066534.

Kabir, Md Rafiul, Dipal Halder, and Sandip Ray. 2024. “Digital Twins for IoT-Driven Energy Systems: A Survey.” IEEE Access 12: 177123–43. doi:10.1109/ACCESS.2024.3506660.

Kumari, N, A Sharma, B Tran, N Chilamkurti, and D Alahakoon. 2023. “A Comprehensive Review of Digital Twin Technology for Grid-Connected Microgrid Systems.” Energies 16(14): 5525. doi:10.3390/en16145525.

Liu, Z, Y Zhang, and H Li. 2021. “Digital Twin for Power Systems: State-of-the-Art and Future Trends.” Renewable and Sustainable Energy Reviews 152: 111675. doi:10.1016/j.rser.2021.111675.

Mohamed, Nahed Belhadj, Georges Kaddoum, and Md Zoheb Hassan. 2026. “Digital Twin-Driven Continual Deep Reinforcement Learning for Coexistence of Multiple Radio Access Technology IoT Links with Nonlinear Receivers.” IEEE Transactions on Machine Learning in Communications and Networking 4(February): 591–611. doi:10.1109/TMLCN.2026.3669837.

Musa, Usman Ibrahim, and Sukanta Ghosh. 2023. “Advancing Digital Twin through the Integration of New AI Algorithms.” International Research Journal of Engineering and Technology: 922–32. https://www.researchgate.net/publication/369693333.

Nneoma Ugwu, Chinyere, Fabian Chukwudi Ogenyi, Jovita Nnenna Ugwu, Paul Chima Okechukwu Ugwu, and Micheal Ben Okon. 2026. “Integration of AI-Driven Digital Twins for Real-Time Optimization of Renewable Energy Grids.” Frontiers in Energy Research 14(February): 1–13. doi:10.3389/fenrg.2026.1748233.

Parisio, A, E Rikos, and L Glielmo. 2014. “A Model Predictive Control Approach to Microgrid Operation Optimization.” IEEE Transactions on Control Systems Technology 22(5): 1813–1827,. doi:10.1109/TCST.2013.2295737.

Ton, T N, P M Le, and T M Le. 2026. “Multi-Objective Optimization of Distributed Generation Placement and Sizing in Active Distribution Networks Considering Harmonic Distortion.” International Journal of Electrical and Computer Engineering 16(2): 598–607,. doi:10.11591/ijece.v16i2.pp598-607.

W. F. Mbasso, A. Harrison, I. Dagal, P. Jangir, M. Khishe, H. Kotb, M. S. Shaikh, A. Smerat, E. F. Donfack, and R. Kumar. 2025. “Digital Twins in Renewable Energy Systems: A Comprehensive Review of Concepts, Applications, and Future Directions.” International Journal of Electrical Power & Energy Systems 61(101814). doi:10.1016/j.esr.2025.101814.

Wang, Y, X Liu, J Zhao, and M Shahidehpour. 2024. “Digital Twin-Driven Resilient Operation of Active Distribution Networks With High Renewable Penetration.” IEEE Transactions on Smart Grid 15(2): 1648–1661,. doi:10.1109/TSG.2023.3334869.

Zhang, Z, J Wang, and B Chen. 2021. “Deep Reinforcement Learning for Energy Management in Microgrids.” Applied Energy 288: 116618. doi:10.1016/j.apenergy.2021.116618.

Zhou, K, S Yang, and Z Shao. 2021. “Digital Twin-Based Optimization for Energy Systems.” Applied Energy 285: 116419. doi:10.1016/j.apenergy.2020.116419.