The economic and ecological impact of stationary energy storage systems is significantly influenced by battery degradation, which can be categorized into calendar and cyclic aging, each driven by different stress factors. The significance of these stress factors and their effects on battery degradation are application-specific, meaning that the development of aging-sensitive operating strategies is becoming increasingly important.
Both model-based and data-driven machine learning operating strategies have gained prominence, highlighting the need for their comparison and evaluation. A key challenge for both approaches are 1st the identification of relevant stress factors and 2nd the efficient and effective incorporation into operating strategies. In stationary applications, such as photovoltaic battery energy storage systems (PV BESS), additional optimization targets—beyond minimizing battery aging—include maximizing system efficiency, increasing self-consumption, and enhancing self-sufficiency. These objectives often involve trade-offs. Multi-objective optimization is therefore a suitable method to balance these competing goals and translate them into effective operating strategies. In the context of multi-objective optimized operating strategies, reinforcement learning agents are particularly suitable.
Comparing model-based multi-objective and e.g. a multi-objective reinforcement learning agent with the same optimization problem becomes essential in this context, as it has not been sufficiently investigated to date.
Both strategies offer different strengths in addressing these trade-offs, with model-based methods providing structured insights and machine learning strategies offering adaptability and real-time learning potential.

For the application, three distinct operating strategies were developed, implemented, and subsequently compared. First, the greedy approach for maximizing self-consumption and serving as the state-of-the-art reference, was implemented and optimized concerning state of charge boundaries of the battery. Second, a model-based algorithm with an integrated load prediction algorithm was designed as an alternative operating strategy. For this, the electrical behavior of the battery is modeled using a 2-RC equivalent circuit model. For the thermal modeling, a zero-dimensional approach was selected. For modeling the aging behavior, a semi-empirical generic model approach was used and parametrized using the results of a multi-stage aging measurement campaign.
As data-driven machine learning algorithm a reinforcement learning agent was developed, based on a double deep Q-network architecture, which was further enhanced with weather forecasts. The environment of the reinforcement agent contains a simplified battery model based on the electrical and thermal models described above, whereby the aging model was adopted without simplifications. This allows battery effects to be included into the training process and thus enables the continuous consideration of aging, extending common implementations.
The three operating strategies were then compared to assess their potential for use in PV BESS systems and to identify suitable trade-offs between the different target indicators. For this purpose, their sensitivities to parameters of the operating strategies, such as e. g. maximum state of charge, depth of discharge, charging and discharging power and various use-case scenarios with different photovoltaic and household profiles are examined and their influence on the target values identified.
This enables a comparison of the reinforcement learning agent’s robustness explicitly against volatile PV generation and varying load profiles.