Stability of microgrids became an important issue especially with increasing the penetration of the inverter-based renewable energy sources (RESs) that characterized by their low inertia and naturally intermittent generated power. These two issues can be tackled by the utilization of the energy storage systems (ESSs), power electronics, and control techniques. Using a single type of ESS may fail to fulfill the system requirements, therefore a hybrid energy storage system (HESS) consists of supercapacitor and battery is employed. The proposed microgrid consists of photovoltaic (PV), diesel generator, HESS, and load. The HESS is controlled by virtual synchronous generator (VSG) technique, which imitates the performance of conventional synchronous generator. However, the frequency response of the microgrid strongly relies on the VSG's parameters. Design based on the small-signal model of VSG requires extensive analysis due to interaction among different generations and loads. Therefore, this work exploits particle swarm optimization (PSO) for designing optimal VSG using two objective functions. The first (OVSG_1) minimizes the integral time absolute error (ITAE) of the frequency, while the second (OVSG_2) considers the ITAE, frequency nadir, and rate of change of frequency (ROCOF) concurrently. Moreover, the proposed control system comprises droop control integrated with VSG, which enables the HESS to operate as a grid forming at outage of the diesel generation and to maintain the microgrid stability. For evaluating the different VSG controllers, several disturbances such as load variation, solar irradiance variation, and grid following/forming contingencies are carried out. In addition, the paper investigates the battery storage system performance with/without supercapacitor under different disturbances. The overall system is modeled and simulated using MATLAB™ platform.