Ok, I have found the issue. It is due to the dual SIM setup!
A recent update seems to have changed something in the phone. If I have both SIM1 and SIM2 on LTE/3G/2G or SIM1 on 5G/LTE/3G/2G and SIM2 on LTE/3G/2G, I will encounter internet connection loss whenever SIM1 (set to be used as the Mobile Data under Dual SIM mode) switches from 4G+ to 4G. Whether I enable or disable Auto data switching does not change the outcome. Even disabling Data Roaming does not affect the outcome.
However, if turn off SIM2, the problem goes away. Alternatively, if I want to keep SIM2 alive, I need switch it to 3G/2G and the issue does not occur.
Pretty weird... it did not have such an issue previously before the recent updates in Google Play.