Perhaps I can share some unsolicited advice.
Like u, I meet all types of people during work, and in my normal course of life. In work, I tend to have higher tolerance level for nonsense simply because there is a certain level of professionalism I need to maintain in a service industry.
In life, I like to spend time with people I can tcss. Consequently, I am also quick to cut off people becaus I have limited time.
Naturally, if u were to spend time with people u like, u will enjoy ur time with them. On the hand, if u find it difficult to reject people, u are going to waste a precious off day and ur social points on the wrong people resulting in social fatigue.