Is it bad that I don't want to hang out with my friend because....?

...When he hangs out with his friend girls he let's them drag him around and they tend to be immature. Even if we have to do something or go somewhere he will just sit there and let them play around with him and he just says "Okay" or "I can't right now" with a smile on his face. Me or my other buddies will have to actually yell at his face until he gets up and then he gets mad at us.