Gym Bandwagon

One thing I have been considering since I made my goal this year to lose weight is to join a gym. My sister and I were all in for joining the gym but now my hubby is wanting to join our gym bandwagon. I do not mind him going to a gym or what not but I honestly had plans for my gym plans to be just my sister and I. I am very shy about exercising and such in front of people and feel stupid to do so in front of my hubby. He has tried to tell me he wanted to purchase testosterone boosters so he could gain muscle while working out and I am excited for him but I still just wanted the gym to be a girls thing. How can I break the news to him without him being upset? I just need a little me time.