Because this hobby is "anime girl streamers" and men don't come as a part of that, >we don't want to watch them, or hear about them, because >we don't care about anyone other than our oshi and a few friends. Those of us with experience with idols or similar stuff know that it's a red flag when you see interaction between the two sides of an idol business (Which is how Hololive operates, so I'm going to use that as the main foundation for explanation) as it's ALWAYS done for a quick buck for the controversy involved and for shippers to get their jollies off.
Idol stuff is all about intimacy between the idol and fan (Even if it's fake) and introducing more people damages that, especially men. It's always seen as the start of the end for the group in question IRL when management decides it's a good thing to try malexfemale interaction. Even Kpop isn't immune to the phenomenon (See Twice and BTS for that), what's to say vtubing isn't next? Hell, just look at Nijisanji and parts of Hololive history to know that people care about this stuff. It's not about "haha male idol fucks female idol" all the time. It genuinely produces some of the worst content out there because the fan is no longer the centre of attention or intimacy. It makes no monetary sense from a long-term point of view because gachikoi leave out of betrayal, and you're left with short-term revenue from shippers who quickly stop being interested because the element of taboo is gone, which is what really attracted them to the idea.
Then there's the fans, it's always some form of "haha homostar is fucking your oshi lol" or "wow aren't these guys interesting?" when they aren't cute, or girly, or GFE whatever you want to call it (The things you want to see from vtubers which are basically CGDCT but streaming or video games or whatever), and it feels insulting to have your oshi compared to a sex object for someone to use, just another hole. You fuckers meme about leeching but the homoposters back on /hlg/ were unanimously hated because they were off-topic; no one wanted to watch them and their posts were awful, no grey areas or anything, yet still posted their garbage to try and drum up numbers any way they could, moderation deleted them and all.
That's why people feel "threatened". They know from experience it is the start of the end, they're cashing out on the group, the investment is no longer worth it.