Things are defined differently in Gensokyo's universe than they are in our own. Magic is an integral and obvious part of life there, and that is even true to some extent outside of the border (after all, we see a lot of magic come in from the outside world in canon).
However, our own universe is non-magical, and does not rely on magic or even define entities or energies in such terms.
Any entity traveling from one universe to another is essentially traveling between sets of definitions and rules for existence, and so the entity would need to be converted by that universe (or convert itself) into a form that is well-understood by the new universe.
Basically, that would reduce all of the horribly powerful youkai, spirits and even humans to some more manageable form... and in some cases the translation may wind up destroying necessary structures for these beings to exist as sentient entities.
So yes, I would say we would not only win, but we'd never know we were being invaded.