The UM2 has a USB connection for connection to a computer, one XLR/Line in combo socket, one line-in socket, and two RCA outputs (by design these are 'unbalanced' outputs) primarily designed for connection to active monitors (or into a power amp driving passive speakers).
There is no XLR output on the UM2 itself, so when you said 'XLR output', did you mean the XLR output of your microphone?
If you have the UM2 connected to active monitors (or a power amp + speakers) and a computer/laptop with a mains connection, then you could be suffering from ground loop noise. The first thing to do is make sure all the equipment is connected to the same power outlet using a multi-way power distribution strip, so that all the equipment has a common ground connection point. This can often cure such noise. Don't use several sockets located around the room to power each item of equipment.
But the noise may be down to a poor ground connection in your room/house/etc. If you've got a plug-type socket mains socket tester, make sure that the mains sockets are all wired correctly. You won't test for a high resistance ground path here, but at least if there are earth connections missing, then this should tell you. Any faults - get them fixed as soon as possible.
As has been mentioned, if you are using a laptop, then running that on its own battery and disconnecting its power lead may cure the noise problem (but is not a long term solution). You may still get some noise issued so disconnecting one speaker's RCA lead will then leave a laptop on battery and only one ground connection path for the UM2 via the remaining speaker. If you still have the noise, then it's not a ground loop problem but something else. If the noise is greatly reduced but there is still some high-pitched noise, then it could be a mix of ground loop and some other issue. But one step at a time.
Assuming that it is ground loop noise, then the easiest way to cure it is by breaking the ground connections between the speakers and the UM2. Because the UM2 only has RCA outputs for the speakers means that you'll need an external line isolator unit that uses a transformer on each signal path to break the connection between the input and output. You can find very cheap RCA line isolators on eBay and the like, but I'd tend to go for a product that's been designed to produce as neutral an effect on the sound as possible and has been tested by a decent audio publication. One such device is the ART DTI box https://www.soundonsound.com/reviews/art-dti
This has all different types of input and output connectors, but it certainly has RCA connectors to make hooking up your UM2 easy. Yes, the unit is more expensive than the UM2 (so you could try cheaper alternatives).
I don't know what your speaker arrangement is, but if it has XLR inputs available, you would probably not have had any noise issues had you spent a bit more money on an interface with balanced XLR or TRS outputs for the speaker connection. This is normally the best connection method for connecting speakers and people rarely end up with any ground loop noise with balanced connection to speakers and a USB connection to a computer.
So if you have speakers or a power amp that have balanced inputs (XLR or TRS) and if you can return the UM2, then getting a different interface with balanced speaker outputs will also probably solve your noise issues (provided that the socket wiring is all OK).