I think we might be having the same problem. Although I had to pause trying to solve it due to preparations for the apocalypse, and getting my old Dad and very old great-auntie ready and stocked up.
The problem sounds the same: the old PC (laptop) was great but was hitting it's limits. Electrically silent though. I never gave it a thought. The new PC, an absolute cracker (custom built for audio production) is great but has the digital noise problem you mention. Like some super-high-speed modem/data transmission type sound. It alters based on what I do, varying with processor load, or what's happening on-screen, like scrolling some text in a window. It is not a quiet noise either. Very prominent. Interestingly running LatencyMon changes it from a chaotic noise to a steady-ish whine. Do you get the same?
Working with the suppliers, they suggested a ground loop was the likely cause, especially after running the LatencyMon test for some reason.
I have gone through all the standard processes, like running all power in a star formation, ultimately coming from a single wall outlet. Removing as much hardware as I can from the system while diagnosing it, etc. I need to re-visit where we got to, because it was all getting really subtle and complex. For example, removing USB cables from various points helped, but just touching the end of the USB cable to various pieces of metal (USB sockets, other audio connectors elsewhere on the back of the interface, etc.) changed the sound.
In short, between the supplier (who has been great) and I, we hadn't completely nailed it before the Corona virus turned the world on its head. However, the system is in a state that I can use in the meantime. What helped was this:
1. Using an ART DTI box https://www.soundonsound.com/reviews/art-dti
between the interface and my active monitor.
2. Using XLR cables in places where I was previously using instrument cables.
What seemed to make no difference was rearranging the power supplies to all come from the same outlet. To be fair, I was only using two outlets before, but I know it's good practice to use one, so I've left it that way.
I confess, I'm worried that I now have a system that can only work well when things are connected by special boxes, isolators, relatively expensive XLR cables, etc. What happens when I just want to plug in something that only has a TRS connection? Does that means a new isolator box, for example? I've gone from having a silent studio to one that seems to be alive with noise unless I'm really careful. Maybe I was just lucky before and everyone who's been doing this for a long time has found out what I'm just learning.
One point the suppliers made was that moving to a bigger power supply can cause these sorts of problems. It's true that the power supply of this new PC is far beefier than my old laptop. Is that the case for you too?
Watch out for dodgy cables as well. My situation was clouded by a brand new XLR cable connector being faulty. That caused all sorts of head-scratching until i accidently knocked the connector and the noise improved a hundred times.
I'll be interested to see how you get on with yours. Working through it methodically as James suggests is the only way.