If I recall correctly, if you're not using localsystem then its much tougher, as by default it won't have access to your interactive desktop, just something internal that you won't see, maybe just a hidden windows station.  You're right that the interact setting is only for localsystem.

It's been many years, but I think I had some code that located and attached to the interactive desktop (if one was currently available) from within a background service that I used in a remote management system.  Our remote service (functioned similarly to rshd) also ran under a specific user.

At least in theory I think you might be able to tweak the local buildbot code to execute any tests from within a child process that does the same thing first.  In the end it's only the executing test code that needs the access, not the buildbot itself.

If I get a chance I'll see if I can scrounge up what I did, but will take that off-line from this ticket since it's a broader question.