I'm surprised about the speed issues you are experiencing. As a matter of fact, the speed at which the X-Tunes window appears is not related at all to X-Tunes: it's MacOS X that somehow caches X-Tunes on disk when it's not used and the system needs memory. When you need it again, as a consequence, it might take some time to appear. However, if you make it appear again just after, it should pop up immediately. X-Tunes uses almost no processing power, except when fading out.

Or maybe there are some specific AppleScript communication problems between X-Tunes and iTunes that would delay everything on your configurations?