Yeah, I've noticed this too.
The best solution price-wise is to just get a crappy second-hand system and run the brain as essentially a MIDI thru, but the pads on cheaper kits are usually cheap and nasty plastic jobs and don't give the same nice rebound as the likes of the Roland ones and the MIDI options tend to be much more limited, and don't always even have MIDI out.
In my experience, controllers have a hard time registering all the velocity changes and hits when you're playing fast stuff with sticks. Every controller I've tried to play really fast taiko style percussion on sounds like shit. I'm assuming you've got to get a nice expensive electric drum set to get the best results.