Part throttle hesitation and shaking sounds like a lean condition.

It's probably a vacuum leak on/around the intake manifold. Something small will lean it way out at idle/cruise where you're not using a bunch of air. Under boost, the effect would be much smaller which is probably why it seems normal.