I think I found the source of the idle issue. My clutch was malfunctioning so I yanked the engine out to inspect and decided to pull the IM off to check everywhere for cracks/uncovered vacuum nipples. I apparently cleaned all of the old gasket of the IM but neglected to take all of it off the head mounting surface.

So that is all fixed now, I hope, now I'm just waiting on a replacement PP and clutch.