Basically correct. The whine is likely characteristic of the DC ripple in the output of the alternator. If it's a single package head unit - radion and amp in one box - a bit of ripple in the DC shouldn't make a whine. If the amp is separate from the head, then it's possible that the audio lines from the head in the dash to the amp - probably in the trunk - can pick up the voltage difference between the front and rear of the car, including any AC ripple.
Please clarify if one or two piece.
If the car has an antenne where the other end of the coax is attached to the car ground - satellite or stick FM / AM antenna - or maybe any kind if antenna, the shield is grounded at back of car - that could cause a ground differential. Speakers are not likely because they are balanced - two wires never touching ground - from the amp output (in the radio head or external amp) to the speakers.

Easy to prove it's alternator. Disconnect the field connector on the alternator. Car will give warning on dash because no charge. See if noise is still present. You may now hear different noise from injectors or spark plugs. Stop engine and hook alternator back up.

You should be able to track this down. You'll need a multimeter. With engine running... Set it to the most sensitive AC - not DC - mode. Ripple (whine) is AC. Start across battery terminals. Should be virtually no - a few millivolts - of AC. If significant, then maybe battery high impedance - or even maybe open diode in alternator. You'd need a scope to prove that.

Then, one meter lead on negative terminal, touch other meter lead to car body, engine, etc getting progressively further away from battery. If you see higher ripple, then likely it's a bad connection somewhere.

This may be a bit complicated, but if you can get the idea, it may be a bit clear.