I understand the idea of gain matching an active pre but not why it matters with a passive pre as there is no gain stage it is just progressive de-attenuation of the fixed-level signal it receives.

If it is just a fetishitic thing about where the volume knob is on the dial then I get that - but how can where the knob is on the dial actually affect the sound adversely unless, as you say. you run out of steps before reaching the required SPL?