For adjusting azimuth, I would think measuring at speaker level is not accurate enough beyond "in the ballpark". There's just so many possible variables at that point, exact positioning of microphone, difference in sensitivity between speakers, power amplifier channel balance (likely very good though), volume pot (will vary across the range of the pot if a regular pot is used) & phono stage accuracy come to mind.

All that ignores a big one which is alignment of various bits in the cartridge & alignment of stylus tip relative to coils (if talking MC cartridges) - there's good reason phono cartridges might only have a specified channel balance within 0.5dB or even worse, though I gotta say I'm not sure I've had any carts which were off by more than that.

So, IMO for azimuth adjustment it's best to do it visually and adjust according to the diamond to get most accurate tracking. Also not all cartridges seem to age gracefully, it would seem sometimes things can get out of whack a bit as the suspension components age.

I do think getting azimuth spot on is very important and here from tracking perspective there are no compromises, the better you do it, the better it gets, that simple. In a perfect world no arm would have azimuth adjustment, but would come out perfect from the factory, like all cartridges and we'd never have to worry about it. Sadly in the real world that is not the case. If the cartridge is not put together perfectly, then adjusting azimuth for the diamond you might end up sacrificing some channel separation at the generator. If that happens, it happens. If you have to twist the cartridge more than just the slightest bit for a very small fine tune, you might want to get in touch with either the seller of the cartridge or the manufacturer (assuming a new cart).