The only concern with your arrangement is that the amplifier that is not selected is not given a load. The danger is that this amplifier could be switched on by accident without a load. To provide 'safety' loading would necessitate the use of an 8-pole double throw 'break before make' switch, and one where the time between 'break' and 'make' is very short.
As to the possible degradation of the signal through the insertion of another set of contacts, well provided the current rating of the switch is high enough the effect will be minimal.
I once built a switch box to switch the output of my amplifier to either speakers or headphones. I used a military (aviation) grade, 20A rated, Cutler-Hammer 4PDT switch. When switched to the headphone position, the amp was loaded by 10W 8Ohm metal-clad resistors, with the headphones fed via series 0.5W 100 Ohm resistors.