That will put the speakers out of phase with one another and will destroy stereo imaging.
Yes I've connected speakers out of phase accidentally before . . . does sound very weird but just as an exercise I might try it although as you say the image will suffer.