100 dB? That's a very high level, have to say I don't listen at anything like that, about 75dB average level. I agree 3% distortion in a speaker is not a large amount but who listens at 100dB? That shouldn't be an issue in normal use surely?
The speakers built into my TV are terrible and intelligibility was massively increased just by replacing them with some JBL Control One which can be bought new for about £70. One of the rare occasions I have heard a true 'night and day' difference.
Replacing the JBLs with the 3 way B&O MC120.2 was another step up. I just can't buy into the idea that using a better speaker will make things worse except for the edge case I described previously.