I've had my TV connected to the audio gear for years, without problems.
It's a great way of bypassing the crappy TV audio internals.
In my case the TV has a Toslink out, so that is connected to my external DAC, which also has my Squeezebox Touch connected to it.

That is 95% of my audio playback on just one amplifier input - the other input I use is for my turntable setup.
I don't have problems with cracks and bangs.
If I did have such problems I would switch inputs or at least mute the amplifier output before doing anything with the TV. A remote controllable amp is a godsend in such instances.