Sorry if this is patronising to many but to begin with I'll assume zero knowledge on the part of the reader...
So what is a power supply and what qualities do we want from it?
Everything powered by the mains needs a PSU (power supply unit) and even in the case of battery power, the batteries are then the power supply. A PSU
is NOT only an external box which supplies power to another piece of equipment. It will often be built into the equipment and even in the case of an external PSU there will often be more circuitry associated with the power supply within the main unit.
The perfect power supply will give a precise DC voltage which does not vary under any circumstances and has zero noise superimposed on the DC.
Imagine a perfect 10V DC PSU. The 10V output would not vary with time, temperature, incoming mains voltage (if it's mains powered of course) and most importantly
it will not vary with the current being drawn from it by the load.
This last quality is known as a PSU's regulation. All PSU's, even batteries, have as one of their characteristics their regulation but this is not the same thing as having a voltage regulator... other terms such as the "stiffness" of a supply, its internal resistance, source resistance or source impedance all refer to
basically the same thing. The perfect supply has 0% regulation, meaning infinitesimally low internal resistance. This means it would maintain exactly 10V whether it had no load on it or 1000 Amps being drawn from it. Think of the 10V being 10 foot of rope. It should stay 10 foot exactly even with 1000kg weight hanging of it. (I shall use analogy here and there to try and make things easier to grasp so apologies for the clumsiness of some of them! In this case the less than perfect rope would stretch to more than 10' but the less than perfect supply "sags" under the load and gets to be less than 10V...)
Why is it so important that a PSU be "stiff", have excellent regulation etc?
Imagine an amplifier with 3 stages of amplification, typically that means that at it's most basic we have 3 transistors (or valves) each with a particular gain (amplification factor). Lets say each has x 10 amplification so in total we have 10x10x10 = 1000 x amplification of the input signal.
Now the problem is that an amplifier has more inputs than just the one labelled "input"... The point where each of the 3 stages takes its power from the PSU is also an "input", of much less sensitivity than the actual input but present nonetheless. Imagine that a 1mV signal in the form of a kick drum beat from a cartridge is the input. This will be amplified 1000 x and appear at the output as a 1V kick drum beat. The final stage obviously has the biggest signal as it's amplified by 10 by each stage and to give this 1V signal out the final stage must draw the most current from the PSU.
If the PSU has poor regulation, ie is not "stiff" and has a high source resistance, then the 10V from the PSU will drop to lets say 9.9V in sympathy with the drum beat... The first stage is being powered from the same PSU and so it also experiences this drop down to 9.9V. The original 10V minus the drop to 9.9V gives us in effect another signal to the first stage of 10-9.9=0.1V or 100mV, 100 x more than the 1mV actual input from the cartridge! This will mainly appear at the output of the first stage, forgoing its x 10 amplification but still then being amplified by 100 x by the two following x10 stages.... In a severe case such as the example given we get "howl round" in the same way as a mic too close to the speaker in a PA system, but of course all happening in the electrical domain rather than via sound. (it would most often be at an ultrasonic frequency beyond even bat hearing in practice).
Noise on the DC Voltage gets into the circuitry by the same method.... Note. In electronics "noise" refers to anything present which should not be there but is not related to the signal. So we mean mains hum at 50Hz, the frequency doubled rectification products at 100Hz, clicks pops, hiss etc etc
Now, depending on circuit topology, real amplifier stages vary hugely in how susceptible to the above problems they are... in the degree to which their power input is also a signal input if you like. This gives us a figure called Power Supply Rejection Ratio, or PSRR. As a brazen generalisation, the simplest "purest" circuits with the fewest parts
tend to have the worst PSRR (with the exception of a simple triode valve stage, which
can have a useful degree of PSRR inherent to it. Still not that good in this respect though) and much more complicated circuitry is needed to give better inherent PSRR. Op amps tend to have extremely good PSRR.
At the two extremes a very simple purist input stage for a very high gain circuit like a phono stage could require a really superb PSU to even work without obvious mains hum, hiss and pops and a state of the art PSU to sound really good. whereas at the other extreme some op amp circuitry could be powered by a really crap PSU with poor regulation and loads of noise and yet be impervious to it.
Sorry to technical types for all the ifs, buts, caveats etc I've glossed over or not mentioned but this is aimed at the layman.
I'll let folks absorb that and answer any questions arising, things needing clarification etc before moving on