The idea of gaskets between drivers and baffles is to make an airtight seal. Not vibration damping. Although there will be a minor element of this. Drive units should (generally) be firmly enough fixed to the baffle that vibration due to looseness cannot happen (although all speaker parts will resonate).

Craft foam sheet from art shops is good for gaskets, as is self adhesive foam strip. Both can be had in black.