Prevent the remote user from hearing my computer sounds

[JVDG]

I want to prevent any remote user, whether it was me or the other computer that initiated the connection, from hearing any sounds from my computer. I know that the other user can change a setting at their end that will prevent them from hearing my sounds, but I can't rely on that and want to be able to set something at my end that will prevent any of my sounds being streamed.
Muting my sound card stops me from hearing my own sounds on my speakers, but doesn't stop them from going down the wire to the other user and it is really annoying for them.
Does anyone know of a way to do this?