This is one that catches everyone out - just hold down the shift key and click on it and the envelope or texture will be removed. Be warned though! There is no undo, or warning, for this action.
