Take off the ASA - drill and tap the hole that you see and put a grub screw in that hole with a ton of loc tite.
Okay - so this is the worst idea ever.
The hole I am speaking of is a relief hole so that if you unscrew your tank any residual air will vent from this hole. It also allows you to unscrew the ASA knob a certain length and vent this pressure as well.
The o-ring on the stem of the On/Off is likely dry or needs to be replaced.