Don't wind the tool down so far, you should have a small mag pick up tool,use that and a seal pick or thin screwdrivers to load the collect into the retainer and then hold the retainer with the screw driver and remove the magnet.
The trick is to get the retainer height just right so it helps grab the collet.