I use a couple of flash drives (one for backup) that I sort of keep out of general view for the ones that I can't remember (it's amazing how one can remember some that sort of relate to old memories, etc.) so somebody that wanted them would have to have physical access to my home and computer and then know where to look, etc.
Of course now that I've said that, I may have to change my thinking ;-)