That is a tough question to answer, as everyone is different and so is their situation. My off-the-cuff answer would be that it would be safe to leave him a. when he feels secure, b. when he is able to SAFELY get up and around without help, c. when he can handle his elimination issues by himself.
You may want to try leaving for shorter times...grocery store/pharmacy runs, hair cut, a cup of coffee at Starbucks (all equally important for you) and see how he does. When you are gone you might feel better if he PROMISES to keep his cell phone on his person so that he could call for help should he need it.
He is lucky to have you as a caregiver!!!
Sara Anne