You could just call him and say something to the effect of "I am so sorry, but something has come up and I am going to have to cancel your tour. I will give you a call when my schedule can accommodate your request."
That will leave it open ended with the ball in your court, in a very polite...