Really dont know what to say and respond on such a question. Its your dog, you know better why it acts like that. The only thing that comes to my mind is, this dog is a male dog who was having a great time with his earlier owner, who was a woman desperate for a dick, any dick. In absence of other males, the dog was getting some fun time, which is illegal act. So he becomes nostalgic and looks for his turn.
What you should do, tie him outside, whatelse can you do? The other option is, sell it off. Rather than spend money on pet animals and feed them, I would rather spend money after feeding poor people.