Firstly, it sounds as if you are drawing conclusions before it's actually clear what is happening, making ou fel maybe more insecure than necessary. Secondly. you were having an affair with a married man --- never assume that they will leave their family for their mistress. That's part of the deal when you choose to become a mistress.
Married men having an affair are using the woman who becomes their mistress. And you were content to risk causing pain and problems to his wife and children, only objecting then the discomfort is caused to you.
btdt, LUST is selfish, love is not. I dont ever buy the idea that it's always totally the fault of the married man, and that all mistresses are wholly innocent little victims who couldn't possibly help themselves --- that's a highly insulting view of women, typical of the married adulterer, but not worthy of anyone who respects women. It takes two to tango.
And you cannot possibly learn from your mistakes, until you acknowledge them as a mistake. And NO married man can "offer security" to his mistress, whatever he says --- it is only insecurity he is offering.