I feel bad for you. This is not a stable situation. IMHO (in my humble opinion) he is using you. From my POV you need to let go of him. You can find someone much better, who will love you and be kind to you and be yours alone. Until you are ready to accept this, you will continue to put up with his b.s.
Some behaviours are just not acceptable. Everyone has their flaws, however cheating is not generally considered a flaw, in my view, it is a reason to break it off and never see him again. How can you trust someone again, who thinks this is ok?
My advice? Tell him it's over, tell him not to communicate with you in any form, remove him from all your social media, block him on your phone & im applications and try to forget about him. Either he will come to his senses and stop wasting his time with that other woman, break up with her & try to work things out with you. Or the truth of the matter, that he was keeping you hanging on as a back up, will come out.
All the best,