How to deal with a teacher that thinks you cheated?

This is my fault but I didn't realize he would do this. Instead of going right after class to discuss why I had lost points unnecessarily (his ta's grade them so they took off points when they weren't supposed to on my test.). The first thing he points out is that my answers look "erased" and he told me that he would check with the TA since she remembers everything and she takes photocopies blah blah. He was trying to threaten me and I said okay check with your TA because I didn't cheat and then says that "oh I don't think you cheat but that has happened before.". Um wtf? Contradicting himself?? Then, when class started he told everyone to write in pen and everyone was angry and he says, "Sorry that ONE student had to ruin it for all of you." He mean me and this is a MATH/STATS class where I need to erase on my quizzes. This is completely unfair/unjust. He falsely accuses me of cheating. I'm going to talk to him today to see if he did talk to the TA and if he thinks I cheated and gives me a 0 what can I do? No on trusts students, the board would probably agree with him since he has been teaching for a long time. So frustrating!