As a photographer, I've always felt like making a model feel comfortable and at ease is one of the most important parts of trying to capture a good set of images. As a human being, it's simply not appropriate or fair to offer any kind of feedback as to a model's appearance based on personal preferences.
I think that oftentimes instances such as those being described in the OP are not typically intended to cause any upset or compromise a model's confidence, but are instead the result of poor social skills or awareness. That's not to say that these comments cannot have an affect on an individual, but in realising that they aren't said with malice is an important part of letting go of any hold the comments may have on you.
It is fair to say that every individual is entitled to their own opinion, and their own preferences. And these will vary wildly person to person. When somebody does pass comment on something inherently personal to you, it's important to frame that in the context of their individual opinion. And quite frankly, who gives a f**k what some random photographer thinks 😂
I appreciate that it's not always easy or immediately productive to say to somebody in the moment "I don't think that comment was fair" or "I was a little uncomfortable with what you said", but I think it would almost certainly benefit the individual if after the shoot, something diplomatic was sent their way such as:
"Thanks for a great shoot, I'm really looking forward to seeing the final images! I did just want to mention that I'm not sure I gave my best work for the entire shoot as I was a little upset that you said x at one point. Obviously I did my best to put it out of my mind, but if we do get the chance to work together again I think it would help me if there were fewer comments that might affect my ability to produce the best work for both of us. I've got some exciting ideas that I think you'd be great at capturing and I know you've got some great ideas as well so I do hope that we do get the chance to work together again at some point!"
Something like the above would be unlikely to be considered an attack or complaint and thus unlikely to illicit a negative/defensive response. The intimation that you would still like to work together again would negate the possibility that raising the issue could cost you future bookings. I also think that the tone of something like the above would be mindful of the fact that the photographer was most likely oblivious to what they were saying. However, giving this feedback is a pretty important step towards trying to change the way they conduct themselves during shoots for your sake and the sake of any other models they work with.
And of course, if they do kick off because of a diplomatic and fair message offering feedback then f**k 'em - negative reference on PP and cut them out of your life because nobody needs to accept that kind of negativity in life.
In regards to positive references - whilst they are a useful indicator as to whether somebody is worth a positive public reputation, they are obviously very biased in nature. Individuals of all disciplines are likely to leave positive references that they otherwise wouldn't for a number of reasons: to be diplomatic, to court a reciprocal positive reference, because they don't want to be contrary, to court future work etc. It might also be that the photographer has legitimately deserved all their positive references, but then - for whatever reason - has a bad day or a lapse in judgement on one specific shoot and says or does something they shouldn't. I think the most important thing with positive references is that you do not let them influence the way you feel about your shoot. If you feel like somebody has behaved or spoken in a way that has otherwise left a negative impression on you, then you need to own that regardless of how other people might have felt about their experiences.
I'm sorry that you've had some less than ideal experiences with your shoot, but I really do feel that you need not take them as a personal reflection on you and merely understand that the way somebody behaves is entirely a reflection on them. Meanwhile, keep up the good work!