sorry to trim your post down to that, but that phrase sort of tends to stand out like it's neon highlighted. I honestly believe I'm not a racist, but I can tell you what happens when that epithet is thrown out to shut you up, which is its most frequent use; it's a way of saying "you just shut up" in a way that has a fair chance of working.
Here's how it worked for me - it hurt my feelings and I shut up - the first time - so I went back and carefully considered my opinions, I thoroughly searched my heart trying to decide if I actually hated people for being "different". The answer was no. the next time I heard "that's racist" My reaction wasn't silence, it was anger - I had said something with absolutely no consideration to anyone's color or background, but was attacked with the powerful "you just shut up" technique.
after going through one more phase of trying to defend how calling me a racist on some issues is a racist act in and of itself sometimes, I don't even bother with that anymore, I don't CARE what I'm called because my usual MO is toward the betterment of everyone - the song is correct - sometimes "you gotta be cruel to be kind" - challenging someone to grow personally seems mean but it, in fact, most of the time, forces them to become better, stronger (on multiple levels), and often, more humble people.
I've often thrown a famous quote of Dr. Martin Luther King Jr. (not the losers cashing in on his legacy and sowing division everywhere they go)
"I have a dream that one day my four children will be judged on the content of their character, not the color of their skin."
My conclusion from that is that you most certainly CAN find a human being unworthy as a leader, and wanting as a human being altogether, so long as what color he is - is not a factor in your decision.