Sure I get that. In isolation that's exactly it, but many aren't in isolation, many of those acts are done whilst waving a Nazi flag, or attending a rally/march, most are accompanied with language and or noises. There are some other attributing factors that you can apply. If you can't apply them or link them, then it makes it hard to prove. But a person in those situations could have a visit by the old bill to their house where they may or may not find something that links him to being a racist, what's on his hard drive, what sites does he visit, what does he say in emails, texts, chatroom and forums. What groups does he belong to etc.etc..that's called an investigation.
The entire point is a photograph frozen in time means fuck all, add to it historical references, a media whose only interest is a) to be First to report it with zero regards for accuracy because clicks mean more $$$ than accuracy b) They might enjoy being a provocative/antagonistic media. Then add a baying public who are absolutely focused on being part of the righteous (no matter what side of the argument they are on) and you have a bloke misjudged, falsely accused, maliciously attacked, all debated by people who weren't even there, who's only reference is a still photograph, a frozen moment in time taken from one angle.
It makes everyone look like utter cunts. I've been there, I've been a cunt I've been judgemental too many times to count, I will probably will do so again, but I've done it enough times to know that tt's just time to chill gather the evidence, asses it calmly explore other angles and trust experts who have access to everything to investigate it. (That last bit is hard for some to do, including me as I've witnessed Police racism first had multiple times but that's a whole other can of worms).