The problem is that the Brentford area is not great to look at and the pubs aren't really that great. Sorry mate, I wish you well but need to be realistic - you're next to Kew and Richmond, they like rugby and they have nice pubs. Not being snobbish - but sadly the lower league football community is socially a bit scabby - spitting and swearing all over the place, half the supporters will probably fight you if you look at them wrong.
I'm not sure I understand your point Escher. Are you saying Brentford shouldn't build a new ground because fans of lower league teams are "scabs", or is it that Brentford probably won't be able to build a stadium due to the opposition of wealthy residents in neighbouring towns, fearing the hordes of drunken working class oiks?
Apart from that, though, I'd also say you'd be amazed at how many lower league games it's possible to go to without being beaten up or spat on.