I think what he's trying to say is to "come at him if you know" what you are talking about and not some random person who thinks he/she knows basketball just by looking at it on TV. The reference to the rafters just has to do with knowledge of the game. You don't get five banners by not knowing...