I have nothing against Austin either. Is he one of my favorite One and Done guys? No, not even close. However, he was a massive recruit that committed to Duke when we still weren't seen as the "cool" program. He was the 2nd domino after Kyrie.
The Duke team Austin played on just wasn't a great team, plain and simple. I think what irritated a lot of people was Austin's type of game. He was a score first, score second, pass third type of guard. He came to Duke to score, not to become a well rounded player (from what it seemed.) And the fact that his Duke team didn't really have a point guard, everything snowballed out of of control and I feel like Austin took a lot of the blame because he was the best player.
Could he have committed more defensively? Yes, definitely. What's funny is his niche' in the NBA has become defense, he's elite on that end of the floor.
Will he always be a part of the brotherhood even though his team's loss was super embarrassing and painful? Yes, 100%.