I know (1st or 2nd degree) a bunch of physicists that were annoyed by him. Mainly by his antics. Gell-Mann has funny stories about this. He was Yin to Schwinger's Yang (I am not punning on CN Yang) in that Schwinger was very much about the mathematical formalism and rigor. Dirac was also like that. Also, many physicists at that level take very seriously the time we have on earth and dillydallying is frowned upon. Feynman messed around quite bit.
For me, personally, he was very influential. I found his curiosity and inquisitiveness really tantalizing.
He's known for diagrams, which technically pre-date Feynman, but that he popularized. The diagrams are really helpful. Solving certain things that keep cropping up in field theory gets really very, very redundant and tedious. Pages and pages of calculations. These techniques made leaning field theory easier and made time for other thinking.
I also liked that Feynman put himself out there. He ran a lecture where he tried to get a theory of gravity from the ground up. Of course, it didn't work, but you get to see his mind attack the problem. Normally a theorist would only show polished solutions to the world. Some kids at Caltech took notes and turned it into a book (Lectures on Gravitation).
Whatever catastrophe or dynamic equilibrium this will eventually lead to will be a mathematical not a moral phenomenon. - A Fryer