I have a 660 and a 640. Since I have smallish hands, I got the 640 because there are things I cannot do well on the 660. The 640 is better than my expectations. I took no hits in any of the respects you mention, and it's easier on the left hand with respect to reach and holding barres (both guitars have the same action, and normal-tension strings). My guitar teacher also compared my two instruments and agreed that the 640 is as good as the 660 in every respect.
Of course it depends on the instrument. Some 640s may not be so good.