Check out these quotes from Yahoo's article on it.
"Delicate artifacts found in the cave were described as "toy-sized" versions of stone tools made by H. erectus. They suggest that Flores Man retained intelligence and dexterity to flake small weapons with sharp edges, even if its body shrunk over time."
If all their tools were scaled down that would mean the entire population was that size, not just a few of them.
And apperently there were enough of them running around to start legends among other islanders.
"Folklore experts have reported persistent legends of little people living on Flores and nearby islands. Islanders called the creature "Ebu Gogo" and say it was about 3 feet tall."
And about that one skeleton doesn't prove anything....am I the only one that remembers Lucy(or whatever that ape's name was)?