They got older. Kazutaka Miyatake is gonna hit 60 next year. Haruka Takachiho? Gonna be 57 in November. Naoyuki Kato? 56 this year. Kawamori was kind of a late-comer to the group by the time these guys set themselves up.
But more importantly, they have less time to spend with each other because of other commitments. Their popularity in the anime community basically pulled them away. But that's the territory, in any field. The more popular you are, the more projects people try to get you on, the less time you get to spend with the people whom you started out with. It's just the way things are.