It's all about good marketing, and a good number of loyal fans who got too much money. In many ways it is about selling mecha design instead of story or character - just look at the popularity of the un-animated S-Gundam when very few have read the novel. The toy+Gunpla marketing machine in Japan is just huge - you got 4 magazines which push Gundam stuff - Gundam Ace, Newtype, Hobby Japan and Dengeki Hobby, and at least 4 brands of sub-companies covering the market: Bandai Toy(MSIA/FIX/SD), Bandai Hobby(Models), Bandpresto(Games/toys), B-Club(Resin) etc. Bandai is the world's 3rd largest toy company after Mattel and Hasbro, and at least 25% (Can't remember the actual figure) of Bandai's sale came from Gundam stuff.
The purpose of the new show is to draw in new fans, and it may happen to generate a few good mecha design so that the toy/model fans would buy it in 10 different scales.
As for the shows themself, they are not that different from other action-scifi like Masked Raider, Transformers, Star Wars or Star Trek - recycle the winning formula. Gundam just being the biggest in Japan, and Bandai works hard to keep it in that position.