Actually, if you use the glue sparingly, it'll be about 90% dry in about 1 hour.  If you cover the part in the glue, then you'll want to wait much much longer. 
When I'm doing repairs with my Yammies that involve glue of any sort, I give it a minimum of 24 hours.  Although, it's not uncommon for me to come back to the toy about a week later.