It would be helpful, but not exactly needed. The only requirement to work in Japan is to have at least a 4 year degree in something or have a spousal visa or something.
What you need and what they look at:
1. 4 Year Degree - Anything will do, but something related to teaching or English is a plus.
2. Visa Type - You need sponsorship, a company / workplace can provide it, or you can get married.
3. Experience - The more you have, the higher you are paid. If you don't have any, volunteer somewhere (Community Center or Special Needs School).
4. Patience, patience and more patience.
Although they can't come out and say that they care about this, but looks do matter. If a school doesn't like the way you look, they won't even bother to respond to your resume no matter how qualified you are or the experience you have. In private school it is not about ability but advertising.
Right now I work as a free-lance teacher for the ministry of education. Schools request me to come in when the JET teacher has totally screwed things up, or when JET teachers refuse to teach at a school. Starting next year the M.O.E. will be phasing out the JET program. And they will be opening up bids for private schools to take over to some extent, but mostly it will be handled by Japanese teachers who have a teaching license for English. So if you want to work inside a public school, your best chance would be to appy to a company called "W5" (pronouced "Dubba-five").
Rob