If you are planning a visit to Japan, short or long term, and intend to drive, then yes you will need an international driving permit. Even if you don't plan to, if you just want to be able to drive should the chance arise, you will need it and the low cost versus the convenience means it's worth getting.
If you didn't know - you have to get your international driving permit from your own country before you leave. Here's a list of links to help you locate somewhere near you to get an IDP.
Find Where to Get an International Driving Permit
United States of America American Automobile Association, Inc.
Canada Canadian Automobile Association
United Kingdom The Automobile Association Limited
Republic of Ireland AA Ireland
Jamaica Jamaica Tax Administration
Australia Australian Automobile Association
New Zealand The New Zealand Automobile Association Inc.
International driving permits aren't expensive to get and if you are planning any kind of working holiday then it's well worth getting one. In fact, some ALT jobs - especially those out in the country, require driving so you need one to get yourself from school to school.