Forms of ID in Japan.

A few years ago, there were basically four forms of legally acceptable Photo ID that a foreigner could obtain in Japan.

First was a "Certificate of alien registration" (ARC), (外国人登録証明書).  This was issued only to foreigners, and contained a lot of information you might not want to show to everyone.  Also, this document screamed "I am a foreigner!", which, even if you look obviously "non-Japanese", is not a good thing.  

Places like Softbank decided that if you register with an ARC, they wanted to check your visa validity period, your passport, make you pay by credit card, etc.  If you feel that this information is none of their business, you wouldn't want to use your ARC to sign up.  (Not only that, but once you registered with an ARC, they keep track of the fact that you are foreign forever, apparently).

The best way to avoid this was to use a health insurance card, or a drivers' license, and sign up like any normal Japanese person would.   Since health insurance cards generally don't have a photo, some places don't accept them for certain purposes.  One official thing you could get as an official proof of address that didn't mention anything about your nationality, visa status, etc., was a seal registration certificate (印鑑登録証).  These also don't have a photo, and are typically only considered valid for only three months, though.

And drivers' licenses are an expensive pain to get.  Basically speaking, unless you have a foreign license to convert that meets the proper conditions, you have to pay something $3000 and spend several weeks to go to driving school, and pass driving tests, etc.  That is a lot of time, energy, and money to spend on getting an ID card!

You might notice that the Japanese DMV does not issue "non-drivers' license" ID cards like the "State ID" cards available in the US.  The closest thing available in Japan is a Basic Resident Registration, or "JUKI" Card ("住民基本台帳カード").  These have been available only to Japanese citizens.

New laws that came into effect in 2012, which abolished the system where foreigners were registered in one system, with Japanese registered in another system.  Under the new rules, everyone is registered in Juki-Net, and foreigners now get a "Zairyu card" (Residence card) instead of an ARC.  The Zairyu card is in some ways similar to a JUKI card, but still a different card that is issued only to foreigners.  Worse yet, although it has less information on it, that is also bad in a sense.  If you had a registered Japanese name on your ARC, it will be omitted from your Zairyu card.  So your Zairyu card might have your photo, but not your Japanese name, and your seal registration certificate would have your Japanese name and address, but not your photo.  If you know anything about Japanese bureaucracy, you know that some places can be picky about this kind of stuff.  Also, since the new laws related to the Zairyu card came into effect, it has been possible for foreign residents to get a copy of their "juminhyo", or residence certificate (this is a paper sheet, not a card), whereas they used to have to get a special "certificate of registered matters" or similar for foreigners.  The fact that two separate documents were used was a source of confusion, as many places weren't familiar with the document for foreigners - so standardizing on the Juminhyo is a step in the right direction.

Still, there was still the problem that foreigners couldn't get a regular Japanese ID - until now.  Starting July 8th, 2013, almost all municipalities will by law have to provide JUKI cards to any registered foreigner who applies.  This means you can get a normal photo ID that isn't specific to foreigners.  Better yet, it will include any registered Japanese name you have, and if you apply, will include an encryption certificate that can be used to file for taxes online.  (e-Tax).  Carrying the card also makes you eligible for simplified move-in/move-out procedures when changing address.  The cost varies by area, but is something like 500 yen, so there isn't much reason *not* to apply.  Note that many municipalities have not updated their web sites, so they may say that the JUKI card is only available for foreignors.  If the person accepting your application says something like that, please point them to the Soumu web site with details about the change.

Update:

Starting from January 2016, the JUKI card system will be replaced by the new "My Number" card system in Japan.  This will be a similar card which can be used as photo ID, however your new "Personal Number" will be printed on the back.  (This number will be assigned to all residents starting in October 2015, and is similar in purpose to the Social Security numbers issues in the USA and other countries).  You can continue to use your existing JUKI card until it expires.