The first thing to note is that like many countries in Europe, receiving calls is free.  The calling party pays for both ends of the connections.

Before you start worrying about this, you need to ask yourself "Do I really need land-line service?".  The reason why is that it is very expensive to set up.  You have to buy a "subscription right", usually from NTT, for what amounts to hundreds of dollars/euros or more.  Aside from that, obviously if you move, you will have to have your land-line moved as well.

Mobile phones have no subscription right, and so are much easier to set up.  

Note: There is a type of cordless-phone/mobile-phone hybrid called PHS that used to be popular.  The advantage was that it was very cheap.  The disadvantage was that the reception was poor compared with normal mobiles, and it wouldn't work at high-speeds.  I won't discuss that here, but I believe Willcom still offers that service.

Major carriers offering normal mobile phone service include: NTT Docomo, Softbank, E-Mobile (Now Y!Mobile, a subsidiary of Softbank), and KDDI Mobile (aka Tuka), and Willcom (also a subsidiary of Softbank).  There are also MVNO carriers, such as OCN, Mineo, Sony Mobile, and b-Mobile.  

Basically there are two ways you can get a normal mobile phone.  You can get a pre-paid unit, and then pay through the nose for minutes - or you can sign up for a normal contract.  If you are only in Japan temporarily, then you will have to go the pre-paid route, since all of the major carriers will only give contracts to Japanese residents.  (If you don't mind a data only SIM card, non-residents can sign up for those - and getting a prepaid one is particularly easy).

If you opt for the prepaid option, you will have to put down about $100-$200 USD for the phone, plus buy some minutes.  At least from Softbank, the minutes are available in denominations of roughly $30 and $50.  Some of the other carriers have stopped offering prepaid service all-together, claiming it is mainly used for criminal activities.  You may be able to buy a second-hand prepaid phone through Tokyo Message Board or Craigslist, and likewise when you are done with it, you can probably sell it there too.  (Buyer beware, if the person selling has purchased their phone on an installment plan and then fails to make payments, the network carrier will probably network block the phone, rendering it unusable.  In order to avoid this, you can by your phone from reputable used phone dealers such as Sofmap or Janpara).

If you go through the normal option, be prepared for some sticker shock.  The US companies usually lie to you about the phone price, claiming it's "free", or reducing the price by several hundred dollars.  Of course, the phones aren't really that cheap in the US, and consumers get ripped off by thinking they are... but we'll get to that in a minute.  An entry level phone will set you back about $400.  One of the fancier touch-screen phones with built-in TV (yes, they have that!) will be $800 or more.  You have the option to pay this all at once, or to have it split up into 24 monthly payments that get added to your bill.  Thus, a typical bill might look like this:

$40 Basic Service

$20 Phone Payment

$20 Usage charges (Minutes/Messages)

------

$80 Total

This is really how it works in the US too, since obviously they have to increase your monthly payment to pay for the phone itself.  The kicker is that in the US, since they don't call it a Phone Payment fee, it never goes away - that is, you never pay off your phone.  Indeed, in the US you actually do usually have the option of getting a new "free" phone when the service runs out, but you unfortunately don't have the option to have a lower bill.  (Europe works much the same way as Japan).  In Japan, once the phone is paid off after 2 years, that part of your bill disappears.

While many US and European carriers offer "Free Minutes", that generally doesn't apply in Japan.  Instead, there is a base rate you pay, and a per-minute or per-message rate in addition to that.  Typically you can pay a higher base-rate for a lower per-minute rate. 

Update: As of 2015, since most of the carriers realized that people don't use the voice calling feature all that much with applications such as LINE (and Skype, Google Talk, etc.) being popular, they have started pushing people to "Unlimited Talk" plans for all new contracts, where you can talk all you like - while simultaneously cutting the data.  Sneaky.  

Update 2: With the recent popularity of cheap MNVO carriers, it is worth noting that their per minute rate for voice calls is actually very expensive, equivilent to the non-discounted rate of Docomo or Softbank.  (i.e. about $0.40 per minute).  One can get around this issue by using VOIP software like 050+ when calling other phone numbers, and otherwise using other software that allows you to use data instead of voice minutes to talk.  If you need to make outgoing calls via an 080 number often, an MVNO may not be your cheapest bet.

Another thing to note - Foreign phones in general don't work in Japan.  (iPhone 3G appears to be an exception to this).  Japanese phones in general don't work over-seas either.  Almost all Softbank models feature GSM compatibility, so they will work overseas, but most phones from the other carriers don't.  This situation has improved in recent years, with the carriers adding in additional frequencies.

Also, speaking of the iPhone, it appears to be popular among foreigners, but I would suggest that's because they don't know what they are missing, with Japanese phones you can many things the iPhone doesn't have:

1. IR sending of contact information

2. DecoMail, etc.

3. Superior Japanese Input (and a keyboard, since Japanese typically use email much more than Westerners).

4. 1Seg mobile TV (And more recently, NotTV and FullSeg HDTV)

5. Standard Japanese chargers (you can buy batteries at 7-11 and such if you need to juice up in an emergency)

6. Saifu Keitai (Phone Wallet)/Mobile Suica (allows you to use your phone to as a train pass and to pay various things)

7. Japanese phone applications (Many available only for the standard Japanese phone Java platform)

8. Proper Japanese Mobile Phone Web site support (no, safari doesn't work well for this)

9. QR Code Scanning

10. Some phones have 5, 8, or even 10 megapixel cameras and HD video recording...

11. TV Call - Yes, video calls on the cell phone!  (This is different from FaceTime in that it is supported by the mobile network directly in much the same way as normal voice calls, and is compatible across phone models from different brands).

and more...

One more thing to note.  Softbank is in general the cheapest major carrier by sticker price, but they have many traps to cost you more in the long run.  (MVNOs may be much cheaper, depending on the plan and your usage patterns).  NTT Docomo is more expensive, but generally has the best coverage if you live in the middle of nowhere.  Messages to people on the same carrier are often free.

The MNOs have a bit of a scam going, when you sign up for a phone contract, they lock you in for two years.  That may be normal, but the trick is that when the two years is up, you have an "update month", where you can cancel with no fees.  Note that there is still usually a $30 fee for porting your number to another carrier even during the update month.  If you don't cancel your contract or port your next number during the update month, then your contract renews automatically and you are locked in for another years.  Yes, that's right, if you cancel your contract after 25 months, you will have to pay a cancellation fee.  It still may better just to pay the cancellation fee and move to a cheaper carrier, however bear in mind that if you purchased a phone by installment payments, you will probably have to pay the remainder as a lump sum, then pay the cancellation fee, and then pay a fee to port your number.  Ouch!  Our suggestion is that if you want to cancel your contract, mark the date on your calendar so you don't forget.

Many people I know have been burned by the carriers because of their free gifts. Softbank seems to be the worst at this kind of thing.  Softbank offers free or greatly discounted second phones, tablets, etc., even to people that may not want them.  Some people figure "Hey, it's free, so why not?"  Then later, they come to regret it.

Case 1: Some of the items are only free for a limited time.  They will start charging later (long after the customer has forgotten), and the customer will only know when they are surprised by the charges on their phone bill.

Case 2: Sometimes they give free phones, with free (or very cheap) service, such as emergency phones for children - but they come as a set with the main contract.  Customers find out later that the real reason was to bind them to their carrier forever.  Cancelling either contract means a cancellation fee, and the update month is often different.  Canceling the main contract means that the sub-contract will also be cancelled, and so there may no no way to escape from the carrier without paying fees.  (Softbank is bad about this kind of stuff).

Case 3: Unwanted Services.  Some of the carriers want to sign you up for all kinds of services you will never use.  First is warranty on your phone (which may be useful), next is "wide support", which means you can call in with questions any time.  Next are dozens of separate services that even if you want are better purchased elsewhere.  For example, they might want you to pay a monthly fee for a navigation app, for a comic book app, for a streaming TV app, etc., etc.  These are usually only a few hundred yen (a couple USD) each, but when you have 20 of them, they will weight down your bill even if you never use any of them.  Often you must sign up for these items in order to receive a discount (which is included in the advertised price), however you will be told "Don't worry, they are free for the first month, and you can cancel them later!"  They are counting on customers being too lazy to cancel them, and it's also not as easy as they say.  Often you have to go through the cancellation process for each and every add-on separately.  You may be asked to sign up for credit cards as well - but this can be a good way to get accepted if you have had trouble making a credit card elsewhere.  Many times third party shops are worse about this than the official shops.  (Docomo is seems to be worse than the others about this).

With cell phone calls costing upwards of $0.20 per minute or more, and more to the point, Japanese International Long distance rates being obscene, Voice-Over-IP is becoming an attractive option.  There are basically two routes you can follow:

a. Voice-Over-IP offered by your Internet Provider - In this plan, you sign up for VOIP with your ISP, and they will usually provide you with a box that you plug into the internet router, and then plug your normal phone into.  This is particularly great for grandmother types who don't want to figure out new-fangled technology, but might not mind having a reduced bill.  It's not the cheapest option, but the call quality should be high, since the ISP is running and supporting it after all.  (An example of this type of option is "Hikari Denwa" from NTT East or "dot 050" from OCN).

b. One of the various VoIP services offered by 3rd parties - My favorite is Skype, but there are plenty of others.  In this model, you use your software on your computer to make calls to normal phones for a small fee (or, to other computers for free).  With many of these services you can pay-as-you go instead of being locked into a monthly fee, which is great if you don't use it too often.  Also, you can pay to rent a number for Incoming calls, and it doesn't have to be a Japanese number.  For example, you can link a Japanese number and/or a US number to your Skype account for a relatively low monthly fee.  Then people in the US can call you for free.  Also note that if you are using the computer on both ends, then Google Talk (Now Hangouts) is another great option.

Note that Japanese numbers follow a pattern, in general, numbers beginning with:

03 are offices or homes in the Tokyo area

070, 080, and 090 are mobile phones or PHS phones.

050 are Voice-over-IP

(Other prefixes for other areas)

0120 indicates a free dial number (like 1-800 in the US).

Which means people will know when they call your Japanese number that it's voice-over-IP, but I don't see that as being a big problem.  What is a problem is that some Japanese phones are blocked against receiving calls from VoIP by default.  I take this as scare-mongering by threatened Japanese companies trying to tell customers that VoIP might be used by spammers.

Also, only 070, 080, and 090 numbers can send/receive SMS messages.  Since a lot of services (like LINE, etc,) require an SMS confirmation to sign up, it means you may have trouble signing up with a phone that can only do 050.

The great thing about Skype in particular is that there are many non-computer devices that support it, including the iPhone, Sony Mylo, Newer versions of the PSP, and other stand-alone wi-fi phones.  You can also buy a box that connects a traditional phone to your network as mentioned in scenario 1 above.  With the Mylo or Wi-Fi phones, they can be on stand-by 24 hours a day, which means they can receive incoming calls, making them great to actually replace a normal home phone.  (With, of course, the added capability of taking them wherever you go if there is an internet connection).  A bit limitation of Skype is that even if you pay for an "online number" for Japan, the outgoing caller ID is not reliable in Japan.

Besides Skype, Gizmo5 was another popular service.  Gizmo5 could be used with their computer client (similar to Skype), and also with SIP phones and boxes.  A SIP "ATA Box" is a special box that you plug into your internet connection, and then you can plug a normal phone into it.  It will act as a translator between the SIP protocol that Gizmo5 uses and your normal phone - meaning that your computer won't have to be on at all.  Of course, you can always use the PC client instead if you don't want to buy anything.  Gizmo5 was also similar to Skype in that you can have free PC to PC voice and video calls and/or paid incoming and outgoing calls to normal phones.  Gizmo lets you rent s US number, or but currently does not offer Japanese numbers.  Also note that if you use Google Voice, you can have a free US number, and connect it to Gizmo.  There are rumors of Google buying Gizmo, so this integration might improve in the future.  Note that Gizmo 5 has been shut down by Google. 

050 plus is a more phone-centric Skype alternative from NTT Communications, please check out more in our review at:

Yamanote Style 050 Plus Review

As mentioned above, Google Voice (Formerly Grand Central) is a phone service offered by google.  While it currently only handles US numbers, you can use it to send and receive SMS numbers with US cell phones from their web page, and it can receive voice mails which are emailed to you as a sound file and/or a transcription.  Since this number is offered free of charge for incoming calls, it may be a good choice for those who only need to use voicemail and/or SMS.  Although you can forward it to another phone number, it has to be a US number as well.  If you forward it to the above mentioned Gizmo5 SIP number, then you can receive unlimited incoming phone calls for free - but you have to be at your computer.   Note that Gizmo 5 has been shut down by Google, but incoming calls can be answered in the web browser with GMail if the proper plug-ins are installed.  Note that Google Voice can be used for SMS and voice calls from the computer (via GMail), and with some special applications for Android and iOS (As with Skype, the Android applications work for incoming voice calls, due to better due to integration limitations in iOS). 

You can use computer-only IM clients like MSN, Google Talk, iChat, AIM, and more, to do domestic and international voice calls for free - provided you want to be at the computer every time.  If you want to to be able to do normal voice calls from these clients, you can use GTalk2VOIP, which provides a paid, but cheap service for connecting these normal IP clients to the phone system.  Despite its name, GTalk2VOIP actually works with many different IM services.  Although their web page is less than fancy, it works, and the price is right.  They offer both US and Japanese numbers for a very cheap rate.  The Japanese numbers are also not area code 50 (VOIP) numbers, but rather Tokyo area code 3 (Office) numbers.  This means anyone in Japan can dial them, and they look familiar/legitimate.  An interesting combination is that you can combine GTalk2VOIP and Gizmo5, meaning that you can receive incoming calls on not only your computer, but your actual phone if you have a SIP ATA box.  Although this combination sounds complicated, it is cheaper than Skype for long term use.

Note while you can't typically easily use Fax services over voice-over-IP,  services like popfax.com offer outoing and even incoming faxes without the use of any fax machine.  You can upload the documents you want to fax as PDF files or similar, and then send them to any number you like for a small fee.  Alternatively, if you subscribe, they will set up an incoming number, and when a document is faxed to it, they will convert it to a PDF and email it to you.