First, though, let's look at some of the other options:

Taxis exist in Japan, and are your main option for getting around after the train service is done for the night.  They are also great if you need to carry some heavy luggage from somewhere other than the airport, and they usually have GPS systems so they can find their way around the byzantine street planning the Japanese officials set up as a practical joke.  On the other hand, they are not cheap, not by a long shot.  For example, if you decide to take a taxi from Narita airport to central Tokyo, you will most certainly be sorry.

Bikes are popular in Japan, and are a great way to go short distances in the places that aren't so crowded.  You can see people running errands on them all the time, and some people even use them to get to work in the country-side.  Many children also ride bike to school.  One down-side is that many city streets don't have a bike lane.  While the official regulations state that bicyclists should use the regular traffic lanes, many people ride on the sidewalk, which is quite dangerous to pedestrians (which are numerous).

If you have a Japanese driver's license, you can rent a car - though this is really only effective for long trips into the countryside.

This is common in the rural areas, but expensive even there.  In the city, you had better want a car pretty badly, because that's probably where a large portion of your paycheck will be going if you buy one.  Among other things, you have to buy a parking spot, you'll also have to pay around $3000 for the mandatory driving school before you can take the driving test, as well as quite a heavy sales tax.  Luckily, in the cities, the train service is very convenient - plus you don't have to worry about drunk driving if you take the train.

There are a number of bus services available, and the payment methods are similar to the train (discussed below).  I am not a big fan of commuter bus service myself, but the busses to/from the airport may be more convenient than the train in some circumstances, and will certainly cost less than a taxi.  There are also long-haul busses that run from Osaka to Tokyo overnight.   While not nearly as fast or sexy as the bullet trains, you can probably get some sleep and avoid breaking your budget at the same time.

Finally...

The first thing to know is that there are many types of trains... usually in any area, there is the local Japan Rail company, and some competing companies.  JR trains are typically above ground, while other companies run above-ground trains, as well as subways.  JR also runs the Shinkansen bullet train service as well.

The old way to take a normal way is to buy a paper ticket first.  Cost is based on distance, so you would figure out your destination, and then pick it on a machine (or look at a map) to determine your fare.  You put in the money and get the magnetic paper ticket.  Before you ride a train, you put your ticket into the ticket gate (and make sure to take it with you when it comes out!), and then you stick it in the ticket gate at your destination station.  This is all well and good, but it becomes more complicated if you transfer, make a mistake, or change your mind about where you are going.  

If you take the train much (and you probably will if you live in a city), then your best bet is to get a Suica IC card, or whatever card your area of Japan uses (f.e. Kansai has the Ikoka card).  With these cards, you simply charge them with money and then touch them each time you enter or exit through a gate.  If you run out of money, you can add more at the ticket machine.  If you want to be really fancy, you can activate the IC Card function of your mobile phone and then just use that.  That has the advantage that you can refill the "card" by credit card without visiting the ticket machine at all.

The bullet trains and major commuter train lines in the big cities usually have the normal announcements in both Japanese and English.  Outside of that, it may be entirely in Japanese.  If you don't speak the language, you will want to get a comprehensive English map of the train lines in your area before you venture out alone.  The station staff are typically very helpful, but again, the smaller stations may or may not have anyone who speaks your native language.  Also, any time there is a problem or delay, the conductor of the train will take over from the computerized announcer to let everyone know what's going on, and you can be sure they won't announce it in English.  If the train stops for apparently no reason, you might have to ask around and hope one of the other passengers can explain it to you.  

"Shuten" means last stop.  Even if you look at your map and find out that a train goes all the way to where you want to go, it might happen that the train you got on doesn't.  For example, the Keihin-Touhoku line goes from north of Ueno, past Tokyo, then Shinagawa, and all the way to Yokohama and beyond - but not every train does.  If you get on at Shinagawa, the train you stepped onto might have its last stop somewhere in-between there and Yokohama.  You can see this from the sign before you get in the train if you are careful (and if you know where the last stop actually is!)  You will know if this situation occurs because everyone will get off the train.  Don't panic, just get off with them, and wait for the next train showing up on the same track in a few minutes.

With such a bewildering array of trains to choose from, there are many ways to get from place A to place B.  Several online route planners can help you find your way.  Among the most popular are Jorudan (Also available in English) and Ekitan.  Google Maps Japan is also useful if you have a destination address (In Japanese) rather than a station name.  Navitime (English) goes a step further, giving bus routes, and also walking guidance via their phone app.  These are all accessible via Japanese cell phones as well as PC web browsers.

The above mentioned methods work on normal commuter trains and subways only.  If you want to take the bullet train, then you have to buy a specific ticket.  The same goes for special express trains like the Narita Airport Express.  Also, many lines have first class "green" cars, which you can ride if you get an additional green car ticket, or add the green car fare to your IC card.  The advantage of the green cars is better seating, often a 2nd floor, and there is food cart service.  You can also pay for the food cart with your IC card.  Note that if you will be staying in Japan as a tourist and plan to visit many far away places, you should most certainly sign up for the JR Pass before you come to Japan.  It costs about US $300 per week for unlimited travel, but that will almost certainly pay for itself quickly if you plan to take the bullet train more than once.