1. If you buy a bicycle that is registered to someone else and you are ever stopped, you will be accused of having stolen it.

2. If you buy a non-registered bicycle without any proof of purchase, you will not be able to register it, and may be accused of having stolen it.

3. If you buy a used bicycle and ride it home, you may be stopped on the way and asked for paperwork.

4. If your bicycle is ever stolen, you will probably wish you had registered it.

Being detained by police isn't fun, and Japanese are particular about doing things the "official" way in general, so it's important to know the rules from the start.

The process is officially called "Jitensha bouhan touroku"/自転車 防犯登録 (Bicycle Crime Prevention Registration) in Japanese.  Technically, by law you must register your bicycle, but there is usually no real punishment if you don't, except warnings and harassment by police.  

When buying a new bicycle from a normal (official) retail shop, the situation is simple.  They will usually offer to register the bicycle when you buy it for a small additional fee, typically around 500 yen.  You will be asked to fill out a form with your name and address, the serial number of the bicycle, and of course show ID to prove who you are.  They will send this information to the police for processing, and hand you a receipt, which you should keep in your wallet for a few months - in case you are stopped before the information has been registered in the system.  Finally, they will slap a shiny sticker on your bike which has a registration number printed on it.

When purchasing a used bicycle, the process is similar to buying a new bike, except that you must bring the bike to the local police or bicycle dealership after purchase.    If you are unsure, you can ask at your local police box, and they can direct you to the proper place.  You will have to fill out the same form, show ID, and pay the same registration fee as when purchasing a new bike - except, that in order to ensure that the bike is indeed yours, they will require a transfer contract from the previous owner.  You can find a blank form for this contract on the Tokyo Bicycle Crime Prevention Assiciation's web site  (In Japanese of course), so print this out ans take it with you when going to buy the bike.  If you buy a used bike and do not get the original owner to fill this out, you will probably not be able to register your bike.  (There is a possibility that the police may contact the previous owner for more information, so faking the form is not a smart idea).

If you are buying a used bike and aren't confident that the existing owner really owns it (i.e. that it wasn't stolen), then you shouldn't buy it.  If it turns out to be registered already, but to someone other than who filled the form out, then the police may take it from you and return it to the original owner.  If you are suspicious, at least check the seller's ID and write down their name matches what's on the contract, and and note the number on their ID. 

If the seller purchased the bicycle new, but never registered it (likely with foreigners, since they may not even know about the bike registration law if they have never been stopped), then ask if they have any of the original paperwork from when they bought the bike, in addition to having them fill out the transfer form.

(If you purchased your bike new, but did not register it at that time, you can register it later, and the process is the same as for a used bicycle, except that you can show your purchase receipt or a special bicycle sales document instead of the transfer contract).

If your bike is ever stolen and then abandoned, then the police will call you to pick it up.  Likewise, if the police stop someone who has stolen your bike, then they will alert you once they have checked the registration.  The main disadvantage that most Japanese people complain about is that if someone steals your bike and leaves it some place very far away, it is your responsibility to go and pick it up from the police - even if you would rather just trash it.