It isn't quite that simple but yes, you can keep the plates with the car indefinitely. California has a program called "Year Of Manufacture". It allows owners of classic cars to register period-correct plates to their car. There are however a few rules.
First, this law only applies to 1962 and earlier cars. This means black/yellow plates, blue/yellow plates and all white plates are excluded and cannot be reassigned to a vehicle. Even if the plates are original to the car, once a newer plate has been assigned the old plate can never be used again. For this reason, if you ever see a car with black/yellow CA plates, you know that car has never been registered outside of California.
In order to register plates in the YOM program, you must own both front and rear plates and they must be in good condition. The numbers must also be clear in the DMV computer. This doesn't mean they only need not be issued to a car, they have to be tagged as either never used, or as unregistered from whatever vehicle they were originally assigned to.
You must also obtain the correct year registration sticker for the vehicle. For example, plates correct for MGA would be orange/black which were originally issued in 1956. If your MGA is registered as a '56, then you can install the plate with the stamped "56" showing. If your car is 57-62 then you need an official CA registration sticker from that year. They are not always easy to find since they were never remanufactured. You have to find an actual period sticker in good condition.
In addition to the period registration sticker, you are also required to mount the modern (current) registration sticker and the month sticker. The DMV will give you two ugly metal tabs for this purpose. Many people get creative and make their own tag plates.
hope this helps!