I would not be afraid to (knowingly) purchase a car with a rebuilt motor, assuming that I was able to find out where it was done, and I was at least reasonably certain that it was rebuilt correctly. I say reasonably certain because: #1) even the best motor builder has to have an off-day occasionally, and #2) you can only be reasonably certain that the motor in your brand-new car, right off the sales floor, was correctly rebuilt. If I was buying a second-hand car from an individual and he told me he had rebuilt the engine in his own garage, I'd be a little more leery, because he has no quality control department that he has to answer to, and there's definitely no warranty from the individual.
Here's something that may be worth considering: how many of us are driving cars with rebuilt motors that have no idea that's what they have? One, if you bought it used, you may never know. Two, if an engine fails on the assembly line, do they throw it away, or do they take it apart and rebuild it? If anyone here has ever worked in a plant building cars and can answer that question, I'm curious to know.