Help show Consumer Reports how it's done

[mkaresh]

Originally Posted by Brett

Of course there are no perfect samples, thats why Random samples are used, so that a margin of error can be mathematically calculated. There is no way to calculate your margin of error. You claim to be a PHD on your site, which means you know all this already, so that begs the question of what your actual motive is. Frankly, I dont care, I dont own a CTS.

Turn your site into a "vehicle issue" blog and monetize it that way. That could be a useful resource.

When you say "you claim to be a PHD," the "you claim" bit implies that I'm dishonest. What have I said anywhere to warrant this suspicion, this presumption of guilt? Why do so many people here feel the need to dole out insults?

I have the Ph.D., but don't generally bring it up because it gets some people all huffy. It's from the University of Chicago, where I worked for a few years in the National Opinion Research Center, a leader in the field of survey research.

When you know a little about an area, you assume that all rules must be followed. Learn more, and you eventually learn when the rules can be bent, and how much. Like in many fields, what the textbooks say to do and what must be done in the real world are two different things.

I have a fair amount of knowledge in the area. People with far more knowledge and experience in survey research than I have have examined what I'm doing, and have been okay with it. Now, I'm sure other experts would disagree with them. But the issue is not nearly as black and white as you believe it to be.

It's generally the people with a smattering of knowledge in the area and who have little or no experience in survey research who raise the objections you are raising.

In the end, the proof is in the results. Are they all over the place, or do they generally fit with others' results and one another?

There are some flukes in my results, and these are noted in the comments you'll see if you hover over some of the results. The first-gen CTS is unusual in that this time around I had to comment both the 2003 and 2005 this way--but then those are asterisked results owing to very small sample sizes.

There aren't more such flukes than you'd expect among the 119 non-asterisked results in the current set. With a 90 percent confidence interval, for example, you must expect about 10 percent of the results to be off.

By and large, the results have been much more consistent and stable than I personally expected them to be, given the small sample sizes, human error, and so forth. And the larger the sample size, the more stable they become.

I feel confident in the 2008 CTS result because it is so high--even a number 2/3 as high would still be high--and because the sample size is over 50.

[Brett]

Originally Posted by mkaresh

When you say "you claim to be a PHD," the "you claim" bit implies that I'm dishonest. What have I said anywhere to warrant this suspicion, this presumption of guilt?

I have no reason to believe or disbelieve, I was just stating that you claimed it.

[mkaresh]

Sorry if I over-reacted to your choice of words. This thread has made me a bit touchy, for some reason.

[RightTurn]

Originally Posted by mkaresh

...I think this message has been lost, in part because the repair rate came in so high for the 2008 CTS that it is one of the few cases where the difference compared to most other models is substantial.

I don't believe anyone is surprised that a first-year model has a few bugs to work out, which is why many of us take a "wait and see" attitude to newly introduced models of any make. Also, most of the people objecting to your methods do not own the '08 CTS.

[MoFex]

I thought I would stay away from this discussion. But I can see an unprecedented attack on makaresh. Why? Because the CTS 08 initial quality rating is so poor?
I know, his survey is not perfect, but the same can be said about J.D. Powers and Consumer Report. All of them have certain error build in.

I purchased CTS because I liked it. I liked the exterior, the interior, the drive train. And I promised Detroit I would be back if they designed a car I liked. And I kept the promise. Despite many warning me against GM quality. I have to honestly say that the CTS has been the worst car in terms of initial quality I have ever had! And by a large margin! I already had numerous visits to the dealer to fix problems that should have been picked up during assembly. It shows that GM can design and engineer a nice car but cannot get quality system in place. So I would say “shame on you GM” for lack of quality on an almost $50k car. Fortunately the dealer has been exceptional and worked to resolve every issue (except the NAV one, but I learned there is finally a bulletin to address this problem).

My experience so far has been that makaresh CTS 08 results reflect my experience. Do I like it? Heck NO! But this is what it is. As much as I hate it I hope the long-term reliability will be there. And this is far more important for me right now.

GM needs to hear it and address problems at the source. I am a firm believer in continuous improvement so hiding head in the sand and pretending as nothing happened will get us nowhere. CTS has it share of problems, more than I wished for, but it is not makaresh fault. It is GM and GM needs to fix it so we all can benefit and enjoy the car….

And this is what I am going to do…..

End of rant…..

[lawfive]

Originally Posted by Brett

What type of verification process is used before your results are processed?

Originally Posted by mkaresh

I run various checks on the responses that are submitted and the user-data collected in the process. Those that fail these tests are investigated. So far, very few such checks have been required, and so far all errant entries discovered have been caused by human error, not malicious intent.

I'm no expert in this area but I can speak as a person who has filled out both the J.D. Powers survey and the Consumer Reports survey: neither firm contacted me to verify my responses. Perhaps this was because my responses didn't constitute outlier data points, or perhaps it was because there's no verification process.

[lawfive]

I don't know anything about the web site in question here nor do I give a crap, but mkaresh has shown some class in this thread with the measured tenor of his responses.

[mkaresh]

No one's going to verify every single response. Aside from upsetting participants who don't feel they have more time to deal with it, it'd be very expensive.

I run checks and compare the repair descriptions with the multiple choice responses to identify likely errors. Many of these I can fix without contacting the respondent. Only in those cases where an error is likely but I don't have enough information to fix it on my own do I contact the respondent. About one in fifteen responses to the repair survey requires this.

Clearly different people define class in different ways. Or maybe it's one of those "do as I say, not as I do" things.

[gothicaleigh]

JD Power released their 2008 Initial Quality Study today:

"Cadillac gains the most rank positions among nameplates, moving up from No. 26 in last year’s study to No. 10 in the 2008 study due to a 16% improvement in quality."

The 2008 CTS is rated 3 and a half out of 5 in intitial quality with their system.



"Each year, the J.D. Power and Associates Initial Quality Study (IQS)SM surveys consumers who have purchased new model-year vehicles and have owned them for at least 90 days. IQS results are based upon approximately 97,000 responses from verified purchasers and lessees of new 2008 model year cars and trucks."


They also have reviews and ratings for individual models (here is the one for the 2008 CTS: http://www.jdpower.com/autos/Cadilla.../Sedan/ratings ).

There is no fee or participation required to view their results. www.jdpower.com

[mkaresh]

The "gains the most" quote is actually a typo. It should read "Audi" rather than "Cadillac." But Cadillac moved up nearly as many spots. In neither case is the improvement as large as it sounds--the brands in the middle are tightly bunched. The absolute change was from 135 last year to 113 this year, or about one-fifth of a problem per car. Average was 125 last year and 118 this year.

The real problem with the IQS, though, is that it combines "design quality" (driveability, usability and such, irritants designed into the car) and "manufacturing quality" (manufacturing defects, things that can be repaired). BMW ends up below average for one simple reason: iDrive. In terms of manufacturing quality, they do much better than their IQS score suggests.

J.D. Power will eventually post subscores for each of these. But if the design score is over 3.5--and it probably is, given how well the Cadillac interior is designed and how nicely it drives--then the manufacturing score will be lower than 3.5 stars. Probably not as low as in my survey, though. Most likely a 2.5, maybe a 3. But if the car gets a 5 for design, then the manufacturing quality score could be as low as 2.

I was surprised by the low score for the Lambdas, a 2.5. The 2007s had problems in my survey, but the 2008s have been better. Apparently the 2008s aren't doing better for J.D. Power. I was not surprised by the low score for the new VUE--it's been doing poorly in my survey as well.

I also was not surprised to hear that the Malibu did very well. I have very limited data on it, but as I think I posted somewhere here a few days ago it appears to have a very low problem rate. It has taken them a very long time to ramp up production on the car--I think they were more careful than usual to get the bugs ironed out with it.

As for no fee or participation required to view their results--that's because they publicly release only a very small portion of their actual results. Try to find an IQS score for a model--a number, not just a number of stars. Or try to find any one-star ratings. You won't, because the lowest rating they'll show is two stars. These days, anything two stars and up is actually pretty good. What you really want to know is what is well below average, and they never release this information at the model level.

There's no fee or participation required to view most of my results, either. I provide far more to the general public than they do. And to participants I provide the level of detail they reserve for their corporate clients.

[mkaresh]

Originally Posted by gothicaleigh

They also have reviews and ratings for individual models (here is the one for the 2008 CTS: http://www.jdpower.com/autos/Cadilla.../Sedan/ratings ).

All of those scores are still for the 2007, not the 2008. The key score to note for those interested in reliability is "Overall Quality Mechanical." The 2007 CTS had a three out of five, which means average.

[Southern]

Congratulations to Cadillac and Audi.
I see Buick is back in the hunt also. They have typically done well.
And Jaguar did a real turnaround after Ford Bought them.
It will be interesting to see how Tata does with them.

[mkaresh]

I think Buick has actually done better in the past.

JD Power is showing a low score for the Lambdas, so the Enclave is dragging Buick down.

[mkaresh]

Elsewhere someone pointed out to me what the dots represent.

5 = top 10 percent of models
4 = 60th to 90th percentile
3 = 30th to 60th
2 = below 30th

Essentially, it's harder to get a 5 than I realized, but easier to get a 3 or a 4.

I had thought they were combining 1s and 2s. But, since the 2 range is no wider than the others, they've really got a four-dot scale, with an extra dot on each score to spare the low-scoring models the shame of having a single dot.

No word on what the halfs mean. But if I had to guess, I'd guess that a 3.5 equals the top half of the 3 range, or 45th to 60th percentile. Maybe 50th to 60th.

Since these are percentiles, it remains unclear how each dot compares to the average (twice as many problems? half as many?). Since most models will be clustered near the average, the range of scores covered by the 3 and 4 dots will be much narrower than that covered by the 2s and 5s, since those include outliers.

It would be nice if they'd at least state the PPH range for each dot.

[TaVern]

Originally Posted by lawfive

I don't know anything about the web site in question here nor do I give a crap, but mkaresh has shown some class in this thread with the measured tenor of his responses.

I will agree, 100%.

mkaresh,

I believe your website is being funded from information you have taken from this forum, CadillacOwners.com, free of charge from you.

I have donated $10 to CadillacOwners.com on your behalf, the equivalent of a 2 month supporting membership. If you question this transaction, please private message me and I will supply the details.

I am personally granting you two months of freely "bumping" this thread whenever you see fit to "advertise" your deceptive website.

I encourage you to match my donation, as you have undoubtedly benefitted from this forum.

Thank you in advance.