I don't really have much of a problem with Consumer Reports ratings. It's not perfect (like democracies) but it's often the best you can get when you want someone to compare products under controlled laboratory circumstances. The test results may not really be helpful when it comes to vacs that are closely rated ("very good" versus "excellent" for bare floors for example), but you can really get an idea of which machines to say away from at the bottom of the ratings. And in fact Consumer Reports tells you that models with close ratings are essentially the same and the gap is not significant. What's great about being an "on-line" subscriber like me, is that you can use their new "interactive" feature filter which allows you to eliminate vacuums on the test list based on your personal priorities. For example, I do not use canisters for carpet cleaning, but I demand that the canister have decent suction through the hose for tool use. So I adjust the slider to eliminate all tested vacs that rated lower than "very good" for "airflow throught the hose" and voila! - about half the tested vacs disappear from the list! Same with uprights - I eliminate any upright that rates lower than "very good" in the carpet cleaning category - and that kills off about 1/3 of the rated vacs. When it comes to uprights, if it can't clean a carpet well, it's off my list - no matter how good the filtration is or how quiet it is! On the other hand, if you are really sensitive to noise, you can decide to bypass the top-rated Hoover Self Propelled Windtunnel - excellent at carpet cleaning, but only "fair" in the noise control department.
Consumer Reports also tells you a little bit about brand reliability which is based on CU members answering questions like: "Did you buy a new vacuum in the past year? What brand was it? Did it require a repair in the first year?" With the results of these questions, you get an idea of which brand is most reliable, and they warn you that this says nothing about differences between a brand's models - just the general history of consumers' experiences with a particular company. For instance, though Hoover uprights often get good ratings in the test results, it is way down on the list for brand reliability and CR tells you that. Dyson and Kirby lovers will be glad to know that these two companies are at the top of the reliability list for upright vacuum cleaners.
I think Consumer Reports wants to be able to help the average guy who walks into Walmart or Sears looking for a vacuum - not the vacuum fanatic or wealthy person who will buy one at a specialty vacuum shop. Vac fanatics like us know enough about vacuum cleaners that we can go into a specialty shop and not be pressured into buying something that we know is not good for us. But CR can help the average guy know that you don't need to spend $1000 to get a good vacuum - and with that knowledge he won't be pressured to do so. It's not a good use of CR's resources and time to test a lot of $1200 machines, when most people would never think of spending that amount on a vacuum cleaner. And most people who can afford a $1200 machine won't care if it does not work well or dies on them very fast - they have the cash to go and buy a replacement every year. And if their carpets aren't getting clean, they either have a cleaning person work longer hours with a bad vac to clean them, or they buy new carpets!
For a guy on a limited budget like me, I thank Consumer Reports for telling me that I can buy an $80 Hoover Tempo and get excellent carpet cleaning from it. It helped me avoid paying $930 for a Lindhaus that can't clean carpets as well. And yes, the Lindhaus may last longer, but I generally don't abuse products I buy, and with care, my Hoover Tempo should offer me many years of good carpet cleaning service.
Just my 2 cents worth - no offence meant to those who love Lindhaus!!!!
For me personally, I also consider the business choices a company makes when it comes to making their vacs - something that does not enter into the ratings of Consumer Reports tests. I have a problem with companies who are making their vacs in countries that don't treat gay people well, or don't treat other minorities well. And though Sweden's Electrolux have a pretty poor "brand reliability" position in CR ratings, I like the fact that they are doing a lot for the environment, including making their UltraSilencer Green out of recycled materials.
The only mystery I have about CR ratings is that I never see a Filter Queen canister tested. If they have the interest in throwing a Rainbow into the mix, not sure why they leave out Filter Queen.
Anyhow, just my personal opinion about Consumer Reports. Most of the time, the test results have really helped me save money and time. I am not good with high pressure sales people in any store, and so I usually buy something with a little list of approved models from the latest CR test in my pocket. I avoid having to listen to bs from salespeople (we all can tell stories about the lies we have heard from dishonest sales people), and most of the time, I end up with a decent product thanks to the work done by CR.