Wanting to Test Vacuums

[z67danco]

I’d like to gradually start testing vacuums, but I don’t know where to start. I don’t want flaws in my experiments, but that’s something I’ll have to learn as I go, refining the process over time. Eventually, I would like to have a way to get my experiment(s) analyzed. What equipment would be ideal to start out with? What tests? ASTM?

Is there any database for vacuum-specific research? Or air purification?

I’m primarily interested in filtration performance, and dirt extraction ability. I’m trying to figure out what the most common type of carpet is in U.S. residential houses so that the carpet samples I acquire for testing will be better representations of carpet performance than if I were to use my own home’s carpet. I do want to test performance differences on special types as well. Some vacuums do well on one carpet, but struggle on another.

As far as filtration, that will probably be more difficult due to the conditions being more challenging to control at home; but I’d like to test for air leaks and particle retention at PM 0.1 and PM 0.3. Maybe I could build an air-tight chamber, but how would I test for PM 0.1? Particle counters are interesting and useful but they don’t cover all the bases. I still want one, though.

What do I have to test currently? Kirby, Rainbow, Miele, Sebo, Dyson. My primary daily driver is a Miele C3, while my V15 supplements daily spot-cleaning and touch-ups. We have a a few indoor pets, a good mix of carpet and hard flooring, and a lot of traffic, so daily vacuuming is a must.

 

[cheesewonton]

Do some reading on how HVAC techs measure turbulent airflow in a tube or pipe. That will give you some insights on how to use an anemometer properly, and the math to correctly calculate airflow from air velocity and the surface area of the hose, tube, pipe or whatever the air is moving though.

Understand that suction and airflow are inversely related. As suction increases airflow decreases. The larger the hole, or orifice, the vacuum is breathing through the greater the airflow but suction is less. As the orifice size is decreased or restricted suction will increase and airflow will decrease. At sealed suction there is no airflow. If you ever wondered why vacuum motors seem to speed up when you put your hand over the hose it is because the airflow is stopped, unloading the fan and allowing the motor to spin faster ( and overheat ! ). When you take your hand off airflow resumes and the motor slows down as it works to spin the fan(s) against the mass of the air flowing through it.

If you look at some vacuum motor performance charts you see that as suction increases amp draw and wattage drop.. That is because more suction means less air on the fan(s) and less load on the motor. You can get a lot of insights into vacuum motor performance from studying their performance charts. The Ametek website has what they call "Product Bulletins" for their motors that show all these relationships as well as how air watts are calculated from suction and airflow at different size orifices.

 

[Absolute Rainbow]

Do some reading on how HVAC techs measure turbulent airflow in a tube or pipe. That will give you some insights on how to use an anemometer properly, and the math to correctly calculate airflow from air velocity and the surface area of the hose, tube, pipe or whatever the air is moving though.

Understand that suction and airflow are inversely related. As suction increases airflow decreases. The larger the hole, or orifice, the vacuum is breathing through the greater the airflow but suction is less. As the orifice size is decreased or restricted suction will increase and airflow will decrease. At sealed suction there is no airflow. If you ever wondered why vacuum motors seem to speed up when you put your hand over the hose it is because the airflow is stopped, unloading the fan and allowing the motor to spin faster ( and overheat ! ). When you take your hand off airflow resumes and the motor slows down as it works to spin the fan(s) against the mass of the air flowing through it.

If you look at some vacuum motor performance charts you see that as suction increases amp draw and wattage drop.. That is because more suction means less air on the fan(s) and less load on the motor. You can get a lot of insights into vacuum motor performance from studying their performance charts. The Ametek website has what they call "Product Bulletins" for their motors that show all these relationships as well as how air watts are calculated from suction and airflow at different size orifices.

In addition, also watch the reviews @Vacuum Facts, Vacuum Wars, RTINGS and such made. They are objective in their own way (Vacuum Facts himself considering himself gold standard; Vacuum Wars no longer doing sponsored review; RTINGS' extensive datas), and look out for important stuffs. Both specs and actual floor- and deep-cleaning performance (across both hard floors and all kinds of carpets) are important for dust extraction. As for the filtration, you'll need a smoke test to test the machine filtration first - you'll want a fully sealed, whole-machine HEPA filtration.

FYI, @z67danco

 

[cheesewonton]

In addition, also watch the reviews @Vacuum Facts, Vacuum Wars, RTINGS and such made. They are objective in their own way (Vacuum Facts himself considering himself gold standard; Vacuum Wars no longer doing sponsored review; RTINGS' extensive datas), and look out for important stuffs. Both specs and actual floor- and deep-cleaning performance (across both hard floors and all kinds of carpets) are important for dust extraction. As for the filtration, you'll need a smoke test to test the machine filtration first - you'll want a fully sealed, whole-machine HEPA filtration.

FYI, @z67danco

Vacuum Wars has the best test procedure by far. The one and only criticism is that their technique for measuring airflow is wrong and over states actual airflow. In every other respect they have a very good, repeatable non subjective series of tests.

 

[centralsweeper63]

I would suggest that you get a few different types of carpet. These being solution dyed nylon, loop pile wool, mohawk plush and then a two pile height plush rug to finish off a set of testing rugs. Make sure they are at least 4x4 feet though not too much larger so as to allow for weighing the carpet. This size lets you run three full size vacuums side by side for comparison. I think that everyone should know the difference between baking soda and flour in vacuum testing. Flour is lighter and finer. Baking soda is sharper and denser. Therefore using a mix of very fine sand, baking soda and flour gives you a good mix of sizes. I use kpok to simulate pet hair as it is very accurate for dog hair. Kinetic sand really gets in the fibres good and stays there. This makes it a perfect test material. Pumice and cat litter are good for larger debris. Also if you want to be precise you should weigh the carpet beforehand. Therefore you can account for what was left in the carpet by the last vacuum. Bear in mind you need space for all the stuff behind the brush roll on your rug, otherwise the machine will be angled which prevents it from adjusting properly. Weighing dust bags or bins is good for measuring pickup. A precise set of small scales is good for measuring dirt. A large set is good for weighing rugs. I won't be much help with air filtration testing. The tricky bit is the testing I suppose.

 

[Vacuum Facts]

Vacuum Wars has the best test procedure by far. The one and only criticism is that their technique for measuring airflow is wrong and over states actual airflow. In every other respect they have a very good, repeatable non subjective series of tests.

This is total steaming baloney but is consistent with your track record on factual matters at least.

I noticed no one has ever said what tests should show, how they achieve it, and why it's important relative to the specific known science. They've never even compared it to industry test standards. Anyone who hasn't, credibly, can be confidently ignored, both here and elsewhere. I've outlined what's important in my videos. The closest approximation to I've seen (that isn't my testing) comes from Frickhelm who is qualitatively consistent with me. There are some very interesting things that have cropped up though since the V16, that expose weaknesses in even our testing, but that will have to wait until the review next year.

 

[cheesewonton]

This is total steaming baloney but is consistent with your track record on factual matters at least.

I noticed no one has ever said what tests should show, how they achieve it, and why it's important relative to the specific known science. They've never even compared it to industry test standards. Anyone who hasn't, credibly, can be confidently ignored, both here and elsewhere. I've outlined what's important in my videos. The closest approximation to I've seen (that isn't my testing) comes from Frickhelm who is qualitatively consistent with me. There are some very interesting things that have cropped up though since the V16, that expose weaknesses in even our testing, but that will have to wait until the review next year.

Based on public documents of Europro ( Shark )'s lawsuit against Dyson each company spent over $1 million to have labs like Intertek and IBS perform just the ATSM F608 Embedded Dirt Pick Up Test no hobbyist is going to reproduce the ATSM tests exactly. But that should be the goal to get as close as possible to these as one can. To their credit Vacuum Wars does conduct the smoke and particle tests IAW the relevant ATSM test standard. Their embedded dirt test follows the standard to a point but they don't use three different machines and four different carpet types to get an average value across the three machines as required by the ATSM standard. But their test is a lot more relevant than the silly subjective, cough cough, "test" conducted by Performance Reviews.

https://www.govinfo.gov/content/pkg...v-09442/pdf/USCOURTS-ilnd-1_14-cv-09442-3.pdf

One thing I found interesting from reading the court docs was that when testing bagless machines the dust bin must return to within 2 grams of it's initial clean pre test weight after each run or the next run is invalid. From reading the court docs this is apparently both difficult on some machines to achieve and often overlooked, skewing the results of subsequent runs as the bin still has residual from the prior run.

Btw, knock yourself out buying copies of the many ATSM test standards. Each one has to be purchased and nobody is giving them away. That is why I turned to court documents, which are free and public. There are enough industry lawsuits out there with discussions of the relevant test standards that are part of the dispute leading to the lawsuit.

I also found it interesting reading another batch of court docs on a related lawsuit by Shark against Dyson that Dyson was originally earning about $300 on a $600 DC16 in pure profit, but that went down to $100 per vacuum ( per vacuum ) when Shark introduced the Lift Away line. In my gut I always suspected Dyson's were grossly over priced for what you get and this more or less confirms it.

 

[centralsweeper63]

I apologize for not explaining the testing process well enough. I will write it out as clearly as I can.

1. weigh the carpet, bin or bag and the dirt. write down these numbers.
2. add x amount of x material onto the carpet as evenly as possible.
3. use your chosen method of measuring movement speed to do x amount of passes.
4. weigh the bag or bins and the carpet again. Note how much changed between the two.
5. now you know what amount of the placed down dirt was extracted.

you could vacuum brand new carpet pieces for about 20 minutes each to remove fuzz and settled dust. Measure what goes in and what comes out of each carpet so you know what builds up over time if you want absolute accuracy. Then you can calculate dust extraction based on the total amount of dirt in the carpet. You could measure pet hair pickup by the total number of passes to remove all of it. My testing is not based on industry standards, instead my opinion of what factors need to be considered when testing a vacuum. Science and theory can prove anything on paper, however, Actual testing accounts for all the little details missed. It also covers variables like particle size and shape. In Christchurch we have a library that has some of the ASTM standards that you can borrow at no charge.

Vacuumfacts, if there is any problems in my testing please let me know and I will try to think of a way to remedy them.

 

[Absolute Rainbow]

I noticed no one has ever said what tests should show, how they achieve it, and why it's important relative to the specific known science. They've never even compared it to industry test standards. Anyone who hasn't, credibly, can be confidently ignored, both here and elsewhere. I've outlined what's important in my videos. The closest approximation to I've seen (that isn't my testing) comes from Frickhelm who is qualitatively consistent with me. There are some very interesting things that have cropped up though since the V16, that expose weaknesses in even our testing, but that will have to wait until the review next year.

@Vacuum Facts, reveal the necessary mod for V16 first, maybe as part of your myth-busting anthology leading up to the full review. That review can still wait for later, most likely until next year (2026). Also, why didn't many reviewers bring themselves to compare their tests to industry standards?

Based on public documents of Europro ( Shark )'s lawsuit against Dyson each company spent over $1 million to have labs like Intertek and IBS perform just the ATSM F608 Embedded Dirt Pick Up Test no hobbyist is going to reproduce the ATSM tests exactly. But that should be the goal to get as close as possible to these as one can. To their credit Vacuum Wars does conduct the smoke and particle tests IAW the relevant ATSM test standard. Their embedded dirt test follows the standard to a point but they don't use three different machines and four different carpet types to get an average value across the three machines as required by the ATSM standard. But their test is a lot more relevant than the silly subjective, cough cough, "test" conducted by Performance Reviews.

https://www.govinfo.gov/content/pkg...v-09442/pdf/USCOURTS-ilnd-1_14-cv-09442-3.pdf

One thing I found interesting from reading the court docs was that when testing bagless machines the dust bin must return to within 2 grams of it's initial clean pre test weight after each run or the next run is invalid. From reading the court docs this is apparently both difficult on some machines to achieve and often overlooked, skewing the results of subsequent runs as the bin still has residual from the prior run.

Btw, knock yourself out buying copies of the many ATSM test standards. Each one has to be purchased and nobody is giving them away. That is why I turned to court documents, which are free and public. There are enough industry lawsuits out there with discussions of the relevant test standards that are part of the dispute leading to the lawsuit.

@cheesewonton, what's the difference between ATSM standards and court documents?

Also, I agree that both @Vacuum Facts and Vacuum Wars are better than Performance Reviews in such regards, because unlike the latter, both tried to be objective. In fact, I wouldn't mind listening to Vacuum Wars wholeheartedly if they fixed their airflow and suction measurements and more stochastic deep cleaning tests, which is something @Vacuum Facts himself already excelled at.

 

[Vacuum Facts]

Vacuumfacts, if there is any problems in my testing please let me know and I will try to think of a way to remedy them.

The weaknesses I can see are as follows:

• No procedure and measurements outlined to ensure consistent initialisation
• No discussion of the distribution and placement of dust within pile — “spread evenly” is extremely vague and far more needs to be considered
• No discussion of the properties of the dust materials and learned explanation of why they’re representative of an environment the machines were designed for
• No discussion of the important properties of the carpet
• No quantification of the cleaning speed and understanding why this is important, including overlap properties
• No discussion of how to ensure accuracy of what’s measured and priming system to account for mass losses within the machine (dust sticking etc. and not being weighed)
• No recognition, appreciation, or understand the statistical nature of particle removal from first order systems and how this is to be captured and correctly interpreted
• No evidence of data reproducibility in any testing methodology, given all the above

Most people don’t have PhDs in experimentalism, so this is not surprising. Much of this is covered in my videos.

Even if you did all the above, without a laboratory and the necessary training, the results still have to be taken with a pinch of salt owing to lack of rigor that not using a laboratory brings.

 

[cheesewonton]

@Vacuum Facts, reveal the necessary mod for V16 first, maybe as part of your myth-busting anthology leading up to the full review. That review can still wait for later, most likely until next year (2026). Also, why didn't many reviewers bring themselves to compare their tests to industry standards?

@cheesewonton, what's the difference between ATSM standards and court documents?

Also, I agree that both @Vacuum Facts and Vacuum Wars are better than Performance Reviews in such regards, because unlike the latter, both tried to be objective. In fact, I wouldn't mind listening to Vacuum Wars wholeheartedly if they fixed their airflow and suction measurements and more stochastic deep cleaning tests, which is something @Vacuum Facts himself already excelled at.

Stochastic means a collection of random variable indexed to a set. You and Vac Facts like to throw around scientific sounding terms without even knowing what they mean. That might fly with people who didn't do math or statistics for a living but I did. You two are not fooling me with your use of terms that have no contextual meaning. Unless you are blind testing a selection of vacuums, meaning you have no idea what vacuum you test and your choice of vacuum is purely random you do not have a stochastic test of anything. In other words all you know is that choice A had this much dirt removal, choice B had this much, etc. It you are only testing one vacuum or one make and model of vacuum you can never have a random variable to index to a set. But go ahead and have fun throwing out scientific sounding words not knowing the underlying math.

 

[centralsweeper63]

To distribute dust you can use a flour sifter ( the type with a pull trigger that is used in dusting a cake in icing sugar.) and count how many times you pull the trigger. This allows you to pull the trigger and release to evenly cover the rug. 50 grams of dirt is about right for a 4x4 foot rug. In a home you usually encounter dust, grit, fluff and larger debris. Dust and grit is replicated by a mix of equal parts flour, baking soda and fine sand. Fluff is replicated by kpok, which is spread by holding a large clump and holding it about a centimetre above the carpet and letting it stick how it wishes. larger debris can be replicated by holding about 100 grams of clean cat litter in your hand and holding MOST but NOT all and throwing it randomly over the rug. I haven't measured how fast I move the vacuum across the rug. You could take the size of the rug and time how long it takes to do 100 passes figure that out and then calculate the average speed based on that. I vacuum in a w being straight forward diagonal back and repeat. Overlapping 50% is my standard. If I understand your second to last point correctly, science and theory has no effect on a practical test other than the hypotheses. This is why we test, to avoid unknowns and practically prove them correct or incorrect. To account for particle sticking etc you simply weigh the whole machine before and after. weighing the carpet proves what goes in and out. Hence the record keeping. The important properties of the carpet are as follows: two pile, This is tricky for vacuums as there is a loose tall pile of handle then also a very denso short pile bellow. This makes cleaning extremely difficult. Wool loop pile, wool loop pile carpet is great for creating traps and block as it is very densely made. Dust in the centre of a loop is difficult to clean!. ZSolution dyed nylon is great to represent a typical average carpet. Mohawk carpet is great for representing thedeeper piles that trap dirt. an area the size of a bottle cap can hold up to roughly 2-3 teaspoons of dust without looking terrible. Hence it is terrible to fully clean and know it is clean.

 

[Vacuum Facts]

Stochastic means a collection of random variable indexed to a set.

Stochastic means it has a random distribution. It should be obvious what this means in the context I've used it (which incidentally is not the meaning others have wrongly used it with).

To distribute dust you can use a flour sifter ( the type with a pull trigger that is used in dusting a cake in icing sugar.) and count how many times you pull the trigger. This allows you to pull the trigger and release to evenly cover the rug. 50 grams of dirt is about right for a 4x4 foot rug. In a home you usually encounter dust, grit, fluff and larger debris. Dust and grit is replicated by a mix of equal parts flour, baking soda and fine sand. Fluff is replicated by kpok, which is spread by holding a large clump and holding it about a centimetre above the carpet and letting it stick how it wishes. larger debris can be replicated by holding about 100 grams of clean cat litter in your hand and holding MOST but NOT all and throwing it randomly over the rug. I haven't measured how fast I move the vacuum across the rug. You could take the size of the rug and time how long it takes to do 100 passes figure that out and then calculate the average speed based on that. I vacuum in a w being straight forward diagonal back and repeat. Overlapping 50% is my standard. If I understand your second to last point correctly, science and theory has no effect on a practical test other than the hypotheses. This is why we test, to avoid unknowns and practically prove them correct or incorrect. To account for particle sticking etc you simply weigh the whole machine before and after. weighing the carpet proves what goes in and out. Hence the record keeping. The important properties of the carpet are as follows: two pile, This is tricky for vacuums as there is a loose tall pile of handle then also a very denso short pile bellow. This makes cleaning extremely difficult. Wool loop pile, wool loop pile carpet is great for creating traps and block as it is very densely made. Dust in the centre of a loop is difficult to clean!. ZSolution dyed nylon is great to represent a typical average carpet. Mohawk carpet is great for representing thedeeper piles that trap dirt. an area the size of a bottle cap can hold up to roughly 2-3 teaspoons of dust without looking terrible. Hence it is terrible to fully clean and know it is clean.

I recommend following my testing procedure if you're interested in having convincing relative comparisons that are real-world representative. But even mine has weaknesses (just fewer than other bedroom testers out there). I'd need a lab, expensive equipment, and a lot more spare time to do it properly.