How We Test Projectors

We test video projectors in our Televisioninfo.com testing lab, which is a light-proofed room in our offices. All of the walls are covered with Duvetyne to avoid any reflections, and all potential light sources (such as screens and LED lights on other devices) are masked to avoid interfering with the projector being tested.

Chroma meter
To analyze the photometry and colorimetry properties of the a projector, we use a Konica Minolta CS-200 Chroma Meter, a laboratory spectroradiometer that provides extremely accurate luminance and color measurements for all display technologies. It has a very narrow one-degree acceptance angle, which is very important for the accurate measurements. The CS-200 can measure light sources in the range of 0.005 to 200,000 cd/m², with an absolute accuracy of +/- 0.02 cd/m². It is significantly more accurate than the instruments used by many other reviewers, which use a set of color filters instead of the light spectrum and generally have a wide acceptance angle that contaminates the luminance measurements. The CS-200 connects to a PC via a USB port, and every data sample is logged.

The CS-200 measures the light reflected from the screen, and our testing process involves capturing many thousands of individual data points, which is done using a customized scripting system that automates the testing process. We then use a number of sophisticated mathematical tools to analyze this data and produce the results and scores that you see in the reviews on this site. For more details of what we test and how we analyze these results, scroll down to the individual test below.

Lux Meter
To measure the lux level that a projector produces, we use a Sinometer LX1330B lux meter.

Noise Meter
We measure the noise that projectors produce using a Check Mate SPL meter, set to use the A weighting.

Screen
Our test screen is an 80-inch Carada Precision Cinema Brilliant White screen with a 1.78:1 aspect ratio (also known as 16:9; the same aspect ratio as a HDTV image). We set the projector to fill as much of this screen as possible, then we measure the size of the active area for our calculations of lumens, throw ratio, etc. This screen has a gain of 1.0, and we have found that it has very neutral color reflective characteristics: it does not boost or diminish any visible frequencies. This screen also includes a black absorbent material on the edges of the frame that avoids reflections.

To measure the lumen output of a projector, we project an all-white screen onto our test screen at the projectors native resolution and measure the lux level at 9 points (the crosses indicated on the screen). We then average these readings and multiply it by the active area of the screen to produce the lumens rating. This process is repeated in all of the image or color modes that the projector offers, and our scoring is based on the lumen output in the brightest mode. The lumen rating in other modes is discussed in the review as well, including our calibrated mode.

One thing to note here is that we use an all-white screen, and some projectors use special modes that boost this at the cost of color accuracy (such as the BrilliantColor mode of DLP projectors). these modes are effective for making presentations and the like look better in an office, we leave them turned on for this test, but we turn them off for the color tests. This test does not indicate the brightness of colors on the screen, and some projectors produce very bright whites at the cost of the brightness of colors. We also test the reduced power mode of the projector in a similar fashion.

The brightness of a projector often comes at the cost of weak blacks: light bouncing around inside the projector mechanism turns the blacks into greys. To measure this effect, we measure the level of a small area of black in the middle of the screen as an increasing amount of the screen is white. If the projector has a problem, the black level will rise as the amount of white increases, and our scoring is based on this increase. This test is carried out using the mode that produced the maximum lumen level in the Peak Brightness test

The color of white that a projector produces can vary significantly with factory settings and picture modes. The exact color of white is specified precisely by its CIE chromaticity coordinates, and more commonly by its correlated color temperature, which is a rough approximation to the light given off by a laboratory black body at a temperature of 5,000 to 15,000 degrees.

The Konica Minolta CS-200 Spectroradiometer that we use can measure the chromaticity coordinates and correlated color temperature very accurately. We use this to measure the performance of the display being tested, measuring the red, green and blue primaries as well as the D65 point. We test by setting the display as close as possible to D65 (a measured 6500k), which is a television and photographic industry standard. D65 approximates the color of daylight at noon on an overcast day and includes components of the both blue sky and direct sunlight.

To test how well the projector reproduces changes in both greyscale and color, we display a number of screens at intensity levels between 255 (the brightest white) and 0 (complete black) for white and the RGB primary colors, measuring both the color temperature and color coordinates of each point in the range. The scoring for this test is based on the amount of variance from the maximum intensity chromaticity values, measured in the CIE 1976 uniform color space (u’, v’). Although we feature both the color temperature variation and the CIE 1976 color space distance in our review, the score is based on the latter, as this provides a better measure of how the white of the display shifts within the color space. We discount any shift of less than 0.004, as this is not noticeable by most observers. This distance is shown on our charts by the green area; any shift within this area is not visible by most users.

We determine the transfer function of a display for each of the primary colors by measuring the luminance of a screen for the range of signal intensities from 0 to 255. We then analyze the curve to determine the granularity and other characteristics. Our scoring is based on this analysis; issues such as excessive stepping, clipping and uneven response reduce the score the projector achieves.

We test how closely the projected image matches the standard primary colors of ITU-R BT.709 (generally referred to as Rec.709), which defines the color gamut of high definition TV signals. The scoring for this test is based on the distance between the measured and standard values; the greater the distance, the lower the score. We plot the measured and recommended gamut in the CIE 1976 Lu’v’ color space. Note that a color gamut that is greater than the standard is also undesirable; this will produce colors that are outside of the standard gamut, producing incorrect colors that are too saturated and not as the content producer intended.

Our motion test uses a variety of test screens and video sources, including the Multimedia with Motion edition of DisplayMate and a number of movie sequences. We use these to judge the quality of the motion on the display, looking for issues with ghosting, shadowing, smearing, the rainbow effect and other common artifacts.

We test the 3:2 pulldown processing (which is also known as 2:3 pulldown) capabilities of the display with the HQV Benchmark test disc. We also evaluate the performance of the display with a video source that has been processed with the telecine effect.

To test the performance of the display with a 24 frames per second signal, we use a PlayStation 3 configured to output a 24 frames per second signal playing a Blu-ray disc.

To measure the noise that a projector produces, we measure the noise level using a Check Mate SPL meter set to the A weighting at a fixed distance of 12 inches from the projector. We measure the noise level at 6 points around the projector (5 around the side of the projector, and one above) and average the result.

To measure the heat produced by the projector, we measure the temperature of the air entering the projector body and the air exiting the body using a temperature probe. We then include the difference between the two in the review, indicating how much heat the projector is likely to add to the environment it is used in.

We test power consumption using a Watts Up Pro power meter connected to a computer. In order to make the test results comparable between projectors, we test 4 of the modes that the projector offers, including the brightest, our calibrated settings, and an “eco” or other power saving mode. We then use these figures to calculate the typical cost of using this HDTV, working on the basis of electricity costing 10.7 cents a Kilowatt Hour (this is the 12-month average for the cost of electricity in the USA up to April 2008 from the EPA), with the viewer watching the TV for five hours a day, seven days a week, and leaving it in standby mode the rest of the time.

Every test that we perform results in a score, which allows us to compare displays directly, even if they are not tested side by side. Our rigorous, scientific scoring system ensures that our results are consistent, accurate and represent the strengths and weaknesses of a display. Many of our scores are open-ended; the score can climb beyond the nominal maximum of 10 as the performance of new models improves. Most reviewers use a fixed 1 to 10 scoring system, but this means that once a product has earned a top score, there is nowhere else to go; the reviewer has to reset the scoring system and start again. Our infinite score system allows us to keep going, so if a new technology comes along that provides radically better color or a more accurate color gamut, we can still score it, and compare it with other models that we tested before the new technology arrived.

To create our overall score for each display, each individual score is multiplied by a weighting, which is based on how important we think the individual factor is to the typical consumer. The weightings are listed below: the majority of the score is based upon the performance tests outlined above, but the features that the display offers also play a part. You can see the individual weightings that we use for each score on the Specs & Ratings page of any review (such as this one).

Projectors are designed to be used in a variety of situations, such as showing a movie in a darkened room or showing a business presentation in a bright office. There is no single set of calibrated settings that will produce the best results for all situations, so we create a set of calibrated settings for one purpose: watching movies. In calibrating this way, our aim is to find the best balance of screen brightness and color accuracy for watching movies and TV in a dark setting. We use this set of settings to analyze the color performance of the display, as this is the most important criteria for performance when watching movies.

We calculate the size and brightness of the image that the projector will produce on a number of different sized screens, all using the mode that produces the maximum lumen output. We also make projections on how big a screen will be usable in a variety of situations by setting minimum levels of screen illuminance: 200 lumens for a darkened room, 400 lux for a dimly lit room, 800 for a moderately well lit room and 1500 lux for a well-lit room. Note that these are guidelines only: the type and origin of the room lighting and the type of screen used will all have an impact on the visibility of an image and the usability of a projector in different lighting conditions. We also assume a 16:9 aspect ratio screen and signal source: the projection screen size and brightness will differ with different sized screens and signal sources.

We also test the performance of projectors by projecting a test image onto a number of surfaces and photographing the projected image. These include:

A standard glossy dry erase white board
Light, unvarnished wood panelling (cedar wood)
A light grey color matte paint (Sierra Grande paint color from Mythic Paints)
A light, cream colored matte paint (Mistic Heaven, from Mythic Paints)