STP1294: The Use of Ranked Versus Rated Data in the Evaluation of Weathered Specimens or Material

    Asher, LB
    Researcher and Statistical Consultant, 3M Company, St. Paul, Mn

    Pages: 9    Published: Jan 1996


    Abstract

    The evaluation of specimens after weathering (in natural outdoor exposures or accelerated cycles) can be handled by a number of approaches; two of these are ranking and rating. This paper will center on these two approaches and will demonstrate how ranking is often superior. Rating requires one to compare the specimens against a scale (ie. 1 through 10 with 1 being “excellent”, 5 being “good” and 10 being “poor”, etc.). Ranking on the other hand requires the direct comparison of the specimens against each other.

    Since people typically do not like to be overly critical or overly generous, the rating approach of ten leads to the specimens being rated in the middle of the scale (with ratings of 4 to 7 out of a scale of 1 to 10). The specimens differentiation or discrimination is therefore greatly reduced. The ranking approach forces people to rank the specimens from best to worst. It is highly recommended that ties not be allowed; this forces the evaluators to clearly state their preferences.

    Since ranking involves a direct comparison, all the specimens must be available at the same time. If this is not possible, the rating approach would be the logical choice. The use of visual examples or standards (that visually demonstrate three ratings,ie. 1,5 & 10) is strongly recommended in these situations.

    The ranked and rated data from a visual comparison of six reflective sheetings for whiteness will be used to demonstrate the effectiveness of each approach. The statistical evaluation of the ranked data will be discussed. This will include both simplified and more comprehensive analysis approaches. The averages and ranges of ranked data will represent the simplified methods. Thurstone Scaling and Analysis of Variance (ANOVA) represent the more comprehensive methods.

    Keywords:

    rank, rate, weathered specimens, weathered material, visual comparison, data, Thurstone Scaling, analysis of variance


    Paper ID: STP16162S

    Committee/Subcommittee: G03

    DOI: 10.1520/STP16162S


    CrossRef ASTM International is a member of CrossRef.