NISTIR 6014

THE ABC's OF THE U.S. CONFORMITY ASSESSMENT SYSTEM

Maureen A. Breitenberg
Office of Standards Services
National Institute of Standards and Technology
Gaithersburg, MD 20899

April 1997

TABLE OF CONTENTS

• Foreword
• Acknowledgments
• Abstract
• Introduction
• Standardization
• Conformity Assessment
• Inspection
• Testing
• Criteria for Laboratory Evaluation
• Laboratory Accreditation
• Laboratory Accreditation Evaluation Criteria
• Other Considerations
• U.S. Laboratory Accreditation System
• Certification
• Types of Certification
• Third Party Certification Programs in the United States
• Choice of Standards
• Certification Methodology
• Certificates or Marks of Conformity (Certification Marks)
• Certification Program Evaluation Criteria
• Other Considerations
• U.S. Certification Program Accreditation System
• Management System Assessment/Registration
• The ISO 9000 Standards
• The ISO 14000 Standards
• Management System Assessors
• Management System Registration
• Accreditation of U.S. Registrars
• Accreditation Program Recognition
• International/Regional Cooperation in Conformity Assessment Summary
• Appendix I --- List of Relevant ISO/IEC Documents
• Appendix II -- List of Acronyms
• Appendix III - Information Available from the Office of Standards Services

FOREWORD

The Office of Standards Services periodically publishes information related to standards and conformity assessment as a service to producers and users of such systems -- both in the government and in the private sector. This report provides those not fully familiar with these fields with an introduction to some of the complexities. We hope that this material will be informative and will serve to stimulate wider understanding of the purpose and nature of the various aspects of conformity assessment, as well as interrelationships among related activities. The interested reader may wish to take advantage of other available publications and services provided by the Office of Standards Services, some of which are described in Appendix III.

ACKNOWLEDGMENTS

I would like to thank John Donaldson, ANSI; Charles Hyer, The Marley Organization; Walter Leight, NIST; and Krista Johnsen Leuteritz, NIST; and many others for their careful review of and comments on this document.

Maureen A. Breitenberg
Office of Standards Services

ABSTRACT

This report is designed to provide the reader with an introduction to conformity assessment and information on how the various conformity assessment activities are interlinked. It highlights some of the field's more important aspects and serves as background for using available documents and services.

Key Words: accreditation; certificates of conformity; certification; certification marks; conformance testing; conformity; environmental management systems; inspection; laboratory accreditation; product listing; quality assurance; quality system; quality system registration; recognition; registration; standardization; standards; testing

INTRODUCTION

The U.S. marketplace is becoming increasingly globalized as evidenced by the vast array of products made in foreign countries which are available to U.S. consumers. This growing complexity has made buyers increasingly dependent on formal methods and procedures for ensuring that the products, services, and systems that they purchase -- whether domestic or foreign -- consistently meet their needs.

Some product characteristics are vital for safe and effective performance, and many of these characteristics cannot be evaluated simply by picking up and examining the product in the marketplace. Such characteristics need to be determined, assessed, and assurance provided to the buyer (or other interested party) that the product conforms to requirements and that conformance is consistent from product to product. For example, if a buyer determines that it is critical that an airplane part be able to withstand pressure of at least 5000 kilograms per square centimeter, then the buyer needs assurance that none of the purchased parts will fail to withstand such pressure. The buyer cannot determine whether a part meets this requirement by simply looking at it.

Conformity assessment is defined in ISO/IEC (1) Guide 2: 1996 as: "any activity concerned with determining directly or indirectly that relevant requirements are fulfilled." Conformity assessment procedures provide a means of ensuring that the products, services, or systems produced or operated have the required characteristics, and that these characteristics are consistent from product to product, service to service, or system to system. Conformity assessment includes: sampling and testing; inspection; certification; and quality and environmental system assessment and registration. It also includes accreditation of the competence of those activities by a third party and recognition (usually by a government agency) of an accreditation program's capability.

While each of these activities is a distinct operation, they are closely interrelated. The inclusion or absence of any of these activities, as well as the quality with which any one of them is performed, can have a significant effect on the confidence and reliance that can be placed on the results of the entire conformity assessment process. In addition, standards, which underlie each of these activities, can also have a major impact on the outcome of each specific conformity assessment activity as well as a cumulative effect on the outcome of the entire process. Conformity assessment activities form a vital link between standards (which define necessary characteristics or requirements for products) and the products themselves. Together standards and conformity assessment activities impact almost every aspect of life in the United States.

This impact is especially evident when one considers specific product examples. A state-of-the-art computer is of no benefit without compatible software. A new and technologically superior appliance is useless if its plug does not fit the outlet; or (worse yet!) appears to fit, but actually increases the potential for fire or electrical shock. We do not want to purchase a product that appears to meet our needs and then discover that it has potentially dangerous undetected defects. If we purchase products on a regular basis, we also don't want to find that they are periodically unacceptable because of unexplained or unexpected variations in production or delivery processes.

Conformity assessment can verify that a particular product meets a given level of quality or safety, and provide the user with explicit or implicit information about its characteristics, the consistency of those characteristics, and/or performance of the product. Conformity assessment can also increase a buyer's confidence in a product, furnish useful information to a buyer, and help to substantiate a company's advertising and labeling claims regarding a product (2). Conformity assessment is therefore an important marketplace communications device -- a means of exchanging information between buyer and seller. It is vital for buyers, sellers, and other interested parties to understand the conformity assessment process to competently judge the value of a particular assessment scheme and to use the information resulting from that scheme to make intelligent marketplace choices.

The quality of the conformity assessment information conveyed depends on: the impartiality and competence of the assessment body; the types of assessment activities included in the scheme; and the adequacy and appropriateness of the standards against which the product is evaluated. Improperly conducted conformity assessment activities may result in widespread buyer deception. If properly conducted, however, conformity assessment can furnish valuable information to the marketplace and can serve as the basis for increased opportunities for national and international trade.

The impact of conformity assessment on both domestic and international trade was prominently noted in the 1994 Agreement of Technical Barriers to Trade (TBT Agreement) of the international General Agreement on Tariffs and Trade of the World Trade Organization (WTO). The TBT Agreement recognized that conformity assessment activities can expedite or seriously hinder the free flow of goods in international commerce and established procedural requirements for conformity assessment schemes to avoid the establishment of unnecessary obstacles to trade. The agreement requires that conformity assessment procedures be "prepared, adopted and applied so as to grant access for suppliers of like products originating in the territories of other Members [signatories to the agreement] under conditions no less favorable that those accorded to suppliers of like product of national origin or originating in any other country... ." The Agreement also requires that such procedures (3) not be "prepared, adopted or applied with a view to or with the effect of creating unnecessary obstacles to international trade." Ideally, a properly conducted conformity assessment program benefits, not hinders, the free flow of goods into the marketplace.

This report will provide the reader an overview of the topic of conformity assessment in order to better understand its impact on the marketplace. The report will: (1) discuss some concerns regarding the standards used in conformity assessment activities; (2) describe each type of conformity assessment activity and examine some of the issues involved in assessing the activity's competence; and (3) identify some of the interrelationships among conformity assessment activities.

To improve readability, this report will use the term product to refer to a product, service, process, and/or system. The reader should also note that while many examples in this paper refer to products (because such examples are generally easier to understand), the implications are equally applicable to services, processes, and systems.

STANDARDIZATION

A standard was defined by the National Standards Policy Advisory Committee (4) as:

"A prescribed set of rules, conditions, or requirements concerning definitions of terms; classification of components; specification of materials, performance, or operations; delineation of procedures; or measurement of quantity and quality in describing materials, products, systems, services, or practices."

Standards (5) are known to have existed as early as 7000 B.C. when cylindrical stones were used as units of weight in Egypt. In more modern times, the great blaze in downtown Baltimore in February 1904 and other similar catastrophes provided tragic and undeniable evidence of the importance of standards. While the fire in Baltimore burned, fire engines from as far away as New York rushed to the scene only to discover that their hoses would not fit Baltimore hydrants. Those "alien" fire engines were useless! The inferno raged for more than thirty hours, destroying 1526 buildings covering more than seventy city blocks. All electric light, telephone, telegraph, and power facilities were also razed. In contrast, 23 years later, help from 20 neighboring towns saved Fall River, Massachusetts from destruction since hydrants and hose couplings had by then been standardized in those communities.

As late as 1927, a color-blind motorist had as good (or as bad) a chance as anyone else when trying to interpret traffic signals. Purple, orange, green, blue, yellow, and red lights greeted him as he drove from state to state. In some states, green meant "Go," in others "Stop." Red, not yellow, lights meant caution in New York City. In 1927 a national code for colors was established through the work of the American Association of State Highway Officials, the National Bureau of Standards (now the National Institute of Standards and Technology (NIST)) and the National Safety Council. Imagine the chaos that would occur during rush hour in any major U.S. city today if newcomers and tourists did not know what traffic signals meant!

Probably the most significant standard ever developed in the United States, however, was the railroads' standard track gage. This standard (now also used in Great Britain, the United States, Canada and much of continental Europe) enables railroad rolling stock to cross the country.

It was the Second World War, however, that brought the urgency of extending domestic standardization to the international level. Allied supplies and facilities were severely strained due to the incompatibility of tools, replacement parts, and equipment. The War highlighted the need for standards aimed at reducing inventories and increasing compatibility.

Standards may be classified in numerous ways, some of which are described here. Standards can define words so that an industry or parties to a transaction may use a common, clearly understood language. Test method standards define the process/procedures to be used to assess the performance or other characteristics of a product. Product standards establish qualities or requirements for a product (or related group of products) to ensure that it will function safely and/or effectively. Process standards specify requirements to be met by a process (e.g., an assembly line operation) in order to function effectively. Service standards (e.g., standards for servicing or repairing a car), establish requirements to be met in order to achieve the designated purpose of this service. Interface standards (e.g., a standard for the point of connection between a telephone and a computer terminal), are concerned with the compatibility of products. Another type of standard lists product data requirements for which values must be obtained.

Still another classification scheme distinguishes between voluntary standards, which by themselves impose no obligations regarding use, and mandatory standards. A mandatory standard is generally published as part of a code, rule or regulation by a regulatory government body and imposes an obligation on specified parties to conform to it. However, the distinction between these two categories may be lost when voluntary consensus standards are referenced in government regulations, effectively making them "mandatory" standards. Voluntary consensus standards may also become "quasi-mandatory" due to conditions in the marketplace. For example, the health care industry is sensitive to the need to use the safest products available to ensure patient safety and to protect manufacturers, vendors and health care providers against lawsuits. Informed buyers of health care products will frequently insist that products meet all appropriate voluntary consensus standards. If manufacturers wish to compete effectively, their products must conform to such standards.

It is clear, then, that standards cover a broad range of types and serve a wide variety of purposes. In the United States alone, there are approximately 49 000 private sector voluntary standards developed by more than 620 organizations (6). This number does not include approximately 44 000 regulatory and procurement standards (developed and used by approximately 80 federal regulatory and procurement authorities) or other codes, rules and regulations containing standards which have been developed and adopted by state and local government authorities. There are also company standards, developed for use by a company or organization for its own products or for the products it purchases. However, some company standards gain such widespread marketplace acceptance that they can and do become de facto industry standards, such as the architecture for the personal computer established by IBM and widely used in the personal computer industry.

In addition, numerous foreign national, regional and international organizations produce standards of interest and importance to U.S. manufacturers and exporters. The International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) produce the largest number of international standards. In its 1996 catalog, ISO had 10 189 international standards, in addition to other technical documents. In 1995 alone, the IEC issued 429 standards totalling 17 000 bilingual pages.

Standards are vital tools of industry and commerce because they promote understanding between buyers and sellers and make possible mutually beneficial commercial transactions. As noted before, buyers cannot always evaluate product specifications or characteristics by inspection or even from prior experience. Information on a product's conformance (or nonconformance) to a particular standard can provide an efficient method of conveying information needed by a buyer on the product's safety and suitability.

While physical measurement standards tend to be objects (e.g., a standard meter, kilogram, etc.,) rather than pieces of paper, these types of standards are outside the scope of this paper. Most standards are written documents. Such standards have little or no significance -- unless they are used. Some standards never receive widespread acceptance and use. Others may have been accepted by industry at one time, but now apply to technologies that have become outdated. One of the most important uses for standards is within a conformity assessment process. Standards provide the basis for conformity assessment activities which, in turn, are the basis for many buyer-seller transactions. Hence standards used in conformity assessment activities can have tremendous impact on companies and nations and even on the economic fabric of the world market. Because standards have such an impact on the validity of conformity assessment activities, it is important to have some familiarity with them.

Standards can cover many aspects of the conformity assessment process. They can describe characteristics of the product for which conformity is sought; the methodology (e.g., test, inspection or other assessment methods) used to assess that conformity; or even the conformity assessment process itself (e.g., how a certification program should be operated). Standards used in conformity assessment should be clearly and concisely written, readily understood, precise, technically credible, and contain only unambiguous requirements - the absence or presence of which can be objectively verified. The use of well written standards in a conformity assessment process lends credibility and validity to the process, increasing its usefulness. Requirements in such standards should be stated in terms of "shall" or "will," rather than "may." In addition, standards for conformity assessment methods (e.g., test methods) used in the conformity assessment process should be capable of evaluating the conformity of a product to the specified requirements in a manner that produces test results that are within an acceptable accuracy range. The results should also be consistent with results produced by the same laboratory when it repeats the test using the same or a similar test method. The results should also be reproducible, i.e., capable of being duplicated by other testing bodies using the same or similar test methods.

Standards used in conformity assessment should not impede innovation. For this reason, performance standards, which describe how a product is supposed to function, are preferred over design (also called prescriptive) standards, which define how the product is to be designed or constructed (7). For example, a performance standard for water pipe might set requirements for the pressure per unit area that a pipe must withstand, along with a test method to determine if a pipe sample meets the requirement (8). Manufacturers are free to choose any product design, material, and manufacturing process as long as the pipe can perform in the specified manner. On the other hand, a standard that requires that a pipe be made of a given gage of copper and have a given diameter is a design standard. Manufacturers trying to comply with such as standard are not free to make the pipe out of stainless steel, for example, or to vary the size of the diameter no matter how such changes impact the pipe's performance. For this reason, including design requirements in standards can discourage innovation.

The example above is rather simplified. Few standards are purely design or performance in nature. Most are a mix of requirements of both types. Requirements in a standard, for example, may be mostly written in terms of performance; while the test method for ensuring conformance might be written in design terms. In fact, design requirements are frequently more appropriate for test methods where the need for accuracy and reproducibility usually outweigh other considerations. It should also be noted that the determination of conformance to performance standards may be more difficult than for design standards. For example, it is usually more difficult to determine that a pipe can perform in the specified manner than it is to determine that a pipe is made of a given gage of copper and has a given diameter. Performance standards are also more difficult to write. Therefore, the use of performance standards, while desirable, may not be practical in all situations. In some cases, the disadvantages associated with the use of performance standards may outweigh other considerations.

However, in general, when products can be defined in terms of required performance characteristics, the resulting performance standards tend to be less restrictive than design standards. Performance standards are also more likely to allow the inclusion of technological innovations in the product and to prevent unnecessary barriers to trade. This is why the United States and the other signatories to the WTO Agreement are encouraged to write technical regulations and standards in terms of performance rather than design (9).

It should also be noted that writing a poor performance standard is as easy as writing a poor design standard. A poorly written standard of EITHER type is unlikely to lead to greater technological innovation, increased trade, or to an acceptable outcome when used in a conformity assessment process.

Standards used in conformity assessment should also be chosen so that they specify all essential characteristics of a product necessary for achieving the objective of the conformity assessment activity. For example, if assurance of the electrical safety of a coffee pot is the objective, a standard that covers only the electrical safety of the coffee pot's cord and does not cover the pot's heating element would not meet the objective. Knowing what aspects of the product will be evaluated in a conformity assessment process and whether there are other aspects which might impact quality, safety, or performance allows the user of the conformity assessment data to evaluate the data's significance.

Given the large number of national and international standards, it is not surprising that a number of standards are redundant or overlapping. In a few areas (particularly building and construction), there are sometimes five or six standards that, while not identical, define functional requirements for the same type of product or material. This redundancy makes it especially important to know not only which aspects of the product are covered in a conformity assessment process, but which standard(s) was used. Requirements in two different standards covering the same characteristics may be very different. To understand the results of a conformity assessment process, the user needs to know what standard(s) was used.

CONFORMITY ASSESSMENT

In addition to information on the standards used, those who rely on conformity assessment results also need to know and understand which types of conformity assessment activities were included in the process. The following sections cover a number of conformity assessment activities including: inspection; testing; laboratory accreditation; certification programs and their accreditation; management system assessment/registration and accreditation; and the recognition (usually by a government agency) of the competence of accreditation programs.

INSPECTION

Inspection is defined in ISO/IEC Guide 2 as "conformity evaluation by observation and judgement accompanied as appropriate by measurement, testing or gauging." In the European Standard, EN 45004, inspection is defined as the "(e)xamination of a product design, product, service, process or plant, and determination of their conformity with specific requirements, or on the basis of professional judgement..." While a number of people regard inspection as an activity which is distinct from other types of conformity assessment activities, most other conformity assessment activities do involve inspection to some degree. For example, it would be unusual to test a product without first looking at it (inspecting it) to determine if it is intact or has undergone rough handing that might have damaged it and could therefore affect the outcome of the testing. Likewise, it would be unusual to assess a company's quality system, without inspecting the elements of the facility which could impact the system's operation. Because some type of inspection is included in most forms of conformity assessment and because the topic of conformity assessment is so complex, this paper will not deal with inspection as a separate type of conformity assessment activity. However, the reader should be aware that this activity exists and that inspection can and often does provide information that is used in conformity assessment. ISO has also published a guide on the topic, ISO Guide 39: 1988, "General requirements for the acceptance of inspection bodies," which is currently being updated.

TESTING

Testing laboratories support billion dollar industries and affect the entire operation of U.S. industry and the U.S. regulatory system. Each day major corporate and regulatory decisions are made based on data produced by testing laboratories.

Test data are used in many tasks including:

o product design and research;

o quality control prior to acceptance of incoming materials/components, during production, and prior to shipment/sale;

o insurance underwriting;

o meeting contractual agreements;

o satisfying government regulatory requirements;

o certification and labeling;

o buyer protection and information;

o product comparisons;

o building and structure design, construction and related engineering tasks;

o medical and health services;

o environmental protection;

o product operation, maintenance and repair;

o legal proceedings; and

o forensic work.

Flawed test data can result in defective products capable of causing serious injury or harm to the user or the environment. Defective products (such as fire detection and mitigation equipment and systems, security alarms, aircraft, and autos) can also result in serious injury or death - not only to users, but also to unsuspecting bystanders.

Testing can be performed by laboratories differing widely in size, legal status, purpose, range of testing services offered, and technical competence. They may be government regulatory laboratories, government research laboratories, or government supported laboratories -- at the federal, state or local levels. They can also be college/university laboratories, independent private sector laboratories, laboratories affiliated with or owned by industrial firms or industry associations, or manufacturers' in-house laboratories. Test laboratories can be for-profit or nonprofit. Laboratories can operate facilities in one or multiple locations; and may even operate in multiple countries. Laboratories can offer only a limited range of testing services or services in many fields. There are almost as many different types of laboratories as there are different types of users of the test data that the laboratories produce.

A test is defined by ISO/IEC Guide 2 as a: "technical operation that consists of the determination of one or more characteristics of a given product, process or service according to a specified procedure." Test data result from the performance of a test. If the test method is well written, it is sufficient that the test data comply with the test method's requirements for accuracy and variability requirements.

Accuracy (or bias) refers to the degree of departure of the test result from the "true value." For example, if a product is weighed and the result is 5.1 kg (when the actual weight is 5.0 kg), the test or measurement is inaccurate by .1 kg. The required degree of accuracy will depend on the characteristic being tested and the impact of test data accuracy on the ability of the product being tested to perform in an acceptable manner.

Variability (or precision) refers to the degree of difference between the results from several repetitions of the same test. For example, if that same product (weighing 5.0 kg) were measured three times and the weights were recorded as 5.1 kg, 4.9 kg, and 5.0 kg., these results vary less than if measurements for that product were 4.5 kg, 5.0 kg and 5.5 kg.

Variability can be further defined in terms of repeatability, which is a measure of the variation among the test results when the same or similar test is repeated within ONE laboratory. Reproducibility (or replicability) is a measure of variation of test results from similar tests conducted in DIFFERENT laboratories. Reproducibility can be a key concern in conformity assessment programs which use multiple laboratories.

Problems in the accuracy and variability of test results occur not only due to errors by the laboratory staff or defects in the test equipment, but also arise from other factors, such as flaws or variables in the test method or in the sample selection process. As noted elsewhere, the selection of good test methods is vital to the production of good test results. Because test results are a vital component of most conformity assessment programs, the use good test data is essential for the credibility of any certification program.

Standards organizations have long recognized the importance of the link between testing and laboratory competence. For example, ISO and IEC have published Guide 25, "General Requirements for the Competence of Calibration and Testing Laboratories," which establishes general requirements for laboratory competence to conduct specific calibrations or tests. Laboratory requirements in this guide include: having an appropriate organizational structure; having adequate resources and using only properly trained staff; having a good quality system; using equipment which has been adequately maintained and calibrated; conducting tests under acceptable environmental conditions and using appropriate test methods; and producing accurate, clear, unambiguous, and objective test reports. The compliance of a laboratory with ISO/IEC Guide 25 or its equivalent provides some assurance of the competence of that laboratory.

CRITERIA FOR LABORATORY EVALUATION

The following list contains general criteria which may be used in evaluating laboratories.

1. Laboratory Organization/Independence (No Conflict- of-Interest)

The laboratory should be a legal entity organized in a manner which permits satisfactory performance of all required functions. In addition, a laboratory and its staff should be impartial or independent (that is free from any outside influence), which might bias the integrity and objectivity of the work performed.

2. Financial Stability

The laboratory should have sufficient resources to enable it to properly use and maintain the test equipment and facility, to satisfactorily perform all required functions, and to adequately indemnify itself against financial liabilities/penalties resulting from its operations. The laboratory's business should also be sufficiently diversified so that the loss of a single client does not seriously jeopardize its financial well being.

3. Staff Qualifications Requirements

Each staff member in the laboratory should have the education, training, knowledge, and experience necessary to perform the tasks assigned and an appropriate level of supervision should be maintained. The training of each staff member should be kept current and documented.

4. Adequate Quality System

The laboratory should have a quality system appropriate to the type and amount of work performed. The system should be reviewed periodically by management and revised as needed to ensure continued acceptable performance. The quality system should be suitably documented in a comprehensive, up-to-date quality manual, which is readily available for consultation by staff. Internal audits should be conducted and documented by the laboratory as needed to evaluate its compliance with the requirements in its quality documentation, as well as to assess the efficiency and effectiveness of the system's operation. The results of such audits should be reported to top management and appropriate action taken and documented.

5. Sampling Requirements

If a laboratory receives test materials in quantities larger than the amount required for the test, the laboratory should sample the material in such a manner as to ensure that the sample tested is representative of the entire quantity of material received. Where sample selection is the responsibility of the test laboratory, the laboratory should use appropriate sampling methods and/or techniques.

6. Sample Control/Integrity Requirements

The laboratory should have an effective system which ensures both the identity and integrity of the test samples. Maintaining the integrity of the sample involves preventing it from being damaged during any stage of its collection, shipment, storage, or handling. Such damage can include: physical damage; loss of part of the sample due to leakage or other causes; contamination by foreign materials; failure to maintain the sample within appropriate temperature or atmospheric conditions; or other deterioration, such as deterioration resulting from samples being held too long before testing. Where the sample may be used as legal evidence, a complete record may be required on who had custody of the sample from collection through testing, and (when needed) up until its disposal.

7. Statistical Methods Requirements

The statistical methods used to interpret or to provide additional information about test data should be appropriate and adequate for the type and level of testing undertaken. Control charts, which help distinguish random errors from systematic (assignable cause) errors or variations, should be employed as needed to alert laboratory personnel to potential problems in test procedures or equipment.

8. Recordkeeping Requirements

A laboratory should maintain all test records, observations, calculations and derived data for all tests it performs for an appropriate time period or as required by law.

9. Test Report Content/Format Requirements

Test reports should include all information relevant to sample selection, test performance, and test results. It may also be useful for a laboratory that is involved in related conformity assessment work or research to note if any relationship exists between that work/research and the results contained on the test report. Test data should be displayed in a format that is easy to read and understand. Data included in such reports should be routinely audited and validated, i.e., checked for questionable values and accepted or rejected based on an established set of criteria. Audit levels (the amount of work subject to review and the extent of those reviews) should be appropriate for the type and amount of work being performed as well as the skill of the analyst or technician conducting the tests.

10. Available Operational Manuals/Instructions

The laboratory should have readily available instructions on the operation and maintenance of all materials and equipment, copies of the test methods and standards employed with any additional instructions needed on their application, sample selection and handling procedures, and any other relevant information necessary to ensure the quality of the work performed.

11. Participation in Proficiency Testing Program

The laboratory should participate in proficiency testing to the extent appropriate to ensure the competence of its testing processes. Proficiency testing allows a laboratory to compare its test results with results obtained from other laboratory(s) from tests performed on the same or similar items. Proficiency testing can provide the laboratory with valuable feedback on the competence of its testing processes.

12. Adequacy of Facilities and Equipment

The laboratory should own or have access to all equipment required to perform all test methods it conducts. In addition, the facility should require test methods to be conducted in a controlled environment to prevent any adverse effects on the accuracy of the test result. Specifically, the testing environment should be free from excessive temperatures, temperature fluctuations, dust, moisture, dryness, vibration, and electromagnetic or other interference. The laboratory should also have adequate lighting, heating and ventilation. When needed, specialized facilities such as clean rooms should be available.

13. Equipment Maintenance/Repair/Calibration Requirements

Equipment calibration, preventative maintenance and repair procedures and the choice of reference materials (10) used for calibration should be appropriate for the nature and amount of work being performed. Equipment calibrations should be traceable (11) to some ultimate or national reference standard (12). Complete records should be maintained on all calibration, maintenance and repair procedures performed.

14. Adequate Control over Subcontractors

The laboratory should have a system to ensure that testing and related work performed by any of its subcontractors is at an acceptable level of quality. The nature of the system should be appropriate for the type and amount of work being performed by the subcontractor.

15. Appeals Procedure

The laboratory should have a mechanism to deal with technical questions, appeals, complaints and challenges, originating either from the customer or from interested regulatory or other parties.

LABORATORY ACCREDITATION

While a laboratory can self-declare or self-certify its compliance with requirements such as those listed above, or laboratory users can validate a laboratory's compliance for themselves, another method to ensure the quality and accuracy of laboratory test results is through the use of an independent third party. A third party (independent of the laboratory and the purchasers or users of laboratory testing and test data) can accredit the competence of a laboratory to conduct specific tests or to operate in specific fields.

Accreditation is defined in ISO/IEC Guide 2 as a: "procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks." In the case of laboratory accreditation, the tasks are tests or specific types of tests. Laboratory accreditation does not provide any guarantees about the test results obtained from the performance of any specific test procedures. Laboratory accreditation means that the laboratory is capable of performing specified test methods and procedures correctly, not that the laboratory has competently tested all products in each and every instance. In addition, accreditation provides assurance about a laboratory's capability SOLELY within the scope or areas for which accreditation was granted. If a laboratory is accredited to test concrete, no assurance is provided regarding that laboratory's ability to test any product other than concrete. It should also be noted that not being accredited does not necessarily imply that the laboratory is not technically competent since not all laboratories seek or require accreditation, and accreditation programs may not exist in the laboratory's field of operation.

Users of test results must understand the laboratory accreditation process, the criteria used, and the scope of accreditation to assess the value of that accreditation. This knowledge is necessary to make intelligent choices regarding the use of test results produced by the accredited laboratories. While laboratory accreditation is related to other aspects of conformity assessment, it is nonetheless a distinct procedure. Unlike product or service certification (which is described later in this report), laboratory accreditation involves neither the review nor assessment of products or services. While accredited laboratories may be used in certification programs, laboratories may be accredited to conduct tests (such as EPA's accreditation program for laboratories testing drinking water) in fields where no certification program exists. Some certification programs also use laboratories that are not accredited.

Accreditation may recognize both the technical competence and impartiality (lack of conflicts-of-interest) of a testing laboratory, or only its technical competence. The inclusion or exclusion of a requirement for impartiality is only one of many differences among the various approval criteria used in U.S. laboratory accreditation schemes. Such differences in requirements or criteria for accreditation must be considered in evaluating a particular scheme.

Since most U.S. laboratory accreditation/designation schemes were designed to meet particular governmental or private sector needs, such schemes tend to take distinctive forms and use different sets of procedures to ensure that a laboratory has sufficient competence to perform the specified testing. Some schemes involve only a simple review of data submitted by a laboratory with no attempt at verification. Others require a full scale on-site evaluation of the laboratory's facilities, staff and equipment, and include a review of the results of internal audits, quality system reviews, and proficiency testing. Comparable programs, even those which are conducted by the same organization or government agency, may include different types or number of assessment procedures and may provide different degrees of assurance regarding a laboratory's competence.

Meeting regulatory requirements is probably the primary reason for a laboratory to participate in an accreditation program. Congress, state, and local governments can mandate that required testing be done by an accredited laboratory. Federal, state and local government agencies may also impose a requirement for laboratory accreditation through regulations issued under their own legal authority. Laboratories may also require accreditation to meet testing requirements imposed by foreign governments on products imported into their countries.

However, laboratories may also wish to participate in an accreditation program as: (1) an outside check of their internal quality control program; (2) proof of competence to higher level management within the organization; (3) a competitive advantage over other unaccredited laboratories; (4) a means of protection in liability proceedings; or (4) a means of establishing credibility with the public. In addition, contracts or procurement requirements may mandate the use of an accredited laboratory to conduct any required testing. A laboratory desiring to compete for such work would have to seek accreditation. Laboratories can also sometimes secure reduced medical care and related insurance rates by providing proof of a safe working environment through accreditation. An accreditation requirement could also be imposed as a condition of sale by a purchaser of a laboratory.

The reasons for and benefits of seeking accreditation are as diverse as the laboratories themselves. In general, however, laboratories participate in accreditation programs in expectation of some type of economic return (such as increased business) on the resources they invest in obtaining the accreditation.

Eligibility requirements for laboratories seeking accreditation vary among programs. Some federal programs restrict eligibility to state government laboratories, such as NIST's program to accredit state weights and measures laboratories. Other programs place different restrictions on eligibility. For example, the U.S. Department of Agriculture (USDA) program for inspecting grain for export restricts the number of approved laboratories in a particular geographical area to ensure that each approved laboratory has an adequate market share.

Some accreditation programs are restricted to laboratories operated by the body doing the accreditation, such as the Department of Veterans Affairs' (VA) program to accredit the laboratories of its medical centers. This type of program falls into a categorical gray area somewhere between laboratory accreditation and internal quality assurance -- since elements of both are present. In other programs, eligibility may be restricted to those holding membership in the accrediting organization. However, it is generally desirable that a program impose as few restrictions on eligibility as possible.

The terminology used to refer to a laboratory which satisfies the criteria established by a laboratory evaluation program also varies greatly. Some programs use the term "accredited," while others use the term "designated." Other programs refer to such a laboratory as "inspected," "accepted," or even "nationally recognized." Even the same term, such as "accredited," can be used by different programs to mean very different types of assessments. Different terms may also have different legal implications or may reflect differences between various agencies' or organizations' legal authority.

The term laboratory "designation" is increasingly being used rather than "accreditation" for schemes in which government agencies, public authorities, certification bodies, companies and others identify or "designate" one or more laboratories to perform specific types of testing for their own use or for the purpose of implementing regulations, standards, or specifications in which the organization or agency has an interest. Regulators, certifiers and others sometimes designate test laboratories through a contractual or similar relationship with the laboratory. The use of the term "designation" may be preferable, particularly if the depth of technical competence assessment is less than what is usually encompassed under accreditation. A designation type program is usually conducted by a body for its own use, rather than for use by others.

One should note the distinction between the acceptance body (the organization responsible for accepting and using the data produced by an accredited laboratory) and the accrediting body (the body that administers the laboratory accreditation program and issues the accreditation). For example, the accreditation body for a laboratory which tests radiation dosimeters may be NIST's National Voluntary Laboratory Accreditation Program (NVLAP), but the acceptance body -- the body which uses the data to regulate dosimeters -- is the Nuclear Regulatory Commission (NRC). This is an important difference that the reader should be aware of when reviewing information on laboratory accreditation programs.

Laboratories can be accredited in several ways. One way is for the laboratory to be accredited to test in an entire field of testing. The American Society for Testing and Materials (ASTM) Standard E 1224-94: Categorizing Fields of Capability for Laboratory Accreditation Purposes defines a field of testing as a "broad sphere of science, engineering, or technology used to describe a general area of testing for classification purposes." ASTM E 1224-94 lists these fields as acoustic and vibration testing, biological testing, chemical testing, electrical testing, ionizing radiation, mechanical testing, metrology,

non-destructive testing, optics and photometry, and thermal testing.

A laboratory can also be accredited in a scientific discipline, such as biochemistry, or for a specific technology (such as gene splicing), or in relation to specific products, such as blood product testing or concrete sample testing. Accreditation can also be limited to the conduct of specified test methods.

In general, the broader the scope of approval, the more difficult and time consuming it is for the accrediting body to thoroughly assess the laboratory's ability to perform all test methods and to test all sizes and types of products within that scope. However, the narrower the scope of accreditation, the more likely it is that a laboratory performing a broad range of testing will have to obtain multiple accreditations.

There are a number of standards or guidelines on how to operate a laboratory accreditation program, such as ISO/IEC Guide 58: 1992 - Calibration and testing laboratory accreditation systems - General requirements for operation and recognition. In the United States, a number of organizations have also attempted to address the issue of judging technical competence of laboratory accreditation bodies through standards documents, such as the American Society for Testing and Materials (ASTM) E 994-1995, "Guide Calibration and Testing Laboratory Accreditation Systems: General Requirements for Operation and Recognition."

Not all laboratory accreditation programs are equally thorough and not all laboratory accreditation bodies are equally competent. The users of accreditation, accredited laboratories, and the test data produced by accredited laboratories should be aware of the following factors when evaluating a laboratory's accreditation or an accreditation process. These factors may affect the ability of the accreditation process to provide assurance that an accredited laboratory is capable of producing accurate and precise test data that is well accepted by all users.

LABORATORY ACCREDITATION EVALUATION CRITERIA

1. Accreditation Body/Independence (No Conflict-of- Interest)

The accreditation body should be a legal entity organized in a manner which permits the satisfactory performance of all required functions. If the process is to be credible, the accrediting body and its assessors should be free from any outside influence which might bias the results of the accreditation process. No relationship should exist between the laboratory and the accreditation body or its assessors which might influence the objectivity and outcome of the accreditation process.

2. Accreditation Body Procedures

The laboratory accreditation body should have published procedures on its operations, including procedures for granting, maintaining, modifying, suspending and withdrawing accreditation; and for maintaining the confidentiality of proprietary information. In general, participation in the accreditation process should not be conditional upon a laboratory's membership in any association or organization. Unjustified fees, financial requirements, or other conditions for application, which restrict participation and are not relevant to the competence of the laboratory, should also be avoided.

3. Financial Stability

The body should have sufficient resources to satisfactorily perform all required functions, and to adequately indemnify itself against financial liabilities/penalties resulting from its operations. The body should have sufficient financial resources to allow it to refuse accreditation should that prove necessary.

4. Staff Qualifications Requirements

The accreditation body should be able to demonstrate that its personnel are qualified. Assessors should have the knowledge and experience necessary to perform all required assessments in the accreditation process, including detailed knowledge of the assessment criteria and the standards, test methods, and equipment involved. Assessors should also be able to effectively communicate the results of the assessment. Each staff member's training should be kept current and documented. Financial benefit accruing to the accreditation body from assessor training should be avoided to discourage the training of marginally qualified assessors.

5. Adequate Quality System

The body should have a quality system appropriate to the type, range and number of accreditations performed. The presence of an effective quality system in the accreditation scheme is important for maintaining confidence in the process. The system should be reviewed periodically by management and revised as needed to ensure continued acceptable performance. The quality system should be suitably documented in a comprehensive, up-to-date quality manual, which is readily available for consultation by staff. The body should conduct and document the results of internal audits as necessary to evaluate compliance with the requirements in its quality documentation and to assess the efficiency and effectiveness of the system's operation.

6. The Application Process

The accreditation body's assessment process should include a detailed application process. The application should include: the scope of accreditation being sought; an agreement by the applicant to comply with all requirements and fees imposed by the accreditation body; information on the laboratory, its staff and facilities; as well as the laboratory's quality manual and other quality documentation as appropriate.

7. Standards/Test Methods Used

The choice of standards and test methods used in a laboratory accreditation scheme has a significant impact on the validity of the scheme and the accuracy, representativeness, and reproducibility of the results. Accreditation programs should only accept and use test methods that are capable of producing accurate and precise test results.

8. The Assessment Process

The accrediting body should have a means of ensuring fair and equitable assessor selection and assignment. The accreditation body should appoint only competent and impartial assessors to conduct the assessment and provide them with any instructions and/or procedures needed. The assessment process should include: an initial meeting between the assessment team and the applicant to discuss the assessment; an evaluation of all areas of the laboratory required to ensure compliance with the accreditation requirements; documentation of the evaluation in a written report; and a follow up meeting between the team and the applicant to discuss the report and results of the assessment.

9. Adequate Control over Subcontractors

The accreditation body should have a system to ensure that any work related to the accreditation performed by a subcontractor is of acceptable quality. The nature of the system should be appropriate for the type and amount of work performed by the subcontractor(s). The body should also be able to provide interested parties with adequate evidence of the competence of all subcontractors.

10. Accreditation Decision

The accreditation body should make a decision on accreditation based on all evidence collected during the accreditation process. Responsibility for accreditation decisions should not be delegated by the accreditation body to another party.

However, in some cases, it may be useful for the accreditation body to accept as equivalent to its own accreditation, an accreditation issued by another accreditation body(s). This should only be done after the accreditation body has thoroughly reviewed the requirements, procedures, and process used by the other accreditation body(s), and the two (or more) bodies have established a formal agreement listing the terms and conditions under which such recognition will occur.

11. Recordkeeping Requirements

The accrediting body should maintain adequate records on the data collected, findings and reports related to all assessments performed; the qualifications, training and experience of each assessor; the assessment procedures used; and the laboratories accredited. All documentation regarding a laboratory should be restricted to persons or organizations considered by the accreditation body as having a legal right to such records.

12. Surveillance System

The accreditation body should have a documented system in place for laboratory surveillance and periodic reassessment (including participation by the laboratory in a proficiency testing program) to ensure that accredited laboratories continue to comply with all accreditation requirements. The laboratory should be required to assist the body as necessary in gauging continued compliance. The accrediting body should also have procedures which permit the swift withdrawal of accredited status from laboratories which fail to comply with the terms and requirements for accreditation, as well as procedures for notifying other interested parties of the change in accreditation status. The body should also require that any significant changes in an accredited laboratory's operations be promptly reported. Based on such information, the body should then decide in a timely manner whether accreditation should be continued or terminated.

13. Control of the Accreditation Document/Logo

The accrediting body should have mechanisms in place which minimize the potential for fraud or deception by a laboratory regarding its accreditation status or the meaning of that status. The body should have requirements regarding the use of its mark and the accreditation document, as well as a system in place to enforce those requirements. The body should take appropriate action against an accredited laboratory which misuses or misrepresents its mark or its accreditation or fails to continue to meet the requirements for accreditation. The body should also take action against unaccredited laboratories which misrepresent their accreditation status.

14. Appeals Procedure

Disagreements may occasionally arise among parties involved in an accreditation program. The program should include an impartial appeals mechanism to handle disagreements that cannot otherwise be resolved. Procedural requirements for appeals should be documented with minimal limitations on their timing and on who may file.

15. Accredited laboratory listing

An accreditation body should periodically publish a listing of the accredited laboratories, the scope of accreditation, and any limitations/restrictions on the availability of the testing laboratories' services.

OTHER CONSIDERATIONS

There are other considerations for evaluating the competence and usefulness of an accreditation program, these include:

1. Recognition of Laboratory Accreditation Program

Widespread recognition or acceptance of a laboratory accreditation program can reduce the need for extensive retesting of a product. Programs that are well accepted are usually more cost effective since the need for multiple and duplicative assessments is reduced. Lack of recognition can result in the need for a laboratory to obtain multiple accreditations of the same (or essentially the same) testing services, sometimes based on all or most of the same accreditation criteria.

In some cases, lack of recognition can result from inadequate knowledge or information on the program, rather than problems in the program itself. Based on information collected by Mr. Charles Hyer, the author of a number of NIST publications in this area, laboratories desiring to be accredited/designated nationwide to conduct electrical safety- related testing of construction materials, for example, have to gain the acceptance of at least 43 states, over 100 local jurisdictions, the International Conference of Building Officials (ICBO), the Building Officials and Code Administrators (BOCA), the Southern Building Code Congress International (SBCCI), a number of federal agencies, as well as of several large corporations.

Recently Congress recognized the importance of problems arising from redundant requirements in laboratory accreditation and other conformity assessment areas in its passage of the Technology Transfer and Advancement Act of 1996. Section 12 of the Act requires that federal agencies coordinate their conformity assessment activities with each other and with those of state government agencies and the private sector with the goal of "eliminating unnecessary duplication and complexity in the development and promulgation of conformity assessment requirements and measures." As a result, an effort is underway in the United States to upgrade the quality of and to facilitate mutual recognition among accreditation programs, known as the National Council for Laboratory Accreditation (NACLA). This joint government/industry program is composed of those in government and industry who actively support development of a system for recognizing the competence of testing and calibration laboratories -- both nationally and internationally. The work of this Council began in early 1997. Its efforts may help to decrease requirements for redundant accreditations and facilitate mutual recognition in the U.S. laboratory accreditation system.

In addition, the National Environmental Laboratory Accreditation Conference (NELAC), sponsored by the U.S. Environmental Protection Agency (EPA), concerned state agencies, and others involved in environmental testing, is working to promote mutual acceptance of environmental test data among states.

2. Program Scope

The scope of a laboratory accreditation program can impact the program's usefulness. Laboratory accreditation programs which are very narrow in scope -- designating or accrediting laboratories to test only a narrowly defined range of products within the agency's or organization's area of responsibility -- are usually of interest to only a few organizations. A laboratory which offers a broad range of testing services may need multiple accreditations which can increase the lab's cost of doing business and may restrict entrance into additional testing areas. On the other hand, accreditation programs which accredit laboratories to conduct too broad a scope of testing may not be able to adequately assess the competence of the laboratory for all test methods or testing areas within the scope, reducing the value of the accreditation.

3. Accreditation Costs

Accreditation costs are of concern to all laboratories, particularly smaller laboratories or laboratories that offer a broad range of testing services. Such costs must be balanced against the amount of new testing work likely to result from accreditation.

Multiple accreditations can be required to perform testing for different countries, or even for more than one state and/or local jurisdiction. Multiple accreditations may also be required for different products or testing fields -- all of which can considerably increase total accreditation costs. These costs are generally passed on to the users of the testing services.

THE U.S. LABORATORY ACCREDITATION SYSTEM

The U.S. laboratory accreditation system is different from that of most foreign countries. The majority of foreign accreditation bodies are public organizations or organizations with some direct government involvement. There is also a growing tendency in foreign countries to run national laboratory accreditation schemes in a coordinated fashion. In the United States, laboratory accreditation schemes are operated by all levels of government and by the private sector. No centralized coordinating body exists, though some coordination does take place between specific programs on their own initiative or through the intervention of other interested bodies, such as government agencies, trade associations or professional societies. This lack of coordination may be reduced as NACLA's and NELAC's work progresses.

The different types of U.S. programs are briefly described below.

1. Federal Government Laboratory Accreditation Programs

Requirements for laboratory accreditation/designation programs within the federal government vary greatly by program. While some programs, such the National Voluntary Laboratory Accreditation Program (NVLAP), are quite comprehensive; others involve only minimal review of a laboratory's qualifications. The requirements and scope of each accreditation program have generally been tailored to meet specific agency needs. With the exception of NVLAP, which was established to provide laboratory accreditation services, regulation or procurement (rather than laboratory accreditation/designation) is usually the agency's primary goal. In some cases, the accredited or designated laboratories provide only an initial product screening, with federal laboratories maintaining final responsibility for producing the test data used in enforcing regulations, such as the Mine Safety and Health Administration's (MSHA) programs for conformity assessment of mining products and equipment.

Eligibility requirements for accreditation also vary among programs. Some programs restrict applications to state laboratories only, such as NIST's program for accrediting state weights and measures laboratories. As noted above, the VA program accredits only the laboratories of its medical centers. Some agencies, such as DOD's Defense Logistics Agency (DLA), designate or accredit laboratories found to be capable of performing specific tests on products procured by that agency. Such programs are open only to potential government suppliers.

Terminology also varies by program. A 1989 GAO report noted the use of 10 different terms for accreditation with at least 18 different meanings by the 20 programs it reviewed. The same term, such as "accredited," used by different programs can mean very different types of assessments and assessment procedures. Different terms may also have different legal implications or may reflect differences between various agencies' legal authority to conduct specific programs (13).

2. State/Local Government Laboratory Accreditation Programs

States administer many laboratory accreditation programs for a variety of reasons. In most cases, state and local government agencies accredit laboratories to test regulated products and require that such products be inspected and/or tested by an approved laboratory. An example is the regulation of building and electrical products at the state or local government level by requiring that the products be tested/inspected and bear the mark of a recognized or approved testing laboratory. In other cases, the state or local government programs, like some federal programs, accredit or designate laboratories that provide a surveillance/screening service for testing product compliance. In such cases, state/local government laboratories usually maintain responsibility for producing the test data used in enforcement situations.

States also evaluate laboratories for other purposes, such as to assist them in enforcing federal regulations. For example, states have primary enforcement responsibility for the national requirements for ensuring the quality of public water systems. In carrying out this obligation, states accredit local laboratories to test drinking water. State and local authorities also accredit or designate laboratories to test products prior to their procurement by a state agency.

As with federal programs, state and local requirements for laboratory accreditation/designation vary greatly by program. Some programs are quite comprehensive, while others involve only minimal review of a laboratory capabilities. The terminology used for accreditation also varies extensively.

3. Private Sector Laboratory Accreditation Programs

Private sector accrediting bodies also administer laboratory accreditation programs for a variety of reasons. These reasons range from assisting laboratories in defending their competence in professional malpractice matters to assisting an industry to avoid government regulation by undertaking self-policing efforts. Many private sector laboratory accreditation programs are operated as an integral part of a private sector certification program. In addition, accreditation programs can be established to assist government agencies in enforcing regulations, such as in the testing of building and construction products. The criteria and terminology used in private sector laboratory accreditation/designation schemes again vary greatly.

Private sector programs also differ in size and scope. The American Association of Blood Banks (AABB) operates an accreditation program for blood bank and transfusion services which is recognized by many state authorities. The College of American Pathologists (CAP) program accredits hospitals and independent medical laboratories, and their program is used by the Department of Veterans Affairs (VA) to ensure the competence of VA medical facilities. The American Association for Laboratory Accreditation (A2LA) operates an accreditation program which accredits laboratories in a number of testing fields.

At the international level, the IEC Quality Assessment System for Electronic Components (IECQ) accredits testing facilities capable of demonstrating compliance of electronic components to prevailing requirements and standards, which are based on IEC standards. This accreditation program is part of a reciprocal worldwide certification system, in which certificates of conformity issued by an approved laboratory in one participating country are accepted by all other participating countries.

The IEC also operates the IEC System for Conformity Testing for Safety of Electrical Equipment (IECEE). The IECEE's objective is the reciprocal recognition of test results (not certification marks or certificates of conformity) among the participating countries. Equipment is tested by approved laboratories against prevailing specifications and requirements, including safety, based on IEC standards. Any electrical products for which there are appropriate IEC standards can be included within the program.

CERTIFICATION (14)

While a close interrelationship exists among standardization, testing, laboratory and certification, these areas are distinct. Certification is the process of providing assurance that a product conforms to a standard or specification or that a person is competent to perform a certain task. The use of certification in the marketplace dates back to the earliest times. An inscribed stone of the 4th Century BC, found in Eleusis, Greece in 1893, cited a decree regarding the manufacture of bronze fittings for the erection of the columns of a new stoa (portico) which became the Philonion Stoa. The bronze specified was copper/tin in a ratio of 11 parts copper to 1 part tin. Professor Varoufakis in his book, Materials Testing in Classical Greece, Technical Specifications of the 4th Century BC, argued that the discovery of such a specification implies the existence of some type of check or verification. Professor Varoufakis tested the possibility that this check could have been conducted based on color differences between polished specimens of bronze differing in tin content. He noted a distinct color difference between two specimens when the tin content differed by only 2%. A similar process for identifying and differentiating gold alloys was already in use in ancient Greece. These may have been two of the earliest forms of certification -- metals of unknown content certified as to their composition on the basis of inspections for color (15).

From 1890 to 1900, when the use of gas for domestic cooking and heating began to gain widespread popularity, many new gas appliances were developed and sold. Early records indicate that some thought was given to establishing a gas appliance testing laboratory as early as 1903. The Baltimore investigation, conducted by the National Bureau of Standards in 1923, analyzed accidents resulting from the use of domestic gas appliances and flexible gas tubing. The City of Los Angeles started a similar laboratory and field study in 1924. Both investigations pointed to the need for a certification program for gas appliances. In 1925, the American Gas Association established a laboratory to test and certify gas appliances. The first certified gas heaters and ranges appeared on the market in 1926.

Just 50 years ago, U.S. drug manufacturers could produce and sell drugs without testing them on either animals or humans and without any kind of governmental approval. Governmental action could only be taken against drugs which were misbranded or adulterated. In 1937, physicians in Tulsa, Oklahoma reported to the American Medical Association (AMA) the deaths of six patients from a liquified version of the then wonder drug sulfa. Ultimately this drug killed 107 people, mostly children, before doctors realized what was happening and the drug was recalled. This tragedy led to the enactment of the 1938 Food, Drug and Cosmetic Act, which requires that drugs be tested and approved by the Food and Drug Administration before marketing. From such early beginnings, government and private sector certification programs have grown both in size and complexity.

Some (but not all) certification programs mandate that accredited laboratories conduct any required testing, but (as noted above) there are laboratory accreditation schemes that are not associated with a certification program. Certification and laboratory accreditation programs both use standards, but not all standards are intended for these uses. Because standards, testing, laboratory accreditation, and certification are linked; strengths as well as deficiencies in any one area can have significant consequences for the other areas. For example, improvements in test method standards can significantly increase the capability of a laboratory to produce valid test data, which may then be used in a certification program.

Product certification schemes range from the simple to the complex. Many private organizations, as well as federal and state agencies in the United States, certify products ranging from electrical cords to meat products. In addition, many certification programs are operated at local government (city, township, county, etc.) levels. Consumers see evidence of the extensiveness of certification-related activities when they note the Underwriters Laboratories (UL) mark on such products as electric coffee pots and fire extinguishers, the U.S. Department of Agriculture (USDA) mark on poultry and other agricultural products, and the International Wool Secretariat's Woolmark or Woolmarkblend on wool and wool blend textile goods - only a few of the many certification marks which may appear on consumer products.

TYPES OF CERTIFICATION

1. First Party Certification or self-certification is the process by which a manufacturer or supplier declares that the product meets one or more standards based on: (1) the manufacturer's confidence in the quality control system, or (2) the results of testing or inspection the manufacturer undertakes or authorizes others to undertake on his/her behalf. This process is also known as a manufacturer's or supplier's declaration of conformity. The manufacturer's capability, integrity, and reputation determine the degree of confidence that can be placed in this type of certification.

In the United States, the criteria and procedures recommended for a self-certification program are established in American National Standards Institute (ANSI) (16) Standard Z-34.2-1987, "American National Standard for Certification - Self-Certification by Producer or Supplier." At the international level, ISO/IEC Guide 22: 1996 - "General Criteria for Supplier's Declaration of Conformity," also contains a list of such criteria and recommended procedures. Such procedures frequently include the requirement that the self-certification or manufacturer's declaration be based on conformance with all requirements of a standard unless full disclosure is made of any limitations of the certification. Usually there are additional requirements for the adequate use and maintenance of test equipment, an effective quality control system, fully-trained staff, written certification procedures, and adequate record keeping.

One of the most familiar certification programs in the United States involves the identification of the weight of motor oils by the manufacturer of conformance to the Society of Automotive Engineers (SAE) standards. Consumers will recognize such weight designations as SAE 10W-40W or 10W-30W on the motor oils that they buy. These SAE designations are placed on the motor oils by the manufacturer based on his/her own testing and quality control mechanisms. This is primarily a self-certification program, though SAE does audit manufacturers' self- certifications to ensure that SAE standards are not being misused.

2. Second Party Certification is also common in the United States. In this case, it is usually the buyer who requires and certifies that the products he/she wishes to purchase from suppliers meet one or more standards. The nature of these programs varies greatly depending on the type of product being purchased and the needs of the buyer. These certifications are generally only available to and mandatory for those companies wishing to become suppliers to that buyer.

3. Third party certification is a type of certification in which the producer's claim of conformity is validated by a technically and otherwise competent third party (a body not controlled by or under the influence of the producer or buyer. The sponsor of the third party program (the certifier) may be responsible for collecting the required data, generating test results or conducting inspections, in addition to reviewing the results of such activities and making a final determination on the product's conformance or lack of conformance. The certifier may also delegate all or part of the data collection and review activities to another party or parties. The degree of confidence that can be placed in third party certification programs varies greatly depending on (1) the number and types of testing/inspection methods used within the program to ensure product conformance, (2) the adequacy of the manufacturer's quality control system, and (3) the competence of the body which conducts the testing and/or inspection and evaluates the test results.

Recommended criteria and procedures for third party certification programs in the United States are established in ANSI Standard Z-34.1-1993, "American National Standards for Certification - Third-Party Certification Programs for Products, Processes, and Services." However, the recently issued ISO/IEC Guide 65: 1996 "General requirements for bodies operating product certification systems," developed by the ISO CASCO Committee (17) is likely to gain considerable acceptance nationally and internationally in the future.

THIRD PARTY CERTIFICATION PROGRAMS IN THE UNITED STATES

1. Private Sector Certification Programs

U.S. private sector third party certification programs, described below, are operated by many different types of organizations including:

o Professional or technical societies, such as the American Dental Association (ADA) or the American Society of Sanitary Engineering (ASSE);

o Trade associations, such as the Association of Home Appliance Manufacturers (AHAM) or the International Safe Transit Association (ISTA);

o Independent testing/inspection organizations, such as Underwriters Laboratories (UL), Factory Mutual Research Corp. (FMRC), ETL Testing Laboratories, and the MET Electrical Testing Company;

o Organizations oriented toward consumers, industrial buyers, or users of the product, such as Good Housekeeping magazine, which offers a consumer protection program for products it approves;

o Organizations composed of producers, testing laboratories, and others concerned with the well- being of an industrial group and/or its customers, such as the Solar Rating and Certification Corp. (SRCC), a nonprofit organization founded in 1980 to test and certify solar thermal panels and systems;

o Organizations comprised of government officials involved in the regulation of an industry, such as the Building Officials and Code Administrators International (BOCA), the International Conference of Building Officials (ICBO), the Southern Building Code Congress International (SBCCI), and the International Association of Plumbing and Mechanical Officials (IAPMO); and

o Other miscellaneous organizations, such as a ship classification society or nonprofit organizations representing the kosher food consumer.

One of the oldest certification programs has been operated since 1870 by the American Bureau of Shipping (ABS), a ship classification society. It is concerned with the design, construction and periodic survey of merchant ships and other marine structures. ABS certifications cover over 12,400 vessels. Other certified products include: cargo containers, mobile offshore drilling and production units, fixed offshore structures, work boats, cranes, yachts, cargo handling engines, and related machinery and materials.

Another well established industry trade association certification program is that of the American Gas Association (AGA), operated since 1925, and which has joined with its Canadian counterpart to form International Approval Services (IAS). Over 700 U.S. and Canadian manufacturers participate in IAS's program for gas appliances and accessories. IAS currently certifies a wide range of products including gas appliances and related accessories and electric and oil appliances. Products are tested in IAS facilities and requirements for certification include: a review of the construction and performance information for the product, factory and quality control inspections and annual product testing/inspections. Additional documentation may also be required. IAS publishes an annual directory of certified appliances and accessories.

More recently established appliance certification programs for room air conditioners, refrigerator-freezers, dehumidifiers, and humidifiers are operated by the Association of Home Appliance Manufacturers (AHAM). The Air-Conditioning and Refrigeration Institute (ARI) certifies appliances ranging from air conditioning equipment to water coolers and solar collectors.

A program familiar to many consumers is that of the American Dental Association (ADA), a professional society which operates a program for certifying dental materials, instruments, and equipment, including toothpaste and tooth brushes. The ADA certification program uses ADA standards and requires the submittal of a detailed application describing the product's specifications. ADA also requires self-certification by the manufacturer that the product meets the standard. Product samples obtained on the open market are tested by ADA. Upon approval, ADA authorizes the use of the ADA Seal of Certification and publishes approved product lists.

Most consumers take for granted the familiar "UL" mark on a variety of products from electrical appliances to fire extinguishers. The Underwriters Laboratories (UL), an independent testing laboratory founded in 1894, is not only a major standards writer, but also tests products and materials with respect to potential hazards to life or property, listing (18) those items which appear to pose no significant hazards. The Factory Mutual Research Corporation (FM), is another "product listing" organization, similar to UL. It is a non- profit testing and research organization whose purpose is to minimize industrial property damage through a number of activities, including product approval. FM issues lists of approved products which can vary from industrial trucks to building materials.

Gardeners might recognize the AARS (All-America Rose Selections) mark on the rose bushes they purchase. Over 40% of the roses sold in the United States bear the AARS symbol indicating that the rose bush meets certain quality requirements. The AAS (All-America Selections) mark is used in connection with seed for flowers and vegetables which meet similar requirements.

The work of other major certification organizations, although equally vital, may be less well known outside the testing/inspection community. The Maple Flooring Manufacturers Association certifies floor finishes and sealants. The National Cotton Council of America, a trade association, certifies wrapping and banding materials for cotton bales. MEA Certifications certifies carpet under the Department of Housing and Urban Development (HUD) carpet certification program developed by the Carpet and Rug Institute.

The American Society of Mechanical Engineers (ASME) accredits manufacturers of boilers, water heaters, pressure vessels and nuclear components, which, in turn, certify that their products meet ASME Code requirements through the completion of a Code data report. The National Board of Boiler and Pressure Vessel Inspectors (NBBI), an organization composed of chief boiler inspectors in the United States and Canada, registers manufacturers Code data reports and commissions the inspectors who provide independent third party inspection of products manufactured under the Code. The value of certifying these products to ASME Codes is recognized internationally.

Several building code organizations, such as the Building Officials and Code Administrators International (BOCA), the International Conference of Building Officials (ICBO), and the Southern Building Code Congress International (SBCCI), engage in certification-related activities. Composed of building, construction, zoning, and inspection officials, these organizations have developed model building codes which have been adopted by hundreds of state and local governments. The organizations evaluate products and materials and approve them for conformance with the requirements established in their building codes.

Over one hundred seventy eight private sector organizations in the United States certify or list different types of products (19).

Many others certify services. The broad range of organizations involved in certification reflects the impact of certification on a vast spectrum of interests and disciplines.

2. Federal Government Certification Programs (20)

Federal government certification programs can be classified into several general categories:

o Programs to certify products directly affecting the health or safety of the user or the general public.

o Programs which test products in order to avoid the necessity for retesting at local levels or prior to each procurement.

o Programs to provide a uniform basis for trade by assessing the quality and condition of products offered for sale.

Examples of the first type of certification program include the evaluation and approval by the Food and Drug Administration (FDA), U.S. Department of Health and Human Services, of new animal and human drugs, medical devices, biologicals, and other products; the certification by the Federal Aviation Administration (FAA), U.S. Department of Transportation, of airplanes, major airplane components, and related services; and the certification by the Mine Safety and Health Administration (MSHA), Department of Labor, of electrical equipment used in mines.

An example of the second type of program is the Department of Defense's (DoD) Qualified Products Listing (QPL) Program for parts, materials and components used in military systems. This program reduces retesting prior to each government purchase by testing products and placing those approved on appropriate QPL's. An extension of this concept also underlies the DoD Qualified Manufacturing Lists (QML's) Program, in which a manufacturer's process controls and manufacturing capabilities are evaluated and approved for an entire range of products.

An example of the third type of program is the U.S. Department of Agriculture's (USDA) voluntary program to grade and certify meat and meat products (on a voluntary basis) using uniform quality grading standards for the buying and selling of such products. The USDA also certifies dairy products, fresh and processed fruits, vegetables, nuts and related products. The National Oceanic and Atmospheric Administration (NOAA), U.S. Department of Commerce, likewise inspects and grades processed fish and shellfish at a seafood processor's request.

3. State Certification Programs

States administer many certification programs which cover a diversity of products for a variety of reasons. In some cases, states inspect or test products under authority delegated by the federal government. For example, many states inspect meat and meat products, certifying those that meet standards established by the USDA. These states then authorize the use of the appropriate USDA marks. Many states also inspect and issue certificates of conformity for manufactured homes under authority delegated by the U.S. Department of Housing and Urban Development (HUD). States may also impose additional state requirements and simultaneously check for conformity to these and federal requirements.

States also regulate products under their own authority for health and safety reasons, including amusement rides and insulation, depending on each state's perception of the health and safety impact of such products on its population. Products may be inspected and/or tested directly by the states themselves, or indirectly through a requirement that such products be inspected and/or tested and certified by an approved body, such as a nationally recognized laboratory. An example of the latter is the regulation of electrical building products by imposing a state requirement that they be tested/inspected and bear the mark of a "nationally recognized testing laboratory." The term "nationally recognized laboratory" is currently defined by each state and/or municipality.

States regulate products of direct or indirect economic importance. Florida and California, for example, inspect products important to their citrus fruit industry. Nebraska, with a considerable agricultural industry, regulates tractors through a testing program at the University of Nebraska and issues certificates of conformity for approved models. California, with its air pollution problem, stringently regulates auto emissions equipment.

States also inspect/test/certify materials, products, systems and services they procure, such as materials for the construction of state roads and bridges. In yet other cases, the states establish standards and leave enforcement (testing, inspection, etc.) to local authorities. This is sometimes true for building and construction materials.

States operate certification (also called licensing programs) to regulate the services offered by professionals in fields ranging from medicine to insurance.

CHOICE OF STANDARDS

As noted before, the standards chosen for use in certification (or other types of conformity assessment) programs should specify accurate and reproducible methods of testing for the specified characteristics if the certification program is to have any validity. The standards should also be clearly written and be capable of being uniformly interpreted. ISO/IEC Guide 7: 1994 has stated that standards suitable for product certification "....should be written in such a way that they facilitate and do not retard the development of technology." However, there are many other elements to consider when choosing standards for a certification program.

One of the most obvious considerations in selecting standards is the determination of what characteristics need to be assessed to achieve the objectives of the program. The standards should specify all those and only those characteristics and requirements that are necessary to define the properties of the product or its performance; if this is not possible, then reference should be made to other relevant standards. Unless the chosen standards do, in fact, adequately specify all product characteristics that need to be assessed, it will not be possible to fully achieve the program's objectives and the program's value may be questionable.

Certification schemes are normally established to indicate that a product meets one or more sets of acceptable or minimum requirements. But what is acceptable? What may be acceptable to one buyer may not be acceptable to another. Certification programs may choose standards that result in certified products that satisfy a particular group or organization, such as the government, or the program may choose standards expected to satisfy a larger group of buyers or users. The choice of standards, therefore, may depend on the needs of the parties likely to use the certification.

Standards containing only one set of requirements and which define only one level of safety or quality are known as "pass- fail" standards. Either the product meets the minimum requirements (it passes and is certified) or it does not (it fails). Little or no attempt is made to convey information about the relative safety or quality of one certified product compared to other certified products (such as is conveyed through a grading system). Unless other methods are used to convey such information to buyers/users, a supplier may be less motivated to produce or supply products which exceed the minimum requirements specified in the standard.

Certifiers may therefore wish to provide information on the quality or safety of one certified product relative to other certified products. This may be accomplished by selecting standards which define several levels of safety or quality and by including a grading scheme in the certification program. Information can be conveyed by stating the expected life of the product, such as the expected number of miles that tires are supposed to travel, or the versatility of the product, such as portable fire extinguishers rated for several specified types of fires, or the efficiency of the product, such as the energy efficiency rating for appliances. Buyers, assured of at least a minimum level of safety and/or quality, can then choose among certified products based on any additional level of quality or safety that they desire, other characteristics or features, and/or price.

A choice must also be made as to whether the establishment of conformity should be based on an assessment of a product's performance or its design. As noted above, performance standards, tend to be difficult to develop and difficult to use in assessing compliance. Problems may also arise in establishing reliable test methods for assessing conformity, thus potentially increasing certification costs.

The choice of standards for a certification program has a significant impact on the validity of the program, the value of the information conveyed by the certification, and the program's cost.

CERTIFICATION METHODOLOGY

Third party certification programs differ greatly from one another, and the degree of confidence in the resultant certification depends on the type of program and its comprehensiveness (the number and types of test/inspection methods used within the program to assess conformity).

The methods used in third party certification programs can be classified as follows:

1. Type-testing/Initial Inspection - This method attempts to determine if the supplier's design for the product can produce a product that conforms to a particular standard. Products from a preproduction run are inspected and/or tested or services or service providers are inspected/tested before being allowed to offer services to the public, but this method provides no information on whether the products or services supplied subsequent to the initial testing and inspection also consistently meet the specification.

2. Surveillance of the Supplier's Process - Assessment of a supplier's materials, production and control processes can, at relatively low cost, provide assurance that the supplier's quality control procedures are adequate. In some cases, a supplier's quality system must have been assessed and approved (registered) by a third party as a pre-requirement for application.

3. Audit-Testing - In this procedure, samples of products, services or individuals are selected at random from the marketplace or population and tested. Extensive testing is usually required to provide adequate assurance that group of objects/populations from which the samples are taken meet the referenced standard.

4. Field Investigations - Alleged failures of products during actual use are investigated to determine the cause of failure and to suggest appropriate corrective action.

5. Batch-testing - A sample is selected from a population or batch and tested for conformance to the standard. If the sampling procedure and the sample size are adequate, batch-testing may be used to estimate, with a specified degree of confidence, that all objects in that batch or population conform to the standard. It does not, however, ensure that an untested product or individual in the batch/population will meet the standard, nor does it furnish information on the quality of products produced in earlier or subsequent batches, or of individual in other populations. Batch testing is used in many certification programs for building products.

6. 100 Percent Testing - In this method, each product/individual is tested to determine compliance with the designated standard. If the testing procedures are adequate, the procedure provides the highest possible level of assurance that the product/individual conforms to a particular standard. It is also usually the most expensive method and can be applied only where the test has no adverse effect on the product/individual.

Many certification programs rely on two or more of these methods for their approval process. The choice of methods depends on the needs of both the buyer and the seller and on the nature of the product or service provided. The chosen methods can greatly affect both the cost of the program and the level of confidence that can be ascribed to it.

CERTIFICATES OR MARKS OF CONFORMITY (CERTIFICATION MARKS)

ISO/IEC Guide 2 defines a certification mark (a mark of conformity for certification) as a: "protected mark, applied or issued under the rules of a certification system, indicating that confidence is provided that the relevant product, process or service is in conformity with a specific standard or other normative document." The marks used in U.S. certification programs are usually (but not always) registered with the U.S. Patent and Trademark Office (PTO) under the Trade Marks Act of 1946 (Lanham Act) as amended (1994). Some of these marks are registered with PTO as certification marks, however, others may be registered as trade or collective marks (21). The requirements established and enforced by PTO for obtaining and using these different types of marks vary by type of mark.

ISO/IEC Guide 2 defines a certificate of conformity, as a: "document issued under the rules of a certification system, providing confidence that a duly identified product, process, or service is in conformity with a specific standard or other normative document."

The marks or accompanying information should indicate the identity of the certification body (and any other testing body if applicable) and any relationship that the body(s) may have to the manufacturer. In addition, the certificate of conformity should contain information on: (1) the lot, batch or other production information to allow traceability to the production source and time of production; (2) the date when the certificate was issued; and (3) the officer of the company responsible for its issuance. Labeling included with the product should identify the supplier, and contain information on the name, type or model number and all instructions necessary for the correct and safe use and maintenance.

Certification marks and certificates of conformity should be used to indicate that all essential characteristics of the product have been assessed. In cases where only one of several aspects of the product have been evaluated, such as flammability or electrical safety, this information should be conveyed in some manner to the buyer lest the mark mislead the buyer into placing more reliance on the certification than is justified. To the extent possible, the symbols used in connection with the certification mark should be capable of being interpreted without further definition.

CERTIFICATION PROGRAM EVALUATION CRITERIA

While certification programs benefit both the seller and buyer of a certified product, certification programs also have the potential for misleading users and for other abuses. Some of the more significant elements to consider when evaluating certification programs are the similar to those for evaluating laboratory accreditation programs including:

OTHER CONSIDERATIONS

There are also other considerations for evaluating the usefulness of a certification program, including:

Despite public perception and assumptions that certification implies legal responsibility for the quality and/or safety of the product certified by a third party, most certifiers are not willing to accept such responsibility. Responsibility for this liability generally rests with the supplier. Buyers and users of certification should understand these limitations. Buyers and users of a certification scheme should be adequately informed as to the purpose, scope, and technical limitations of the resultant certification so as not to be misled as to the value and degree of confidence that can be placed on the mark or certificate of conformity.

2. Compatibility with and Recognition by Other Certification Schemes.

When U.S. certification schemes for the same product differ significantly, this may result in the need for extensive retesting at considerable expense in terms of both time and money, such as having to have a product certified by multiple certifiers to meet various state requirements. The potential barriers to trade which can result from differences among U.S. and foreign national certification programs can also be considerable. Preference should be given to the use of certification programs which are compatible with other national or foreign national certification schemes while still meeting the objectives of the scheme.

In some cases, lack of recognition results from inadequacies in specific programs which need to be addressed by the responsible certifiers. However, lack of recognition can also result from inadequate knowledge or information on the program by certification users. It may also result from the vested interests of competing certifiers or others, rather than problems in the program itself. Widespread recognition or acceptance of competent certification programs should be encouraged to reduce the need for extensive retesting/inspection/certification of a product.

3. Appropriate Use of Certification in Conjunction with Quality System Requirements

As W. Edwards Deming (22) often pointed out, quality must be designed into the product and ensured through an effective and efficient manufacturing process. Inspection, testing, and certification will provide information on whether the desired end result has been achieved, but usually that information comes too late in the supplier's process to improve the quality of the inspected/tested products. Certification should

not be used as a substitute for a supplier having an effective quality system.

The costs involved in the certification process should be reasonable and in balance with the benefits likely to accrue to the supplier from having the product certified. Excessive costs in a certification process can be cause for serious concern for all suppliers (particularly for smaller suppliers), and can serve to restrict trade and competition. In most cases, certification costs are ultimately passed on to the buyers of the product.

U.S. CERTIFICATION PROGRAM ACCREDITATION SYSTEM

One of the criticisms of self-certification or manufacturer's declaration, namely that the manufacturer does not have a third party checking the competence and integrity of the results, fails to recognize that many third party certification programs also do not have anyone looking over their shoulder to ensure the quality of their programs. Like a laboratory, a certifier can self-declare or self-certify its compliance with the requirements, such as those listed in ISO Guide 65: 1996 "General requirements for bodies operating product certification systems." Users of certification results can also validate a certifier's compliance for themselves. Yet another method for ensuring the competence of certifiers is through an accreditation program conducted by an independent third party.

A third party (independent of the certifier and the purchasers or users of the certification) can accredit the competence of a certifier to conduct specific certification tasks for specific types of products. Like laboratory accreditation, certification program accreditation does not necessarily give any guarantees about the certification results obtained from the performance of specific certification procedures. Certification program accreditation means that the certifier is capable of performing specified certification procedures correctly, not that the certifier has competently certified all products in each and every instance. Again, like laboratory accreditation, certification program accreditation only provides assurance about a certifier's capability within the scope or areas for which accreditation was granted. It should also be noted that lack of accreditation does not necessarily imply that the certifier is not technically competent since not all certifiers want or require accreditation. In some cases, accreditation programs may not exist in the certifier's field of operation.

Meeting regulatory requirements is again probably the primary reason for a certifier to participate in an accreditation program. Congress, state and local governments can mandate that certifications be done by accredited certifiers. OSHA, for example, is responsible for the regulation of all electrical products used in the work place. Electrical products under OSHA's jurisdiction require approval and marking/listing by certifiers accredited by OSHA, known as Nationally Recognized Testing Laboratories (NRTL).

State and local government agencies may also impose such a requirement through regulations issued under their own legal authority. Certifiers may also have to be accredited to meet mandatory requirements imposed by foreign governments on products imported into their countries.

Like laboratories, certifiers may also participate in an accreditation program as an outside check of their internal quality control program, as proof of competence to higher level management within the organization, as a competitive advantage over other unaccredited programs, as a means of protection in liability proceedings, or as a means of establishing credibility with the public.

The reasons for and benefits of seeking accreditation are many. In general, however, certifiers (and laboratories) participate in accreditation programs in expectation of some type of economic return on the resources they invest in obtaining the accreditation, such as new or expanded business opportunities.

Again like laboratory accreditation programs, not all certifier accreditation programs are equally thorough and not all accrediting bodies are equally competent. Some factors that can be used to evaluate a certification program accreditation process were previously discussed under laboratory accreditation. These factors affect the ability of the certification program accreditation process to provide assurance that an accredited certifier is capable of competently conducting certain certification procedures for one or more products as defined in the scope of the accreditation.

The U.S. certification program accreditation system is less complex than that for laboratory accreditation, primarily because the U.S. has significantly fewer programs for accrediting certifiers than for accrediting laboratories. The two most prominent programs are the ones run by OSHA (which was mentioned above) and by ANSI.

OSHA's program covers electrical equipment/materials used in the work place. All electrical products used in the work place must be tested and listed or labelled by a certifier (known as a Nationally Recognized Testing Laboratory or NRTL) which is recognized/approved by OSHA. Participation in the OSHA program is voluntary for certifiers; however, if they wish to test such products, they must be recognized/approved by OSHA (23).

ANSI also conducts an accreditation program for certifiers which was established in the 1970s. ANSI's policies and procedures were revised in 1992 to be consistent with both relevant international standards and guides and with anticipated obligations under the WTO Agreement. ANSI accredits certifiers in a number of areas ranging from construction materials to drinking water additives to gas and electrical appliances and accessories.

MANAGEMENT SYSTEM ASSESSMENT/REGISTRATION

At the national and international level, interest has mushroomed in quality and environmental management systems. Having a quality management system helps a supplier ensure that he/she is able to produce products which consistently conform to a given set of standards or expectations. Product quality depends on many variables, such as the caliber of the components or materials used; type of equipment used in design, production, handling, installation, testing and shipping; the equipment calibration and maintenance procedures employed; the training and experience of production and supervisory personnel; the level of "workmanship;" and sometimes the environmental conditions (temperature, humidity, level of dust particles) in the area where the product is produced. The process, organizational structure, procedures, and resources that suppliers use to control these variables to produce a product of consistent quality which meets defined specifications is called a quality system (24). The standards for quality systems that are best known and that are being adopted globally are the ISO 9000 standards.

Environmental management systems (EMS) are designed to provide a comprehensive approach which can be used by businesses and other organizations to manage the impact of their activities on the environment. Environmental management system standards generally do not establish requirements for environmental compliance nor for specific levels of pollution prevention or performance (this being essentially the responsibility of the appropriate regulatory agency(s) and/or the organization's management).

However, management system standards do establish a framework which an organization can use to evaluate its performance against such levels. EMS standards, such as ISO 14001, generally stress identifying key environmental impacts (aspects) associated with the operations of an organization, setting targets and objectives for these aspects, and implementing the system via training and documentation of procedures. EMS standards, like ISO 14001, also require that the company implement a program to conduct internal audits of its EMS, to conduct periodic management reviews of the system's effectiveness, and to have a commitment to continual improvement of the EMS.

Recent publication of the first two international environmental management system standards (ISO 14001 and ISO 14004) is likely to increase awareness in the United States of the usefulness of management systems in overseeing and improving environmental performance.

THE ISO 9000 STANDARDS

The original ISO 9000 standards were a series of five international standards (ISO 9000, 9001, 9002, 9003, and 9004), developed by ISO Technical Committee (TC) 176 and published in 1987 to provide guidance on selection of an appropriate quality management program (system) for a supplier's operations. The Series is generic in scope covering both service and manufacturing industries. The 1994 edition of ISO 9001 contains requirements in twenty areas covering: management responsibility; quality system requirements; contract review; design control; document and data control; purchasing; requirements for customer-supplied product; product identification and traceability; process control; inspection and testing; control of inspection, measuring and test equipment; inspection and test status; control of nonconforming product; corrective and preventative action; handling, storage, packaging, preservation and delivery; control of quality records; internal quality audits; training; servicing; and the use of statistical techniques. ISO 9002 covers all elements except design control. ISO 9003 (which is not used to any extent in the United States) covers significantly fewer elements. The titles and content of the 1994 edition of the series are as listed in Appendix I.

The United States has been an active participant in ISO TC 176 since 1987, and the 1994 edition of the ISO 9000 Standard Series has been adopted in the United States as the ANSI/ASQC Q 9000 Standard Series. The ISO 9000 standards, which were initially intended to be advisory in nature and which were developed primarily for use in two- party contractual situations or for internal auditing, are currently being applied under a much broader range of conditions and circumstances.

The reader should also note that some national and regional standards bodies are developing supplemental guidance for the application of the ISO 9000 series to specific industries. For example, CEN and CENELEC are developing more specific requirements for the application of the ISO 9001 to the medical device industry; and the U.S. Food and Drug Administration (FDA) is revising its Good Manufacturing Practice (GMP) (25) regulations for medical devices to follow ISO 9001 with appropriate additional requirements.

THE ISO 14000 STANDARDS

In June 1992, the United Nations' Conference on Environmental Development was held in Brazil. At that Conference, ISO made a commitment to continue the work it had begun the prior year in the area of environmental management. The previous year, ISO had formed the Strategic Advisory Group on the Environment (SAGE) to develop an international consensus of what needed to be done in this area. The work of SAGE ultimately resulted in the formation of TC 207 in 1993 to develop standards in the field of environmental management tools and systems.

The United States has been an active participant in ISO TC 207 since 1993. U.S. participation in TC 207 and its subcommittees is jointly managed by the American Society of Testing and Materials (ASTM) and ASQC under the aegis of ANSI. The work of ISO TC 207 is divided into seven areas: (1) management systems; (2) audits; (3) labeling; (4) environmental performance evaluation; (5) life cycle assessment; (6) terms and definitions, and (7) environmental aspects in product standards.

Work is ongoing in all of these areas. In the management system area, ISO 14001 and 14004 were released in the fall of 1996, and have been the subject of much review and analysis in the United States. Some U.S. companies have already begun to seek conformance with the standards, with a smaller number seeking third party registration of that conformance. While it is still unclear as to the extent that these standards will ultimately gain acceptance with U.S. industry, there appears to be growing acceptance of these standards in Europe and elsewhere.

MANAGEMENT SYSTEM ASSESSORS

1. First party assessment or self-assessment is the process by which a manufacturer or supplier declares that the quality or environmental management system conforms to the requirements of ISO 9001, 9002 or 9003; ISO 14001; or some comparable standard. Such self-audits are usually major components of the system itself. Such self-audits can increase the confidence of management in its system and demonstrate to its personnel that the firm is committed to quality or environmental management. This process is also known as a manufacturer's declaration of conformance.

2. Second Party Assessment is also common in the United States. In this case, it is usually the buyer who requires and assesses the quality or environmental management systems of the suppliers from which the buyer wishes to purchase products. The nature of these programs varies greatly depending on the type of product being purchased and the needs and concerns of the buyer. These assessments are generally only available to and mandatory for those companies wishing to become suppliers to that buyer.

3. Third party Assessment (better known as management system registration), is a form of assessment in which the supplier's claim of conformity is validated by a technically competent body. "Third party" quality or environmental management system evaluations and registrations may be voluntary or mandatory and are conducted by persons or organizations independent of both the supplier and the buyer.

4. A new type of assessment has also been proposed within the United States and internationally -- the Supplier's Audit Confirmation (SAC) route -- which involved elements of both first party and third party evaluations. The SAC route would include a supplier's declaration of conformity to the appropriate ISO 9000 or ISO 14000 standard combined with a third party assessment of the maturity of the manufacturer's quality or environmental management system and the effectiveness of the internal audit program. The details of how this new type of assessment would operate are still under development.

MANAGEMENT SYSTEM REGISTRATION

Management system registration or approval (sometimes misnamed "management system certification" (26)) involves the assessment and periodic audit of the adequacy of a supplier's management system by a third party, known as a registrar. When a supplier's system conforms to the registrar's interpretation of an ISO 9000 or 14000 standard, the registrar issues the supplier a "certificate of registration."

The reader should note that it is the supplier's system that is registered, not an individual product or a company's compliance with an established level of environmental performance. Consequently, management system registration does not imply product conformity to any given set of requirements or compliance wit