CRM assessment

Thirdly, if one is not careful, a crude risk benefit analysis can lead to gross over simplification - almost to the point of justifying a product s risk in nothing more than a few sentences. When one is faced with the effort and expense of undertaking a careful CRM assessment it might on occasion be tempting to take the following position ... 

Organisations may decide (or be required) to operate an SMS which is compliant with a specific local or international standard. By developing a consistent process that is compliant with the standard or standards, its requirements can be flowed down and inherited by the SMS. Those who are undertaking a CRM assessment can follow the SMS without necessarily concerning themselves with the specific details of individual standards - ultimately achieving compliance by abstraction. 

A CRM assessment will generally be a cost to the project (although of course one can argue the ultimate value add of the assessment more than recuperates that cost). The cost needs to be included in the project business case but quantifying it early in the product lifecycle can be a challenge. By developing a repeatable SMS, one is able to better predict the likely effort involved in applying that process to a particular product. 

In any CRM assessment one needs to define the boundaries of the analysis (as discussed in Sect. 11.1.1). Those systems, modules, components or areas of functionality which have the potential to impact care are often described as being safety-related . Thus, an entity can be considered safety-related if it ... 

Perhaps the most important interface is the user interface with which the user operates the system. User interface design is a complex topic and beyond the scope of this book. However, it is key to understand that subtle features of the user interface have the potential to be causes of hazards. Is important clinical information difficult to access, truncated, inappropriately labelled or subject to excessive scrolling Ideally one would identily these potential issues from design material or early prototypes. In some cases the full feel of the user interface can only be assessed once the system has been at least partially built. Projects should therefore plan to assess, redesign and re-assess parts of the user interface where necessary and a CRM assessment of any shortcomings should influence the prioritisation of any areas for re-design. 

Where an issue is complex, associated with significant clinical risk or needs to be communicated to a wider audience it can be beneficial to document and distribute a formal CRM assessment to project stakeholders. This is a convenient way of summarising the problem in one or two pages, setting out the rationale for the clinical risk evaluation and making any recommendations. The approach set out in Sect. 19.2 can be used for these purposes. 

Each defect needs to be subject to a CRM assessment and a level of clinical risk assigned to each to inform the fix priority. It should be possible to map each defect back to an appropriate hazard in the hazard register. If this is not possible then the team should discuss whether in fact a previously unforeseen hazard has been detected. 

This section will nonnaUy set the scene and rationale for undertaking the CRM assessment. This serves to put the report into the context of the overall project and product lifecycle with reference to key milestones. It is often appropriate at this point to refer back to the original CRM Plan for further context setting. 

Alternatively we may decide to create more specific claims, perhaps that the system performs at a level that is conducive to practical operation in a clinical environment and/or with a defined degree of availability. Whatever measure is selected it should be clear and concise and, most importantly, be capable of being demonstrated within the scope of the planned CRM assessment process. In other words it should be possible to draw lines of logical inference between each claim and the arguments set out in the body of the safety case. 

Where manufacturers have contractually agreed resolution times, there exists a clear chain of influence from the CRM assessment to the commercial pressure to resolve it. 

Not all faults impact safety to the same extent so a CRM assessment will be required. The degree of clinical risk should be one of the factors which influence fix priority. 

Objectively documenting a CRM assessment for a significant fault is a powerful tool in gaining stakeholder buy-in and effecting its resolution. This is particularly the case when a healthcare organisations needs to escalate a fault to the manufacturer. 

Number of technical breakdowns in communication, Level of relevant team training (CRM), assessment of quality of critical communication- related to miscommunication between actors during critical operations. 

ISO Guide 33 (1998) deals with other uses of RMs. It elaborates on various uses of RMs, excluding calibration, which is the subject of ISO Guide 32. In most cases, RMs are used as a quality control measure, i.e. to assess the performance of a measurement method. Most matrix RMs are produced with this purpose in mind. Other purposes of RMs are the maintenance of conventional scales, such as the octane number and the pH scale. ISO Guide 33 provides guidance on the proper use of RMs, and therefore it is together with ISO Guide 32 the most important document for users of CRMs. 

To validate the analytical procedure recovery experiments are performed. To this end, the CRM is spiked with a known mass of the analytes at a variety of concentration levels (at least three different levels) and the concentrations measured are compared to the expected concentrations in at least three separate experiments. The extraction step has been shown to be a critical step in the analytical procedure and it may be responsible for poor recoveries. The efficiency of this step can be assessed either by repetitive extraction of the sample or by the addition of internal standards prior to the extraction step with the assumption that the latter actually represent the behavior of the analytes of interest. 

To discuss the development of CRMs for the emerging use of microanalytical techniques, one has to be concerned chiefly with the degree of homogeneity of the components in the material at the designated sample size. Basic indications for the homogeneity properties of a CRM for microanalytical methods and the assessment of these properties can be derived from the general requirements ... 

To assess homogeneity, the distribution of chemical constituents in a matrix is at the core of the investigation. This distribution can range from a random temporal and spatial occurrence at atomic or molecular levels over well defined patterns in crystalline structures to clusters of a chemical of microscopic to macroscopic scale. Although many physical and optical methods as well as analytical chemistry methods are used to visualize and quantify such spatial distributions, the determination of chemical homogeneity in a CRM must be treated as part of the uncertainty budget affecting analytical chemistry measurements. 

In their broadest application, CRMs are used as controls to verify in a direct comparison the accuracy of the results of a particular measurement parallel with this verification, traceability may be demonstrated. Under conditions demonstrated to be equal for sample and CRM, agreement of results, e.g. as defined above, is proof. Since such possibilities for a direct comparison between samples and a CRM are rare, the user s claims for accuracy and traceability have to be made by inference. Naturally, the use of several CRMs of similar matrix but different analyte content will strengthen the user s inference. Even so, the user stiU has to assess and account for all uncertainties in this comparison of results. These imcertainty calculations must include beyond the common analytical uncertainty budget (i) a component that reflects material matrix effects, (2) a component that reflects differences in the amount of substance determined, (3) the uncertainty of the certified or reference value(s) used, and 4) the uncertainty of the comparison itself AU this information certainly supports the assertion of accuracy in relation to the CRM. However, the requirement of the imbroken chain of comparisons wiU not be formally fulfilled. 

This confirmation is often provided by an external audit or assessment (u.s.). Quality management systems are often certified for conformation with ISO 9000. Probably the most common term with respect to our topic here is certified reference materiaT (CRM). 

The analytical quality in such cases can be better assessed when CRMs of similar nature are available. For example, samples of arterial blood weighing 5 to 15 mg were withdrawn from different parts of a rabbit and were subjected to mineralization with hot 50% (v/v) HNO3 in a closed vessel microwave device. After adequate dilution end analysis of Ca, Mg and Fe was carried out by ICP/AES . 

One or more of these bias components are encountered when analyzing RMs. In general, RMs are divided into certified RMs (CRMs, either pure substances/solu-tions or matrix CRMs) and (noncertified) laboratory RMs (LRMs), also called QC samples [89]. CRMs can address all aspects of bias (method, laboratory, and run bias) they are defined with a statement of uncertainty and traceable to international standards. Therefore, CRMs are considered useful tools to achieve traceability in analytical measurements, to calibrat equipment and methods (in certain cases), to monitor laboratory performance, to validate methods, and to allow comparison of methods [4, 15, 30]. However, the use of CRMs does not necessarely guarantee trueness of the results. The best way to assess bias practically is by replicate analysis of samples with known concentrations such as reference materials (see also Section 8.2.2). The ideal reference material is a matrix CRM, as this is very similar to the samples of interest (the latter is called matrix matching). A correct result obtained with a matrix CRM, however, does not guarantee that the results of unknown samples with other matrix compositions will be correct [4, 89]. 

The usefulness of CRMs for validation (in particular for trueness assessment) and traceability purposes has been debated for years. This is illustrated by the enor-... 

To determine Sb in marine sediments by ETAAS, a direct method was developed based on quantitating the analyte in the liquid phase of the slurries (prepared directly in autosampler cups). The variables influencing the extraction of Sb into the liquid phase and the experimental setup were set after a literature search and a subsequent multivariate optimisation procedure. After the optimisation, a study was carried out to assess robustness. Six variables were considered at three levels each (see Table 2.13). In addition, two noise factors were set after observing that two ions, which are currently present into marine sediments, might interfere in the quantitations. In order to evaluate robustness, a certified reference material was used throughout, BCR-CRM 277 Estuarine Sediment (guide value for Sb 3.5 0.4pgg ). Table 2.13 depicts the experimental setup. 

The stability is another critical aspect for RM use it has to be verified in relation to the purpose of the study, e.g. over the duration of an interlaboratory study, or over long-term storage periods (for CRMs). The (in)stability should be studied or known before the RM is produced and should be monitored on the batch of RM. Studies may be performed under accelerated ageing conditions, e.g. elevated temperatures, or at various temperatures over defined periods of time. The BCR has developed a strategy with respect to stability assessment which has been successfully used for a wide variety of chemical species as described later in this chapter. Full details on the organisation of such studies are described elsewhere (Quevauviller et al., 1996a Quevauviller, 1998b). 

A common way to assess the matrix bias is to analyse a drinking water (therefore containing a matrix) in the proficiency testing. For this analysis, laboratories calibrate their instruments using standard, commercial or in-house solutions. If a CRM is available, a possible matrix effect can be corrected by adjusting operational instrument parameters to match the certified value. 

If a suitable (adequate) CRM is not available, participation of a second laboratory in an IHRM characterization is desirable, even when an unbiased validated method is used, to assess the laboratory bias. Traceability chains of the method form in this case a traceability scheme of the value carried by the IHRM. 

Several techniques can be used for validation, the most highly recommended ones being (i) evaluation of uncertainty (i.e., a systematic assessment of the quantities influencing the result) (ii) performing CRM analysis (iii) participation in ILCs/PTs and (iv) comparison of results with other analytical... 

As already mentioned, the analytical parameters required for method validation and for the estimation of measurement uncertainty can be evaluated without assigned values. But to assess the accuracy of delivered results, as stated in ISO/IEC 17025, there is a requirement for assigned values with a stated uncertainty, which are traceable to the same reference as the analytical results of the method used. In physics, measurements are made in accordance with the SI units, which were introduced under the convention of the meter. In chemical measurements, traceability of results to SI units is not always possible. Therefore, the role of chemical standards is decisive in establishing the comparability of results between laboratories. During the validation of the analytical procedure, traceability of the result can be demonstrated by comparison against the certified value of a CRM, which provides exactly this traceable assigned value with a stated uncertainty. 

---