Randomized trial external validity

The Stroke-Thrombolytic Predictive Instrument (Stroke-TPI) has recently been developed in order to provide patient-specific estimates of the probability of a more favorable outcome with rt-PA, and has been proposed as a decision-making aid to patient selection for rt-PA." The estimates from this tool should, however, be treated with caution. The prediction rule is dependent on post hoc mathematical modeling, uses clinical trial data from subjects randomized beyond 3 hours who are not rt-PA-eligible according to FDA labeling and current best practice, and has not been externally validated. It is, therefore, not appropriate to exclude patients from rt-PA treatment based solely on Stroke-TPI predictions. 

Fig. 14.2. An external validation of the model detailed in Table 14.2 for the five-year risk of stroke on medical treatment in an independent randomized trial of endarterectomy versus medical treatment for symptomatic carotid stenosis (Rothwell et ai. 2005). Predicted risk of stroke on medical treatment is plotted against the observed risk of stroke in patients randomized to medical treatment in the trial (squares) and against the observed operative risk of stroke and death in patients randomized to surgical treatment (diamonds). Groups are quintiles of predicted risk.

Assessment of the external validity of a randomized controlled trial... 

Table 18.4. Some of the factors that can affect external validity of randomized controlled trials and should be addressed in reports of the results and be considered by clinicians...

External validity can also be affected if trials have protocols that differ from usual clinical practice. For example, prior to randomization in the trials of endarterectomy for symptomatic carotid stenosis patients had to be diagnosed by a neurologist and to have conventional arterial angiography, neither of which are routine in many centers. The trial intervention itself may also differ from that used in current practice, such as in the formulation and bioavailability of a drug, or the type of anesthetic used for an operation. The same can be true of the treatment in the control group in a trial, which may use a particularly low dose of the comparator drug or fall short of best current practice in some... 

The external validity refers to applicability and generalization and is outlined in the section, Applying the Results. The remainder of this section focuses on critically appraising the quality— that is, the internal validity—of individual trials. The internal validity is determined by how well the trial ensures that the known and unknown risk factors are equally distributed between the treatment and control groups. To ensure validity, the conduct of the trial should minimize systematic bias and random error as much as possible to provide results that are as accurate and close to the truth as possible. Four sources of bias are possible in trials of health care interventions selection bias, performance bias, attrition bias, and detection bias. Bias can result in an overestimation or underestimation of the effectiveness of a drug therapy and mislead the reader. While it is beyond... 

---