Full analysis set

The previous section clearly indicates the need to conform to the principle of intention-to-treat to ensure that the statistical comparison of the treatment groups remains valid. In practice compliance with this principle is a little more difficult and the regulators, recognising these difficulties, allow a compromise. This involves the definition in particular trials of the full analysis set which gets us as close as we possibly can get to the intention-to-treat ideal. 

ICH E9 (1998) Note for Guidance on Statistical Principles for Clinical Trials  

The regulators are telling us therefore, to get as close as possible and they go on in the ICH E9 guideline to outline circumstances where it will usually be acceptable to omit subjects without causing bias. 

These omissions will not cause bias only under some circumstances. In particular, subjects in each of the treatment groups should receive equal scrutiny for protocol violations and all such violators should be excluded, in relation to the first point. For the second and third points, the fact that patients do not take study medication or do not provide any post-baseline data should be unrelated to the treatments to which such subjects were assigned. Any potential bias arising from these exclusions should be fully investigated. 

The term full analysis set was introduced in order to separate the practice of intention-to-treat from the principle, but practitioners still frequently use the term intention-to-treat population when referring to this set. The term modified intention-to-treat population is also in common use within particular companies and also by regulators in some settings where exclusions from strict intention-to-treat are considered. 

---

The intention-to-treat principle implies that the primary analysis should include all randomised subjects. Compliance with this principle would necessitate complete follow-up of all randomised subjects for study outcomes. In practice this ideal may be difficult to achieve, for reasons to be described. In this document the term full analysis set is used to describe the analysis set which is as complete as possible and as close as possible to the intention-to-treat ideal of including all randomised subjects. ... 

The per-protocoT set of subjects, sometimes described as the valid cases, the efficacy sample or the evaluable subjects sample, defines a subset of the subjects in the full analysis set who are more compliant with the protocol... ... 

The definition of a per-protocol set of subjects allows us to get closer to the scientific question by including only those patients who comply with the protocol to a defined extent. The per-protocol set, like the full analysis set, must be prespecified in the protocol and then defined at the patient level at the blind review, following database lock, but before breaking the blind. It must be noted, however, that the per-protocol set is subject to bias and further, tends to overestimate the treatment effect. For this reason it is usually used only as a secondary analysis, supportive hopefully of the findings based on the full analysis set. 

In general, it is advantageous to demonstrate a lack of sensitivity of the principle trials results to alternative choices of the set of subjects analysed. In confirmatory trials it is usually appropriate to plan to conduct both an analysis of the full analysis set and a per-protocol analysis, so that any differences between them can be the subject of explicit discussion and interpretation. ... 

This regulatory statement is not saying that the analyses based on the full analysis set and the per-protocol set are in any sense co-primary. The full analysis set will provide the primary analysis and usually this analysis must give p < 0.05 for a positive result. The per-protocol set, however, does not need to give p < 0.05, but should provide results which are qualitatively similar in terms of the direction of the treatment effect and with effect size not too dissimilar from that seen for the full analysis set. 

When the full analysis set and the per-protocol set lead to essentially the same conclusions, confidence in the trial results is increased... ... 

The considerations so far in this chapter have been on the evaluation of efficacy. For safety we usually define the safety set as the set of subjects who receive at least one dose of study medication. Usually the safety set will coincide with the full analysis set, but not always. There may well be a patient who started on medication, but withdrew immediately because of a side effect. This patient is unlikely to have provided post baseline efficacy data and so could be excluded from the full analysis set. 

No It often makes sense to power for the per-protocol set and then factor upwards to allow for dropouts as this will also ensure that there is enough power for the full analysis set providing any extra patient-to-patient variation in the full analysis set does not counterbalance the increase in sample size, but the analysis based on the per-protocol set is still subject to bias. See Section 8.5.2 for further discussion on this point. 

Generally speaking we power based on the per-protocol set and then increase the sample size requirement to give the number required in the full analysis set. Under some circumstances, for example in anti-infective trials, we factor up further to take into account the patients that are recruited, but are not eligible for the full analysis set. 

In a similar way it may be that the crd seen in the analysis based on the per-protocol set is larger than that seen in the full analysis set and this anticipated difference may also need to be factored in. 

In a superiority trial the primary analysis will be based on the full analysis set with the per-protocol set being used as the basis for a supportive secondary analysis, and in this sense there will be no multiplicity issues. The form of the analysis, however, depends in addition on the methods to be used to account for missing data and these should clearly be pre-specified. It is also good practice to explore the robustness of the conclusions to both the choice of the per-protocol set and the methods to be used for missing data. These analyses again will be supportive (or not) of the main conclusions and no multiplicity aspects arise. 

In equivalence and non-inferiority trials (see Chapter 12), the full analysis set and the per-protocol set have equal status and are treated as co-primary. The requirement, therefore, is to show significance for each of these analyses. This is another case where significance is needed on all endpoints with both analyses being conducted at the usual 5 per cent significance level. 

In superiority trials, the full analysis set is the basis for the primary analysis. As discussed in Section 7.2, the regulators prefer this approach, in part, because it gives a conservative view of the new treatment. In equivalence/non-inferiority trials, however, it is not conservative and will tend to result in the treatments looking more similar than, in reality, they are. This is because the full analysis set will include the patients who have not complied with the medication schedules and who have not followed the study procedures and the inclusion of such patients will tend to weaken treatment differences. 

For equivalence and non-inferiority trials, therefore, the regulators like to see analyses undertaken on both the full analysis set and the per-protocol set with positive conclusions being drawn from both. In this sense these two analyses are considered co-primary. There is a common misconception here that for equivalence/non-inferiority trials the per-protocol set is primary. This is not the case. The per protocol set is still potentially subject to bias because of the exclusion of randomised patients and so cannot supply the complete answer both analysis sets need to be supporting equivalence/non-inferiority in order to have a robust conclusion. 

As with sample size in superiority trials we generally power on the basis of the per-protocol set and increase the sample size to account for the non-evaluable patients. This is particularly important in non-inferiority trials where the full analysis set and the per-protocol set are co-primary analyses. Note also, as before in superiority trials further factoring up may be needed if there are randomised patients who are being systematically excluded from the full analysis set, as is the case, for example, in anti-infective trials. 

For superiority the full analysis set is usually the basis for the primary analysis so the emphasis in the superiority claim would then need to be based around this. 

Definition of analysis sets (full analysis set, per-protocol set, safety set)... 

---