The statistical analysis plan

The statistical analysis plan (SAP) is a more detailed elaboration of the statistical methods of analysis contained in the protocol. The SAP is written as the trial 

The SAP will also often contain table templates that allow the precise way in which the statistical analysis will be presented to be set down well in advance of running the analyses on the final trial data. 

---

In order to reduce unnecessary data queries, the statistics group should be consulted early in the clinical database development process to identify variables critical for data analysis. Optimally, the statistical analysis plan would already be written by the time of database development so that the queries could be designed based on the critical variables indicated in the analysis plan. However, at the database development stage, usually only the clinical protocol exists to guide the statistics and clinical data management departments in developing the query or data management plan. 

Ideally, rules for combining centres should be detailed in the Statistical Analysis Plan. 

We will discuss the decision-making process with regard to the Statistical Analysis Plan and the Blind Review in Section 16.3. 

Covariates to be included in the analysis must be pre-specified in the protocol or in the statistical analysis plan. ... 

If new knowledge becomes available regarding important covariates after completion of the statistical analysis plan then modify the plan at the blind review stage. 

A key aspect of the definition of analysis sets and the way that missing data is to be handled is pre-specification. Usually these points will be covered in the protocol, if not, in the statistical analysis plan. If methods are not pre-specified then there will be problems as the way that these issues are dealt with could then be data driven, or at least there may be suspicion of that. This is, of course, not unique to analysis sets and missing data, but is true more generally in relation to the main methods of statistical analysis. 

It is good practice to pre-specify in the protocol, or certainly in the statistical analysis plan, the statistical method to be used for analysis for each of the endpoints within the confirmatory part of the trial. This avoids the potential for bias at the analysis stage, which could arise if a method were chosen, for example, which maximised the treatment difference. As a consequence changing the method of analysis following unblinding of the study in an unplanned way, even if there seem sound statistical reasons for doing so, is problematic. Such a switch could only be supported if there was a clear algorithm contained within the statistical analysis plan which specified the rules for the switch. An example of this would be as follows ... 

The aim of the book is not to turn non-statisticians into statisticians. I do not want you to go away from this book and do statistics. It is the job of the statistician to provide statistical input to the development plan, to individual protocols, to write the statistical analysis plan, to analyse the data and to work with medical writing in producing the clinical report also to support the company in its interactions with regulators on statistical issues. 

Several summary tables are commonly presented to report safety data. Two examples of typical formats are provided here. Table 10.3 shows the format for the overall summary of adverse events falling within several adverse event categories. Such table shells are typically prepared by medical writers in advance of the study results being available and are based on the clinical study protocol and/or the statistical analysis plan written before the study started. Preparation in advance of the availability of the data saves time during the preparation of the clinical study report once the data are available. 

In the fixed sample clinical trial approach, one analysis is performed once all of the data have been collected. The chosen nominal significance level (the Type I error rate) will have been stated in the study protocol and/or the statistical analysis plan. This value is likely to be 0.05 As we have seen, declaring a finding statistically significant is typically done at the 5% p-level. In a group sequential clinical trial, the plan is to conduct at least one interim analysis and possibly several of them. This procedure will also be discussed in the trial s study protocol and/or the statistical analysis plan. For example, suppose the plan is to perform a maximum of five analyses (the fifth would have been the only analysis conducted had the trial adopted a fixed sample approach), and it is planned to enroll 1,000 subjects in the trial. The first interim analysis would be conducted after data had been collected for the first fifth of the total sample size, i.e., after 200 subjects. If this analysis provided compelling evidence to terminate the trial, it would be terminated at that point. If compelling evidence to terminate the trial was not obtained, the trial would proceed to the point where two-fifths of the total sample size had been recruited, at which point the second interim analysis would be conducted. All of the accumulated data collected to this point, i.e., the data from all 400 subjects, would be used in this analysis. 

Apart from compliance with SOPs for biostatistics and report writing, the statistical analysis plan, the trial protocol, regulatory requirements and guidelines (ICH E3, 1995 ICH E9, 1998 ISO 9000 2005, 2005), QA auditors check the internal consistency of the trial report and appendices and between data in tables, figures and graphs and numbers cited in the text. All numbers and percentages must be substantiated by attached tables and listings. In summary, the trial report should be an accurate representation of the clinical data. Allocation of trial... 

A study protocol is often supplemented with another very important document called the statistical analysis plan (sometimes referred to by similar names such as a data analysis plan or reporting analysis plan). The statistical analysis plan often supplements a study protocol by providing a very detailed account of the analyses that will be conducted at the completion of data acquisition. The statistical analysis plan should be written in conjunction with (and at the same time as) the protocol, but in reality this does not always happen. At the very least it should be finalized before the statistical analysis and breaking of the blind. In many instances (for example, confirmatory trials) it may be helpful to submit the final statistical analysis plan to the appropriate regulatory authorities for their input. 

The handling and analysis of pharmacoeco-nomic data should be along the lines familiar to those observing good clinical practices (GCP) for other purposes. Data collection instruments need to be selected, or created and incorporated into case report forms, just as for any other end-point. Data analysis plans should be created prospectively. The statistical analysis plan should be prospective, and should help put the pharmacoe-conomic measures in the context of other properties of the test medication (Table 19.3). Are they... 

The [statistical] plan should be reviewed and possibly updated as a result of the blind review of the data... and should be finalized before breaking the blind. Formal records should be kept of when the statistical analysis plan was finalized, as well as when the blind was subsequently broken. If the blind review suggests changes to the principal features stated in the protocol, these should be... 

Completeness and transparency are key principles in the reporting of a meta-analysis for the evaluation of safety in the regulatory framework. The protocol and the statistical analysis plan should be completed and finalized prior to the conduct of the meta-analysis. Any deviations or additional investigations after the finalization of the protocol should be clearly identified as such. 

SMQs) might be an interesting future direction as the SMQs represent more well-established medical concepts. Because there is flexibility in modeling different levels of AEs, as a practical advice, it is important to prespecify the multiplicity adjustment method that will be used. Grouping and definitions should be described in the study protocol or the statistical analysis plan, rather than on a post hoc basis. 

A total of 16,492 participants with type 2 diabetes or at risk for cardiovascular events were randomized to saxagliptin or placebo (on a background of standard of care for diabetes and cardiovascular risk factors) and followed for a median of 2.1 years. The primary endpoint was occurrence of MACE outcomes. A primary endpoint event occurred in 613 participants in the saxagliptin group and in 609 participants in the placebo group. The statistical analysis plan prespecified that a test for noninferiority would be conducted first, followed by a test for superiority. Saxagliptin did not increase or decrease the rate of MACE outcomes (hazard ratio=1.00, 95% CI=0.89-1.12 p<0.001 for noninfeiiority, p=0.99 for superiority). 

Adaptive methodologies, such as sample size or outcome re-estimations and early stopping decision rules, could be specified in the statistical analysis plans for cardiovascular safety outcome trials, potentially to reduce study duration, increase the chances of success, and facilitate earlier submissions. Many different types of adaptive designs are in use. The most common adaptation is to increase the sample size if the rate at which outcomes are accruing is slow. This modification raises no statistical issues if done in a blinded manner. 

---