The Data Analysis Plan

good modeling practices dictate that a data analysis plan (DAP) be written prior to any modeling being conducted and prior to any unblinding of the data. The reason being that model credibility is increased to outside reviewers when there is impartiality in the model development process. The DAP essentially provides a blueprint for the analysis. It should provide details 

The 1999 guidance on Population Pharmacokinetics issued by the Food and Drug Administration (FDA) differentiates two types of DAPs, which they refer to as Study Protocols. The first type is an add-on DAP which is seamlessly interwoven into the clinical protocol from which the pharmacokinetic data will be derived. The other type is a stand-alone DAP, which is independent of any clinical protocols, and can standalone by itself without reference to other protocols. Stand-alone protocols are useful when data from many different studies will be analyzed. In the guidance issued by the ICH, it is suggested that the essential features of the analysis are included in the clinical protocol, but that the details of the analysis are identified in a stand-alone SAP. 

In the stand-alone DAP, the key essential feature is prespecification of the analysis in which the primary analysis variable(s) are defined and methods for dealing with anticipated problems are defined. A DAP differs from the concept of a SAP as defined by the ICH in one important aspect. Modeling is largely an exercise in exploratory data analysis. There are rarely specific hypotheses to be tested. Hence, any PopPK analysis cannot be described in the detail outlined by a SAP under ICH guidelines. Nevertheless, certain elements can be predefined and identified prior to conducting any analysis. But keep in mind that a DAP devoid of detail is essentially meaningless, whereas a DAP that is so detailed will inevitably force the analyst to deviate 

---

From the perspective of the time required for modeling, it is apparent that a very important aspect of the data collection phase is ensuring that the pharmacometri-cian takes the time to prepare for the modeling. This preparatory work should include finalization of the data analysis plan, preparation of model building procedures, and construction of a template or templates for the report. In this way, the data collection phase can shorten the time required for modeling. 

Given the level of research activity devoted to identification of influential observations, considerably less effort has been devoted to what to do about them. Under guidelines (E9 Statistical Principles for Clinical Trials) developed by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (1997), more commonly called ICH, several principles for dealing with outliers or influential observations are presented. First, data analysis should be defined prior to analyzing the data, preferable before data collection even begins. The data analysis plan should specify in detail how outliers or influential observations will be handled. Second, in the absence of a plan for handling outliers or influential observations, the analyst should do two analyses, one with and the other without the points in question, and the differences between the results should be presented in the discussion of the results. Lastly, identification of outliers should be based on statistical, as well as scientific rationale, and that the context of the data point should dictate how to deal with it. 

To see what impact these observations might have on the parameter estimates, these observations were removed and the best model after backwards stepwise model development was refit. The results are shown in Table 9.15. Removal of these observations resulted in a decrease in the OFV, AIC, and condition number with little to no change in the parameter estimates. Although to be fair, direct comparison of the OFVs and AICs is not valid because of the unequal number of observations in the data sets. More importantly, although the distribution of weighted residuals was not normally distributed, the distribution was no longer skewed. Also, the standard error of the estimates all decreased. Whether to remove these observations from the data set is not immediately clear and if removal of observations was not specified a priori in the data analysis plan then their removal should probably not be made. In this case, it was decided to remove the observations in the data set. 

Table 15.9 lists the common graphical representations with the typical usage and interpretation in a population approach setting. Some specific examples of representative plots are shown in Figures 15.2 through 15.5. Ette and Ludden have provided an informative example of how these and other diagnostic techniques are integrated into the data analysis plan of a population pharmacokinetic analysis. 

The data analysis plan describing how the data will be analyzed how often by whom how the results will be presented, discussed, and used for internal program decision making and what information will be shared with external groups... 

In order to reduce unnecessary data queries, the statistics group should be consulted early in the clinical database development process to identify variables critical for data analysis. Optimally, the statistical analysis plan would already be written by the time of database development so that the queries could be designed based on the critical variables indicated in the analysis plan. However, at the database development stage, usually only the clinical protocol exists to guide the statistics and clinical data management departments in developing the query or data management plan. 

A key aspect of the definition of analysis sets and the way that missing data is to be handled is pre-specification. Usually these points will be covered in the protocol, if not, in the statistical analysis plan. If methods are not pre-specified then there will be problems as the way that these issues are dealt with could then be data driven, or at least there may be suspicion of that. This is, of course, not unique to analysis sets and missing data, but is true more generally in relation to the main methods of statistical analysis. 

Sometimes the blind review can throw up data issues that require further evaluation by the data management group with data queries being raised, and these perhaps may result in changes to the database. This sequence of events can cause major headaches and delays in the data analysis and reporting, and so it is important in the planning phase to get the data validation plan correct so that issues are identified and dealt with in an ongoing way. 

The aim of the book is not to turn non-statisticians into statisticians. I do not want you to go away from this book and do statistics. It is the job of the statistician to provide statistical input to the development plan, to individual protocols, to write the statistical analysis plan, to analyse the data and to work with medical writing in producing the clinical report also to support the company in its interactions with regulators on statistical issues. 

Several summary tables are commonly presented to report safety data. Two examples of typical formats are provided here. Table 10.3 shows the format for the overall summary of adverse events falling within several adverse event categories. Such table shells are typically prepared by medical writers in advance of the study results being available and are based on the clinical study protocol and/or the statistical analysis plan written before the study started. Preparation in advance of the availability of the data saves time during the preparation of the clinical study report once the data are available. 

Safety analyses are not typically prespecified in the study protocol and/or the study analysis plan. Studies are typically powered on efficacy outcomes (the primary objective in therapeutic confirmatory trials see Chapter 9), and the sample size that results from this sample-size estimation may be considerably smaller than would be needed for a thorough investigation of safety data. 

In the fixed sample clinical trial approach, one analysis is performed once all of the data have been collected. The chosen nominal significance level (the Type I error rate) will have been stated in the study protocol and/or the statistical analysis plan. This value is likely to be 0.05 As we have seen, declaring a finding statistically significant is typically done at the 5% p-level. In a group sequential clinical trial, the plan is to conduct at least one interim analysis and possibly several of them. This procedure will also be discussed in the trial s study protocol and/or the statistical analysis plan. For example, suppose the plan is to perform a maximum of five analyses (the fifth would have been the only analysis conducted had the trial adopted a fixed sample approach), and it is planned to enroll 1,000 subjects in the trial. The first interim analysis would be conducted after data had been collected for the first fifth of the total sample size, i.e., after 200 subjects. If this analysis provided compelling evidence to terminate the trial, it would be terminated at that point. If compelling evidence to terminate the trial was not obtained, the trial would proceed to the point where two-fifths of the total sample size had been recruited, at which point the second interim analysis would be conducted. All of the accumulated data collected to this point, i.e., the data from all 400 subjects, would be used in this analysis. 

At the Pine Bluff Chemical Agent Disposal Facility (PBCDF), process knowledge, quality assurance data, and analytical data are used to make waste characterization decisions. Under the PBCDF RCRA permit, the term chemical agent free refers to contaminated or potentially contaminated solid materials that have been tested per the PBCDF waste analysis plan and found to be below the WCL or to have been thermally treated for 15 minutes at 1000°F. Under the waste analysis plan, waste may be shipped off-facility for treatment and/or disposal only if... 

Following construction of the conceptual model, problem formulation continues by developing a plan to implement the conceptual model of the ERA. The resulting analysis plan further characterizes the stressors, identifies specific ecological effects of concern, and identifies applicable data, as well as measures or models that can be used to quantitatively relate the stressors to the expected ecological effects. 

Apart from compliance with SOPs for biostatistics and report writing, the statistical analysis plan, the trial protocol, regulatory requirements and guidelines (ICH E3, 1995 ICH E9, 1998 ISO 9000 2005, 2005), QA auditors check the internal consistency of the trial report and appendices and between data in tables, figures and graphs and numbers cited in the text. All numbers and percentages must be substantiated by attached tables and listings. In summary, the trial report should be an accurate representation of the clinical data. Allocation of trial... 

Phase IV studies can also take the form of retrospective pooled analyses which are designed to reflect the totality of clinical research experience with a new drug which is usually obtained via analysis of pooled clinical trial databases. Such data-mining efforts should not be considered inferior to data obtained from the conduct of an individual clinical trial. In fact, there are substantial benefits of such an approach, as the results of a given trial, especially if the end point is not prespecified to be primary because of power limitations, can be a function of chance. To assure that the results obtained from pooled analyses are not biased, a prespecified data analysis plan is often formulated as the first step, outlining the clear goals of the proposed analysis as well as its methodology. Key to this type of analysis is the definition of the outcome measure. Both efficacy and safety measures can be the focus of these types of analyses. 

A study protocol is often supplemented with another very important document called the statistical analysis plan (sometimes referred to by similar names such as a data analysis plan or reporting analysis plan). The statistical analysis plan often supplements a study protocol by providing a very detailed account of the analyses that will be conducted at the completion of data acquisition. The statistical analysis plan should be written in conjunction with (and at the same time as) the protocol, but in reality this does not always happen. At the very least it should be finalized before the statistical analysis and breaking of the blind. In many instances (for example, confirmatory trials) it may be helpful to submit the final statistical analysis plan to the appropriate regulatory authorities for their input. 

Table 11.1 lists some useful headings in a data analysis plan. The specific outline will vary from company to company and/or from project to project. Clearly, there is a great deal of information included in this type of document, so it is necessary to realize that writing it will require a considerable amount of time. On the other... 

Another important point to remember is that the analysis plan should be finalized before the data from the study becomes accessible. On the other hand, it does not need to have been finalized before the actual study (the data collection) starts, meaning that some sections and many details can be added while the study is running. However, since the analysis plan describes how the analysis should be performed and as this, in turn, depends on the goals of the analysis, it is necessary to start thinking about the intended form of the analysis and start considering the plan at the beginning of the project. At the start of the data collection the analysis plan need only be sufficiently detailed to ensure that goals of the analysis can be met. The plan can then be finalized some time later, but it must be fully specified before the start of the data analysis. 

---