The performance of a prediction or measurement model is often evaluated by computing the correlation coefficient and/or the regression of predictions on true (reference) values. These provide, however, only a poor description of predictive performance. The mean square prediction error (precision) and the mean prediction error (bias) provide better descriptions of predictive performance. These quantities are easily computed, and can be used to compare prediction methods to absolute standards or to one another. The measures, however, are unreliable when the reference method is imprecise. The use of these measures is discussed and illustrated.

No abstract

Dose [-concentration]-effect relationships can be obtained by fitting a predictive pharmacokinetic (PK)-pharmacodynamic (PD) model to both concentration and effect observations. Either a model can befit simultaneously to all the data ("simultaneous" method), or first a model can befit to the PK data and then a model can be fit to the PD data, conditioning in some way on the PK data or on the estimates of the PK parameters ("sequential" method). Using simulated data, we compare the performance of the simultaneous method with that of three sequential method variants with respect to computation time, estimation precision, and inference. Using NONMEM, under various study designs, observations of one type of PK and one type of PD response from different numbers of individuals were simulated according to a one-compartment PK model and direct Emax PD model, with parameters drawn from an appropriate population distribution. The same PK and PD models were fit to these observations using simultaneous and sequential methods. Performance measures include computation time,fraction of cases for which estimates are successfully obtained, precision of PD parameter estimates, precision of PD parameter standard error estimates, and type-I error rates of a likelihood ratio test. With the sequential method, computation time is less, and estimates are more likely to be obtained. Using the First Order Conditional Estimation (FOCE) method, a sequential approach that conditions on both population PK parameter estimates and PK data, estimates PD parameters and their standard errors about as well as the "gold standard" simultaneous method, and saves about 40% computation time. Type-I error rates of likelihood ratio test for both simultaneous and sequential approaches are close to the nominal rates.

Pharmacokinetic data consist of drug concentration measurements, as well as reports of some measured concentrations being below the quantification limit of the assay (BQL). A pharmacokinetic model may befit to these data, and for this purpose, the BQL observations must be either discarded or handled in a special way. In this paper, seven methods for dealing with BQL observations are evaluated. Both single-subject and population data are simulated from a one-compartment model. A moderate amount of data is simulated for each individual. The actual cv of concentration measurements at the quantification limit is assumed to be no greater than 20%, in accord with the FDA Guidance. The results of this paper should be interpreted in this context. The methods include handling BQL observations as fixed-point censored observations, i.e., by using the likelihoods that these observations are in fact BQL. This method is shown to have some overall statistical advantage. However, the gain in using this method over that of simply discarding the BQL observations is not always much, and this is especially so when the frequency of BQL observations is small. Some simple methods entailing (i) replacing one or more BQL observations with the value 0, or (ii) replacing them with the value QL/2, where QL is the quantification limit, are also included. The first of these two approaches should not be used With population data, use of the second approach can result in some noticeably improved estimation of the typical value of a parameter, but then there is also marked degradation in the estimation of the population variance of the parameter.

Individual pharmacokinetic par parameters quantify the pharmacokinetics of an individual, while population pharmacokinetic parameters quantify population mean kinetics, interindividual variability, and residual intraindividual variability plus measurement error. Individual pharmacokinetics are estimated by fitting individual data to a pharmacokinetic model. Population pharmacokinetic parameters are estimated either by fitting all individual's data together as though there was no individual kinetic differences (the naive pooled data approach), or by fitting each individual's data separately, and then combining the individual parameter estimates (the two-stage approach). A third approach, NONMEM, takes a middle course between these, and avoids shortcomings of each of them. A data set consisting of 124 steady-state phenytoin concentration-dosage pairs from 49 patients, obtained in the routine course of their therapy, was analyzed by each method. The resulting population parameter estimates differ considerably (population mean Km, for example, is estimated as 1.57, 5.36, and 4.44 micrograms/ml by the naive pooled data, two-stage, and NONMEN approaches, respectively). Simulations of the data were analyzed to investigate these differences. The simulations indicate that the pooled data approach fails to estimate variabilities and produces imprecise estimates of mean kinetics. The two-stage approach produces good estimates of mean kinetics, but biased and imprecise estimates of interindividual variability. NONMEN produces accurate and precise estimates of all parameters, and also reasonable confidence intervals for them. This performance is exactly what is expected from theoretical considerations and provides empirical support for the use of NONMEM when estimating population pharmacokinetics from routine type patient data.