Use of posterior predictive assessments to evaluate model fit in multilevel logistic regression

Assessing the fit of a model is an important final step in any statistical analysis, but this is not straightforward when complex discrete response models are used. Cross validation and posterior predictions have been suggested as methods to aid model criticism. In this paper a comparison is made between four methods of model predictive assessment in the context of a three level logistic regression model for clinical mastitis in dairy cattle; cross validation, a prediction using the full posterior predictive distribution and two “mixed” predictive methods that incorporate higher level random effects simulated from the underlying model distribution. Cross validation is considered a gold standard method but is computationally intensive and thus a comparison is made between posterior predictive assessments and cross validation. The analyses revealed that mixed prediction methods produced results close to cross validation whilst the full posterior predictive assessment gave predictions that were over-optimistic (closer to the observed disease rates) compared with cross validation. A mixed prediction method that simulated random effects from both higher levels was best at identifying the outlying level two (farm-year) units of interest. It is concluded that this mixed prediction method, simulating random effects from both higher levels, is straightforward and may be of value in model criticism of multilevel logistic regression, a technique commonly used for animal health data with a hierarchical structure.

Granulocyte-colony stimulating factor in experimental stroke and its effects on infarct size and functional outcome: a systematic review

Background Granulocyte-colony stimulating factor (G-CSF) shows promise as a treatment for stroke. This systematic review assesses G-CSF in experimental ischaemic stroke. Methods Relevant studies were identified with searches of Medline, Embase and PubMed. Data were extracted on stroke lesion size, neurological outcome and quality, and analysed using Cochrane Review Manager using random effects models; results are expressed as standardised mean difference (SMD) and odds ratio (OR). Results Data were included from 19 publications incorporating 666 animals. G-CSF reduced lesion size significantly in transient (SMD -1.63, p<0.00001) but not permanent (SMD -1.56, p=0.11) focal models of ischaemia. Lesion size was reduced at all doses and with treatment commenced within 4 hours of transient ischaemia. Neurological deficit (SMD -1.37, p=0.0004) and limb placement (SMD -1.88, p=0.003) improved with G-CSF; however, locomotor activity (>4 weeks post ischaemia) was not (SMD 0.76, p=0.35). Death (OR 0.27, p<0.0001) was reduced with G-CSF. Median study quality was 4 (range 0-7/8); Egger’s test suggested significant publication bias (p=0.001). Conclusions G-CSF significantly reduced lesion size in transient but not permanent models of ischaemic stroke. Motor impairment and death were also reduced. Further studies assessing dose-response, administration time, length of ischaemia and long-term functional recovery are needed.