3D Face Reconstruction from 2D Pictures: First Results of a Web-Based Computer Aided System for Aesthetic Procedures

The human face is a vital component of our identity and many people undergo medical aesthetics procedures in order to achieve an ideal or desired look. However, communication between physician and patient is fundamental to understand the patient’s wishes and to achieve the desired results. To date, most plastic surgeons rely on either “free hand” 2D drawings on picture printouts or computerized picture morphing. Alternatively, hardware dependent solutions allow facial shapes to be created and planned in 3D, but they are usually expensive or complex to handle. To offer a simple and hardware independent solution, we propose a web-based application that uses 3 standard 2D pictures to create a 3D representation of the patient’s face on which facial aesthetic procedures such as filling, skin clearing or rejuvenation, and rhinoplasty are planned in 3D. The proposed application couples a set of well-established methods together in a novel manner to optimize 3D reconstructions for clinical use. Face reconstructions performed with the application were evaluated by two plastic surgeons and also compared to ground truth data. Results showed the application can provide accurate 3D face representations to be used in clinics (within an average of 2 mm error) in less than 5 min.

Comparative Evaluation of Pelvic Allograft Selection Methods

This paper presents a firsthand comparative evaluation of three different existing methods for selecting a suitable allograft from a bone storage bank. The three examined methods are manual selection, automatic volume-based registration, and automatic surface-based registration. Although the methods were originally published for different bones, they were adapted to be systematically applied on the same data set of hemi-pelvises. A thorough experiment was designed and applied in order to highlight the advantages and disadvantages of each method. The methods were applied on the whole pelvis and on smaller fragments, thus producing a realistic set of clinical scenarios. Clinically relevant criteria are used for the assessment such as surface distances and the quality of the junctions between the donor and the receptor. The obtained results showed that both automatic methods outperform the manual counterpart. Additional advantages of the surface-based method are in the lower computational time requirements and the greater contact surfaces where the donor meets the recipient.

Hierarchical segmentation-assisted multimodal registration for MR brain images

Information theory-based metric such as mutual information (MI) is widely used as similarity measurement for multimodal registration. Nevertheless, this metric may lead to matching ambiguity for non-rigid registration. Moreover, maximization of MI alone does not necessarily produce an optimal solution. In this paper, we propose a segmentation-assisted similarity metric based on point-wise mutual information (PMI). This similarity metric, termed SPMI, enhances the registration accuracy by considering tissue classification probabilities as prior information, which is generated from an expectation maximization (EM) algorithm. Diffeomorphic demons is then adopted as the registration model and is optimized in a hierarchical framework (H-SPMI) based on different levels of anatomical structure as prior knowledge. The proposed method is evaluated using Brainweb synthetic data and clinical fMRI images. Both qualitative and quantitative assessment were performed as well as a sensitivity analysis to the segmentation error. Compared to the pure intensity-based approaches which only maximize mutual information, we show that the proposed algorithm provides significantly better accuracy on both synthetic and clinical data.

Integrated Segmentation of Brain Tumor Images for Radiotherapy and Neurosurgery

Image-based modeling of tumor growth combines methods from cancer simulation and medical imaging. In this context, we present a novel approach to adapt a healthy brain atlas to MR images of tumor patients. In order to establish correspondence between a healthy atlas and a pathologic patient image, tumor growth modeling in combination with registration algorithms is employed. In a first step, the tumor is grown in the atlas based on a new multi-scale, multi-physics model including growth simulation from the cellular level up to the biomechanical level, accounting for cell proliferation and tissue deformations. Large-scale deformations are handled with an Eulerian approach for finite element computations, which can operate directly on the image voxel mesh. Subsequently, dense correspondence between the modified atlas and patient image is established using nonrigid registration. The method offers opportunities in atlasbased segmentation of tumor-bearing brain images as well as for improved patient-specific simulation and prognosis of tumor progression.

SerpinB1 is critical for neutrophil survival through cell-autonomous inhibition of cathepsin G

Bone marrow (BM) holds a large reserve of polymorphonuclear neutrophils (PMNs) that are rapidly mobilized to the circulation and tissues in response to danger signals. SerpinB1 is a potent inhibitor of neutrophil serine proteases neutrophil elastase (NE) and cathepsin G (CG). SerpinB1 deficiency (sB1(-/-)) results in a severe reduction of the BM PMN reserve and failure to clear bacterial infection. Using BM chimera, we found that serpinB1 deficiency in BM cells was necessary and sufficient to reproduce the BM neutropenia of sB1(-/-) mice. Moreover, we showed that genetic deletion of CG, but not NE, fully rescued the BM neutropenia in sB1(-/-) mice. In mixed BM chimera and in vitro survival studies, we showed that CG modulates sB1(-/-) PMN survival through a cell-intrinsic pathway. In addition, membrane permeabilization by lysosomotropic agent l-leucyl-l-leucine methyl ester that allows cytosolic release of granule contents was sufficient to induce rapid PMN death through a CG-dependent pathway. CG-mediated PMN cytotoxicity was only partly blocked by caspase inhibition, suggesting that CG cleaves a distinct set of targets during apoptosis. In conclusion, we have unveiled a new cytotoxic function for the serine protease CG and showed that serpinB1 is critical for maintaining PMN survival by antagonizing intracellular CG activity.

Fully automatic X-ray image segmentation via joint estimation of image displacements.

We propose a new method for fully-automatic landmark detection and shape segmentation in X-ray images. Our algorithm works by estimating the displacements from image patches to the (unknown) landmark positions and then integrating them via voting. The fundamental contribution is that, we jointly estimate the displacements from all patches to multiple landmarks together, by considering not only the training data but also geometric constraints on the test image. The various constraints constitute a convex objective function that can be solved efficiently. Validated on three challenging datasets, our method achieves high accuracy in landmark detection, and, combined with statistical shape model, gives a better performance in shape segmentation compared to the state-of-the-art methods.

Statistical model-based segmentation of the proximal femur in digital antero-posterior (AP) pelvic radiographs.

PURPOSE    Segmentation of the proximal femur in digital antero-posterior (AP) pelvic radiographs is required to create a three-dimensional model of the hip joint for use in planning and treatment. However, manually extracting the femoral contour is tedious and prone to subjective bias, while automatic segmentation must accommodate poor image quality, anatomical structure overlap, and femur deformity. A new method was developed for femur segmentation in AP pelvic radiographs. METHODS    Using manual annotations on 100 AP pelvic radiographs, a statistical shape model (SSM) and a statistical appearance model (SAM) of the femur contour were constructed. The SSM and SAM were used to segment new AP pelvic radiographs with a three-stage approach. At initialization, the mean SSM model is coarsely registered to the femur in the AP radiograph through a scaled rigid registration. Mahalanobis distance defined on the SAM is employed as the search criteria for each annotated suggested landmark location. Dynamic programming was used to eliminate ambiguities. After all landmarks are assigned, a regularized non-rigid registration method deforms the current mean shape of SSM to produce a new segmentation of proximal femur. The second and third stages are iteratively executed to convergence. RESULTS    A set of 100 clinical AP pelvic radiographs (not used for training) were evaluated. The mean segmentation error was [Formula: see text], requiring [Formula: see text] s per case when implemented with Matlab. The influence of the initialization on segmentation results was tested by six clinicians, demonstrating no significance difference. CONCLUSIONS    A fast, robust and accurate method for femur segmentation in digital AP pelvic radiographs was developed by combining SSM and SAM with dynamic programming. This method can be extended to segmentation of other bony structures such as the pelvis.

Comparison of partial least squares regression and principal component regression for pelvic shape prediction.

This paper studied two different regression techniques for pelvic shape prediction, i.e., the partial least square regression (PLSR) and the principal component regression (PCR). Three different predictors such as surface landmarks, morphological parameters, or surface models of neighboring structures were used in a cross-validation study to predict the pelvic shape. Results obtained from applying these two different regression techniques were compared to the population mean model. In almost all the prediction experiments, both regression techniques unanimously generated better results than the population mean model, while the difference on prediction accuracy between these two regression methods is not statistically significant (α=0.01).

Pelvic tilt is minimally changed by total hip arthroplasty

BACKGROUND While surgical navigation offers the opportunity to accurately place an acetabular component, questions remain as to the best goal for acetabular component positioning in individual patients. Overall functional orientation of the pelvis after surgery is one of the most important variables for the surgeon to consider when determining the proper goal for acetabular component orientation. QUESTIONS/PURPOSES We measured the variation in pelvic tilt in 30 patients before THA and the effect of THA on pelvic tilt in the same patients more than a year after THA. METHODS Each patient had a CT study for CT-based surgical navigation and standing and supine radiographs before and after surgery. Pelvic tilt was calculated for each of the radiographs using a novel and validated two-dimensional/three-dimensional matching technique. RESULTS Mean supine pelvic tilt changed less than 2°, from 4.4° ± 6.4° (range, -7.7° to 20.8°) before THA to 6.3° ± 6.6° (range, -5.7° to 19.6°) after THA. Mean standing pelvic tilt changed less than 1°, from 1.5° ± 7.2° (range, -13.1° to 12.8°) before THA to 2.0° ± 8.3° (range, -12.3° to 16.8°) after THA. Preoperative pelvic tilt correlated with postoperative tilt in both the supine (r(2) = 0.75) and standing (r(2) = 0.87) positions. CONCLUSIONS In this population, pelvic tilt had a small and predictable change after surgery. However, intersubject variability of pelvic tilt was high, suggesting preoperative pelvic tilt should be considered when determining desired acetabular component positioning on a patient-specific basis.

3D volumetric intensity reconsturction from 2D x-ray images using partial least squares regression

Reconstruction of shape and intensity from 2D x-ray images has drawn more and more attentions. Previously introduced work suffers from the long computing time due to its iterative optimization characteristics and the requirement of generating digitally reconstructed radiographs within each iteration. In this paper, we propose a novel method which uses a patient-specific 3D surface model reconstructed from 2D x-ray images as a surrogate to get a patient-specific volumetric intensity reconstruction via partial least squares regression. No DRR generation is needed. The method was validated on 20 cadaveric proximal femurs by performing a leave-one-out study. Qualitative and quantitative results demonstrated the efficacy of the present method. Compared to the existing work, the present method has the advantage of much shorter computing time and can be applied to both DXA images as well as conventional x-ray images, which may hold the potentials to be applied to clinical routine task such as total hip arthroplasty (THA).

Robust Proximal Femur Segmentation in Conventional X-Ray Images via Random Forest Regression on Multi-resolution Gradient Features

In this paper, we propose a fully automatic, robust approach for segmenting proximal femur in conventional X-ray images. Our method is based on hierarchical landmark detection by random forest regression, where the detection results of 22 global landmarks are used to do the spatial normalization, and the detection results of the 59 local landmarks serve as the image cue for instantiation of a statistical shape model of the proximal femur. To detect landmarks in both levels, we use multi-resolution HoG (Histogram of Oriented Gradients) as features which can achieve better accuracy and robustness. The efficacy of the present method is demonstrated by experiments conducted on 150 clinical x-ray images. It was found that the present method could achieve an average point-to-curve error of 2.0 mm and that the present method was robust to low image contrast, noise and occlusions caused by implants.

Expectation Conditional Maximization-Based Deformable Shape Registration

This paper addresses the issue of matching statistical and non-rigid shapes, and introduces an Expectation Conditional Maximization-based deformable shape registration (ECM-DSR) algorithm. Similar to previous works, we cast the statistical and non-rigid shape registration problem into a missing data framework and handle the unknown correspondences with Gaussian Mixture Models (GMM). The registration problem is then solved by fitting the GMM centroids to the data. But unlike previous works where equal isotropic covariances are used, our new algorithm uses heteroscedastic covariances whose values are iteratively estimated from the data. A previously introduced virtual observation concept is adopted here to simplify the estimation of the registration parameters. Based on this concept, we derive closed-form solutions to estimate parameters for statistical or non-rigid shape registrations in each iteration. Our experiments conducted on synthesized and real data demonstrate that the ECM-DSR algorithm has various advantages over existing algorithms.