Skip to main content
Download PDF
Download ePub

Methods for verification of 3D printed anatomic model accuracy using cardiac models as an example

3D Printing in Medicine volume 5, Article number: 6 (2019) Cite this article

Abstract

Background

Medical 3D printing has brought the manufacturing world closer to the patient’s bedside than ever before. This requires hospitals and their personnel to update their quality assurance program to more appropriately accommodate the 3D printing fabrication process and the challenges that come along with it.

Results

In this paper, we explored different methods for verifying the accuracy of a 3D printed anatomical model. Methods included physical measurements, digital photographic measurements, surface scanning, photogrammetry, and computed tomography (CT) scans. The details of each verification method, as well as their benefits and challenges, are discussed.

Conclusion

There are multiple methods for model verification, each with benefits and drawbacks. The choice of which method to adopt into a quality assurance program is multifactorial and will depend on the type of 3D printed models being created, the training of personnel, and what resources are available within a 3D printed laboratory.

Introduction

3-dimensional (3D) printing is a method of fabrication that allows for the creation of graspable 3D objects from a digital blueprint. One of the most compelling use cases for 3D printing in medicine is the creation of patient-specific anatomical models for presurgical planning [1,2,3]. In these cases, the digital blueprint used for printing is created from a patient’s medical imaging data [4]. The resultant physical model can be given to a surgeon/ interventionalist, offering him or her an opportunity to plan the surgery before stepping into the operating room/catheterization lab. Due to positive feedback from the use of these 3D printed anatomical models during pre-surgical consultations, there has been a push to explore additional ways that 3D models can be used. Other examples of using 3D printed models in the hospital include benchtop surgical simulation [5,6,7], sizing of devices prior to a surgery or procedure [8,9,10,11,12,13], and designing patient-matched surgical cutting guides [14, 15] or implants [16,17,18,19,20,21].

Established quality assurance (QA) programs exist for many areas of medicine, including medical imaging. In radiology, these include Quality Control (QC) programs for ensuring optimal performance of imaging acquisition hardware [22, 23] and QA programs for dose reduction, appropriate use, radiologist interpretations, and reporting of results [24]. A handful of hospitals have led the way in adapting and extending imaging QA programs for use in 3D printing [25,26,27], including the creation of new phantoms that test the performance and accuracy of 3D printers and materials [1, 26]. As 3D printing programs push past enhanced visualization as a product deliverable and begin to create true medical devices, there is a need to expand the existing QA programs within hospitals to create a more robust QA program inclusive of 3D printing as a clinical resource.

Verification refers to ensuring that a part is physically made to product specifications within a given tolerance (e.g., a model of a patient’s heart matches the dimensions of the patient’s actual heart within ±1 mm), while validation ensures that the model will fulfill its intended purpose and meet the customer’s needs and expectations (e.g., the model of a patient’s heart will allow the surgeon the opportunity to practice the procedure prior to surgery). Verification that a part is built to pre-defined specifications may seem straightforward on the surface, but it can pose multiple challenges in the 3D printing space. Measuring organic shapes can be complicated; several measurement techniques have been employed to date with variable results, including manual measurements with calipers [28,29,30], photogrammetry, optical [31,32,33] and contact-based surface scanning [34], and x-ray/ CT scanning [26, 35] of a part. Another challenge is that verification requires knowing the 3D printed part’s desired dimensions, which in the case of patient-specific anatomical models means knowing the accurate dimensions of the patient’s anatomy. These reference dimensions are often impossible to physically measure if the anatomy is internal, and therefore the reference standard for a given 3D model or part is often based on medical imaging, which becomes a stand-in for ground truth.

In this paper, we explore several measurement methods for verifying the accuracy of 3D-printed cardiac models and discuss some of the unique challenges we encountered with each method. This work is not meant to determine the superiority of one technique over the other nor to make a statement about what technique should ultimately be used. Instead, it is meant as an overview and a conversation starter concerning the challenges inherent in part verification.

Methods

Subjects

Patient-specific models were created at the University of Washington School of Medicine as part of an ongoing, IRB-approved research project to determine the utility of 3D printed heart models for pre-procedural planning. A subset of these hearts underwent additional quality assurance testing, along with 3D printed models of cadaveric donor hearts, to establish a standard operating procedure for model verification.

Creation of 3D printed anatomical models from patient CT scans

Cardiac CT scans were acquired using a GE Revolution scanner (GE, Waukesha, WI) in axial plane at 0.625 mm slice thickness, 100–120 kV, and variable mA with dose modulation. Pixel spacing was 0.488 × 0.488 mm, with image matrix 512 × 512. Images were reconstructed using the GE “Standard” convolution kernel.

Next, the Digital Imaging and Communications in Medicine (DICOM) images were imported into Mimics Medical software version 21.0 (Materialise, Leuven Belgium) and were manually segmented using thresholding-based strategies. For blood pool segmentation, the blood pool mask was edited to exclude non-cardiac structures (e.g. ribs, pulmonary vasculature). For myocardium models, a second mask was segmented based on myocardial Hounsfield units and the blood pool mask was subtracted from the myocardial mask. In a subset of cases included in this study, the CT scanner table was also segmented using a thresholding strategy. Heart volume and CT table masks were exported from Mimics Medical into 3-Matic Medical Software (Version 13.0) as STL files for further STL processing, including smoothing and correction of vertex errors. Once STL editing was complete, all files were imported back into Mimics software for final qualitative verification of positioning and mask accuracy with respect to the original DICOM imaging.

Finally, all models were 3D-printed on a Form2 printer (Formlabs, Cambridge, MA). Models were printed out of clear, white and gray resins.

Creation of 3D printed anatomical models from cadaveric hearts

Cadaveric hearts were placed into plastic containers and fully submerged in a saline bath spiked with iodinated contrast. The contrast: saline ratio was titrated so that the saline bath measured approximately 400 Hounsfield units (±25 HU, Standard Deviation) on test scans. This provided sufficient contrast between the cadaveric heart tissue and the surrounding saline for segmentation. Cadaveric hearts were imaged using a GE Revolution scanner (GE, Waukesha, WI) in the axial plane at 0.625 mm slice thickness, 100 kV, and 375 mA. Pixel spacing was 0.488 × 0.488 mm, with image matrix 512 × 512. Images were reconstructed using the GE “Standard” convolution kernel.

Segmentation methods and STL file creation were essentially as described above. Slicing software used for g-code creation was Cura (Ultimaking Ltd., London, England). 3D printed heart models were printed on a material extrusion printer (Printrbot simple; Printrbot, Sacramento CA) using flexible white filament (Ninjaflex; Ninjatek, Manheim, PA).

Measurement of discrete features on 3D printed models

In the first set of experiments, we explored the feasibility of physically measuring complex organic shapes. Discrete features of interest were measured on 5 cadaveric hearts and their corresponding 3D printed models. Features were as follows: sinotubular junction perimeter (3 models), aortic annulus maximum and minimum diameter (3 models), mitral valve maximum and minimum diameter (2 models), and mitral valve annulus perimeter (2 models). Diameter measurements were made with sliding digital calipers (Neiko, Taiwan) and recorded to the nearest tenth of a millimeter; perimeter measurements were made with a bendable wire marked at length with a needle driver and subsequently measured against a ruler to the nearest tenth of a millimeter (Fig. 1). Four experimenters measured each feature (cardiothoracic surgeon JH, interventional cardiologist MM, radiologist with cardiovascular imaging training BR, and cardiac anatomist DL).

Fig. 1
figure 1

Physical measurements of 3D printed cardiac models and cadaveric hearts. a-c Three examples of cadaveric hearts and their corresponding 3D printed models, used for measuring features of the aortic valve (a, b) and the mitral valve (b, c). d-e Measurement of perimeters was accomplished using a malleable metal wire which was marked with a needle driver and then straightened and measured against a ruler. f Organic shapes present challenges to measurement with linear rulers or calipers

Full size image

In the second set of experiments, we explored two different techniques for measuring discrete features on 3D models - manual measurements made with a digital caliper and measurements made on digital photographs. We identified four discrete features of interest that were accessible by calipers (Fig. 2) and obtained a total of 7 linear measurements from these 4 features (wall thickness, maximum and minimum diameter of openings in the model). Manual and digital photographic measurements were made by a single experimenter (M.O.) blinded to actual feature dimensions. Digital photographs were acquired by placing a marker of known length parallel to and at the same height as the feature in the model and then capturing a digital photograph using an iPhone X with the stock camera application (12Megapixels and f/1.8 aperture). The photographs were imported into Image J digital measurement software (National Institutes of Health, Bethesda, Maryland) and were calibrated by performing a ratio between the pixel count and known physical length of the marker. The linear measurement tool in image J was then used to repeat the same 7 measurements that were made with calipers.

Fig. 2
figure 2

Digital photographic measurement technique. a-d Four discrete 3D model features (a Feature 1, b Feature 2, c Feature 3, and d Feature 4) were chosen for measurement, based on their accessibility on an exterior surface and the fact that they had distinguishing features that would allow for repeat measurements. All digital photographs were calibrated before measurement. This was achieved by placing a marker of known length at the same height and angle as the feature of interest before photographing the model. Once the photograph was imported into an image analysis software (in this case, ImageJ), the length of the known marker was measured in pixels, and a conversion factor was calculated for pixels to millimeters. Of note, insufficient illumination for feature 3 (panel c) caused blending of edges, most pronounced along the inferior aspect of the photograph. This affected accuracy of measurements for this feature (see Table 2)

Full size image

Measurement of discrete features from an STL file

We investigated the feasibility of making discrete measurements on an STL file to serve as the standard against which to compare physical and digital photographic measurements of 3D printed models. Measurements of discrete features on the original model STL file were made in Autodesk 3D Studio Max 2018 (Autodesk, San Rafael, CA), a professional 3D computer graphics program for digital vertex to vertex measurement. Digital photographs with embedded measurements were exported from Image J, imported into Autodesk 3D Studio Max, and overlaid on the STL file for reference to ensure that the STL file features were measured at the same location as the digital photographic measurements. A sample of the output produced by 3D Studio Max along with the steps is presented in Fig. 3.

Fig. 3
figure 3

Vertex to vertex edge STL file measurement. a Digital photograph exported from Image J with measurement included for landmark reference. b The digital photograph was imported into Autodesk 3D Studio Max and overlaid on the STL file (rendered in red). The STL file was carefully aligned so that the meshed feature of interest aligned with the photograph. c Vertex to vertex edge measurement was made of the meshed STL file

Full size image

Comparison of discrete feature measurements across measurements styles

We did not design or power this study to formally compare differences between the measurement techniques described above, and therefore no formal statistical analysis was performed. Instead, the data is presented in raw table format and expressed as standard deviation or percent error where applicable. For digital caliper and digital photographic measurements compared against STL file measurements, the percent error was defined as the absolute value of (STL measurement – experimental measurement) divided by STL measurement and multiplied by 100.

Photogrammetry and surface scanning of 3D printed models

We explored two methods for optical imaging of 3D model surfaces - photogrammetry and surface scanning. We chose a total of 3 models (printed in gray, white and clear resin, respectively) for optical imaging experiments (Fig. 4). For photogrammetry, a series of photographs were taken at different angles and orientations (a total of 40 on average) with an iPhone X with the stock camera application (12MP and f/1.8 aperture). The auto white-balance/focus was kept on to maintain consistency between all images taken. Images were taken at a constant distance from the object of interest. Images were processed using VisualSFM (a GUI application for 3D reconstruction created by Changchang Wu, available at http://ccwu.me/vsfm/) to create a computed set of three-dimensional coordinates needed for photogrammetry. For surface scanning, the same 3 models were scanned using a structured light Artec Space Spider scanner with a resolution up to 0.1 mm (Artec, Santa Clara, CA).

Fig. 4
figure 4

Surface scanning success was dependent on model surface characteristics. Models printed in clear (a), white (b) and gray (c) resin were used for surface scan testing. Clear and white models were not successfully scanned, due to challenges related to scanning of transparent (clear model) and reflective (white model) surfaces. The model printed in the gray resin was successfully scanned, with the resultant surface model shown in (d). However, fine model features were degraded in the surface scan, such as the transcatheter aortic valve depicted in (e)

Full size image

CT scanning of 3D printed models

3D printed heart models were CT scanned and compared to patient DICOM datasets. A unique heart holder was created for each model to ensure that the model was scanned in the same spatial orientation within the CT scanner as the patient’s heart in the original scan. The first step in creating the heart holder was segmenting the CT table from the patient DICOM at the time of heart segmentation. Segmented CT tables were imported into 3-Matic as STL files, as described above. The table was cloned and offset in the Y-axis by the table thickness (measured on the DICOM images using standard picture archiving and communication system [PACS] measurement tools) so that the cloned table sat perfectly flush with the CT table. Next, a set of cylinders were created which transected both the heart and the cloned table. The cylinders were labeled 1–4 and were notched to ensure appropriate orientation with the table piece. Holes with 0.1 mm offset were created by Boolean subtracting the cylinders from the heart model as well as the table piece (Fig. 5). One heart model and accompanying heart holder was printed on both a Form2 printer as well as a Connex3 Objet 350 printer (Stratasys, Rehovot, Israel) to allow for comparison of the technique across printers.

Fig. 5
figure 5

3D model-specific heart holder for optimal positioning during CT scanning. a The CT table was segmented and exported as an STL file at the time of model segmentation and creation. b A custom table adaptor piece created in CAD software was designed to sit flush with the CT Table. c The corresponding heart model was imported into CAD software, and 4 cylinders were created and positioned so that they intersected the model and the table adaptor piece. d Holes fitted to the cylinders with 0.1 mm offset were created in the model, and the table adaptor piece and the cylinders were shortened to the appropriate length. e Example of finished model-specific heart holder in CAD software. f 3D printed heart and heart holder positioned in the CT gantry in the same orientation that the patient was scanned in (feet first)

Full size image

Heart models (in their respective heart holders) were scanned on the same Revolution Scanner (GE, Waukesha WI) that the initial CT images were acquired on with the following parameters: axial mode with 0.625 mm slice thickness, at 120 kVP, 375 mA, reconstructed using GE “Standard” convolution kernel. Pixel spacing was 0.488 × 0.488 mm with image matrix 512 × 512.

Alignment of patient DICOM and 3D model DICOM datasets

We aligned the patient DICOM dataset and the 3D model DICOM dataset in Osirix MD (Pixmeo, Geneva, Switzerland) by creating a fusion image of the two DICOM files. Detailed instructions for making fusion images are available online (https://en.wikibooks.org/wiki/Online_OsiriX_Documentation/Making_fusion_images).

The heart holder positioned the model in the same spatial coordinates and alignment as the patient’s heart on the initial scan, and thus there was no requirement for non-rigid deformation. Minimal differences in X, Y and Z coordinates between the two DICOM datasets were corrected using the pan and rotate tools in Osirix.

Results

Physical measurements of discrete 3D printed model features

Physical measurements of discrete model features presented some challenges (Fig. 1). Nonlinear measurements such as perimeters could not be accomplished with straight rulers or digital calipers. Instead, we used a thin malleable wire to mark the perimeter length and then straightened and measured the wire. The standard deviation (SD) between observers for perimeter measurements was approximately twice as large as the SD for linear measurements made with calipers (SD range of ±2.1 mm to ±9.2 mm for perimeter measurements, compared to SD range of ±0.8 mm to ±4.4 mm for linear measurements with calipers) (Table 1). Physical measurements were not possible for features that were internal to the model or recessed beyond the reach of calipers (e.g., the aortic annulus in a model that includes portions of the ascending aorta and the left ventricular outflow tract).

Table 1 Physical measurements of cadaveric hearts and 3D printed models
Full size table

Digital photography-based measurements of discrete 3D printed model features

Digital photography-based measurements could only be made for external features and were constrained by the requirement that a measurement marker be positioned in the same spatial plane as the feature of interest. We found that appropriate lighting was critical for measurement accuracy. For example, improper illumination in the digital photograph of feature 3 led to blending of edges and inaccurate measurements for this feature when compared to the reference STL (Fig. 2c, Table 2).

Table 2 Measurements of discrete 3D model features
Full size table

Surface scanning and photogrammetry of 3D printed models

3D printed model surfaces were variably challenging to capture with structured light surface scanning and photogrammetry approaches. The clear resin model was not amenable to surface scanning because the emitted light from the 3D scanner passed through the model and was not reflected onto the scanner’s registration sensor; therefore, no points were registered on the surface of the object. The white resin model had a reflective surface which caused light to scatter and resulted in a sparse and inaccurate point cloud. Surface scanning of the gray resin model was largely successful, but some fine features were blurred (Fig. 4). For photogrammetry approaches, the calculated point cloud was dependent on the number of images obtained, the percentage of overlap between images, appropriate centering of the object of interest in the image frame, and the consistency in lighting (no hard shadows). As with surface scanning, almost no surface points were collected from the model printed in clear resin. For the white resin and gray resin models, the point cloud was sparse, with more useable data obtained from the gray model relative to the white. In the end, however, the point cloud was not sufficient for comparison to the model STL files.

CT scanning of 3D printed models

CT scanning of 3D models was relatively simple to perform but required the creation of a model-specific holder to minimize image registration challenges. With the heart holder, DICOM image stacks of the 3D model and the source DICOM images could be aligned using simple pan and rotate tools in Osirix MD. A qualitative check of the model contours relative to the anatomy of interest was accomplished by scrolling through the dataset (Fig. 6). Quantitative measurements were also made on the final 3D model DICOM images for critical features (Fig. 7).

Fig. 6
figure 6

3D heart models CT scanned using a custom heart holder. Heart models were CT scanned for an end-to-end verification of the final 3D model back to the original DICOM images. A custom heart holder was created from source DICOM images to ensure that the model was imaged in the same orientation and spatial position as the original anatomy. Top row: (left) 3D printed heart and 3D printed holder, created on a Form2 printer, (middle) DICOM image from the CT scan of the 3D model, and (right) DICOM image from the original patient study, with the 3D model DICOM overlaid for comparison. (Of note, this model was hollowed to save material, and the specular foci within the center of the model represent internal support structures.) Bottom row: same as above, but in this case, the heart and holder were printed on a Connex 3 Objet 350 printer. This strategy allowed for verification of the model creation pipeline for 2 separate printing techniques. (Of note, this model is filled with internal support material, which on this printer is laid down uniformly, in contrast to the strut-like supports of the Form2 model)

Full size image
Fig. 7
figure 7

Quantitative measurements of a 3D printed model by CT. Measurements of critical features (e.g., left ventricular outflow tract just inferior to aortic annulus) were made on final 3D printed model DICOM images (a, in yellow) and checked against the source patient DICOM data (b)

Full size image

Discussion

A quality assurance (QA) program takes years to establish and build and is comprised of numerous components, each with its inherent intricacies. Here, we explored one component of a QA program - verification of part accuracy. Verification requires measurement of final model dimensions and comparison of those measurements to the physical geometry of the patient’s anatomy. In cases of internal anatomy, the patient’s medical imaging often serves as a proxy for the actual anatomy. The substitution of imaging data for ground truth inherently introduces some degree of error into the verification process, which is mitigated by established radiology department QC/QA practices for ensuring image accuracy. It is important to remember that the quality of the imaging always constrains the quality of the model.

Unfortunately, each step in the process of 3D-printing an anatomical model can introduce error (Fig. 8). Because of this, it is necessary to minimize and continue to monitor errors created by the fabrication process to ensure safe and impactful use of these models in a clinical setting. Checkpoints in the design process are often used for intermediate verification checks. For example, overlaying a final STL on the original DICOM image or comparing a point cloud obtained from surface scanning of the final model to the STL file are examples of these types of checks. The challenge is that none of these methods fully verify the model, since they depend on derivatives of the patient’s anatomy and are, in of themselves, subject to the introduction of errors.

Fig. 8
figure 8

Infographic overview of 3D printed model / part verification

Full size image

There are multiple strategies for part measurement, including manual measurements, digital photographic measurements, surface imaging strategies (3D surface scanning and photogrammetry) and CT scanning. We sampled various options as part of feasibility testing in building our quality assurance program. It is important to emphasize that the data presented here is not a comprehensive assessment of the accuracy of these measurements in of themselves (which has been done elsewhere). Instead, the goal was to discover the strengths and weaknesses of each technology in our hands to find the best fit for clinical heart model verification in our hospital-based program (Table 3).

Table 3 Different measurement techniques have advantages and disadvantages
Full size table

Physical measurements presented early challenges, as critical non-linear cardiac anatomy such as the aortic and mitral valve annulus could not be measured with calipers or a ruler. We used a malleable wire to mark perimeter lengths but found the inter-observer variability to be more than twice that of measurements made with calipers. A portion of that variability is likely technical, as small bends in the wire that were not perfectly smoothed out introduced error. More importantly, however, it required personnel with extensive anatomical training to make the measurements, and concepts such as how to measure the mitral valve annulus were dependent on the background and training of the person performing the measurements. Digital photographic measurements could potentially mitigate the challenges of nonlinear measurements but were still limited to being utilized for external structures, and the requirements for marker placement and camera set-up were non-trivial.

We investigated two methods for measuring the surface of models- structured light surface scanning and photogrammetry. We learned that model material color heavily influenced the success of these methods in our hands; we were unable to obtain usable datasets for clear and white resin models. Marking the models with talcum powder or a developer spray may have helped with scanning, but we were concerned that those options would have affected the overall part geometry, and for that reason, they were not used in this study. While there were multiple additional steps we could have taken to improve results with these techniques, including testing different types of scanners (for surface scanning) or improving image acquisition techniques (for photogrammetry), we determined that the time and staff training needed was outside of the scope of what was feasible for our program. These techniques are likely to perform well in hospital-based programs that have optimized the process and have staff who are experienced with these techniques.

We found that CT scanning circumvented many of the challenges faced by other measurement techniques. Perhaps most importantly, CT scanning of the 3D printed model allowed end-to-end verification of the entire 3D printing workflow by comparing the final 3D printed model directly to the patient’s anatomy in the original DICOM images (Fig. 8). Overlaid 3D printed model and patient DICOM images were easily scrolled through by a radiologist, allowing for overall qualitative confirmation of accuracy and additional quantitative measurements of critical parts of the model at the discretion of the radiologist (e.g., aortic annulus measurements in a model intended for pre-procedural planning for transcatheter aortic valve replacement, Fig. 7). An additional benefit was that our radiology staff have baseline expertise in CT technology, and therefore required no additional training to incorporate model verification into their daily workflow.

One potential challenge of using CT to verify 3D models is the requirement for DICOM image registration. Fiducial markers or distinct landmarks are often necessary for image registration, but these require foresight to add to the patient before a scan. To minimize the challenges of landmark registration, we used the CT table itself as a fiducial marker. This strategy allowed for rigid image registration with a minimum of time and effort (less than 5 min, on average), but at the cost of the design and printing time that came with the creation of the heart holder. A portion of the design work may be automated going forward.

Limitations

This work is limited by the small sample size and lack of statistical analysis. We did not test contact-dependent methods of surface analysis, such as a coordinate measurement machine. We did not troubleshoot initial challenges with surface scanning and photogrammetry approaches, and we do not present the steps required for registration of surface point cloud data with STL files needed for model verification with these strategies. The model-specific heart holder presented here for CT scanning is a prototype, and has not been tested with flexible models or anatomy outside of cardiac models.. Finally, while the methods described are detailed with respect to cardiac models, it is very likely that modifications to ensure accuracy would be necessary for other clinically appropriate 3D printed models. [36]

Conclusion

Verification of 3D printed model accuracy is an essential, but nontrivial, component of a quality assurance program. Numerous methods for measuring 3D printed model dimensions exist, each with benefits and drawbacks. The choice of verification method should be made after considering the unique clinical requirements of the 3D model being verified, as well as the expertise and preferences of program staff. Specific recommendations to the reader on the number or frequency of models to verify for a given workflow is beyond our purview; instead, readers should look to existing regulatory guidance [37, 38].

References

  1. Matsumoto JS, Morris JM, Foley TA, Williamson EE, Nesberg LE, Vrtiska TJ. Three-dimensional physical modeling : applications and experience at Mayo Clinic. RadioGraphics. 2015;35:1989–2006.

    Article  Google Scholar 

  2. Mitsouras D, Liacouras P, Imanzadeh A, Giannopoulos AA, Cai T, Kumamaru KK, et al. Medical 3D printing for the radiologist. Radiographics. 2015;35(7):1965–88.

    Article  Google Scholar 

  3. Bastawrous S, Wake N, Levin D, Ripley B. Principles of three-dimensional printing and clinical applications within the abdomen and pelvis. Abdom Radiol. 2018;43:2809–22.

  4. Ripley B, Levin D, Kelil T, Hermsen JL, Kim S, Maki JH, et al. 3D printing from MRI data: harnessing strengths and minimizing weaknesses. J Magn Reson Imaging. 2017;45(3):635–45.

  5. Hermsen JL, Burke TM, Seslar SP, Owens DS, Ripley BA, Mokadam NA, et al. Scan, plan, print, practice, perform: development and use of a patient-specific 3-dimensional printed model in adult cardiac surgery. J Thorac Cardiovasc Surg. 2017;153(1):132–40.

  6. Hsieh TY, Cervenka B, Dedhia R, Strong EB, Steele T. Assessment of a patient-specific, 3-dimensionally printed endoscopic sinus and skull base surgical model. JAMA Otolaryngol - Head Neck Surg. 2018;144(7):574–9.

    Article  Google Scholar 

  7. Park HJ, Wang C, Choi KH, Kim HN. Use of a life-size three-dimensional-printed spine model for pedicle screw instrumentation training. J Orthop Surg Res. 2018;13(1):86.

    Article  Google Scholar 

  8. Ripley B, Kelil T, Cheezum MK, Goncalves A, Di Carli MF, Rybicki FJ, et al. 3D printing based on cardiac CT assists anatomic visualization prior to transcatheter aortic valve replacement. J Cardiovasc Comput Tomogr. 2016;10(1):28–36 Elsevier Ltd.

    Article  Google Scholar 

  9. Pellegrino PL, Fassini G, Di Biase M, Tondo C. Left atrial appendage closure guided by 3D printed cardiac reconstruction: emerging directions and future trends. J Cardiovasc Electrophysiol. 2016;27(6):768–71.

  10. Farooqi KM, Saeed O, Zaidi A, Sanz J, Nielsen JC, Hsu DT, et al. 3D printing to guide ventricular assist device placement in adults with congenital heart disease and heart failure. JACC Hear Fail Elsevier. 2016;4(4):301–11.

    Article  Google Scholar 

  11. Ciobotaru V, Combes N, Martin CA, Marijon E, Maupas E, Bortone A, et al. Left atrial appendage occlusion simulation based on three-dimensional printing: new insights into outcome and technique. EuroIntervention. 2018;14(2):176–84.

    Article  Google Scholar 

  12. Qian Z, Wang K, Liu S, Zhou X, Rajagopal V, Meduri C, et al. Quantitative prediction of paravalvular leak in Transcatheter aortic valve replacement based on tissue-mimicking 3D printing. JACC Cardiovasc Imaging. 2017;10(7):719–31.

    Article  Google Scholar 

  13. Hosny A, Dilley JD, Kelil T, Mathur M, Dean MN, Weaver JC, et al. Pre-procedural fit-testing of TAVR valves using parametric modeling and 3D printing. J Cardiovasc Comput Tomogr. 2018;13(1):21–30.

  14. Park JW, Kang HG, Lim KM, Park DW, Kim JH, Kim HS. Bone tumor resection guide using three-dimensional printing for limb salvage surgery. J Surg Oncol. 2018;118 (6):898–905.

  15. Sys G, Eykens H, Lenaerts G, Shumelinsky F, Robbrecht C, Poffyn B. Accuracy assessment of surgical planning and three-dimensional-printed patient-specific guides for orthopaedic osteotomies. J Eng Med. 2017;231(6):499–508.

    Article  Google Scholar 

  16. Beltrami G, Ristori G, Scoccianti G, Tamburini A, Capanna R, Campanacci D, et al. Latissimus dorsi rotational flap combined with a custom-made scapular prosthesis after oncological surgical resection: a report of two patients. BMC Cancer. 2018;18(1):1003.

    Article  Google Scholar 

  17. Girolami M, Boriani S, Bandiera S, Barbanti-Bródano G, Ghermandi R, Terzi S, et al. Biomimetic 3D-printed custom-made prosthesis for anterior column reconstruction in the thoracolumbar spine: a tailored option following en bloc resection for spinal tumors Preliminary results on a case-series of 13 patients. Eur Spine J. 2018;1:3.

  18. Zhang Y, Zhang L, Sun R, Jia Y, Chen X, Liu Y, et al. A new 3D printed titanium metal trabecular bone reconstruction system for early osteonecrosis of the femoral head. Medicine (Baltimore). 2018;97(26).

  19. Ackland D, Robinson D, Vee P, Lee S, Dimitroulis G. Design and clinical outcome of a novel 3D-printed prosthetic joint replacement for the human temporomandibular joint. Clin Biomech. 2018;56:52–60.

  20. Dekker TJ, Steele JR, Federer AE, Hamid KS, Adams SB. Use of patient-specific 3D-printed titanium implants for complex foot and ankle limb salvage, deformity correction, and arthrodesis procedures. Foot Ankle Int. 2018;39(8):916–21.

    Article  Google Scholar 

  21. Morrison RJ, Hollister SJ, Niedner MF, Mahani MG, Park AH, Mehta DK, et al. Mitigation of tracheobronchomalacia with 3D-printed personalized medical devices in pediatric patients. Sci Transl Med. 2015;7(285):285ra64.

    Article  Google Scholar 

  22. Dillon C, Breeden IIIW, Clements J, Cody D, Gress D, Kanal K, et al. Computed tomography quality control manual: American College of Radiology; 2017. p. 2017. https://www.acr.org/-/media/ACR/NOINDEX/QC-Manuals/CT_QCManual.pdf

  23. Price R, Allison J, Clarke G, Dennis M, Reeve D. 2015 American College of Radiology MRI quality control manual: American College of Radiology; 2015. https://shop.acr.org/Default.aspx?TabID=55&ProductId=731499117

  24. Broder JC, Cameron SF, Korn WT, Baccei SJ. Creating a radiology quality and safety program: principles and pitfalls. RadioGraphics. 2018;38(6):1786–98.

    Article  Google Scholar 

  25. George E, Liacouras P, Rybicki FJ, Mitsouras D. Measuring and establishing the accuracy and reproducibility of 3D printed medical models. RadioGraphics. 2017;37(5):1424–50.

    Article  Google Scholar 

  26. Leng S, McGee K, Morris J, Alexander A, Kuhlmann J, Vrieze T, et al. Anatomic modeling using 3D printing: quality assurance and optimization. 3D Print Med. 3D Print Med. 2017;3(1):6.

    Article  Google Scholar 

  27. Morrison RJ, Kashlan KN, Flanangan CL, Wright JK, Green GE, Hollister SJ, et al. Regulatory considerations in the design and manufacturing of implantable 3D-printed medical devices. Clin Transl Sci. 2015;8(5):594–600.

    Article  Google Scholar 

  28. Carew RM, Morgan RM, Rando C. A Preliminary Investigation into the accuracy of 3D modeling and 3D printing in forensic anthropology evidence reconstruction. J Forensic Sci. 2018;64(2):342–52.

  29. Brouwers L, Teutelink A, van Tilborg FAJB, de Jongh MAC, Lansink KWW, Bemelman M. Validation study of 3D-printed anatomical models using 2 PLA printers for preoperative planning in trauma surgery, a human cadaver study. Eur J Trauma Emerg Surg. 2018;0(0):1–8 Springer Berlin Heidelberg.

    Google Scholar 

  30. Rebong RE, Stewart KT, Utreja A, Ghoneima AA. Accuracy of three-dimensional dental resin models created by fused deposition modeling, stereolithography, and Polyjet prototype technologies: A comparative study. Angle Orthod. 2018;88(3):363–9.

    Article  Google Scholar 

  31. Jeong Y-G, Lee W-S, Lee K-B. Accuracy evaluation of dental models manufactured by CAD/CAM milling method and 3D printing method. J Adv Prosthodont. 2018;10(3):245.

    Article  Google Scholar 

  32. Kim SY, Shin YS, Jung HD, Hwang CJ, Baik HS, Cha JY. Precision and trueness of dental models manufactured with different 3-dimensional printing techniques. Am J Orthod Dentofac Orthop. 2018;153(1):144–53 American Association of Orthodontists.

    Article  Google Scholar 

  33. Kernen F, Benic GI, Payer M, Schär A, Müller-Gerbl M, Filippi A, et al. Accuracy of three-dimensional printed templates for guided implant placement based on matching a surface scan with CBCT. Clin Implant Dent Relat Res. 2016;18(4):762–8.

    Article  Google Scholar 

  34. Revilla-León M, Ceballos L, Martínez-Klemm I, Özcan M. Discrepancy of complete-arch titanium frameworks manufactured using selective laser melting and electron beam melting additive manufacturing technologies. J Prosthet Dent. 2018;120(6):942–47.

  35. Liang X, Liao W, Cai H, Jiang S, Chen S. 3D-printed artificial teeth: accuracy and application in root canal therapy. J Biomed Nanotechnol. 2018;14(8):1477–85.

    Article  CAS  Google Scholar 

  36. Chepelev L, Wake N, Ryan J, Althobaity W, Gupta A, Arribas E. Radiological Society of North America (RSNA) 3D printing special interest group (SIG): guidelines for medical 3D printing and appropriateness for clinical scenarios. 3D Print Med. 2018;4(11):1–38.

  37. Di Prima M, Coburn J, Hwang D, Kelly J, Khairuzzaman A, Ricles L. Additively manufactured medical products – the FDA perspective. 3D Print Med. 2015;2(1):1.

    Article  Google Scholar 

  38. Ricles LM, Coburn JC, Di Prima M, Oh SS. Regulating 3D-printed medical products. Sci Transl Med. 2018;10(461):eaan6521.

    Article  Google Scholar 

Download references

Acknowledgements

The article processing charge for this publication has been funded generously by an unrestricted educational grant from Stratasys.

Funding

There was no funding source for the research reported.

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Author information

Authors and Affiliations

  1. Institute for Simulation and Training, University of Central Florida, Orlando, FL, USA

    Mohammad Odeh, Jim Inziello, Fluvio Lobo Fenoglietto & Jack Stubbs

  2. Department of Medicine, Division of Cardiology, University of Washington School of Medicine, Seattle, WA, USA

    Dmitry Levin

  3. Structural Interventional Cardiology, Virginia Mason Hospital, Edmonds, WA, USA

    Moses Mathur

  4. Department of Surgery, Division of Cardiothoracic Surgery, University of Wisconsin School of Medicine, Madison, WI, USA

    Joshua Hermsen

  5. VA Puget Sound Health Care System, Seattle, WA, USA

    Beth Ripley

  6. Department of Radiology, University of Washington School of Medicine, Seattle, WA, USA

    Beth Ripley

Authors
  1. Mohammad Odeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dmitry Levin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jim Inziello
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fluvio Lobo Fenoglietto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Moses Mathur
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joshua Hermsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jack Stubbs
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Beth Ripley
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MO, JI, FLF and JS designed and performed experiments related to optical scanning, digital photographic measurements, and measurement of STL files and wrote portions of the paper related to these experiments. DL, MM, JH and BR designed and performed experiments comparing 3D printed models and cadaveric hearts, and wrote portions of the paper related to these experiments. DL and BR designed and performed experiments related to CT scanning of models, including design of a custom 3D model heart holder, and segmented and printed all 3D models used in the study. BR wrote the manuscript and created the figures. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Beth Ripley.

Ethics declarations

Ethics approval and consent to participate

IRB-approved study.

Consent for publication

N/A.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Odeh, M., Levin, D., Inziello, J. et al. Methods for verification of 3D printed anatomic model accuracy using cardiac models as an example. 3D Print Med 5, 6 (2019). https://doi.org/10.1186/s41205-019-0043-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41205-019-0043-1

Keywords

3D Printing in Medicine

ISSN: 2365-6271

Contact us