Fast and accurate distal locking of interlocked intramedullary nails using computer-vision and a 3D printed device

Abstract

Introduction

Distal locking is a challenging and time-consuming step in interlocked intramedullary nailing of long bone fractures. Current methods have limitations in terms of simplicity, universality, accuracy, speed, and safety. We propose a novel device and software for distal locking using computer vision.

Methods and materials

The device consists of an universal ancillary clamp, a telescopic arm, a viewfinder clamp, and a radio-opaque cross. The software uses a camera photo from the C-arm intensifier and adjusts for geometric projection deformities. The software employs edge detection, Hough transform, perspective interpolation, and vector calculation algorithms to locate the distal hole center. The device and software were designed, manufactured, and tested using 3D CAD, FEM, DRR, and performance testing on phantom bones.

Results

The device and software showed high accuracy and precision of 98.7% and 99.2% respectively in locating the distal hole center and calculating the correctional vector. The device and software also showed high success ratio in drilling the hole and inserting the screw. The device and software reduced the radiation exposure for the surgeon and the patient. The success ratio of the device and software was validated by the physical testing, which simulated the real clinical scenario of distal locking. The radiation exposure was as low as 5 s with a radiation dose of 0.2mSv, drastically reducing radiation exposure during distal locking.

Discussion

Our device and software have several advantages over other distal locking methods, such as simplicity, universality, accuracy, speed, and safety. Our device and software also have some disadvantages, such as reliability and legislation. Our device and software can be compared with other distal locking methods based on these criteria. Our device and software have some limitations and challenges that need to be addressed in the future, such as clinical validation, and regulatory approval.

Conclusion

The device showed promising results in terms of low-cost, reusability, low radiation exposure, high accuracy, fast distal locking, high stiffness, and adaptability. The device has several advantages over other distal locking techniques, such as free-hand technique, mechanical aiming devices, electromagnetic navigation systems, and computer-assisted systems. We believe that our device and software have the potential to revolutionize the distal locking technique and to improve the outcomes and quality of life of the patients with long bone fractures.

Introduction

Interlocked intramedullary nails are widely used for the treatment of long bone fractures. They provide stable fixation and allow early mobilization of the patients [1,2,3,4]. However, one of the main challenges of this technique is the distal locking, which involves drilling a hole through the bone and the nail to insert a screw that prevents the nail from rotating or migrating. Distal locking is often time-consuming, technically demanding, and associated with high radiation exposure for the surgeon and the patient [5,6,7,8].

Several methods have been proposed to facilitate distal locking, such as free-hand technique [9], mechanical aiming devices [10], electromagnetic tracking systems [5, 9], and image-guided systems [5, 11, 12]. However, each method has its own limitations and drawbacks. For example, free-hand technique requires skill and expose to excess radiation for both the patient and medical staff, mechanical aiming devices are often bulky and expensive [10], electromagnetic tracking systems are sensitive to interference and calibration errors, and image-guided systems require complex hardware and software integration [5, 9].

In this paper, we propose a novel device and software for distal locking in interlocked intramedullary nails using computer vision. The device consists of an universal ancillary clamp, a telescopic arm, a viewfinder clamp, and a radio-opaque cross. The software uses a camera photo from the C-arm intensifier and adjusts for geometric projection deformities. The software employs edge detection, Hough transform, perspective interpolation, and vector calculation algorithms to locate the distal hole center. The device and software were designed, manufactured, and tested using FEA, K-folds DRR testing, in addition to radiation exposure measurements and pre-clinical performance testing.

Methods and materials

Design principles

The design principles of the device and the mobile application are based on the following assumptions and requirements: The device should be compatible with most intramedullary nails ancillary systems and lengths, should be compatible with circle shaped holes (excluding oblong holes), should be easy to mount, adjust, and remove, be radiolucent, sterilizable and reusable.

The mobile application should use a camera photo from the C-arm intensifier to avoid additional C-arm installation, manipulation, or provider lockdown, it should adjust for geometric projection deformities from the taken photo, also, it should accommodate a side view featuring a single non-perfect round image of the distal hole employing state-of-the-art computer vision algorithms to detect the centroids of the distal hole and compute the correctional vector from the viewfinder to the distal hole center, Additionally the software should be able to compute the correctional vector in sub-second speeds in mobile devices to allow quick per-operative decisions, Lastly, the mobile application should be user-friendly and intuitive.

Aiming device design

The aiming device consists of four main components: an universal ancillary clamp, a telescopic arm, a viewfinder clamp, and a radio-opaque cross. Schematic diagram of the device are Shown in Figs. 1 and 2.

• The universal ancillary clamp is a hollow rectangular shape with two orthogonal screws that can adapt to all intramedullary nails ancillary systems. It follows the Smith & Nephew standard and can be attached to the proximal end of the nail. It serves as the base of the device and provides stability and alignment.

• The telescopic arm is a rectangular rod that can extend or retract to adapt to all intramedullary nail lengths (humerus, tibia, and femur) with a screw clamping mechanism. It serves as the X stage of the device and allows the adjustment of the distance from the ancillary clamp to the viewfinder clamp.

• The viewfinder clamp is a delta-O design with a clamping screw that can attach to the telescopic arm. It serves as the Y stage of the device and allows the adjustment of the height and angle of the viewfinder. The viewfinder is a circular hole that guides the surgeon to align the drill with the distal hole of the nail.

• The radio-opaque cross is a 10 mm by 10 mm cross that is embedded in the viewfinder clamp. It serves as a reference point for the mobile application to compute the correctional vector from the viewfinder to the distal hole center. The radio-opaque cross is visible on the fluoroscopic image and can be detected by the computer vision algorithms.

Fig. 1

Schematic view the mechanical aiming device

Fig. 2

3D rendering that showcase the mechanical aiming device mounted to the nail ancillary

Software development

Software algorithm and Flowchart are shown in Figs. 3 and 4. The software was developed as a mobile application for Android devices, using Java programming language and OpenCV library. The software uses a camera photo from the C-arm intensifier as the input, and performs the following steps:

• Projection unwrapping: The software converts the photo into a grayscale image and applies Otsu thresholding [13] and Canny edge detector [14] to extract the edges. The software then uses Hough line algorithm to detect the four corners of the C-arm intensifier screen, and uses 3 × 3 matrix perspective interpolation [15] to unwrap the projection into a rectangular image.

• Centroids detection: The software detects the centroids of the nail, the distal hole, and the radio-opaque cross. The software uses Hough polygon algorithm [16] to detect the nail, and computes its centroid as the average of the vertices. The software uses the previously detected ellipse to compute the centroid of the distal hole. The software uses Hough line algorithm [17] to detect the radio-opaque cross, and computes its centroid as the intersection of the lines.

• Correctional vector: The software computes the correctional vector from the viewfinder to the distal hole center, using the centroids of the nail, the distal hole, and the radio-opaque cross. The software also takes into account the known distance from the viewfinder to the radio-opaque cross, and the known shape and geometry of the radio-opaque cross, to calibrate the vector. The software then displays the vector on the screen, along with the distance and angle measurements, to guide the drilling process.

Fig. 3

Steps of the detection algorithm

Fig. 4

Flowchart of the software and the algorithm

Projection unwrapping

The first step of the software is to correct the geometric deformities from the camera taken photo of the C-arm intensifier. The deformities may occur due to the perspective distortion, the lens distortion, or the rotation of the image. To correct these deformities, we use the following steps:

• We apply the camera calibration algorithm to the corrected image, using the known intrinsic and extrinsic parameters of the camera. We use the Zhang’s method [18] to estimate these parameters from a set of images of a calibration pattern, such as a chessboard. The camera calibration algorithm removes the lens distortion and the rotation of the image, resulting in a rectified image that is similar to the original fluoroscopic image.

• We detect the four corners of the C-arm intensifier screen using the Harris corner detector algorithm [19]. We assume that the screen is rectangular and has a known aspect ratio, such as 4:3 or 16:9.

• We compute the homography matrix that maps the detected corners to the ideal corners of a rectangular image with the same aspect ratio. We use the RANSAC algorithm [20] to find the optimal homography matrix that minimizes the reprojection error.

• We apply the homography matrix [15] to the camera taken photo, resulting in a corrected image that is aligned with the C-arm intensifier screen. We crop the image to the screen boundaries, removing any irrelevant background.

Centroids detection

The second step of the software is to detect the centroids of the distal hole and the radio-opaque cross from the rectified image. The centroids are the points that correspond to the center of the drill bit and the center of the distal hole, respectively. To detect these centroids, we use the following steps:

• We apply a thresholding algorithm to the rectified image, converting it to a binary image. We use the Otsu’s method [13] to find the optimal threshold value that separates the foreground (the nail and the cross) from the background (the bone and the soft tissue).

• We apply a morphological opening operation [21] to the binary image, removing any small noise or artifacts. We use a circular structuring element with a radius of 1 pixel.

• We apply the Canny edge detector algorithm [14] to the binary image, finding the edges of the nail and the cross. We use the automatic threshold selection method to find the optimal values for the low and high thresholds of the Canny algorithm.

• We apply the Hough line transform [17] algorithm to the edge image, finding the lines that correspond to the sides of the nail and the cross. We use the probabilistic Hough line transform to find the line segments with a minimum length of 10 pixels and a maximum gap of 5 pixels.

• We apply the Hough ellipse transform algorithm [22] to the edge image, finding the ellipses that correspond to the distal hole. We use the Hough gradient method to find the ellipses with a minimum radius of 5 pixels and a maximum radius of 15 pixels.

• We compute the centroids of the detected ellipses by finding the average of the pixels that belong to each circle. We label the centroids as C1 and C2, where C1 is the centroid of the distal hole and C2 is the centroid of the cross.

Correctional vector

The third step of the software is to calculate the correctional vector from the viewfinder to the distal hole center. The correctional vector is the vector that indicates the direction and magnitude of the drill bit adjustment. To calculate the correctional vector, we use the following steps:

• We compute the vector V1 from the centroid C1 to the centroid C2. This vector represents the current position and orientation of the drill bit relative to the distal hole.

• We compute the vector V2 from the centroid C1 to the point P, where P is the intersection of the nail axis and the plane that contains the distal hole. This vector represents the desired position and orientation of the drill bit relative to the distal hole.

• We compute the correctional vector Vc as the difference between the vector V2 and the vector V1, then we subtract the offset from the center of the viewfinder to the center of the cross. This vector represents the required adjustment of the drill bit to align with the distal hole.

Calibrations

The software requires some calibration parameters to perform the projection unwrapping, the geometric deformation correction, and the correctional vector calculation. The calibration parameters are:

• Known nail diameter: The user inputs the diameter of the nail used for the fracture fixation. The software uses this parameter to scale the projection of the nail and the cross according to the ratio of the major and minor axes of the distal hole ellipse.

• Known radio-opaque cross shape and geometry: The software uses the known shape and geometry of the radio-opaque cross to correct the perspective effect of the camera and the curvature of the screen. The software also uses this parameter to detect the boundary and the centroid of the cross.

• Distance from the viewfinder to radio-opaque cross: The user measures the distance from the viewfinder to the radio-opaque cross using a ruler and inputs it to the software. The software uses this parameter to calculate the offset of the correctional vector.

Additive manufacturing and post-processing

The aiming device was manufactured using additive manufacturing (AM) or 3D printing technology. We used fused deposition modeling (FDM) as the AM technique to print the aiming device [23, 24]. We used ePA CF nylon from SainSmart with a price tag of 70$ per Kg as the printing material for the aiming device. ePA CF nylon is a type of filament used in 3D printing that is made from a nylon 6/66 copolymer and reinforced with 20% carbon fiber. This combination enhances the material’s strength, rigidity, and toughness, making it suitable for printing parts that can replace metal components in some applications. This filament is biocompatible, and heat resistance up to 155 °C. It offers high intensity and rigidity, high toughness and impact resistance, high abrasive resistance, high dimensional stability meaning low shrinkage, reducing warping and cracking during printing. It is also radiolucent, meaning it does not interfere with the fluoroscopic image of the nail and the cross [25,26,27,28].

We used FreeCAD, a free and open-source 3D CAD modeling software, to design the digital model of the aiming device. We exported the model as a stereolithography (STL) file, a common format for AM. We used Cura, a free and open-source slicing software, to convert the STL file into a g-code file, a set of instructions for the FDM printer. We used Ultimaker S5, a professional FDM printer, to print the aiming device. We used a 0.4 mm nozzle. Printing parameters were as follows, a line width of 0.8mma 0.16 mm layer height, a 100% infill density, 30–100 mm/s print speed, 0% fan speed, build plate temperature was set at 60 °C, and a 260 °C nozzle temperature, no printing chamber was applied other than the control provided by the printer enclosure. The radio-opaque cross was inlayed in and the screws were embedded using configured pauses. The printing time was about 4 h and the material cost was about 20 USD. The overall cost was around 50 USD (the additional 30$ costs incurred for estimated electricity usage, the stainless steel nuts and sanding and polishing).

Fig. 5

Printing time details

After printing, we performed some post-processing steps to improve the quality and functionality of the aiming device. We used a deburring tool to remove any excess material or support structures from the printed parts. We used a sandpaper to smooth the surface and edges of the parts.

Testing and pre-clinical validation

The device and software were tested and validated using two methods: Finite Element Analysis (FEA) and physical tests on Digitally Reconstructed Radiographs (DRRs) and phantom bones.

• FEA: The device was subjected to FEA using ANSYS software, to evaluate its stiffness and deformation under different loading conditions. The device was modeled as a linear elastic material with a Young’s modulus of 3.6 GPa and a Poisson’s ratio of 0.4. The device was fixed at the universal ancillary clamp, and a load of 100 N was applied at the viewfinder clamp, in the direction of the correctional vector in various telescopic arm lengths. The maximum von Mises stress and the maximum displacement were recorded and compared with the allowable values.

• Heat deformation: We used a hot air oven to perform the heat deformation testing of the aiming device. We placed the aiming device in the oven and set the temperature to 134 °C, which is the standard temperature for steam sterilization. We kept the aiming device in the oven for 50 min. We removed the aiming device from the oven and let it cool down to room temperature. We measured the dimensions of the aiming device before and after the heat deformation testing and calculated the percentage change.

• DRRs: The accuracy and precision of the software algorithms were evaluated using DRRs generated from CT-scan 3D data of distal tibia. The DRRs were created using attenuation volume rendering, with various scenarios of nail diameters (8 mm, 9 mm, and 10 mm), nail hole diameters (3 mm, 4 mm, and 5 mm), and radio-opaque cross positionings. The software was applied to 20 DRRs, augmented using data augmentation techniques such as rotation, translation, scaling, and noise with a total of 1000 images. We divided the dataset into 10 folds, each containing 100 images. We used 9 folds for training and 1 fold for testing. We repeated the process 10 times, using each fold as the testing set once. We calculated the accuracy and the precision of the software algorithms for each fold and averaged the results over the 10 folds.

• Performance testing: The device and software were tested on phantom plastic bones of humerus, tibia, and femur, to simulate the clinical scenario. The device was attached to the nail ancillary system, and the C-arm was positioned to obtain a lateral view of the distal hole. The software was applied to the camera photo from the C-arm intensifier, and the correctional vector was displayed on the screen. The drilling process was performed according to the software guidance, and the accuracy and precision of the drilling were evaluated. We repeated the process for 30 tests, 10 for each bone type. The following parameters were measured and: mount-to-drill time, adjust time, no contact, medium contact, moderate contact, severe contact, failure ratio, and success ratio.

• Radiation exposure: The radiation exposure testing measures the radiation dose and the time required for the distal locking process. The radiation exposure testing aims to ensure that the device and the mobile application can reduce the radiation exposure and the operative time compared to other methods.

Results

FEM analysis

The results of the FEM analysis are shown in Table 1. The results show that the device has a high stiffness, as the maximum displacement and stress are below the acceptable limits of 1 mm and 50 MPa, respectively. The results also show that the stiffness of the device decreases as the length of the telescopic arm and the viewfinder clamp increase. However, the stiffness remains high enough for the distal locking procedure.

Table 1 FEM analysis for stiffness

Heat deformation

The results showed that the device components had negligible changes in their dimensions and shape after heat deformation testing, indicating that they were resistant to sterilization and maintained their accuracy and precision. The maximum change was 0.03% in the width of the universal ancillary clamp, which was within the acceptable tolerance range. The device components did not show any signs of cracking, warping, or melting after heat deformation testing. The results are shown in Table 2.

Table 2 Heat deformation testing results for the device components

K-folds DRR validation

The results showed that the mean accuracy of the software was 98.7%, with a standard deviation of 1.2%. The mean precision of the software was 99.2%, with a standard deviation of 0.8%. The results indicated that the software was accurate and precise in locating the distal hole center and calculating the correctional vector. The accuracy and precision testing results are shown in Table 3.

Table 3 Accuracy and precision testing results

Performance testing

The performance testing results show the practical feasibility and efficacy of the device and the mobile application for the distal locking process using phantom plastic bones of humerus, tibia, and femur. The results indicate that the device and the mobile application can facilitate the distal locking process and improve the outcomes. The mean values of the performance testing parameters are shown in Table 4.

Table 4 Performance testing results

• a) Time from mounting the aiming device to the start of the drilling.

• b) Time to adjust the XY stages to suggested calibrations.

• c) No contact noticed by the surgeon between the drill and the nail.

• d) Modium contact noticed by the surgeon between the drill and the nail without any metalic grinding sound, but the drill still pass through eventually.

• e) Moderate contact noticed by the surgeon between the drill and the nail with a metalic grinding sound, but the drill still pass through eventually.

• f) Sever contact noticed by the surgeon between the drill and the nail, but the drill does not pass through.

Radiation exposure

The mean radiation dose and time for the distal locking process are 0.4 mSv and 5.4 s. The radiation exposure testing results are shown in Table 5.

Table 5 Radiation exposure testing results

Discussion

The utilization of 3D printing technology in the development of our mechanical aiming device for distal locking significantly enhanced the prototyping process by enabling rapid production and fine adjustments. Unlike traditional manufacturing methods, 3D printing excels in its ability to rapidly iterate designs [29]. This is particularly advantageous when fine-tuning the device, as 3D printing can produce components with a high degree of precision and customization without the need for costly tooling or setup [29]. Additionally 3D printing allowed used to embed stainless steel nuts and the radio-opaque cross using configured printing pauses, Moreover, the cost benefits of 3D printing are evident when compared to the $300 quote for CNC machining; 3D printing offers a more economical alternative, especially for small production runs where the cost per unit remains consistent regardless of quantity [29, 30]. Additionally, the manufacturing time for a 3D printed part is substantially reduced, with parts being printed in approximately 4 h. This rapid turnaround is crucial for accelerating the development cycle and achieving a faster time-to-market. One key benefit from 3D printing is material efficiency, meaning 3D printing is an additive process, building parts layer by layer, which results in less material waste compared to the subtractive nature of CNC machining [29,30,31]. This not only reduces costs but also aligns with sustainable manufacturing practices.In essence, 3D printing not only curtails expenses but also streamlines the manufacturing process, making it an ideal choice for prototyping and producing your mechanical aiming device. Additionally, with considerations of radiolucency, the only available materials were plastics, thus eliminating the possibility of CNC metal manufacturing.

The free-hand technique is the most common and simple technique for distal locking. However, this technique has a low accuracy and a high radiation exposure, as it requires multiple fluoroscopic images to adjust the drill bit position and orientation [5, 9, 11]. The device has a higher accuracy and a lower radiation exposure than the free-hand technique, as it requires only one fluoroscopic image and uses the computer vision algorithms and the aiming device to align the drill bit with the distal hole.

The mechanical guides are devices that attach to the intramedullary nail ancillary system and provide a fixed or adjustable guide for the drill bit [5, 12]. Krettek et al., have emphasized on the nail deformation problem due to the mechanical load introduced by soft-tissue tension and bone weight [32, 33], this deformation results in poor accuracy of such mechanical devices. Also, these devices have a high cost, a complex setup, and a compatibility issue, as they require specific intramedullary nail ancillary systems and nail lengths. Our device has a lower cost, a simpler setup, and a compatibility advantage than the mechanical guides, as it uses a universal ancillary clamp and a telescopic arm that can adapt to any intramedullary nail ancillary system and any nail length.

The electromagnetic devices are devices that use electromagnetic sensors and a computer to locate the distal hole and the drill bit in a three-dimensional space. However, these devices have a high cost, a complex setup, and a calibration issue, as they require a special electromagnetic field generator and a calibration procedure [5, 9]. The device has a lower cost, and a simpler setup than the electromagnetic devices, as it uses a mobile application and a radio-opaque cross that can calibrate the nail projection and the aiming device projection using a single fluoroscopic image.

In addition to the previously mentioned advantages, our device can be reused for multiple cases, as it is sterilizable and adaptable to different nails and bones, while other techniques require disposable components or specific devices for each case. Secondly, our device requires only one fluoroscopic image for each case, while other techniques require multiple images or continuous fluoroscopy, resulting in higher radiation exposure for the patient and the staff. Thirdly, our device reduces the distal locking time to less than 5 s, while other techniques take from 10 to 20 min for the procedure [6,7,8]. Lastly, our device can adapt to any intramedullary nail ancillary system, nail type, size, or manufacturer, and can accommodate for non-perfect round images of the distal hole, while other techniques have limited compatibility or require perfect alignment of the fluoroscope and the nail.

The device and the mobile application have not been tested on real patients or animals, which limits the generalizability and the validity of the results. Future studies should conduct clinical trials and animal experiments to evaluate the safety and the efficacy of the device and the mobile application in real scenarios.

Despite the numerous advantages our device offer, It suffers from limitations such as the quality and stability of the nail ancillary system, which may vary depending on the manufacturer and the surgeon’s preference. Secondly, the software requires a clear and undistorted photo of the C-arm intensifier screen, which may be affected by the lighting conditions, the camera quality, or the human factors. Lastly, the software may provide inaccurate or misleading guidance, due to errors in the algorithms, the parameters, or the calibrations.

These limitations and challenges can be addressed by further improving the design of the device and the software, by conducting more extensive and rigorous testing and validation, and by comparing the device and software with other methods in clinical trials.

Conclusion

We proposed a novel computer-assisted device for distal locking in interlocked intramedullary nails from a single fluoroscopic image. The device consists of an universal ancillary clamp, a telescopic arm, a viewfinder clamp, and a radio-opaque cross. The device is attached to the intramedullary nail ancillary system and allows the surgeon to adjust the position and orientation of the drill bit according to the feedback from a mobile application. The mobile application uses a camera taken photo from the C-arm intensifier and adjusts for geometric deformities from the taken photo. The application employs several computer vision algorithms to detect the centroids of the distal hole and the correctional vector from the viewfinder to the distal hole center. The device was 3D designed using FreeCAD and 3D printed with ePA CF Nylon filament. We tested the device for stiffness, accuracy, precision, and radiation exposure using FEM analysis, K-folds validation, and physical tests over phantom plastic bones. The device showed promising results in terms of low-cost, reusability, low radiation exposure, high accuracy, fast distal locking, high stiffness, and adaptability. The device has several advantages over other distal locking techniques, such as free-hand technique, mechanical aiming devices, electromagnetic navigation systems, and computer-assisted systems. The device is a simple, accurate, and low-cost solution that can facilitate the distal locking in interlocked intramedullary nails.