Deformation Expression of Soft Tissue Based on BP Neural Network

This paper proposes a soft tissue grasping deformation model, where BP neural network optimized by the genetic algorithm is used to realize the real-time and accurate interaction of soft tissue grasping during virtual surgery. In the model, the soft tissue epidermis is divided into meshes, and the meshes generate displacements under the action of tension. The relationship between the tension and displacement of the mesh is determined by the proposed cylindrical spiral spring model. The optimized BP neural network is trained based on the sample data of the mesh point and vertical tension, so as to obtain the force and displacement of any mesh point on the soft tissue epidermis. The virtual experiment platform is built using a PHANTOM OMNI haptic hand controller and the 3D Max software, by which the simulation experiment of grasping the human abdomen is realized. The experimental results show that the proposed model has good visual interaction and real-time force feedback, which can meet the requirements of deformation simulation for soft tissue grasping in virtual surgery.


Introduction
With the rapid development of the information technology, the intelligent era is coming. Various emerging technologies and applications are increasingly widespread. The aggravation of the global aging population [1], the shortage of medical staff [2,3], and the tediouness of surgical training task [4] have brought severe challenges to the traditional surgical training. The traditional surgical training has such shortcomings as few specimens, long cycle, slow effect, and high cost; therefore, the intellectualization of the virtual surgical training system is needed urgently. The emergence of virtual reality technologies and haptic interactive equipments has brought new opportunities to the virtual surgery training system, and its development and application have promoted the intellectualization of virtual surgery [5][6][7].
The key of the virtual surgery system is designing virtual soft tissue deformation model. In virtual surgery, the soft tissue model needs to simulate various deformation operations and to feedback the deformation operation, so as to simulate the operation process of real soft tissue. Generally, it is common to grasp the soft tissue with surgical instruments during the surgical simulation. For example, it is often necessary to grasp the wound when removing tumors in soft tissue.
With the rapid development of artificial intelligence technology, many researchers apply deep learning, neural network, and other optimization algorithms to deformable soft tissue models. In order to achieve the requirements of real-time performance and accuracy of the virtual models, Zhong et al. [8] proposed a cellular neural method for interaction of soft tissue deformation by virtue of real-time computing of cellular neural networks, where the soft tissue deformation is executed based on the potential energy propagation [9]. According to the law of conservation of energy, the external force applied to soft tissue is regarded as equivalent potential energy and propagates in soft tissue through nerve propagation based on CNN [10]. De et al. [11] also studied the neural network technology based on machine learning for soft tissue deformation. The calculation process is divided into off-line and on-line stages, in which the response of the FEM model under specified displacement is calculated in advance in the off-line stage, and the coefficients of neurons are optimized by training the radial basis function network (RBFN); the deformation field is reconstructed by using the trained RBFN in the online stage. Lorente et al. [12] further explored the method based on machine learning for calculating soft tissue deformation, by which liver deformation simulation was realized during respiration. In this method, a supervised machine learning model based on deformation data is used to construct the mapping function of input variables to approximate the known model output. The mapping function constructed in the training stage can generate exact output for unknown input in following testing stage, that is, the mapping function can reconstruct the overall deformation of soft tissue. Therefore, the performance of the machine learning model highly depends on the training data and the selected learning algorithm. Tonutti et al. [13] further studied the ability of machine learning method in real-time modeling of soft tissue deformation and experimental results showed that the position error is less than 0.3 mm, which exceeds the general threshold of surgical accuracy in images.
In this paper, a novel variable model of soft tissue deformation is proposed to model and simulate soft tissue grasping deformation in interactive surgical simulation. In this model, the grasping deformation of soft tissue is studied by using BP neural network optimized by the genetic algorithm, and the varieties of soft tissue in the direction of tension are determined. The paper is organized as follows: Section 2 introduces the establishment of the soft tissue grasping deformation model and determination of the functional relationship between key points and stress. Section 3 describes the construction of the virtual surgery simulation system based on the model proposed in this paper. In Section 4, we analyze and compare the experimental results through adjusting the stress range of the soft tissue model and the load of the model.

System Design
The soft tissue grasping deformation model consists of four major components: the establishment of a rectangular coordinate system, the study of key points, the functional relationship between key points and stress, and the establishment of the model. The following sections describe in detail the major components of the soft tissue grasping deformation model.

Establishment of a Rectangular Coordinate System
Under the action of given tension F, when the virtual robot manipulator collides with any point on the virtual soft tissue epidermis with arbitrary shape, under the collision point, it is assumed that the plane of the soft tissue epidermis with arbitrary shape connected with the collision point is XY plane. The midpoint of the longest distance between any two points is regarded as the centroid point O. The centroid point O is considered as the origin point and the line passing through the origin O and perpendicular to the XY plane is considered as the Z-axis. Based on the above definition, the space coordinate system is established. It is assumed that m lines are parallel to the X -axis, and the distance between adjacent lines is 1 mm. The m lines include the X -axis , where the value range of m is m ¼ 2; 3; 4; Á Á Á ; 300. It is assumed that n lines parallel to the Y -axis, and the distance between adjacent lines is 1 mm. The n lines include the Y -axis, where the value range of n is n ¼ 2; 3; 4; Á Á Á ; 300. Through m lines are parallel to the X -axis and n lines are parallel to the Y -axis, which is shown in Fig. 1, the soft tissue epidermis without deformation can be divided into a series of regions, the number of the divided regions is s. The number of regular block areas and irregular edge areas are s 1 and s 2 , respectively.
The soft tissue epidermis can be divided into a series of areas. which denotes as, where the number of regular block areas s 1 < s, s 1 is not less than 80% of the total number.
Note that s is the number of the divided a series of regions in soft tissue epidermis with any shape. m is a straight line parallel to the X -axis and includes the X -axis. N is a line parallel to the Y -axis and includes the Y -axis. If the soft tissue epidermis with arbitrary shape is under tension, only the regular block area is considered in the deformation calculation, and the irregular edge area is mainly generated by the expansion internal force of the soft tissue epidermis.

Research on Key Points
The origin point O of the XYZ space coordinate system where the soft tissue epidermis with arbitrary shape is placed is the starting point. Parallel to the X -axis, the total number of rows is m. With the direction paralleling to the Y -axis as the column, the total number of columns is n. The soft tissue epidermis with arbitrary shape is divided into a series of regular block regions, with the intersection of rows and columns as the vertices. The four vertices of each block area are marked with spacial coordinates. In this way, the coordinates of the vertices of each block area in the four quadrants can be expressed as A 0;0 ðx 0;0 ; y 0;0 ; 0Þ, A À1;À1 ðx À1;À1 ; y À1;À1 ; 0Þ, A À1;1 ðx À1;1 ; y À1;1 ; 0Þ, A 1;À1 ðx 1;À1 ; y 1;À1 ; 0Þ…Am 2 ; n 2 ðxm 2 ; n 2 ; ym 2 ; n 2 ; 0Þ, respectively, where m ¼ 2; 3; 4; Á Á Á ; 300, n ¼ 2; 3; 4; Á Á Á ; 300, and the distance d i;j from each vertex to the origin point O can be expressed as: where i and j represent the abscissa and ordinate of the coordinate system, respectively, i ¼ 2; 3; 4; …; 300, and j ¼ 2; 3; 4; …; 300. X ; Y ; Z are the vertex coordinates. If the vertices of each block mesh are subjected to a slight tension in the Z-axis direction at the origin, the displacement changes of the mesh vertices in the four quadrants in the Z direction are DZ 0;0 , DZ À1;À1 , DZ À1;1 , DZ 1;À1 , DZ 1;1 ,…DZ À m 2 ; À n 2 , DZ À m 2 ; n 2 , DZm 2 ; n 2 , and DZm 2 ; À n 2 , respectively. If the tension F i is applied at the origin O, the displacement changes of all points on the circle with the same distance to the origin are the same, so when the coordinate points are selected as the key points, only the XY plane of the cross-sectional coordinate system of the soft tissue epidermis is opted. The vertices on Y ¼ X in the first quadrant can be regarded as key points, which can be expressed as A 1;1 ;A 1;2 ; Á Á Á ;A n;n , where n ¼ 2; 3; 4; Á Á Á ; 300, as shown in Fig. 2.

The Functional Relationship Between Key Points and Stress
If any key point on the line Y ¼ X in the first quadrant of XY plane is subjected to the tension F i in different Z directions at the center of the circle, the change in the vertical direction of Z axis can be measured by experiments, thus can establish the database of key points. The key points are selected according to the vertex of Y ¼ X in the first quadrant of XY plane of the cross-section coordinate system where the soft tissue epidermis is located, that is the vertical displacement variation of any point and the tension F i in the Z-axis direction applied at point O.
In order to study the relationship between the Z-axis change and the tension F i , the micro-cylindrical compact coil spring is used as the deformation model to study the relationship between the Z-axis change DZ and the tension F i . The spring model accords with the elastic characteristics of soft tissue epidermis Y X Z Figure 2: Deformation of key points in spacial coordinate system and is suitable for analysis. If any point on the soft tissue epidermis is stressed, the vertical spring can be simulated as the micro-tension in the direction of the vertical spring cross-section. Through the spring model, the following relationship between the magnitude of tension F i and DZ in the Z-axis direction on the soft tissue epidermis can be obtained as Eq. (4): In Eq. (4), spring tension is proportional to the elongation of deformation, that is, the tension F i received by the soft tissue epidermis is proportional to the change DZ in the Z-axis direction. k is the spring stiffness coefficient, N is the shear modulus, R is the radius of the spring, t is the number of turns of the spring, and r is the radius of the spring cylinder.
The database is used to help the genetic algorithm optimize the BP neural network. Through the BP neural network, the change amount DZ i of any point A i on the soft tissue epidermis on the Z-axis and the tension F i in the Z-axis direction at the origin O can be obtained. There is a non-linear functional relationship between the spatial distances d i;j from the vertex of the mesh divided on the soft tissue epidermis to the origin O. The genetic algorithm is used to optimize the initial weights and thresholds of the BP neural network so that the optimized BP neural network can better predict the output, that is, the BP neural network optimized by the genetic algorithm can obtain the exact functional relationship between DZ i;j , F i and d i;j , as is shown in Fig. 3. The genetic algorithm to optimize BP neural network includes five elements, i.e., population initialization, fitness function, selection function, crossover operation, and mutation operation. The detailed flow chart is shown in Fig. 4.

Model Construction
If a tension F i in the Z-axis direction is employed at the origin point of the soft tissue epidermis, the change amount DZ in the Z-axis direction of any coordinate point on the soft tissue epidermis can be obtained by optimizing the BP neural network through the genetic algorithm and can be obtained through the micro spring model. The force of each point in the Z-axis direction is obtained, so that a planar soft tissue epidermal mesh model can be established and the change DZ in the Z-axis direction of any point on the soft tissue epidermis can be obtained.
In order to test the usability of the virtual platform, operate the PHANTOM OMNI hand controller to grasp the modeled lower abdomen. The process of simulated grasping operation is shown in Fig. 5. Experiments show that this model is effective, can grasp the modeled soft tissues, and can provide visual and tactile feedback.
In the grasping process, the color map is used to display the deformation characteristics of the soft tissue, which more intuitively and clearly shows the deformation of the soft tissue under local stress. The colors from blue to red indicate the strength of the force, blue indicates no force, and red indicates the area that is deformed by the force, as shown in Fig. 6.

System Realizations
The virtual surgery simulation system is composed of three modules, including the input module, information processing module, and output module. The input module mainly uses force-tactile interaction devices to input the tension force, including the magnitude, direction, and duration of the force. The information processing module is a soft tissue model programmed by 3D MAX software, which relies on the computer operating system for processing, including force signal processing and model shape processing. The output module is for feeding back the information to the operator after the signal processing by the input module, including the feedback force from the force-tactile interactive device and the deformation of the model displayed on the displayer. The pipeline of the simulation test is shown in Fig. 7.
In order to realize the algorithm, a virtual simulation platform is first established. Our platform consists of a computer and a haptic interaction facility called PHANTOM OMNI. The computer is based on Windows 10 with an Intel (R) Xeon (R) CPU, E5-1650 v3 @ 3.5 GHz processor, and NVIDIA GeForce GT 720M graphics. The simulation is carried out on VC++ 2019 and 3DS MAX 2019 software with OpenGL graphics libraries. The PHANTOM OMNI allows the operators to touch and operate on the virtual object simulated by our method. The experimental environment is shown in Fig. 8.
A three-dimensional model of 65536 lower abdomens and forceps is used, which is shown in Fig. 9. Import the 3D model into the 3D MAX software, and render the entire 3D model surface with the colorful lower abdomen CT data obtained from the First Affiliated Hospital of Southeast University, including 36,542 color rendering triangles. The rendering effect is shown in Fig. 10.

Collision Detection Validation
The top-K algorithm in the bounding box hierarchy method has better performance and high real-time detection accuracy [14,15], therefore, the study selects it as the collision detection method. The basic idea of the bounding box hierarchy method is to use a slightly larger bounding box and more simple geometric   structure characteristics than the simulation model to approximately represent the complex simulation object, and gradually approach the object by constructing a tree-like hierarchy until the geometric characteristics of the object is completely obtained. If the collision of the object is detected, the bounding box is intersected. Since finding the intersection of the object is easy by the intersection ratio of the bounding box, it is possible to quickly exclude many disjoint objects. The overlapping parts of the bounding boxes are further tested for intersection, thereby speeding up the algorithm.

Simulation Deformation Time
Under the same experimental conditions, 50 grasping operations were performed on the virtual abdomens. The virtual abdomens are constructed by the model proposed in this paper and the springmass model [16], respectively. Additionally, we also select fresh living soft tissues for 50 real grasping operations, and the deformation time of the 50 grasping operations is shown in Fig. 11, where the deformation time is from the beginning of the grasping operation on the abdomen to the end of the model deformation. The blue grid points denote the results of the proposed model in this paper, green grid points denote the results based on the Spring-mass model and red grid points denote the results based on living soft tissues.

Stress Range of Soft Tissue Model
The soft tissue has the maximum stress range during the deformation process after being stressed. In order to make the deformation of this model more accurate, it is necessary to determine its maximum stress. This paper used real living soft tissues to grab and obtain real data, which is shown in Fig. 12. Based on the obtained real data, we improved the parameter settings of the proposed model.

Stability of Deformation
The stability of model deformation is an important metric of the performance of the model. In order to verify the stability of the proposed model, the traditional spring-mass model [16] and the proposed model are simulated, respectively. The proposed model achieves stable at about 50ms for the soft tissue deformation of 12 cm, but traditional spring-mass model has a slight increase after 75 ms without reaching a steady state. As shown in Fig. 13, the proposed model has better stability after the grasping operation.

Model Load Comparison
The finite element model with better load performance [17][18][19][20][21] and the model proposed in this paper are simulated, respectively, and the deformation results are shown in Fig. 14. The load-displacement of the local deformation of the proposed model is also in good agreement with the finite element model, and the maximum deviation between the load and displacement is about 5%. The experimental results show that the proposed model is excellently similar to the finite element model in the load performance.

Conclusions
This study proposed a soft tissue grasping deformation model to simulate the grasping deformation process on the human abdomen soft tissue by virtual human-computer interaction. Using 3DS MAX 2019 with OpenGL graphics libraries and VC++ 2019 software, the abdominal soft tissue model training system for grasping operation was built through the PHANTOM OMNI force-tactile interactive device. The proposed model combines the displacement and deformation of any point of the irregular crosssection elastic cylinder under grasping action, where the BP neural network optimized by the genetic algorithm is used to calculate the deformation of the elastic tissues intuitively, accurately and quickly. A real-time deformation simulation system was constructed to improve the fidelity of virtual force tactile interaction. The real-time performance and accuracy of the proposed model are verified by comparative experiments. The experimental results show that the proposed model has better stability and on-site perception. Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.