This article can be interesting for people working with modeling and simulation of flexible multibody systems, as well as for all Modelica enthusiasts.


PowerGrab - Modeling the musculoskeletal system of the human hand in Modelica

Prehensile and precision grasps with Modelica

Several years ago, while volunteering at a nursing home and observing senior citizens having trouble opening doors and holding objects, I started to think of how such people can be assisted.

To put this blog into the right perspective, I would like to mention that I am currently a 17-year-old high school senior, with a curiosity in modeling, controls, and technical design.

The project described below is entirely the result of my work. Dr. Johan Andreasson from Modelon has served as a project supervisor, providing comments and feedback; meanwhile the code for the model and the text for the connected article accepted for Modelica Conference 2017 belong to me entirely.

To learn more about what led me to this project, jump to the end of this article.

So what is PowerGrab?

Being interested in engineering, my thoughts on assisting hand functionalities turned naturally towards prosthetic assist devices. Multiple assist technologies including orthotic gloves have already been constructed: NASA and GM jointly developed one, as did a lab at MIT, to name only two examples.

The designs used by the developers, however, varied greatly, from soft-muscle pneumatic tube gloves, to spring-actuated braces. This variability in design led me to wonder how to figure out the optimal glove. Having already experienced the wonders of Modelica, I obviously turned towards modeling and simulation.

Lately, through a summer internship at Modelon I got the chance for creating a mathematical model that could be used towards the development and improvement of orthotic assist gloves. This is what PowerGrab is about.

Goal of PowerGrab

My goal for PowerGrab was to develop a musculoskeletal model of the hand, a model that could be used to describe contracting and extending motion, both alone and around a contact object.

I chose to test the model's ability to perform two types of grasps - prehensile [1] (full hand grab) and precision [2] (fingertip hold).

Models

To develop the model, I needed two sets of information: knowledge of the anatomy and physiology of the hand, as well as the basic physics describing the contact. For implementation, I used Modelon’s Multi Body Library.

I hence took my high school's anatomy course - an interesting undertaking, to say the least - and based on the literature in the textbook, as well as what was learned in class, I developed the musculoskeletal part of the model.

The basic physics for the contact object I had already studied previously in high school, so I spent more time just on how to represent the contact in Modelica.

Finger component model

The finger and the hand in general are extremely complex systems, something I came to quickly realize in my anatomy class. Other previous musculoskeletal models attempt to address the multiple components of the hand, in their full detail - tendons, muscles, bones, joints, and more.

To me, however, I only needed a model that could demonstrate the motion of the hand, based on its general composition. Hence, I made an array of assumptions to shape the system down into something I could handle.

The end result was that the hand was made of three major systems: bones, muscles (and tendons), and joints. I simplified the bones to cylindrical rigid bodies, and the joints to be composed of one or more revolute joints. The muscles became nonlinear springs that acted between preset connection points along the bones.

The structure within the finger is better shown in Figure 1: 

finger model Modelon
Fig. 1 The finger model

Contact object

The contact object was a major object of concern for a sizeable part of the PowerGrab project. Currently, instead of a formal 'contact object' model, the PowerGrab package has an ad-hoc model of a contact sphere, developed on an as-need basis.

To tackle the problem of the contact object, I split the model into two parts: what occurs at contact, and where the contact actually happens. As the contact would happen in 3D space, to make dealing with multiple directions easier, I made the contact object a sphere.

The contact forces were relatively easy to describe - I compared it to a nonlinear spring, which acted between the object and the finger, the 'displacement' of this 'spring' being how much the finger compressed the object from its normal state.

Finding the contact point required more work - I eventually resolved it using vectors to continually update the vector representing the shortest distance from the object and the finger, and then using that vector's magnitude to determine contact.

With that done, the contact object was thus represented by a point in the space which represented the object's center and a set of connections between the object the finger.

As contact could occur on any of the finger's subsections, I placed connections to each phalange in the finger, leading to the previously mentioned clutter.

Apart from aesthetic issues, there were problems regarding slipping. The friction model of the contact was created by 'dampening' motion across the surface of the object, but ultimately proved ineffectual. While the issue of slipping was addressed using different muscle actuation profiles, the contact object still has much left to accomplish with regards to the friction model.

Testing

The main models coming out of the PowerGrab project are a finger, a finger interacting with a contact object, and a hand - also interacting with a contact object.

I proceeded to test the models in three stages.

First, I checked the three-dimensional animation generated from the Multibody models, to see if the general setup and motion seemed reasonable.

Second, I analyzed the actual joint angles, contact profiles, and muscle actuation curves.

After going back and tweaking the model to pass the first two checkpoints, I altered the actuation profiles to produce the two distinct types of grasps - this last step is specific to the hand model. The data and animation images from the finger and hand grasp testing are shown in Figure 2.

Fig. 2 Visualization of the finger-model contacting spherical object (front and side views)

Analysis and results

Figure 3 and 4 display the results obtained from testing the prehensile and precision grasps, respectively. In each figure, the following data per each finger is shown:

  • Four joint angles

    • Three of the joint angles (proximal, middle, and distal) are for the joints in the fingers that allow forwards and backwards motion
    • The fourth joint angle representing the sideways movement of the finger (think shaking your finger from side to side when gesturing no)

  • Muscle activation signals

    • The abbreviation used, PCS, comes from the model – the actuation signals are pulse models, and they ‘control’ the activation, so I called them Pulse Control Signals (PCS)
    • The three separate PCS curves correspond to which of the three segments of the bone the muscle is actuating (the proximal, middle, and/or distal).

The joint angles show us how the finger moves over the course of the grasp. 

Fig.3 Hand Prehensile Grasp Simulation Data
Fig. 4 Hand Precision Grasp Simulation Data

The distinction between the prehensile and precision grasps is visible when considering how little the other joints of the finger move in comparison to the base joint (the middle and distal joint angles as compared to the proximal joint).

Of course, there is movement – but this occurs after contact with the ball, when the fingers are pressed against the surface, and thus pushing on the other joints as well. For a better idea of how the precision grasp looks, Figure 5 shows an animation generated by the precision grasp simulation.

Fig. 5: Visualization of the hand, before and after performing a precision grasp

In comparison, as seen in Figure 6, the prehensile grasp is messy – there is more movement of the fingers, and there is slight slipping. However, overall, the fingers move to the ball, and retain their grasp over the ball – even though they pass through the others doing so!

(Note that the little finger actually completely misses the ball – the ball’s position was such that when the finger touched the ball, it skidded off the surface of, instead of pressing into, the ball.)

Fig. 6: Visualization of the hand, before and after performing a prehensile grasp

When considering the PCS curves, the main point of interest is if the finger responds to the actuation, which is clearly displayed by the movement in the joint angles. One interesting result does occur – when one muscle actuates, the joints apart from the one directly affected by the muscle also react.

The results of the analysis indicated that the grasps were successfully accomplished with the models. Therefore, I am confident that the main goals of the project were accomplished and that the project can stand as proof of concept. 

Further work

That said, there is much left for improvement. I list below the steps I see needed.

  • The structure of the hand is limited, as it is based on the knowledge from a high school anatomy course.
  • The contact object model includes many assumptions of the object's composition, and has a weak friction model. Even more importantly, as the contact object model can only be used for the actual spherical contact object, none of the fingers can actually contact each other, leading to overlapping fingers.
  • Apart from the finger's relative unawareness and the simplified musculoskeletal geometry, the testing also did not integrate existing datasets of either actuation profiles, contact forces, or contracting motion.

Want to learn more?

Do not hesitate to contact us.

Extra background of the project

Having built a functional flat linear motor during my freshman year at high school, I would happily gaze at the cart, as it moved down the metal tracks due to the force of the solenoids – a scenario that occurred only when it moved in a straight line, and when it was placed the proper distance from the first solenoid. Yet, after finishing the linear motor, I decided to leave it alone, with its faults, as the process of revising such an ad-hoc construction would be quite tedious and repetitive. However, during an internship in the following summer (2014) under Dr. Tiller at Xogeny, I was introduced to Modelica, and modeling and simulation. Looking back at the linear motor, I thought of modeling the system. And a few months later, after working in OpenModelica and then Wolfram SystemModeler, I had a three-dimensional model of the linear motor using the Modelica.Multibody library.

Later, experiences gained while volunteering at a nursing home - observing senior citizens having trouble opening doors and holding objects – led me to start thinking of ways to devise an orthotic glove to assist these people. To gain a better understanding of the hand system, I decided to first develop a mathematical model for it. I began working on the PowerGrab project as a summer intern at Modelon, under the supervision of Dr. Andreasson. Since then, I have continued to work on developing the code for the PowerGrab library, as well as writing the corresponding paper.

The hand model in PowerGrab, while not the most elegant possible, it was enough to point me in the direction of how to optimize the linear motor. With that, I realized how useful modeling and simulation was in design and control optimization, if not for more areas.

Reference

  • Shashank Swaminathan & Johan Andreasson, Musculoskeletal Modeling of the Hand and Contact Object in Modelica, paper accepted for presentation at the International Modelica Conference - May 2017, Prague, Czech Republic.

[1] think open a door by its handle

[2] think hold a glass of water


Shashank Swaminathan was born in 1999, in Michigan, USA, and is currently a senior in high school. He has placed in the top 100 in the Michigan Math Prize Competition, is a National AP Scholar, and was named a U.S. Physics Olympiad Semifinalist in both 2015 and 2017.

He was a summer intern at Modelon in 2015, and currently is in the Wolfram Mentorship Program. Shashank will present the work highlighted in this blog at the Modelica Conference 2017 in Prague.

by Shashank Swaminathan