Modeling and Calibration of a Tactile Sensor for Robust Grasping*

Modeling and Calibration of a Tactile Sensor for Robust Grasping*

Proceedings of the 20th World Congress The International Federation of Congress Automatic Control Proceedings of the 20th World The International Fede...

1MB Sizes 12 Downloads 58 Views

Proceedings of the 20th World Congress The International Federation of Congress Automatic Control Proceedings of the 20th World The International Federation of Congress Automatic Control Proceedings of the 20th9-14, World Toulouse, France, July 2017 The International Federation of Automatic Control Available online at www.sciencedirect.com Toulouse, France,Federation July 9-14, 2017 The International of Automatic Control Toulouse, France, July 9-14, 2017 Toulouse, France, July 9-14, 2017

ScienceDirect

A. A. A. A.

IFAC PapersOnLine 50-1 (2017) 6843–6850 Modeling and Calibration of a Tactile Modeling and Calibration of a Tactile Modeling and Calibration of a ⋆⋆ Modeling and Calibration of a Tactile Tactile Sensor for Robust Grasping Sensor for Robust Grasping ⋆⋆ Sensor for Robust Grasping Sensor for Robust Grasping ∗ ∗ ∗ ∗

∗ Cirillo Cirillo ∗∗ P. P. Cirillo Cirillo ∗ G. G. De De Maria Maria ∗ C. C. Natale Natale ∗ S. S. Pirozzi Pirozzi ∗ Cirillo ∗ P. Cirillo ∗∗ G. De Maria ∗∗ C. Natale ∗∗ S. Pirozzi ∗∗ Cirillo P. Cirillo G. De Maria C. Natale S. Pirozzi ∗ ∗ Dipartimento di Ingegneria Industriale e dell’Informazione, di Ingegneria Industriale e dell’Informazione, ∗ Dipartimento Dipartimento di Ingegneria Industriale e dell’Informazione, Universit` a della “Luigi Via ∗ Universit` a degli degli Studi Studi della Campania Campania “Luigie Vanvitelli”, Vanvitelli”, Via Roma Roma Dipartimento di Ingegneria Industriale dell’Informazione, Universit` a degli Studi della Campania “Luigi Vanvitelli”, Via Roma 29, 81031 Aversa, Italy 29, 81031 Aversa, Italy Universit` a degli Studi della Campania “Luigi Roma 29, 81031 Aversa, Italy Vanvitelli”, Via (e-mail: pasquale.cirillo, giuseppe.demaria, ciro.natale, (e-mail: {andrea.cirillo, {andrea.cirillo,29, pasquale.cirillo, giuseppe.demaria, ciro.natale, 81031 Aversa, Italy (e-mail: {andrea.cirillo, pasquale.cirillo, giuseppe.demaria, ciro.natale, salvatore.pirozzi}@unicampania.it). salvatore.pirozzi}@unicampania.it). (e-mail: {andrea.cirillo, pasquale.cirillo, giuseppe.demaria, ciro.natale, salvatore.pirozzi}@unicampania.it). salvatore.pirozzi}@unicampania.it). Abstract: Abstract: Robust Robust grasping grasping of of everyday everyday objects objects is is still still an an open open problem problem in in robotics robotics due due to to Abstract: Robust grasping ofphysical everyday objects is still an open problem in robotics due to uncertainties affecting object properties like weight and friction. The present paper uncertainties affecting object physical properties like weight and friction. The present paper Abstract: Robust grasping of everyday objects is still an open problem in robotics due to uncertainties affecting object physical propertiesbylike weightsensor and to friction. The present paper proposes the data obtain useful information proposes to to exploit exploit the perception perception data provided provided by aa tactile tactile sensor to obtainThe useful information uncertainties affecting object physical properties weight and friction. present paper proposes to exploit the like perception data provided bylike a tactile sensor to obtain useful information on the contact state, normal and tangential components of the contact force. A on the contact state, like normal andprovided tangential components of theobtain contact force. A novel novel proposes to exploit thethe perception by components a fingertip tactile sensor useful information on the contact state, like normaldata and tangential of to the contact force. Ais novel mechanical model of contact between the soft and grasped object here mechanical model of the contact between the softcomponents fingertip and grasped force. objectAis novel here on the contact state, like normal and tangential of the contact mechanical model of the by contact between the soft fingertip and the grasped object is here presented and supported both FEM analysis and experimental verification. The proposed presented and supported by both FEM analysis andfingertip experimental verification. The proposed mechanical model of the contact between the soft and the grasped object is here presented and supported by both FEMfrom analysis andraw experimental verification. The proposed algorithm to such tactile data, on is algorithm and to extract extract such information information from tactile raw data, based basedverification. on this this model, model, is simple simple presented supported both FEM analysis and experimental Theembedded proposed algorithm to extract suchby information from tactile raw data, based on this model, is simple enough to allow implementation of the grasp control strategy on the control hardware enough to allow implementation of the grasp control strategy on based the control hardware embedded algorithm to extract such information from tactile raw data, on this model, is simple enough to allow implementation of the grasp control strategy on the controlgrip hardware embedded into robotic gripper, so to human reactive responses, which into a a standard standard robotic parallel parallel of gripper, so as as to mimic mimic human reactive grip responses, which enough to allow implementation the grasp control strategy on the controlgrip hardware embedded into a when standard robotic of parallel gripper, so as to uncertain. mimic human reactive responses, which occur the control an held object appears occura when the control of an held object so appears uncertain. into standard robotic parallel gripper, as to mimic human reactive grip responses, which occur when the control of an held object appears uncertain. occur control ofFederation an held object appears uncertain. © 2017,when IFACthe (International of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Keywords: Robotics Robotics technology, technology, Perception Perception and and sensing, sensing, Robots Robots manipulators, manipulators, Embedded Embedded Keywords: Robotics technology, Perception and sensing, Robots manipulators, Embedded robotics, Autonomous robotic systems. robotics, Autonomous robotic systems. Keywords: Robotics technology, Perception and sensing, Robots manipulators, Embedded robotics, Autonomous robotic systems. robotics, Autonomous robotic systems. 1. and 1. INTRODUCTION INTRODUCTION and that that it it brings brings into into play play neural neural processes processes involving involving and that it brings into play neural processes involving 1. INTRODUCTION representations of fingertip forces and weight of the object representations of fingertip forces and weight of the object 1. INTRODUCTION and that it brings into play neural processes involving representations of fingertip forces and force. weight ofconclusion, the object as well as information about the slip In Humans start developing their manipulation abilities from well as information about the slip force. Inofconclusion, Humans start developing their manipulation abilities from as representations of fingertip forces and weight the object as well asthat information about the slip force. In conclusion, it seems humans adjust the grip force based on Humans start developing their manipulation abilities from birth. They quickly learn how to firmly grasp a large it seems that humans adjust the slip gripforce. force In based on two two birth. They quickly learn how to firmly grasp a large as well as information about the conclusion, Humans start developing their manipulation abilities from seems that humans adjust the fast grip reflex force motion based on twoa different neural processes, aa first and birth. They quickly learn how to firmly external grasp a forces large it variety of and able to different neural processes, first fast reflex motion and varietyThey of objects objects and are are able to counteract counteract external forces it seems that humans adjust the grip force based on twoa birth. quickly learn how to firmly grasp a large different neural processes, a first fast reflex motion and a slower adaptation of the grip force based on more complex variety of objects and are able to counteract external forces that tend to take the object away from their hand. Many slower adaptation of the grip force based on more complex that tend to take the object away from their hand. Many different neuralrepresentations processes, a first fast reflex motion and a variety of objects and are ableaway to counteract external forces slower adaptation of the grip force based on more complex sensory-motor of the grasping action. that tend to take the object from their hand. Many neuroscience studies demonstrated that such capability is representations of the grasping action. neuroscience studies demonstrated that such capability is sensory-motor slower adaptation of the grip force based on more complex that tend automatic to take thedemonstrated object away perception from hand. Many neuroscience studies that their such capability is sensory-motor representations of the grasping action. based responses information based on on automatic responses to to perception information Inspired by these studies on human manipulation, sensory-motor representations of the grasping action. neuroscience studies demonstrated that such capability is Inspired by these studies on human manipulation, robotirobotibased on by automatic responses to afferents perception information provided the innervate by these studies on processes human manipulation, robotiprovided by the mechanoreceptive mechanoreceptive afferents that that innervate Inspired cists started to such for purbased on automatic responses to perception information cists started to imitate imitate such processes for slip slip control control purprovided by the mechanoreceptive afferents that innervate Inspired by these studies on human manipulation, robotithe hairless skin of the hand and in particular the fingerstarted to imitate such processes for of slipslip control purthe hairless skin of the hand and in particular the finger- cists poses. Early approaches to the problem detection provided by the mechanoreceptive afferents that innervate poses. Early to approaches to the problem of slip detection the hairless skin of the hand and in particular the fingercists started imitate such processes for slip control purtips (Enoka, 2015). The human nervous system elicits this poses. Early approaches to the problem of slip detection tips (Enoka, 2015). The human nervous system elicits this relied on frequency content of tactile or force measurethe hairless skin of The the hand and in particular the fingerrelied on frequency content of problem tactile orofforce measuretips (Enoka, 2015). human nervous system elicits this poses. Early approaches to the slip detection response, sometimes called reactive grip force responses, response, sometimes called reactive gripsystem force responses, relied on frequency tactile or force et 1996), while, more recent studies tips (Enoka, 2015). The human nervous this ment ment (Holweg (Holweg et al., al.,content 1996), of more recentmeasurestudies response, sometimes called reactive grip force elicits responses, onto frequency ofwhile, tactile or force measurewhen appears subject to Usually, ment (Holweg et models al.,content 1996), while, more recent studies when the the grasp grasp appears subject to disturbances. disturbances. Usually, relied started exploit based on contact mechanics to response, sometimes called reactive grip force responses, started(Holweg to exploit models based on contact mechanics to when the grasp appears subject to disturbances. Usually, ment et al., 1996), while, more recent studies the response is composed by two parts, a first shortto exploit models based onslippage contactavoidance mechanics to the response is appears composed by two parts, a firstUsually, short- started both detect slippage and to devise conwhen the grasp subject to disturbances. both detect slippage and tobased deviseonslippage avoidance conthe response is composed by two parts, a first shortstarted to exploit models contact mechanics to latency reaction with bell-shaped rate of change in the both detect slippage and to devise slippage avoidance conlatency reaction with bell-shaped rate of change in the trol algorithms (Melchiorri, 2000; De Maria et al., 2013). the response isa composed byresponse tworate parts, aslow firstincrease shorttrol algorithms (Melchiorri, 2000; slippage De Mariaavoidance et al., 2013). latency reaction with bell-shaped of change in the both detect slippage and to devise congrip force, and long-latency with algorithms (Melchiorri, 2000; De Maria etofal., 2013). grip force, and a long-latency response with slow increase Such class require measurement both norlatency reaction withinbell-shaped ratewith of change inhave the trol Such class of of algorithms algorithms require measurement both norgrip a long-latency response slow increase trol algorithms (Melchiorri, 2000; De Maria etof al., 2013). in rate of change force. latencies class of algorithms require measurement of both norin the theforce, rate and of change in the the grip grip force. The The latencies have Such mal and tangential components of the contact force vector grip force, and a long-latency response with slow increase mal and tangential components of the contact of force vector in the rate of change in the grip force. The latencies have Such class of algorithms require measurement both norbeen also measured by Macefield and Johansson (2003), mal and tangential components of the contact force vector been also measured by Macefield and Johansson (2003), with respect to the contact plane, therefore suitable senin thefound ratemeasured ofthat change independ the griponforce. The latencies have with respect to thecomponents contact plane, therefore suitable senbeen also by Macefield and Johansson (2003), mal and tangential of the contact force vector who they the rate of change of who found that they depend on the rate of change of with respect to the contact plane, therefore suitable sensors are needed to properly setup these control strategies. been also measured bydepend Macefield and Johansson (2003), sors are needed to properly setup these control strategies. who found that they on the rate of change of with respect to the contact plane, therefore suitable senthe external load. The short-latency response is about sors are needed to properly setupon these control strategies. the external load.they Thedepend short-latency response is about Many tactile sensors exist based different technologies, who found that on the rate of change of Manyare tactile sensors exist based on different technologies, the external load. The short-latency response is about sors needed to properly setup these control strategies. 35 ms, while the long-latency response is about 59 ms. Many tactile sensors exist based on different technologies, 35 ms, while the long-latency responseresponse is about 59 ms. aa comprehensive review can be found in et the external The short-latency is 59 about comprehensive review be on found in (Dahiya (Dahiya et al., al., 35 ms, while load. the long-latency responseat ms. Many tactile sensors existcan based different technologies, More debated is neural the basis this a2010). comprehensive review can be found in (Dahiya et al., More debated is the the neural mechanism mechanism at is theabout basis of of this 35 ms, while the long-latency response is about 59 ms. 2010). More debated is the neural mechanism at the basis of this a comprehensive review can be found in (Dahiya et al., response. A more recent study by Ehrsson et al. (2007) response. A more study by Ehrsson al. (2007) More debated is therecent neuralstudy mechanism at Imaging theet of this 2010). response. A more recent by Ehrsson etbasis al. (fMRI) (2007) The authors of the present paper developed tactile sensors 2010). based on functional Magnetic Resonance The authors of the present paper developed tactile sensors based on functional Magnetic Resonance Imaging (fMRI) response. A more recent studyResonance by between Ehrsson et al. (fMRI) (2007) The authors of the present paper developed tactile sensors based on functional Magnetic Imaging on technologies both for demonstrated a correlation grip adbased on optoelectronic optoelectronic technologies both tactile for manipulamanipulademonstrated a strong strong correlation between grip force force ad- based The authors of the present paper developed sensors based on functional Magnetic Resonance Imaging (fMRI) based on Maria optoelectronic technologies both for interaction manipulademonstrated a strong correlation between grip force adtion (De et al., 2012) and human-robot justments during loading and unloading and activation of (De Maria et al., 2012) and human-robot interaction justments during loading and unloading andgrip activation of tion based on optoelectronic technologies both for manipulademonstrated a strong correlation between force ad(De Mariaetet al., al., 2012) and human-robot interaction justments during loading andand unloading andareas. activation of tion tasks (Cirillo 2014). A version of both motor cortex cerebellum In tasks(De (Cirillo etet al., 2014).and A modified modified version of the the both primary primary motor cortex and cerebellum In any any Maria al., 2012) human-robot interaction justments during loading and unloading andareas. activation of tion tasks (Cirillo et al., 2014). A modified version of the both primary motor cortex and cerebellum areas. In grip any first sensor is presented here, with enhanced measurement case, this neurophysiology study demonstrated that first sensor is presented here, with enhancedversion measurement case, primary this neurophysiology study demonstrated that grip tasks (Cirillo et al., 2014). A modified of the both motor cortex and cerebellum areas. In any sensor is presentedrobustness, here, with enhanced measurement case, neurophysiology study demonstrated that grip first range and which it force adjustment is activated by inputs range and mechanical mechanical robustness, which make make it suitable suitable force this adjustment is strongly strongly activated by tactile tactile inputs first sensor is presented here, with enhanced measurement case, this neurophysiology study demonstrated that grip and mechanical robustness, which make it suitable force adjustment is strongly activated by tactile inputs range for real industrial manipulation tasks. With respect to for realand industrial manipulation tasks. With respect to the the ⋆ This adjustment range mechanical robustness, which make it suitable force is strongly activated by tactile inputs was supported by the European Commission within the ⋆ This work for real industrial manipulation tasks. With respect to the previous works, the aim here is to setup a calibration alwork was supported by the European Commission within the ⋆ This previous works, the aim here is tasks. to setup a calibration alfor real industrial manipulation With respect to the FP7 projects EUROC (GA by n. the 608849) and ECHORD++ (WIRES work was supported European Commission within the previous works, the aim here isenough to setup abecalibration alFP7 projects EUROC (GA by n. the 608849) and ECHORD++ (WIRES ⋆ This gorithm computationally light to implemented work was supported European Commission within the gorithm computationally light enough to be implemented Experiment n. 601116) H2020and REFILLS project(WIRES (ID n. FP7 projectsGA EUROC (GA n.and 608849) ECHORD++ previous works, the aim here is to setup a calibration alExperiment GA n. 601116) and H2020 REFILLS project (ID n. gorithm computationally light enough to be implemented on the control board of the gripper itself. To this purpose, FP7 projectsGA EUROC (GA n.and 608849) ECHORD++ 731590). Experiment n. 601116) H2020and REFILLS project(WIRES (ID n. on the control board of thelight gripper itself. To this purpose, gorithm computationally enough to be implemented 731590). on the control board of the gripper itself. To this purpose, Experiment GA n. 601116) and H2020 REFILLS project (ID n. 731590). on the control board of the gripper itself. To this purpose, 731590).

Copyright © 2017 IFAC 7047 Copyright 2017 IFAC 7047Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2017, IFAC (International Federation of Automatic Control) Copyright © 2017 IFAC 7047 Peer review under responsibility of International Federation of Automatic Copyright © 2017 IFAC 7047Control. 10.1016/j.ifacol.2017.08.1205

Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 6844 A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850

an extensive FEM analysis has been carried out to identify mechanical properties of the device that could be usefully exploited to simplify the original calibration algorithm that was based on a neural network. The result of this analysis is a gray-box model with very low computational burden and whose parameters can be easily identified with a quick calibration procedure. The calibrated sensor has been integrated into an industrial gripper whose control unit has been endowed with both the calibration algorithm necessary to reconstruct the force vector as explained above and with a slipping control algorithm, which already proved to be effective in slippage control experiments in Cavallo. et al. (2014); De Maria et al. (2015). As it happens in human manipulation, the control action is composed by a static control action based on the well-known Coulomb friction model and by a dynamic control action activated by slipping events due to uncertain conditions and external perturbations. Real robotic manipulation tasks have been executed with a Kuka LBR iiwa with the developed smart gripper that has no connection to robot control unit except for open/close commands. Nevertheless, the slip control algorithm implemented on the gripper, that automatically adjusts the grip force, allowed the robot to robustly grasp objects of unknown weight and friction conditions, under timevarying external perturbations. 2. TACTILE SENSOR Starting from the previous version in (De Maria et al., 2012), the authors present an optoelectronic force/tactile sensor suitably developed to be integrated into a standard parallel robotic gripper. 2.1 Sensor Technology The sensor is constituted by three layers, an optoelectronic layer a rigid mechanical layer and a deformable silicone layer. Differently from (De Maria et al., 2012), the electronic PCB integrates 25 taxels, organized in a 5 × 5 matrix, and each taxel is constituted by an unique SMT photo-reflector (see Fig. 1(c)). In particular, the presented prototype uses the NJL5908AR photo-reflector (manufactured by New Japan Radio Co.), that integrates in a single device both an infrared LED (working at 920 nm) and a phototransistor (PT) (working at 880 nm), with a surface encumbrance of 1.06 × 1.46 mm for a single device. The new component allows an easy PCB assembly process with a robotized pick-and-place procedure and also to reduce the performance degradation associated to uncertainties on the relative orientation between LED and PT, when separated components were used. Above the PCB is positioned a mechanical structure, constituted by a rigid layer with a grid shape and a deformable silicone layer bonded to the rigid one. The rigid layer is fixed by soldering rigid pins on the PCB (see Fig. 1(b)). The deformable silicone layer is a spherical cap squared at the base, as shown from pictures in Fig. 1 and sketches in Fig. 4. On the bottom side there are twenty-five empty cells, which present the surfaces in front of photo-reflectors made of white silicone, while the walls separating the cells are black, to avoid cross-talk effects (see Fig. 1(a)). The rigid grid is positioned so that

(a)

(b)

(c)

(d)

Fig. 1. Pictures of the tactile sensor: a bottom view of the deformable layer (a), the rigid layer bonded to the deformable one (b), the optoelectronic layer (c) and the assembled sensor (d). the optical centers of photo-reflectors are aligned with the cell centers of deformable layer (see Fig. 1(d)). When an external force is applied to the deformable layer, it produces vertical displacements of the ceilings of the cells for all taxels. The distances between photo-reflectors and the white surfaces change, by producing variations of the reflected light and, accordingly, of the photocurrents measured by the PTs. The cells in the silicone layer have been designed so that, in rest condition, the white ceilings are positioned at a distance d0 = 1 mm from the emitting surface of the components. Since the NJL5908AR component has a non-monotonic characteristic with respect to the distance from the white surface, taking into account that the height of the components is 0.5 mm, the rigid layer has been designed with a thickness of 0.8 mm (corresponding to a minimum reachable distance dm = 0.3 mm), to force the component to work in the monotonic working area, highlighted in red in Fig. 2. The deformable black walls, bonded on the grid, have been designed so that their distance from the photo-reflector optical centers is sufficiently high to avoid optical interactions during cell deformations. With these choices, each taxel results sensitive only to vertical displacements and not to lateral ones. The mechanical properties of the silicone cap determine the full-scale of the sensor. The presented prototype uses a hardness of 26 shore-A, with a full-scale for the measured forces up to 30 N. 2.2 Sensor Modeling and Finite Element Analysis One of the objectives of this paper is to propose a phenomenological model to estimate the contact force components from the tactile map, with a small number of parameters to identify. The model should be computationally light enough so that it could be implemented on a low-cost microcontroller embedded into the gripper. The proposed model has been devised based on a Finite Element (FE) analysis of the sensor deformable layer

7048

Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850

6845

1

z

0.9 Normalized voltage

0.8

f

contact plane

P

working area

0.7 0.6

d

0.5

d

0.4

x,y

0

m

0

0

C

Fx=0 N, Fy=0 N, Fz=-3.8 N

0.3

-0.1

0.2

0.4 0.6 0.8 1 Reflective surface distance [mm]

1.2

1.4

fzi [N]

0.2 0

Fig. 2. Characteristic for a single taxel: normalized voltage vs reflective surface distance.

-0.2 -0.3 -0.4 -0.5 -10

-5

0

5

10

x (y) [mm]

z [mm]

Fig. 4. Sketch for the proposed model with a normal applied force (top) with the fzi corresponding to taxels along central row (x) and column (y) (bottom).

y [mm] x [mm]

20

Fig. 3. Mesh used for the FE model.

15

Let consider a sketch of the sensor deformable layer (see Fig. 4-top) with the reference frame fixed to the center of its base and the z-axis orthogonal to the base itself. Since the sensor is mounted on a parallel gripper, only cases with the contact plane parallel to the sensor base will be considered in the present paper. In addition, the effect of the actual force distribution on the contact surface can be considered equivalent to the effect of a concentrated contact force f = ( fx fy fz )T applied in the centroid of the force distribution on the contact surface. First of all, a single taxel has been experimentally tested to evaluate the voltage variations measured by the photo-reflector with respect to the normal force applied on the taxel itself. The results in Fig. 5 show that for a single taxel this relation is linear, and, as a consequence, the normal force is proportional to the measured voltage. By extending this observation to the whole sensor, the following linear model is assumed for the estimation of the normal component fz 25  fz = kz ∆vi , (1) i=1

where ∆vi is the voltage variation measured by the i-th taxel and kz is a calibration parameter to be identified.

Normal force [N]

carried out in COMSOL Multiphysics. The extra fine mesh used in the model is reported in Fig. 3, for a total of 103.059 elements. The silicone of the deformable layer has been simulated with a Young’s modulus E = 0.6 MPa and a Poisson’s ratio ν = 0.49. The silicone layer has been constrained on its base, while a rigid plane, simulating the manipulated object, comes into contact with the silicone by following a prescribed displacement. The simulation has been carried out in stationary conditions using the penalty method to solve the contact problem.

10

5

0 0

0.2

0.4 0.6 0.8 Voltage variations [V]

1

1.2

Fig. 5. Characteristic for a single cell: normal force vs voltage variations. With this choice, the voltage measured by each taxel can be considered as the measurement of a local normal force fzi = kz ∆vi . The 25 measured fzi represent a force field of normal forces applied at the sensor taxels. To verify assumption (1), the vertical displacement of each cell, for different values of the applied normal force, has been estimated as the average displacement over the taxel area. By using the characteristic of the optoelectronic component reported in Fig. 2, the voltage variations have been computed from the displacements. The parameter kz has been estimated with a standard least mean square method. Figure 6 reports the comparison between the normal force estimated by the FE model and the fz values reconstructed by using the model (1), which shows a good matching. Given kz , model (1) allows to reconstruct also the force field fzi , i = 1, . . . , 25. When a pure normal force is applied to the sensor (see Fig. 4), this force field is axial symmetric with respect to the z-axis. As a consequence, by reporting the fzi values corresponding to the taxels of the central

7049

Proceedings of the 20th IFAC World Congress 6846 A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850 Toulouse, France, July 9-14, 2017

0

z

FEM data Proposed Model

−1

r

fz [N]

−2

f

contact plane

P

−3 −4

y

−5

0

−6

0

−7

-0.2

2

4

6

8

samples

Fx=1.80 N, Fy=3.38 N, Fz=-8.14 N

-0.4

10

fzi [N]

−8

yC

Fig. 6. Proposed model vs FEM data for the normal force.

-0.6 -0.8 -1

z -10

r

5

10

contact plane

P

Fig. 8. Sketch for the proposed model with a tangential fy applied force (top) with the fzi corresponding to taxels along the central column (bottom).

x 0

xC

4 3.5

Fx=1.80 N, Fy=3.38 N, Fz=-8.14 N

xC , yC [mm]

-0.2 -0.4

fzi [N]

0

y [mm]

f

0

-5

-0.6 -0.8

x−Data y−Data Linear interpolation

3 2.5 2 1.5 1

-1

0.5

-10

-5

0

5

10

0 0

x [mm]

0.2

0.4

0.6

0.8

1

xCt , yCt [mm]

1.2

1.4

Fig. 7. Sketch for the proposed model with a tangential fx applied force (top) with the fzi corresponding to taxels along the central row (bottom).

Fig. 9. Relationship between centroid of tactile map and point C.

row and column, the same symmetric values are obtained (see Fig. 4-bottom). On the contrary, when the contact force f includes also tangential components, the fzi force field becomes asymmetric. In particular, the more the tangential force value the more significant the asymmetry is. From the FE model fzi have been computed for different values of fx and fy , applied to the deformable layer. By considering the case with fx = 1.80 N and fy = 3.38 N, Figs. 7,8 show the fzi values on the taxels along the central row and the central column, respectively.

To verify the equation above, the coordinates of the point C have been computed from the force vector as xC = −zP fx /fz , yC = −zP fy /fz , (4)

A simple way to quantify this asymmetry is through the location of the centroid of the force field, which, obviously, coincides with the centroid Ct of the tactile map, i.e., 25 25 xi ∆vi yi ∆vi , y = (2) xCt = i=1 i=1 Ct 25 25 ∆v i i=1 i=1 ∆vi where (xi , yi ) are the coordinates of the i−th taxel. By analyzing the results of several FE simulations, we discovered that the force vector is aligned with the direction of the line from the point P on the top of the undeformed

sensor tip to a point C on the sensor base (see Figs. 7,8) related to the centroid above as xC = αxCt , yC = αyCt . (3)

where zP is the z coordinate of the point P , and compared to the coordinates of the centroid computed as in (2),(3) with a scale factor α = 2.7, estimated via a leastsquare algorithm. The result is shown in Fig. 9, which demonstrates that our assumption of the force aligned with the line from P to C is quite accurate. This allows us to assume as phenomenological model to estimate the tangential components based on the centroid of the tactile map, the relationships in (2)–(4), i.e., 25 25 yi ∆vi α α i=1 xi ∆vi fx = − f , f = − fz . (5) 25 i=1 z y 25 zP z P ∆v i i=1 i=1 ∆vi The model above has been validated against the FE model and the results are reported in Figs. 10,11 for the x and y

7050

Proceedings of the 20th IFAC World Congress A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850 Toulouse, France, July 9-14, 2017

5

5

FEM data Model

4

6847

Validation data Model

3

4

fx [N]

fx [N]

2

3

1 0 −1

2

−2 −3

1

−4 −5 0

0

2

4

6

samples

8

20

4

fy [N]

fz = kz 3

fx =

2

1

fy = 2

4

6

samples

8

10

Fig. 11. Proposed model vs FEM data for fy . 2

Validation data Model

0

fz [N]

−2 −4 −6 −8 −10 0

10

40

50

by multiplying each taxel voltage by an equalizing scaling factor βi , such that the complete model to identify is

FEM data Model

0

30

time [s]

Fig. 13. Experimental calibration of the sensor: tangential component along x axis.

Fig. 10. Proposed model vs FEM data for fx . 5

10

10

20

30

time [s]

40

50

Fig. 12. Experimental calibration of the sensor: normal component. components of the force vector, respectively. It is evident how the phenomenological model is able to reconstruct the tangential components of the force vector. 2.3 Sensor calibration The phenomenological model presented so far has been exploited to calibrate the sensor experimentally. To compensate for the unavoidable differences among the various taxels in terms of sensitivity, due to differences in the phototransistor gains, collector and LED resistances, and to the different initial distances from the reflective surface, the relationships in (1) and (5) have been slightly modified

25 

βi ∆vi i=1 25 xi βi ∆vi fz i=1 25 i=1 βi ∆vi 25 yi βi ∆vi fz . i=1 25 i=1 βi ∆vi

(6) (7) (8)

Therefore, the proposed calibration algorithm has 26 parameters to be estimated. The estimation has been carried out by applying a set of force vectors on the sensor tip (placed horizontally on a rigid support) through a rigid plane attached to the robot end effector and moving it along a spiral path with an amplitude linearly increasing with respect to the vertical displacement, so as to apply tangential forces with increasing amplitude proportionally (through the friction coefficient) to the amplitude of the normal force. A standard optimization algorithm has been adopted to estimate the best calibration parameters that minimize the total square error between modeled forces and forces measured through the reference sixaxis force/torque sensor ATI mini45 mounted below the tactile sensor. The calibration model has been validated on a completely different set of force vectors applied by manually touching the sensor with a rigid flat object. The results, reported in Figs. 12,13,14 for the normal and tangential components, respectively, demonstrate the good accordance between measured and reconstructed forces. 3. SMART GRIPPER The smart gripper presented in this paper is a sensorized parallel industrial gripper that integrates the sensor technology proposed by the authors. It integrates a motor control board that drives the motors, a microcontroller board that represents the interface between the user and the system motors plus sensors, a DC-DC voltage conversion board to power all the electronics, and the sensors technology proposed by the authors. The proposed system provides an easy and smart way for acquiring the sensor data and for controlling the finger position through a unique ethernet interface, nowadays available in many industrial control systems. Figure 15 shows the smart

7051

Proceedings of the 20th IFAC World Congress 6848 A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850 Toulouse, France, July 9-14, 2017

5 4

Validation data Model

3

fy [N]

2 1 0 −1 −2 −3 −4 −5 0

10

20

30

time [s]

40

50

Fig. 14. Experimental calibration of the sensor: tangential component along y axis.

(a) Experimental Setup

(b) Smart Gripper

Fig. 15. The Smart Gripper.

Fig. 16. The software and hardware architecture. gripper on the right and the complete experimental setup on the left, while, Fig. 16 reports the proposed architecture detailed in Section 3.1.

possible to send commands to the motors by using the TMCL protocol by an RS232 interface. The two fingers of the gripper have been re-designed and 3D-printed in order to allow perfect mounting of two force sensors. Both sensors and the TMCM-KR-842 board are connected to an ARM-based microcontroller board provided by Olimex (Olimex STM32-E407), via an SPI interface and a USART interface, respectively. Such a board represents the interface between the control unit and/or the user and the system constituted by the sensors and the gripper. Finally, a DC-DC voltage conversion circuit provides the required voltage adjusting in order to power all the electronics from a single 24 V input voltage, taken from the control unit of the robot, to a 3.3 V and a 5 V output voltages used to power the tactile sensor and the Olimex board, respectively. In detail, the tactile sensor provides a digital interface for acquiring 25 taxel voltages. Two AD7490 16-channels Analog to Digital converters (ADCs) by Analog Devices are integrated on the sensor board in order to convert the taxel signals on board. The acquired data are, then, sent to the Olimex board over the SPI interface. For this purpose, the typical Master-Slave SPI configuration is considered, in which the Olimex board is the Master and the two ADCs are the slaves. Moreover, in order to acquire in sequence the signals connected to the ADCs independently, two independent chip select pins are used to activate the two converters separately. So, only 7 pins have been used to connect each sensor to the Olimex board, namely the Vcc (power supply), GND (ground pin), MISO (Master Input Slave Output pin), MOSI (Master Output Slave Input), SCLK (Slave Clock pin) and two CS (Chip Select pin). The Olimex STM32-E407 implements two threads in order to send the sensor data to the control unit and to receive the motor control input from the control unit. A first thread has been exploited to acquire the 25 taxel voltages of the tactile sensor using the SPI interface and to compute the proposed model in order to estimate the three components of the applied force vector as well as the coordinates xCt and yCt . The five estimated quantities are sent over the ethernet at a rate of 400 Hz. A second thread is exploited to receive via the ethernet interface the control input from the control unit in terms of finger positions at a rate of 40 Hz, limited by the gripper control board. The finger positions are, then, translated into TMCL packets and sent to the TMCM-KR-842 board over the USART interface that is responsible to drive the motors. Finally, the smart gripper has been installed on a KUKA LBR iiwa robot, connected to a control PC via ethernet and programmed, under the ROS framework, to execute the task. The control PC computes the slipping control algorithm and provides the control signal to the gripper to avoid slipping of the manipulated object during task execution. Of course, this control algorithm could be implemented directly on the Olimex board, but it was implemented on the PC just for easy testing provided by ROS.

3.1 Hardware Architecture 4. EXPERIMENTS The smart gripper is based on the KUKA youBot twofinger gripper that integrates a TMCM-KR-842 24 V board provided by TRINAMIC to control two linear stepper motors (see the KUKA youBot specifications in (KUKA, 2015)). Each motor moves one gripper finger, and it is

In order to prove the effectiveness of the presented approach, few experiments have been carried out by considering the slipping avoidance control law presented by the authors in (De Maria et al., 2015), briefly recalled below.

7052

Proceedings of the 20th IFAC World Congress A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850 Toulouse, France, July 9-14, 2017

The considered control action is composed by a static control action based on the Coulomb friction model and by a dynamic control action activated by slipping events due to uncertain conditions, i.e., an unknown friction coefficient, and external perturbations, i.e., dynamic changing of the manipulated object weight. The total actuation force is u = us + ud = ft /µ + fnd ,

Forces [N]

2

where fnd is an integral action on the residual of a linear Kalman Filter (KF)used to filter the estimated tangential

In the first three case studies, an object of 0.2 kg and estimated friction coefficient µ = 0.65 positioned on a table, is considered. Figure 17 reports results of case study 1. During phase (a), the object is grasped with an initial force of 2 N in order to avoid any slipping when the object will be lifted. At the beginning of phase (b), the object is lifted for 1 cm in order to correctly measure its weight and the static contribution of the slipping control algorithm is activated. By the knowledge of the friction coefficient and the measured tangential force, through the Coulomb friction model, it derives that a force of about 1.5 N is sufficient to avoid object slippage. In fact, after the activation of the static control action, the grasping force decreases from 2 N to 1.5 N (see vertical red bar). Then, the object is moved to the filling position. Phase (c) starts and the object is slowly filled by pouring 0.1 kg of rice. Given the quasi-static condition of the experiment, the static contribution allows to firmly grasp the object

(c)

0 −2 fz 20

40 time [s]

60

80

Fig. 17. Case study 1.

Forces [N]

2

(a)

(b)

(c)

0 −2 ft −4 0

fz 20

Slip 40 time [s]

60

80

Fig. 18. Case study 2. (c)

2 Forces [N]

Grasping robustness of the smart gripper has been tested by considering five case studies, i.e., • Case 1: the weight and the friction coefficient of the object are known. Only the static contribution of the control law is computed and the rice is slowly poured in the object. • Case 2: the weight and the friction coefficient of the object are known. Only the static contribution is activated and the rice is quickly poured in the object. • Case 3: the weight and the friction coefficient of the object are known. Both the static and dynamic contributions are computed and the object is quickly filled. • Case 4: the friction coefficient of the object is known but its weight is supposed unknown. Both the static and dynamic contributions are activated and the object is quickly filled. • Case 5: both the weight and the friction coefficient of the object are supposed unknown. The static and dynamic contributions are activated and the object is quickly filled.

(b)

ft

fx2 + fy2 of the contact force. This component ft = slipping avoidance algorithm has been exploited to test the smart gripper with the novel sensor calibration based on the new contact model during a robotic pick and place task of an aluminium box. In detail, the task consists of three main phases, i.e., • Phase (a): the object is grasped by the gripper; • Phase (b): the robot lifts the object and moves it forward to a filling position; • Phase (c): the object is filled with rice in order to dynamically change its weight.

(a)

−4 0

(9)

6849

0 −2 −4 40

fnd

fz 50

ft 60 time [s]

70

80

Fig. 19. Case study 3. avoiding any slippage. Differently, in case study 2 (see Fig. 18), when the quasi-static condition is no longer valid, the static contribution of the control law does not guarantee the avoidance of object slippage. In fact, by quickly pouring the rice in the object, the latter falls down and the estimated tangential force ft is equal to the Coulomb friction force (1.2 N about) and not to the total weight. From the previous discussion, it turns out that, in order to ensure a robust grasping in dynamic conditions, the contribution fnd of the actuation force is necessary as demonstrated in case study 3. In fact, by repeating the previous experiment, and considering this time the total actuation force, no slipping event occurs. Figure 19 reports the dynamic contribution fnd during phase (c), as well as the tangential force ft and the measured normal force fz . At the time instant 65 s, namely when the rice have being poured in the object, the controller provides a dynamic contribution fnd of about 1 N , in order to reject the external perturbation. Case studies 4 and 5 have been carried out with the aim of showing the ability of the smart gripper to handle significantly uncertain conditions. In both cases, an heavier object has been considered with unknown weight. Hence, the slipping avoidance algorithm is activated before phase (b) in order to adjust the grasping force and avoid object slippage during the lifting phase. In case study 4 (see Fig. 20), the friction coefficient is considered known. The initial grasping force is set to 1.5 N (phase (a)), less than the actual force needed to lift the object avoiding its slippage. Then, the total control law is activated and phase

7053

Proceedings of the 20th IFAC World Congress 6850 A. Cirillo et al. / IFAC PapersOnLine 50-1 (2017) 6843–6850 Toulouse, France, July 9-14, 2017

Forces [N]

5

(a)

(b)

hardware of a standard industrial gripper so to endow this device with a slipping avoidance capability. This smart capability is based on a control algorithm which makes use of the force measurements provided by a tactile sensor mounted on the gripper itself and thus does not require any interaction with the robot central control unit. Such characteristic makes this technology attractive for many robotics applications both in the industrial and professional service domains. Future developments will concern the extension to non parallel grippers, thus requiring the relaxation of the simplifying assumption of a purely horizontal contact surface. Further investigations will be also devoted to the case of contact with deformable objects.

(c)

0

−5

0

ft

fz

5

10

15 time [s]

20

25

30

(a)

Forces [N]

5

(c)

(b)

0

−5 fnd

fz

10

REFERENCES

ft 15

20 time [s]

25

30

(b)

Fig. 20. Case study 4: whole task (a) and detail of lifting and filling phases (b).

Forces [N]

(a)

(b)

(c)

0

−5 ft −10 0

fz 5

10

15 time [s]

20

25

(a)

Forces [N]

5

(c)

(b)

0 −5 −10

fnd 10

fz

ft 15

time [s]

20

25

(b)

Fig. 21. Case study 5: whole task (a) and detail of lifting and filling phases (b). (b) starts. The dynamic contribution, differently from case study 3, acts during both object lifting and rice pouring as shown in Fig. 20(b) around time instants 10 s and 24 s, respectively. In case study 5 (see Fig. 21), also the friction coefficient has been supposed unknown. Talcum has been applied to the object surface in order to intentionally decrease the friction coefficient. Notice that, this time, the computed dynamic contribution fnd has a positive bias due to the uncertainty on the friction coefficient (see Fig. 21(b)), which allows to avoid the slippage during the movement. 5. CONCLUSION

Cavallo., A., De Maria, G., Natale, C., and Pirozzi, S. (2014). Slipping detection and avoidance based on kalman filter. Mechatronics, 24, 489–2499. Cirillo, A., Cirillo, P., De Maria, G., Natale, C., and Pirozzi, S. (2014). An artificial skin based on optoelectronic technology. Sensors and Actuators A: Physical, 212, 110–122. Dahiya, R.S., Metta, G., Valle, M., and Sandini, G. (2010). Tactile sensing-from humans to humanoids. IEEE Trans. on Robotics, 26, 1–20. De Maria, G., Falco, P., Natale, C., and Pirozzi, S. (2015). Integrated force/tactile sensing: The enabling technology for slipping detection and avoidance. In Proc. of the 2015 IEEE Int. Conf. on Robotics and Automation, 3883–3889. Seattle. De Maria, G., Natale, C., and Pirozzi, S. (2012). Force/tactile sensor for robotic applications. Sensors and Actuators A: Physical, 175, 60–72. De Maria, G., Natale, C., and Pirozzi, S. (2013). Slipping control through tactile sensing feedback. In Proc. of the 2013 IEEE Int. Conf. on Robotics and Automation, 3508–3513. Karlsruhe, DE. Ehrsson, H., Fagergren, A., Ehrsson, G., and Forssberg, H. (2007). Holding an object: Neural activity associated with fingertip force ajdustments to external perturbations. J. Neurophysiol., 97, 1342–1352. Enoka, R. (2015). Neuromechanics of Human Movement. Human Kinetics, 5th edition. Holweg, E., Hoeve, H., Jongkind, W., Marconi, L., Melchiorri, C., and Bonivento, C. (1996). Slip detection by tactile sensors: Algorithms and experimental results. In Proc. of the 1996 IEEE Int. Conf. on Rob. and Aut., 3234–3239. Minneapolis. KUKA (2015). Youbot detailed specifications. URL http://www.youbot-store.com/wiki/index.php. Macefield, V. and Johansson, R. (2003). Loads applied tangential to a fingertip during an object restraint task can trigger short-latency as well as long-latency emg responses in hand muscles. Exp. Brain Res., 152, 143– 149. Melchiorri, C. (2000). Slip detection and control using tactile and force sensors. IEEE/ASME Trans. on Mechatronics, 5(3), 235–242.

The phenomenological model presented in this paper to calibrate a force/tactile sensor demonstrated simple and effective enough to be implemented on the embedded 7054