Workshop Speakers
Human Sense of Touch
Prof. Etienne Burdet
Human Robotics Lab Department of Bioengineering, Imperial College London, UK
Title: Haptic communication between humans and with robots
Abstract: Robotic systems are increasingly used to work in physical contact with humans, but these contact robots have so far made little use of the opportunities of interactive control. We have found recently that mechanically connected humans benefit from the interaction force to exchange task information, identify each other’s control, and improve own performance. This presentation will present these results on human-human behaviour and their computational modelling. It will derive robotic translation of the control principles, enabling sensory augmentation and optimally shared effort between interacting humans or/and robots.
Bio: Dr. Etienne Burdet is Professor and Chair of Human Robotics at the Imperial College of Science, Technology and Medicine in UK. He is also a visiting Professor at University College London. He holds an MSc in Mathematics (1990), an MSc in Physics (1991), and a PhD in Robotics (1996), all from ETH-Zürich. He was a postdoctoral fellow with TE Milner from McGill University, Canada, JE Colgate from Northwestern University, USA and Mitsuo Kawato of ATR in Japan. Professor Burdet’s group uses an integrative approach of neuroscience and robotics to: i) investigate human motor control, and ii) design efficient systems for training and rehabilitation, which are tested in clinical trials.
Tactile Sensing Technologies
Prof. Takao Someya
Riken Center for Emergent Matter Science and University of Tokio, Japan
Title: Smart Skins with Stretchable Electronics
Dr. Benjamin Tee
The Institute of Materials Research and Engineering, NUS, Singapore
Bio: Dr. Benjamin C.K. Tee is appointed President’s Assistant Professor in Materials Science and Engineering Department at the National University of Singapore, where he leads the Sensor.AI Labs. He was awarded the prestigious NRF Fellowship in 2017. He obtained his doctoral degree in Electrical Engineering from Stanford University and applied a needs-driven R&D approach as a Stanford Biodesign Global Innovation Fellow in 2014. He has developed and patented several award-winning electronic skin sensor technologies. He is an MIT TR35 Innovator (Global) in 2015 and Singapore National Research Foundation (NRF) Fellow. His research group aims to develop technologies at the intersection of materials science, mechanics, electronics and biology, with a focus on sensitive electronic skins that has tremendous potential to advance global healthcare technologies in an increasingly Artificial Intelligence (AI) era. www.benjamintee.com
Dr. Dae-Hyeong Kim
Translational Flextronics Group, Seoul National University, South Korea
Title: Bioinspired curved image sensor array
Abstract: Although recent efforts in device designs and fabrication strategies have resulted in meaningful progresses to the goal of soft optoelectronics, significant challenges still remain in fabricating a soft form of the image sensor array, particularly in the case of system level integration with inspiration from the human eye. In this presentation, we report our recent achievement in a high-density soft curved image sensor (CurvIS) array by using a heterostructure of inherently soft 2D materials (MoS2 and graphene), by employing an ultrathin device structure, and by applying strain-isolating/-releasing array designs. This high-density soft CurvIS array with the single-lens optics successfully acquires pixelated images without optical aberration and infrared noises. We also present a human-eye-inspired soft implantable optoelectronic device based on the developed CurvIS array. Theoretical analysis in conjunction with supporting experiments corroborates the validity of the proposed soft materials and device designs.
Bio: Dae-Hyeong Kim obtained B.S. and M.S. degree in Chemical Engineering from Seoul National University, Korea, in 2000 and 2002, respectively. He received Ph. D. degree in Materials Science and Engineering from University of Illinois at Urbana Champaign in 2009. From 2009 to 2011, he was a post-doctoral research associate at University of Illinois. He joined Seoul National University in 2011 and is currently an associate professor in the School of Chemical and Biological Engineering of Seoul National University. He has published more than 75 papers and 25 international and domestic patents. He has been recognized with several awards including George Smith Award (2009), MRS Graduate Student Award (2009), Green Photonics Award (2011), TR 35 award (2011), Hong Jin-ki Creative Award (2015), SCEJ Award (2016), and Korea Young Scientist Award (2017).
Prof. Hubert Zangl
Institute of Smart System Technologies, Sensors and Actuators Lab, Alpen-Adria-Universitaet Klagenfurt, Austria
Title: Integration of Tactile and Proximity Perception with Capacitive Sensing Technologe
Abstract: The combination of Tactile Perception and Proximity Perception technologies has the potential to play an essential role for service, industrial robotics, HRC, compliant robotics applications and even bio-inspired robotics in the near future as it allows for novel control strategies for exploration, grasping, manipulation and interaction with humans. However, sensor elements for both perception tasks need to share the space when they are integrated e.g. in a sensor skin. The sensor elements of capacitive sensors are just conductive areas that can be extremely thin, bendable and stretchable and can be used for both, proximity and tactile perception. Beyond that, the sensor elements may also be part of a wireless communication and even of energy harvesting. The presentation will address such integration aspects and benefits of the combination of Tactile and Proximity Perception.
Bio: Univ.-Prof. Dr. Hubert Zangl received the Dipl.Ing. degree in telematics, the Dr.Tech. degree in electrical en-gineering, and the Venia Docendi for sensors and instrumentation from Graz University of Technology (TUG), Graz, Austria, in 2001, 2005, and 2009, respectively. From 2010 to 2013 he was an Associate Professor for Sensors and Instrumentation, Institute of Electrical Measurement and Measurement Signal Processing, TUG. Since September 2013 he is a Professor chairing Sensors and Actuators at the Institute of Smart System Technologies, Alpen-Adria-Universität Klagenfurt, Austria. His current research interests focus on design and optimization of smart sensors and actuators, robustness and reliability of sensors and actuators, sensor signal processing, autarkic wireless sensors, energy harvesting and applications in robot sensing and IoT.
Perception, Interaction, & Learning
Prof. Bruno Siciliano
Director of PRISMA Lab and ICAROS Center Università degli Studi di Napoli Federico II , Italy
Title: Tactile Sensing and Manipulation in Medical Applications
Abstract:Tactile sensing and bio-inspired design of human-like tools will enhance manipulation skills and humans’ abilities reproduction. Besides humanoid robots and prosthetics applications, other areas such as minimally invasive laparoscopic surgery could benefit from the use of suitably designed and sensorized tools with reconfiguratio capabilities. This talk will present the PRISMA Hand II, an anthropomorphic prosthetic hand endowed with force sensors at the fingertips. Using novel COmpliant Rolling-contact Element (CORE) joints, the hand has a simple structure and exhibits compliance in multiple directions. A surgical tool, the MUSHA Hand II will be presented as well. The tool has three miniature underactuated fingers composed of multiple vertebrae and internal elastic elements; the six degrees of actuation allow the hand to be configured to grasper mode or re-tractor mode for different organ manipulation tasks.Three-dimensional force sensors are integrated into the fingertips to provide the surgeon with the sense of touch and force information.
Bio:
Prof. Fanny Ficuciello
Title: Tactile Sensing and Manipulation in Medical Applications
Abstract: Tactile sensing and bio-inspired design of human-like tools will enhance manipulation skills and humans’ abilities reproduction. Besides humanoid robots and prosthetics applications, other areas such as minimally invasive laparoscopic surgery could benefit from the use of suitably designed and sensorized tools with reconfiguratio capabilities. This talk will present the PRISMA Hand II, an anthropomorphic prosthetic hand endowed with force sensors at the fingertips. Using novel COmpliant Rolling-contact Element (CORE) joints, the hand has a simple structure and exhibits compliance in multiple directions. A surgical tool, the MUSHA Hand II will be presented as well. The tool has three miniature underactuated fingers composed of multiple vertebrae and internal elastic elements; the six degrees of actuation allow the hand to be configured to grasper mode or re-tractor mode for different organ manipulation tasks.Three-dimensional force sensors are integrated into the fingertips to provide the surgeon with the sense of touch and force information.
Bio: Fanny Ficuciello received the Laurea degree magna cum laude in Mechanical Engineering and the Ph.D. degree in Computer and Automation Engineering both at the University of Naples Federico II in 2007 and 2010 respectively. From September 2009 to March 2010 she was a visiting scholar in the Control Engineering Group at the University of Twente (Netherlands). Currently, she is Assistant Professor of Industrial Bioengineering at the University of Naples Federico II. Her research activity is focused on biomechanical design and bio-aware control strategies for anthropomorphic artificial hands, grasping and manipulation with hand/arm and dual arm robotic systems, human-robot interaction control, variable impedance control and redundancy resolution strategies. Recently she is involved also on surgical robotics research projects, as a member of the ICAROS center (Interdepartmental Center for Advances in Robotic Surgery) of the University of Naples Federico II. From 2008 she is member of the IEEE Robotics and Automation Society and from 2017 she received the upgrade to IEEE Senior Member. From 2018 she is in the Technology Committee of the European Association of Endoscopic Surgery (EAES). Currently she serves as associate editor for Journal of Intelligent Service Robotics (JIST) and for Transactions on Robotics (T-Ro).
Prof. Hyungpil Moon
Robotics and Intelligent Systems Lab, Sungkyunkwan University, South Korea
Title: Delicate and Firm Tactile Sensing Toward Dexterous Manipulation
Bio: Hyungpil Moon is an Associate Professor and the director of Robotics Engineering Research Center(RERC) at Sungkyunkwan University(SKKU), Korea.
Hyungpil received BS and MS degrees in mechanical engineering from POSTECH, Korea and Ph.D. in mechanical engineering from University of Michigan, Ann Arbor in 2005. Before joining SKKU, he was a post-doctoral fellow with the Robotics Institute, Carnegie Mellon University.
He was an Editor of ICRA, a technical editor of IEEE/ASME Trans. Mechatronics, and the Editor-in-Chief of Journal of Korea Robotics Society. He is now serving as a co-chair of IEEE RAS TC on Robotic Hand, Grasping and Manipulation and an associate editor of Intelligent Service Robotics. He is also a creator of the Autonomous Drone Racing Competition.
He leads the RISE LAB in RERC researching dexterous manipulation, task-motion planning, and autonomous drones.
Dr. Michael Wuertenberger
Vice President and Director of AI and Robotics Center of Excellence tat he BMW Research, Germany
Title: Vehicle Interiors as Sensate Environments – Tactile Robotics and Artificial Intelligence Transforming the Future in-Car Experience
Abstract: A vehicle’s interior can already now be considered a highly interactive sensate environment. With the advent of highly automated driving, even more emphasis will be on this smart space and the corresponding user experience. New interior layouts become possible, with the attention shifting from the driver to the wellbeing and comfort of rider passengers in highly reconfigurable interior layouts. Smart materials, an array of sensors, robotic mechatronics, and intelligent perception will provide vehicles with more capabilities to sense and interact with their human counterparts. Tactile intelligence in particular will add an exciting new modality and help address challenges of safe human-robot coexistence.
Grasping and Manipulation
Prof. Oliver Brock
Professor of Robotics in the School of Electrical Engineering and Computer Science at the Technische Universität Berlin, Germany
Title: Sensing for Soft Manipulation: If, What, and How — and Can You Hear It?
Abstract: Soft Manipulation is blurring the boundaries between sensing, perception, control, morphology, and planning. For example: soft hands, by complying to the environment in interesting ways, perform sensing and control without physical sensors or explicit computation. This simplifies perception, control, and planning — but it challenges existing system design wisdom. In this presentation, I will explore the opportunities and consequences of blurring the boundaries between sensing, perception, control, morphology, and planning in the context of manipulation. New questions arise: (when) do we need sensing? What should we sense? How should we integrate sensors into new materials and soft-material robots?
Bio: Oliver Brock is the Alexander-von-Humboldt Professor of Robotics in the School of Electrical Engineering and Computer Science at the Technische Universität Berlin in Germany, which is a German University of Excellence. He received his Ph.D. from Stanford University in the year 2000 and held post-doctoral positions at Rice University and Stanford University. He was an Assistant Professor and Associate Professor in the Department of Computer Science at the University of Massachusetts Amherst, before to moving back to the Technische Universität Berlin in 2009. The research of Brock’s lab, the Robotics and Biology Laboratory, focuses on robot intelligence, mobile manipulation, interactive perception, grasping, manipulation, soft material robotics, interactive machine learning, deep learning, motion generation, and the application of algorithms and concepts from robotics to computational problems in structural molecular biology. OliverBrock is the coordinator of the Research Center of Excellence “Science of Intelligence”. He is an IEEE Fellow and was president of the Robotics: Science and Systems Foundation from 2012 until 2019.
Prof. Alberto Rodriguez Garcia
Prof. in Robotics, Massachusetts Institute of Technology (MIT), US
Title: A vision for tactile dexterity and reactive manipulation
Abstract: Tactile sensing plays a privileged role in the manipulation chain: it is in direct contact with the world, potentially offering direct observations of shape, motion and force at contact. Robot’s limited tactile reasoning, however, is one of the main limitations of current robotic manipulation systems, and a long-standing challenge in the robotics research community. After decades of advances in sensing instrumentation and processing power, basic questions remains: How should robots make effective use of sensed contact information? How do we plan interactions that render useful tactile information? How de we exploit tactile feedback to simplify planning and control? In this talk I will describe current work on developing a reactive manipulation planning framework to handle objects using tactile feedback.
Bio: Alberto Rodriguez is an Associate Professor (without tenure) at the Mechanical Engineering Department at MIT. Alberto graduated in Mathematics (’05) and Telecommunication Engineering (’06) from the Universitat Politecnica de Catalunya, and earned his PhD (’13) from the Robotics Institute at Carnegie Mellon University. Alberto has received the Best Student Paper Awards at conferences RSS’11, ICRA’13, and RSS’18, the Best Cognitive Paper Award at IROS’18, and has been finalist for best paper awards at IROS’16 and IROS’18. He has lead Team MIT-Princeton in the Amazon Robotics Challenge between 2015 and 2017, and is recipient of the Amazon Research Award. He has also received the 2018 Best Manipulation System Paper Award and has been finalist to the 2018 Best Manipulation Technical Paper Award from Amazon. He leads the Manipulation and Mechanisms Lab at MIT (MCube) researching autonomous dexterous manipulation, robot automation, and end-effector design.
Prof. Peter Allen
Title: MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning
Abstract: Vision-based grasping systems typically adopt an open-loop execution of a planned grasp. This policy can fail due to many reasons, including ubiquitous calibration error. Recovery from a failed grasp is further complicated by visual occlusion, as the hand is usually occluding the vision sensor as it attempts another open-loop regrasp. This work presents MAT, a tactile closed-loop method capable of realizing grasps provided by a coarse initial positioning of the hand above an object. Our algorithm is a deep reinforcement learning (RL) policy optimized through the clipped surrogate objective within a maximum entropy RL framework to balance exploitation and exploration. The method utilizes tactile and proprioceptive information to act through both fine finger motions and larger regrasp movements to execute stable grasps. A novel curriculum of action motion magnitude makes learning more tractable and helps turn common failure cases into successes. Careful selection of features that exhibit small sim-to-real gaps enables this tactile grasping policy, trained purely in simulation, to transfer well to real world environments without the need for additional learning. Experimentally, this methodology improves over a vision-only grasp success rate substantially on a multi-fingered robot hand. When this methodology is used to realize grasps from coarse initial positions provided by a vision-only planner, the system is made dramatically more robust to calibration errors in the camera-robot transform.
Bio: Peter K. Allen is Professor of Computer Science at Columbia University. He received the A.B. degree from Brown University in Mathematics-Economics, the M.S. in Computer Science from the University of Oregon and the Ph.D. in Computer Science from the University of Pennsylvania, where he was the recipient of the CBS Foundation Fellowship, Army Research Office fellowship and the Rubinoff Award for innovative uses of computers. His current research interests include robotic grasping, 3-D vision and modelling, and medical robotics. In recognition of his work, Professor Allen has been named a Presidential Young Investigator by the National Science Foundation.
Prof. Alexander Schmitz
Title: Distributed 3D tactile sensing for robotic grasping and manipulation
Abstract: I will present an overview of the research in our lab on developing tactile sensors for robot hands, grippers and arms, and using those sensors for tasks such as grasp stability analysis and dexterous in-hand manipulation. In particular, we will present a skin sensor system, which is soft, can cover various parts of the robot hand and measure force in 3-axis. Each 3-axis sensor is about 4x4x4mm3 big and has digital output. 272 of such 3-sensors have been used to cover all segments of the Allegro hand fingers, with minimal wiring. We will also present skin sensors for safe robot arms.
Bio: Alexander Schmitz is an Associate Professor (without tenure) at the Sugano Lab, Waseda University, Japan. He is also one of the founders of www.xelarobotics.com, which sells tactile sensors. He received his master’s degree with honors from the University of Vienna, Austria, in 2007, and PhD degree from the University of Sheffield, England in 2011. He performed his PhD research as part of a joint location program with the Italian Institute of Technology, Italy. His research interests include tactile sensing, human symbiotic robotics, and robotic object handling. He has applied for 11 patents. He has received a grant of 117 million JPY from the JST START Program in 2016.
Prof. Van Ho
Title: Tactile sensing based on soft morphology: from small scale to large scale
Abstract: In our laboratory, we focus on science and technology of morphological computation for recognition of efficient motory and sensory system, especially haptic interaction and tactile sensing. Human’s skin is not only the largest organ, but also a very sophisticated touch system that help humans perceive surroundings’ acute contact, resulting in unique feedback for dexterous manipulation, texture exploration (fingers, hands) and emotional communication (whole body). Human skin is also soft, and has it own unique morphology (finger print, wrinkles), and its mechanism contributing to tactile sensation remained interesting. In this talk, I will present two representative development at small and large scale. First, I will introduce the role of wrinkled morphology toward implementation of small-scale active sensing based on dynamic changing of the skin, which results in variation of underneath mechanoreceptors. Second, a newly-developed whole-arm tactile sensing technology will be introduced. This large-scale robotic skin has high resolution and can sense both deformation and force distribution under contact with the surrounding. These two development shows the potential of softness and morphological computation in development of soft, flexible sensing devices for robotic manipulation and human-robot interaction.
Bohan Wu
Title: MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning
Abstract: Vision-based grasping systems typically adopt an open-loop execution of a planned grasp. This policy can fail due to many reasons, including ubiquitous calibration error. Recovery from a failed grasp is further complicated by visual occlusion, as the hand is usually occluding the vision sensor as it attempts another open-loop regrasp. This work presents MAT, a tactile closed-loop method capable of realizing grasps provided by a coarse initial positioning of the hand above an object. Our algorithm is a deep reinforcement learning (RL) policy optimized through the clipped surrogate objective within a maximum entropy RL framework to balance exploitation and exploration. The method utilizes tactile and proprioceptive information to act through both fine finger motions and larger regrasp movements to execute stable grasps. A novel curriculum of action motion magnitude makes learning more tractable and helps turn common failure cases into successes. Careful selection of features that exhibit small sim-to-real gaps enables this tactile grasping policy, trained purely in simulation, to transfer well to real world environments without the need for additional learning. Experimentally, this methodology improves over a vision-only grasp success rate substantially on a multi-fingered robot hand. When this methodology is used to realize grasps from coarse initial positions provided by a vision-only planner, the system is made dramatically more robust to calibration errors in the camera-robot transform.
Bio: Bohan Wu is a thesis-tracked M.S. Candidate in Computer Science at Columbia University and a Research Assistant at the Columbia Robotics Lab advised by Prof. Peter K. Allen. His current and future research centers around robotic learning. Previously he was a Visiting Student Researcher at Stanford Intelligent Systems Laboratory advised by Prof. Mykel J. Kochenderfer. He received a Bachelor’s degree in Computer Science and Economics from Duke University in 2016, and was also a Non Degree Option student at Stanford University from 2016 to 2018.
