Deep Transfer Learning of Pick Points on Fabric for Robot Bed-Making

Seita, Daniel & Jamali, Nawid & Laskey, Michael & Tanwani, Ajay & Berenstein, Ron & Baskaran, Prakash & Iba, Soshi & Canny, John & Goldberg, Kenneth. (2022). Deep Transfer Learning of Pick Points on Fabric for Robot Bed-Making. 10.1007/978-3-030-95459-8_17.

Abstract A fundamental challenge in manipulating fabric for clothes folding and textiles manufacturing is computing “pick points” to effectively modify the state of an uncertain manifold. We present a supervised deep transfer learning approach to locate pick points using depth images for invariance to color and texture. We consider the task of bedmaking, where a robot sequentially grasps and pulls at pick points to increase blanket coverage. We perform physical experiments with two mobile manipulator robots, the Toyota HSR and the Fetch, and three blankets of different colors and textures. We compare coverage results from (1) human supervision,(2) a baseline of picking at the uppermost blanket point, and (3) learned pick points. On a quarter-scale twin bed, a model trained with combined data from the two robots achieves 92% blanket coverage compared with 83% for the baseline and 95% for human supervisors. The model transfers to two novel blankets and achieves 93% coverage. Average coverage results of 92% for 193 beds suggest that transfer-invariant robot pick points on fabric can be effectively learned.