Full Text:   <267>

Summary:  <109>

CLC number: TP242.6

On-line Access: 2019-08-29

Received: 2018-05-03

Revision Accepted: 2018-07-30

Crosschecked: 2019-08-09

Cited: 0

Clicked: 699

Citations:  Bibtex RefMan EndNote GB/T7714


Mei-qin Liu


-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2019 Vol.20 No.8 P.1036-1048


Finding misplaced items using a mobile robot in a smart home environment

Author(s):  Qi Wang, Zhen Fan, Wei-hua Sheng, Sen-lin Zhang, Mei-qin Liu

Affiliation(s):  College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   liumeiqin@zju.edu.cn

Key Words:  Home service robot, Smart home, Heterogeneous sensors, Autonomous robot retrieval

Qi Wang, Zhen Fan, Wei-hua Sheng, Sen-lin Zhang, Mei-qin Liu. Finding misplaced items using a mobile robot in a smart home environment[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(8): 1036-1048.

@article{title="Finding misplaced items using a mobile robot in a smart home environment",
author="Qi Wang, Zhen Fan, Wei-hua Sheng, Sen-lin Zhang, Mei-qin Liu",
journal="Frontiers of Information Technology & Electronic Engineering",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Finding misplaced items using a mobile robot in a smart home environment
%A Qi Wang
%A Zhen Fan
%A Wei-hua Sheng
%A Sen-lin Zhang
%A Mei-qin Liu
%J Frontiers of Information Technology & Electronic Engineering
%V 20
%N 8
%P 1036-1048
%@ 2095-9184
%D 2019
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.1800275

T1 - Finding misplaced items using a mobile robot in a smart home environment
A1 - Qi Wang
A1 - Zhen Fan
A1 - Wei-hua Sheng
A1 - Sen-lin Zhang
A1 - Mei-qin Liu
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 20
IS - 8
SP - 1036
EP - 1048
%@ 2095-9184
Y1 - 2019
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.1800275

smart homes can provide complementary information to assist home service robots. We present a robotic misplaced item finding (MIF) system, which uses human historical trajectory data obtained in a smart home environment. First, a multi-sensor fusion method is developed to localize and track a resident. Second, a path-planning method is developed to generate the robot movement plan, which considers the knowledge of the human historical trajectory. Third, a real-time object detector based on a convolutional neural network is applied to detect the misplaced item. We present MIF experiments in a smart home testbed and the experimental results verify the accuracy and efficiency of our solution.




Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Ahern SJ, Carter J, Wilson P, 2015. Autonomous robot retrieval system. SAI Intelligent Systems Conf, p.280-282.

[2]Castellanos JA, Neira J, Tardós JD, 2001. Multisensor fusion for simultaneous localization and map building. IEEE Trans Rob Autom, 17(6):908-914.

[3]Choi YS, Deyle T, Chen T, et al., 2009. A list of household objects for robotic retrieval prioritized by people with ALS. IEEE Int Conf on Rehabilitation Robotics, p.510-517.

[4]Ekvall S, Kragic D, Jensfelt P, 2007. Object detection and mapping for service robot tasks. Robotica, 25(2):175-187.

[5]Fujii T, Ueda H, Minoh M, 2007. A looking-for-objects service in ubiquitous home. J Nat Inst Inform Commun Technol, 54(3):175-181.

[6]Huynh SM, Parry D, Fong ACM, et al., 2014. Home localization system for misplaced objects. IEEE Int Conf on Consumer Electronics, p.462-463.

[7]Kollar T, Roy N, 2009. Utilizing object-object and object-scene context when planning to find things. IEEE Int Conf on Robotics and Automation, p.2168-2173.

[8]Komatsuzaki M, Tsukada K, Siio I, et al., 2011. IteMinder: finding items in a room using passive RFID tags and an autonomous robot (poster). Proc 13th Int Conf on Ubiquitous Computing, p.599-600.

[9]Lin YC, Wei ST, Yang SA, et al., 2015. Planning on searching occluded target object with a mobile robot manipulator. IEEE Int Conf on Robotics and Automation, p.3110-3115.

[10]Meng Q, Lee MH, 2006. Design issues for assistive robotics for the elderly. Adv Eng Inform, 20(2):171-186.

[11]PIR, 2018. Passive Infrared Sensor. https://en.wikipedia.org/wiki/Passive_infrared_sensor

[12]Ravi K, Bhavani KJ, Vinayak M, et al., 2016. Identification of misplaced objects. Indian J Sci Technol, 9(17):1-4.

[13]Redmon J, Divvala S, Girshick R, et al., 2016. You only look once: unified, real-time object detection. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.779-788.

[14]Rogers JG, Christensen HI, 2013. Robot planning with a semantic map. IEEE Int Conf on Robotics and Automation, p.2239-2244.

[15]ROS, 2018. Robot Operating System. http://www.ros.org/

[16]Sadhu C, Abhiram MH, Chandan B, et al., 2013. Cognitive learning enabled real time object search robot. Int Conf on Control, Automation, Robotics and Embedded System, p.1-6.

[17]Schwarz D, Schwarz M, Stückler J, et al., 2014. Cosero, find my keys! Object localization and retrieval using bluetooth low energy tags. Proc 18th RoboCup Int Symp, p.195-206.

[18]Susi M, Renaudin V, Lachapelle G, 2013. Motion mode recognition and step detection algorithms for mobile phone users. Sensors, 13(2):1539-1562.

[19]Tesoriero R, Gallud JA, Lozano MD, et al., 2009. Tracking autonomous entities using RFID technology. IEEE Trans Consum Electron, 55(2):650-655.

[20]Wang Q, 2018. Youtube Video on Robotic Misplaced Item Finding System. https://youtu.be/jEDKbr71pis

[21]Wang Q, Zhang SL, Liu MQ, et al., 2017. Retrieval of misplaced items using a mobile robot via visual object recognition. IEEE 7th Conf on CYBER Technology in Automation, Control, and Intelligent Systems, p.1188-1193.

[22]Xu D, Huang Q, Liu HW, 2016. Object detection on robot operation system. IEEE 11th Conf on Industrial Electronics and Applications, p.1155-1159.

[23]Xunfei, 2018. Voice Dictation. http://www.xfyun.cn/services/voicedictation

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE