Full Text:   <3061>

CLC number: TP399; R318

On-line Access: 2011-05-09

Received: 2010-06-18

Revision Accepted: 2010-11-10

Crosschecked: 2011-03-15

Cited: 16

Clicked: 4862

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE C 2011 Vol.12 No.5 P.351-361


A hybrid brain-computer interface control strategy in a virtual environment

Author(s):  Yu Su, Yu Qi, Jian-xun Luo, Bian Wu, Fan Yang, Yi Li, Yue-ting Zhuang, Xiao-xiang Zheng, Wei-dong Chen

Affiliation(s):  Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou 310027, China, School of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China, Department of Biomedical Engineering, Zhejiang University, Hangzhou 310027, China, Key Laboratory of Biomedical Engineering of the Ministry of Education, Hangzhou 310027, China

Corresponding email(s):   qaas@zju.edu.cn

Key Words:  Hybrid brain-computer interface (BCI) control strategy, P300 potential, Sensorimotor rhythms, Virtual environment

Share this article to: More |Next Article >>>

Yu Su, Yu Qi, Jian-xun Luo, Bian Wu, Fan Yang, Yi Li, Yue-ting Zhuang, Xiao-xiang Zheng, Wei-dong Chen. A hybrid brain-computer interface control strategy in a virtual environment[J]. Journal of Zhejiang University Science C, 2011, 12(5): 351-361.

@article{title="A hybrid brain-computer interface control strategy in a virtual environment",
author="Yu Su, Yu Qi, Jian-xun Luo, Bian Wu, Fan Yang, Yi Li, Yue-ting Zhuang, Xiao-xiang Zheng, Wei-dong Chen",
journal="Journal of Zhejiang University Science C",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T A hybrid brain-computer interface control strategy in a virtual environment
%A Yu Su
%A Yu Qi
%A Jian-xun Luo
%A Bian Wu
%A Fan Yang
%A Yi Li
%A Yue-ting Zhuang
%A Xiao-xiang Zheng
%A Wei-dong Chen
%J Journal of Zhejiang University SCIENCE C
%V 12
%N 5
%P 351-361
%@ 1869-1951
%D 2011
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.C1000208

T1 - A hybrid brain-computer interface control strategy in a virtual environment
A1 - Yu Su
A1 - Yu Qi
A1 - Jian-xun Luo
A1 - Bian Wu
A1 - Fan Yang
A1 - Yi Li
A1 - Yue-ting Zhuang
A1 - Xiao-xiang Zheng
A1 - Wei-dong Chen
J0 - Journal of Zhejiang University Science C
VL - 12
IS - 5
SP - 351
EP - 361
%@ 1869-1951
Y1 - 2011
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.C1000208

This paper presents a hybrid brain-computer interface (BCI) control strategy, the goal of which is to expand control functions of a conventional motor imagery or a p300 potential based BCI in a virtual environment. The hybrid control strategy utilizes p300 potential to control virtual devices and motor imagery related sensorimotor rhythms to navigate in the virtual world. The two electroencephalography (EEG) patterns serve as source signals for different control functions in their corresponding system states, and state switch is achieved in a sequential manner. In the current system, imagination of left/right hand movement was translated into turning left/right in the virtual apartment continuously, while p300 potentials were mapped to discrete virtual device control commands using a five-oddball paradigm. The combination of motor imagery and P300 patterns in one BCI system for virtual environment control was tested and the results were compared with those of a single motor imagery or P300-based BCI. Subjects obtained similar performances in the hybrid and single control tasks, which indicates the hybrid control strategy works well in the virtual environment.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Allison, B.Z., Brunner, C., Kaiser, V., Müller-Putz, G.R., Neuper, C., Pfurtscheller, G., 2010. Toward a hybrid brain-computer interface based on imagined movement and visual attention. J. Neural Eng., 7(2):026007.

[2]Bayliss, J.D., 2003. Use of the evoked potential P3 component for control in a virtual apartment. IEEE Trans. Neural Syst. Rehabil. Eng., 11(2):113-116.

[3]Bayliss, J.D., Ballard, D.H., 2000. A virtual reality testbed for brain-computer interface research. IEEE Trans. Rehabil. Eng., 8(2):188-190.

[4]Bishop, C.M., 2006. Pattern Recognition and Machine Learning. Springer, USA, p.179-192.

[5]Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Müller, K.R., 2008. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process. Mag., 25(1):41-56.

[6]Brunner, C., Allison, B.Z., Krusienski, D.J., Kaiser, V., Müller-Putz, G.R., Pfurtscheller, G., Neuper, C., 2010. Improved signal processing approaches in an offline simulation of a hybrid brain-computer interface. J. Neurosci. Meth., 188(1):165-173.

[7]Chen, W.D., Zhang, J.H., Zhang, J.C., Li, Y., Qi, Y., Su, Y., Wu, B., Zhang, S.M., Dai, J.H., Zheng, X.X., et al., 2010. A P300 based online brain-computer interface system for virtual hand control. J. Zhejiang Univ.-Sci. C (Comput. & Electron.), 11(8):587-597.

[8]Cristianini, N., Shawe-Taylor, J., 2000. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge, UK, p.103-131.

[9]Croft, R.J., Barry, R.J., 2000. Removal of ocular artifact from the EEG: a review. Clin. Neurophysiol., 30(1):5-19.

[10]Donchin, E., Spencer, K.M., Wijesinghe, R., 2000. The mental prosthesis: assessing the speed of a P300-based brain-computer interface. IEEE Trans. Rehabil. Eng., 8(2):174-179.

[11]Edlinger, G., Krausz, G., Groenegress, C., Holzner, C., Guger, C., Slater, M., 2008. Brain-Computer Interfaces for Virtual Environment Control. Proc. 13th Int. Conf. on Biomedical Engineering, p.366-369.

[12]Farwell, L.A., Donchin, E., 1988. Taking off the top of your head-toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol., 70(6):510-523.

[13]Kronegg, J., Chanel, G., Voloshynovskiy, S., Pun, T., 2007. EEG-based synchronized brain-computer interfaces: a model for optimizing the number of mental tasks. IEEE Trans. Neural Syst. Rehabil. Eng., 15(1):50-58.

[14]Leeb, R., Pfurtscheller, G., 2004. Walking Through a Virtual City by Thought. Proc. 26th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society, p.4503-4506.

[15]Leeb, R., Lee, F., Keinrath, C., Scherer, R., Bischof, H., Pfurtscheller, G., 2007a. Brain-computer communication: motivation, aim, and impact of exploring a virtual apartment. IEEE Trans. Neural Syst. Rehabil. Eng., 15(4):473-482.

[16]Leeb, R., Friedman, D., Müller-Putz, G.R., Scherer, R., Slater, M., Pfurtscheller, G., 2007b. Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic. Comput. Intell. Neurosci., 2007:79642.

[17]Middendorf, M., McMillan, G., Calhoun, G., Jones, K.S., 2000. Brain-computer interfaces based on the steady-state visual-evoked response. IEEE Trans. Rehabil. Eng., 8(2):211-214.

[18]Obermaier, B., Neuper, C., Guger, C., Pfurtscheller, G., 2001. Information transfer rate in a five-classes brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng., 9(3):283-288.

[19]Pfurtscheller, G., Neuper, C., 2001. Motor imagery and direct brain-computer communication. Proc. IEEE, 89(7):1123-1134.

[20]Pfurtscheller, G., Allison, B.Z., Brunner, C., Bauernfeind, G., Solis-Escalante, T., Scherer, R., Zander, T.O., Müller-Putz, G.R., Neuper, C., Birbaumer, N., 2010a. The hybrid BCI. Front. Neurosci., 4:30.

[21]Pfurtscheller, G., Solis-Escalante, T., Ortner, R., Linortner, P., Müller-Putz, G.R., 2010b. Self-paced operation of an SSVEP-based orthosis with and without an imagery-based “brain switch”: a feasibility study towards a hybrid BCI. IEEE Trans. Neural Syst. Rehabil. Eng., 18(4):409-414.

[22]Piccione, F., Priftis, K., Tonin, P., Vidale, D., Furlan, R., Cavinato, M., Merico, A., Piron, L., 2008. Task and stimulation paradigm effects in a P300 brain computer interface exploitable in a virtual environment: a pilot study. PsychNol. J., 6(1):99-108.

[23]Ramoser, H., Müller-Gerking, J., Pfurtscheller, G., 2000. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng., 8(4):441-446.

[24]Ron-Angevin, R., Diaz-Estrella, A., 2009. Brain-computer interface: changes in performance using virtual reality techniques. Neurosci. Lett., 449(2):123-127.

[25]Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R., 2004. BCI2000: a general-purpose, brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng., 51(6):1034-1043.

[26]Scherer, R., Lee, F., Schloegl, A., Leeb, R., Bischof, H., Pfurtscheller, G., 2008. Toward self-paced brain-computer communication: navigation through virtual worlds. IEEE Trans. Biomed. Eng., 55(2):675-682.

[27]Su, Y., Wu, B., Chen, W., Zhang, J., Jiang, J., Zhuang, Y.,Zheng, X., 2008. P300-based brain computer interface: prototype of a Chinese speller. J. Comput. Inform. Syst., 4(4):1515-1522.

[28]Velasco-Álvarez, F., Ron-Angevin, R., 2009. Asynchronous brain-computer interface to navigate in virtual environments using one motor imagery. LNCS, 5517:698-705.

[29]Wolpaw, J.R., Birbaumer, N., Heetderks, W.J., McFarland, D.J., Peckham, P.H., Schalk, G., Donchin, E., Quatrano, L.A., Robinson, C.J., Vaughan, T.M., 2000. Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng., 8(2):164-173.

[30]Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M., 2002. Brain-computer interfaces for communication and control. Clin. Neurophysiol., 113(6):767-791.

[31]Zhao, Q.B., Zhang, L.Q., Cichockis, A., 2009. EEG-based asynchronous BCI control of a car in 3D virtual reality environments. Chin. Sci. Bull., 54(1):78-87.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE