CLC number: TP391.4
On-line Access: 2021-07-20
Received: 2020-03-27
Revision Accepted: 2020-06-07
Crosschecked: 2021-07-05
Cited: 0
Clicked: 4806
Citations: Bibtex RefMan EndNote GB/T7714
Sihan Zhu, Jian Pu. A self-supervised method for treatment recommendation in sepsis[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2000127 @article{title="A self-supervised method for treatment recommendation in sepsis", %0 Journal Article TY - JOUR
自监督脓毒症治疗推荐算法1华东师范大学计算机科学与技术学院,中国上海市,200062 2复旦大学类脑智能科学与技术研究院,中国上海市,200433 摘要:由于每个脓毒症患者治疗反应可能不同,为病人提供量身定制的治疗建议来帮助医生有效、准确地做出决定,并采取有效治疗方案,是降低医院重症监护病房死亡率的一项极具挑战性的工作。本文将强化学习应用于个人治疗推荐,采用对样本不确定性进行建模并评估的方法,根据患者对治疗的反应和状态,将患者样本分为两个域,然后使用辅助迁移学习任务重建两个域的样本,使用特权学习的蒸馏方法与用于迁移学习的变分自动编码器框架关联低质量域和高质量域间的任务。通过结合自监督方式获得更好的状态和动作表示,本文提出一种针对引起较高风险的不确定性进行控制的深度强化学习方法;模型提供一定的灵活性使之可以在不同场景对模糊样本做出保守预测或明确判断,并降低预期死亡率。在大规模公开可用的真实医疗数据集MIMIC-III上的实验表明,所提模型将总体估计死亡率降低了2.3%,并将主要估计死亡率降低到9.5%。 关键词组: Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article
Reference[1]Almirall D, Compton SN, Gunlicks-Stoessel M, et al., 2012. Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Stat Med, 31(17):1887-1902. [2]Asiain E, Clempner JB, Poznyak AS, 2018. A reinforcement learning approach for solving the mean variance customer portfolio in partially observable models. Int J Artif Intell Tools, 27(8):1850034. [3]Bajor JM, Lasko TA, 2017. Predicting medications from diagnostic codes with recurrent neural networks. Int Conf on Learning Representations, p.1-19. [4]Chen JG, Li KL, Rong HG, et al., 2018. A disease diagnosis and treatment recommendation system based on big data mining and cloud computing. Inform Sci, 435:124-149. [5]Chen Z, Marple K, Salazar E, et al., 2016. A physician advisory system for chronic heart failure management based on knowledge patterns. Theory Pract Log Progr, 16(5-6):604-618. [6]Futoma J, Hariharan S, Heller KA, et al., 2017. An improved multi-output Gaussian process RNN with real-time validation for early sepsis detection. Proc 2nd Machine Learning for Healthcare Conf, p.243-254. [7]Gidaris S, Singh P, Komodakis N, 2018. Unsupervised representation learning by predicting image rotations. Int Conf on Learning Representations, p.1-16. [8]Gunlicks-Stoessel M, Mufson L, Westervelt A, et al., 2016. A pilot smart for developing an adaptive treatment strategy for adolescent depression. J Clin Child Adolesc Psychol, 45(4):480-494. [9]Hendrycks D, Mazeika M, Kadavath S, et al., 2019. Using self-supervised learning can improve model robustness and uncertainty. Proc 33rd Conf on Neural Information Processing Systems, p.1-13. [10]Hinton G, Vinyals O, Dean J, 2015. Distilling the knowledge in a neural network. https://arxiv.org/abs/1503.02531 [11]Jiang N, Li LH, 2016. Doubly robust off-policy value evaluation for reinforcement learning. Proc 33rd Int Conf on Machine Learning, p.652-661. [12]Johnson AEW, Pollard TJ, Shen L, et al., 2016. MIMIC-III, a freely accessible critical care database. Sci Data, 3:160035. [13]Kaelbling LP, Littman ML, Moore AW, 1995. An introduction to reinforcement learning. In: Steels L (Ed.), The Biology and Technology of Intelligent Autonomous Agents. Springer, Berlin, p.90-127. [14]Katzman JL, Shaham U, Cloninger A, et al., 2018. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Method, 18(1):24. [15]Kingma DP, Welling M, 2014. Auto-encoding variational Bayes. Int Conf on Learning Representations Ithacap, p.1-14. [16]Kingma DP, Salimans T, Jozefowicz R, et al., 2016. Improved variational inference with inverse autoregressive flow. Proc 30th Int Conf on Neural Information Processing Systems, p.4743-4751. [17]Komorowski M, Celi LA, Badawi O, et al., 2018. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med, 24(11):1716-1720. [18]Li Y, Zeng JB, Shan SG, et al., 2019. Self-supervised representation learning from videos for facial action unit detection. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.10924-10933. [19]Long M, Cao Y, Wang J, et al., 2015. Learning transferable features with deep adaptation networks. Int Conf on Machine Learning, p.97-105. [20]Lopez-Paz D, Bottou L, Schölkopf B, et al., 2016. Unifying distillation and privileged information. https://arxiv.org/abs/1511.03643 [21]Mnih V, Kavukcuoglu K, Silver D, et al., 2015. Playing Atari with deep reinforcement learning. https://arxiv.org/abs/1312.5602 [22]Nemati S, Ghassemi MM, Clifford GD, 2016. Optimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach. Proc 38th Annual Int Conf of the IEEE Engineering in Medicine and Biology Society, p.2978-2981. [23]Peng XF, Ding Y, Wihl D, et al., 2018. Improving sepsis treatment strategies by combining deep and kernel-based reinforcement learning. American Medical Informatics Association® Annual Symp, p.887-896. [24]Raghu A, Komorowski M, Ahmed I, et al., 2017. Deep reinforcement learning for sepsis treatment. Proc 31st Conf on Neural Information Processing Systems, p.1-9. [25]Raghu A, Komorowski M, Singh S, 2018. Model-based reinforcement learning for sepsis treatment. https://arxiv.org/abs/1811.09602 [26]Saria S, 2018. Individualized sepsis treatment using reinforcement learning. Nat Med, 24(11):1641-1642. [27]Shortreed SM, Laber E, Lizotte DJ, et al., 2011. Informing sequential clinical decision-making through reinforcement learning: an empirical study. Mach Learn, 84(1-2):109-136. [28]Singer M, Deutschman CS, Seymour CW, et al., 2016. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA, 315(8):801-810. [29]Vapnik V, Izmailov R, 2015. Learning using privileged information: similarity control and knowledge transfer. J Mach Learn Res, 16(1):2023-2049. [30]Vondrick C, Pirsiavash H, Torralba A, 2016. Anticipating visual representations from unlabeled video. IEEE Conf on Computer Vision and Pattern Recognition, p.98-106. [31]Wang L, Zhang W, He XF, et al., 2018. Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation. Proc 24th ACM SIGKDD Int Conf on Knowledge Discovery &Data Mining, p.2447-2456. [32]Wang ZY, Schaul T, Hessel M, et al., 2016. Dueling network architectures for deep reinforcement learning. Proc 33rd Int Conf on Machine Learning, p.1995-2003. [33]Weng WH, Gao MW, He Z, et al., 2017. Representation and reinforcement learning for personalized glycemic control in septic patients. Proc 31st Conf on Neural Information Processing Systems, p.1-5. [34]Yu C, Liu JM, Nemati S, 2019. Reinforcement learning in healthcare: a survey. https://arxiv.org/abs/1908.08796 [35]Zhai XH, Oliver A, Kolesnikov A, et al., 2019. S4L: self-supervised semi-supervised learning. IEEE/CVF Int Conf on Computer Vision, p.1476-1485. [36]Zhang YT, Chen R, Tang J, et al., 2017. LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. Proc 23rd ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.1315-1324. [37]Zhao SJ, Song JM, Ermon S, 2017. InfoVAE: information maximizing variational autoencoders. https://arxiv.org/abs/1706.02262 Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou
310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE |
Open peer comments: Debate/Discuss/Question/Opinion
<1>