JZUS - Journal of Zhejiang University SCIENCE

Frontiers of Information Technology & Electronic Engineering

Accepted manuscript available online (unedited version)

Three trustworthiness challenges in large language model based financial systems: real-world examples and mitigation strategies

Author(s): Shurui XU¹, Feng LUO², Shuyan LI¹, Mengzhen FAN³, Zhongtian SUN⁴
Affiliation(s): ¹School of Electronics, Electrical Engineering and Computer Science, Queen's University, Belfast BT7 1NN, UK ²Department of Computer Science, Rice University, Houston 77005, USA ³HSBC Business School, Peking University, University Town, Shenzhen 518055, China ⁴School of Computing, University of Kent, Canterbury, CT2 7NZ, UK
Corresponding email(s): li-sy16@tsinghua.org.cn
Key Words: Trustworthy artificial intelligence; Large language models; Finance; Fintech

Share this article to： More <<< Previous Paper \|Next Paper >>>

Shurui XU¹, Feng LUO², Shuyan LI¹, Mengzhen FAN³, Zhongtian SUN⁴. Three trustworthiness challenges in large language model based financial systems: real-world examples and mitigation strategies[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2500421

@article{title="Three trustworthiness challenges in large language model based financial systems: real-world examples and mitigation strategies",
author="Shurui XU¹, Feng LUO², Shuyan LI¹, Mengzhen FAN³, Zhongtian SUN⁴",
journal="Frontiers of Information Technology & Electronic Engineering",
year="in press",
publisher="Zhejiang University Press & Springer",
doi="https://doi.org/10.1631/FITEE.2500421"
}

%0 Journal Article
%T Three trustworthiness challenges in large language model based financial systems: real-world examples and mitigation strategies
%A Shurui XU¹
%A Feng LUO²
%A Shuyan LI¹
%A Mengzhen FAN³
%A Zhongtian SUN⁴
%J Frontiers of Information Technology & Electronic Engineering
%P
%@ 2095-9184
%D in press
%I Zhejiang University Press & Springer
doi="https://doi.org/10.1631/FITEE.2500421"

TY - JOUR
T1 - Three trustworthiness challenges in large language model based financial systems: real-world examples and mitigation strategies
A1 - Shurui XU¹
A1 - Feng LUO²
A1 - Shuyan LI¹
A1 - Mengzhen FAN³
A1 - Zhongtian SUN⁴
J0 - Frontiers of Information Technology & Electronic Engineering
SP -
EP -
%@ 2095-9184
Y1 - in press
PB - Zhejiang University Press & Springer
ER -
doi="https://doi.org/10.1631/FITEE.2500421"

Abstract
Chinese Summary
Academic Network
Reviewer Comment

Abstract: The integration of large language models (LLMs) into financial applications has demonstrated remarkable potential for enhancing decision-making processes, automating operations, and delivering personalized services. However, the high-stakes nature of financial systems demands a very high level of trustworthiness that current LLMs often fail to meet. This study identifies and examines three major trustworthiness challenges in LLM-based financial systems: (1) Jailbreak Prompts that exploit vulnerabilities in model alignment to produce harmful or noncompliant responses; (2) Hallucination, where models generate factually incorrect outputs that can mislead financial decision- making; and (3) Bias and Fairness concerns, where demographic or institutional bias embedded in LLMs may result in unfair treatment of individuals or regions. To make these risks concrete, we designed three finance-relevant probes and evaluated a set of mainstream LLMs spanning both proprietary and open-source families. Across models, we observed risky behavior in at least one scenario per probe.Based on these findings, we systematically summarized the existing mitigation strategies that aim to address these risks. We argued that resolving these issues is not only vital for ensuring the responsible use of artificial intelligence (AI) in the financial sector but also for enabling its safe and scalable deployment.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

Open peer comments: Debate/Discuss/Question/Opinion

<1>

- Go to

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference