Abstract: Current financial large language models (FinLLMs) exhibit two major limitations: the absence of standardized evaluation metrics for stock analysis quality and insufficient analytical depth. We address these limitations with two contributions. First, we introduce AnalyScore, a systematic framework for evaluating the quality of stock analysis. Second, we construct Stocksis, an expert-curated dataset designed to enhance the financial analysis capabilities of large language models (LLMs). Building on Stocksis, together with a novel integration framework and quantitative tools, we develop FinSphere, an artificial intelligence (AI) agent that generates professional-grade stock analysis reports. Evaluations with AnalyScore show that FinSphere consistently surpasses general-purpose LLMs, domain-specific FinLLMs, and existing agent-based systems, even when the latter are enhanced with real-time data access and few-shot guidance. The findings highlight FinSphere's significant advantages in analytical quality and real-world applicability.

FinSphere：一款搭载指令微调大语言模型及集成领域专用工具的实时股票分析代理

[1]Bhat R, Jain B, 2024. Stock price trend prediction using emotion analysis of financial headlines with distilled LLM model. Proc 17^th Int Conf on Pervasive Technologies Related to Assistive Environments, p.67-73.

[2]Chen J, Zhou PL, Hua YN, et al., 2024. FinTextQA: a dataset for long-form financial question answering. Proc 62^nd Annual Meeting of the Association for Computational Linguistics, p.6025-6047.

[3]Chen ZY, Chen WH, Smiley C, et al., 2021. FinQA: a dataset of numerical reasoning over financial data. Proc Conf on Empirical Methods in Natural Language Processing, p.3697-3711.

[4]DeepSeek-AI, 2024. DeepSeek-V3 technical report. https://arxiv.org/abs/2412.19437

[5]Ding H, Li YH, Wang JH, et al., 2024. Large language model agent in financial trading: a survey. https://arxiv.org/abs/2408.06361

[6]Guo X, Xia HT, Liu ZW, et al., 2025. FinEval: a Chinese financial domain knowledge evaluation benchmark for large language models. Proc Conf of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, p.6258-6292.

[7]Gupta U, 2023. GPT-InvestAR: enhancing stock investment strategies through annual report analysis with large language models.

[8]Han SJ, Kang HQ, Jin B, et al., 2024. XBRL agent: leveraging large language models for financial report analysis. Proc 5^th ACM Int Conf on AI in Finance, p.856-864.

[9]Huang AH, Wang H, Yang Y, 2023. FinBERT: a large language model for extracting information from financial text. Contemporary Acc Res, 40(2):806-841.

[10]Islam P, Kannappan A, Kiela D, et al., 2023. FinanceBench: a new benchmark for financial question answering.

[11]Kim A, Muhn M, Nikolaev V, 2024. Financial statement analysis with large language models.

[12]Krause D, 2023. Large language models and generative AI in finance: an analysis of ChatGPT, Bard, and Bing AI. SSRN Electr J.

[13]Lei Y, Li JT, Jiang M, et al., 2023. CFBenchmark: Chinese financial assistant benchmark for large language model.

[14]Li HX, Gao HY, Wu CZ, et al., 2025. Extracting financial data from unstructured sources: leveraging large language models. J Inform Syst, 39(1):135-156.

[15]Li YH, Wang SF, Ding H, et al., 2023. Large language models in finance: a survey. Pro 4^th ACM Int Conf on AI in Finance, p.374-382.

[16]Lin CY, 2004. ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, p.74-81. https://aclanthology.org/W04-1013

[17]Liu CH, Arulappan A, Naha R, et al., 2024. Large language models and sentiment analysis in financial markets: a review, datasets and case study. IEEE Access, 12:134041-134061.

[18]Liu Z, Huang DG, Huang KY, et al., 2020. FinBERT: a pre-trained financial language representation model for financial text mining. Proc 29^th Int Joint Conf on Artificial Intelligence, p.4513-4519.

[19]Ni HW, Meng SC, Chen XP, et al., 2024. Harnessing earnings reports for stock predictions: a QLoRA-enhanced LLM approach.

[20]Nie YQ, Kong YX, Dong XW, et al., 2024. A survey of large language models for financial applications: progress, prospects and challenges.

[21]Papineni K, Roukos S, Ward T, et al., 2002. BLEU: a method for automatic evaluation of machine translation. Proc 40^th Annual Meeting of the Association for Computational Linguistics, p.311-318.

[22]Park T, 2024. Enhancing anomaly detection in financial markets with an LLM-based multi-agent framework.

[23]Wu SJ, Irsoy O, Lu S, et al., 2023. BloombergGPT: a large language model for finance.

[24]Xie QQ, Han WG, Zhang X, et al., 2023. PIXIU: a large language model, instruction data and evaluation benchmark for finance. Proc 37^th Int Conf on Neural Information Processing Systems, Article 1454. https://dl.acm.org/doi/10.5555/3666122.3667576

[25]Yang HY, Liu XY, Wang CD, 2023. FinGPT: open-source financial large language models.

[26]Yang HY, Zhang BY, Wang N, et al., 2024. FinRobot: an open-source AI agent platform for financial applications using large language models.

[27]Yang Y, Uy MCS, Huang A, 2020. FinBERT: a pretrained language model for financial communications.

[28]Yang Y, Tang YX, Tam KY, 2023. InvestLM: a large language model for investment using financial domain instruction tuning.

[29]Yu YY, Li HH, Chen Z, et al., 2024. FinMem: a performance-enhanced LLM trading agent with layered memory and character design. Proc AAAI Symp Series, 3(1):595-597.

[30]Zhang BY, Yang HY, Liu XY, 2023. Instruct-FinGPT: financial sentiment analysis by instruction tuning of general-purpose large language models.

[31]Zhang WT, Zhao LX, Xia HC, et al., 2024. A multimodal foundation agent for financial trading: tool-augmented, diversified, and generalist. Proc 30^th ACM SIGKDD Conf on Knowledge Discovery and Data Mining, p.4314-4325.

[32]Zhao HQ, Liu ZL, Wu ZH, et al., 2024. Revolutionizing finance with LLMs: an overview of applications and insights.

[33]Zhu FB, Lei WQ, Huang YC, et al., 2021. TAT-QA: a question answering benchmark on a hybrid of tabular and textual content in finance. Proc 59^th Annual Meeting of the Association for Computational Linguistics and the 11^th Int Joint Conf on Natural Language Processing, p.3277-3287.