A Comprehensive Survey on Large Language Models: Architectures, Applications, and Ethical Considerations
Downloads
The rapid progress and revolutionary potential of large language models (LLMs) are examined in this survey, with particular attention paid to transformer-based designs and scaling strategies that power models such as GPT-4, Gemini, and LLaMA. Focusing on advancements in natural language comprehension, translation, text production, and information retrieval, this study examines various applications in the fields of healthcare, finance, education, and public administration. Along with addressing ethical issues such as algorithmic bias, disinformation, privacy threats, and environmental impacts, the study promotes robust auditing frameworks, consistent evaluation measures, and environmentally friendly practices. The study offers a comprehensive framework for future research and the responsible development of large language models (LLMs), outlining the complex supply chain and development processes with a focus on ethical integration, accountability, and transparency.
J. A. Goldstein, G. Sastry, M. Musser, R. DiResta, "Generative language models and automated influence operations: Emerging threats and potential mitigations, 2023. [PDF]
T. Teubner, C. M. Flath, C. Weinhardt, et al., "Welcome to the era of ChatGPT et al. the prospects of large language models," Business & Information Systems Engineering, vol. 2023, Springer. springer.com
C. Ling, X. Zhao, J. Lu, C. Deng, C. Zheng, J. Wang, "Domain specialization as the key to make large language models disruptive: A comprehensive survey,", 2023. [PDF]
X. Zhao, J. Lu, C. Deng, C. Zheng, J. Wang, "Beyond One-Model-Fits-All: A Survey of Domain Specialization for Large Language Models," arXiv preprint, 2023. academia.edu.
Hasan, D. A., Hussan, B. K., Zeebaree, S. R. M., Ahmed, D. M., Kareem, O. S., & Sadeeq, M. A. M. (2021). The impact of test case generation methods on the software performance: A review. International Journal of Science and Business, 5(6), 33–44. https://doi.org/10.5281/zenodo.4623940
S. B. Shah, S. Thapa, A. Acharya, K. Rauniyar, "Navigating the web of disinformation and misinformation: Large language models as double-edged swords," IEEE, 2024. ieee.org
D. Barman, Z. Guo, and O. Conlan, "The dark side of language models: Exploring the potential of llms in multimedia disinformation generation and dissemination," Machine Learning with Applications, 2024. sciencedirect.com
I. Augenstein, T. Baldwin, M. Cha, et al., "Factuality challenges in the era of large language models and opportunities for fact-checking," *Nature Machine Intelligence*, 2024. google.com
Mohammed, S. M., Jacksi, K., & Zeebaree, S. R. M. (2021). A state-of-the-art survey on semantic similarity for document clustering using GloVe and density-based algorithms. Indonesian Journal of Electrical Engineering and Computer Science, 22(1), 552–562.
https://doi.org/10.11591/ijeecs.v22.i1.pp552-562.
Raaian, M. A. K., Mukta, M. S. H., Fatema, K., and Fahad, N. M., "A review on large language models: Architectures, applications, taxonomies, open issues and challenges," IEEE, 2024. ieee.org.
Abdulazeez, A. M., Zeebaree, S. R. M., & Sadeeq, M. A. M. (2018). Design and implementation of electronic student affairs system. Academic Journal of Nawroz University, 7(3).
https://doi.org/10.25007/ajnu.v7n3a201.
M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, "Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects," Authorea, 2023. researchgate.net.
Zebari, R. R., Zeebaree, S. R. M., Jacksi, K., & Shukur, H. M. (2019). E-business requirements for flexibility and implementation enterprise system: A review. International Journal of Scientific & Technology Research, 8(11), 655. Retrieved from http://www.ijstr.org.
D. Rothman, "Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, Hugging Face, and …," 2022. 5.202.73.55.
N. Patwardhan, S. Marrone, and C. Sansone, "Transformers in the real world: A survey on nlp applications," Information, 2023. mdpi.com.
H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, "A comprehensive overview of large language models,", 2023. [PDF]
M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, "A survey on large language models: Applications, challenges, limitations, and practical usage," Authorea, 2023. figshare.com
Y. Xie, C. Yu, T. Zhu, J. Bai et al., "Translating natural language to planning goals with large-language models," arXiv preprint arXiv:2302.05128, 2023. [PDF]
Y. Wang, Y. S. Lin, R. Huang, J. Wang et al., "Enhancing user experience in large language models through human-centered design: Integrating theoretical insights with an experimental study to meet diverse …," arXiv preprint arXiv:2405.11505, 2024. [PDF]
Z. Tan and M. Jiang, "User modeling in the era of large language models: Current research and future directions," arXiv preprint arXiv:2312.11518, 2023. [PDF]
R. M. Samant, M. R. Bachute, S. Gite, and K. Kotecha, "Framework for deep learning-based language models using multi-task learning in natural language understanding: A systematic literature review and future directions," IEEE Access, 2022. ieee.org
P. Liu, Y. Ren, J. Tao, and Z. Ren, "Git-mol: A multi-modal large language model for molecular science with graph, image, and text," Computers in biology and medicine, 2024. [PDF].
Ibrahim, R. K., Zeebaree, S. R. M., & Jacksi, K. F. S. (2019). Survey on semantic similarity based on document clustering. Advances in Science Technology and Engineering Systems Journal, 115. https://doi.org/10.25046/aj040515
D. M. Petroșanu, A. Pîrjan, and A. Tăbușcă, "Tracing the influence of large language models across the most impactful scientific works," Electronics, 2023. mdpi.com
S. Ateia and U. Kruschwitz, "Can open-source LLMs compete with commercial models? Exploring the few-shot performance of current GPT models in biomedical tasks," arXiv preprint arXiv:2407.13511, 2024. [PDF]
Z. Liu, A. Zhang, H. Fei, E. Zhang, and X. Wang, "Prott3: Protein-to-text generation for text-based protein understanding,", 2024. [PDF]
E. Kasneci, K. Seßler, S. Küchemann, M. Bannert, "ChatGPT for good? On opportunities and challenges of large language models for education," Learning and Individual Differences, vol. 2023, Elsevier. osf.io
X. Wu, R. Duan, and J. Ni, "Unveiling security, privacy, and ethical concerns of ChatGPT," Journal of Information and Intelligence, 2024.
sciencedirect.com.
Jacksi, K., Dimililer, N., & Zeebaree, S. R. M. (2015). A survey of exploratory search systems based on LOD resources. In Proceedings of the 5th International Conference on Computing and Informatics (ICOCI 2015) (Paper No. 112). Istanbul, Turkey: Universiti Utara Malaysia.
O. Oviedo-Trespalacios, A. E. Peden, T. Cole-Hunter, et al., "The risks of using ChatGPT to obtain common safety-related information and advice," Safety Science, vol. 2023, Elsevier.
sciencedirect.com
S. Tan, B. Knott, Y. Tian, and D. J. Wu, "CryptGPU: Fast privacy-preserving machine learning on the GPU," in *2021 IEEE Symposium on …*, 2021. [PDF]
Y. Shen, L. Heacock, J. Elias, K. D. Hentel, B. Reig, and G. Shih, "ChatGPT and other large language models are double-edged swords," Radiology, 2023. rsna.org.
Wang, S., Zhao, Y., Hou, X., & Wang, H. (2024). Large Language Model Supply Chain: A Research Agenda. ACM Transactions on Software Engineering and Methodology.
https://doi.org/10.1145/3708531.
Javaid, S., Fahim, H., He, B., & Saeed, N. (2024). Large Language Models for UAVs: Current State and Pathways to the Future. IEEE Open Journal of Vehicular Technology. https://doi.org/10.1109/OJVT.2024.3446799.
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023). Auditing Large Language Models: A Three-Layered Approach. AI and Ethics, 4, 1085–1115. https://doi.org/10.1007/s43681-023-00289-2.
Papageorgiou, G., Sarlis, V., Maragoudakis, M., & Tjortjis, C. (2024). Enhancing E-Government Services through State-of-the-Art, Modular, and Reproducible Architecture over Large Language Models. Applied Sciences, 14(8259). https://doi.org/10.3390/app14188259.
Kong, Y., Nie, Y., Dong, X., Mulvey, J. M., Poor, H. V., Wen, Q., & Zohren, S. (2024). Large Language Models for Financial and Investment Management: Applications and Benchmarks. The Journal of Portfolio Management.
Omiye, J. A., Gui, H., Rezaei, S. J., Zou, J., & Daneshjou, R. (2022). Large Language Models in Medicine: The Potentials and Pitfalls.
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., Zheng, K., Lian, D., & Chen, E. (2024). When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities. World Wide Web, 27(42).https://doi.org/10.1007/s11280-024-01276-1.
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2024). Large Language Models in Medicine: Current Potential and Opportunities for Development.
Yan, Y., & Liu, H. (2024). Ethical Framework for AI Education Based on Large Language Models. Education and Information Technologies. https://doi.org/10.1007/s10639-024-13241-6.
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. (2024). Bias in large language models: Origin, evaluation, and mitigation. arXiv. http://arxiv.org/abs/2411.10915v1.
Zheng, Z., Ning, K., Zhong, Q., Chen, J., Chen, W., Guo, L., Wang, W., & Wang, Y. (2024). Towards an understanding of large language models in software engineering tasks [Preprint]. arXiv. http://arxiv.org/abs/2308.11396.
Desai, B., Patil, K., Patil, A., & Mehta, I. (2023). Large language models: A comprehensive exploration of modern AI’s potential and pitfalls. Journal of Innovative Technologies, 6. Retrieved from https://academicpinnacle.com.
Yuan, X., Hu, J., & Zhang, Q. (2023). A comparative analysis of cultural alignment in large language models in bilingual contexts.
Langston, O., & Ashford, B. (2024, August 02). Automated summarization of multiple document abstracts and contents using large language models. https://doi.org/10.36227/techrxiv.17262754.45577350/v1.
Zhao, J., Huang, C., & Li, X. (2024). A comparative study of cultural hallucination in large language models on culturally specific ethical questions [Preprint]. Research Square.
https://doi.org/10.21203/rs.3.rs-4566507/v1.
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., Weller, J., Kuhn, J., & Kasneci, G. (2024). ChatGPT for good? On opportunities and challenges of large language models for education.
Wong, S. M., Leung, H., & Wong, K. Y. (2024). Efficiency in language understanding and generation: An evaluation of four open‐source large language models [Preprint]. Research Square. https://doi.org/10.21203/rs.3.rs-4063228/v1.
Wang, S., Ouyang, Q., & Wang, B. (2024). Comparative evaluation of commercial large language models on PromptBench: An English and Chinese perspective [Preprint]. Research Square. https://doi.org/10.21203/rs.3.rs-3987793/v1.
Ong, J. C. L., Chang, S. Y.-H., Liu, N., Ting, D. S. W., Chew, L. S. T., et al. (2024). Ethical and Regulatory Challenges of Large Language Models in Medicine. Lancet Digital Health, 6, e428–e432. https://doi.org/10.1016/S2589-7500(24)00061-X.
Karabacak, M., & Margetis, K. (2023). Embracing large language models for medical applications: Opportunities and challenges. Cureus, 15(5), e39305. https://doi.org/10.7759/cureus.39305.
