SE4LLM Literature Search

This repository is used for literature retrieval and manual review for Software Engineering for Large Language Models: Research Status, Challenges and the Road Ahead.

Cited Publications

• "Vorbești Românește?" A Recipe to Train Powerful Romanian LLMs with English Instructions
• EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection
• (Why) Is My Prompt Getting Worse? Rethinking Regression Testing for Evolving LLM APIs
• A comprehensive overview of large language models
• A First Look at Bugs in LLM Inference Engines
• A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage
• A First Look at Model Supply Chain: From the Risk Perspective
• A pretrainer's guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity
• A review on code generation with llms: Application and evaluation
• A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation
• A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)
• A survey of large language models
• A survey of resource-efficient llm and multimodal foundation models
• A Survey of the Model Context Protocol (MCP): Standardizing Context to Enhance Large Language Models (LLMs)
• A survey on evaluation of large language models
• A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency
• A survey on large language model (llm) security and privacy: The good, the bad, and the ugly
• A survey on large language models for code generation
• A survey on large language models: Applications, challenges, limitations, and practical usage
• A survey on llm-as-a-judge
• A survey on model compression for large language models
• A systematic evaluation of large language models of code
• A systematic literature review on large language models for automated program repair
• AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling
• Adaptive contextualization in large language models using dynamic semantic drift encoding
• Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents
• AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents
• Alignment faking in large language models
• An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection
• An empirical analysis and resource footprint study of deploying large language models on edge devices
• An empirical study of catastrophic forgetting in large language models during continual fine-tuning
• An empirical study of llama3 quantization: From llms to mllms
• An Empirical Study of the Non-Determinism of ChatGPT in Code Generation
• An Empirical Study on Influence-Based Pretraining Data Selection for Code Large Language Models
• Angel-ptm: A scalable and economical large-scale pre-training system in tencent
• Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study
• Are you still on track!? Catching LLM Task Drift with Activations
• Arithmetic control of llms for diverse user preferences: Directional preference alignment with multi-objective rewards
• Art or artifice? large language models and the false promise of creativity
• Autom3l: An automated multimodal machine learning framework with large language models
• Automatic dataset construction (adc): Sample collection, data curation, and beyond
• Aya dataset: An open-access collection for multilingual instruction tuning
• Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol Ecosystem
• Bias and unfairness in information retrieval systems: New challenges in the llm era
• C2KD: Bridging the Modality Gap for Cross-Modal Knowledge Distillation
• Cache side-channel attacks and defenses of the sliding window algorithm in TEEs
• Can llms reason like humans? assessing theory of mind reasoning in llms for open-ended questions
• Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering
• Can we trust large language models generated code? a framework for in-context learning, security patterns, and code evaluations across diverse llms
• CAT-LM training language models on aligned code and tests
• Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks
• CausalBench: A Comprehensive Benchmark for Evaluating Causal Reasoning Capabilities of Large Language Models
• CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaboration
• Challenges and applications of large language models
• Challenges in Testing Large Language Model Based Software: A Faceted Taxonomy
• Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems
• ChatGPT for good? On opportunities and challenges of large language models for education
• Cipherfix: Mitigating ciphertext Side-Channel attacks in software
• Complexcodeeval: A benchmark for evaluating large code models on more complex code
• Concerned with Data Contamination? Assessing Countermeasures in Code Language Model
• Confidential Computing Across Edge-To-Cloud for Machine Learning: A Survey Study
• CoSec: On-the-Fly Security Hardening of Code LLMs via Supervised Co-decoding
• Cost-Effective Online Multi-LLM Selection with Versatile Reward Models
• Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
• Curlora: Stable llm continual fine-tuning and catastrophic forgetting mitigation
• CVE-Poisoning: Towards Human-Guided and Cost-Effective Detection of a Novel AI Data Poisoning Attack
• DataRecipe - How to Cook the Data for CodeLLM?
• Datasets for large language models: A comprehensive survey
• DebUnc: mitigating hallucinations in large language model agent communication with uncertainty estimations
• Deep learning model reuse in the huggingface community: Challenges, benefit and trends
• DeepSeek-V3 Technical Report
• Demystifying rce vulnerabilities in llm-integrated apps
• Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation
• Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
• Dynamic Evaluation of Large Language Models by Meta Probing Agents
• Dynamollm: Designing llm inference clusters for performance and energy efficiency
• Eagle: Efficient training-free router for multi-llm inference
• Edgeshard: Efficient llm inference via collaborative edge computing
• Effective test generation using pre-trained large language models and mutation testing
• Effi-Code: Unleashing Code Efficiency in Language Models
• Efficient continual pre-training by mitigating the stability gap
• Efficient large multi-modal models via visual context compression
• Efficient large-scale language model training on gpu clusters using megatron-lm
• Efficient Mixed-Precision Large Language Model Inference with TurboMind
• Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters
• Efficient training of large language models on distributed infrastructures: a survey
• Embodied Question Answering via Multi-LLM Systems
• Empowering AI-Powered Industrial Design Software: Introducing Constraint Satisfaction Problem for Requirement Alignment
• Enhancing Security Awareness and Education for LLMs
• Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies
• Evaluating clone detection tools with BigCloneBench
• Evaluating cognitive maps and planning in large language models with CogEval
• Evaluating large language models: A comprehensive survey
• Examining forgetting in continual pre-training of aligned large language models
• Exegpt: Constraint-aware resource scheduling for llm inference
• Exploring large language model based intelligent agents: Definitions, methods, and prospects
• FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
• Federated Large Language Model: A Position Paper
• Federatedscope-llm: A comprehensive package for fine-tuning large language models in federated learning
• Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making
• Fine-tuning large language models with user-level differential privacy
• From generation to judgment: Opportunities and challenges of llm-as-a-judge
• From pre-training corpora to large language models: What factors influence llm performance in causal discovery tasks?
• Fuzz4all: Universal fuzzing with large language models
• Green LLM Techniques in Action: How Effective Are Existing Techniques for Improving the Energy Efficiency of LLM-Based Applications in Industry?
• Grounding and evaluation for large language models: Practical challenges and lessons learned (survey)
• Groundinggpt: Language enhanced multi-modal grounding model
• Healai: A healthcare llm for effective medical documentation
• Hidden Technical Debt in Machine Learning Systems.
• Hierarchical Prediction-based Management for LMaaS Systems
• Holistic evaluation of language models
• How Can Synthetic Data Improve Multilingual Language Model Pretraining? A Data Quality Perspective
• How does Machine Learning Change Software Development Practices?
• How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients
• How is ChatGPT's behavior changing over time?
• How mature is requirements engineering for AI-based systems? A systematic mapping study on practices, challenges, and future research directions
• Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent
• Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads
• Hybrid slm and llm for edge-cloud collaborative inference
• Improving Data Efficiency via Curating LLM-Driven Rating Systems
• Improving multimodal large language models using continual learning
• Inadequacies of large language model benchmarks in the era of generative artificial intelligence
• InferDPT: Privacy-preserving inference for black-box large language model
• Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents
• Intent Assurance using LLMs guided by Intent Drift
• Investigating bias in llm-based bias detection: Disparities between llms and human perception
• Investigating Energy Efficiency and Performance Trade-offs in LLM Inference Across Tasks and DVFS Settings
• Investigating Markers and Drivers of Gender Bias in Machine Translations
• Investigating the catastrophic forgetting in multimodal large language model fine-tuning
• Is Attention All You Need? Toward a Conceptual Model for Social Awareness in Large Language Models
• Is poisoning a real threat to LLM alignment? Maybe more so than you think
• Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation
• It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization
• Jatmo: Prompt injection defense by task-specific finetuning
• Justice or prejudice? quantifying biases in llm-as-a-judge
• Large language model (llm) ai text generation detection based on transformer deep learning algorithm
• Large language model (llm) for telecommunications: A comprehensive survey on principles, key techniques, and opportunities
• Large Language Model Agent: A Survey on Methodology, Applications and Challenges
• Large language model as attributed training data generator: A tale of diversity and bias
• Large Language Model Operations (LLMOps): Definition, Challenges, and Lifecycle Management
• Large language model-based agents for software engineering: A survey
• Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches
• Large language models for software engineering: A systematic literature review
• Large language models for software engineering: Survey and open problems
• Large Language Models in Cybersecurity: Threats, Exposure and Mitigation
• Large language models in medicine
• Learning to generate instruction tuning datasets for zero-shot task adaptation
• Lessleak-bench: A first investigation of data leakage in llms across 83 software engineering benchmarks
• Llama 2: Open foundation and fine-tuned chat models
• LLM-Datasets: An Open Framework for Pretraining Datasets of Large Language Models
• LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Services
• LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
• LLMGuard : Safeguarding Real-Time Inference for Large Language Models on Edge Devices
• LLMRCA: Multilevel Root Cause Analysis for LLM Applications Using Multimodal Observability Data
• Llms-as-judges: a comprehensive survey on llm-based evaluation methods
• LoRAMoE: Alleviating world knowledge forgetting in large language models via MoE-style plugin
• Lost in the Middle: How Language Models Use Long Contexts
• Maintainability and Scalability in Machine Learning: Challenges and Solutions
• MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
• MARS: A Benchmark for Multi-LLM Algorithmic Routing System
• MCP Guardian: A Security-First Layer for Safeguarding MCP-Based AI System
• MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits
• Medbench: A large-scale chinese benchmark for evaluating medical large language models
• Medical large language models are vulnerable to data-poisoning attacks
• Meta-requirements for LLM-Based Knowledge Exploration Tools in Information Systems Research
• Metamorphic testing of large language models for natural language processing
• Mftcoder: Boosting code llms with multitask fine-tuning
• Mitigating hallucination in large multi-modal models via robust instruction tuning
• MNN: A universal and efficient inference engine
• Mobile edge intelligence for large language models: A contemporary survey
• Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers
• Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions
• Model editing at scale leads to gradual and catastrophic forgetting
• Modular pluralism: Pluralistic alignment via multi-llm collaboration
• Monkey: Image resolution and text label are important things for large multi-modal models
• More than catastrophic forgetting: Integrating general capabilities for domain-specific llms
• Mpc-minimized secure llm inference
• Mtlora: Low-rank adaptation approach for efficient multi-task learning
• MULTI-CASE: A Transformer-based Ethics-aware Multimodal Investigative Intelligence Framework
• Multi-tier privacy protection for large language models using differential privacy
• Mutation-guided llm-based test generation at meta
• No free lunch theorem for privacy-preserving llm inference
• Non-functional requirements for machine learning: Understanding current use and challenges in industry
• Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection
• On llms-driven synthetic data generation, curation, and evaluation: A survey
• On the creativity of large language models
• On the Effectiveness of LLM-as-a-Judge for Code Generation and Summarization
• Optimizing llm inference clusters for enhanced performance and energy efficiency
• PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Training
• Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance
• Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change
• PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning
• Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models
• Poster: An Exploration of Large Language Models in Malicious Source Code Detection
• Power-softmax: Towards secure llm inference over encrypted data
• Practical and ethical challenges of large language models in education: A systematic scoping review
• Pride and prejudice: LLM amplifies self-bias in self-refinement
• Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap
• Prompt infection: Llm-to-llm prompt injection within multi-agent systems
• Prompt Injection attack against LLM-integrated Applications
• Prompt-to-SQL Injections in LLM-Integrated Web Applications: Risks and Defenses
• PromptDebt: A Comprehensive Study of Technical Debt Across LLM Projects
• Promptware Engineering: Software Engineering for Prompt-Enabled Systems
• PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)
• ProxyWar: Dynamic Assessment of LLM Code Generation in Game Arenas
• Pytorch: An imperative style, high-performance deep learning library
• QuixBugs: a multi-lingual program repair benchmark set based on the quixey challenge
• Qwen3 technical report
• Real-World Traceability Patterns for Generative AI Systems: With Insights from the Safa Dataset
• Red Teaming LLMs via Linguistic-Aware Fuzzing
• Reflections on the Reproducibility of Commercial LLM Performance in Empirical Software Engineering Studies
• Repairing LLM Executions for Secure Automatic Programming
• Repomastereval: Evaluating code completion via real-world repositories
• RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models
• Reporting LLM Prompting in Automated Software Engineering: A Guideline Based on Current Practices and Expectations
• Requirements engineering for artificial intelligence systems: A systematic mapping study
• Requirements Engineering for LLM: The Case of Digital Inquiries Application
• Requirements Engineering for Machine Learning: Perspectives from Data Scientists
• Rethinking benchmark and contamination for language models with rephrased samples
• Rethinking Software Engineering in the Era of Foundation Models: A Curated Catalogue of Challenges in the Development of Trustworthy FMware
• Rethinking the Evaluation of Secure Code Generation
• Revisiting Catastrophic Forgetting in Large Language Model Tuning
• RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage
• Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
• Scaling laws for forgetting when fine-tuning large language models
• Schrödinger's Update: User Perceptions of Uncertainties in Proprietary Large Language Model Updates
• Scieval: A multi-level large language model evaluation benchmark for scientific research
• Securing AI Inference in the Cloud: Is CPU-GPU Confidential Computing Ready?
• Security and privacy challenges of large language models: A survey
• Security Considerations for Large Language Model Use: Implementation Research in Securing LLM-Integrated Applications
• Self-Admitted Technical Debt in LLM Software: An Empirical Comparison with ML and Non-ML Software
• Self-Distillation Enables Continual Learning
• Self-Pluralising Culture Alignment for Large Language Models
• Semantic drift mitigation in large language model knowledge retention using the residual knowledge stability concept
• Serverlessllm: Low-latency serverless inference for large language models
• Sglang: Efficient execution of structured language model programs
• Shadowcast: Stealthy data poisoning attacks against vision-language models
• Sharegpt4v: Improving large multi-modal models with better captions
• She elicits requirements and he tests: Software engineering gender bias in large language models
• Simple and scalable strategies to continually pre-train large language models
• Small llms are weak tool learners: A multi-llm agent
• Software Engineering for AI-Based Systems: A Survey
• Software Engineering for Machine Learning: A Case Study
• Software Performance Engineering for Foundation Model-Powered Software (FMware)
• Source2synth: Synthetic data generation and curation grounded in real data sources
• Specification and Detection of LLM Code Smells
• Splitwise: Efficient generative llm inference using phase splitting
• Spurious Forgetting in Continual Learning of Language Models
• State of ai in business 2025
• TaintP2X: Detecting Taint-Style Prompt-to-Anything Injection Vulnerabilities in LLM-Integrated Applications
• TensorOpera Router: A Multi-Model Router for Efficient LLM Inference
• The Dark Side of LLMs Agent-based Attacks for Complete Computer Takeover
• The devil is in the tails: How long-tailed code distributions impact large language models
• The era of 1-bit llms: All large language models are in 1.58 bits
• The EU AI Act: a summary of its significance and scope
• The good, the bad, and the greedy: Evaluation of llms should not ignore non-determinism
• The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study
• The Large Language Model (LLM) Bias Evaluation (Age Bias)
• The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only
• The state of AI
• To tune or not to tune? adapting pretrained representations to diverse tasks
• Toolformer: Language Models Can Teach Themselves to Use Tools
• Top Leaderboard Ranking= Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM
• Torrance tests of creative thinking
• Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure
• Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap
• Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
• Towards semantic consistency: Dirichlet energy driven robust multi-modal entity alignment
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• Transformer-lite: High-efficiency deployment of large language models on mobile phone gpus
• Transfusion: Predict the next token and diffuse images with one multi-modal model
• TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deployment
• TsCA: On the Semantic Consistency Alignment via Conditional Transport for Compositional Zero-Shot Learning
• Turning Generative Models Degenerate: The Power of Data Poisoning Attacks
• Two Heads are Better than One: Zero-shot Cognitive Reasoning via Multi-LLM Knowledge Fusion
• Understanding Knowledge Drift in LLMs through Misinformation
• Understanding silent data corruption in LLM training
• Understanding social reasoning in language models with language models
• Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE
• Unifiedmllm: Enabling unified representation for multi-modal multi-tasks with large language model
• Unique security and privacy threats of large language model: A comprehensive survey
• Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Study
• Vit-lens: Towards omni-modal representations
• Vulnerabilities in ai code generators: Exploring targeted data poisoning attacks
• Vulnerabilities introduced by llms through code suggestions
• What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners' Perspective
• What Makes Code Generation Ethically Sourced?
• What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use
• When AI Coding Assistants Leak Training Data: A Study of LLM Memorization in Code Generation
• When moe meets llms: Parameter efficient fine-tuning for multi-task medical applications
• When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions
• Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy
• xcodeeval: A large scale multilingual multitask benchmark for code understanding, generation, translation and retrieval
• Xkd: Cross-modal knowledge distillation with domain alignment for video representation learning
• Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation
• Zero-Trust Architecture (ZTA): Designing an AI-Powered Cloud Security Framework for LLMs' Black Box Problems

Cited Tools and Technical Sources

• [New Model]: Codestral Mamba Issue
• Accelerate Kunpeng Ascend Native Innovation and Help Build New Quality Productivity
• Advance your maturity level for GenAIOps
• AI principles
• Announcing the Agent2Agent Protocol (A2A)
• Architecting Efficient Context-Aware Multi-Agent Framework for Production
• Azure OpenAI Service
• Cloud TPUs: Accelerate AI development with Google Cloud
• ComfyUI
• CVE-2025-6514 Detail
• DeepSpeed: Accelerating Deep Learning Training and Inference
• Effective Context Engineering for AI Agents
• Executorch Overview
• FMOps/LLMOps: Operationalise Generative AI using MLOps principles
• Gemma-2-2B JPN-IT (Flax) — Google
• Grand Teton: Our next-generation platform for AI at scale
• How Anthropic uses Surge AI to Train and Evaluate Claude
• Huawei Releases Full-Stack Ascend AI Software to Bridge the Divide Between AI Computing and Application
• Intel® Software Guard Extensions (Intel® SGX)
• LangChain: Build context-aware reasoning applications
• llama.cpp: Inference of LLaMA models in pure C/C++
• LlamaIndex Python Framework Documentation
• MCP: Agents and Tools Overview
• MLC-LLM
• MLflow Model Registry
• NVIDIA Nsight Systems
• Ollama Official Website
• Operationalizing Generative AI: How It Differs from MLOps
• PEFT: Parameter-Efficient Fine-Tuning
• TensorRT-LLM: A TensorRT Toolbox for Optimized Large Language Model Inference
• TensorRT: High-Performance Deep Learning Inference Library
• Text Generation Inference
• Transformers: State-of-the-art Natural Language Processing
• Triton: A Language and Compiler for Efficient Deep Learning Primitives
• vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
• Weights & Biases: The AI Developer Platform
• XLA: A machine learning compiler for GPUs, CPUs, and ML accelerators