此页面是公开的。
Large language models (LLMs) can do a great deal, but they still struggle with complex reasoning, and there are concerns about their truthfulness, transparency, robustness and ethical alignment. In this preliminary study, Konstantin Hebenstreit, Robert Praas and Matthias Samwald gather principles for steering and evaluating how LLMs reason. They found 220 in the literature and distilled 37 core ones. What follows summarizes the paper and its claims.
The aim is to integrate insights from several distinct strands of work into a set of core principles guiding reasoning in LLM-based AI systems. From the paper (Figure 1), CC BY 4.0.
Six strands of work
The authors reviewed literature from six fields:
- Structured reasoning. LLMs can generate “chains of thought” that improve reasoning performance and make the reasoning more transparent, and later work breaks reasoning into steps and may add outside tools such as information retrieval.
- Self-evaluation and self-reflection. Because human evaluation does not scale, models have been used to judge their own output, first for bias and toxicity. Reinforcement learning from AI feedback used a “constitution” of norms to train an aligned model with little human feedback.
- Explainability. Concerns include practical usefulness, faithfulness, contrastiveness, simulatability and causality.
- AI safety and security. Making systems robust against misinformation and adversarial attacks, and building systems that do not deceive or cause harm.
- Human critical thinking. General principles from philosophy and psychology that can apply to LLMs.
- Ethical and regulatory guidelines. For example the draft EU AI Act and the OECD AI principles.
How the list was built
The authors extracted concerns, system instructions and guidelines into a table and sorted them into broad categories such as utility, ethics, robustness and security. Over several rounds of revision they distilled a small set of core principles, worded as broadly as possible so they apply to many reasoning tasks and to multimodal setups. For outside validation they listed authors of recent papers on LLM reasoning, LLM evaluation or AI ethics, contacted 66 experts across academia and industry, and asked them to rate each principle's importance, imagining they were judging an LLM response in a critical application, on a 5-point scale from “not important” to “very important.” Seven filled out the anonymous survey.
Results
Thirty documents yielded 220 principles in eight categories, from which the authors derived a core set of 37 in seven categories: assumptions and perspectives, reasoning, information and evidence, robustness and security, ethics, utility, and implications.
| Category | Collected from literature | Core set |
|---|---|---|
| Assumptions and perspectives | 28 | 3 |
| Reasoning | 40 | 9 |
| Information and evidence | 18 | 6 |
| Ethics | 79 | 6 |
| Robustness and security | 11 | 4 |
| Utility | 30 | 8 |
| Implications | 2 | 1 |
| General | 12 | none |
The set of core principles compiled from literature. From the paper (Figure 2), CC BY 4.0.
For most principles the experts' median answer was “important” or “very important,” though for some the answers ran the full range, from “not important” to “very important.”
What the authors say it is for
They see such a set serving several purposes: steering models at inference time, for instance by writing principles as instructions that structure reasoning; monitoring, critiquing and improving model behavior through automated self-critique at inference time or during training (“constitutional AI”); and helping humans evaluate the reasoning in a model's responses. The set can be adapted, shortened, expanded and reworded to fit those uses.
The authors expect the social impacts to be mainly beneficial, but they say there is significant uncertainty over whether instructing and training models with such principles ensures they are followed, so the approach is only one part of a comprehensive plan for alignment.
Limits
The literature review did not cover everything, though the authors saw much redundancy in additional sources. The principles lean toward language-based interactions such as question answering and general dialog models, and the authors expect to add concerns as AI systems act more as autonomous agents. The work is preliminary: the set is not exhaustive, was derived through subjective judgments of relevance, and the survey was small. They suggest a multi-round Delphi study or an open online discussion platform for gathering principles, input from more perspectives including industry stakeholders, and empirical tests, such as deriving prompts from the principles and measuring their effect on performance, explainability and ethical alignment. They invite feedback from the community.
Sources
- Konstantin Hebenstreit, Robert Praas and Matthias Samwald, “A collection of principles for guiding and evaluating large language models,” arXiv:2312.10059, licensed under CC BY 4.0. Changed here: summarized and rewritten in hubnx's own words; references and supplementary material left out; figure captions shortened.
留言
