Tratopedia
Side N繁中
Settings

Text size

Language

EN繁中BMES

Theme

High contrast

Version

v1.184.0

The release this page was built from. It is what the service worker caches under.

What Happened

Terminology · LLM Engineering · Language Models · Benchmarking · AI · ExplainerThe term and its practice, June 2018 to September 2026

First of five · the innermost layer

The Tricks Moved Into the Model. The Writing Stayed.

Prompt engineering is writing a model’s instructions so that it does what you meant. The phrase is in the research literature by February 2021. By March 2023 it was a job advertised at up to $335,000 a year, and by 2024 and 2025 the trade press was calling that job obsolete: tools that search for prompts were beating people at it, and models were getting better at working out what people meant. The practice did not go with the title. Anthropic, OpenAI and Google all maintain prompting guides in 2026. What changed in them is where the tricks live: “let’s think step by step” became a built-in thinking setting, prefilled answers gave way to structured outputs, and hand-tuned wording gave way to tests. The reason it mattered in the first place has not been reported fixed — changing nothing but a prompt’s formatting has moved accuracy by up to 76 points.

  • 76accuracy points moved by changing nothing but a prompt’s formatting, on one open model (2023)
  • 17.7 → 78.7per cent on a maths benchmark, before and after adding “Let’s think step by step” (2022)
  • $335Ktop of an advertised salary range for a prompt engineer, March 2023
  • 0 / 10top new AI roles in Microsoft’s 2025 survey of 31,000 workers that were “prompt engineer”

Three definitions, five years apart A researcher’s, a vendor’s and a dictionary’s. Only the dictionary calls it a profession.

  • Liu et al. · Jul 2021

    “Creating a prompting function”

    From when a prompt was a template with blanks in it.

    • A field survey’s section title: prompt engineering is “the process of creating a prompting function” that gives “the most effective performance on the downstream task”.
    • Five months earlier, Laria Reynolds and Kyle McDonell had written that prompt engineering “may be conceived as programming in natural language” — the earliest use of the phrase found here.

  • OpenAI and Anthropic · 2025–26

    “Writing effective instructions”

    From when a prompt is a set of instructions to a model that can reason.

    • OpenAI: “the process of writing effective instructions for a model, such that it consistently generates content that meets your requirements” — “a mix of art and science”.
    • Anthropic: “methods for writing and organizing LLM instructions”, with context engineering as its natural progression.

  • Oxford English Dictionary · 2025

    “…the discipline or profession concerned with this”

    As Wikipedia quotes it; the entry itself could not be opened.

    • “The action or process of formulating and refining prompts for an artificial intelligence program … in order to optimize its output or to achieve a desired outcome.”
    • In 2023 “prompt” in its AI sense was a finalist for Oxford’s Word of the Year. The word chosen was rizz.

How firmly each part stands Sorted by standing. A measurement and a vendor’s advice are not the same kind of fact.

StandingWhatOn what
MeasuredFormatting alone moved accuracy by up to 76 points on LLaMA-2-13B, and the sensitivity “remains even when increasing model size, the number of few-shot examples, or performing instruction tuning”.Sclar, Choi, Tsvetkov and Suhr, Oct 2023. Abstract read at source
MeasuredThe order of the examples alone made “the difference between near state-of-the-art and random guess performance”, across model sizes; a good order for one model did not carry over to another.Lu et al., Apr 2021, and Zhao et al., Feb 2021. Abstracts read at source
MeasuredAdding “Let’s think step by step” took one model from 17.7% to 78.7% on MultiArith and from 10.4% to 40.7% on GSM8K, with no worked examples.Kojima et al., May 2022. Abstract read at source
The vendor, about its own productStep-by-step reasoning is now built in: Anthropic’s newest models think by default, and Google says its Gemini 2.5 and 3 models generate internal thinking, so having them lay out reasoning steps in the answer is “generally not necessary”.The vendors’ own current documentation, read in full
The vendor, about its own productOpenAI tells developers to pin production apps to a model snapshot and build evaluations, because “even different snapshots of models within the same family could produce different results”.OpenAI’s prompt engineering guide, read in full
DocumentedAnthropic advertised a Prompt Engineer and Librarian at $175,000 to $335,000 a year.David Ramel, Virtualization Review, 13 Mar 2023, reporting the listing
DocumentedIn Microsoft’s survey of 31,000 workers in 31 markets, the ten new AI roles companies were most considering run from AI trainer (32%) to chief AI officer (27%). Prompt engineer is not among them.Microsoft Work Trend Index, 23 Apr 2025, read in full
Reported second-handMicrosoft’s Jared Spataro: “Two years ago, everybody said, ‘Oh, I think prompt engineer is going to be the hot job.’ It’s not turning out to be true at all.”The Wall Street Journal, as reported by Erin Davis, Entrepreneur, 29 Apr 2025
Self-reportedPutting long documents first and the question last “can improve response quality by up to 30 percent in tests”.Anthropic, about its own tests; not independently measured
Our readingThree things that began as wording moved into the model or the API: step-by-step reasoning, prefilled answers, and hand-tuning.Assembled across the sources above. No single source says it

Timeline

Eight years, from a question to a guide Every date is a publication. When people changed how they worked is not measured by any of them.

  1. Jun 2018A prompt is a question that picks the task. decaNLP casts ten language tasks “as question answering over a context”, for one model.
  2. 2019The prompt replaces training. GPT-2 performs tasks zero-shot, “without any parameter or architecture modification”; writing “TL;DR:” after an article is enough to get a summary.
  3. May 2020From asking to showing. GPT-3, at 175 billion parameters, takes tasks and worked examples “specified purely via text”.
  4. 15 Feb 2021The phrase, in print. Reynolds and McDonell: prompt engineering “may be conceived as programming in natural language”. A prompt holding only the names of two languages and a colon beat a ten-example prompt at translation.
  5. 19 Feb 2021The instability is measured. Format, choice of examples and their order move GPT-3 “from near chance to near state-of-the-art”.
  6. Apr 2021Order alone does it, at every model size, and a good order does not transfer between models.
  7. 28 Jul 2021A definition. A survey of the field gives prompt engineering its own section: finding the prompt that gives the most effective performance.
  8. 28 Jan 2022From the answer to the method. Chain of thought: eight examples that show their reasoning take a 540-billion-parameter model to state of the art on GSM8K.
  9. Mar–May 2022Self-consistency votes across several reasoning paths; then “Let’s think step by step” is shown to lift reasoning scores with no examples at all.
  10. 12 Sep 2022Its shadow gets a name. After Riley Goodside shows GPT-3 obeying “Ignore the above directions”, Simon Willison proposes calling it prompt injection, after SQL injection.
  11. Nov 2022A model writes the prompt. Automatic Prompt Engineer: model-written instructions match or beat human-written ones on 19 of 24 tasks.
  12. Feb 2023Indirect prompt injection: instructions planted in data an application retrieves. Effective defences are “currently lacking”.
  13. 13 Mar 2023A job. Prompt-engineer listings are reported at up to $335,000 a year. A Microsoft researcher says he is inclined to think the techniques will end up in existing roles instead.
  14. May 2023Tree of thoughts: search over reasoning steps, with backtracking. On one puzzle, 4% solved with chain of thought, 74% with the tree.
  15. Oct 2023Formatting alone: up to 76 points. And it does not go away with bigger models, more examples or instruction tuning.
  16. 2023“Prompt” is a finalist for Oxford’s Word of the Year; its AI sense had seen “very little use” in Oxford’s corpus before 2022.
  17. Feb–Mar 2024The obituary. Across three models, an automated optimiser proves the most effective way to tune a system message; IEEE Spectrum headlines the result “AI Prompt Engineering Is Dead”.
  18. Jun 2024The Prompt Report catalogues 58 text prompting techniques; in its case study an optimiser beats a 20-hour hand-built prompt, and the authors conclude the two work best combined.
  19. 19 Dec 2024Anthropic: tool definitions deserve “just as much prompt engineering attention” as prompts — it spent more time on its tools than its prompt.
  20. Apr 2025The job fades. Microsoft’s top ten new AI roles leave it out; reporting of a Wall Street Journal piece quotes Microsoft’s Jared Spataro calling the hot-job prediction “not … true at all”.
  21. 29 Sep 2025Anthropic calls context engineering the “natural progression” of prompt engineering — the later terms are in the next four articles of this series.
  22. 10 Nov 2025Anthropic publishes its prompting best-practices post. Revised since, it now says “prompting is converging with context engineering” — “less scaffolding, more curation”.
  23. 3 Jun 2026OpenAI begins de-emphasising reusable prompt objects: prompts are to live in application code, with tests, like any other code.
  24. 17 Sep 2026Google’s Gemini prompting guide is last updated. It still says to always include examples, and that the order of a prompt’s content can change the answer.
  25. 30 Nov 2026Scheduled: OpenAI’s stored-prompt endpoint shuts down.

The Argument

Is it dead? The obituaries are about a job. The guides are about a practice. Both are right about their own subject.

The case for the obituary is made by named people with evidence. In March 2024 IEEE Spectrum’s Dina Genkina reported a VMware study in which, in almost every case, an automatically generated prompt did better than the best one found by trial and error; the winning prompt was a Star Trek captain’s order, and the study’s Rick Battle concluded, in the article’s words, that no human should manually optimise prompts ever again. The Prompt Report’s case study, three months later, reached the same point more carefully: an optimiser’s prompt did better on held-out data than one a person had spent 47 steps and about 20 hours building. By April 2025 Microsoft’s survey of 31,000 workers did not list prompt engineer among the ten new AI roles companies were most considering, and the Wall Street Journal, as Entrepreneur reported it, quoted Microsoft’s Jared Spataro: models are now more “aware of context”, so “the perfect prompt” matters less. The case against is that none of this is about the practice. The same IEEE article quotes Red Hat’s Tim Cramer — “there are going to be prompt engineers for quite some time” — and the Prompt Report’s authors ended their study concluding that automation combined with human revision “was the most successful approach”. In March 2023, when the job was new, Microsoft Research’s James McCaffrey had said he was inclined to think the techniques would become part of the toolkit of existing roles rather than a job of their own. That is close to what Microsoft’s own 2025 report describes: every employee needs to learn “prompting with context and intent”. And the vendors’ guides, which are the practice written down, were still being updated in September 2026.

  • 47 → 16steps a person took to build a prompt, against the optimiser’s iterations that beat it on held-out data (2024)
  • ~3,000Indeed listings with “prompt engineer” in title or description in April 2025, a number Indeed called flatlining
  • 3vendors whose prompting guides were read in full, all current in 2026

What Others Add

The techniques, each at its own paper What each one added, in the words and numbers of the people who introduced it.

TechniqueIntroducedWhat it showed
Zero-shotRadford et al., 2019An instruction alone can specify the task; “TL;DR:” after an article yields a summary.
Few-shotBrown et al., May 2020Worked examples in the prompt, no retraining; “sometimes even reaching competitiveness” with fine-tuning.
Wording over examplesReynolds and McDonell, Feb 2021“0-shot prompts can significantly outperform few-shot prompts” — shown, by the authors’ own account, on one illustrative task.
CalibrationZhao et al., Feb 2021Correcting a model’s bias towards certain answers raised average accuracy by up to 30.0 points.
Chain of thoughtWei et al., Jan 2022Examples that show their reasoning; state of the art on GSM8K, “surpassing even finetuned GPT-3 with a verifier”.
Self-consistencyWang et al., Mar 2022Sample several reasoning paths, keep the most consistent answer: GSM8K +17.9%.
Zero-shot chain of thoughtKojima et al., May 2022One sentence, no examples: MultiArith 17.7% → 78.7%, GSM8K 10.4% → 40.7%.
Automatic prompt engineeringZhou et al., Nov 2022A model proposes and selects instructions; as good as or better than humans’ on 19 of 24 tasks.
Tree of thoughtsYao et al., May 2023Search over reasoning steps with backtracking: Game of 24 solved 4% → 74% with GPT-4.

What the vendors tell people to do now Three guides read in full. Where they agree, and the one place a vendor disagrees with itself.

AdviceWho says it, and how
Be clear and explain whyAnthropic: treat the model as “a brilliant but new employee who lacks context”, and give the reason behind a rule. OpenAI: GPT models are like “a junior coworker” who needs explicit instructions; reasoning models like “a senior co-worker” who can be given a goal.
Show examplesAnthropic: “Include 3–5 examples for best results.” Google: “always include few-shot examples”, formatted consistently. OpenAI: a diverse range of inputs with the desired outputs.
Give it structureAll three: separate instructions, context and examples with XML tags or Markdown. Anthropic and OpenAI both put long material in a fixed place — Anthropic at the top with the question last, OpenAI’s variable context near the end.
Let the model do the reasoningAnthropic: extended or adaptive thinking is “generally preferable to manual chain of thought prompting”; on its newest models thinking is always on. Google: asking Gemini 2.5 or 3 to lay out its reasoning is “generally not necessary”.
Test it like codeOpenAI: pin a model snapshot, build evaluations, keep prompts in application code with fixtures and tests. Anthropic: a technique measured on one model should be re-checked “against your own evals” before it is applied to another.
Prompt the tools tooAnthropic, December 2024: its coding agent made mistakes with relative file paths, so the tool was changed to require absolute ones, which the model then used “flawlessly” — and it spent more time on tools than on the prompt.
Prefill: one vendor, two answersAnthropic’s blog still teaches starting the model’s answer for it. Anthropic’s documentation says that from Claude 4.6 models a prefilled final turn returns an error, and points to structured outputs instead.

Two things that did not go away The sensitivity that made the practice necessary, and the attack it made possible.

The measurements run from 2021 to 2023 and point one way. Tony Zhao and colleagues found that format, choice of examples and their order could move GPT-3 from near chance to near state of the art. Yao Lu’s group found order alone was enough, at every model size, and that a good order for one model did not transfer to another; the Prompt Report later summarised that on some tasks order moved accuracy from under 50% to over 90%. Melanie Sclar’s group then held content fixed and changed only formatting: up to 76 points, persisting through bigger models, more examples and instruction tuning, and only weakly correlated between models — which, they note, makes comparing models on one fixed prompt format questionable. No later measurement was found. What is current is advice: Google’s guide, updated in September 2026, still tells users to try changing the order of a prompt’s content, and Anthropic still reports that where the question sits changes the quality of the answer. The second thing is security. In September 2022 Simon Willison named prompt injection after watching GPT-3 obey text telling it to ignore its instructions, and compared it to SQL injection, where the fix is to keep code and data apart. In February 2023 Kai Greshake and colleagues showed the indirect form — instructions hidden in a web page or email an application reads — and wrote that such applications “blur the line between data and instructions”, with effective mitigations “currently lacking”. A prompt is at once the program and the input, which is what makes it powerful and what makes it attackable.

  • <50% → 90%+accuracy on some tasks from reordering the same examples (2021)
  • 76accuracy points from formatting alone, persisting with scale and instruction tuning (2023)
  • 2022the year prompt injection was named, after SQL injection

Conclusion

What to take away Three things, and the last one is what to hold loosely.

A job faded. The practice was written down.

The evidence that prompt engineering is dead is evidence about hiring: listings, a survey that omits the role, an executive’s quote. The evidence that it is alive is Anthropic’s, OpenAI’s and Google’s prompting guides, all current in 2026. Read “prompt engineering is dead” as a sentence about job titles.

The tricks moved; the writing did not

“Let’s think step by step” is now a thinking setting, prefill is giving way to structured outputs, and search for the best wording is increasingly done by optimisers. What the 2026 guides still ask of a person is to say clearly what they want, show examples, structure the input and test the result — and examples, order and format are what the 2021 to 2023 studies found models are sensitive to. That summary is this site’s reading across the guides; no single guide puts it that way.

Hold the size of the sensitivity loosely

The 76-point and order effects were measured on models of 2021 to 2023. No measurement on 2025–26 models was found. The vendors’ current advice treats order, examples and format as still consequential, but advice is not a measurement, and the size of the effect on today’s models is not known from anything here.

What could not be verified

Unconfirmed report — that prompt engineer ranked second to last among the new roles companies were considering in Microsoft’s 2025 survey. The ranking is attributed to the Wall Street Journal’s reporting; Microsoft’s published report lists only its top ten, and the role is not among them.

Take-home and sources

  • McCann et al., arXiv (20 Jun 2018)
  • Radford et al., OpenAI (2019)
  • Brown et al., arXiv (28 May 2020)
  • Reynolds and McDonell, arXiv (15 Feb 2021)
  • Zhao et al., arXiv (19 Feb 2021)
  • Lu et al., arXiv (18 Apr 2021)
  • Liu et al., arXiv (28 Jul 2021)
  • Wei et al., arXiv (28 Jan 2022)
  • Wang et al., arXiv (21 Mar 2022)
  • Kojima et al., arXiv (24 May 2022)
  • Simon Willison (12 Sep 2022)
  • Zhou et al., arXiv (3 Nov 2022)
  • Greshake et al., arXiv (23 Feb 2023)
  • David Ramel, Virtualization Review (13 Mar 2023)
  • Yao et al., arXiv (17 May 2023)
  • Sclar et al., arXiv (17 Oct 2023)
  • Battle and Gollapudi, arXiv (9 Feb 2024)
  • Dina Genkina, IEEE Spectrum (6 Mar 2024)
  • Schulhoff et al., The Prompt Report, arXiv (6 Jun 2024)
  • Erik S. and Barry Zhang, Anthropic (19 Dec 2024)
  • Microsoft Work Trend Index (23 Apr 2025)
  • Erin Davis, Entrepreneur (29 Apr 2025)
  • Rajasekaran, Dixon, Ryan and Hadfield, Anthropic (29 Sep 2025)
  • Anthropic, Claude blog (10 Nov 2025, revised since)
  • Anthropic, Prompting best practices
  • OpenAI, Prompt engineering guide
  • Google, Gemini API prompt design strategies (updated 17 Sep 2026)
  • Oxford Languages, Word of the Year 2023
  • Wikipedia, “Prompt engineering”, for its quotation of the Oxford English Dictionary. All read 28 Sep 2026. This term is one of five covered by the standing topic LLM engineering, which sets it beside the other four.

Reliable

Reliable, but with an interestWorth believing and reliableWeak on both, and used with careHonest, but thin

Worth believing

Each point is one source used, placed by two judgements on a six-step scale. Points carry no names: the chart shows the shape of the evidence, not a verdict on any publisher.

Sharing a subject

Sharing a source domain

Versions

This document is rewritten when what it says has to change. Every version stays published at its own address.

  1. v0002 current
  2. v0001 superseded

The current version is also at latest/.