Skip to content
13 min read

Improving Enterprise Search Experience: Six Paths for the AI Era

The daily interactions of search establish user behavior patterns setting expectations, criteria for success and expectations of how search and ai should work.

So many paths
Photo by Jens Lelie on Unsplash

Improving search experiences has long been of interest to me. In the enterprise, my argument was simple: consumer search had trained everyone what “good” felt like, and enterprise search kept failing to come close to that bar. It appears my argument is evergreen. The consumer bar has moved again, and it has moved further in the last three years than in the previous ten.

An old benchmark was a 2012 Pew Research study, which found that 73 percent of Americans used search engines and that 91 percent of search users said they always or mostly found what they were looking for. Those numbers set user expectations that enterprise systems were graded against.

Now that enterprise solutions have finally provided lists of links. Expectations have been set further by the consumer space. Users no longer expect a list of links. They expect the answer. Pew’s 2025 study of Google search behavior, drawn from the browsing data of 900 U.S. adults, found that about 18 percent of searches produced an AI-generated summary, and that when one appeared, users clicked a traditional result on only 8 percent of visits, compared with 15 percent when no summary was present. Just 1 percent clicked a link inside the summary itself. Read that as a behavioral fact rather than a publishing controversy: people increasingly treat the answer as the destination.

Meanwhile, the execution gap is as wide as ever. Baymard Institute’s 2026 search benchmark, covering more than 170 sites and apps and over 10,000 usability scores, found that 56 percent of sites deliver a “mediocre or worse” search experience. Their testing repeatedly captured participants concluding that a product was not “in stock” when it was in fact available and findable by browsing or wayfinding.

Expectations are higher, patience is shorter, and the technology that raised expectations is now sitting in your development team’s backlog with an executive sponsor attached that may not fully understand the technology.

Here is my updated Paths to Improving Search Experiences in the enterprise:

1. Understand the Complexity and Resource It Honestly

A two-person team can still stand up a proof of concept in a sprint. Putting search into production across diverse content silos, with document-level security, sub-second latency, and defensible relevance, is still hard every single time. The original pitfall was under-resourcing. That pitfall now has a much larger and better-documented body count.

MIT’s NANDA initiative published The GenAI Divide: State of AI in Business 2025, based on roughly 150 leader interviews, a survey of employees, and analysis of 300 public deployments. It found that about 95 percent of enterprise generative AI pilots delivered no measurable P&L impact. The researchers attributed the failures not to model quality but to integration and organizational learning. Of enterprise-grade custom or vendor tools evaluated, only a fraction reached pilot, and fewer still reached production.

The study drew methodological criticism, and it deserves some. Treat the exact percentage as directional rather than precise. The mechanism is what matters, and the mechanism is the same one that killed enterprise search projects in 2014: a demo is not a system, and the work between them is mostly unglamorous integration, data preparation, and access control.

Two practical implications:

Budget for the retrieval layer, not the model. When a retrieval-augmented system produces a confident wrong answer, the failure is usually upstream. The generator faithfully summarized the wrong documents. Teams that spend their effort on prompt tuning while leaving chunking, metadata, and ranking untouched are optimizing the cheap half of the pipeline.

Scope narrowly and ship. The pattern that separated the successful minority in MIT’s data was tight scope and domain specificity. One content domain, one user population, one measurable job. Enterprise-wide “search everything” programs have been failing for twenty years, and adding a language model does not change their odds.

2. Avoid Inventing. Use the (Updated) Patterns

Thousands of development hours have laid a path, so do not reinvent the wheel. Some of those wheels have been replaced.

Still valid, now cheaper. Silent query expansion and spelling correction were once bespoke engineering. Modern dense retrieval absorbs a good deal of that work automatically, since semantic similarity is tolerant of phrasing and misspelling in ways that lexical matching never was. Do not skip the capability; just stop hand-building it.

Autocomplete has gotten more important, not less. It is still the cheapest intent-shaping surface you own. Baymard’s work on scope suggestions is worth reading here, particularly for catalogs where users do not know the vocabulary.

Hybrid retrieval is the new default. This is the single largest pattern shift since the original article. Pure vector search blurs exact identifiers, which is precisely the wrong failure mode in an enterprise, where SKUs, part numbers, policy numbers, contract IDs, and error codes are the queries that matter most. The current standard is to run lexical retrieval (BM25) and dense retrieval in parallel, fuse the result sets with a method such as Reciprocal Rank Fusion, and then apply a cross-encoder reranker to the top candidates. Retrieve broadly, rerank aggressively, pass a small number of high-precision passages forward.

Design for query types, not for queries. Baymard’s taxonomy is the most useful practical framework I have seen for this, and it transfers cleanly out of retail into any catalog or knowledge domain. Their 2026 findings on failure rates by type are instructive:

Query type Sites with issues Enterprise analogue
Exact (model number, title) 12% Part numbers, doc IDs, error codes
Product type 20% Document class, policy category
Feature (attribute qualifiers) 39% Faceted attribute queries
Use case (how, where, when) 43% Task and scenario queries
Abbreviation and symbol 54% Internal acronyms, units, notation
Compatibility 44% Version, system, and dependency queries
Symptom (describes the problem) 37% Troubleshooting and diagnostic queries
Non-product (policies, help) 66% HR policy, process, and reference content

Two of those deserve special attention in an enterprise. Abbreviation handling is nearly free to fix and is failing on more than half of tested sites; every organization runs on acronyms that mean nothing outside the building. And symptom searches are the user’s last resort. If someone can only describe the problem and your system cannot bridge from problem to solution, they have no remaining move.

What I got wrong about advanced search. I told you to skip it, and I stand by the underlying point that a dedicated advanced-search form earns almost no usage and rarely repays its build cost. But the need it served did not disappear, it relocated. Users still want to express complex, multi-constraint intent. They now do it in natural language and expect the system to decompose it. The modern implementation of advanced search is query understanding plus visible, editable filters applied to the result set. Baymard’s testing supports this directly: when a system interprets a constraint, it should show that interpretation as a toggleable filter rather than applying it invisibly. That gives you the power of advanced search with the transparency and control that a hidden form never provided.

3. Search is still a Transaction. Simplify the Interface, and Earn Trust

“Search is a transaction” is still the right frame. Minimize clutter that does not move the user through it. But the interface picked up a new job, because a results list and a generated answer are not the same artifact and cannot be evaluated the same way.

A list of ten links is self-evidencing. The user scans, judges, and clicks. A generated answer asks the user for belief and trust. This is a fundamentally different ask, and the interface has to support it.

Design implications:

  • Cite at the claim level, not the response level. A block of text followed by four source links tells the user nothing about which claim came from where. Inline attribution lets a skeptical user verify one sentence without re-reading four documents.
  • Make the retrieved evidence reachable in one action. Users who need to verify are usually the users whose work carries consequences. Do not bury the passage that produced the answer.
  • Show the interpretation. If the system narrowed, expanded, filtered, or rewrote the query, say so and make it reversible. Silent reinterpretation is the single fastest way to erode confidence in a search system.
  • Say “I do not know” out loud. A grounded system that declines when retrieval comes back weak is more valuable than one that always produces fluent prose. Train the abstention behavior and design a real state for it.
  • Keep the list. The answer layer is an addition, not a replacement. Exploratory work, comparison, and browsing all still need results. So does every user who does not trust the summary yet.

Remember that attention is front-loaded and expensive. Spend it on the answer and the evidence.

4. Avoid Over-Configuration and Replace Faith with Evaluation

I still see it in every relevance engagement I have walked into: a stakeholder complains about one query, someone adds a boost rule, three other queries quietly get worse, and eighteen months later nobody can explain why the system behaves the way it does.

Do not trust the algorithm. Measure it.

“Trust the algorithm” was defensible advice when the alternative was arbitrary manual boosting. It is not sufficient guidance for a stack where retrieval quality is a tunable and consequential variable thanks to the measurement tooling becoming accessible.

Build a golden set. A few hundred representative queries drawn from real logs, spanning the query types above, with judged relevance. This is the highest-leverage artifact in a relevance program, and the one teams skip most often. It is also the thing that converts a relevance argument from an opinion contest into a measurement.

Instrument offline metrics. Precision@k, nDCG, MRR, and MAP each answer a different question. nDCG is usually the right default when graded relevance matters and position matters. MRR is right when there is one correct answer, and you care how fast the user reaches it. Pick deliberately, because the metric you choose determines what your team optimizes.

Use LLM-as-judge to scale judgment, with calibration. Human relevance judgments are the gold standard and are expensive to produce at volume. Model-based judging makes continuous evaluation affordable. It also introduces its own bias, so validate the judge against a human-labeled subset before you let it drive decisions.

Separate retrieval evaluation from generation evaluation. Ask two questions independently. Did we retrieve the right evidence? Did we faithfully use what we retrieved? Conflating them makes every regression untraceable.

The content point from the original article survives intact and has gotten sharper. Poor relevancy is more often a content problem than an algorithm problem. In a retrieval-augmented system, the modern version of “content that is not rich enough for successful indexing” is content that chunks badly: tables split across boundaries, headings orphaned from bodies, passages that cannot be understood standing alone. Audit your content before you tune your ranker. Every chunk should be independently comprehensible, because that is the unit your system will actually retrieve.

5. Collect and Use Data. Users Are Still Telling You Exactly What They Want

Search analytics remains a key data point where users state their intent in their own words, at scale, for free.

Previously, I recommended the following to clients: Capture search failure. Capture common misspellings. Identify jargon worth expanding. Identify indexing gaps. Now, I would add:

  • Zero-result rate and low-engagement rate, tracked separately. Zero results is the obvious failure. The more dangerous one is the query that returns 400 results and gets no clicks, because it looks healthy on a dashboard.
  • Query reformulation chains. When a user searches three times in ninety seconds, the first two queries are a failure report. The vocabulary they abandon is a synonym list you have not written yet.
  • Abandonment after an answer. In a generative system, a session that ends immediately after an answer is ambiguous. It may mean satisfaction or it may mean surrender. Pair it with explicit feedback or source-click behavior before you interpret it.
  • The queries nobody is running. Absence is a signal. If a content domain gets indexed and never retrieved, either your users do not need it, or your system cannot surface it. Both are worth knowing.

One addition on governance. In a system that synthesizes across silos, permissions cannot be an afterthought at the presentation layer. Document-level security has to be enforced at retrieval time, or your search tool becomes an excellent instrument for aggregating information people were not entitled to see. This was true in 2014 and mattered less, because a link the user could not open failed safely. A generated summary of a document the user cannot open does not fail safely.

6. Design for the Agent as a User

Your search system now has a second user population, and it does not have eyes. AI agents query enterprise retrieval systems programmatically, through APIs and increasingly through standardized tool interfaces, then reason over what comes back and take action. Microsoft, Google, and most major search vendors have shipped agentic retrieval products on this premise. Whatever you think of the timeline, the interface implication is concrete.

Agents have different requirements than humans:

  • They do not scan, so precision at the top matters more. A human recovers from a mediocre result at position three by glancing past it. An agent may feed it straight into a downstream decision.
  • They issue decomposed queries. One user question may become five retrieval calls. Latency budgets and rate limits designed for human pacing will not hold.
  • They need structure, not layout. Clean metadata, stable identifiers, explicit relationships, and machine-readable provenance matter more than presentation.
  • They inherit the user’s permissions, or they should. An agent acting on someone’s behalf must be constrained by that person’s access rights, evaluated per call.

The useful reframe is that your retrieval layer is becoming a product with two consumers: a user interface and an agent interface. If retrieval only exists as a component embedded inside a web application, you will end up rebuilding it, badly, the first time someone asks you to connect an assistant to it. Design the retrieval service to stand on its own.

Guideposts, Still Not Rules

The paths discussed above are guideposts. The underlying observation has not changed in a decade. Search provides access to information that people need in order to do their work. Most enterprise search systems are still mediocre. Research says so consistently, and mediocre search is a tax that every employee and every customer pays every day without anyone assigning it a line item.

What has changed is the ceiling and the risk. The best systems are meaningfully better than they were three years ago. The worst ones now fail in a new way, by generating confident, fluent, well-formatted answers grounded in the wrong documents. That is a harder failure to detect than an empty results page, and it is a harder one to recover trust from.

Measure it, ground it, cite it, and let people see the seams. That will get you further than any model choice.

Sources