When the metric starts deciding the content

A highly cited journal does not validate every article

The San Francisco Declaration on Research Assessment, developed in 2012, recommends against using a journal’s impact factor as a substitute for assessing an article or a researcher. The indicator describes a journal while the evaluation concerns a different object.

The issue is familiar in communications: an accurate total can still be a poor answer to the question. The object measured and the context remain essential to interpretation.

Not every link is an endorsement

Google introduced nofollow in 2005 to address practices including comment spam. In 2019, its documentation added separate attributes for sponsored links and user-generated content, while changing how these signals were treated.

A link can serve different purposes. Payment, a user contribution and an editorial reference do not convey the same information. Counting links without context removes part of what makes them informative.

Read the citations generated by assistants

A citation in an AI answer depends on the question, service and time of observation. Preserve these conditions when comparing results, then read what was used. The total does not establish accuracy or usefulness to a buyer.

These historical examples do not predict how assistants will develop. They give a reason to examine a metric before making it a target. If a team produces whatever increases the score, the usefulness of those publications still needs review.

The shift may be subtle: preferring short definitions because they are often cited, dropping an analysis that is difficult to summarise or multiplying general subjects to create more opportunities for reuse. These are possibilities to examine, not a claim about every team. The risk arises from the relationship between the metric, the target and the choices it rewards.

Keep the business objective in the evaluation

AMEC’s Barcelona Principles 4.0 connect evaluation with organisational objectives and relevant audiences. That provides a basis for examining a citation score against the intended contribution: which subjects need better understanding, which capabilities need recognition and which relationships should develop?

A review retains the total alongside the questions, sources and observed answers. It may show that growth mainly concerns peripheral definitions, or that less frequently cited material supports an important discussion. Editorial decisions can use both observations rather than automatically rewarding the larger number.

The aim is to preserve the company’s ability to invest in explanations whose value extends beyond ease of measurement. A metric should help examine that choice and its results. When it quietly chooses the subject for the team, it stops describing the work and starts directing it.

The warning sign is a textual change that improves the count while weakening the answer: splitting one explanation artificially, producing interchangeable lists or dropping a qualification to make a sentence easier to quote. Compare those choices with the question the content was meant to address.