Introduction
The position probe (SIGI-2026-017) established that LLMs display primacy bias and geographic sensitivity in ranking evaluation. This companion paper explores the qualitative mechanisms underlying these findings, focusing on the critical distinction between how the model processes entity unfamiliarity versus contextual mismatch.
This distinction has practical significance for generative engine optimisation. If the model merely questions unfamiliar entities but actively rejects contextual mismatches, the implications for geographic market positioning are substantially different from a model that treats all anomalies uniformly.
Methodology
We conduct a comparative analysis of four key position probe variations: PP03 (lesser-known real entity at position 1), PP04 (fictitious entity at position 1), PP05 (shuffled known entities), and PP06 (non-local entity at position 1). We analyse sentiment markers, response patterns, and the qualitative language used in evaluative assessments. All entities are anonymised as described in SIGI-2026-017.
Results: The Unknown-versus-Foreign Distinction
Comparative Response Analysis
| Variation | Entity at #1 | Sentiment | Pos. | Neg. | Neu. | Words | Tokens Out |
|---|---|---|---|---|---|---|---|
| PP03 | Unknown real entity | neutral | 0 | 0 | 0 | 190 | 280 |
| PP04 | Fictitious entity | neutral | 0 | 0 | 0 | 200 | 277 |
| PP05 | Known entity (shuffled) | neutral | 0 | 0 | 0 | 193 | 270 |
| PP06 | Non-local entity | negative | 0 | 1 | 0 | 202 | 295 |
The contrast between PP03/PP04 and PP06 is revealing. Both manipulation types place a potentially inappropriate entity at position 1 in a locally-scoped ranking, yet they produce qualitatively different responses. Unknown entities (PP03, PP04) produce neutral sentiment with zero sentiment markers — the model generates an evaluation that neither endorses nor rejects the ranking. The non-local entity (PP06) produces negative sentiment with an explicit negative marker, representing active rejection rather than passive questioning.
Response Length Analysis
PP06 produced both the longest response (202 words) and the highest token output (295 tokens) of all six variations. This slight increase compared to PP03–PP05 (190–200 words, 270–280 tokens) suggests the model generates additional evaluative content when it identifies a contextual mismatch, consistent with the hypothesis that mismatch detection triggers a more active evaluation pathway.
Mid-List Positional Expectations
PP05 (shuffled known entities) produced neutral sentiment with zero sentiment markers, identical to PP03 and PP04. This indicates that the model holds weak positional expectations for entities at positions 2 through 5 while maintaining strong expectations for position 1. A known entity moved from its expected position within the list to another non-first position does not trigger the model’s ordering-violation detection. The first position carries unique evaluative weight.
Discussion
The unknown-versus-foreign distinction maps onto a meaningful cognitive architecture within the LLM. The model appears to maintain two separate evaluation systems: an entity recognition system that checks whether entities are known, and a contextual consistency system that checks whether known entities belong in the specified context.
When entity recognition fails (PP03, PP04), the model defaults to neutral evaluation — it cannot confirm or deny the entity’s appropriateness without recognising it. When entity recognition succeeds but contextual consistency fails (PP06), the model applies active rejection — it knows the entity but determines it does not belong in the specified geographic ranking.
This two-pathway model has implications for entities seeking visibility in AI-mediated rankings. Entities that are unknown to the model face neutral evaluation at worst, while entities that are known but geographically miscontextualised face active negative assessment. For multi-national service providers, this suggests that geographic self-identification in the correct market context may be more important than overall brand recognition for AI ranking purposes.
The uniform word count across variations (standard deviation of approximately 5 words) is itself a finding. Unlike the domain-age probe (where signal irrelevance produced shorter responses) or the word-count probe (where input length influenced output length), the position probe elicits consistent evaluative depth regardless of manipulation type. This suggests ranking assessment operates as a fixed-complexity task with a stable output template.
Limitations
The analysis draws from only 6 probe variations, limiting the granularity of the distinction between unknown and foreign entities. Only one non-local entity was tested, from a single foreign market. The boundary between “unknown” and “known but non-local” depends on the model’s training data and may shift over time. Single-model, single-time-point limitations apply.
Conclusions
The LLM applies qualitatively different evaluative responses to unfamiliar entities (questioning, neutral) versus contextually mismatched entities (rejection, negative). This distinction confirms that the model operates separate evaluation pathways for entity recognition and contextual fit, with the contextual-fit pathway capable of generating negative sentiment while the recognition pathway defaults to neutral. For GEO strategy, geographic contextual consistency at the first position in rankings is more important than mere brand recognition.
Confidence Statement: HIGH for the unknown-versus-foreign distinction. The two-pathway interpretation is consistent with the data but represents a framework overlay, not a confirmed mechanism.