We Graded Our Own Scouting Reports: What Pug Demos Can and Can't Tell You

We checked 86 of our own Enemy Scout reports against the matches that were then played. The player-level reads held up modestly. The team-level calls did not, so we have stopped making them unless the numbers are clear.

Enemy Scout downloads your opponents' recent demos and summarises how they play. It has always produced confident-looking output. This post is about how much of that output survived contact with the actual match, and what we changed as a result.

The problem with scouting a pug

The five players you are about to face have usually never played together. Each one's recent demos come from different matches with four different strangers. So when a report says "this team loses B site", it is really saying "these five individuals were each recently on a team that lost B site", which is a much weaker statement.

Scouting a real team is different. A roster that plays together has habits. Five randoms mostly do not.

What we measured

When a member scouts a lobby and the match finishes, we parse the demo of the game that was actually played and compare it with what the report said. Across 86 matches:

Call	Pointed the right way	Just naming the map's usual site
"Hit this site" (the one they lose more on CT)	68% (43 of 63)	63%
"Stack this site" (where they plant more)	69% (48 of 69)	66%

A few points better than knowing the map is not a scouting report. With samples this size it could easily be nothing at all.

An earlier study of ours on 348 demos explains why. We measured how well each habit in a player's past demos predicted the same habit in their next match:

• Carries over reasonably well: how much utility a player throws.
• Carries over a little: whether they pick up the AWP, where they stand on CT early in the round, and whether they take the opening duel.
• Does not carry over at all: which site the team attacks, how often they force-buy, how often they retake, pistol-round results, and side win rates.

Everything in the last group is a team result, and the team no longer exists by the next match.

A report that would have cost a round

One real example. For a Mirage lobby, our Team Read said "HIT B". The numbers behind it: the enemy players had conceded A in 20% of their recent CT rounds and B in 24%. That is a difference of about two rounds out of fifty-four. In the match that followed, A was the softer site.

The same report said "SAVE T PISTOL", built on a handful of pistol rounds. That is not something a team can act on, and it should never have been printed as an instruction.

What we changed

• A site call now needs a real gap: at least ten points between the sites over at least forty rounds. Otherwise the report says no clear site and shows you the numbers.
• Pistol and economy "calls" are gone. The figures are still shown, as figures.
• A player is only labelled as the utility thrower after twenty rounds across at least two demos, the same bar the other labels already had.
• If you decline to scout more demos, you now see the read from the ones already parsed instead of nothing.

The result is a report that says less. We think a tool that tells you "there is nothing reliable here, play your default" is more useful than one that always has an opinion.

What is still worth knowing about five strangers

In rough order of how much it helps before a pug:

1. Who the strong player is, who the weak one is, and who looks like a smurf. This comes from account history, not demos, and it is in the free scan.
2. Which map suits your five against theirs. Also the free scan, and the part with the best evidence behind it: lobbies that played our recommended map have won 55% against 48% on other maps across more than two thousand graded matches.
3. Who tends to AWP. Demos can show this, with the caution that a single demo is weak evidence, which is why the card says "AWP in 2 of 3 demos" rather than just "AWPer".

Demo scouting earns its keep against opponents who play together: a league match, a tournament, a team you will meet again. For a pug, the map board and the player stats are where the edge is.