Google found a way to test Gemini without seeing the questions

Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it The post Google found a way to test Gemini without seeing the questions appeared first on The New Stack.

Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it The post Google found a way to test Gemini without seeing the questions appeared first on The New Stack.

In context

• Topic: Inteligencia Artificial — IA aplicada al negocio: casos, límites, costos y gobierno.
• Source: The New Stack
• Published: 27/08/2026

Continue reading at the original source →

Excerpt published automatically by the site radar. The full text belongs to its publisher and is linked above.

Why it matters

Every week there is an announcement that promises to change everything, and every week most organisations are still fighting the same battle: messy data, processes nobody documented, and expectations running faster than capability. This story reads better with that in the background.

I measure AI maturity with an uncomfortable question: can anyone explain, in one line, what decision the model makes and who answers if it gets it wrong? Where that is clear, the project moves. Where it is not, it stays in permanent pilot and gets cancelled with the excuse that the technology was not ready.

What usually goes wrong

The point where it usually breaks is the handover to operations. The team that built the pilot knows how to read the results; the team receiving it does not. Without training and written criteria, people start accepting everything the model says or ignoring it entirely, and both are equally expensive.

What to watch

• Who reviews the model outputs and how often — the point at which AI becomes auditable.
• Cost per query at real volume, not pilot volume: the variable that produces the most surprises.
• Where the data ends up and under what contract, especially when customer information is involved.

How I read this entry

I would use it for a talent conversation rather than a purchasing one. The capability that needs installing is not operating a tool, which changes every six months, but framing the problem well and judging the answer. That one stays in the house.

This entry is an excerpt from the original source, selected by the site radar. The commentary above is the site's own and does not belong to the cited publisher.