Tool Review: Google’s What-If Tool
Seeing the Model for What It Is
I’ve never fully trusted metrics. Accuracy, precision, recall they hint at something useful, but they don’t tell you enough. A model that’s 92% accurate doesn’t explain who’s in the 8%. Or why they were excluded. Or if those exclusions form a pattern. And once you realise they do consistently, structurally, across datasets it’s hard to look away.
That’s where the What-If Tool comes in. Not just as a visualisation layer, but as a provocation. It lets you ask: What happens if I tweak this feature? What if the same person were just one year older? What if we ran this input again with a different postcode? The simplicity is deceptive. What it’s really doing is forcing the question: Why did the model make this decision? And perhaps more importantly, would it do it again?
Google What if
What is it?
It’s been around since 2018, quietly maintained by Google’s PAIR (People + AI Research) team. You won’t see it trending. It doesn’t generate synthetic faces or hallucinate poetry. But if you want to understand how your model thinks - and what assumptions it’s hiding - this is one of the best tools available.
I stumbled across it years ago, buried in a demo. Within minutes, I was deep in it: tweaking features, analysing performance across groups, generating counterfactuals. I wasn’t watching outputs; I was watching decisions. And more than that I was watching how decisions changed with context. It felt less like debugging and more like holding a mirror up to the system.
The What-If Tool is a model-agnostic interface for visually exploring trained machine learning models. It was designed to make complex concepts like fairness, sensitivity, and explainability something you can actually see. It runs in TensorBoard or standalone via Jupyter notebooks and works with TensorFlow, scikit-learn, XGBoost, LightGBM, and other frameworks. It’s open-source, available on GitHub, and very deliberately built for exploration, not production pipelines.
But here’s what matters: it lets you manipulate your inputs and see how the model responds. That might sound basic. It’s not. It’s radical. Because what you often find is that small, seemingly irrelevant features have massive downstream impact. A £2,000 change in income tips a decision. A change in race or gender reshapes the entire prediction landscape. These aren’t bugs. They’re reflections of the data, the assumptions, and the world the model was trained in.
This isn’t about explainability for developers. It’s about insight for everyone else - the policy team, the ethics advisor, the compliance reviewer who has no idea how XGBoost works but needs to sign off on a system. It’s one of the few tools that lets technical and non-technical users sit in front of the same screen and have a real conversation.
So why use it?
Because most models look fair until you ask the right questions. And most tools aren’t built to ask them. The What-If Tool helps you explore your model across subgroups by race, gender, age, postcode, or any feature you define. It lets you see how predictions shift between groups and how your performance metrics (accuracy, false positive rates, etc.) behave under the hood. That’s vital. A model with 90% accuracy might still systematically underperform for one subgroup. You’d never see it unless you slice the data.
It also lets you define fairness metrics demographic parity, equal opportunity, equalised odds and test how your model performs against each. There’s no single definition of fairness. That’s part of the challenge. But this tool lets you explore trade-offs. And it makes it painfully obvious when a model is structurally unequal, even if no one intended it to be.
But perhaps its most powerful feature is the generation of counterfactuals. You can select any individual in your dataset and ask: What would need to change for the model to reach a different decision? It shows you minimal alterations income, age, job title that flip the prediction. It surfaces the implicit rules the model has learned. And those rules often don’t align with your intentions.
Take a hiring model. You load your dataset. You run the tool. You see that applicants with career gaps are consistently downgraded. You test counterfactuals by adjusting employment history by a few months. The prediction jumps. You do the same for male and female applicants. The differences aren’t just measurable they’re glaring. The model has learned bias. Not because anyone coded it in, but because the data carried it in, and the model reinforced it.
Without a tool like this, it’s easy to miss. You rely on validation metrics and trust the averages. But averages hide too much. This tool lets you watch the edge cases. And often, the edge cases are where harm lives.
There’s a deeper point here. The What-If Tool isn’t just for testing your model it’s for understanding it. That matters because regulatory scrutiny is increasing. Under the EU AI Act, high-risk systems must be transparent, fair, and explainable. ISO/IEC 23894 and ISO/IEC 42001 both require risk controls, documentation, and evaluation of model behaviour. The What-If Tool doesn’t solve governance, but it gives you artefacts recorded outputs, fairness evaluations, counterfactual traces that support compliance and help defend decisions under audit.
Of course, it has limitations. It’s not scalable for production monitoring. It doesn’t offer alerts or pipeline integration. It doesn’t validate fairness in a legally binding way. And if you misuse it cherry-picking inputs or ignoring the hard cases you’ll come away with a false sense of security. But that’s true of every tool. It still requires thought. It still requires judgement.
So where do you start?
Go to the GitHub repo. Open a demo in Colab. Use the sample dataset. Then load your own. Test a model you already trust and see if it holds up. If you’re in a regulated industry, explore how it maps to ISO and EU audit requirements. If you’re in a start-up, use it to explain model behaviour to your design and marketing teams. If you’re in policy, this tool might be the clearest way you’ve ever seen what algorithmic bias actually looks like.
You don’t need to be an ML engineer. You need to be curious. You need to care. And you need to be willing to confront what the model is really doing.
This tool doesn’t guarantee ethical outcomes. It doesn’t fix your model. But it does give you a better map. A visual, interactive, transparent way to ask: What if we did things differently? What if we built for scrutiny rather than scale? What if we refused to be surprised by harm that was always there, just invisible?
The What-If Tool doesn’t feel like software. It feels like a lens. And in a field where so much still operates in the dark, that’s a meaningful thing to carry.