How to Check Numbers and Calculations in AI Answers

Evergreen preparation guide.

An AI answer gives you a percentage, a neat formula, and a confident explanation. Your calculator confirms its division. Does that make the answer correct?

Only if it divided the right quantities to answer the user's question. A calculation can be accurate while using the wrong population, unit, or starting value. Evaluating numerical answers means checking both the setup and the arithmetic.

You can practise this skill without advanced mathematics. Begin with percentages, counts, and unit conversions, then write a precise explanation of the first step that goes wrong.

What the research supports

The 2023 paper Let's Verify Step by Step notes that final-answer grading can mistakenly accept solutions whose reasoning is incorrect. Its experiments concern mathematical reward models; they do not establish a universal rule for grading every AI response. Lightman et al., section 2.5.

In GSM-Symbolic, first submitted in 2024, researchers tested variations of grade-school mathematical questions, including changed numbers and added irrelevant information, and reported sensitivity in the models they studied. That supports checking the particular problem in front of you rather than trusting familiar-looking wording. It does not prove that all models lack reasoning ability. Mirzadeh et al..

The routine below is an original practice method. It checks the written calculation and explanation you can see; it does not reveal a model's hidden internal process.

A five-step numerical check

1. Name the quantity the user requested

Before reading the AI's working closely, complete this sentence: “The answer should tell me ___, measured in ___.”

For a percentage, identify what counts as the whole group. For a rate, identify what it is per hour, person, or item. Notice whether the user requests a total, an average, a difference, or a percentage change. These are different calculations even when they use the same facts.

2. Extract the relevant facts with their units

Make a small scratch list of the supplied values and what each represents. Keep “registered participants” separate from “staff,” and “millilitres” separate from “litres.”

Check whether a number comes from the prompt, an allowed source, or an unsupported assumption. Do not automatically use every number in a word problem. Explain why a detail belongs in the calculation before adding or subtracting it.

3. Calculate independently

Work out the answer yourself before following the model's formula. Use a calculator if the assessment's rules allow it.

Write the setup in ordinary words first: “registered attendees divided by all registered participants, multiplied by 100.” Then insert the values. This helps you avoid checking only the arithmetic inside an already incorrect formula.

Add a quick reasonableness check. A fraction representing attendance within one fixed group should fall between zero and 100%. Keep rounding until the end unless the task specifies otherwise.

4. Locate the first unsupported step

Compare the visible working with your independent calculation. Classify the actual problem:

• Setup: the answer uses the wrong group, denominator, or operation.
• Arithmetic: the operation is appropriate but computed incorrectly.
• Units: the conversion or unit label is wrong.
• Assumption: a required value or adjustment has no support.

You may find more than one issue. Start with the earliest decisive error, then explain how it affects the conclusion. Avoid a vague label such as “bad maths” when you can identify the exact mismatch.

5. Apply the supplied rubric

Separate final-answer accuracy from the quality of any required explanation. A correct number can have invalid working. A sound setup can also end in a multiplication mistake.

If the task only requests a final value, do not invent a requirement for lengthy intermediate steps. If it asks for working, check that the shown steps support the result. Valid alternative methods deserve credit; the response need not use your preferred formula arrangement.

Worked example: who belongs in the percentage?

Fictional practice scenario. All figures and responses are invented.

The user asks:

Eighty people registered for a practice webinar. Sixty-four of those registrants attended. Eight staff members also joined. What percentage of registered participants attended? Show the calculation.

Response A: “There were 72 attendees out of 88 people: 72 ÷ 88 × 100 = 81.8%. Therefore, 81.8% of registered participants attended.”

Response B: “Of the 80 registered participants, 64 attended. The rate is 64 ÷ 80 × 100 = 80%. Staff are outside the registered-participant group.”

B is better. The requested group is registered participants, so the numerator is 64 and the denominator is 80. The staff count changes neither value.

A's division is approximately correct for 72 divided by 88. Its error happens earlier: it adds staff to both counts and answers a question about a combined group instead. Repeating its calculation on a calculator would confirm the arithmetic while missing the wrong setup.

A useful justification is:

B uses the requested population: 64 attendees among 80 registrants, giving 80%. A includes staff in both counts, so its 81.8% answers a different question. The decisive error is the group definition, not the division.

As a cross-check, 16 registrants were absent. That is 20% of 80, leaving an attendance rate of 80%.

Exercise: a correct result with a faulty conversion

Fictional exercise. The user asks for the combined volume of two 750 mL bottles in litres, with working. The rubric checks both the final value and valid calculations.

The AI writes: “2 × 750 = 1,500 mL. Divide by 100 to convert to litres, giving 1.5 L.”

Which parts pass, and what should be corrected?

Explained answer: The total of 1,500 mL and final value of 1.5 L are correct. The conversion step is wrong: 1 L equals 1,000 mL, so divide 1,500 by 1,000. Dividing by 100 would give 15, not 1.5. Under this rubric, final-value accuracy passes while the written conversion fails. Do not infer why the model made the mistake; evaluate the text it supplied.

Common mistakes

• Checking a formula before checking what its quantities mean.
• Treating fluent explanation as evidence that the calculation works.
• Calling acceptable rounding an error without reading the requested precision.
• Giving full reasoning credit just because the final number matches.
• Rejecting an equally valid method because it differs from yours.

Practise precise evaluation

Build the habit of identifying the requested outcome, checking the evidence, and explaining the decisive difference between responses.

Take your first AtomicEval challenge to practise response comparison and written reasoning. It has a five-minute target and can be started without signup. A free profile is required to view and save your result.

When you want to continue preparing, create a free AtomicEval profile.

Sources

• Lightman et al.: Let's Verify Step by Step, submitted May 31, 2023.
• Mirzadeh et al.: GSM-Symbolic, submitted October 7, 2024; revised August 27, 2025.