How accurate is AI calorie counting from a photo?

How AI estimates calories from a photo

Counting calories from a picture takes three steps, and modern AI models do them together.

1. Recognition. The model looks at the image and names what it sees: rice, salmon, avocado, a fried egg. This is the part AI does best, because it has learned from huge numbers of food images.
2. Portion estimation. It then guesses how much of each food there is, usually in grams. It uses visual clues such as the size of the plate, a fork next to it or the height of a pile of rice.
3. Nutrition. Finally it turns each amount into calories, protein, carbs and fat using what it knows about typical nutrition values.

Errors can creep in at every step, and they add up. If the model names the food correctly but guesses 150 g of pasta where you have 250 g, the calories will be about 40% low however good the recognition was.

Where photo estimates struggle

Portion size. A photo is flat, so depth is hard to judge. A deep bowl of rice and a shallow one can look alike from above. Research on general-purpose AI models found that they underestimated more as portions got bigger.

Oils, butter and dressings. Fat added in cooking is mostly invisible, and it's calorie-dense. One tablespoon of olive oil has about 119 kcal and one tablespoon of butter about 102 kcal (USDA). The University of Sydney researchers call the inability to detect ingredients added in cooking "a recognised limitation of current food image recognition technology".

Mixed dishes and sauces. Curries, stews, casseroles and layered bowls hide what's inside. In the same study, energy estimates for dishes like bibimbap, beef pho and eggs on toast varied widely between apps, with some well over and others well under.

Drinks. A glass of milk tea or a latte doesn't show how much sugar or what kind of milk is in it. In the Sydney study, apps underestimated the energy in pearl milk tea by up to 76%.

Less familiar foods. The Sydney team found that apps often failed to pick out the parts of Asian dishes such as beef pho and pearl milk tea, and called for more AI training on "mixed dishes and culturally diverse foods".

What the research says

These are peer-reviewed studies you can read yourself. Each tested different apps or models under different conditions, so the numbers aren't directly comparable.

University of Sydney, 2024. Researchers tested 18 nutrition apps, 7 of them with AI photo recognition, published in Nutrients. The best two apps correctly identified 97% and 92% of the food components in the test photos. The authors still concluded that "automatic energy estimations from AI-enabled food image recognition were inaccurate", especially for mixed dishes. Their test photos were taken in good light at a consistent angle, so everyday photos may do worse.

ChatGPT and meal photos, 2025. A study in Nutrients gave ChatGPT-4 photos of 114 meals. It identified the foods correctly 93% of the time. Its estimate of meal weight agreed well with reality for small meals and poorly for medium and large ones, and it underestimated 11 nutrients.

Three AI models compared, 2025. A study in Current Developments in Nutrition tested ChatGPT-4o, Claude 3.5 Sonnet and Gemini 1.5 Pro on 52 standardized food photos. ChatGPT and Claude had an average energy error of about 36%, and Gemini 1.5 Pro had larger errors. All three underestimated more as portions grew. CaloriePix uses newer Google Gemini models, which that study didn't test.

Manual logging isn't exact either. In the same Sydney study, the 16 apps with manual food logging overestimated a day of Western meals by about 250 kcal (1,040 kJ) on average and underestimated a day of Asian meals by about 360 kcal (1,520 kJ). A 2019 systematic review of 59 studies compared self-reported food intake with a precise lab method (doubly labeled water) and found that most studies showed significant under-reporting. A photo estimate and a hand-typed log both carry error. What helps most is logging consistently and correcting the parts you know are off.

How CaloriePix handles it

Because the hard parts are portions and hidden ingredients, CaloriePix is built so you can fix them quickly.

• An item list you can edit. A scan returns each food on its own, with its estimated amount and calories, protein, carbs and fat. Tap any item to change its quantity or unit, or correct its nutrition, down to zero if the AI added something that isn't there. You can also rename the meal or scale the whole thing up or down.
• What the estimate assumed. The review screen includes notes on the assumptions behind the estimate, so you can see what it took for granted before you accept it.
• The pack's own numbers for packaged food. Scanning a barcode looks the product up in the Open Food Facts database, and scanning the nutrition label reads the numbers from the pack.
• Words when a photo can't show it. Describe the meal or say it out loud, with amounts like "2 tbsp pesto" or "a large oat latte". You can also search foods by name in USDA FoodData Central.

When you analyze a photo or description, it's sent to Google's Gemini API to return the estimate. Your diary itself stays on your phone. The estimate is for everyday tracking and isn't medical advice.

Tips for better estimates

• 
  Use good light and get the whole plate in the frameShadows and cropped edges hide food.
• 
  Keep the plate and a fork or spoon in viewThey give a sense of scale, which helps with portion size.
• 
  One plate per photoShared dishes and crowded tables are harder to split.
• 
  Add what the camera can't seeLog cooking oil, butter, dressings and sauces as a separate entry, or describe the meal in words and mention them.
• 
  Check big portionsEstimates tend to come out low as portions grow, so look twice at a large bowl of pasta or rice.
• 
  Use the barcode or label for packaged foodThe pack's numbers beat any estimate.
• 
  Describe drinksSay the size, the milk and whether it's sweetened.
• 
  Weigh now and thenWeighing a few common foods once teaches you what 100 g looks like, and makes your edits faster.

FAQ

Sources

1. Li X, Yin A, Choi HY, Chan V, Allman-Farinelli M, Chen J. Evaluating the Quality and Comparative Validity of Manual Food Logging and Artificial Intelligence-Enabled Food Image Recognition in Apps for Nutrition Care. Nutrients. 2024;16(15):2573. doi:10.3390/nu16152573. University of Sydney news release: AI food tracking apps need improvement to address accuracy, cultural diversity (29 August 2024)
2. O'Hara C, Kent G, Flynn AC, Gibney ER, Timon CM. An Evaluation of ChatGPT for Nutrient Content Estimation from Meal Photographs. Nutrients. 2025;17(4):607. doi:10.3390/nu17040607
3. Fridolfsson J, Sjöberg E, Thiwång M, Pettersson S. Performance Evaluation of 3 Large Language Models for Nutritional Content Estimation from Food Images. Current Developments in Nutrition. 2025;9(10):107556. doi:10.1016/j.cdnut.2025.107556
4. Burrows TL, Ho YY, Rollo ME, Collins CE. Validity of Dietary Assessment Methods When Compared to the Method of Doubly Labeled Water: A Systematic Review in Adults. Frontiers in Endocrinology. 2019;10:850. doi:10.3389/fendo.2019.00850
5. Dalakleidi KV, Papadelli M, Kapolos I, Papadimitriou K. Applying Image-Based Food-Recognition Systems on Dietary Assessment: A Systematic Review. Advances in Nutrition. 2022;13(6):2590–2619. doi:10.1093/advances/nmac078
6. U.S. Department of Agriculture. FoodData Central: olive oil (171413), butter, unsalted (173430)
7. Cal AI FAQ: www.calai.app/faq (checked 2026-10-09)
8. CaloriePix privacy policy: shape-up-me-privacy-policy.vercel.app

Start with your next meal
Snap it, check it, and see your day add up.