Anyone with an Instagram account is never more than a few seconds away from a photo of someone else’s dinner. Cashing in on this food envy, a computer scientist has developed a program that can find the recipe for a dish based on a single photograph.
Using artificial intelligence that was “fed” more than a million recipes and 80,000 images of food, the neural network has learnt to recognise combinations of ingredients based on a single picture of a finished dish.
Users can upload a photograph of a dish and the technology pings back a recipe — just like Shazam, the music recognition app, but for food. In tests, the app had a 65 per cent success rate at finding correct recipes.
The website, Pic2Recipe!, was created by Nick Hynes, an MIT electrical engineering and computer science student, who sourced the recipes from Food.com and AllRecipes sites.
From this his team developed a database with more than a million annotated recipes.
However, more work is needed to hone the AI’s ability to identify more ambiguous foods. Tests have shown that it is very good at picking out the recipes for baked food but less adept at identifying what goes into more complex foods such as sushi or smoothies. Early tests reveal that it gets confused when there are many similar recipes for the same dish and could not yet identify several simple dishes, including ramen, crisps and rice and beans, which all returned “no matches”.
“It still has a lot of difficulty identifying certain food types, including blended and mixed foods like smoothies, soups and sushi rolls,” Mr Hynes told Wiredmagazine. “It makes a big difference if photos are taken from close-up or far away, or if the photo is of a single item, multiples or part of a dish.”
Scientists have also created a system that can estimate the volume of food from a photograph to help diabetics to monitor their carbohydrate intake.