How computer vision for autonomous cars will help you eat healthier.

There have been huge advances in artificial intelligence in recent years and we can reasonably expect to see practical, autonomous cars a reality very soon. If we can expect cars to drive themselves, how is tracking our nutrition still a problem?

It shouldn’t be and we’re here to fix that.

So, how are the two problems even similar? They‘re as different as chaulk and cheese, but you’ll be surprised that the methods used to solve the tidious problem of driving a car can be applied to how we solve the problem of tracking our nutrition.

Well, what does it take to make a car autonomous? Everyone imagines that eventually artificial intelligence will see the world like we do and then learn to drive like us. Easy peasy. Right?

Before I go any further, let me define some terms.

Artificial intelligence, machine learning, deep learning? What’s up with all these terms? Here’s a simple way of thinking about them. They all refer to methods used to take input data to train a model, so we can use the model to predict output from new input data. The popularity of each term stems from history. Nvidia has a perfect chart to clarify what I mean:

As you can see, deep learning is the latest advancements in A.I., so from henceforth, I will use this term.

When you imagine that a car just takes sensory input and simply learns how to drive, that might be possible with what’s called “end-to-end” deep learning, but that is hard to do in the real world because end-to-end deep learning is a data hog. You need a lot of data, not to mention the right, high quality data.

Instead, car companies today are tackling the problem stepwise. On a very high level, the problem of making a car drive itself is broken down into three sub-problems.

1. Take input sensory data (imagery, radar, lidar) to detect pedestrians, vehicles, traffic lights, road signs, etc.
2. Plan a route based on input data, also called motion planning in robotics.
3. Steer the car to execute route, also known as the control systems.

For each sub-problem, they apply the latest techniques in deep learning to model, predict, and perform subtasks. For example, they may use deep learning to develop computer vision to detect objects on the road or they may apply deep reinforcement learning to motion planning. By breaking the problem down to sub-problems, data scientists can hand-design algorithms that requires less data to achieve human level performance, which in deep learning speak is a proxy for the Bayes error rate — the lowest possible error rate or irreducible error for any classifier.

Okay, so how does this apply to nutrition?

If you can imagine a day where your car drives itself, why can’t it help us drive this body of ours? If an A.I. can make decisions on when and where to steer a car, then it can certainly inform us on when and what to eat.

Imagine this scenario:

At 7am, you wake up and put your Google ContactLenz on and go to the kitchen to fix breakfast. You grab the Captain Crunch box and your heads-up display flashes a quick warning that if you eat this cereal, your blood sugar will spike, which historically results in a sudden drop of energy around 11:26am. You’re also low on protein and vitamin C, so your A.I., Tali, recommends Greek yogart with granola and lemon curd. Oh, and your favorite Northern Spy apple is also available at the corner Amazon Go store. Tali says, “Shall I order for you?” You reply, “Yes, and add a chai latte to the order.” She says, “You got it. It’ll be ready when you get there.”

After breakfast, you go to work and there’s a box of donuts in the office. As you grab the donut, a heads-up display shows you its nutritional content. 353 calories, 7 grams of saturated fat. You think to yourself, “maybe not, I’ve got to fit into my jeans for the reunion.”

Come lunch time, you go down to the cafeteria and browse the selection. Margherita pizza, maki sushi, pesto penne, stir-fry beef noodles, miso ramen, almond curry and naan. Each time an entree comes into view, your heads up display shows nutritional information. You decide on the margherita pizza. As you place each slice on your plate, Tali approximates the nutritional intake you’ll be eating in your heads up display. You think to yourself, “wow, I didn’t know three slices was almost 700 calories.”

This is the future I see. A day when you can know in real-time what nutrients you’re going to put in your body. It won’t be easy, but with deep learning, I think it’s possible. The advancements of autonomous cars gives me hope and their strategy could be a blueprint that we can follow. It’s an audacious goal, I know, but it seems worth shooting for.

Thanks for reading my ramblings. We’re building cool, new tools to help people track their nutrition. I’d love to hear your thoughts! Please leave a comment below!