Reinventing Food

The world is growing and faster than ever. Scientist predict we will cross 9B people this century. And while there are arguments about when that growth will stop and how quickly, one thing is clear: we are going to run out of food soon.

Recent research proposes that in order to accommodate such a large population, we are going to have to change our diet. Specifically, move away from meat and consume mostly (if not exclusively) fruit and vegetable.

Meat consumption has dire effects on the environment. It is expensive to grow, raises moral concerns, but primarily it is extremely ineffective. An average cow will consume 30 times more calories in its lifetime than it will provide. The number is much smaller (about 5 times) for poultry. And of course, the most effective way is to consume vegetable based calories.

The worldwide trend is clear: as countries get wealthier, their meat consumption increases. Together with the fact that the world’s population is larger than ever and growing, we are heading towards an environmental crisis and likely famine.

What can we do? Other than switching to a vegetarian diet, we also need to start thinking about optimizing our farming. This is where robotics, computer vision and machine learning can help. For example, fully automated farms that are controlled by algorithms can be extremely effective and minimize waste, space and therefore cost and environmental footprint.

The Future of Employment

The workplace is changing faster than ever. Manual labor was already largely replaced by automation. The impact of automation created a more educated workforce. People were required to get more advanced education so they could perform complex tasks that were beyond what machines could do.

The recent advances in AI and robotics may lead to a future where education is not enough. More importantly, AI may make people unnecessary. If in past revolutions people became under qualified and had to adjust, with AI there may be no adjustment to make. A large portion of the workforce will be taken over by machines, and people will have to find something else to do with their now significant free time.

There are certain professions that are likely to last longer than others. For example, surgeons will be easier to replace by robots than nurses. In fact, while automation took over manual labor, AI may initially take over highly specialized professions. But, ultimately, we can expect machines to take over the entire workforce.

What will people do in a world where their labor isn’t needed? It’s going to be a rough adjustment. Work has been central to our culture and lives for a very long time. But, not always. Hunter gatherer communities certainly had to spend time getting food and supplies. However, outside of that, they could spend most of their time on social and artistic activities.

I don’t know what the workforce will look like in a 100 years. But, I’m certain it will be very different. While many things will become easier and simpler, the quest for purposes and satisfaction will likely become increasingly challenging.

The Curse of Dimensionality

You can find more info about the following post on my website: Dov Katz

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces (often with hundreds or thousands of dimensions) that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience.” (https://en.wikipedia.org/wiki/Curse_of_dimensionality)

This is a funny definition if I’ve ever seen one. It implies that our everyday experience is low dimensional and therefore the data associated with it is easy to analyze and organize. But if the 3D world we live in was so simple, why is AI struggling to make sense of it?

The answer is that the above definition is both right and wrong. It’s wrong because the world we live in is very very very high dimensional. Sure, our space is three dimensional. But our visual perception of it is composed of millions of pixels refreshed multiple times per second. Therefore our visual data of the simple 3D space around us actually has millions of dimensions.

At the same time, this definition is exactly right. Or at least, it exposes a brilliant truth: it must be possible to organize the data pertaining to the world around us in a low dimensional form. Otherwise, how can any intelligent creature with finite resources (read: humans) make sense of it?

One of my favorite example is the dimensionality of a line. We all remember from school that a line can be described using the equation y=ax+b. In short, a line can be described using two parameters: a & b. Now, imagine tilting your head such that the line rotates to be parallel with the ground (or: x axis). Now, you can describe the line using a single number: y=c (or: the height of the line parallel to the x axis).

This simple example illustrates that perspective or representation is key. If you look at the data the right way, a high dimensional state space becomes lower dimensional. And, in the lower dimension representation, problems are easier to solve, and it becomes possible to make sense of a high dimensional world.

Object Segmentation

What is object segmentation?

Object segmentation is computer vision speak for “draw a bounding box around an object”. What it really means is separating the background from the foreground (the object).

If you’re new to computer vision, you’re going to think this is the easiest problem in the world. I mean, it’s pretty trivial for us to tell where one object ends and another begins. In fact, there’s good reason to believe our ability to quickly segment a scene into objects is the result of natural selection. If it takes you too long to identify the tiger hiding in the woods, you’re not going to survive long enough to tell the story.

So why is image segmentation so hard for computers? The main reason is that there are no rules. Objects have varying texture, size, colors and shapes. It’s practically impossible to hard-code a set of rules to identify where an object “ends”. It gets much easier if the computer has prior knowledge.

How to solve segmentation?

Prior knowledge can come in the shape of specific knowledge. For example, if you’re focusing on detecting screws in images taken on a factory floor, you can make some assumptions that render the problem simpler. You can also train a neural net to identify specific classes of objects.

A little less common is the idea of motion. If an object is moving, segmenting it becomes trivial. Think about watching cars on a highway. Segmentation becomes image subtraction: look at two consecutive images, the only thing changing is a car (because the highway looks the same). If you simply subtract the first image from the second, you’ll get a precise segmentation of the moving car!

You can also take this idea one step further. What if segmentation isn’t a passive task? What if we have a robot that can create motion? Well, in that case image segmentation becomes much easier. This is a great application of Interactive Perception.

Children learning segmentation

What I find most fascinating here is that a robot could bootstrap object segmentation through interaction. Eventually, it has seen enough examples that it can learn to do segmentation even without poking objects. And, if this sounds familiar, it’s probably because this is how visual tasks are learned by children.

You can read more about this type of work on my websites: Dov Katz and Dubi Katz.

The Scientist in the Crib

The Scientist in the Crib

My name is Dov Katz. My training is in roboticis. My research and work are focused on the intersection of computer vision, machine learning and robotics. I also enjoy writing about these topics.

Here are some thoughts about Interactive Perception as I originally published on Medium:

Human Learning

Humanity’s advantage over the rest of the biosphere is out amazing abaility to learn and adapt. Alison Gopnik, Andrew N. Meltzoff and Patricia K. Kuhl. wrote a book called “The Scientist in the Crib” that discusses how babies learn.

Scientific Learning

Scientists often start with a crude, intuitive understanding of a phenomena. They then explore it by conducting some experiments. But, to conduct insightful experiments, scientists spend considerable amount of time learning what others have done. Typically, a scientific experiment is the last floor in a very tall building, sometimes extending decades or even centuries back.

Children Learning

So how do children learn? They begin with some crude knowledge provided by our genes. For instance, researchers have demonstrated that newborns already understand the concept of momentum. If you show a newborn a moving object that disappears behind a blanket, they expected it to reappear on the other based on its velocity and direction.

Of course, there’s only so much knowledge that can be hardcoded. Most things have to be learnt. Children appear to be spending much time playing. But, really, they are conducting serious experiments. What happens when I paint the wall? Will the glass break if I drop it? Is twisting the doorknob the right way to move the door?

Finally, there are teacher, also known as “parents”. The world is pretty complex. Making sense of it by running experiments alone isn’t practical. A parent, however, can give us a shove in the right direction to maximize our learning. This is what researchers refer to as “structure”. When you have a sense of how to organize things, discovery the logic behind a phenomena becomes easier.

The "Scientist in the Crib" book

The “Scientist in the Crib” does a good job describing these three components of learning: what we’re born with, what we learn from experimenting, and the role of teachers. It includes some fascinating examples demonstrating we are born with quite a bit of knowledge.

An interesting topic in the book is the brain’s plasticity. This is the notion that our brain is extremely adaptive. For instance, Japanese and English speakers are sensitive to different sounds. An adult speaker might not be able to hear a certain sound because of their cultural background. Babies, however, are able to hear all sounds regardless of their culture. This demonstrates brain plasticity — because culturally we don’t need to distinguish between two sounds, the adult brain adapts and can no more hear the unnecessary sound,

If you find this topic interesting, I highly recommend reading the book!

Check out my social media profile: https://www.linkedin.com/in/katzdov/ for more information. Also, my personal webpage: http://www.dovkatz-bio.com and http://www.dubikatz.com

Interactive Perception / Dov Katz

Interactive Perception

My name is Dov Katz. I am a roboticist by training. My research and work focuses on the intersection of machine learning, computer vision and robotics. I also like to write about these topics. 

Here are some thoughts about Interactive Perception as I originally published on Medium:

Introduction

I’ve been fascinated with robots forever. The idea of a machine that can think like a person but is physically far superior is intriguing. As I started doing research in robotics I quickly realized a surprising fact: building a brilliant machine is easy. Designing a machine that can do everything we do effortlessly is hard.

Here’s the truth about how far we’ve gotten in about 50 years of robotics: machines can beat almost every living person in chess, they can drive cars and navigate on Mars. And yet, they suck at opening a drawer, taking out some forks and placing them on the dinner table. This seems… weird. Why would robots excel at things so far out of what any 3 years old cares about and be so clueless in what every 3 years old can do effortlessly?

I believe the answer is that robots don’t get to grow up. They’ve never had to build from the grounds up, they never developed curiosity. Human children in the first three years of life are consumed by a desire to explore and experiment with objects. They are fascinated by causal relations between objects, and quite systematically explore the way one object can influence another object. They persistently explore the properties of objects using all their senses. For example, a child might gently tap a new toy car against the floor, listening to the sounds it makes, then try banging it loudly, and then try banging it against the soft sofa. This kind of playing around with the world, while observing the outcome of actions, is more than just play. It actually contributes to babies’ ability to solve the big, deep problems of disappearance, causality, and categorization.

Action and Perception

This explanatory drive tightly couples action and perception. This coupling was first observed in the 80s by the psychologist Gibson. Gibson’s research views perception as an active process, highly coupled with motor activities. Motor activities are necessary to perform perception and perception is geared towards detecting opportunities for motor activities. Gibson called these opportunities “affordances”. In my research in robotics I referred to this process as Interactive Perception.

Perceiving the world, making decisions, and acting to change the state of the world seem to be three independent processes. This is exactly why most people consider action and perception as separate. However, “enactive” approach to perception may be essential for surviving in a high-dimensional and uncertain world. Interactive Perception provides a straightforward way to formulate theories about the state of the world and directly test these theories through interactions.

For example, think about the first time a child encounters a pair of scissors. She has no sense of what this object does or how it works. Yes, she could spend some time looking at it and making educated guesses. But, what the child is most likely going to do is poke and probe it. This interaction will create motion, and this motion will make it easy to determine what scissors can do.

Interactive Perception imposes structure. It limits what needs to be perceived and explained. If we have hope of building robots that can do what toddlers do, I believe making them curious and letting them interact with the world to learn about it is essential.

And, maybe once they are expert toddlers, we can start thinking about sending them to preschool :-)