I was recently asked to answer a few questions about cognitive robotics in the context of industrial and factory applications. It is always interesting to explore the possible connections between academic research and real-world applications and so, here’s how the first round of Q-A went.

*[part of the text used in the answer to some questions is taken from the webpage of the Cognitive Robotics group at UofT]

1. What is your definition of cognitive robotics?

The term “cognitive robotics” is used to refer to robots with higher level cognitive functions that involve knowledge representation and reasoning. A cognitive robot has an internal representation of its own abilities to act and perceive information as well as the current state of the environment. This allows the robot to reason and synthesize an appropriate course of action for each of the situations it may encounter as a result of reasoning about goals that need to be fulfilled, the available actions that can be performed, sensor data that the robot perceives from the environment, the mental states of other robots, etc.

- What makes a robot a cognitive one? What sets it apart from other robots?

Every robotic system has a kind of core program that specifies what are the actions that the robot should perform at every point in time (e.g. move arm towards object, open gripper, etc). This program is done by a programmer who takes into account the task that the robot should do, the mechanical actions it can perform, the information the robot can get from the environment, etc, and constructs a program that when executed will realize the intended behavior.

In a sense, most of the logic behind the structure of the program is implicit as it only exists in the mind of the programmer. The intended behavior for the robot might be something like “whenever there exists an object in some specific boundaries, the robot should pick it up and move it to the appropriate box”. Even though this might be apparent when inspecting the code, the actual logic behind this is not explicit in the program.

The difference with cognitive robotics is that instead of programming a robot with the sequences of mechanical actions it should perform when certain conditions arise, a cognitive robot is programmed with the logic behind the mechanical actions it can perform and how to synthesize these mechanical actions in order to bring the environment to a specific state. Any task that the robot should do can then be expressed in a more abstract way and the actual sequences of actions that should be performed will be computed by the program itself.

- How is it different from vision-guided robotics?

The notion of cognitive robotics is somewhat orthogonal to vision-guided robotics. The central feature of cognitive robotics is the explicit representation of the interaction between the robot and its environment, and the ability of the robot to reason within this representation so as to specify the actions that should be performed in order to complete a task. As the the robot is reasoning to compute the course of action the sensor data is also taken into account and can be used to guide it. This sensor data can be a set of features extracted by visual cameras, laser or infrared sensors, or any other kind of sensors that are appropriate.

2. What is the goal of cognitive robotics?

Similarly to question 1, cognitive robotics is concerned with endowing robots with higher level cognitive functions that involve reasoning, for example, reasoning about goals, perception, actions, the mental states of other robots, collaborative task execution, etc.

3. What industrial applications are we seeing cognitive robotics applied in today?

To my knowledge there are no industrial applications where the cognitive robotics design paradigm is used.

- What applications will we see in the future?

In the past robots have replaced humans in many tasks in the industry, mainly tasks that involved certain skills and minimal reasoning capabilities. It is possible that as the techniques related to cognitive robotics become more powerful and the implementations of cognitive robots more robust, we will see cognitive robots replace humans in more advanced tasks that currently robots cannot safely perform.

Also, the cognitive robotics design paradigm offers some advantages with respect to modularity and adaptability, even though reasoning may not be necessary for a robot in an industrial application. The fact that the behavior of a cognitive robot is a result of reasoning about the ways it can interact with the environment and the specification of the task it should do, enables it to adapt to different settings. For instance, a futuristic scenario may involve cognitive robots that consist of modules that come with a representation of their parts and the dynamics of the available mechanical actions they can perform. The modules can then be combined and incorporated into industrial applications by programming only at the level of task specification, while the low-level details are sorted out by reasoning that the cognitive robot does by itself.

4. What is needed in order for cognitive robotics to perform? (i.e. software, hardware, etc.)

Cognitive robotics is a design paradigm that when applied to robotic agents (in contrast to software agents) it involves taking care of issues that lie in several different fields of research and application. In particular there are the following parts:

1. The mechanical part of the robot that is responsible for movement and affecting the environment (e.g. mechanical arms, the body, the motors).
2. The software and hardware part responsible for getting meaningful information from the environment that the robot is situated (e.g. the hardware and the information processing software for doing feature extraction from visual images and sound).
3. The software part responsible for the representation of the environment and the way that the robot can interact with it (e.g. a logical specification of the properties of the environment as well as how these are affected by the available actions that the robot can perform).
4. The software part responsible for the specification of the task that the robot should do based on the previous representation.
5. The software and hardware part that makes use of the representation of the environment and the specification of the task that the robot should do (numbers 3 and 4) in order to compute the behavior of the robot at any given moment.
6. The software and hardware part that provides the interface between the reasoning component (number 5) and the actual sensors and actuators in the environment (numbers 1 and 2).

5.What are the challenges associated with cognitive robotics?

Even though each of the parts discussed in the previous question comes with its own set of challenges, the synthesis of those parts into a robust cognitive robotics architecture is probably the biggest challenge. In particular for the case of cognitive robots for industrial applications it is needed that people from the relevant fields of academic research and the industry are brought together in order to identify a specific real-world industrial application that will motivate approaches that deal with the problem as a whole.

6. What robotics “problems” does cognitive robotics fix?

Cognitive robotics is a design paradigm that separates the “high-level” intended behavior of the robot from the “low-level” details of how it can be realized. Depending on the requirements for a robotic application this may be beneficial with respect to adaptability and modularity. Also, the knowledge representation and reasoning part provides a clean framework for specifying advanced tasks related to proactive behavior (e.g. short-term planning before acting in order to find a particular sequence of actions that can achieve a specific goal).

7. What are some of the really amazing applications you’ve seen cognitive robotics applied in?

(Not an industrial application). There is an ongoing project regarding unmanned aerial vehicles that uses fully autonomous helicopters for finding survivors in rescue missions.

8. Tell me about the research your group is doing in cognitive robotics.

Part of the group’s focus is research related to a high-level programming language for controlling robots called IndiGolog. In this language one can specify the following:

• The available actions that the robot can do, including actions that affect the environment and actions that extract information from the environment using its sensors.
• The objects and properties of the environment and how these change for each of the possible actions the robot can do.
• A high-level control program that specifies the behavior of the robot. This looks like an ordinary imperative program but uses the properties of the environment as control-flow expressions and the actions that the robot can do as basic statements. Moreover, the program may include statements expressing that the robot should act accordingly in order to make a certain property of the environment become true. For this kind of statements the interpreter will have to reason at run-time in order to compute a sequence of actions that will realize the intended property based on the conditions that hold at run-time.

A prototype implementation of the language has been developed and experiments have been conducted using the language to build a high-level controller for simple robots such as the Sony ERS7 Aibo.

- Do you partner with industry/manufacturers for research? Find out what they’d like robots to do?

There is no close connection with the industry or manufacturers at the moment. As mentioned also in question 5, it is one of the current challenges to bring the two communities closer together.