| dc.description.abstract | Perception and action have a close functional relationship. While perception provides the means for actions, actions generate more opportunities for perception. This relationship between perception and action, between the living organism and its environment, has been actively studied. However, a few aspects have not been investigated sufficiently including the influence of motor responses on perceptual organization and functions, the role of sensorimotor in the development of tool-use capability, and the emergence of multiple-tool-use behavior in sensorimotor learning. In this study, I will address three questions to examine several different angles of perception and action. These include: (1) can motor control affect perceptual performance? (2) can proprioception and kinesthesia accelerate the development of tool-use behavior?, and (3) can a reinforcement learning agent identify different types of tools and learn to use these tools in different manners? To answer the first question, we investigated the origin of the oblique effect, where it is harder to judge the angle of oblique lines than horizontal/vertical lines. We proposed that the basis of the effect is due to the mechanical makeup of the oculomotor system that affects the perceptual performance in tasks involving oblique patterns. The idea was tested by using a mechanically based mathematical model to generate the motions of the eye and a multilayer perceptron as a simple surrogate of the visual cortex to recognize obliques vs. vertical and horizontal visual patterns. Our results indicated that the specific mechanical properties of the motor system could systematically influence perceptual function. On the second question, we proposed the use of proprioceptive and kinesthetic sensory feedback to speed up the tool-use learning process. For this, we conducted a series of experiments in which a reinforcement learning agent is trained to pick up a tool placed in the environment and utilize this tool to reach a target object and drag it down. These experiments enabled us to analyze the impact of different types of sensory input on the development of tool-use behavior. The experimental results show a higher success rate and faster training convergence time when proprioceptive and kinesthetic sensory feedback is used in addition to the visual input.
Finally, on the last question, I studied the ability of a sensorimotor learning agent to use multiple types of tools. To achieve this objective, I upgraded the original tool use environment to vary the type of tool and randomize their positions. I then trained the agent to learn to use these different tools to complete the same task. Finally, I examined and analyzed the behaviors of the agent while handling these tools to figure out if it can learn to use the tools based on the different perceived properties of the tools. Analyzing the behaviors of the trained agent provide insights on the strategies that the agent developed to solve the task for each tool and demonstrated how multiple tools could be used to solve the same task by a single model. Overall, the studies in this research are expected to help us gain better understandings over the intertwined relationship between perception and action ranging from the effect of motor responses on perception to the emergence of tool-use behaviors in sensorimotor learning systems. | en |
