Behavior-based Control
Part 1: Ethological Background
CIS548
Recall:
A dominant theme in early intelligent robotics – hierarchical, KR-intensive, oriented toward logic & theorem-proving, heavily influenced by GPS & STRIPS. Various proposals for reactive control systems, e.g., Brooks, Agre, Chapman [Pengi].
Another influence – behavioral studies of animals spurred by early work of Lorenz & Tinbergen.
The material below gains a lot of information from Parker’s notes:
Animal Studies:
Late 1970s and early 1980s
Michael Arbib
investigation of animal intelligence from biological and cognitive sciences
Ethology – the scientific study of animal behavior
Heinroth & Lorenz – recognized and pioneered study of fixed action patterns
Fixed action pattern
From Wikipedia, the free encyclopedia
“In ethology, a fixed action pattern (FAP) is a complex behavioral sequence that is indivisible and runs to completion. FAPs are invariant and are produced by the innate releasing mechanism that responds to a external sensory stimulus (sign stimulus).
Examples
A mating dance may be used as an example. Many species of birds engage in a specific series of elaborate movements, usually by a brightly colored male. How well they perform the "dance" is then used by females of the species to judge their fitness as a potential mate. The key stimulus is typically the presence of the female.
Human infants grasp strongly with their hands as a response to tactile stimulus.
Fixed action pattern is used in biology to classify ethology”
Lorenz also recognized imprinting: “Imprinting (psychology)
From Wikipedia, the free encyclopedia
Imprinting is the term used in psychology and ethology to describe any kind of phase-sensitive learning (learning occurring at a particular age or a particular life stage) that is rapid and apparently independent of the consequences of behavior. It was first used to describe situations in which an animal or person learns the characteristics of some stimulus, which is therefore said to be "imprinted" onto the subject.”
Intelligence:
Parker: “Open question: where intelligence begins and ends
Intelligence (our working definition): the ability to improve an animal or
human’s likelihood of survival within the real world, and, whereappropriate,
to compete or cooperate successfully with other agents to do so.
For our purposes, we call certain animal behaviors “intelligent”
Merriam-Webster: “the ability to learn or understand or to deal with new or trying
situations : REASON; also : the skilled use of reason (2) : the ability to apply
knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (as tests)”
Encarta: “ability to think and learn: the ability to learn facts and skills and apply
them, especially when this ability is highly developed”
Free dictionary: “The capacity to acquire and apply knowledge; The faculty of
thought and reason; Superior powers of mind.”
Our definition: The ability to think, reason, plan and act according to plan.
Cognitive Psychology:
Wikipedia: “the school of psychology that examines internal mental processes such as problem solving, memory, and language. It had its foundations in the Gestalt psychology of Max Wertheimer, Wolfgang Köhler, and Kurt Koffka, and in the work of Jean Piaget, who studied intellectual development in children. Cognitive psychologists are interested in how people understand, diagnose, and solve problems, concerning themselves with the mental processes which mediate between stimulus and response. Cognitive theory contends that solutions to problems take the form of algorithms—rules that are not necessarily understood but promise a solution, or heuristics—rules that are understood but that do not always guarantee solutions. In other instances, solutions may be found through insight, a sudden awareness of relationships.”
For us: We extend the scope of cog. psych. In several ways:
Memory: We include all aspects of knowledge representation, including
learning and knowledge structuring, acquisition, updating, storage &
retrieval.
Problem solving: We include all aspects of reasoning including proof,
cognitive processing of input, projection and planning.
Language: Because of the incredible breadth and depth of this subject, we
focus only on the essentials of communicating information; ultimately
language is intimately related to both memory and problems solving as outlined above, though, unfortunately it is beyond the scope of this course.
Study of animal behaviors:
Why study? See Parker’s list, p. 11 (Sept. 5)
Examples: (from Parker)
Frog’s visual system
Homing pigeon
Autonomous agents:
Wikipedia: “An autonomous agent is a system situated in, and part of, an environment, which senses that environment, and acts on it, over time, in pursuit of its own agenda. This agenda evolves from drives (or programmed goals). The agent acts to change the environment and influences what it senses at a later time.
Non-biological examples include intelligent agents, autonomous robots, and various software agents, including artificial life agents, and many computer viruses.”
Intelligent agent (wikipedia): “I n computer science, an intelligent agent (IA) is a software agent that exhibits some form of artificial intelligence that assists the user and will act on their behalf, in performing repetitive computer-related tasks. While the working of software agents used for operator assistance or data mining (sometimes referred to as bots) is often based on fixed pre-programmed rules, "intelligent" here implies the ability to adapt and learn.
In some literature IAs are also referred to as autonomous intelligent agents, which means they act independently, and will learn and adapt to changing circumstances. According to Nikola Kasabov[1] IA systems should exhibit the following characteristics:
* learn and improve through interaction with the environment (embodiment)
* adapt online and in real time
* learn quickly from large amounts of data
* accommodate new problem solving rules incrementally
* have memory based exemplar storage and retrieval capacities
* have parameters to represent short and long term memory, age, forgetting, etc.
* be able to analyze itself in terms of behavior, error and success.”
Autonomous robots (wikipedia): “Autonomous robots are robots which can perform desired tasks in unstructured environments without continuous human guidance. Many kinds of robots are autonomous to some degree. Different robots can be autonomous in different ways. A high degree of autonomy is particularly desirable in fields such as space exploration, where communication and delays and interruptions are unavoidable.”
For us: An autonomous agent and/or robot: an entity that is intelligent (as defined above) and which, when given a task specification, is capable of understanding the specification and of planning and carrying out that task on its own.
Development of behaviors:
What is a behavior?
Parker – Sept. 5, p. 17: “Mapping of sensory inputs to a pattern of motor actions that are used to achieve a task”
For us: A trigger-action sequence pair, where
Trigger: An input which initiates a behavior. It can be a simple sensory input,
the output of a control mechanism, or a prescribed combination of the one or the other or both.
Action sequence: a prescribed sequence of actuator impulses, behaviors, or a
prescribed combination of the one or the other or both.
Simple example: Flee contact
Trigger: Sensor reading indicating contact established at orientation O.
Action sequence: orient motors anti-O; actuate motors.
Another example: GoTo loc
Trigger: Control mechanism output: GoTo(loc)
Action sequence: Generate path; Foreach(pathsegment) GoFrom(A,B)
Types of behaviors (Parker, 9/5, p.17):
“Three broad categories of behaviors:
–Reflexive behaviors:
•Stimulus-response
•Hard-wired for fast response
•Example: (physical) knee-jerk reaction
–Reactive behaviors:
•Learned
•“Compiled down” to be executed without conscious thought
•Examples: “muscle memory” –playing piano, riding bicycle, running, etc.
–Conscious behaviors:
•Require deliberative thought
•Examples: writing computer code, completing your tax returns, etc.”
Types of reflexive behaviors (Parker, 9/5, p.18):
“Three Types of Reflexive Behaviors
•Reflexes:
–Rapid, automatic, involuntary responses triggered by certain environmental
stimuli
–Response persists only as long as the duration of the stimulus
–Response intensity correlates with the stimulus’ strength
–Used for locomotion and other highly coordinated activities
•Taxes:
–Behavioral responses that orient animal toward or away from a stimulus
–Occur in response to visual, chemical, mechanical, and electromagnetic phenomena
–Example: pheromone trail following of ants
•Fixed-Action Patterns:
–Time-extended response patterns triggered by a stimulus
–Last longer than the stimulus itself
–Intensity and duration of response is not governed by the strength and duration of the stimulus
–May be motivated, may result from a much broader range of stimuli
–Examples: the song of crickets, predator fleeing, etc.”
[Note: see reference to fixed action pattern in Wikipedia article on ethology, above].
What is a taxis?
From Wikipedia, the free encyclopedia [http://en.wikipedia.org/wiki/Taxis]
“A taxis (plural taxes, pronounced "takseez") is an innate behavioral response by an organism (or cell) to a directional stimulus (a stimulus from a particular direction) whereby an organism moves (orientation movement) either towards (positive taxis) or away from (negative taxis) the stimulus. A taxis differs from a turning response or tropism in that the organism has mobility and demonstrates guided movement to or away from the stimulus (Kendeigh, 1961). For example, flagellate protozoans of the Genus Euglena move towards a light source. Here the directional stimulus is light, and the orientation movement is towards the light. This reaction or behaviour is a positive one to light and specifically termed a positive phototaxis. Phototaxis, then, is a response to a light stimulus. Many other types of taxes that have been identified, and named using prefixes to specify the stimulus eliciting a response:
* Anemotaxis, stimulated by wind
* Barotaxis, stimulated by pressure
* Chemotaxis, stimulated by chemicals
* Galvanotaxis, stimulated by current (electricity)
* Geotaxis, stimulated by gravity
* Hydrotaxis, stimulated by moisture
* Phototaxis, stimulated by light
* Rheotaxis, stimulated by current (flow)
* Thermotaxis, stimulated by temperature changes
* Thigmotaxis, stimulated by contact (touch)
Depending on the type of sensory organs present, taxes can be classified as klinotaxes, where an organism continuously samples the environment to determine the direction of a stimulus, tropotaxes, where bilateral sense organs are utilized to determine the stimulus direction, and telotaxes, similar to tropotaxes but a single organ suffices to establish the orientation movement.”
Behavior acquisition (Parker, 9/5, p.19):
“Lorenz and Tinbergen identified four ways to acquire a behavior:
–Innate:
•Born with it
•Simple, effective, computationally inexpensive
–Sequence of innate behaviors:
•Born with a sequence of behaviors
•Several steps follow one after the other
–Innate with memory:
•Born with behaviors that need initialization, customization
•Agent has to learn parameters of behaviors
–Learned:
•Can be complex behaviors
•Learn the releasing mechanism and the actions, and how to put them together”
Robotic behavior acquisition:
–Innate:
• Programmed via hardware, software or combination of
–Learned:
• Learning process is itself a behavior
• Learning process is innate
• An example of machine learning
Behavioral triggers:
Releasing mechanism – Neil Greenberg [https://notes.utk.edu/bio/greenberg.nsf/bbac4e3d4d2ddbf285256d79006e5465/41ad26eddefc9ae285256e0000564bbf?OpenDocument]
a functionally organized, neural circuit that recognizes a specific sign stimulus and produces the appropriate response. previously the RM was called the "Innate releasing mechanism", see for discussion of "Innate" See Stimulus.
Innate releasing mechanism – Parker, p. 20
• IRM: a combination of stimuli that elicit a specific, perhaps complex, response to a
particular biological situation
• Releaser:
– Similar to a latch or Boolean variable that has to be set by a stimulus
– Acts as a control signal to activate a behavior
Innate releasing mechanism – reference.com
[http://www.reference.com/browse/wiki/Instinct]
“Technically speaking, any event that initiates an instinctive behavior is termed a key stimulus (KS). Key stimuli in turn lead to innate releasing mechanisms (IRM), which in turn produce fixed action patterns (FAP). More than one key stimuli may be needed to trigger a FAP. Sensory receptor cells are critical in determining the type of FAP which is initiated. For instance, the reception of pheromones through nasal sensory receptor cells may trigger a sexual response, while the reception of a "frightening sound" through auditory sensory receptor cells may trigger a fight or flight response. The neural networks of these different sensory cells assist in integrating the signal from many receptors to determine the degree of the KS and therefore produce an appropriate degree of response. Several of these responses are determined by carefully regulated chemical messengers called hormones. The endocrine system, which is responsible for the production and transport of hormones throughout the body, is made up of many secretory glands that produce hormones and release them for transport to target organs. Specifically in vertebrates, neural control of this system is funneled through the hypothalamus to the anterior and posterior pituitary gland. Whether or not the behavioral response to a given key stimuli is either learned, genetic, or both is the center of study in the field of behavioral genetics. Researchers use techniques such as inbreeding and knockout studies to separate learning and environment from genetic determination of behavioral traits. And humans as a matter of speaking have no instincts past the early stages of infancy. Instinct should not be confused with responses that an organism is born with such as breathing, hunger, sex drive etc. These are no different than sight, aural ability, tactility or taste perception.”
Implementation of behaviors:
Coding examples:
Parker – pp. 21-23
Interaction of concurrent behaviors: Parker – p. 24
- Equilibrium
- Dominance
- Cancellation
Role of perception: Parker – p. 25
- Release
- Inform
Mutual interaction: Parker – p. 25
Execution of behavior triggers filtering mechanism on information stream
- Information yields potential for initiation of behavior [affordance]
Human perception: Parker – p. 26
Hypothesis: Two kinds of Perceptual Systems in the Brain
•Speculated by Neisser
•Direct perception:
–Doesn’t require memory, inference, or interpretation
–Minimal computation
–Rapid execution time
–Example: Optic flow
•Recognition:
–Ties in problem solving and other cognitive activities
–Deals with more “top-down”, “model-based” perception
–Example: find your car in the parking lot
Schema theory: Parker – p. 27
Schemas & bahaviors: Parker – p. 28
Execution of behavior triggers filtering mechanism on information stream
- Information yields potential for initiation of behavior [affordance]
Correlation – schemas & ethology/cogn. psch.: Parker – p. 32
[Note: schemas are problematic].
Open issues: Parker – p. 33
- Conflicting concurrent behaviors
- Explicit knowledge representation [Note: see Brooks/Kirsh debate]
- Learning behaviors
Brooks/Kirsh debate:
- Elephants don’t play chess http://people.csail.mit.edu/brooks/papers/elephants.pdf
- Today the Earwig, Tomorrow Man http://interactivity.ucsd.edu/articles/Earwig/earwig-cleaned.html
http://hci.ucsd.edu/102a/05-lectures/earwig.pdf
- From Earwigs to Humans http://people.csail.mit.edu/brooks/papers/ascona.pdf