CIS 2050 Lecture Notes - Lecture 4: Affective Computing, Sensor Fusion, Human Computer
Differentiate between intelligent and routine techniques in computing
1.
Describe and characterize the approaches of intelligent computing
using Human-Computer Cooperation, Affective Computing and Bio-
inspired Design
2.
Explain the differences in emphasis between the approaches of
intelligent computing using Human-Computer Cooperation, Affecting
Computing and Bio-inspired Design to intelligent computing
3.
Identify applications in Affective Computing, Human-Computing
Cooperation and Bio-inspired design
4.
Learning Outcomes:
Key question in building an "intelligent machine" is to ask: what do
we mean by "intelligent technology"
!
Even if it can, is what a computer conveys real human
emotions, or just human emotion-like?
○
Or is it just an emotion experienced by the human viewer?
○
Can a computer convey emotions like humans?
!
Can imitating nature help in building an intelligent machine?
○
What other designs work towards building an "intelligent machine"?
!
E.g. combining human intelligence with the power of creative
algorithms in cooperation
○
What about other approaches to "intelligent machines"?
!
Is an "intelligent machine" conscious?
!
Key Points:
Systems/programs that determine a user's mood/emotion
through visual, auditory, physiological, or behavioural cues
○
These cues make up the users "affect"
○
Programs aim to study a combination of short term emotions
and long term moods
○
Affective Computing:
!
600BC, Sun Tzu refers to the "art of studying moods" in The
Art of War
○
Found that "emotions are physiological and universal in
nature"
!
Sir Charles Bell's The Anatomy and Philosophy of Expression
(1824) and Charles Darwin's The Expression of Emotions in
Man and Animals (1872) used empirical studies to connect
emotions and physical responses
○
1884: William James' "What is an Emotion?" found "emotions
have distinct bodily expressions (affects) which can be
categorized, measured and analyzed
○
1978: Paul Ekman's Facial Action Coding System (FACS)
provided a bases for analyzing affect in facial expression
○
1995: Rosalind Picard's Affective Computing was the firest
paper on the value of affective computing systems
○
History:
!
Used by individuals with autism who have difficulty
expressing their stress before having a meltdown
!
Q sensor analyzes electrodermal activity and uses Q live
software to analyze the data and notify the user when
they are becoming overly stressed
!
Non-visual sensors such as the Q sensor
○
Beyond Verbal (an Israel-based company) extrapolates
emotional content in speech with automated voice
analysis and can help with decision making
!
Verbal/Auditory sensor recognizes emotions in the tone of a
voice
○
First uses Action Units (AUs) which are specific, easy
identifiable point s on the face such as mouth or eyes in
Active Appearance Models (AAM) and Constrained
Local Models (CLM) to identify emotion
!
Second uses "low level vision" features such as Gabor
Filters (scans face in a single plane) or Local Binary
Patterns (LBPs) to track general shapes and movement
!
Used in the most advanced FACs
"
The most advanced Affective Computing currently uses
Computer Expression Recognition Toolbox (CERT)
which uses AUs as well as multiple Gabor filters in
different orientations to map the face in 3D and monitor
specific facial features
!
Facial Recognition software as two main methods of operation
○
Types of Affective Computing:
!
Massive amounts of data is required to make the software
effective
○
Beyond Verbal "crowdsources" data in surveys from user's who
try their software demos
○
Another company, Emotion Sense, uses an app to collect mass
data from users
○
Development:
!
Potential for misuse of recorded data, info could be leaked or
used without consent
○
Multi sensor interface (sensor fusion) could lead to secure
information being sent between devices and therefore third-
parties without the users knowledge
○
Concerns:
!
Share emotion data from young students with ADHD or autism
to teachers/parents
○
Private person notifications of mood when in public settings
○
Marketing campaigns get genuine, real-time reactions to ads
○
Educators can see when students are confused or lose focus in
lectures
○
Cars can tell when driver gets distracted or dangerously tired
○
Applications:
!
Read: Communications of the ACM
This field is about making robots that are inspired by biological
systems
○
Biomimicry is copying nature while bio-inspired design
is learning from nature and making a mechanism that is
simpler and more effective than the system observed in
nature
!
The biological systems have been optimized for
specific tasks according to their habitat
"
However, they are multifunctional and are not
designed for only one specific functionality
"
Biomimicry has led to the development of a different
branch of robotics called soft robotics
!
Biomimicry and bio-inspired design are sometimes confused
○
Designer should then try to simplify and enhance that
mechnism for the specific task of interest
!
Bio-inspired robotics is about studying biological systems, and
look for the mechanisms that may solve a problem in the
engineering field
○
Most of the robots have some type of locomotion system
!
Bio-inspired roboticists are usually interested in biosensors,
bioactuators, or biomaterials
○
Bio-inspired robotic locomotion is about learning concepts from
nature and applying them to the design of real-world engineered
systems
!
May include terrestrial locomotion and arboreal
locomotion
!
iSprawl = hexapedal robot inspired by the
cockroach
"
Legged locomotion -can move on an uneven
environment more effectively
!
Passive/active wheels
"
Active treads
"
Undulating robots using vertical waves or linear
expansions
"
Robots can actively control their scales
to modify their frictional properties
and move on a variety of surfaces
efficiently
◊
Scalybot focuses on the role of snake ventral
scaled on adjusting the frictional properties
in different directions
!
Modular robots can mimic several snake gaits
"
Limbless locomotion -snake like robotics
!
Most robots have been built around a single
functionality
"
Geckobots use van der waals forces that only work
on smooth surfaces
"
Stickybots use directional dry adhesives that work
best on smooth surface
"
Spinybot and the RiSE robot are more insect-like
and use spines
"
Cannot handle large obstacles -are not
flexible and require a wide space for moving
!
Usually cannot climb both smooth and rough
sufraces or handle vertical to horizontal
transitions
!
Legged climbing robots have several limitations
"
Climbing
!
It uses torsion springs as energy storage and
includes a wire and latch mechanism to
compress and release the springs
!
A jumping robot inspired by a locust has been
developed, with the jump event induced by
releasing the tension of a spring
"
Thermal gas expansion inside the soft
combusting chamber drastically increases the
chamber volume
!
The sot robot inspired by a roly-poly toy then
reorients itself into an upright position after
landing
!
ETH Zurich has reported a soft jumping robot
based on the combusting of methane and laughing
gas
"
Jumping
!
On a surface
○
Can accelerate and maneuver far better than
any boat/submarine, and produce less noise
and water disturbance
!
Fish can achieve a propulsive efficiency >90%
"
Essex University Computer Science Robotic Fish
G9 and Robot Tuna analyze and mathematically
model thunniform motion
"
The Aqua Penguin copies the stream-line shape and
propulsion by the front flippers
"
Aqua Ray and Aqua Jelly emulate the locomotion
of these species
"
iSplash-II was first robotic fish capable of
outperforming real carangiform fish in terms of
average maximum velocity and endurance
"
Swimming or flying robotics (piscine)
!
In a fluid
○
Biolocomotion:
!
Typically capable of performing several tasks and are
specifically useful for search and rescue or exploratory
missions
!
Salamander inspired robot that can walk and swim
"
Snake inspired robot that has 4 different modes of
terrestrial locomotion
"
Cockroach inspired robot that can run and climb
"
Examples:
!
Modular
○
Robots that look human-like or are inspired by the human
form
!
Personal assistance
"
Reception
"
Work at industries
"
Companionship
"
Applications:
!
Used for research and were originally developed to build
better orthosis and prosthesis for human beings
!
Petman -one of the first and most advanced
"
Honda Asimo -over actuated
"
*also some that do not have any actuators and walk
passively descending a shallow slope
"
Examples:
!
Humanoid
○
Goal is to make robots that can work together and
transfer data, make structures as a group…etc
!
Swarming
○
Robots composed of entirely soft material and moved
through pneumatic pressure (similar to octopus or
starfish)
!
They are flexible enough to move in very limited spaces
(such as the human body)
!
First multigait soft robots were developed in 2011
!
First fully integrated, independent soft robot was
developed in 2015
!
Soft
○
Morphological Classification
!
Read: Bio-inspired Robotics
Weak human + machine + better process > strong human + machine
+ inferior process
!
More about the power and efficiency of communciation
between human and computer than it is about the brute
strength of humans and computers along
!
Human-Computer Cooperation (HCC) is the concept of humans
working together with computer systems to solve problems that
each could not solve on their own
○
All about the right type of cooperation
!
He determined that it is better to augment human skills with
computers than it is to make computers more human-like
○
Joseph Carl Robnett Licklider coined the terms "human-computer
symbiosis" and "intelligence augemtation"
!
Enable man and machine to cooperate
○
Without dependence on pre-determined technology
○
--> intelligence augmentation (IA)
!
Humans can focus on design
○
Human-computer symbiosis makes us more capable
○
Computers do work required for insights
!
Human-Computer Symbiosis= Gestalt of (Human &
Computer)/Friction
○
Analytic Capability = Human + Computer + Human-Computer
Symbiosis
!
*need to minimize friction between computer and humans
○
Better process --> less friction
!
The best results come from the most efficient human-computer
interface
○
HCC works based on computers' exceptional computing and
processing speed/power and human creativity/decision making
!
Watch: Rise of human-computer cooperation
Ideally, human's want robots to be less robotic (more fluid and life-
like)
!
It should be more like a duet, where both members are working
together on the same task at the same time
○
Human-robot interaction is like a chess game, human makes a 'move'
and then waits for computer to analyze and respond with another
'move'
!
The 'traditional' robot would take a command from the human
and carry out the exact task
○
The 'adventurous' robot tried to anticipate ways to help humans,
and even though it made more mistakes, humans liked
interacting with it better
○
People even referred to the adventurous robot as 'he/she' but the
traditional robot as 'it'
○
A designer made a robot that could be programmed as a 'traditional'
robot (receives orders and carrier out a specific task repeatedly) and
as an adventurous/risk taking robot
!
This was a big step toward a more human-like robot, because
improvisation cant' be pre-planned or calculated
○
The robot was also given a head with a camera, so it could
express what is was playing/listening to with 'body language',
as well as play an improvised duet with a human by watching
the human with the camera
○
The designer created another robot that could improvise music
!
Using this programming, the designer made a speaker dock that plays
music from your phone, as well as moves/dances to the music to
make it seem more life-like
!
Using this technology in other applications will hopefully make
human-robot interactions less like a chess game and more like playing
music
!
Watch: Robots with "soul"
One of the biggest differences between natural and human
technologies is robustness
!
Cockroach's flexible legs allow them to self stabilize on
uneven ground, can perform multiple tasks with the same
bodily structure
!
Stable in new and complex environments
○
Cockroach feet have barbs/hooks and sticky pads ont heir
feet for grips while moving and can still walk/climb if
they lose their feet/legs
!
Fault tolerant and failsafe
○
Cockroach exoskeleton is made of strong/rigid plates and
tubes connect with flexible/compliant membranes which
allows them to compress their body by 40% and
withstand 800x their body weight in compressive force
!
Damage Resistant
○
Robust systems are:
!
Team at Berkeley has created hexapod robots made of very
compliant/flexible materials, which allows it to mimic a lot of the
basic movements of a cockroach
!
Ex. For search and rescue efforts
○
By mimicking successful technologies in nature (biomimicry), we can
greatly improve human-made technologies
!
Watch: Secrets of nature's grossest creatures, channeled into robots
Bio-inspired robotics in the process of studying mechanisms in
nature or natural things, and from the understanding, adapting
to engineering design to solve practical problems
○
It is arguably "intelligent", because it provides many clever and
novel solutions that are efficient, and often effective in many
engineering tasks
○
Ex. Better locomotion robotics, prosthetics, and other
biomaterials (like muscle)
○
What is bio-inspired robotics? In what ways is it intelligent?1.
Affective computing is the study and development of
computing that simulates the emotional construct of human
affects
○
The emotion is usually not considered as real in the sense that
the emotion is consciously felt by the system
○
It is nevertheless often interpreted as such by the human
participants and help understanding inducing human emotions
in artifacts, and machines
○
Affective computing is found to be important in some clinical
studies (such as autism)
○
What is affective computing and what is its significance to Artificial
Intelligence?
2.
Practice Questions:
Intelligence Computing in Technology
#$%&'()*+, -./&%)&*, 0+,1203
4514,67
Differentiate between intelligent and routine techniques in computing1.
Describe and characterize the approaches of intelligent computing
using Human-Computer Cooperation, Affective Computing and Bio-
inspired Design
2.
Explain the differences in emphasis between the approaches of
intelligent computing using Human-Computer Cooperation, Affecting
Computing and Bio-inspired Design to intelligent computing
3.
Identify applications in Affective Computing, Human-Computing
Cooperation and Bio-inspired design
4.
Learning Outcomes:
Key question in building an "intelligent machine" is to ask: what do
we mean by "intelligent technology"
!
Even if it can, is what a computer conveys real human
emotions, or just human emotion-like?
○
Or is it just an emotion experienced by the human viewer?
○
Can a computer convey emotions like humans?
!
Can imitating nature help in building an intelligent machine?
○
What other designs work towards building an "intelligent machine"?
!
E.g. combining human intelligence with the power of creative
algorithms in cooperation
○
What about other approaches to "intelligent machines"?
!
Is an "intelligent machine" conscious?
!
Key Points:
Systems/programs that determine a user's mood/emotion
through visual, auditory, physiological, or behavioural cues
○
These cues make up the users "affect"
○
Programs aim to study a combination of short term emotions
and long term moods
○
Affective Computing:
!
600BC, Sun Tzu refers to the "art of studying moods" in The
Art of War
○
Found that "emotions are physiological and universal in
nature"
!
Sir Charles Bell's The Anatomy and Philosophy of Expression
(1824) and Charles Darwin's The Expression of Emotions in
Man and Animals (1872) used empirical studies to connect
emotions and physical responses
○
1884: William James' "What is an Emotion?" found "emotions
have distinct bodily expressions (affects) which can be
categorized, measured and analyzed
○
1978: Paul Ekman's Facial Action Coding System (FACS)
provided a bases for analyzing affect in facial expression
○
1995: Rosalind Picard's Affective Computing was the firest
paper on the value of affective computing systems
○
History:
!
Used by individuals with autism who have difficulty
expressing their stress before having a meltdown
!
Q sensor analyzes electrodermal activity and uses Q live
software to analyze the data and notify the user when
they are becoming overly stressed
!
Non-visual sensors such as the Q sensor
○
Beyond Verbal (an Israel-based company) extrapolates
emotional content in speech with automated voice
analysis and can help with decision making
!
Verbal/Auditory sensor recognizes emotions in the tone of a
voice
○
First uses Action Units (AUs) which are specific, easy
identifiable point s on the face such as mouth or eyes in
Active Appearance Models (AAM) and Constrained
Local Models (CLM) to identify emotion
!
Second uses "low level vision" features such as Gabor
Filters (scans face in a single plane) or Local Binary
Patterns (LBPs) to track general shapes and movement
!
Used in the most advanced FACs
"
The most advanced Affective Computing currently uses
Computer Expression Recognition Toolbox (CERT)
which uses AUs as well as multiple Gabor filters in
different orientations to map the face in 3D and monitor
specific facial features
!
Facial Recognition software as two main methods of operation
○
Types of Affective Computing:
!
Massive amounts of data is required to make the software
effective
○
Beyond Verbal "crowdsources" data in surveys from user's who
try their software demos
○
Another company, Emotion Sense, uses an app to collect mass
data from users
○
Development:
!
Potential for misuse of recorded data, info could be leaked or
used without consent
○
Multi sensor interface (sensor fusion) could lead to secure
information being sent between devices and therefore third-
parties without the users knowledge
○
Concerns:
!
Share emotion data from young students with ADHD or autism
to teachers/parents
○
Private person notifications of mood when in public settings
○
Marketing campaigns get genuine, real-time reactions to ads
○
Educators can see when students are confused or lose focus in
lectures
○
Cars can tell when driver gets distracted or dangerously tired
○
Applications:
!
Read: Communications of the ACM
This field is about making robots that are inspired by biological
systems
○
Biomimicry is copying nature while bio-inspired design
is learning from nature and making a mechanism that is
simpler and more effective than the system observed in
nature
!
The biological systems have been optimized for
specific tasks according to their habitat
"
However, they are multifunctional and are not
designed for only one specific functionality
"
Biomimicry has led to the development of a different
branch of robotics called soft robotics
!
Biomimicry and bio-inspired design are sometimes confused
○
Designer should then try to simplify and enhance that
mechnism for the specific task of interest
!
Bio-inspired robotics is about studying biological systems, and
look for the mechanisms that may solve a problem in the
engineering field
○
Most of the robots have some type of locomotion system
!
Bio-inspired roboticists are usually interested in biosensors,
bioactuators, or biomaterials
○
Bio-inspired robotic locomotion is about learning concepts from
nature and applying them to the design of real-world engineered
systems
!
May include terrestrial locomotion and arboreal
locomotion
!
iSprawl = hexapedal robot inspired by the
cockroach
"
Legged locomotion -can move on an uneven
environment more effectively
!
Passive/active wheels
"
Active treads
"
Undulating robots using vertical waves or linear
expansions
"
Robots can actively control their scales
to modify their frictional properties
and move on a variety of surfaces
efficiently
◊
Scalybot focuses on the role of snake ventral
scaled on adjusting the frictional properties
in different directions
!
Modular robots can mimic several snake gaits
"
Limbless locomotion -snake like robotics
!
Most robots have been built around a single
functionality
"
Geckobots use van der waals forces that only work
on smooth surfaces
"
Stickybots use directional dry adhesives that work
best on smooth surface
"
Spinybot and the RiSE robot are more insect-like
and use spines
"
Cannot handle large obstacles -are not
flexible and require a wide space for moving
!
Usually cannot climb both smooth and rough
sufraces or handle vertical to horizontal
transitions
!
Legged climbing robots have several limitations
"
Climbing
!
It uses torsion springs as energy storage and
includes a wire and latch mechanism to
compress and release the springs
!
A jumping robot inspired by a locust has been
developed, with the jump event induced by
releasing the tension of a spring
"
Thermal gas expansion inside the soft
combusting chamber drastically increases the
chamber volume
!
The sot robot inspired by a roly-poly toy then
reorients itself into an upright position after
landing
!
ETH Zurich has reported a soft jumping robot
based on the combusting of methane and laughing
gas
"
Jumping
!
On a surface
○
Can accelerate and maneuver far better than
any boat/submarine, and produce less noise
and water disturbance
!
Fish can achieve a propulsive efficiency >90%
"
Essex University Computer Science Robotic Fish
G9 and Robot Tuna analyze and mathematically
model thunniform motion
"
The Aqua Penguin copies the stream-line shape and
propulsion by the front flippers
"
Aqua Ray and Aqua Jelly emulate the locomotion
of these species
"
iSplash-II was first robotic fish capable of
outperforming real carangiform fish in terms of
average maximum velocity and endurance
"
Swimming or flying robotics (piscine)
!
In a fluid
○
Biolocomotion:
!
Typically capable of performing several tasks and are
specifically useful for search and rescue or exploratory
missions
!
Salamander inspired robot that can walk and swim
"
Snake inspired robot that has 4 different modes of
terrestrial locomotion
"
Cockroach inspired robot that can run and climb
"
Examples:
!
Modular
○
Robots that look human-like or are inspired by the human
form
!
Personal assistance
"
Reception
"
Work at industries
"
Companionship
"
Applications:
!
Used for research and were originally developed to build
better orthosis and prosthesis for human beings
!
Petman -one of the first and most advanced
"
Honda Asimo -over actuated
"
*also some that do not have any actuators and walk
passively descending a shallow slope
"
Examples:
!
Humanoid
○
Goal is to make robots that can work together and
transfer data, make structures as a group…etc
!
Swarming
○
Robots composed of entirely soft material and moved
through pneumatic pressure (similar to octopus or
starfish)
!
They are flexible enough to move in very limited spaces
(such as the human body)
!
First multigait soft robots were developed in 2011
!
First fully integrated, independent soft robot was
developed in 2015
!
Soft
○
Morphological Classification
!
Read: Bio-inspired Robotics
Weak human + machine + better process > strong human + machine
+ inferior process
!
More about the power and efficiency of communciation
between human and computer than it is about the brute
strength of humans and computers along
!
Human-Computer Cooperation (HCC) is the concept of humans
working together with computer systems to solve problems that
each could not solve on their own
○
All about the right type of cooperation
!
He determined that it is better to augment human skills with
computers than it is to make computers more human-like
○
Joseph Carl Robnett Licklider coined the terms "human-computer
symbiosis" and "intelligence augemtation"
!
Enable man and machine to cooperate
○
Without dependence on pre-determined technology
○
--> intelligence augmentation (IA)
!
Humans can focus on design
○
Human-computer symbiosis makes us more capable
○
Computers do work required for insights
!
Human-Computer Symbiosis= Gestalt of (Human &
Computer)/Friction
○
Analytic Capability = Human + Computer + Human-Computer
Symbiosis
!
*need to minimize friction between computer and humans
○
Better process --> less friction
!
The best results come from the most efficient human-computer
interface
○
HCC works based on computers' exceptional computing and
processing speed/power and human creativity/decision making
!
Watch: Rise of human-computer cooperation
Ideally, human's want robots to be less robotic (more fluid and life-
like)
!
It should be more like a duet, where both members are working
together on the same task at the same time
○
Human-robot interaction is like a chess game, human makes a 'move'
and then waits for computer to analyze and respond with another
'move'
!
The 'traditional' robot would take a command from the human
and carry out the exact task
○
The 'adventurous' robot tried to anticipate ways to help humans,
and even though it made more mistakes, humans liked
interacting with it better
○
People even referred to the adventurous robot as 'he/she' but the
traditional robot as 'it'
○
A designer made a robot that could be programmed as a 'traditional'
robot (receives orders and carrier out a specific task repeatedly) and
as an adventurous/risk taking robot
!
This was a big step toward a more human-like robot, because
improvisation cant' be pre-planned or calculated
○
The robot was also given a head with a camera, so it could
express what is was playing/listening to with 'body language',
as well as play an improvised duet with a human by watching
the human with the camera
○
The designer created another robot that could improvise music
!
Using this programming, the designer made a speaker dock that plays
music from your phone, as well as moves/dances to the music to
make it seem more life-like
!
Using this technology in other applications will hopefully make
human-robot interactions less like a chess game and more like playing
music
!
Watch: Robots with "soul"
One of the biggest differences between natural and human
technologies is robustness
!
Cockroach's flexible legs allow them to self stabilize on
uneven ground, can perform multiple tasks with the same
bodily structure
!
Stable in new and complex environments
○
Cockroach feet have barbs/hooks and sticky pads ont heir
feet for grips while moving and can still walk/climb if
they lose their feet/legs
!
Fault tolerant and failsafe
○
Cockroach exoskeleton is made of strong/rigid plates and
tubes connect with flexible/compliant membranes which
allows them to compress their body by 40% and
withstand 800x their body weight in compressive force
!
Damage Resistant
○
Robust systems are:
!
Team at Berkeley has created hexapod robots made of very
compliant/flexible materials, which allows it to mimic a lot of the
basic movements of a cockroach
!
Ex. For search and rescue efforts
○
By mimicking successful technologies in nature (biomimicry), we can
greatly improve human-made technologies
!
Watch: Secrets of nature's grossest creatures, channeled into robots
Bio-inspired robotics in the process of studying mechanisms in
nature or natural things, and from the understanding, adapting
to engineering design to solve practical problems
○
It is arguably "intelligent", because it provides many clever and
novel solutions that are efficient, and often effective in many
engineering tasks
○
Ex. Better locomotion robotics, prosthetics, and other
biomaterials (like muscle)
○
What is bio-inspired robotics? In what ways is it intelligent?1.
Affective computing is the study and development of
computing that simulates the emotional construct of human
affects
○
The emotion is usually not considered as real in the sense that
the emotion is consciously felt by the system
○
It is nevertheless often interpreted as such by the human
participants and help understanding inducing human emotions
in artifacts, and machines
○
Affective computing is found to be important in some clinical
studies (such as autism)
○
What is affective computing and what is its significance to Artificial
Intelligence?
2.
Practice Questions:
Intelligence Computing in Technology
#$%&'()*+, -./&%)&*, 0+,1203 4514,67
Differentiate between intelligent and routine techniques in computing1.
Describe and characterize the approaches of intelligent computing
using Human-Computer Cooperation, Affective Computing and Bio-
inspired Design
2.
Explain the differences in emphasis between the approaches of
intelligent computing using Human-Computer Cooperation, Affecting
Computing and Bio-inspired Design to intelligent computing
3.
Identify applications in Affective Computing, Human-Computing
Cooperation and Bio-inspired design
4.
Learning Outcomes:
Key question in building an "intelligent machine" is to ask: what do
we mean by "intelligent technology"
!
Even if it can, is what a computer conveys real human
emotions, or just human emotion-like?
○
Or is it just an emotion experienced by the human viewer?
○
Can a computer convey emotions like humans?
!
Can imitating nature help in building an intelligent machine?
○
What other designs work towards building an "intelligent machine"?
!
E.g. combining human intelligence with the power of creative
algorithms in cooperation
○
What about other approaches to "intelligent machines"?
!
Is an "intelligent machine" conscious?
!
Key Points:
Systems/programs that determine a user's mood/emotion
through visual, auditory, physiological, or behavioural cues
○
These cues make up the users "affect"
○
Programs aim to study a combination of short term emotions
and long term moods
○
Affective Computing:
!
600BC, Sun Tzu refers to the "art of studying moods" in The
Art of War
○
Found that "emotions are physiological and universal in
nature"
!
Sir Charles Bell's The Anatomy and Philosophy of Expression
(1824) and Charles Darwin's The Expression of Emotions in
Man and Animals (1872) used empirical studies to connect
emotions and physical responses
○
1884: William James' "What is an Emotion?" found "emotions
have distinct bodily expressions (affects) which can be
categorized, measured and analyzed
○
1978: Paul Ekman's Facial Action Coding System (FACS)
provided a bases for analyzing affect in facial expression
○
1995: Rosalind Picard's Affective Computing was the firest
paper on the value of affective computing systems
○
History:
!
Used by individuals with autism who have difficulty
expressing their stress before having a meltdown
!
Q sensor analyzes electrodermal activity and uses Q live
software to analyze the data and notify the user when
they are becoming overly stressed
!
Non-visual sensors such as the Q sensor
○
Beyond Verbal (an Israel-based company) extrapolates
emotional content in speech with automated voice
analysis and can help with decision making
!
Verbal/Auditory sensor recognizes emotions in the tone of a
voice
○
First uses Action Units (AUs) which are specific, easy
identifiable point s on the face such as mouth or eyes in
Active Appearance Models (AAM) and Constrained
Local Models (CLM) to identify emotion
!
Second uses "low level vision" features such as Gabor
Filters (scans face in a single plane) or Local Binary
Patterns (LBPs) to track general shapes and movement
!
Used in the most advanced FACs
"
The most advanced Affective Computing currently uses
Computer Expression Recognition Toolbox (CERT)
which uses AUs as well as multiple Gabor filters in
different orientations to map the face in 3D and monitor
specific facial features
!
Facial Recognition software as two main methods of operation
○
Types of Affective Computing:
!
Massive amounts of data is required to make the software
effective
○
Beyond Verbal "crowdsources" data in surveys from user's who
try their software demos
○
Another company, Emotion Sense, uses an app to collect mass
data from users
○
Development:
!
Potential for misuse of recorded data, info could be leaked or
used without consent
○
Multi sensor interface (sensor fusion) could lead to secure
information being sent between devices and therefore third-
parties without the users knowledge
○
Concerns:
!
Share emotion data from young students with ADHD or autism
to teachers/parents
○
Private person notifications of mood when in public settings
○
Marketing campaigns get genuine, real-time reactions to ads
○
Educators can see when students are confused or lose focus in
lectures
○
Cars can tell when driver gets distracted or dangerously tired
○
Applications:
!
Read: Communications of the ACM
This field is about making robots that are inspired by biological
systems
○
Biomimicry is copying nature while bio-inspired design
is learning from nature and making a mechanism that is
simpler and more effective than the system observed in
nature
!
The biological systems have been optimized for
specific tasks according to their habitat
"
However, they are multifunctional and are not
designed for only one specific functionality
"
Biomimicry has led to the development of a different
branch of robotics called soft robotics
!
Biomimicry and bio-inspired design are sometimes confused
○
Designer should then try to simplify and enhance that
mechnism for the specific task of interest
!
Bio-inspired robotics is about studying biological systems, and
look for the mechanisms that may solve a problem in the
engineering field
○
Most of the robots have some type of locomotion system
!
Bio-inspired roboticists are usually interested in biosensors,
bioactuators, or biomaterials
○
Bio-inspired robotic locomotion is about learning concepts from
nature and applying them to the design of real-world engineered
systems
!
May include terrestrial locomotion and arboreal
locomotion
!
iSprawl = hexapedal robot inspired by the
cockroach
"
Legged locomotion -can move on an uneven
environment more effectively
!
Passive/active wheels
"
Active treads
"
Undulating robots using vertical waves or linear
expansions
"
Robots can actively control their scales
to modify their frictional properties
and move on a variety of surfaces
efficiently
◊
Scalybot focuses on the role of snake ventral
scaled on adjusting the frictional properties
in different directions
!
Modular robots can mimic several snake gaits
"
Limbless locomotion -snake like robotics
!
Most robots have been built around a single
functionality
"
Geckobots use van der waals forces that only work
on smooth surfaces
"
Stickybots use directional dry adhesives that work
best on smooth surface
"
Spinybot and the RiSE robot are more insect-like
and use spines
"
Cannot handle large obstacles -are not
flexible and require a wide space for moving
!
Usually cannot climb both smooth and rough
sufraces or handle vertical to horizontal
transitions
!
Legged climbing robots have several limitations
"
Climbing
!
It uses torsion springs as energy storage and
includes a wire and latch mechanism to
compress and release the springs
!
A jumping robot inspired by a locust has been
developed, with the jump event induced by
releasing the tension of a spring
"
Thermal gas expansion inside the soft
combusting chamber drastically increases the
chamber volume
!
The sot robot inspired by a roly-poly toy then
reorients itself into an upright position after
landing
!
ETH Zurich has reported a soft jumping robot
based on the combusting of methane and laughing
gas
"
Jumping
!
On a surface
○
Can accelerate and maneuver far better than
any boat/submarine, and produce less noise
and water disturbance
!
Fish can achieve a propulsive efficiency >90%
"
Essex University Computer Science Robotic Fish
G9 and Robot Tuna analyze and mathematically
model thunniform motion
"
The Aqua Penguin copies the stream-line shape and
propulsion by the front flippers
"
Aqua Ray and Aqua Jelly emulate the locomotion
of these species
"
iSplash-II was first robotic fish capable of
outperforming real carangiform fish in terms of
average maximum velocity and endurance
"
Swimming or flying robotics (piscine)
!
In a fluid
○
Biolocomotion:
!
Typically capable of performing several tasks and are
specifically useful for search and rescue or exploratory
missions
!
Salamander inspired robot that can walk and swim
"
Snake inspired robot that has 4 different modes of
terrestrial locomotion
"
Cockroach inspired robot that can run and climb
"
Examples:
!
Modular
○
Robots that look human-like or are inspired by the human
form
!
Personal assistance
"
Reception
"
Work at industries
"
Companionship
"
Applications:
!
Used for research and were originally developed to build
better orthosis and prosthesis for human beings
!
Petman -one of the first and most advanced
"
Honda Asimo -over actuated
"
*also some that do not have any actuators and walk
passively descending a shallow slope
"
Examples:
!
Humanoid
○
Goal is to make robots that can work together and
transfer data, make structures as a group…etc
!
Swarming
○
Robots composed of entirely soft material and moved
through pneumatic pressure (similar to octopus or
starfish)
!
They are flexible enough to move in very limited spaces
(such as the human body)
!
First multigait soft robots were developed in 2011
!
First fully integrated, independent soft robot was
developed in 2015
!
Soft
○
Morphological Classification
!
Read: Bio-inspired Robotics
Weak human + machine + better process > strong human + machine
+ inferior process
!
More about the power and efficiency of communciation
between human and computer than it is about the brute
strength of humans and computers along
!
Human-Computer Cooperation (HCC) is the concept of humans
working together with computer systems to solve problems that
each could not solve on their own
○
All about the right type of cooperation
!
He determined that it is better to augment human skills with
computers than it is to make computers more human-like
○
Joseph Carl Robnett Licklider coined the terms "human-computer
symbiosis" and "intelligence augemtation"
!
Enable man and machine to cooperate
○
Without dependence on pre-determined technology
○
--> intelligence augmentation (IA)
!
Humans can focus on design
○
Human-computer symbiosis makes us more capable
○
Computers do work required for insights
!
Human-Computer Symbiosis= Gestalt of (Human &
Computer)/Friction
○
Analytic Capability = Human + Computer + Human-Computer
Symbiosis
!
*need to minimize friction between computer and humans
○
Better process --> less friction
!
The best results come from the most efficient human-computer
interface
○
HCC works based on computers' exceptional computing and
processing speed/power and human creativity/decision making
!
Watch: Rise of human-computer cooperation
Ideally, human's want robots to be less robotic (more fluid and life-
like)
!
It should be more like a duet, where both members are working
together on the same task at the same time
○
Human-robot interaction is like a chess game, human makes a 'move'
and then waits for computer to analyze and respond with another
'move'
!
The 'traditional' robot would take a command from the human
and carry out the exact task
○
The 'adventurous' robot tried to anticipate ways to help humans,
and even though it made more mistakes, humans liked
interacting with it better
○
People even referred to the adventurous robot as 'he/she' but the
traditional robot as 'it'
○
A designer made a robot that could be programmed as a 'traditional'
robot (receives orders and carrier out a specific task repeatedly) and
as an adventurous/risk taking robot
!
This was a big step toward a more human-like robot, because
improvisation cant' be pre-planned or calculated
○
The robot was also given a head with a camera, so it could
express what is was playing/listening to with 'body language',
as well as play an improvised duet with a human by watching
the human with the camera
○
The designer created another robot that could improvise music
!
Using this programming, the designer made a speaker dock that plays
music from your phone, as well as moves/dances to the music to
make it seem more life-like
!
Using this technology in other applications will hopefully make
human-robot interactions less like a chess game and more like playing
music
!
Watch: Robots with "soul"
One of the biggest differences between natural and human
technologies is robustness
!
Cockroach's flexible legs allow them to self stabilize on
uneven ground, can perform multiple tasks with the same
bodily structure
!
Stable in new and complex environments
○
Cockroach feet have barbs/hooks and sticky pads ont heir
feet for grips while moving and can still walk/climb if
they lose their feet/legs
!
Fault tolerant and failsafe
○
Cockroach exoskeleton is made of strong/rigid plates and
tubes connect with flexible/compliant membranes which
allows them to compress their body by 40% and
withstand 800x their body weight in compressive force
!
Damage Resistant
○
Robust systems are:
!
Team at Berkeley has created hexapod robots made of very
compliant/flexible materials, which allows it to mimic a lot of the
basic movements of a cockroach
!
Ex. For search and rescue efforts
○
By mimicking successful technologies in nature (biomimicry), we can
greatly improve human-made technologies
!
Watch: Secrets of nature's grossest creatures, channeled into robots
Bio-inspired robotics in the process of studying mechanisms in
nature or natural things, and from the understanding, adapting
to engineering design to solve practical problems
○
It is arguably "intelligent", because it provides many clever and
novel solutions that are efficient, and often effective in many
engineering tasks
○
Ex. Better locomotion robotics, prosthetics, and other
biomaterials (like muscle)
○
What is bio-inspired robotics? In what ways is it intelligent?1.
Affective computing is the study and development of
computing that simulates the emotional construct of human
affects
○
The emotion is usually not considered as real in the sense that
the emotion is consciously felt by the system
○
It is nevertheless often interpreted as such by the human
participants and help understanding inducing human emotions
in artifacts, and machines
○
Affective computing is found to be important in some clinical
studies (such as autism)
○
What is affective computing and what is its significance to Artificial
Intelligence?
2.
Practice Questions:
Intelligence Computing in Technology
#$%&'()*+, -./&%)&*, 0+,1203 4514,67