FP7 Project

Objective This tightly integrated focused project aims at fundamental breakthroughs in understanding and synthesising the mechanisms achieving cognition and language. It engages in carefully controlled experiments in which autonomous humanoid robots self-organise rich conceptual frameworks and communication systems with similar features as those found in human languages. Language and cognition are seen as complex adaptive systems that are continuously shaped by the actions of their users.

The project takes a 'whole systems' approach and tackles the complete chain from embodiment and sensori-motor action to conceptualisation and language. Concept formation and language invention and acquisition are embedded in situated interactions. The inventory of concepts, the strategies for grounding them in the world, and action co-evolve with the emergent artificial languages. Next to the required physical and cognitive capacities of each robotic agent, we also focus on the complex systems phenomena that appear when a group of such agents starts to interact in a distributed fashion.

The machinery required for these experiments will heavily push the state-of-the-art in all relevant technologies, particularly robotics, concept formation, computational linguistics, and A.I. We need rich sensors, actuators and robust, real-time performance of vision and motor control subsystems. We need sophisticated constraint-based conceptualisation of the world and effective parsing and production systems. Above all we need to discover how these systems can build up their competence autonomously and remain adaptive to cope with changing environments.

The project's orientation towards robotic experimentation is complemented by a search for an encompassing framework in which we seek to identify the principles underlying the evolution of human language-like cognition and to investigate in how far these principles are relevant to understand the most magnificent achievement of our own minds: language.

---

Objective ARAKNES stems from the innovative idea to transfer the technologies of bi-manual laparoscopic surgery to the endoluminal surgical approach, thus further reducing the operative trauma and enhancing the therapeutic outcome of minimally invasive surgical procedures for morbid obesity and gastro-esophageal reflux. The worldwide number of bypass surgeries for obesity is predicted to rise up to 1.3 million by 2015, hence the social and industrial impact of ARAKNES is considerable. In particular, European health-systems can benefit from major reductions in costs and hospitalisation periods, the effects of which translate directly to improvements in the quality of life for many EU citizens.
Through SandT excellence in micro-robotics and micro-system technologies, ARAKNES will facilitate the combination of current state-of-the-art and breakthrough innovations focusing on integrated micro-nano-bio-info devices. Specifically ARAKNESS will exploit the convergence of:
- established laparoscopic techniques;
- over 20 years of clinical experience with robotic and computer assisted surgery;
- the maturity of micro-, nano- and bio-technologies;
- the trend towards wireless, wearable and swallowable devices.
This combination enables the conception of a comprehensive micro-robotic based smart operating system for advanced endoluminal surgery, which will reform established endoluminal techniques.
The objectives and impact of ARAKNES are broad and ambitious. Consequently, they require a 4-year European project in the format of an IP managed by a strong and committed Consortium. The ARAKNES Consortium is a unique blend of European pioneers in all the involved disciplines (surgery, robotics, information technologies, micro- and nano-technologies) which have the vision, the commitment and the capabilities to make ARAKNES successful in terms of scientific innovation, industrial demonstrations and, ultimately, clinical application.

---

Objective BIOTACT will develop novel biomimetic computational methods and technologies for active touch sensing. The project is inspired by the functionality, morphology, and neural processing architectures of biological sensory systems that use tactile hairs known as whiskers or vibrissae. It will focus on two animals that are tactile specialists--the Norwegian rat and the Etruscan shrew. Both species make actively-controlled sweeping movements of their facial whiskers ("whisking") as they explore their environments. Specific objectives are to understand, and then 'reverse-engineer' in the form of computational models, the ability of these animals to:
i. generate fast rhythmic whisking behaviour, exquisitely modulated by sensory feedback, that allows precise positioning of the vibrissae in a manner that maximises information uptake;
ii. accurately determine object properties such as position, shape and texture, encode tactile memories that can support recognition of familiar items and places, and make rapid and accurate decisions about object identity and spatial location;
iii. determine the parameters of movement, shape, and position of detected prey animals that allow their tracking and capture using tactile sense data alone.
The achievement of these goals will be demonstrated and extended through the development of two biomimetic artefacts:
i. a novel active tactile sensing array--termed a BIOTACT sensor--with many hundreds of vibrissa-like sensing elements, that can fulfil a wide variety of sensing functions well beyond the capacity of existing tactile sensory devices.
ii. an autonomous whiskered robot with the ability to seek-out, identify, track, and capture fast-moving target objects.
Further objectives are to foster and accelerate research within the wider community of active touch researchers and to train young scientists in the multi-disciplinary skills needed to ensure rapid future progress in this field.

---

Objective Versatile and robust systems able to respond sensibly to challenges not precisely specified in their design, as envisaged in Challenge 2 target outcome (a), depend on (amongst other competences) effective perception systems able to function in complex and variable conditions. The principal objective of the proposed project is to discover how to engineer such systems, specifically embodied active sonar perception systems which can serve as a complement to vision and facilitate the deployment of robotic systems in situations where vision is infeasible.

For understanding the implementation of sonar systems able to function in demanding and open-ended environments, the Chiroptera (bats) are an obvious source of ideas. Their astounding diversity of diet and habitat attests to their success in integrating morphological, acoustic and behavioural parameters to enable robust and versatile hunting behaviours --- the bat equivalent of tangible object handling.

The project will implement and evaluate two demonstration sensors built as biomimetic models of an insect gleaning and a water-trawling bat species respectively. It will use a classic biomimetic methodology, involving close collaboration between bat ethologists and roboticists. It will proceed by identifying and measuring the relevant acoustic and morphological parameters of a few carefully selected bat species, reconstructing from that the bat's acoustic
experience as it flies through natural hunting tasks. From this data, computational models of how the bat coordinates its acoustic, behavioural and morphological choices during hunting will be elicited and implemented on appropriate robotic systems. The robustness, versatility and generality of these implemented models will be evaluated from an engineering standpoint as example embodied active
sonar perception systems, using tasks analogous to the hunting tasks of their living prototypes; evaluation from a biological standpoint will also be carried out.

---

Objective The challenge laid out in this call is to understand the principles according to which cognitive
systems should be built if they are to handle novelty, situations unforeseen by their designers, and
open-ended, challenging environments with uncertainty and change. Our aim is to meet this challenge by creating a theory -- evaluated in robots -- of how a cognitive system can model its own knowledge; use this to cope with uncertainty and novelty during task execution; extend its own abilities and knowledge; and extend its own understanding of those abilities. Imagine a cognitive system that models not only the environment, but its own understanding of the environment and how this understanding changes under action. It identifies gaps in its own understanding and then plans how to fill those gaps so as to deal with novelty and uncertainty in task execution, gather information necessary to complete its tasks, and to extend its abilities and knowledge so as to perform future tasks more efficiently.

---

Objective CHRIS will address the fundamental issues which would enable safe Human Robot Interaction (HRI). Specifically this project addresses the problem of a human and a robot performing co-operative tasks in a co-located space, such as in the kitchen where your service robot stirs the soup as you add the cream. These issues include communication of a shared goal (verbally and through gesture), perception and understanding of intention (from dextrous and gross movements), cognition necessary for interaction, and active and passive compliance. These are the prerequisites for many applications in service robotics and through research will provide the scientific foundations for engineering cognitive systems. The project is based on the essential premise that it will be ultimately beneficial to our socio-economic welfare to generate service robots capable of safe co-operative physical interaction with humans. The key hypothesis is that safe interaction between human and robot can be engineered physically and cognitively for joint physical tasks requiring co-operative manipulation of real world objects. A diverse set of disciplines have been brought together to realise an inter-disciplinary solution. The starting point for understanding cooperative cognition will be from the basic building blocks of initial interactions, those of young children. Engineering principles of safe movement and dexterity will be explored on the 3 available robot platforms, and developed with principles of language, communication and decisional action planning where the robot reasons explicitly with its human partner. Integration of cognition for safe co-operation in the same physical space will provide significant advancement in the area, and a step towards service robots in society.

---

Objective The DEXMART project is focused on artificial systems reproducing smart sensory-motor human skills, which operate in unstructured real-world environments. The emphasis is on manipulation capabilities achieved by dexterous and autonomous, and also human aware dual-arm/hand robotic systems. The goal is to allow a dual-arm robot including two multi-fingered redundant hands to grasp and manipulate the same objects used by human beings. The objects shall be allowed to have different shape, dimension and weight. The manipulation will take place in an unsupervised, robust and dependable manner so as to allow the robot to safely cooperate with humans for the execution of given tasks. The robotic system has to possess the ability to autonomously decide between different manipulation options. It has to properly and quickly react to unexpected situations and events as well as understand changes in the behaviour of humans cooperating with it. Moreover, in order to act in a changing scenario, the robot should be able to acquire knowledge by learning new action sequences so as to create a consistent and comprehensive manipulation knowledge base through an actual reasoning process. The possibility to exploit the high power-to-weight ratio of smart materials and structures will be explored aimed at the design of new hand components (finger, thumb, wrist) and sensors that will pave the way for the next generation of dexterous robotic hands.

---

Objective The research intends to investigate the interplay existing between vision and motion control, and to study how to exploit this interaction to achieve a knowledge of the surrounding environment that allows a robot to act properly. Robot perception can be flexibly integrated with its own actions and the understanding of planned actions of humans in a shared workspace. The research relies upon the assumption that a complete and operative cognition of visual space can be achieved only through active exploration of it: the natural effectors of this cognition are the eyes and the arms.
Crucial but yet unsolved issues we address are object recognition, dynamic shifts of attention, 3D space perception including eye and arm movements including action selection in unstructured environments. We propose a flexible solution based on the concept of visual fragments, which avoids a central representation of the environment and rather uses specialized components that interact with each other and tune themselves on the task at hand.
In addition to a high standard in engineering solutions the development and application of novel learning rules enables our system to acquire the necessary information directly from the environment.
The study and models of human/primate behavior, based on specific experiments, guide many of our envisaged solutions.
Three main objectives will be addressed:
- a robotic system for interactive visual stereopsis {composed of: an anthropomorphic mechatronic binocular system; and software vision modules based on cortical-like population, to be used as an experimental platform}.
- a model of a multisensory egocentric representation of the 3D space {constructed on binocular visual cues, signals from the oculomotor systems, signals about reaching movements performed by the arm}.
- a model of human-robot cooperative actions in a shared workspace {relaying on the concept of shared attention to understand the intention or goal of the communicating partner}.

---

Objective Compared to humans or primates, the ability of contemporary robotic hands is suprisingly limited, they can grasp only few objects in constricted poses with limited grasping postures and position. The aim of GRASP is the design of a cognitive system capable of performing tasks in open-ended environments, dealing with uncertainty and novel situations. We have decided to study the problem of object manipulation and grasping, by providing theoretical and measurable basis for system design that are valid in both human and artificial systems. We believe that this is of utmost importance for the design of artificial cognitive systems that are to be deployed in real environments and interact with humans and other artificial agents. Such systems need the ability to exploit the innate knowledge and self-understanding to gradually develop cognitive capabilities. To demonstrate the feasibility of our approach, we will instantiate, implement and evaluate our theories on robot systems with different emobodiments and levels of complexity. These systems will operate in real-world scenarios, with and without human intervention and tutoring.

GRASP will develop means for robotic systems to reason about graspable targets, to explore and investigate their physical properties and finally to make artificial hands grasp any object. We will use theoretical, computational and experimental studies to model skilled sensorimotor behavior based on known principles governing grasping and manipulation tasks performed by humans. Therefore, GRASP sets out to integrate a large body of findings from disciplines such as neuroscience, cognitive science, robotics, multi-modal perception and machine learning to achieve a core capability: Grasping any object by building up relations between task setting, embodied hand actions, object attributes, and contextual knowledge.

---

Objective The ITALK project aims to develop artificial embodied agents able to acquire complex behavioural, cognitive, and linguistic skills through individual and social learning. This will be achieved through the development of cognitive robotic agents, such as the iCub humanoid platform, that learn to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions.
The main theoretical hypothesis behind the project is that the parallel development of action, conceptualisation and social interaction permits the bootstrapping of language capabilities, which on their part enhance cognitive development. This is possible through the integration and transfer of knowledge and cognitive processes involved in sensorimotor learning and the construction of action categories, imitation and other forms of social learning, the acquisition of grounded conceptual representations and the development of the grammatical structure of language.
The project will lead to the development of: (a) new theoretical insights, models and scientific explanations of the integration of action, social and linguistic skills and in particular on the hypothesis that action, social and linguistic knowledge co-develop and further bootstrap cognitive development, (b) new interdisciplinary sets of methods for analysing the interaction of language, action and cognition in humans and artificial cognitive agents, (c) new cognitively-plausible engineering principles and approaches for the design of robots with behavioural, cognitive, social and linguistic skills.
Overall, the project proposes visionary research that will provide a new standard in embodied cognitive science and will demonstrate the effectiveness of the method proposed by integrating interdisciplinary theoretical and experimental research on a single advanced robotic platform.

---

Objective The objective of the LAMPETRA proposal is developing lamprey/salamander bioinspired artefacts to perform neuroscientific studies related to goal-directed locomotion and to find new solutions for high-performance artificial locomotion, in terms of fast-response, adaptability, reliability, energy efficiency, and control. These issues will be pursued by implementing a bio-inspired artificial system (mimicking the neuronal level, the biomechanical structure, the control functions) and by performing advanced numerical simulations focused on the neural control system of locomotion. The bio-inspired system will be composed by a high number of segments, a compliant body structure, muscle-like actuators, legs-like appendages, artificial stretch receptors, vestibular and vision sensors, and architectures for low/high level control functions. The numerical simulations are aimed at formulating a model of central pattern generator, at developing a software platform for replicating body movements, at implementing sensors inputs, visuo-motor coordination, posture control and legs movements. Experiments will include also a bio-hybrid approach, interfacing a lamprey nervous system with acquisition and control hardware. The project is expected to achieve advances in neurosciences, technology, and engineering. Neuroscience will provide information to be transferred to mathematical models, creating a bridge between biology and technology. Numerical simulations will be used to analyse these models and to study system properties in well defined situations. A number of features from lampreys and salamanders will be replicated in biomimetic artefacts. The project is also expected to bring developments in the field of control, i.e. the behaviour-based control inspired to animals, by rethinking control methods and involving the combination of digital and analogue circuits.

---

Objective LIREC aims to establish a multi-faceted (memory, emotions, cognition, communication, learning, etc.) theory of artificial long-term companions, embody it in robust and innovative technology, verify the theory and technology experimentally in real social environments, and provide resulting guidelines for designing such companions. Whether as robots, social toys or graphical and mobile synthetic characters, interactive and sociable technology is advancing rapidly. However, the social, psychological and cognitive foundations and consequences of such technological artefacts entering our daily lives - at work, or in the home - are less well understood. The technology is commonly based on evaluation of short-term interactions or even first encounters, and limited to one specific embodiment. Research shows that the novelty effect, which may attract interest in the first encounter, quickly runs out and that people's preferences and attitudes towards the system change. Successful technology can only be delivered on the basis of strong scientific foundations, and with partners in psychology, ethology, human-computer interaction, human-robot interaction, robotics and graphical characters, LIREC will advance understanding of the concepts of embodiment, autobiographic memory and social interactions in the context of companions where the 'mind' might migrate to differently embodied 'bodies'. Experimental human-human and human-animal studies and longitudinal evaluation of the developed technology in social settings will support the development and delivery of mechanisms for verbal and non-verbal social interaction and communication; an autobiographic emotionally-tagged memory; mechanisms for detecting and responding sensitively to the user's affective state, motives and intentions; an autonomous cognitive-affective architecture and support for migrating companions. These will be combined in case-study long-life companions that will take social technology to a new state-of-the-art.

---

Objective The main hypothesis of this project is that movement training for neurorehabilitation can be substantially improved through immersive and multimodal sensory feedback. The approach is real-time acquisition of behavioural and physiological data from patients and the use of this to adaptively and dynamically change the displays of an immersive virtual reality system, with the goal of maximising patient motivation. This will result in complex systems that are natural, user-friendly and easy to use.
In this project two exemplary systems will be developed for robot-assisted rehabilitation of upper and lower extremities. The systems will be able to record multi-sensory data (motion, forces, voice, muscle activity, heart rate, skin conductance etc.) and process this data in real-time to infer the intention of the patient and the overall psycho-physiological state. The computed information will be used to drive the therapy robots in combination with immersive virtual reality systems including 3D graphics and 3D sound. Several individually adjustable scenarios will be set up and the patient will be immersed into the corresponding virtual environments. Experimental tests on humans will evaluate to which such systems can increase the motivation of the patients and, thus, the therapeutic outcome of the training.
New basic insights will be obtained about presence and motivation in humans. Furthermore, MIMICS technology will enter clinical routine so that large patient populations (e.g. stroke, spinal cord injury) can benefit. Motor rehabilitation will become more efficient and may reduce costs of the European health care systems. A multi-national network is required to better compensate regional differences and increase international acceptance. The MIMICS consortium is characterised by an ideal balance of engineers, computer scientists, physicians and one industrial partner, who are all word-leaders in their respective fields.

---

Objective "The objective of this proposal is the development of an open architecture for future autonomous vehicles to become a standard platform shared by car makers in the design of next generation intelligent vehicles. It is based on 360 degrees sensorial suite which includes perceptual and decision making modules, with the ultimate goal of providing the vehicle with autonomous driving capabilities and/or supervise the driver's behavior. The perception module also includes vehicle-to-vehicle and vehicle-to-infrastructure subsystems, to increment the vehicle s sensing capabilities. The research is based on the extended know-how and experience of the Principal Investigator s group at the Univ of Parma, which already marked fundamental milestones worldwide in the field of vehicular robotics. Car manufacturers and automotive suppliers are extremely interested in this research stream, but at the same time are very cautious in investing in long term and risky research like this. Besides providing clear advantages on safety for road users, the availability of an open architecture will encourage and make possible the sharing of knowledge between public and private research communities (academic and automotive industry) and thus speed up the design of a standard platform for future vehicles. Further research steps will be eased -and therefore made more effective-thanks to the common and open architectural layer proposed by this project."

---

Objective "The objective of this proposal is the development of an open architecture for future autonomous vehicles to become a standard platform shared by car makers in the design of next generation intelligent vehicles. It is based on 360 degrees sensorial suite which includes perceptual and decision making modules, with the ultimate goal of providing the vehicle with autonomous driving capabilities and/or supervise the driver's behavior. The perception module also includes vehicle-to-vehicle and vehicle-to-infrastructure subsystems, to increment the vehicle s sensing capabilities. The research is based on the extended know-how and experience of the Principal Investigator s group at the Univ of Parma, which already marked fundamental milestones worldwide in the field of vehicular robotics. Car manufacturers and automotive suppliers are extremely interested in this research stream, but at the same time are very cautious in investing in long term and risky research like this. Besides providing clear advantages on safety for road users, the availability of an open architecture will encourage and make possible the sharing of knowledge between public and private research communities (academic and automotive industry) and thus speed up the design of a standard platform for future vehicles. Further research steps will be eased -and therefore made more effective-thanks to the common and open architectural layer proposed by this project."

---

Objective TThe REPLICATOR project focuses on the development of an advanced robotic system, consisting of a super-large-scale swarm of small autonomous mobile micro-robots that are capable of self-assembling into large artificial organisms. These robotic organisms possess common energy and information buses as well as reliable legged, wheeled or climbing locomotion, based on modular sub-systems which can be autonomously reconfigured. Thanks to the heterogeneity of the elementary robots and their capability to share resources and communicate, the robotic organisms are able to achieve a large computational power, and rich close- and far-range sensing. The energy is autonomously harvested from external power sources.
The main goal of the project is to develop novel principles underlying these robotic organisms, such as self-configuration, self-adjustment and self-learning. The bio-inspired evolutionary approach and evolvable hardware structure adopted in this project enable the robotic organisms to emerge new functionalities, to develop their own cognitive and control structures and, finally, to work autonomously in uncertain situations without any human supervision. Ultimately, these robotic organisms, which are extremely adaptive, robust, scalable and rich in sensing and actuating capabilities, will be used to build autonomous sensor networks, capable of self-spreading and self-maintaining in open-ended, even hazardous, environments.

---

Objective The ROBOCAST project aims to develop ICT scientific methods and technologies which focus on robot assisted keyhole neurosurgery. A modular system, allowing a reduction of the footprint, will be developed with two robots and one active bio-mimetic probe, able to cooperate among themselves in a biomimetic sensory-motor integrated framework. A gross positioning 3-axes robot will support a miniature parallel robot holding the probe to be introduced through a "keyhole" opening into the skull of the patient. Optical trackers (tracking the end effector and the patient), an imaging endoscope camera, and electromagnetic position and force sensors (on the probe) will extend robot perception by providing the control system with position and force feedback from the operating tools, and with visual information of the surgical field. Path planning outside and inside the body will be autonomously proposed by the control system by processing of preoperative diagnostic information. The path inside the brain will be planned on the basis of a "risk atlas" reproducing a fuzzy representation of a brain atlas, relating structures to a "level of danger". Construction of the atlas will rely on cognitive learning, where the system will be able to provide the surgeon with explanations for any suggested action. Semi-autonomous plan updating, following unforeseen changes occurring during surgery and based on processing of information gathered intra-operatively (e.g. ultrasonic images), will be negotiated between the system and the surgeon, where the latter will be allowed to specify any additional constraints to the planner. Ex- ante- and final path plan inside and outside the body will thus stem from the interaction between the surgeon and the intelligent core of the system. The interface between the system and the user will require minimal interaction while providing maximum information i.e. an intuitive interface which relies on context-based interpretation of surgeon commands.

---

Objective Starting from the assumption that cognition is embodied, the ROSSI project addresses the question how the possibility of communication between agents (e.g. humans and robots) is effected by differences in sensorimotor capacities.
This is an important issue, given that robots are expected to become more common in non-structured environments, such as homes or hospitals. While there is a trend towards humanoid robots, it is clear that for the foreseeable future there will still be substantial differences in physical embodiment between robots and humans.
To explore to what extent concepts must be shared to facilitate communication, we will build robots with sensorimotor systems structurally roughly similar to human beings. Furthermore, the control mechanisms for these robots will be based on insights into the neural mechanisms underlying human concepts and language.
In particular, two types of neurons in premotor cortex will be modelled: (1) canonical neurons, which are active during both the execution of specific object-directed actions and the mere visual observation of the same objects, and (2) mirror neurons, which are involved in both an agent's own actions and the visual observation of such actions performed by others.
In this framework, the project's aims are twofold. First, behavioural and neurophysiological experiments will provide new evidence and insights into the grounding of human conceptualization and language. Second, computational modelling of the underlying neural mechanisms will provide novel approaches to grounding of robotic conceptualization and language.
More specifically, the project aims at building robots endowed with the sensorimotor and neural/computational mechanisms that allow them to: (a) flexibly manipulate and use objects in the environment, (b) use a simple form of language, i.e. nouns and verbs referring to objects and object-oriented actions, (c) use such concepts and verbal labels in social interaction with humans.

---

Objective The single overarching goal of the SF consortium is to identify the neuronal, cognitive and behavioral principles underlying optimal foraging in rodents and to implement these principles in a real-world foraging artefact or the Synthetic Forager (SF.01). SF.01 constitutes a novel biologically based cognitive technology for autonomous exploration and foraging in real-world man-made indoor and outdoor environments. SF exploits our growing understanding of exploration and foraging behavior in rodents, advances current theories of the neuronal and behavioral organization of foraging and transfers this understanding towards the construction of novel real-world synthetic cognitive technologies.
The behavior and neurophysiology of foraging will be studied in rodents behaving in automatically controlled multi-modal environments, fully controlled using an advanced experimental technology developed by the consortium. The overall integration of the perceptual, cognitive and behavioral control systems will be accomplished using a Distributed Adaptive Control (DAC). The perceptual, cognitive and behavioral control systems of SF will be based on statistical analysis and detailed game theoretic models. The SF control systems are validated against the behavioral and physiological data. The SF phenotype comprises a high-mobility robotic platform equipped with visual, auditory, olfactory and tactile sensors. The SF will be evaluated in a number of stringent benchmarks ranging from robot equivalents of rodent foraging tasks to simulated de-mining.
The approach and technologies developed in SF will have long-term implications to areas including: cleaning robots, search and rescue systems, terrestrial and planetary exploration, delivery systems, autonomous transportation systems, military intelligence and battle field information control systems, environmental monitoring, internet information analysis and retrieval, information and communication networks and humanitarian de-mining.

---

Objective The aim of SPARK II is to develop, evaluate, optimise and generalise a new, insect brain inspired, computational model. The architecture will be hierarchical, based on parallel sensory-motor pathways, implementing reflex-driven basic behaviours. These will be enriched with higher and complex insect brain structural models (Mushroom Bodies, Central Complex) and more physically-inspired nonlinear lattices. The latters will be able to generate "self-organizing" complex dynamics, while the formers will reproduce relevant cognitive functions in insects such as attention-like processes, short-term memory and rewarding mechanisms. Both of them will work concurrently to generate cognitive behaviours at the output motor layer.
The architecture will exploit a number of different sensors, processing signals distributed in space and time and also showing nonlinear dynamics. Perceptual processes are conceived as emerging pattern flows (result of a nonlinear spatial-temporal dynamics). Pattern meaning (concept generation) will be incrementally built upon information derived from sensors as well as from some basic "inherited" behaviours mediated through the environment. Emerged patterns, with the concurrent dynamics generated by models of the relevant perception centres in insects, will influence the particular associated motor behaviour.
To show the generality of the approach, the model will be applied to different robotics architectures, asked to work in real life, unstructured, cluttered and dynamically changing environment. This will demonstrate that perception for action, formulated by suitably merging complex systems with dynamics derived by insect brain models, can lead to the emergence of general archetypes of wide applicability.
Finally another challenging task will consist in using the same model, applied to different robots, to lead to the emergence of cooperation capabilities to perform tasks unable to be fulfilled by one robot alone: further proof of generality.

---

Objective The main focus of this project is to investigate and develop novel principles of adaptation and evolution for symbiotic multi-robot organisms based on bio-inspired approaches and modern computing paradigms. Such robot organisms will consist of super-large-scale swarms of robots, which can dock with each other and symbiotically share energy and computational resources within a single artificial-life-form. When it is advantageous to do so, these swarm robots can dynamically aggregate into one or many symbiotic organisms and collectively interact with the physical world via a variety of sensors and actuators. The bio-inspired evolutionary paradigms combined with robot embodiment and swarm-emergent phenomena, enable the organisms to autonomously manage their own hardware and software organization. In this way, artificial robotic organisms become self-configuring, self-healing, self-optimizing and self-protecting from both hardware and software perspectives. This leads not only to extremely adaptive, evolve-able and scalable robotic systems, but also enables robot organisms to reprogram themselves without human supervision and for new, previously unforeseen, functionality to emerge. In addition, different symbiotic organisms may co-evolve and cooperate with each other and with their environment.
The extraordinary potential and capability of autonomous large-scale symbiotic self-aggregation, reprogramming and evolution would open-up a wide range of current and future applications. The main application scenarios of such artificial organisms would be human-free environments with a high degree of danger or uncertainty as e.g. hazardous or space environments. The consortium represents leading organizations in different research fields and provides the critical mass of expertise and resources. The core of consortium is already participating in other robotic projects, as e.g. I-Swarm or MiCRoN thus demonstrating the high level of competence and credibility of this proposal.

---

Objective "In recent breakthrough work at the boundary of robotics and computer vision, I demonstrated that the core `Simultaneous Localisation and Mapping' (SLAM) approach of probabilistic map-building for mobile robots can be applied to real-time 3D motion estimation from the image stream from a single agile camera. Now I propose to take this line of research to its logical conclusion by investigating a paradigm of `Instant SLAM'. Can monocular visual SLAM be pushed far enough to estimate in real-time the motion of cameras potentially moving and accelerating very rapidly, turned on at arbitrary times and pointed at arbitrary scenes? Is is possible to develop a real-time vision algorithm, running on the standard processors of today or the near future, which can accurately track the motion of a camera attached to flying or bouncing robot, carried by a running person or even just thrown across a room? The ability to track camera motion with so few restrictions on dynamics and prior knowledge would turn simple webcam-type or embedded camera modules into truly flexible, low cost, go-anywhere position sensors with any number of applications in robotics, wearable computing and beyond. Achieving this goal requires both theoretical and practical research on extracting with ultimate efficiency the motion information available in an image sequence, with emphasis on high frame-rate capture, information-theoretic analysis, optimised probabilistic filtering and the relaxation of prior assumptions."

---

Objective The aim of the ANGELS project is to design and build a prototype of a reconfigurable Anguilliform swimming robot able to split into smaller agents (and vice-versa), each equipped with a bio-inspired "electric sense" used both for recognition of objects and communication between agents. This mode of active perception, present in some fish, is based on the polarisation of certain regions of their body, so generating an electric field flowing through an electroreceptive skin.The robot will exploit both "mechanical re-configurability", by changing body topology, and a new concept of "electric re-configurability" that will allow the robots to self-adapt their perception to their environment by changing the location of emitters and receptors on the robot boundaries. The electric field generated around the robots can be considered as an "electric-body" shaped through electric reconfigurations. ANGELS will explore the range of abilities conferred by different mechanical and electric morphologies, from the shaping of the common electric body shared by the agents navigating in formation, to design of behavioural cooperative rules inspired by fish for improving multi-agent perception through emergent collective behaviours.Thus the intelligence encoded in the animals' morphology will be applied to the design of a new generation of Autonomous Underwater Vehicles able to adapt to their environment and suited to a wide spectrum of uses, in particular in situations where vision cannot be used. To achieve these goals, the ANGELS will form a multidisciplinary team combining the complementary expertise of biologists, roboticians, image-processing specialists and nuclear physicists specialised in the design of particle detectors.

---

Objective BRICS addresses a very urgent need of the research community, namely the need for common research platforms, which support integration of research results and which support the evaluation, comparison and benchmarking of result and the promotion of best practice in robotics. In a poll in the robotics research community in December 2007 95% of the participants have voted for such platforms.

Such common research platform would be beneficial for the robotics community, both for the academic community as much as for the industrial community. The academic community would save a significant amount of resources, which typically have to be invested in "from scratch developments" and "me-too approaches". Furthermore the scientific results would become better comparable which might promote a culture of sound experimentation and comparative evaluation. Jointly specified platforms will foster a rapid technology transfer. A rapid technology transfer will support the development of new systems and applications, which is to the benefit of the industrial community.

To achieve the our objectives we propose the development of a design methodology which focuses on three fundamental major research and development issues, which will be implemented in three highly interwoven lines of technical activities:

•\tIdentification of best practice in robotics hardware and software components
•\tDevelopment of a tool chain that supports a rapid and flexible configuration of
new robot platforms
•\tCross-sectional activities addressing robust autonomy, system openness, and harmonisation and benchmarking

Kuka in BRICS takes the leadership in a community initiative, which is not free of risk for the company, but which is badly needed for a convergence of technology in the field of robotics. This endeavour is of fundamental importance to ensure the success and competitiveness of the European robotics industry as well as research.

---

Objective Autonomous Underwater Vehicles (AUVs) represent one of the most challenging frontiers for robotics research. AUVs work in an unstructured environment and face unique perception, decision, control and communications difficulties. Currently, the state of the art is dominated by single AUVs limited to open-sea preplanned trajectories with offline postprocessing of the data gathered during the mission. The use of multiple AUVs as propagated in this project is still in a very early research phase. Some of the research issues addressed in this project are even completely uncharted territory, especially the development of functionalities to seamlessly monitor critical underwater infrastructures and to detect anomalous situations (e.g., missions related to harbour safety and security) and the study of advanced AUVs capable of interacting with humans to perform such functions as companion/support platforms during scientific and commercial dives. The aim of the Co3-AUVs project is to develop, implement and test advanced cognitive systems for coordination and cooperative control of multiple AUVs. Several aspects will be investigated including 3D perception and mapping, cooperative situation awareness, deliberation and navigation as well as behavioral control strictly linked with the underwater communication challenges. As a result, the team of AUVs will cooperate in challenging scenarios in the execution of missions where all data is online processed. In doing so, the team will be robust with respect to failures and environmental changes. These key features will be tested in a harbor scenario where additional difficulties with respect to open sea applications arise and in a human diver assistance scenario that also illustrates human robot interaction issues.

---

Objective Standard humanoid robots mimic the human form, but the mechanisms used in such robots are very different from those in humans, and the characteristics of the robots reflect this. This places severe limitations on the kinds of interactions such robots can engage in, on the knowledge they can acquire of their environment, and therefore on the nature of their cognitive engagement with the environment. However, a new kind of robot is just beginning to emerge - the anthropomimetic robot. Instead of just copying the outward form of a human, it copies the inner structures and mechanisms - bones, joints, muscles, and tendons, and thus has the potential for human-like action and interaction in the world. Unfortunately, there are as yet no established methods for controlling such robots, nor even for describing their intrinsic movement patterns. This project has three goals: to design and build a robot using anthropomimetic principles; to characterise its dynamics and control it; and to exploit its human-like characteristics to produce some human-like cognitive features. The consortium already has access to an excellent anthropomimetic torso design; this will be further developed to include significant engineering refinements, and will be mounted on a powered mobile chassis to permit mobile manuipulation. Advanced techniques of motion capture and causal analysis will characterise the robot's intrinsic dynamics. Three major approaches to control will then be assessed: classical control theory, the use of physics based internal models, and sensory-motor strategies. These will be combined in a single architecture to cope with different types of tasks, and the consortium will then focus on the use the robot can make of its human-like interaction with the world to begin to cognise the world in a human-like way. Finally, the performance of the robot will be evaluated, both absolutely, and in comparison with the relevant outputs from the EU RobotCub project.

---

Objective The European robotics industry plays a key role in maintaining our continent's industrial base. The robotics industry is strong, but fragmented and dispersed. In the future, cutting-edge technology resulting from top-level research will be the decisive factor for success. Europe not only has a powerful robotics industry, but can also boast superb research. By drawing on these resources, ECHORD aims at producing new knowledge through advancing the state of the art in selected research foci and developing novel technology from which new products can be derived. Within ECHORD, opportunities for knowledge advancement and technology transfer between academia and industry will be created across the whole continent. This will be achieved through the solicitation of focused, small-size RTD projects, so-called experiments, which can be rapidly negotiated, funded and executed. Via these experiments, ECHORD will bring about a large-scale introduction of robotic equipment into research institutions. This is expected to result in both tangible and measurable out-comes in terms of the accelerated development of technologies, as well as the deployment of robotics technology into new scenarios for the direct application of research results. For ECHORD, three such scenarios have been defined: human-robot co-working, hyper flexible cells, and cognitive factories. The foremost purpose of the scenarios is to define an environment that is both scientifically challenging to research institutions and commercially relevant to robot manufacturers.

---

Objective There has been a tremendous progress in the development of autonomous robots that offer different services to their users. Typical services include the support of elderly people, cleaning, delivery tasks, autonomous driving, or surveillance. Most of the systems developed so far, however, are restricted to indoor scenarios, to non-urban outdoor environments, or to autonomous driving on roads. Furthermore, substantial efforts are required to deploy such robots in new environments.
Urban areas are highly dynamic and complex and introduce numerous challenges to autonomous robots. They require solutions to several complex problems regarding the perception of the environment, the representation of the robot's workspace, models for the expected interaction with users to plan actions, state estimation as well as the states of all dynamic objects, the proper interpretation of the gathered information including semantic information as well as long-term operation.
The goal of the EUROPA2 project, which builds on top of the results of the successfully completed FP7 project EUROPA, is to bridge this gap and to develop the foundations for robots designed to autonomously navigate in urban environments outdoors as well as in shopping malls and shops, for example, to provide various services to humans. Based on the combination of publicly available maps and the data gathered with the robot's sensors, it will acquire, maintain, and revise a detailed model of the environment including semantic information, detect and track moving objects in the environment, adapt its navigation behavior according to the current situation and anticipate interactions with users during navigation. A central aspect in the project is life-long operation and reduced deployment efforts by avoiding to build maps with the robot before it can operate. EUROPA2 is targeted at developing novel technologies that will open new perspectives for commercial applications of service robots in the future.

---

Objective The goal of the EVRYON project is to develop a novel approach for the design of Wearable Robots (WRs) that can be used for such applications as rehabilitation, assistance, human augmentation and more. Such systems should aim at the optimal trade-off between performance, i.e. the level of assistance to be provided to the user, and some critical requirements, such as minimal weight and encumbrance, low energy consumption and several other factors that can impact the effectiveness of WRs. Under this regard, better WRs can be developed if 'embodied intelligence', and particularly 'structural intelligence', are properly exploited.
To this aim, EVRYON will develop an open-ended design process where both robot morphology and control are co-evolved and optimized in a simulation environment, where also the dynamical properties of the human body are taken into account.
The EVRYON design methodology will originate advanced tools for assisted mechatronic design, that will be validated by developing an active orthosis for the lower limbs. The orthosis will include a novel hip module for improved ergonomics and a distributed sensory apparatus that will also monitor the motions of the upper body for an improved user intention detection.
The EVRYON WR will integrate kinematic, dynamic and control solutions produced by the co-evolutionary optimization process. Custom variable impedance modules will allow the tuning of the dynamical properties of the robot so that walking will arise as an emerging dynamic behaviour.
The WR prototype will be tested on a group of elderly subjects with age-related motor decay so to assess its acceptability and its ability to restore proper walking and increase personal autonomy.

---

Objective The overall aim of FILOSE is acquiring a deeper understanding of the principles underlying fish locomotion and sensing, in order to develop new technologies for underwater vehicles on the basis of biological evidence. More specifically, FILOSE focuses on shedding light on how fish exploit lateral line sensing in underwater environments. The lateral line provides fish with the ability to detect hydrodynamic patterns in the surrounding environment, thereby playing a key role in adapting to environmental changes. FILOSEs main goals are captured by the following objectives: 1. investigate fish locomotion in a controlled hydrodynamic environment, in particular addressing the issue of how fish react to changes in hydrodynamic patterns; 2. develop a novel mechanical design of an underwater fish robot, characterized by high maneuverability and low complexity; 3. develop a MEMS-based artificial lateral line; 4. develop a control method for the artificial fish, aimed at reproducing locomotion patterns found in biological fish and based on a central pattern generator (CPG); 5. develop a method to characterize and classify hydrodynamic images, making use of a mechanosensory array; 6. develop a classification method to couple detected hydrodynamic events with locomotion patterns found in biological fish; 7. conduct comparative experiments in a controlled hydrodynamic environment to assess behaviour of an artificial fish equipped with artificial lateral line sensing with respect to the behaviour of a biological fish. We believe that the proposed investigations are key to building underwater robots that improve on the existing by exhibiting a greater degree of autonomy, adaptability to environmental changes, maneuverability, stability and overall lower complexity, as well as moving more efficiently and quietly.

---

Objective The field of robotics is undergoing a major revolution as it is increasingly being applied to general purposes outside the production line: for health, rehabilitation and professional services, in domestic and leisure environments, as well as hazardous environments. There, one keystone for robots to carry out accurate and intelligent tasks, with and for people, is their ability both to handle autonomously all sorts of objects and to use human tools. However, today's robots are unable to achieve dexterous and fine manipulation, especially when this requires in-hand manipulation. They are far from being able to understand and reason about their environments, their goals and their own capabilities, to learn skills and improve their performance by what they have been taught and their own experience, to interact with their environments with the efficiency of humans.
The HANDLE project aims at understanding how humans perform the manipulation of objects in order to replicate grasping and skilled in-hand movements with an anthropomorphic artificial hand, and thereby move robot grippers from current best practice towards more autonomous, natural and effective articulated hands. The project implies not only focusing on technological developments but also working with fundamental multidisciplinary research aspects in order to endow the robotic hand system with advanced perception capabilities, high level feedback control and elements of intelligence that allow recognition of objects and context, reasoning about actions and a high degree of recovery from failure during the execution of dexterous tasks.
Integrating findings from disciplines such as neuroscience, developmental psychology, cognitive science, robotics, multimodal perception and machine learning, the method we will develop is based on an original blend of learning and predicting behaviours from imitation and "babbling" to allow the robot to be capable of responding to gaps in its knowledge.

---

Objective The HUMOUR project will investigate and develop efficient robot strategies to facilitate the acquisition of motor skills. We will address both the (human) trainee and the (robot) trainer sides, by combining behavioural studies on motor learning and its neural correlates with design, implementation, and validation of robot agents that behave as 'optimal' trainers, which efficiently exploit structure and plasticity of the human sensorimotor systems. On the human trainee side, we will focus on the cognitive and neural mechanisms underlying the acquisition of a variety of motor skills, by specifically aiming at understanding the way humans physically cooperate in acquiring a motor skill and how physical assistance affects motor learning. Experiments will enable us to identify determinants and dynamics of the learning process in representative motor tasks, and will provide the foundations for designing efficient schemes of assistance. On the robot trainer side, we will develop several robot agents for the acquisition of a variety of motor skills. They will be capable of generating appropriate schemes of assistance, based on the specific task needs and possibly learned from human experts. Robots will continuously adapt assistance, in terms of the observed performance and of neural and cognitive correlates of adaptation, to the specific user (e.g. patient) and his or her state. Robot trainers will be validated in the context of motor skill learning and robot-assisted rehabilitation. Robot agents that facilitate the capture of new motor skills may potentially benefit large groups of individuals, by helping professionals, e-g- surgeons, to acquire delicate motor skills; by providing older persons greater access to activities like fitness, sports, and arts, thus ultimately improving their quality of life. Furthermore, more effective training agents would also represent an innovative approach to robot therapy, which would likely increase its impact and extend its scope.

---

Objective IM-CLeVeR aims to develop a new methodology for designing robots controllers that can: (1) cumulatively learn new efficient skills through autonomous development based on intrinsic motivations, and (2) reuse such skills for accomplishing multiple, complex, and externally-assigned tasks. During skill-acquisition, the robots will behave like children at play which acquire skills autonomously on the basis of "intrinsic motivations". During skill-exploitation, the robots will exhibit fast learning capabilities and a high versatility in solving tasks defined by external users due to their capacity of flexibly re-using, composing and re-adapting previously acquired skills.
This overall goal will be pursued investigating three fundamental scientific and technological issues: (1) the mechanisms of abstraction of sensory information; (2) the mechanisms underlying intrinsic motivations, e.g. "curiosity drives" that learn to focus attention and learning capabilities on "zones of proximal development"; (3) hierarchical recursive architectures which permit cumulative learning. The study of these issues will also be fuelled by a reverse-engineering effort aiming at reproducing with bio-mimetic models the results of empirical experiments run with monkeys, children, and human adults. The controllers proposed will be validated with challenging demonstrators based on a single humanoid robotic platform (iCub).
As a main outcome, the project will significantly advance the scientific and technological state of the art, both in terms of theory and implementations, in autonomous learning systems and robots. This overall goal will be achieved on the basis of the integrated work of a highly interdisciplinary Consortium involving leading international neuroscientists, psychologists, roboticists and machine-learning researchers.

---

Objective Locomorph's goal is to push beyond the state of the art in robotic locomotion and movements, by increasing efficiency, robustness, and thus usability in unknown environments. As robotic research and industry are competing to increase robots' usability towards the highly-in-demand service robotics, advancements in robotic locomotion today would give Europe a significant competitive advantage. Locomorph combines multidisciplinary approaches from biology, biomechanics, neuroscience, robotics, and embodied intelligence to investigate locomotion and movements in animals and robots, focusing on two concepts: morphology and morphosis. We will build many diverse robots using heterogeneous modules to explore various morphological factors (shape, materials, sensors, compliance, etc) and sensory-motor control strategies, in order to generate novel and optimal robotic designs which exploit the physical dynamics emerging from the interaction among the physical morphology, control, and environment. The second concept, morphosis (changing of morphology), extends this concept further. Voluntary morphosis is a valuable skill for robots, as it can increase their adaptivity to current tasks/environments. We will adopt two complementary approaches. We will conduct animal/human experiments to study biological strategies in dealing with voluntary/involuntary morphosis. We will extract insights from the results to develop strategies for effective robotic morphosis and motor control solutions for dealing with morphosis. This, combined with the robot's modularity, will create highly robust robots, able to deal with body changes e.g. limb loss. Through an exploration of morphology and morphosis, we aim to develop robots with increased maneuverability, self-stabilization, energy efficiency, and adaptivity to unknown environments. These advances will bring us closer to service robotics, as a large part of these robots must able to locomote safely, regardless of surfaces, layouts, or terrains.

---

Objective The octopus is a marine invertebrate with amazing motor capabilities and intelligent behaviour. Its body has no rigid structures and has interesting characteristics, from an engineering viewpoint: infinite number of degrees of freedom (dof), bending in many different directions, variable and controllable stiffness, high dexterity, fine manipulation, highly distributed control. The octopus represents a biological demonstration of how effective behaviour in the real world is tightly related to the morphology of the body.
The grand challenge of the OCTOPUS IP is investigating and understanding the principles that give rise to the octopus sensory-motor capabilities and at incorporating them in new design approaches and ICT and robotics technologies to build an embodied artefact, based broadly on the anatomy of the 8-arm body of an octopus, and with similar performance in water, in terms of dexterity, speed, control, flexibility, and applicability.
The new technologies expected to result from the IP concern actuation (soft actuators), sensing (distributed flexible tactile sensors), control and robot architectures (distributed control, coordination of many dof), materials (variable stiffness), mechanisms (soft-bodied hydrostat structures), kinematics models.
The final robotic octopus prototype will be capable of locomotion on different substrates, of squeezing into small apertures, of dextrous manipulation by coordinating the eight arms, of anchoring in order to exert forces on external environment; of controlling a flexible structure with virtually infinite dof.
This IP is expected to achieve new science and new technology, demonstrated by joint publications and patents, through a truly interdisciplinary research work programme, by a consortium of engineering and biology groups, each involving diverse disciplines and all experienced in biomimetics, possessing, as a consortium, the best expertise available in Europe for pursuing the objectives of the IP.

---

Objective This project will develop and demonstrate a range of new robot capabilities based on robot skin tactile feedback from large areas of the robot body. An investigation of these issues until now has been limited by the lack of tactile sensing technologies enabling large scale experimental activities, since so far skin technologies and embedded tactile sensors have been mostly demonstrated only at prototype stage. The new capabilities will improve the ability of robots to operate effectively and safely in unconstrained environments and also their ability to communicate and co-operate with each other and with humans. To support this aim, one part of the project will focus on the investigation of methods and technologies enabling the implementation of skin sensors that can be used with existing robots. Other parts of the project will develop new structures for representing and integrating tactile data with existing cognitive architectures in order to support skin-based cognition, behaviour and communication. As a result, the project will address three main objectives. The first is to develop new sensor technologies that can provide tactile feedback from large areas of the robot body. This development process will be incremental and will take advantage of feedback produced from the application of these technologies in the work done on the other project objectives. The second objective is to develop and integrate fundamental cognitive structures for efficient and safe utilisation of tactile data in terms of a robot body image, safe reflexive reactions to tactile events and flexible representations of spatially and temporally distributed patterns of physical contact. The last objective is to develop cognitive mechanisms that use tactile feedback to improve human-robot interaction capabilities particularly in the domains of programming through demonstration and robot assisted play.

---

Objective The proposal objective is to investigate new approaches to the design and development of humanoid robots with advanced perception and action capabilities, showing robust, adaptive, predictive and effective behaviour in the real world. The proposed new approaches are strongly based on the concept of human sense of movement by Alain Berthoz, a key partner in this proposal. There are two main ideas related to this concept, which are relevant to robotics: 1) the vestibular unified reference frame, as set by the vestibular system in the centre of the head; 2) Expected Perception (EP), or the capability to make predictions of consequences of actions, which is at the basis of the human predictive control. The expected robot behaviour is the capability to follow a visual target by coordinating eye, head, and leg movements, with head stabilization, walking smoothly and effectively in an unstructured environment, with a robust reactive behaviour, improved by predictions. This behaviour is a fundamental, but quite novel, capability for humanoid robots, and it may result in a truly robust and effective behaviour in many helpful tasks in real-world scenarios. The proposed project will use the existing and fully operational biped humanoid robotic platform named Sabian, available at SSSA. It is a copy of the Wabian robot developed at WUT. The legs have functional hip and waiste DOF that allow to de-couple the leg and head movements. This unique feature guarantees a truly human-like walking behaviour with head stabilization. IST has developed and has in its lab the same head as the Sabian robot. It is strong belief of proposers that the service robotics markets needs a new generation of robotic systems with a better behaviour in real world, in terms of sensory-motor performance, adaptability, robustness, and that understanding the principles underlying the biological brain sense of movement can lead to the design of robots that represent one important step in this direction.

---

Objective The ROSETTA project develops a new generation of robot controller technology for human-like industrial robots working together with humans. The control system uses a variety of sensor-based skills, together with knowledge about how to adapt them on-line to the concrete situation. The controller executes complex tasks with a high degree of autonomy, because it has network access to a knowledge repository and management system. The knowledge repository receives feedback from a population of installed robot systems, that have received their programmes from the repository earlier, and that now share their individual improvements and experiences. The knowledge repository can thus improve and extend itself at an unprecedented fast rate.
This will result in:
- A new paradigm for engineering & instructing of robots, based on robot task and skill representations. Human programmers will be able to build a robot application using task-level instructions, embedding knowledge about the sensing, planning, control and decision making that is appropriate for the task. A breakthrough will be finding representations of all types of knowledge required in the envisaged production application. These results will be implemented on a controller and tested in an assembly application with human-like robots, with on-line error detection and error recovery, using adaptive hybrid control for self-improvement, and natural language interaction.
- New sensing, control and decision making methods for safe physical human-robot interaction. This includes new functionalities that are "human-centred": the controller has knowledge about how it could inflict injuries on humans, and how to avoid them, and it is equipped with sensors, sensor processing and reasoning capabilities to find the optimal trade-off between effective progress in its task and safe and psychologically acceptable physical interaction with the human co-worker(s), which will be subject to standardisation efforts.

---

Objective We have identified a cutting-edge method for monitoring pollution in ports as specified in EU Directive 2005/35. This monitoring process is currently costing approximately 350 million Euros per year in the EU. SHOAL will develop a shoal of robotic fish to analyse contaminants in water and produce a real-time map of which pollutants are in the water, in what concentrations and where these are on a 3D map of the port. SHOAL will use advanced swarm intelligence techniques to control the robots, utilising hybrid particle swarm/ant colony optimisation techniques in order to coordinate the group efficiently and adapt quickly to changes in the environment. This will benefit not only monitoring operations in ports across the EU, but also lead to important advances in robotics, chemical analysis, underwater communications and robot intelligence. At present there are no fully autonomous systems for monitoring pollution in ports. SHOAL is innovative in that it can analyse chemicals not only on the surface of the water (e.g. oil) but also those that are dissolved in the water (e.g. nitrates). This will allow the fish to find pollution from agriculture as well as leaks from vessels in a port.
SHOAL will build robot fish which will function independently and as part of a larger group to analyse and monitor pollution in a port. These robotic fish will be equipped with chemical sensors to find pollutants in the water and modems to create an ad hoc network for communication within the swarm. This will allow the shoal of robot fish to build up a broad map of the pollutants moving through the port in real time whilst adapting naturally to changes in environmental conditions in the port. Beyond this, due to the design of the robots, they will be able to search underwater rather than simply on the surface, meaning that if a leak is still occurring they will be able to isolate it even if it originates underwater (for example from the hull of a ship or an underwater pipeline).

---

Objective Autonomous micro helicopters are about to play major roles in tasks like reconnaissance for search and rescue, environment monitoring, security surveillance, inspection, law enforcement, etc. The ability to fly allows easily avoiding obstacles on the ground and to have an excellent bird's eye view. Therefore flying robots are the logical heir of ground based mobile robots. Its navigational and hovering advantages make them the ideal platform for exploration, mapping and monitoring tasks. If they are further realized in small scale, they can also be used in narrow out- and indoor environment and they represent only a limited risk for the environment and people living in it. However, for such operations today's systems navigating on GPS information only are not sufficient any more. Fully autonomous operation in cities or other dense environments requires the micro helicopter to fly at low altitude or indoors where GPS signals are often shadowed and to actively explore unknown environments while avoiding collisions and creating maps. This involves a number of challenges on all levels of helicopter design, perception, actuation, control, navigation and power supply that have yet to be solved.
Our S and T endeavor proposed in this project will therefore focus on micro helicopter design, visual 3D mapping and navigation, low power communication including range estimation and multi-robot control under environmental constraints. It shall lead to novel micro flying robots that are:
• Inherently safe due to very low weight (around 500g) and appropriate propeller design;
• capable of vision-based fully autonomous navigation and mapping;
• able of coordinated flight in small swarms in constrained and dense environments.

---

Objective The goal of the proposed study is to equip a highly biomimetic robot hand-arm system with the agility, robustness and versatility that are hallmarks of the human motor system by understanding and mimicking the variable stiffness paradigms that are so effectively employed by the human CNS. A key component of the study will be the anatomically accurate musculoskeletal modelling of the human arm and hand. The project will develop novel methodologies to comprehend how the human arm can adapt its impedance, e.g. by changing the co-contraction level or by adapting the reflex gains. The impedance of the arm and of the hand will be investigated using powerful robot manipulators capable of imposing force perturbations. The existing closed-loop system identification techniques will be extended with non-linear time-variant techniques which can identify the behaviour during reaching and grasping tasks. The grasp force modulation and hand muscle activity correlations will be learned for use on the robotic system. Finally, optimization techniques gleaned and validated on the detailed biophysical model will be transferred to the variable impedance actuation of the novel biomorphic robotic system The central question that this proposal focuses on for both the human and robotic arm is: how is stiffness used to enhance performance?; and this project represents one of the first attempts where targeted modelling studies will go the full circle by exploiting these results for optimal control of an embodied, high dimensional, variable impedance robotic system.

---

Objective Most of today's robots have rigid structures and actuators which require complex software control algorithms and sophisticated sensor systems in order to behave adaptable, compliant, and safe in contact with unknown environments or with humans. Moreover, in terms of energy efficiency, peak force and speed, these robots are still considerably weaker than their biological archetypes. An alternative design approach is to build actuators with physically adjustable compliance and damping, which are able to store and release mechanical energy, react softly when touching the environment, and provide an intrinsic degree of safety, just like muscles do. The ability to vary the impedance (i.e., stiffness and damping) is crucial in order to optimally adapt to a large variety of situations.However, the technological realisation of such systems is very challenging, having many open problems in terms of materials, design, actuation, and control concepts. VIACTORS addresses the development and use of safe, energy-efficient, and highly dynamic variable-impedance actuation systems which will permit the embodiment of natural characteristics, found in biological systems, into a new generation of mechatronic systems. Target outcome of the project is that of obtaining the intended physical interaction and motion behaviours of the robotic system intrinsically by its physical structures to the maximum extent possible. This will not only save computational and communication costs for controlling the robot motion; it will also allow us to match the task requirements in a natural and highly dynamic way as it can be observed in biology. And, perhaps most important, to provide a powerful, human-like physical interface which can be accessed by higher (cognitive) intelligence levels without having to care for basic motion generation principles.This advance in technology will pave the way towards new application fields, such as industrial co-workers, household robots, advanced prostheses and rehabilitation devices, and autonomous robots for exploration of space and hostile environments. Therefore, results of this project will deeply impact applications where successful task completion requires people and robots to collaborate directly in a shared workspace or robots to move autonomously and as efficiently as humans.

---

Objective Compared to animals and humans, the motor skills of today's robots still must be qualified as poor. Their behavioural repertoire is typically limited to a narrow set of carefully engineered motor patterns that operate a rigid mechanics and lack situated adaptivity, learnability and dynamical fusion of motor primitives into complex, task-oriented behavioural patterns.
The AMARSi Integrated Project aims at a qualitative jump toward biological richness of robotic motor skills, a jump to complex, task-oriented interaction sequences between a robot and a human caretaker.
It comprises leading groups from robot engineering, compliant mechanics, morphological computing, human motor research; biomechanics, theoretical biology, machine learning. The project deploys:
- coordinated and simultaneous development of compliant mechanics, pervasive learning and dynamical-systems based control architectures, centered on the concept of adaptive modules;
- mutually informing research in human motor behavior and robotics;
- reliance on compliant mechanics and morphological computing for flexibility, computational and motoric speed, safety and damage-robust learning;
- novel learning paradigms (unsupervised, reinforcement and imitation) drawing from principles of reservoir computing;
- control architectures based on dynamical (neural) systems throughout, also on the higher cognitive levels.
Robotic demonstration with a compliant version of the iCub robot and a compliant quadruped Cheetah platform will manifest progress. The robots will engage in an interaction with a human caretaker at the level of a young child playing open-ended in a cluttered and rough environment. Hardware and software solutions will be made publicly available as open sources.
Ultimately, the naturalness of such compliant robots will let them blend into the everyday routines of human society, physically safe and psychologically acceptable.

---

Objective The project entitled Highly Customizable and Flexible Packaging Station for mid- to upper sized Electronic Consumer Goods using Industrial Robots (CustomPacker) aims at developing and integrating a scalable and flexible packaging assistant that aids human workers while packaging mid to upper sized and mostly heavy goods.Electronic consumer goods, e.g. TV sets, have a large number of variants and are packaged manually. Only in single-variant production lines with high lot sizes, an automation of the packaging step has been introduced. However, automating the packaging process will decrease the production cycle time and costs also for mixed variant production lines, thus allowing that several production lines can be merged to a reduced number of flexible packaging stations. This also allows an optimization with regard to the actual demands of the (various) goods (i.e. number of items produced per day). In order to achieve the realization of these challenging goals for a highly flexible packaging station, CustomPacker will bring together the highly adaptable skills of a human worker together with the precision and ability of robots to carry heavy goods.The main goal of CustomPacker is to design and assemble a packaging workstation mostly using standard hardware components resulting in a universal handling system for different products. Ideally one setup for packaging a high variety of products and components can be implemented, which can be achieved by a teachable system architecture. This will open a new dimension of today's way in how industrial robots are deployed, namely the collaboration of human workers together with robot co-workers. Only by driving the reliability and precision of today's available technology to the limits and by additionally forcing the use of highly sophisticated software modules for worker detection and intention recognition, the cycle times can be reduced in order to justify the investment costs for additional complexity.

---

Objective "CROPS will develop scientific know-how for a highly configurable, modular and clever carrier platform that includes modular parallel manipulators and “intelligent tools” (sensors, algorithms, sprayers, grippers) that can be easily installed onto the carrier and are capable of adapting to new tasks and conditions. Several technological demonstrators will be developed for high value crops like greenhouse vegetables, fruits in orchards, and grapes for premium wines. The CROPS robotic platform will be capable of site-specific spraying (targets spray only towards foliage and selective targets) and selective harvesting of fruit (detects the fruit, determines its ripeness, moves towards the fruit, grasps it and softly detaches it). Another objective of CROPS is to develop techniques for reliable detection and classification of obstacles and other objects to enable successful autonomous navigation and operation in plantations and forests. The agricultural and forestry applications share many research areas, primarily regarding sensing and learning capabilities."

---

Objective Over the last few years successful coordination activities have been undertaken within the academic and industrial robotics
communities (EURON and EUROP), but both communities still struggle with overcoming some remaining gaps: the
community-internal gaps of confusion about terminology, suboptimally coordinated transfer of research visions, technology
and people, and the limited coordinated communication with both the general public and professional audiences. This
includes the popular science media, European Commission, national funding bodies, and representatives from neighbouring
technology and market domains, such as cognitive science, mechatronics, automotive, aerospace, security, computer vision,
embedded control systems.
This project’s ambition is to create sustainable solutions to all of the above-mentioned gaps, following a policy of targeted
stimulation of relevant grass-roots initiatives that both communities have already experimented with during the last couple of
years, but that have previously seen little success because of a lack of committed, professional and coordinated support. The
driver behind these stimulations will always be the robotics industry (since its needs for innovation and strong positioning in
the worldwide robotics market are greatest), but the academic research community will be heavily involved via a system of
flexible, targeted expert contributions whose short-term benefits are easy to identify and communicate.
The following activities are planned: improved industry-academia cooperation by giving more structure to commonly
organised events (administration, annual meetings, web portal on Robotics in Europe, advanced training, roadmapping,
and entrepreneurship advocacy) and by coordinated communication to the general public (press releases, visibility at major
robotics events worldwide, robotics competitions related to the shared research and development roadmap, layman’s
explanation of robotics technology in combination with semantic search support on the web portal).

---

Objective The development of flexible mobile manipulation systems is a promising area for the robotics industry as it allows to combine the success of manipulation robots with the flexibility of mobile robots. In the past, there has been a tremendous success in the areas of robotic manipulators and mobile robots. Many industrial processes highly depend on the reliability and robustness of robotic manipulators. On the other hand, research on mobile robots has led to systems that demonstrated the capability of safe and accurate navigation. The goal of this project is to integrate these two areas in the context of a real-world application scenario to build the basis for a new generation of autonomous mobile manipulation robots that can flexibly be instructed to perform complex manipulation and transportation tasks. The project will develop a novel robot programming environment that allows even non-expert users to specify complex manipulation tasks in real-world environments. In addition to a task specification language, the environment includes concepts for probabilistic inference and for learning manipulation skills from demonstration and from experience. The project will build upon and extend recent results in robot programming, navigation, manipulation, perception, learning by instruction, and statistical relational learning to develop advanced technology for mobile manipulation robots that can flexibly be instructed even by non-expert users to perform challenging manipulation tasks in real-world environments.The project results will be evaluated in a real-world setting involving mobile manipulation platforms built from state-of-the-art components and controlled by a fully integrated software system containing all developed components. The integrated system will, starting from a task specification, be capable of acquiring necessary low-level manipulation skills, imitating mobile manipulation behaviors demonstrated by a human it interacts with and, most importantly, will be able to generalize over such demonstrated behaviors to autonomously solve other tasks.

---

Objective In order to work naturally in human environments robots of the future will need to be much more flexible and robust in the face of novelty than those of today. In GeRT we will develop new methods to cope with novelty in manipulation tasks. Humans cope seamlessly with novel objects: we do not think of grasping a new cup, or screwing the lid off a jar we haven't seen before as challenging. This kind of everyday novelty is hard for a robot. The most advanced robots can perform a task such as making a drink, which involves grasping, pouring and twisting off a cap from a jar. But the rules must be precisely programmed. The ability to manipulate fifty different objects means writing fifty different programs. When the robot encounters an object it hasn't seen before, it can't grasp the object. If one object in a task changes then the program for the whole task may need to be rewritten. If we substitute a mug for a glass in a task that involved pouring liquid from the mug or glass into another object, the pouring position changes, as changes the grasping position. Perhaps we would grasp the mug by the handle, and then tip it sideways to pour. All of this means that if robots are ever going to be useful in natural settings where manipulation is involved that they need ways of generalising on the fly to cope with novel objects, and perhaps novel tasks. Our approach is to take a small set of existing robot programs, for a certain robot manipulation task, such as serving a drink and to give the robot the ability to adapt them to a novel version of the task. These programs constitute a database of prototypes representing that class of task. When confronted with a novel instance of the same task the robot needs establishing correspondences between objects and actions in the prototypes and their counterparts in the novel scenario. To achieve that, GeRT will use a variety of techniques from machine perception, machine learning and Artificial Intelligence such as automated planning.

---

Objective Humanoids expected to collaborate with people should be able to interact with them in the most natural way. This involves significant perceptual, communication, and motor processes, operating in a coordinated fashion. Consider a social gathering scenario where a humanoid is expected to possess certain social skills. It should be able to explore a populated space, to localize people and to determine their status, to decide to join one or two persons, to synthetize appropriate behavior, and to engage in dialog with them. Humans appear to solve these tasks routinely by integrating the often complementary information provided by multi sensory data processing, from low-level 3D object positioning to high-level gesture recognition and dialog handling. Understanding the world from unrestricted sensorial data, recognizing people's intentions and behaving like them are extremely challenging problems. The objective of HUMAVIPS is to endow humanoid robots with audiovisual (AV) abilities: exploration, recognition, and interaction, such that they exhibit adequate behavior when dealing with a group of people. Proposed research and technological developments will emphasize the role played by multimodal perception within principled models of human-robot interaction and of humanoid behavior. An adequate architecture will implement auditory and visual skills onto a fully programmable humanoid robot. An open-source software platform will be developed to foster dissemination and to ensure exploitation beyond the lifetime of the project.

---

Objective The inability to cope with abstract commands confines today s robots to very constrained, well-controlled environments. To overcome these limitations, a-priori knowledge - preprogrammed or learned - is required. However, as objectives and situations may radically change over time there will always be knowledge gaps. Even if provided with sophisticated cognitive capabilities, sufficient information will not always be available in the environment to fill these gaps. Humans, however, are a rich source of information to be utilized, e. g. by asking for directions. Access to this source provides robots with a powerful means to improve its adaptability and cope with new situations as they arise. The IURO project explores the integration of information retrieval from humans into robot control architectures to complement their perception and action control capabilities. IURO follows a multi-disciplinary approach combining environment perception, communication, navigation, knowledge representation and assessment, and human factors studies as well as a novel robot platform for human-centred urban environments as a pre-industrial development. IURO focuses on several main aspects, which address core objectives of the call: perception and appropriate representation of dynamic urban environments, identification of knowledge gaps arising from dynamically changing situations and contexts not specified a priori, retrieval of missing information from humans in a natural way by pro-actively approaching them and initiating communication situations. Quantitative and comparative benchmark measures considering flexibility and robustness regarding navigation and interaction capabilities in real-world scenarios are developed. IURO targets novel technologies for new commercial service robots with improved flexibility and dependability. The project consortium combines research in environmental and human action perception, human-robot interaction and communication aspects, navigation and planning, knowledge representation and assessment, and human factors studies as well as robot platform development.

---

Objective We propose research and development of a Knowledgeable Service Robot for Aging (KSERA) that will serve several related purposes for elderly persons in general and those with pulmonary disease in particular. Specifically KSERA provides (1) a mobile assistant to follow and monitor the health and behavior of a senior, (2) useful communication (video, internet) services including needed alerts to caregivers and emergency personnel, and (3) a robot integrated with smart household technology to monitor the environment and advise the senior or caregivers of anomalous or dangerous situations. KSERA aims at an adaptive technical aid that will provide needed and useful services in a pleasant, easy-to-use format via a robot that also acts as a companion and assistant. The problems to be addressed by the research and field trials include: (1) robot mobile behavior, i.e. machine navigation and following a target person through a variable and cluttered environment, (2) ubiquitous monitoring of physiological and behavioral data through direct measurements and interaction with household sensors, and (3) human-robot interaction including new developments in shared environmental processing, affective technology, and adaptable multimodal interfaces. A single robotic device hosting entertainment and communication aids, and at the same time providing an assistant that monitors the environment and the user's behavior, contributes to the user's health and quality of life (QoL). It will be designed to use contextual information and adaptive decision making algorithms to continually update the monitoring and mobile behavior for improved interaction with its user and to provide information and support at the right time and place. Real user scenarios and participative design drive the research. Two prototypes will be developed and validated in real end-user environments by qualitative and quantitative validation metrics including measures of safety, user acceptance, care efficiency and QoL.

---

Objective "The European automotive industry and their component manufacturers are facing the biggest shift in their history. The transition from combustion engines to electric drives (e-vehicle) requires pro-duction facilities that can initially deal with low and varying production volumes and can quickly be up-scaled to large numbers at need. LOCOBOT provides a solution to this problem by developing a flexible robotic assistant platform to support manual production processes and increase the produc-tivity and precision of such tasks. LOCOBOT does not only include the robot itself but also the en-gineering tools that are required for quickly building the robot, setting up its control structure and defining its tasks. Facing the demographic change, a further goal of LOCOBOT is the improvement of ergonomics in industrial production processes. A group of key players in the automotive industry, in automation components, advanced robots and engineering software will be supported by a group of excellent researchers to solve the technical and scientific challenges in LOCOBOT. The results will be demonstrated by setting up 3 typical and highly relevant use cases in a pilot production line of Audi AG. The automotive industry will benefit from LOCOBOT by having a robot assistant that can be quick-ly reconfigured in terms of its kinematic structure as well as its tasks. This allows them to make manual production processes much more efficient, to quickly up-scale a production process if re-quired and to improve working conditions by reducing the need to lift heavy objects (such as wheel hub drives for the e-vehicle). This will enable the industry to achieve a leading position in the the e-vehicle market and to keep up with the expected customer demand. The immediate impact (2-5 years) of LOCOBOT will be about 150 MEur in savings due to increased flexibility and efficiency, and will be 10 times as much in the following years depending on how the production numbers of the e-vehicle evolve."

---

The priority objective of the MOW-BY-SAT project is to support the development of a GNSS based navigation and guidance system to be integrated into an autonomous lawnmower, paving the way for industrialisation and commercialisation of GNSS applications oriented to domestic service robot, operating outdoor. Up to now, GNSS technologies are not much used in robotic applications. Beyond this concrete application the project aims to increase the adoption of the GNSS technologies towards robotics application, in a relation M2MM (Machine to Mobile Machine). In this context the project will study the benefits of European GNSS (EGNOS and Galileo).

---

Objective The first autopilots in airplanes can be traced back to the beginning of the twentieth century. These devices greatly reduced the pilot's workload by taking over parts of the navigation. The success of autopilots in reducing navigational complexity and improving safety explains the recent interest to introduce navigational assistance in other transportation means as well. However, implementing robotic navigation correction on a large scale also represents a potential safety risk for its users. For example, some plane crashes have been attributed to the incorrect estimation by pilots of the state of the plane's automatic pilot, an effect known as mode confusion.RADHAR therefore proposes a novel framework to design human-aware adaptive autonomy that avoids mode confusion by embedding a thorough understanding of diver behaviour and estimated intention into the decision making. Through lifelong, unsupervised learning, the robot will fuse the inherently uncertain information from environment and driver perception sensors; autonomously estimate the user model and intention and calculate a human-friendly trajectory. Since human characteristics vary over time a continuous interaction between two learning systems will emerge, hence RADHAR: Robotic ADaptation to Humans Adapting to Robots. In order to apply this framework to realistic real-world scenarios, sensor models will be developed to build 3D models of the environment with estimation of dynamic obstacles' motion and terrain traversability. To verify driver model assumptions such as focus-of-attention, the driver's posture and facial expression will be estimated with a camera and a haptic interface. The framework will be demonstrated on a wheelchair platform that navigates in an everyday environment with everyday objects. Tests on various levels of autonomy can be performed easily and safely on wheelchairs. Evaluation will happen by a diverse and challenging population of wheelchair users who currently drive unsafely.

---

Objective "In the last two decades, a precise management of agricultural land has been made possible due to the availability of new technologies, including global positioning systems (GPS), geographic information systems (GIS), sensors, automation of agricultural machinery, and high resolution image sensing. As a result, the concept of Precision Agriculture has emerged as the management strategy that uses information technologies to collect and process data from multiple sources in order to facilitate decisions associated with crop production. Moreover, the EU’s sixth environmental action programme addresses the need to encourage farmers to change their use of plant protection products’ .RHEA is focused on the design, development, and testing of a new generation of automatic and robotic systems for both chemical and physical –mechanical and thermal– effective weed management focused on both agriculture and forestry, and covering a large variety of European products including agriculture wide row crops (processing tomato, maize, strawberry, sunflower and cotton), close row crops (winter wheat and winter barley) and forestry woody perennials (walnut trees, almond trees, olive groves and multipurpose open woodland).RHEA aims at diminishing the use of agricultural chemical inputs in a 75%, improving crop quality, health and safety for humans, and reducing production costs by means of sustainable crop management using a fleet of small, heterogeneous robots –ground and aerial– equipped with advanced sensors, enhanced end-effectors and improved decision control algorithms. RHEA can be considered as a cooperative robotic system, falling within an emerging area of research and technology with a large number of applications as reported by the FP6 Network of Excellence EURON, Special Interest Group on Cooperative Robotics, funded by the European Commission.RHEA will be a unique opportunity to gather a very large number of multidisciplinary research groups with adequate funds to accomplish an authentic step forward in applying precision agriculture techniques in a massive way.This consortium joints a number of multidisciplinary, experienced researchers capable of improving individual scientific knowledge, but a large cooperation project is demanded to sum up the individual efforts in a holistic manner. The success of RHEA could bring a new means of applying automatic systems to agriculture and forestry crops with an important impact in improving the economy and environment as well as in maintaining the sustainability of rural areas by launching new technological jobs."

---

Objective The RoboEarth-project exploits a new approach towards endowing robots with advanced perception and action capabilities, thus enabling robots to carrying out useful tasks autonomously. The core of the innovation involves a world-wide web-style database: RoboEarth. RoboEarth will allow robots to share any reusable knowledge independent from their hardware and configuration. When a robot starts performing a task, it is able to download available high-level knowledge on both task and environment; next, it can use and translate this knowledge to its hardware specifications and its configuration and will improve it by learning during the task. Finally, it will upload its knowledge to RoboEarth again. As a result of the proposed project, major innovations are expected in the fields of: 3D sensing, control strategies and learning. Next to these, RoboEarth will contribute to a more modular design of robotic systems; robot hardware with a hardware abstraction layer for RoboEarth will be enough to build a functional robot. A series of six demonstrators will show the contributions of the project. The Action Recipe demonstrator will show robots creating/uploading and downloading/executing action recipes (abstract behaviour or algorithm descriptions in the RoboEarth database) to and from RoboEarth. The Ask Meal demonstrator will present meal options to a patient in a hospital room. The Serve Drink demonstrator will integrate 3D-sensing in the task by presenting a drink to the patient. The Screw Cap demonstrator will add improvement of the task by learning to remove the screw cap from a bottle. The Family Visit demonstrator focuses on the integration of dynamic object tracking into RoboEarth. Finally, the A Week In The Hospital demonstrator will show that history data from RoboEarth will enable a robot to improve its performance.
The research and technological development the RoboEarth consortium proposes to undertake will pave the way for completely new markets.

---

Objective Footwear production is still mainly handcrafted. Currently, opposite to other manufacturing sectors like automotive, food or metal processing, robots are still out of Footwear industry: only technical shoe producers have introduced robots to assist in the injection moulding process but there are not other relevant applications in use. The introduction of robotics will contribute to overcome the complexity in the automation of the processes of this industry that accounts for some of the shortest production runs to be found. The main difficulties to achieve this goal are:
•The high number of products variants.
•Complex manufacturing process.
•Complex assembly process.
•Extensive labour demand on some processes.

To achieve this objective, a consortium composed by 4 Industrial companies, 4 Research centres and 2 Shoe manufacturers will research and develop:
•New manipulation strategies and devices for non-rigid parts that allowed grasping, handling and packaging of shoes without damaging them.
•Sensor based robot programming and controlling tools that will exploit the information coming from CAD systems and all sensors available, in particular visual sensors as the base for visual servoing, making possible easy to program and flexible robotic applications.
•Re-design of some shoe production processes to allow robot assisted manufacturing and assembly, in particular selective heating, visual inspection and packaging.

The consortium has identified a set of six operations in the shoe manufacturing process as the more suitable for short-medium term robotics introduction. They will be packed into three prototypes that will be scheduled through the 30 months duration of the project in such a way that, from early phases, the Footwear Industry may get aware of the potential applications and benefits of robotics in their sector.

This research activity is carried out in the framework of the ''Factories of the Future'' call promoted by the ''Cognitive Systems and Robotics'' ICT Challenge.

---

Objective This proposal addresses the development of technologies for patient safety in robotic surgery. We define patient safety metrics for surgical procedures and then develop methods that abide by safety requirements, formulated in terms of our metrics. We aim at demonstrating that a properly controlled robotic surgery carried out in accordance to our safety criteria can improve the level of patient safety currently achievable by traditional surgery. The main innovative aspects of this project are: 1-\tResearch driven by patient-safety requirements 2-\tEmphasis on methodological rigor: development of a methodology founded in evidence-based medicine 3-\tScope: the entire surgical workflow is considered for development and validation. SAFROS focuses on innovative development of methods for the following technologies: \tSoft organ modeling and calibration, considering patient pathologies and anatomical variants \t Simulation planning in deformable environments\tIntra-operative registration and workflow monitoring\t Robot modeling and performance monitoring\tSurgeon training\tOperator interface with integrated stereovision and haptics New methods are integrated and validated on two distinct surgical robots (MIRO and RAMS), with respect to two inherently different contexts (pancreatic and vascular surgery). We quantitatively validate the adherence of our methods to the safety criteria, using surgical phantoms and animals. By comparing across robots and surgical contexts we draw conclusions about the generality of our approach. The SAFROS consortium comprises:\tHospitals with worldwide reputation, which provide medical knowledge and can validate our approach\tEurope's leading research groups in telerobotics and surgical robotics \tInnovative companies, to develop new technologies for surgical simulators\tWorld Health Organization, with global expertise in patient safety surgical safety guidelines\tRenowned educational organizations, to innovate surgeon training

---

Objective The project focuses on the development and prototyping of remotely-controlled, semi-autonomous robotic solutions in domestic environments to support elderly people. In particular, the SRS project will demonstrate an innovative, practical and efficient system called "shadow robot" for personalised home care.

Most elderly people want to live in the familiar environment of their own residence for as long as possible. However, not many can live with their adult children and therefore, at some stage, often late in life, have to live alone. Studies show that some forms of home care are usually required as they advance in years.
SRS solutions are designed to enable a robot to act as a shadow of its controller. For example, elderly parents can have a robot as a shadow of their children or carers. In this case, adult children or carers can help them remotely and physically with tasks such as getting up or going to bed, doing the laundry and setting up ICT equipment etc. as if the children or carers were resident in the house. This objective will be realised through the following SRS innovations:
1) A new intent-based remote control mechanism to enable the robots to be tele-operated over a real-world communication network robustly. 2) An adaptive automation mechanism to enable a highly efficient task execution for remotely controlled service robots. 3) A new robotic self-learning mechanism to enable the robots to learn from their experience. 4) A safety-oriented framework derived through extensive usability and user acceptance studies that enable service robots to be effectively deployed into home care applications.
The shadow robot prototypes created with EU support in the SRS project will be tested at the "S.Maria Nascente" Centre in Milano and the IZA Care Center in San Sebastián. The final solution will be further developed by Hewlett-Packard and other industrial partners of the consortium for a worldwide market with significant potential and volume.

---

Objective Today's automation and logistics paradigms make it difficult, time consuming, and costly to change the type of the product manufactured and to scale the production up and down in response to market volatility. Consequently, and with the increasing market uncertainties, it becomes more and more difficult to justify new automation lines. To keep production in Europe instead of shifting it to low-wage countries, this project will break new ground in robot-based automation and logistics as the backbone of a transformable factory of the future, enabling an economic production regardless of changes in volumes and product type.TAPAS pioneers the following tasks in real production environments: mobile robots with manipulation arms will make logistic tasks more flexible and more complete by not only transporting, but also collecting the parts needed and delivering them right to the place were needed. Since moving parts around the shop floor does not create value by itself, TAPAS robots go even beyond: they will automate assistive tasks that naturally extend the logistic tasks, such as preparatory and post-processing works, e.g., pre-assembly or machine tending with inherent quality control. Through this additional creation of value and by a faster adaptation to changes with new levels of robustness, availability, and completeness of jobs TAPAS robots promise to yield an earlier return of investment.To reach the objectives, the TAPAS consortium will iteratively test and validate the developments with two pilot installations of increasing complexity and scale. The drivers behind TAPAS are a robot manufacturer and a system integrator, providing both their production environments for intensive testing and validation, and a software technology provider. Teaming up with three excellent research partners they will develop logistic and assistive robotic solutions for transformable automation that are generally applicable and scalable.

---

Objective The scientific goals of the proposal revolve around the reciprocal linkages between the physical hand and its high-level control functions, and about the way that the embodiment enables and determines its behaviours and cognitive functions. THE Hand Embodied refers to the hand as both a cognitive entity standing for the sense of active touch and as the physical embodiment of such sense, the organ, comprised of actuators and sensors that ultimately realize the link between perception and action. The study of the intrinsic relationship between the hand as a cognitive abstraction and its bodily instance will be made possible by: (a) performing neuroscientific and perceptual behavioural studies with participants engaged in controlled manual activities; and (b) the parallel development of a theoretical framework to lay the foundations for design and control of robotic hands and haptic interfaces. The general idea is to study how the embodied characteristics of the human hand and its sensors, the sensorimotor transformations, and the very constraints they impose, affect and determine the learning and control strategies we use for such fundamental cognitive functions as exploring, grasping and manipulating. The ultimate goal of the present proposal is to learn from human data and hypotheses-driven simulations how to devise improved system architectures for the hand as a cognitive organ, and eventually how to better design and control robot hands and haptic interfaces. The project hinges about the conceptual structure and the geometry of such enabling constraints, or synergies: correlations in redundant hand mobility (motor synergies), correlations in redundant cutaneous and kinaesthetic receptors readings (multi-cue integration), and overall sensorimotor system synergies. These are also our key ideas for advancing the state of the art in artificial systems for robotic manipulation and haptic and neuroprosthetic interfaces.

---

Objective The TRIDENT project is aimed at developing a prototype system (in the following named also system), integrating advanced technologies of aero-stereographic image acquisition, digital three-dimensional cartography restitution, data base, internet/intranet networking, in order to improve services provision to EU administrations and citizens. The end-product will be represented by a remotely accessible 3D reconstruction of a sample urban area extracted from three pilot sites (Madrid (Spain), Helsinki (Finland) and Rome (Italy)), correlated with relevant geo-referenced information, and by a set of specific applications exploiting these data to provide innovative IT services to the citizens and to the local administrations.OBJECTIVESA widely accessible 3D urban cartography represents an "enabling technology?" to achieve a better knowledge, management and control of the territory. Within the scope of the TRIDENT Project selected applications, exploiting this technology, will be addressed in response to specific needs coming from users directly involved in the project: Urban Development Planning (UDP); Urban Monitoring (UM); Telecommunication Services (TS). The project scope is limited to specific sample areas of Rome, Madrid and Helsinki, as TRIDENT purpose is not to build extensive data sets of the pilot sites but to demonstrate feasibility of an innovative technological approach, which is the basis for any further commercial exploitation of project results.DESCRIPTION OF WORKThe TRIDENT Project will be developed, in a two and half year period, performing the following activities: WP 1: Management; where the project management and co-ordination will be performed by the co-ordinator; WP 2: System; which includes the activities related to the specification and design of the Core System, and the activities related to the customisation, integration and test of the TRIDENT system; WP 3: Production Environment; which includes the development of the city data set acquisition and preparation; WP 4: Data Base Systems Integration, which includes the activities related to the preparation and management of the City Data Base, and develops the basic Internet data accessibility; WP 5: Urban Development Planning Application; which develops the specific application dedicated to the urban planning; WP 6: Telecomm Services Application; which develops the specific application related to the telecommunication services; WP 7: Urban Monitoring Application, which develops the specific application for urban monitoring; WP 8: Dissemination and Implementation; which performs the activity to inform the community on the results of the project, and the activities related to the possible exploitation of the TRIDENT system. Both in the Core System and in the Specific Applications the following main conceptual phases are considered: - Definition Phase; where all the general requirements will be assessed, and the system specification and preliminary design will be performed. - Design Phase; where the detailed design of the whole project will be done. - Development Phase; where the Core System and the Applications will be developed. - Customisation, Integration and Testing Phase; where each application will be integrated in the Core System. The whole system will be tested and the applications will be adapted to the other countries.

---

Objective The ACTIVE project exploits ICT and other engineering methods and technologies for the design and development of an integrated redundant robotic platform for neurosurgery. A light and agile redundant robotic cell with 20 degrees-of-freedom (DoFs) and an advanced processing unit for pre- and intra-operative control will operate both autonomously and cooperatively with surgical staff on the brain, a loosely structured environment. As the patient will not be considered rigidly fixed to the operating table and/or to the robot, the system will push the boundaries of the state of the art in the fields of robotics and control for the accuracy and bandwidth required by the challenging and complex surgical scenario.

Two cooperating robots will interact with the brain that will deform for the tool contact, blood pressure, breathing and deliquoration. Human factors are considered by allowing easy interaction with the users through a novel haptic interface for tele-manipulation and by a collaborative control mode ("hands-on"). Force and video feedback signals will be provided to surgeons. Active constraints will limit and direct tool tip position, force and speed preventing damage to eloquent areas, defined on realistic tissue models updated on-the-field through sensors information. The active constraints will be updated (displaced) in real time in response to the feedback from tool-tissue interactions and any additional constraints arising from a complex shared workspace. The overarching control architecture of ACTIVE will negotiate the requirements and references of the two slave robots.

The operative room represents the epitome of a dynamic and unstructured volatile environment, crowded with people and instruments. The workspace will thus be monitored by environmental cameras, and machine learning techniques will be used for the safe workspace sharing. Decisions about collision avoidance and downgrading to a safe state will be taken autonomously, the movement of the head of the patient will be filtered by a bespoke active head frame, while fast and unpredictable patient motion will be compensated by a real-time cooperative control system. Cognitive skills will help to identify the target location in the brain and constrain robotic motions by means of on-field observations.

---

Objective This ambitious project aims at creating a swarm of interacting, cognitive, autonomous robots. We will develop a swarm of autonomous underwater vehicles (AUVs) that are able to interact with each other and which can balance tasks (interactions between/within swarms). These tasks are: ecological monitoring, searching, maintaining, exploring and harvesting resources in underwater habitats. The swarm will maintain swarm integrity under conditions of dynamically changing environments and will therefore require robustness and flexibility. This will be achieved by letting the AUVs interact with each other and exchange information, resulting in a cognitive system that is aware of its environment, of local individual goals and threats and of global swarm-level goals and threats. Our consortium consists of both, biological and technical institutions and is therefore optimally qualified to achieve this goal.
By a combination of locally acting and globally acting self-organizing mechanisms, information from the global level flows into the local level and influences the behaviour of individual AUVs. Such a cognitive-based scheme creates a very fast reaction of the whole collective system when optimizing the global performance. As shown by natural swimming fish swarms, such mechanisms are also flexible and scalable. The usage of cognition-generating algorithms can even allow robotic swarms to mimic each other's behaviour and to learn from each other adequate reactions to environmental changes. In addition, we plan to investigate the emergence of artificial collective pre-consciousness, which leads to self-identification and further improvement of collective performance. In this way we explore several general principles of swarm-level cognition and can assess their importance in real-world applications. This can be exploited for improving the robustness, flexibility and efficiency of other technical applications in the field of ICT.

---

Objective As the introduction of robots into our daily life becomes a reality, the social compatibility of such robots gains importance. In order to meaningfully interact with humans, robots must develop an advanced real-world social intelligence that includes novel perceptual, behavioural, emotional, motivational and cognitive capabilities. The Experimental Functional Android Assistant (EFAA) project will contribute to the development of socially intelligent humanoids by advancing the state of the art in both single human-like social capabilities and in their integration in a consistent architecture.
The EFAA project proposes a biomimetic, brain-inspired approach. The central assumption of EFAA is that social robots must develop a sense of self as to overcome the fundamental problem of social inference. It is only in possessing the core aspects of a human-like self, that inferences about others can be made through analogy. The EFAA Biomimetic Architecture for Situated Social Intelligence Systems, called BASSIS, is based on our growing understanding of the neuronal mechanisms and psychological processes underlying social perception, cognition and action and will exploit the availability, amongst the members of the consortium, of a number of complementary prototype robot-based perceptual, cognitive and motor architectures. By integrating across these existing architectures, by directing focused effort on specific core problems, and by exploiting the availability of unique advanced real-time neuronal simulation and hardware, the impact of the EFAA project is assured.
The EFAA project will apply and benchmark BASSIS on a mobile humanoid assistant based on the iCub platform. The resultant system, EFAA, will actively engage in social interactions and interactive cognitive tasks. To facilitate the realization of these game like interactions and the performance analysis will be facilitated through a mixed-reality interaction paradigm using a table-top tangible interface system.

---

Objective The EMICAB consortium takes a holistic approach to the engineering of artificial cognitive systems. Our goal is to integrate smart body mechanics in intelligent planning and control of motor behaviour. To achieve this goal the consortium accounts equally for problems in neuroscience (e.g., multi-sensory integration, internal body models, intelligent action planning) and technology (smart body mechanics, distributed embodied sensors and brain-like controllers). Our approach starts with a strongly sensorised bionic body with redundant whole-body kinematics and then designs the technological infrastructure such that cognitive mechansims emerge from distributed sensorimotor intelligence. The concept is based on neuroscience research on insects whose motor dexterity, adaptiveness and pre-rational abilities in learning and memory rival those of lower mammals: stick insects orchestrate a wide range of dexterous motor behaviours and flies can maintain object locations in short-term memory during navigation tasks, just to mention paradigms that are studied by UNIBI and JGUM. The partners UNICT and SDU will devise bio-inspired models and, in turn, guide ongoing experimental research in order to achieve the overall technological goal: a dexterous hexapod robot that exploits its bodily resources for cognitive functions. Two levels of analysis and modelling will be accounted for: the smart brain that captures various aspects of motion intelligence (motor learning, context-dependent actions, multi-sensory integration) and the smart body equipped with distributed proprioceptors and muscle-like compliance, allowing for novel, highly adaptive, neurobionic control strategies. The EMICAB robot will draw from its complex body features and learn by use of a manipulable internal body model. This will be monitored by an ambitious set of benchmarking scenarios. We expect mutual benefit for applied research on autonomous mobile robots and for basic research in neuroscience.

---

Objective IntellAct addresses the problem of understanding and exploiting the meaning (semantics) of manipulations in terms of objects, actions and their consequences for reproducing human actions with machines. This is in particular required for the interaction between humans and robots in which the robot has to understand the human action and then to transfer it to its own embodiment. IntellAct will provide means to allow for this transfer not by copying movements of the human but by transferring the human action on a semantic level. IntellAct will demonstrate the ability to understand scene and action semantics and to execute actions with a robot in two domains. First, in a laboratory environment (exemplified by a lab in the International Space Station (ISS)) and second, in an assembly process in an industrial context.
IntellAct consists of three building blocks: (1) Learning: Abstract, semantic descriptions of manipulations are extracted from video sequences showing a human demonstrating the manipulations; (2) Monitoring: In the second step, observed manipulations are evaluated against the learned, semantic models; (3) Execution: Based on learned, semantic models, equivalent manipulations are executed by a robot.

The analysis of low-level observation data for semantic content (Learning) and the synthesis of concrete behaviour (Execution) constitute the major scientific challenge of IntellAct.
Based on the semantic interpretation and description and enhanced with low-level trajectory data for grounding, two major application areas are addressed by IntellAct: First, the monitoring of human manipulations for correctness (e.g., for training or in high-risk scenarios) and second, the efficient teaching of cognitive robots to perform manipulations in a wide variety of applications.

To achieve these goals, IntellAct brings together recent methods for (1) parsing scenes into spatio-temporal graphs and so-called „semantic Event Chains‟, (2) probabilistic models of objects and their manipulation, (3) probabilistic rule learning, and (4) dynamic motion primitives for trainable and flexible descriptions of robotic motor behaviour. Its implementation employs a concurrent-engineering approach that includes virtual-reality-enhanced simulation as well as physical robots. Its goal culminates in the demonstration of a robot understanding, monitoring and reproducing human action.

---

Objective The JAMES project ("Joint Action for Multimodal Embodied Social Systems") aims to develop a socially intelligent humanoid robot combining efficient task-based behaviour with the ability to understand and respond in a socially appropriate manner to a wide range of multimodal communicative signals in the context of realistic, open-ended, multi-party interactions.

To direct our research in JAMES, we will focus on five core objectives: (1) analysing natural human communicative signals, (2) building a model of social interaction, (3) extending the model to manage learning and uncertainty, (4) implementing the model on a physical robot platform, and (5) evaluating the implemented system.

The work in JAMES will build on state-of-the-art results and techniques in seven areas: social robotics, social signal processing, machine learning, multimodal data collection, planning and reasoning, visual processing, and natural language interaction.

JAMES will combine the analysis of human social communicative behaviour, the development and integration of state-of-the-art technical components, and the evaluation of integrated systems. Work on these threads will be interleaved: the results of the human data analysis will be used in the development of the technical components, while the robot will be used for further data collection and evaluation studies.

JAMES will extend the state-of-the-art in social robotics by moving beyond one-on-one, long-term relationships to deal with more open-ended, multi-party, short-term situations. The research in JAMES will also increase our understanding of how humans use multimodal social cues to communicate and coordinate their interactions in task-driven, joint-action contexts. The individual technical contributions to the system components will also provide
state-of-the-art results in their respective research areas.

---

Objective Endowing robots with cognition is a long-standing and difficult objective. Substantial progress in cognitive science and neuroscience has led to the insight that cognition is tightly linked to the sensory and motor surfaces, and that cognition emerges during development from relatively low-level mechanisms when situated in a structured environment.
Building on basic functions such as detection and selection, we will develop a set of elements of cognition and techniques for combining such elements, allowing us to scale towards such cognitive capabilities as scene representation and sequence generation. We will implement and evaluate these elements of cognition in scenarios inspired by the development of cognition in early childhood.

---

Objective Globalization causes an increasing transport of goods. Nowadays, most goods are shipped in containers and are transferred onto trucks for further transport. The containers are unloaded manually since they are nearly always packed chaotically, the variety of transported goods is high, and time requirements are strict. Unloading of containers is a strenuous task as goods have a weight up to 70 kg that poses health risks, which include the effects of pesticides and poisonous gases as well as injuries through unexpectedly falling objects. Human labor is hence a high cost factor combined with unhealthy working conditions, making automated solutions highly desirable. Existing systems for automated unloading are restricted to specific scenarios and still have drawbacks in their flexibility, adaptability and robustness. A robotic system suited for any unloading task of containers requires a high amount of cognitive capabilities. RobLog aims at developing appropriate methods and technologies meeting the requirements to automate logistics processes.The RobLog system has to be capable of 3D perception in a challenging scenario (high variability of objects, dynamic scene, deformable objects). The perceived environment has to be integrated into a 3D model in real-time. Grasping hypotheses, decisions and path-plans have to be generated and executed in an adaptive manner including obstacle avoidance and re-planning, if necessary. The actions must be grounded in a physical set-up capable of handling a large variety of potentially deformable items. Finally, there is the need to provide an interface suited for a human operator to provide high-level instructions to a multitude of systems operating at several unloading docks in parallel. All these advances are demonstrated within the project in close cooperation with an industrial end-user in a realistic application scenario; thus opening the potential to reach a completely new level of automation in the logistics chain.

---

Objective The goal of the proposed research is to enable a generational leap in the techniques and scalability of motion synthesis systems. Motion synthesis is a key component of future robotic and cognitive systems to enable their physical interaction with humans and physical manipulation of their environment. Existing motion synthesis algorithms are severely limited in their ability to cope with real-world objects such as flexible objects or objects with many degrees of freedom. The high dimensionality of the state and action space of such objects defies existing methods for perception, control and planning and leads to poor generalisability of solutions in such domains. These limitations are a core obstacle of current robotic research.We propose to solve these problems by learning and exploiting appropriate topological representations and testing them on challenging domains of flexible, multi-object manipulation and close contact robot control and computer animation. Topological representations describe motion in terms of more abstract, more appropriate, and better generalizing features: for instance, an embracing motion can better be described, controlled and planned in coordinates quantifying the `wrappedness' of arms or fingers around the object (as opposed to joint angle coordinates). Such topological representations exist on different levels of abstraction and reduce the dimensionality of the state and action spaces. This proposal investigates existing topological metrics (similar to the mentioned `wrappedness') and uses data driven methods to discover new mappings that capture key invariances. Given topological representations, we will develop methods for sensing, control and planning using on these representations.This proposal, for the first time, aims to achieve this at all the three levels of sensing, representation and action generation -- by developing novel object-action representations for sensing based on manipulation manifolds and refining metamorphic manipulator design in a complete cycle. The methods and hardware developed will be tested on challenging real world robotic manipulation problems ranging from domains with many rigid objects to articulated carton folding or origami and all the way to full body humanoid interactions with flexible objects. The results of this project provide the necessary key technologies for future robots and computer vision systems to enable fluent interaction with their environment -- as well as provide answers to the basic scientific question of the `right' representation in sensorimotor control.

---

Objective Current research in embodied cognition builds on the idea that physical interaction with and exploration of the world allows an agent to acquire intrinsically grounded, cognitive representations which are better adapted to guiding behavior than human crafted rules. Exploration and discriminative learning, however, are relatively slow processes. Humans are able to rapidly create new concepts and react to unanticipated situations using their experience. They use generative mechanisms, like imagining and internal simulation, based on prior knowledge to predict the immediate future. Such generative mechanisms increase both the bandwidth and speed of cognitive development, however, current artificial cognitive systems do not yet use generative mechanisms in this way.The Xperience project addresses this problem by structural bootstrapping, an idea taken from language acquisition research: knowledge about grammar allows a child to infer the meaning of an unknown word from its grammatical role together with understood remainder of the sentence. Structural bootstrapping generalizes this idea for general cognitive learning: if you know the structure of a process the role of unknown actions and entities can be inferred from their location and use in the process. This approach will enable rapid generalization and allow agents to communicate effectively.Xperience will implement a complete robot system for automating introspective, predictive, and interactive understanding of actions and dynamic situations based on structural bootstrapping. Xperience will evaluate and benchmark this on state-of-the-art humanoid robots demonstrating rich interactions with humans. By equipping embodied artificial agents with the means to exploit prior experience via generative inner models, XPERIENCE will have a major impact in a wide range of autonomous robotics applications that benefit from efficient learning through exploration, predictive reasoning and external guidance.

---

Objective The goal of the AIRobots project is to develop a new generation of aerial service robots capable to support human beings in all those activities which require the ability to interact actively and safely with environments not constrained on ground but, indeed, freely in air. The step forward with respect to the classical field of aerial robotics is to realize aerial vehicles able to accomplish a large variety of applications, such as inspection of buildings and large infrastructures, sample picking, aerial remote manipulation, etc.The starting point is an aerial platform whose aeromechanical configuration allows the vehicle to interact with the environment in a non-destructive way and to hover close to operating points. Rotary-wing aerial vehicles with shrouded propellers represent the basic airframes which will be then equipped with appropriate robotic end-effectors and sensors in order to transform the aerial platform into an aerial service robot, a system able to fly and to achieve robotic tasks. Advanced automatic control algorithms will be conceived to govern the aerial platform which will be remotely supervised by the operator with the use of haptic devices. Particular emphasis will be given to develop advanced human-in-the-loop and autonomous navigation control strategies relying upon a cooperative and adaptive interaction between the on-board automatic control and the remote operator. Force and visual feedback strategies will be investigated in order to transform the aerial platform in a flying hand suitable for aerial manipulation.The consortium is composed of four academic groups and an SME with the role of end-user and evaluator of the project outcomes. Prototypes of aerial service robots will be developed and tested on experimental setups which will be constructed in order to reproduce typical industrial scenarios for which aerial inspection robotics can be beneficial (docking, cleaning, inspection and repairing of infrastructures, payload lifting, etc.).

---

Objective The goal of ALIZ-E is to develop methods for developing and testing interactive, mobile robots which will be able to interact with human users over extended periods of time, i.e. a possibly non-continuous succession of interactions which can refer back to, and build forth on, previous experiences.To achieve this aim, ALIZ-E will address three related issues in developing interactive robots capable of self-sustaining medium- to long-term autonomous operation in real-world indoor environments. One, ALIZ-E will address how long-term experience can be acquired, to ground actions and interactions across time. Two, ALIZ-E will address how a system can deal robustly with inevitable differences in quality in perceiving and understanding a user and her environment. To this end, novel methods for adaptively controlling how a system invokes and adaptively balances a hybrid ensemble of processing and behaviours. Third, ALIZ-E will address how a system can adapt its interaction based on how user behaviour changes over time and contexts.To demonstrate and evaluate scientific methods, ALIZ-E will instantiate and evaluate these methods in working systems that interact with hospitalized children undergoing diabetes treatment. Long-term interaction in this context means interactions over a period of up to 5 days (possibly longer). Choosing this scenario, ALIZ-E makes it possible to bring existing extensive experience in conducting clinical trials of IT technology to the field of cognitive systems and human-robot interaction, to help develop novel methods for evaluating interactive robots at system-level.The theory and practice of ALIZ-E will impact on theoretical cognitive systems research (eg. memory, long-term affective interaction), implementation (eg. adaptive deployment of processing and behaviour for robust interaction, cloud computing for cognitive systems, speech processing for young users) and commercial applications of these technologies.

---

Objective The DIPLECS project aims at designing an Artificial Cognitive System architecture that allows for learning and adapting hierarchical perception-action cycles in dynamic and interactive real-world scenarios. The architectural progress will be evaluated within the scenario of a driver assistance system that continuously improves its capabilities by observing the human driver, the car data, and the environment.
The system is expected to emulate and predict the behaviour of the driver, to extract and analyse relevant information from the environment, and to predict the future state of the car in relation to its context in the world. Starting from a rudimentary, pre-specified, i.e., man-modelled system, the architecture is expected to successively replace manually modelled knowledge with learned models, thus improving robustness and flexibility. Bootstrapping and learning is applied at all levels, in a dynamic and interactive context.
Dynamic and interactive context means that the system needs to react at any time to any relevant event and that the action comprises communication to the human driver or direct car control. The architecture applies a hierarchical design principle, where adjacent levels are connected by feedback-loops that require time for processing. Therefore, the potential reaction becomes more advanced through time, i.e., the system provides nested strategies. A real-time operation requires feed-forward mappings, which use the information learned in feedback operation.
The developed methods will be evaluated in three different settings: off-line with data recorded in a real vehicle, online in the real vehicle, and online for a model car. The first setting allows for evaluating methods that take into account the dynamics of the environment, but which are not real-time capable. The second setting allows for testing of passive assistance capabilities by communicating real-time information. The third setting allows for testing of active capabilities.

---

Objective This project aims at developing new machine learning methods relying on very large number of hand-designed heuristics, together with statistical tools to facilitate the design of these heuristics in an open and collaborative framework.We define an heuristic to be any algorithm processing raw inputs to produce values relevant to the problem at hand. This purposely very general definition encompasses techniques spanning from simple signal processing to symbolic modeling or locally trained predictors. Since we assume high performance can only be achieved by combining hundreds of such heuristics, we propose to develop them collaboratively, in a way similar to the successful development process of open-source software or collaborative encyclopedia.We will assess the performance of that strategy on the control of an avatar in a realistic 3D simulator and on the control of a real robotic arm, and we aim at creating a generic software platform usable on alternative applications.Hence, the key aspects of this proposal are to:- develop novel statistical techniques for prediction and goal-planning with a very large heterogeneous set of features,- develop statistical tools such as similarity measures in the space of features to help the design of very large sets of heuristics by many contributors,- assess the efficiency of this approach on a series of complex tasks in a realistic simulated 3D environment and with a real robot arm.The five partners of the consortium are from the fields of applied and theoretical statistical learning, reinforcement learning, artificial vision and robotics.

---

Objective NIFTi puts the human factor into cognitive architectures. NIFTi investigates how natural behavior in human-robot cooperation can arise. NIFTi operationalizes natural cooperation as balancing operational and cooperation demands in a cognitive architecture (CA), to minimize human cognitive task load and optimize joint work flow.
NIFTi designs CAs by closely coupling cognitive user models to how the architecture understands the environment, how it performs actions, how it communicates. The architecture acquires maps which combine perceptual- and conceptual information. These maps capture where what is in the environment, and project this to how acting is affected. The CA combines projections with cognitive user models and plans to predict why changes in human behavior (due to attention, task load) may occur. The CA uses these predictions to anticipate how it should adapt acting and communication to align with the human. The CA bases planning and execution in a cognitive control model. Control uses attention, characteristics of agent morphology, and skill acquisition, to guide autonomous action execution in a cooperative context. The CA achieves balance by actively interconnecting content across modules. Content in modules is not isolated. In the NIFTi CA design, controllers interconnect content across modules, percolating content changes throughout the CA. Changes guide processing in a module; interconnectivity ensures behavior changes coherently system-wide. Controllers are learnt off- and online, using reinforcement learning and statistical (relational) learning.
Adapting to a human thus permeates the entire architecture. NIFTi focuses on cooperation in the USAR domain, to restrict what actions, forms of communication, and user aspects need to be taken into account. USAR end users join NIFTi to yearly evaluate its approach in real settings, using an integrated CA on a new robot with adaptive active/passive locomotion to jointly explore a disaster area.

---

Objective Current multi-AUV systems are far from being capable of fully autonomously taking over real-life complex situation-awareness operations. As such operations require advanced reasoning and decision-making abilities the current designs have to heavily rely on human operators. The involvement of humans, however, is by no means a guarantee of performance; humans can easily be overwhelmed by the information overload, fatigue can act detrimentally to their performance, properly coordinating vehicles actions is hard, and continuous operation is all but impossible. Within NOPTILUS we take the view that an effective fully-autonomous multi-AUV concept/system, is capable of overcoming these shortcomings, by replacing human-operated operations by a fully autonomous one. To successfully attain such an objective, significant advances are required, involving cooperative & cognitive-based communications and sonars (low level), Gaussian Process-based estimation as well as perceptual sensory-motor and learning motion control (medium level), and learning/cognitive-based situation understanding and motion strategies (high level).Of paramount importance is the integration of all these advances and the demonstration of the NOPTILUS system in a realistic environment at the Port of Leixões, utilizing a team of 6 AUVs that will be operating continuously on a 24hours/7days-a-week basis. As part of this demonstration another important aspect of the NOPTILUS system that of (near-) optimality will be shown. Evaluation of the performance of the overall NOPTILUS system will be performed with emphasis on its robustness, dependability, adaptability and flexibility especially when it deals with completely unknown underwater environments and situations never taught before as well as its ability to provide with arbitrarily-close-to-the-optimal performance.

---

Objective Reproducing an act with sensorimotor means and using fine natural language for communicating the intentionality behind the act is what Aristotle called "Poetics". POETICON explores the "poetics of everyday life", i.e. the synthesis of sensorimotor representations and natural language in everyday human interaction. This is related to an old problem in Artificial Intelligence on how meaning emerges, which is approached here in a new way. POETICON follows an empirical approach for discovering the "languages" of sensorimotor representations and the correspondences with natural language; guided by experiments in psychology and neuroscience, it employs cutting-edge equipment and established cognitive protocols for collecting face and body movement measurements, visual object information and associated linguistic descriptions from interacting human subjects, with a two-fold objective:
a)\tThe creation of the PRAXICON, an extensible computational resource which associates symbolic representations (words/concepts) with corresponding sensorimotor representations and that is enriched with information on patterns among these representations for forming conceptual structures.
b)\tThe exploration of the association of symbolic and sensorimotor representations through cognitive and neurophysiological experiments and experimentation with a humanoid as driving forces and implementation tools for the development of the PRAXICON, respectively.
POETICON views a cognitive system as a set of different languages (the spoken, the motor, the vision language and so on) and provides a set of tools for parsing, generating and translating among them. Through inter-disciplinary research, it contributes to the exploration of what integration in human cognition is and how it can be reproduced by intelligent agents. This is an ambitious first step for revealing and conquering the "poetics of everyday life".

---

Objective SCOVIS will investigate weakly supervised learning algorithms and self-adaptation strategies for analysis of visually observable procedures. SCOVIS research directly affects ease of deployment and minimises effort of operation of monitoring systems and is unique in the sense that it links object learning using low-level object descriptors and procedure learning with adaptation mechanisms and active camera network coordination. SCOVIS advocates a synergistic approach that combines largely unsupervised learning and model evolution in a bootstrapping process; it involves continuous learning from visual content in order to enrich the models and, inversely, the direct use of these models to enhance the extraction. In the SCOVIS application scenario user interaction will be significantly reduced compared to current methods. The system will be able to calculate the camera spatial relations automatically (self-configuration) for coupled, uncoupled and active cameras. The user will define a set of objects and procedures of interest during a very short supervised learning phase, while the associations with low-level descriptors will be automatically learnt. The resulting models will be significantly enhanced through online data acquisition and unsupervised learning (adaptation). The enhanced models will be able to be verified and potentially adapted through relevance feedback. The main measurable objective of SCOVIS will be to significantly improve the versatility and the performance of current monitoring systems. The resulting technology will enable the easy installation of intelligent supervision systems, which has not been possible so far, due to the prohibitively high manual effort and the inability to model complex visual processes. The produced technology will be evaluated through realistic scenarios related to industry and public infrastructure. The proposed research will be performed with absolute respect to privacy and personal data of monitored individuals.

---

"Please note that the datasets are the only one in the world from industrial environments (NISSAN in Barcelona). So please provide this capability too."

---

Objective TACO develops a 3D sensing system with real 3D foveation properties endowing service robots with a higher level of motion and affordance perception and interaction capabilities with respect to everyday objects and environments. By 3D foveation properties we mean properties based on the process of acquiring 3D images with coarse level of details, applying fast object recognition techniques to identify areas of interest in the coarse 3D image and then concentrate the image acquisition on details of interest allowing for higher resolution 3D sampling of these details. This new 3D foveation concept will allow robots to interact with everyday environments in a more natural and human-like manner, increasing the level of detail whenever needed for interaction between the robot and everyday objects and humans. These 3D foveation properties are achieved by utilising the power of micro-mirror MEMS technology combined with state-of-the-art time-of-flight methods to ensure a system that is small, light-weight and easily mounted on an ordinary-sized service robot or even a robot arm. The project will explore control strategies for 3D foveation allowing 3D robot vision that is adaptable with space- and time-variant sampling, processing and understanding. The project will verify and test the 3D sensing system in a robotic environment, exploring the capabilities of the system to allow the robot to navigate autonomously and interact with a diverse number of everyday objects. The TACO consortium has RTD partners and industrial end users: Fraunhofer IPMS (micro-mirror scanning device), Fraunhofer IPM (3D range sensors), CTR (Electronics), SINTEF (3D foveation software), TU Wien (benchmarking with state-of-the-art 3D sensing methods), Shadow Robotics (application towards robot grippers) and Oxford Technologies (application towards robots for harsh environments)

---

Objective The aim of VANAHEIM is to study innovative surveillance components for autonomous monitoring of complex audio/video surveillance infrastructure, such as the ones prevalent in shopping malls or underground stations. To do so, VANAHEIM addresses three main application-driven research questions: 1.Scene activity modelling algorithms for automatic sensor selection in control room. In everyday practice , surveillance video wall monitors frequently show empty scenes, while there are obviously many cameras looking at scenes in which something (even normal) is happening. Performing a sensor selection at the control room level to autonomously select the streams to display therefore seems required. While this scenario is trivial when dealing with empty vs occupied scenes, building models to characterise the streams content, in terms of usual and unusual activities, turns out to be necessary when dealing with almost all occupied scenes. Furthermore, the need for such selection is even more explicit when dealing with audio streams, for which mosaicing of data is not possible due to the transparent nature of sound. VANAHEIM thus targets the development of such automatic audio/video components allowing to select the audio/video streams to present to operators in control room.2.Investigation of behavioural cues for human-centred monitoring and reporting. VANAHEIM investigates the use of subtle human behavioural cues (head pose, body shape) and social models (e.g. about space occupancy) to perform the live detection of well-defined scenarios of interest. In addition, the project also targets three specific levels of monitoring: 1. individuals, 2. groups of people and 3. crowd/people flow. Last, VANAHEIM targets the development of a situational awareness reporting, which aims at translating the ongoing activities of people into meaningful user-oriented figures, through for example a map-based overlay of the approximate location/number/behaviour of people in the infrastructure.3.Collective behaviour building and online learning from long-term analysis of passenger activities. By combining cognitive science and ethological analysis, VANAHEIM aims at designing models for the identification and characterization of the structures inherent in collective human behaviour. In other words, by continuously analyzing, learning and clustering information about users' locations, routes, activities, interactions with others passengers and/or equipments, and contextual data (time of day, density of people...), VANAHEIM targets a subsystem able to estimate of the long-term trends of large-scale human behaviour, thus allowing the discovery of collective comprehensive daily routines.The evaluation/assessment of the sub-systems developed within the project is covered thanks to the participation of Turin and Paris transport operators; VANAHEIM deployment in both sites will allow to demonstrate the scalability, as well as the performance of the developed system.

---

Objective The project V-Charge is based on the vision that, due to required drastic decrease of CO2 production and energy consumption, mobility will undergo important changes in the years to come. This includes new concept for an optimal combination of public and individual transportation as well as the introduction of electrical cars that need coordinated recharging. A typical scenario of such a concept might be automatic drop-off and recovery of a car in front of a train station without taking care of parking or re-charging. Such new mobility concepts require among other technologies autonomous driving in designated areas. The objective of this project is to develop a smart car system that allows for autonomous driving in designated areas (e.g. valet parking, park and ride) and can offer advanced driver support in urban environments. The final goal in four years is the demonstration and implementation of a fully operational future car system including autonomous local transportation, valet parking and battery charging on the campus of ETH Zurich and TU Braunschweig. The envisioned key contribution is the development safe and fully autonomous driving in city-like environments using only low-cost GPS, camera images, ultrasonic sensors and radar. Within the proposed project, the focus will therefore be set on the following main topics: Development of machine vision systems based upon close-to-market sensor systems (such as stereo vision, radar, ultrasonic etc.) as well as the integration and fusion of each sensors data into a detailed world model describing static and dynamic world contents by means of online mapping and obstacle detection and tracking.\tComputer-base situation assessment within the world model as well as describing dependencies and interactions between separate model components (e.g. separate dynamic objects). For this purpose, the integration of market-ready map-material (i.e. originating from navigation systems) as well as the use of vehicle-to-infrastructure communication shall be explored. Precise low-cost localization in urban environments through the integration of standard satellite-based technologies with visual map-matching approaches combining both the onboard-perception system and available map material. Highly adaptive global and local planning considering dynamic obstacles (cars, pedestrians) and their potential trajectory.

---

Objective This project will create a self-learning robotic ecology, called RUBICON (for Robotic UBIquitous COgnitive Network), consisting of a network of sensors, effectors and mobile robot devices.

Enabling robots to seamlessly operate as part of these ecologies is an important challenge for robotics R&D, in order to support applications such as ambient assisted living, security, etc.

Current approaches heavily rely on models of the environment and on human configuration and supervision and lack the ability to smoothly adapt to evolving situations. These limitations make these systems hard and costly to deploy and maintain in real world applications, as they must be tailored to the specific environment and constantly updated to suit changes in both the environments and in the applications where they are deployed.

A RUBICON ecology will be able to teach itself about its environment and learn to improve the way it carries out different tasks. The ecology will act as a persistent memory and source of intelligence for all its participants and it will exploit the mobility and the better sensing capabilities of the robots to verify and provide the feedback on its own performance.

As the nodes of a RUBICON ecology will mutually support one another's learning, the ecology will identify, commission and fulfil tasks more effectively and efficiently.

The project builds on many years of experience across a world-leading consortium. It combines robotics, multi-agent systems, novelty detection, dynamic planning, statistical and computational neuroscience methods, efficient component & data abstraction, robot/WSN middleware and three robotic test-beds. Validation will take place using two application scenarios

Impact: The project will reduce the amount of preparation and pre-programming that robotic and/or wireless sensor network (WSN) solutions require when they are deployed. In addition, RUBICON ecologies will reduce the need to maintain and re-configure already-deployed systems, so that changes in the requirements of such systems can be easily implemented and new components can be easily accommodated.

The relative intelligence and mobility of a robot, when compared to those of a typical wireless sensor node, means that WSN nodes embedded in a RUBICON ecology can learn about their environment and their domain application, through the 'training' that is provided by the robot. This means that the quality of service which is offered by WSNs can be significantly improved, without the need for extensive human involvement.

---

Objective The SpaceBook project will prototype a speech-driven, hands-free, eyes-free device for pedestrian navigation and exploration. SpaceBook will be developed as open source and progress will be bench marked through controlled task-based experiments with real users in central Edinburgh. The SpaceBook project will generate concrete technical and scientific advances for eyes-free, hands-free navigation and exploration systems which will support applications in tourism, elderly care and tools for urban workers.

In addition to advances in navigation and exploration systems, SpaceBook provides a task environment in which more fundamental scientific and technical knowledge will be generated. Specifically we seek to advance the state of the art in model-based approaches to plan generation and recognition, statistical learning techniques for interaction management and machine learning of semantic analysis components.

The interdisciplinary SpaceBook team brings a wealth of complementary expertise necessary to realize the SpaceBoook vision. Umeå University (Sweden), the project's overall coordinator, brings expertise in natural language interfaces to database systems and spatial databases. The University of Edinburgh (UK) brings expertise in location-based services, dynamic 3D modelling, geographical information systems, system evaluation methodology, discourse processing, and information access and delivery (including question answering). Heriot Watt University (UK) brings experience in machine learning, spoken dialogue systems, data-collection, and evaluation of interactive systems. KTH (Sweden) brings expertise in dialogue management and the design and development of spoken dialogue systems. Our industrial partner, Liquid Media (Sweden), brings software engineering and commercial exploitation expertise and experience. Cambridge University (UK) brings expertise in the combination of linguistic theory and machine learning and the development of real-world language processing applications. Finally, the AI group from the Universitat Pompeu Fabra (Spain) brings expertise in the areas of autonomous behavior, and in particular, model-based methods for plan generation and plan recognition.

---

Objective The majority of current robot architectures is based on a perception-then-action control strategy. In this project, we will adopt a theoretical perspective that turns this classical view upside-down and emphasizes the constitutive role of action for perception. The key concept our project is based on that of sensorimotor contingencies, that is, law-like relations between actions and associated changes in sensory input. We will advance this concept further and suggest that actions not only play a key role for perception, but also in developing more complex cognitive capabilities. We suggest that extended sensorimotor contingencies (eSMCs) may be exploited for the definition of object concepts and action plans and that their mastery can lead to goal-oriented behavior. The project pursues the following objectives: We will employ this approach to establish computational models that are suitable as controllers for autonomous robots; we will implement these eSMCs-based models on robotic platforms with different sensor-actuator equipment; we will investigate learning and adaptivity of eSMCs in artificial systems, focussing on sensorimotor interactions, object recognition and action planning; we will investigate and validate the concept of eSMCs in natural cognitive systems, by carrying out behavioural and neurophysiological studies on healthy human subjects; finally, we will test predictions derived from this concept in patients with movement dysfunctions, where ensueing changes in perceptual and cognitive processing will be tested. A set of benchmarks and task scenarios will be developed serving as demonstrators for the enhanced performance of artificial systems based on the eSMCs approach. Moreover, the usefulness of the approach for the development of applications in augmenting human behaviour will be demonstrated.

---

Objective The emergence of networked embedded systems and sensor/actuator networks has made possible the collection of large amount of real-time data about a monitored environment. Depending on the application, such data may have different characteristics: multidimensional, multi-scale, spatially distributed, time series. Moreover, the data values may be influenced by controlled variables, as well as by external environmental factors. However, in many cases the collected data may be incomplete, or it may not make sense for various reasons, thus compromising the sensor-environment interaction and possibly affecting the ability to manage and control key variables of the environment. Such problems are generally the result of some fault in the sensor/actuator system itself or an abnormality in the monitored environment, which may be either permanent or temporary, developing abruptly or incipiently. These problems become more pronounced as sensing/actuation systems get older. The main objective of this project is to develop intelligent methods for analyzing and interpreting the data such that faults are detected, isolated and identified as soon as possible, and accommodated for in future decisions or actuator actions. The problem becomes more challenging when these sensing/actuation systems are used in a wide range of environments which are not known a priori and, as a result, it is unrealistic to assume the existence of an accurate model for the behavior of various components in the monitored environment. Therefore, this project will focus on cognitive system approaches that can learn characteristics or system dynamics of the monitored environment and can adapt their behavior and predict missing or inconsistent data to achieve fault tolerant monitoring and control.

---

Objective The GARNICS project aims at 3D sensing of plant growth and building perceptual representations for learning the links to actions of a robot gardener. Plants are complex, self-changing systems with increasing complexity over time. Actions performed at plants (like watering), will have strongly delayed effects. Thus, monitoring and controlling plants is a difficult perception-action problem requiring advanced predictive cognitive properties, which so far can only be provided by experienced human gardeners. Sensing and control of a plants actual properties, i.e. its phenotype, is relevant to e.g. seed production and plant breeders. We address plant sensing and control by combining active vision with appropriate perceptual representations, which are essential for cognitive interactions. Core ingredients for these representations are channel representations, dynamic graphs and cause-effect couples (CECs). Channel representations are a wavelet-like, biologically motivated information representation, which can be generalized coherently using group theory. Using these representations, plant models -- represented by dynamic graphs -- will be acquired and by interacting with a human gardener the system will be taught the different cause-effect relations resulting from possible treatments. Employing decision making and planning processes via CECs, our robot gardener will then be able to choose from its learned repertoire the appropriate actions for optimal plant growth. This way we will arrive at an adaptive, interactive cognitive system, which will be implemented and tested in an industrially-relevant plant-phenotyping application.

---

Objective The Goal-Leaders project aims at developing biologically-constrained architectures for the next generation of adaptive service robots, with unprecedented levels of goal-directedness and proactivity. Goal-Leaders will realize builder robots able to realize externally assigned tasks (e.g., fetching objects, composing building parts) and, at the same time, keeping their homeostatic drives within a safe range (e.g., never end up without energy or get hurt), by operating autonomously and for prolonged periods of time in open-ended environments. To this aim, Goal-Leaders will pursue a combined neuroscientific, computational and robotic study of three key sets of competences:-the biological and computational mechanisms behind an agent's goal system, which integrates somatic, affective, and cognitive elements, and realizes the setting and selection of the agent's goals;-the biological and computational mechanisms that support the assignment of situation-dependent value to an agent's state and action representations, and therefore realize the link between the agent's goal system and perceptual-motor processes;-the biological and computational mechanisms behind an agent's anticipatory and mental simulation abilities.The Goal-Leaders achievements beyond the state of the art will be assessed against behavioral and neuroscientific data, and by realizing three demonstrators in which robots will perform autonomous navigation and construction tasks, and will readapt without reprogramming to novel task allocations or changes in their environment.The project consortium includes a highly complementary, interdisciplinary team of top European researchers who focus on neuroscience, cognitive science, and robotics. Our robotic design methodology will have a significant impact both for the understanding of goal-directed action in living organisms, and for the realization of goal-directed service robots by combining navigation and manipulation within an unconstrained environment.

---

Objective This project will develop advanced technologies for automation in minimally invasive and open surgery. The introduction of more and more complex surgical devices, such as surgical robots, and single-port minimally invasive instruments, highlights the need of new control technologies in the operating room. On the one hand, the complexity of these devices requires new coordination methods to ensure their smooth operation; on the other hand, it also requires new interfaces that could simplify their use for surgeons. Automation may thus provide a solution to improve performance and efficiency in the operating room without increasing operating costs.
Currently, automation is not used in the operating room for a number of technical and legal reasons. The anatomical environment is particularly difficult to handle by classical automation. Furthermore, the execution of a surgical intervention is not only controlled by a set of physical and geometrical set points, describing the anatomical area and its properties, but also and especially by the medical and surgical knowledge that the surgeon uses in deciding what to do and how to do it, during the intervention. These control and cognitive challenges are also coupled to a legal barrier that currently prevents the use of an automatic intervention device in the operating room. In fact, liability issues of automatic products are known to have stopped many successful research projects in several Countries.
Thus, the I-SUR project aims at breaking new ground in the above areas related to automation in surgical intervention, in particular design of robotic surgical instruments, task modeling and control in highly uncertain and variable environments, medical situation awareness and its interaction with task control, surgeon-robot communication, and legal barrier identification.
To narrow the scope of the work the project will focus on simple surgical actions, such as puncturing, cutting and suturing. Success metrics will be defined for those actions and methods developed that abide by safety requirements, formulated in terms of those metrics. The project will demonstrate that an autonomous robotic surgical action, carried out with the developed technologies, can be as safe as currently achievable by traditional surgery. Furthermore, pre-operative task planning will be included in the project, to make sure that each surgeon is able to develop automatic procedures with his/her own surgical style.
The I-SUR consortium is composed of the following partners: two University Hospitals, at the University of Verona and at the San Raffaele Institute (Milano); the e-Services department of the San Raffaele Hospital; the ETH in Zurich; the BioRobotics Laboratory of the Tallinn University of Technology; the Interventional Center of the Oslo University Hospital; the Yeditepe University in Istanbul; and the Universities of Verona Ferrara and Modena-Reggio Emilia. The University of Verona coordinates the project.

---

Objective One of the aspirations of machine learning is to develop intelligent systems that can address a wide variety of control problems of many different types. However, although the community has developed successful technologies for many individual problems, these technologies have not previously been integrated into a unified framework. As a result, the technology used to specify, solve and analyse one control problem typically cannot be reused on a different problem. The community has fragmented into a diverse set of specialists with particular solutions to particular problems. The purpose of this project is to develop a unified toolkit for intelligent control in many different problem areas. This toolkit will incorporate many of the most successful approaches to a variety of important control problems within a single framework, including bandit problems, Markov Decision Processes (MDPs), Partially Observable MDPs (POMDPs), continuous stochastic control, and multi-agent systems. In addition, the toolkit will provide methods for the automatic construction of representations and capabilities, which can then be applied to any of these problem types. Finally, the toolkit will provide a generic interface to specifying problems and analysing performance, by mapping intuitive, human-understandable goals into machine-understandable objectives, and by mapping algorithm performance and regret back into human-understandable terms.

---

Objective The aim of the SEMAINE project is to draw together the current research on non-verbal signs and to produce a system that capitalises on them to achieve genuinely sustained, emotionally coloured interactions between a person and a machine. To do that, it develops an idea which was introduced in the FP5 project ERMIS and pursued in HUMAINE. The idea is to build a 'Sensitive Artificial Listener', or SAL for short, which interacts with human beings in a shallow but emotion-rich way, using interactions observed in chat shows and at parties as a model. The verbal capabilities of such a system are very limited and superficial, similar to 'chatbots' from ELIZA on. Like them, it uses rules to respond to what a listener says without understanding the content in any depth. Unlike chatbots, however, a SAL system has sophisticated, robust, real-time multi-modal interaction capabilities – in an ongoing interaction, it detects the emotional colouring communicated by the user, and adapts its responses to that. In SAL, the system's responses are themselves emotionally loaded, and are chosen to propel the user towards an emotional state that the system currently 'wants' the user to be in.

---

Objective Significant advances in human-computer interaction will require systems which can exhibit truly cognitive behaviour. This is particularly so in spoken dialogue systems (SDS) where, despite wide deployment and significant investment, current systems are still limited in capability and fragile to changes in environment or application.

Recent advances in statistical modelling and machine learning offer the potential for making a significant step forward in SDS. By both exploiting and extending these advances, the CLASSiC project will improve generalization to unexpected situations. By modelling the whole end-to-end system as an integrated statistical process, the CLASSiC project will demonstrate a qualitative leap in the adaptivity, flexibility, robustness, and naturalness of SDS.

The CLASSiC partners will develop a modular processing framework with an explicit representation of uncertainty which connects the various sources of uncertainty (understanding errors, ambiguity, etc) to the constraints to be exploited (task, dialogue, and user contexts). This architecture will support a layered hierarchy of supervised learning and reinforcement learning in order to facilitate mathematically principled optimisation and adaptation techniques. The architecture will be developed in close cooperation with our industrial partner in order to ensure that it provides a practical deployment platform as well as a flexible research test-bed.

The resulting CLASSiC SDS will be able to adapt autonomously both to the needs of different users and to changing operating environments, and to learn through experience. The data-driven methodology will also enable faster and lower-cost system implementation through automatic optimisation. Overall, the project will demonstrate not only a step-change in the capability of practical spoken dialogue systems, it will also mark a significant step forward in the longer term goal of endowing autonomous systems with truly human-like capabilities.

---

Objective Co-FRIEND aims to design a framework for understanding human activities in real environments, through an artificial cognitive vision system, identifying objects and events, and extracting sense from scene observation. It will manage uncertainty and change, and will create analysis meaning. A heterogeneous sensor network (wide angle and PTZ cameras in airport immediate area, and GPS wide area vehicle monitoring) will be deployed on Toulouse AIRPORT by SILOGIC and READING. The cognitive capabilities developed by INRIA will be demonstrated by monitoring outdoor airport activities. Deficits of current approaches to scene understanding will be addressed by CSL and LEEDS through machine learning, requiring explicit domain modelling. This challenge will provide generic research insights and demonstrate the impact of our conceptual and technical innovations.
We will improve the performance and integration of relevant cognitive functions: learning, dynamic context adaptation, perception, tracking, and recognition. Integrating them in Co-FRIEND will address the creation and exploitation of knowledge by. Our innovations will provide relevant and flexible learning and reasoning capabilities that adapt in a largely unsupervised way to change and new events. Feedback and multi-data fusion will be exploited to achieve robust detection and efficient tracking of objects in real and complex scenes.
With dynamic understanding contextualized to a learnt domain, porting and exploiting the technology in other installations, new contexts or other domains will be demonstrated. Research in uncertainty management will focus on maintaining long-term coherence in variable and complex scene environment. Cognitive and active visions will highlight human activity understanding by interpreting gestures and wide area monitoring. Co-FRIEND will evaluate and illustrate that robustness and adaptability in crowded environments can be obtained by multi-modal sensor fusion in a cognitive architecture.

---

Objective The automatic capture, recognition and rendering of human sensory-motor activities represent essential technologies in many diverse applications, ranging from 3D virtual manuals through to training simulators and novel computer games. Although capture systems already exist on the market, they focus primarily on capturing raw motion data, matched to a coarse model of the human body. Moreover, the recorded data is organised as a single cinematic sequence, with little or no reference to the underlying task activity or workflow patterns exhibited by the human subject. The result is data that is difficult to use in all but the most straightforward of applications, requiring extensive editing and user manipulation, especially when cognitive understanding of human action is a key concern, such as in virtual manuals or training simulators.The aim of the COGNITO project is to address these issues by advancing both the scope and the capability of human activity capture, recognition and rendering. Specifically, we propose to develop novel techniques that will allow cognitive workflow patterns to be analysed, learnt, recorded and subsequently rendered in a user-adaptive manner. Our concern will be to map and closely couple both the afferent and efferent channels of the human subject, enabling activity data to be linked directly to workflow patterns and task completion. We will focus particularly on tasks involving the hand manipulation of objects and tools due to their importance in many industrial applications.

---

Objective One of the aspirations of machine learning is to develop intelligent systems that can address a wide variety of control problems of many different types. However, although the community has developed successful technologies for many individual problems, these technologies have not previously been integrated into a unified framework. As a result, the technology used to specify, solve and analyse one control problem typically cannot be reused on a different problem. The community has fragmented into a diverse set of specialists with particular solutions to particular problems. The purpose of this project is to develop a unified toolkit for intelligent control in many different problem areas. This toolkit will incorporate many of the most successful approaches to a variety of important control problems within a single framework, including bandit problems, Markov Decision Processes (MDPs), Partially Observable MDPs (POMDPs), continuous stochastic control, and multi-agent systems. In addition, the toolkit will provide methods for the automatic construction of representations and capabilities, which can then be applied to any of these problem types. Finally, the toolkit will provide a generic interface to specifying problems and analysing performance, by mapping intuitive, human-understandable goals into machine-understandable objectives, and by mapping algorithm performance and regret back into human-understandable terms.

---

Objective CORBYS focus is on robotic systems that have symbiotic relationship with humans. Such robotic systems have to cope with highly dynamic environments as humans are demanding, curious and often act unpredictably. CORBYS will design and implement a cognitive robot control architecture that allows the integration of 1) high-level cognitive control modules, 2) a semantically-driven self-awareness module, and 3) a cognitive framework for anticipation of, and synergy with, human behaviour based on biologically-inspired information-theoretic principles. These modules, supported with an advanced multi-sensor system to facilitate dynamic environment perception, will endow the robotic systems with high-level cognitive capabilities such as situation-awareness, and attention control. This will enable the adaptation of robot behaviour, to the user's variable requirements, to be directed by cognitively adapted control parameters. CORBYS will provide a flexible and extensible architecture to benefit a wide range of applications; ranging from robotised vehicles and autonomous systems such as robots performing object manipulation tasks in an unstructured environment to systems where robots work in synergy with humans. The latter class of systems will be a special focus of CORBYS innovation as there exist important classes of critical applications where support for humans and robots sharing their cognitive capabilities is a particularly crucial requirement to be met. CORBYS control architecture will be validated within two challenging demonstrators: i) a novel mobile robot-assisted gait rehabilitation system CORBYS; ii) an existing autonomous robotic system. The CORBYS demonstrator to be developed during the project, will be a self-aware system capable of learning and reasoning that enables it to optimally match the requirements of the user at different stages of rehabilitation in a wide range of gait disorders.

---

Objective Targeting both assembly and service industry the DARWIN project aims to develop an 'acting, learning and reasoning' assembler robot that will ultimately be capable of assembling complex objects from its constituent parts. First steps will also be taken to explore the general reparation problem (seen as a sequence of assembly-disassembly actions). Functionally the robotic artefact will operate in three modes: a) slave, where the necessary sequence of operations will be provided by a CAD-CAM system; b) semi-autonomous, where the sequence will be provided either through demonstration by a teacher performing the same task or by describing it in a higher level 'language', suitable for a human operator. In this mode the system's perception will provide closure of the perception-action loop, so eliminating the need for detailed spatial information as in (a); c) fully autonomous mode, where an executive process will generate the necessary sequence by reasoning and mental simulation of consequences of actions on objects. Effort will be put to keep the resulting cognitive architecture domain agnostic. This directly implies that the robot should be able to effectively generalize and transfer previously gained knowledge to new tasks (or performing the same task in a slightly modified world). The objects in its world will not be known apriori. Rather, their knowledge will be built by interacting with them. Thus categorization, object affordances, accurate manipulation and discovery of the naive physics will be acquired gradually by the robot. In addition, it will also learn the solution steps for an assembly problem by observing snapshots of the assembled object during various construction phases. The reasoning system will exploit all these experiences in order to allow the robot to go beyond experience when confronted with novel situations. A series of demonstrators of increasing complexity will be developed in correlation with the maturation of the cognitive architecture.

---

Objective The EMIME project will help to overcome the language barrier by developing a mobile device that performs personalized speech-to-speech translation, such that the a user's spoken input in one language is used to produce spoken output in another language, while continuing to sound like the user's voice. Personalisation of systems for cross-lingual spoken communication is an important, but little explored, topic. It is essential for providing more natural interaction and making the computing device a less obtrusive element when assisting human-human interactions. We will build on recent developments in speech synthesis using hidden Markov models, which is the same technology used for automatic speech recognition. Using a common statistical modelling framework for automatic speech recognition and speech synthesis will enable the use of common techniques for adaptation and multilinguality. Significant progress will be made towards a unified approach for speech recognition and speech synthesis: this is a very powerful concept, and will open up many new areas of research. In this project, we will explore the use of speaker adaptation across languages so that, by performing automatic speech recognition, we can learn the characteristics of an individual speaker, and then use those characteristics when producing output speech in another language.

Our objectives are to: Personalise speech processing systems by learning individual characteristics of a user's speech and reproducing them in synthesised speech; Introduce a cross-lingual capability such that personal characteristics can be reproduced in a second language not spoken by the user; Develop and better understand the mathematical and theoretical relationship between speech recognition and synthesis; Eliminate the need for human intervention in the process of cross-lingual personalisation; Evaluate our research against state-of-the art techniques and in a practical mobile application.

---

Objective The project intends to establish a link between fundamental sensing tasks and automated cognition processes that concern the understanding a short-term prediction of human behaviour as well as complex human interaction. The analysis of human behaviour is unrestricted environments, including localization and tracking of multiple people and recognition of their activities, currently constitutes a topic of intensive research in the signal processing and computer vision communities. This research is driven by diferent important applications, including unattended surveillance and intelligent space monitoring.
The overall goal of the project is the development of principled methods to link fundamental sensing tasks using multiple modalities, and automated cognition regarding the understanding of human behaviour in complex indoor environments, at both individual and collective levels. Given the two above principles, the consortium will conduct research on three core scientific and technological objectives:
1) sersor modelling and information fusion from multiple, heterogeneous perceptual modalities
2) modelling, localization, and tracking of multiple people
3) modelling, recognition, and short-term prediction of continuous complex human behaviour

---

Objective Simple search engines have turned out to be successful in text retrieval. This is partly due to advances in algorithms, but additionally because the typical tasks are relatively straightforward as it is possible to come up with a good list of keywords with a reasonable effort. All information needs cannot however be solved with such searches. For the more and more commonly available audio-visual data people are not able to specify efficient non-textual search keys. Moreover, if the information needs of the user are tacit it is very laborious to create a query.
We develop new information retrieval principles needed for replacing or complementing explicit queries. The new techniques will be applied to construct prototypes of proactive information navigators, the next generation systems able to adapt to the user and her tacit information needs. During browsing and searching with a task-dependent interface, the goals of the user will be inferred from implicit feedback signals and interaction, complemented with social filtering.
New advanced machine learning methods are needed for inferring the implicit topic of interest and the sense in which it is interesting. We will augment the standard two-valued relevance feedback from the user by richer feedback derived from point-and-click and point-and-speak interfaces and gaze patterns over items that include texts, images and movies. Eye tracking technology is rapidly maturing to be usable on a massive scale.
Our consortium combines pioneering application expertise with solid machine learning background. We have built a top content-based image retrieval system and used eye tracking in information retrieval. Our machine learning expertise is suited for the core issues of learning the comparison metrics and solving the exploration-exploitation problem. Proactive information navigators will bring up a new spectrum of application areas; the system will be tested in a set of targeted applications by two industry partners.

---