Beyond Behaviorism

Reinforcement learning has been used extensively in robotics for many years, ranging from the simple types of obstacle avoidance seen in the seven dwarfs all the way up to the elaborate brain based Darwin automata capable of more sophisticated associative learning  This is predicated upon a psychological school of thought known as behaviorism, which was extremely popular for most of the 20th century but which has fallen out of favor in recent decades.

In the realm of learning robotics there don't seem to have been many attempts thus far to go beyond the limited bounds of behaviorism.  Clearly in animals classical conditioning does play an important role, but it's by no means the whole story.  For example, young children don't learn language by being explicitly rewarded or punished every time some new concept is acquired.  Much of learning seems to be self-organised, where the creature has a spontaneous ability to classify and is engaging in a kind of synchronization with its environment where the resulting conceptual framework converges towards stable forms, like eddies in the flow of causality.

Is human language Turing complete?

Even reinforcement learning, when combined with self-organization aren't sufficient to fully explain complex behaviors described as "intelligent".  To be capable of a wide range of adaptation the system needs to be Turing complete in a manner which supports the generation of a population of virtual machines capable of implementing arbitrary transformations or behaviors.

Turing machines can of course be implemented in many different kinds of ways, and there are many possible systems which are computationally equivalent to them.  As noted by people like Chomsky, human language seems to exhibit qualities similar to Turing completeness, such as the ability to generate recursive structures.  If human language is a Turing complete system, as I suspect that it is, then this may explain why it is such a powerful tool for regimenting thought processes, and has class 4 type behavior.  Interactions between multiple such systems, both within a single mind and between multiple minds could be the way in which the problems of classical incompleteness are overcome, since at the interfaces between machines undefined behavior can occur and the systems themselves may comprise heterogeneous but symbiotic formalisms which like the bimetallic strip of a mechanical clock work against each other in a complementary way.

The language augmented nature of human thought is probably also the reason why reinforcement learning can have more extensive consequences than you might see in other creatures, up to and including the complete reorganization of a person's world view based upon one or a small number of learning instances.  So instead of just reinforcing some single isolated categorization the reinforcing event becomes the input to a Turing machine which has consequences that are not always easy to foresee (without running the program) and may spawn new virtual machines or erase existing ones.

Dimensions of Affect

Another issue with reinforcement learning as typically practiced in robotics is that learning itself tends to be unidimensional.  The learning rate may vary, but there are not usually more than one learning parameter.  Hence it's difficult to simulate fading affect memory biases, where memories associated with positive or negative consequences are differentially weighted and can have multidimensional results.  A possible research programme would be to apply the biologically based affect theory of Silvian Tomkins to robotics and try to replicate learning scenarios which have similar outcomes to those seen in people.

A Musical Seasons Greetings from Freiburg

Seasons Greetings from Freiburg's Humanoid Robots Lab!

ROS Documentation Contest: ROSOSC

Our plan to trade t-shirts for documentation continues to pay off, with Michael Carroll sending in his documentation for his new ROS to Open Sound Control Bridge stack. We suggest watching the videos below for how this can be used for teleoperation and remote diagnostics from the convenience of your iDevice or Android device using TouchOSC.







Tutorials can be found here and if anyone is looking for electrical or software
engineers with ROS experience Michael is available for employment.

Turning robots into products

Many good points about the commercial prospects for robotics are made in a recent episode of the robots podcast.  I think it is true that we're at a turning point where mobile robotics is becoming much more practical than it was in the past, and where a hunt for new kinds of applications can commence.

It's not that sensor technologies comparable to the Kinect sensor didn't exist before, since time-of-flight sensors have been around for at least five years and scanning laser rangefinders have been around for longer, but that the current generation of depth sensing technology has a far more favourable price/performance such that applications other than traditional high cost industrial automation ones can be considered.

One way to address the unfocused research problem would be to have some percentage of research activities biased towards application areas.  However, the issue remains that what constitutes good application areas for mobile robotics at this point remains largely undefined.  What's going to happen with robotics will I think be similar to what happened with computers.  In the early days of home computers people didn't really know what they would be used for.  At first they didn't do much more than run simple kinds of games, then they were used for spreadsheets, accountancy and wordprocessing.  In the early 1990s there was a vague idea that computers would be used for "multi media" education, interactive TV or "virtual reality", then in the late 1990s with the rise of the internet the computer became a machine for communicating and doing shopping.

Even in the late 1990s there were still people who asked the question "why would I want a computer in my home?" or "why would I ever want or need to look at the internet?".  People say similar things today about robotics.

So whenever new technology arrives within a consumer price range there is always a long, messy and meandering process of discovery which unfolds and which is not easy to foresee in the research lab.  This means that it's not necessarily a simple task to pick applications towards which research may be biased without being able to foresee the future, and what looks like the hot application of today (elder care or hospital work perhaps) may not be quite so hot by the time something actually gets to the stage of being built and sold.

When drones fail

Here is an amusing tale of how a military drone was hijacked and forced to land at what it believed to be it's home airfield.  Drones like this havn't been around for very long, and their guidance systems are presumably quite simple - similar to the growing number of DIY drones based upon Arduino and similar microcontrollers or DSPs - so there is probably lots of low hanging fruit in terms of security vulnerabilities.

As I supposed many years ago there will now ensue an evolution of telerobot and counter-telerobot technologies.  The most obvious thing which could be done to avoid denial of GPS or fake GPS scenarios would be to add visual localization.  This could be as simple as dead reckoning based upon optical flow, similar to the method by which an optical mouse works. More accurate visual localization would combine an internal map with standard planar object recognition techniques based upon things like SIFT or SURF features in order to have the drone recognise certain locations and dead reckon between them.  Those bag of words-like techniques can scale to thousands of objects, so there could be many landmark locations upon which the drone could localize.  Of course, this assumes that the drone is able to see features on the ground, but if they're primarily used for surveillance then that's likely to be true most of the time anyway.

Sensor fusion between GPS and visual localization/odometry is the most likely solution.  Also, this isn't only a problem which applies to military drones.  In future I expect that most civilian aircraft will be flown in the same way, without human pilots, so increasing the robustness of the navigation is a worthwhile goal.

Speedy

There are a variety of reasons why it's probably not a good idea to have robots moving around at high speed indoors.

• There's likely to be more wear on the carpet, and "tracks" will become conspicuous.
• There's a tripping hazard.  One second there is no robot and the next there is.
• If anything goes wrong and the robot hits something, it could cause non-trivial damage - possibly including generating fire hazards.
• Last but not least, more speed usually means more noise.  Noise level is something often forgotten, but for robots operating indoors I think it will be a factor.  People already fuss over noisy PC cooling fans.

However, in industrial settings such as warehouses where there are rarely people around and it's a more controlled environment I expect that it will be normal for robots to be moving at this kind of speed.

For most domestic or office space service robots probably a very pedestrian Turtlebot-like speed of 0.3m/s will be about as fast as it gets, and may even be slower if handling objects.

Last chance for standard shipping sale

A quick note from I Heart Engineering. Please order by Dec 14th at 7pm EST to have purchases guaranteed to arrive via standard shipping in time for Xmas. Orders placed after the 14th via standard shipping may or may not arrive in time. You will have until the 19th for guaranteed delivery of express shipments.

To help loosen your wallet, we have some things on sale including new Neon Green tripod mount for the Kinect.

Kevin Warwick and the Seven Dwarfs

Here is some video which I took at an exhibition in Glasgow in 1996. Kevin Warwick demonstrates machine learning and flocking behavior with the "Seven Dwarf" robots.

This is not the "cognitive" robotics which you can now see in robots running ROS, where they're mapping the environment and planning paths through it, but is a much simpler and more behavioristic system in which sensors and actuators are connected fairly directly with only a negligible amount of internal state.

It is interesting to think about these types of robots, which only have a small amount of computation onboard, in terms of Stephen Wolframs classes of automata, and whether or not they could converge towards collective behavior systems which are Turing complete - facilitating an open-ended cultural evolution.  It's a conjecture of mine that something like this happened in human evolution, such that whereas most social creatures have quite predictable behaviors and a fairly static culture some accident of circumstance or mutation meant that the collective human socio-cognitive system fell into class 4 type behaviors, and what followed was the emergence of history and technology.

There was a great deal of interest in behavioristic robotics throughout the 1990s, particularly promoted by Rodney Brooks, in spite of behaviorism's having fallen out of favor within the realm of psychology by that time.  Within the last decade robotics systems have tended to move back towards GOFAI, with much more internal state and elaborate environmental modeling and planning.

Celebrity chefs

Sandwich making activities. This is pretty much the state of the art at present, but maybe in another five or ten years this will be something that you could buy and install in a Café. There may also be an industry in creating plans for the robots to follow which are customized to the particular working environment.

An interview with Sebastian Thrun

Here's an interview with Sebastian Thrun. He's one of the professors running the current online Introduction to AI course, and a co-author of Probabilistic Robotics. I think that the implementation of Monte Carlo Localisation within ROS is based upon the description of the algorithm from that book.
 

I'm not a very enthusiastic driver myself, so I hope that within the next couple of decades it will become possible to get into a car and just select the destination, then not have to be concerned about the details of manual driving.

Conversations such as the above are ok up to a point, although there are some confusions and omissions along the way. The familiar standard narrative about technology is wheeled out once again, with the talk about the transition from agrarianism to industrialism. Although this transition undoubtedly did take place in more or less the manner described this doesn't mean that a similar transition is occurring today. In fact the current data suggests the opposite, with employment rising in low skilled jobs and stagnant or falling in higher skilled ones within newer high technology or software-related industries.

There is also the question of the use of AI to help resolve social problems at a political level, and unfortunately here I think the responses from Thrun and Diamandis were weak. It's not the case that political elites (governments and their various agencies) aren't interested in data collection or analytics, which typically involves some amount of AI. Governments obsessively collect and analyze data, using it to guide policy in a wide variety of ways. It's just that this activity is rarely conveyed in any intelligible form to the general populace, and almost never features within popular debate. Recent examples of use of AI by governments would be eigenbehavior analysis of entire nations via mobile phone location logs, drone aircraft, persona management, "screening" at national borders and of civil service employees, and so on.

From what I've seen, Thrun and Norvig's online AI course has been pretty good so far - although definitely rough around the edges - and I hope that this is the way that higher education goes in future. However, there is also the question of over-production of highly educated elites (people with good education and high expectations about their future status), which according to things I've been reading more recently tends to lead to political instability.  So I don't think it's the case that we're facing an inevitable exponential march towards a Star Trek style future, as many of the Singularitarians seem to assume.  The future is likely to be complex, and will include strange loops, paradigmatic phase changes and all sorts of unexpected challenges.

Graphical launch files

The new package called rxdeveloper-ros-pkg looks quite interesting, and may help to make the process of creating and editing ROS launch files easier.  The appearance is similar to the function block editing UIs found in many industrial control systems.