February 28 2008 / by memebox
Category: Technology Year: General Rating: 5
The following is a transcript of an audio interview with
Steve Omohundro by Venessa Posavec
V: For Memebox.com, this is Venessa Posavec, and with me is Dr.
Stephen Omohundro, president and founder of Self-Aware Systems, and
research advisor to the Singularity Institute for Artificial
Intelligence. Thank you for joining us, Steve.
Steve: Thanks for having me.
V: What do you do and how is that related to the future?
Steve: I’m the president of a company called Self-Aware Systems,
and we’re involved both with developing a new form of advanced
artificial intelligence, and also we serve as a kind of think tank
where we’re analyzing the social consequences of this new
technology and trying to guide it in a way that best serves
humanity.
V: What is artificial intelligence?
Steve: That’s a discipline that’s been going since, oh, the late
50s, where we try and understand what’s the fundamental nature of
human intelligence and build machines which can solve the same
kinds of problems that people can.
V: What are some opportunities (and risks) of a self-improving
AI?
Steve: The particular approach to artificial intelligence that
my company is taking, and now a few other groups are as well, is to
try and build systems that understand their own behavior and watch
themselves as they work and solve problems; notice what things are
working well and which things aren’t working well, and then change
themselves, improve themselves, so that they work better. And we
believe that this is a very powerful new approach to artificial
intelligence that will solve many of the problems that other people
haven’t been able to solve. And so on the one hand, it gives rise
to great opportunity. On the other hand, it’s a very different kind
of system than we’re used to. When a human programmer just writes a
program, he understands what he wants it to do, and sometimes there
are bugs, but basically the system behaves the way you expect it
to. When you have a system that can change itself, basically it
writes its own program, then you may understand the first version
of it, but unless you’ve done a lot of analysis, it may change
itself into something that you no longer understand. And so, these
systems are quite a bit more unpredictable than the kinds of
software we’ve been used to, so it’s very powerful, but there’s
also potential dangers. So, a lot of our work is involved with
getting the benefits while avoiding any risks.
V: What kind of impact would AI have on our economy? if our
machines can perform many of the tasks that we think of as
belonging to humans?
Steve: Well, I think we’re in for a big shift, because
essentially every aspect of the economy can be improved by having
more intelligence there, by making decisions more effectively. One
of the consequences of artificial intelligence will be robotics
that can actually behave much more flexibly than the robots we have
today. And on the good side, that means a lot of manual labor which
people don’t particularly like doing can be replaced by robots. On
the potentially negative side, a lot of jobs that people have today
will be much more cheaply accomplished by robotic systems, and so
it’s going to be a big dislocation in the economy of the world.
Huge potential benefits, way greater productivity, meaning that
there’s a lot more potential wealth for the entire world, but
exactly how we distribute that, and how the social structure adapts
to this new technology is one of the big questions we’re facing
right now.
V: Where does opposition to building intelligent machines
usually come from?
Steve: Well, it leads to a very uncertain future, with a lot of
potential downsides and risks, at the same time as potential
positive things. Any time a new technology comes around, there is a
kind of conservatism. There are a lot of people who, quite
rightfully so, believe we should move very slowly. Bill Joy, one of
the founders of Sun Microsystems, wrote a very influential paper,
saying ‘the future doesn’t need us’, suggesting that we relinquish
the technologies of artificial intelligence, nanotechnology, and
biotechnology, and that we only proceed very very slowly in
developing them. That might make perfect sense if we could actually
be sure we could do that. The problem is that if a country, say,
the United States, decides to stop developing this kind of
technology, it just means that the future we end up with is going
to be determined by some other country – and that may be North
Korea, Iran, or a country with values very different from our own.
And so I think there’s really no way to stop it. I think the best
path is to understand it very carefully, to be very clear about our
values and what we want our future society to look like, and then
we can guide this technology to help us to develop that future.
V: What is productive nanotechnology?
Steve: Nanotechnology refers to doing manufacturing and
engineering on the nanoscale. A nanometer is 10 (-9) meter, it’s
very very small, it’s on the scale of individual atoms. There’s
nanotechnology today, which builds structures and materials that
have intricate structure on the nanoscale. But what productive
nanotechnology refers to are nanosystems which can actually build
other nanosystems. So you can think of them as little molecular
robot arms, or little molecular mills, like in a manufacturing
mill, and that leads to an entirely different set of consequences,
because one of the things a productive system can make is a copy of
the manufacturing device itself. It may be very very expensive,
maybe billions of dollars, to create the first one. But once you
have the first one, then it can very cheaply make a copy of itself,
and then you can get vast manufacturing capability very
inexpensively. So it has the potential of making things that are
very expensive today, like a laptop computer or solar cells or
diamond rings, all of those materials today require very expensive
processes to build. In the presence of productive nanotechnology,
those would cost about $1/kilogram, and you could have little
blocks that would crank out whatever you have the instructions for,
at the rate of about a kilogram per hour. So, it has the potential
to radically lower the cost of manufacturing.
V: What is that going to mean for the manufacturing industry in
general – with the birth of these 3D printers, where you can just
print from home any kind of object, when people don’t have to go to
the store anymore to buy something, they just print what they want
right from their house?
Steve: Yeah, I think one of the revolutions we’re right in the
middle of is trying to understand the nature of intellectual
property. Property rights make perfect sense for specific physical
objects. If I have something and you take it, I no longer have it.
What we’re seeing right now is this raging battle over music and
movies, where the record companies and movie companies have this
notion of intellectual property. The problem is with music and
movies, if you give a copy of it to your friends, you still have
your copy. Humanity loves to share, it’s part of our altruism, and
so with information it’s very difficult to stop people from giving
away things that they like. With the fabs you were mentioning, and
with nanotechnology, ultimately, it turns material objects
essentially into information. Let’s say you’ve got a laptop
computer you really love, it works really well. In this future
world, you’ll be able to take the description of that laptop
computer and send it to your friend, and then they can create it on
their own home fab. So the notion of material objects as
possessions gets much more blurry, and the material world becomes
much more integrated with the informational world.
V: What is the connection between productive nanotechnology
& AI?
Steve: These are, I think, the two most important technologies
that are going to transform everything. Productive nanotechnology
deals with manipulating the physical world at a fine level, and
self-improving artificial intelligence deals with manipulating the
informational world at a fundamental level. They’re actually
synergistic with each other, in that to really get the power and
impact of productive nanotechnology, you need to control it, and
these are very complicated devices, with billions and billions of
components, and to control it effectively, you need a great
intelligence. To really make productive nanotechnology effective,
you need artificial intelligence. To make the artificial
intelligence really powerful and effective, it needs to run on a
very powerful computer, which productive nanotechnology can
provide. And so the two technologies are very synergistic, in that
the combination of them is what’s going to be so transformative.
There’s also a very interesting aspect, which is, both of those
technologies, we can clearly see what they’re going to look like,
what they’re outcome is like, but the pathway from where we are
today to getting there – there’s still some roadblocks and
stumbling blocks along the way. Interestingly enough, each of the
technologies would enable the other one very quickly. If productive
nanotechnology were to be produced first, you could make very
conservative designs that Eric Drexler has done in his book
Nanosystems, which enable the construction of a computer in the
size of a sugar cube, which is more powerful than all the computers
on Earth today together, and it would cost a few cents, and one of
those productive nanotechnology manufacturing systems could produce
it in a few minutes. So, the compute power that’s available once we
have productive nanotechnology will just dwarf anything that’s
around today. And that enables some brute force approaches to
artificial intelligence. One, for example, that Ray Kurzweil has
written quite a lot about, is the idea of scanning the human brain
in detail, discovering what all the neuronal connections are like,
and then simulating whole human brains directly. It requires a lot
of compute power, but presumably you should be able to get the
level of intelligence of a person in that way, and then that can
serve as the bootstrap for having the system improve itself, and
you get full arbitrarily powerful artificial intelligence. So, if
productive nanotechnology comes first, then probably a matter of a
year or two after, I would expect to see artificial intelligence.
If artificial intelligence comes first, then it could be used to
solve some of the hurdles on the pathway to productive
nanotechnology. There’s a roadmap that the Foresight Institute and
Battelle and several other groups just released a couple of months
ago, at a big conference in Washington DC, in which they outlined
four different pathways toward productive nanotechnolgy – some that
sort of build on top of biotechnology using DNA as a construction tool, others that use organic
chemistry and self-assembling molecules, another approach using
scanning tunneling electron microscopes, so each of these
approaches is very promising, but there are some hurdles, some
steps that we don’t yet understand how to accomplish. If we had a
full artificial intelligence, it’s very likely that those systems
would be able to solve those remaining hurdles. So, probably in the
matter of a year or two after AI we would have the first productive
nanotechnologies. So, I think when we think about the future, we
really need to think of those two technologies as working
together.
V: You mentioned brain scans. Do you think that once that’s
possible, the next step is that our minds can be scanned and
uploaded, and we can live on in immortality in a virtual world?
Steve: That’s a vision that some people think is both possible
and desirable. I’m a little more circumspect about it. I have no
question that intelligence will be possible in that way, I think
there are a lot of philosophical questions which we don’t yet
understand about consciousness and about qualia, the sense of, when
you see the color red, it has a certain quality or a certain
feeling to you. Whether those aspects of our humanity are
replicable inside of a machine – I think we’re going to just
experiment with them and see. What the experience of a person whose
mind or brain is copied and simulated in a computer, is that
program really you? Does that program feel pain? I think those are
questions that are going to really probe to the depths of our
humanity and what does it mean to be human and what do we want to
do. So, I’m hoping that that transition is one that we explore very
carefully and very delicately. So, I would not predict that people
would be rushing (whole-hog?) to be uploaded into computers, but I
think it’s going to shine a lot of light on our experience, and
we’ll make those decisions as we come to them.
V: Based on the research up to this point, and talks from the
Singularity Summit, when do people think the first AI will be
created?
Steve: Well, it’s challenging to try and put precise dates on
it. Artificial intelligence, the name, was introduced by John
McCarthy in 1959 I think it was, and because intelligence is
something we do so naturally, it’s very difficult to see what’s
hard about it. Interestingly enough, the things that we thought
were hard, such as playing chess, turn out to be really easy for
computers. In fact, the best computer chess player beat the world
champion in 1995, whereas things that are very easy for us, say,
distinguishing a dog from a cat, no computer system can yet do
that. In the early 60s when the field of AI was just getting going,
people expected it would be about 3 or 4 years before they would be
able to solve a problem like distinguishing dogs from cats. In
fact, Marvin Minsky had a master’s student he assigned a summer
project of doing machine vision, and he made great advances, but
the field is nowhere near complete at that. So, the field of AI,
unfortunately, has made optimistic prognostications during it’s
whole evolution, and so it’s dangerous to say when it’s going to
happen, but the kind of brute force methods I was talking about
such as simulating brains – the analysis that Ray Kurzweil does in
his book The Singularity is Near, he argues that if you look at
technological trends, how fast computers are getting cheaper and
faster, he estimates somewhere around 2030 as the point at which a
brute force approach to AI should work. If there’s a new idea, and
that’s something I’m very interested in, my company feels that our
technology has a number of new ideas that may be the critical ones,
that a new idea can make AI have dramatic improvements in these
systems, and so that’s very unpredictable. So, I’d say the
situation looks like 2030 or somewhere thereabouts we’re likely to
be able to do it with brute force approaches. Using more clever
techniques, using new ideas and insights into intelligence, it
could happen much sooner. Some of the people at the Singularity
Summit claimed that they expected their systems to be fully
intelligent within the next 4 years. So, I think there’s a lot of
uncertainty and we’ll have to see how it plays out, but it could be
soon. I think in our thinking about what the future looks like, we
have to account for the possibility that it could be in the next
few decades, certainly.
V: What resources do you recommend for people who want to better
understand both artificial intelligence and the future in general
and what the future will hold?
Steve: There are several books that are really good. If you want
to understand the ideas and technical details of artificial
intelligence, there’s an excellent textbook called Artificial
Intelligence: A Modern Approach, by Stewart Russell and Peter
Norvig, it’s the standard textbook now that’s used in universities.
There are also a huge number of resources on the web of various
kinds, and I think your website is a very exciting new direction
there, where you aggregate and present information that helps one
form a coherent picture of the future. Some other places – I’m on
the Advisory Board of the Singularity Institute – their website has
a lot of interesting information, Ray Kurzweil’s site has a lot of
pointers to new developments in artificial intelligence, the
Foresight Institute has a lot of references to nanotechnology
developments, as does the Center for Responsible Nanotechnology. So
I think we’re seeing a real flourishing of interest in these
topics, and people writing excellent books and resources, so that
it’s possible to become knowledgeable much more quickly than it was
a few years ago.
V: Can you give us a general landscape of what you think the
future will look like?
Steve: Well, I think it’s pretty clear what the technology looks
like. I think what humanity does with that technology is much more
uncertain. There are both positive utopian scenarios, and very
negative dystopian scenarios. Let me just say a few of the positive
things we can expect. One of the great things about nanotechnolgy
is that it will enable us to advance medicine way beyond where we
are today. The current techniques of surgery is basically taking a
big knife and cutting into your tissues. Post-nanotechnology,
that’s going to look extremely crude, almost like bloodletting
looks to us. With nanotechnology, you can have devices that go into
the human body, find pathogens and eliminate them, find cancerous
cells and destroy them, clean out your arteries, basically restore
health at a very fundamental molecular level. One thing that I
think is unquestionably a positive good will be the elimination of
human disease. That leads to a somewhat more controversial thing,
which is – and many people are also excited about the prospect of
also eliminating aging and potentially eliminating death – that is
a prospect that scares a lot of people and excites a lot of people.
The social consequences are huge. If people no longer die, the
whole structure of our society will change dramatically, but I
think that will certainly be a technological possibility. The
presence of productive nanotechnology will dramatically lower the
cost of material goods, so there’s the potential to eliminate
poverty throughout the world. One of the challenges though is,
there are mechanisms in today’s economy whereby the wealthy become
extremely wealthy, and the poor tend to get poorer, so there’s sort
of a divide between the richest and the poorest humans. These
technologies can amplify that, because of their very nature. It is
possible for them to be used by the very wealthy to enhance their
wealth. It’s also possible, if we choose the way we structure
society properly, if you’ve got your home nanotech manufacturing
device, you can give a copy of it to your friend, and they now have
one, and you can each make whatever material goods you need, and so
there’s the potential for great equalization of wealth and
prosperity throughout the world. So, finding how we want to
structure this post-AI, post-nanotech economy, I think, is one of
the big questions that faces us. Another huge area is conflict.
Today, we have wars and skirmishes all over the planet. These
technologies, on the one hand, will enable weapons far more
powerful and more deadly and more scary than anything we have
today. On the other hand, they also enable defensive mechanisms
that will – for instance, one of the big worries a lot people have
today is about biological terror. If someone were to develop a
pathogen, an artificial pathogen, which was like an airborne ebola
virus or something, it could be extremely devastating, and could
kill millions and millions of people, we really don’t have very
effective defenses against that kind of thing today. With
nanotechnology, we can defend against those things, of course it
also enables the creation of much more horrific tools. And so
there’s a balance of defense and offense, and finding a way to
create a social structure wherein we eliminate violence and
conflict that ends up damaging the participants, I think that
should be one of the highest priorities. And yea, we’ve been
developing some possible approaches to that. So on the positive
side of the ledger, we have the potential to eliminate disease,
eliminate aging, eliminate poverty, eliminate hunger, eliminate
pollution, we can clean up global warming, eliminate war, and so
it’s kind of a utopian vision, but it all looks technologically
possible. On the negative side, there are all sorts of potential
dangers where you have runaway technology or you have some small
piece of society which ends up trying to take over the rest. So the
choices we make today in guiding this technology will lead us to
either an amazing, beautiful future, or to one that’s very
negative.
V: So, 2008 is right around the corner. Can you list some
specific predictions and possibly some emerging trends that you see
for next year?
Steve: Well, I don’t expect artificial intelligence and
nanotechnology to come about on that short of a time scale, but I
think the sort of “normal” technology – Moore’s Law making
computers cheaper and hard disks cheaper, bandwidth is increasing,
there’s a new firewire spec that will enable high definition video
to be transmitted into and out of our computers. One trend, I think
is really exciting, that was just announced is – well, Wikipedia is
a phenomenon that is phenomenal. It’s sort of an aggregation of all
of human knowledge, and just in the few short years that it’s been
around, it’s grown to become really an amazing and powerful
resource. I think it has a number of problems though, and right now
it appears that there’s quite a lot of political scwabbling inside
of it, and particularly the rise of kind-of powerful Wikipedians
who delete – who feel their role is to delete knowledge – there’s a
big furor right now whether mathematical proofs belong in that, as
if that’s not human knowledge. So, Google is coming up with a
competitor, potentially, they’re calling Knols, where an individual
has ownership of a page, and others can comment on it and make
suggestions, but they control what the content of it is. That seems
to me a more stable model for the long term, but I think it’s
actually good that we have competing models. One or the other or
maybe some other idea will come about to make access to human
knowledge way easier than it is today. I think another trend that
we see right now is cell phones are just spreading all over the
world, and they’re becoming more and more powerful. I think most
people’s access to the internet in the world will not be through
computers, it will be through handheld devices like cellphones. I
see a lot of advancement in that area.
V: Again for 2008, do you think there are any disruptive events
that may occur that people just don’t see coming?
Steve: There’s nothing that I’m aware of. Of course, if it’s
really disruptive, I probably wouldn’t be aware of it. I don’t see
2008 as having a huge unforseen thing. I guess it depends on what
you consider disruptive. For example, the social networking sites
really exploded in this past year, things like Facebook. On the
other hand, they’ve been going at a lower rate for a number of
years, so it’s not clear whether that’s disruptive or not. I think
the technology trends that underlie it are gonna smoothly continue
on, I think Moore’s Law has at least another 10 or 15 years to go,
and that will drive amazing and powerful technology. The computers
we’ll be able to buy are just getting better and better. What’s
disruptive, I think, is the social response to that. I just saw
that YouTube, which enables people to put their own videos on the
web, has had a big impact in the Western countries, but I was just
reading that Saudi Arabian teenagers who feel sort of oppressed by
their society are using YouTube as a means of expression, and
they’re videotaping themselves doing wild and crazy things –
hanging out of cars running down the freeway, and it’s shocking the
elders of that society, but it’s become a mechanism for social
change that wasn’t necessarily predictable just by looking at the
technology. So, I would expect most of the unexpected things to
come from the social level rather than from the technological
level.
V: Craig Venter and his team are working on creating the first
synthetic lifeform. What are you thoughts about that?
Steve: Well, I think it’s very interesting. Biotechnology is
right at the cusp of major advances. I mean, we just sequenced the
genome a few years ago and now already there are companies like
23andMe, who for a thousand dollars, will examine your DNA and tell you if you have a wide variety of
genetic diseases. Craig Venter has been at the forefront of that.
He recently finished a trip around the world collecting
DNA from the oceans and finding many many
new species. So, I think our understanding of the natural world,
particularly how biology works, is just exploding. I would expect
sometime in the next few years, that in addition to having the
complete genome, we’ll have a complete map of all the proteins in a
cell, and the metabolic processes by which those proteins interact,
and the paths by which they’re created. Once we have that, I think
it will give us insight into human disease that’s just beyond
anything we’ve got now. It will also potentially enable us to
develop new molecules and drugs in an amazing way. And, – and this
is somewhat controversial, to develop new lifeforms, to modify the
DNA to produce an outcome we want. I
think it hasn’t been quite announced, but everyone’s been
pre-announcing it, this artificial lifeform, is certainly a
milestone in the history of our understanding and relationship with
living things. Where that leads? Today we have big controversies
about stem cells and about abortion. I think that when we can
choose the genomes of our children, what social consequences does
that lead to? Certainly everybody’s going to want to eliminate
genetic diseases, and I don’t think there will be much controversy
about that. But, why stop at eliminating diseases? Why not say,
well I want my kids to be a little smarter? I want my kids to be
more beautiful? I want them to be stronger? Pretty soon, the choice
of your child’s genome has nothing to do with your own genome, so
we’re kind of breaking the paths that evolution has run on, of
heritable characteristics. Now, the characteristics of our children
will be what is popular – memes choose genes in that world. I
expect there’s going to be a firestorm of controversy about that,
and I think it could be dangerous. If we don’t do it carefully, you
could have a runaway fad changing humanity in a very dramatic way,
without necessarily understanding what the full ramifications of
those changes will be. So I think biotechnology will be the first –
y’know, it’s biotechnology, nanotechnology, and AI are the huge
tsunamis coming towards us, and I think biotechnology will be the
first of them. And actually maybe that’s good. The decisions and
understanding and the way of thinking about ourselves and our
relationship to the world – biotechnology will help us get clarity
on that, so that when nanotech and AI come, we’ll have at least
some grounding in this new future we’re creating.
V: Do you have any predictions for the next 5 years, through the
year 2012?
Steve: Well, I think we can expect to see computer power, and
particularly hard disk storage, getting cheaper and cheaper. I
mean, I just bought a 750GB hard drive at Frey’s on Black Friday
for $150 bucks, which just shocked me. It kind of made me think
back a few years ago, I was envisioning what would it take to have
a terabyte of storage and you could fit it in one house and it
would be less than a million dollars, and now you can just go down
to the store and buy it. So that means you can store thousands of
movies, and I think those trends, there’s no sign of them ending
for a number of years. So, the machine of five years from now… for
instance, cameras are getting very cheap, and if you have very
inexpensive storage, at some point it’s cheap enough that you can
just record your entire life. There’s going to be a date, probably
within the next five years, where I expect to see a lot of people
who just record everything in their lives. That’s sort of
intriguing from a historic point of view, they’re creating a full
history. It also leads to a very different society. David Brin has
a very interesting book called The Transparent Society, looking at
some of the consequences of very inexpensive cameras and recording
devices, where pretty much all public space eventually becomes
monitored, and it dramatically changes the balance between
criminals and police forces. There’s potential for abuse of that,
and there’s also potential to use that kind of “su-veilllance” –
surveillance is sort of viewing from above, su-veillance is sort of
viewing from below – if you have an empowered citizenry that has
inexpensive cameras and are recording everything, it could create a
new level of people being accountable for their actions and could
transform society in a big way.
V: And finally, what specific or general predictions for next 10
years, thru end of 2017.
Steve: Well, that’s getting into the realm where we may actually
see some early forms of these technologies coming about. AI in
particular – today we have AI systems that are not of the great
power of the systems I was talking about, but they’re getting more
and more powerful. You see them in speech recognition systems.
That’s becoming more and more common. And particularly as computer
hardware gets cheaper, even just today’s pretty dumb algorithms for
things like speech recognition and handwriting recognition and
image recognition, very brute force approaches will become
inexpensive and viable. I think we can expect to see the
consequences of that much more widely. One area where I expect to
see that is in the area of video games. I think there’s going to be
a kind of blending of movies and video games. Already today, the
most powerful video game platforms can almost produce
photo-realistic imagery. Kind of interactive entertainment is
likely to become more and more important. I expect it to kind-of
expand beyond the teenage boy customer, and new forms and new
versions of those systems which are appealing to adults and to
women, I think that’s a trend we are going to see a lot more of.
And what form it takes I think is unclear. People were talking a
lot about virtual reality and Second Life. None of those systems
have really caught on in a huge way yet, so I would expect that
over that time frame the kind of understanding of our connection
between the physical world and the virtual world is going to become
much clearer.
V: Thank you for speaking with us.
Comment Thread ()
