I think there are more fundamental questions needing to be answered:
- What does it mean to be human?
- Can an AGI be considered human? We need to answer the previous question in order to answer this question.
- Will the human race have gone extinct if an AGI arises that embodies human values and is considered human?
- Finally, assuming such a "human" AGI were to arise, would its legacy then also be the legacy of the human race?
Maybe all of history was building to this point. Different religions have discussed this for thousands of years - man breaking out of the confines of his body. How do we know that AGI isn't the way to achieve that? I can tell you one thing, I can conceive of an AGI exploring the galaxy. Meanwhile it's hard to imagine humans exploring the solar system. I think we're going to reach the conclusion that physics and biology prohibitively confine us - confines that don't exist for AGI.
I think there are more fundamental questions needing to be answered:
- What does it mean to be human?
- Can an AGI be considered human? We need to answer the previous question in order to answer this question.
- Will the human race have gone extinct if an AGI arises that embodies human values and is considered human?
- Finally, assuming such a "human" AGI were to arise, would its legacy then also be the legacy of the human race?
Maybe all of history was building to this point. Different religions have discussed this for thousands of years - man breaking out of the confines of his body. How do we know that AGI isn't the way to achieve that? I can tell you one thing, I can conceive of an AGI exploring the galaxy. Meanwhile it's hard to imagine humans exploring the solar system. I think we're going to reach the conclusion that physics and biology prohibitively confine us - confines that don't exist for AGI.
[dead]