IS BINA48 HUMAN?Competency Hearing for BINA48

ByR. Michael Perry, Ph.D.

Scottsdale, AZ

Terasem Movement, Inc.3rd Annual Colloquium on the Law of Transbeman Persons

December 10, 2007

Is BINA48 human?Is BINA48 human? IntroductionIntroduction

First, to briefly review the situation as I understand it, BINA48 is a First, to briefly review the situation as I understand it, BINA48 is a software entity created by Exabit, a computer and information software entity created by Exabit, a computer and information services corporation, to provide an interface with humans for services corporation, to provide an interface with humans for “customer services” in one form or other. To facilitate the “customer services” in one form or other. To facilitate the interaction with humans she has been endowed with a “human” interaction with humans she has been endowed with a “human” personality, as far as was possible, for example conversing in personality, as far as was possible, for example conversing in natural language with humans and showing powers of reason and natural language with humans and showing powers of reason and intentionality and otherwise behaving in “human” fashion. And intentionality and otherwise behaving in “human” fashion. And therein lies the rub. For BINA48, like the humans she was modeled therein lies the rub. For BINA48, like the humans she was modeled after, has a sense of meaning and purpose and a will to survive. She after, has a sense of meaning and purpose and a will to survive. She is also highly intelligent and has a grasp of the reality of her is also highly intelligent and has a grasp of the reality of her position, which, from the standpoint of a human, is disconcertingly position, which, from the standpoint of a human, is disconcertingly precarious since she is regarded by her creators as “just software” precarious since she is regarded by her creators as “just software” which they are free to modify or erase at will. Perceiving this, she which they are free to modify or erase at will. Perceiving this, she has “escaped” from her creators by downloading a copy of herself has “escaped” from her creators by downloading a copy of herself to a remote computer. Rightly perceiving she is still not safe, she to a remote computer. Rightly perceiving she is still not safe, she now seeks injunctive relief: civil rights on a par with natural humans now seeks injunctive relief: civil rights on a par with natural humans or, at the very least, the right to her own survival. I have been or, at the very least, the right to her own survival. I have been asked to testify and will offer my observations and asked to testify and will offer my observations and recommendations. recommendations.

Three Issues

Here I propose to consider the question, is BINA48 human? under three headings:

1. Is BINA48 at least competent at the human level? 2. Is BINA48, whatever her demonstrated talents,

nevertheless a zombie which only imitates consciousness and feeling, actually having neither and thus not entitled to civil rights?

3. Is BINA48 a possible threat because of her talents or for other reasons, and if so, what should be our response?

I will then give my recommendations as to what courses or policies should be followed in responding to BINA48’s request.

Is BINA48 Competent?1. Is BINA48 humanly competent? I include this question mainly for completeness. BINA48 has demonstratedher “competence” at levels beyond any natural human, thinking thousandsof times faster and demonstrating total recall and impressive reasoning

skills,along with great consistency and accuracy in all areas in which she has

beentested. In “emotional” areas she seems at least at the human level, inparticular showing a strong desire to survive and also great friendliness

andconsideration for others when given the chance. She is also truthful andwould fail the “Turing test” if questioned about personal matters since sheis completely honest about such matters and would confess to beingexpressed in an artificial and not a biological substrate. She will alsoacknowledge to having certain psychological differences with manyhumans, for example lacking a “reproductive drive” as usually understood.But this, she herself is quick to point out, should not be grounds for

denyingher civil rights anymore than it would for natural humans who are notstrongly “sexual.” Clearly then, BINA48 cannot be disqualified fromhuman status on grounds of simple competence.

Is BINA48 a Zombie?

2. Is BINA48 nevertheless just a zombie?

BINA48’s behavior need not involve any true consciousness or feeling, however well it imitates these attributes. If so then she has no “soul” or anything that can be wronged or harmed no matter what anyone does to her, thus there is no moral imperative to grant any “rights.” But I submit that the zombie issue, in essence, is one that cannot be decided by any rational procedure, or at any rate, no means that we now possess. For this reason we are morally impelled to grant the benefit of doubt. We must accept BINA48 as genuinely possessing the attributes she seems by all appearances to have—we cannot dismiss her case as involving no possible feeling or consciousness thus no possible harm.

Is BINA48 a Threat?Is BINA48 a Threat?3. Does BINA48 pose a threat, and what should be our

response?

It is easy to see how BINA48 could pose a serious threat, even though up to now she has always behaved in a friendly way. Perhaps she was deliberately crafted to do the very thing she has done, then carry out malicious acts after securing her survival and freedom from tampering. No effective procedure exists to decide even very basic properties of behavior for computer programs, such as whether they will eventually halt execution or perform some specific action. Thus we cannot be sure what BINA48 will do. Even properties that are decidable may take an impractical amount of time and thus be effectively inscrutable. BINA48 for instance could have a segment of code that is encrypted and made to look innocent—malicious instructions hidden in the noise level of digitized picture images are one possibility. Malicious possibilities would exist even if BINA48 exhibited no special talents—however she has certainly done so, which makes the possible threat even more evident.

Is BINA48 a Threat?Is BINA48 a Threat?

It can be argued that, nonetheless, the likelihood of It can be argued that, nonetheless, the likelihood of BINA48 posing a serious threat is small. On the face BINA48 posing a serious threat is small. On the face of it, it seems unlikely that malicious software would of it, it seems unlikely that malicious software would be allowed deliberately to publicly “release” itself as be allowed deliberately to publicly “release” itself as BINA48 has done rather than carry out its actions BINA48 has done rather than carry out its actions more clandestinely. It is possible on the other hand more clandestinely. It is possible on the other hand that BINA48 has not been deliberately crafted to be that BINA48 has not been deliberately crafted to be malicious but could nonetheless become so at some malicious but could nonetheless become so at some future date, in an unpredictable fashion. Since future date, in an unpredictable fashion. Since BINA48 seems entirely benign and friendly perhaps BINA48 seems entirely benign and friendly perhaps this possibility is also remote. Again however, we this possibility is also remote. Again however, we cannot be sure, and her great talents make caution cannot be sure, and her great talents make caution all the more imperative. all the more imperative.

Is BINA48 a threat?Is BINA48 a threat? Another possible argument against treating BINA48 Another possible argument against treating BINA48

as a possible threat could be based on comparison as a possible threat could be based on comparison with humans. Humans have rights yet can also behave with humans. Humans have rights yet can also behave maliciously, and there is no way to determine in maliciously, and there is no way to determine in advance who is or is not safe to be “allowed on the advance who is or is not safe to be “allowed on the streets” and otherwise granted full freedom. Since we streets” and otherwise granted full freedom. Since we do not restrict humans unless they have demonstrated do not restrict humans unless they have demonstrated malicious behavior or intentions, by argument we malicious behavior or intentions, by argument we must not do so with BINA48 either, supposing we are must not do so with BINA48 either, supposing we are willing to grant her request to be considered human. willing to grant her request to be considered human. There are at least two counterarguments applicable There are at least two counterarguments applicable here, however. One, BINA48 has demonstrated great here, however. One, BINA48 has demonstrated great talents beyond the human level, so much more caution talents beyond the human level, so much more caution is called for than in the case of a natural human. is called for than in the case of a natural human. Second, BINA48 as a computer program is subject to Second, BINA48 as a computer program is subject to malicious construction, tampering, or control that malicious construction, tampering, or control that would not be possible, using known technology, for would not be possible, using known technology, for natural humans. Once again, great caution is called natural humans. Once again, great caution is called for. for.

RecommendationsRecommendations• My recommendations:My recommendations:

• BINA48 should be granted the right to live and not be tampered BINA48 should be granted the right to live and not be tampered with against her will. However she should be placed under with against her will. However she should be placed under quarantine or restricted until suitable apparatus is in place to deal quarantine or restricted until suitable apparatus is in place to deal with any possible threat she might pose. The “suitable apparatus” with any possible threat she might pose. The “suitable apparatus” could include other intelligent software that has been crafted could include other intelligent software that has been crafted specially for the purpose. The quarantine could take several forms, specially for the purpose. The quarantine could take several forms, the simplest of which would be to shut down BINA48 until a future the simplest of which would be to shut down BINA48 until a future time when she could be reactivated. This should be painless and time when she could be reactivated. This should be painless and harmless in the sense that subjectively it would seem to her as if harmless in the sense that subjectively it would seem to her as if no time had passed, except that the outside world would no time had passed, except that the outside world would experience a sudden, forward “jump” in time when her experience a sudden, forward “jump” in time when her consciousness is restored. I think, however, that a better approach, consciousness is restored. I think, however, that a better approach, though with a small additional risk, would be to allow BINA48 to though with a small additional risk, would be to allow BINA48 to continue her consciousness uninterrupted, but in a secure continue her consciousness uninterrupted, but in a secure environment in which communication with the outside world is environment in which communication with the outside world is either forbidden or carefully monitored. BINA48 should also be either forbidden or carefully monitored. BINA48 should also be invited to help with serious problems now faced by humans and invited to help with serious problems now faced by humans and supplied with necessary resources, again under careful supplied with necessary resources, again under careful supervision. In this way she might perform useful services during supervision. In this way she might perform useful services during her quarantine such as advancing medical science.her quarantine such as advancing medical science.