People have emotional attachments to things. This is not new. Psychologists call it "cathexis", I think. We imbue things with "personalities" and create imaginary relationships with them. (We do something similiar with pets, and even very young children.)
Some work in AI enourages this. Robotics for example. The AIBO robots from Sony are designed as robotic pets. I've seen it with my own work also - the robot "Stumpy" I programmed for my honours for example. People attribute much more intelligence and personality to it than is really there. People would often call it "he". They are emotionally moved by its behaviour.
There is what I consider a very interesting research area growing in AI. It is called "Expressive AI" or "Interactive Narrative" and is about building interactive storytelling tools. It includes building software agents which are believeable characters and to which people can have emotional responses.
Again, this is nothing new. Stories, film and television have been pushing our emotional buttons for years. It's what we expect, even what we desire. Most people don't have any problem distinguishing fiction from reality. (Although the "cult of celebrity" and celebrity stalking may make us wonder otherwise.)
Still, the things we are making have potential to be much more "real" than these. A character in a novel doesn't react to us, doesn't tell us it loves us.
The comparison to pets is also valid. Clearly pets do have real emotional reactions to us, at some level. Most people understand this and value it, without inflating it beyond its appropriate level. (Although, again, we can think of cases which go beyond this in what is an uncomfortably unhealthy way.)
But a pet is still a pet, not a person. We can dress them up in people- clothes, but they'll never actually talk to us. Computer simulations, on the other hand, can be arbitrarily realistic. No that I think we will ever solve the strong AI problem (human-level intelligence) but we can certainly make the appearance strong - and people are more easily deceived than you'd think. We naturally model complex systems as personalities. Put a face on it, give it a voice, and see how many people will tell it their life story.
So my question is: What kind of ethical concerns should we have about developing these kinds of emotionally rich agents? What are the dangers for individuals, and for society?
Loving relationships, with people and with God, are the foundation of Christian ethics. "Love the LORD your God" is the first and greatest commandment. And the second is like it "Love your neighbour as yourself." (Matt 22:34-40). How do relationships with things, with pets, with comptuers, with fictional characters, fit into this scheme?
It's easy to cast ethical questions in AI as pie-in-the-sky science-fiction scenarios and pointless scaremongering. But this research is happening now and the results are already visible. Is this no more harmful than loving a good book, or your pet dog? Or is it turning into something else? I honestly don't know.