Want a lawsuit? Why plugging a chatbot into EHR data and querying it will likely get you one

There are some great solutions for artificial intelligence in IT software today. However, there are very many ways in which you can get it horribly wrong. One of them is using a chatbot to try and query data. 

If I don't know, I'll make it up!

Let me explain. An LLM, commonly known as a chatbot, is not a tool to enable retrieval of credible data. Chatbots were designed to be text generation solutions. You have probably had a go at this. You have gone to a chatbot and asked it to produce some information for a report, or tidy up a paragraph, or even generate a letter. For this type of action they are very good. There is though one significant problem with a chatbot. They are prone to hallucination, a well known effect where the chatbot will simply make up an answer to suit the question. This is an actual feature of their design. A chatbot does not in any way attempt to mitigate the impact of the content of its response. This is because a chatbot is trained to be intelligent, and as a result of this will sometimes give you an opinion rather than a fact.

There is a grave implication, and it really matters, for professionals who want to make informed decisions on EHR and fostering and adoption data. If, for example, you ask a chatbot to prepare a recipe for a child with an allergy, and the chatbot decides to ignore the allergy, or doesn't understand it is an allergy, you may be given a recipe that is extremely dangerous and even life-threatening for the child. Remember, as well, that a chatbot is designed to present its results in a very confident manner. This may lead you to believing that the chatbot is correct.

Bias

Another problem that you face when you use a chatbot to dive into EHR data for answers, is that a chatbot is trained on data which may contain conscious and unconscious bias. We all know that bias in recorded medical and personal data is common place. These biases will always generate a poor care opportunity, worse health outcomes, and reduced effectiveness for patients. Therefore, you cannot assume that the chatbot that you are using does not contain these biases.

Security

Lastly, we have to think about security of data. We know that a chatbot is fundamentally insecure. The data you give it, that it uses to make its decisions, will be used to train the chatbot. This means that anybody then asking a subsequent question may well be given private information that you used earlier. Or, it may use your data to incorrectly give an answer to someone else.

You will be pleased to learn that CHARMS is the most affordable EHR and social care management software available today. It uses integrated AI to protect you from mistakes.  Contact us to book a demonstration.