Healthcare organizations are continuously looking for ways to reduce the costs associated with repeated rounds of testing of their information systems. Many investigate automated testing tools as a way to take the burden of their analysts.
These organizations know how much clinical and non-clinical staff alike depend on the software solutions they use to help in delivering and supporting the care they provide to the patients they serve.
However, many of them do not understand the importance of having the clinical and non-clinical end-user staff participate in testing new software solutions prior to moving them into production use. While there is a place for automated testing, it is not in the integration testing or user acceptance testing phases of the implementation. There is a great risk in not involving end-users in the system and end user readiness processes.
This whitepaper will define what integration testing (IT) and user acceptance testing (UAT) are, why every healthcare organization should ensure that both clinical and non-clinical staff participate in these types of testing and why that eliminates automated testing for these testing types.
What is Integration Testing?
IT tests that a group of solutions work together to achieve the desired outcomes. The testing is workflow based and includes all integrated and interfaced solutions.
In healthcare, it includes everything from the time a patient first encounters the healthcare organization to the time that the claim is paid. It tests that end users, from scheduling & registration, to the clinicians, and finally to the billing office, can efficiently get their jobs done.
IT usually includes the subject matter experts (SMEs) that helped in the overall design of the solutions. However, some organizations choose to have the application analysts that assisted with configuring the solutions execute IT, eliminating the SMEs from the equation.
What is User Acceptance Testing?
UAT is like IT in that it is workflow based and includes everything from the time a patient first encounters the healthcare organization to the time that the claim is paid. However, it is different from IT in that it confirms that it meets the end users needs.
For this reason, UAT must be done by SMEs or other selected end users who can evaluate the system with a critical and practical perspective. It is recommended that UAT be done utilizing the same test scripts, or a sub-set of the scripts that were utilized for IT.
Some organizations think of UAT as the “final round” of IT. Sadly, some healthcare organizations eliminate UAT all together.
Why Clinical and Non-Clinical Staff Should be Testers
The 2013 article called “Go-live Gone Wrong”, by Bernie Monegain, in HealthcareIT News, outlined just how devastating to a large-scale software implementation it can be to forgo the use of clinical and non-clinical staff in system and end user readiness.
In the case of this implementation there was nothing broken in the system build, but the end users did not understand the new workflows at go-live, and did not understand that the surgical nurses in the maternity area, for example, would need to manually enter charges into the system. Because of this, the healthcare organization did not charge for C‑section births for a period of 6 weeks.
Had they based their training off of workflows, the surgical nurses in the maternity area would have seen that the workflow had changed, and that they needed to manually enter their charges for the charges to appear on the claims. In this case the training was not workflow based and only taught feature functionality to the end users.
Since it is not referenced, we can only assume that the SMEs (such as the maternity unit nurse anonymously quoted in the article, or any of their peers) did not participate in IT or UAT. Had they done so; they would have realized how different it would be for these nurses to need to begin entering charges into the system and would have incorporated it into their training.
Although this is only one example, there are many, many examples just like this, across all vendors, that can be pointed to which led to millions of dollars of loss for healthcare organizations.
Whether it is claims that went unbilled because coders did not know the workflow to remove billing holds that has been manually placed by someone else, or a medication error because the home medications were not appropriately documented at the time the Physician completed medication reconciliation.
It seems clear that to reduce the number of workflow issues at go-live and enhance user readiness and adoption, SMEs should be utilized throughout the entire implementation:
- Documenting current state workflows so that important details do not go missing
- Working with the vendor to understand and document future state workflows
- Building IT and UAT scripts that are based off the highest volume, highest revenue generating patient care scenarios that incorporate all the most important workflows and details
- Participating in IT and UAT
- Creating training materials that are workflow based and incorporate all the most important workflows
- Providing at-the-elbow support at go-live to those that were not involved in the implementation.
Why Automated Testing Should Not be Used for IT/UAT
Healthcare organizations are always looking for ways to save money. Adding tools to help facilitate and improve the quality of testing is absolutely a way to save resource costs during implementations and reduce the number of issues encountered post-go-live.
However, the right tools for the given purpose need to be put in place. While there are tools out there that automate the running of test scripts to remove humans from the testing process, and while this might be helpful to use such tools for volume testing, it is never the right thing to do with IT or UAT.
Only humans will be able to walk through a test script and confirm that they understand the workflows in the test script and would be able to repeat them in the real world. Remember that in the case of the organization in “Go‑Live Gone Wrong”, the system worked perfectly as designed. The issue was that the workflow was not completed correctly by the end users, leaving millions of dollars of charges unbilled.
Organizations are better served picking a tool for IT and UAT that is designed to make it easier for testers to test, rather than eliminating the testers.
Talk to Our Experts
You don't have to try to figure this out on your own. Get the help you need to ensure better outcomes and be confident in your testing approach.