The Indian Council of Medical Research (“ICMR”), in March 2023, released the Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare (“AI Guidelines”). The AI Guidelines have been developed after extensive discussions with stakeholders and provide guidance and a framework on, inter alia, the ethical principles to be followed, governance of artificial intelligence (“AI”) use and seeking informed consent while deploying AI based solutions in biomedical research and healthcare.
Some of the ethical principles addressed in the AI Guidelines are as follows:
- Autonomy: As per the ICMR, AI technologies can potentially undermine human autonomy by taking decision making into its hands. Accordingly, humans should have oversight and control over medical decision making and the AI based healthcare system. Patients should be provided with the option of relying on the clinical decisions of the physician and the AI technology and a consent process must be formulated to give complete autonomy to the patients to reject AI technology.
- Safety and risk minimization: Safety of a patient is of paramount importance. Accordingly, patient’s safety would be the responsibility of the stakeholders that have developed and deployed the AI technology. The ICMR suggests, inter alia, a robust set of control mechanism to prevent unintended misuse; to have secure systems and software given the sensitive nature of the data; a pre-emptive design for AI technologies to perform consistently with minimum unintended consequences; anonymization of data; development of AI technologies as per the legal and data protection requirements of India etc.
- Trustworthiness: The ICMR recognizes trustworthiness as the most desirable quality of a diagnostic or prognostic tool for use in AI technologies such that their validity and reliability can be tested. Further, as per the ICMR, such AI-based solution must be lawful, ethical, reliable and valid, explainable, transparent, and independent of a physician’s views/decision on a particular disease.
- Data Privacy: The AI Guidelines place a lot of emphasis on data privacy particularly since health-related data is of sensitive nature. As per the ICMR, privacy and protection of personal data must be ensured at all stages of development and deployment of the AI based technology. Given the sensitive nature of the data, ICMR suggests anonymization of patient’s data especially since such data can be present as Metadata and on-image data. Issues pertaining to ownership of such data would vary based on the national/regional laws governing privacy and data protection. The ICMR acknowledges that the Information Technology Act, 2000 (and the rules thereunder) is the existing law governing data privacy in the country and states that the upcoming Digital Information Security in Healthcare Act (DISHA) Bill and the Digital Personal Data Protection Bill, 2022 will be binding on the AI Guidelines. Some of the salient points on data privacy identified in the AI Guidelines include, inter alia, users’ control over the data collected for developing and designing AI technology for healthcare and the provision to access, modify, or remove such data, informing the users about the data being collected and the purpose of such collection, seeking consent of the users for taking and sharing data, taking extra security measures for technologies requiring biometric data, impact assessment by relevant authorities etc.
- Accountability and Liability: As per the ICMR, AI technologies should undergo audits to ensure optimum functioning and the audit reports should be made public. Further, representatives of the health sector should be involved in all stages of the development and deployment of such technologies. The onus of responsibility of the deployment and use of AI based technologies would depend on the facts of the case. For instance, if the harm is caused due to a malfunction, then the designer or the manufacturer will be held responsible.
- Optimization of data quality: Since AI is data driven technology, it is pertinent that due diligence is carried out to ensure that the training data is free from biases and represents all sections of the Indian population. This would be crucial in eliminating errors while using AI based technology.
- Accessibility, equity and inclusiveness: Fairness in distribution of AI technology should be maintained and equal opportunity and access to different user groups should also be provided particularly to socially and economically disadvantaged classes. Further, ICMR suggests use of local languages in the user interface to overcome language related accessibility barriers.
- Collaboration: Given the ever-changing nature of AI, it is important to collaborate with AI experts. ICMR also encourages inter-disciplinary as well as international collaboration.
- Non-discrimination and fairness: The training dataset must be accurate and free from biases. Special attention must be given to promoting and protecting the equality of people. A safe mechanism to raise concerns regarding the issues with the AI based technology should also be developed.
- Validity: Before application, the AI technology must undergo clinical and field validation to ensure safety and efficacy of the patients.
The AI Guidelines further provide elaborate guiding principles for the development phase, validation phase, and for clinical and other health related deployment. The ICMR also suggests deployment of ethical review procedures in medical AI stating that the members of the Ethical Committee should include experts from the legal field and be trained in emerging AI technologies such as big data, internet of things etc. to be informed about the developments in the field. Additionally, the ICMR states that a researcher must obtain written informed consent from the study participant for any health research. This informed consent should consist of certain components namely, providing relevant information to prospective participants, ensuring individual’s competence, ensuring that the information is easily understood and ensuring voluntariness of participation.
The AI Guidelines is a living document and will be updated by the ICMR from time to time in light of the evolution of artificial intelligence and the ethical principles governing it.
Our Take
The ICMR has taken a welcome and timely step in recognizing the increasing use of AI in the healthcare sector and the need to regulate this development. Understanding the ethical implications of AI become more relevant with the increasing popularity and use of newer AI technologies. The AI Guidelines adopt industry based best practices and provide for a robust framework for development and deployment of AI based solutions in the healthcare sector. Given that development and deployment of AI in healthcare is at a nascent stage, AI Guidelines emphasize on human intervention and oversight at every stage. The AI Guidelines are centered around protection of patients and their data and highlight, in great detail, the potential threats from the use and misuse of AI based solutions in healthcare. In light of the above, the AI Guidelines provide much needed guidance for regulation, governance, and accountability of the players involved in the biomedical research and healthcare sector and in respect of sensitive personal data.
While the ICMR has also taken upcoming legislations into consideration making the AI Guidelines dynamic and adaptable in nature, it seems to have ignored the fact that the Draft Digital Personal Data Protection Bill, 2022 has done away with the subcategorization of personal data into sensitive personal data.
Having said the above, the intention behind introducing the AI Guidelines is certainly vested in the interest of the patients/participants, providing adequate safeguards for ensuring consumer protection and the effectiveness of these guidelines will be crucial in guiding the government in developing better standards for AI and their application in healthcare as well as other sectors.
Links
Link to the Ethical Guidelines on the Application of Artificial Intelligence in Biomedical Research and Healthcare developed by the ICMR: https://main.icmr.nic.in/sites/default/files/upload_documents/Ethical_Guidelines_AI_Healthcare_2023.pdf
