Welcome to today's educational event titled AI 101, Terminology, Applications, and Considerations for Sleep Medicine. It's sponsored by the American Academy of Sleep Medicine. This webinar was developed by ASM's AI and Sleep Medicine Committee in collaboration with ASM Sleep Technologist and Respiratory Therapist Education Committee. Before I have our speakers introduce themselves, I'd like to remind our attendees that your audio is muted and your video is off, but we will definitely want your input. During the presentation, I encourage you to ask questions using the Q&A function of the Zoom platform, and we will address those questions at the conclusion of our presentation. So now I'll have each speaker present, give a brief introduction of themselves. So Sam, let me start with you. Awesome. Thanks, Dr. Oakes. My name is Sam Rusk. I'm co-founder and president at Enso Data. I developed a passion for signal processing, machine learning, and healthcare after receiving a degree in electrical engineering from the University of Wisconsin, and I currently lead our AI team at Enso Data where I support both our automated scoring product, Enso Sleep, and our other research initiatives. Thank you. Johan? Yes. Hi. My name is Johan Augustson. I'm the VP of Artificial Intelligence and Data Research here at Knox Medical. My background is in electrical engineering and physics. I've been very much working in the field of sensors, data acquisition, and data processing, and here I lead the team who is developing our AI products, doing the clinical validations, and participating in research projects. Thank you. Hauke? Yes. Hi. I am an instructor at Mass General Hospital and Harvard Medical School at Boston. I did my postdoc with a focus on AI applications in sleep medicine to extract brain health sleep signals. I have served two years in the AI in Sleep Medicine Committee in ASM. Anuja? Hi, everyone. My name is Anuja Bangipatai. I'm a pediatric pulmonologist and sleep physician at Riley Children's Hospital in Indiana. I currently serve as the chair of the AI in Sleep Medicine ASM Committee, and I'm so happy to be here today. Thank you. And Joseph? Good morning, everybody. Whatever you are, my name is Joseph Castillo. I hold a medical degree for the National University of El Salvador. I am a sleep technician with a 15-year career as a sleep educator. I work for several sleep labs in California, for Sunset Sleep Lab in Texas, for Ascension Syphon Medical Center, because I live in Austin. And also, I am a clinical instructor for the first school for polysomnography for Latin America, which is based in Colombia and Puerto Rico. Thank you. Thank you. And last but not least, Mark? Hey. Good morning. My name is Mark Spiceland. I'm located at Ascension St. Thomas in Nashville, Tennessee, sleep lab here in Middle Tennessee. I serve on the ASM Sleep Tech and Respiratory Committee. I'm also the secretary of the PSG Standards Committee of Tennessee. I'm happy to be here today and look forward to this panel. Awesome. Thank you. And I'll introduce myself. I'm Dr. Margarita Oaks. I'm a sleep medicine attending at Lenox Hill Hospital in New York City, as well as the associate medical director of our local sleep center. So let's get started. Okay. Hao-Ki, so let me start with you. So if you can please explain to us, what is artificial intelligence? Yes. Thanks, Dr. Oaks. Yeah. So AI is an area that gives machines the ability to seem like they have human intelligence. So this can include at least four domains, such as learning, planning and motion, reasoning, and general intelligence. So we can first look at learning. So learning can mean learning to classify whether it's a cat or dog image, or learning to predict whether it will rain tomorrow or not, or learning to find phenotypes, for example, the OSA with resistant hypertension. And then for planning and motion, it basically means how do you plan optimally and act optimally to optimize a future outcome, such as a adaptive strategy to improve CPAP adherence. And then in terms of reasoning, it just means the ability to explain why these patients, for example, will have a bad outcome. Is this due to some end organ failure or not? And general intelligence, it means the machine can solve a wide range of problems and make past curing tests to replace a doctor. So what I think many people find confusing is the subdivisions of artificial intelligence. So Hao-Ki, if you can please explain the difference between machine and deep learning to our listeners. Yes. Yeah. So as you see from the figure, there are three circles. So the outside circle, that's artificial intelligence. Let me explain them one by one. So there are multiple definitions for artificial intelligence, but in general, it is the field that makes a machine to perform like human to solve a problem. But AI itself does not have to confine itself to methods that are biologically feasible. And inside AI, there's a machine learning, as you can see from the figure, which is a subset, which focuses on the use of data and algorithms to imitate the way that human learns so that it can gradually improve its accuracy. And deep learning is a further subset of machine learning. And deep learning automates much of the algorithm design, eliminates some of the required human knowledge, and enables the use of larger and less pre-processed data sets. Okay. Thank you for that. So if you can explain in a little bit more detail exactly how does machine learning work? Yes. So basically, machine learning algorithms, it learns from data with human-specified features without explicitly programmed commands, and it can improve performance with experience. So here on this slide, we can see on the right-hand side, there are several steps of machine learning. So let's say you're looking at a picture of your house, and you try to decide whether that picture contains or not contain your car. So to determine whether it contains a car, machine learning will need to know some patterns in the picture, which we call features. For example, a car can have wheels, can have headlights, taillights, and doors in the picture. And these features or patterns sometimes can be noisy. So if you are viewing your car from an unusual angle, they could be noisy. But the machine learning algorithms can have a way to combine these noisy features into a yes or no answer, whether there's a car or not, or the probability of having a car in the picture. Okay. So then... If we go to the next... Yeah. Yeah. So then if machine learning allows computers to act almost like humans, then what does deep learning entail? Yes. So deep learning is a type of machine learning. It's a subset. And deep learning algorithms themselves can learn the important feature from the raw data. So just now, in machine learning, we have noticed that having those good quality features, for example, the car, the shape of the car, can make it easier to make decisions. But these good quality features, they require our human knowledge. Deep learning takes one more step forward by removing this need for human-engineered features. So to do so, deep learning networks, usually they have additional steps or additional layers to first learn the important features from the raw data. Once these learned features are extracted, we can combine these features in the same way as in machine learning. So the fact they need these additional layers to learn the features, that's exactly the reason why they are called deep learning. Got it. So then, so that I understand and our listeners understand clearly, deep learning is essentially training a computer to make human-like decisions. And then the computer can then independently use those artificial neural networks to answer further questions. So exactly how does this process work? How do machines actually learn this information? Right. Yeah. So as you can see in this slide, in general, we first need some sensing technology to collect the data. For example, as we are all familiar with images or sounds or text or sleep signals. And then we can pass these data to be processed by AI algorithms. And these AI algorithms can run either on CPU, you know, the central processing unit, or GPU, which is graphical processing unit in your computer. Actually part of the reason why GPU becomes popular in AI algorithms is because of its extraordinary ability to run multiple and simultaneous computations, which significantly speed up the model training. So and the last step in terms of learning, there could be different types. For example, this can include supervised learning, where an answer is provided during the training, so that the machine can learn what is the right answer, or it can be unsupervised learning, where there's no answer provided during the training, so that the machine itself needs to find some hidden structure or hidden pattern in the data sets. And the last type is called reinforcement learning, which is pretty interesting. It's the type where machine learns to learn the optimal action to interact with the real world to maximize some reward in the future. So, yeah, so if you can just tell us a little bit more about the deep learning networks. Right, yeah, so as you can see from the slide, how does deep learning work? Then deep learning networks have a goal in their mind, and in terms of terminology, it's called loss function. So and it has a goal in the mind, and also it has a way to slowly adjust their parameters to achieve that goal by using something called error-backed propagation. And by looking at or learning more and more examples, the network will measure the number of errors made, and then adjust its parameters to get closer to that goal. So, for example, if we try to classify wake versus sleep from PSU signals, the network will actually slowly learn the boundary between wake and sleep, which can be very nonlinear as we show in the figure below. Yeah, that's one example. And on the right-hand side of the figure, I just want to show that this is an example of deep learning neural network, which is called Inception Network, which it has multiple versions, and the first version has 27 layers. So I just want to mention that recent deep neural networks can have many layers. Here is 27, and some network can even have more than 100 layers. Got it. Thank you. I know that was a lot to explain. So now that we have the basics of AI so eloquently explained by Hauke, I want to switch gears a little bit and talk about some of the applications of AI. So Anuja, where is AI being used now, and is it really a tool that's available only in sleep medicine? So glad you asked that question. So AI is everywhere. I can guarantee you between the moment we woke up until right now when we came to work, each one of us have used some form of AI-assisted algorithm or not, be it the facial recognition for your phone, or if you have a fancy Tesla, then be it your self-driving car. It is being used regularly in your day-to-day life. We don't even have to look that far. If you look at medicine in general, look at the natural language processing being used for voice recognition to complete clinic notes, look at the healthcare data being stored in the cloud, AI literally is everywhere, even in medicine. Now, looking at sleep medicine, so where are we in terms of AI applications? So sleep medicine has a very unique position because it completely can leverage that eight hour of uninterrupted physiologic data that we get on our polysaminograms. That combined with the data that we get from sleep tracking devices combined with our EHR data can all help us towards our goal of personalized treatment. The most common application possibly of AI application in sleep medicine, though, would be AI-assisted algorithms, which help us in PSG scoring. So PSG scoring, if you look at what do we know from the ISR, we know that our typical interscorer rate of reliability is around 82 to 83%. There was a meta-analysis which came out a couple of years ago, which showed that the interscore or the score of reliability when compared between AI-assisted algorithms was comparable at around 85%. So in the last five years, AI-assisted PSG scoring has grown by leaps and bounds, and I'm really excited to see where it's going to go from here. Absolutely. It's a fast moving field. So Sam, let's expand a little bit on what Anuja just said. So can you please tell us how artificial intelligence is being employed in polysomnography scoring? Yeah, absolutely. As mentioned, computer-assisted scoring of in-lab polysomnograms and home sleep tests are growing in adoption. So the automated scoring of all sleep staging, respiratory events, arousal events like movement events, and many others allow for technicians to review studies faster and allow for more time with direct patient care and education and coaching. And these tools are being used to address the epidemic scale sleep apnea population in the US and around the globe. And so computer-assisted scoring is helping to lower the technical burden required to successfully diagnose and treat our patients. Absolutely. And John, can you maybe discuss a little bit about how the PSG data is actually analyzed? Using artificial intelligence and the automated scoring systems available? Yeah, there are different approaches being used. And there are several mentors who are providing medical devices who are applying AI for sleep diagnostics. We have mentors which are providing cloud-based services, while others are using a locally installed PC software. Some of them are using data from many recording devices, while others are specifying for specific devices. So they're applying different approaches. And then also depending on the scoring task at hand, for example, scoring oxygen desaturation does not require the same complexity of a model as maybe scoring cortical arousals. So we are also seeing that mentors are using simple models for simple tasks and really big, deep learning models for complicated tasks. Got it. So we know that AI conquered PSG, so to speak. So what other areas of sleep medicine do you think are going to be targeted by AI next. So Sam, why don't you take this one? Yeah, I think going beyond the computer-assisted of scoring of sleep studies, machine learning has shown promise to assist in the diagnosis of other sleep disorders. So drawing from examples in the literature in a 2018 publication from Nature Communication titled Neural Network Analysis of Sleep Stages Enables Efficient Diagnosis of Narcolepsy, they demonstrate the ability to detect type one narcolepsy from an in-lab polysomnogram. And so what they did is leverage various novel features ranging from EEG-based sleep fragmentation characterization to the standard sleep study report values and other clinical data. They were able to show comparative sensitivity and specificity when compared to the clinical standard of care in diagnosing type one narcolepsy to the MSLT diagnostic test. And so, yeah, where else do we have eight hours of patient's physiologic data recorded in a hospital? And so I think seeing papers like this and other evidence polysomnography may hold a wealth of unused information that only machine learning tools can extract. So how can the data that we mine with AI be used to advance the sleep medicine arena? Yeah, I think even taking a step back from polysomnography, human health and disease are incredibly complex systems. And so sleep is certainly a player, but not always the single cause. So biomarkers from sleep, whether from the standard report level indices or other machine learning outputs can help us to understand the role of sleep in the entire spectrum of human health and disease. So making sleep data available as another dimension in epidemiological research will help uncover the importance of sleep health and how it's affecting the rest of the human body and certainly negative health outcomes. Yeah, absolutely. So John, as you know, the idea of endotyping and phenotyping in sleep medicine has been a very active area of research in our community and especially in the management of obstructive sleep apnea. So how can AI pave the way for personalized medicine? Yeah, it's like Sam mentioned, there's a wealth of data collected during a eight-hour sleep study. And where else do you get access to uninterrupted physiological signals for eight hours? It's very rare. And the human scoring and annotation of sleep data is very limited to what a human can do in a reasonable amount of time. But with machine learning, we are not bound to what the human can comprehend in their brain, but we can actually detect much more subtle patterns. We can detect maybe interactions between different physiological signals, which are maybe not so obvious to the humans. So actually by applying AI, and I would just call them kind of advanced analytics, we can actually start to connect what we see in the sleep study more directly to the health outcomes and maybe the physiology of the disease. For example, for OSA, I mean, sleep apnea, it's not one thing. There are many causes of sleep apnea and different presentations of sleep apnea, they can also have different clinical outcomes. So by being able to kind of use the wealth of data available from the sleep study to determine what is the underlying physiology for the sleep apnea, we can start to think about them providing tools for treatment developers to develop treatments to target specific causes of sleep apnea and also for clinicians and researchers to understand what are the clinical implications of different presentations of, for example, sleep apnea. Got it, thank you for that. So I wanna switch gears a little bit once again and discuss, I think, what is one of the most important aspects of artificial intelligence in sleep medicine, and that's the concern surrounding it. With any new technology in life or in medicine, it's natural for there to be some hesitation, right? And even fear of anything new. And a lot of this fear is surrounding kind of how we're gonna impact the established status quo. So I wanna talk about some of these concerns that have been raised regarding AI. So Mark, let me go to you. So do you think that AI can coexist within the traditional framework of established sleep medicine practices? Yeah, thanks, Dr. Rhodes. I think that's a tough question. Like, I do think that it can. It already is coexisting within the traditional status, but AI also has the ability within it to change the landscape of sleep medicine. So, you know, it's hard to answer. You know, what I'd like to suggest to our techs out there is that the incorporation of AI to sleep isn't a bad thing. It's just a new tool that we can use and we need to absorb into our traditional status. You know, most technicians that I speak with, they harbor some kind of fear about AI. They're really just worried about it taking their job. You know, I have to remind them that I have yet to experience one year as a sleep tech in which I didn't hear my job was being threatened or becoming obsolete, whether it's because of HSATs or APAPs or new oral appliance, implantable devices, drugs on the horizon. And, you know, now it's just AI. I think I saw an advertisement for an ear pap recently, you know, so it never stops. There's always something. And what's funny is when you talk to sleep techs who have been doing this for a few decades, they share their stories of the novel procedures or technologies in their time that were going to take their job decades ago. The UPPP is one example of that. I also heard from a sleep tech recently that in the early 2000s in their hospital, they introduced split night studies and this to the techs thought, you know, that it was going to take away half their jobs because they needed half of their studies. You know, my personal opinion is that we as sleep techs need to realize and accept that all these change boogeymen are just potential tools in our arsenal that are going to provide better, quicker, and more expensive care to our patients. And they're not direct competitors for our job. I, you know, change is scary. AI could change the landscape a bit in the future, but if we're ready to embrace that change and realize it's a tool to be utilized and not feared, we'll be better served by it. You know, and us as sleep techs will move with that change as opposed to standing against it. So what do you think are some of the barriers that exist to kind of making that transition so that there's less fear in the community? Besides accepting that it's just a tool, for me, it's just trust. It's trust that it won't take your job, but it's more importantly, it's trust that it works. It's trust that it can help serve the patient better. I believe it'll take some time to earn that trust, but everything does, right? I, you know, at some point decades ago, there was a salesman that probably went to a physician's office and tried to sell a pulse ox and said, you know, I have this revolutionary new technology that shines infrared lights, you know, through your finger into, excuse me, your nail beds and using diodes, it gives you an estimated amount of your O2 saturation. And the physicians probably thought it sounded like science fiction. And, you know, the salesman said, well, this is gonna save you time. And it's almost as accurate. And there was some RT in the distance who said, hey, you know, you're trying to take my job. But through the years, you begin to trust it. You begin to realize it works. And AI is just like that, right? It'll take a few years to gain trust. And it's, you know, it's up to the manufacturer and the developer of these AI platforms to prove that it works in order for it to gain that trust. But once sleep as an entity knows that it works and trust that it isn't here to take away jobs and it helps patients, I don't think we'll have many people left on the fence. Yeah, I completely agree. So Joseph, what's your opinion on this? Do you think that AI will actually replace sleep and respiratory technologists at some point in the future? Well, thank you for the question. Thank you for the opportunity to share my experience. The short answer is no, but let me explain you why. Let's go back a little bit. Let's talk about in the past, what we, with the sleep tech, before we were used to score a sleep study in three hours, four hours, we were reviewing hundreds of pages and pages and pages. Then the software, the sleep studies came with the integration with the sleep software. Then we started scoring sleep studies from scratch, possibly from one hour to one hour and a half. And with the AI, the interfacial intelligence, it can take us sometimes a few minutes. That one was one of the fear for my colleagues. This is on a sleep technician. They didn't want to embrace the new technologies. And they said, a computer is not going to do my job. He cannot, a computer cannot do it better, or cannot help me to do it as close as the way that, the best that I can for patient care. So now, instead for me to score a sleep study per hour, I can score 20, review 40, and I can get focused in other areas. The AI, it replaced my job as a scorer, but it opened other doors. It allowed me to do more patient care, to talk with a patient, to practice sleep caution, to aid the sleep physician. And now, the AI is not the future itself. The future is the people or the technician, the physician, the old and the new generation who is going to embrace it. Currently, we have in the sleep lab, a lot of tools with AI to help us to expedite our patient care or to get more focused because of sleep. For example, we used to say, apnea is not just numbers, right? It's more than that. So AI is not just a machine that is going to take away your job. It's more than that. It's the right tool that is going to benefit you, is going to benefit your patient, but you need to carefully oversight. AI is not going to do the job by itself alone, and you never are going to review the sleep study. You are not going to put your clinical input, your clinical assessment. You are not, it's not going to talk to your patient. It might help you to fill out a questionnaire, to select the right mask or the, we are not with a rule measuring nose. We used to send the patient before with three types of mask and let the patient know, provide us, give us a call and let us know which size is right for you. Now we can select the right mask with the aid of the artificial intelligence. And then we can use a smartwatches to measure or to use this as an actigraph, as a sleep tool. And I can go over and over with a lot of example, but the future of AI is not the AI itself. The future is you, who is behind and listen us, the people who needs to embrace it. It doesn't matter if you are as big as sleep lab with 20 bed. It doesn't matter if you are a small sleep lab with two beds. It's going to benefit you. It's going to make you grow to make, or to grow in different way for the patient care, and also do yourself. It's going to be a good interaction. So going back to the answer before I ask them more, because I love sleep medicine, it's not going to replace the job. It's going, you are the one who is going to direct how it's going to make an integration with your clinic, with your practice, and what the jobs are going to be chief, but the jobs are not going to disappear. And that is the typical example with myself. I have a score over a little bit, 30,000 sleep studies in the past eight years, I guess. I score between 5,000, 6,000 a year. It's not because I have, I am an octopus. I have some clients to help me. I use different computers and also, but in a few years, I might say, oh, I just score a hundred sleep study, but it's not just a number. It allow me to take care of a hundred patients. I help people. That's why we are here, right? Absolutely. Thank you. Thank you for that. I think it's a very valuable experience. So I have a question for you, John. So you have the honor of being based in Iceland. So have you noticed a difference? I noticed on the state side, sometimes there's a lot of discrepancy about the adaptation of AI in some of our sleep centers. What about internationally? And obviously you can only speak for your home base, so to speak, do you notice that there's a lot of reservation between within the individual players within the sleep practices? So from the clinicians and the sleep practices, I don't see systematic differences in the interest. I mean, most of us are just humans and we are very intrigued by new technology and we find it very exciting and we all get excited about the possibilities. There are, however, systematic differences in regulatory environments. So for example, in the USA, you have the FDA. Now in Europe, we have the medical device regulation, which is a quite strict regulation, which is just started taking place. There are also differences in reimbursement. So there may be different external influences, which influence the adoption of AI. There is also, of course, in Europe, we have the GDPR, which is the regulation regarding personal data, which is also restricting a lot of cloud access for data. On top of that, we have, of course, like health data, which is even more sensitive. So it's more like kind of these external things which are influencing it, but overall I think the interest is there. And of course, I mean, you have people who are more excited than Lexus, less excited, but yeah, systematically not really big differences, but maybe a little bit more kind of in the environment. Got it, got it. So Sam, one of the biggest questions that I often get is what are the privacy aspects associated with AI? How do we ensure that there's no HIPAA violations, et cetera? How are companies even such as yours dealing with some of those aspects? Yeah, I think the simple answer is that all the software medical device companies that handle personal health information are expected to be HIPAA compliant. And so there are a ton of different requirements to maintain HIPAA compliance. A couple of them are both at rest and in transit encryption, as well as minimum necessary access protocols and others like that. So security is a critical component in designing AI systems and in software systems more broadly that handle healthcare data. Got it, thank you for that. So Anuja, this one is going to go to you, okay? So how easy is it to actually incorporate AI into a well-established sleep medicine practice? You know, for the people who are listening, who are interested and who want to kind of get the ball rolling, how easy is it to do it? Well, Margarita, the question itself is not easy to answer. And I would say that the short answer is that it is not an easy process. It's a big investment. It's a big investment by several stakeholders. You need buy-in from the hospital, you need buy-in from the administrators, you need buy-in from the providers, and you need buy-in from the patients. And I think Mark hit that on the head when he said that we need to get the trust. Unless we can earn the trust, it will be very difficult to incorporate AI in our practice. So let's take a step back. How do we earn this trust? Let's see, how does, as Haki had explained, how does the machine learn? The machine learns from datasets which utilize labeled healthcare data. Now, healthcare data is notorious for not being very clean in nature. So if you were, for instance, to go back and then look at retrospective studies, and you'd always see that there are things which are either not documented or they are not mixed. So if you're going to train a machine on that dataset, you have to make sure that the dataset is getting reliable data to build that set. Now, once you do have these datasets, you want to make sure it's generalizable. You want to make sure there's no racial disparity. You don't want that algorithm to be trained on a certain population and only cater to that population, because when you're going to market that, you're going to market it for the whole world to use. So there has to be some accountability that this dataset is generalizable. And now, when you talk about accountability, that's where the regulations and transparency comes in. So I know FDA usually has a process in place where it will go over how these algorithms work, and they will give you an approval or a clearance. But the truth is that we, as clinicians, We are looking at not just like the validity of the algorithm, but we are also looking at the context where it can be used. And so it is very important for us to be able to regulate that algorithm in our own clinical context. And I think ASM has already taken a step towards that. So what we have been working on is a pilot certification program for all these AI-assisted algorithms or these companies which utilize these algorithms. So they can, once we have our own testing data set, we can have the companies send their software and we can help see whether it meets up to certain benchmarks or not. And by setting those benchmarks, I think we have already raised the bar where we can then talk the same language and say that, yes, this meets our criteria, this can be used. And I think it's very important that all of us, as we start using any form of assistive AI or any form of augmentative AI, we need to start having our own sets of benchmarks. And once we have vetted these algorithms, I think we'll get a better chance of buy-in. Because integrating AI in any system, we would need huge financial resources, we would need a dedicated IT team. So all of that can only come from something which has been vetted properly. Yeah, absolutely. But it's definitely worth the effort. I think that the sleep centers that have started on this endeavor have seen a lot of gain and it's a huge undertaking, but definitely worth it from the patient perspective, from the technologist, from the physician perspective. So because I want to make sure that we answer all of our questions, I think we're going to open it up. But in summary, we discussed some of the definitions of AI, some of the applications of artificial intelligence and sleep medicine, and also focused a good chunk of our time discussing the concerns. So I will go to the questions now. And we have the first one. So maybe this one is for Anuja. So has AI been used to determine treatment of, for example, CPAP versus mandibular advancement devices? So the short answer to that one is that yes, there have been several research studies which have worked at just looking at CPAP adherence. I am not aware of any studies which have looked at oral appliance devices, but again, I have to do a deeper dive for that. But most of the research is at that stage now where they are looking at, is this feasible or not? So it hasn't necessarily been pushed to that level where you can incorporate it in clinical practice. From PAP adherence, there are several applications which increase patient engagement. So it has been shown that AI-assisted algorithm, which increases patient engagement, helps with more PAP adherence. But I don't think there has been any comparative data in the literature which compares one mode of treatment versus the other. Got it. Thank you. We have another question from Chuck. So what are some ways to identify bias and algorithms that are used in machine learning? And is there a way to mitigate the impact of these biases in machine learning and AI? So I think that either Haoqi, maybe Sam or John can take that one. Yeah, I can take this one. Yeah, others welcome to try me. So yeah, so definitely, I guess when you say bias, I guess it's mainly assumed they are, for example, bias from different races and bias from different age groups and gender groups. Yeah, so there are definitely ways to identify biases. So for example, in machine learning, after you fit the model, usually, as we see in the paper, usually people just say, OK, what is the accuracy overall? But it's also very important to report the accuracy in these sub groups. Let's say if you're writing a paper or doing research on sleep staging, and you want to know, OK, this particular algorithm, how does that work on older people? How does that work on younger people? So I think it's a very good practice to really report these performance metrics in the different groups, at least in your data set. So that's a way to identify how well this model, this algorithm can generalize to different groups. And are there ways to reduce the impact? Yes, definitely. So as you might know already, the most straightforward way is to make sure that in your data set, you have enough representation from different groups. So I think there are multiple data sets out there where the ratio distribution and the age distribution are quite rich. For example, the MESA, M-E-S-A, the Multi-Asthmatic Study of Atherosclerosis. Sorry. Yeah. So that's a very good example of having different people to represent in the data set. I think that's the major way. And of course, you can have some other algorithm ways to migrate these, for example, applying different weights to the different groups, like weighing higher to the minor groups and weighing lower to the major groups. That's all possible. Yeah. Thanks. Sam or John, anything to add? Yeah, I would love to add to that. I agree with everything that Heike said, but I think we can even think about bias in even a broader sense. I think there's risk of bias whenever you have a patient population which is underrepresented in the training and the validation data that you use. And this might be genders, age, races, but it might also be medical conditions or some kind of physiology, which is different. I also have an example where we trained a deep learning network using US data and tried to validate it using European data and it didn't work. And why didn't it work? Because in the US you have 60 Hertz electricity and in Europe we have 50 Hertz electricity. And the model learned that 60 Hertz, there's no information there, but then you apply 50 Hertz and you get biased. I mean, there are many ways that you can run into bias in your models. And I think the unexpected ones, these are the ones that you have to really look out for. I think the best way to mitigate this is that, because there are so many ways that you can run into biases that you cannot probably test everything, but as a provider of the AI models, you have to be quite transparent in labeling the AI algorithms, how they were developed, how they were validated so that the users can actually read the instructions and they can make up their own mind and see if they think that the algorithm is appropriate for the patient population that they're using it for. Absolutely. So we have another question from Robert Miller. So there's an immediate need for AI to identify populations of patients who are at risk and undiagnosed. And this would of course create significant demand for sleep programs, 100% agreed. So how soon before that's available, if anybody wants to take that? That's the million dollar question. Hi, go ahead, go ahead. Okay. Thanks. Yeah. Very briefly. Yes. Of course, nobody can answer how soon, but actually, I would say, sorry, I am not directly answering how soon, but I am answering how can we identify a population of patients who are at high risk? So for example, you can, like in our data set, when we collect the data, we can have follow-ups so that from the data set, you know, after this sleep study, how many years will they develop some kind of outcome? And you can use, for example, survival analysis or machine learning to predict that hazard ratio, that hazard, and people having high hazards, even if this person do not develop the outcome yet at the time of sleep study, we can still do a prediction into the future so that if the, you know, the risk, the hazard is high, then probably it's an alarming sound. And then we can individualize, you know, different ways to pay more attention to this group or other approaches. Yeah. So my answer is basically it's on the method part. I think we definitely have some work to do because that's, you know, it's the holy grail, right, to identify those populations, but there's no timeline that I'm aware of either. We have a question from Melissa. Has the Somnalyzer software already been approved by ASM for the scoring of HSDs? Do we know? I don't think so. Yeah. So we're going to have to get back to you, Melissa. I don't know. Anuja, right? We don't know. I don't know. What I, could you clarify what is meant by approval by the ASM? Did you mean in the scoring manual or? I guess as a, probably more an FDA approval. I thought they were FDA approved. Again, I could be wrong. Because the ASM does not independently, right. We don't independently approve any kind of software for analysis. Right. And then the pilot certification that we have currently, it's the full in-lab sleep PSG that we are looking at. We are not looking at HSATs yet, but definitely if this is successful, I think those would be the next steps because HSATs during the pandemic has gained an incredible amount of popularity. So yes, the short answer to that would be, I think it is approved by FDA, but we don't have any mechanism in ASM as yet. Yeah, absolutely. So Sam, I think you'll like this question. So this is from anonymous. It's a small sleep lab and how far along are we in AI to use AI based scoring to skip employing scorers and to just have the doctors review the AI scoring before reporting and interpretation? Yeah. So as the, as the indicated by the FDA clearances for these automated scoring devices, there is a technical review required by a clinical team of the AI scoring. So whether that be the RPSGTs or the doctor or the physicians themselves, I think I've seen, we've seen groups do either one of those. So I think it is at the point where, you know, doctors can kind of interpret and edit those directly from the scoring. But you know, I think there's multiple ways to do that. Yeah. So we, in our lab, we actually have AI score, and then we still have the same number of techs. They actually review, and this significantly cuts down their time. And then that will go to us as the, you know, the reporting physicians. So we have that extra layer. So we're able, you know, to obviously go through a significantly larger volume of studies. But we, you know, you know, our scores are still going through to make sure and verify because, you know, there is sometimes some disagreement and, you know, the human always wins in these situations. Okay. So from Chuck, once a health data set has been used to train a machine, can you add the data set or does the machine need to be trained again on the new data set that's available? So do you have to retrain the machines basically? Yeah, I can take this one as well. Yeah, typically with how you're seeing machine learning applied in healthcare, to update the algorithms, you would need to retrain if you had additional data sets. So typically, yeah, you'd take the data, add it to the existing data set, and then you would retrain. Okay. And how reliable? Sorry, just to add to that. Yeah, for certain type of algorithms, you can avoid retraining. For example, the deep learning neural networks, and you can continue training by unfixing those weights. But yes, I agree with Sam that most of our current algorithms are machine learning or sometimes it's just linear regression. In that case, you must retrain the data set. Okay. Sorry, Margarita. I know there's one more question pending, but on lines of this, I had a question in terms of software updates. Like for instance, if you are making minor changes, I know FDA doesn't necessarily always need to re-approve, but what is your system in your company? If you're making minor changes, say to the oxygen desaturation algorithm rule, would you still retrain your whole software again? Yeah, I just wanted to mention this because when you retrain an artificial neural network, or if you just change your analysis, you may trigger the need for a new FDA application. The FDA, however, is of course evolving with the times and they are incorporating a program which allows vendors to change protocols. They say, okay, if we get new data, we are allowed to retrain the models and release them without applying again to the FDA. That's a new program that the FDA is taking up. It's like you can declare what are small changes to your algorithms. As long as you're playing within this scope, the FDA will presumably allow you to update this, but this is really new. I don't know, maybe Sam has already experienced with this. We are kind of starting out with this, but I don't know if it's become like an official program of the FDA yet, or if they're still in pilot phase. Yeah, you're exactly right, John. It's still in the pilot phase. In terms of the FDA providing a direct guidance as to how this should be done, I think this is kind of where the field is at the cutting edge right now and trying to establish the safest way to do this and continually validate the different algorithms is what we'll see in the next couple of years. I think this will be our last question. How reliable is AI for scoring sleep studies of different ages? Primarily referring to the pediatric population and young adults, I guess. To my knowledge, again, correct me if I'm wrong, most of the commercially available systems, I think there's, I can think of maybe one which has been approved in young children, but I don't think there are many. And that can stem from the fact that the data sets that have been used by most of these companies have relied on existing research data, and that's why they're not existing research data or data sets, and those have been primarily adult population. And that's been my inference. That's why probably there's such few. And plus, if you look at the literature on performing home sleep apnea testing in pediatrics, ASM's statement has been that we are not ready yet to use HZAT. So unless I think we can use, unless there is a shift in that sphere, or unless we can come up with a system where we have dry EEG and you're also getting EEG data where you can mark arousals and you're getting a more in-depth home sleep study for kids. Once we get to that level, I think then we can start scoring. But at the moment, I don't think it's reliable. Thank you. Okay. And there's a question for John, the name of this program, but I'm not sure what this is referring to. So, but thank you guys. So this was great. Thank you for joining us. If any of our listeners have any questions, you can feel free to reach out to the AI committee. And we hope to have many more webinars on this topic in the future. So thank you for joining. Have a good day.

AI 101

Sleep Medicine

Webinar

Polysomnograms

Sleep Disorders

Machine Learning

Deep Learning

Trust

Data Privacy