Session 15: AI Part 2 - Q&A
Questions for Dr. Dan Nayot
- Giles Palmer: Do you ever use this “selectively” to select some for donation ( in egg sharing cases..i.e Own and donor eggs) or to just validate a good egg from freezing?
- Peter He: What was the false positive/negative rate in the VIOLET vs embryologist study?
- Mark Adamowicz: If you have a low-scoring egg, i.e. 1, 2, or 3, do you discard or NOT perform ICSI? Or do you ICSI/inseminate all mature eggs regardless of score?
- Answered live
- Peter Nagy: We at RBA would still go ahead and do ICSI, as sometimes, even "poor morphology" eggs will fertilize and develop to good quality embryo
- JORGE RAMOS: How many eggs are needed to be analyzed to consider the result more reliable and accurate?
- Alex Parker: Great question Jorge! Ultimately, the more eggs, the more data, the more reliable and accurate the Machine Learning model will be. It is possible to create a Machine Learning model with as little as 10 data points, however, the reliability of that model would be questionable. Typically you’d want a minimum of hundreds of data points, but preferably 1,000s or 10,000s of data points to train the model.
- Dr Debajani Mahapatra: Can we see 2nd polar body in GV stage...?
- Peter Nagy: That is extremely unlikely (normally you should not be able to see any polar body for an egg that is in the GV stage). If you see any "polar body" like structure in the PVS for a GV egg, that is likely of cytoplasmic fragments
- ICC: Subjective between embryologists is always a concern of judging the quality of embryos, especially for those in between bad and good. Does the tool eliminate this concern and how?
- Alex Parker: AI can certainly take away the subjectivity of any particular embryologist. As long as the data provided gives an outcome of the embryo, the Machine Learning model would simply use that as a result to assess against, without any personal bias.
- Robert Heudes: So was the AI system used on fresh oocytes only or was it used for vitrified/warmed oocytes and if so was any difference noted between the two types of oocytes.
- Dan Nayot: The AI system was based on fresh oocytes. Ideally we can evolve into using vitrified / thawed oocytes data, but this scarce (not many overall cases and capturing images of oocytes are not routinely done, so there is minimal retrospective data) and fragmented (would need many contributing IVF clinics)
- Estrella Rosemberg: did you see differences between egg donors and patients and fertility etiology?
- Dan Nayot:The AI system was based on fertility patients, but from across the spectrum. We would anticipate oocytes from a healthy young female with male factor infertility (that requires IVF due to sperm-related issues) should resemble donor oocytes. We have not specifically compared the oocytes between this subgroup to the rest of the data set. Thanks for the good idea
- Lonyl Palao: Do you foresee egg banks using AI on oocytes before vitrifying them so they can assure or guarantee patients that they are receiving eggs that have the highest potential?
- Dan Nayot: I think there are 3 pillars where an egg assessment tool would be of great clinical benefit
- 1. Egg freezing —> Being able to predict the outcomes of the frozen eggs
- 2. IVF/ICSI —> Being able to provide feedback on the eggs used, or even using egg quality as part of the embryo selection process
- 3. Egg donation —> Being able to predict stratify eggs (ag 18 eggs into 3 lots of 6 with 1 live birth per lot), or statistically model assurance programs (eg 6 or 8 or 10 eggs to lead to 1 live birth depending on the egg quality)
I think the egg donor domain will be an early adopter to any egg quality assessment tool, as we currently don't have any tools to guide us
- Dr. Vikash Verma: (What is the) Underlying AI Engine? OpenAI, DeepBlue.. or Proprietary
- Dan Nayot: The initial phase of this project we tried to use open source AI platforms, but had minimal success. We built a de-novo convoluted neural network thats specific to the clinic question being asked (so its proprietary)
- Luis Delgado: Hi, great talk, Thanks. Can we try this VIOLET system? I mean if there is available a demo?
- Dan Nayot: looking for strategic partners for data collection, for real feedback (patients, embryologists, and clinicians) and early adopters. Please email me
- Deepti Tiwari: What were culture conditions to predict and evaluate the data? What were the stimulation protocols, Demographic population please
- Dan Nayot:The data was based on multiple IVF clinics, each with their own specific lab conditions and culture media. As well, we aimed at obtaining as much data as possible, so the patient population and stimulation protocols were wide ranging. In general, the more diverse the data, the more robust the AI engine
Questions for Dr. Michelle Perugini
- Margaret Wozniak: Have multiple 2D images been analyzed for score at different rotations of the blastocyst? Did scores vary at the different rotations of the blastocyst?
- Answered live
- Michelle Perugini: Rotations of the blastocyst will result in a different score being given because the image attributes will differ. However, the key is in the diversity of the data used to train the AI which should include representative images from different vantage points.
- Richard Rawlins: HOw does your AI on static images compare in predictive ability against existing morphokinetic prediction of outcome?
- Answered live
- Michelle Perugini: International clinical studies have demonstrated at least a 25% accuracy advantage of the AI compared with traditional morphological grading. AI on static images has not been directly compared to morphokinetic analysis, however, AI on static images is highly predictive of implantation potential suggesting it is an accurate alternative to morphokinetics.
- Diego ezcurra: Did you test LW in a RCT having a hard clinical endpoint like implantation rate?
- Answered live
- Michelle Perugini: LW has not been tested in an RCT.. Ranking cohort studies have been done on transferred embryos showing the AI has a ranking advantage and can shorten time to pregnancy in simulated cohort studies. Prediction comparisons for implantation with standard grading versus AI have been completed internationally and show a significant increase in accuracy of the AI suggesting that implantation rates will be improved by virtue of using the AI for embryo selection. LW is already in commercial use and so ongoing post market clinical evaluation will be conducted to test the impact on pregnancy rates in a true clinical environment.
- Mayank Patel: What are the ways to increase AI accuracy?
- Alex Parker: Great question, Mayank! There are a number of ways that can increase AI accuracy.
So AI and Machine Learning models are built off of datasets. Both quality and quantity of the datasets play a role here.
So quality-wise, if there is a clear outcome and a clear coefficient (grading) for any variables, then the dataset could be considered high quality. Sometimes this can be tough, for example, if we try to create a model based off of handwritten doctors notes, some of the variables might be missing and some of the outcomes might be tough to read - low quality data.
Quantity-wise, the more the better, ultimately. If we build a model using 10 data points vs 10,000, the larger data set will give a far more accurate prediction.
Michelle Perugini: Data quality and diversity are key. This is what drives the variability seen in clinical practice to be effectively trained in the AI.
- Giles Palmer:Transparency you say? Can this data be given to the patient and does it vary from country to country..FDA reg/ confidentiality etc
- Michelle Perugini: Yes, transparency and protection is provided in multiple ways. Transparency can be provided through publication of research results. Reports can also be provided to patients that show the score given to each embryo by the AI. This can help manage expectation of patients, and gives them confidence that an objective measure has been used to assist in embryo selection. Our system is HIPAA compliant, ISO13485 compliant, and also GDPR compliant. Per GDPR if a patient requires their information to be removed from the system we can do so. Every country’s regulation is different for our product.
- Dr. Vikash Verma: Well said, Michelle, it'd be a collective value eventually as the game changer. EEVA has been struggling to become mainstream for quite some years now. What challenges do you see in EEVA's or any AI Algorithm's acceptance in the near-future - is it Quality of DataSet as Quantity does not seem to be an issue at present or other combination of factors?
- Michelle Perugini: AI is still new and there are relatively few clinically utilised AI systems. EEVA is a very different system with different clinical utility. It predicts blastocyst development from Day 3 embryos which is very different to our system which predicts implantation potential, an enpoint that we believe is of greater clinical utility. For any AI system, trust needs to be gained, and the systems need to be adequately tested, and to prove themselves in practice. There will always be those clinicians that are first movers and enjoy being at the frontier of technological changes and those that do not. We align with those that do, and we collaboratively build our products with those end users. We even give royalties to those that contribute data to new products. This is our way of gaining trust and having the end users understand and input into the AI.
- Luis Delgado: Hi, great talk, Thanks. Can we try this grading system? I mean if there is available a demo?
- Michelle Perugini: Yes, absolutely. It is very simple for us to set up a new user anywhere in the world. You can email us at email@example.com. We are approved for use in UK, Europe, India, New Zealand, Canada, and Australia, and undergoing approvals in USA, and SE Asia.
- Adolfo Flores Saiffe: How robust is the embryo assessment to changes in brightness or flips or different image sizes or microscopes?
- Michelle Perugini: AI is not always robust to these changes. Our AI is robust to these changes. It has been trained with a very diverse dataset and we have pre-processing stages that are automated and deal with different brightness and orientations, focal lengths in images. The proof of this is in double blind testing and clinical utilisation in clinics/laboratories in different countries that never contributed data to the system initially. This demonstrates the LW AI is robust and generalisable despite the large amount of variability in imaging.
- Estrella Rosemberg: Is there a correlation with aneuploidy?
- Michelle Perugini: Genetic health of an embryo is very different to implantation potential. We have unpublished works that show these do not overlap as much as you might expect them to. This was the driver for us creating our second application with AI for non-invasive image based genetic assessment of embryos. The combination of this AI with the current LW viability assessment will be very powerful.
- Jayaprakash Divakaran: Could you find 25% increase in terms of pregnancy in correlation with AI Vs Embryologist assessment? to confirm the finding?
- Michelle Perugini: Yes, this is what our studies show, please refer to the April edition of Human Reproduction for our published study. The accuracy is even greater since submission.
Questions for Daniella Gilboa
- Anabella Marconetto Daniella, what about collapsed embryos? could the software manage it?
- Asher Owusu-Tok Which parameters does the AI machine really focus on in choosing the best embryo for transfer and how deep are the details Answered in QnA
- Alex Parker: Great question, Asher. Depending on the type of model, it sometimes can not be known which parameters the AI machine rates highly. It will give different weight or preference to different parameters based on the results from the given dataset. The results can sometimes be very counterintuitive to a human observer.
- Nikica Zaninovic: Can AI models be universally applied in different labs ?
- Feras AlNahhal Which better to use frozen or fresh semen to avoid DNA fragmentation?
- Irfan Mahfooz: do you suggest if semen parameters are normal, perform swim up instead of doing centrifugation?
- Dara Berger: Many labs will centrifuge and then swim up