Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1 of 23

Feedbackdriven radiologyreportretrieval ichi2015-v2

1

Share

Download to read offline

Feedback-Driven Radiology Exam Report Retrieval with Semantics

Feedbackdriven radiologyreportretrieval ichi2015-v2

  1. 1. Feedback-Driven Radiology Exam Report Retrieval with Semantics Sarasi Lalithsena Amit Sheth Kno.e.sis Center Wright State University Luis Tari Steven Gustafson GE Global Research Ann von Reden Benjamin Wilson Brian Kolowitz UPMC Enterprises John Kalafut GE Healthcare
  2. 2. 2 Motivation Reason for exam Secondary diagnoses related via anatomical region or disease type Follow up exam for an existing condition Specific hypothesis or inquiry mentioned To do an informed diagnosis radiologists might need:
  3. 3. 3 Problem Reason for exam: Foot pain Access to wealth of data EMR, Prior Radiology report R1:………………………. R2:………………………. R3:History Diabetes, Smoker …………………………… …………………………… Rn:........................................... ….. R3:History Diabetes, Smoker ……
  4. 4. 4 Challenges • Capture contextually relevant data Patient X : Current Study Reason for Exam: Foot Pain ………………………………………………………………………………… R1 : Lower Extremity Venous Insufficient Doppler Ultra sound R2 : History: Diabetes Former Smoker, History of Hypertension RX : Generalized Abdominal Pain …………………………………………………………………………………… …………………………………………………………………………………… Patient X : Prior Reports Lower Extremity Venous Insufficient Doppler Ultra sound History: Diabetes Former Smoker, History of Hypertension Generalized Abdominal Pain Foot partOf Lower Extremity Foot Pain associatedTo Diabetes Related by anatomical context Related by disease context
  5. 5. 5 Challenges • Capture contextually relevant data Patient X : Current Study Reason for Exam: Foot Pain ………………………………………………………………………………… R1 : Lower Extremity Venous Insufficient Doppler Ultra sound R2 : History: Diabetes Former Smoker, History of Hypertension RX : Generalized Abdominal Pain …………………………………………………………………………………… …………………………………………………………………………………… Patient X : Prior Reports Lower Extremity Venous Insufficient Doppler Ultra sound History: Diabetes Former Smoker, History of Hypertension Generalized Abdominal Pain Foot partOf Lower Extremity Foot Pain associatedTo Diabetes • Lexical Similarity Measures fall behind when there is no lexical similarity • Knowledge-based similarity measure only uses taxonomical relationships Related by anatomical context Related by disease context
  6. 6. 6 Challenges • Personalize the relevancy based on radiologist's need Patient X : Current Study Reason for Exam: XR Chest, Rib Fracture Abdomen X-RAY Neuro SpecialistMusculoskeletal Specialist More Relevant MRI CHEST WALL More Relevant XR Left Ribs including CHEST CT Thorax without Contrast All radiologists find these two records as relevant Abdomen X-Ray was found relevant by Chest specialist and Musculoskeletal specialist but less relevant by Neuro specialist, Generalists and Abdominal specialists
  7. 7. 7 Our Approach Capture contextually relevant data Knowledgebase and Semantic Similarity Personalize the relevancy Explicit Relevant Feedback Semantic Vector Builder Similarity Calculator …………………… R1 : Lower Extremity R2 : History: Diabetes …………….. ……………… Prior Reports …………………… R1 R2 Prior Report Vectors Reason for exam vector Feedback Incorporator R1 : Lower Extremity R2 : History: Diabetes …………
  8. 8. 8 Our Approach – Semantic Vector Creation • Uses RadLex Ontology to capture semantic relationships with NLP techniques CT THORAX WITHOUT CONTRAST Reason for Exam: Chest pain Reason for Exam: Foot pain Lower Extremity Venous Insufficient Doppler Ultra sound
  9. 9. 9 Our Approach – Semantic Vector Creation • Creates the vector of concepts using semantic similarity methods for each relationship 𝑺𝒊𝒎 𝒄𝒊, 𝒄 𝒎 = 𝟎, 𝒊𝒇 𝑷𝒂𝒕𝒉𝑳𝒆𝒏𝒈𝒕𝒉 𝒄𝒊, 𝒄 𝒎 = 𝟎 𝟏 𝑷𝒂𝒕𝒉𝑳𝒆𝒏𝒈𝒕𝒉 𝒄𝒊, 𝒄 𝒎 , 𝑶𝒕𝒉𝒆𝒓𝒘𝒊𝒔𝒆 Lower Extremity Venous Insufficient Doppler Ultra sound 1 0.5 bpi bpj bpi HasPart bpj bpi – Lower Extremity bpj - Foot
  10. 10. 10 Our Approach – Semantic Vector Creation • Handles multiple relation 𝒕𝒊 = 𝒎𝒂𝒙(𝑯𝒂𝒔𝑷𝒂𝒓𝒕𝒊 , 𝑰𝒔𝑨𝒊, … . 𝑪𝒂𝒖𝒔𝒆𝒔𝒊) Lower Extremity Venous Insufficient Doppler Ultra sound 1 0.5 1 0 1 0 0.2 0.02 HasPart Isa Causes 1 0.5 0.020.2
  11. 11. 11 Our Approach: Explicit Relevance Feedback • Explicit relevance feedback is used in information retrieval to improve the relevance of the results • Useful when users have a general conception on what they are looking for • Adopted an existing algorithm “Rocchio Algorithm” to work with the ratings provided by the users * ! * * * ! * * * ! ! * Relevant Record ! Non-Relevant Record OO MO Original Query Vector Modified Query Vector
  12. 12. 12 Our Approach: Explicit Relevance Feedback Modified Rocchio query vector 𝒒 𝒎 = 𝒂𝒒 𝒐 + 𝒃 𝟏 𝑫 𝒓 𝒘𝒊 𝟓 𝒊=𝟏 𝒅𝒋𝒅 𝒋ε𝑫 𝒊 + 𝒄 𝟏 𝑫 𝒏𝒓 𝒅 𝒌𝒅 𝒌ε𝑫 𝒏𝒓 𝑫 𝒓 = 𝑫 𝟏 𝑫 𝟐 𝑫 𝟑 𝑫 𝟒 𝑫 𝟓 qm - modified query vector qo - original query vector Dr - relevant patient records Dnr - non relevant patient records Di - patient records rated as i a = 1 b = 0.6 c = 0.1
  13. 13. 13 Evaluation • We use a radiology exam corpus consists of 6600+ anonymous exam reports to evaluate our approach • Two evaluation strategies have been used, – Domain expert based evaluation – Intra-class and Inter-class similarity
  14. 14. 14 Domain Expert Evaluation – Part 1 • Evaluates the relevant patient records with 7 radiologists for 2 queries • Measures the effectiveness of relevance retrieval via semantic vector generation (without feedback) using Rprecision • 𝑹𝑷𝒓𝒆𝒄@𝑹 = 𝒓 𝑹 r = number of relevant documents in top R Q1 Q2 Rprec@2 Rprec@5 Rprec@8 Rprec@2 Rprec@5 Rprec@8 BOC 0.5 0.59 0.67 1 0.71 0.85 ISA 0.5 0.73 0.72 1 0.71 0.85 VSS 1 0.73 0.72 1 0.88 0.93 BOC: Only uses bag of concepts and no relationships ISA: Only uses taxonomical relationships VSS: Our approach which uses taxonomical relationships and HasPart relationships
  15. 15. 15 Domain Expert Evaluation – Part 2 • Evaluates the effectiveness for relevance feedback • Re ranks the results with explicit feedback from domain experts • R precision value increased from 0.72 to 0.78 for Q1 Study Description Before Feedback BOC Ranking After Feedback BOC (R1) Ranking XR LEFT RIBS INCLUDIND CHEST 1 1 ABDOMEN X-RAY 2 4 XR RIGHT FINGER(S) 2 3 CT THORAX WITHOUT CONTRAST 2 2 NUCLEAR MEDICINE PARATHYROID SCAN 5 9 BILITERAL VASCULAR ANKLE BRACHIAL INDEX 5 9 MR RIGHT KNEE 5 7 MR THORACIC SPINE 5 7 CT LEFT FOOT WITHOUT CONTRAST 5 5 MRI CHEST WALL WITHOUT CONTRAST 5 6
  16. 16. 16 Intra-class and Inter-class similarity Hypothesis 1 All cases our approach would show a higher similarity score for Intra- class than Inter-class. Hypothesis 2 Differences will be increased with semantic enhancement for Intra-class similarity and the differences will be decreased for Inter-class similarity
  17. 17. 17 Intra-class and Inter-class similarity Experimental Set Up ● Group all patient records based on the exam code such as CTCHEST, MAMSCRNDIG, DEXABOD and USABDCOMP ● Select 10 exam codes with the highest number of reports covering about 2200+ reports ● Calculate the intra class and inter class similarity among these 10 exam codes (10 intra class similarities and 45 inter class similarities)
  18. 18. 18 Intra-class and inter-class similarity • We show that intra-class similarities are always higher than inter- class similarities • Our approach was able to show that differences between intra-class similarities are increased against the baseline (taxonomy approach) • Our approach was able to show that differences between inter-class similarities are decreased against the baseline (taxonomy approach) Approach Avg Intra-class Sim ISA 0.50 VSS 0.55 Approach Avg Inter-class Sim ISA 0.178 VSS 0.162
  19. 19. 19 Conclusion ● We developed a technique that identifies relevant information from prior radiology reports to help radiologists to improve their interpretation ● We showed a semantic approach using Radlex ontology to find relevant information from prior radiology reports was effective ● We showed the potential of using explicit relevant feedback to personalize each radiologists need ● In future we plan to, ● Extend our corpora outside radiology reports ● Improve the coverage of the queries
  20. 20. Thank You! Questions? http://knoesis.wright.edu/researchers/sarasi sarasi@knoesis.org Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing Wright State University, Dayton, Ohio, USA
  21. 21. 21 Hypothesis 1 Testing All Intra-class similarities are higher than the inter-class similarities A B C D E F G H I J (A)CTCHEST 0.5625 0.1022 0.0902 0.0930 0.5193 0.1218 0.1183 0.1289 0.1383 0.1081 (B)MAMSCRNDIG 0.1022 0.7147 0.0841 0.2655 0.1070 0.0805 0.3519 0.2117 0.0862 0.3450 (C)DEXABOD 0.0902 0.0841 0.6031 0.0582 0.0914 0.1923 0.1410 0.1409 0.0766 0.1381 (D)USABDCOMP 0.0930 0.2655 0.0582 0.4496 0.0962 0.0959 0.1925 0.2059 0.1697 0.2101 (E)OSRCHESTCR 0.5193 0.1070 0.0914 0.0962 0.5380 0.1257 0.1231 0.1266 0.1361 0.1139 (F)LUMBARSP 0.1218 0.0805 0.1923 0.0959 0.1257 0.3482 0.1074 0.1021 0.1123 0.1003 (G)MAMDIAGDIG 0.1183 0.3519 0.1410 0.1925 0.1231 0.1074 0.5954 0.2742 0.1364 0.2916 (H)USSOFTISSU 0.1289 0.2117 0.1409 0.2059 0.1266 0.1021 0.2742 0.5782 0.1364 0,2916 (I)CTABDPEL 0.1383 0.0862 0.0766 0.1697 0.1361 0.1123 0.1364 0.1364 0.4995 0.1474 (J)USBREASTLT 0.1081 0.3450 0.1381 0.2101 0.1139 0.1003 0.5165 0.2916 0.1474 0.5623
  22. 22. 22 Hypothesis 2 Testing Approach Avg Intra-class Sim ISA 0.50 OUR 0.55 Differences are increased for intra-class similarity
  23. 23. 23 Hypothesis 2 Testing Approach Avg Inter-class Sim ISA 0.178 OUR 0.162 Differences are decreased for inter-class similarity

×