Hyperspectral Retinal Imaging
•
2 gefällt mir•1,352 views
Short intro for some design considerations around hyperspectral retinal imaging. Both for research-grade desktop setups built around supercontinuum laser and AOTF tunable filter, and for mobile low-cost retinal imagers. Available also from: https://www.dropbox.com/s/5brchl9ntqno0i9/hyperspectral_retinal_imaging.pdf?dl=0
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Hyperspectral Retinal Imaging
Ähnlich wie Hyperspectral Retinal Imaging (20)
Mehr von PetteriTeikariPhD
Mehr von PetteriTeikariPhD (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Hyperspectral Retinal Imaging
- 1. Petteri Teikari, PhD Singapore Eye Research Institute (SERI) Visual Neurosciences group http://petteri-teikari.com/ Version “Tue 16 April 2019“ Hyperspectral Retinal Imaging From scientific desktop design to low-cost embedded mobile imaging Hyperspectral fundus structure, oximetry, metabolomics, etc.
- 4. Filter the illuminant side with AOTFs Limitations: Need two AOTFs, ot a lot ofselection for these guysandfaster switchingspeed isnice forpost-processing NIR band used less forhyperspectral fundusimaging, practical implications ● More to discover ● Video-rate NIR non-mydriatic fundus imagingallowsmultiframe reconstruction as no pupillaryreaction, but NIR cameras slower:( ● Fast frame rates more interesting deep learning -wise then VISCamera:XimeaCB019MG-CM-X8G3-EF (31,650SGD) NIRCamera:XimeaCB120RG-CM-X8G3 (10,500SGD?) “Test”Camera: Reference funduscameratogetstarted Remidio,3nethra, Nexy,orHorus Low-costfordeployment e-Con RGB+IRorPi Noir With the 2,500 fpscamera,one framethen is 400 µs,thus141bands take 56.5msintotal 16msforNIR 40.5msforVIS17.73fps 3,525fps required for 25 fpshyperspectral video
- 5. Imaging Speed with the AOTF System With the 2,500 fps camera,one frame then is 400 µs, thus101 visible light bandstake 40.4 ms Cube:17.73 fpsat thisspeed With the 330 fps camera,one frame then is 3,030 µs, thus40 NIR bands take 121.2ms Cube:2.34 fps at thisspeed 161.62msper hyperspectral cube =6.19fps for hyperspectral video 121.2msofnon-PLR lightwith 40msof pupilconstricting lightso “easilyget” onewhole cube before pupil constriction Characteristicsofthe pupillary light reflex in the alert rhesusmonkey https://doi.org/10.1152/jn.01131.2002 Thedatafromnormalsubjectswere fittedwithamodelcomposedofa cascadeddelayandfirst-andsecond- order filters. For therhesusmonkey,thedata werebestfitbyadelayof160ms,a first-order filterwithatimeconstantof 0.12s,andasecond-order filterwitha resonantfrequencyof1.9Hzanda dampingconstantof0.5. For thehuman,thedatawerebestfit byadelayof280ms,afirst-order filter withatimeconstantof0.15s,and asecond-order filterwitharesonant frequencyof1.3Hzandadamping constantof0.7. Although thismodel adequately fitted both the gain and phase datafor the rhesusmonkey, it failed tofullyfit the phaseadvance seeninthe human dataat low temporal frequencies.
- 6. Optical Design Contact-free trans-pars-planarillumination enables snapshot fundus camerafornonmydriatic wide fieldphotography https://doi.org/10.1038/s41598-018-27112-x Benquan Wang,DevrimToslak,MinhajNurAlam, R.V.PaulChan &XinchengYao B.W., D.T. and X.Y. have pending patent (US 62/518,801) relative to this work. M.N.A. and R.V.P.C. have no competing interests. DepartmentsofOphthalmology and Visual Sciences/ Bioengineering, University of Illinoisat Chicago, Chicago ~ 508mm
- 7. Optical Design Details Contact-free trans-pars-planarillumination enablessnapshot fundus camera for nonmydriatic wide field photography https://doi.org/10.1038/s41598-018-27112-x Benquan Wang,Devrim Toslak,Minhaj NurAlam,R.V.PaulChan &XinchengYao B.W., D.T. and X.Y. have pending patent (US 62/518,801) relative to this work. M.N.A. and R.V.P.C. have no competing interests. Departments of Ophthalmology and Visual Sciences / Bioengineering, University of IllinoisatChicago, Chicago
- 8. Design Constraints What is wanted from the system in the end? 141 bands AOTFs 161.62msper hyperspectral cube With the 330fps NIRcamera as the bottleneck for faster imaging ● Timeframe of thecommercialization? 960fps low cost camera a realityin 2 years? ● Your basic 40° fundus cameraeven with multispectral extension provides very littlevalue? ● Google rolls outtheir fundus camera andhave alarge database collected in notime.Who wants non-deep learning multispectral fundus camera? ● Cliniciandonot like multitude of devices,nor really the patients, sowhatis the point in screening multispectral camera?
- 9. COMPONENTS
- 12. ILLUMINATION Supercontinuum white light lasers NKTPhotonicsSuperK EVO https://www.nktphotonics.com/lasers-fibers/product/superk-evo-industrial-supercontinuum-l asers/ NKTPhotonicsSuperK MIR https://www.nktphotonics.com/lasers-fibers/product/superk-mir-mid-ir-supercontinuum-laser/ Relativelyflatspectrum withhighpower Visiblelightandmid-IR availablefrom NKT PhotonicsSuperK MIR deliversmorethan450mW of continuouslightfrom1.1 to4.2 µm. the new SuperK EVO supercontinuum fiberlaser platform isdesignedfor OEM integration in demandingcommercial applications.
- 13. FILTERS Gooch & Housego Notalotofoptions inthis spaceforquickilluminant- sidefiltering ACOUSTO-OPTICTUNABLE FILTERS(AOTF) Acousto-optic tunable filters (AOTF) are used to rapidly and dynamically select a specific wavelength from a broadband or multi-line laser source. As the applied RF frequency is varied, the transmitted wavelength changes, “tuning” the wavelength of the beam or image in tens of microseconds or less. We offer an extensive line of AOTFs for wavelength regions from the UV through mid-IR, with resolution bandwidths of less than 1nm. We also offer options such as large-aperture imaging filtering and sideband suppression. Fiber-coupledAOTF devicesare availableuponrequest. https://goochandhousego.com/product-categories/tunable-filters-aotf/
- 14. MOTORIZED STAGE Trans-pars-planar illumination issensitive topositioningsee need some automagicstage that optimizesthe lightingentry Cost-efficientsolution for opticalbench? http://www.optics-focus.com/6axis-motorized-positioning-stage-p-661.html#.W2gaH59fhpo
- 15. RELAY LENSES L1, L2, L3 Found fromthe optics set tobeordered / compiledby Bingyao? https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=688 https://www.edmundoptics.com/c/ir-lenses/655/
- 16. Apertures 2.5mm Standard component? Arc Lasercut suitablematerial asmask/ GOBO Fixation Cross https://www.thorlabs.com/navigation.cfm?guide_id=78
- 19. Why multispectral fundusimaging? http://www.annidis.com/page/technolog y http://doi.org/10.1167/iovs.14-15783 Spectralbiomarkersfor variouspathologies? Simplifiesdeeplearningimage segmentation as now forexample macula is very visible at 460nm, blood andoptic at near-infrared.
- 22. Mining for Ocular Spectral Signatures Early work without imaging#2 FovealreflectionanalysisinaclinicalsettingbyMartijnJacobKanis Co-authorsJan vandeKraats, Martijn J. Kanis,Stijn W. GendersandDirkvan Norren. Utrecht University, Facultyof Medicine, TheNetherlands (2008) Cited by3 -Related articles Theprimaryaimofthisthesiswastoinvestigatetheclinical-diagnosticvalueofthe FovealReflection Analyzer (FRA).Thiswasachievedbystudyingcone-photoreceptor involvementinseveralophthalmicdiseases.A secondaryaimwastoinvestigatetheputativeprotectiveroleofmacular pigmentandmelaninonthe ... Reduced reflectance caused by increased absorption of the eye lens is visible at short wavelengths (<450 nm) in the diabetes patient. Around 460 nm macular pigment reduces reflectance. At longer wavelengths (>550 nm) reflectance is seen to increase because of decreasing absorption of melanin and blood. At yet longer wavelengths,waterabsorptionreducesreflectance. Spectral reflectancecurvesof thesamesubjectsatthepupil position wherethe optical SCE is atitsmaximum. Attheshortwavelengths,thereflectionisverylowbecauseof theabsorption in theeyelens. Around 460nmthemacular pigmentreducesreflectance.Atlonger wavelengths, thereflectanceisseen to increasebecauseof decreasing absorptionby melaninand blood. Atyetlongerwavelengthswater absorption reducesthereflectance.Notethatthereflectance around 540nmislowerin thePOAG patient(lower dottedcurve) compared to thecontrol subject(upperdotted curve) 66year oldwoman withearlyAMD
- 23. Ocular Spectral Signatures Absorption spectra of major endogenous contrast agents in biological tissue at normal concentrations. Oxy-hemoglobin (HbO2) and deoxy- hemoglobin (HbR), 150g/L in blood; lipid, 20% by volume in tissue; water, 80% by volume in tissue; DNA and RNA, 1g/L in cellnuclei; melanin, 14.3g/Lin medium human skin; reduced myoglobin (MbR) and oxy- myoglobin (MbO2), 0.5% by mass in skeletal muscle; bilirubin,12mg/L in blood.- YaoandWang(2014) Highlighting IR SpectrochemicalImagingof the Retina https://doi.org/10.1016/j.tibs.2018.04.005 SpatiallyresolvedIRimageshighlightsimultaneous chemicalcompositionoftheentirespanoftheretinain a label-freemanner(e.g. invivometabolomics). * k = 1/ =1/kλ → λ = 1/k→ λ = 1/kλ → λ = 1/k λ = 1/3500cm-1 ~ 2.9 μm; λ = 1/2500cmm; λ = 1/2500cm-1 ~ 4 μm; λ = 1/2500cmm; Note! Compare this to max wavelength ~ 4 μm; λ = 1/2500cmm of “NKT Photonics SuperK MIR” λ = 1/1000cm-1 ~ 10 μm; λ = 1/2500cmm
- 24. Retinal Oximetry Themostobvious application 27August2018 Optimalwavelengthsforsubdiffuse scanninglaseroximetryofthe humanretina https://doi.org/10.1117/1.JBO.23.8.086003 MathiDamodaran;ArjenAmelink; JohannesF.deBoer VrijeUniv.Amsterdam(TheNetherlands) a) Absorption spectrum of oxy- and deoxyhemoglobin from 350 to 1000 nm assuming a concentration of 150 mg of hemoglobin in 1 mL of blood. The inset shows the absorption spectrum from 450 to 650 nm. The absorption of these two types of hemoglobins is different for most wavelengths except for the isosbestic points (denoted by black dashed circles: 506, 522, 548, 569, and 586 nm in the inset) where the absorption depends on factors other than the oxygen saturation. (b) Fundus reflectance of the right eye of a healthy adult male at 570 nm recorded using a Fundus camera (Oxymap T1, Oxymapehf.,Iceland,fittedtoaTopconfunduscamera,TopconTRC50-DX,Topconcorp.,Japan).
- 26. Retinal Oximetry Fordiabetic retinopathy (DR) October2017 RetinalOximetryandVesselDiameterMeasurementsWithaCommerciallyAvailableScanning LaserOphthalmoscopeinDiabeticRetinopathy http://doi.org/10.1167/iovs.17-21934 NormanP.Blair; JustinWanek; AnthonyE.Felder; CharlotteE.Joslin; JacobK.Kresovich;JenniferI.Lim;FelixY.Chau; YannekLeiderman; MahnazShahidi Fourth, we did not account for measurement variations as a function of the cardiac cycle. While Knudtson et al. have found that image quality is more important than the cardiac cycle as a source of measurement variability, Chen et al. have reported changes in vessel diameter during the heart cycle as 3.46% and 4.82% in arteries and veins, respectively. This suggests that the heart cycle may have been a major source of variability in our data over and above the error of measurement. In the future, taking this into account may reducethevariabilitysubstantially. Fifth, aging has been shown to affect retinal SO2 measurements in healthy and diabetic subjects. Although control and diabetic subjects were age matched, adjustments for age were taken into account in the models. While changes in the optical properties of the eye due to disease were minimized by the calculation of optical density ratios, future studies are needed for rigorous determination of the effects of alterations in lenticular light transmission on SO2 measurements. In summary, we demonstrated that alterations in retinal vascular diameters and hemoglobin oxygen saturations according to the stages of diabetic retinopathy can be detected with awidely available SLO and that statistical modeling can reveal the influencesof covariates such asrace on theresults
- 27. Retinal Oximetry Forglaucoma management 2018Jan4 Glaucomarelated retinaloximetry:atechnologyupdate https://dx.doi.org/10.2147%2FOPTH.S128459 ZhuLiYap,SushmaVerma,YiFangLee,CharlesOng,AditiMohla,andShamiraAPerera “… a few studies found that in patients with treated POAG, deeper visual field defects were associated with increased oxygen saturation in venules and decreased arteriovenous (AV) difference in retinal oxygen saturation. The most likely explanation is that glaucomatous atrophy decreases the demand for oxygen, resulting in a higher retinal venous oxygen saturation. The data suggested that oxygen metabolism is affected in the glaucomatous retina, possibly related to tissue atrophy, adding further to the evidence for ischemia/ hypoxia in glaucoma “From the literature, it is known that some medications can have an effect on the retinal oxygen saturation by influencing ocular blood flow. In a literature review and meta-analysis, Siesky et al looked at an 8-month treatment with dorzolamide– timolol or timolol alone and found that the decrease in IOP with dual therapy decreased IOP and increased retinal blood flow in the superficial retinal vasculature in both glaucomatous and healthy patients.”
- 28. Retinal Oximetry Proxyfor cerebral oxygenation measurement January5,2018 Non-invasiveassessmentofcerebral oxygenation:Acomparisonof retinaland transcranialoximetry https://doi.org/10.1371/journal.pone.0190612Karel VanKeer, Jan Van Keer, João Barbosa Breda, Vahid Nassiri,Cathy De Deyne, CorneliaGenbrugge, Luís Abegão Pinto, Ingeborg Stalmans, EvelienVandewalle This is the first study to show the correlation between retinal and cerebral oxygen saturation, measured by near-infrared spectroscopy (NIRS) and retinal oximetry. The average relative weight of arterial and venous retinal oxygen saturation to obtain the measured transcranial oxygen saturation as measured by NIRS, approximates the established arterial:venous ratio of 30:70 closely, but shows substantial inter-individual variation. These findings provide a proof of concept for the role of retinal oximetry in evaluating cerebral oxygenation Current retinal oximeters have several limitations that inhibit the use in the settings mentioned above. First, retinal oximetry provides a snapshot in time but not a continuous measurement. Next, the size of the current retinal oximeters limits its bedside usability. In addition, the price is steep. However, recently developed hyperspectral image sensors at the size of a regular color image sensor will likely provide an answer to both issues. By replacing the color image sensor with a hyperspectral sensor, any portable fundus camera could be modified into a portable retinal oximeter. Provided that retinal oximetry could be performed at the bedside, this technology could become an interesting tool in the hemodynamic and cerebral assessment of critically ill patients and hence also contribute to a better understandingofNIRS
- 29. Retinal Pigment Epithelium May2016 SpatialandSpectralCharacterizationofHumanRetinalPigmentEpitheliumFluorophoreFamiliesby ExVivoHyperspectralAutofluorescenceImaging http://doi.org/10.1167/tvst.5.3.5 Tal Ben Ami;Yuehong Tong; Alauddin Bhuiyan; CarrieHuisingh;ZsoltAblonczy;ThomasAch;ChristineA.Curcio;R.TheodoreSmith Finally, our long-term goal isto translate these methodsto human eyes invivo,using a snapshot hyperspectral camera, and provide a straightforward and noninvasive method for evaluatingthe RPE fluorophores. Altered and/or newfluorophoresmay be expected inAMD, aswell as redistribution of normal fluorophores. Thus,althoughthe usefulness of hyperspectral imaging in AMD is asyetunproven, the successof the method presented here in normal tissues exvivo suggeststhe potential of spectral, molecularbiopsy of the RPE in vivo forearlydetection, longitudinal follow-up, andtarget discoveryforAMD
- 30. Retinal Pigment Epithelium and Drusen Dec2016 HyperspectralAutofluorescenceImagingofDrusenandRetinalPigmentEpitheliuminDonorEyes withAge-RelatedMacularDegeneration https://dx.doi.org/10.1097%2FIAE.0000000000001325 Our initial effort to decompose total AF emission tissue spectra of RPE, drusen, and sub-RPE deposits is a step toward molecular identification of these fluorophores in vivo. Such data may aid the understanding of AMD biogenesis. A clinical hyperspectral AF camera that could detect the earliest formation of drusen and sub-RPE deposits in vivo could provide an early warning for individuals at risk for AMD, as well as facilitate the development and monitoring of new therapies basedon specific moleculardefects
- 31. Fundus Autofluorescence and RPE 22May2018 Spectralanalysisof fundusautofluorescencepatternasatooltodetectearlystagesof degenerationin theretinaand retinalpigmentepitheliu https://doi.org/10.1038/s41433-018-0109-0TatianaB.Feldman,MarinaA.Yakovleva,AndreyV.Larichev,PatimatM. Arbukhanova,AlexandraSh.Radchenko,SergeyA.Borzenok,VladimirA.Kuzmin&MikhailA.Ostrovsky
- 32. Flavoprotein Fluorescence for the Diagnosis and Monitoring of Retinal Disease 2016 Development and Commercializationof Functional, Non-Invasive Retinal Imaging Device Utilizing Quantification of Flavoprotein Fluorescence forthe Diagnosisand Monitoring of Retinal Disease http://rave.ohiolink.edu/etdc/view?acc_nu m=case1458921113 Heise,ErichA Masterof Sciences,CaseWestern Reserve University, Biology. Diabetes and its related complications are becoming increasingly more prevalent in the United States and other developed countries. Additionally, an increasingly earlier age of onset results in more complications, such as diabetic retinopathy, leading to increased annual medical costs and poorer quality of life. Therefore, the ability to rapidly and non-invasively detect diabetes and other retinal diseases at the earliest stages is critical to saving patients’visionandimprovingtheiroverallqualityoflife. Retinal metabolic analysis, a measure that the detects the fluorescence offlavoproteinscritical toATPgeneration,allowsophthalmologistsand optometrists to identity retinal diseases earlier in the disease process and intervenebeforeirreversibletissuedamageandlossofvisualacuityoccurs. Excitation and emission spectrum, respectively, of free FAD in neutral aqueoussolution PNASChristie etal. 1999 Absorption spectrum of flavin adeninedinucleotide (FAD) inoxidized (a)and reduced forms(b,c) Nishinaet al. 2007
- 33. Hyperspectral OCT 2DFundusimagingdoesnot “depthlocalize” thespectral signature addingambiguityfor analysis,OCTaddsthe 3rd dimension. WhitelaserOCTswithNKT Photonicssupercontinuum lasers Structural and functionalhumanretinalimagingwithafiber-basedvisible light OCT ophthalmoscope (2017)https://doi.org/10.1364/BOE.8.000323 In conclusion, we described a multi- functional fiber-based visible light OCT system for human retinal imaging with < 2 µm axialresolution intissue,anddemonstrated in vivo imaging of Doppler velocity, hemoglobin saturation, and concentration, as well as spectroscopic imaging ofthe inner and outer retina with high resolution. These results suggest that visible light OCT is a promising avenue for future investigation, with potential advantages over near-infrared OCT for high- resolution structural and functional retinal imaging.
- 35. Alzheimer’s Diagnosis VisualizingADdepositswithouttagging Newer imaging modalities thattargetspecificreflectance characteristics of biological material hold thepromise of being non-invasive,contrast agentfree andpotentially specific. Onesuch method is hyperspectral imaging, whereby tissue reflectance to a wide range of incident wavelengths are quantified (More and Vince, 2015). Using this technique, More and Vince (2015) were able to determine that A has a unique hyperspectral signatureβ has a unique hyperspectral signature , capable of distinguishing these deposits in ex vivo preparations of brain and retina from a mouse model of AD. More recently, the authors applied this technology to a live mouse retina, demonstrating that the hyperspectral signatures are preserved even when imaged through the ocular media. Thisdiscovery provides the firstevidence for the non-invasive invivo detectionof AD using hyperspectralimaging withouttheneedforanextraneousagent(Moreetal., 2016). Another emerging imaging technique is the use of cross-polarizers to distinguish ex vivo human donor retinas with AD from controls (Campbell et al., 2015; Hamel et al., 2016). This method capitalizes on the fibrillary arrangement of A resulting in specific changes toβ resulting in specific changes to birefringence thatarequantifiable using Mueller matrix polarimetry. Along theselines, theadventof PS-OCT mayalsoprovetobeuseful. Direct visualization of AD hallmarks with or without a contrast enhancing agent in the retina may be considered the most promising biomarker due to its specificity for the disease. However, ongoing work is needed to verify that A plaques or accumulations, are indeed present in humanβ resulting in specific changes to retinal tissues;andthatsuch depositsindicate or are predictiveof braindeposits. https://dx.doi.org/10.3389%2Ffnins.2016.00536 → Cited by 18
- 36. Fundus Imaging Multispectral imaging identifying spectral characteristicsof Alzheimer’sdisease Multi-spectral imaging for in vivo imagin g of oxygen tension and β-amyloid Dr. Tos TJM Berendschot, Prof. dr. Carroll AB Webers University Eye Clinic Maastricht https://goo.gl/i1nLbe Moreetal., 2016:[Citedby12] Inbluelight(510nm),nervefibersspreading from theopticdiscstrongly reflectlight.In green light(555nm) wherehemoglobinlight absorptionisstrong,retinalbloodvessels appeardarkagainstthefundusbackground, theveinsbeingdarkerthanthearteriesdue to theirrelativelyhighercontentofdeoxy- hemoglobin.Thespectralimage alsoshowsa reversal inthevesselcontrast,fromdarkerto brighter,betweengreen(555nm) tored (615 nm)wavelengths.Inredandnearinfrared light(615and705nm,lowerleftandright), both vesseltypesnowappearbrightagainst thebackground. Opticalspectrarecordedfromhumanandmouse retinasamples.(A) Comparisonofspectralshapes fromAlzheimer'sandnormalretina The upper curves showspectrafromhumanretinalwhole mounts,obtainedpostmortemfromanindividual diagnosedwithAD(n =1)comparedwithanormal subject(n =1).
- 39. Modify a fundus camera Spectrallyoptimalilluminationsfor diabeticretinopathydetection inretinalimaging Bartczak,P.,Fält, P., Penttinen,N. etal. OptRev (2017)24:105 https://doi.org/10.1007/s10043-016-0300-0 The DMD-based spectrally tunable light source used in this study: a optical setup of the designed DMD-based system, and b real implementation, where a Light source with focusing optics, b entrance slit with a UV filter, c collimating lens, d diffraction grating, e focusing lenses, f DMD, g focusing lens, and h liquid light guide Modified fundus camera used for retinal imaging: a Retiga 4000DC monochrome CCD camera (2048x2048 px), b illumination collection optics, c DMD’s liquid light guide, d forehead and chin rest, and e display
- 40. Field-of-View Requirements At least 45º field-of-view (FOV) of the retina. For larger destop devices Optos does 200º FOV and that of course could be nice but not necessary https://www.optos.com/en/products/the-benefits-of-optomap / http://www.boatmansoptical.co.uk/our-services/retinal_imagi ng?next=&id=18
- 41. Resolution Requirements Atleast30px/deg, and in general fundus imaging, the 5 MP resolution (at least 2 MP for NHS screening standard) is more than adequate. More important the dynamic range BEST would be non- mydriatic fundus camera so that we don’t need to pharmacologically dilate the subjects https://doi.org/10.1155/2017/4526243 Dr. BaileyShen’sdesign isso simple he includesa shopping list ofnecessarypartsand instructionsfor assemblyin his paper publishedin Journal ofOphthalmology. He saysthe camera can be replicated from partseasily found online forabout $185.
- 42. Dynamicrange Requirements Retina Typically optic disc gets overexposed making its quantitative analysis harder. Could “ROI HDR” of just that region be implemented by us? Fast acquisition allowed super-resolution techniques as well to be used Improved dynamic range Super-resolution fromretinal fundusvideos(Köhleretal.2014)
- 43. FlickerVessel Responseand retinal neurovascular functioning A schematic representation of the neurovascular unit (NVU) showing cellular elements regulating cerebral blood flow along the vascular tree. - Kisler et al. (2017): Cerebral blood flow regulation and neurovascular dysfunction in Alzheimer disease
- 44. Zoom/ROI Requirements One could want to zoom for vessels for functional response analysis. In other measure the width of vessels Amplitude ofsuchchanges are typically around -5%to+5% And if the vessel (vein orartery)takesfor example 10 pixelsinwidth,thequantification ofsuchchanges becomesimpossible in practice. Theaveragemean timecourseof arterial flicker reactionin age-relatedmaculardegeneration patients before(pink line)andafter(red line)thetreatmentaswellas in olderhealthycontrols(blueline).Thereaction ofyounger medicallyhealthyvolunteers(green line) isshown for comparison.Notethattheyoungervolunteersdemonstrate atwo-peaked flickerresponse http://doi.org/10.1111/j.1755-3768.2009.01718.x Characteristicexamplesofindividualsmoothed averagedretinalarterialandvenousreactionsto flickeringlight:(A) healthycontrol;(B)MCI- nonAD; (C)ADD; (D)MCI-AD. http://doi.org/10.1038/s41598-017-13349-5
- 45. VesselResponse withthe multispectral imagingaswell Measure oxygenization Meanmetricratiomeasurementsofretinalarterialandvenousoxygensaturation(SO2A R – redcircles;SO2V R– bluecircles)duringlightflickerandafter itscessationfromallsubjects(N= 16). Retinal Vascular and Oxygen Temporal Dynamic Responses to Light Flicker in Humans http://doi.org/10.1167/iovs.17-22174 Ahmed etal.(2014); Abayand Kyriacou (2017)
- 47. Spectral Requirements hbw = 10 nm bands should be good (or even slightly wider) from 400 nm to 810 nm or even further? Annidis RHA does 480–940 nm What sort of technical tradeoffs are there from this? e.g. 400nm goodforlensmediameasurements 460 nmgood formacularsegmentation 440-600 nm goodforAD 802nmusedinindocyaninegreenangiographyprocedures xnmyetunknownspectralsignatures 400-1000nmrangealreadybackin2010,withgooddynamicrangeandspectralresolution
- 48. Filter wheel with monochrome camera Somechallenges emergenow Chromaticaberrations, filternonidealities, chromatic focus differences Exact CorrectionofImageDistortioninaFilterWheel Multispectral Camerawithfocusadjustment RafaelCelestre-TechnischeUniversitätIlmenau https://vimeo.com/234484273 http://doi.org/10.1088/1742-6596/1044/1/012031
- 49. Tunable filters Youwould liketo switch abitfaster for human eyethan a mechanicalfilter wheelcoulddo 9March2018 Hyperspectralimaging:comparison ofacousto- opticandliquidcrystaltunablefilters RamyAbdlaty; Samir Sahli; JosephHayward; Qiyin Fang ProceedingsVolume 10573, Medical Imaging 2018:Physicsof Medical Imaging; 105732P (2018) https://doi.org/10.1117/12.2282532 Event: SPIE Medical Imaging, 2018, Houston, Texas,United States LCTF:Varispec VIS-10-20, (CambridgeResearch& Instrumentation). 400 – 720 nm AOTF:Gooch&HousegoTF625- 350-2-11-BR1A . 450– 800 nm + 8-channel digitalfrequency synthesizer (Gooch& Housego MSD040-150-0.2ADM-A5H-8x 1 Spectrophotometer: highresolution (~1nm) spectrophotometer(NEWPORT Model OSM400,Irvine,Ca,USA) Another distinct advantage of AOTF is the ultrafast (>1000 times) random access to spectral bands, compared to LCTF. This advantage makes AOTF a better candidate for imaging at videorate.
- 51. Digital Micromirror Devices Usedby Bartczak etal. (2017) for example forfundusimaging DMD’smicromirror array( Discovery™1100Chip Set,Texas Instruments,Inc.,USA)which contains 1024 ×768 micromirrors DLP®TechnologyforSpectroscopy DLP4500NIR[166–655SGD] http://www.ti.com/lit/wp/dlpa048a/dlpa048a.pdf Two-Dimensional MEMS ArraysPave Wayfor Mobile Spectrometers https://www.photonics.com/Articles/Two-Dimensional_MEMS_Arrays_ Pave_Way_for_Mobile/a61058 https://doi.org/10.1117/12.2291369 (2018) https://doi.org/10.1117/12.682232
- 55. Acousto- optical tunable filters (AOTF) Valle,Stefano(2017) Design and applicationofhigh performanceAcousto Optic TunableFilters. EngD thesis. UniversityofGlasgow http://theses.gla.ac.uk/8323/ Large aperture AOTFs are typically used as tunable filters for imaging systems or when a large etendue is required due to the spectralpropertiesofthesourceorthetarget. High quality crystals with linear dimensions larger than 30 mm are difficult to grow and processing significantly increases the cost of the final device. Therefore the design process is a trade-off between specificationandcost NKT Photonics for example is developing a source covering the wavelength range above 2 µm within the MINERVA Project (FP-7), consequently AOTFs are the typically used due to the fast random access capability and narrow bandwidth.
- 56. Ophthalmic example of illuminant filtering Spectral Imaging of the Human Retina and Computationally Determined Optimal Illuminants for Diabetic Retinopathy Lesion Detection Authors: Fält, Pauli1; Hiltunen, Jouni1; Hauta-Kasari, Markku1; Sorri, Iiris2; Kalesnykiene, Valentina2; Pietilä, Juhani3; Uusitalo, Hannu Journal of Imaging Science and Technology Volume 55, Number 3, May2011, pp. 30509-1-30509-10(10) https://doi.org/10.2352/J.ImagingSci.Technol.2011.55.3.030509
- 57. Non-Ophthalmic example of illuminant filtering 02February2018 Optimizedlightingmethodof applyingshaped-functionsignalfor increasingthedynamicrangeof LED-multispectralimagingsystem https://doi.org/10.1063/1.5022700 AIP ReviewofScientificInstruments Lightemittingdiode(LED) illumination-based multispectralimaging (LEDMSI)isoneofthe promisingtechniquesoffast andeffectivespectralimage acquisition. Highdynamicrange(HDR) imagingisahighdynamic rangetechniqueusedin imagingandphotographyto reproduceagreater dynamic rangeofluminosity Studiesarebeingcarriedout thatthemono-LEDMSI systematafixedexposure time,theLEDmodulatedbya shaped-functionsignal,and theframeaccumulate techniqueareintroducedto improvethegray-scale resolutionanddecreasethe lightintensity
- 58. Non-Ophthalmic example of illuminant filtering 25August2016 Afastmultispectrallight synthesiserbasedonLEDsanda diffractiongrating https://doi.org/10.1038/srep32012 Gregor Belušič,MarkoIlić,AndrejMeglič &PrimožPirih
- 59. Non-Ophthalmic example of LED Swept source April2018 Amultispectrallinearopticalsweepsource,based ontunableInGaN/GaNQWLEDs https://doi.org/10.1016/j.ijleo.2017.12.156 DipankarBiswas,AnupGorai,ApuMistry Citedby1 -Relatedarticles→ https://doi.org/10.1038/srep32012
- 62. Image Quality Assessment 04 September2017 QualityAssessmentofRetinalHyperspectralImagesUsingSURFandIntensity Featureshttps://doi.org/10.1007/978-3-319-66185-8_14 FatenM’hiri,ClaudiaChevrefils,Jean- PhilippeSylvestreOptinaDiagnostics Relatedarticles Havingagoodquality ofacquisitioniskeyfor betterautomatic evaluationof pathologiesinthe retina. Futureworkswill includetestsona largerdatasetto representpatients withdifferent pathologiesandwill includetheapplication ofaclustering approachforabetter qualitycategorization. Finally,morework shouldbemadeto reduceprocessing times
- 65. Example of a Prototyping System
- 66. JETSON + Dual Camera + Vitals
- 67. NVIDIA Jetson I/O + Triggering ● GPIOs,I2 C,I2 S,SPI,CAN ● TTLUARTwithflowcontrol ● Display ExpansionHeader ● CameraExpansionHeader DISPLAY The Jetson carrier board includes a 120-pin (2x60, 0.5mm pitch) Display Expansion Connector (J23). The connector used on the carrier board is a Samtec QSH-060-01-H-D-A. ThematingconnectorisaSamtecQTH-060-01-H-D-A. This expansion connector includes interface options for an embeddeddisplayandtouchcontroller including: CAMERA The Jetson carrier board includes a 120-pin (2x60, 0.5mm pitch) Camera Expansion Connector (J22). The connector used on the carrier board is a Samtec QSH-060-01-H-D-A. The mating connector is a Samtec QTH-060-01-H-D-A. The expansion connector includes interface options for multiple camerasaswellassomefor audio(I2S&DMIC): ▪ CSI up to 6x2 lane, CAM_I2C, Clock & Control GPIOs for▪ the Cameras, Digital Microphone IF, I2S, SPI, I2C,▪ ▪ ▪ ▪ ▪ UART NVIDIAJetsonTX1/TX2DeveloperKitCarrier Board| https://developer.nvidia.com/embedded/downloads
- 68. Jetson Nano $99,smaller Jetsonvariant GettingStarted withtheNVIDIA JetsonNano DeveloperKit Gettingstarted withNVIDIA’s GPU-basedhardware Alasdair Allan Apr152019 https://medium.com/@aallan/getting-started-with-t he-nvidia-jetson-nano-developer-kit-43aa7c29879 7
- 69. Google’s TPU Edge Accelerator Handsonwiththe CoralDev Board Gettingstartedwith Google’snewEdge TPU hardware Alasdair Allan Mar 262019 Ialsogo“HandsonwiththeCoralUSBAccelerator ”inacompanionarticle. https://medium.com/@aallan/hands-on-with-the- coral-dev-board-adbcc317b6af 5 mega-pixel (2582×1933 pixel) Omnivision OV5645 sensor, the camera isdesigned to connect to the board’sMIPI-CSI connector whichlocated on the underside of the Dev Board.More detailsof the camera hardware can be found in the datasheet. Everythingyouneedtogetstarted settingupthetheCoralUSB AcceleratorwithaRaspberryPi board
- 70. Jetson TK1 Obsolete e-CAM40_CUTK1-4MPRGB-IRCamera BoardforNVIDIATegraK1(JetsonDevKit) https://www.e-consystems.com/OV4682-RGB-I R-MIPI-JetsonTK1-Camera.asp note —JetsonTK1DeveloperKitisEOL. TK1chips and 3rd-partymodules remainavailable through2024.-https://elinux.org/Jetson_TK1 Mightofferenoughcomputingpowerforsomeapplications,but isalready EndofLife(EOL), sowouldnot designanewdevicearoundthis https://www.slideshare.net/NVIDIATaiwan/4-4-gtc2016embeddedclhungv30914 NVIDIAJetson Platform Characterization http://doi.org/10.1007/978-3- 319-64203-1_7
- 71. Intro for Embedded Camera Tech
- 72. USB or CSI camera?
- 73. MIPI CSI-2, GigE, USB 3.1, PCIe USB3.0isnowcalledUSB3.1Gen1(Speedupto5Gbps) USB3.1isnowcalledUSB3.1Gen2(Speedupto10Gbps) MIPICameraSerialInterface2(MIPICSI-2) -MIPI Alliance MIPICSI-2SM isthemostwidelyusedcamerainterfaceinthemobileindustry. https://mipi.org/specifications/csi-2 USB2.0 GigE USB3.1Gen 1 USB3.1Gen 2 MIPI CSI-2 D-PHY MIPI CSI-2 C-PHY PCIe Gen3 0 10 20 30 40 50 60 70 Bitrate [Gbit/s] USBCameras: ● Work finein manyusecases ● Areeasytointegrate ● Can doalotoftheimageworkoff- board(exposurecontrol,framerate, etc) ● Manyprovideinputs/interruptsthat canhelptimeyourapplication (e.g. interrupton newframe) ● UseCPU timeduetoUSBbus, thiswillimpactyourapplication if it uses100%CPU ● Arenotoptimalforuseofhardware vision pipeline(hardwareencoders, etc) ● Can workoverlong distances(upto maxofUSBstandard) ● Can supportlargerimagesensors (1"andhigherforbetterimage qualityandlessnoise) ● Givesyouhighlevelcontrolofthe sensor/camerathrougha manufacturer'sAPI ● Often havetools/GUIaround their APIsothatyoucan testoutfeatures withouthavingtowritecode CSIBusCameras: ● OptimizedintermsofCPUandmemory usageforgettingimagesprocessedand intomemory ● Cantakefulladvantageofhardware vision pipeline ● Canbea paininthearse software/controlwisesincethelibraries andsupporton TXplatformsarenot mature ● Barebonesfeatures(exposure,etc done inTX1) ● Short distancesfromTX1only(10cm maxusually)unlessyouuseserialization systems(GMSL,FPDLink,COAXPress, Ambarella)whichareimmatureand highly customat themomentthough someofthevendorsbellowcurrently carrysomemodels ● Aremostlysmallersensorsfromphone cameramodules butcustomonescan bemadeataprice.Theaddednoise fromthesmallersensorcan bemitigated abit throughthehardwaredenoisein TX1/2 ● Gives youaccesstolowlevelcontrolof thesensor/camera https://devtalk.nvidia.com/default/topic/1005570/jetson-tx2/best-cameras-for-jetson/
- 74. Same sensors used by most of the manufacturers
- 75. Global Shutter ones for fast motion SONY® Pregius™–CMOSGlobalShutter https://www.framos.com/media/pdf/2d/69/6d/FRAMOS-Sony-brochure.pdf Thesemodelsgoodfor embeddeddeeplearningfor multiframereconstruction purposes IMX287, IMX273,IMX174,IMX252, IMX250,IMX255,IMX253 Sony IMX250 provideshighresolutionof 5 MP,whichisperfectlysuitablefor applicationsunder low-light conditions andinmicroscopy.Framerate:86fps. MultiAOI FLIRBlackflyS 1,520SGD
- 76. Specific sensors for custom use cases HDR (High Dynamic Range) sensors for multi-lens HD video applications https://www.sony-semicon.co.jp/products_en/IS/sensor2/technology/dol-hdr.html https://www.framos.com/en/imx477-aack-c-19503 The Sony IMX477 can support up to 12.3 megapixel @ 60 frames per second video shooting under the ADC 10bit mode, the resolution format is 4056x3040. Enabling the SME- HDR mode, the camera can support 60fps, while it supports 30fps if enablingtheDOL-HDRmode. ThisnewCMOSimagesensor featuresSME-HDR (Spatially MultiplexedExposureHDR) and DOL-HDR (DigitalOverlapHDR), supportsdualsensorsynchronization operation,IMX477alsoadoptsthe back-illuminatedtechnologycan achievebetterimagequality underlow lightilluminationcondition.
- 77. Due to implementation pains The embedded camerawith best SDK/API/libraries will“win” the race, but which ofthese is it? NVIDIAPreferredPartners to startfrom?
- 78. e-con Systems https://youtu.be/w1igAYnbAmA https://www.e-consystems.com/13mp-autofocus-nvidia-jetson-tx2-camera-board.asp e-con Systems is a pioneer in bringing out high performance cameras for NVIDIA Jetson Developer kits. We are continuously launching many Jetson Cameras which utilizes the MIPI CSI-2 interface available on the NVIDIA development platforms. These Jetson cameras can be accessed by using standard V4L2 APIs. https://www.e-consystems.com/nvidia-jetson-camera.asp US$249 ExcludingShipmentCharges
- 79. Leopard Imaging Support forvarietyof sensors: IMX274 IMX185 IMX477 IMX377 IMX577 IMX290 MT9V024 IMX327 IMX334 IMX335 IMX390-GMSL2 IMX390-FPDLINKIII IMX390-GW5200-FPDLINKIII OV2311-GMSL2 AR0231-GMSL2 AR0231-FPDLINKIII AR0820-FPDLINKIII AR023Z-YUV OV10650-FPDLINKIII
- 80. Ximea XIMEA Linux APIon NvidiaJetsonTX1/TX2 On64-bitversionofLinuxFor Tegra(availablesinceR24.1)itispossibletousePCIecameras. xiAPIsupportstwoCPUfamilies: Intelx86-testedonLinuxUbuntux86_64: 14.04, 16.04, 18.04 ARM -testedon: LinuxUbuntu 14.04 on ODROIDXUBoard Linuxfor Tegra on NVIDIA Jetson TX1 and Jetson TX2 - see Linux TX1 andTX2Support and Multimedia UserGuide Linuxfor Tegra on NVIDIA Jetson TK1 (experimentalsupport) SupportedVisionLibraries, openCV https://www.ximea.com/support/wiki/vision-libraries/OpenCV_Installation_in_Linux SonyCMOSPregiuscamerashighestspeed xiX-Platformfor MulticamerasystemsandEmbeddedvision https://www.ximea.com/en/pci-express-camera/sony-cmos-pregius-fastest-smallest-cameras-multi Partialimage readout: ROI(region ofinterest), Multiple AOI(areaof interest) and Skipping /Binning modes on request MX120MG-CM9,5kSGD MX120CG-CM9,5kSGD MX120RG-CM10,5kSGD NVIDIAJetsonTX1/TX2carrier board https://www.ximea.com/en/products/xilab-application-specific-custom-oem/Embedded-vision-cameras-xiX
- 81. IDS NXT Platform NXTVegashttps://en.ids-imaging.com/ids-nxt.html IDSNXTplatformkicksoffwiththe IDSNXTvegas,withlight-sensitive1.3MP CMOSsensor (colour ormonochrome),liquidlenswith autofocus,integratedTOF (timeofflight,depth)sensorandnumerousother functionsandfeatures.Thecolour sensor modelcanberecognizedbyitswhiteLEDs.
- 82. Allied Vision 130/140Serieshttps://www.alliedvision.com/en/products/embedded-vision/embedded-vision-cameras.html MIPI CSI-2with enhanced functionality Buildingonitsexperienceinthe industrialandscientificfield,Allied Visionhasbuiltadvanced triggeringandframeratecontrol capabilitiesintoitsCSI-2cameras. Thatmeansthatembeddedusers nowhaveaccesstoafullycompliant MIPICSI-2cameramodule enhancedwiththetriggerprecision knowninindustrialinspection. SingaporeAlliedVisionTechnologiesAsiaPte.Ltd. 82Playfair Road #07-02D’Lithium Singapore368001T//+65-6634-9027ONSemiAR1820HS 3840x2160 48fps 1920x1080 60fps
- 83. Allied Vision 130/140Series Interfacing The 130 U and 140 U camera families can be operated on Linux x86/x64, Linux for ARM, and Windows. APIs:C,C++,and .NET
- 84. Basler dartBCON https://www.baslerweb.com/en/embedded-vision/embedded-vision-portfolio/embedded-vision-cameras/dart-bcon-for-mipi/ Starting from €49 (78 SGD), the dart camera module allows for leanest embedded systems with Machine Vision standards and excellent image quality. Easy code migration thanks to GenICam compliance and thanks to theunified pylon SDK Development kit with reference implementationavailable 5Megapixel,60fps,ONSemiconductor AR0521sensor https://www.baslerweb.com/en/products/cam eras/area-scan-cameras/dart/daa2500-60mc/ A 5MP (2592×1944pixels) Omnivision 5647 sensor in a fixed focusmodule Support 1080p30, 720p60 and 640x480p60/90 video record vs
- 86. Multi / Hyper / Ultra -spectral? Nostandardized nomenclature, the more bands, the morebuzzwordy sounding HyperspectralImagingfor LandmineDetection December 2017 IhabMakki https://www.researchgate.net/publication/323 443295_Hyperspectral_Imaging_for_Landmin e_Detection
- 87. Multispectral Sensor Types Snapshotimaging with custom filter arrays inpractice for ophthalmology https://www.e-education.psu.edu/geog480/node/494 Reviewofsnapshot spectral imagingtechnologies https://doi.org/10.1117/1.OE.52.9.090901 NAHagen, MWKudenov Optical Engineering, 2013 –spiedigitallibrary.org Citedby235 Relatedarticles https://www.vision-systems.com/articles/print/volume-20/issue-1/departments/technology- trends/image-capture-cmos-cameras-employ-hyperspectral-sensors.html
- 88. Commercial multispectral sensors LightHaus https://www.lighthaus.com.sg/hyperspectral.html Imec's missionistodrive the effective development and commercialization ofCMOSbased hyperspectral imagingvision solutions, together with a broad ecosystem ofpartnersatsensor, camera, software, illumination toapplication levels. Get in touch with one of ecosystem partner to explore potential collaboration for OEM business: https://www.imec-int.com/en/hyperspectral-imaging xiSpec -HyperspectralCameraswithUSB3Vision Compact,lightweightwithonly26.4x26.4 x31mm,32grams https://www.ximea.com/usb3-vision-camera/hyperspectral -usb3-cameras http://www.vttresearch.com/media/news/vtts-hyperspectral-imaging-technology- enables-new-artificial-intelligence-applications-as-a-part-of-consumer-devices MEMSfilterfromVTT Finland
- 89. Commercial Snapshot Spectral Imaging:TheArtof thePossible Michael West JohnGrossmann ChrisGalvan September 2018 https://www.mitre.org/sites/default /files/publications/pr-18-3832-com mercial-snapshot-spectral-imaging -art-of-possible.pdf
- 90. Modulating only the illumination for monochromecamera Ifyouwouldhave fast enoughcameraornon-movingtarget,youcan use “normal”spatial resolutionsusing off-the-shelf industrialcameraswithmultispectral light engines Spectrallytunablelight sourcesforimplementing computationallydesigned illuminations http://doi.org/10.13140/RG.2.2.20484.22400 Piotr Bartczak PhD Thesis Advisor:Markku Hauta-Kasari, JussiParkkinen, Project: ReVision https://youtu.be/EFbuwGS2cYE HyperCam: Hyperspectral Imaging forUbiquitous Computing Applications https://doi.org/10.1145/2750858.2804282 https://www.channelsystems.ca/news/cost-effecti ve-multispectral-solutions-mst-vision-and-xenics-i nfrared-solutions
- 91. Examples of Multispectral Illumination Systems A fastmultispectrallightsynthesiserbasedon LEDsandadiffractiongrating https://doi.org/10.1038/srep32012 TuLUMIS- atunableLED-based underwater multispectral imagingsystem https://doi.org/10.1364/OE.26.007811 Non-contact multi-spectralimaging combined withthermographytodeterminephysiological changes in perfusion duringclinicalinterventions https://www.researchgate.net/publication/3 05200249_Non-contact_multi-spectral_im aging_combined_with_thermography_to_d etermine_physiological_changes_in_perfusi on_during_clinical_interventions
- 92. Active field of research checkoutthespaceforlow-costoptionssuitableforscreeningdevices→ TunablePlasmonic Filters http://dx.doi.org/10.1021/acsphotonics.6b01003 Multispectral andpolarimetric photodetectionusinga plasmonicmetasurface JournalofAppliedPhysics 123,043107(2018); https://doi.org/10.1063/1.5011167 Citedby2 Relatedarticles We distinguish our FPI technology by means of two different manufacturing platforms: mass-producible optical MEMS- based chips (MOEMS FPI) for high-volume applications, and the separately assembled piezo-actuated tunable filter structures (Piezo FPI) for customised high-performance PFPI optical instruments. https://www.vttresearch.com/services/smart-industry/space- technologies/sensors-imaging-and-data-analysis/optical-mi crospectrometer https://doi.org/10.1364/OE.26.006021 Miniature integratedmicro-spectrometer arrayforsnap shotmultispectralsensing OpticsExpressVol.27,Issue4,pp.5719-5728(2019) https://doi.org/10.1364/OE.27.005719
- 93. (“Pseudo”)multispectral sensors RGB+NIRarraybyreplacingoneGreenpixelwithNIRpixel Canonannounces newultra-high-resolution 120-megapixel CMOSsensorcapableof simultaneous acquisitionofvisible-lightand near-infraredwavelengthimages TOKYO, March26,2019—Canon Inc.andCanon Marketing Japan Inc.announcedtoday thelaunchin Japan ofthe 120MXSI,anultra-high-resolution 120-megapixelCMOS sensorcapableofsimultaneousimagecapturein visiblelight andnear-infrared regions. https://canon-cmos-sensors.com/canon-120mxs-c mos-sensor/ https://www.ovt.com/purecel-pixel-tech/rgb-ir-technology RGB-IrTechnology OmniVision’sRGB-Ir technologycapturesbothRGBandinfrared(IR) imagesinasinglesensor
- 96. Full System forBiosignals: MySignals http://www.my-signals.com/#what-is-mysignals
- 98. MySignals hardware,addownsensors http://www.libelium.com/downloads/documentation/mysignals_technical_guide.pdf MySignalsSW BLE Complete Kit (eHealth Medical Development Platform) https://www.the-iot-marketplace.com/mysignal s-sw-ehealth-medical-biometric-complete-kit- ble used in https://doi.org/10.1155/2018/9054758 for example 3,050 SGD MySignalsSW BLE Complete Kit includes17 sensorswhichcan be usedto monitor more than 20 biometric parameters.
- 99. Pulse Oximetry Biopachttps://www.biopac.com/product/pulse-oximetry-transducers/ Research-gradesystem Part #: TSD124A,TSD124B, TSD124C The TSD124 series human oximetry transducers are reliable and simple to use on a wide range of subjects for both short-term and continuous noninvasive monitoring; use with the OXY100E oximetry amplifier or the OXYSSH-SYS Human Oximetry System (note: OXYSSH-SYSincludesone TSD124A). OXYSSH-SYS Human Oximetry System https://www.biopac.com/product/human-pulse-oxi metery-spo2-system/ In addition to SpO2, the OXYSS module supports threeauxiliaryoutputs: Pulse: A pulse plethysmogram signal that reflects changes in blood density on the finger tip caused by varying blood pressure Rate: HeartRatein BeatsPer Minute(BPM) Status optional: Outputs voltage levels that reflect the SpO2 signal status; requires BSLCBL14A interfacecable, purchasedseparately https://www.biopac.com/product/sp o2-pulse-oximeter-amplifier/ TheOXY100Emeasuresbeat-by- beat,bloodoxygensaturation (SpO2)levelinanoninvasivefashion. TheOXY100Eoutputsfour signalssimultaneously: A:SpO2 value(Ch1,2,3,or 4) B:PulsePlethysmogram(Ch5,6,7, or 8) C: Heartpulserate(Ch9,10,11,or 12) D:ModuleStatus(Ch13,14,15,or16) Thesesignalsaredirectedto switchableblocksofdifferentMP inputchannels.Uptofour OXY100E modulescanbeusedwithasingle MPSystem.
- 101. BluetoothLow Energy datastreamingnotnecessarily thebest option? e.g. MAX30100 / MAX30101 / MAX30102 PulseOximeter andHeart-Rate Sensor UsetheI2 C for commu- nication UserGuide6409 MAX3010xEVKits: RecommendedCon figurationsandOperatingProfiles https://www.maximintegrated.com/en/products/sensors/MAX30101A CCEVKIT.html/tb_tab0 Pulse Oximetry Build yourown?
- 102. MAXREFDES101#:Health SensorPlatform 2.0 The MAXREFDES101# is a unique evaluation and development platform in a wrist-worn wearable form factor that demonstrates the functions of a wide range of Maxim’s products for health-sensing applications. This second-generation health sensor platform (a follow-on to the MAXREFDES100#) integratesa PPG analog-front-end(AFE)sensor (MAX86141), a biopotential AFE (MAX30001) , a human body temperature sensor (MAX30205), a microcontroller (MAX32630), a power- managementIC (MAX20303),anda 6-axisaccelerometer/gyroscope. The complete platform includes a watch enclosure and a biometric sensor hub with an embedded heart-rate algorithm (MAX32664). Algorithm output and raw data can be streamed through Bluetooth® to an Android® app or PC GUI for demonstration, evaluation, and customizeddevelopment. Introducing the Health Sensor Platfor m 2.0 (MAXREFDES101) 2:45 September 20, 2018 https://www.maximintegrated.com/en/design/reference-design-center/system-board/6779.html Download All Design Files HardwareFiles: Schematic; BillofMaterials(BOM); PCBLayout;FabPackage;PCBCAD Software MAXREFDES101Software:Windows10 MAXREFDES101Software:AndroidAPK MAXREFDES101Software:Windows7
- 103. MAXREFDES101 withdual-modeBluetooth https://www.maximintegrated.com/en/design/reference-design-center/system-board/6779.html Algorithmoutputandrawdata canbe streamedthrough Bluetooth® to anAndroid® app orPC GUIfordemonstration, evaluation,andcustomizeddevelopment. BluetoothMultiMode:PAN1326B/1316B featuringboth BluetoothLowEnergy and Bluetoothconnectivity,basedonTexas Instruments’NEW CC2564Bcontroller, The Bluetooth Low Energy (BLE) standard is designed to reduce power consumption by establishing very fast connections (few ms) and transferring small amounts of data. Using these techniques, energy consumption is reduced to a tenth of a Classic Bluetooth device. Bluetooth Smart Ready devices can communicate with both Bluetooth Classic and Bluetooth Low Energy devices. Smart Ready devices take advantage of the data rate of Bluetooth Classic (3Mb/s) and ultra fast connection time of Bluetooth Low Energy (3mS).
- 104. MAX30001 is an analogue front-end for bio-potential electrocardiogram (ECG) and bio-impedance (BioZ), measuring these on the chest or wrist to detect heart rate, respiration (BioZ) and arrhythmias – detecting anomaliessufficientlywelltopromptthe user to get a medical-grade multi- electrode ECG, said Shieh. It meets IEC60601clinicalECGstandards. MAXREFDES101 Respirationsensing https://www.youtube.com/watch?v=55GS_3m8VNM Respiration (BioZ) Withelectrodeson chest(thesameas usedfor ECG,low- noiseamplifier) Canbeusedalsofor GSR(GalvanicSkin Response)
- 105. Optical Design of fundus camera
- 107. How about image quality through lenticular opacities e.g. cataractandincreased intraocularscatter with aging
- 108. Lightscatteringcontrol withneuralnetworksin transmissionandreflection AlexTurpin,IvanVishniakou,and JohannesD. Seelig (2018) https://arxiv.org/abs/1805.05602 Spatial light modulator (SLM, DMD,768 × 1024 pixels, pixel size = 13.7 µm2 model V-7000 from Vialux)) at a maximum frame rate of 22.7 kHz DMDs1-bit for(AO)SLO SLM 8-bit insteadofDMD
- 109. AdaptiveOptics Inpractice still expensive for“low-cost”designs Trans-pars-planar illumination https://doi.org/10.1038/s41598-018-27112-x polynomialwitharesponsetime<5ms.
- 110. DeepLearning ofcourse then tomake cheaperhardware performance betterorin combinationwith high-qualityhardwaredepending on yourbudgetand desired use Deepspecklecorrelation:adeeplearningapproachtowardscalableimagingthroughscatteringmedia YunzheLi,YujiaXue,andLeiTian Optica 5(10)1181-1190(2018) https://doi.org/10.1364/OPTICA.5.001181|Citedby14 -Relatedarticles
- 111. How about then other type of spectroscopy? Ramanspectroscopy fortheeye?
- 112. OCT + Raman Spectroscopy filter,M:mirror,NDF: neutral-density filter,NF: notchfilter,PC:polarization control,SLD:superluminescent diode.TheCCDinthesamplearmis usedasaviewfinder. Normalized Raman spectra from the foveal region of the human sample (similar peaks are observedtothe work of Beattie et al. [14]) compared with an average Raman spectrum obtained from asingle B-scan of porcine retinalsample.
- 113. Raman Spectroscopy for inflammation measurement MolecularImagingofthehuman retinabyRamanSpectroscopy (S26.001) NeurologyApr 2015,84 (14Supplement) S26.001 https://n.neurology.org/content/84/14_S upplement/S26.001 https://doi.org/10.1002/jbio.201300101 Retina is part of the Central Nervous System (CNS) and it can be easily assessed with photonic devices such as Optical Coherence Tomography (OCT) that offers a good in-vivo structural retinal image. In Multiple Sclerosis, OCT reveals Ganglion Cell Layer thinning that parallels brain atrophy. However, Inner Nuclear Layer thickening has been associated with inflammation and neurodegeneration. Molecular imaging may better and earlier discriminate these mechanisms since functional changes usually appear in an early phase of disease, a key period for neuroprotectivedrugs. Raman spectroscopy analysis of the Retinal Ganglion Cell layer of the retina. (A) Design of the analysis of the Ganglion cell layer (GCL) and Retinal Nerve Fiber Layer (RNFL) of the retina based in the physical properties of laser light [785 nm (5-100 mW)] and anatomical structure ofretinallayers. (B) Examples of raw Raman spectra from representative retina sample after 10 hours incubation time (black) and LPS treated retina sample after 10 hours incubation time (red) and12hoursincubationtime(blue).
- 114. Raman Spectroscopy asAlzheimer’sand neurodegenerative biomarker Label-freeimagingof amyloid plaquesinAlzheimer’sdisease withstimulatedRamanscattering microscopy https://sci-hub.tw/10.1126/sciadv.aat7715 (2018) ...the existence of extracellular deposition of amyloid plaques formed with misfolded amyloid- (A ). Theβ resulting in specific changes to β resulting in specific changes to conformational change of proteins leads to enriched contents of sheets,β resulting in specific changes to resulting in remarkable changes of vibrational spectra, especially the spectral shifts of the amide I mode. Here, we applied stimulated Raman scattering (SRS) microscopy to image amyloid plaques in the brain tissue of an ADmousemodel. E. Ryzhikova et al. (2015). Raman spectroscopy of blood serum for Alzheimer’s disease diagnostics: specificity relative to other types of dementia. J Biophotonics, Vol. 8, Issue 7, pp. 584- 596. https://doi.org/10.1002/jbio.201400060,from: https://www.photonics.com/a63331/Raman_Hyperspectroscopy_Shows_Promise_for https://doi.org/10.1007/s12035-018-1151-4
- 115. Raman useful for retinopathies from bloodsamplesalso Fourier-transform-infrared- spectroscopybasedmetabolomic spectralbiomarkerselection towardsoptimaldiagnostic differentiationofdiabeteswith andwithoutretinopathy https://doi.org/10.1080/00387010.2018.1 471510 Again, lipidic signatures were strongly corroborated with glycosylated hemoglobin levels in diabetic retinopathy and diabetic subjects. Spectral signatures also revealed an elevated level of -sheet containingβ resulting in specific changes to proteins in serum in diabetic retinopathy condition. The method was validated through spectral biomarker selection by the DBMS technique. Thus, this method has the capability of diagnostic cost minimization for detection of diabetic retinopathy by label-free spectralbiomarker identification. Raman spectral probe and unique fractal signatures for human serum with diabetes and early stage diabetic retinopathy https://doi.org/10.1088/2057-1976/aaed0e Most importantly, we observe unique arc-fractals by gold particles in presence of diabetes retinopathy serum samples. We quantify the fractal geometry by measuring the persistence length, which can be used as a biomarker in pathological reporting systems. We believe that our findings will motivate the medical researchers to employ the proposed methodologies for diagnosing diabetes/early stage diabetic retinopathy on larger number of subjectsin clinical trials.
- 118. Mobile spectrophotometry withouttheimaging (a) The smartphone spectrometer with the inset (top-left) of a plastic fiber assembled into the built-in flash. (b) The spectral images from different-sized plastic fibers are shown. https://doi.org/10.5772/intechopen.82227 The communication between the smartphone and a remote server is illustrated in (a) and the developed ChemTrainer (for testing peroxide levels from urine) app is presented in (b). Similar systems exist for albumin, pH, protein and glucose valuesfrom samples. https://doi.org/10.5772/intechopen.82227 https://spectralworkbench.org/ Luminit One Luminit Two Spectruino https://myspectral.com/ Spectrometryoflight sources,lasers, environment Radiometer – Measure irradiance oflightsources, LED, laser, sun, Internet ofThings Sensor for Sunlight, public lighting. Spectrometer – Measure reflected spectrum from four spectrallybalanced lights reflected from surfaces. IoT NDVIindex, Normalized DifferenceVegetationIndex, whiteness ofpapers, walls, chemical and environmental agents, and more. Spectrophotometer – Measure light passing through a cuvettecontaining liquids ofvarious colors. With spectrometric cameramode, you can even observe kinetics of reactions, measure adding ofconcentrates to beverages online, or measure turbidityand fluorescence ofliquids, algae, phytoplankton. Programmable – Spectrometer, spectroradiometer with Arduino, microbit and RaspberryPi. LuminiTWOSpectrometer 1 599.00 € excl. VAT
- 119. Fundus+PPG Enter Arduino Uno for PPG. Does notsolve thebulkiness of the camera May2018 Assessingblood vesselperfusionandvital signsthroughretinalimaging photoplethysmography https://doi.org/10.1364/BOE.9.002351 HarnaniHassan,SheilaJaidka,VincentM.Dwyer,and SijungHu The work presented here demonstrates a simple retinal imaging photoplethysmography (iPPG) system with the potential to provide screening, diagnosis, monitoring and assessment that is non-invasive, painless and radiationless. Time series of individual retinal blood vessel images, captured with an eye fundus camera, are processed using standard filtering, amplitude demodulation and principle component analysis (PCA) methods to determine the values of the heart rate (HR) and respiration rate (RR), which are in compliance with simultaneously obtained measurements using commercialpulseoximetry.
- 120. “Spectral Selfie” June2018 ExperimentalDemonstrationofRemoteand CompactImagingSpectrometerBased on MobileDevices https://doi.org/10.3390/s18071989 JieChen,FuhongCai,RongxiaoHe,andSailingHe Citedby4 -Relatedarticles In this paper, we develop a compact imaging spectrometer (35 g in weight, 18 mm in diameter, and 72 mm in length) that can be equipped on a motorized selfiestick toperformremote sensing. Oursystemoffers a route toward mobile imaging spectrometers used in daily life AWi-FiCMOScamerawitha20mmclosed-circuit television(CCTV)lensisutilizedtocapturethespectral image.(Inset) onespectralimagecapturedbytheWi- Ficamerawhenmeasuringamercurylamp. The selfie stick-based imaging spectrometer was utilized to scan environmental objects on campus. (a) Photo of the scanned object. The scanning was performed in the horizontal direction; (b) the spatial image obtained from the scanning result; (c) the reflectance spectrum from the tree region; (d) the reflectance spectrum from the building region; (e) the reflected light ratio index (RLRI) image based on the hyperspectral data cube; (f) the overlap of the spatial image and the binary image based on RLRI value.
- 121. Low-cost tunable LED based system This paper presents a full end-to-end analysis of a low-cost reflectance-mode multispectral imaging system operating in the visible and near-infrared spectra. The system consists of 13 LEDs mounted on a printed circuit board, a monochrome machine vision camera, and a tablet computer to control the hardware. The bill of materials for the system is less than $1000. Hardware design and implementation are detailed. December 2018 Portable,low-costmultispectralimagingsystem:design,development,validation,andutilization FrankJ.Bolton; AmirS.Bernat; Kfir Bar-Am; DavidLevitz; StevenJacquesAuthorAffiliations- FrankJ.Bolton , AmirS.Bernat , KfirBar-Am , DavidLevitz,StevenJacques J.ofBiomedicalOptics,23(12),121612(2018). https://doi.org/10.1117/1.JBO.23.12.121612 Additional future work will critically assess the combination of LEDs used for the spectral illumination to maximize performance at reduced cost. The current implementation is limited by assumptions made in the fitting method, which employs a limited set of tissue parameters. More extensive Monte Carlo methods will be implemented to better optimize the method, as a larger lookup table can provide finer estimates of tissue parameters.
- 122. Design considerations for tunable LED based system April2018 Methodusinginvivoquantitativespectroscopytoguidedesignandoptimizationoflow-cost, compactclinicalimagingdevices:emulationand evaluationofmultispectralimagingsystems RolfB.Saager;MelissaL.Baldado;RebeccaA.Rowland; KristenM.Kelly;AnthonyJ.Durkin https://doi.org/10.1117/1.JBO.23.4.046002|Citedby2 -Relatedarticles Spectralresponsefunctionsforthe emulatedimagers:(a) thespectral bandwidthoftheeightLEDsusedinthe OxImager, (b) thespectralsensitivityof theBayerfilterfromthecolorimager, and(c) thespectralsensitivityofthe16 channelsoftheXiSpechyperspectral snapshotimager.Panel(d) callsout fourindividualchannelsfromthe XiSpecimager,illustratingthebest(top figures)andworst(bottom)casesof spectralselectivityfromthisimager.