Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Corneal transparency
Corneal transparency
Loading in …3
1 of 66

Raskar stanfordextremecompuimagingapr2016



Download to read offline

Talk at Stanford SCIEN meeting 2016

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Raskar stanfordextremecompuimagingapr2016

  1. 1. Camera Culture Ramesh Raskar Ramesh Raskar Asso. Professor MIT Media Lab Extreme Computational Imaging
  2. 2. Mohit, Shah, Angelino, Sinha
  3. 3. Huang, Wetzstein, Barsky, Raskar. SIGGRAPH 2014. Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays.
  4. 4. EyeSelfie, Swedish et al SIGGRAPH 2015
  5. 5. Replacement Expansion Prediction Predictive Modeling of Future Health
  6. 6. Theia: API for Diabetic Retinopathy Screening Open API and Documentation Drag and Drop Web Demo Swedish et al
  7. 7. Visual Urban Sensing Naik, Glaeser, Hidalgo, Raskar (email: Crowdsourcing Computer Vision + 3.6/10 Sentiment Analysis for Street-level Images Quantifying Urban Change Applications: Real estate, Data-driven urban policy and planning Decline Improvement How to use computer vision and crowdsourcing to evaluate the appearance of cities across the world at street-level resolution and global scale?
  8. 8. Smartphone spectroscopy for consumer applications Rechargeable battery Arduino pro mini Bluetooth module Spectrometer chip LED FilterLight switch Power switch Application in rapid testing of food quality: Chlorophyll fluorescence in fruits Anshuman Das, Akshat Wahi, Ishan Kothari and Ramesh Raskar
  9. 9. Hand-held 3D Imager for biomedical applications 3D imaging of throat and tonsils for evaluation of obstructive sleep apnea Anshuman Das, Tulio Valdez, Ramesh Raskar et. Al., Biomed. Opt. Express 7, 1445-1457 (2016) 3D imaging of the tympanic membrane for accurately detecting ear infections Anshuman Das, Julio Estrada, Ramesh Raskar et. Al., Proc. SPIE, 9303, 93031F (2015)
  10. 10. Health and Wellness Interfaces
  11. 11. Positions Open ( Collaboration/Expt Co-funding
  12. 12. Raskar, Camera Culture, MIT Media Lab MIT Media Lab
  13. 13. Bit Hacking PhotonHacking Computer Vision Optics Sensors Signal Processing Displays Machine Learning Computational Light Transport Computational PhotographyIllumination
  14. 14. Diffraction Dynamic Range Descattering Defocus Depth
  15. 15. Milli Micro Nano Pico Femto Atto Conquer .. Time
  16. 16. Milli Micro Nano Pico Atto Femto-graphy Conquer .. Time Noise Signal
  17. 17. Light in Slow Motion .. 10 Billion x Slow
  18. 18. Velten et al, Siggraph 2013
  19. 19. Andreas Velten, Di Wu, Christopher Barsi, Everett Lawson
  20. 20. Raw Data
  21. 21. Can you see around corners ?
  22. 22. 2nd Bounce Multi-path Analysis 1st Bounce 3rd Bounce 28
  23. 23. Collision avoidance, robot navigation, … 29 DARPA REVEAL Program by Dr. Predrag Milojkovic
  24. 24. Motion beyond line of sight Pandharkar, Velten, Bardagjy, Lawson, Bawendi, Raskar, CVPR 2011 30
  25. 25. …, bronchoscopies, … 31
  26. 26. Identify Materials
  27. 27. Naik, Zhao, Velten, Raskar, Bala, (SIGGRAPH Asia 2011) BRDF from a single viewpoint
  28. 28. Single Photon Sensitive Imaging Gariepy et al. Nature Comm (2015)
  29. 29. Millisecond Microsecond NanoSecond PicoSecond FemtoSecond AttoSecond RFlow CAT- scan Phone3D Reading Book Frequency TimeResolution Look Around Corners Audio RF XrayLightTHz FLIM
  30. 30. Multi-Dimensional Light Transport 5-D Transport
  31. 31. Nanophotography 70 picosecond resolution [Kadambi et al 2013]
  32. 32. Real-time Localization Imaging Real Time Localization Kadambi, Zhao, Shi, Raskar. "Occluded Imaging with Time of Flight Sensors." ACM ToG 2016 (Pres. at SIGGRAPH)
  33. 33. Wavelength vs Shininess
  34. 34. Macroscopic Interferometry Optical Interferogram Electronic Interferogram Kadambi, Schiel, Raskar CVPR 2016 (Oral)
  35. 35. Macroscopic Interferometry SNR Benefits Works at long-range Obeys theoretic bounds from optical coherence
  36. 36. Macroscopic Interferometry
  37. 37. Depth Sensor Kadambi, Taamazyan, Shi, Raskar, ICCV 2015 Polarized 3D
  38. 38. Mitigate Exploit Create Scattered Light
  39. 39. FLIM Satat et al. Nature Comm 2015 Bhandari et al. 2015
  40. 40. Application: Fluorescence Lifetime Imaging ToF Imaging : Relative Phase Ecodes Distance Info FLIM : Relative Phase Ecodes Lifetime Info Probing Function Response Time Domain FLIM ToF Imaging FLIM Imaging Frequency Domain FLIM Excitation Fluorescence Lifetime Parameterizes Lifetime
  41. 41. Application: Fluorescence Lifetime Imaging Bhandari, Barsi, Raskar, 2016 Blind and reference-free fluorescence lifetime estimation via consumer time-of-flight sensors
  42. 42. Location + Fluoro Lifetime Behind Diffuser Satat et al. Nature Comm (2015) Bhandari et al. 2015
  43. 43. Imaging Behind Thick Scattering Media 1.5cm Thick Diffuser Hidden Scene Reconstruction with ToF Satat et al. “Imaging Through Thick Medium Using Time-Resolved Measurement” COSI (2015)
  44. 44. Mohit, Shah, Angelino, Sinha
  45. 45. Optical Brush Heshmat, Lee, Raskar,, Nature Scientific Reports, 2016
  46. 46. Heshmat, Lee, Raskar,, Nature Scientific Reports, 2016 Optical Brush
  47. 47. Sanchez, Heshmat, Reza, Romberg, Raskar 2015
  48. 48. Sanchez, Heshmat, Reza, Romberg, Raskar 2015
  49. 49. Bedri, Gupta, Mohit et al Time-of-Flight Microwave Camera Charvat, Temme, Feigin, Raskar Nature SciRep 2015
  50. 50. Kadambi, Bhandari et al
  51. 51. Millisecond Microsecond NanoSecond PicoSecond FemtoSecond AttoSecond RFlow CAT- scan Phone3D Reading Book Frequency TimeResolution Look Around Corners Audio RF XrayLightTHz FLIM
  52. 52. Multi-Dimensional Light Transport 5-D Transport

Editor's Notes

  • Provide oral health solutions via a tooth brush that takes achieves x-ray like screening
  • Vision is important for day labor for his employment
  • Siggraph 2015 Paper: EyeSelfie: Swedish, Roesch, …, Raskar

  • My work involves creative new ways to play with light by co-designing optical and digital processing.

    My work lies at the INTERSECTION of processing of photons and processing of bits.

    At MERL, I transformed the field of computational photography, with key papers and impact on products
    At Media Lab, I invented a new field ‘computational light transport’

  • If I turn this laser point ON and OFF in a trillionth of a second .. i.e. in a few femto-second ..

  • I will create a packate of photons ..
    less than a milimeter long ..
    Bullet of light .. Traveling a million times faster than ord bullet
  • What if we send this bullet of photons to shatter into this bottle
    How does light look in slow motion
    The whole sequence is less than a nanosecond stretched to about 10 seconds i.e. 10 billion times slowed down
    If an oridinary bullet, same distance at such a slow motion video, do you know how long you will have to sit here to watch the movie? A day, a week .. a whole year .. it will be a very boring movie ..
  • so we have to use a stroboscopic sampling method to fire the bullet millions of times and do clever sync and record raw data, and use computational techniques to to weave together femto-video.

    World’s fastest videos
    Same photo about 10 million times to get enough photons
    World’s fastest slowest camera ..

  • Superman can fly and heroes can becomes invisible .. But what about a new power for a future super-hero ..
    To see around corners ..
  • My idea is to use the multiple bounces of light i.e. echoes of light.

    Echoes of sound -> Echoes of light

    We all know about echoes of sound.
    But sounds travels slow and we can actually hear the echoes
    Light travels fast so we need specialized hardware to ‘listen’ to these echoes.
    So we end up using light sources and cameras that run at a trillion frames per second (not a million and not a billion, but trillion)
  • And to see deep inside the body with endoscopes that look in narrow channels in lungs
    Or colons

  • Provide oral health solutions via a tooth brush that takes achieves x-ray like screening
  • Ancient text witout damaging books
  • How can we drive on foggy, crowded streets
    As if it is a sunny day on a clear road
  • My Photo 24 roll film for one year
  • ×