Bringing Computer Vision to the Masses David Moloney, CTO HiPEAC CSW HIPP 07 October 2014 © Copyright Movidius 2014 Movidius Vision Processor - Computational Imaging Visual Awareness Slide©2Copyright of 2 Movidius 2014 Where do current phone cameras fall short? • • • • Poor light conditions Shot delay Thickness going against slim design trends Enabling real vision based user experiences Slide©3Copyright of 3 Movidius 2014 The opportunity: the 3rd imaging revolution From Digital Imaging to Computational Imaging Computational Optics Smaller f# (better night picture) Thinner Camera Computational Photography N x M aperture Camera Smaller Pixel (higher noise) >2014 New Color Patterns (clear…) Increasing Resolution/MP Vision Processing Today <2013 Higher Image Quality New Imaging Experience Intelligence from Scene Lens (Optics) Image Sensor Image Processing Image Experience Transition of the entire imaging value chain Why Digital > Computational Imaging ? the only way to overcome physical, mechanical and computational limitations of today’s mobile cameras © Copyright Movidius 2014 Why are computational cameras a game changer ? Highway to the next generation user experiences Visual Awareness Computational Photography Image Capture 79.3 mph Image Capture The old paradigm “optical” zoom, depth, ultra-fast AF, panorama, HDR, spherical capture, extreme low light 3D modeling, scanning, visual search, in-door navigation, augmented reality, object detection, object recognition… Vision Processing: the new imaging paradigm © Copyright Movidius 2014 Google – Project Tango Bringing Visual Awareness to next generation Android devices Mobile Applications • • • • • • • • Enhanced Photography Natural User Interfaces Immersive Gaming Augmented Reality Indoor Navigation Visual Search 3D Scanning Robotics 2x Myriad1 VPUs © Copyright Movidius 2014 Need for Special Purpose Vision Processor GOPS 1500 Computational Photography 400 GB/sec Computer Vision 1000 200 GB/sec 500 ISP 1W Application Processors’ Typical Performance Given Thermal Limitations and Limited Battery Life 100 GB/sec Computational Complexity Source: Movidius © Copyright Movidius 2014 Introducing the Myriad 2 Vision Processor SOC Optimized configurable imaging and vision hardware engines (framework) Vector VLIW processors designed to crunch complex vision and imaging algorithms at high performance and low power Interfaces Computational Imaging Hardware Accelerators Vector Processors RISCRT x12 RISCRTOS RISCs run RTOS, Firmware, RunTime Scheduler… Memory designed for low power, zero latency, sustained high performance through data locality Memory Fabric Nominal 600 Mpixels/sec throughput enables connection to multiple cameras, world-class computational imaging pipelines, and complex vision applications © Copyright Movidius 2014 Myriad 2 Detailed System Diagram SW Controlled I/O Multiplexing CIF SDIO NAL I2C SPI x3 x3 USB3 SIPP Hardware Accelerators RAW LSC Harris Corner Chroma Denoise Debayer Colour Combination Median Filter Luma Denoise Polyphase Scaler Sharpen Filter LUT Conv Kernel Edge Oper I2S SPI x3 x3 UART RISCRT L1 4/4 kB L2 32kB CMX Memory Fabric 2MB Multi-Ported RAM Subsystem 17 independent power islands power 32x HW Mutex Arbiter & 16:1 mux SHAVE 11 SHAVE 10 SHAVE 9 SHAVE 8 SHAVE 7 SHAVE 6 SHAVE 5 SHAVE 4 SHAVE 3 SHAVE 2 Inter-SHAVE Interconnect (ISI) SHAVE 1 RISCRTOS L1 32/32k B L2 128kB ROM 64kB AMC Crossbar SHAVE 0 ETH 1Gb 128 LCD SPI SPI SPI x3 x3 x3 Bridge Main Bus MIPI D-PHY x12 lanes L2 cache 256kB PLL & CPM DDR Controller Stacked KGD 1-8Gbit LP-DDR2/3 © Copyright Movidius 2014 The Always-On Challenge Total System Power in mW range: • Sensor innovation • Interface innovation • Processor innovation • Wake up system by degrees in response to events of interest When Running: • Massive Processing Requirements • Low Latency Response • Low Power Requirement • Broad applications require programmability Copyright Movidius 2014 Slide©10 of 10 Do I need an Always-on camera? © Copyright Movidius 2014 Issues with Always-on Cameras? • Privacy – Do we want always-on access to our daily lives? – Do we bring the cloud to bear on hard problems? • Power/Heat – Related to resolution, frame-rate, algorithms, HW … • Bandwidth – The “elephant in the room” • Latency – Interactive services useless if latency is 100s of msec – Augmented reality requires <7msecs (Abrash) © Copyright Movidius 2014 • H2020 EoT Consortium: © Copyright Movidius 2014 Changing our perspective • Always-on use-cases require constant view of world • Wearable button camera only alerts user when trigger-event or sequence occurs • The “pixels stay in the camera” paradigm – guarantees privacy – energy efficient – conserves precious data bandwidth – enables low-latency services EoT button BT LE Myriad 2 (WLCSP) IMU NanEye © Copyright Movidius 2014 No More Free Lunch! And Logic scales @ 50% Vs Memory @ 20-30% https://n3xt.stanford.edu/system/files/b_cronquist_-_monolithic_3d.pdf Challenges to Research & Industry • You can no longer optically scale your way out of trouble ($ or mW) • More heterogeneous HW to replace Dennard scaling • Software needs to fill the scaling gap but historically 3% over 50yrs © Copyright Movidius 2014 Thank you! Q&A © Copyright Movidius 2014
© Copyright 2024 ExpyDoc
