{"id":9854,"date":"2018-02-27T11:48:17","date_gmt":"2018-02-27T11:48:17","guid":{"rendered":"https:\/\/www.techdesignforums.com\/practice\/?p=9854"},"modified":"2018-03-05T02:03:52","modified_gmt":"2018-03-05T02:03:52","slug":"optimizing-power-and-performance-trade-offs-in-cnn-implementations-for-embedded-vision","status":"publish","type":"post","link":"https:\/\/www.techdesignforums.com\/practice\/technique\/optimizing-power-and-performance-trade-offs-in-cnn-implementations-for-embedded-vision\/","title":{"rendered":"Optimizing power and performance trade-offs in CNN implementations for embedded vision"},"content":{"rendered":"
\n
\n

High-performance, low-power embedded vision processors are used in many applications: face recognition in mobile phones; identifying hands and room layouts to enable augmented or mixed-reality headsets to incorporate them into game-play; and in\u00a0self-driving cars to see and interpret the road ahead.<\/p>\n

These applications demand the use of deep-learning algorithms, such as convolutional neural networks (CNNs), which demand\u00a0a lot of processing power yet need to operate at the lowest possible power consumption. As in many other areas of computing, such as digital signal processing, achieving the best trade-off between power and performance demands the use of dedicated CNN engines.<\/p>\n<\/section>\n<\/div>\n

\n
\n
\n
\n
\n
\n
\n

No URL for image<\/em><\/p>\n<\/div>\n

Chip options for deep learning<\/span><\/h2>\n<\/section>\n<\/div>\n<\/div>\n
\n
\n
\n

The computational power of GPUs has provided the horsepower needed to explore different deep-learning algorithms and train the best candidate architectures on real-world tasks. However, these processors, originally built to handle\u00a0graphics,\u00a0have limited utility\u00a0in power-sensitive embedded applications.<\/p>\n

\n

Vector DSPs \u2013 very large instruction word SIMD processors \u2013 were designed as general-purpose engines to execute conventionally programmed computer-vision algorithms. A vector DSP\u2019s ability to perform simultaneous multiply-accumulate (MAC) operations allows it to run the two-dimensional convolutions needed to execute a CNN graph more efficiently than a GPU. Adding more MACs to a vector DSP will allow it to process more CNN calculations per cycle, enabling higher frame rates, but even greater power and area efficiency can be gained by adding dedicated CNN accelerators.<\/p>\n

The best of both worlds can be achieved by using a\u00a0vector DSP \u00a0for pre- and post-processing of video images, and then closely coupling it to a\u00a0dedicated CNN engine, designed to support all common CNN operations such as convolutions and pooling, at minimum die area and power consumption.<\/p>\n

Such a dedicated CNN engine, optimized for memory and register reuse, is as important as increasing the number of MAC operations that the CNN engine can perform each second. If the processor doesn\u2019t have the bandwidth and memory architecture to feed those MACs, the system cannot achieve optimal performance.<\/p>\n

Even lower power can be achieved with a hardwired ASIC design, which may become desirable\u00a0when the industry agrees on embedded vision processing standards, as happened when the video industry settled on the H.264 compression standard. At the moment, though, CNN architectures are evolving so quickly\u00a0that\u00a0designers are better off building flexible solutions.<\/p>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n

\n
\n
\n
\n

Measuring in-system power before silicon<\/span><\/h2>\n<\/div>\n
\n

For the most power-sensitive embedded vision applications, its important to be able to predict power consumption before silicon is available.<\/p>\n

Consider an application with a tight power budget, such as a battery-powered IoT smart home or mobile device running a facial-recognition algorithm that may require several\u00a0hundred GMAC\/s of processing power. The power budget for all this computation may be limited to several hundred mW.<\/p>\n

Comparing different vision processor IP blocks is complex. Bleeding-edge IP solutions often haven\u2019t reached silicon yet, and every implementation is different, making it difficult to compare their power and\u00a0performance. There aren\u2019t any standard benchmarks for comparing CNN solutions, and although an FPGA prototyping platform might provide accurate performance benchmarks it won\u2019t provide\u00a0\u00a0accurate power estimates.<\/p>\n

One way to calculate power consumption is to run an RTL or netlist-based simulation to capture how all the design’s logic is toggled. This information, using the layout of the design, can provide a good power estimate, and for smaller designs, the simulation may\u00a0complete in hours. However, a simulation of a high\u00a0frame-rate CNN graph could take weeks to reach a steady state in which to measure power. There is a risk that when IP vendors skip such detailed power analysis\u00a0in favour of using estimates based on\u00a0smaller simulation models, the resultant power problem will be\u00a0pushed downstream to the SoC vendors to sign-off on the IP vendor\u2019s power analysis claim.<\/p>\n

Synopsys\u2019 ZeBu Server <\/a>can help both IP developers and SoC designers by supporting advanced use modes including power-management verification, comprehensive debug and Verdi integration, hybrid emulation with virtual prototypes, and architectural exploration and optimization. ZeBu has additional facilities to compute power accurately for hundreds of millions of clock cycles, in hours instead of months. Access to a ZeBu emulator therefore makes it easier to explore power\/performance tradeoffs in running application software on various candidate hardware architectures.<\/p>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n

\n
\n
\n
\n

Systemic optimizations for lower power<\/span><\/h2>\n<\/div>\n
\n

There are various other ways to cut systemic power, for example by moving to a denser\u00a0processing node, lowering the operating frequency of the design, or applying near-threshold logic techniques\u00a0to\u00a0greatly reduce the energy needed to switch a\u00a0transistor. Another approach is to minimize the amount of external bus bandwidth the design needs, by increasing the amount of on-chip memory. Designers can also minimize bandwidth\u00a0requirements by applying\u00a0compression techniques to the key parameters of their CNN graphs.<\/p>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n

\n
\n
\n
\n

Reducing energy use is obviously\u00a0important for battery-powered applications, but even designs for\u00a0<\/span>autonomous vehicles need to take care of power consumption. An autonomous\u00a0vehicle, running multiple\u00a08Mpixel cameras at 60frame\/s, could require 20 to 30 TMAC\/s of computational power \u2013 within the lowest possible power budget.<\/p>\n

In\u00a0this context, designers need access to an optimized CNN engine whose performance can scale up. In\u00a0Synopsys\u2019 EV6x family, this is done in two ways: by scaling the number of MACs in\u00a0each CNN engine, and also by allowing multiple instances of the CNN engine to share a bus fabric. You can see this in\u00a0Figure 2, which shows (at top) an\u00a0EV61 processor with an 880 MAC CNN (which can \u00a0be scaled up to 1760 or 3520 MACs), and (below) multiple EV processors on an AXIbus or other network-on-chip fabric.<\/p>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n

\n
\n
\n
\n
No URL for image<\/em><\/div>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n
\n
\n
\n
\n

Deep-learning algorithms for embedded vision are developing very rapidly at the moment. Dedicated CNN engines can provide\u00a0power-efficient implementations of such algorithms, especially when tightly coupled to supporting vector processors as in the DesignWare EV6x Embedded Vision Processor<\/a> family. Embedded system developers can use tools such as the\u00a0ZeBu Server to\u00a0compare power consumption between embedded processor offerings.<\/p>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n<\/div>\n

Author<\/b><\/h2>\n

Gordon Cooper is product marketing manager for EV processors at Synopsys.<\/p>\n

Company info<\/b><\/h2>\n
Synopsys Corporate Headquarters<\/i><\/address>\n
690 East Middlefield Road<\/i><\/address>\n
Mountain View, CA 94043<\/i><\/address>\n
(650) 584-5000<\/i><\/address>\n
(800) 541-7737<\/i><\/address>\n
\u00a0<\/i>www.synopsys.com<\/a><\/i><\/address>\n

Sign up for more<\/b><\/h2>\n

If\u00a0this was useful to you, why not make sure you’re getting our regular digests of \u00a0Tech Design Forum’s technical content? Register<\/a> and\u00a0receive our newsletter free.<\/p>\n","protected":false},"excerpt":{"rendered":"

High-performance vision-processing algorithms need optimized CNN engines to deliver the right performance within the power budget of embedded applications.<\/p>\n","protected":false},"author":329,"featured_media":9495,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1242,1385,35],"tags":[1924,1903,1695,1047,1690],"coauthors":[996],"class_list":["post-9854","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-architecture-design","category-choose-buy-ip","category-design-verification","tag-cnn","tag-deep-learning","tag-embedded-vision","tag-emulation","tag-power-analysis","workflow-expert-blog","workflow-up-to-date","organization-synopsys"],"_links":{"self":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/posts\/9854","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/users\/329"}],"replies":[{"embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/comments?post=9854"}],"version-history":[{"count":0,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/posts\/9854\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/media\/9495"}],"wp:attachment":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/media?parent=9854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/categories?post=9854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/tags?post=9854"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/coauthors?post=9854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}