![\"Figure](\"https:\/\/www.techdesignforums.com\/practicefiles\/2019\/03\/HardwareAndSoftwareMarketsfor-CV.png\")
<\/a>Figure 1. Hardware and software markets for computer vision (Tractica)<\/p><\/div>\n
Computer vision challenges<\/strong><\/h2>\nDesign for computer vision is tough. Here are some of the more important challenges.<\/p>\n
\n- Computer vision systems use computationally intensive convolutional neural networks (CNNs) for training and inference. These are testing the performance limits of standard platforms (e.g., GPU, CPU), and making the case for \u2018domain-specific\u2019 alternatives.<\/li>\n
- Because many computer vision applications are safety-critical or time-sensitive, they undertake more edge processing than other AI technologies and thus must be done within a tight power budget.<\/li>\n
- The market is still evolving rapidly. Alongside the computational load, algorithmic complexity is increasing all the time.<\/li>\n<\/ul>\n
According to Mike Fingeroff, HLS Technologist at Mentor, a Siemens Business, this type of environment requires a design flow that has three core qualities:<\/p>\n
\n- It accelerates schedules.<\/li>\n
- It allows for efficient hardware evaluation.<\/li>\n
- It enables late stage changes.<\/li>\n<\/ol>\n
HLS for computer vision<\/strong><\/h2>\nLet\u2019s consider those three requirements of a computer vision design infrastructure in turn.<\/p>\n
Schedules<\/strong><\/h3>\nHLS shortens design cycles by raising design abstraction above RTL, typically using SystemC, C or C++ to define the project before synthesis. This is especially useful for computer vision because of a useful alignment between the hardware and algorithmic design environments. Badru Agarwala, General Manager with Mentor, explains:<\/p>\n
\u201cAlgorithm developers prefer to write code in C++, do not want to learn register transfer languages \u2013 such as Verilog or VHDL \u2013 and they do not want to use the tools and methodologies required for the hardware implementation process,\u201d he writes. \u201cTo address this problem, some algorithm developers write their code in C++ and then use high-level synthesis tools.\u201d<\/p>\n
In this way, the traditional benefits of HLS are further amplified.<\/p>\n
Evaluation<\/strong><\/h3>\nThe rate of change in computer vision algorithms makes system definition tricky. CNNs are computationally intensive and as algorithms are refined, it can therefore be difficult to get the optimal alignment between hardware and software. Here, as Agarwala again explains, some of the inherent advantages of HLS in terms of \u2018what if\u2019 analysis are important.<\/p>\n
\u201cDevelopers\u2026 quickly make tradeoffs and then automatically generate RTL.<\/p>\n
\u201cThese developers can try out multiple implementations of new algorithms, explore the performance and power consumptions of implementing these algorithms in hardware or software, and examine the tradeoffs of running on ASICs, FPGAs, CPUs,or GPUs.<\/p>\n
\u201cTradeoffs are also explored by varying the number of processor cores and experimenting with memory sizes and configurations.\u201d<\/p>\n
Time saved through design abstraction and the speed of the HLS process gives hardware and software a lot of latitude here.<\/p>\n
Late changes<\/strong><\/h3>\nThe evolution of computer vision implies that sometimes projects will need to cope with changes very late in the day. They may need a final tweak to the algorithm, or an increase\/reallocation in resources to realize an unfamiliar application.<\/p>\n
\u201cTeams need a way to design and verify an algorithm while the specifications and requirements evolve without starting over every time there is a change. HLS flows allow this by using constraints and directivews that guide the process, while leaving the algorithm unchanged,\u201d writes Agarwala.<\/p>\n
\u201cFor example, at the last minute, a team decides to change the clock speed. That constraint is changed and the HLS tool regenerates the RTL according to the new clock speed, with no change to the C++ algorithm.<\/p>\n
\u201cThis type of late-stage change is not possible in a traditional RTL glow, yet with HLS the RTL is rebuilt at the push of a button.\u201d<\/p>\n
A tried-and-tested HLS flow captures the three qualities (Figure 2).<\/p>\n
![\"Figure](\"https:\/\/www.techdesignforums.com\/practicefiles\/2019\/03\/HLSFlow_Computer_Vision_Feature.png\")
<\/a>Figure 2. The HLS flow (Mentor)<\/p><\/div>\n
Given the complexity of all AI development \u2013 and particularly the challenges computer vision poses \u2013 HLS provides a platform that brings together the software\/algorithm and hardware teams so that they can collaborate directly on an essentially consistent platform.<\/p>\n
Part Two of this series<\/a> looks at how these project management and methodological goals can be realized in practice, and refers specifically to the development of computer vision IP for deep learning and inference object detection at Chips&Media.<\/p>\nFurther reading<\/strong><\/h2>\n
According to Mike Fingeroff, HLS Technologist at Mentor, a Siemens Business, this type of environment requires a design flow that has three core qualities:<\/p>\n
- \n
- It accelerates schedules.<\/li>\n
- It allows for efficient hardware evaluation.<\/li>\n
- It enables late stage changes.<\/li>\n<\/ol>\n
HLS for computer vision<\/strong><\/h2>\n
Let\u2019s consider those three requirements of a computer vision design infrastructure in turn.<\/p>\n
Schedules<\/strong><\/h3>\n
HLS shortens design cycles by raising design abstraction above RTL, typically using SystemC, C or C++ to define the project before synthesis. This is especially useful for computer vision because of a useful alignment between the hardware and algorithmic design environments. Badru Agarwala, General Manager with Mentor, explains:<\/p>\n
\u201cAlgorithm developers prefer to write code in C++, do not want to learn register transfer languages \u2013 such as Verilog or VHDL \u2013 and they do not want to use the tools and methodologies required for the hardware implementation process,\u201d he writes. \u201cTo address this problem, some algorithm developers write their code in C++ and then use high-level synthesis tools.\u201d<\/p>\n
In this way, the traditional benefits of HLS are further amplified.<\/p>\n
Evaluation<\/strong><\/h3>\n
The rate of change in computer vision algorithms makes system definition tricky. CNNs are computationally intensive and as algorithms are refined, it can therefore be difficult to get the optimal alignment between hardware and software. Here, as Agarwala again explains, some of the inherent advantages of HLS in terms of \u2018what if\u2019 analysis are important.<\/p>\n
\u201cDevelopers\u2026 quickly make tradeoffs and then automatically generate RTL.<\/p>\n
\u201cThese developers can try out multiple implementations of new algorithms, explore the performance and power consumptions of implementing these algorithms in hardware or software, and examine the tradeoffs of running on ASICs, FPGAs, CPUs,or GPUs.<\/p>\n
\u201cTradeoffs are also explored by varying the number of processor cores and experimenting with memory sizes and configurations.\u201d<\/p>\n
Time saved through design abstraction and the speed of the HLS process gives hardware and software a lot of latitude here.<\/p>\n
Late changes<\/strong><\/h3>\n
The evolution of computer vision implies that sometimes projects will need to cope with changes very late in the day. They may need a final tweak to the algorithm, or an increase\/reallocation in resources to realize an unfamiliar application.<\/p>\n
\u201cTeams need a way to design and verify an algorithm while the specifications and requirements evolve without starting over every time there is a change. HLS flows allow this by using constraints and directivews that guide the process, while leaving the algorithm unchanged,\u201d writes Agarwala.<\/p>\n
\u201cFor example, at the last minute, a team decides to change the clock speed. That constraint is changed and the HLS tool regenerates the RTL according to the new clock speed, with no change to the C++ algorithm.<\/p>\n
\u201cThis type of late-stage change is not possible in a traditional RTL glow, yet with HLS the RTL is rebuilt at the push of a button.\u201d<\/p>\n
A tried-and-tested HLS flow captures the three qualities (Figure 2).<\/p>\n
<\/a>Figure 2. The HLS flow (Mentor)<\/p><\/div>\n
Given the complexity of all AI development \u2013 and particularly the challenges computer vision poses \u2013 HLS provides a platform that brings together the software\/algorithm and hardware teams so that they can collaborate directly on an essentially consistent platform.<\/p>\n
Part Two of this series<\/a> looks at how these project management and methodological goals can be realized in practice, and refers specifically to the development of computer vision IP for deep learning and inference object detection at Chips&Media.<\/p>\n
Further reading<\/strong><\/h2>\n