How HLS is giving shape to glasses-free 3DTV
High level synthesis (HLS) was adopted to realize innovative display IP as developed by a small core engineering team.
High level synthesis (HLS) has become a popular choice for designs with a heavy bias toward video processing. One of the latest companies to follow the likes of Google and Nvidia down this path is SeeCubic, the R&D subsidiary of Stream TV Networks.
SeeCubic makes an interesting case study, and not just because it has developed an innovative glasses-free 3D display technology, Ultra-D. The company runs lean; certainly it does not today boast the same level of resources as some of HLS’ better-known users.
The initial Ultra-D’s implementation was achieved by a team comprising four core engineers, and three specialists who were brought aboard later on. There was consequently little scope to adopt HLS in a leisurely way
The business challenges facing SeeCubic’s were also highly relevant to its adoption of HLS, and again arguably more influential than they might have been for larger users. The salesforce needed an FPGA-based IP demonstrator to convince potential clients to incorporate Ultra-D as a block within their own SoCs. So, the demonstrator needed to show the technology working and provide metrics for potential clients (e.g. memory bandwidth requirements and silicon area costs).
Here the fact that, as engineer Eric Leenman explains, “HLS decouples the design from the technology” was very important.
That decoupling allowed SeeCubic to develop a C++ ‘golden reference’ template for its IP that could be fed into an HLS tool (here, Mentor’s Catapult) to generate RTL aimed at different process technology libraries. These could range from the Altera FPGA used for the demonstrator to the various processes at major foundries that could ultimately be used for customers’ own silicon.
Ultra-D
Ultra-D is an innovative format for glasses-free 3D viewing and rendering. Existing 3D digital content can be converted to the format on-the-fly. Meanwhile, 2D content can undergo stereo conversion to 3D Ultra-D for later viewing.
SeeCubic describes the its processing algorithm like this: “The light of individual sub-pixels is directed from the screen and projected into space; this generates virtual pixels. Virtual pixels merge in the space and form a complete view for each eye of the viewer. From that point the viewer’s brain takes over processing both views in the same natural way, as it is used to in the real world, creating a natural 3D experience.”
The converted material can be viewed on a standard display once an extra stack of layers of refractive and diffractive optical elements has been bonded to it.
Figure 1. Ultra-D overview (SeeCubic)
The technology can be used across a range of panels from just a few inches to several feet. As noted earlier, SeeCubic’s commercial objective is to market Ultra-D as an IP block for integration on multifunctional display SoCs.
Choosing HLS
As 2016 drew to a close, the Ultra-D technology was generating increasing interest. Companies as varied as Orange and Toyota had hosted demonstrations of the company’s initial hardware prototype. The time had come to progress to an IP demonstrator.
HLS rapidly emerged as an attractive design option because it would allow the team to address multiple challenges. As noted earlier, there were numerous commercial considerations. To those they now also needed to add, an ability to perhaps tweak the specification according to a potentially unforeseen customer request. Meanwhile from a technological point of view, the small design team wanted as much of its effort to be ready-for-primetime as possible.
Eric Leenman particularly cites a presentation at DAC 2016 by Nvidia on the use of HLS in a video-heavy implementation “We knew the guy who was giving the presentation,” he says, “and that he knew what he was talking about.”
The main deliverables SeeCubic aimed to benchmark with HLS included:
- Faster design turnaround time.
- Reduced code output and thereby IP footprint.
- A portable model for use on different processes.
To these ends, the company took an evaluation license for the Catapult HLS tool from Mentor, A Siemens Business, in January 2017. By February, the company had familiarized itself sufficiently with Catapult that it felt ready to move forward with a trial design covering one block within its entire IP package.
At that point, SeeCubic felt able to compare a ‘traditional’ design flow (in which an algorithm design team would pass from a ‘golden’ C model for implementation in a traditional HDL (e.g., VHDL/Verilog/SystemVerilog)) with a new HLS-driven one (where algorithm development and hardware design took place into the same simultaneous environment).
Figure 2. High level HLS flow with Mentor Catapult (SeeCubic/Mentor)
The comparison suggested that Catapult would deliver the Ultra-D block as clean RTL within two weeks, against an expectation of four-to-five weeks for the traditional flow. However, it was apparent that getting the best results would require some old tricks and some new thinking.
In the latter case, Leenman says those members of the team working on software and algorithm development needed to understand that they were no longer working simply toward a C-compiler but as part of a combined hardware/software team designing in C. This important distinction was inherent in the development of the entire C part of the project as a Khan Process Network.
It was clear some manual design work would be required. C is untimed, so the engineers needed to provide the timing constraints, determine what types of buffer to use, how state machines would be generated and so on. Going further, the nuances of PLLs and CDCs also required some knowledge of RTL to finesse the code that the HLS tool would automatically generate.
With the evaluation complete and a strategy for going forward determined, SeeCubic purchased a full Catapult license. It extended the use of HLS from the initial trial block to the entire project. The demonstrator was delivered based on predominantly C-to-RTL generation in May.
Working with HLS
The results SeeCubic achieved using Catapult continue to impress the company. With its carefully developed HLS implementation strategy, Leenman estimates that the overall development project was realized in half the time that would have been taken using the traditional flow. This went a long way to justifying the significant financial investment made in adopting HLS.
Although manual work was required, Catapult generated 95% of the RTL that was required. By contrast, manual RTL coding was responsible for 80% of the functional bugs, with C++ coding accounting for just 20%. This showed a clear advantage for the C++ approach when it comes to bug rate.
While pleased with the result, Leenman did note that a number of lessons needed to be learned during the company’s first major HLS project.
Direct consultation with Mentor’s Catapult team was, Leenman says, essential. The vendor’s engineers visited with SeeCubic, got to closely understand the client’s objectives, and could therefore provide invaluable help as the user became familiar with the technology. Leenman believes that had SeeCubic been forced to adopt HLS within the traditional ‘ticket’ customer support system, the project would have failed.
SeeCubic found that some aspects of the design were harder to realize than others in HLS. The technique is, as noted earlier, extremely well suited to video projects. However, control blocks in a design are, Leenman says, trickier. However, these challenges can be overcome. HLS’ other benefits make this worth the effort, he adds.
Other factors – the need to recognize some HLS limitations, the need to see C-coding as a hardware design effort from day one – have been discussed earlier.
One final issue Leenman raises is flexibility.
SeeCubic started out seeing HLS as the answer to a ‘Catch 22’. It allowed the company to balance various questions that a marketing team and potential customers might pose against the need to constrain a design: a demonstrator for all seasons. However, he now adds that working in C with HLS provides more flexibility to accommodate both ‘what if’ explorations of a specification as well as late-stage changes or tweaks that a client may seek.
Overall, Leenman and his colleagues now see HLS as the foundation for their future design work.
