Driving 4K smartphone and AR/VR device displays

Displays for high-end smartphones and augmented reality (AR)/virtual reality (VR) devices are becoming more sophisticated as the demand for high resolution content and visual quality increases. To meet such demands, high-end smartphones are moving from Wide Quad HD (WQHD) displays to ultra-high-resolution 4K displays. AR/VR applications require more pixels and high refresh rates as the display is closer to the eyes and must maintain visual quality. Higher refresh rates of 90frame/s, 120frame/s or more, minimize motion sickness and provide a vivid and immersive visual experience by reducing the discrepancy between what your eye sees and what your brain expects to see.

These unique features, along with the associated bandwidth requirements, pose a system-level challenge to designers. This article illustrates the different display subsystem architectures, and describes an interoperable Display Processing Unit (DPU) and MIPI Display Serial Interface (DSI) IP solution that enables 4K embedded displays for smartphones and AR/VR devices.

Anatomy of a display subsystem in an application processor

The multimedia pipeline in an application processor architecture encompasses multiple IP blocks such as a graphics processing unit (GPU), video processing unit (VPU), DPU, and MIPI DSI, shown in Figure 1. As the displayed content becomes more complex in terms of resolution and frame rate, the GPU/CPU must offload more functionality to meet increasing graphics performance within the ever-shrinking silicon area and thermal design power requirements.

The Arm Mali-D71 DPU is a fundamental processor and GPU companion in a multimedia subsystem that works seamlessly behind the Android Hardware Composer HAL. Mali-D71 offers a super-efficient, multi-layer hardware composition solution in terms of system power and performance. It also performs in-line orthogonal rotation, high quality upscaling and downscaling, ambient light adaptation, and high-quality HDR local and global tone-mapping and color/gamut management, when combined with Arm Assertive Display 5. It performs all of the above operations in a single pass through memory, translating to significant system bandwidth (power) savings. The latest version of Arm Frame Buffer Compression, version 1.2, is supported throughout the multimedia pipeline, adding significant performance and system bandwidth benefits. Arm Corelink MMU-600 is also an essential component of the Arm Display Solution. It is optimized to work with Mali-D71 on 4Kbyte paged memory to guarantee real-time performance for latency-sensitive applications, such as VR. Mali-D71 supports all major display industry standards for all kinds of panels (mobile, tablet, TV, notebooks, etc.), including MIPI DSI.

Mali-D71 connects to the display device through the display serial interface like the MIPI DSI. The interface must support all the integral requirements of a 4K display in terms of bandwidth and speed. According to MIPI Alliance, the DSI specification provides “brilliant color rendering for the most demanding imagery and video scenes and supports transmission of stereoscopic content.”

Designers can integrate the MIPI DSI specification into their application processor or system-on-chip (SoC) with highly-optimized IP such as the Synopsys DesignWare MIPI DSI Controller.

Synopsys’ MIPI DSI controller is a fully verified and configurable IP that converts the incoming pixel data, which in this case is Arm’s DPU, into MIPI DSI packets which are transmitted to the MIPI D-PHY link connected to the embedded display. The Synopsys DSI IP supports dual DSI link use-cases by providing additional bandwidth for ultra-high-resolution mobile systems. It also supports the Video Electronics Standards Association (VESA) Display Stream Compression (DSC) standard to reduce the required data transmission bandwidth for 4K resolution.

From WQHD to 4K displays

System integrators must select a DPU and a DSI solution that delivers the bandwidth, speed and power required for 4K embedded displays, and allows for seamless interoperability with one another. Figure 2 shows an example of a WQHD display at 60Hz with 24bit/pixel needing 6.3Gbit/s to transfer data from the application processor to the display device.

In this example, one link of one display output unit connects to one MIPI DSI host controller, which then connects to the display device using a 4-lane MIPI D-PHY. The DSI host controller integrates a VESA DSC encoder with the capability to perform visually lossless compression by a factor of 2x or 3x, reducing the required bandwidth to 3.16Gbit/s. The compressed stream converts into MIPI DSI packets passed down to the D-PHY link. Using four-lane operation at 1Gbit/s per lane results in a total available bandwidth of 4Gbit/s, enough to transmit the compressed stream. This entire process reverses on the display device side. The DSI packets are recovered by the receiver controller and a VESA DSC decoder decodes the data before the image is displayed on the panel.

In contrast, shifting to a 4K display requires a different architecture, shown in Figure 3. A 4K display resolution at 90Hz with 30bit/pixel requires 26.6Gbit/s bandwidth with two display drivers and associated MIPI display interfaces. The display drivers synchronize to enable the two halves of the image to display properly.

In this example, two links of one display output unit connect to two distinct MIPI DSI host controllers, which then connect to the display device using two by four-lane MIPI D-PHYs. The DSC, with a compression factor of 3x, reduces the required bandwidth for each DSI link to 4.4Gbit/s. The compressed stream converts into MIPI DSI packets that are passed down to the D-PHY link. Using two sets of four lanes operating at 1.5Gbit/s per lane for a total available bandwidth of 6Gbit/s per DSI link is enough to transfer the compressed stream to each display port.

Both examples show the advantages of using a DSI solution with an integrated VESA DSC encoder. The encoder reduces: the amount of data transmission sent to the display device, the D-PHY operation frequency, the number of D-PHY lanes, and the display frame-buffer size. This combination results in significant power, area, and cost savings.

A fully interoperable embedded display solution

MIPI-enabled displays support video mode, command mode or both. In video mode, the host must constantly refresh the display. Synchronizing the information and image data transmit over the MIPI bus as it uses DSI packets happens in real-time. Because the host must refresh the display, the display does not need a frame buffer. In command mode, the MIPI host transmits pixel data stream to the display using display command set (DCS), which is transmitted over the MIPI bus using DSI packets. The display has a full frame buffer to store the entire pixel data.

Once the data is in the display’s frame buffer, a local timing controller on the display retrieves data from the frame buffer and shows it on the display. The MIPI host does not need to refresh the display constantly.

Arm and Synopsys have collaborated to ensure Arm Mali DPU and Synopsys DesignWare DSI Controller IP, with DSC encoder, are fully interoperable for both video and command mode displays. Besides the fundamental clock and reset control, the companies provide the detailed interface connectivity, including all the IP-specific slide band signals for each display mode in both single- and dual-link mode of operation. Dual-link operations double the available bandwidth, which is particularly important for ultra-high-resolution applications. Table 1 shows the numerous resolutions that the interoperable embedded display solution supports.

The combined display solution also supports several advanced features such as variable refresh rate, dynamic resolution change, and partial update.

Variable refresh rate enables changes of total frame time during frame transfers and therefore effective frame rate. This dynamic refresh rate change is especially useful when displaying static content (i.e. static screen scenarios) and playing back video of fixed frame rate (i.e. 24frame/s or 48frame/s.)

Dynamic resolution change is a solution that enables the transmission of multi-resolution video streams between the Mali-D71 DPU and Synopsys DesignWare MIPI DSI Host Controller IP without the need for reprogramming or reconfiguring the IP components.

If the display peripheral supports partial update, then there is the option to program the Mali-D71 DPU and Synopsys DesignWare MIPI DSI Host Controller IP to update only a selected part of a frame buffer from the display peripheral. This feature is especially useful when only part of a screen is updated from frame to frame and the rest of the screen is static during most of the display time, which is during most frames. Figure 4 shows video playback displayed together with the status bar, navigation bar, and background. Only the video content changes between the frames, while other display layers remain static. With partial update support, only the video region of the display frame transmits to the display resulting in overall power savings.

Synopsys and Arm ensure complete and optimized display solution

For smartphones and AR/VR devices, there is a shift in embedded display resolution from WQHD to 4K. However, each application requires its own unique set of features when it comes to high bandwidth and refresh rates. Synopsys and Arm have collaborated to provide a complete embedded display solution with a fully interoperable DPU and MIPI DSl interface that enables designers to minimize integration risk. The solution supports command and video modes, and incorporates several advanced features such as variable refresh rate, dynamic resolution change, and partial update.

Further information

Synopsys DesignWare MIPI DSI Host Controllers

Arm Mali-D71 DPU

Author

Licinio Sousa is a technical marketing manager at Synopsys. Vassilis Androutsopoulos is a senior product manager at Arm.