{"id":8246,"date":"2015-09-23T01:44:42","date_gmt":"2015-09-23T01:44:42","guid":{"rendered":"https:\/\/www.techdesignforums.com\/practice\/?p=8246"},"modified":"2015-09-24T09:11:44","modified_gmt":"2015-09-24T09:11:44","slug":"mounting-fiji-how-amd-realized-the-first-volume-interposer","status":"publish","type":"post","link":"https:\/\/www.techdesignforums.com\/practice\/technique\/mounting-fiji-how-amd-realized-the-first-volume-interposer\/","title":{"rendered":"Mounting Fiji: How AMD realized the first volume interposer"},"content":{"rendered":"

In\u00a0June, AMD launched its Radeon R9 graphics cards<\/a>. It is the company\u2019s first family to feature die-stacking and its proprietary High-Bandwidth Memory (HBM). AMD further claims these are the first \u2018volume\u2019 interposer products.<\/p>\n

There have been\u00a0two overarching commercial goals. The first and most obvious was to introduce\u00a0AMD\u2019s latest generation GPU \u2013 previously codenamed \u2018Fiji\u2019 \u2013 for the 4k and VR gaming markets. The second was (and is) to create and mature an interposer\/stacking technology platform that AMD can extend across all its products.<\/p>\n

The flagship of the range so far, the Fury X, stacks the GPU on top of four HBM die. The GPU has 1050Mhz clockspeed with 4096 SPs organized into 64 CUs. The memories, thanks to the TSV construction, have a fat 4096-bit memory bus clocked at 1Gbps (500MHz DDR). This makes for 512GB\/sec of memory bandwidth, fed by 4GB of HBM organized in stacks of 1GB each. It\u2019s a 60% increase in memory bandwidth achieved\u00a0without the kind of power penalty that has been dogging GPU configurations<\/p>\n

The gaming performance of the cards \u2013 so far, the Fury and Nano have also been launched \u2013 has been widely discussed on gaming and benchmarking sites. Less has been said about Fiji\u2019s overall development process and what lessons it might hold for others planning for\u00a03D-IC.<\/p>\n

During the SiP Global Summit earlier this month at Semicon Taiwan<\/a>, AMD Fellow Bryan Black, one of the project’s leads, shared some war stories \u2013 or more accurately pointed to one interconnected story that he sees as representing the greatest challenge presented by 3D-IC.<\/p>\n

Motivation for 3D-IC<\/strong><\/h3>\n

Dr Black referred to a graph that has appeared in similar variations within the GPU community over the last few years to remind his audience why AMD has moved to interposer-based boards. The shift has long been planned.<\/p>\n

\"Figure<\/a>

Figure 1. GDDR5 can’t keep up with GPU performance growth (Source: AMD)<\/p><\/div>\n

Figure 1\u00a0shows three lines. In purple, it depicts the GPU flavor of Moore\u2019s Law which envisages roughly a 40% performance increase every year. The orange line represents memory system power, and how it would be likely to increase based on a traditional graphics board architecture. The blue line shows what is left after meeting memory\u2019s demands for computation.<\/p>\n

\u201cYou can see that over time we\u2019re running into trouble,\u201d Dr Black explained. \u201cWe starting with an interesting distribution where the vast majority of power is allocated to compute. But over time you see the compute allocation going down very fast and the memory allocation going up very fast. Before you know it, you have a system where you can no longer improve performance.\u201d<\/p>\n

He added that having identified this problem \u201cabout seven or eight years ago\u201d, AMD decided to put memory integration with GPU at the forefront of its existing research effort into die-stacking and other aspects of 3D-IC. This was the technology that would head off that power crunch.<\/p>\n

The company\u00a0ultimately achieved that\u00a0by exploiting the key features of 3D-IC. “We took the technologies that you’ve heard about so many times – the microbump interface, the very narrow pitches – and we went to an extremely wide DRAM interface [4,096 bit], slowed down the interface to 500MHz , and then we could simplify everything,” explained Dr Black.<\/p>\n

“By doing that, we went from that kind of non-linear power growth to a very linear power regime. Very simple structures consume very little power, and what they do consume they take extremely efficiency. So, we were able to get 60% higher bandwidth and 60% lower power than our product from the year before.<\/p>\n

“In terms of gigabyte of bandwidth per watt, it’s a 4X improvement over the previous generation of GDDR5. That’s massive. And it’s not just the DRAM. The interfaces on the GPU also consume much less power.”<\/p>\n

\"Figure<\/a>

Figure 2. GDDR5 compared against interposer-based HBM\u00a0(Source: AMD)<\/p><\/div>\n

So, job done. But, Dr Black’s main argument was that it took\u00a0a lot of work, investment and risk to get there.<\/p>\n

Interposer prototyping for feasibility<\/strong><\/h3>\n

\u201cWe predicted that the problem would hit in the 2015-16 timeframe and that we had to do something new,\u201d said Dr Black.<\/p>\n

That something new emerged after a number of 3D-IC prototyping and delivery stages. These were divided into:<\/p>\n

    \n
  1. 2007-11: Concept, first steps.<\/li>\n
  2. 2011-2014: Primary learning to feasibility and supply chain definition<\/li>\n
  3. 2014-2015: Final product development, bring-up and delivery<\/li>\n<\/ol>\n

    AMD undertook numerous prototypes within each of these stages. Dr Black estimates \u00a0around 15 in total.<\/p>\n

    For example, during the initial\u00a0phase, AMD concentrated on\u00a0a vehicle that placed GDDR3 memory next to a Griffin (Turion X2 Ultra) generation CPU in a mechanical stack. R&D at this point was conducted internally and chiefly with Germany\u2019s Fraunhofer Institute.<\/p>\n

    \u201c[This early work] taught us the real challenges and not just the technical ones like warpage, but the real challenges in how you acquire the silicon, glue it all together and then test it,\u201d said Dr Black.<\/p>\n

    As this work progressed, AMD added more partners. Still in the first phase \u2013 around 2009 \u2013 the company began working with SK Hynix on what would become HBM: the existing ATi RV635 GPU was combined with a custom DDR3 die-stack with TSVs.<\/p>\n

    Over time, OSAT, foundry, packaging and more partners were added, features were shrunk, further reliability and test vehicles were added. But it was not until 2012 that the team had a prototype structure that “convinced us we could build this thing and should spend serious money”.<\/p>\n

    More refinement and productization followed as work entered the critical third phase. “Toward mid-2014, we realized that the GPU was going to be quite large, so we started moving to larger devices. Specifically all the functional devices needed to be hooked up to a 500 sq mm ASIC in the prototype,” Dr Black explained.<\/p>\n

    “We didn’t have a 500 sq mm ASIC at the time, but we did have one where if you chopped out four of them at a time, they would match a 500 sq mm ASIC. So that’s what we did, then we powered all four die – some of them fully functional – and moved forward. The functional aspect was important there for our electrical test requirements.”<\/p>\n

    This was followed by the ‘Fiji Replica’, a 600 sq mm ASIC sitting on a 1,000 sq mm, with essentially the same dimensions as the final product.<\/p>\n

    “We realized this was quite expensive, so we went through a tremendous cost-down effort, eliminating complexity, anything we could get rid of,” said Dr Black. “I think we were able to reduce about 50% of the cost. That gave us a shippable solution.”<\/p>\n

    As its efforts neared fruition, AMD concluded that the main challenges in the development process had proved to be significantly different from those it had expected.<\/p>\n

    \u201cWhen we started the program we thought it was all going to be about developing TSVs, developing microbumps and figuring out how to attach die,\u201d Dr Black told the SiP Forum. \u201cWhat we quickly learned is that it was actually about developing a supply chain, creating new technology partners and changing the roles of the companies we worked with.<\/p>\n

    \u201cOur foundries started doing different things. Our OSATs started doing different things. Our DRAM vendors were taking on much different activities. Even our approach to how we\u2019d test a part. All these things changed.\u201d<\/p>\n

    The combination of skills required to realize an advanced interposer product introduced an inevitable domino effect: change one thing and many more were altered. Both in the prototypes and in the finished product, AMD concluded\u00a0that with a complex and shifting supply chain in place, \u201cIf we came up against a business or technical challenge, we went back and changed the design to get rid of it.\u201d<\/p>\n

    Record breaking bring-up<\/strong><\/h3>\n

    \u201cI don\u2019t think any one challenge was hard. It was their interaction,\u201d reiterated Dr Black.<\/p>\n

    Yet when AMD looked to apply its interposer learning in anger, things went remarkably smoothly.<\/p>\n

    \u201cThe program\u2019s key accomplishments are that 12 hours from having our first sample in hand, we were booting Windows and we were running games,\u201d said Dr Black.<\/p>\n

    \u201cWithin five days we had completed our entire bring-up of the product and could shut those activities down. Usually they had taken three months.\u00a0And we were shipping within six months of the first ASIC wafer.<\/p>\n

    “This was the fast bring-up and execution of a program ever at AMD. Development of the first R9 cards began in mid-2014 and they were going out the door by this June.”<\/p>\n

    But it was only possible because of the eight years of work before hand. And thereby hangs a tale.<\/p>\n

    AMD is unquestionably a Tier One player, the kind of company expected to take a lead in the implementation of new technologies. But it is still significant that as it moved through a lengthy 3D-IC development process, it was essentially making it up (or figuring it out, if you prefer) as it went along.<\/p>\n

    Conclusion<\/h3>\n

    A recurring theme in the SiP world is the current lack of standards. Given that the technology involved remains difficult – though not, as AMD’s work shows, insurmountable – users essentially have to develop their own flows and infrastructure.<\/p>\n

    It is unlikely today that a company would need to spend seven or eight years doing that, but there is still a lot to mix and match. As the Radeon R9 shows, change one thing and the results cascade through the project.<\/p>\n

    The upside, of course, is that AMD’s ambitions have drawn suppliers along with it. UMC’s TSV technology can now be considered somewhat battle-hardened. SK Hynix has further demonstrated its capabilities in stacked memory.<\/p>\n

    There is know-how in place. Nevertheless, the main message from AMD’s experience stands: it is still likely to take some effort and skill to knit all that together.<\/p>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    AMD’s Radeon R9 family is the result of eight years developing 3D-IC and interposer technology. What lessons did the company learn?<\/p>\n","protected":false},"author":138,"featured_media":8252,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30,43,38],"tags":[1070,1166,1436,1257,1783,1782,1073,949],"coauthors":[1441],"class_list":["post-8246","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dfm-dfy","category-test-design","category-ic-implementation","tag-3dic","tag-advanced-packaging","tag-dram","tag-gpu","tag-high-bandwidth-memory","tag-osat","tag-silicon-interposer","tag-tsv","workflow-featured","workflow-technique","workflow-up-to-date","organization-amd","organization-amkor","organization-ase","organization-sk-hynix","organization-umc"],"_links":{"self":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/posts\/8246","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/users\/138"}],"replies":[{"embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/comments?post=8246"}],"version-history":[{"count":0,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/posts\/8246\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/media\/8252"}],"wp:attachment":[{"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/media?parent=8246"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/categories?post=8246"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/tags?post=8246"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.techdesignforums.com\/practice\/wp-json\/wp\/v2\/coauthors?post=8246"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}