Applying agile development techniques borrowed from the world of software development to hardware design is increasingly important, according to speakers at a DAC panel earlier this month.
“We need to get design times down from 18 months to six,” said Randy Smith, vice president of marketing for onchip network IP company Sonics.
He argued that the increasing focus on consumer ICs means that more projects are being started without a full specification, an approach which breaks the traditional ‘waterfall’ development strategy of making an orderly progression from specification through prototype, design, verification and physical design. Smith said that the same applies for IoT devices: “Almost every one of those products has to start without a complete specification.”
Other drivers for a change of methodology include the need to develop low-power, always-on devices; to improve their security; and to reduce their cost. Smith also argued that the IC industry is currently in one of its regular cycles towards greater product customisation, demanding a greater volume of design work.
The desired attributes of a new methodology therefore include the ability to make progress without a full specification; to tackle modern design challenges such as power, security and time to market; to make it possible to ship products on time, or even early with a reduced feature set; and to improve the relationship between architectural, logical, physical and software design.
The agile manifesto
The agile development manifesto demands a shift of outlook from its users, who should change what they value. Development teams using agile methodologies should value:
- individuals and interactions over processes and tools
- working software over comprehensive documentation
- customer collaboration over contract negotiation
- the ability to respond to change over following a plan
“We are so used to specifying everything on paper before we ship a part, but it’s more important to ship the product and have it work,” said Smith. “To do this better we are going to need major EDA vendors to develop tools that support this methodology.”
Smith said Sonics is promoting agile IC development methodologies because its role in providing interconnect IP means it spends a lot of time working with system architects, who are responsible for many views of a new architecture. He said such architects don’t have a single language for overall hardware/software co-design and so end up working in Word and Excel files and having to make clumsy translations between the two formats.
“New tools, please”
Smith argued that IC developers need to adopt ‘requirements management’ tools, which help teams document, analyse, trace, prioritise and agree project requirements, and then track and communicate how these requirements change. This process continues throughout the development process, and helps teams move from working with a fixed specification to working with a well-communicated conversation about what a project should achieve. Such a tool should also link to bug tracking, enhancement requests and other relevant information.
In IC development, a requirements management tool could help IC designers refine requirements into architectural terms, perhaps using formats such as the next release of IP-XACT and UPF as a common language for defining the system architecture. With these facilities in place “you could almost do an ECO flow for the systems design flow,” said Smith. He called on Accellera, EDAC or the IEEE to help with enabling the development of an agile IC development methodology.
“It’s time to take off the shackles of what we have been doing before and work out what we can do better,” he added.
Learning agile techniques from software developers
Davis Frank is director of internal applications at Pivotal Software, which provides software and services for the development of custom applications for data and analytics based on cloud-computing technology.
Frank has been applying agile techniques in his development team for years, and says that he can’t really use the waterfall methodology, because it distances him from customers: “What you’ve done is hidden yourself from the market, the users.
“We tend to use waterfalls because we think we are designers, but it prevents us from facing the actual problems you meet when you get started. Long feedback loops hide reality from you and you end up operating in your own bubble.
“The universe tends to entropy, and it is out to get you and kill your project.”
Frank argued that, in this context, short feedback loops are better than long, frequent is better than eventually, and good is better than perfect.
Frank outlined the agile process that Pivotal uses for its development:
- Every project starts with an ‘inception’ meeting, in which everyone gets together in one room to review the goals of the engagement, talk about what the project is about, who the users are and how they use the app.
“Inception is really key for big release phases in projects,” said Frank. “The goal is to set the tone of having high-bandwidth communications across the team.”
- ‘Build the business first’. This means always writing user stories, and talking about how individual parts of the project add value to the user or the business.
“If we’re not building business value we’re doing science experiments,” said Frank.
- Use pair programming, to keep the quality high and enable teaching
- Rotate people a lot.
“We find rotating people among the various subsystems on big projects helps share knowledge and enables the circulation of ideas.”
- Test drive the code
- Do continuous integration
- Have shared ownership across the whole project
“This enables people to go on holiday, be sick, look after sick families and helps reduce the impact of ego.”
- Do short iterations, of maybe a week, planning small and planning often
- Always be shipping
“You need a good pace of releases to enable feedback.”
- At the end of every week run a retrospective, which improves team communication and progress
What about applying this to hardware? One problem with IC development is that its impossible to ship a new iteration of an SoC each week in order to get customer feedback. But Frank argues that aspects of agile methodologies have worked effectively in other disciplines with long time horizons, such as self-published novels. The key is to enable short feedback loops where possible.
Frank’s final advice was pragmatic: “Do what works: if it is not working for your team, stop doing it and find something else. In the end, though, your process is your product.”
A personal approach to agile IC development
Neil Johnson, principal consultant at XtremeEDA, discussed the way his company had been using agile techniques for hardware development over the past few years.
Johnson argued that taking on the agile manifesto in his work has been about “figuring out where you are, and picking things you can change to become more agile as a result.
“Quality is a big thing for me as a verification engineer, because if I have to spend days doing bug fixes I can’t be responsive.”
Johnson outlined his approach to applying agile techniques in his work:
- Have a weekly cycle during which you focus on a single feature, promising it on a Monday and delivering it on a Friday
- Do incremental development: write a test bench, write the RTL, and test it immediately: “This constrains the scope of the work and therefore enables focus, and you can give feedback early.”
- Practice test-driven development, in which you write a unit test for a new feature; check that the test fails, to avoid false positives; implement the feature; and then ensure it passes your test.
“These three practices cement the idea entirely,” Johnson said.
Agile development of FPGA accelerators at Microsoft
Andrew Putnam, senior research hardware design engineer in the Microsoft Research Technologies lab at Microsoft Research, discussed using agile techniques to develop FPGA based accelerators for Microsoft’s cloud services.
His group set out to accelerate the Bing search engine by building an experimental server cluster using FPGAs programmed to accelerate the Bing page-ranking service. The experiment was able to halve the number of servers necessary to provide the service while reducing its latency by 30%. Using a similar approach to accelerate the convolutional neural networks used for image classification delivered a tenfold performance improvement, while cutting power consumption ten times, and so delivering a 100fold increase in efficiency by using FPGAs.
The accelerators were built using Altera Stratix 5 FPGAs, with one FPGA per server board and 48 servers per half-rack. To offload the configuration and intercommunication load from the server backplane, Microsoft implemented a 6 x 8 toroidal network among the FPGAs in each half rack.
Figure 1 Microsoft Research's Catapult fabric for cloud service acceleration (source: MSR)
The FPGAs were programmed using agile development techniques, and Putnam first outlined how not to develop an accelerator: “Take a code branch from the software development team. Work alone for six months. Present the software team with the accelerator and hope.”
He added: “You have to work at the software team’s pace.”
Software development at Microsoft is already done using agile approaches, so his team moved to a ‘scrum’ development strategy for the hardware development, in which:
- The team works in one- or two-week sprints
- The tasks are defined by the end customer
- The requirements change as the work goes along
- There are short daily progress meetings
- The teams are small and physically co-located
- The team delivers something after every sprint
For this project, the teams included three hardware developers and two software developers, who jointly defined the software and hardware requirements. The team favoured development over documentation to help keep things flexible, and limited ownership of parts of the project so that everyone could work on anything, although Putnam says this has been harder for the hardware teams than the software teams.
Managers promised to turn around code reviews within a day.
Putnam highlighted the parts of this approach that he felt worked well, including:
- Continuous integration, with every code check-in going through a rigorous assessment process
- Use of a variant of pair programming, in which several people work on a task with one of them acting as a leader
- Use of test-driven development, in which developers really focus on what has to be accomplished by the module
- Long compilation and timing-closure runs for the FPGAs, of up to 24 hours, which shift the the dynamics of what you can do and make the planning harder
- Poor debug visibility
- The tendency for designs to ‘break’ suddenly, as they reach the limits of an FPGA’s resources
- Lack of binary compatibility among FPGAs, making it more difficult to move a design to a new architecture or device
- The fact that security and reliability can end up as afterthoughts: “That’s a problem because you can do so much more damage with hardware”
The Microsoft team adjusted their approach to hardware design to enable the use of agile strategies. They divided the design into a shell, a reusable portion of programmable logic that is common across applications, and roles, the application logic itself, which was restricted to a large, fixed region of the infrastructure to enable an agile approach.
Figure 2 Splitting the design into a common shell and application-specific roles helped the use of agile techniques (Source: MSR)
Putnam said that this shell/role split enabled the creation of a common API for hardware/software interaction across all applications, and helped define clear hardware/software partitioning boundaries. The system’s private network between FPGAs enables the applications to scale.
Putnam concluded that although this agile project had worked it remains to be seen whether the approach can be scaled to bigger projects and teams.
The diagrams are taken from a presentation given at the 41st Annual International Symposium on Computer Architecture in 2014, entitled “A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services”, by Andrew Putnam et al.