The software food chain
Software has become a ubiquitous part of many products and services, some of which are distributed and used on a massive scale. Think of cloud computing, online banking and the engines for today’s cellphones and cameras. Competition requires that such new products are developed more quickly than ever before and yet brought to market at lower and lower cost.
Project managers therefore demand the design efficiency inherent in the reuse of existing, proven software code, be it internally generated or acquired from third parties. Most companies have lots of their own code files. Many, many more can now easily be sourced online and downloaded from open source libraries or acquired from external developers at a fraction of the time and cost it would take to do the work from scratch in-house.
Open source code is also attractive because it tends to grow exponentially in many respects—e.g., functionality, richness, stability, quality and security—while being subjected to rigorous peer review. Yet it is also thought to involve no monetary cost. However, such software is still governed by a wide range of complex license regulations and usage terms that require expert interpretation.
Keeping track of the use of such software is also difficult. Companies rarely create the entire software load in their products or services. Rather, most players in the ‘food chain’ (Figure 1) assemble software from a mixture of suppliers, contractors and open sources, then couple it with their own value-add code and pass it along to another stage or competence—at which point, the same process of creative software combination from various sources will take place all over again.
Today’s typical end product or service is the sum of code provided by many organizations and individuals at multiple stages of a supply chain. Software modules are treated as commodities, as has happened for longer with many hardware components. However, there is one major difference: software code comes laden with licensing and intellectual property (IP) terms that can seriously affect a product’s commercialization and/or certain corporate transactions.
Without a complete and up-to-date view of all the software components and contributors to its products or services, a company runs the risk of encountering unexpected costs, missed deadlines and significant business risks. In particularly egregious scenarios, software components may not meet specification and need to be repaired, or proper IP and copyright obligations may be overlooked, yet the problems they create only come to light after a product has been released. Imagine that a security fault is discovered in the protocol stack supplied by one of the players. The end-user now has to ascertain which products are affected (potentially thousands), and meet the cost of solving the problem and correcting it in each instance.
Software governance and development discipline
The risks outlined above can be greatly reduced, even eliminated if you have proper, accurate records for all the code in a product and know the pedigree of all its components. Alas, few companies have traditionally managed to capture such a ‘software bill of materials’ because those tools that could determine code’s content and manage IP correctly were mainly retrospective. They also either involved cumbersome manual audits or were expensive tools for automatic code analysis.
These problems can now be avoided by implementing real-time records-gathering and preventive IP management processes based on appropriate design flow policies. Moreover, recently released preventive tools allow for the real-time analysis, recording and compliance-checking of legacy code as well as any new code brought into a project.
While good software development practices have evolved to include systems for checking syntax, managing software versions and tracking software bugs, disciplines that are standard practice in structured hardware development flows have yet to be adopted. This latter group includes:
- Approved vendor lists that should contain approved components and licenses, including the commercial terms, vendor history, version details, historic pricing, and so on. A developer can then select components freely from the list without concern.
- Automatic notification in case attempts are made to use code with unapproved licenses, or code modules that are governed by incompatible licenses.
- A bill of materials that fully records which components feature in the final product and that includes necessary details to enable production, determine costs, set export terms, track vendor upgrades and manage other post-design activities.
- Such advanced practices must be applied to software development processes in order to ensure proper software governance. They will give organizations a more effective overview of their source code assets and streamline product commercialization. To establish the necessary culture of software development discipline and good IP management, product and development managers must therefore:
- understand the economics of software governance;
- work with legal counsel to create and administer appropriate policies for software IP management;
- acquire full knowledge of licensing and other IP characteristics of the existing code in the institution and clean it where necessary; and
- automate the record keeping and IP management process as an integral part of the software development process going forward.
The latest automoted tools can help managers take all these necessary steps and create an effective source code management flow (Figure 2).
The economics of software IP management
The economic justifications for software IP management include the cost incurred to fix bugs and delays in time-to-market caused by the amount of time that elapses between the detection of IP issues and their correction. The earlier any external IP obligations are detected and managed, the less expensive they are to address and fix.
Until recently, the industry had only limited options. The main preventive solutions are still often entirely manual, and basically involve training developers to avoid certain types of software code. This is time-consuming and expensive because the number and range of code suppliers and sources keep getting larger and more complex. And still, there is no certainty that the end software will be ‘clean’.
On the other hand, after-the-fact corrective measures waste resources on removing the offending code, identifying a replacement third-party code (or worse, rewriting code internally), and making changes to the application program interfaces (APIs) so that the new code fits the software. The time required for correction lengthens time-to-market and a project’s commercial window can even close before its release.
It is better to handle record gathering and software IP management during the original code development and quality assurance (QA) processes. Here, automatic tools now place minimal training demands on R&D staff with regards to the minutiae of IP issues. They are easy to adopt and structured so that they do not disturb normal creative development flows. Examples of these tools include the offerings from our company. As shown in Figure 3,they cover the time-efficiency spectrum, from On-Command to On-Schedule and Continuous Real-Time. These tools are:
The Enterprise IP Analyzer
It helps establish corporate records-keeping and corresponding IP policies, analyzes legacy code in a single project or across an entire organization, creates an associated pedigree database, and reports violations of declared policies.
The Build IP Analyzer
It identifies all software modules that actually feature in a product and checks them against established IP policies.
The Developer IP Assistant.
It automatically builds a record of the code’s pedigree as a project develops and provides real-time notifications to ensure disciplined records-keeping and that the software contains ‘clean’ IP.
Software IP management policies and compliance administration
Software managers who want to maintain disciplined development environments and deliver products with clean IP must collaborate toward these goals with legal counsel. Together, they can develop appropriate coding policies that ensure IP compliance and the presence of downstream safeguards, while enabling the constructive use and adoption of open source or third-party software. A sound, clear and enforceable IP policy will contain an effective list of approved software vendors, together with acceptable external-content license attributes and measures to be taken in various situations:
- What is allowed and what is restricted.
- What to do if a piece of code with a hitherto-unknown license is desired.
- What to do in case of unknown code.
- What are the potential distribution restrictions, including those on exports.
Such policies should be consistent with corporate goals, easy to enforce and allow compliance monitoring throughout a project’s lifetime from development to commercialization and on to post-sales support. An organization should also be able to adopt and enforce appropriate IP policies for each class of projects it handles.
Retrospective code mapping and IP issue corrections
To ascertain the IP cleanliness of the software that has already been developed or acquired, users need to map the existing (legacy) code and ensure that it is in line with the appropriate IP policy. This can be done by engaging expert teams to perform an IP audit or by using automatic tools—e.g., Enterprise IP Analyzer—to perform the code mapping and check all the components.
Retrospective tools—available from companies like Protecode, BlackDuck and Palamida—establish the IP pedigree of existing code by analyzing it and decomposing it into modules that can be matched against extensive databases of known software.
Real-time preventive IP software management
The final and most cost- and time-effective measure is for management to embed the records-keeping and software IP management process as an intrinsic element of the company’s development and QA processes. Tools that unobtrusively operate at each development workstation are preferrable where they do not perturb the normal process of software creation and assembly. Our Developer IP Assistant product is an example of this kind of approach.
Such tools automatically detect and log in real time each piece of code a developer brings into a project. Various techniques are used to determine the code’s pedigree (e.g., its ‘signature’ when compared against a database of known code modules, identifying which URL it came from, identifiers within the code itself, requesting the developer to provide a valid record of how the code was created and brought in). The resulting pedigree is captured in the overall code map and henceforth always associated with that piece of code. The Developer IP Assistant also checks the licensing obligations of the identified code against the IP Policy for that project and takes any necessary enforcement steps.
The integration of various analysis and policy enforcement tools (Figure 4) ensures proper record keeping and clean IP software development. Such automatic tools are already being used in academia and by industry. Their utilization here will accelerate in the face of enhanced demands for IP indemnification, open source enforcement of copyright infringement and reduced product development cost.