An open-source framework for greater flexibility in machine-learning development

Allen Watson is a senior product marketing manager for Synopsys’ ARC development tools and ecosystem, and brings to the role many years of experience in hardware and software product development. Prior to Synopsys, Watson held various marketing and business development positions at MIPS Technologies and Mentor Graphics. Allen holds a BSc in electronic and electrical engineering from the University of Surrey.

Data scientists and engineers working in machine learning today have a wide variety of tools available to them. Many of them are open source, free and available from technology giants such as Facebook, Google and Microsoft. A key choice in any machine-learning project is which deep-learning framework to use. These frameworks are used to develop, train and deploy neural-network models and include Caffe, Caffe2, Microsoft Cognitive Toolkit, MXNet, PyTorch and Tensorflow.

Each framework offers different features and approaches to developing neural networks and each user may prefer to work with a particular framework. However, it is important when choosing a framework to consider how the resultant models can be deployed. Most frameworks make it easy to support deployment to the cloud or to a farm of workstations. However, many of the neural networks being developed today are targeted at the network edge, for use on constrained devices that may not support a chosen framework. This is where a recent initiative called the Open Neural Network Exchange (ONNX) comes in.

ONNX was started by Facebook and Microsoft and has since gathered widespread support from other companies including Amazon, Intel, Qualcomm and others. It is an open interchange format for describing neural-network models. This means that a data scientist can develop and train a model in his or her favorite framework and then export it to the ONNX format (Figure 1). If the tools for the deployment target also support ONNX as an input format then that model can be deployed to the target. Thus, the choice of framework in which to develop and train a model becomes much less constrained by the final deployment target. For example, Synopsys recently announced support for ONNX in the MetaWare EV Development Toolkit for use with its EV6x Processor IP, which includes a CNN inference engine. This support gives users targeting that CNN engine much more freedom to choose which framework to start with on their project.

There may also be stages in a machine-learning project where one framework has more useful tools than another, e.g. when visualizing or profiling neural-network models. Users could switch to that framework by exporting the model in the ONNX format for that phase of the project and then switch back again as necessary.

ONNX is not the only emerging standard in this area. The Khronos Group has the Neural Network Exchange Format (NNEF). Many of the same companies supporting ONNX also participate in the NNEF standard. ONNX certainly seems to have the momentum at the moment, but both formats offer more flexibility for data scientists and engineers, and that has to be a good thing!

Allen is a senior product marketing manager for Synopsys’ ARC development tools and ecosystem, and brings to the role many years of experience in hardware and software product development. Prior to Synopsys, Allen held various marketing and business development positions at MIPS Technologies and Mentor Graphics. Allen holds a BSc in electronic and electrical engineering from the University of Surrey.