Eliminating Performance Bottlenecks on Web-Based AI
The end-to-end cloud ecosystem must mature rapidly to support enterprise deployment of AI and machine learning applications.
For AI developers, TensorFlow.js’ support for Node.js lets them build ML, DL, natural language processing, and other models on a “write once run anywhere” basis. They can author these models in TensorFlow.js script that runs in both the front-end browser and any back-end website that implements Node.js. In addition to executing AI inside any standard front-end browser, TensorFlow.js now includes Node.js bindings for Mac, Linux, and Windows back-ends. Training and inferencing of TensorFlow.js models may execute either entirely inside the browser, in the back-end website, or a combination of both, with their data distributed across these locations.
Tap into locally installed graphics processing units (GPUs) and other AI-optimized hardware on the platforms where they’re deployed — browser and/or web server — in order to speed model execution
Include high-performance built-in and pretrained models to speed development of regression, classification, image recognition, and other AI-powered tasks in a web client or back-end, thereby reducing the need for resource-intensive training during the development process
Allow developers to convert an existing model that was built elsewhere so that it can be imported into the web browser or Node.js back-end
Export models from the browser so that they may be processed in higher-performance platforms
Dynamically model from external databases, file systems, and other back-ends via HTTP requests and so that they needn’t consume inordinate browser or back-end platform storage resources
Send models to external HTTP servers through multipart form data requests.
Large TensorFlow.js models typically train up to 10-15x slower in the browser than standard TensorFlow models written in Python on non-web platforms, though some small TensorFlow.js models may train faster in browsers than equivalent TensorFlow Python models on other platforms. Some TensorFlow.js developers are resorting to external training environments — such as IBM Watson — to get around the library’s performance constraints.
Though client-side training can accelerate some AI development scenarios, it may not greatly reduce the elapsed training time on many of the most demanding scenarios. Accelerating a particular AI DevOps workflow may require centralization, decentralization, or some hybrid approach to preparation, modeling, training, and so on. For example, most client-side training depends on the availability of pretrained — and centrally produced — models as the foundation of in-the-field adaptive tweaks.
Even as they emerge, edge-AI benchmarking suites may not be able to keep pace with the growing assortment of neural-net workloads being deployed to browsers in every type of front-end (browser, mobile, IoT, embedded and back-end Web application. In addition, rapid innovation in browser-based AI inferencing algorithms, such as real-time human-pose estimation, may make it difficult to identify stable use cases around which such performance benchmarks might be defined.
The Web is AI’s new frontier. However, the end-to-end cloud application ecosystem must mature rapidly to support enterprise deployment of browser-based ML applications in production environments.
Jim is Wikibon's Lead Analyst for Data Science, Deep Learning, and Application Development. Previously, Jim was IBM's data science evangelist. He managed IBM's thought leadership, social and influencer marketing programs targeted at developers of big data analytics, machine ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
2018 State of the CloudCloud adoption is growing, but how are organizations taking advantage of it? Interop ITX and InformationWeek surveyed technology decision-makers to find out, read this report to discover what they had to say!