Databricks, corporate provider of support and development for the Apache Spark in-memory big data project, has spiced up its cloud-based implementation of Apache Spark with two additions that top IT’s current hot list.
The new features — GPU acceleration and integration with numerous deep learning libraries — can in theory be implemented in any local Apache Spark installation. But Databricks says its versions are tuned to avoid the resource contentions that complicate the use of such features.
Apache Spark isn’t configured out of the box to provide GPU acceleration, and to set up a system to support it, users must cobble together several pieces. To that end, Databricks offers to handle all the heavy lifting.