The data and artificial intelligence landscape is undergoing a transformative shift with the integration of no-code technologies to enhance efficiency and innovation.

One of the companies leading this charge is Databricks Inc., whose recent developments promise to streamline data management and processing through open-source initiatives and advanced automation.

DbInsight’s Tony Baer talks to theCUBE about no-code technologies.

“We’ve seen the waves of innovation,” said John Furrier, executive analyst at the CUBE Research. “Another era you’re seeing a lot of potential big moves coming, and Databricks is laying down a lot of strong game here by going for the long game by saying, ‘Hey, we’re going to open-source things.’”

Furrier spoke with Sanjeev Mohan (pictured, left), principal at SanjMo, and Tony Baer (right), principal at dbInsight LLC, at the Data + AI Summit, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the recent advancements in data and AI integration by Databricks, with a focus on open-source initiatives, automation and the competitive landscape between the Databricks and Snowflake Inc.

Open source vs. proprietary control

Databricks is pushing the envelope with its commitment to open-source formats, a move poised to push industry boundaries. However, this approach comes with challenges, particularly the perception of Databricks maintaining too much control over these projects, according to Baer.

“I think there’s a lot of interest on the open-source side,” he said. “I think the challenge for Databricks is that their history of open source, aside from Apache Spark, has been projects that Databricks has been largely at the helm of. I think the open source, especially the open sourcing of Unity, is an excellent development. On the other hand, I think there’s some perceptions here that Databricks has to overcome the perception that they’re actually controlling this.”

The open-source strategy of Databricks, especially with tools such as Lakehouse Federation, allows for the connection of various data sources such as SAP SE and Salesforce Inc. This integration facilitates data sharing and ETL processes without requiring extensive coding, according to Mohan.

“Picture what Databricks is doing with Lakehouse Federation or the new ETLs that they’re doing with LakeFlow, where you can connect to not just Snowflake, but SAP Salesforce,” he said. “You can do ETL, you can do sharing of data, you can do clean rooms. I really feel that both Snowflake and Databricks are really innovating and they’re helping their customers.”

The future of integration, automation and no-code technologies

Integration and automation are at the forefront of recent innovations in the data and AI landscape. Databricks has introduced new initiatives to unify data and AI stacks, moving beyond traditional table formats to enable complex data interactions seamlessly, Mohan pointed out.

“You do need formats, but not for everything,” he said. “You need a table format if you’re going to have structured data and you want any compute engine on top. But when you do AI, then the table formats won’t come in the picture. When you do data sharing, you need data sharing as a standard but not the table format. There’s a lot of standardization going on of access points.”

The competition between Databricks and Snowflake is expected to drive further innovation, with both companies catering to different user bases but working toward similar goals. The evolving ecosystem around these platforms will also encourage third-party innovations, contributing to the overall growth and advancement of the industry, according to Baer.

“We have Snowflake and we have Databricks, and they aren’t going to continue to coexist,” he said. “We have to figure out how do we rationalize this. I think that’s still a very dominant thing. The fact is within any organization, you’re going to be having these dueling platforms.”

As Databricks continues to innovate with no-code technologies, the company is setting new standards for efficiency and accessibility in data management. This focus on user-friendly solutions is crucial for empowering businesses to leverage advanced data capabilities without the need for extensive technical expertise.

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the Data + AI Summit:

Photo: SiliconANGLE