Apache Spark - Allow Sharing a cluster across multiple users
Currently, an entire new cluster is spun for every user who starts an Apache Spark session using Notebooks. Please add the ability to share a single physical cluster across multiple users. Spinning up a new cluster (with a minimum of 3 nodes) for every user is very expensive.
Santosh Veluguri commented
This is really important, otherwise there's no way organisations will migrate to Synapse Analytics Workspace. Can we get an ETA on a better way to handle this? Much appreciated!
If you could interact with Spark via regular, Open Source means, you would be able to do this. Come on MS, SQL Server would have died if the only client you could use was Management Studio.
Euan Garden commented
Thanks for the feedback.