This error is caused by resource limits in Databricks. The workaround is to retry the query. If the error persists, restart the Spark cluster in Databricks to clear the resources being used. Additionally, the resource limit can be increased at the Databricks account level. Please contact Databricks Support for this.