Spark job management operations submit, monitor, and cancel Spark jobs.
Submitting a Spark Job
To submit a job when Spark is running in YARN, Aster uses the client spark-submit, either locally (without ssh) or remotely (with remote ssh).
Aster and Spark can run on the same or different clusters. Aster runs spark-submit locally if Aster and Spark run on the same cluster, and remotely (with remote ssh) if they run on different clusters.
Aster can run spark-submit locally when Aster and Spark are on different clusters (that is, when Spark is on a remote cluster).
Monitoring a Spark Job
To monitor a Spark job, Aster uses the REST API.
Canceling a Spark Job
To cancel a Spark job, Aster uses either the REST API or the YARN client.