Problem: You see this message:
hadoop fs -mkdir -p /user/sparkJobSubmitter/tmp ... Cannot create directory ... Name node is in safe mode.
Reason: Hadoop nodes do not have enough free disk space.
Solution:
- Free more disk space.
- Leave safe mode, using this command:
hdfs dfsadmin -safemode leave
- Ensure that Hadoop cleans up cache and log files before running out of disk space:
yarn.nodemanager.localizer.cache.cleanup.interval-ms yarn.nodemanager.localizer.cache.target-size-mb