Amazon EC2 – disk full /dev/nvme0n1p1 full how to fix

Problem: While running my Jenkins build, build failed with following error.

[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.1.1.RELEASE:repackage (repackage) on project <project name>: No space left on device -> [Help 1]

To investigate, when I run df -h on my Amazon EC2 server, this is the output:

[root@ip-172-*-*-3 builds]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 516K 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/nvme0n1p1 20G 20G 76K 100% /
tmpfs 389M 0 389M 0% /run/user/1000

For some reason my space is almost full and I really can’t find the big files or directory is easily.

To get the files/directory eating up my space I used the following command to find all the files greater than 10 MB:

sudo find / -type f -size +10M -exec ls -lh {} \;

This command return you list of files with complete path, which helps me to identify which ones are causing the trouble and deal with them accordingly.

After analyzing the result form the above command, I found that Jenkins keeps archiving all the previous builds which are eating up most of my memory.

Once I have the list of files I deleted all the large files using rm -r commands and removed all the files inside directory using rm -rf *.

Once I have deleted the large files, I ran the Jenkins build again and it work perfectly.

So, while using Jenkins I think we should also think about the clean up process for the archives, if not the entire one we can delete the significantly old archives and have enough space for builds to execute successfully.

Read more about Java here.

 

Leave a Comment