When a Linux disk fills up: a calm checklist
A full disk takes services down in confusing ways. This is the order I check things in, including the deleted-but-open file trap.
Sample article to show the layout. Replace it with your own writing.
A full disk rarely announces itself. Services fail to write logs, databases refuse connections, and the error messages point everywhere except the disk.
Confirm it's really space
df -h
df -iThe second command matters. A filesystem can run out of inodes while showing free space.
Find what grew
sudo du -xh / --max-depth=1 2>/dev/null | sort -hThe -x flag keeps du on one filesystem, so mounted volumes don't confuse the totals.
The deleted-but-open trap
If df says the disk is full but du can't find the files, a process is probably still holding a deleted file open.
sudo lsof +L1Restarting that process releases the space. Truncating the file in place also works when a restart isn't possible.
Prevent the next one
Set up log rotation for anything that writes continuously, and alert at 80% usage rather than 95%.