Truncating a large log file

This is actually one of my favorite questions to ask Jr. Systems Administrators. I believe the way they answer this question really helps me gauge where they are at in their Administration skills.

How do you clear a large log file that an application is actively writing to?

Some will answer honestly and say “by removing the file” and others will pretend like they careful of everything and say “move the file out of the way and create a new one”.

The correct answer is “truncate the file using the redirect symbol” or if you really want to pretend like you are super careful “make a copy of the file then truncate it using the redirect symbol”


# > /tmp/big.log

One thing most will neglect to tell me is why do they truncate the file rather than simply using the move command?

Well the answer is because of open file handles, if a running process has an open file handle on the log file in question when you perform the mv command the file handle will follow the file to the new location. If that happens all of the new data will be written to the new location rather than the correct location.


# fuser /tmp/big.log  
/tmp/big.log: 2246  
# mv /tmp/big.log /tmp/big.log2  
# fuser /tmp/big.log2 
/tmp/big.log2: 2246

This can mean misplaced log data which could be a very big headache for you later. Especially if you had monitoring software reading /tmp/big.log periodically.
If you use the redirect symbol to truncate the file than you will empty the contents of the file without disrupting any existing open file handles.


# fuser /tmp/big.log2  
/tmp/big.log2: 2246  
# > /tmp/big.log2 
# fuser /tmp/big.log2  
/tmp/big.log2: 2246
Benjamin Cane
Principal Engineer, Vice President

Benjamin Cane is Principal Engineer at American Express. He has more than 16 years of experience with roles in both systems and software engineering. He leverages both his systems and software skills to build end-to-end platforms. Platforms, purpose built for performance and resiliency. Benjamin is also the author of Red Hat Enterprise Linux - Troubleshooting Guide (2015, Packt Publishing), and he has published many popular articles on topics such as Linux, Docker, Python, Go and Performance Tuning. Thoughts and Opinions expressed in my articles are my own.