“Intricacy”of logrotate (HIGH CPU)

So i noticed logrotate process is chewing a lot of CPU on a brand new Hardware (Oracle Database Appliance)

tail /var/lib/logrotate.status

"/u01/app/oracle/diag/crs/zsc-oda0-0/crs/trace/oifcfg_45354.trm" 2018-11-29
"/u01/app/12.2.0.1/oracle/rdbms/audit/+ASM1_ora_89696_20190201115423042224143795.aud-20190209.gz-20190215.gz-20190217.gz" 2019-2-20
"/u01/app/12.2.0.1/oracle/rdbms/audit/+ASM1_ora_65203_20181209123013319320143795.aud-20190209.gz-20190215.gz-20190217.gz-20190220.gz-20190226.gz" 2019-3-5
"/u01/app/12.2.0.1/oracle/rdbms/audit/+ASM1_ora_77256_20190122165831610096143795.aud-20190209.gz-20190215.gz-20190217.gz-20190220.gz" 2019-2-26
"/u01/app/12.2.0.1/oracle/rdbms/audit/+ASM1_ora_66149_20180615010539814998143795.aud-20190209.gz-20190215.gz-20190217.gz-20190220.gz-20190226.gz-20190305.gz" 2019-3-13

cp/rm/backup the /var/lib/logrotate.status file

logrotate -vf /etc/logrotate.conf   (Manually run logrotate)

Main culprit (For my case), “/u01/app/12.2.0.1/oracle/rdbms/audit”. As you can see it is rotating all the files including the files that are already rotated.

Hints: “Date stamp.gz” kept getting appended to already rotated file.

[oracle@zsc-oda0-0 audit]$ ls -ltr /u01/app/12.2.0.1/oracle/rdbms/audit | wc -l

2220964

Over 2 million files under
/u01/app/12.2.0.1/oracle/rdbms/audit

Kill the current running logrotate process

Let’s take a look at my logrotate script under /etc/logrorate.d/grid_rdbms

/u01/app/12.2.0.1/oracle/rdbms/audit/*
{
        daily
        rotate 15
        compress
        copytruncate
        notifempty
}

Every files under /u01/app/12.2.0.1/oracle/rdbms/audit/ get rotated over and over again.

Correct logrotate script ( change logrotate script to just rotate only the log file with specific extension)

/u01/app/12.2.0.1/oracle/rdbms/audit/*.aud
{
        daily
        rotate 15
        compress
        copytruncate
        notifempty
}

Of course i want to check if this would resolved the issue, so i tried to delete some of the old log rotate files using rm

[oracle@zsc-oda0-0 audit]$ rm *.gz
-bash: /bin/rm: Argument list too long

[oracle@zsc-oda0-0audit]$ find /u01/app/12.2.0.1/oracle/rdbms/audit/. - name "*.gz" -delete

[oracle@zsc-oda0-0audit]$ find /u01/app/12.2.0.1/oracle/rdbms/audit/. -name "*.aud" -mtime +7-delete    (delete current *aud extension with files older than 7day)

Check back a few days later to make sure it isn’t exponentially growing.

[oracle@zsc-oda0-0audit]$ find . -name "*.aud"-mtime +7-delete    (delete current *aud extension with files older than 7day)