[torqueusers] limiting stdout/stderr files

[Glen Beane]

On Wed, Dec 15, 2010 at 2:24 PM, Chad Vizino <vizino at psc.edu> wrote:
> We've hacked David's "hacky bit of code" (see below) to work with Torque
> at our site and find it quite useful.  Thanks Dave!
>
> Here's our Torque version currently in use with 2.3.12:
>
> #if !defined(NO_SPOOL_OUTPUT)
> #define VARSPOOLUSERLIM_KB 20480
>
>   /* check file sizes in PBS spool area */
>   if (pjob->ji_qs.ji_svrflags&JOB_SVFLG_HERE) { /* only on MS */
>     char path[64];
>     char *suf;
>     struct stat sbuf;
>
>     (void)strcpy(path, path_spool);
>     (void)strcat(path, pjob->ji_qs.ji_fileprefix);
>     suf = path+strlen(path);
>
>     (void)strcat(path, JOB_STDOUT_SUFFIX);
>     if ( (stat(path, &sbuf)==0) &&
>       (sbuf.st_size>>10 > (off_t)VARSPOOLUSERLIM_KB) ){
>       sprintf(log_buffer, "stdout file size %luKB exceeded limit %luKB",
>       ((unsigned long)(sbuf.st_size>>10)), (unsigned
> long)VARSPOOLUSERLIM_KB);
>       return (TRUE);
>     }
>
>     (void)strcpy(suf, JOB_STDERR_SUFFIX);
>     if ( (stat(path, &sbuf)==0) &&
>       (sbuf.st_size>>10 > (off_t)VARSPOOLUSERLIM_KB) ){
>       sprintf(log_buffer, "stderr file size %luKB exceeded limit %luKB",
>       ((unsigned long)(sbuf.st_size>>10)), (unsigned
> long)VARSPOOLUSERLIM_KB);
>       return (TRUE);
>     }
>   }
> #endif
>
>  -Chad


it might be nice to make something like this configurable in the
pbs_mom config file - the default behavior would be the same
(unlimited), but if the admin sets a maximum size then apply this
limit and terminate jobs that go over.  I would probably be willing to
do this if there were enough interested.