Bug 168

Summary: 2.5(.9) qsub does not seem to accept comma seperated -W argument
Product: TORQUE Reporter: ramon.bastiaans
Component: clientsAssignee: Ken Nielson <knielson>
Status: RESOLVED FIXED    
Severity: major CC: chris, rea+maui, torquedev
Priority: P5    
Version: 2.5.x   
Hardware: PC   
OS: Linux   
Attachments: Fixes handling of multiple comma-separated arguments for -W
Fixes old regression: qsub -W won't accept directive like stagein=a@h:b,c@h:d

Description ramon.bastiaans 2011-12-14 01:48:18 MST
When submitting a job that contains:

#PBS -W stagein=file1@host:/some/path/file1,stagein=file2@host:/some/path/file2

or:

#PBS -W stagein=file1@host:/some/path/file1,file2@host:/some/path/file2

or:

#PBS -W
'stagein=file1@host:/some/path/file1','stagein=file2@host:/some/path/file2'

etc etc;

We get the error:
* qsub: illegal -W value

However, submitting like this:

#PBS -W stagein=file1@host:/some/path/file1
#PBS -W stagein=file2@host:/some/path/file2

It works fine.

I believe 2.4 suffered from a similar bug, which was fixed in 2.4.9 (from
changelog):

b - restore ability to parse -W x=geometry:{...,...} 

Is it possible that this bug was not fixed in 2.5 ? Or maybe this is an
completely different bug.
Comment 1 Chris Samuel 2011-12-14 15:54:14 MST
(In reply to comment #0)

> I believe 2.4 suffered from a similar bug, which was fixed in 2.4.9 (from
> changelog):
> 
> b - restore ability to parse -W x=geometry:{...,...} 

That was SVN commit 3774, "Earlier patch broke ability to parse geometry:{...}
now fixed", or git commit 98d76c1a7f4da64fb38e2846a473baf9116023c1 for those
following along at home with git-svn.

> Is it possible that this bug was not fixed in 2.5 ? Or maybe this is an
> completely different bug.

Looks like the fix was never applied to 2.5, the 2.4 change still applies
cleanly (at least to qsub.c, the CHANGELOG part fails of course)..

samuel@eris:~/Downloads/Torque$ git show
98d76c1a7f4da64fb38e2846a473baf9116023c1 > /tmp/qsub.patch

samuel@eris:~/Downloads/Torque$ cd branches/2.5-fixes/

samuel@eris:~/Downloads/Torque/branches/2.5-fixes$ patch --dry-run -p3 <
/tmp/qsub.patch 
patching file CHANGELOG
Hunk #1 FAILED at 28.
1 out of 1 hunk FAILED -- saving rejects to file CHANGELOG.rej
patching file src/cmds/qsub.c
Hunk #1 succeeded at 491 (offset 19 lines).
Comment 2 Eygene Ryabinkin 2012-01-30 23:24:04 MST
Created an attachment (id=101) [details]
Fixes handling of multiple comma-separated arguments for -W

I have the same issue for 2.5.10.  The attached patch fixes the problem for me.
 Please, try it (though I am not sure that this patch will work for the whole
2.5 line).
Comment 3 Eygene Ryabinkin 2012-01-31 04:35:07 MST
Created an attachment (id=102) [details]
Fixes old regression: qsub -W won't accept directive like stagein=a@h:b,c@h:d

And another patch that fixes the long-standing regression in the qsub
behaviour: as the manual says, -W stagein has the form 'stagein=file_list' with
file_list being "local_file@hostname:remote_file[,...]", but qsub from 2.5.10
won't allow such syntax.

Since Torque is widely used by the European Grid middleware
(http://repository.egi.eu/) and it used to rely on the old behaviour of -W, it
will be good to restore that (though, just now it uses a workaround that sticks
-W stagein=a@h:b,stagein=c@h:d' to the submission scripts).  And the previous
patch that allows for '-W stagein=a@h:b,stagein=c@h:d' is an absolute must:
without it nothing will be submitted to the batch system by the EGI CREAM
middleware.
Comment 4 Eygene Ryabinkin 2012-01-31 04:36:19 MST
Forgot to say: these patches are currently deployed at our production
infrastructure and are OK already for some hours ;))
Comment 5 Eygene Ryabinkin 2012-01-31 08:09:46 MST
Found out that processing of double and single quotes and backslashes from
smart_strtok was dropped in my patch.  Will try to resurrect them and prepare
the updated fix.
Comment 6 Ken Nielson 2012-03-13 10:07:36 MDT
Fixed in 2.5.11 revision 5803