The torque.spec changes are being rejected. We feel that source builds should
be n /usr/local by default.

The PBS_CONFIGURE_ARGS will not be accepted in their form. We are considering
options.

The remaining patches, all of which regarding freebsd, kfreebsd, and manpages
were merged into trunk and 2.4-fixes.

Created an attachment (id=36) [details]
Sample torque spec file

(In reply to comment #1)
> The torque.spec changes are being rejected. We feel that source builds should
> be n /usr/local by default.

I don't see the patch attached to this bug, but assuming it modifies
torque.spec to install in FHS-compliant paths (e.g., /usr), this is absolutely
the correct thing to do.  RPM's should NEVER install to /usr/local.  For source
builds (./configure && make install), you're absolutely right, but building an
RPM isn't a source build.  You're integrating with the system-level package
manager.

So again, I haven't seen the patch, but having the spec file use %configure or
./configure --prefix=%{_prefix} is the correct way to create RPM's.

For reference, I'm including my torque spec file which packages correctly
according to established best practices.  RPM's built with this spec will
properly go into /usr, but manual source builds will still use /usr/local.

Actually, it just occured to me that this whole issue is moot because
torque.spec is generated from torque.spec.in by ./configure. torque.spec gets
whatever options were passed to configure. There are no defaults in
torque.spec.in.

The patches in question were linked from the original email to the torqueusers
list, specifically:
http://svn.debian.org/wsvn/pkg-escience/torque/trunk/debian/patches/spec_fhs_paths.patch

This patch doesn't actually change anything because it is patching a file
generated by ./configure.

(In reply to comment #3)
> Actually, it just occured to me that this whole issue is moot because
> torque.spec is generated from torque.spec.in by ./configure. torque.spec gets
> whatever options were passed to configure. There are no defaults in
> torque.spec.in.

FWIW, that's not really advisable.  Paths in RPM's should be FHS-compliant as
dictated by the macro settings in /usr/lib/rpm/macros.  As I said previously,
RPM's are not a source build.  They are managed by the system-level package
manager and should adhere to its settings.

So I disagree that the issue is moot.  It's fine to generate torque.spec and
populate things like Name and Version, but paths should really be left up to
RPM.

It is too a source build. Look, there, at the source files! And look at them
building!

The user can run configure with whatever paths they like. If they want
./configure --prefix=/usr, then that's great. If they want ./configure
--prefix=/opt/torque, then Bob's your uncle.

This mechanism is specifically designed to build an rpm (and subrpms) for the
builder following whatever options they want.

$ tar zxvf torque-whatever.tar.gz
$ cd torque-whatever
$ ./configure --prefix=/great --uncle=bob --what=ever
$ make rpm

Are tpackages also not source builds? Should we enforce some prefix on them
too?

Distros don't use the spec file in the tarball, so would they even care? Why
did Debian even look at the spec file, that they don't use, and don't
distribute in their .debs?

(In reply to comment #5)
> It is too a source build. Look, there, at the source files! And look at them
> building!

Yes, that's excellent.  I can be snide too, but it's not going to accomplish
anything.  This know-it-all attitude thing really discourages people who are
trying to help the project and the community educate/improve themselves by
sharing expertise that others may lack.  You are certainly a guru when it comes
to TORQUE, but that doesn't mean that others don't have knowledge that might be
of use to TORQUE, or to you, or to Adaptive.

That said, I'm going to try anyway, because I'm stubborn like that.

When you are building RPM's, you are creating packages that will most likely be
installed at the system level by an established package manager that has
established practices, paths, and configuration settings.  Almost no one
installs RPM into their own private directory, and 100% of those that do have
to manipulate the aforementioned macros anyway or else it won't work.  So the
macros values are ALWAYS right.

So if you want to get pedantic about terminology, it would be more correct to
say that building RPM's is not a source "install."  But the point remains that
it is inherently distinct from direct use of ./configure and make install.

> The user can run configure with whatever paths they like. If they want
> ./configure --prefix=/usr, then that's great. If they want ./configure
> --prefix=/opt/torque, then Bob's your uncle.

The point is that "whatever paths they like" doesn't work in the world of RPM. 
The macros exist for a reason.

> This mechanism is specifically designed to build an rpm (and subrpms) for the
> builder following whatever options they want.
> 
> $ tar zxvf torque-whatever.tar.gz
> $ cd torque-whatever
> $ ./configure --prefix=/great --uncle=bob --what=ever
> $ make rpm

You've completely ignored people wanting to build using, for example, "rpmbuild
-ta."  There IS a torque.spec in the distribution tarball, and its paths are,
by default, wrong for this very reason.

> Distros don't use the spec file in the tarball, so would they even care?

And there's a reason for that:  Because most software authors don't understand
RPM packaging and therefore don't make very good spec files.  This is precisely
why.  :-)

Furthermore, you're not just creating the spec file for distros.  You're also
creating it for people who want to use it with their own RPM-based distro which
might not supply torque packages directly.  And for them, supplying a proper
spec file which uses RPM and its settings correctly would be a huge benefit.

> Why did Debian even look at the spec file, that they don't use, and don't
> distribute in their .debs?

Perhaps because it's autogenerated by running configure but also included in
the distribution tarball, meaning whatever paths they happened to use when they
./configure'd propogated into the resultant distribution channel, inflicting
their chosen paths on an unsuspecting recipient?  :-)

(In reply to comment #6)
> (In reply to comment #5)
> > It is too a source build. Look, there, at the source files! And look at them
> > building!
> 
> Yes, that's excellent.  I can be snide too,

Great! We'll make beautiful music together.


> When you are building RPM's, you are creating packages that will most likely be
> installed at the system level by an established package manager that has
> established practices, paths, and configuration settings.  Almost no one
> installs RPM into their own private directory, and 100% of those that do have
> to manipulate the aforementioned macros anyway or else it won't work.  So the
> macros values are ALWAYS right.

But you don't know what the builder wants to do. It depends on the situation.



> You've completely ignored people wanting to build using, for example, "rpmbuild
> -ta."  There IS a torque.spec in the distribution tarball, and its paths are,
> by default, wrong for this very reason.

Actually, -ta isn't ignored at all. I went through great effort to make sure
-ta works. Corner cases like 'make snap' took extra work.


> > Distros don't use the spec file in the tarball, so would they even care?
> 
> And there's a reason for that:  Because most software authors don't understand
> RPM packaging and therefore don't make very good spec files.  This is precisely
> why.  :-)
> 
> Furthermore, you're not just creating the spec file for distros.  You're also
> creating it for people who want to use it with their own RPM-based distro which
> might not supply torque packages directly.  And for them, supplying a proper
> spec file which uses RPM and its settings correctly would be a huge benefit.
> 
> > Why did Debian even look at the spec file, that they don't use, and don't
> > distribute in their .debs?
> 
> Perhaps because it's autogenerated by running configure but also included in
> the distribution tarball, meaning whatever paths they happened to use when they
> ./configure'd propogated into the resultant distribution channel, inflicting
> their chosen paths on an unsuspecting recipient?  :-)

I have a solution. configure should still generate a torque.spec that exactly
matches what the user specified, but the torque.spec shipped with a distributed
tarball from Adaptive should have prefix=/usr.

This could be done easily if whoever at Adaptive was preparing the tarball
(release or snapshot) always remembered to run './configure --prefix=/usr'
before 'make dist'. Unfortunately that won't work because he or she won't
remember to do it!

This solution must be automatic. configure must know when to write a
torque.spec with --prefix=/usr and when to do what the builder is doing. I
propose two ways to automate this: 

1) configure looks for a .svn directory (if .svn exists, then --prefix=/usr is
written into the torque.spec). 

2) a pre-written buildutils/torque.spec.dist that only exists in the svn repo
is used in the tarball (copied to torque.spec before writing the tarball).
buildutils/torque.spec.dist would not, itself, be distributed.

I was just playing around with the method #2 and I now propose a method #3: 
./configure always writes a new torque.spec that has whatever the builder
chooses, but 'make dist' (or distcheck) copies in torque.spec.dist conditional
on the existance of .svn/. torque.spec.dist (and torque.spec.dist.in) would be
distributed.

The one downside is that a builder can't create their own tarball, from a
tarball, with their own custom torque.spec without editing
buildutils/torque.spec.dist

Alright, I think I've done it and checked it into trunk, r3802.

(In reply to comment #7)

> But you don't know what the builder wants to do. It depends on the situation.

No, it doesn't.  That's what I'm trying very hard to explain.  The macros are
ALWAYS right because they comprise the configuration for RPM, and as the
package manager, its configuration is authoritative.  The builder wants the RPM
they build to be properly installable on their system and/or the systems of
others (which, due to the nature of RPM, must the similar).  In order to
accomplish this, the RPM configuration needs to be honored, not ignored.

> Actually, -ta isn't ignored at all. I went through great effort to make sure
> -ta works. Corner cases like 'make snap' took extra work.

And yet, in reality, it does not work properly.  I should be able to run
rpmbuild -ta torque-2.4.8.tar.gz and get proper, working RPM's out the other
side.  The way it is now, assuming the RPM's build properly at all, the paths
will be whatever the person who built the tarball last ran ./configure with
(which may or may not be correct) rather than what RPM is configured to use via
macros (which is ALWAYS correct).

If someone building RPM's wants to put them in a different location, they
expect to build able to use something like this:

rpmbuild -ta --define '_prefix /opt/torque/2.4.8' torque-2.4.8.tar.gz

and have it build and install properly into that path.  Correctly-written spec
files, such as the one I attached, do this.  Incorrectly-written spec files,
such as the one supplied with torque, do not.

If I have to untar the tarball, run configure, and remake the tarball just to
use rpmbuild -ta, the spec is wrong.

> I have a solution. configure should still generate a torque.spec that exactly
> matches what the user specified, but the torque.spec shipped with a distributed
> tarball from Adaptive should have prefix=/usr.

No, it shouldn't.  configure should be used only to replace values in
torque.spec.in which are immutable for a given tarball.  In other words, I
should be able to untar the tarball, run ./configure with ANY options I choose,
run make dist, and get a tarball whose contents are IDENTICAL to the original
tarball.  That's the whole point.

The way things are being done right now is incorrect.

> This could be done easily if whoever at Adaptive was preparing the tarball
> (release or snapshot) always remembered to run './configure --prefix=/usr'
> before 'make dist'. Unfortunately that won't work because he or she won't
> remember to do it!

And I thank you for proving my point.  There is absolutely nothing stopping
someone from Adaptive who has been doing some testing with ./configure
--prefix=/home/foo/test/torque from running "make dist" and handing it off to
an unsuspecting user or customer (or doing a release) and making the generation
of RPM's from this release tarball impossible.

> This solution must be automatic. configure must know when to write a
> torque.spec with --prefix=/usr and when to do what the builder is doing.

My solution *is* automatic.  It defaults to the most correct option (using the
macros RPM supplies for exactly this purpose) and still allows the builder to
specify any values (s)he chooses by overriding the macros as shown above. 
There is NO CORRELATION WHATSOEVER between options passed to ./configure by
someone building source and the paths into which an RPM should be installed.

Again, someone who builds RPM's themselves will already have to have all the
correct paths setup in their .rpmmacros or system-wide macros files. 
Overriding them in the spec file is NEVER correct and almost always causes more
hassles for the person doing the building.

> 1) configure looks for a .svn directory (if .svn exists, then --prefix=/usr is
> written into the torque.spec). 
> 
> 2) a pre-written buildutils/torque.spec.dist that only exists in the svn repo
> is used in the tarball (copied to torque.spec before writing the tarball).
> buildutils/torque.spec.dist would not, itself, be distributed.

Either of these, based on the current spec file, will still be wrong.  A
correct spec file USES the macro values supplied by RPM; it does not replace
them.

As a customer, I am very curious to hear someone from Adaptive comment on
whether or not they are willing to supply a correct spec file for their
customers or if we will continue having to maintain our own independent spec.

Officially Adaptive Computing only supports the source tarballs found on the
web site. We do not officially support spec files or RPMs. Even so, we do try
to accommodate and promote best practices with respect to spec files and RPMs.
But officially we only support installing using ./configure make and make
install. 

Why don't you bring this up on the torquedev mailing list and poll the
community.

(In reply to comment #11)

> Officially Adaptive Computing only supports the source tarballs found on the
> web site. We do not officially support spec files or RPMs. Even so, we do try
> to accommodate and promote best practices with respect to spec files and RPMs.
> But officially we only support installing using ./configure make and make
> install. 
> 
> Why don't you bring this up on the torquedev mailing list and poll the
> community.

Fair enough.  Thank you, Ken. :)

(In reply to comment #10)
> (In reply to comment #7)
> 
> > But you don't know what the builder wants to do. It depends on the situation.
> 
> No, it doesn't.  That's what I'm trying very hard to explain.  The macros are
> ALWAYS right because they comprise the configuration for RPM, and as the
> package manager, its configuration is authoritative.  The builder wants the RPM
> they build to be properly installable on their system and/or the systems of
> others (which, due to the nature of RPM, must the similar).  In order to
> accomplish this, the RPM configuration needs to be honored, not ignored.

You don't know that. You have this "there is ONE proper way" mentality towards
system administration. I say that whoever is building the software knows best.


> > Actually, -ta isn't ignored at all. I went through great effort to make sure
> > -ta works. Corner cases like 'make snap' took extra work.
> 
> And yet, in reality, it does not work properly.  I should be able to run
> rpmbuild -ta torque-2.4.8.tar.gz and get proper, working RPM's out the other
> side.  The way it is now, assuming the RPM's build properly at all, the paths
> will be whatever the person who built the tarball last ran ./configure with
> (which may or may not be correct) rather than what RPM is configured to use via
> macros (which is ALWAYS correct).

"properly" according to your narrow definition of how things should work; is
not always correct.


> If someone building RPM's wants to put them in a different location, they
> expect to build able to use something like this:
> 
> rpmbuild -ta --define '_prefix /opt/torque/2.4.8' torque-2.4.8.tar.gz

So these same admins, the ones you characterized as barely understanding rpm,
are now using --defines? 


> and have it build and install properly into that path.  Correctly-written spec
> files, such as the one I attached, do this.  Incorrectly-written spec files,
> such as the one supplied with torque, do not.
> 
> If I have to untar the tarball, run configure, and remake the tarball just to
> use rpmbuild -ta, the spec is wrong.

Actually, you have an extra step there. untar, run configured, and run 'make
rpm'. Seems pretty simple.


> > I have a solution. configure should still generate a torque.spec that exactly
> > matches what the user specified, but the torque.spec shipped with a distributed
> > tarball from Adaptive should have prefix=/usr.
> 
> No, it shouldn't.  configure should be used only to replace values in
> torque.spec.in which are immutable for a given tarball.  In other words, I
> should be able to untar the tarball, run ./configure with ANY options I choose,
> run make dist, and get a tarball whose contents are IDENTICAL to the original
> tarball.  That's the whole point.

I should be able to untar the tarball, run ./configure with ANY options I
choose, run make rpm, and get rpms that exactly match what I want.

We can't have both ways.  You can't have something that customizes and stays
the same.


> The way things are being done right now is incorrect.
> 
> > This could be done easily if whoever at Adaptive was preparing the tarball
> > (release or snapshot) always remembered to run './configure --prefix=/usr'
> > before 'make dist'. Unfortunately that won't work because he or she won't
> > remember to do it!
> 
> And I thank you for proving my point.  There is absolutely nothing stopping
> someone from Adaptive who has been doing some testing with ./configure
> --prefix=/home/foo/test/torque from running "make dist" and handing it off to
> an unsuspecting user or customer (or doing a release) and making the generation
> of RPM's from this release tarball impossible.

Not impossible at all. Just run ./configure again.



> > This solution must be automatic. configure must know when to write a
> > torque.spec with --prefix=/usr and when to do what the builder is doing.
> 
> My solution *is* automatic.  It defaults to the most correct option (using the
> macros RPM supplies for exactly this purpose) and still allows the builder to
> specify any values (s)he chooses by overriding the macros as shown above. 
> There is NO CORRELATION WHATSOEVER between options passed to ./configure by
> someone building source and the paths into which an RPM should be installed.
> 
> Again, someone who builds RPM's themselves will already have to have all the
> correct paths setup in their .rpmmacros or system-wide macros files. 
> Overriding them in the spec file is NEVER correct and almost always causes more
> hassles for the person doing the building.
> 
> > 1) configure looks for a .svn directory (if .svn exists, then --prefix=/usr is
> > written into the torque.spec). 
> > 
> > 2) a pre-written buildutils/torque.spec.dist that only exists in the svn repo
> > is used in the tarball (copied to torque.spec before writing the tarball).
> > buildutils/torque.spec.dist would not, itself, be distributed.
> 
> Either of these, based on the current spec file, will still be wrong.  A
> correct spec file USES the macro values supplied by RPM; it does not replace
> them.

My solution is shipping a default torque.spec that uses the macros. This is
what you want and you are still fighting it? You are just arguing with me.


> As a customer, I am very curious to hear someone from Adaptive comment on
> whether or not they are willing to supply a correct spec file for their
> customers or if we will continue having to maintain our own independent spec.

(In reply to comment #13)

> You don't know that. You have this "there is ONE proper way" mentality towards
> system administration.

No, I'm saying there is one proper way with respect to RPM.  There's a big
difference.  :-)  And as an RPM developer, I have a pretty good idea what I'm
talking about.

> I say that whoever is building the software knows best.

And >10 years of experience with RPM and RPM-based distributions tells me
otherwise.  Most users want things to Just Work(tm), and proper use of the
macros does that.

> "properly" according to your narrow definition of how things should work; is
> not always correct.

That's like saying that it's okay to build a part for a car and completely
ignore the type of connectors required to attach the part to the rest of the
car.  You're trying to create something for use with RPM while completely
ignoring the interface.

> So these same admins, the ones you characterized as barely understanding rpm,
> are now using --defines? 

Now you're mixing up my words.  The people I previously characterized as
"barely understanding RPM" are typical software authors who try to create their
own spec files.

Users who build RPM's and install them into /opt already know how to set up the
macros to facilitate that.  They have to; if they didn't, they couldn't build
arbitrary RPM's that install elsewhere.

And the users who don't know how to do that want their RPM's to just work,
which means going to the correct locations for RPM, not the defaults for GNU
autoconf.

> Actually, you have an extra step there. untar, run configured, and run 'make
> rpm'. Seems pretty simple.

Also non-standard and not as simple as:

wget http://www.clusterresources.com/downloads/torque/torque-2.4.8.tar.gz
rpmbuild -ta torque-2.4.8.tar.gz

Or, even better:

mzbuild http://www.clusterresources.com/downloads/torque/torque-2.4.8.tar.gz

> I should be able to untar the tarball, run ./configure with ANY options I
> choose, run make rpm, and get rpms that exactly match what I want.
> 
> We can't have both ways.  You can't have something that customizes and stays
> the same.

Just out of curiosity, do you actually use RPM's in general, or of torque?  Are
you saying this because it's something you, personally, do on a regular basis,
or because that's your view of what the community of RPM users wants?

Most RPM users expect to have a workflow similar to what I've described.  I
think it's great that you provide a macro for the configure arguments, and it's
very easy for someone to supply those to rpmbuild.  I have no problem with
that, per se.  The problems arise from propagating ./configure arguments into
the built RPM's.  This is not typical, expected behavior, and is contrary to
best practices.

> > And I thank you for proving my point.  There is absolutely nothing stopping
> > someone from Adaptive who has been doing some testing with ./configure
> > --prefix=/home/foo/test/torque from running "make dist" and handing it off to
> > an unsuspecting user or customer (or doing a release) and making the generation
> > of RPM's from this release tarball impossible.
> 
> Not impossible at all. Just run ./configure again.

The same reply could've been made to your point, and we both know that's not
the point.  rpmbuild -ta would not work properly, even with _prefix defined
properly.

> My solution is shipping a default torque.spec that uses the macros. This is
> what you want and you are still fighting it? You are just arguing with me.

Your solution does fix the specific concerns I voiced with overriding the
macros, so thank you for that.

My goals when I originally wrote my torque spec file were:
  1.  Fix adherence to best practices and standards for RPM packaging.
  2.  Provide a working, functional torque out-of-the-box as soon as the RPM is
installed.
  3.  Use standard conditional build directives and parameters to allow
build-time customizations rather than non-standard macros.
  4.  Reduce the number of unnecessary subpackages by consolidating where it
makes sense and using existing RPM features (e.g., --excludedocs).

I think I've accomplished those goals pretty well.  If anyone with commit
access wants to take a look at the spec I supplied and maybe consider adopting
some of the other stuff in there, please feel free.  Otherwise, I'm okay to
agree to disagree and consider the issue closed.