[torqueusers] Torque 3.0.3 and chroot environment

Kunal Rao kunalgrao at gmail.com
Thu Jun 7 11:55:14 MDT 2012


Hi David,

Thanks for your response.

This is exactly why I was thinking of a chroot kind of environment. Here
each mom can have its own "/proc " directory and there could be an external
script which populates various information for example loadavg, meminfo
etc. and pbs_mom would read that and report to the head node. Thus there
won't be any source code change at all and it would mimic a real large
cluster.

Were there any problems with the chroot environment settings that you had.
If there is any documentation related to Torque configuration with chroot
env. could you point me to that ?

Coming back to the multi-mom feature, for each mom to report different
load, memory etc. can we have :

pbs_mom for node1 read from /proc/node1/          (we dump some loadavg ,
meminfo, cpuinfo etc. files here )

pbs_mom for node2 read from /proc/node2/          (here also we dump some
loadavg, meminfo etc. files)

'
'
I'm guessing for this, in the source code, wherever it is hard-coded to
read from "/proc" path, it should take the path as an argument when pbs_mom
is started.

e.g. pbs_mom -u <path from where to read resource usage info>

This will probably fix that issue and each mom will be able to report
different resource information to the head node and will mimic a real large
cluster with a smaller set of nodes.

Let me know your thoughts on that. Are there other approaches that you have
in mind ?

Thanks,
Kunal

On Thu, Jun 7, 2012 at 1:08 PM, David Beer <dbeer at adaptivecomputing.com>wrote:

> Kunal,
>
> As of now each will report the same thing. If you wanted them to change
> each one, you'd have to modify the code. It wouldn't be too hard to do (the
> mom daemons know that they're running multi-mom) but it would take some
> customization.
>
> David
>
>
> On Thu, Jun 7, 2012 at 10:50 AM, Kunal Rao <kunalgrao at gmail.com> wrote:
>
>> Hi David,
>>
>> Thanks for your quick response and for pointing to the multi-mom feature.
>> The idea is similar i.e. make a small cluster look bigger with being as
>> realistic as possible.
>>
>> I read through that page and seems like it will do what I want. I had a
>> follow up question on that :
>>
>> - Does each mom read from /proc and report to the head node (pbs_server)
>> ? In that case the total cpus , memory, load etc. will be reported same
>> from each of them. Can that be isolated and different for each of them to
>> mimic
>>   actual large cluster i.e. each having different number of cpus, memory,
>> load etc.
>>
>> Thanks,
>> Kunal
>>
>>
>>
>> On Thu, Jun 7, 2012 at 12:16 PM, David Beer <dbeer at adaptivecomputing.com>wrote:
>>
>>> Kunal,
>>>
>>> I have done a chroot environment with TORQUE - it worked fine. I was
>>> doing this for testing with sleep jobs, and the chroot was because I didn't
>>> want it to interact with anything else on the machine. I'm not sure what
>>> you're attempting to accomplish, but you may want to consider looking into
>>> the multi-mom feature (available starting in 3.0.0) that we also use a lot
>>> for testing. I have actually abandoned my chroot environment in favor of
>>> using the multi-moms.
>>>
>>>
>>> http://www.adaptivecomputing.com/resources/docs/torque/3-0-3/1.8multimom.php
>>>
>>> David
>>>
>>> On Thu, Jun 7, 2012 at 10:05 AM, Kunal Rao <kunalgrao at gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Has anyone tried chroot environment for Torque 3.0.3 or later version ?
>>>> I'm thinking of having multiple chroot environment on the same system, each
>>>> representing a compute node and build a cluster.
>>>>
>>>> So, even though there are say only 2 physical machines ( 1 server and 1
>>>> compute node), we should be able to make a cluster of say 4 nodes. Assuming
>>>> that the 1 physical compute node can have 3 chroot environment,
>>>> each having its own virtual IP and communicating with the master as 3
>>>> independent compute nodes. Head node / server will see as if there are 4
>>>> nodes and the scheduler will aallocate jobs accordingly.
>>>>
>>>> Is this feasible and can work without any source code modifications to
>>>> pbs server  / pbs mom ?
>>>>
>>>> Thanks,
>>>> Kunal
>>>>
>>>> _______________________________________________
>>>> torqueusers mailing list
>>>> torqueusers at supercluster.org
>>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>>>
>>>>
>>>
>>>
>>> --
>>> David Beer | Software Engineer
>>> Adaptive Computing
>>>
>>>
>>> _______________________________________________
>>> torqueusers mailing list
>>> torqueusers at supercluster.org
>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>>
>>>
>>
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>
>>
>
>
> --
> David Beer | Software Engineer
> Adaptive Computing
>
>
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20120607/9d3abc6c/attachment.html 


More information about the torqueusers mailing list