[torqueusers] Torque 3.0.3 and chroot environment

Kunal Rao kunalgrao at gmail.com
Mon Jun 11 15:16:59 MDT 2012


Any further thoughts / inputs on this ?

Thanks,
Kunal

On Thu, Jun 7, 2012 at 1:55 PM, Kunal Rao <kunalgrao at gmail.com> wrote:

> Hi David,
>
> Thanks for your response.
>
> This is exactly why I was thinking of a chroot kind of environment. Here
> each mom can have its own "/proc " directory and there could be an external
> script which populates various information for example loadavg, meminfo
> etc. and pbs_mom would read that and report to the head node. Thus there
> won't be any source code change at all and it would mimic a real large
> cluster.
>
> Were there any problems with the chroot environment settings that you had.
> If there is any documentation related to Torque configuration with chroot
> env. could you point me to that ?
>
> Coming back to the multi-mom feature, for each mom to report different
> load, memory etc. can we have :
>
> pbs_mom for node1 read from /proc/node1/          (we dump some loadavg ,
> meminfo, cpuinfo etc. files here )
>
> pbs_mom for node2 read from /proc/node2/          (here also we dump some
> loadavg, meminfo etc. files)
>
> '
> '
> I'm guessing for this, in the source code, wherever it is hard-coded to
> read from "/proc" path, it should take the path as an argument when pbs_mom
> is started.
>
> e.g. pbs_mom -u <path from where to read resource usage info>
>
> This will probably fix that issue and each mom will be able to report
> different resource information to the head node and will mimic a real large
> cluster with a smaller set of nodes.
>
> Let me know your thoughts on that. Are there other approaches that you
> have in mind ?
>
> Thanks,
> Kunal
>
>
> On Thu, Jun 7, 2012 at 1:08 PM, David Beer <dbeer at adaptivecomputing.com>wrote:
>
>> Kunal,
>>
>> As of now each will report the same thing. If you wanted them to change
>> each one, you'd have to modify the code. It wouldn't be too hard to do (the
>> mom daemons know that they're running multi-mom) but it would take some
>> customization.
>>
>> David
>>
>>
>> On Thu, Jun 7, 2012 at 10:50 AM, Kunal Rao <kunalgrao at gmail.com> wrote:
>>
>>> Hi David,
>>>
>>> Thanks for your quick response and for pointing to the multi-mom
>>> feature. The idea is similar i.e. make a small cluster look bigger with
>>> being as realistic as possible.
>>>
>>> I read through that page and seems like it will do what I want. I had a
>>> follow up question on that :
>>>
>>> - Does each mom read from /proc and report to the head node (pbs_server)
>>> ? In that case the total cpus , memory, load etc. will be reported same
>>> from each of them. Can that be isolated and different for each of them to
>>> mimic
>>>   actual large cluster i.e. each having different number of cpus,
>>> memory, load etc.
>>>
>>> Thanks,
>>> Kunal
>>>
>>>
>>>
>>> On Thu, Jun 7, 2012 at 12:16 PM, David Beer <dbeer at adaptivecomputing.com
>>> > wrote:
>>>
>>>> Kunal,
>>>>
>>>> I have done a chroot environment with TORQUE - it worked fine. I was
>>>> doing this for testing with sleep jobs, and the chroot was because I didn't
>>>> want it to interact with anything else on the machine. I'm not sure what
>>>> you're attempting to accomplish, but you may want to consider looking into
>>>> the multi-mom feature (available starting in 3.0.0) that we also use a lot
>>>> for testing. I have actually abandoned my chroot environment in favor of
>>>> using the multi-moms.
>>>>
>>>>
>>>> http://www.adaptivecomputing.com/resources/docs/torque/3-0-3/1.8multimom.php
>>>>
>>>> David
>>>>
>>>> On Thu, Jun 7, 2012 at 10:05 AM, Kunal Rao <kunalgrao at gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Has anyone tried chroot environment for Torque 3.0.3 or later version
>>>>> ? I'm thinking of having multiple chroot environment on the same system,
>>>>> each representing a compute node and build a cluster.
>>>>>
>>>>> So, even though there are say only 2 physical machines ( 1 server and
>>>>> 1 compute node), we should be able to make a cluster of say 4 nodes.
>>>>> Assuming that the 1 physical compute node can have 3 chroot environment,
>>>>> each having its own virtual IP and communicating with the master as 3
>>>>> independent compute nodes. Head node / server will see as if there are 4
>>>>> nodes and the scheduler will aallocate jobs accordingly.
>>>>>
>>>>> Is this feasible and can work without any source code modifications to
>>>>> pbs server  / pbs mom ?
>>>>>
>>>>> Thanks,
>>>>> Kunal
>>>>>
>>>>> _______________________________________________
>>>>> torqueusers mailing list
>>>>> torqueusers at supercluster.org
>>>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> David Beer | Software Engineer
>>>> Adaptive Computing
>>>>
>>>>
>>>> _______________________________________________
>>>> torqueusers mailing list
>>>> torqueusers at supercluster.org
>>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>>>
>>>>
>>>
>>> _______________________________________________
>>> torqueusers mailing list
>>> torqueusers at supercluster.org
>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>>
>>>
>>
>>
>> --
>> David Beer | Software Engineer
>> Adaptive Computing
>>
>>
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20120611/7eb93fd8/attachment.html 


More information about the torqueusers mailing list