[torquedev] saving inodes with lots of TM tasks
ake.sandgren at hpc2n.umu.se
Sat Feb 17 01:52:37 MST 2007
On Fri, 2007-02-16 at 20:33 -0700, Garrick Staples wrote:
> I have an immediate requirement to support hundreds of thousands of TM
> tasks within a single job. The limiter at this point is that pbs_mom
> saves a TM struct in a file per task. After a few hundred thousand
> tasks, the filesystem runs out of inodes!
> The purpose of the disk files are only to read in on a pbs_mom restart.
> In normal use, they are never read.
> So the solution seems simple: use 1 file. I've attached a patch that
> implements this idea by opening 1 file and seeking (sizeof(struct) *
> task_id) bytes into the file before writing. I didn't bother trying to
> be backwards compatible at this point because I think this will only be
> for 2.2.x.
Don't you have to make a copy of task_save in the read loop?
while (read(fds,(char *)&task_save,sizeof(task_save)) ==
pt->ti_qs = task_save;
You overwrite task_save each round...
Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
Internet: ake at hpc2n.umu.se Phone: +46 90 7866134 Fax: +46 90 7866126
Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se
More information about the torquedev