[torqueusers] Empty output/error log file

[FyD]

Dear Michael

>> So all the jobs go in the queue, the first 8th ones directly runs
>> while the others are queued. The first 8th jobs end well and the
>> wanted data files/results are generated.
>
> Okay, so everything is fine with torque. Your problem is directly on
> your node(s).

ok

>> _Nothing_ is done/generated! There is obviously no problem of space;
>> we have plenty of room in the scratch directory.
>
> So if you are using 8 jobs and each of this job creates about 1M files
> you might run into filesystem limitations, possibly not enough inodes.
> Have you checked this? What operating system/filesystem/partition size
> are you using?

We use Linux Centos 5.4. 64 bits with the ext3 file system:

[xxxx at node2 ~]$ df -Th
Sys. de fich. Type     Tail. Occ. Disp. %Occ. Monté sur
/dev/sda3     ext3    2,9G  2,6G  105M  97% /
/dev/sda5     ext3    219G  188M  208G   1% /tmp
/dev/sda1     ext3     99M   18M   77M  19% /boot
tmpfs        tmpfs    5,9G     0  5,9G   0% /dev/shm
/dev/sdb1     ext3    917G  414G  457G  48% /scratch
master0:/home  nfs    5,5T  2,2T  3,4T  40% /home
master0:/usr/local
                nfs     27G   16G   11G  60% /usr/local
master0:/opt   nfs     15G  7,6G  5,9G  57% /opt

as you can see the /scratch partition is not full...

I think we do not have file system/room limitation because when I  
re-run the job that fails it usually works.

> As mentioned already, please also post the log files of your pbs_mom
> directory on your node.

oh oh I need to read about that here...

thanks, regards, Francois