[torquedev] Leaking Memory ?

ifeghali at petrobras.com.br ifeghali at petrobras.com.br
Fri Nov 6 06:22:54 MST 2009



Hello,

Following the same path as pbs_python
(https://subtrac.sara.nl/oss/pbs_python), I have used SWIG
(www.swig.org) to build a PHP wrapper for libtorque. The whole stuff
works great for standalone applications, but iterating too many times
(on a daemon for instance) makes the memory usage sky rocket. It does
not happens for python, maybe because python's garbage collection
policy I guess, but I am not sure. But it does happen with PHP and I
still haven't found a workaround.

For instance, the following code:

<?php
require_once 'build/openpbs.php';

while (true) {
   $con = pbs_connect("pbsserver");
   pbs_statjob($con, '', null, null);
   pbs_disconnect($con);

   print number_format(memory_get_usage(),2)."\n";
   sleep(2);
}
?>

will output:

199,784.00
199,792.00
199,800.00
199,808.00
199,816.00
etc.

The memory usage always increase each loop. For a single loop, the
relevant output of valgrind is as follows:

==13242== 156 (36 direct, 120 indirect) bytes in 1 blocks are
definitely lost in loss record 26 of 58
==13242==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==13242==    by 0x88DC47: nss_parse_service_list (in /lib/libc-2.5.so)
==13242==    by 0x88E505: __nss_database_lookup (in /lib/libc-2.5.so)
==13242==    by 0x4559F6B: ???
==13242==    by 0x455B0BC: ???
==13242==    by 0x83BC82: getpwuid_r@@GLIBC_2.1.2 (in /lib/libc-2.5.so)
==13242==    by 0x83B657: getpwuid (in /lib/libc-2.5.so)
==13242==    by 0xA6F4CF: pbs_connect (in /usr/lib/libtorque.so.0.0.0)
==13242==    by 0x4018E1A: _wrap_pbs_connect (openpbs_wrap.c:2405)
==13242==    by 0x82A8935: zend_do_fcall_common_helper_SPEC
(zend_vm_execute.h:200)
==13242==    by 0x82AED90: ZEND_DO_FCALL_SPEC_CONST_HANDLER
(zend_vm_execute.h:1729)
==13242==    by 0x82A8430: execute (zend_vm_execute.h:92)
==13242==
==13242==
==13242== 24,244 (16 direct, 24,228 indirect) bytes in 1 blocks are
definitely lost in loss record 51 of 58
==13242==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==13242==    by 0xA717CD: (within /usr/lib/libtorque.so.0.0.0)
==13242==    by 0xA7196F: PBSD_status_get (in /usr/lib/libtorque.so.0.0.0)
==13242==    by 0xA71A15: PBSD_status (in /usr/lib/libtorque.so.0.0.0)
==13242==    by 0xA71609: pbs_statjob (in /usr/lib/libtorque.so.0.0.0)
==13242==    by 0x40155D1: _wrap_pbs_statjob (openpbs_wrap.c:3397)
==13242==    by 0x82A8935: zend_do_fcall_common_helper_SPEC
(zend_vm_execute.h:200)
==13242==    by 0x82AED90: ZEND_DO_FCALL_SPEC_CONST_HANDLER
(zend_vm_execute.h:1729)
==13242==    by 0x82A8430: execute (zend_vm_execute.h:92)
==13242==    by 0x827FF74: zend_execute_scripts (zend.c:1134)
==13242==    by 0x82252D5: php_execute_script (main.c:2023)
==13242==    by 0x8306397: main (php_cli.c:1133)
==13242==

The first leak seems to come from pbs_connect when it calls GLIB, that
is probably querying NIS for my username. The next one seems to occur
exclusively inside pbs_statjob() code.

Currently I have a PHP daemon that queries "qstat" to cache jobs
information over time. For performance reasons I am attempting to make
PHP talks to libtorque directly, but the memory issue is being a
killer.

Any help would be much appreciated.

Best Regards,
~IF. 
"O emitente desta mensagem é responsável por seu conteúdo e endereçamento. Cabe ao destinatário cuidar quanto ao tratamento adequado. Sem a devida autorização, a divulgação, a reprodução, a distribuição ou qualquer outra ação em desconformidade com as normas internas do Sistema Petrobras são proibidas e passíveis de sanção disciplinar, cível e criminal."
 
"The sender of this message is responsible for its content and addressing. The receiver shall take proper care of it. Without due authorization, the publication, reproduction, distribution or the performance of  any other action not conforming to Petrobras System internal policies and procedures is forbidden and liable to disciplinary, civil or criminal sanctions."
 
"El emisor de este mensaje es responsable por su contenido y direccionamiento. Cabe al destinatario darle el tratamiento adecuado. Sin la debida autorización, su divulgación, reproducción, distribución o cualquier otra acción no conforme a las normas internas del Sistema Petrobras están prohibidas y serán pasibles de sanción disciplinaria, civil y penal."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torquedev/attachments/20091106/3266e854/attachment.html 


More information about the torquedev mailing list