1Following are examples of running py_mallocstk.d. This traces malloc() from 2Python, printing byte distributions by user stack trace. 3 4Here we see the script runnin on the program Code/Python/func_abc.py 5 6# py_mallocstk.d -c ./func_abc.py 7 8Tracing... Hit Ctrl-C to end. 9Function A 10Function B 11Function C 12 13Python malloc byte distributions by stack trace, 14 15 16 17 libc.so.1`malloc 18 libpython2.4.so.1.0`r_object+0x52f 19 libpython2.4.so.1.0`r_object+0x491 20 libpython2.4.so.1.0`r_object+0xd3 21 libpython2.4.so.1.0`r_object+0x491 22 libpython2.4.so.1.0`r_object+0xd3 23 libpython2.4.so.1.0`r_object+0x491 24 libpython2.4.so.1.0`r_object+0xd3 25 libpython2.4.so.1.0`PyMarshal_ReadObjectFromString+0x36 26 libpython2.4.so.1.0`PyMarshal_ReadLastObjectFromFile+0x6a 27 libpython2.4.so.1.0`read_compiled_module+0xf 28 libpython2.4.so.1.0`load_source_module+0x63 29 libpython2.4.so.1.0`load_module+0xac 30 libpython2.4.so.1.0`import_submodule+0xfb 31 libpython2.4.so.1.0`load_next+0xee 32 libpython2.4.so.1.0`import_module_ex+0x48 33 libpython2.4.so.1.0`PyImport_ImportModuleEx+0x1d 34 libpython2.4.so.1.0`builtin___import__+0x4e 35 libpython2.4.so.1.0`PyCFunction_Call+0x15f 36 libpython2.4.so.1.0`PyObject_Call+0x1d 37 libpython2.4.so.1.0`PyEval_CallObjectWithKeywords+0xb8 38 libpython2.4.so.1.0`PyEval_EvalFrame+0xd3c 39 [ /usr/lib/python2.4/encodings/__init__.py:28 (?) ] 40 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 41 libpython2.4.so.1.0`PyEval_EvalCode+0x22 42 libpython2.4.so.1.0`PyImport_ExecCodeModuleEx+0xc0 43 libpython2.4.so.1.0`load_source_module+0xe6 44 libpython2.4.so.1.0`load_module+0xac 45 libpython2.4.so.1.0`load_package+0xef 46 libpython2.4.so.1.0`load_module+0x6a 47 libpython2.4.so.1.0`import_submodule+0xfb 48 libpython2.4.so.1.0`load_next+0xa2 49 libpython2.4.so.1.0`import_module_ex+0x48 50 libpython2.4.so.1.0`PyImport_ImportModuleEx+0x1d 51 libpython2.4.so.1.0`_PyCodecRegistry_Init+0xce 52 libpython2.4.so.1.0`_PyCodec_Lookup+0x2a 53 libpython2.4.so.1.0`PyCodec_Encoder+0xf 54 libpython2.4.so.1.0`Py_InitializeEx+0x257 55 libpython2.4.so.1.0`Py_Initialize+0xd 56 libpython2.4.so.1.0`Py_Main+0x4db 57 python`main+0x11 58 python`_start+0x7a 59 60 value ------------- Distribution ------------- count 61 0 | 0 62 1 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 63 2 | 0 64 65 66 libc.so.1`malloc 67 libpython2.4.so.1.0`_PyUnicode_New+0xb2 68 libpython2.4.so.1.0`_PyUnicodeUCS2_Init+0x19 69 libpython2.4.so.1.0`Py_InitializeEx+0x11c 70 libpython2.4.so.1.0`Py_Initialize+0xd 71 libpython2.4.so.1.0`Py_Main+0x4db 72 python`main+0x11 73 python`_start+0x7a 74 75 value ------------- Distribution ------------- count 76 1 | 0 77 2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 78 4 | 0 79 80 81 libc.so.1`malloc 82 libc.so.1`_real_gettext_u+0x81 83 libc.so.1`dgettext+0x5e 84 libc.so.1`strerror+0x40 85 libpython2.4.so.1.0`PyErr_SetFromErrnoWithFilenameObject+0x2d 86 libpython2.4.so.1.0`PyErr_SetFromErrnoWithFilename+0x27 87 libpython2.4.so.1.0`posix_error_with_allocated_filename+0x17 88 libpython2.4.so.1.0`posix_do_stat+0x21f 89 libpython2.4.so.1.0`posix_stat+0x1f 90 libpython2.4.so.1.0`PyCFunction_Call+0x15f 91 libpython2.4.so.1.0`call_function+0x406 92 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 93 [ /usr/lib/python2.4/posixpath.py:195 (isdir) ] 94 libpython2.4.so.1.0`fast_function+0xa8 95 libpython2.4.so.1.0`call_function+0xda 96 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 97 [ /usr/lib/python2.4/site.py:202 (addsitepackages) ] 98 libpython2.4.so.1.0`fast_function+0xa8 99 libpython2.4.so.1.0`call_function+0xda 100 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 101 [ /usr/lib/python2.4/site.py:382 (main) ] 102 libpython2.4.so.1.0`fast_function+0xa8 103 libpython2.4.so.1.0`call_function+0xda 104 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 105 [ /usr/lib/python2.4/site.py:397 (?) ] 106 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 107 libpython2.4.so.1.0`PyEval_EvalCode+0x22 108 libpython2.4.so.1.0`PyImport_ExecCodeModuleEx+0xc0 109 libpython2.4.so.1.0`load_source_module+0xe6 110 libpython2.4.so.1.0`load_module+0xac 111 libpython2.4.so.1.0`import_submodule+0xfb 112 libpython2.4.so.1.0`load_next+0xa2 113 libpython2.4.so.1.0`import_module_ex+0x48 114 libpython2.4.so.1.0`PyImport_ImportModuleEx+0x1d 115 libpython2.4.so.1.0`builtin___import__+0x4e 116 libpython2.4.so.1.0`PyCFunction_Call+0x15f 117 libpython2.4.so.1.0`PyObject_Call+0x1d 118 libpython2.4.so.1.0`PyObject_CallFunction+0x90 119 libpython2.4.so.1.0`PyImport_Import+0x163 120 libpython2.4.so.1.0`PyImport_ImportModule+0x1f 121 libpython2.4.so.1.0`initsite+0x10 122 libpython2.4.so.1.0`Py_InitializeEx+0x1ea 123 libpython2.4.so.1.0`Py_Initialize+0xd 124 libpython2.4.so.1.0`Py_Main+0x4db 125 python`main+0x11 126 python`_start+0x7a 127 128 value ------------- Distribution ------------- count 129 1 | 0 130 2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 131 4 | 0 132 133[... around 17000 lines truncated ...] 134 135 136 libc.so.1`malloc 137 libpython2.4.so.1.0`PyObject_Malloc+0x126 138 libpython2.4.so.1.0`fixstate+0x26 139 libpython2.4.so.1.0`fixdfa+0x2a 140 libpython2.4.so.1.0`PyGrammar_AddAccelerators+0x1b 141 libpython2.4.so.1.0`PyParser_New+0x18 142 libpython2.4.so.1.0`parsetok+0x17 143 libpython2.4.so.1.0`PyParser_ParseStringFlagsFilename+0x72 144 libpython2.4.so.1.0`PyParser_ParseStringFlags+0x1c 145 libpython2.4.so.1.0`PyParser_SimpleParseStringFlags+0x23 146 libpython2.4.so.1.0`PyRun_StringFlags+0x2c 147 libpython2.4.so.1.0`builtin_eval+0x273 148 libpython2.4.so.1.0`PyCFunction_Call+0x15f 149 libpython2.4.so.1.0`call_function+0x406 150 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 151 [ /usr/lib/python2.4/os.py:503 (_exists) ] 152 libpython2.4.so.1.0`fast_function+0xa8 153 libpython2.4.so.1.0`call_function+0xda 154 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 155 [ /usr/lib/python2.4/os.py:509 (?) ] 156 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 157 libpython2.4.so.1.0`PyEval_EvalCode+0x22 158 libpython2.4.so.1.0`PyImport_ExecCodeModuleEx+0xc0 159 libpython2.4.so.1.0`load_source_module+0xe6 160 libpython2.4.so.1.0`load_module+0xac 161 libpython2.4.so.1.0`import_submodule+0xfb 162 libpython2.4.so.1.0`load_next+0xa2 163 libpython2.4.so.1.0`import_module_ex+0x48 164 libpython2.4.so.1.0`PyImport_ImportModuleEx+0x1d 165 libpython2.4.so.1.0`builtin___import__+0x4e 166 libpython2.4.so.1.0`PyCFunction_Call+0x15f 167 libpython2.4.so.1.0`PyObject_Call+0x1d 168 libpython2.4.so.1.0`PyEval_CallObjectWithKeywords+0xb8 169 libpython2.4.so.1.0`PyEval_EvalFrame+0xd3c 170 [ /usr/lib/python2.4/site.py:58 (?) ] 171 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 172 libpython2.4.so.1.0`PyEval_EvalCode+0x22 173 libpython2.4.so.1.0`PyImport_ExecCodeModuleEx+0xc0 174 libpython2.4.so.1.0`load_source_module+0xe6 175 libpython2.4.so.1.0`load_module+0xac 176 libpython2.4.so.1.0`import_submodule+0xfb 177 libpython2.4.so.1.0`load_next+0xa2 178 libpython2.4.so.1.0`import_module_ex+0x48 179 libpython2.4.so.1.0`PyImport_ImportModuleEx+0x1d 180 libpython2.4.so.1.0`builtin___import__+0x4e 181 libpython2.4.so.1.0`PyCFunction_Call+0x15f 182 libpython2.4.so.1.0`PyObject_Call+0x1d 183 libpython2.4.so.1.0`PyObject_CallFunction+0x90 184 libpython2.4.so.1.0`PyImport_Import+0x163 185 libpython2.4.so.1.0`PyImport_ImportModule+0x1f 186 libpython2.4.so.1.0`initsite+0x10 187 libpython2.4.so.1.0`Py_InitializeEx+0x1ea 188 libpython2.4.so.1.0`Py_Initialize+0xd 189 libpython2.4.so.1.0`Py_Main+0x4db 190 python`main+0x11 191 python`_start+0x7a 192 193 value ------------- Distribution ------------- count 194 256 | 0 195 512 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 331 196 1024 | 0 197 198 199 libc.so.1`malloc 200 libpython2.4.so.1.0`new_arena+0x13 201 libpython2.4.so.1.0`PyObject_Malloc+0x91 202 libpython2.4.so.1.0`string_concat+0x109 203 libpython2.4.so.1.0`PyString_Concat+0x3b 204 libpython2.4.so.1.0`string_concatenate+0x150 205 libpython2.4.so.1.0`PyEval_EvalFrame+0x27cc 206 [ /usr/lib/python2.4/posixpath.py:62 (join) ] 207 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 208 libpython2.4.so.1.0`function_call+0x15e 209 libpython2.4.so.1.0`PyObject_Call+0x1d 210 libpython2.4.so.1.0`ext_do_call+0xfb 211 libpython2.4.so.1.0`PyEval_EvalFrame+0xb4f 212 [ /usr/lib/python2.4/site.py:66 (makepath) ] 213 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 214 libpython2.4.so.1.0`fast_function+0x112 215 libpython2.4.so.1.0`call_function+0xda 216 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 217 [ /usr/lib/python2.4/site.py:138 (addpackage) ] 218 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 219 libpython2.4.so.1.0`fast_function+0x112 220 libpython2.4.so.1.0`call_function+0xda 221 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 222 [ /usr/lib/python2.4/site.py:166 (addsitedir) ] 223 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 224 libpython2.4.so.1.0`fast_function+0x112 225 libpython2.4.so.1.0`call_function+0xda 226 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 227 [ <string>:1 (?) ] 228 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 229 libpython2.4.so.1.0`PyEval_EvalCode+0x22 230 libpython2.4.so.1.0`run_node+0x35 231 libpython2.4.so.1.0`run_err_node+0x1f 232 libpython2.4.so.1.0`PyRun_String+0x27 233 libpython2.4.so.1.0`exec_statement+0x2b0 234 libpython2.4.so.1.0`PyEval_EvalFrame+0x15d6 235 [ /usr/lib/python2.4/site.py:134 (addpackage) ] 236 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 237 libpython2.4.so.1.0`fast_function+0x112 238 libpython2.4.so.1.0`call_function+0xda 239 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 240 [ /usr/lib/python2.4/site.py:166 (addsitedir) ] 241 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 242 libpython2.4.so.1.0`fast_function+0x112 243 libpython2.4.so.1.0`call_function+0xda 244 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 245 [ /usr/lib/python2.4/site.py:203 (addsitepackages) ] 246 libpython2.4.so.1.0`fast_function+0xa8 247 libpython2.4.so.1.0`call_function+0xda 248 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 249 [ /usr/lib/python2.4/site.py:382 (main) ] 250 libpython2.4.so.1.0`fast_function+0xa8 251 libpython2.4.so.1.0`call_function+0xda 252 libpython2.4.so.1.0`PyEval_EvalFrame+0xbdf 253 [ /usr/lib/python2.4/site.py:397 (?) ] 254 libpython2.4.so.1.0`PyEval_EvalCodeEx+0x732 255 libpython2.4.so.1.0`PyEval_EvalCode+0x22 256 libpython2.4.so.1.0`PyImport_ExecCodeModuleEx+0xc0 257 libpython2.4.so.1.0`load_source_module+0xe6 258 libpython2.4.so.1.0`load_module+0xac 259 libpython2.4.so.1.0`import_submodule+0xfb 260 libpython2.4.so.1.0`load_next+0xa2 261 libpython2.4.so.1.0`import_module_ex+0x48 262 libpython2.4.so.1.0`PyImport_ImportModuleEx+0x1d 263 libpython2.4.so.1.0`builtin___import__+0x4e 264 libpython2.4.so.1.0`PyCFunction_Call+0x15f 265 libpython2.4.so.1.0`PyObject_Call+0x1d 266 libpython2.4.so.1.0`PyObject_CallFunction+0x90 267 libpython2.4.so.1.0`PyImport_Import+0x163 268 libpython2.4.so.1.0`PyImport_ImportModule+0x1f 269 libpython2.4.so.1.0`initsite+0x10 270 libpython2.4.so.1.0`Py_InitializeEx+0x1ea 271 libpython2.4.so.1.0`Py_Initialize+0xd 272 libpython2.4.so.1.0`Py_Main+0x4db 273 274 value ------------- Distribution ------------- count 275 131072 | 0 276 262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 277 524288 | 0 278 279 280 libc.so.1`malloc 281 libpython2.4.so.1.0`new_arena+0x13 282 libpython2.4.so.1.0`PyObject_Malloc+0x91 283 libpython2.4.so.1.0`_PyObject_GC_Malloc+0x13 284 libpython2.4.so.1.0`_PyObject_GC_NewVar+0x24 285 libpython2.4.so.1.0`PyTuple_New+0x78 286 libpython2.4.so.1.0`PyType_Ready+0x98 287 libpython2.4.so.1.0`PyType_Ready+0x60 288 libpython2.4.so.1.0`_Py_ReadyTypes+0x10 289 libpython2.4.so.1.0`Py_InitializeEx+0xed 290 libpython2.4.so.1.0`Py_Initialize+0xd 291 libpython2.4.so.1.0`Py_Main+0x4db 292 python`main+0x11 293 python`_start+0x7a 294 295 value ------------- Distribution ------------- count 296 131072 | 0 297 262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 298 524288 | 0 299 300 301This output looks a little unusual at first glance, and can be confusing for 302people unfamiliar with stack tracing and Python engine internals. 303 304Start by looking at the distribution plots below each stack trace - each plot 305shows how many bytes were requested as a histogram by byte size. This should 306indicated to you if python is malloc()ing much memory or not, and whether it 307is doing so in a few large malloc()s or many small ones. 308 309With this information in mind you can inspect the stack traces - these explain 310why Python called malloc() in that instance, along with translations of Python 311functions buried in the stack trace. The stack traces can be hard to read at 312first (or even at second or at third) - since you are examining Python engine 313internals. Try looking for lines in square brackets - those are Python language 314frames, and will show where (or if) the malloc() was caused by Python code. 315