"Fossies" - the Fresh Open Source Software Archive

Member "SAOImageDS9/libxml2/doc/xmlmem.html" (13 Nov 2019, 14441 Bytes) of package /linux/misc/ds9.8.1.tar.gz:


The requested HTML page contains a <FORM> tag that is unusable on "Fossies" in "automatic" (rendered) mode so that page is shown as HTML source code syntax highlighting (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 <?xml version="1.0" encoding="UTF-8"?>
    2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    3 <html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><link rel="SHORTCUT ICON" href="/favicon.ico" /><style type="text/css">
    4 TD {font-family: Verdana,Arial,Helvetica}
    5 BODY {font-family: Verdana,Arial,Helvetica; margin-top: 2em; margin-left: 0em; margin-right: 0em}
    6 H1 {font-family: Verdana,Arial,Helvetica}
    7 H2 {font-family: Verdana,Arial,Helvetica}
    8 H3 {font-family: Verdana,Arial,Helvetica}
    9 A:link, A:visited, A:active { text-decoration: underline }
   10 </style><title>Memory Management</title></head><body bgcolor="#8b7765" text="#000000" link="#a06060" vlink="#000000"><table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr><td width="120"><a href="http://swpat.ffii.org/"><img src="epatents.png" alt="Action against software patents" /></a></td><td width="180"><a href="http://www.gnome.org/"><img src="gnome2.png" alt="Gnome2 Logo" /></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo" /></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo" /></a><div align="left"><a href="http://xmlsoft.org/"><img src="Libxml2-Logo-180x168.gif" alt="Made with Libxml2 Logo" /></a></div></td><td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center"><h1>The XML C parser and toolkit of Gnome</h1><h2>Memory Management</h2></td></tr></table></td></tr></table></td></tr></table><table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr><td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Developer Menu</b></center></td></tr><tr><td bgcolor="#fffacd"><form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><input name="query" type="text" size="20" value="" /><input name="submit" type="submit" value="Search ..." /></form><ul><li><a href="index.html" style="font-weight:bold">Main Menu</a></li><li><a href="html/index.html" style="font-weight:bold">Reference Manual</a></li><li><a href="examples/index.html" style="font-weight:bold">Code Examples</a></li><li><a href="guidelines.html">XML Guidelines</a></li><li><a href="tutorial/index.html">Tutorial</a></li><li><a href="xmlreader.html">The Reader Interface</a></li><li><a href="ChangeLog.html">ChangeLog</a></li><li><a href="XSLT.html">XSLT</a></li><li><a href="python.html">Python and bindings</a></li><li><a href="architecture.html">libxml2 architecture</a></li><li><a href="tree.html">The tree output</a></li><li><a href="interface.html">The SAX interface</a></li><li><a href="xmlmem.html">Memory Management</a></li><li><a href="xmlio.html">I/O Interfaces</a></li><li><a href="library.html">The parser interfaces</a></li><li><a href="entities.html">Entities or no entities</a></li><li><a href="namespaces.html">Namespaces</a></li><li><a href="upgrade.html">Upgrading 1.x code</a></li><li><a href="threads.html">Thread safety</a></li><li><a href="DOM.html">DOM Principles</a></li><li><a href="example.html">A real example</a></li><li><a href="xml.html">flat page</a>, <a href="site.xsl">stylesheet</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>API Indexes</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="APIchunk0.html">Alphabetic</a></li><li><a href="APIconstructors.html">Constructors</a></li><li><a href="APIfunctions.html">Functions/Types</a></li><li><a href="APIfiles.html">Modules</a></li><li><a href="APIsymbols.html">Symbols</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li><li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li><li><a href="http://phd.cs.unibo.it/gdome2/">DOM gdome2</a></li><li><a href="http://www.aleksey.com/xmlsec/">XML-DSig xmlsec</a></li><li><a href="ftp://xmlsoft.org/">FTP</a></li><li><a href="http://www.zlatkovic.com/projects/libxml/">Windows binaries</a></li><li><a href="http://opencsw.org/packages/libxml2">Solaris binaries</a></li><li><a href="http://www.explain.com.au/oss/libxml2xslt.html">MacOsX binaries</a></li><li><a href="http://lxml.de/">lxml Python bindings</a></li><li><a href="http://cpan.uwinnipeg.ca/dist/XML-LibXML">Perl bindings</a></li><li><a href="http://libxmlplusplus.sourceforge.net/">C++ bindings</a></li><li><a href="http://www.zend.com/php5/articles/php5-xmlphp.php#Heading4">PHP bindings</a></li><li><a href="http://sourceforge.net/projects/libxml2-pas/">Pascal bindings</a></li><li><a href="http://libxml.rubyforge.org/">Ruby bindings</a></li><li><a href="http://tclxml.sourceforge.net/">Tcl bindings</a></li><li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">Bug Tracker</a></li></ul></td></tr></table></td></tr></table></td><td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd"><p>Table of Content:</p><ol>
   11   <li><a href="#General3">General overview</a></li>
   12   <li><a href="#setting">Setting libxml2 set of memory routines</a></li>
   13   <li><a href="#cleanup">Cleaning up after using the library</a></li>
   14   <li><a href="#Debugging">Debugging routines</a></li>
   15   <li><a href="#General4">General memory requirements</a></li>
   16   <li><a href="#Compacting">Returning memory to the kernel</a></li>
   17 </ol><h3><a name="General3" id="General3">General overview</a></h3><p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>
   18 provides the interfaces to the libxml2 memory system:</p><ul>
   19   <li>libxml2 does not use the libc memory allocator directly but xmlFree(),
   20     xmlMalloc() and xmlRealloc()</li>
   21   <li>those routines can be reallocated to a specific set of routine, by
   22     default the libc ones i.e. free(), malloc() and realloc()</li>
   23   <li>the xmlmemory.c module includes a set of debugging routine</li>
   24 </ul><h3><a name="setting" id="setting">Setting libxml2 set of memory routines</a></h3><p>It is sometimes useful to not use the default memory allocator, either for
   25 debugging, analysis or to implement a specific behaviour on memory management
   26 (like on embedded systems). Two function calls are available to do so:</p><ul>
   27   <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet
   28     ()</a> which return the current set of functions in use by the parser</li>
   29   <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>
   30     which allow to set up a new set of memory allocation functions</li>
   31 </ul><p>Of course a call to xmlMemSetup() should probably be done before calling
   32 any other libxml2 routines (unless you are sure your allocations routines are
   33 compatibles).</p><h3><a name="cleanup" id="cleanup">Cleaning up after using the library</a></h3><p>Libxml2 is not stateless, there is a few set of memory structures needing
   34 allocation before the parser is fully functional (some encoding structures
   35 for example). This also mean that once parsing is finished there is a tiny
   36 amount of memory (a few hundred bytes) which can be recollected if you don't
   37 reuse the library or any document built with it:</p><ul>
   38   <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser
   39     ()</a> is a centralized routine to free the library state and data. Note
   40     that it won't deallocate any produced tree if any (use the xmlFreeDoc()
   41     and related routines for this). This should be called only when the library
   42     is not used anymore.</li>
   43   <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser
   44     ()</a> is the dual routine allowing to preallocate the parsing state
   45     which can be useful for example to avoid initialization reentrancy
   46     problems when using libxml2 in multithreaded applications</li>
   47 </ul><p>Generally xmlCleanupParser() is safe assuming no parsing is ongoing and
   48 no document is still being used, if needed the state will be rebuild at the
   49 next invocation of parser routines (or by xmlInitParser()), but be careful
   50 of the consequences in multithreaded applications.</p><h3><a name="Debugging" id="Debugging">Debugging routines</a></h3><p>When configured using --with-mem-debug flag (off by default), libxml2 uses
   51 a set of memory allocation debugging routines keeping track of all allocated
   52 blocks and the location in the code where the routine was called. A couple of
   53 other debugging routines allow to dump the memory allocated infos to a file
   54 or call a specific routine when a given block number is allocated:</p><ul>
   55   <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a>
   56     <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>
   57     and <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>
   58     are the memory debugging replacement allocation routines</li>
   59   <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump
   60     ()</a> dumps all the information about the allocated memory block lefts
   61     in the <code>.memdump</code> file</li>
   62 </ul><p>When developing libxml2 memory debug is enabled, the tests programs call
   63 xmlMemoryDump () and the "make test" regression tests will check for any
   64 memory leak during the full regression test sequence, this helps a lot
   65 ensuring that libxml2  does not leak memory and bullet proof memory
   66 allocations use (some libc implementations are known to be far too permissive
   67 resulting in major portability problems!).</p><p>If the .memdump reports a leak, it displays the allocation function and
   68 also tries to give some information about the content and structure of the
   69 allocated blocks left. This is sufficient in most cases to find the culprit,
   70 but not always. Assuming the allocation problem is reproducible, it is
   71 possible to find more easily:</p><ol>
   72   <li>write down the block number xxxx not allocated</li>
   73   <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , the easiest
   74     when using GDB is to simply give the command
   75     <p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p>
   76     <p>before running the program.</p>
   77   </li>
   78   <li>run the program under a debugger and set a breakpoint on
   79     xmlMallocBreakpoint() a specific function called when this precise block
   80     is allocated</li>
   81   <li>when the breakpoint is reached you can then do a fine analysis of the
   82     allocation an step  to see the condition resulting in the missing
   83     deallocation.</li>
   84 </ol><p>I used to use a commercial tool to debug libxml2 memory problems but after
   85 noticing that it was not detecting memory leaks that simple mechanism was
   86 used and proved extremely efficient until now. Lately I have also used <a href="http://developer.kde.org/~sewardj/">valgrind</a> with quite some
   87 success, it is tied to the i386 architecture since it works by emulating the
   88 processor and instruction set, it is slow but  extremely efficient, i.e. it
   89 spot memory usage errors in a very precise way.</p><h3><a name="General4" id="General4">General memory requirements</a></h3><p>How much libxml2 memory require ? It's hard to tell in average it depends
   90 of a number of things:</p><ul>
   91   <li>the parser itself should work  in a fixed amount of memory, except for
   92     information maintained about the stacks of names and  entities locations.
   93     The I/O and encoding handlers will probably account for a few KBytes.
   94     This is true for both the XML and HTML parser (though the HTML parser
   95     need more state).</li>
   96   <li>If you are generating the DOM tree then memory requirements will grow
   97     nearly linear with the size of the data. In general for a balanced
   98     textual document the internal memory requirement is about 4 times the
   99     size of the UTF8 serialization of this document (example the XML-1.0
  100     recommendation is a bit more of 150KBytes and takes 650KBytes of main
  101     memory when parsed). Validation will add a amount of memory required for
  102     maintaining the external Dtd state which should be linear with the
  103     complexity of the content model defined by the Dtd</li>
  104   <li>If you need to work with fixed memory requirements or don't need the
  105     full DOM tree then using the <a href="xmlreader.html">xmlReader
  106     interface</a> is probably the best way to proceed, it still allows to
  107     validate or operate on subset of the tree if needed.</li>
  108   <li>If you don't care about the advanced features of libxml2 like
  109     validation, DOM, XPath or XPointer, don't use entities, need to work with
  110     fixed memory requirements, and try to get the fastest parsing possible
  111     then the SAX interface should be used, but it has known restrictions.</li>
  112 </ul><p></p><h3><a name="Compacting" id="Compacting">Returning memory to the kernel</a></h3><p>You may encounter that your process using libxml2 does not have a
  113 reduced memory usage although you freed the trees. This is because
  114 libxml2 allocates memory in a number of small chunks. When freeing one
  115 of those chunks, the OS may decide that giving this little memory back
  116 to the kernel will cause too much overhead and delay the operation. As
  117 all chunks are this small, they get actually freed but not returned to
  118 the kernel. On systems using glibc, there is a function call
  119 "malloc_trim" from malloc.h which does this missing operation (note that
  120 it is allowed to fail). Thus, after freeing your tree you may simply try
  121 "malloc_trim(0);" to really get the memory back. If your OS does not
  122 provide malloc_trim, try searching for a similar function.</p><p></p><p><a href="bugs.html">Daniel Veillard</a></p></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td></tr></table></body></html>