This is Info file /home/pdm/tmp/Python-1.5.2p1/Doc/lib/python-lib.info, produced by Makeinfo version 1.68 from the input file lib.texi. July 6, 1999 1.5.2  File: python-lib.info, Node: sys, Next: types, Prev: Python Services, Up: Python Services System-specific parameters and functions ======================================== Access system-specific parameters and functions. This module provides access to some variables used or maintained by the interpreter and to functions that interact strongly with the interpreter. It is always available. `argv' The list of command line arguments passed to a Python script. `argv[0]' is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the `-c' command line option to the interpreter, `argv[0]' is set to the string `'-c''. If no script name was passed to the Python interpreter, `argv' has zero length. `builtin_module_names' A tuple of strings giving the names of all modules that are compiled into this Python interpreter. (This information is not available in any other way -- `modules.keys()' only lists the imported modules.) `copyright' A string containing the copyright pertaining to the Python interpreter. `exc_info()' This function returns a tuple of three values that give information about the exception that is currently being handled. The information returned is specific both to the current thread and to the current stack frame. If the current stack frame is not handling an exception, the information is taken from the calling stack frame, or its caller, and so on until a stack frame is found that is handling an exception. Here, "handling an exception" is defined as "executing or having executed an except clause." For any stack frame, only information about the most recently handled exception is accessible. If no exception is being handled anywhere on the stack, a tuple containing three `None' values is returned. Otherwise, the values returned are `(TYPE, VALUE, TRACEBACK)'. Their meaning is: TYPE gets the exception type of the exception being handled (a string or class object); VALUE gets the exception parameter (its "associated value" or the second argument to `raise', which is always a class instance if the exception type is a class object); TRACEBACK gets a traceback object (see the Reference Manual) which encapsulates the call stack at the point where the exception originally occurred. *Warning:* assigning the TRACEBACK return value to a local variable in a function that is handling an exception will cause a circular reference. This will prevent anything referenced by a local variable in the same function or by the traceback from being garbage collected. Since most functions don't need access to the traceback, the best solution is to use something like `type, value = sys.exc_info()[:2]' to extract only the exception type and value. If you do need the traceback, make sure to delete it after use (best done with a `try' ... `finally' statement) or to call `exc_info()' in a function that does not itself handle an exception. `exc_type' `exc_value' `exc_traceback' *This is deprecated in Python 1.5. Use `exc_info()' instead.* Since they are global variables, they are not specific to the current thread, so their use is not safe in a multi-threaded program. When no exception is being handled, `exc_type' is set to `None' and the other two are undefined. `exec_prefix' A string giving the site-specific directory prefix where the platform-dependent Python files are installed; by default, this is also `'/usr/local''. This can be set at build time with the `-'`-exec-prefix' argument to the `configure' script. Specifically, all configuration files (e.g. the `config.h' header file) are installed in the directory `exec_prefix + '/lib/pythonVERSION/config'', and shared library modules are installed in `exec_prefix + '/lib/pythonVERSION/lib-dynload'', where VERSION is equal to `version[:3]'. `executable' A string giving the name of the executable binary for the Python interpreter, on systems where this makes sense. `exit([arg])' Exit from Python. This is implemented by raising the `SystemExit' exception, so cleanup actions specified by finally clauses of `try' statements are honored, and it is possible to intercept the exit attempt at an outer level. The optional argument ARG can be an integer giving the exit status (defaulting to zero), or another type of object. If it is an integer, zero is considered "successful termination" and any nonzero value is considered "abnormal termination" by shells and the like. Most systems require it to be in the range 0-127, and produce undefined results otherwise. Some systems have a convention for assigning specific meanings to specific exit codes, but these are generally underdeveloped; Unix programs generally use 2 for command line syntax errors and 1 for all other kind of errors. If another type of object is passed, `None' is equivalent to passing zero, and any other object is printed to `sys.stderr' and results in an exit code of 1. In particular, `sys.exit("some error message")' is a quick way to exit a program when an error occurs. `exitfunc' This value is not actually defined by the module, but can be set by the user (or by a program) to specify a clean-up action at program exit. When set, it should be a parameterless function. This function will be called when the interpreter exits. Note: the exit function is not called when the program is killed by a signal, when a Python fatal internal error is detected, or when `os._exit()' is called. `getrefcount(object)' Return the reference count of the OBJECT. The count returned is generally one higher than you might expect, because it includes the (temporary) reference as an argument to `getrefcount()'. `last_type' `last_value' `last_traceback' These three variables are not always defined; they are set when an exception is not handled and the interpreter prints an error message and a stack traceback. Their intended use is to allow an interactive user to import a debugger module and engage in post-mortem debugging without having to re-execute the command that caused the error. (Typical use is `import pdb; pdb.pm()' to enter the post-mortem debugger; see the chapter "The Python Debugger" for more information.) The meaning of the variables is the same as that of the return values from `exc_info()' above. (Since there is only one interactive thread, thread-safety is not a concern for these variables, unlike for `exc_type' etc.) `maxint' The largest positive integer supported by Python's regular integer type. This is at least 2**31-1. The largest negative integer is `-maxint-1' - the asymmetry results from the use of 2's complement binary arithmetic. `modules' This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks. Note that removing a module from this dictionary is *not* the same as calling `reload()' on the corresponding module object. `path' A list of strings that specifies the search path for modules. Initialized from the environment variable `PYTHONPATH', or an installation-dependent default. The first item of this list, `path[0]', is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), `path[0]' is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted *before* the entries inserted as a result of `PYTHONPATH'. `platform' This string contains a platform identifier, e.g. `'sunos5'' or `'linux1''. This can be used to append platform-specific components to `path', for instance. `prefix' A string giving the site-specific directory prefix where the platform independent Python files are installed; by default, this is the string `'/usr/local''. This can be set at build time with the `-'`-prefix' argument to the `configure' script. The main collection of Python library modules is installed in the directory `prefix + '/lib/pythonVERSION'' while the platform independent header files (all except `config.h') are stored in `prefix + '/include/pythonVERSION'', where VERSION is equal to `version[:3]'. `ps1' `ps2' Strings specifying the primary and secondary prompt of the interpreter. These are only defined if the interpreter is in interactive mode. Their initial values in this case are `'>>> '' and `'... ''. If a non-string object is assigned to either variable, its `str()' is re-evaluated each time the interpreter prepares to read a new interactive command; this can be used to implement a dynamic prompt. `setcheckinterval(interval)' Set the interpreter's "check interval". This integer value determines how often the interpreter checks for periodic things such as thread switches and signal handlers. The default is `10', meaning the check is performed every 10 Python virtual instructions. Setting it to a larger value may increase performance for programs using threads. Setting it to a value `<=' 0 checks every virtual instruction, maximizing responsiveness as well as overhead. `setprofile(profilefunc)' Set the system's profile function, which allows you to implement a Python source code profiler in Python. See the chapter on the Python Profiler. The system's profile function is called similarly to the system's trace function (see `settrace()'), but it isn't called for each executed line of code (only on call and return and when an exception occurs). Also, its return value is not used, so it can just return `None'. `settrace(tracefunc)' Set the system's trace function, which allows you to implement a Python source code debugger in Python. See section "How It Works" in the chapter on the Python Debugger. `stdin' `stdout' `stderr' File objects corresponding to the interpreter's standard input, output and error streams. `stdin' is used for all interpreter input except for scripts but including calls to `input()' and `raw_input()'. `stdout' is used for the output of `print' and expression statements and for the prompts of `input()' and `raw_input()'. The interpreter's own prompts and (almost all of) its error messages go to `stderr'. `stdout' and `stderr' needn't be built-in file objects: any object is acceptable as long as it has a `write()' method that takes a string argument. (Changing these objects doesn't affect the standard I/O streams of processes executed by `os.popen()', `os.system()' or the `exec*()' family of functions in the `os' module.) `__stdin__' `__stdout__' `__stderr__' These objects contain the original values of `stdin', `stderr' and `stdout' at the start of the program. They are used during finalization, and could be useful to restore the actual files to known working file objects in case they have been overwritten with a broken object. `tracebacklimit' When this variable is set to an integer value, it determines the maximum number of levels of traceback information printed when an unhandled exception occurs. The default is `1000'. When set to 0 or less, all traceback information is suppressed and only the exception type and value are printed. `version' A string containing the version number of the Python interpreter.  File: python-lib.info, Node: types, Next: UserDict, Prev: sys, Up: Python Services Names for all built-in types. ============================= Names for all built-in types. This module defines names for all object types that are used by the standard Python interpreter, but not for the types defined by various extension modules. It is safe to use `from types import *' -- the module does not export any names besides the ones listed here. New names exported by future versions of this module will all end in `Type'. Typical use is for functions that do different things depending on their argument types, like the following: from types import * def delete(list, item): if type(item) is IntType: del list[item] else: list.remove(item) The module defines the following names: `NoneType' The type of `None'. `TypeType' The type of type objects (such as returned by `type()'). `IntType' The type of integers (e.g. `1'). `LongType' The type of long integers (e.g. `1L'). `FloatType' The type of floating point numbers (e.g. `1.0'). `ComplexType' The type of complex numbers (e.g. `1.0j'). `StringType' The type of character strings (e.g. `'Spam''). `TupleType' The type of tuples (e.g. `(1, 2, 3, 'Spam')'). `ListType' The type of lists (e.g. `[0, 1, 2, 3]'). `DictType' The type of dictionaries (e.g. `{'Bacon': 1, 'Ham': 0}'). `DictionaryType' An alternate name for `DictType'. `FunctionType' The type of user-defined functions and lambdas. `LambdaType' An alternate name for `FunctionType'. `CodeType' The type for code objects such as returned by `compile()'. `ClassType' The type of user-defined classes. `InstanceType' The type of instances of user-defined classes. `MethodType' The type of methods of user-defined class instances. `UnboundMethodType' An alternate name for `MethodType'. `BuiltinFunctionType' The type of built-in functions like `len()' or `sys.exit()'. `BuiltinMethodType' An alternate name for `BuiltinFunction'. `ModuleType' The type of modules. `FileType' The type of open file objects such as `sys.stdout'. `XRangeType' The type of range objects returned by `xrange()'. `SliceType' The type of objects returned by `slice()'. `EllipsisType' The type of `Ellipsis'. `TracebackType' The type of traceback objects such as found in `sys.exc_traceback'. `FrameType' The type of frame objects such as found in `tb.tb_frame' if `tb' is a traceback object.  File: python-lib.info, Node: UserDict, Next: UserList, Prev: types, Up: Python Services Class wrapper for dictionary objects ==================================== Class wrapper for dictionary objects. This module defines a class that acts as a wrapper around dictionary objects. It is a useful base class for your own dictionary-like classes, which can inherit from them and override existing methods or add new ones. In this way one can add new behaviours to dictionaries. The `UserDict' module defines the `UserDict' class: `UserDict()' Return a class instance that simulates a dictionary. The instance's contents are kept in a regular dictionary, which is accessible via the `data' attribute of `UserDict' instances. In addition to supporting the methods and operations of mappings (see section *Note Mapping Types::), `UserDict' instances provide the following attribute: `data' A real dictionary used to store the contents of the `UserDict' class.  File: python-lib.info, Node: UserList, Next: operator, Prev: UserDict, Up: Python Services Class wrapper for list objects ============================== Class wrapper for list objects. This module defines a class that acts as a wrapper around list objects. It is a useful base class for your own list-like classes, which can inherit from them and override existing methods or add new ones. In this way one can add new behaviours to lists. The `UserList' module defines the `UserList' class: `UserList([list])' Return a class instance that simulates a list. The instance's contents are kept in a regular list, which is accessible via the `data' attribute of `UserList' instances. The instance's contents are initially set to a copy of LIST, defaulting to the empty list `[]'. LIST can be either a regular Python list, or an instance of `UserList' (or a subclass). In addition to supporting the methods and operations of mutable sequences (see section *Note Sequence Types::), `UserList' instances provide the following attribute: `data' A real Python list object used to store the contents of the `UserList' class.  File: python-lib.info, Node: operator, Next: traceback, Prev: UserList, Up: Python Services Standard operators as functions. ================================ This section was written by Skip Montanaro . All Python's standard operators as built-in functions. The `operator' module exports a set of functions implemented in C corresponding to the intrinsic operators of Python. For example, `operator.add(x, y)' is equivalent to the expression `x+y'. The function names are those used for special class methods; variants without leading and trailing `__' are also provided for convenience. The `operator' module defines the following functions: `add(a, b)' `__add__ a, b' Return A `+' B, for A and B numbers. `sub(a, b)' `__sub__ a, b' Return A `-' B. `mul(a, b)' `__mul__ a, b' Return A `*' B, for A and B numbers. `div(a, b)' `__div__ a, b' Return A `/' B. `mod(a, b)' `__mod__ a, b' Return A `%' B. `neg(o)' `__neg__ o' Return O negated. `pos(o)' `__pos__ o' Return O positive. `abs(o)' `__abs__ o' Return the absolute value of O. `inv(o)' `__inv__ o' Return the inverse of O. `lshift(a, b)' `__lshift__ a, b' Return A shifted left by B. `rshift(a, b)' `__rshift__ a, b' Return A shifted right by B. `and_(a, b)' `__and__ a, b' Return the bitwise and of A and B. `or_(a, b)' `__or__ a, b' Return the bitwise or of A and B. `xor(a, b)' `__xor__ a, b' Return the bitwise exclusive or of A and B. `not_(o)' `__not__ o' Return the outcome of `not' O. (Note that there is no `__not__()' discipline for object instances; only the interpreter core defines this operation.) `truth(o)' Return `1' if O is true, and 0 otherwise. `concat(a, b)' `__concat__ a, b' Return A `+' B for A and B sequences. `repeat(a, b)' `__repeat__ a, b' Return A `*' B where A is a sequence and B is an integer. `contains(a, b)' `sequenceIncludes a, b' Return the outcome of the test B `in' A. Note the reversed operands. `countOf(a, b)' Return the number of occurrences of B in A. `indexOf(a, b)' Return the index of the first of occurrence of B in A. `getitem(a, b)' `__getitem__ a, b' Return the value of A at index B. `setitem(a, b, c)' `__setitem__ a, b, c' Set the value of A at index B to C. `delitem(a, b)' `__delitem__ a, b' Remove the value of A at index B. `getslice(a, b, c)' `__getslice__ a, b, c' Return the slice of A from index B to index C`-1'. `setslice(a, b, c, v)' `__setslice__ a, b, c, v' Set the slice of A from index B to index C`-1' to the sequence V. `delslice(a, b, c)' `__delslice__ a, b, c' Delete the slice of A from index B to index C`-1'. Example: Build a dictionary that maps the ordinals from `0' to `256' to their character equivalents. >>> import operator >>> d = {} >>> keys = range(256) >>> vals = map(chr, keys) >>> map(operator.setitem, [d]*len(keys), keys, vals)  File: python-lib.info, Node: traceback, Next: linecache, Prev: operator, Up: Python Services Print or retrieve a stack traceback =================================== Print or retrieve a stack traceback. This module provides a standard interface to extract, format and print stack traces of Python programs. It exactly mimics the behavior of the Python interpreter when it prints a stack trace. This is useful when you want to print stack traces under program control, e.g. in a "wrapper" around the interpreter. The module uses traceback objects -- this is the object type that is stored in the variables `sys.exc_traceback' and `sys.last_traceback' and returned as the third item from `sys.exc_info()'. The module defines the following functions: `print_tb(traceback[, limit[, file]])' Print up to LIMIT stack trace entries from TRACEBACK. If LIMIT is omitted or `None', all entries are printed. If FILE is omitted or `None', the output goes to `sys.stderr'; otherwise it should be an open file or file-like object to receive the output. `print_exception(type, value, traceback[, limit[, file]])' Print exception information and up to LIMIT stack trace entries from TRACEBACK to FILE. This differs from `print_tb()' in the following ways: (1) if TRACEBACK is not `None', it prints a header `Traceback (innermost last):'; (2) it prints the exception TYPE and VALUE after the stack trace; (3) if TYPE is `SyntaxError' and VALUE has the appropriate format, it prints the line where the syntax error occurred with a caret indicating the approximate position of the error. `print_exc([limit[, file]])' This is a shorthand for ``print_exception(sys.exc_type,' `sys.exc_value,' `sys.exc_traceback,' LIMIT`,' FILE`)''. (In fact, it uses `sys.exc_info()' to retrieve the same information in a thread-safe way.) `print_last([limit[, file]])' This is a shorthand for ``print_exception(sys.last_type,' `sys.last_value,' `sys.last_traceback,' LIMIT`,' FILE`)''. `print_stack([f[, limit[, file]]])' This function prints a stack trace from its invocation point. The optional F argument can be used to specify an alternate stack frame to start. The optional LIMIT and FILE arguments have the same meaning as for `print_exception()'. `extract_tb(traceback[, limit])' Return a list of up to LIMIT "pre-processed" stack trace entries extracted from the traceback object TRACEBACK. It is useful for alternate formatting of stack traces. If LIMIT is omitted or `None', all entries are extracted. A "pre-processed" stack trace entry is a quadruple (FILENAME, LINE NUMBER, FUNCTION NAME, TEXT) representing the information that is usually printed for a stack trace. The TEXT is a string with leading and trailing whitespace stripped; if the source is not available it is `None'. `extract_stack([f[, limit]])' Extract the raw traceback from the current stack frame. The return value has the same format as for `extract_tb()'. The optional F and LIMIT arguments have the same meaning as for `print_stack()'. `format_list(list)' Given a list of tuples as returned by `extract_tb()' or `extract_stack()', return a list of strings ready for printing. Each string in the resulting list corresponds to the item with the same index in the argument list. Each string ends in a newline; the strings may contain internal newlines as well, for those items whose source text line is not `None'. `format_exception_only(type, value)' Format the exception part of a traceback. The arguments are the exception type and value such as given by `sys.last_type' and `sys.last_value'. The return value is a list of strings, each ending in a newline. Normally, the list contains a single string; however, for `SyntaxError' exceptions, it contains several lines that (when printed) display detailed information about where the syntax error occurred. The message indicating which exception occurred is the always last string in the list. `format_exception(type, value, tb[, limit])' Format a stack trace and the exception information. The arguments have the same meaning as the corresponding arguments to `print_exception()'. The return value is a list of strings, each ending in a newline and some containing internal newlines. When these lines are contatenated and printed, exactly the same text is printed as does `print_exception()'. `format_tb(tb[, limit])' A shorthand for `format_list(extract_tb(TB, LIMIT))'. `format_stack([f[, limit]])' A shorthand for `format_list(extract_stack(F, LIMIT))'. `tb_lineno(tb)' This function returns the current line number set in the traceback object. This is normally the same as the `TB.tb_lineno' field of the object, but when optimization is used (the -O flag) this field is not updated correctly; this function calculates the correct value. * Menu: * Traceback Example::  File: python-lib.info, Node: Traceback Example, Prev: traceback, Up: traceback Traceback Example ----------------- This simple example implements a basic read-eval-print loop, similar to (but less useful than) the standard Python interactive interpreter loop. For a more complete implementation of the interpreter loop, refer to the `code' module. import sys, traceback def run_user_code(envdir): source = raw_input(">>> ") try: exec source in envdir except: print "Exception in user code:" print '-'*60 traceback.print_exc(file=sys.stdout) print '-'*60 envdir = {} while 1: run_user_code(envdir)  File: python-lib.info, Node: linecache, Next: pickle, Prev: traceback, Up: Python Services Random access to text lines =========================== This section was written by Moshe Zadka . This module provides random access to individual lines from text files. The `linecache' module allows one to get any line from any file, while attempting to optimize internally, using a cache, the common case where many lines are read from a single file. This is used by the `traceback' module to retrieve source lines for inclusion in the formatted traceback. The `linecache' module defines the following functions: `getline(filename, lineno)' Get line LINENO from file named FILENAME. This function will never throw an exception -- it will return `''' on errors. If a file named FILENAME is not found, the function will look for it in the module search path, `sys.path'. `clearcache()' Clear the cache. Use this function if you know that you do not need to read lines from many of files you already read from using this module. `checkcache()' Check the cache for validity. Use this function if files in the cache may have changed on disk, and you require the updated version. Example: >>> import linecache >>> linecache.getline('/etc/passwd', 4) 'sys:x:3:3:sys:/dev:/bin/sh\012'  File: python-lib.info, Node: pickle, Next: cPickle, Prev: linecache, Up: Python Services Python object serialization =========================== Convert Python objects to streams of bytes and back. The `pickle' module implements a basic but powerful algorithm for "pickling" (a.k.a. serializing, marshalling or flattening) nearly arbitrary Python objects. This is the act of converting objects to a stream of bytes (and back: "unpickling"). This is a more primitive notion than persistency -- although `pickle' reads and writes file objects, it does not handle the issue of naming persistent objects, nor the (even more complicated) area of concurrent access to persistent objects. The `pickle' module can transform a complex object into a byte stream and it can transform the byte stream into an object with the same internal structure. The most obvious thing to do with these byte streams is to write them onto a file, but it is also conceivable to send them across a network or store them in a database. The module `shelve' provides a simple interface to pickle and unpickle objects on DBM-style database files. *Note:* The `pickle' module is rather slow. A reimplementation of the same algorithm in C, which is up to 1000 times faster, is available as the `cPickle' module. This has the same interface except that `Pickler' and `Unpickler' are factory functions, not classes (so they cannot be used as base classes for inheritance). Unlike the built-in module `marshal', `pickle' handles the following correctly: * recursive objects (objects containing references to themselves) * object sharing (references to the same object in different places) * user-defined classes and their instances The data format used by `pickle' is Python-specific. This has the advantage that there are no restrictions imposed by external standards such as XDR (which can't represent pointer sharing); however it means that non-Python programs may not be able to reconstruct pickled Python objects. By default, the `pickle' data format uses a printable ASCII representation. This is slightly more voluminous than a binary representation. The big advantage of using printable ASCII (and of some other characteristics of `pickle''s representation) is that for debugging or recovery purposes it is possible for a human to read the pickled file with a standard text editor. A binary format, which is slightly more efficient, can be chosen by specifying a nonzero (true) value for the BIN argument to the `Pickler' constructor or the `dump()' and `dumps()' functions. The binary format is not the default because of backwards compatibility with the Python 1.4 pickle module. In a future version, the default may change to binary. The `pickle' module doesn't handle code objects, which the `marshal' module does. I suppose `pickle' could, and maybe it should, but there's probably no great need for it right now (as long as `marshal' continues to be used for reading and writing code objects), and at least this avoids the possibility of smuggling Trojan horses into a program. For the benefit of persistency modules written using `pickle', it supports the notion of a reference to an object outside the pickled data stream. Such objects are referenced by a name, which is an arbitrary string of printable ASCII characters. The resolution of such names is not defined by the `pickle' module -- the persistent object module will have to implement a method `persistent_load()'. To write references to persistent objects, the persistent module must define a method `persistent_id()' which returns either `None' or the persistent ID of the object. There are some restrictions on the pickling of class instances. First of all, the class must be defined at the top level in a module. Furthermore, all its instance variables must be picklable. When a pickled class instance is unpickled, its `__init__()' method is normally *not* invoked. *Note:* This is a deviation from previous versions of this module; the change was introduced in Python 1.5b2. The reason for the change is that in many cases it is desirable to have a constructor that requires arguments; it is a (minor) nuisance to have to provide a `__getinitargs__()' method. If it is desirable that the `__init__()' method be called on unpickling, a class can define a method `__getinitargs__()', which should return a *tuple* containing the arguments to be passed to the class constructor (`__init__()'). This method is called at pickle time; the tuple it returns is incorporated in the pickle for the instance. Classes can further influence how their instances are pickled -- if the class defines the method `__getstate__()', it is called and the return state is pickled as the contents for the instance, and if the class defines the method `__setstate__()', it is called with the unpickled state. (Note that these methods can also be used to implement copying class instances.) If there is no `__getstate__()' method, the instance's `__dict__' is pickled. If there is no `__setstate__()' method, the pickled object must be a dictionary and its items are assigned to the new instance's dictionary. (If a class defines both `__getstate__()' and `__setstate__()', the state object needn't be a dictionary -- these methods can do what they want.) This protocol is also used by the shallow and deep copying operations defined in the `copy' module. Note that when class instances are pickled, their class's code and data are not pickled along with them. Only the instance data are pickled. This is done on purpose, so you can fix bugs in a class or add methods and still load objects that were created with an earlier version of the class. If you plan to have long-lived objects that will see many versions of a class, it may be worthwhile to put a version number in the objects so that suitable conversions can be made by the class's `__setstate__()' method. When a class itself is pickled, only its name is pickled -- the class definition is not pickled, but re-imported by the unpickling process. Therefore, the restriction that the class must be defined at the top level in a module applies to pickled classes as well. The interface can be summarized as follows. To pickle an object `x' onto a file `f', open for writing: p = pickle.Pickler(f) p.dump(x) A shorthand for this is: pickle.dump(x, f) To unpickle an object `x' from a file `f', open for reading: u = pickle.Unpickler(f) x = u.load() A shorthand is: x = pickle.load(f) The `Pickler' class only calls the method `f.write()' with a string argument. The `Unpickler' calls the methods `f.read()' (with an integer argument) and `f.readline()' (without argument), both returning a string. It is explicitly allowed to pass non-file objects here, as long as they have the right methods. The constructor for the `Pickler' class has an optional second argument, BIN. If this is present and nonzero, the binary pickle format is used; if it is zero or absent, the (less efficient, but backwards compatible) text pickle format is used. The `Unpickler' class does not have an argument to distinguish between binary and text pickle formats; it accepts either format. The following types can be pickled: * `None' * integers, long integers, floating point numbers * strings * tuples, lists and dictionaries containing only picklable objects * classes that are defined at the top level in a module * instances of such classes whose `__dict__' or `__setstate__()' is picklable Attempts to pickle unpicklable objects will raise the `PicklingError' exception; when this happens, an unspecified number of bytes may have been written to the file. It is possible to make multiple calls to the `dump()' method of the same `Pickler' instance. These must then be matched to the same number of calls to the `load()' method of the corresponding `Unpickler' instance. If the same object is pickled by multiple `dump()' calls, the `load()' will all yield references to the same object. *Warning*: this is intended for pickling multiple objects without intervening modifications to the objects or their parts. If you modify an object and then pickle it again using the same `Pickler' instance, the object is not pickled again -- a reference to it is pickled and the `Unpickler' will return the old value, not the modified one. (There are two problems here: (a) detecting changes, and (b) marshalling a minimal set of changes. I have no answers. Garbage Collection may also become a problem here.) Apart from the `Pickler' and `Unpickler' classes, the module defines the following functions, and an exception: `dump(object, file[, bin])' Write a pickled representation of OBECT to the open file object FILE. This is equivalent to `Pickler(FILE, BIN).dump(OBJECT)'. If the optional BIN argument is present and nonzero, the binary pickle format is used; if it is zero or absent, the (less efficient) text pickle format is used. `load(file)' Read a pickled object from the open file object FILE. This is equivalent to `Unpickler(FILE).load()'. `dumps(object[, bin])' Return the pickled representation of the object as a string, instead of writing it to a file. If the optional BIN argument is present and nonzero, the binary pickle format is used; if it is zero or absent, the (less efficient) text pickle format is used. `loads(string)' Read a pickled object from a string instead of a file. Characters in the string past the pickled object's representation are ignored. `PicklingError' This exception is raised when an unpicklable object is passed to `Pickler.dump()'. See also: *Note copy_reg:: pickle interface constructor registration *Note shelve:: indexed databases of objects; uses `pickle' *Note copy:: shallow and deep object copying *Note marshal:: high-performance serialization of built-in types  File: python-lib.info, Node: cPickle, Next: copy_reg, Prev: pickle, Up: Python Services Alternate implementation of `pickle' ==================================== Faster version of `pickle', but not subclassable. This module was documented by Jim Fulton . This section was written by Fred L. Drake, Jr. . The `cPickle' module provides a similar interface and identical functionality as the `pickle' module, but can be up to 1000 times faster since it is implemented in C. The only other important difference to note is that `Pickler()' and `Unpickler()' are functions and not classes, and so cannot be subclassed. This should not be an issue in most cases. The format of the pickle data is identical to that produced using the `pickle' module, so it is possible to use `pickle' and `cPickle' interchangably with existing pickles. (Since the pickle data format is actually a tiny stack-oriented programming language, and there are some freedoms in the encodings of certain objects, it's possible that the two modules produce different pickled data for the same input objects; however they will always be able to read each others pickles back in.)  File: python-lib.info, Node: copy_reg, Next: shelve, Prev: cPickle, Up: Python Services Register `pickle' support functions =================================== Register `pickle' support functions. The `copy_reg' module provides support for the `pickle' and `cPickle' modules. The `copy' module is likely to use this in the future as well. It provides configuration information about object constructors which are not classes. Such constructors may be factory functions or class instances. `constructor(object)' Declares OBJECT to be a valid constructor. `pickle(type, function[, constructor])' Declares that FUNCTION should be used as a "reduction" function for objects of type or class TYPE. FUNCTION should return either a string or a tuple. The optional CONSTRUCTOR parameter, if provided, is a callable object which can be used to reconstruct the object when called with the tuple of arguments returned by FUNCTION at pickling time.  File: python-lib.info, Node: shelve, Next: copy, Prev: copy_reg, Up: Python Services Python object persistency ========================= Python object persistency. A "shelf" is a persistent, dictionary-like object. The difference with "dbm" databases is that the values (not the keys!) in a shelf can be essentially arbitrary Python objects -- anything that the `pickle' module can handle. This includes most class instances, recursive data types, and objects containing lots of shared sub-objects. The keys are ordinary strings. To summarize the interface (`key' is a string, `data' is an arbitrary object): import shelve d = shelve.open(filename) # open, with (g)dbm filename -- no suffix d[key] = data # store data at key (overwrites old data if # using an existing key) data = d[key] # retrieve data at key (raise KeyError if no # such key) del d[key] # delete data stored at key (raises KeyError # if no such key) flag = d.has_key(key) # true if the key exists list = d.keys() # a list of all existing keys (slow!) d.close() # close it Restrictions: * The choice of which database package will be used (e.g. `dbm' or `gdbm') depends on which interface is available. Therefore it is not safe to open the database directly using `dbm'. The database is also (unfortunately) subject to the limitations of `dbm', if it is used -- this means that (the pickled representation of) the objects stored in the database should be fairly small, and in rare cases key collisions may cause the database to refuse updates. * Dependent on the implementation, closing a persistent dictionary may or may not be necessary to flush changes to disk. * The `shelve' module does not support *concurrent* read/write access to shelved objects. (Multiple simultaneous read accesses are safe.) When a program has a shelf open for writing, no other program should have it open for reading or writing. UNIX file locking can be used to solve this, but this differs across UNIX versions and requires knowledge about the database implementation used. See also: *Note anydbm:: Generic interface to `dbm'-style databases. *Note dbhash:: BSD `db' database interface. *Note dbm:: Standard UNIX database interface. *Note dumbdbm:: Portable implementation of the `dbm' interface. *Note gdbm:: GNU database interface, based on the `dbm' interface. *Note pickle:: Object serialization used by `shelve'. *Note cPickle:: High-performance version of `pickle'.  File: python-lib.info, Node: copy, Next: marshal, Prev: shelve, Up: Python Services Shallow and deep copy operations ================================ Shallow and deep copy operations. This module provides generic (shallow and deep) copying operations. Interface summary: import copy x = copy.copy(y) # make a shallow copy of y x = copy.deepcopy(y) # make a deep copy of y For module specific errors, `copy.error' is raised. The difference between shallow and deep copying is only relevant for compound objects (objects that contain other objects, like lists or class instances): * A *shallow copy* constructs a new compound object and then (to the extent possible) inserts *references* into it to the objects found in the original. * A *deep copy* constructs a new compound object and then, recursively, inserts *copies* into it of the objects found in the original. Two problems often exist with deep copy operations that don't exist with shallow copy operations: * Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. * Because deep copy copies *everything* it may copy too much, e.g., administrative data structures that should be shared even between copies. The `deepcopy()' function avoids these problems by: * keeping a "memo" dictionary of objects already copied during the current copying pass; and * letting user-defined classes override the copying operation or the set of components copied. This version does not copy types like module, class, function, method, stack trace, stack frame, file, socket, window, array, or any similar types. Classes can use the same interfaces to control copying that they use to control pickling: they can define methods called `__getinitargs__()', `__getstate__()' and `__setstate__()'. See the description of module `pickle' for information on these methods. The `copy' module does not use the `copy_reg' registration module. In order for a class to define its own copy implementation, it can define special methods `__copy__()' and `__deepcopy__()'. The former is called to implement the shallow copy operation; no additional arguments are passed. The latter is called to implement the deep copy operation; it is passed one argument, the memo dictionary. If the `__deepcopy__()' implementation needs to make a deep copy of a component, it should call the `deepcopy()' function with the component as first argument and the memo dictionary as second argument. See also: *Note pickle:: Discussion of the special disciplines used to support object state retrieval and restoration.  File: python-lib.info, Node: marshal, Next: imp, Prev: copy, Up: Python Services Alternate Python object serialization ===================================== Convert Python objects to streams of bytes and back (with different constraints). This module contains functions that can read and write Python values in a binary format. The format is specific to Python, but independent of machine architecture issues (e.g., you can write a Python value to a file on a PC, transport the file to a Sun, and read it back there). Details of the format are undocumented on purpose; it may change between Python versions (although it rarely does).(1) This is not a general "persistency" module. For general persistency and transfer of Python objects through RPC calls, see the modules `pickle' and `shelve'. The `marshal' module exists mainly to support reading and writing the "pseudo-compiled" code for Python modules of `.pyc' files. Not all Python object types are supported; in general, only objects whose value is independent from a particular invocation of Python can be written and read by this module. The following types are supported: `None', integers, long integers, floating point numbers, strings, tuples, lists, dictionaries, and code objects, where it should be understood that tuples, lists and dictionaries are only supported as long as the values contained therein are themselves supported; and recursive lists and dictionaries should not be written (they will cause infinite loops). *Caveat:* On machines where C's `long int' type has more than 32 bits (such as the DEC Alpha), it is possible to create plain Python integers that are longer than 32 bits. Since the current `marshal' module uses 32 bits to transfer plain Python integers, such values are silently truncated. This particularly affects the use of very long integer literals in Python modules -- these will be accepted by the parser on such machines, but will be silently be truncated when the module is read from the `.pyc' instead.(2) There are functions that read/write files as well as functions operating on strings. The module defines these functions: `dump(value, file)' Write the value on the open file. The value must be a supported type. The file must be an open file object such as `sys.stdout' or returned by `open()' or `posix.popen()'. If the value has (or contains an object that has) an unsupported type, a `ValueError' exception is raised -- but garbage data will also be written to the file. The object will not be properly read back by `load()'. `load(file)' Read one value from the open file and return it. If no valid value is read, raise `EOFError', `ValueError' or `TypeError'. The file must be an open file object. *Warning:* If an object containing an unsupported type was marshalled with `dump()', `load()' will substitute `None' for the unmarshallable type. `dumps(value)' Return the string that would be written to a file by `dump(VALUE, FILE)'. The value must be a supported type. Raise a `ValueError' exception if value has (or contains an object that has) an unsupported type. `loads(string)' Convert the string to a value. If no valid value is found, raise `EOFError', `ValueError' or `TypeError'. Extra characters in the string are ignored. ---------- Footnotes ---------- (1) The name of this module stems from a bit of terminology used by the designers of Modula-3 (amongst others), who use the term "marshalling" for shipping of data around in a self-contained form. Strictly speaking, "to marshal" means to convert some data from internal to external form (in an RPC buffer for instance) and "unmarshalling" for the reverse process. (2) A solution would be to refuse such literals in the parser, since they are inherently non-portable. Another solution would be to let the `marshal' module raise an exception when an integer value would be truncated. At least one of these solutions will be implemented in a future version.