TPIE's namespace. More...

Namespaces
	ami
	The namespace within TPIE for the Access Method Interface elements.

	pipelining
	TPIE pipelining framework.

Classes
struct	allocation_scope_magic

class	allocator
	A allocator object usable in STL containers, using the TPIE memory manager. More...

class	array
	A generic array with a fixed size. More...

struct	array_allocation_scope_magic

class	array_iter_base
	Shared implementation of array iterators. More...

class	array_view
	Encapsulation of two pointers from any random access container. More...

class	array_view< const T >

class	array_view_base
	Base class for array_view. More...

class	auto_ptr
	like std::auto_ptr, but delete the object with tpie_delete. More...

struct	auto_ptr_ref

struct	binary_argument_swap
	A binary functor with the arguments swapped. More...

class	chaining_hash_table
	Hash table handling hash collisions by chaining. More...

class	cpu_timer

struct	default_unused

struct	default_unused< std::pair< T1, T2 > >

class	disjoint_sets
	Internal memory union find implementation. More...

struct	disjunction
	Class to compute the disjunction between two boost true/false types. More...

struct	dummy_fraction_progress
	A fractional progress indicator that is conceptually compatible with tpie::fractional_progress. More...

struct	dummy_progress_indicator
	A progress indicator that is conceptually compatible with tpie::progress_indicator_base and tpie::fractional_subindicator. More...

struct	end_of_stream_exception

struct	exception

class	execution_time_predictor

class	file
	Central file abstraction. More...

class	file_base

class	file_base_crtp
	Base class of classes that access files. More...

class	file_log_target
	A simple logger that writes messages to a tpie temporary file. More...

class	file_stream
	Simple class acting both as file and a file::stream. More...

class	file_stream_base

class	fractional_progress
	Fractional progress reporter. More...

class	fractional_subindicator
	Subindicator for fractional progress reporting. More...

struct	get_arg_help

struct	get_arg_help< bool >

struct	hash
	Default hashing function for integral (size_t-castable) types. More...

struct	hash< const char * >
	Default hashing function for C-style strings. More...

struct	hash< std::pair< T1, T2 > >
	Default hashing function for std::pair. More...

struct	hash< std::string >
	Default hashing function for std::string. More...

class	hash_map
	Hash map implementation backed by a template parameterized hash table. More...

class	hash_set
	Hash set implementation backed by a template parameterized hash table. More...

class	internal_priority_queue
	Standard binary internal heap. More...

class	internal_queue
	A generic internal circular queue. More...

class	internal_stack
	A generic internal stack. More...

class	internal_stack_vector_base
	A base class for a generic internal fixed size stack and vector. More...

class	internal_vector
	A generic internal vector. More...

struct	invalid_argument_exception

struct	invalid_file_exception

struct	io_exception

class	is_simple_iterator
	Checks if an iterator is simple. More...

struct	is_simple_iterator_enable_if

struct	is_simple_iterator_enable_if< true >

struct	is_trivially_serializable

struct	is_trivially_serializable_enable_if

struct	is_trivially_serializable_enable_if< true >

class	job

struct	job_manager_exception

struct	linear_memory_base
	Base class of data structures with linear memory usage. More...

struct	linear_memory_structure_concept
	Check if a structure adheres to the linear_memory_structure concept. More...

struct	linear_memory_structure_doc
	Description of the methods that must be implemented to support the linear_memory_structure_concept. More...

class	linear_probing_hash_table
	Hash table handling hash collisions by linear probing. More...

class	log_stream_buf

struct	log_target

class	logmanip
	The logmanip template is based on the omanip template from iomanip.h in the libg++ sources. More...

class	logstream
	A log is like a regular output stream, but it also supports messages at different priorities, see log_level. More...

struct	magic_cast_help

struct	magic_cast_help< bool, std::string >

struct	magic_cast_help< std::string, bool >

class	memory_manager
	Memory management object used to track memory usage. More...

class	merge_sort_not_ready

class	merge_sorter
	Merge sorting consists of three phases. More...

class	merger

class	op2TPIE_cmp
	Convert a class with a comparison operator < to a TPIE comparison object with a compare() function. More...

struct	out_of_memory_error
	Thrown when trying to allocate too much memory. More...

struct	out_of_space_exception

class	packed_array
	An array storring elements of type T using B bits to to store a element. More...

class	packed_array_iter_facade
	Base class for the iterators. More...

class	parallel_sort_impl
	A simple parallel sort implementation with progress tracking. More...

class	pq_merge_heap

class	pq_overflow_heap
	Overflow Priority Queue, based on a simple Heap. More...

class	priority_queue
	External memory priority queue implementation. More...

struct	priority_queue_error

class	progress_indicator_arrow
	A class that indicates the progress by expanding an arrow. More...

class	progress_indicator_base
	The base class for indicating the progress of some task. More...

class	progress_indicator_null
	a dummy progress indicator that produces no output More...

class	progress_indicator_spin
	A class that indicates the progress by a spinning cross. More...

class	progress_indicator_subindicator

class	progress_indicator_terminal
	A class that indicates the progress by a simple counter that is printed to the terminal. More...

struct	progress_types
	For applications where you wish to disable progress indicators via a template parameter, refer to progress_types<use_progress> members names sub, fp and base. More...

struct	progress_types< false >

class	queue
	Basic Implementation of I/O Efficient FIFO queue. More...

class	scoped_log_enabler

struct	serialization_error

class	serialization_reader

class	serialization_reverse_reader

class	serialization_reverse_writer

class	serialization_sort

class	serialization_writer

class	serializer
	Class providing binary serialization to a std::ostream. More...

struct	sign

struct	sign< uint16_t >

struct	sign< uint32_t >

struct	sign< uint64_t >

struct	sign< uint8_t >

class	sort_manager
	A class of manager objects for merge sorting objects of type T. More...

struct	sort_parameters

class	stack
	An implementation of an external-memory stack. More...

class	stderr_log_target
	A simple logger that writes messages to stderr. More...

class	STL2TPIE_cmp
	Convert STL comparison object with operator() to a TPIE comparison object with a compare() function. More...

class	stream_crtp

struct	stream_exception

struct	stream_header_t

struct	sysinfo
	Class providing system and platform info. More...

class	temp_file
	Class representing the existence of a temporary file. More...

struct	tempfile_error

struct	template_log
	Computes the least integer strictly greater than log(t). More...

struct	template_log< 1 >

class	tempname
	Static methods for generating temporary file names and finding temporary file directories. More...

class	testmanip

class	tests

class	teststream

class	teststream_buf

class	TPIE2STL_cmp
	Convert a TPIE comparison object with a compare() function to STL comparison object with operator(). More...

struct	trivial_same_size

class	unique_id_type

class	unserializer
	Class for unserializing binary data serialized with the serializer Data can be unserialized using the >> operators. More...

struct	unsign

struct	unsign< int16_t >

struct	unsign< int32_t >

struct	unsign< int64_t >

struct	unsign< int8_t >

Typedefs
typedef packed_array< bool, 1 >	bit_array
	Packed array of bits. More...

typedef boost::uint64_t	time_type

typedef file_accessor::raw_file_accessor	default_raw_file_accessor

typedef file_accessor::file_accessor	default_file_accessor

typedef int	merge_flag
	Intended to signal in a merge which of the input streams are non-empty. More...

typedef TPIE_OS_SIZE_T	arity_t
	Intended to signal the number of input streams in a merge. More...

typedef boost::uint64_t	hash_type

typedef boost::uint64_t	stream_size_type

typedef boost::int64_t	stream_offset_type

typedef ::size_t	memory_size_type

typedef ::ssize_t	memory_offset_type

typedef stream_offset_type	offset_type

typedef ssize_t	ssize_type

typedef size_t	size_type

Enumerations
enum	access_type { access_read, access_write, access_read_write }
	Type describing how we wish to access a file. More...

enum	cache_hint { access_normal, access_sequential, access_random }

enum	log_level { LOG_FATAL = 0, LOG_ERROR, LOG_WARNING, LOG_INFORMATIONAL, LOG_APP_DEBUG, LOG_DEBUG, LOG_MEM_DEBUG }
	TPIE logging levels, from higest priority to lowest. More...

enum	description_importance { IMPORTANCE_NONE, IMPORTANCE_LOG, IMPORTANCE_MINOR, IMPORTANCE_MAJOR }

enum	stream_usage { STREAM_USAGE_OVERHEAD = 1, STREAM_USAGE_BUFFER, STREAM_USAGE_CURRENT, STREAM_USAGE_MAXIMUM, STREAM_USAGE_SUBSTREAM }

enum	subsystem { MEMORY_MANAGER =1, DEFAULT_LOGGING =2, PROGRESS =4, PRIMEDB =8, JOB_MANAGER =16, CAPTURE_FRACTIONS =32, ALL =MEMORY_MANAGER \| DEFAULT_LOGGING \| PROGRESS \| PRIMEDB \| JOB_MANAGER }
	Subsystems of TPIE. More...

Functions
template<typename T >
std::ostream &	operator<< (std::ostream &o, const array< T > &a)

template<typename T >
array_view< typename boost::remove_pointer < typename std::iterator_traits< T > ::pointer >::type >	make_array_view (T start, T end)

template<typename T >
array_view< T >	make_array_view (T start, T end)

template<typename T >
array_view< typename T::value_type >	make_array_view (T &s)

template<typename T >
array_view< typename T::value_type >	make_array_view (T &s, size_t start, size_t end)

template<typename T >
array_view< const typename T::value_type >	make_array_view (const T &s)

template<typename T >
array_view< const typename T::value_type >	make_array_view (const T &s, size_t start, size_t end)

void	backtrace (std::ostream &out, int depth=1024)
	Output a function call backtrace for debugging. More...

void	__softassert (const char expr, const char file, int line)
	Do not use this directly; use the softassert() macro instead. More...

std::ostream &	operator<< (std::ostream &s, cpu_timer &ct)
	Enable outputting the queriable values of this timer. More...

void	init_execution_time_db ()
	Used by tpie_init to initialize the execution time database. More...

void	finish_execution_time_db ()
	Used by tpie_finish to deinitialize the execution time database. More...

	TPIE_DEPRECATED (inline void increment_open_file_count())
	Deprecated; does nothing. More...

memory_size_type	open_file_count ()
	Return the current number of open files. More...

memory_size_type	available_files ()
	Return the additional number of files that can be opened before running out of file descriptors. More...

void	init_fraction_db (bool capture_progress=false)
	Used by tpie_init to initialize the fraction database. More...

void	finish_fraction_db ()
	Used by tpie_finish to deinitialize the fraction database. More...

void	update_fractions (const char *name, float frac, stream_size_type n)

void	load_fractions (const std::string &path)

void	save_fractions (const std::string &path, bool force=false)

memory_size_type	default_worker_count ()
	Return the number of job threads initialized by the job framework in init_job(). More...

void	init_job ()
	Used by tpie_init to initialize the job subsystem. More...

void	finish_job ()
	Used by tpie_finish to deinitialize the job subsystem. More...

logstream &	manip_level (logstream &tpl, log_level p)

logmanip< log_level >	setlevel (log_level p)

void	init_memory_manager ()
	Used by tpie_init to initialize the memory manager. More...

void	finish_memory_manager ()
	Used by tpie_finish to deinitialize the memory manager. More...

memory_manager &	get_memory_manager ()
	Return a reference to the memory manager. More...

void	__register_pointer (void *p, size_t size, const std::type_info &t)

void	__unregister_pointer (void *p, size_t size, const std::type_info &t)

void	assert_tpie_ptr (void *p)
	In a debug build, assert that a given pointer has been allocated with tpie_new. More...

template<typename D , typename T >
D	ptr_cast (T *t)
	Cast between pointer types. More...

template<typename T >
T *	__allocate ()

template<typename T >
size_t	tpie_size (T *p)

template<typename T >
T *	tpie_new_array (size_t size)
	Allocate a new array and register its memory usage. More...

template<typename T , typename Args >
T *	tpie_new (Args args)
	Allocate an element of the type given as template parameter, and register its memory usage with TPIE. More...

template<typename T >
void	tpie_delete (T *p) throw ()
	Delete an object allocated with tpie_new. More...

template<typename T >
void	tpie_delete_array (T *a, size_t size) throw ()
	Delete an array allocated with tpie_new_array. More...

template<typename T >
bool	operator== (const tpie::allocator< T > &, const tpie::allocator< T > &)

template<typename T >
bool	operator!= (const tpie::allocator< T > &, const tpie::allocator< T > &)

size_t	consecutive_memory_available (size_t granularity=5 1024 1024)
	Find the largest amount of memory that can be allocated as a single chunk. More...

template<typename CT , bool f, typename RT >
CT	operator+ (ptrdiff_t n, const packed_array_iter_facade< CT, f, RT > &i)

template<bool Progress, typename iterator_type , typename comp_type >
void	parallel_sort (iterator_type a, iterator_type b, typename tpie::progress_types< Progress >::base &pi, comp_type comp=std::less< typename boost::iterator_value< iterator_type >::type >())
	Sort items in the range [a,b) using a parallel quick sort. More...

template<typename iterator_type , typename comp_type >
void	parallel_sort (iterator_type a, iterator_type b, comp_type comp=std::less< typename boost::iterator_value< iterator_type >::type >())
	Sort items in the range [a,b) using a parallel quick sort. More...

void	init_prime ()
	Used by tpie_init to initialize the prime number database. More...

void	finish_prime ()
	Used by tpie_finish to deinitialize the prime number database. More...

bool	is_prime (size_type i)
	Check if i is a prime. More...

hash_type	prime_hash (const std::string &s)
	Calculate a fairly good string hash based on prime numbers. More...

size_t	next_prime (size_t i)
	Get next prime. More...

template<typename D >
void	serialize (D &dst, const foo &v)
	Sample tpie::serialize prototype. More...

template<typename S >
void	unserialize (S &src, foo &v)
	Sample tpie::unserialize prototype. More...

template<typename D , typename T >
void	serialize (D &dst, const T &v, typename boost::enable_if< is_trivially_serializable< T > >::type =0, typename boost::disable_if< boost::is_pointer< T > >::type =0)
	tpie::serialize for POD/array types. More...

template<typename S , typename T >
void	unserialize (S &src, T &v, typename boost::enable_if< is_trivially_serializable< T > >::type =0, typename boost::disable_if< boost::is_pointer< T > >::type =0)
	tpie::unserialize for POD/array types. More...

template<typename D , typename T >
void	serialize (D &dst, T start, T end)
	Serialize an array of serializables. More...

template<typename D , typename T >
void	unserialize (D &dst, T start, T end)
	Unserialize an array of serializables. More...

template<typename D , typename T , typename alloc_t >
void	serialize (D &dst, const std::vector< T, alloc_t > &v)
	tpie::serialize for std::vectors of serializable items. More...

template<typename S , typename T , typename alloc_t >
void	unserialize (S &src, std::vector< T, alloc_t > &v)
	tpie::unserialize for std::vectors of unserializable items. More...

template<typename D , typename T >
void	serialize (D &dst, const std::basic_string< T > &v)
	tpie::serialize for std::basic_strings of serializable items, including std::strings. More...

template<typename S , typename T >
void	unserialize (S &src, std::basic_string< T > &v)
	tpie::unserialize for std::basic_strings of unserializable items, including std::strings. More...

template<typename T >
size_t	serialized_size (const T &v)
	Given a serializable, serialize it and measure its serialized size. More...

template<typename T , typename Compare >
void	sort (file_stream< T > &instream, file_stream< T > &outstream, Compare comp, progress_indicator_base &indicator)
	Sort elements of a stream using the given STL-style comparator object. More...

template<typename T >
void	sort (file_stream< T > &instream, file_stream< T > &outstream, tpie::progress_indicator_base *indicator=NULL)
	Sort elements of a stream using the less-than operator. More...

template<typename T , typename Compare >
void	sort (file_stream< T > &instream, Compare comp, progress_indicator_base &indicator)
	Sort elements of a stream in-place using the given STL-style comparator object. More...

template<typename T >
void	sort (file_stream< T > &instream, progress_indicator_base &indicator)
	Sort elements of a stream in-place using the less-than operator. More...

template<typename T >
void	sort (file_stream< T > &instream)
	Sort elements of a stream in-place using the less-than operator and no progress indicator. More...

stream_size_type	get_temp_file_usage ()
	Return the number of bytes currently being used by temporary files. More...

void	increment_temp_file_usage (stream_offset_type delta)
	Increment (possibly by a negative amount) the number of bytes being used by temporary files. More...

stream_size_type	get_bytes_read ()
	Return the number of bytes read from disk since program start. More...

stream_size_type	get_bytes_written ()
	Return the number of bytes written to disk since program start. More...

void	increment_bytes_read (stream_size_type delta)
	Inform that stats module that an additional delta bytes have been read from disk. More...

void	increment_bytes_written (stream_size_type delta)
	Inform the stats module that an additional delta bytes have been written to disk. More...

std::ostream &	operator<< (std::ostream &s, const sysinfo &info)
	Report default system info to the specified `ostream`. More...

void	tpie_init (int subsystems=ALL)
	Initialize the given subsystems of TPIE. More...

void	tpie_finish (int subsystems=ALL)
	Deinitialize the given subsystems of TPIE. More...

memory_size_type	get_block_size ()
	Get the TPIE block size. More...

void	set_block_size (memory_size_type block_size)
	Set the TPIE block size. More...

const std::string &	log_name ()
	Returns the file name of the log stream. More...

void	init_default_log ()
	Used by tpie_init to initialize the log subsystem. More...

void	finish_default_log ()
	Used by tpie_finish to deinitialize the log subsystem. More...

logstream &	get_log ()
	Returns the only logstream object. More...

logstream &	log_fatal ()
	Return logstream for writing fatal log messages. More...

logstream &	log_error ()
	Return logstream for writing error log messages. More...

logstream &	log_info ()
	Return logstream for writing info log messages. More...

logstream &	log_warning ()
	Return logstream for writing warning log messages. More...

logstream &	log_app_debug ()
	Return logstream for writing app_debug log messages. More...

logstream &	log_debug ()
	Return logstream for writing debug log messages. More...

logstream &	log_mem_debug ()
	Return logstream for writing mem_debug log messages. More...

testmanip< bool >	result (bool success)

testmanip< bool >	success ()

testmanip< bool >	failure ()

template<typename dst , typename src >
dst	magic_cast (const src &s)

template<typename T >
void	unused (const T &x)
	Declare that a variable is unused on purpose. More...

template<typename T , typename C >
void	pop_and_push_heap (T a, T b, C lt)
	Restore heap invariants after the first element has been replaced by some other element. More...

template<typename T >
void	pop_and_push_heap (T a, T b)
	Restore heap invariants after the first element has been replaced by some other element. More...

void	atomic_rename (const std::string &src, const std::string &dst)

template<typename T >
void	free_structure_memory (T &v)
	Free the memory assosiated with a stl or tpie structure by swapping it with a default constructed structure of the same type. More...

Variables
enum TPIE_DEPRECATED_CLASS_B tpie::persistence	TPIE_DEPRECATED_CLASS_C

const char *	git_commit
	The Git commit hash (40 hexadecimal characters) that TPIE was built from. More...

const char *	git_refspec
	The Git refspec that TPIE was built from. More...

logstream	log_singleton

const char	directory_delimiter = '/'

Detailed Description

TPIE's namespace.

Typedef Documentation

typedef TPIE_OS_SIZE_T tpie::arity_t

Intended to signal the number of input streams in a merge.

Definition at line 59 of file merge.h.

typedef packed_array<bool, 1> tpie::bit_array

Packed array of bits.

Definition at line 31 of file bit_array.h.

typedef int tpie::merge_flag

Intended to signal in a merge which of the input streams are non-empty.

Definition at line 57 of file merge.h.

Enumeration Type Documentation

enum tpie::access_type

Type describing how we wish to access a file.

Enumerator
access_read	Open a file for reading.
access_write	Open a file for writing only, content is truncated.
access_read_write	Open a file for reading or writing.

Definition at line 29 of file access_type.h.

                  {
     access_read,
     access_write,
     access_read_write
 };

enum tpie::cache_hint

Enumerator

access_normal

Neither sequential access nor random access is intended.

Corresponds to POSIX_FADV_NORMAL.

access_sequential

Sequential access is intended.

Default for file_stream. Corresponds to POSIX_FADV_SEQUENTIAL and FILE_FLAG_SEQUENTIAL_SCAN (Win32).

access_random

Random access is intended.

Corresponds to POSIX_FADV_RANDOM and FILE_FLAG_RANDOM_ACCESS (Win32).

Definition at line 28 of file cache_hint.h.

                 {
     access_normal,
 
     access_sequential,
 
     access_random
 };

enum tpie::log_level

TPIE logging levels, from higest priority to lowest.

Enumerator
LOG_FATAL	LOG_FATAL is the highest error level and is used for all kinds of errors that would normally impair subsequent computations; LOG_FATAL errors are always logged.
LOG_ERROR	LOG_ERROR is used for none fatal errors.
LOG_WARNING	LOG_WARNING is used for warnings.
LOG_INFORMATIONAL	LOG_INFORMATIONAL is used for informational messagse.
LOG_APP_DEBUG	LOG_APP_DEBUG can be used by applications built on top of TPIE, for logging debugging information.
LOG_DEBUG	LOG_DEBUG is the lowest level and is used by the TPIE library for logging debugging information.
LOG_MEM_DEBUG	Logging level for warnings concerning memory allocation and deallocation.

Definition at line 33 of file loglevel.h.

                {
     LOG_FATAL = 0,  
     
     LOG_ERROR,
     
     LOG_WARNING,    
     
     LOG_INFORMATIONAL,
     
     LOG_APP_DEBUG,     
     
     LOG_DEBUG,
     
     LOG_MEM_DEBUG
 };

enum tpie::stream_usage

Enumerator
STREAM_USAGE_OVERHEAD	Overhead of the object without the buffer.
STREAM_USAGE_BUFFER	Max amount ever used by a buffer.
STREAM_USAGE_CURRENT	Amount currently in use.
STREAM_USAGE_MAXIMUM	Max amount that will ever be used.
STREAM_USAGE_SUBSTREAM	Maximum additional amount used by each substream created.

Definition at line 28 of file stream_usage.h.

                       {
         STREAM_USAGE_OVERHEAD = 1,
         STREAM_USAGE_BUFFER,
         STREAM_USAGE_CURRENT,
         STREAM_USAGE_MAXIMUM,
         STREAM_USAGE_SUBSTREAM
     };

enum tpie::subsystem

Subsystems of TPIE.

Enumerator
MEMORY_MANAGER	Needed for tpie_new and tpie_delete and implicitly needed by all TPIE algorithm and data structure implementations.
DEFAULT_LOGGING	TPIE logging framework. See tpie_log.h.
PROGRESS	Progress tracking. Needed for the fraction database.
PRIMEDB	Prime number database, for prime.h.
JOB_MANAGER	Job manager, for job.h and the parallel quick sort.
CAPTURE_FRACTIONS	Capture fractions.
ALL	Alias for all default subsystems.

Definition at line 38 of file tpie.h.

                {
     MEMORY_MANAGER=1,
     DEFAULT_LOGGING=2,
     PROGRESS=4,
     PRIMEDB=8,
     JOB_MANAGER=16,
     CAPTURE_FRACTIONS=32,
     ALL=MEMORY_MANAGER | DEFAULT_LOGGING | PROGRESS | PRIMEDB | JOB_MANAGER
 };

Function Documentation

void tpie::__register_pointer	(	void *	p,
		size_t	size,
		const std::type_info &	t
	)

inline

Register a pointer, for debugging memory leaks and such.

Definition at line 187 of file memory.h.

References get_memory_manager(), and unused().

                                                                               {
 #ifndef TPIE_NDEBUG
     get_memory_manager().register_pointer(p, size, t);
 #else
     unused(p);
     unused(size);
     unused(t);
 #endif
 }

void tpie::__softassert	(	const char *	expr,
		const char *	file,
		int	line
	)

Do not use this directly; use the softassert() macro instead.

See also: softassert()

void tpie::__unregister_pointer	(	void *	p,
		size_t	size,
		const std::type_info &	t
	)

inline

Unregister a registered pointer.

Definition at line 201 of file memory.h.

References get_memory_manager(), and unused().

Referenced by tpie_delete(), and tpie_delete_array().

                                                                                 {
 #ifndef TPIE_NDEBUG
     get_memory_manager().unregister_pointer(p, size, t);
 #else
     unused(p);
     unused(size);
     unused(t);
 #endif
 }

void tpie::assert_tpie_ptr ( void * p )

inline

In a debug build, assert that a given pointer has been allocated with tpie_new.

Definition at line 215 of file memory.h.

References get_memory_manager(), and unused().

                                       {
 #ifndef TPIE_NDEBUG
     if (p)
         get_memory_manager().assert_tpie_ptr(p);
 #else
     unused(p);
 #endif
 }

memory_size_type tpie::available_files ( )

Return the additional number of files that can be opened before running out of file descriptors.

Referenced by tpie::ami::stream< T >::available_streams().

void tpie::backtrace	(	std::ostream &	out,
		int	depth = `1024`
	)

Output a function call backtrace for debugging.

Does not support Windows. On Linux, uses cxxabi.h and execinfo.h to inspect the stack at runtime.

size_t tpie::consecutive_memory_available ( size_t granularity = 5 *1024 *1024 )

Find the largest amount of memory that can be allocated as a single chunk.

Referenced by tpie::ami::main_mem_merge(), tpie::ami::merge(), and tpie::ami::partition_and_merge().

memory_size_type tpie::default_worker_count ( )

Return the number of job threads initialized by the job framework in init_job().

When hardware concurrency is less than four, for instance on a dual-core processor without hyperthreading, or a single-core processor, the number of threads launched is equal to the number of cores. When hardware concurrency is four or greater, the job framework spares one core for the user interface.

Referenced by tpie::pipelining::parallel().

void tpie::finish_default_log ( )

Used by tpie_finish to deinitialize the log subsystem.

void tpie::finish_execution_time_db ( )

Used by tpie_finish to deinitialize the execution time database.

void tpie::finish_fraction_db ( )

Used by tpie_finish to deinitialize the fraction database.

void tpie::finish_job ( )

Used by tpie_finish to deinitialize the job subsystem.

void tpie::finish_memory_manager ( )

Used by tpie_finish to deinitialize the memory manager.

void tpie::finish_prime ( )

Used by tpie_finish to deinitialize the prime number database.

template<typename T >

void tpie::free_structure_memory ( T & v )

inline

Free the memory assosiated with a stl or tpie structure by swapping it with a default constructed structure of the same type.

Definition at line 177 of file util.h.

                                          {
     T t;
     std::swap(v, t);
 }

memory_size_type tpie::get_block_size ( )

Get the TPIE block size.

This can be changed by setting the TPIE_BLOCK_SIZE environment variable or by calling the set_block_size method.

The default is 2 MiB (2**21 bytes).

Referenced by tpie::file_base_crtp< file_stream_base >::block_size().

stream_size_type tpie::get_bytes_read ( )

Return the number of bytes read from disk since program start.

stream_size_type tpie::get_bytes_written ( )

Return the number of bytes written to disk since program start.

logstream& tpie::get_log ( )

inline

Returns the only logstream object.

Definition at line 94 of file tpie_log.h.

Referenced by log_app_debug(), log_debug(), log_error(), log_fatal(), log_info(), log_mem_debug(), and log_warning().

94 {return log_singleton;}

memory_manager& tpie::get_memory_manager ( )

Return a reference to the memory manager.

May only be called when init_memory_manager has been called. See tpie_init().

Referenced by __register_pointer(), __unregister_pointer(), assert_tpie_ptr(), tpie::merge_sorter< T, UseProgress, pred_t >::end(), tpie_delete(), and tpie_delete_array().

stream_size_type tpie::get_temp_file_usage ( )

Return the number of bytes currently being used by temporary files.

void tpie::increment_bytes_read ( stream_size_type delta )

Inform that stats module that an additional delta bytes have been read from disk.

void tpie::increment_bytes_written ( stream_size_type delta )

Inform the stats module that an additional delta bytes have been written to disk.

void tpie::increment_temp_file_usage ( stream_offset_type delta )

Increment (possibly by a negative amount) the number of bytes being used by temporary files.

void tpie::init_default_log ( )

Used by tpie_init to initialize the log subsystem.

void tpie::init_execution_time_db ( )

Used by tpie_init to initialize the execution time database.

void tpie::init_fraction_db ( bool capture_progress = false )

Used by tpie_init to initialize the fraction database.

void tpie::init_job ( )

Used by tpie_init to initialize the job subsystem.

void tpie::init_memory_manager ( )

Used by tpie_init to initialize the memory manager.

void tpie::init_prime ( )

Used by tpie_init to initialize the prime number database.

bool tpie::is_prime ( size_type i )

Check if i is a prime.

Parameters

i	number to check, must be less then 4294967295.

Returns: true if and only if i is a prime number.

Referenced by tpie::chaining_hash_table< value_t, hash_t, equal_t, index_t >::resize(), and tpie::linear_probing_hash_table< value_t, hash_t, equal_t, index_t >::resize().

logstream& tpie::log_app_debug ( )

inline

Return logstream for writing app_debug log messages.

Definition at line 119 of file tpie_log.h.

References get_log(), and LOG_APP_DEBUG.

119 {return get_log() << setlevel(LOG_APP_DEBUG);}

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

tpie::LOG_APP_DEBUG

LOG_APP_DEBUG can be used by applications built on top of TPIE, for logging debugging information...

Definition: loglevel.h:50

logstream& tpie::log_debug ( )

inline

Return logstream for writing debug log messages.

Definition at line 124 of file tpie_log.h.

References get_log(), and LOG_DEBUG.

Referenced by tpie::merge_sorter< T, UseProgress, pred_t >::begin(), tpie::merge_sorter< T, UseProgress, pred_t >::end(), tpie::pipelining::node::forward_any(), tpie::pipelining::serialization_bits::sort_calc_t< Traits >::go(), tpie::merge_sorter< T, UseProgress, pred_t >::set_items(), tpie::merge_sorter< T, UseProgress, pred_t >::set_parameters(), and tpie::progress_indicator_base::step().

124 {return get_log() << setlevel(LOG_DEBUG);}

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

tpie::LOG_DEBUG

LOG_DEBUG is the lowest level and is used by the TPIE library for logging debugging information...

Definition: loglevel.h:54

logstream& tpie::log_error ( )

inline

Return logstream for writing error log messages.

Definition at line 104 of file tpie_log.h.

References get_log(), and LOG_ERROR.

Referenced by tpie::pipelining::parallel_bits::producer< T1, T2 >::end(), tpie::pipelining::virtual_chunk_end< Input >::operator=(), tpie::pipelining::virtual_chunk< Input, Output >::operator=(), tpie::pipelining::virtual_chunk_begin< Output >::operator=(), tpie::pipelining::node::set_steps(), and tpie::pipelining::parallel_bits::state_base::transition_state().

104 {return get_log() << setlevel(LOG_ERROR);}

tpie::LOG_ERROR

LOG_ERROR is used for none fatal errors.

Definition: loglevel.h:40

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

logstream& tpie::log_fatal ( )

inline

Return logstream for writing fatal log messages.

Definition at line 99 of file tpie_log.h.

References get_log(), and LOG_FATAL.

99 {return get_log() << setlevel(LOG_FATAL);}

tpie::LOG_FATAL

LOG_FATAL is the highest error level and is used for all kinds of errors that would normally impair s...

Definition: loglevel.h:37

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

logstream& tpie::log_info ( )

inline

Return logstream for writing info log messages.

Definition at line 109 of file tpie_log.h.

References get_log(), and LOG_INFORMATIONAL.

109 {return get_log() << setlevel(LOG_INFORMATIONAL);}

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

tpie::LOG_INFORMATIONAL

LOG_INFORMATIONAL is used for informational messagse.

Definition: loglevel.h:46

logstream& tpie::log_mem_debug ( )

inline

Return logstream for writing mem_debug log messages.

Definition at line 129 of file tpie_log.h.

References get_log(), and LOG_MEM_DEBUG.

129 {return get_log() << setlevel(LOG_MEM_DEBUG);}

tpie::LOG_MEM_DEBUG

Logging level for warnings concerning memory allocation and deallocation.

Definition: loglevel.h:57

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

const std::string& tpie::log_name ( )

Returns the file name of the log stream.

This assumes that init_default_log has been called.

logstream& tpie::log_warning ( )

inline

Return logstream for writing warning log messages.

Definition at line 114 of file tpie_log.h.

References get_log(), and LOG_WARNING.

Referenced by tpie::pipelining::serialization_bits::sort_pull_output_t< Traits >::go(), tpie::pipelining::bits::sort_pull_output_t< item_type, pred_t >::go(), tpie::pipelining::node::go(), and tpie::pipelining::node::step().

114 {return get_log() << setlevel(LOG_WARNING);}

tpie::LOG_WARNING

LOG_WARNING is used for warnings.

Definition: loglevel.h:43

tpie::get_log

logstream & get_log()

Returns the only logstream object.

Definition: tpie_log.h:94

size_t tpie::next_prime ( size_t i )

Get next prime.

Parameters

i	Subject to same restrictions as in is_prime.

Returns: Least prime greater than or equal to i.

memory_size_type tpie::open_file_count ( )

Return the current number of open files.

Returns: The current number of open files.

std::ostream& tpie::operator<<	(	std::ostream &	s,
		cpu_timer &	ct
	)

Enable outputting the queriable values of this timer.

On Windows, just output the elapsed real time in seconds. On Linux, output user, system and wall clock time in seconds.

std::ostream& tpie::operator<<	(	std::ostream &	s,
		const sysinfo &	info
	)

inline

Report default system info to the specified ostream.

See also: sysinfo::printinfo

Definition at line 181 of file sysinfo.h.

References tpie::sysinfo::blocksize(), tpie::sysinfo::commit(), tpie::sysinfo::hostname(), tpie::sysinfo::localtime(), tpie::sysinfo::platform(), and tpie::sysinfo::refspec().

                                                                      {
     return s
         << "Hostname:       " << info.hostname() << '\n'
         << "Platform:       " << info.platform() << '\n'
         << "Git branch:     " << info.refspec() << '\n'
         << "Git commit:     " << info.commit() << '\n'
         << "Local time:     " << info.localtime() << '\n'
         << "Block size:     " << info.blocksize() << '\n'
         << "Parallel sort:  "
 #ifdef TPIE_PARALLEL_SORT
         << "Enabled"
 #else
         << "Disabled"
 #endif
         << '\n'
         ;
 }

template<bool Progress, typename iterator_type , typename comp_type >

void tpie::parallel_sort	(	iterator_type	a,
		iterator_type	b,
		typename tpie::progress_types< Progress >::base &	pi,
		comp_type	comp = `std::less<typename boost::iterator_value<iterator_type>::type>()`
	)

Sort items in the range [a,b) using a parallel quick sort.

Parameters

a	Iterator to left boundary.
b	Iterator to right boundary.
pi	Progress tracker. No thread-safety required.
comp	Comparator.

See also: parallel_sort_impl

Definition at line 295 of file parallel_sort.h.

References tpie::progress_indicator_base::done(), tpie::progress_indicator_base::init(), and sort().

                                                                                               {
 #ifdef TPIE_PARALLEL_SORT
     parallel_sort_impl<iterator_type, comp_type, Progress> s(&pi);
     s(a,b,comp);
 #else
     pi.init(1);
     std::sort(a,b,comp);
     pi.done();
 #endif
 }

template<typename iterator_type , typename comp_type >

void tpie::parallel_sort	(	iterator_type	a,
		iterator_type	b,
		comp_type	comp = `std::less<typename boost::iterator_value<iterator_type>::type>()`
	)

Sort items in the range [a,b) using a parallel quick sort.

Parameters

a	Iterator to left boundary.
b	Iterator to right boundary.
comp	Comparator.

See also: parallel_sort_impl

Definition at line 317 of file parallel_sort.h.

References sort().

                                                                                               {
 #ifdef TPIE_PARALLEL_SORT
     parallel_sort_impl<iterator_type, comp_type, false> s(0);
     s(a,b,comp);
 #else
     std::sort(a, b, comp);
 #endif
 }

template<typename T , typename C >

void tpie::pop_and_push_heap	(	T	a,
		T	b,
		C	lt
	)

Restore heap invariants after the first element has been replaced by some other element.

Definition at line 119 of file util.h.

Referenced by tpie::internal_priority_queue< std::pair< T, size_t >, tpie::merger::predwrap >::pop_and_push(), and pop_and_push_heap().

                                        {
     size_t i=0;
     size_t n=(b-a);
     while (true) {
         size_t c=2*i+1;
         if (c+1 >= n) {
             if (c < n && lt(*(a+i), *(a+c)))
                 std::swap(*(a+c), *(a+i));
             break;
         }
         if (lt(*(a+c+1), *(a+c))) {
             if (lt(*(a+i), *(a+c))) {
                 std::swap(*(a+c), *(a+i));
                 i=c;
             } else break;
         } else {
             if (lt(*(a+i), *(a+c+1))) {
                 std::swap(*(a+c+1), *(a+i));
                 i=c+1;
             } else break;
         }
     }
 }

template<typename T >

void tpie::pop_and_push_heap	(	T	a,
		T	b
	)

Restore heap invariants after the first element has been replaced by some other element.

Uses std::less as the heap property.

Definition at line 148 of file util.h.

References pop_and_push_heap().

                                  {
     pop_and_push_heap(a,b, std::less<typename T::value_type>());
 }

hash_type tpie::prime_hash ( const std::string & s )

Calculate a fairly good string hash based on prime numbers.

Parameters

s	The string to hash.

Returns: The hash value.

template<typename D , typename T >

D tpie::ptr_cast ( T * t )

inline

Cast between pointer types.

If the input pointer is polymorphic, its base address is found, and that is then casted to the output type.

Template Parameters

D	A non-polymorphic pointer type.
T	Any type.

Definition at line 250 of file memory.h.

Referenced by tpie_delete().

250 { return reinterpret_cast<D>(__object_addr<T>()(t)); }

template<typename D >

void tpie::serialize	(	D &	dst,
		const foo &	v
	)

Sample tpie::serialize prototype.

To enable serialization of your own type, overload tpie::serialize. This docstring is an example for a type named foo, but it is for exposition purposes only.

The implementation of tpie::serialize(dst, v) shall call dst.write(src, n) a number of times. Each time, src is a const pointer to a byte buffer of size n (bytes) that represents a piece of the serialized object.

A common idiom for polymorphic and/or variable-sized objects is to first serialize a constant-size tag or length and then serialize the variably sized payload. For this purpose, you may want to use tpie::serialize(dst, a, b) to serialize all elements in the range [a, b).

Referenced by tpie::serialization_writer::serialize(), serialize(), tpie::serialization_reverse_writer::serialize(), and serialized_size().

template<typename D , typename T >

void tpie::serialize	(	D &	dst,
		const T &	v,
		typename boost::enable_if< is_trivially_serializable< T > >::type *	= `0`,
		typename boost::disable_if< boost::is_pointer< T > >::type *	= `0`
	)

tpie::serialize for POD/array types.

Definition at line 108 of file serialization2.h.

                                                                         {
     dst.write((const char *)&v, sizeof(T));
 }

template<typename D , typename T >

void tpie::serialize	(	D &	dst,
		T	start,
		T	end
	)

Serialize an array of serializables.

This uses direct memory copying for POD typed arrays, and tpie::serialize for proper objects.

Definition at line 192 of file serialization2.h.

                                         {
     bits::array_encode_magic<D, T> magic;
     magic(dst, start, end);
 }

template<typename D , typename T , typename alloc_t >

void tpie::serialize	(	D &	dst,
		const std::vector< T, alloc_t > &	v
	)

tpie::serialize for std::vectors of serializable items.

Definition at line 213 of file serialization2.h.

References serialize().

                                                          {
     using tpie::serialize;
     serialize(dst, v.size());
     serialize(dst, v.begin(), v.end());
 }

template<typename D , typename T >

void tpie::serialize	(	D &	dst,
		const std::basic_string< T > &	v
	)

tpie::serialize for std::basic_strings of serializable items, including std::strings.

Definition at line 236 of file serialization2.h.

References serialize().

                                                       {
     using tpie::serialize;
     serialize(dst, v.size());
     serialize(dst, v.begin(), v.end());
 }

template<typename T >

size_t tpie::serialized_size ( const T & v )

Given a serializable, serialize it and measure its serialized size.

Definition at line 259 of file serialization2.h.

References serialize().

Referenced by tpie::serialization_bits::internal_sort< T, pred_t >::push().

                                     {
     using tpie::serialize;
     bits::counter c;
     serialize(c, v);
     return c.size;
 }

void tpie::set_block_size ( memory_size_type block_size )

Set the TPIE block size.

It is not safe to change the block size once TPIE has been initialized.

template<typename T , typename Compare >

void tpie::sort	(	file_stream< T > &	instream,
		file_stream< T > &	outstream,
		Compare	comp,
		progress_indicator_base &	indicator
	)

Sort elements of a stream using the given STL-style comparator object.

Definition at line 59 of file sort.h.

References tpie::sort_manager< T, I, M >::sort().

Referenced by tpie::parallel_sort_impl< iterator_type, comp_type, Progress, min_size >::qsort_job::operator()(), tpie::parallel_sort_impl< iterator_type, comp_type, Progress, min_size >::operator()(), parallel_sort(), sort(), and tpie::sort_manager< T, I, M >::sort().

                                                              {
 
     ami::Internal_Sorter_Obj<T,Compare> myInternalSorter(comp);
     ami::merge_heap_obj<T,Compare>      myMergeHeap(comp);
     sort_manager< T, ami::Internal_Sorter_Obj<T,Compare>, ami::merge_heap_obj<T,Compare> > 
     mySortManager(&myInternalSorter, &myMergeHeap);
 
     mySortManager.sort(&instream, &outstream, &indicator);
 }

template<typename T >

void tpie::sort	(	file_stream< T > &	instream,
		file_stream< T > &	outstream,
		tpie::progress_indicator_base *	indicator = `NULL`
	)

Sort elements of a stream using the less-than operator.

Definition at line 74 of file sort.h.

References sort().

                                                        {
     std::less<T> comp;
     sort(instream, outstream, comp, *indicator);
 }

template<typename T , typename Compare >

void tpie::sort	(	file_stream< T > &	instream,
		Compare	comp,
		progress_indicator_base &	indicator
	)

Sort elements of a stream in-place using the given STL-style comparator object.

Definition at line 93 of file sort.h.

References sort().

                                                {
     sort(instream, instream, comp, indicator);
 }

template<typename T >

void tpie::sort	(	file_stream< T > &	instream,
		progress_indicator_base &	indicator
	)

Sort elements of a stream in-place using the less-than operator.

Definition at line 102 of file sort.h.

References sort().

                                               {
     sort(instream, instream, &indicator);
 }

template<typename T >

void tpie::sort ( file_stream< T > & instream )

Sort elements of a stream in-place using the less-than operator and no progress indicator.

Definition at line 112 of file sort.h.

References sort().

                                      {
     sort(instream, instream);
 }

template<typename T >

void tpie::tpie_delete	(	T *	p	)
throw	(
	)

inline

Delete an object allocated with tpie_new.

Parameters

p	the object to delete

Definition at line 380 of file memory.h.

References __unregister_pointer(), get_memory_manager(), ptr_cast(), and tpie::memory_manager::register_deallocation().

Referenced by tpie::pipelining::bits::delayed_buffer_output_t< dest_t >::end(), tpie::pipelining::bits::reverser_output_t< dest_t >::end(), and tpie::ami::partition_and_merge().

                                        {
     if (p == 0) return;
     get_memory_manager().register_deallocation(tpie_size(p));
     uint8_t * pp = ptr_cast<uint8_t *>(p);
     __unregister_pointer(pp, tpie_size(p), typeid(*p));
     p->~T();
     if(!boost::is_polymorphic<T>::value) 
         delete[] pp;
     else
         delete[] (pp - sizeof(size_t));
 }

template<typename T >

void tpie::tpie_delete_array	(	T *	a,
		size_t	size
	)
throw	(
	)

inline

Delete an array allocated with tpie_new_array.

Parameters

a	The array to delete.
size	The size of the array in elements as passed to tpie_new_array.

Definition at line 398 of file memory.h.

References __unregister_pointer(), get_memory_manager(), and tpie::memory_manager::register_deallocation().

Referenced by tpie::file_stream_base::close(), tpie::ami::main_mem_merge(), tpie::ami::partition_and_merge(), tpie::packed_array< T, B >::resize(), and tpie::ami::single_merge().

                                                           {
     if (a == 0) return;
     get_memory_manager().register_deallocation(sizeof(T) * size);
     __unregister_pointer(a, sizeof(T) * size, typeid(T) );
     delete[] a;
 }

tpie::TPIE_DEPRECATED ( inline void increment_open_file_count() )

Deprecated; does nothing.

Definition at line 33 of file file_count.h.

                                                          {
     // Does nothing.
 }

void tpie::tpie_finish ( int subsystems = ALL )

Deinitialize the given subsystems of TPIE.

You MUST pass the same bitmask of subsystems to tpie_finish as you did to tpie_init.

Parameters

subsystems Logical OR of subsystem entries.

void tpie::tpie_init ( int subsystems = ALL )

Initialize the given subsystems of TPIE.

Parameters

subsystems Logical OR of subsystem entries.

template<typename T , typename Args >

T* tpie::tpie_new ( Args args )

inline

Allocate an element of the type given as template parameter, and register its memory usage with TPIE.

The implementation of tpie_new either uses variadic templates (if supported by the compiler) or a bunch of tpie_new overloads to support a variable number of constructor parameters.

Template Parameters

T	The type of element to allocate
Args	The variadic number of types of constructor parameters.

Parameters

args	The variadic number of arguments to pass to the constructor of T.

template<typename T >

T* tpie::tpie_new_array ( size_t size )

inline

Allocate a new array and register its memory usage.

Parameters

size	The number of elements in the new array.

Returns: The new array.

Definition at line 338 of file memory.h.

                                        {
     array_allocation_scope_magic<T> m(size);
     m.allocate();
     return m.finalize();
 }

template<typename S >

void tpie::unserialize	(	S &	src,
		foo &	v
	)

Sample tpie::unserialize prototype.

To enable unserialization of your own type, overload tpie::unserialize. This docstring is an example for a type named foo, but it is for exposition purposes only.

The implementation of tpie::unserialize(src, v) shall call src.read(dst, n) a number of times. Each time, src is a pointer to a byte buffer that can hold at least n bytes, where n is the number of bytes to be read.

Variable Documentation

const char* tpie::git_commit

The Git commit hash (40 hexadecimal characters) that TPIE was built from.

Referenced by tpie::sysinfo::commit().

const char* tpie::git_refspec

The Git refspec that TPIE was built from.

Usually of the form "refs/heads/"+branch, for instance "refs/heads/master".

Referenced by tpie::sysinfo::refspec().

void tpie::unserialize	(	S &	src,
		std::vector< T, alloc_t > &	v
	)

TPIE

Namespaces

Classes

Typedefs

Enumerations

Functions

Variables

Detailed Description

Typedef Documentation

Enumeration Type Documentation

Function Documentation

Variable Documentation