tpie::ami::stream< T > Class Template Reference

A Stream<T> object stores an ordered collection of objects of type T on external memory. More...

#include <tpie/stream.h>

Public Types
typedef T	item_type

Public Member Functions
	stream ()
	A new stream of type READ_WRITE_STREAM is constructed on the given device as a file with a randomly generated name, prefixed by "". More...
	stream (const std::string &path_name, stream_type st=READ_WRITE_STREAM)
	A new stream is constructed and named and placed according to the given parameter pathname. More...
err	new_substream (stream_type st, stream_offset_type sub_begin, stream_offset_type sub_end, stream< T > **sub_stream)
	A substream is a TPIE stream that is part of another TPIE stream. More...
stream_status	status () const
	Returns the status of the stream instance; the result is either STREAM_STATUS_VALID or STREAM_STATUS_INVALID. More...
bool	is_valid () const
	Returns wether the status of the stream is STREAM_STATUS_VALID. More...
bool	operator! () const
	Returns true if the block's status is not BLOCK_STATUS_VALID. More...
err	read_item (T **elt)
	Reads the current item from the stream and advance the "current item" pointer to the next item. More...
err	write_item (const T &elt)
	Writes elt to the stream in the current position. More...
err	read_array (T *mm_space, stream_offset_type *len)
	Reads *len items from the current position of the stream into the array mm_array. More...
err	read_array (T *mm_space, memory_size_type &len)
	Reads len items from the current position of the stream into the array mm_array. More...
err	write_array (const T *mm_space, memory_size_type len)
	Writes len items from array |mm_array to the stream, starting in the current position. More...
stream_offset_type	stream_len (void) const
	Returns the number of items in the stream. More...
std::string	name () const
	Returns the path name of this stream in newly allocated space. More...
err	seek (stream_offset_type offset)
	Move the current position to off (measured in terms of items. More...
stream_offset_type	tell () const
	Returns the current position in the stream measured of items from the beginning of the stream. More...
err	truncate (stream_offset_type offset)
	Resize the stream to off items. More...
err	main_memory_usage (size_type *usage, stream_usage usage_type) const
	This function is used for obtaining the amount of main memory used by an Stream<T> object (in bytes). More...
size_t	available_streams (void)
	Returns the number of globally available streams. More...
memory_size_type	chunk_size (void) const
	Returns the maximum number of items (of type T) that can be stored in one block. More...
void	persist (persistence p)
	Set the stream's persistence flag to p, which can have one of two values: PERSIST_DELETE or PERSIST_PERSISTENT. More...
persistence	persist () const
	Set the stram's persistence flag to PERSIST_PERSISTENT, thereby ensuring it is not deleted when destructed. More...
std::string &	sprint ()
	Return a string describing the stream. More...
file_stream< T > &	underlying_stream ()
	Get the underlying file_stream<T> More...

Static Public Member Functions
static memory_size_type	memory_usage (memory_size_type count)
	Returns the number of bytes that count streams will maximaly consume. More...

Detailed Description

template<class T>
class tpie::ami::stream< T >

A Stream<T> object stores an ordered collection of objects of type T on external memory.

The type of a Stream indicates what operations are permitted on the stream. Stream types provided in TPIE are the following:

READ_STREAM:

Input operations on the stream are permitted, but output is not permitted.

WRITE_STREAM:

Output operations are permitted, but input operations are not permitted.

APPEND_STREAM:

Output is appended to the end of the stream. Input operations are not permitted. This is similar to WRITE_STREAM except that if the stream is constructed on a file containing an existing stream, objects written to the stream will be appended at the end of the stream.

READ_WRITE_STREAM:

Both input and output operations are permitted.

Definition at line 99 of file stream.h.

Constructor & Destructor Documentation

template<class T >

tpie::ami::stream< T >::stream

(

)

A new stream of type READ_WRITE_STREAM is constructed on the given device as a file with a randomly generated name, prefixed by "".

Definition at line 366 of file stream.h.

References tpie::temp_file::path(), and tpie::ami::STREAM_STATUS_VALID.

                     : m_stream(block_factor()), m_status(STREAM_STATUS_INVALID)
    {
        TP_LOG_DEBUG_ID("Temporary stream in file: ");
        TP_LOG_DEBUG_ID( m_temp.path() );
        try {
            m_stream.open( m_temp );
        } catch(const stream_exception &e) {
            TP_LOG_FATAL_ID("Open failed: " << e.what());
            return;
        }
        //  Set status to VALID.
        m_status = STREAM_STATUS_VALID;
    };

template<class T >

tpie::ami::stream< T >::stream	(	const std::string &	path_name,
		stream_type	st = READ_WRITE_STREAM
	)

A new stream is constructed and named and placed according to the given parameter pathname.

Its type is given by st which defaults to READ_WRITE_STREAM.

Definition at line 384 of file stream.h.

References tpie::access_read, tpie::access_read_write, and tpie::ami::STREAM_STATUS_VALID.

                                                                :
        m_temp(path_name, true), m_stream(block_factor()), m_status(STREAM_STATUS_INVALID) {
        try {
            m_stream.open(m_temp, st==READ_STREAM ? access_read: access_read_write);
            if (st == APPEND_STREAM) m_stream.seek(0, file_stream_base::end);
        } catch(const stream_exception &e) {
            TP_LOG_FATAL_ID("Open failed: " <<  e.what());
            return;
        }
        m_status = STREAM_STATUS_VALID;
    };

Member Function Documentation

template<class T>

size_t tpie::ami::stream< T >::available_streams ( void  )

inline

Returns the number of globally available streams.

The number should resemble the the maximum number of streams allowed (which is OS-dependent) minus the number of streams currently opened by TPIE.

Definition at line 293 of file stream.h.

References tpie::available_files().

Referenced by tpie::ami::partition_and_merge().

                                   {
        return available_files();
    }

template<class T>

memory_size_type tpie::ami::stream< T >::chunk_size ( void  ) const

inline

Returns the maximum number of items (of type T) that can be stored in one block.

Definition at line 301 of file stream.h.

References tpie::file_base_crtp< file_base >::block_size().

Referenced by tpie::ami::partition_and_merge().

                                            { 
        return file_base::block_size(1.0) / sizeof(T);
    }

template<class T>

bool tpie::ami::stream< T >::is_valid ( ) const

inline

Returns wether the status of the stream is STREAM_STATUS_VALID.

See also

status()

Definition at line 160 of file stream.h.

References tpie::ami::STREAM_STATUS_VALID.

Referenced by tpie::ami::stream< T >::operator!().

                                 { 
        return m_status == STREAM_STATUS_VALID; 
    }

template<class T>

err tpie::ami::stream< T >::main_memory_usage	(	size_type *	usage,
		stream_usage	usage_type
	)		const

This function is used for obtaining the amount of main memory used by an Stream<T> object (in bytes).

Parameters

[in]

usage_type

of type MM_stream_usage and is one of the following:

MM_STREAM_USAGE_CURRENT

Total amount of memory currently used by the stream.

MM_STREAM_USAGE_MAXIMUM

Max amount of memory that will ever be used by the stream.

MM_STREAM_USAGE_OVERHEAD

The amount of memory used by the object itself, without the data buffer.

MM_STREAM_USAGE_BUFFER

The amount of memory used by the data buffer.

MM_STREAM_USAGE_SUBSTREAM

The additional amount of memory that will be used by each substream created.

Parameters

[out]

usage

amount of memory in bytes used by the stream

Definition at line 449 of file stream.h.

References tpie::ami::BTE_ERROR, tpie::file_stream< T >::memory_usage(), tpie::ami::NO_ERROR, tpie::STREAM_USAGE_BUFFER, tpie::STREAM_USAGE_CURRENT, tpie::STREAM_USAGE_MAXIMUM, tpie::STREAM_USAGE_OVERHEAD, and tpie::STREAM_USAGE_SUBSTREAM.

Referenced by tpie::ami::merge(), and tpie::ami::partition_and_merge().

                                                    {

        switch (usage_type) {
        case STREAM_USAGE_OVERHEAD:
            *usage = sizeof(*this) + file_stream<T>::memory_usage(0.0);
            return NO_ERROR;
        case STREAM_USAGE_CURRENT:
        case STREAM_USAGE_MAXIMUM:
        case STREAM_USAGE_SUBSTREAM:
            *usage =  memory_usage(1);
            return NO_ERROR;
        case STREAM_USAGE_BUFFER:
            *usage = file_stream<T>::memory_usage(block_factor()) - file_stream<T>::memory_usage(0.0); 
            return NO_ERROR;
        }
        return BTE_ERROR;
    }

template<class T >

memory_size_type tpie::ami::stream< T >::memory_usage ( memory_size_type  count )

static

Returns the number of bytes that count streams will maximaly consume.

Definition at line 442 of file stream.h.

References tpie::file_stream< T >::memory_usage().

                                                               {
    return count*(file_stream<T>::memory_usage(block_factor()) + sizeof(stream<T>));
}

template<class T >

std::string tpie::ami::stream< T >::name ( ) const

inline

Returns the path name of this stream in newly allocated space.

Definition at line 413 of file stream.h.

                                           {
        return m_stream.path();
    }

template<class T >

err tpie::ami::stream< T >::new_substream	(	stream_type	st,
		stream_offset_type	sub_begin,
		stream_offset_type	sub_end,
		stream< T > **	sub_stream
	)

A substream is a TPIE stream that is part of another TPIE stream.

More precisely, a substream B of a stream A is defined as a contiguous range of objects from the ordered collection of objects that make up the stream A. If desired, one can construct substreams of substreams of substreams ad infinitum. Since a substream is a stream in its own right, many of the stream member functions can be applied to a substream. A substream can be created via this pseudo-constructor. The reason we do not use a real constructor is to get around the fact that constructors can not be virtual.

Parameters

[in]	st	specifies the type of the substream
[in]	sub_begin	offset, that defines the begin of the substream within the original stream
[in]	sub_end	offset, that defines the end of the substream within the original stream
[out]	sub_stream	upon completion points to the newly created substream.

Definition at line 399 of file stream.h.

References tpie::ami::BTE_ERROR, and tpie::unused().

    {
        unused(st);
        unused(sub_begin);
        unused(sub_end);
        unused(sub_stream);
        return BTE_ERROR;
    }

template<class T>

bool tpie::ami::stream< T >::operator! ( ) const

inline

Returns true if the block's status is not BLOCK_STATUS_VALID.

See also

is_valid(), status()

Definition at line 168 of file stream.h.

References tpie::ami::stream< T >::is_valid().

                                  { 
        return !is_valid(); 
    }

template<class T>

void tpie::ami::stream< T >::persist ( persistence  p )

inline

Set the stream's persistence flag to p, which can have one of two values: PERSIST_DELETE or PERSIST_PERSISTENT.

}

Definition at line 309 of file stream.h.

References PERSIST_PERSISTENT, and tpie::temp_file::set_persistent().

Referenced by tpie::ami::partition_and_merge().

                                {
        m_temp.set_persistent(p == PERSIST_PERSISTENT);
    }

template<class T>

persistence tpie::ami::stream< T >::persist ( ) const

inline

Set the stram's persistence flag to PERSIST_PERSISTENT, thereby ensuring it is not deleted when destructed.

Definition at line 317 of file stream.h.

References tpie::temp_file::is_persistent(), PERSIST_DELETE, and PERSIST_PERSISTENT.

                                { 
        return m_temp.is_persistent() ? PERSIST_PERSISTENT : PERSIST_DELETE;
    }

template<class T >

err tpie::ami::stream< T >::read_array	(	T *	mm_space,
		stream_offset_type *	len
	)

Reads *len items from the current position of the stream into the array mm_array.

The "current position" pointer is increased accordingly.

Definition at line 484 of file stream.h.

Referenced by tpie::ami::main_mem_merge(), and tpie::ami::partition_and_merge().

                                                                  {
        size_type l=(size_t)*len;
        err e = read_array(mm_space, l);
        *len = l;
        return e;
    }

template<class T >

err tpie::ami::stream< T >::read_array	(	T *	mm_space,
		memory_size_type &	len
	)

Reads len items from the current position of the stream into the array mm_array.

The "current position" pointer is increased accordingly.

Definition at line 492 of file stream.h.

References tpie::ami::END_OF_STREAM, and tpie::ami::NO_ERROR.

                                                                 {
        size_type l = static_cast<size_type>(std::min(
            static_cast<stream_size_type>(len), 
            static_cast<stream_size_type>(m_stream.size() - m_stream.offset())));
        m_stream.read(mm_space, mm_space+l);
        return (l == len)?NO_ERROR:END_OF_STREAM;
    }

template<class T >

err tpie::ami::stream< T >::read_item ( T **  elt )

inline

Reads the current item from the stream and advance the "current item" pointer to the next item.

The item read is pointed to by *elt. If no error has occurred, return NO_ERROR. If the ``current item'' pointer is beyond the last item in the stream, ERROR_END_OF_STREAM is returned

Definition at line 469 of file stream.h.

References tpie::ami::END_OF_STREAM, and tpie::ami::NO_ERROR.

Referenced by tpie::ami::single_merge().

                                           {
        if (!m_stream.can_read())
        return END_OF_STREAM;

        *elt = &(const_cast<T &>(m_stream.read()));
        return NO_ERROR;
    }

template<class T >

err tpie::ami::stream< T >::seek ( stream_offset_type  offset )

inline

Move the current position to off (measured in terms of items.

Definition at line 419 of file stream.h.

References tpie::ami::BTE_ERROR, and tpie::ami::NO_ERROR.

Referenced by tpie::ami::main_mem_merge(), and tpie::ami::partition_and_merge().

                                                        {
        try {
            m_stream.seek(offset);
        } catch(const stream_exception &e) {
            TP_LOG_WARNING_ID("BTE error - seek failed: " << e.what());
            return BTE_ERROR;
        }
        return NO_ERROR;
    }

template<class T >

std::string & tpie::ami::stream< T >::sprint

(

)

Return a string describing the stream.

Definition at line 507 of file stream.h.

                                 {
        static std::string buf;
        std::stringstream ss;
        ss << "STREAM " << name() <<  " " << static_cast<long>(stream_len());
        ss >> buf;
        return buf;
    }

template<class T>

stream_status tpie::ami::stream< T >::status ( ) const

inline

Returns the status of the stream instance; the result is either STREAM_STATUS_VALID or STREAM_STATUS_INVALID.

The only operation that can leave the stream invalid is the constructor (if that happens, the log file contains more information). No items should be read from or written to an invalid stream.

Definition at line 152 of file stream.h.

                                        { 
        return m_status; 
    }

template<class T>

stream_offset_type tpie::ami::stream< T >::stream_len ( void  ) const

inline

Returns the number of items in the stream.

Definition at line 213 of file stream.h.

Referenced by tpie::ami::main_mem_merge(), and tpie::ami::partition_and_merge().

                                                     { 
        return m_stream.size();
    }

template<class T>

stream_offset_type tpie::ami::stream< T >::tell ( ) const

inline

Returns the current position in the stream measured of items from the beginning of the stream.

Definition at line 231 of file stream.h.

                                           { 
        return m_stream.offset(); 
    }

template<class T >

err tpie::ami::stream< T >::truncate ( stream_offset_type  offset )

inline

Resize the stream to off items.

If off is less than the number of objects in the stream, truncate() truncates the stream to off objects. If off is more than the number of objects in the stream, truncate() extends the stream to the specified number of objects. In either case, the "current item" pointer will be moved to the new end of the stream.

Definition at line 431 of file stream.h.

References tpie::ami::BTE_ERROR, and tpie::ami::NO_ERROR.

                                                            {
        try {
            m_stream.truncate(offset);
        } catch(const stream_exception & e) {
            TP_LOG_WARNING_ID("BTE error - truncate failed: " << e.what());
            return BTE_ERROR;
        }
        return NO_ERROR;
    }

template<class T>

file_stream<T>& tpie::ami::stream< T >::underlying_stream ( )

inline

Get the underlying file_stream<T>

Definition at line 330 of file stream.h.

                                        {
        return m_stream;
    }

template<class T>

err tpie::ami::stream< T >::write_array	(	const T *	mm_space,
		memory_size_type	len
	)

Writes len items from array |mm_array to the stream, starting in the current position.

The "current item" pointer is increased accordingly.

Definition at line 501 of file stream.h.

References tpie::ami::NO_ERROR.

Referenced by tpie::ami::main_mem_merge(), and tpie::ami::partition_and_merge().

                                                               {
        m_stream.write(mm_space, mm_space+len);
        return NO_ERROR;
    }

template<class T >

err tpie::ami::stream< T >::write_item ( const T &  elt )

inline

Writes elt to the stream in the current position.

Advance the "current item" pointer to the next item. If no error has occurred NO_ERROR is returned.

Definition at line 478 of file stream.h.

References tpie::ami::NO_ERROR.

Referenced by tpie::ami::single_merge().

                                                 {
        m_stream.write(elt);
        return NO_ERROR;
    }

---

The documentation for this class was generated from the following file:

• stream.h