dmlc::Parser< IndexType, DType > Class Template Reference

parser interface that parses input data used to load dmlc data format into your own data format Difference between RowBlockIter and Parser: RowBlockIter caches the data internally that can be used to iterate the dataset multiple times, Parser holds very limited internal state and was usually used to read data only once More...

#include <data.h>

Inheritance diagram for dmlc::Parser< IndexType, DType >:

Collaboration diagram for dmlc::Parser< IndexType, DType >:

Public Types
typedef Parser< IndexType, DType > *(*	Factory) (const std::string &path, const std::map< std::string, std::string > &args, unsigned part_index, unsigned num_parts)
	Factory type of the parser. More...

Public Member Functions
virtual size_t	BytesRead (void) const =0
Public Member Functions inherited from dmlc::DataIter< RowBlock< IndexType, DType > >
virtual	~DataIter (void) DMLC_THROW_EXCEPTION
	destructor More...
virtual void	BeforeFirst (void)=0
	set before first of the item More...
virtual bool	Next (void)=0
	move to next item More...
virtual const RowBlock< IndexType, DType > &	Value (void) const =0
	get current data More...

Static Public Member Functions
static Parser< IndexType, DType > *	Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type)
	create a new instance of parser based on the "type" More...

Detailed Description

template<typename IndexType, typename DType = real_t>
class dmlc::Parser< IndexType, DType >

parser interface that parses input data used to load dmlc data format into your own data format Difference between RowBlockIter and Parser: RowBlockIter caches the data internally that can be used to iterate the dataset multiple times, Parser holds very limited internal state and was usually used to read data only once

See also

RowBlockIter

Template Parameters

IndexType	type of index in RowBlock
DType	type of label and value in RowBlock Create function was only implemented for IndexType uint64_t and uint32_t and DType real_t and int

Member Typedef Documentation

template<typename IndexType, typename DType = real_t>

typedef Parser<IndexType, DType>*(* dmlc::Parser< IndexType, DType >::Factory) (const std::string &path, const std::map< std::string, std::string > &args, unsigned part_index, unsigned num_parts)

Factory type of the parser.

Member Function Documentation

template<typename IndexType, typename DType = real_t>

virtual size_t dmlc::Parser< IndexType, DType >::BytesRead ( void  ) const

pure virtual

Returns

size of bytes read so far

template<typename IndexType, typename DType = real_t>

static Parser<IndexType, DType>* dmlc::Parser< IndexType, DType >::Create ( const char *  uri_, unsigned  part_index, unsigned  num_parts, const char *  type  )

static

create a new instance of parser based on the "type"

Parameters

uri_	the uri of the input, can contain hdfs prefix
part_index	the part id of current input
num_parts	total number of splits
type	type of dataset can be: "libsvm", "auto", ...

When "auto" is passed, the type is decided by format argument string in URI.

Returns

the created parser

---

The documentation for this class was generated from the following file:

• /work/mxnet/3rdparty/dmlc-core/include/dmlc/data.h