mxnet.recordio

Read and write for the RecordIO data format.

Classes

IRHeader

An alias for HEADER.

MXIndexedRecordIO(idx_path, uri, flag[, …])

Reads/writes RecordIO data format, supporting random access.

MXRecordIO(uri, flag)

Reads/writes RecordIO data format, supporting sequential read and write.

Functions

pack(header, s)

Pack a string into MXImageRecord.

pack_img(header, img[, quality, img_fmt])

Pack an image into MXImageRecord.

unpack(s)

Unpack a MXImageRecord to string.

unpack_img(s[, iscolor])

Unpack a MXImageRecord to image.

mxnet.recordio.IRHeader

An alias for HEADER. Used to store metadata (e.g. labels) accompanying a record. See mxnet.recordio.pack and mxnet.recordio.pack_img for example uses.

Parameters
  • flag (int) – Available for convenience, can be set arbitrarily.

  • label (float or an array of float) – Typically used to store label(s) for a record.

  • id (int) – Usually a unique id representing record.

  • id2 (int) – Higher order bits of the unique id, should be set to 0 (in most cases).

Attributes

flag

Alias for field number 0

id

Alias for field number 2

id2

Alias for field number 3

label

Alias for field number 1

alias of mxnet.recordio.HEADER

class mxnet.recordio.MXIndexedRecordIO(idx_path, uri, flag, key_type=<class 'int'>)[source]

Bases: mxnet.recordio.MXRecordIO

Reads/writes RecordIO data format, supporting random access.

Examples

Methods

close()

Closes the record file.

open()

Opens the record file.

read_idx(idx)

Returns the record at given index.

seek(idx)

Sets the current read pointer position.

tell()

Returns the current position of write head.

write_idx(idx, buf)

Inserts input record at given index.

>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r')
>>> record.read_idx(3)
record_3
Parameters
  • idx_path (str) – Path to the index file.

  • uri (str) – Path to the record file. Only supports seekable file types.

  • flag (str) – ‘w’ for write or ‘r’ for read.

  • key_type (type) – Data type for keys.

close()[source]

Closes the record file.

open()[source]

Opens the record file.

read_idx(idx)[source]

Returns the record at given index.

Examples

>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w')
>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r')
>>> record.read_idx(3)
record_3
seek(idx)[source]

Sets the current read pointer position.

This function is internally called by read_idx(idx) to find the current reader pointer position. It doesn’t return anything.

tell()[source]

Returns the current position of write head.

Examples

>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w')
>>> print(record.tell())
0
>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
...     print(record.tell())
16
32
48
64
80
write_idx(idx, buf)[source]

Inserts input record at given index.

Examples

>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
Parameters
  • idx (int) – Index of a file.

  • buf – Record to write.

class mxnet.recordio.MXRecordIO(uri, flag)[source]

Bases: object

Reads/writes RecordIO data format, supporting sequential read and write.

Examples

Methods

close()

Closes the record file.

open()

Opens the record file.

read()

Returns record as a string.

reset()

Resets the pointer to first item.

write(buf)

Inserts a string buffer as a record.

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'w')
<mxnet.recordio.MXRecordIO object at 0x10ef40ed0>
>>> for i in range(5):
...    record.write('record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(5):
...    item = record.read()
...    print(item)
record_0
record_1
record_2
record_3
record_4
>>> record.close()
Parameters
  • uri (string) – Path to the record file.

  • flag (string) – ‘w’ for write or ‘r’ for read.

close()[source]

Closes the record file.

open()[source]

Opens the record file.

read()[source]

Returns record as a string.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(5):
...    item = record.read()
...    print(item)
record_0
record_1
record_2
record_3
record_4
>>> record.close()
Returns

buf – Buffer read.

Return type

string

reset()[source]

Resets the pointer to first item.

If the record is opened with ‘w’, this function will truncate the file to empty.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(2):
...    item = record.read()
...    print(item)
record_0
record_1
>>> record.reset()  # Pointer is reset.
>>> print(record.read()) # Started reading from start again.
record_0
>>> record.close()
write(buf)[source]

Inserts a string buffer as a record.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'w')
>>> for i in range(5):
...    record.write('record_%d'%i)
>>> record.close()
Parameters

buf (string (python2), bytes (python3)) – Buffer to write.

mxnet.recordio.pack(header, s)[source]

Pack a string into MXImageRecord.

Parameters
  • header (IRHeader) – Header of the image record. header.label can be a number or an array. See more detail in IRHeader.

  • s (str) – Raw image string to be packed.

Returns

s – The packed string.

Return type

str

Examples

>>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3]
>>> id = 2574
>>> header = mx.recordio.IRHeader(0, label, id, 0)
>>> with open(path, 'r') as file:
...     s = file.read()
>>> packed_s = mx.recordio.pack(header, s)
mxnet.recordio.pack_img(header, img, quality=95, img_fmt='.jpg')[source]

Pack an image into MXImageRecord.

Parameters
  • header (IRHeader) – Header of the image record. header.label can be a number or an array. See more detail in IRHeader.

  • img (numpy.ndarray) – Image to be packed.

  • quality (int) – Quality for JPEG encoding in range 1-100, or compression for PNG encoding in range 1-9.

  • img_fmt (str) – Encoding of the image (.jpg for JPEG, .png for PNG).

Returns

s – The packed string.

Return type

str

Examples

>>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3]
>>> id = 2574
>>> header = mx.recordio.IRHeader(0, label, id, 0)
>>> img = cv2.imread('test.jpg')
>>> packed_s = mx.recordio.pack_img(header, img)
mxnet.recordio.unpack(s)[source]

Unpack a MXImageRecord to string.

Parameters

s (str) – String buffer from MXRecordIO.read.

Returns

  • header (IRHeader) – Header of the image record.

  • s (str) – Unpacked string.

Examples

>>> record = mx.recordio.MXRecordIO('test.rec', 'r')
>>> item = record.read()
>>> header, s = mx.recordio.unpack(item)
>>> header
HEADER(flag=0, label=14.0, id=20129312, id2=0)
mxnet.recordio.unpack_img(s, iscolor=-1)[source]

Unpack a MXImageRecord to image.

Parameters
  • s (str) – String buffer from MXRecordIO.read.

  • iscolor (int) – Image format option for cv2.imdecode.

Returns

  • header (IRHeader) – Header of the image record.

  • img (numpy.ndarray) – Unpacked image.

Examples

>>> record = mx.recordio.MXRecordIO('test.rec', 'r')
>>> item = record.read()
>>> header, img = mx.recordio.unpack_img(item)
>>> header
HEADER(flag=0, label=14.0, id=20129312, id2=0)
>>> img
array([[[ 23,  27,  45],
        [ 28,  32,  50],
        ...,
        [ 36,  40,  59],
        [ 35,  39,  58]],
       ...,
       [[ 91,  92, 113],
        [ 97,  98, 119],
        ...,
        [168, 169, 167],
        [166, 167, 165]]], dtype=uint8)