org.apache.gora.util

Class IOUtils

java.lang.Object

org.apache.gora.util.IOUtils

public class IOUtils
extends Object

An utility class for I/O related functionality.

Field Summary

Fields

Modifier and Type	Field and Description
static int	BUFFER_SIZE

Constructor Summary

Constructors

Constructor and Description
IOUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static <K,T> T	deserialize(byte[] bytes, org.apache.avro.specific.SpecificDatumReader<T> datumReader, org.apache.avro.Schema schema, T object) Deserializes the field object using the datumReader.
static <K,T extends org.apache.avro.specific.SpecificRecord> T	deserialize(byte[] bytes, org.apache.avro.specific.SpecificDatumReader<T> datumReader, org.apache.avro.Schema schema, T object) Deserializes the field object using the datumReader.
static <T> T	deserialize(org.apache.hadoop.conf.Configuration conf, byte[] in, T obj) Deserializes the object in the given datainput using available Hadoop serializations.
static <T> T	deserialize(org.apache.hadoop.conf.Configuration conf, DataInput in, T obj) Deserializes the object in the given datainput using available Hadoop serializations.
static <T> T	deserialize(org.apache.hadoop.conf.Configuration conf, DataInput in, T obj, Class<T> objClass) Deserializes the object in the given datainput using available Hadoop serializations.
static <T> T	deserialize(org.apache.hadoop.conf.Configuration conf, DataInput in, T obj, String objClass) Deserializes the object in the given datainput using available Hadoop serializations.
static <K,T extends org.apache.avro.specific.SpecificRecord> T	deserialize(InputStream is, org.apache.avro.specific.SpecificDatumReader<T> datumReader, org.apache.avro.Schema schema, T object) Deserializes the field object using the datumReader.
static byte[]	getAsBytes(List<ByteBuffer> buffers) Copies the contents of the buffers into a single byte[]
static <T> T	loadFromConf(org.apache.hadoop.conf.Configuration conf, String dataKey) Loads the object stored by storeToConf(Object, Configuration, String) method from the configuration under the given dataKey.
static boolean[]	readBoolArray(DataInput in) Reads a boolean[] from input
static boolean[]	readBoolArray(org.apache.avro.io.Decoder in) Reads a boolean[] from input
static byte[]	readFully(InputStream in) Reads until the end of the input stream, and returns the contents as a byte[]
static boolean[]	readNullFieldsInfo(DataInput in) Reads the data written by writeNullFieldsInfo(DataOutput, Object...) and returns a boolean array representing whether each field is null or not.
static Object	readObject(DataInput in)
static String[]	readStringArray(DataInput in) Reads and returns a String array that is written by writeStringArray(DataOutput, String[]).
static <T> void	serialize(org.apache.hadoop.conf.Configuration conf, DataOutput out, T obj) Serializes the object to the given dataoutput using available Hadoop serializations
static <T> void	serialize(org.apache.hadoop.conf.Configuration conf, DataOutput out, T obj, Class<T> objClass) Serializes the object to the given dataoutput using available Hadoop serializations
static <T> byte[]	serialize(org.apache.hadoop.conf.Configuration conf, T obj) Serializes the object to the given dataoutput using available Hadoop serializations
static <T> void	serialize(OutputStream os, org.apache.avro.specific.SpecificDatumWriter<T> datumWriter, org.apache.avro.Schema schema, T object) Serializes the field object using the datumWriter.
static <T extends org.apache.avro.specific.SpecificRecord> void	serialize(OutputStream os, org.apache.avro.specific.SpecificDatumWriter<T> datumWriter, org.apache.avro.Schema schema, T object) Serializes the field object using the datumWriter.
static <T> byte[]	serialize(org.apache.avro.specific.SpecificDatumWriter<T> datumWriter, org.apache.avro.Schema schema, T object) Serializes the field object using the datumWriter.
static <T extends org.apache.avro.specific.SpecificRecord> byte[]	serialize(org.apache.avro.specific.SpecificDatumWriter<T> datumWriter, org.apache.avro.Schema schema, T object) Serializes the field object using the datumWriter.
static <T> void	storeToConf(T obj, org.apache.hadoop.conf.Configuration conf, String dataKey) Stores the given object in the configuration under the given dataKey
static void	writeBoolArray(DataOutput out, boolean[] boolArray) Writes a boolean[] to the output.
static void	writeBoolArray(org.apache.avro.io.Encoder out, boolean[] boolArray) Writes a boolean[] to the output.
static void	writeNullFieldsInfo(DataOutput out, Object... fields) Writes a byte[] to the output, representing whether each given field is null or not.
static void	writeObject(DataOutput out, Object obj)
static void	writeStringArray(DataOutput out, String[] arr) Writes the String array to the given DataOutput.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BUFFER_SIZE

public static final int BUFFER_SIZE

See Also:

Constant Field Values

Constructor Detail

IOUtils

public IOUtils()

Method Detail

readObject

public static Object readObject(DataInput in)
                         throws ClassNotFoundException,
                                IOException

Throws:

ClassNotFoundException

IOException

writeObject

public static void writeObject(DataOutput out,
               Object obj)
                        throws IOException

Throws:

IOException

serialize

public static <T> void serialize(org.apache.hadoop.conf.Configuration conf,
                 DataOutput out,
                 T obj,
                 Class<T> objClass)
                      throws IOException

Serializes the object to the given dataoutput using available Hadoop serializations

Throws:

IOException

serialize

public static <T> void serialize(org.apache.hadoop.conf.Configuration conf,
                 DataOutput out,
                 T obj)
                      throws IOException

Serializes the object to the given dataoutput using available Hadoop serializations

Throws:

IOException

serialize

public static <T> byte[] serialize(org.apache.hadoop.conf.Configuration conf,
                   T obj)
                        throws IOException

Serializes the object to the given dataoutput using available Hadoop serializations

Throws:

IOException

serialize

public static <T extends org.apache.avro.specific.SpecificRecord> void serialize(OutputStream os,
                                                                 org.apache.avro.specific.SpecificDatumWriter<T> datumWriter,
                                                                 org.apache.avro.Schema schema,
                                                                 T object)
                      throws IOException

Serializes the field object using the datumWriter.

Throws:

IOException

serialize

public static <T> void serialize(OutputStream os,
                 org.apache.avro.specific.SpecificDatumWriter<T> datumWriter,
                 org.apache.avro.Schema schema,
                 T object)
                      throws IOException

Serializes the field object using the datumWriter.

Throws:

IOException

serialize

public static <T extends org.apache.avro.specific.SpecificRecord> byte[] serialize(org.apache.avro.specific.SpecificDatumWriter<T> datumWriter,
                                                                   org.apache.avro.Schema schema,
                                                                   T object)
                        throws IOException

Serializes the field object using the datumWriter.

Throws:

IOException

serialize

public static <T> byte[] serialize(org.apache.avro.specific.SpecificDatumWriter<T> datumWriter,
                   org.apache.avro.Schema schema,
                   T object)
                        throws IOException

Serializes the field object using the datumWriter.

Throws:

IOException

deserialize

public static <T> T deserialize(org.apache.hadoop.conf.Configuration conf,
                DataInput in,
                T obj,
                String objClass)
                     throws IOException,
                            ClassNotFoundException

Deserializes the object in the given datainput using available Hadoop serializations.

Throws:

IOException

ClassNotFoundException

deserialize

public static <T> T deserialize(org.apache.hadoop.conf.Configuration conf,
                DataInput in,
                T obj,
                Class<T> objClass)
                     throws IOException

Deserializes the object in the given datainput using available Hadoop serializations.

Throws:

IOException

deserialize

public static <T> T deserialize(org.apache.hadoop.conf.Configuration conf,
                DataInput in,
                T obj)
                     throws IOException,
                            ClassNotFoundException

Deserializes the object in the given datainput using available Hadoop serializations.

Throws:

IOException

ClassNotFoundException

deserialize

public static <T> T deserialize(org.apache.hadoop.conf.Configuration conf,
                byte[] in,
                T obj)
                     throws IOException,
                            ClassNotFoundException

Deserializes the object in the given datainput using available Hadoop serializations.

Throws:

IOException

ClassNotFoundException

deserialize

public static <K,T extends org.apache.avro.specific.SpecificRecord> T deserialize(InputStream is,
                                                                  org.apache.avro.specific.SpecificDatumReader<T> datumReader,
                                                                  org.apache.avro.Schema schema,
                                                                  T object)
                                                                     throws IOException

Deserializes the field object using the datumReader.

Throws:

IOException

deserialize

public static <K,T extends org.apache.avro.specific.SpecificRecord> T deserialize(byte[] bytes,
                                                                  org.apache.avro.specific.SpecificDatumReader<T> datumReader,
                                                                  org.apache.avro.Schema schema,
                                                                  T object)
                                                                     throws IOException

Deserializes the field object using the datumReader.

Throws:

IOException

deserialize

public static <K,T> T deserialize(byte[] bytes,
                  org.apache.avro.specific.SpecificDatumReader<T> datumReader,
                  org.apache.avro.Schema schema,
                  T object)
                     throws IOException

Deserializes the field object using the datumReader.

Throws:

IOException

writeNullFieldsInfo

public static void writeNullFieldsInfo(DataOutput out,
                       Object... fields)
                                throws IOException

Writes a byte[] to the output, representing whether each given field is null or not. A Vint and ceil( fields.length / 8 ) bytes are written to the output.

Parameters:

out - the output to write to

fields - the fields to check for null

Throws:

IOException

See Also:

readNullFieldsInfo(DataInput)

readNullFieldsInfo

public static boolean[] readNullFieldsInfo(DataInput in)
                                    throws IOException

Reads the data written by writeNullFieldsInfo(DataOutput, Object...) and returns a boolean array representing whether each field is null or not.

Parameters:

in - the input to read from

Returns:

a boolean[] representing whether each field is null or not.

Throws:

IOException

writeBoolArray

public static void writeBoolArray(DataOutput out,
                  boolean[] boolArray)
                           throws IOException

Writes a boolean[] to the output.

Throws:

IOException

readBoolArray

public static boolean[] readBoolArray(DataInput in)
                               throws IOException

Reads a boolean[] from input

Throws:

IOException

writeBoolArray

public static void writeBoolArray(org.apache.avro.io.Encoder out,
                  boolean[] boolArray)
                           throws IOException

Writes a boolean[] to the output.

Throws:

IOException

readBoolArray

public static boolean[] readBoolArray(org.apache.avro.io.Decoder in)
                               throws IOException

Reads a boolean[] from input

Throws:

IOException

writeStringArray

public static void writeStringArray(DataOutput out,
                    String[] arr)
                             throws IOException

Writes the String array to the given DataOutput.

Parameters:

out - the data output to write to

arr - the array to write

Throws:

IOException

See Also:

readStringArray(DataInput)

readStringArray

public static String[] readStringArray(DataInput in)
                                throws IOException

Reads and returns a String array that is written by writeStringArray(DataOutput, String[]).

Parameters:

in - the data input to read from

Returns:

read String[]

Throws:

IOException

storeToConf

public static <T> void storeToConf(T obj,
                   org.apache.hadoop.conf.Configuration conf,
                   String dataKey)
                        throws IOException

Stores the given object in the configuration under the given dataKey

Parameters:

obj - the object to store

conf - the configuration to store the object into

dataKey - the key to store the data

Throws:

IOException

loadFromConf

public static <T> T loadFromConf(org.apache.hadoop.conf.Configuration conf,
                 String dataKey)
                      throws IOException

Loads the object stored by storeToConf(Object, Configuration, String) method from the configuration under the given dataKey.

Parameters:

conf - the configuration to read from

dataKey - the key to get the data from

Returns:

the store object

Throws:

IOException

getAsBytes

public static byte[] getAsBytes(List<ByteBuffer> buffers)

Copies the contents of the buffers into a single byte[]

readFully

public static byte[] readFully(InputStream in)
                        throws IOException

Reads until the end of the input stream, and returns the contents as a byte[]

Throws:

IOException