org.apache.gora.avro.store

Class AvroStore<K,T extends PersistentBase>

java.lang.Object

org.apache.gora.store.impl.DataStoreBase<K,T>

org.apache.gora.store.impl.FileBackedDataStoreBase<K,T>

org.apache.gora.avro.store.AvroStore<K,T>

Type Parameters:

T -

All Implemented Interfaces:

Closeable, AutoCloseable, DataStore<K,T>, FileBackedDataStore<K,T>, org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.Writable

Direct Known Subclasses:

DataFileAvroStore

public class AvroStore<K,T extends PersistentBase>
extends FileBackedDataStoreBase<K,T>
implements org.apache.hadoop.conf.Configurable

An adapter DataStore for binary-compatible Avro serializations. AvroDataStore supports Binary and JSON serializations.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AvroStore.CodecType The type of the avro Encoder/Decoder.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CODEC_TYPE_KEY The property key specifying avro encoder/decoder type to use.
static org.slf4j.Logger	LOG

Fields inherited from class org.apache.gora.store.impl.FileBackedDataStoreBase

inputPath, inputSize, inputStream, outputPath, outputStream

Fields inherited from class org.apache.gora.store.impl.DataStoreBase

autoCreateSchema, beanFactory, conf, fieldMap, keyClass, persistentClass, properties, schema

Constructor Summary

Constructors

Constructor and Description
AvroStore()

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close the DataStore.
protected org.apache.avro.io.DatumReader<T>	createDatumReader()
protected org.apache.avro.io.DatumWriter<T>	createDatumWriter()
protected org.apache.avro.io.Decoder	createDecoder()
protected org.apache.avro.io.Encoder	createEncoder()
boolean	delete(K key) Deletes the object with the given key
long	deleteByQuery(Query<K,T> query) Deletes all the objects matching the query.
protected Result<K,T>	executePartial(FileSplitPartitionQuery<K,T> query) Executes a PartitialQuery, reading the data between start and end.
protected Result<K,T>	executeQuery(Query<K,T> query) Executes a normal Query reading the whole data.
void	flush() Forces the write caches to be flushed.
T	get(K key, String[] fields) Returns the object corresponding to the given key.
org.apache.hadoop.conf.Configuration	getConf()
org.apache.avro.io.DatumReader<T>	getDatumReader()
org.apache.avro.io.DatumWriter<T>	getDatumWriter()
org.apache.avro.io.Decoder	getDecoder()
org.apache.avro.io.Encoder	getEncoder()
String	getSchemaName() Returns the schema name given to this DataStore
void	initialize(Class<K> keyClass, Class<T> persistentClass, Properties properties) Initializes this DataStore.
AvroQuery<K,T>	newQuery() Constructs and returns a new Query.
void	put(K key, T obj) Inserts the persistent object with the given key.
void	readFields(DataInput in)
void	setCodecType(AvroStore.CodecType codecType)
void	setDatumReader(org.apache.avro.io.DatumReader<T> datumReader)
void	setDatumWriter(org.apache.avro.io.DatumWriter<T> datumWriter)
void	setDecoder(org.apache.avro.io.Decoder decoder)
void	setEncoder(org.apache.avro.io.Encoder encoder)
void	write(DataOutput out)

Methods inherited from class org.apache.gora.store.impl.FileBackedDataStoreBase

createInputStream, createOutputStream, createSchema, deleteSchema, execute, getInputPath, getInputStream, getOrCreateInputStream, getOrCreateOutputStream, getOutputPath, getOutputStream, getPartitions, schemaExists, setInputPath, setInputStream, setOutputPath, setOutputStream

Methods inherited from class org.apache.gora.store.impl.DataStoreBase

equals, get, getBeanFactory, getFields, getFieldsToQuery, getKeyClass, getOrCreateConf, getPersistentClass, getSchemaName, newKey, newPersistent, setBeanFactory, setConf, setKeyClass, setPersistentClass, truncateSchema

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

setConf

Methods inherited from interface org.apache.gora.store.DataStore

get, getBeanFactory, getKeyClass, getPersistentClass, newKey, newPersistent, setBeanFactory, setKeyClass, setPersistentClass, truncateSchema

Field Detail

CODEC_TYPE_KEY

public static final String CODEC_TYPE_KEY

The property key specifying avro encoder/decoder type to use. Can take values "BINARY" or "JSON".

See Also:

Constant Field Values

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

AvroStore

public AvroStore()

Method Detail

initialize

public void initialize(Class<K> keyClass,
              Class<T> persistentClass,
              Properties properties)

Description copied from interface: DataStore

Initializes this DataStore.

Specified by:

initialize in interface DataStore<K,T extends PersistentBase>

Overrides:

initialize in class FileBackedDataStoreBase<K,T extends PersistentBase>

Parameters:

keyClass - the class of the keys

persistentClass - the class of the persistent objects

properties - extra metadata

setCodecType

public void setCodecType(AvroStore.CodecType codecType)

setEncoder

public void setEncoder(org.apache.avro.io.Encoder encoder)

setDecoder

public void setDecoder(org.apache.avro.io.Decoder decoder)

setDatumReader

public void setDatumReader(org.apache.avro.io.DatumReader<T> datumReader)

setDatumWriter

public void setDatumWriter(org.apache.avro.io.DatumWriter<T> datumWriter)

close

public void close()

Description copied from interface: DataStore

Close the DataStore. This should release any resources held by the implementation, so that the instance is ready for GC. All other DataStore methods cannot be used after this method was called. Subsequent calls of this method are ignored.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface DataStore<K,T extends PersistentBase>

Overrides:

close in class FileBackedDataStoreBase<K,T extends PersistentBase>

delete

public boolean delete(K key)

Description copied from interface: DataStore

Deletes the object with the given key

Specified by:

delete in interface DataStore<K,T extends PersistentBase>

Parameters:

key - the key of the object

Returns:

whether the object was successfully deleted

deleteByQuery

public long deleteByQuery(Query<K,T> query)

Description copied from interface: DataStore

Deletes all the objects matching the query. See also the note on visibility.

Specified by:

deleteByQuery in interface DataStore<K,T extends PersistentBase>

Parameters:

query - matching records to this query will be deleted

Returns:

number of deleted records

executeQuery

protected Result<K,T> executeQuery(Query<K,T> query)
                                                   throws IOException

Executes a normal Query reading the whole data. #execute() calls this function for non-PartitionQuery's.

Specified by:

executeQuery in class FileBackedDataStoreBase<K,T extends PersistentBase>

Throws:

IOException

executePartial

protected Result<K,T> executePartial(FileSplitPartitionQuery<K,T> query)
                                                     throws IOException

Executes a PartitialQuery, reading the data between start and end.

Specified by:

executePartial in class FileBackedDataStoreBase<K,T extends PersistentBase>

Throws:

IOException

flush

public void flush()

Description copied from interface: DataStore

Forces the write caches to be flushed. DataStore implementations may optimize their writing by deferring the actual put / delete operations until this moment. See also the note on visibility.

Specified by:

flush in interface DataStore<K,T extends PersistentBase>

Overrides:

flush in class FileBackedDataStoreBase<K,T extends PersistentBase>

get

public T get(K key,
    String[] fields)

Description copied from interface: DataStore

Returns the object corresponding to the given key.

Specified by:

get in interface DataStore<K,T extends PersistentBase>

Parameters:

key - the key of the object

fields - the fields required in the object. Pass null, to retrieve all fields

Returns:

the Object corresponding to the key or null if it cannot be found

newQuery

public AvroQuery<K,T> newQuery()

Description copied from interface: DataStore

Constructs and returns a new Query.

Specified by:

newQuery in interface DataStore<K,T extends PersistentBase>

Returns:

a new Query.

put

public void put(K key,
       T obj)

Description copied from interface: DataStore

Inserts the persistent object with the given key. If an object with the same key already exists it will silently be replaced. See also the note on visibility.

Specified by:

put in interface DataStore<K,T extends PersistentBase>

getEncoder

public org.apache.avro.io.Encoder getEncoder()
                                      throws IOException

Throws:

IOException

getDecoder

public org.apache.avro.io.Decoder getDecoder()
                                      throws IOException

Throws:

IOException

getDatumReader

public org.apache.avro.io.DatumReader<T> getDatumReader()

getDatumWriter

public org.apache.avro.io.DatumWriter<T> getDatumWriter()

createEncoder

protected org.apache.avro.io.Encoder createEncoder()
                                            throws IOException

Throws:

IOException

createDecoder

protected org.apache.avro.io.Decoder createDecoder()
                                            throws IOException

Throws:

IOException

createDatumWriter

protected org.apache.avro.io.DatumWriter<T> createDatumWriter()

createDatumReader

protected org.apache.avro.io.DatumReader<T> createDatumReader()

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

Overrides:

getConf in class DataStoreBase<K,T extends PersistentBase>

write

public void write(DataOutput out)

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class FileBackedDataStoreBase<K,T extends PersistentBase>

readFields

public void readFields(DataInput in)

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class FileBackedDataStoreBase<K,T extends PersistentBase>

getSchemaName

public String getSchemaName()

Description copied from interface: DataStore

Returns the schema name given to this DataStore

Specified by:

getSchemaName in interface DataStore<K,T extends PersistentBase>

Returns:

schema name