org.apache.gora.store.impl

Class FileBackedDataStoreBase<K,T extends PersistentBase>

java.lang.Object

org.apache.gora.store.impl.DataStoreBase<K,T>

org.apache.gora.store.impl.FileBackedDataStoreBase<K,T>

All Implemented Interfaces:

Closeable, AutoCloseable, DataStore<K,T>, FileBackedDataStore<K,T>, org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.Writable

Direct Known Subclasses:

AvroStore

public abstract class FileBackedDataStoreBase<K,T extends PersistentBase>
extends DataStoreBase<K,T>
implements FileBackedDataStore<K,T>

Base implementations for FileBackedDataStore methods.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	inputPath
protected long	inputSize
protected InputStream	inputStream
static org.slf4j.Logger	LOG
protected String	outputPath
protected OutputStream	outputStream

Fields inherited from class org.apache.gora.store.impl.DataStoreBase

autoCreateSchema, beanFactory, conf, datumReader, datumWriter, fieldMap, keyClass, persistentClass, properties, schema

Constructor Summary

Constructors

Constructor and Description
FileBackedDataStoreBase()

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close the DataStore.
protected InputStream	createInputStream() Opens an InputStream for the input Hadoop path
protected OutputStream	createOutputStream() Opens an OutputStream for the output Hadoop path
void	createSchema() Creates the optional schema or table (or similar) in the datastore to hold the objects.
void	deleteSchema() Deletes the underlying schema or table (or similar) in the datastore that holds the objects.
Result<K,T>	execute(Query<K,T> query) Executes the given query and returns the results.
protected abstract Result<K,T>	executePartial(FileSplitPartitionQuery<K,T> query) Executes a PartitialQuery, reading the data between start and end.
protected abstract Result<K,T>	executeQuery(Query<K,T> query) Executes a normal Query reading the whole data.
void	flush() Forces the write caches to be flushed.
String	getInputPath()
InputStream	getInputStream()
protected InputStream	getOrCreateInputStream()
protected OutputStream	getOrCreateOutputStream()
String	getOutputPath()
OutputStream	getOutputStream()
List<PartitionQuery<K,T>>	getPartitions(Query<K,T> query) Partitions the given query and returns a list of PartitionQuerys, which will execute on local data.
void	initialize(Class<K> keyClass, Class<T> persistentClass, Properties properties) Initializes this DataStore.
void	readFields(DataInput in)
boolean	schemaExists() Returns whether the schema that holds the data exists in the datastore.
void	setInputPath(String inputPath)
void	setInputStream(InputStream inputStream)
void	setOutputPath(String outputPath)
void	setOutputStream(OutputStream outputStream)
void	write(DataOutput out)

Methods inherited from class org.apache.gora.store.impl.DataStoreBase

equals, get, getBeanFactory, getConf, getFields, getFieldsToQuery, getKeyClass, getOrCreateConf, getPersistentClass, getSchemaName, newKey, newPersistent, setBeanFactory, setConf, setKeyClass, setPersistentClass, truncateSchema

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.gora.store.DataStore

delete, deleteByQuery, get, get, getBeanFactory, getKeyClass, getPersistentClass, getSchemaName, newKey, newPersistent, newQuery, put, setBeanFactory, setKeyClass, setPersistentClass, truncateSchema

Field Detail

inputSize

protected long inputSize

inputPath

protected String inputPath

outputPath

protected String outputPath

inputStream

protected InputStream inputStream

outputStream

protected OutputStream outputStream

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

FileBackedDataStoreBase

public FileBackedDataStoreBase()

Method Detail

initialize

public void initialize(Class<K> keyClass,
              Class<T> persistentClass,
              Properties properties)

Description copied from interface: DataStore

Initializes this DataStore.

Specified by:

initialize in interface DataStore<K,T extends PersistentBase>

Overrides:

initialize in class DataStoreBase<K,T extends PersistentBase>

Parameters:

keyClass - the class of the keys

persistentClass - the class of the persistent objects

properties - extra metadata

setInputPath

public void setInputPath(String inputPath)

Specified by:

setInputPath in interface FileBackedDataStore<K,T extends PersistentBase>

setOutputPath

public void setOutputPath(String outputPath)

Specified by:

setOutputPath in interface FileBackedDataStore<K,T extends PersistentBase>

getInputPath

public String getInputPath()

Specified by:

getInputPath in interface FileBackedDataStore<K,T extends PersistentBase>

getOutputPath

public String getOutputPath()

Specified by:

getOutputPath in interface FileBackedDataStore<K,T extends PersistentBase>

setInputStream

public void setInputStream(InputStream inputStream)

Specified by:

setInputStream in interface FileBackedDataStore<K,T extends PersistentBase>

setOutputStream

public void setOutputStream(OutputStream outputStream)

Specified by:

setOutputStream in interface FileBackedDataStore<K,T extends PersistentBase>

getInputStream

public InputStream getInputStream()

Specified by:

getInputStream in interface FileBackedDataStore<K,T extends PersistentBase>

getOutputStream

public OutputStream getOutputStream()

Specified by:

getOutputStream in interface FileBackedDataStore<K,T extends PersistentBase>

createInputStream

protected InputStream createInputStream()
                                 throws IOException

Opens an InputStream for the input Hadoop path

Throws:

IOException

createOutputStream

protected OutputStream createOutputStream()

Opens an OutputStream for the output Hadoop path

getOrCreateInputStream

protected InputStream getOrCreateInputStream()
                                      throws IOException

Throws:

IOException

getOrCreateOutputStream

protected OutputStream getOrCreateOutputStream()
                                        throws IOException

Throws:

IOException

getPartitions

public List<PartitionQuery<K,T>> getPartitions(Query<K,T> query)

Description copied from interface: DataStore

Partitions the given query and returns a list of PartitionQuerys, which will execute on local data.

Specified by:

getPartitions in interface DataStore<K,T extends PersistentBase>

Parameters:

query - the base query to create the partitions for. If the query is null, then the data store returns the partitions for the default query (returning every object)

Returns:

a List of PartitionQuery's

execute

public Result<K,T> execute(Query<K,T> query)

Description copied from interface: DataStore

Executes the given query and returns the results.

Specified by:

execute in interface DataStore<K,T extends PersistentBase>

Parameters:

query - the query to execute.

Returns:

the results as a Result object.

executeQuery

protected abstract Result<K,T> executeQuery(Query<K,T> query)
                                                            throws IOException

Executes a normal Query reading the whole data. #execute() calls this function for non-PartitionQuery's.

Throws:

IOException

executePartial

protected abstract Result<K,T> executePartial(FileSplitPartitionQuery<K,T> query)
                                                              throws IOException

Executes a PartitialQuery, reading the data between start and end.

Throws:

IOException

flush

public void flush()

Description copied from interface: DataStore

Forces the write caches to be flushed. DataStore implementations may optimize their writing by deferring the actual put / delete operations until this moment. See also the note on visibility.

Specified by:

flush in interface DataStore<K,T extends PersistentBase>

createSchema

public void createSchema()

Description copied from interface: DataStore

Creates the optional schema or table (or similar) in the datastore to hold the objects. If the schema is already created previously, or the underlying data model does not support or need this operation, the operation is ignored.

Specified by:

createSchema in interface DataStore<K,T extends PersistentBase>

deleteSchema

public void deleteSchema()

Description copied from interface: DataStore

Deletes the underlying schema or table (or similar) in the datastore that holds the objects. This also deletes all the data associated with the schema.

Specified by:

deleteSchema in interface DataStore<K,T extends PersistentBase>

schemaExists

public boolean schemaExists()

Description copied from interface: DataStore

Returns whether the schema that holds the data exists in the datastore.

Specified by:

schemaExists in interface DataStore<K,T extends PersistentBase>

Returns:

whether schema exists

write

public void write(DataOutput out)

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class DataStoreBase<K,T extends PersistentBase>

readFields

public void readFields(DataInput in)

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class DataStoreBase<K,T extends PersistentBase>

close

public void close()

Description copied from interface: DataStore

Close the DataStore. This should release any resources held by the implementation, so that the instance is ready for GC. All other DataStore methods cannot be used after this method was called. Subsequent calls of this method are ignored.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface DataStore<K,T extends PersistentBase>