public class GoraInputFormat<K,T extends PersistentBase>
extends org.apache.hadoop.mapreduce.InputFormat<K,T>
implements org.apache.hadoop.conf.Configurable
InputFormat
to fetch the input from Gora data stores. The
query to fetch the items from the datastore should be prepared and
set via setQuery(Job, Query)
, before submitting the job.
The InputSplit
s are prepared from the PartitionQuery
s
obtained by calling DataStore.getPartitions(Query)
.
Hadoop jobs can be either configured through static
setInput()
methods, or from GoraMapper
.
GoraMapper
Constructor and Description |
---|
GoraInputFormat() |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.mapreduce.RecordReader<K,T> |
createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
org.apache.hadoop.mapreduce.TaskAttemptContext context) |
org.apache.hadoop.conf.Configuration |
getConf() |
Query<K,T> |
getQuery(org.apache.hadoop.conf.Configuration conf) |
List<org.apache.hadoop.mapreduce.InputSplit> |
getSplits(org.apache.hadoop.mapreduce.JobContext context) |
void |
setConf(org.apache.hadoop.conf.Configuration conf) |
static <K1,V1 extends Persistent> |
setInput(org.apache.hadoop.mapreduce.Job job,
Class<? extends DataStore<K1,V1>> dataStoreClass,
Class<K1> inKeyClass,
Class<V1> inValueClass,
boolean reuseObjects)
Sets the input parameters for the job
|
static <K1,V1 extends Persistent> |
setInput(org.apache.hadoop.mapreduce.Job job,
Query<K1,V1> query,
boolean reuseObjects)
Sets the input parameters for the job
|
static <K1,V1 extends Persistent> |
setInput(org.apache.hadoop.mapreduce.Job job,
Query<K1,V1> query,
DataStore<K1,V1> dataStore,
boolean reuseObjects)
Sets the input parameters for the job
|
static <K,T extends Persistent> |
setQuery(org.apache.hadoop.mapreduce.Job job,
Query<K,T> query) |
public static final String QUERY_KEY
public org.apache.hadoop.mapreduce.RecordReader<K,T> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
createRecordReader
in class org.apache.hadoop.mapreduce.InputFormat<K,T extends PersistentBase>
IOException
InterruptedException
public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context) throws IOException, InterruptedException
getSplits
in class org.apache.hadoop.mapreduce.InputFormat<K,T extends PersistentBase>
IOException
InterruptedException
public org.apache.hadoop.conf.Configuration getConf()
getConf
in interface org.apache.hadoop.conf.Configurable
public void setConf(org.apache.hadoop.conf.Configuration conf)
setConf
in interface org.apache.hadoop.conf.Configurable
public static <K,T extends Persistent> void setQuery(org.apache.hadoop.mapreduce.Job job, Query<K,T> query) throws IOException
IOException
public Query<K,T> getQuery(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public static <K1,V1 extends Persistent> void setInput(org.apache.hadoop.mapreduce.Job job, Query<K1,V1> query, boolean reuseObjects) throws IOException
job
- the job to set the properties forquery
- the query to get the inputs fromreuseObjects
- whether to reuse objects in serializationIOException
public static <K1,V1 extends Persistent> void setInput(org.apache.hadoop.mapreduce.Job job, Query<K1,V1> query, DataStore<K1,V1> dataStore, boolean reuseObjects) throws IOException
job
- the job to set the properties forquery
- the query to get the inputs fromdataStore
- the datastore as the inputreuseObjects
- whether to reuse objects in serializationIOException
public static <K1,V1 extends Persistent> void setInput(org.apache.hadoop.mapreduce.Job job, Class<? extends DataStore<K1,V1>> dataStoreClass, Class<K1> inKeyClass, Class<V1> inValueClass, boolean reuseObjects) throws IOException
job
- the job to set the properties fordataStoreClass
- the datastore classinKeyClass
- Map input key classinValueClass
- Map input value classreuseObjects
- whether to reuse objects in serializationIOException
Copyright © 2010-2014 The Apache Software Foundation. All Rights Reserved.