org.apache.flume.sink.kite

Class DatasetSinkConstants

java.lang.Object

org.apache.flume.sink.kite.DatasetSinkConstants

public class DatasetSinkConstants
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
static String	AUTH_KEYTAB
static String	AUTH_PRINCIPAL
static String	AUTH_PROXY_USER Hadoop authentication settings
static String[]	AVAILABLE_PARSERS
static String[]	AVAILABLE_POLICIES
static String	AVRO_ENTITY_PARSER Built-in entity parsers
static String	AVRO_SCHEMA_LITERAL_HEADER Headers where avro schema information is expected.
static String	AVRO_SCHEMA_URL_HEADER
static String	CONFIG_ENTITY_PARSER Parser used to parse Flume Events into Kite entities.
static String	CONFIG_FAILURE_POLICY Policy used to handle non-recoverable failures.
static String	CONFIG_FLUSHABLE_COMMIT_ON_BATCH Flag for committing the Flume transaction on each batch for Flushable datasets.
static String	CONFIG_KITE_BATCH_SIZE Number of records to process from the incoming channel per call to process.
static String	CONFIG_KITE_DATASET_NAME Name of the Kite Dataset to write into.
static String	CONFIG_KITE_DATASET_NAMESPACE Namespace of the Kite Dataset to write into.
static String	CONFIG_KITE_DATASET_URI URI of the Kite Dataset
static String	CONFIG_KITE_ERROR_DATASET_URI The URI to write non-recoverable Flume events to in the case of an error.
static String	CONFIG_KITE_REPO_URI URI of the Kite DatasetRepository.
static String	CONFIG_KITE_ROLL_INTERVAL Maximum time to wait before finishing files.
static String	CONFIG_SYNCABLE_SYNC_ON_BATCH Flag for syncing the DatasetWriter on each batch for Syncable datasets.
static long	DEFAULT_BATCH_SIZE
static String	DEFAULT_ENTITY_PARSER
static String	DEFAULT_FAILURE_POLICY
static boolean	DEFAULT_FLUSHABLE_COMMIT_ON_BATCH
static String	DEFAULT_NAMESPACE
static int	DEFAULT_ROLL_INTERVAL
static boolean	DEFAULT_SYNCABLE_SYNC_ON_BATCH
static String	RETRY_FAILURE_POLICY Retry non-recoverable Flume events.
static String	SAVE_FAILURE_POLICY Write non-recoverable Flume events to a Kite dataset.

Constructor Summary

Constructors

Constructor and Description
DatasetSinkConstants()

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CONFIG_KITE_DATASET_URI

public static final String CONFIG_KITE_DATASET_URI

URI of the Kite Dataset

See Also:

Constant Field Values

CONFIG_KITE_REPO_URI

public static final String CONFIG_KITE_REPO_URI

URI of the Kite DatasetRepository.

See Also:

Constant Field Values

CONFIG_KITE_DATASET_NAME

public static final String CONFIG_KITE_DATASET_NAME

Name of the Kite Dataset to write into.

See Also:

Constant Field Values

CONFIG_KITE_DATASET_NAMESPACE

public static final String CONFIG_KITE_DATASET_NAMESPACE

Namespace of the Kite Dataset to write into.

See Also:

Constant Field Values

DEFAULT_NAMESPACE

public static final String DEFAULT_NAMESPACE

See Also:

Constant Field Values

CONFIG_KITE_BATCH_SIZE

public static final String CONFIG_KITE_BATCH_SIZE

Number of records to process from the incoming channel per call to process.

See Also:

Constant Field Values

DEFAULT_BATCH_SIZE

public static long DEFAULT_BATCH_SIZE

CONFIG_KITE_ROLL_INTERVAL

public static final String CONFIG_KITE_ROLL_INTERVAL

Maximum time to wait before finishing files.

See Also:

Constant Field Values

DEFAULT_ROLL_INTERVAL

public static int DEFAULT_ROLL_INTERVAL

CONFIG_FLUSHABLE_COMMIT_ON_BATCH

public static final String CONFIG_FLUSHABLE_COMMIT_ON_BATCH

Flag for committing the Flume transaction on each batch for Flushable datasets. When set to false, Flume will only commit the transaction when roll interval has expired. Setting this to false requires enough space in the channel to handle all events delivered during the roll interval. Defaults to true.

See Also:

Constant Field Values

DEFAULT_FLUSHABLE_COMMIT_ON_BATCH

public static boolean DEFAULT_FLUSHABLE_COMMIT_ON_BATCH

CONFIG_SYNCABLE_SYNC_ON_BATCH

public static final String CONFIG_SYNCABLE_SYNC_ON_BATCH

Flag for syncing the DatasetWriter on each batch for Syncable datasets. Defaults to true.

See Also:

Constant Field Values

DEFAULT_SYNCABLE_SYNC_ON_BATCH

public static boolean DEFAULT_SYNCABLE_SYNC_ON_BATCH

CONFIG_ENTITY_PARSER

public static final String CONFIG_ENTITY_PARSER

Parser used to parse Flume Events into Kite entities.

See Also:

Constant Field Values

AVRO_ENTITY_PARSER

public static final String AVRO_ENTITY_PARSER

Built-in entity parsers

See Also:

Constant Field Values

DEFAULT_ENTITY_PARSER

public static final String DEFAULT_ENTITY_PARSER

See Also:

Constant Field Values

AVAILABLE_PARSERS

public static final String[] AVAILABLE_PARSERS

CONFIG_FAILURE_POLICY

public static final String CONFIG_FAILURE_POLICY

Policy used to handle non-recoverable failures.

See Also:

Constant Field Values

SAVE_FAILURE_POLICY

public static final String SAVE_FAILURE_POLICY

Write non-recoverable Flume events to a Kite dataset.

See Also:

Constant Field Values

CONFIG_KITE_ERROR_DATASET_URI

public static final String CONFIG_KITE_ERROR_DATASET_URI

The URI to write non-recoverable Flume events to in the case of an error. If the dataset doesn't exist, it will be created.

See Also:

Constant Field Values

RETRY_FAILURE_POLICY

public static final String RETRY_FAILURE_POLICY

Retry non-recoverable Flume events. This will lead to a never ending cycle of failure, but matches the previous default semantics of the DatasetSink.

See Also:

Constant Field Values

DEFAULT_FAILURE_POLICY

public static final String DEFAULT_FAILURE_POLICY

See Also:

Constant Field Values

AVAILABLE_POLICIES

public static final String[] AVAILABLE_POLICIES

AVRO_SCHEMA_LITERAL_HEADER

public static final String AVRO_SCHEMA_LITERAL_HEADER

Headers where avro schema information is expected.

See Also:

Constant Field Values

AVRO_SCHEMA_URL_HEADER

public static final String AVRO_SCHEMA_URL_HEADER

See Also:

Constant Field Values

AUTH_PROXY_USER

public static final String AUTH_PROXY_USER

Hadoop authentication settings

See Also:

Constant Field Values

AUTH_PRINCIPAL

public static final String AUTH_PRINCIPAL

See Also:

Constant Field Values

AUTH_KEYTAB

public static final String AUTH_KEYTAB

See Also:

Constant Field Values

Constructor Detail

DatasetSinkConstants

public DatasetSinkConstants()