org.apache.pig

Class PigConfiguration

java.lang.Object

org.apache.pig.PigConfiguration

public class PigConfiguration
extends Object

Container for static configuration strings, defaults, etc. This is intended just for keys that can be set by users, not for keys that are generally used within pig.

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARTAGG_MINREDUCTION Controls the minimum reduction in-mapper Partial Aggregation should achieve in order to stay on.
static String	PIG_DEFAULT_LOAD_FUNC This key is used to define the default load func.
static String	PIG_DEFAULT_STORE_FUNC This key is used to define the default store func.
static String	PIG_ENABLE_TEMP_FILE_COMPRESSION This key is used to define whether to have intermediate file compressed
static String	PIG_INTERSTORAGE_SYNCMARKER_INTERVAL Defines the interval (in bytes) when a sync marker should be written into the binary file
static long	PIG_INTERSTORAGE_SYNCMARKER_INTERVAL_DEFAULT
static String	PIG_INTERSTORAGE_SYNCMARKER_SIZE Sets the length of record markers in binary files produces by Pig between jobs The longer the byte sequence means less chance of collision with actual data, shorter sequence means less overhead
static int	PIG_INTERSTORAGE_SYNCMARKER_SIZE_DEFAULT
static int	PIG_INTERSTORAGE_SYNCMARKER_SIZE_MAX
static int	PIG_INTERSTORAGE_SYNCMARKER_SIZE_MIN
static String	PIG_OUTPUT_COMMITTER_RECOVERY This key is used to define whether to support recovery to handle the application master getting restarted.
static String	PIG_STORE_SCHEMA_DISAMBIGUATE If set to false, automatic schema disambiguation gets disabled i.e.
static String	PIG_STORE_SCHEMA_DISAMBIGUATE_DEFAULT
static String	PIG_STREAMING_ENVIRONMENT This key can be used to defined what properties will be set in the streaming environment.
static String	PIG_TEMP_FILE_COMPRESSION_CODEC Compression codec used by intermediate storage TFileStorage only support gzip and lzo.
static String	PIG_TEMP_FILE_COMPRESSION_STORAGE This key is used to set the storage type used by intermediate file storage If pig.tmpfilecompression, default storage used is TFileStorage.
static String	PROP_CACHEDBAG_MEMUSAGE Controls the fraction of total memory that is allowed to be used by cached bags.
static String	PROP_EXEC_MAP_PARTAGG Controls whether partial aggregation is turned on
static String	PROP_NO_COMBINER
static String	SCHEMA_TUPLE_SHOULD_ALLOW_FORCE
static String	SCHEMA_TUPLE_SHOULD_USE_IN_FOREACH
static String	SCHEMA_TUPLE_SHOULD_USE_IN_FRJOIN
static String	SCHEMA_TUPLE_SHOULD_USE_IN_MERGEJOIN
static String	SCHEMA_TUPLE_SHOULD_USE_IN_UDF
static String	SHOULD_USE_SCHEMA_TUPLE This key must be set to true by the user for code generation to be used.
static String	TIME_UDFS_PROP Controls whether execution time of Pig UDFs should be tracked.

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROP_CACHEDBAG_MEMUSAGE

public static final String PROP_CACHEDBAG_MEMUSAGE

Controls the fraction of total memory that is allowed to be used by cached bags. Default is 0.2.

See Also:

Constant Field Values

PIG_INTERSTORAGE_SYNCMARKER_SIZE

public static final String PIG_INTERSTORAGE_SYNCMARKER_SIZE

Sets the length of record markers in binary files produces by Pig between jobs The longer the byte sequence means less chance of collision with actual data, shorter sequence means less overhead

See Also:

Constant Field Values

PIG_INTERSTORAGE_SYNCMARKER_SIZE_MAX

public static final int PIG_INTERSTORAGE_SYNCMARKER_SIZE_MAX

See Also:

Constant Field Values

PIG_INTERSTORAGE_SYNCMARKER_SIZE_DEFAULT

public static final int PIG_INTERSTORAGE_SYNCMARKER_SIZE_DEFAULT

See Also:

Constant Field Values

PIG_INTERSTORAGE_SYNCMARKER_SIZE_MIN

public static final int PIG_INTERSTORAGE_SYNCMARKER_SIZE_MIN

See Also:

Constant Field Values

PIG_INTERSTORAGE_SYNCMARKER_INTERVAL

public static final String PIG_INTERSTORAGE_SYNCMARKER_INTERVAL

Defines the interval (in bytes) when a sync marker should be written into the binary file

See Also:

Constant Field Values

PIG_INTERSTORAGE_SYNCMARKER_INTERVAL_DEFAULT

public static final long PIG_INTERSTORAGE_SYNCMARKER_INTERVAL_DEFAULT

See Also:

Constant Field Values

PROP_EXEC_MAP_PARTAGG

public static final String PROP_EXEC_MAP_PARTAGG

Controls whether partial aggregation is turned on

See Also:

Constant Field Values

PARTAGG_MINREDUCTION

public static final String PARTAGG_MINREDUCTION

Controls the minimum reduction in-mapper Partial Aggregation should achieve in order to stay on. If after a period of observation this reduction is not achieved, in-mapper aggregation will be turned off and a message logged to that effect.

See Also:

Constant Field Values

TIME_UDFS_PROP

public static final String TIME_UDFS_PROP

Controls whether execution time of Pig UDFs should be tracked. This feature uses counters; use judiciously.

See Also:

Constant Field Values

SHOULD_USE_SCHEMA_TUPLE

public static final String SHOULD_USE_SCHEMA_TUPLE

This key must be set to true by the user for code generation to be used. In the future, it may be turned on by default (at least in certain cases), but for now it is too experimental.

See Also:

Constant Field Values

SCHEMA_TUPLE_SHOULD_USE_IN_UDF

public static final String SCHEMA_TUPLE_SHOULD_USE_IN_UDF

See Also:

Constant Field Values

SCHEMA_TUPLE_SHOULD_USE_IN_FOREACH

public static final String SCHEMA_TUPLE_SHOULD_USE_IN_FOREACH

See Also:

Constant Field Values

SCHEMA_TUPLE_SHOULD_USE_IN_FRJOIN

public static final String SCHEMA_TUPLE_SHOULD_USE_IN_FRJOIN

See Also:

Constant Field Values

SCHEMA_TUPLE_SHOULD_USE_IN_MERGEJOIN

public static final String SCHEMA_TUPLE_SHOULD_USE_IN_MERGEJOIN

See Also:

Constant Field Values

SCHEMA_TUPLE_SHOULD_ALLOW_FORCE

public static final String SCHEMA_TUPLE_SHOULD_ALLOW_FORCE

See Also:

Constant Field Values

PROP_NO_COMBINER

public static final String PROP_NO_COMBINER

See Also:

Constant Field Values

PIG_STREAMING_ENVIRONMENT

public static final String PIG_STREAMING_ENVIRONMENT

This key can be used to defined what properties will be set in the streaming environment. Just set this property to a comma-delimited list of properties to set, and those properties will be set in the environment.

See Also:

Constant Field Values

PIG_DEFAULT_LOAD_FUNC

public static final String PIG_DEFAULT_LOAD_FUNC

This key is used to define the default load func. Pig will fallback on PigStorage as default in case this is undefined.

See Also:

Constant Field Values

PIG_DEFAULT_STORE_FUNC

public static final String PIG_DEFAULT_STORE_FUNC

This key is used to define the default store func. Pig will fallback on PigStorage as default in case this is undefined.

See Also:

Constant Field Values

PIG_OUTPUT_COMMITTER_RECOVERY

public static final String PIG_OUTPUT_COMMITTER_RECOVERY

This key is used to define whether to support recovery to handle the application master getting restarted.

See Also:

Constant Field Values

PIG_ENABLE_TEMP_FILE_COMPRESSION

public static final String PIG_ENABLE_TEMP_FILE_COMPRESSION

This key is used to define whether to have intermediate file compressed

See Also:

Constant Field Values

PIG_TEMP_FILE_COMPRESSION_STORAGE

public static final String PIG_TEMP_FILE_COMPRESSION_STORAGE

This key is used to set the storage type used by intermediate file storage If pig.tmpfilecompression, default storage used is TFileStorage. This can be overriden to use SequenceFileInterStorage by setting following property to "seqfile".

See Also:

Constant Field Values

PIG_TEMP_FILE_COMPRESSION_CODEC

public static final String PIG_TEMP_FILE_COMPRESSION_CODEC

Compression codec used by intermediate storage TFileStorage only support gzip and lzo.

See Also:

Constant Field Values

PIG_STORE_SCHEMA_DISAMBIGUATE

public static final String PIG_STORE_SCHEMA_DISAMBIGUATE

If set to false, automatic schema disambiguation gets disabled i.e. group::name will be just name

See Also:

Constant Field Values

PIG_STORE_SCHEMA_DISAMBIGUATE_DEFAULT

public static final String PIG_STORE_SCHEMA_DISAMBIGUATE_DEFAULT

See Also:

Constant Field Values