Cloudera Enterprise 5.15.x | Other versions

Apache Kudu Configuration

To configure the behavior of each Kudu process, you can pass command-line flags when you start it, or read those options from configuration files by passing them using one or more --flagfile=<file> options. You can even include the --flagfile option within your configuration file to include other files. Learn more about gflags by reading its documentation.

You can place options for masters and tablet servers in the same configuration file, and each will ignore options that do not apply.

Flags can be prefixed with either one or two - characters. This documentation standardizes on two: --example_flag.

Only the most common configuration options are documented in this topic. For a more exhaustive list of configuration options, see the Kudu Configuration Reference. To see all configuration flags for a given executable, run it with the --help option.

Experimental Flags

Some configuration flags are marked 'unsafe' and 'experimental'. Such flags are disabled by default. You can access these flags by enabling the additional flags, --unlock_unsafe_flags and --unlock_experimental_flags. Note that these flags might be removed or modified without a deprecation period or any prior notice in future Kudu releases. Cloudera does not support using unsafe and experimental flags. As a rule of thumb, Cloudera will not support any configuration flags not explicitly documented in the Kudu Configuration Reference Guide.

Directory Configurations

Every Kudu node requires the specification of directory flags. The --fs_wal_dir configuration indicates where Kudu will place its write-ahead logs. The --fs_metadata_dir configuration indicates where Kudu will place metadata for each tablet. It is recommended, although not necessary, that these directories be placed on a high-performance drives with high bandwidth and low latency, e.g. solid-state drives. If --fs_metadata_dir is not specified, metadata will be placed in the directory specified by --fs_wal_dir. Since a Kudu node cannot tolerate the loss of its WAL or metadata directories, you might want to mirror the drives containing these directories in order to make recovering from a drive failure easier. However, mirroring may increase the latency of Kudu writes.

The --fs_data_dirs configuration indicates where Kudu will write its data blocks. This is a comma-separated list of directories; if multiple values are specified, data will be striped across the directories. If not specified, data blocks will be placed in the directory specified by --fs_wal_dir.

Additionally, --fs_wal_dir and --fs_metadata_dir may be the same as one of the directories listed in --fs_data_dirs, but must not be sub-directories of any of them.

CAUTION:
Once --fs_data_dirs is set, extra tooling is required to change it. For more details, see the Changing Directory Configuration.
  Note: The --fs_wal_dir and --fs_metadata_dir configurations can be changed, provided the contents of the directories are also moved to match the flags.

Configuring the Kudu Master

To see all available configuration options for the kudu-master executable, run it with the --help option:

$ kudu-master --help
Table 1. Supported Configuration Flags for Kudu Masters
Flag Valid Options Default Description

--master_addresses

string

localhost

Comma-separated list of all the RPC addresses for Master consensus-configuration. If not specified, assumes a standalone Master.

--fs_data_dirs

string

  

List of directories where the Master will place its data blocks.

--fs_wal_dir

string

  

The directory where the Master will place its write-ahead logs.

--log_dir

string

/tmp

The directory to store Master log files.

For the complete list of flags for masters, see the Kudu Master Configuration Reference.

Configuring Tablet Servers

To see all available configuration options for the kudu-tserver executable, run it with the --help option:

$ kudu-tserver --help
Table 2. Supported Configuration Flags for Kudu Tablet Servers
Flag Valid Options Default Description

--fs_data_dirs

string

  

List of directories where the Tablet Server will place its data blocks.

--fs_wal_dir

string

  

The directory where the Tablet Server will place its write-ahead logs.

--log_dir

string

/tmp

The directory to store Tablet Server log files

--tserver_master_addrs

string

127.0.0.1:7051

Comma separated addresses of the masters that the tablet server should connect to. The masters do not read this flag.

--block_cache_capacity_mb

integer

512

Maximum amount of memory allocated to the Kudu Tablet Server’s block cache.

--memory_limit_hard_bytes

integer

4294967296

Maximum amount of memory a Tablet Server can consume before it starts rejecting all incoming writes.

For the complete list of flags for tablet servers, see the Kudu Tablet Server Configuration Reference.

Page generated May 18, 2018.