Cloudera Enterprise 5.15.x | Other versions

Static Service Pools

Static service pools isolate the services in your cluster from one another, so that load on one service has a bounded impact on other services. Services are allocated a static percentage of total resources—CPU, memory, and I/O weight—which are not shared with other services. When you configure static service pools, Cloudera Manager computes recommended memory, CPU, and I/O configurations for the worker roles of the services that correspond to the percentage assigned to each service. Static service pools are implemented per role group within a cluster, using Linux control groups (cgroups) and cooperative memory limits (for example, Java maximum heap sizes). Static service pools can be used to control access to resources by HBase, HDFS, Impala, MapReduce, Solr, Spark, YARN, and add-on services. Static service pools are not enabled by default.

  Note:
  • I/O allocation only works when short-circuit reads are enabled.
  • I/O allocation does not handle write side I/O because cgroups in the Linux kernel do not currently support buffered writes.

Viewing Static Service Pool Status

Select Clusters > Cluster name > Static Service Pools.If the cluster has a YARN service, the Static Service Pools Status tab displays and shows whether resource management is enabled for the cluster, and the currently configured service pools.

See Monitoring Static Service Pools for more information.

Enabling and Configuring Static Service Pools

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

  1. Select Clusters > Cluster name > Static Service Pools.
  2. Click the Configuration tab. The Step 1 of 4: Basic Allocation Setup page displays. In each field in the basic allocation table, enter the percentage of resources to give to each service. The total must add up to 100%. Click Continue to proceed.
  3. Step 2: Review Changes - The allocation of resources for each resource type and role displays with the new values as well as the values previously in effect. The values for each role are set by role group; if there is more than one role group for a given role type (for example, for RegionServers or DataNodes) then resources will be allocated separately for the hosts in each role group. Take note of changed settings. If you have previously customized these settings, check these over carefully.
    • Click the to the right of each percentage to display the allocations for a single service. Click to the right of the Total (100%) to view all the allocations in a single page.
    • Click the Back button to go to the previous page and change your allocations.

    When you are satisfied with the allocations, click Continue.

  4. Step 3 of 4: Restart Services - To apply the new allocation percentages, click Restart Now to restart the cluster. To skip this step, click Restart Later. If HDFS High Availability is enabled, you will have the option to choose a rolling restart.
  5. Step 4 of 4: Progress displays the status of the restart commands. Click Finished after the restart commands complete.
  6. After you enable static service pools, there are three additional tasks.
    1. Delete everything under the local directory path on NodeManager hosts. The local directory path is configurable, and can be verified in Cloudera Manager with YARN > Configuration > NodeManager Local Directories.
    2. Enable cgroups for resource management. You can enable cgroups in Cloudera Manager with Yarn > Configuration > Use CGroups for Resource Management.
    3. If you are using the optional Impala scratch directory, delete all files in the Impala scratch directory. The directory path is configurable, and can be verified in Cloudera Manager with Impala > Configuration > Impala Daemon Scratch Directories.

Disabling Static Service Pools

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

To disable static service pools, disable cgroup-based resource management for all hosts in all clusters:
  1. In the main navigation bar, click Hosts.
  2. Click the Configuration tab.
  3. Select Scope > Resource Management.
  4. Clear the Enable Cgroup-based Resource Management property.
  5. Click Save Changes.
  6. Restart all services.
Static resource management is disabled, but the percentages you set when you configured the pools, and all the changed settings (for example, heap sizes), are retained by the services. The percentages and settings will also be used when you re-enable static service pools. If you want to revert to the settings you had before static service pools were enabled, follow the procedures in Viewing and Reverting Configuration Changes.
Page generated May 18, 2018.