Db2 11.5.4 on Openshift デプロイのための前提作業 (2) CRI-O PID 拡張


Db2 on OpenShift インストールの前提

Db2 11.5.4 をRed Hat OpenShift上で稼働させるための前提要件は、こちらのサイトに整理されています。

Db2 Community Edition 11.5.4.0
https://github.com/IBM/charts/tree/master/stable/ibm-db2

前提要件

  • Kubernetes Level - ">=1.11.0"
  • Helm Level:
    • Power: ">=2.12(*) and < 3.0"
    • X86: ">=2.14(*) and < 3.0"
      • (*) Tested Integrations
  • OpenShift Version - "3.11, 4.3"
  • PersistentVolume requirements - requires one of the following:
    • NFS
    • IBM Cloud File Storage (gold storage class)
    • Portworx
    • Red Hat OpenShift Container Storage 4.3 and above
    • or a hostPath PV that is a mounted clustered filesystem
  • An IBM Cloud account

その他、推奨される設定項目

他にも、Db2 on OpenShift 導入前提手順としては下記の項目があります。

  • container_manage_cgroup 有効化
  • CRI-O PID拡張(オプション)

 (※container_manage_cgroup 有効化は Db2 11.5.4 on OpenShiftインストーラ内で実施されるようになり不要となった)

PID拡張は、必要に応じて(Db2コンテナ内で多くのプロセスが起動される可能性がある場合)に行う位置づけのようです。
ここでは、OCPコンソール上で設定変更を行いました。

CRI-O PID 拡張の作業手順

コンテナで使用可能なプロセスのPIDの上限が16K未満の場合、db2oltpデプロイ前に、システム構成変更を行い、コンテナで使用可能なプロセスのPIDの上限を変更する。

OCP Web コンソールにログインし、左のナビゲーション・メニューで Compute -> Machine Configs を選択する。(Administrator パースペクティブ で作業する)

「01-worker-container-runtime (デフォルト) 」をクリックする

YAMLタブを選択すると、YAMLファイルが表示される。
コンソール上でYAML文書を全選択し、テキストエディタに添付する。

01-worker-container-runtime そのものを編集するわけではないため、OCP Web コンソールは Machine Config 一覧の画面に戻っておく。

コピーした Yaml ファイルのうち、編集箇所は下記2か所となる。

[1] metadata.name の編集

テキストエディタ上で、metadata.name を 01-worker-container-runtime から 10-worker-container-runtime に更新する。

補足:

Machine Config Operator が Machineconfig を適用するときの動作として、Machineconfig は (00* から 99* までの) 順序で読み取られます。
Machineconfig 内のラベルは、それぞれのノードのタイプ (マスターまたはワーカー) を特定します。同じファイルが複数の Machineconfig ファイルに表示される場合、最後のファイルが有効になります。
たとえば、99* ファイルに出現するファイルは、00* ファイルに出現する同一のファイルを置き換えます。入力された Machineconfig オブジェクトは「レンダリングされた」Machineconfig オブジェクトに結合されます。
これは Operator のターゲットとして使用、Machine Config Poolで確認できる値です。
https://access.redhat.com/documentation/ja-jp/openshift_container_platform/4.3/html/architecture/digging-into-machine-config_architecture-rhcos
https://docs.openshift.com/container-platform/4.3/architecture/architecture-rhcos.html

[2] CRI-O構成の編集

data:,XXXXXX の箇所の CRI-O 構成を変更する
テキストエディタ上で「data:,」の後ろの文字列をすべてコピーする。 (「data:」の後ろのカンマはコピーしない)

「data:,」 の後ろの文字列はURLエンコードされているため、下記サイトにてデコードする。
https://www.urldecoder.org

urldecorder.orgのページ上でデコードする様子↓

「pids_limit」のエントリーを探し、現行値を確認の上、数値を書き換える。(変更前の値はサンプルです)

  • (変更前) pids_limit = 1024
  • (変更後) pids_limit = 16384

※ デコード済みの CRI-O configuration file の全量も、参考情報として末尾に貼っておきます(けっこう長い)。

テキストエディタ上の文字列をすべてコピーし、先ほどと同じリンク先から、コンテキストを再度URLエンコード形式に変更する。
https://www.urldecoder.org

urldecorder.orgのページ上でエンコードする様子↓

URLエンコード済みの文字列を、YAMLファイルの「data:」の後ろに書き戻す。

(編集後イメージ)

OCP Web コンソールの「Compute」メニューから「Machine Configs」を選択し、Machine Configs に戻る。
「Create Machine Config」をクリックし、上の手順で作成した yaml ファイル 内容を反映し、machine config を作成する。

(Create Machine Config 画面)

参考:CRI-O Configuration (編集前)

今回はPIDのみ編集していますが、OCP 4.3 環境の CRI-O configuration file の、デコードした全量はこちら。

[crio]

# The default log directory where all logs will go unless directly specified by
# the kubelet. The log directory specified must be an absolute directory.
log_dir = "/var/log/crio/pods"

# Location for CRI-O to lay down the version file
version_file = "/var/lib/crio/version"

# The crio.api table contains settings for the kubelet/gRPC interface.
[crio.api]

# Path to AF_LOCAL socket on which CRI-O will listen.
listen = "/var/run/crio/crio.sock"

# Host IP considered as the primary IP to use by CRI-O for things such as host network IP.
host_ip = ""

# IP address on which the stream server will listen.
stream_address = ""

# The port on which the stream server will listen.
stream_port = "10010"

# Enable encrypted TLS transport of the stream server.
stream_enable_tls = false

# Path to the x509 certificate file used to serve the encrypted stream. This
# file can change, and CRI-O will automatically pick up the changes within 5
# minutes.
stream_tls_cert = ""

# Path to the key file used to serve the encrypted stream. This file can
# change and CRI-O will automatically pick up the changes within 5 minutes.
stream_tls_key = ""

# Path to the x509 CA(s) file used to verify and authenticate client
# communication with the encrypted stream. This file can change and CRI-O will
# automatically pick up the changes within 5 minutes.
stream_tls_ca = ""

# Maximum grpc send message size in bytes. If not set or <=0, then CRI-O will default to 16 * 1024 * 1024.
grpc_max_send_msg_size = 16777216

# Maximum grpc receive message size. If not set or <= 0, then CRI-O will default to 16 * 1024 * 1024.
grpc_max_recv_msg_size = 16777216

# The crio.runtime table contains settings pertaining to the OCI runtime used
# and options for how to set up and manage the OCI runtime.
[crio.runtime]

# default_runtime is the _name_ of the OCI runtime to be used as the default.
# The name is matched against the runtimes map below.
default_runtime = "runc"

# If true, the runtime will not use pivot_root, but instead use MS_MOVE.
no_pivot = false

# Path to the conmon binary, used for monitoring the OCI runtime.
# Will be searched for using $PATH if empty.
conmon = "/usr/libexec/crio/conmon"

# Cgroup setting for conmon
conmon_cgroup = "pod"

# Environment variable list for the conmon process, used for passing necessary
# environment variables to conmon or the runtime.
conmon_env = [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
]

# If true, SELinux will be used for pod separation on the host.
selinux = true

# Path to the seccomp.json profile which is used as the default seccomp profile
# for the runtime. If not specified, then the internal default seccomp profile
# will be used.
seccomp_profile = ""

# Used to change the name of the default AppArmor profile of CRI-O. The default
# profile name is "crio-default-" followed by the version string of CRI-O.
apparmor_profile = "crio-default"

# Cgroup management implementation used for the runtime.
cgroup_manager = "systemd"

# List of default capabilities for containers. If it is empty or commented out,
# only the capabilities defined in the containers json file by the user/kube
# will be added.
default_capabilities = [
    "CHOWN",
    "DAC_OVERRIDE",
    "FSETID",
    "FOWNER",
    "NET_RAW",
    "SETGID",
    "SETUID",
    "SETPCAP",
    "NET_BIND_SERVICE",
    "SYS_CHROOT",
    "KILL",
]

# List of default sysctls. If it is empty or commented out, only the sysctls
# defined in the container json file by the user/kube will be added.
default_sysctls = [
]

# List of additional devices. specified as
# "<device-on-host>:<device-on-container>:<permissions>", for example: "--device=/dev/sdc:/dev/xvdc:rwm".
#If it is empty or commented out, only the devices
# defined in the container json file by the user/kube will be added.
additional_devices = [
]

# Path to OCI hooks directories for automatically executed hooks.
hooks_dir = [
    "/etc/containers/oci/hooks.d",
]

# List of default mounts for each container. **Deprecated:** this option will
# be removed in future versions in favor of default_mounts_file.
default_mounts = [
]

# Maximum number of processes allowed in a container.
pids_limit = 1024

# Maximum sized allowed for the container log file. Negative numbers indicate
# that no size limit is imposed. If it is positive, it must be >= 8192 to
# match/exceed conmon's read buffer. The file is truncated and re-opened so the
# limit is never exceeded.
log_size_max = -1

# Whether container output should be logged to journald in addition to the kuberentes log file
log_to_journald = false

# Path to directory in which container exit files are written to by conmon.
container_exits_dir = "/var/run/crio/exits"

# Path to directory for container attach sockets.
container_attach_socket_dir = "/var/run/crio"

# The prefix to use for the source of the bind mounts.
bind_mount_prefix = ""

# If set to true, all containers will run in read-only mode.
read_only = false

# Changes the verbosity of the logs based on the level it is set to. Options
# are fatal, panic, error, warn, info, and debug. This option supports live
# configuration reload.
log_level = "error"

# The UID mappings for the user namespace of each container. A range is
# specified in the form containerUID:HostUID:Size. Multiple ranges must be
# separated by comma.
uid_mappings = ""

# The GID mappings for the user namespace of each container. A range is
# specified in the form containerGID:HostGID:Size. Multiple ranges must be
# separated by comma.
gid_mappings = ""

# The minimal amount of time in seconds to wait before issuing a timeout
# regarding the proper termination of the container.
ctr_stop_timeout = 0

# ManageNetworkNSLifecycle determines whether we pin and remove network namespace
# and manage its lifecycle.
manage_network_ns_lifecycle = false

# The "crio.runtime.runtimes" table defines a list of OCI compatible runtimes.
# The runtime to use is picked based on the runtime_handler provided by the CRI.
# If no runtime_handler is provided, the runtime will be picked based on the level
# of trust of the workload. Each entry in the table should follow the format:
#
#[crio.runtime.runtimes.runtime-handler]
#  runtime_path = "/path/to/the/executable"
#  runtime_type = "oci"
#  runtime_root = "/path/to/the/root"
#
# Where:
# - runtime-handler: name used to identify the runtime
# - runtime_path (optional, string): absolute path to the runtime executable in
#   the host filesystem. If omitted, the runtime-handler identifier should match
#   the runtime executable name, and the runtime executable should be placed
#   in $PATH.
# - runtime_type (optional, string): type of runtime, one of: "oci", "vm". If
#   omitted, an "oci" runtime is assumed.
# - runtime_root (optional, string): root directory for storage of containers
#   state.
[crio.runtime.runtimes.runc]
runtime_path = ""
runtime_type = "oci"
runtime_root = "/run/runc"

# CRI-O reads its configured registries defaults from the system wide
# containers-registries.conf(5) located in /etc/containers/registries.conf. If
# you want to modify just CRI-O, you can change the registries configuration in
# this file. Otherwise, leave insecure_registries and registries commented out to
# use the system's defaults from /etc/containers/registries.conf.
[crio.image]

# Default transport for pulling images from a remote container storage.
default_transport = "docker://"

# The path to a file containing credentials necessary for pulling images from
# secure registries. The file is similar to that of /var/lib/kubelet/config.json
global_auth_file = "/var/lib/kubelet/config.json"

# The image used to instantiate infra containers.
# This option supports live configuration reload.
pause_image = "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee50534c2f97c6c9a6baeacf120cf7b4f13a8a59f6b673b152006ad7fb8702d6"

# The path to a file containing credentials specific for pulling the pause_image from
# above. The file is similar to that of /var/lib/kubelet/config.json
# This option supports live configuration reload.
pause_image_auth_file = "/var/lib/kubelet/config.json"

# The command to run to have a container stay in the paused state.
# When explicitly set to "", it will fallback to the entrypoint and command
# specified in the pause image. When commented out, it will fallback to the
# default: "/pause". This option supports live configuration reload.
pause_command = "/usr/bin/pod"

# Path to the file which decides what sort of policy we use when deciding
# whether or not to trust an image that we've pulled. It is not recommended that
# this option be used, as the default behavior of using the system-wide default
# policy (i.e., /etc/containers/policy.json) is most often preferred. Please
# refer to containers-policy.json(5) for more details.
signature_policy = ""

# Controls how image volumes are handled. The valid values are mkdir, bind and
# ignore; the latter will ignore volumes entirely.
image_volumes = "mkdir"

# The crio.network table containers settings pertaining to the management of
# CNI plugins.
[crio.network]
# Path to the directory where CNI configuration files are located.
network_dir = "/etc/kubernetes/cni/net.d/"

# Paths to directories where CNI plugin binaries are located.
plugin_dirs = [
    "/var/lib/cni/bin",
]

# A necessary configuration for Prometheus based metrics retrieval
[crio.metrics]

# Globally enable or disable metrics support.
enable_metrics = true

# The port on which the metrics server will listen.
metrics_port = 9537


PID拡張手順は以上となります。
前提作業の完了後、Db2 11.5.4 on OpenShift を導入した時の手順・ログはこちらに書きました。
 ---> Db2 11.5.4 on OpenShift をデプロイしてみた