k8s ServiceNodePortRange 是什麼

在 Kubernetes 中，ServiceNodePortRange 是一個用於指定 NodePort 服務端口範圍的參數。該參數定義了可以分配給 NodePort 服務的端口範圍。默認情況下，NodePort 服務使用的端口範圍是 30000 到 32767。

如何查看 k8s ServiceNodePortRange

k8s master 節點上的 kube-apiserver 進程啓動時，會指定參數 --service-node-port-range=xxx-xxx，該參數的值就是 ServiceNodePortRange 的值

使用 net.ipv4.ip_local_reserved_ports 配置保留端口範圍

文檔

ip_local_reserved_ports - list of comma separated ranges
    Specify the ports which are reserved for known third-party
    applications. These ports will not be used by automatic port
    assignments (e.g. when calling connect() or bind() with port
    number 0). Explicit port allocation behavior is unchanged.

    The format used for both input and output is a comma separated
    list of ranges (e.g. "1,2-4,10-10" for ports 1, 2, 3, 4 and
    10). Writing to the file will clear all previously reserved
    ports and update the current list with the one given in the
    input.

    Note that ip_local_port_range and ip_local_reserved_ports
    settings are independent and both are considered by the kernel
    when determining which ports are available for automatic port
    assignments.

    You can reserve ports which are not in the current
    ip_local_port_range, e.g.:

    $ cat /proc/sys/net/ipv4/ip_local_port_range
    32000    60999
    $ cat /proc/sys/net/ipv4/ip_local_reserved_ports
    8080,9148

    although this is redundant. However such a setting is useful
    if later the port range is changed to a value that will
    include the reserved ports.

    Default: Empty

ref: https://www.kernel.org/doc/Documentation/networking/ip-sysctl...

net.ipv4.ip_local_reserved_ports 是 Linux 內核參數，用於指定保留的本地端口範圍，這些端口不會被隨機分配給普通用户程序

防止衝突
防止普通用户程序佔用 NodePort 服務端口範圍，導致 NodePort 服務無法正常使用

net.ipv4.ip_local_reserved_ports="30000–32768"

這麼寫在我的系統上會報錯 sysctl: setting key "net.ipv4.ip_local_reserved_ports": Invalid argument

這麼寫是可以的

sysctl -w net.ipv4.ip_local_reserved_ports=31000,32222,30080

修改NodePort端口範圍時必須十分謹慎。務必保證NodePort端口範圍與集羣節點上Linux內核提供的net.ipv4.ip_local_port_range參數中的端口範圍不衝突。該內核參數ip_local_port_range控制了Linux系統上任意應用程序可以使用的本地端口號範圍。ip_local_port_range的默認值為32768~60999。

查看已經使用的 NodePort

kubectl get svc -A -o jsonpath='{range .items[*]}{.spec.ports[*].nodePort}{","}'

ref

• https://www.cnblogs.com/charlieroro/p/17387611.html
• https://www.cnblogs.com/zhangmingcheng/p/15031143.html
• https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/...
• https://www.kernel.org/doc/Documentation/networking/ip-sysctl...
• https://github.com/kubernetes/kubernetes/pull/115374
• https://github.com/kubernetes/kubernetes/issues/111144