learn.lianglianglee.com/专栏/Kubernetes 从上手到实践/13 庖丁解牛:etcd.md.html
2022-05-11 19:04:14 +08:00

698 lines
20 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<!-- saved from url=(0046)https://kaiiiz.github.io/hexo-theme-book-demo/ -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=no">
<link rel="icon" href="/static/favicon.png">
<title>13 庖丁解牛etcd.md.html</title>
<!-- Spectre.css framework -->
<link rel="stylesheet" href="/static/index.css">
<!-- theme css & js -->
<meta name="generator" content="Hexo 4.2.0">
</head>
<body>
<div class="book-container">
<div class="book-sidebar">
<div class="book-brand">
<a href="/">
<img src="/static/favicon.png">
<span>技术文章摘抄</span>
</a>
</div>
<div class="book-menu uncollapsible">
<ul class="uncollapsible">
<li><a href="/" class="current-tab">首页</a></li>
</ul>
<ul class="uncollapsible">
<li><a href="../">上一级</a></li>
</ul>
<ul class="uncollapsible">
<li>
<a href="/专栏/Kubernetes 从上手到实践/01 开篇: Kubernetes 是什么以及为什么需要它.md.html">01 开篇: Kubernetes 是什么以及为什么需要它.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/02 初步认识Kubernetes 基础概念.md.html">02 初步认识Kubernetes 基础概念.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/03 宏观认识:整体架构.md.html">03 宏观认识:整体架构.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/04 搭建 Kubernetes 集群 - 本地快速搭建.md.html">04 搭建 Kubernetes 集群 - 本地快速搭建.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/05 动手实践:搭建一个 Kubernetes 集群 - 生产可用.md.html">05 动手实践:搭建一个 Kubernetes 集群 - 生产可用.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/06 集群管理:初识 kubectl.md.html">06 集群管理:初识 kubectl.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/07 集群管理:以 Redis 为例-部署及访问.md.html">07 集群管理:以 Redis 为例-部署及访问.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/08 安全重点 认证和授权.md.html">08 安全重点 认证和授权.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/09 应用发布:部署实际项目.md.html">09 应用发布:部署实际项目.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/10 应用管理:初识 Helm.md.html">10 应用管理:初识 Helm.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/11 部署实践:以 Helm 部署项目.md.html">11 部署实践:以 Helm 部署项目.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/12 庖丁解牛kube-apiserver.md.html">12 庖丁解牛kube-apiserver.md.html</a>
</li>
<li>
<a class="current-tab" href="/专栏/Kubernetes 从上手到实践/13 庖丁解牛etcd.md.html">13 庖丁解牛etcd.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/14 庖丁解牛controller-manager.md.html">14 庖丁解牛controller-manager.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/15 庖丁解牛kube-scheduler.md.html">15 庖丁解牛kube-scheduler.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/16 庖丁解牛kubelet.md.html">16 庖丁解牛kubelet.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/17 庖丁解牛kube-proxy.md.html">17 庖丁解牛kube-proxy.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/18 庖丁解牛Container Runtime Docker.md.html">18 庖丁解牛Container Runtime Docker.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/19 Troubleshoot.md.html">19 Troubleshoot.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/20 扩展增强Dashboard.md.html">20 扩展增强Dashboard.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/21 扩展增强CoreDNS.md.html">21 扩展增强CoreDNS.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/22 服务增强Ingress.md.html">22 服务增强Ingress.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/23 监控实践:对 K8S 集群进行监控.md.html">23 监控实践:对 K8S 集群进行监控.md.html</a>
</li>
<li>
<a href="/专栏/Kubernetes 从上手到实践/24 总结.md.html">24 总结.md.html</a>
</li>
</ul>
</div>
</div>
<div class="sidebar-toggle" onclick="sidebar_toggle()" onmouseover="add_inner()" onmouseleave="remove_inner()">
<div class="sidebar-toggle-inner"></div>
</div>
<script>
function add_inner() {
let inner = document.querySelector('.sidebar-toggle-inner')
inner.classList.add('show')
}
function remove_inner() {
let inner = document.querySelector('.sidebar-toggle-inner')
inner.classList.remove('show')
}
function sidebar_toggle() {
let sidebar_toggle = document.querySelector('.sidebar-toggle')
let sidebar = document.querySelector('.book-sidebar')
let content = document.querySelector('.off-canvas-content')
if (sidebar_toggle.classList.contains('extend')) { // show
sidebar_toggle.classList.remove('extend')
sidebar.classList.remove('hide')
content.classList.remove('extend')
} else { // hide
sidebar_toggle.classList.add('extend')
sidebar.classList.add('hide')
content.classList.add('extend')
}
}
function open_sidebar() {
let sidebar = document.querySelector('.book-sidebar')
let overlay = document.querySelector('.off-canvas-overlay')
sidebar.classList.add('show')
overlay.classList.add('show')
}
function hide_canvas() {
let sidebar = document.querySelector('.book-sidebar')
let overlay = document.querySelector('.off-canvas-overlay')
sidebar.classList.remove('show')
overlay.classList.remove('show')
}
</script>
<div class="off-canvas-content">
<div class="columns">
<div class="column col-12 col-lg-12">
<div class="book-navbar">
<!-- For Responsive Layout -->
<header class="navbar">
<section class="navbar-section">
<a onclick="open_sidebar()">
<i class="icon icon-menu"></i>
</a>
</section>
</header>
</div>
<div class="book-content" style="max-width: 960px; margin: 0 auto;
overflow-x: auto;
overflow-y: hidden;">
<div class="book-post">
<p id="tip" align="center"></p>
<div><h1>13 庖丁解牛etcd</h1>
<h2>整体概览</h2>
<pre><code>+----------------------------------------------------------+
| Master |
| +-------------------------+ |
| +-------&gt;| API Server |&lt;--------+ |
| | | | | |
| v +-------------------------+ v |
| +----------------+ ^ +--------------------+ |
| | | | | | |
| | Scheduler | | | Controller Manager | |
| | | | | | |
| +----------------+ v +--------------------+ |
| +------------------------------------------------------+ |
| | | |
| | Cluster state store | |
| | | |
| +------------------------------------------------------+ |
+----------------------------------------------------------+
</code></pre>
<p>在第 3 节《宏观认识:整体架构》 中,我们也认识到了 <code>etcd</code> 的存在,知道了 Master 是 K8S 是集群的大脑,而 <code>etcd</code> 则是大脑的核心。为什么这么说?本节我们一同来看看 <code>etcd</code> 为何如此重要。</p>
<h2><code>etcd</code> 是什么</h2>
<p>先摘录<a href="https://etcd.readthedocs.io/en/latest/faq.html#what-is-etcd">官方文档</a>的一句说明:</p>
<blockquote>
<p>etcd is a consistent distributed key-value store. Mainly used as a separate coordination service, in distributed systems. And designed to hold small amounts of data that can fit entirely in memory.</p>
</blockquote>
<p><code>etcd</code> 是由 CoreOS 团队发起的一个分布式,强一致的键值存储。它用 Go 语言编写,使用 <code>Raft</code> 协议作为一致性算法。多数情况下会用于分布式系统中的服务注册发现,或是用于存储系统的关键数据。</p>
<h2><code>etcd</code> 有什么作用</h2>
<p><code>etcd</code> 在 K8S 中,最主要的作用便是其高可用,强一致的键值存储以及监听机制。</p>
<p><code>kube-apiserver</code> 收到对应请求经过一系列的处理后,最终如果是集群所需要存储的数据,便会存储至 <code>etcd</code> 中。主部分主要是集群状态信息和元信息。</p>
<p>我们来实际操作 K8S 集群中的 <code>etcd</code> 看下真实情况。</p>
<ul>
<li>查找集群中的 <code>etcd</code> Pod</li>
</ul>
<pre><code># 默认集群中的 etcd 会放到 kube-system Namespace 中
master $ kubectl -n kube-system get pods | grep etcd
etcd-master 1/1 Running 0 1h
</code></pre>
<ul>
<li>进入该 Pod 并查看 <code>etcd</code> 集群的 <code>member</code></li>
</ul>
<pre><code>master $ kubectl -n kube-system exec -it etcd-master sh
/ # ETCDCTL_API=3 etcdctl --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt --cacert=/etc/kubernetes/pki/etcd/ca.crt member list
a874c87fd42044f, started, master, https://127.0.0.1:2380, https://127.0.0.1:2379
</code></pre>
<p>这里由于在 K8S 1.11.3 中默认使用的是 <code>etcd</code> 3.2 版本,所以需要加入 <code>ETCDCTL_API=3</code> 的环境变量,且 <code>etcd</code> 从 2 到 3 很明显的一个变化也是使用上的变化,在 2 中是 HTTP 接口的。</p>
<p>我们通过传递证书等相关参数进去,完成校验,查看 <code>member</code></p>
<ul>
<li>查看存储的元信息</li>
</ul>
<p><code>etcd</code> 中存储的 K8S 集群元信息基本都是 <code>/registry</code> 下,我们可通过下面的方式进行查看</p>
<pre><code>/ # ETCDCTL_API=3 etcdctl --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt --cacert=/etc/kubernetes/pki/etcd/ca.crt get /registry --prefix --keys-only
/registry/apiregistration.k8s.io/apiservices/v1.
/registry/clusterroles/system:aggregate-to-admin
/registry/configmaps/kube-public/cluster-info
/registry/masterleases/172.17.0.53
/registry/namespaces/default
/registry/namespaces/kube-public
/registry/namespaces/kube-system
/registry/pods/kube-system/etcd-master
/registry/pods/kube-system/kube-apiserver-master
/registry/ranges/serviceips
/registry/ranges/servicenodeports
...
# 篇幅原因,删掉了很多输出
</code></pre>
<p>可以看到有各种类型的资源。我们直接以 Namespaces 为例。</p>
<ul>
<li>查看 Namespaces 信息</li>
</ul>
<pre><code>/ # ETCDCTL_API=3 etcdctl --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt --cacert=/etc/kubernetes/pki/etcd/ca.crt get /registry/namespaces --prefix --keys-only
/registry/namespaces/default
/registry/namespaces/kube-public
/registry/namespaces/kube-system
</code></pre>
<ul>
<li>使用 <code>kubectl</code> 创建名为 <code>moelove</code> 的 Namespaces</li>
</ul>
<pre><code>master $ kubectl create ns moelove
namespace/moelove created
</code></pre>
<ul>
<li>查看 Namespaces 信息</li>
</ul>
<pre><code>/ # ETCDCTL_API=3 etcdctl --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt --cacert=/etc/kubernetes/pki/etcd/ca.crt get /registry/namespaces --prefix --keys-only
/registry/namespaces/default
/registry/namespaces/kube-public
/registry/namespaces/kube-system
/registry/namespaces/moelove
</code></pre>
<p>发现刚创建的 <code>moelove</code> 的 Namespaces 已经在 <code>etcd</code> 中了。</p>
<h2><code>etcd</code> 是如何被使用的</h2>
<p>首先,在 <code>staging/src/k8s.io/apiserver/pkg/server/options/etcd.go</code> 中,存在一处声明:</p>
<pre><code>var storageTypes = sets.NewString(
storagebackend.StorageTypeUnset,
storagebackend.StorageTypeETCD2,
storagebackend.StorageTypeETCD3,
)
</code></pre>
<p><strong>在 1.13 发布时etcd 2 的相关代码已经移除,其中就包含上面声明中的 storagebackend.StorageTypeETCD2</strong></p>
<p>这里是在过渡期间为了同时兼容 <code>etcd</code> 2 和 3 而增加的。</p>
<p>我们来看下实际对各类资源的操作,还是以对 <code>Namespace</code> 的操作为例:代码在 <code>pkg/registry/core/namespace/storage/storage.go</code> 中,</p>
<p>比如,<code>Get</code></p>
<pre><code>func (r *REST) Get(ctx context.Context, name string, options *metav1.GetOptions) (runtime.Object, error) {
return r.store.Get(ctx, name, options)
}
</code></pre>
<p><code>REST</code> 则是这样定义的:</p>
<pre><code>type REST struct {
store *genericregistry.Store
status *genericregistry.Store
}
</code></pre>
<p>通过 <code>REST</code> 实现了一个 <code>RESTStorage</code>,实际使用时,也是调用了 <code>staging/src/k8s.io/apiserver/pkg/registry/generic/registry/store.go</code> 对接口的封装。</p>
<pre><code>func (e *Store) Get(ctx context.Context, name string, options *metav1.GetOptions) (runtime.Object, error) {
obj := e.NewFunc()
key, err := e.KeyFunc(ctx, name)
if err != nil {
return nil, err
}
if err := e.Storage.Get(ctx, key, options.ResourceVersion, obj, false); err != nil {
return nil, storeerr.InterpretGetError(err, e.qualifiedResourceFromContext(ctx), name)
}
if e.Decorator != nil {
if err := e.Decorator(obj); err != nil {
return nil, err
}
}
return obj, nil
}
</code></pre>
<p>在该处实现了对各类基本方法的封装,各种资源类型都会统一去调用。而更上层则是对 <code>storagebackend</code> 的统一封装,最终会调用 <code>etcd</code> 客户端的实现完成想对应的操作,这里就不再多展开了。</p>
<h2>总结</h2>
<p>在本节中,我们介绍了 <code>etcd</code> 以及它在 K8S 中的作用以及如何使用。我们还介绍了如何通过 <code>etcdctl</code> 工具去操作 <code>etcd</code></p>
<p>在某些极端情况下,也许你需要通过直接操作 <code>etcd</code> 集群去变更数据,这里没有介绍所有的操作命令,感兴趣的可以自行通过下方的链接看官方文档进行学习。</p>
<p>但通常情况下,不建议直接操作 <code>etcd</code> ,除非你已经明确自己在做什么。</p>
<p>另外,由于 <code>etcd</code> 集群使用 <code>Raft</code> 一致性算法,通常情况下 <code>etcd</code> 集群需要部署奇数个节点,如 357 等。<code>etcd</code> 集群维护也相对容易,很容易可以做成高可用集群。(这也是保障 K8S 集群高可用的重要一环)</p>
<p>学习了 <code>etcd</code> 之后,下节我们来学习同样很重要的一个组件 <code>Controller Manager</code>,学习它是如何保障集群符合预期状态的。</p>
<h2>参考链接</h2>
<ul>
<li><a href="https://etcd.readthedocs.io/en/latest/faq.html#what-is-etcd">What is etcd</a></li>
</ul>
</div>
</div>
<div>
<div style="float: left">
<a href="/专栏/Kubernetes 从上手到实践/12 庖丁解牛kube-apiserver.md.html">上一页</a>
</div>
<div style="float: right">
<a href="/专栏/Kubernetes 从上手到实践/14 庖丁解牛controller-manager.md.html">下一页</a>
</div>
</div>
</div>
</div>
</div>
</div>
<a class="off-canvas-overlay" onclick="hide_canvas()"></a>
</div>
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/v652eace1692a40cfa3763df669d7439c1639079717194" integrity="sha512-Gi7xpJR8tSkrpF7aordPZQlW2DLtzUlZcumS8dMQjwDHEnw9I7ZLyiOj/6tZStRBGtGgN6ceN6cMH8z7etPGlw==" data-cf-beacon='{"rayId":"70997244683b3d60","version":"2021.12.0","r":1,"token":"1f5d475227ce4f0089a7cff1ab17c0f5","si":100}' crossorigin="anonymous"></script>
</body>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-NPSEEVD756"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag() {
dataLayer.push(arguments);
}
gtag('js', new Date());
gtag('config', 'G-NPSEEVD756');
var path = window.location.pathname
var cookie = getCookie("lastPath");
console.log(path)
if (path.replace("/", "") === "") {
if (cookie.replace("/", "") !== "") {
console.log(cookie)
document.getElementById("tip").innerHTML = "<a href='" + cookie + "'>跳转到上次进度</a>"
}
} else {
setCookie("lastPath", path)
}
function setCookie(cname, cvalue) {
var d = new Date();
d.setTime(d.getTime() + (180 * 24 * 60 * 60 * 1000));
var expires = "expires=" + d.toGMTString();
document.cookie = cname + "=" + cvalue + "; " + expires + ";path = /";
}
function getCookie(cname) {
var name = cname + "=";
var ca = document.cookie.split(';');
for (var i = 0; i < ca.length; i++) {
var c = ca[i].trim();
if (c.indexOf(name) === 0) return c.substring(name.length, c.length);
}
return "";
}
</script>
</html>