atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>林夕水共</title>
  
  <subtitle>关于技术,关于生活</subtitle>
  <link href="/atom.xml" rel="self"/>
  
  <link href="https://rebootcat.com/"/>
  <updated>2021-04-02T15:26:54.797Z</updated>
  <id>https://rebootcat.com/</id>
  
  <author>
    <name>Smaug</name>
    
  </author>
  
  <generator uri="https://hexo.io/">Hexo</generator>
  
  <entry>
    <title>P2P 打洞技术详解</title>
    <link href="https://rebootcat.com/2021/03/28/p2p_nat_traversal/"/>
    <id>https://rebootcat.com/2021/03/28/p2p_nat_traversal/</id>
    <published>2021-03-28T15:50:58.000Z</published>
    <updated>2021-04-02T15:26:54.797Z</updated>
    
    <content type="html"><![CDATA[<h1 id="何为打洞？-what"><a href="#何为打洞？-what" class="headerlink" title="何为打洞？(what)"></a>何为打洞？(what)</h1><h2 id="英文翻译"><a href="#英文翻译" class="headerlink" title="英文翻译"></a>英文翻译</h2><ul><li>NAT traversal : NAT 穿越</li><li>NAT Hole Punching : NAT 打孔</li></ul><h2 id="定义"><a href="#定义" class="headerlink" title="定义"></a>定义</h2><p>(UDP) 打洞技术是通过<strong>中间公网服务器</strong>的协助在通信双方的 <strong>NAT 网关</strong>上建立相关的<strong>映射表项</strong>，使得双方发送的报文能直接<strong>穿透</strong>对方的 NAT 网关（防火墙），实现 P2P 直连。</p><blockquote><p>洞：所谓的洞就是映射规则，外部能够主动与之通信的规则</p></blockquote><h1 id="为何要打洞？-why"><a href="#为何要打洞？-why" class="headerlink" title="为何要打洞？(why)"></a>为何要打洞？(why)</h1><h2 id="直接连不行吗？"><a href="#直接连不行吗？" class="headerlink" title="直接连不行吗？"></a>直接连不行吗？</h2><ul><li><strong>NAT</strong> 技术的存在，一方面减缓了 IPV4 的需求，使得私网 IP 地址通过映射成公网 IP 地址的方式与外界通信</li><li>但另外一方面， <strong>NAT</strong> 也对安全性做了限制（防火墙），<strong>外界不能主动与私网 IP 进行通信</strong></li></ul><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210324201346047.png" alt="image-20210324201346047"></p><h2 id="打洞有什么好处？"><a href="#打洞有什么好处？" class="headerlink" title="打洞有什么好处？"></a>打洞有什么好处？</h2><ul><li>节省流量</li><li>降低中心服务器压力</li><li>下载速度快（比如迅雷、直播等）</li><li>安全性和私密性</li></ul><a id="more"></a><h1 id="如何打洞？（how"><a href="#如何打洞？（how" class="headerlink" title="如何打洞？（how)"></a>如何打洞？（how)</h1><p>通常我们说的打洞技术基本上都是使用 UDP 来实现的，当然 TCP 也行，只不过会复杂一点（后面我们讨论一下 TCP 打洞）。</p><h2 id="使用中继设备（proxy"><a href="#使用中继设备（proxy" class="headerlink" title="使用中继设备（proxy)"></a>使用中继设备（proxy)</h2><p>主要利⽤第三⽅的服务器作为中转服务器，⽐如Application Level Gateway (ALG),application server, application server with agent, <strong>TURN</strong>。</p><p>有点是稳定可靠，缺点是延迟较⼤，不适合在p2p的⽹络中使⽤。</p><blockquote><p>本质上其实不算 P2P 打洞的范畴了。</p></blockquote><h2 id="直连"><a href="#直连" class="headerlink" title="直连"></a>直连</h2><p>基本思路是：<strong>A 和 B 互相知道对方的公网 IP:Port，使用对方公网 IP:Port 通信</strong>。</p><p>通过修改通信协议，或者 NAT 设备来帮助节点之前的直接通信，⽐如： tunnel, NAT-PMP, <strong>UPnP</strong>, MidCom, <strong>STUN</strong> hole punching, ICE, Teredo等。 其中STUN hole punching⽅法在实践中应⽤最为 广泛。</p><ul><li>UPnP: 把内网 IP 直接映射为 NAT 外网 IP (纯转发模式)，类似公网 IP</li><li>STUN: 比较出名的打洞协议，本质上就是利用一台或多台公网服务器协助位于不同私网的两个节点 A 和 B 进行打洞。</li></ul><h1 id="NAT-设备类型"><a href="#NAT-设备类型" class="headerlink" title="NAT 设备类型"></a>NAT 设备类型</h1><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210324201449678.png" alt="image-20210324201449678"></p><ul><li><p>Full Cone NAT： 允许任何外部 IP 任何端⼝连⼊ NAT，只要 NAT 内部 host 产⽣过 IP 端⼝映射。 (<strong>不限制任何 IP</strong>)</p></li><li><p>Restricted Cone NAT： 只允许外部指定 IP 连⼊当前 NAT，即 NAT 内 host 主动连 接过的 IP。（<strong>限制 IP</strong>)</p></li><li><p>Port-Restricted Cone NAT：只允许内⽹设备主动 连接过的外⽹ IP 和 Port 连⼊。（<strong>限制 IP + Port</strong>）</p></li><li><p>Symmetric NAT：这种类型的 NAT ⾏为跟端⼝限制型的 NAT 类型相似，不同的是，对于向外连接的不同的 IP 和 Port，NAT 随机分配⼀个 Port 来完成地址转换，完成对外连接。（<strong>只要向外的 IP:Port 不一致则映射到不同端口</strong>）</p></li></ul><p>注意：<strong>对于前三种 NAT 设备，内网节点连接不同的 server {IP, Port} 对映射外网端口一致</strong>；</p><blockquote><p>主要有以上几种，市面上还有少量的其他类型</p></blockquote><h1 id="Full-Cone-NAT-全锥形"><a href="#Full-Cone-NAT-全锥形" class="headerlink" title="Full Cone NAT (全锥形)"></a>Full Cone NAT (全锥形)</h1><ul><li>NAT 映射规则形成后，外部其他主机能够主动与之通信</li></ul><p>举例如下：</p><ol><li>A 位于私网内部</li><li>A 访问 <a href="https://www.google.com/" target="_blank" rel="noopener">https://www.google.com/</a> </li><li>NAT-A 记录了映射规则，比如 {local_host,local_port,public_port,dest_host} 等信息</li><li>外部其他主机（比如 google.com) 能主动与 A 通信（通过 A  的 public_ip:public_port)</li></ol><h1 id="Restricted-Cone-NAT（限制型）"><a href="#Restricted-Cone-NAT（限制型）" class="headerlink" title="Restricted Cone NAT（限制型）"></a>Restricted Cone NAT（限制型）</h1><p>在 Full Cone NAT 的基础上多加了一条限制规则：</p><ul><li>只允许访问过的 server IP 与之通信</li></ul><p>举例如下：还是上面的例子，后续<strong>只允许</strong> <a href="https://www.google.com/" target="_blank" rel="noopener">https://www.google.com/</a>  (对应 IP 203.107.53.50) 与之通信，而<strong>不 care 203.107.53.50 的端口号</strong>（比如从 6666 端口过来的数据）。</p><h1 id="Port-Restricted-Cone-NAT（端口限制型）"><a href="#Port-Restricted-Cone-NAT（端口限制型）" class="headerlink" title="Port-Restricted Cone NAT（端口限制型）"></a>Port-Restricted Cone NAT（端口限制型）</h1><p>在 Restricted Cone NAT 的基础上多加了一条限制规则：</p><ul><li>只允许访问过的 server {IP, Port} 与之通信</li></ul><p>举例如下：还是上面的例子，后续<strong>只允许</strong> <a href="https://www.google.com/" target="_blank" rel="noopener">https://www.google.com/</a>  (对应 IP 203.107.53.50) 的 80 端口与之通信，<strong>其他端口不行</strong>。</p><h1 id="Symmetric-NAT（对称型）"><a href="#Symmetric-NAT（对称型）" class="headerlink" title="Symmetric NAT（对称型）"></a>Symmetric NAT（对称型）</h1><p>访问规则同 Port-Restricted Cone NAT：</p><ul><li>只允许访问过的 server {IP, Port} 与之通信</li></ul><p>区别是连接不同的 server {IP, Port}，映射到 NAT 上的公网 Port 不一致，且<strong>映射规则不确定</strong>。</p><p>有些 NAT 设备会进行简单的 +1 操作实现端口映射，比如：{local_port: 6000, public_port: 6000, Server1}，{local_port: 6000, public_port: 6001, Server2}，{local_port: 6000, public_port: 6003, Server3}</p><p>有些 NAT 设备为了安全性，可能会随机进行端口映射，提高端口猜测的难度。</p><h1 id="场景分析"><a href="#场景分析" class="headerlink" title="场景分析"></a>场景分析</h1><p>(假设已经获取到⽬标邻居的IP Port 等信息)</p><ol><li>先向公⽹服务器发起请求，探测⾃⼰是不是在⼀个 NAT 下</li><li>如果不在，说明自己是公网节点，可以与目标建立连接（让目标主动连接）</li><li>如果在，则获取⾃⼰的 NAT 类型是什么，根据不同的类型采取相应不同的策略</li></ol><blockquote><p>注意：A 和 B 均要与 server 一致保持连接心跳，确保 NAT 映射端口有效</p></blockquote><h2 id="节点-A-在-NAT-下面，节点-B-在公网环境中"><a href="#节点-A-在-NAT-下面，节点-B-在公网环境中" class="headerlink" title="节点 A 在 NAT 下面，节点 B 在公网环境中"></a>节点 A 在 NAT 下面，节点 B 在公网环境中</h2><p>打洞策略：</p><ul><li>只要保证节点 A 主动向节点 B 发起连接，两者就可以连接成功</li></ul><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210324212632428.png" alt="image-20210324212632428"></p><h2 id="节点-A-和-B-在相同的-NAT-下"><a href="#节点-A-和-B-在相同的-NAT-下" class="headerlink" title="节点 A 和 B 在相同的 NAT 下"></a>节点 A 和 B 在相同的 NAT 下</h2><p>打洞策略：</p><ol><li>A 从 server 获取 B 的信息，server 同时发给 A 和 B 对方的信息（公网/私网信息）</li><li>A 与 B 各自收到 server 的信息后，同时向对方发起连接，公网和私网都发（理论上私网会快一点），一旦收到回复，则停止</li><li>建立连接成功</li></ol><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210324213025505.png" alt="image-20210324213025505"></p><h2 id="节点-A-和-B-在不同的-NAT-下（这是我们要重点讨论的）"><a href="#节点-A-和-B-在不同的-NAT-下（这是我们要重点讨论的）" class="headerlink" title="节点 A 和 B 在不同的 NAT 下（这是我们要重点讨论的）"></a>节点 A 和 B 在不同的 NAT 下（这是我们要重点讨论的）</h2><p><strong>节点 A 和 B 的 NAT 可能是任意一种 NAT 类型</strong></p><ul><li>A 和 B 均是锥形 NAT</li><li>A 和 B 分别是对称型和普通锥形(全锥形，限制型锥形)</li><li>A 和 B 分别是对称型和 Port-Restricted Cone NAT （端口限制型）</li><li>A  和 B 都是对称型</li></ul><h2 id="A-和-B-均是锥形-NAT"><a href="#A-和-B-均是锥形-NAT" class="headerlink" title="A 和 B 均是锥形 NAT"></a>A 和 B 均是锥形 NAT</h2><p>锥形 NAT 之间可以容易的打洞成功，具体流程如下：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210324222800953.png" alt="image-20210324222800953"></p><p>以下流程假定已经完成了 NAT 类型探测，A/B 知道自己的 NAT 类型，以及通过 NAT 映射出去的端口 PA1/PB1。</p><p>打洞策略：</p><ol><li>A 向 server 发起与 B 的打洞请求，server 向 B  转发打洞请求，同时A 向 PB1 直接发送探测包，那么 A 为 B 在 PA1 已经成功打洞，但是 A 的消息无法到达，因为 B 的 NAT 会将不明的地址(PA1) <strong>丢弃</strong>。（注意：<strong><em>这里有可能不是丢弃，而是拒绝</em></strong>）</li><li>B 收到从 server  转发过来的打洞请求后，向 PA1 直接发送探测包，这时 B 的 NAT 可以放行  PA1 的消息了，也就是 B 为 A 在 PB1 上完成了打洞。</li><li>至此，A 和 B 消息能够互通，打洞成功</li></ol><blockquote><p>注意，上面斜体部分，NAT 对不明地址的行为可能是拒绝，待会会讨论。</p></blockquote><h2 id="A-和-B-分别是对称型和普通锥形-全锥形，限制型锥形"><a href="#A-和-B-分别是对称型和普通锥形-全锥形，限制型锥形" class="headerlink" title="A 和 B 分别是对称型和普通锥形(全锥形，限制型锥形)"></a>A 和 B 分别是对称型和普通锥形(全锥形，限制型锥形)</h2><p>假设 A 是对称 NAT，B  是普通锥形：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210325095314524.png" alt="image-20210325095314524"></p><p>打洞策略：</p><ol><li>A 向 server 发起与 B 的打洞请求，server 向 B  转发打洞请求，同时发送探测包到 PB1，这个<strong>探测包是从 PA2 发出</strong>的，不是 PA1(因为对称型)。也就是 A 在端口 PA2 为 PB1 完成打洞，同时 B 的 NAT 会<strong>丢弃</strong>来自不明地址 PA2 的包。（注意：<strong><em>这里有可能不是丢弃，而是拒绝</em></strong>）</li><li>B 收到从 server 转发过来的打洞请求，向 PA1 发送初始探测包（一开始不知道 PA2)，这个时候 <span style="color:red">B 已经为 A 在 PB1 打好洞</span>，至此 PA2 的消息能够通过 PB1 到达 B。（注意：<strong>因为是普通锥形，不对端口做限制，所以从不同端口 PA2 过来的包能被 B 接受</strong>）</li><li>经过步骤2，B 可以收到 PA2 的消息，同时结合 A 的 NAT 类型，重新改发探测包到 PA2，于是 A 在 PA2 能收到 PB1 的探测包，至此 A 和 B 消息可以互通，打洞成功</li></ol><blockquote><p>如果 A 和 B 正好角色相反，那么可以调整打洞的方向即可</p></blockquote><h2 id="A-和-B-分别是对称型和-Port-Restricted-Cone-NAT-（端口限制型）"><a href="#A-和-B-分别是对称型和-Port-Restricted-Cone-NAT-（端口限制型）" class="headerlink" title="A 和 B 分别是对称型和 Port-Restricted Cone NAT （端口限制型）"></a>A 和 B 分别是对称型和 Port-Restricted Cone NAT （端口限制型）</h2><p>原本大致过程是同上面一种场景，但是由于 B 是端口限制型 NAT，会导致 PB1 只允许 PA1 通过（上面红色字体部分<span style="color:red">B 已经为 A 在 PB1 打好洞</span>），从而 PA2 过来的包会被 B 的 NAT 拒绝，导致<strong>打洞失败</strong>。</p><h2 id="A-和-B-都是对称型"><a href="#A-和-B-都是对称型" class="headerlink" title="A  和 B 都是对称型"></a>A  和 B 都是对称型</h2><p>由于 A  和 B 均是对称型 NAT，那么比上面一种场景更严格，A  和 B 探测得到的公网 Port 均会被修改，<strong>无法完成打洞</strong>。</p><h1 id="对称型打洞真的没有办法了吗？"><a href="#对称型打洞真的没有办法了吗？" class="headerlink" title="对称型打洞真的没有办法了吗？"></a>对称型打洞真的没有办法了吗？</h1><p>我们再来考虑<strong>对称型和端口限制型</strong>的打洞，由于 B 收到 server 转发过来的打洞请求后，是向  PA1 发送探测包的，因为 B 只知道 PA1（PA1 是 A 与 server 连接是映射的端口号，server 也只知道 PA1)，但是 A 由于是对称型 NAT，会从一个新端口 PA2 向 B 发包，但是 B 由于是端口限制型，只允许 PA1 端口的包通过，所以 B 会拒绝 PA2。</p><p>还是上面那张图：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210325102706630.png" alt="image-20210325102706630"></p><p>A 从 PA2 发向 B  的包一直会被 B  拒绝，也就是说 B 无法在 NAT-B  上为 A 打洞。</p><p>那<strong><em>假如 B 探测包不是发往  PA1 而是 PA2 呢</em></strong>？那 A 和 B 就能打洞成功。</p><p><span style="color:red">那么问题来了，B 如何知道 PA2 呢？</span></p><p>通常来讲，有两种办法：</p><ul><li>端口探测</li><li>端口预测</li></ul><h2 id="端口探测"><a href="#端口探测" class="headerlink" title="端口探测"></a>端口探测</h2><p>对于对称型的 NAT在映射内网端口的时候，有一些 NAT 设备会采取比较傻瓜的端口分配方法，比如进行<strong>简单的线性变化</strong>。</p><ul><li>比如每次分配的端口号递增  1</li><li>PA2 = (PA1 + PB1 + IPA + IPB) % 65535</li></ul><p>对于这种 NAT，要探测这种特性需要用到两台及以上的公网 server，通过与不同的 server 连接映射的公网 Port，归纳总结自己的 NAT  映射规律，那么对于 B 来说，打洞的时候第一次向  A 发包，就直接往 PA2 发包就好了。</p><h2 id="端口预测"><a href="#端口预测" class="headerlink" title="端口预测"></a>端口预测</h2><p>有一些对称型 NAT 为了安全考虑，分配端口的方法难以预测，比如随机分配端口，那么对于这种情况，如何预测端口号呢？</p><ul><li>基于一个理论：生日攻击理论</li></ul><blockquote><p>生日攻击理论讲的是在一个班级里，每个人的生日可能是 365 天里的任何一天，每年有 365 天，如果要让 <strong>至少有两人的生日相同的概率超过 50%</strong>，问这个班级最少需要多少人？</p></blockquote><p>答案是：(xx)</p><p>是不是出乎预料？</p><p>生日攻击理论说的直白点就是，<span style="color:red">利用了远小于样本集的尝试次数，就能够很大概率获得两个相同的碰撞采样结果</span>。</p><p>那么针对端口号的样本集 65535，实际是 (1025, 65535]，双方随机打洞需要尝试多少次（打多少洞）才能刚好碰撞成功呢？</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&gt;&gt;&gt; </span>cat nat_birth_attack.py                                                                                                                                                                                         </span><br><span class="line"></span><br><span class="line"><span class="comment">#!/usr/bin/env python</span></span><br><span class="line"><span class="comment">#-*- coding:utf8 -*-</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> math</span><br><span class="line"><span class="keyword">from</span> decimal <span class="keyword">import</span> *</span><br><span class="line"></span><br><span class="line">getcontext().prec = <span class="number">6</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">birthday_attack</span><span class="params">(total, rate)</span>:</span></span><br><span class="line">    k = <span class="number">0</span></span><br><span class="line">    <span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line">        k += <span class="number">1</span></span><br><span class="line">        a,t = <span class="number">1</span>,<span class="number">1</span></span><br><span class="line">        <span class="keyword">for</span> i <span class="keyword">in</span> range(k):</span><br><span class="line">            a = a * (total - i)</span><br><span class="line">            t = t *  total</span><br><span class="line"></span><br><span class="line">        np = Decimal(a) / Decimal(t)</span><br><span class="line">        p = <span class="number">1</span> - np</span><br><span class="line">        <span class="keyword">if</span> p &gt; rate:</span><br><span class="line">            print(<span class="string">'total:&#123;0&#125; trytimes:&#123;1&#125;  result:&#123;2&#125; &gt; target_rate:&#123;3&#125; success'</span>.format(total, k, p, rate))</span><br><span class="line">            <span class="keyword">break</span></span><br><span class="line">        <span class="keyword">else</span>:</span><br><span class="line">            print(<span class="string">'total:&#123;0&#125; trytimes:&#123;1&#125;  result:&#123;2&#125; &lt; target_rate:&#123;3&#125; failed, continue...'</span>.format(total, k, p, rate))</span><br><span class="line">            <span class="keyword">continue</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> k</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line">    total_samples = <span class="number">365</span></span><br><span class="line">    rate = <span class="number">0.5</span></span><br><span class="line">    min_k =  birthday_attack(total_samples, rate)</span><br><span class="line">    print(<span class="string">"when total_samples is &#123;0&#125;, if request for rate greater than &#123;1&#125;, then try at least &#123;2&#125; times is ok\n"</span>.format(total_samples, rate, min_k))</span><br></pre></td></tr></table></figure><p>运行结果：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line">&gt;&gt;&gt; python nat_birth_attack.py                                                                                                                                                                                      ‹git:master ✘› <span class="number">11</span>:<span class="number">00.53</span> 四  <span class="number">3</span> <span class="number">25</span> <span class="number">2021</span> &gt;&gt;&gt;</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">1</span>  result:<span class="number">0</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">2</span>  result:<span class="number">0.002740</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">3</span>  result:<span class="number">0.008204</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">4</span>  result:<span class="number">0.016356</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">5</span>  result:<span class="number">0.027136</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">6</span>  result:<span class="number">0.040462</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">7</span>  result:<span class="number">0.056236</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">8</span>  result:<span class="number">0.074335</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">9</span>  result:<span class="number">0.094624</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">10</span>  result:<span class="number">0.116948</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">11</span>  result:<span class="number">0.141141</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">12</span>  result:<span class="number">0.167025</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">13</span>  result:<span class="number">0.194410</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">14</span>  result:<span class="number">0.223103</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">15</span>  result:<span class="number">0.252901</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">16</span>  result:<span class="number">0.283604</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">17</span>  result:<span class="number">0.315008</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">18</span>  result:<span class="number">0.346911</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">19</span>  result:<span class="number">0.379119</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">20</span>  result:<span class="number">0.411438</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">21</span>  result:<span class="number">0.443688</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">22</span>  result:<span class="number">0.475695</span> &lt; target_rate:<span class="number">0.5</span> failed, <span class="keyword">continue</span>...</span><br><span class="line">total:<span class="number">365</span> trytimes:<span class="number">23</span>  result:<span class="number">0.507297</span> &gt; target_rate:<span class="number">0.5</span> success</span><br><span class="line">when total_samples <span class="keyword">is</span> <span class="number">365</span>, <span class="keyword">if</span> request <span class="keyword">for</span> rate greater than <span class="number">0.5</span>, then <span class="keyword">try</span> at least <span class="number">23</span> times <span class="keyword">is</span> ok</span><br></pre></td></tr></table></figure><p>把 total 修改成 65535，概率 rate 修改成 80%，计算得到尝试次数为 460 次。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">total:<span class="number">65536</span> trytimes:<span class="number">460</span>  result:<span class="number">0.801039</span> &gt; target_rate:<span class="number">0.8</span> success</span><br><span class="line">when total_samples <span class="keyword">is</span> <span class="number">65536</span>, <span class="keyword">if</span> request <span class="keyword">for</span> rate greater than <span class="number">0.8</span>, then <span class="keyword">try</span> at least <span class="number">460</span> times <span class="keyword">is</span> ok</span><br></pre></td></tr></table></figure><p>也就是说<span style="color:red">对于 B 来说，可以尝试随机往  A 的 460 个不同的端口发探测包，就有 80% 的概率能够正好预测到 NAT-A 随机分配的 PA2</span>。</p><p>460 个探测包的代价基本可以忽略不计。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210325113426160.png" alt="image-20210325113426160"></p><p>至此，可以完美实现<strong>对称型和端口限制</strong>型的打洞。然而遗憾的是，对于<strong>对称型和对称型</strong>打洞，依然无法实现。</p><h1 id="再来讨论下-NAT-对陌生地址包的行为"><a href="#再来讨论下-NAT-对陌生地址包的行为" class="headerlink" title="再来讨论下 NAT 对陌生地址包的行为"></a>再来讨论下 NAT 对陌生地址包的行为</h1><p>上面能够打洞成功的场景下，都是基于一个前提是 NAT 对陌生地址发来的包采用的是丢弃策略。这里的陌生地址指的是自己没有主动往外发包的 {dest_ip, dest_port} 对。</p><p>如果不是丢弃，而是采用黑名单机制呢？为了安全考虑，有一些 NAT 在收到陌生地址的包后，会触发防火墙模块，并且在自己的 deny 列表中增加一项{PA2, PB1}，随后自己再往 A 发包的时候，本来打算使用 PB1 进行发包，但是发现 deny 列表里已经存在了 PB1，于是会重新选择一个端口号 PB2 发包。于是对于这种锥形 NAT 会退化成对称型的 NAT。</p><p>知道了这个原理，要解决也很容易。</p><ul><li><strong>设置有限 TTL，避免惊动对方防火墙</strong></li></ul><ol><li><p>一开始 A 往 B 发包，可以设置 TTL 为 3，这个数大到足够通过自己的外网 NAT(可能有多层)，又会被中间的某个运营商 router 丢弃，从而不会惊动  B 的防火墙模块，同时为 B 打好了洞。</p></li><li><p>同理，B 也做类似的操作，为 A 打好洞</p></li><li><p>A  和 B 两边都等待一段时间，比如 2 s</p></li><li><p>再互相发探测包，不用设置 TTL</p></li><li><p>打洞成功</p></li></ol><blockquote><p>关于 TTL 的值设置为多少，需要做一定的探测，不然可能设置过小，也许都没有走出自己的 NAT，设置过大，可能导致惊动了对方的防火墙</p></blockquote><h1 id="总结一下"><a href="#总结一下" class="headerlink" title="总结一下"></a>总结一下</h1><table><thead><tr><th>NAT</th><th>全锥形</th><th>限制型锥形</th><th>端口限制型锥形</th><th>对称型</th></tr></thead><tbody><tr><td>全锥形</td><td>Direct</td><td>Direct</td><td>Direct</td><td>Direct</td></tr><tr><td>限制型锥形</td><td>Direct</td><td>Hole Punch</td><td>Hole Punch</td><td>Hole Punch</td></tr><tr><td>端口限制型锥形</td><td>Direct</td><td>Hole Punch</td><td>Hole Punch</td><td>Hole Punch</td></tr><tr><td>对称型</td><td>Direct</td><td>Hole Punch</td><td>Hole Punch</td><td><span style="color:blue">Relay</span></td></tr></tbody></table><h1 id="扩展：讨论一下-TCP-打洞的可行性"><a href="#扩展：讨论一下-TCP-打洞的可行性" class="headerlink" title="扩展：讨论一下 TCP 打洞的可行性"></a>扩展：讨论一下 TCP 打洞的可行性</h1><p>TCP 也能实现 NAT 打洞，只不过相比 UDP 会更复杂一点。原因是：</p><ul><li>一个UDP套接字由一个二元组来标识，该二元组包含一个目的地址和一个目的端口号；而一个TCP套接字是由一个四元组来标识，包括源IP地址、源端口号、目的IP地址、目的端口号</li><li>TCP  套接字仅允许建立 1 对 1 的响应，即应用程序将一个套接字绑定到本地的端口后，试图将第二个套接字绑定到该端口的操作都会失败</li></ul><p>基本打洞策略如下：</p><ol><li>A 和 B 分别位于不同的 NAT 下面</li><li>A 启动 tcp client，bind 一个  local_port  PA1’，执行 connect 连接公网 tcp server，server 获取 A 的映射公网端口 PA</li><li>B 同上，B 启动 tcp client，bind 一个  local_port  PB1’，执行 connect 连接公网 tcp server，server 获取 B 的映射公网端口 PB</li><li>A  和 B 保持和 server 的连接，不断开，避免各自的 NAT  上的映射规则过期 {PA1’ -&gt; PA} 和 {PB1’ -&gt; PB1}</li><li>A 和 B 通过 server 互相获取对方的公网 Port PA1 和 PB1，准备开始打洞</li><li>A 新启动一个 tcp 套接字，使用 SO_REUSEADDR/SO_REUSEPORT 绑定到之前与 server 连接的本地端口，也就是 PA1’，并且调用 listen 处于等待监听状态</li><li>B  同上，bind PB1’ 并且调用 listen 处于监听状态</li><li>A 再<strong>新建一个套接字 bind 到之前的端口</strong>，调用 connect 发起向 PB1 的连接，也就是 A 往 PB1 发送 syn 包，也就是为 B 打洞，NAT-B 会丢弃这个包</li><li>同时 B也<strong>再新建一个套接字 bind 到之前的端口</strong>， 也调用 connect 发起向 PA1 的连接，也就是 B 往 PA1 发送 syn 包，也就是为 A 打洞</li><li>假设 A 发送完 syn 之后，B 的 syn 包达到了 NAT-A，NAT-A 能通过，这个时候有的 linux 系统上 A 会认为自己的异步 connect 调用成功，同时利用相同的 seq 发送 SYN+ACK 包 到 PB1，NAT-B 也能顺利通过，再返回 ACK 包，连接建立成功；有的 linux 系统会走正常的 accept 操作，也能顺利建立连接</li></ol><h1 id="扩展：讨论一下打洞的有效时间"><a href="#扩展：讨论一下打洞的有效时间" class="headerlink" title="扩展：讨论一下打洞的有效时间"></a>扩展：讨论一下打洞的有效时间</h1><p>在 NAT  上的映射规则有失效时间，如果要保持洞口的有效性，需要保持打洞双方的心跳。比如在手机上，这个洞口可能会在 1 min 后失效</p><h1 id="扩展：讨论一下多层-NAT-的打洞"><a href="#扩展：讨论一下多层-NAT-的打洞" class="headerlink" title="扩展：讨论一下多层 NAT 的打洞"></a>扩展：讨论一下多层 NAT 的打洞</h1><p>我感觉其实单层 NAT 应该是类似的。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#105;&#x6e;&#x75;&#120;&#x63;&#x6f;&#100;&#101;&#x32;&#x6e;&#105;&#x6b;&#x69;&#x40;&#x67;&#109;&#x61;&#105;&#108;&#46;&#99;&#x6f;&#109;">&#x6c;&#105;&#x6e;&#x75;&#120;&#x63;&#x6f;&#100;&#101;&#x32;&#x6e;&#105;&#x6b;&#x69;&#x40;&#x67;&#109;&#x61;&#105;&#108;&#46;&#99;&#x6f;&#109;</a></p></li></ul><p>2021-03-28 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;何为打洞？-what&quot;&gt;&lt;a href=&quot;#何为打洞？-what&quot; class=&quot;headerlink&quot; title=&quot;何为打洞？(what)&quot;&gt;&lt;/a&gt;何为打洞？(what)&lt;/h1&gt;&lt;h2 id=&quot;英文翻译&quot;&gt;&lt;a href=&quot;#英文翻译&quot; class=&quot;headerlink&quot; title=&quot;英文翻译&quot;&gt;&lt;/a&gt;英文翻译&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;NAT traversal : NAT 穿越&lt;/li&gt;
&lt;li&gt;NAT Hole Punching : NAT 打孔&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;定义&quot;&gt;&lt;a href=&quot;#定义&quot; class=&quot;headerlink&quot; title=&quot;定义&quot;&gt;&lt;/a&gt;定义&lt;/h2&gt;&lt;p&gt;(UDP) 打洞技术是通过&lt;strong&gt;中间公网服务器&lt;/strong&gt;的协助在通信双方的 &lt;strong&gt;NAT 网关&lt;/strong&gt;上建立相关的&lt;strong&gt;映射表项&lt;/strong&gt;，使得双方发送的报文能直接&lt;strong&gt;穿透&lt;/strong&gt;对方的 NAT 网关（防火墙），实现 P2P 直连。&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;洞：所谓的洞就是映射规则，外部能够主动与之通信的规则&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h1 id=&quot;为何要打洞？-why&quot;&gt;&lt;a href=&quot;#为何要打洞？-why&quot; class=&quot;headerlink&quot; title=&quot;为何要打洞？(why)&quot;&gt;&lt;/a&gt;为何要打洞？(why)&lt;/h1&gt;&lt;h2 id=&quot;直接连不行吗？&quot;&gt;&lt;a href=&quot;#直接连不行吗？&quot; class=&quot;headerlink&quot; title=&quot;直接连不行吗？&quot;&gt;&lt;/a&gt;直接连不行吗？&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;NAT&lt;/strong&gt; 技术的存在，一方面减缓了 IPV4 的需求，使得私网 IP 地址通过映射成公网 IP 地址的方式与外界通信&lt;/li&gt;
&lt;li&gt;但另外一方面， &lt;strong&gt;NAT&lt;/strong&gt; 也对安全性做了限制（防火墙），&lt;strong&gt;外界不能主动与私网 IP 进行通信&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/p2p_nat_traversal/image-20210324201346047.png&quot; alt=&quot;image-20210324201346047&quot;&gt;&lt;/p&gt;
&lt;h2 id=&quot;打洞有什么好处？&quot;&gt;&lt;a href=&quot;#打洞有什么好处？&quot; class=&quot;headerlink&quot; title=&quot;打洞有什么好处？&quot;&gt;&lt;/a&gt;打洞有什么好处？&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;节省流量&lt;/li&gt;
&lt;li&gt;降低中心服务器压力&lt;/li&gt;
&lt;li&gt;下载速度快（比如迅雷、直播等）&lt;/li&gt;
&lt;li&gt;安全性和私密性&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
    
      <category term="linux" scheme="https://rebootcat.com/categories/linux/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="tcp" scheme="https://rebootcat.com/tags/tcp/"/>
    
      <category term="p2p" scheme="https://rebootcat.com/tags/p2p/"/>
    
      <category term="nat" scheme="https://rebootcat.com/tags/nat/"/>
    
      <category term="stun" scheme="https://rebootcat.com/tags/stun/"/>
    
      <category term="udp" scheme="https://rebootcat.com/tags/udp/"/>
    
      <category term="upnp" scheme="https://rebootcat.com/tags/upnp/"/>
    
  </entry>
  
  <entry>
    <title>深入浅出paxos</title>
    <link href="https://rebootcat.com/2020/12/05/paxos/"/>
    <id>https://rebootcat.com/2020/12/05/paxos/</id>
    <published>2020-12-05T03:23:58.000Z</published>
    <updated>2020-12-13T07:14:15.720Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>这是前段时间在公司内部关于 paxos 做的一次技术分享，主要围绕 basic-paxos/multi-paxos 协议进行，并会对 raft 协议进行一些对比，简单提及了一下 pbft。</p><p>取名“深入浅出 paxos”，意思是<strong>从分布式模型的简化和抽象系统，讲到分布式数据一致性的核心问题，再引出 paxos 协议的核心，再从纯理论的 basic-paxos 到落地工程实践的 multi-paxos，最后对比 raft、pbft 协议，从简入深，再从深到核心，再到工程实践</strong>。</p><blockquote><p>由于是一次技术分享，所以和我之前的技术博文不太一样，有些东西并没有完全写到博客里，包括一些现场讨论等，所以可能读完之后对 paxos 理解效果会差一点。</p></blockquote><h1 id="名词介绍"><a href="#名词介绍" class="headerlink" title="名词介绍"></a>名词介绍</h1><ul><li>paxos: 应该是分布式领域较早出现的数据一致性协议（本文研究的正是此协议）</li><li>basic paxos: 通常说的 paxos 就是指 basic paxos，或者称为 classical paxos</li><li>multi-paxos: paxos 的改进，迈出了工程实践的步伐（性能改善，工程落地）</li><li>epaxos/fast-paxos: 其他一些 paxos 的改进，特别是 epaxos 最近几年得到较多的讨论和重视</li><li>raft: 从 paxos 而来，类似于 multi-paxos，但是更为简单，容易理解，更为简单</li><li>quorum: 英文翻译：法定人数，可以理解为多数派，大多数，超过半数的一个集合，更为精确的定义是 ”任意两个 quorum 必须有交集)</li><li>state machine replication model: 复制状态机模型</li><li>Crash Fault Tolerance： 故障容错（节点离线，网络延迟等）</li><li>Byzantine Fault Tolerance： 拜占庭容错（节点离线，网络延迟，节点作恶）</li><li>pbft: 实用拜占庭容错算法</li><li>hotstuff: 也是一种拜占庭容错共识算法</li><li>libraBFT: 基于 hotstuff</li></ul><p>先认识名词，从整体上有一些概念，战略上藐视。</p><h1 id="预先准备"><a href="#预先准备" class="headerlink" title="预先准备"></a>预先准备</h1><p>本文重点分析 paxos (basic paxos) 算法。顺带会提及 multi-paxos 以及 raft 算法。</p><p>paxos 很难理解？争取听完本次分享，大家能彻底理解 paxos! </p><p>先忘记区块链，忘记 pbft，忘记 hotstuff.</p><h1 id="单机？分布式？"><a href="#单机？分布式？" class="headerlink" title="单机？分布式？"></a>单机？分布式？</h1><p>为什么要有分布式系统？ 单机容易故障，无法保证服务高可用。</p><p>于是出现多副本模型，但多副本模型就存在两个问题：</p><ul><li>如何确保复制是成功的？（高可用）</li><li>如何确保值是唯一的？（一致性）</li></ul><a id="more"></a><h1 id="复制是否成功"><a href="#复制是否成功" class="headerlink" title="复制是否成功"></a>复制是否成功</h1><p>我们首先把整个模型抽象一下，到最简单的模型。为此，我们定义两个操作：</p><ul><li>SET X</li><li>GET X</li></ul><p><strong>在这里先不考虑并发，不考虑正确性，不考虑其他操作，也不考虑多个值</strong>。</p><p>只有这两个操作，而且只操作数据 X，X 初始值为 null.</p><p>根据上面的抽象，我们定义复制成功的要求就是：</p><p><strong>如果执行了 SET X，那么 GET X 一定能取到值</strong></p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/FBCBAA9D-DAC4-40F5-B6E2-B2584B368E98.png" alt="853d0560b1902e4cdfdb262ea3bb36cd"></p><h1 id="基础的复制策略"><a href="#基础的复制策略" class="headerlink" title="基础的复制策略"></a>基础的复制策略</h1><ul><li>主从异步复制</li><li>主从同步复制</li><li>主从半同步复制</li><li>多数派写（读）</li></ul><h1 id="主从异步复制"><a href="#主从异步复制" class="headerlink" title="主从异步复制"></a>主从异步复制</h1><p>不满足复制成功的要求</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/3B72E5D3-E91F-41CF-82F5-37FE3EBCFA5E.png" alt="6b12d103f46233405e073bea5c809287"></p><h1 id="主从同步复制"><a href="#主从同步复制" class="headerlink" title="主从同步复制"></a>主从同步复制</h1><p>失联节点不确定，写入 slave 个数也不确定，不满足复制成功的条件</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/9F4AF7BD-E0FF-46AB-9190-5BAA4C650328.png" alt="451a6fac1f1fd7251e0c8909319fc79b"></p><h1 id="主从半同步复制"><a href="#主从半同步复制" class="headerlink" title="主从半同步复制"></a>主从半同步复制</h1><p>写入 slave 个数不确定，最少 1 个，最多 n 个，有概率性存在无法获取 X， 不满足复制成功的要求</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/22DC0B51-C03B-4675-B22E-8E1BBE29BDDD.png" alt="2ed4ace628eff82e0d0a8471d7d41ce1"></p><h1 id="多数派写"><a href="#多数派写" class="headerlink" title="多数派写"></a>多数派写</h1><p>满足复制成功的要求，<strong>先多数派写，再多数派读</strong>。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/6E49F7D6-E957-4BEC-BCB4-51F1C017D12F.png" alt="2490ea5e4182cdce6fd949edf96b5626"></p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/C043C142-6C05-43D2-88C0-4ECF508575A7.png" alt="eb08526a5b83ea77e6dfe4424a657e10"></p><p>优点：</p><ul><li>高可靠性</li><li>高可用性</li><li>数据完整性有保证</li></ul><p>到此，我们解决了成功复制的问题。</p><h1 id="值是否唯一"><a href="#值是否唯一" class="headerlink" title="值是否唯一"></a>值是否唯一</h1><p>首先，策略我们已经确定，要保证复制成功，需要先多数派写，再多数派读。</p><p>扩展一下刚才的抽象，考虑有多个客户端并发的来写（SET X)，那么读到的值将不唯一。</p><p><strong>这里我们依然不考虑正确性的问题，我们只考虑唯一性的问题</strong>。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/C91DD687-DFDE-45E7-A4BF-0242565943B5.png" alt="3939ce9a3cd096a884a38bb9daf6d4bb"></p><h1 id="再重复一遍系统模型的抽象"><a href="#再重复一遍系统模型的抽象" class="headerlink" title="再重复一遍系统模型的抽象"></a>再重复一遍系统模型的抽象</h1><ul><li>只有两个操作 SET 和 GET</li><li>只操作一个数据 X，不考虑其他数据</li><li>不考虑 X 的正确性，只考虑唯一性</li><li>允许并发 SET X，但最终目标是 X 值唯一</li></ul><p>抽象是很重要的，能够帮我们简化模型，思考本质的问题。</p><p>如何解决并发的问题，或者说多个 client SET X 的问题？</p><h1 id="如何解决并发-SET-X"><a href="#如何解决并发-SET-X" class="headerlink" title="如何解决并发 SET X"></a>如何解决并发 SET X</h1><p>如果 SET X 前运行一次多数派读，知道 X 的值可能已经有别人写入了，那么就不写，如果还没有人写入，那么就写入。</p><p>道理很简单，但问题是这涉及到两次 rtt，所以其他 client 就有可能插入进来。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/9331A6AF-AA1D-4E27-83CB-D15A9675BAC6.png" alt="f5dd74498d599979d90abb3b4402433c"></p><p>怎么解决呢?</p><h1 id="拒绝其他写前读取"><a href="#拒绝其他写前读取" class="headerlink" title="拒绝其他写前读取"></a>拒绝其他写前读取</h1><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/FBB45C5A-8D87-400C-BFBE-982B3B7399D3.png" alt="c429070b1d73815c294c8a6ba9755919"></p><p>这里依然有多数派读的要求，Client A 必须收到 a quorum (&gt;n/2) 个响应才认为本次写前读取 true，后续可以放心大胆的写入；否则判定为 false,认为已经有其他节点优先读取了，放弃后续的写。</p><p>有了这个保证，那么 Client A 在第二个 rtt 就可以放心的写了。（参考上面的图）</p><p>到这里其实就已经讲到  paxos 算法的核心了。(终于讲到 paxos 了）。</p><p>也许，我们还可以问两个问题: </p><ul><li>如果第一个 rtt 失败了呢？（思考）</li><li>如果第二个 rtt 失败了呢？（思考）</li></ul><h1 id="初识-paxos"><a href="#初识-paxos" class="headerlink" title="初识 paxos"></a>初识 paxos</h1><p>图灵奖大牛 Leslie Lamport （莱斯利·兰伯特）</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/10C0A8F6-29EA-4A83-B4D7-D5421F291CAB.png" alt="6719640cd8cb047cddfda040cf8d2580"></p><p>来自知乎：</p><p>Lamport在分布式系统理论方面有非常多的成就，比如Lamport时钟、拜占庭将军问题、Paxos算法等等。除了计算机领域之外，其他领域的无数科研工作者也要成天和Lamport开发的一套软件打交道：著名的LaTeX。这是目前科研行业应用最广泛的论文排版系统，名字中的”La”就是指Lamport。</p><p>实际上Paxos在1990年就被提出了。当时Lamport写了一篇名为《The Part-time Parliament》的论文，在这篇文章中作者讲了一个虚构的故事。这个故事发生在希腊的神话中的一个名叫Paxos的岛屿（也就是算法名称的来由），作者将分布式一致性的问题比喻为岛上的立法机构如何对一项决议达成一致的问题。Lamport本来是觉得用故事加以描述更易理解；但其结果完全相反。这篇文章当时的评审几乎没有人看懂，只有一位名叫Butler Lampson的计算机科学家读懂了故事，并意识到这是一篇解决分布式一致性问题的论文。当然这篇论文就被埋没了多年，原文1998年才得以发表，后来Lamport也又重新“正儿八经”地写了一篇《Paxos Made Simple》。</p><p><a href="https://lamport.azurewebsites.net/pubs/paxos-simple.pdf" target="_blank" rel="noopener">https://lamport.azurewebsites.net/pubs/paxos-simple.pdf</a></p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/7CC70C2A-5766-4DEE-9572-3129F982A663.png" alt="49f7d2e95c37f59d61cbc47e286cf531"></p><h1 id="liveness-and-safety"><a href="#liveness-and-safety" class="headerlink" title="liveness and safety"></a>liveness and safety</h1><p>from wiki:</p><p>In order to guarantee safety (also called “consistency”), Paxos defines three properties and ensures the first two are always held, regardless of the pattern of failures:</p><p>Validity (or non-triviality)<br>Only proposed values can be chosen and learned.[15]<br>Agreement (or consistency, or safety)<br>No two distinct learners can learn different values (or there can’t be more than one decided value)[15][16]<br>Termination (or liveness)<br>If value C has been proposed, then eventually learner L will learn some value (if sufficient processors remain non-faulty).[16]<br>Note that Paxos is not guaranteed to terminate, and thus does not have the liveness property. This is supported by the Fischer Lynch Paterson impossibility result (FLP)[6] which states that a consistency protocol can only have two of safety, liveness, and fault tolerance. <strong>As Paxos’s point is to ensure fault tolerance and it guarantees safety, it cannot also guarantee liveness</strong>.</p><h1 id="（Basic-Classical-Paxos"><a href="#（Basic-Classical-Paxos" class="headerlink" title="（Basic/Classical) Paxos"></a>（Basic/Classical) Paxos</h1><p>开始之前，记住刚才讲到的系统抽象（实际上 paxos 也仅仅只是解决了这个问题，想清楚这个系统抽象，理解起来会容易得多，避免走弯路）</p><ul><li>只有两个操作 SET 和 GET</li><li>只操作一个数据 X，不考虑其他数据</li><li>不考虑 X 的正确性，只考虑唯一性</li><li>允许并发 SET X，但最终目标是 X 值唯一</li></ul><p>paxos 定义了 3 种角色：</p><ul><li>Proposer: 发起提案 value（一个提案可以理解为一个操作，或者一个值）</li><li>Acceptor: 接受一个 value 或者驳回</li><li>Learners: 当某个提案被大多数 acceptor 接受后，则认为 value 被选定，然后复制 value</li></ul><blockquote><p>Leaners 暂时不重要，可以不用关心</p></blockquote><p>Tips: 这里有一些概念可能比较难以理解，由于中英文的差异，阅读论文的时候可能会比较苦恼</p><p>举个例子：<br>如果你阅读 paxos 的一些文档，可能会经常看到 ”value be choosen” 等一些概念，这里的 choosen 就可以理解为值已经确定，唯一。</p><p>主要过程分为两个阶段（如同上述讨论的两个 rtt)</p><h1 id="paxos-phase1"><a href="#paxos-phase1" class="headerlink" title="paxos phase1"></a>paxos phase1</h1><h2 id="phase-1a-Prepare"><a href="#phase-1a-Prepare" class="headerlink" title="phase 1a: Prepare"></a>phase 1a: Prepare</h2><p>先来看几个概念：</p><ul><li>proposal number n: 全局唯一且递增，用来给一个提案编号</li><li>proposal v: 一个提案，通常用 value(v) 表示</li></ul><p>proposer 发起 “prepare” 请求，请求里携带一个 proposal number n，这个 n 要比它之前的 prepare 消息使用的值大。然后它发送（广播）这个 “prepare” 消息给 a Quorum of Acceptors（大多数的 acceptors)。</p><p>值得注意的是，这个 prepare 消息通常不包含具体的提案 v。（正如上面的写前读取，不需要携带将要写的值 x)</p><ul><li>proposer 需要持久化它看到的最大的 proposal number</li></ul><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/F58DCFA7-C4E1-4F6C-B533-3BFCC6CDC516.png" alt="69d4c2b689907d300d26e3df7f921a7e"></p><h2 id="phase-1b-Promise"><a href="#phase-1b-Promise" class="headerlink" title="phase 1b: Promise"></a>phase 1b: Promise</h2><p>acceptor 收到从某个 proposer 发过来的 prepare 消息，拿到消息里的 proposal number n.</p><p>先看几个概念：</p><ul><li><strong>minProposal: 之前已经接收到的 prepare 消息里的最大 proposal number</strong></li><li>acceptedProposal: 之前已经接受过的 proposal number</li><li>acceptedValue: 之前已经接受过的 proposal</li></ul><ol><li>如果 n &gt; minProposal</li></ol><ul><li>那么 acceptor 必须返回一个响应，同时做出两个 promise<ul><li>promise1: <strong>它将拒绝后续所有 proposal number &lt;= n 的其他 prepare 请求</strong></li><li>promise2: <strong>它将拒绝后续所有 proposal number &lt; n 的其他 accept 请求（第二阶段会讲到）</strong></li></ul></li><li>如果 acceptor 之前已经接受过一个 proposal，那么返回的响应里还会携带 (acceptedProposal, acceptedValue)，否则返回(nil, nil)</li><li>同时 acceptor 记录 minProposal = n</li></ul><ol start="2"><li>如果 n &lt;= minProposal</li></ol><ul><li>拒绝这个 Prepare 消息（<strong>然后这个 proposer 会以更大的 proposal number 发起 prepare</strong>)</li></ul><p>这里要注意:</p><ul><li>acceptor 需要持久化 (minProposal, acceptedProposal, acceptedValue)，以防止崩溃或者重启</li></ul><p>第一阶段对比之前的讨论，就相当于之前的写前读取，用来判断谁准备写入。但是区别是： 上面的例子中只接受第一个 client 的写前读取，后续其他的 client 的写前读取全部拒绝；而这里的 acceptor 其实是允许接收多个 prepare(写前读取) 的，<strong>想想看为什么</strong>？ 相同的地方是都需要对写前读取做记录 (minProposal = n)</p><h1 id="paxos-phase2"><a href="#paxos-phase2" class="headerlink" title="paxos phase2"></a>paxos phase2</h1><h2 id="phase-2a-Accept"><a href="#phase-2a-Accept" class="headerlink" title="phase 2a: Accept"></a>phase 2a: Accept</h2><p>如果 proposer 收到了 a Quorum of Acceptors返回的 promise，那么说明他可以开始提案了。</p><p>如果 promise 里含有 (acceptedProposal, acceptedValue)，那么就<strong>放弃自己原本的提案</strong>，从返回的这些 promise 里挑出 acceptedProposal 最大的 acceptedValue 作为本次的提案 value。</p><p>（很重要，我自己初看的时候对这里放弃自己的提案很是想不通，还是那句话，把系统抽象到最简模型，提案的具体值不重要，重要的是达成一致）</p><p>这里的理解： accepted 并不表示某个提案 v 被确定了（be choosen)，之所以放弃自己的提案，其实是相当于继续了未完成的 paxos 过程。后面会讲到。</p><p>如果返回的 promise 里都没有 acceptedValue 值，那么才使用自己的提案 value （说明这是第一次提案）</p><p>然后 proposer 发起 “Accept” 请求 (n, v)，广播给大多数 acceptor。</p><h2 id="phase-2b-Accepted"><a href="#phase-2b-Accepted" class="headerlink" title="phase 2b: Accepted"></a>phase 2b: Accepted</h2><p>acceptor 收到 “Accept” 消息后(n, v)，取出里面的 proposal number n.</p><ol><li>如果 n &gt;= minProposal</li></ol><ul><li>acceptor 更新 acceptedProposal = minProposal = n</li><li>acceptor 更新 acceptedValue = v</li><li>acceptor 响应 “Accepted” 消息，消息里携带 minProposal</li></ul><ol start="2"><li>如果 n &lt; minProposal:</li></ol><ul><li>拒绝这个 “Accept” 请求，同样也会回复一个响应，消息里携带 minProposal</li></ul><p>注意，acceptor 是可以接受多次 acceptedValue 的，只要满足上面的条件。</p><p>当 proposer 收到了大多数的 acceptor 返回来的 “Accepted” 消息，则认为这个提案已经确定(be choosen)。</p><p>如果返回的消息有任何 minProposal &gt;n，则重新以更大的 proposal number n执行 paxos.</p><p>接下来，就是 <strong>Leaner 发挥作用的时候了，Leaner 就会复制这个提案</strong>。实际的做法可能有多种，比如 Leaner 和  proposer 集成到一个角色，再通知其他的 Leaner 这个被 choosen 的提案v; 或者 Leaner 自己执行一遍 paxo是就能知道被 choosen 的提案。</p><p>讨论一下上面关于 n 与 minProposal 的大小关系：</p><ul><li>n == minProposal</li><li>n &lt; minProposal</li><li>n &gt; minProposal</li></ul><h3 id="phase-2b-Accepted-n-minProposal"><a href="#phase-2b-Accepted-n-minProposal" class="headerlink" title="phase 2b: Accepted (n == minProposal)"></a>phase 2b: Accepted (n == minProposal)</h3><ul><li><strong>n == minProposal: 这个很好理解，假设 paxos 系统只有一个 proposer，那么到了 accept 这里，必然 minProposal 就是 prepare 消息里的 n，此次 accept 消息自然也是 n</strong></li></ul><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/9626B90A-2D5B-4B6E-9875-6849C62F516E.png" alt="6877fa072749429a24f8204472dedccb"></p><h3 id="phase-2b-Accepted-n-lt-minProposal"><a href="#phase-2b-Accepted-n-lt-minProposal" class="headerlink" title="phase 2b: Accepted (n &lt; minProposal)"></a>phase 2b: Accepted (n &lt; minProposal)</h3><ul><li><strong>n &lt; minProposal: 也好理解，如果 proposerA 成功执行完 prepare 消息后，并确定自己可以开始写入之前，有 ProposerB 发起了 n 更大的 Prepare 消息，那么 acceptor 处的 minProposal 则会更新到这个值，当 ProposerA 的 Accept 过来时， n &lt; minProposal</strong></li></ul><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/C1F2F726-5F4E-48EC-9073-816CE97A6F56.png" alt="40e0ff85898782cd037c7898b9da1710"></p><h3 id="phase-2b-Accepted-n-gt-minProposal"><a href="#phase-2b-Accepted-n-gt-minProposal" class="headerlink" title="phase 2b: Accepted (n &gt; minProposal)"></a>phase 2b: Accepted (n &gt; minProposal)</h3><ul><li><strong>n &gt; minProposal: 假设 proposerA 已经成功发起 prepare 消息了，但是在发起 Accept 请求这段空隙时间，有 proposerB 又以更大的 proposal number n 成功发起了 prepare 消息，那么大多数的 Acceptor 的 minProposal 必然会更新到这个更大的 n，然后又抢在 proposerA 前发起了 Accept 消息，假设这个 AcceptB 消息被上一步没有收到 prepareB 的 acceptor 收到了，那么这些 acceptor 的 minProposal 依然记录的是 proposerA 的n，自然本次的值更大，n &gt; minProposal</strong></li></ul><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/BFCB7670-2ED9-4487-98F3-242A0FD913F8.png" alt="33007908fdef0ea6a4911fd9f5cbf2fd"></p><h1 id="paxos-完整过程"><a href="#paxos-完整过程" class="headerlink" title="paxos 完整过程"></a>paxos 完整过程</h1><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/A1E59E88-A7E4-40B3-9D07-5810D4952F2F.png" alt="2b1ff557b681a0b9287ac941aebd4c4a"></p><p>或者类似 pbft 的流程图：<br><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/9E5808BD-DCB1-432E-86FB-D709783A4D85.png" alt="3535daaaeb09cc36bf08ecd4a50119a1"></p><p>到这里，其实已经把 paxos 的核心算法讲完了。</p><p>来讨论一下涉及到的几个过程中可能出现的场景。</p><h1 id="Basic-Paxos-without-failures"><a href="#Basic-Paxos-without-failures" class="headerlink" title="Basic Paxos without failures"></a>Basic Paxos without failures</h1><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/C81FB194-C79B-4D33-8499-ADBFD2949B43.png" alt="64d844ec4d0616e4122b44d510bcff18"></p><h1 id="Basic-Paxos-when-an-Acceptor-fails"><a href="#Basic-Paxos-when-an-Acceptor-fails" class="headerlink" title="Basic Paxos when an Acceptor fails"></a>Basic Paxos when an Acceptor fails</h1><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/9391A510-EE8A-47C9-9A3D-CE4BA84B886C.png" alt="dfb6e85ed9956a641bb037da4147d9d0"></p><h1 id="Basic-Paxos-when-a-Proposer-fails"><a href="#Basic-Paxos-when-a-Proposer-fails" class="headerlink" title="Basic Paxos when a Proposer fails"></a>Basic Paxos when a Proposer fails</h1><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/B3A69F52-B128-4E65-95BF-13A63274B913.png" alt="b13d31bcc83ff94d7ba3274dfe2c3c8f"></p><h1 id="Basic-Paxos-when-multiple-Proposers-conflict-livelock-活锁）"><a href="#Basic-Paxos-when-multiple-Proposers-conflict-livelock-活锁）" class="headerlink" title="Basic Paxos when multiple Proposers conflict (livelock 活锁）"></a>Basic Paxos when multiple Proposers conflict (livelock 活锁）</h1><p>对比 deadlock，叫法很形象</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/9854B8FA-710E-46C7-9C2B-37442F52235B.png" alt="0c0ca0a9c0e20cdbd6e9f4229d91bcfb"></p><h1 id="是否还有很多疑问？"><a href="#是否还有很多疑问？" class="headerlink" title="是否还有很多疑问？"></a>是否还有很多疑问？</h1><p>到了这里，你是否还有很多疑问？</p><p>我自己看到这的时候，我的疑问就是<strong>假设提案v 已经被确定了，即 be choosen, acceptor 保存的 acceptedValue 和 acceptedProposal 难道要一直保存吗？如果我打算发起第二个提案 v’, 按照上面的算法，promise 还是会把 (acceptedValue, acceptedProposal) 返回来，我始终无法发起第二个提案 v’</strong>.</p><p>更直白的话是，实际的系统肯定是一些连续的不同的提案，比如add, sub, jump, mov, cmp 等等，或者举一个更容易理解的数据库的例子，我们是持续的往数据库里写入数据的：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">x = <span class="number">1</span> ; y = <span class="number">2</span>; z= <span class="number">100</span>; x + <span class="number">100</span>; z - <span class="number">10</span> ...</span><br></pre></td></tr></table></figure><p>当我们成功发起 x=1 这个提案之后，按照 paxos 的流程发起第二个提案 y=2，还是会得到 x=1 这个提案的值。</p><p>basic/classical paxos 仅仅只是解决了一个提案（一次操作）的数据一致性问题，这也是纯理论的 basic/classical paxos 解决的核心问题。</p><p>你是否跟我一样？如果你能想到这里，说明你已经理解了上面的算法！（👏👏）</p><p>那么应用到实际中，怎么解决呢？</p><h1 id="从理论到工程实践"><a href="#从理论到工程实践" class="headerlink" title="从理论到工程实践"></a>从理论到工程实践</h1><p>根据上面的讨论我们可以知道，执行一个 paxos 流程可以解决一个提案的一致性问题，如果想要发起第二个提案，就势必要处理 (acceptedValue, acceptedProposal) 的问题。</p><p>在 Acceptor 上记录了 3 个值 (minProposal, acceptedValue, acceptedProposal)，当第一个提案（比如 x=1) 成功被 choosen 之后，Acceptor 上必须要有一个机制去清除掉这 3 个值，就像最初什么也没发生过时的状态，此时发起第二个提案（y=2） 就和发起第一个提案(x=1) 一样了。</p><p>那么问题就来了，Acceptor 什么时候清除 (minProposal, acceptedValue, acceptedProposal) 呢？它必须知道提案已经被 choosen 了，并且要记录下这个已经被 choosen 的提案 (x=1);否则不能清除。</p><p>所以当一个提案(x=1) 被 choosen 之后，<strong>发起提案的 proposer 需要广播这个 choosen 结果给 Acceptor，直到大多数 Acceptor 返回成功</strong>。（先提一下：这里这个过程是不是和 raft 里 leader 日志复制的第二个过程类似，即 leader 本地 commit 之后，广播给所有 follower，告诉他们这个 command 已经处于 commited 状态了，然后 follower commited command）</p><p>Ok, 到这里 Acceptor 已经知道提案被 choosen(commited) 了，实际做法是记录下这个 choosen 的提案(x=1), 然后它可以放心大胆的清除(minProposal, acceptedValue, acceptedProposal) 这 3 个值了。那么当 proposer 发起第二个提案 y=1后，Acceptor 就能正常处理了。</p><p>但， Acceptor 怎么区分这两个不同的提案(x=1 和 y=1） 呢？（靠 proposal number n ?)</p><p>我们还得给每一个不同的提案一个编号，或者说 index，即每一个不同的提案具有唯一的序号 index。所以一个 proposer 发起一个提案(prepare 消息）时需要包含 (n,v, index)，那么 Acceptor 这里呢，记录的 3 个值 (minProposal, acceptedValue, acceptedProposal) 和每一个 index 唯一绑定，或者说每一个不同的 index 有自己单独的 3 个值, 即（minProposal<sub>index</sub>, acceptedValue<sub>index</sub>, acceptedProposal<sub>index</sub>)</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/38E08D1F-5A7C-47CC-BCD1-42BBA1CCEEE9.png" alt="75fdb0f04337d5e5383a35e3b877d7ef"></p><p>是不是可以并发了呢？</p><h1 id="multi-paxos"><a href="#multi-paxos" class="headerlink" title="multi-paxos"></a>multi-paxos</h1><p>经过上面的讨论，我们看到如果给每一个不同的提案 <code>x = 1 ; y = 2; z= 100; x + 100; z - 10 ...</code> 做一个编号 index，我们就能够独立的执行 paxos 协议，实现我们实际工程中的连续写操作。</p><p><strong>由纯理论到工程实践了！！！！</strong></p><p>我们把上面每个运行独立 paxos 协议的实体称为一个 <strong>paxos instance</strong>，每个 paxos instance 独立互不影响。实际的工程中我们通常也会把 proposer/acceptor/leaner 3 个角色放到一个节点上，即一个 paxos 节点具有多重身份，每个身份可以作为一个线程运行，方便数据共享(比如 choosen value)。</p><p>性能问题的解决：</p><ul><li>两阶段的 paxos 性能较差，如何减少 rpc 请求数？</li><li>livelock(活锁) 问题的解决？</li></ul><p>据此 multi-paxos 提出一个 Leader 机制，避免 proposer 的冲突，这样一样，所有发起的提案全部由这个 Leader 来发起（至于如何选取 leader，问题不大，有很多做法，就如同 raft 的 leader 选举一样），同时也解决了 livelock 问题。</p><p>那么既然有了 leader，第一阶段的 prepare/promise 还有必要吗？ 第一阶段的 prepare/promise 到底是干嘛的，上面其实已经讲的很清楚了，就是确认哪一个 proposer 准备写，那么既然有了 leader，自然这个过程就可以省略了。</p><p>所以 multi-paxos 就只有 accept/accepted 过程了。</p><p>那么我们再重新思考一下：basic paxos 的两个阶段到底解决了什么问题？</p><ul><li>第一阶段(prepare/promise): 解决了选择唯一 proposer 的问题</li><li>第二阶段(accept/accepted): 解决了提案被大多数 acceptor 接受的问题(accepted)，结果最初只有发起提案的 proposer 知道</li></ul><p>通过 multi-paxos leader 机制，我们简化了第一阶段：</p><ul><li>直接发起 accep/accepted: 最终大多数 acceptor 会接受这个提案，并且 leader 汇总结果后知道了大多数 acceptor 已经接受了这个提案，于是被 choosen</li><li>然后广播这个 choosen 结果给所有 acceptor，acceptor 再更新这个提案为 choosen（前面提到过）</li></ul><p>这两个过程是不是和 raft 如出一辙，在 raft 里叫  log replication？😁😁 <strong>multi-paxos ≈ raft</strong>.</p><h1 id="状态机"><a href="#状态机" class="headerlink" title="状态机"></a>状态机</h1><p>先看一个状态机：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/32455F4B-7C62-427A-9308-3DD25949094D.png" alt="5ff678a971801fb6786fe2ef15a977c6"></p><h1 id="multi-paxos-多提案过程"><a href="#multi-paxos-多提案过程" class="headerlink" title="multi-paxos 多提案过程"></a>multi-paxos 多提案过程</h1><p>client 发起一个新的 command jmp，进入共识模块执行 multi-paxos 算法，会去找到最小的已经被 choosen(commited) 的 index， 找到 index = 3（放弃自己的提案 jmp，执行 index=3 的 paxos)， and so on…</p><p>注意，实际可能同时并发发起 3 ~ 20 index 的提案。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/2AE547E9-BE46-4B9E-8F27-C384BA3ADA27.png" alt="74b81298f3a056969f3d3b2aef808b2c"></p><h1 id="multi-paxos-日志空洞"><a href="#multi-paxos-日志空洞" class="headerlink" title="multi-paxos 日志空洞"></a>multi-paxos 日志空洞</h1><p>multi-paxos 是允许日志空洞的，也就是不连续的，每次 leader 并发发起 index 任意多个的不同提案，每个提案独立进行，所以会有成功和失败。</p><h1 id="multi-paxos-幽灵复现日志问题"><a href="#multi-paxos-幽灵复现日志问题" class="headerlink" title="multi-paxos 幽灵复现日志问题"></a>multi-paxos 幽灵复现日志问题</h1><p>”幽灵复现日志“造成的原因就是”日志空洞“和 Leader 切换。</p><p>举个例子：</p><ul><li>用户 A 发起一笔转账，超时没有等到转账结果</li><li>于是用户 A 再次查询一下是否转账成功，如果再次超时，用户 A 可能重新发起新的转账</li><li>但是最后结果是 A 成功执行了两次转账</li></ul><h2 id="multi-paxos-Leader-切换"><a href="#multi-paxos-Leader-切换" class="headerlink" title="multi-paxos Leader 切换"></a>multi-paxos Leader 切换</h2><p>leader 选举的过程可以简单理解为某个节点 A 发起一轮 basic paxos（提案就是选 A 作为 leader)，最终提案被 choosen，广播给所有 acceptor，于是 leader 产生，并利用 lease 机制保持自己的 leader 身份，避免其他的 proposer 发起竞选 leader.</p><blockquote><p>lease 机制：即租约机制，声明 leader 有效期，在有效期内，不允许发起竞选 leader；超过 lease，随意进入选举。</p></blockquote><p>leader 切换必然伴随日志不一致的问题，即当前 leader 的日志和前任 leader 的日志不一致。就有可能造成 client 查询的时候返回 false。详细就不展开讲了。</p><h1 id="multi-paxos-总结"><a href="#multi-paxos-总结" class="headerlink" title="multi-paxos 总结"></a>multi-paxos 总结</h1><ul><li>选了一个 leader，避免 proposer 竞争(避免livelock)</li><li>同时可以并发发起 index = 1,2,3,4… 的提案，每个提案独立运行 paxos 算法</li><li>每个提案被 accepted 之后记录到本地(例如 mysql binlog)</li><li>每个提案被 choosen 之后更新这个 index 的状态为 choosen（比如设置这个 index 对应的 minProposal<sub>index</sub> = ∞）</li><li>被 choosen(commited) 的提案放到 state machine 里执行</li><li>如果同时并发发起 index = 1…10 的 共 10 个提案，中间有一些提案失败了(accepted but not choosen)，下一次会触发继续执行未完成的提案</li><li>multi-paxos 允许提案空洞(不连续） (相反 raft 就必须是连续的，不允许日志空洞）</li></ul><p>上面每个 index 构成的本地提案记录，类似于一个列表，raft 里就 log entry，index 称为 logid.</p><h1 id="raft"><a href="#raft" class="headerlink" title="raft"></a>raft</h1><p>先看一个动画：</p><p><a href="http://thesecretlivesofdata.com/raft/" target="_blank" rel="noopener">http://thesecretlivesofdata.com/raft/</a></p><p>raft 协议和 multi-paxos 很像。</p><p>先不讨论 leader 选举的问题，应该很简单。就讨论 log replication(日志复制）的问题。raft 过程如下：</p><p>定义了 3 种角色：</p><ul><li>leader: 就是 leader</li><li>follower: 系统中其他节点，接收 leader 消息</li><li>candidate: follower 到 leader 转换的中间状态</li></ul><p>过程如下：</p><ul><li>client 发起一个 command (redirect to leader)</li><li>leader 广播这个日志到其余的 follower，称为 AppendEntries rpc (对比 multi-paxos 的 accept/accepted 过程）</li><li>leader 收到大多数 follower 成功响应后，执行 apply entry， 即 commite log，然后广播给其他的 follower（对比 multi-paxos proposer 广播 choosen 的提案）</li><li>leader 回应 client</li></ul><h1 id="raft-leader-election"><a href="#raft-leader-election" class="headerlink" title="raft leader election"></a>raft leader election</h1><p>很简单，每个 follower 持有一个计数器，比如 [100, 300]ms，在这段时间内只要收不到 leader 的 hearbeat，就认为 leader 挂掉（实际有可能是他自己出了问题），然后由 follower 状态转为 candidate 状态，开始竞选 leader.</p><p>每个节点只能投一票，如果这个 candidate 收到大多数节点的 vote，则成为 leader，更新任期号 term number.</p><h1 id="异常情况：raft-多个节点同时竞选"><a href="#异常情况：raft-多个节点同时竞选" class="headerlink" title="异常情况：raft 多个节点同时竞选"></a>异常情况：raft 多个节点同时竞选</h1><p>没关系，只要达不成 quorum vote，就继续下一轮投票； </p><p>为了避免出现无休止的重复这个过程（类似 livelock)，每次重新开始竞选时，随机延迟一段时间，避免出现两个  candidate 竞选。</p><h1 id="异常情况：脑裂"><a href="#异常情况：脑裂" class="headerlink" title="异常情况：脑裂"></a>异常情况：脑裂</h1><p>脑裂就是网络分裂，形成两个或者多个独立的孤岛网络，假设分裂成 A 和 B 网络。</p><ul><li>A 满足多数派条件</li><li>B 不满足多数派条件</li><li>A 和 B 可能会发生各自网络内部的 leader 重新选举，term 增 1</li></ul><p>对于 B 来说，由于不满足多数派，故日志始终处于 uncommited 状态，所以是安全的；</p><p>对于 A 来说，正常进行 raft 共识；</p><p>一旦网络恢复：</p><ul><li>A: 继续进行 raft</li><li>B: 退化为 follower，日志回滚</li></ul><p>所以也是安全的。</p><h1 id="raft-log-replication"><a href="#raft-log-replication" class="headerlink" title="raft log replication"></a>raft log replication</h1><p>基本和 multi-paxos 一样，但区别是 raft 要求日志是连续的，不允许出现日志空洞。即如果 logid = index 处于 commited 状态，那么 logid &lt;index 的一定都处于 commited 状态。<br><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/F38F9AC0-EC20-4463-A956-1AF5B5EA445A.png" alt="50b7cc551cfd8a792d981418b3e7b0b4"></p><h1 id="paxos-vs-raft"><a href="#paxos-vs-raft" class="headerlink" title="paxos vs raft"></a>paxos vs raft</h1><p>先看一下概念上的一些区别：</p><ul><li>raft leader 基本等同于 multi-paxos leader，但区别是 multi-paxos leader 不是强 leader 性质，实际上两个 leader 也可以（退化成 basic-paxos)</li><li>raft follower 相当于 multi-paxos acceptor</li><li>raft 两阶段的 rpc 分别是 appendEntries 和 applyEntries；分别对应 multi-paxos 中的 accept 和广播 choosen 的消息</li><li>raft 日志 等同于 paxos 提案</li><li>raft log entry 等同于 multi-paxos proposal index.</li></ul><p>再看一下其他方面：</p><ul><li>multi-paxos 允许日志空洞； raft 不允许日志空洞</li><li>leader election: multi-paxos 弱 leader 性质；raft 强 leader，且要求竞选 leader 的 follower 必须有较大的 logindex</li></ul><h1 id="pbft"><a href="#pbft" class="headerlink" title="pbft"></a>pbft</h1><p>祭一张图吧 （分享时间有限，先提一下 pbft，以后再分享。）<br><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/95F9A78D-049F-4D69-ABB4-25D1786D6230.png" alt="545718f98e5638166df86c479bbb14ea"><br><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/paxos/BD5B040D-02BD-4B21-8CE2-65E9BD8CF2C4.png" alt="8ec7b4bee535c677e782df0630d85d13"></p><h1 id="完"><a href="#完" class="headerlink" title="完"></a>完</h1><p>关于 paxos， 欢迎和我一起交流。如果上面有讲的不对的地方，欢迎指正。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#x69;&#110;&#x75;&#x78;&#x63;&#x6f;&#x64;&#101;&#x32;&#x6e;&#x69;&#107;&#x69;&#64;&#103;&#109;&#97;&#x69;&#x6c;&#x2e;&#99;&#x6f;&#109;">&#x6c;&#x69;&#110;&#x75;&#x78;&#x63;&#x6f;&#x64;&#101;&#x32;&#x6e;&#x69;&#107;&#x69;&#64;&#103;&#109;&#97;&#x69;&#x6c;&#x2e;&#99;&#x6f;&#109;</a></p></li></ul><p>2020-12-05 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;这是前段时间在公司内部关于 paxos 做的一次技术分享，主要围绕 basic-paxos/multi-paxos 协议进行，并会对 raft 协议进行一些对比，简单提及了一下 pbft。&lt;/p&gt;
&lt;p&gt;取名“深入浅出 paxos”，意思是&lt;strong&gt;从分布式模型的简化和抽象系统，讲到分布式数据一致性的核心问题，再引出 paxos 协议的核心，再从纯理论的 basic-paxos 到落地工程实践的 multi-paxos，最后对比 raft、pbft 协议，从简入深，再从深到核心，再到工程实践&lt;/strong&gt;。&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;由于是一次技术分享，所以和我之前的技术博文不太一样，有些东西并没有完全写到博客里，包括一些现场讨论等，所以可能读完之后对 paxos 理解效果会差一点。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h1 id=&quot;名词介绍&quot;&gt;&lt;a href=&quot;#名词介绍&quot; class=&quot;headerlink&quot; title=&quot;名词介绍&quot;&gt;&lt;/a&gt;名词介绍&lt;/h1&gt;&lt;ul&gt;
&lt;li&gt;paxos: 应该是分布式领域较早出现的数据一致性协议（本文研究的正是此协议）&lt;/li&gt;
&lt;li&gt;basic paxos: 通常说的 paxos 就是指 basic paxos，或者称为 classical paxos&lt;/li&gt;
&lt;li&gt;multi-paxos: paxos 的改进，迈出了工程实践的步伐（性能改善，工程落地）&lt;/li&gt;
&lt;li&gt;epaxos/fast-paxos: 其他一些 paxos 的改进，特别是 epaxos 最近几年得到较多的讨论和重视&lt;/li&gt;
&lt;li&gt;raft: 从 paxos 而来，类似于 multi-paxos，但是更为简单，容易理解，更为简单&lt;/li&gt;
&lt;li&gt;quorum: 英文翻译：法定人数，可以理解为多数派，大多数，超过半数的一个集合，更为精确的定义是 ”任意两个 quorum 必须有交集)&lt;/li&gt;
&lt;li&gt;state machine replication model: 复制状态机模型&lt;/li&gt;
&lt;li&gt;Crash Fault Tolerance： 故障容错（节点离线，网络延迟等）&lt;/li&gt;
&lt;li&gt;Byzantine Fault Tolerance： 拜占庭容错（节点离线，网络延迟，节点作恶）&lt;/li&gt;
&lt;li&gt;pbft: 实用拜占庭容错算法&lt;/li&gt;
&lt;li&gt;hotstuff: 也是一种拜占庭容错共识算法&lt;/li&gt;
&lt;li&gt;libraBFT: 基于 hotstuff&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;先认识名词，从整体上有一些概念，战略上藐视。&lt;/p&gt;
&lt;h1 id=&quot;预先准备&quot;&gt;&lt;a href=&quot;#预先准备&quot; class=&quot;headerlink&quot; title=&quot;预先准备&quot;&gt;&lt;/a&gt;预先准备&lt;/h1&gt;&lt;p&gt;本文重点分析 paxos (basic paxos) 算法。顺带会提及 multi-paxos 以及 raft 算法。&lt;/p&gt;
&lt;p&gt;paxos 很难理解？争取听完本次分享，大家能彻底理解 paxos! &lt;/p&gt;
&lt;p&gt;先忘记区块链，忘记 pbft，忘记 hotstuff.&lt;/p&gt;
&lt;h1 id=&quot;单机？分布式？&quot;&gt;&lt;a href=&quot;#单机？分布式？&quot; class=&quot;headerlink&quot; title=&quot;单机？分布式？&quot;&gt;&lt;/a&gt;单机？分布式？&lt;/h1&gt;&lt;p&gt;为什么要有分布式系统？ 单机容易故障，无法保证服务高可用。&lt;/p&gt;
&lt;p&gt;于是出现多副本模型，但多副本模型就存在两个问题：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;如何确保复制是成功的？（高可用）&lt;/li&gt;
&lt;li&gt;如何确保值是唯一的？（一致性）&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
    
      <category term="blockchain" scheme="https://rebootcat.com/categories/blockchain/"/>
    
    
      <category term="blockchain" scheme="https://rebootcat.com/tags/blockchain/"/>
    
      <category term="paxos" scheme="https://rebootcat.com/tags/paxos/"/>
    
      <category term="basic-paxos" scheme="https://rebootcat.com/tags/basic-paxos/"/>
    
      <category term="classical-paxos" scheme="https://rebootcat.com/tags/classical-paxos/"/>
    
      <category term="multi-paxos" scheme="https://rebootcat.com/tags/multi-paxos/"/>
    
      <category term="raft" scheme="https://rebootcat.com/tags/raft/"/>
    
      <category term="pbft" scheme="https://rebootcat.com/tags/pbft/"/>
    
      <category term="bft" scheme="https://rebootcat.com/tags/bft/"/>
    
      <category term="hotstuff" scheme="https://rebootcat.com/tags/hotstuff/"/>
    
      <category term="librabft" scheme="https://rebootcat.com/tags/librabft/"/>
    
      <category term="statemachine" scheme="https://rebootcat.com/tags/statemachine/"/>
    
      <category term="mysql" scheme="https://rebootcat.com/tags/mysql/"/>
    
      <category term="binlog" scheme="https://rebootcat.com/tags/binlog/"/>
    
      <category term="consensus" scheme="https://rebootcat.com/tags/consensus/"/>
    
  </entry>
  
  <entry>
    <title>fork会复制线程吗</title>
    <link href="https://rebootcat.com/2020/11/21/fork_copy_thread/"/>
    <id>https://rebootcat.com/2020/11/21/fork_copy_thread/</id>
    <published>2020-11-21T03:23:58.000Z</published>
    <updated>2020-11-21T14:59:30.109Z</updated>
    
    <content type="html"><![CDATA[<h1 id="诡异的死锁"><a href="#诡异的死锁" class="headerlink" title="诡异的死锁"></a>诡异的死锁</h1><p>事情是这样的，观察到某台机器上出现了卡死的现象，即没有刷新日志，cpu 使用也较低，怀疑是不是出现了死锁。</p><p>由于程序采用的是 <code>master + worker</code> 的模式，首先 gdb attach 观察 master 情况，发现 master 执行正常，没有 <strong>lock wait</strong> 相关的堆栈；然后 gdb attach 观察 worker 情况，结果发现 worker 堆栈上有 <strong>lock wait</strong> 的情况，果然是出现了死锁，但 worker 上的其他线程并没有发现在等待锁的情况。</p><p>根据堆栈，找到 worker 的代码，重新梳理了一下代码，检查了 std::mutex 相关的函数调用，并没有出现嵌套调用的情况，也没有出现递归调用的情况，和上面发现 worker 其他线程没有等待锁的情况相吻合。</p><p>说明 worker 的死锁，并非由于 worker 内部的多线程造成的。那么就很诡异了，不是 worker 内部死锁，难道是多进程死锁？</p><h1 id="排查验证"><a href="#排查验证" class="headerlink" title="排查验证"></a>排查验证</h1><p>重新又检查了 worker 各个线程的堆栈情况，发现确实只有一个线程出现 <strong>lock wait</strong> 相关的堆栈； 并且又检查了一下 master 进程内部的各个线程，堆栈也都正常。</p><p>那 worker 锁住的这个线程，到底是因为什么原因？梳理 worker 代码，找到 std::mutex 相关的函数调用，发现 master 调用的一个函数使用到了 std::mutex，但是该函数内部逻辑也较为简单，不会一直占用这把锁。</p><p>没有头绪，谷歌搜索了一些类似的问题，找到了一点端倪。<strong>主进程 fork 之后，仅会复制发起调用的线程，不会复制其他线程，如果某个线程占用了某个锁，但是到了子进程，该线程是蒸发掉的，子进程会拷贝这把锁，但是不知道谁能释放，最终死锁</strong>。</p><p>确实符合这个程序的行为，并且确实是多进程下子进程的死锁，而且找不到其他线程也在等待锁。</p><p>接下来，写一个 demo 验证一下，是否 fork 不会复制子线程，并且有可能造成死锁。</p><h2 id="fork-demo-验证"><a href="#fork-demo-验证" class="headerlink" title="fork demo 验证"></a>fork demo 验证</h2><p>简单写一个 demo:</p><a id="more"></a><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// file: fork_copy_thread.cc</span></span><br><span class="line"><span class="comment">// g++ fork_copy_thread.cc  -o fork_copy_thread -std=c++11 -lpthread -ggdb</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;iostream&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;memory&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;string&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;thread&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;chrono&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;mutex&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Event</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    Event() = <span class="keyword">default</span>;</span><br><span class="line">    ~Event() = <span class="keyword">default</span>;</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> str_;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TaskHandler</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    TaskHandler() = <span class="keyword">default</span>;</span><br><span class="line">    ~TaskHandler() = <span class="keyword">default</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">start</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">auto</span> lam = [&amp;]() -&gt; <span class="keyword">void</span> &#123;</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="built_in">std</span>::unique_lock&lt;<span class="built_in">std</span>::mutex&gt; lock(ev_mutex_);</span><br><span class="line">                <span class="keyword">this</span>-&gt;ev_ = <span class="built_in">std</span>::make_shared&lt;Event&gt;();</span><br><span class="line">                <span class="keyword">this</span>-&gt;ev_-&gt;str_ = <span class="string">"hello fork"</span>;</span><br><span class="line">                <span class="comment">// hold this lock for 10 seconds</span></span><br><span class="line">                <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">10</span>));</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"father thread done, exit"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        &#125;;</span><br><span class="line"></span><br><span class="line">        <span class="function"><span class="built_in">std</span>::thread <span class="title">th</span><span class="params">(lam)</span></span>;</span><br><span class="line">        th.<span class="built_in">detach</span>();</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">print_str</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="function"><span class="built_in">std</span>::unique_lock&lt;<span class="built_in">std</span>::mutex&gt; <span class="title">lock</span><span class="params">(ev_mutex_)</span></span>;</span><br><span class="line">        <span class="keyword">if</span> (!ev_) &#123;</span><br><span class="line">            <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"event  not ready"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"event:"</span> &lt;&lt; ev_-&gt;str_ &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span>:</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;Event&gt; ev_ &#123; <span class="literal">nullptr</span> &#125;;</span><br><span class="line">    <span class="built_in">std</span>::mutex ev_mutex_;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;TaskHandler&gt; tsk = <span class="literal">nullptr</span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    tsk = <span class="built_in">std</span>::make_shared&lt;TaskHandler&gt;();</span><br><span class="line">    tsk-&gt;start();</span><br><span class="line">    <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line"></span><br><span class="line">    <span class="comment">// for child process</span></span><br><span class="line">    <span class="keyword">pid_t</span> pid = fork();</span><br><span class="line">    <span class="keyword">switch</span> (pid) &#123;</span><br><span class="line">        <span class="keyword">case</span> <span class="number">-1</span>:</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        <span class="keyword">case</span> <span class="number">0</span>:</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"this is child process"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">                    <span class="comment">// will core here, because tsk-&gt;ev_ is created in father-thread, not copyed,</span></span><br><span class="line">                    <span class="comment">// so in child process, tsk-&gt;ev_ is nullptr</span></span><br><span class="line">                    tsk-&gt;print_str();</span><br><span class="line">                    <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">        <span class="keyword">default</span>:</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="comment">// this is father</span></span><br><span class="line">                <span class="keyword">break</span>;</span><br><span class="line">            &#125;</span><br><span class="line">    &#125; <span class="comment">// end switch</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>上面的代码简单解释一下：</p><ul><li>TaskHandler::start() 中会创建一个线程，线程会申请一把互斥锁，并且睡眠 10s，目的是为了在 fork 的时候依然占用这把互斥锁</li><li>TaskHandler::print_str() 会申请这把互斥锁，然后打印字符串</li><li>程序 main 开始，调用 start() 创建子线程</li><li>然后 fork() 子进程</li><li>子进程死循环执行 print_str() 函数打印字符串</li></ul><p>使用编译命令：</p><figure class="highlight brainfuck"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">$</span> <span class="comment">g</span>++ <span class="comment">fork_copy_thread</span><span class="string">.</span><span class="comment">cc</span>  <span class="literal">-</span><span class="comment">o</span> <span class="comment">fork_copy_thread</span> <span class="literal">-</span><span class="comment">std=c</span>++<span class="comment">11</span> <span class="literal">-</span><span class="comment">lpthread</span> <span class="literal">-</span><span class="comment">ggdb</span></span><br></pre></td></tr></table></figure><p>运行后与预期不符，子进程并没有死循环打印字符串，死锁了。</p><p>然后使用 gdb attach 子进程：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">(gdb) bt</span><br><span class="line">#<span class="number">0</span>  <span class="number">0x00007f4154e1a54d</span> <span class="keyword">in</span> __lll_lock_wait () <span class="keyword">from</span> /lib64/libpthread.so<span class="number">.0</span></span><br><span class="line">#<span class="number">1</span>  <span class="number">0x00007f4154e15e9b</span> <span class="keyword">in</span> _L_lock_883 () <span class="keyword">from</span> /lib64/libpthread.so<span class="number">.0</span></span><br><span class="line">#<span class="number">2</span>  <span class="number">0x00007f4154e15d68</span> <span class="keyword">in</span> pthread_mutex_lock () <span class="keyword">from</span> /lib64/libpthread.so<span class="number">.0</span></span><br><span class="line">#<span class="number">3</span>  <span class="number">0x000000000040128c</span> <span class="keyword">in</span> __gthread_mutex_lock (__mutex=<span class="number">0x1d2fc48</span>) at /usr/include/c++/<span class="number">4.8</span><span class="number">.5</span>/x86_64-unknown-linux-gnu/bits/gthr-<span class="keyword">default</span>.h:<span class="number">748</span></span><br><span class="line">#<span class="number">4</span>  <span class="number">0x0000000000401730</span> <span class="keyword">in</span> std::mutex::lock (<span class="keyword">this</span>=<span class="number">0x1d2fc48</span>) at /usr/include/c++/<span class="number">4.8</span><span class="number">.5</span>/mutex:<span class="number">134</span></span><br><span class="line">#<span class="number">5</span>  <span class="number">0x0000000000401f99</span> <span class="keyword">in</span> std::unique_lock&lt;std::mutex&gt;::lock (<span class="keyword">this</span>=<span class="number">0x7fff168c43a0</span>) at /usr/include/c++/<span class="number">4.8</span><span class="number">.5</span>/mutex:<span class="number">511</span></span><br><span class="line">#<span class="number">6</span>  <span class="number">0x0000000000401b13</span> <span class="keyword">in</span> std::unique_lock&lt;std::mutex&gt;::unique_lock (<span class="keyword">this</span>=<span class="number">0x7fff168c43a0</span>, __m=...) at /usr/include/c++/<span class="number">4.8</span><span class="number">.5</span>/mutex:<span class="number">443</span></span><br><span class="line">#<span class="number">7</span>  <span class="number">0x0000000000401988</span> <span class="keyword">in</span> TaskHandler::print_str (<span class="keyword">this</span>=<span class="number">0x1d2fc38</span>) at fork_copy_thread.cc:<span class="number">43</span></span><br><span class="line">#<span class="number">8</span>  <span class="number">0x00000000004013ff</span> <span class="keyword">in</span> main () at fork_copy_thread.cc:<span class="number">76</span></span><br><span class="line">(gdb)</span><br></pre></td></tr></table></figure><p>果然可以看到子进程卡在了 print_str() 函数上。</p><p>上面的代码，父进程创建线程后，占用了锁，此时 fork 了子进程，子进程拷贝了父进程空间的内存，包括锁，但是没有复制子线程，造成子进程无法获取锁，最终死锁。</p><h2 id="fork-copy-thread"><a href="#fork-copy-thread" class="headerlink" title="fork copy thread?"></a>fork copy thread?</h2><p>上面已经验证了死锁的产生原因是由于 fork 时并没有把父进程里的线程复制到子进程，导致子进程无法获取锁。那么简单修改一下上面的代码，来验证一下子进程确实是没有复制父进程的子线程。</p><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// file: fork_copy_thread.cc</span></span><br><span class="line"><span class="comment">// g++ fork_copy_thread.cc  -o fork_copy_thread -std=c++11 -lpthread -ggdb</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;iostream&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;memory&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;string&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;thread&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;chrono&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;mutex&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Event</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    Event() = <span class="keyword">default</span>;</span><br><span class="line">    ~Event() = <span class="keyword">default</span>;</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> str_;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TaskHandler</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    TaskHandler() = <span class="keyword">default</span>;</span><br><span class="line">    ~TaskHandler() = <span class="keyword">default</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">start</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">auto</span> lam = [&amp;]() -&gt; <span class="keyword">void</span> &#123;</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="built_in">std</span>::unique_lock&lt;<span class="built_in">std</span>::mutex&gt; lock(ev_mutex_);</span><br><span class="line">                <span class="keyword">this</span>-&gt;ev_ = <span class="built_in">std</span>::make_shared&lt;Event&gt;();</span><br><span class="line">                <span class="keyword">this</span>-&gt;ev_-&gt;str_ = <span class="string">"hello fork"</span>;</span><br><span class="line">                <span class="comment">// hold this lock for 10 seconds</span></span><br><span class="line">                <span class="comment">//std::this_thread::sleep_for(std::chrono::seconds(10));</span></span><br><span class="line">            &#125;</span><br><span class="line">            <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"this threadid:"</span> &lt;&lt; <span class="built_in">std</span>::this_thread::get_id() &lt;&lt; <span class="string">" run"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;;</span><br><span class="line"></span><br><span class="line">        <span class="function"><span class="built_in">std</span>::thread <span class="title">th</span><span class="params">(lam)</span></span>;</span><br><span class="line">        th.<span class="built_in">detach</span>();</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">print_str</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="function"><span class="built_in">std</span>::unique_lock&lt;<span class="built_in">std</span>::mutex&gt; <span class="title">lock</span><span class="params">(ev_mutex_)</span></span>;</span><br><span class="line">        <span class="keyword">if</span> (!ev_) &#123;</span><br><span class="line">            <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"event  not ready"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"event:"</span> &lt;&lt; ev_-&gt;str_ &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line"></span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span>:</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;Event&gt; ev_ &#123; <span class="literal">nullptr</span> &#125;;</span><br><span class="line">    <span class="built_in">std</span>::mutex ev_mutex_;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;TaskHandler&gt; tsk = <span class="literal">nullptr</span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    tsk = <span class="built_in">std</span>::make_shared&lt;TaskHandler&gt;();</span><br><span class="line">    tsk-&gt;start();</span><br><span class="line">    <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line"></span><br><span class="line">    <span class="comment">// for child process</span></span><br><span class="line">    <span class="keyword">pid_t</span> pid = fork();</span><br><span class="line">    <span class="keyword">switch</span> (pid) &#123;</span><br><span class="line">        <span class="keyword">case</span> <span class="number">-1</span>:</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        <span class="keyword">case</span> <span class="number">0</span>:</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"this is child process"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">                    <span class="comment">// will core here, because tsk-&gt;ev_ is created in father-thread, not copyed,</span></span><br><span class="line">                    <span class="comment">// so in child process, tsk-&gt;ev_ is nullptr</span></span><br><span class="line">                    <span class="comment">//tsk-&gt;print_str();</span></span><br><span class="line">                    <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">        <span class="keyword">default</span>:</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="comment">// this is father</span></span><br><span class="line">                <span class="keyword">break</span>;</span><br><span class="line">            &#125;</span><br><span class="line">    &#125; <span class="comment">// end switch</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>简单解释一下修改了啥：</p><ul><li>父进程启动一个线程，循环打印字符串</li><li>父进程 fork，子进程保持睡眠</li><li>验证子进程是否有线程打印字符串(如果复制了的话，理应会打印）</li></ul><p>执行结果：</p><figure class="highlight kotlin"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ ./fork_copy_thread</span><br><span class="line"><span class="keyword">this</span> threadid:<span class="number">139674369169152</span> run</span><br><span class="line"><span class="keyword">this</span> threadid:<span class="number">139674369169152</span> run</span><br><span class="line"><span class="keyword">this</span> <span class="keyword">is</span> child process</span><br><span class="line"><span class="keyword">this</span> threadid:<span class="number">139674369169152</span> run</span><br><span class="line"><span class="keyword">this</span> threadid:<span class="number">139674369169152</span> run</span><br><span class="line"><span class="keyword">this</span> threadid:<span class="number">139674369169152</span> run</span><br></pre></td></tr></table></figure><p>可以看到只有一个线程在打印，也就是父进程创建的那个线程；fork 之后父进程的线程在子进程蒸发了。</p><p><strong>多线程程序使用 fork 一定要谨慎，再谨慎，并且也不推荐这样的做法。</strong></p><h1 id="fork-到底复制了什么"><a href="#fork-到底复制了什么" class="headerlink" title="fork 到底复制了什么"></a>fork 到底复制了什么</h1><blockquote><p><a href="https://linux.die.net/man/3/fork" target="_blank" rel="noopener">https://linux.die.net/man/3/fork</a></p></blockquote><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">pid_t</span> <span class="title">fork</span><span class="params">(<span class="keyword">void</span>)</span></span>;</span><br></pre></td></tr></table></figure><h2 id="Copy-On-Write"><a href="#Copy-On-Write" class="headerlink" title="Copy On Write"></a>Copy On Write</h2><blockquote><p>Copy On Write(写时复制）技术大大提高了 fork 的性能。fork 之后，内核会把父进程中的所有内存页都设置为 read-only，然后子进程的地址空间指向父进程。如果父进程和子进程都没有涉及到内存的写操作，那么父子进程保持这样的状态，也就是子进程并不会复制父进程的内存空间；如果父进程或者子进程产生了写操作，那么由于内存页被设置为 read-only，所以会触发页异常中断，然后中断程序会把该内存页复制一份，至此父子进程就拥有不同的内存页；而其他没有操作的内存页依然共享。</p></blockquote><p>上面这段话不太好理解，涉及到的东西其实比较深也比较多。我们把它拆开来说。</p><p><strong>虚拟内存空间</strong>，进程是看不见物理内存地址的，进程的内存空间称为虚拟内存，默认从 0 到 max，虚拟内存空间也就是逻辑内存地址，进程操作的都是逻辑内存地址。</p><p>虚拟内存地址到真实的物理内存地址的转换或者映射称为<strong>地址重定向</strong>，有专门的中断程序来负责处理，作为进程本身不需要关心。</p><p><strong>物理内存的单位是页，也就是内核使用页为单位来管理物理内存</strong>，数据结构上页其实是一个 struct，大小好像是 4KB。虚拟内存地址映射到物理内存以页的方式进行，并且<strong>内核管理一个页映射表</strong>。</p><p><strong>malloc 分配内存，其实操作的是虚拟内存</strong>，也即使用 malloc 分配了一段内存后，在未赋值之前，其实是没有物理内存占用的，当真正向 malloc 分配的内存写数据的时候，内核才会分配真实的物理内存页，并让这段虚拟内存指向实际的物理内存页。并且进程管理一个页表。</p><p>进程的虚拟内存空间，由地地址到高地址空间大致分为代码段、数据段、BSS 段、堆、栈，详情如下：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/free_memory/1.png" alt=""></p><p>fork 之后，子进程复制了父进程的虚拟内存空间，即复制了代码段、堆栈等，所以变量的地址也是一样的。并且父子进程各自有一份页映射表，它们都指向父进程的物理内存地址。</p><p>当父子进程只读时，不会发生真实的物理内存拷贝；但是当父子进程写入时，由于物理页 read-only，会触发页异常中断，中断程序会把该页面复制一份，其他的页保持不动。至此父进程和子进程的页映射表就出现了一点不一致了，但其他部分还是一致的。</p><h2 id="简单总结下-fork"><a href="#简单总结下-fork" class="headerlink" title="简单总结下 fork"></a>简单总结下 fork</h2><p>要理解 fork 的原理，Copy On Write 的原理，重点是理解虚拟内存和物理内存的关系。</p><p>fork 之后，<strong>子进程会复制父进程的虚拟内存空间，也就是代码段、数据段、堆栈等，虚拟内存空间里表达的就是程序里各个变量的地址，所以子进程里各个变量的地址和父进程里各个变量的地址是一样的</strong>。</p><p><strong>父子进程只读时，不会发生真实的物理内存拷贝，他们的页映射表内容一致，即同样的虚拟内存地址指向同样的物理内存地址；但当有一方写入数据时，内核会复制要写入的页，此时修改数据的一方的页映射表就发生了变化，即同样的虚拟内存地址指向了不同的物理内存地址，但其他部分还是一样的</strong>；</p><p>另外，<strong>fork 仅会将发起调用的线程复制到子进程中，所以子进程中的线程 ID 与主进程线程 ID 有一致的情况。其他线程不会被复制</strong>。</p><h1 id="The-End"><a href="#The-End" class="headerlink" title="The End"></a>The End</h1><p>关于 fork 的细节，还有很多值得深入研究的东西。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#105;&#x6e;&#117;&#120;&#x63;&#x6f;&#100;&#x65;&#50;&#110;&#105;&#x6b;&#105;&#x40;&#103;&#x6d;&#97;&#x69;&#x6c;&#46;&#99;&#x6f;&#109;">&#x6c;&#105;&#x6e;&#117;&#120;&#x63;&#x6f;&#100;&#x65;&#50;&#110;&#105;&#x6b;&#105;&#x40;&#103;&#x6d;&#97;&#x69;&#x6c;&#46;&#99;&#x6f;&#109;</a></p></li></ul><p>2020-11-21 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;诡异的死锁&quot;&gt;&lt;a href=&quot;#诡异的死锁&quot; class=&quot;headerlink&quot; title=&quot;诡异的死锁&quot;&gt;&lt;/a&gt;诡异的死锁&lt;/h1&gt;&lt;p&gt;事情是这样的，观察到某台机器上出现了卡死的现象，即没有刷新日志，cpu 使用也较低，怀疑是不是出现了死锁。&lt;/p&gt;
&lt;p&gt;由于程序采用的是 &lt;code&gt;master + worker&lt;/code&gt; 的模式，首先 gdb attach 观察 master 情况，发现 master 执行正常，没有 &lt;strong&gt;lock wait&lt;/strong&gt; 相关的堆栈；然后 gdb attach 观察 worker 情况，结果发现 worker 堆栈上有 &lt;strong&gt;lock wait&lt;/strong&gt; 的情况，果然是出现了死锁，但 worker 上的其他线程并没有发现在等待锁的情况。&lt;/p&gt;
&lt;p&gt;根据堆栈，找到 worker 的代码，重新梳理了一下代码，检查了 std::mutex 相关的函数调用，并没有出现嵌套调用的情况，也没有出现递归调用的情况，和上面发现 worker 其他线程没有等待锁的情况相吻合。&lt;/p&gt;
&lt;p&gt;说明 worker 的死锁，并非由于 worker 内部的多线程造成的。那么就很诡异了，不是 worker 内部死锁，难道是多进程死锁？&lt;/p&gt;
&lt;h1 id=&quot;排查验证&quot;&gt;&lt;a href=&quot;#排查验证&quot; class=&quot;headerlink&quot; title=&quot;排查验证&quot;&gt;&lt;/a&gt;排查验证&lt;/h1&gt;&lt;p&gt;重新又检查了 worker 各个线程的堆栈情况，发现确实只有一个线程出现 &lt;strong&gt;lock wait&lt;/strong&gt; 相关的堆栈； 并且又检查了一下 master 进程内部的各个线程，堆栈也都正常。&lt;/p&gt;
&lt;p&gt;那 worker 锁住的这个线程，到底是因为什么原因？梳理 worker 代码，找到 std::mutex 相关的函数调用，发现 master 调用的一个函数使用到了 std::mutex，但是该函数内部逻辑也较为简单，不会一直占用这把锁。&lt;/p&gt;
&lt;p&gt;没有头绪，谷歌搜索了一些类似的问题，找到了一点端倪。&lt;strong&gt;主进程 fork 之后，仅会复制发起调用的线程，不会复制其他线程，如果某个线程占用了某个锁，但是到了子进程，该线程是蒸发掉的，子进程会拷贝这把锁，但是不知道谁能释放，最终死锁&lt;/strong&gt;。&lt;/p&gt;
&lt;p&gt;确实符合这个程序的行为，并且确实是多进程下子进程的死锁，而且找不到其他线程也在等待锁。&lt;/p&gt;
&lt;p&gt;接下来，写一个 demo 验证一下，是否 fork 不会复制子线程，并且有可能造成死锁。&lt;/p&gt;
&lt;h2 id=&quot;fork-demo-验证&quot;&gt;&lt;a href=&quot;#fork-demo-验证&quot; class=&quot;headerlink&quot; title=&quot;fork demo 验证&quot;&gt;&lt;/a&gt;fork demo 验证&lt;/h2&gt;&lt;p&gt;简单写一个 demo:&lt;/p&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="fork" scheme="https://rebootcat.com/tags/fork/"/>
    
      <category term="thread" scheme="https://rebootcat.com/tags/thread/"/>
    
      <category term="memory" scheme="https://rebootcat.com/tags/memory/"/>
    
  </entry>
  
  <entry>
    <title>TCP全连接和半连接的问题探讨</title>
    <link href="https://rebootcat.com/2020/11/14/tcp_accept/"/>
    <id>https://rebootcat.com/2020/11/14/tcp_accept/</id>
    <published>2020-11-14T03:23:58.000Z</published>
    <updated>2020-11-12T15:15:17.676Z</updated>
    
    <content type="html"><![CDATA[<h1 id="从何说起"><a href="#从何说起" class="headerlink" title="从何说起"></a>从何说起</h1><p>说起 tcp 的连接过程，想必 “<strong>3次握手4次挥手</strong>”是大家广为熟知的知识，那么关于更细节更底层的连接过程也许就很少人能讲清楚了。</p><p>所以本文会先简单回顾一下 tcp 的 3次握手过程，然后重点聊一下 tcp accept 的过程，涉及到 tcp 半连接队列、全连接队列等的内容。</p><h1 id="回顾一下"><a href="#回顾一下" class="headerlink" title="回顾一下"></a>回顾一下</h1><h2 id="3-次握手"><a href="#3-次握手" class="headerlink" title="3 次握手"></a>3 次握手</h2><p>要了解 3 次握手的过程，可能需要先熟悉一下 tcp 协议的格式：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/tcp_accept/1.png" alt=""></p><ul><li>tcp segment 的头部有两个 2字节的字段 <code>source port</code> 和 <code>dest port</code>，分别表示本机端口以及目标端口，在 tcp 传输层是没有 IP 的概念的，那是 IP 层 的概念，IP 层协议会在 IP 协议的头部加上 <code>src ip</code> 和 <code>dest ip</code>；</li><li>4 个字节的 seq，表示序列号，tcp 是可靠连接，不会乱序；</li><li>4 个字节的 ack，表示确认号，表示对接收到的上一个报文的确认，值为 seq + 1;</li><li>几个标志位：ACK,RST,SYN,FIN 这些是我们常用的，比较熟悉的。其中 ACK 简写为 “.”; RST 简写为 “R”; SYN 简写为 “S”; FIN 简写为 “F”;</li></ul><blockquote><p>注意： ack 和 ACK 是不一样的意思，一个是确认号，一个是标志位</p></blockquote><a id="more"></a><p>了解了 tcp 协议的头部格式，那么再来讲一下 3 次握手的过程：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/tcp_accept/2.png" alt=""></p><ol><li>客户端对服务端发起建立连接的请求，发送一个 SYN 包（也就是 SYN 标志位设置为 1），同时随机生成一个 seq 值 x，然后客户端就处于 SYN_SENT 状态；</li><li>服务端收到客户端的连接请求，回复一个 SYN+ACK包（也就是设置 SYN 和 ACK 标志位为 1），同时随机生成一个 seq 值 y，然后确认号 ack = x + 1，也就是 client 的 seq +1，服务端进入 SYN_RECV 阶段；</li><li>客户端收到服务端的 SYN+ACK 包，会回复一个 ACK 包（也就是设置 ACK 标志位为 1），设置 seq = x + 1，ack 等于 服务端的 seq +1，也就是 ack = y+1，然后连接建立成功；</li></ol><h2 id="tcpdump-抓包"><a href="#tcpdump-抓包" class="headerlink" title="tcpdump 抓包"></a>tcpdump 抓包</h2><p>开一个终端执行以下命令作为服务端：</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 服务端</span></span><br><span class="line"><span class="variable">$ </span>nc -l <span class="number">10000</span></span><br></pre></td></tr></table></figure><p>然后打开新的终端用 tcpdump 抓包：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># -i 表示监听所有网卡；</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># -t 表示不打印 timestamp;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># -S 表示打印绝对的 seq 而不是相对的 seq number;</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># port 10000 表示对 10000 端口进行抓包</span></span><br><span class="line"></span><br><span class="line">$ tcpdump  -i any -t -S<span class="built_in"> port </span>10000</span><br></pre></td></tr></table></figure><p>然后再打开一个终端模拟客户端:</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ nc <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> <span class="number">10000</span></span><br></pre></td></tr></table></figure><p>观察 tcpdump 的输出如下：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">IP Jia<span class="number">.22921</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>.ndmp: Flags [S], seq <span class="number">614247470</span>, win <span class="number">29200</span>, options [mss <span class="number">1460</span>,sackOK,TS val <span class="number">159627770</span> ecr <span class="number">0</span>], length <span class="number">0</span></span><br><span class="line">IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>.ndmp &gt; Jia<span class="number">.22921</span>: Flags [S.], seq <span class="number">1720434034</span>, ack <span class="number">614247471</span>, win <span class="number">65160</span>, options [mss <span class="number">1460</span>,sackOK,TS val <span class="number">3002840224</span> ecr <span class="number">159627770</span>], length <span class="number">0</span></span><br><span class="line">IP Jia<span class="number">.22921</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>.ndmp: Flags [.], ack <span class="number">1720434035</span>, win <span class="number">29200</span>, options [nop,nop,TS val <span class="number">159627770</span> ecr <span class="number">3002840224</span>], length <span class="number">0</span></span><br></pre></td></tr></table></figure><p>分析以下上面的结果可以看到：</p><ol><li><p>第一个包 Flags [S] 表示 SYN 包，seq 为随机值 614247470；</p></li><li><p>然后服务端回复了一个 Falgs [S.]，也就是 SYN+ACK 包，同时设置 seq 为随机值 1720434034，设置 ack 为 614247470 + 1 = 614247471；</p></li><li><p>客户端收到之后，回复一个 Flags [.]，也就是 ACK 包，同时设置 ack 为 1720434034 + 1 = 1720434035；</p></li></ol><h2 id="假如3次握手丢包了？"><a href="#假如3次握手丢包了？" class="headerlink" title="假如3次握手丢包了？"></a>假如3次握手丢包了？</h2><p>上面是正常情况的握手情况，假如握手过程中的任何一个包出现丢包呢会怎么样？比如受到了攻击，比如服务端宕机，服务端超时，客户端掉线，网络波动等。</p><p>所以接下来我们分析下 3 次握手过程中涉及到的连接队列。</p><h1 id="tcp-内核参数"><a href="#tcp-内核参数" class="headerlink" title="tcp 内核参数"></a>tcp 内核参数</h1><h2 id="backlog-参数"><a href="#backlog-参数" class="headerlink" title="backlog 参数"></a>backlog 参数</h2><blockquote><p><a href="https://linux.die.net/man/3/listen" target="_blank" rel="noopener">https://linux.die.net/man/3/listen</a></p><p>The backlog argument provides a hint to the implementation which the implementation shall use to limit the number of outstanding connections in the socket’s listen queue. Implementations may impose a limit on backlog and silently reduce the specified value. Normally, a larger backlog argument value shall result in a larger or equal length of the listen queue. Implementations shall support values of backlog up to SOMAXCONN, defined in &lt;sys/socket.h&gt;.</p></blockquote><figure class="highlight perl"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">int</span> <span class="keyword">listen</span>(<span class="keyword">int</span> <span class="keyword">socket</span>, <span class="keyword">int</span> backlog);</span><br></pre></td></tr></table></figure><p>backlog 参数是用来限制 tcp listen queue 的大小的，真实的 listen queue 大小其实也是跟内核参数 somaxconn 有关系，somaxconn 是内核用来限制同一个端口上的连接队列长度。</p><h2 id="全连接队列"><a href="#全连接队列" class="headerlink" title="全连接队列"></a>全连接队列</h2><p>完成 3 次握手的连接，也就是服务端收到了客户端发送的最后一个 ACK 报文后，这个连接会被放到这个端口的全连接队列里，然后等待应用程序来处理，对于 epoll 来说就是内核触发 EPOLLIN 事件，然后应用层使用 <code>epoll_wait</code> 来处理 accept 事件，为连接分配创建 socket 结构，分配 file descriptor 等；</p><p>那么假如应用层没有来处理这些就绪的连接呢？那么这个全连接队列有可能就满了，导致后续的连接被丢弃，发生<strong>全连接队列溢出</strong>，丢弃这个连接，对客户端来说就无法成功建立连接。</p><p>所以为了性能的考虑，我们有必要尽可能的把这个队列的大小调大一点。</p><h3 id="查看全连接队列大小"><a href="#查看全连接队列大小" class="headerlink" title="查看全连接队列大小"></a>查看全连接队列大小</h3><p>可以通过一下命令来查看当前端口的全连接队列大小：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ ss -antl</span><br><span class="line">State   Recv-Q   Send-Q     Local Address:Port     <span class="built_in"> Peer </span>Address:Port  Process            </span><br><span class="line">LISTEN  0        5           192.168.1.7:10000          0.0.0.0:*</span><br></pre></td></tr></table></figure><p>在 ss 输出中：</p><p>LISTEN 状态：Recv-Q 表示当前 listen backlog 队列中的连接数目（等待用户调用 accept() 获取的、已完成 3 次握手的 socket 连接数量），而 Send-Q 表示了 listen socket 最大能容纳的 backlog。</p><p>非 LISTEN 状态：Recv-Q 表示了 receive queue 中存在的字节数目；Send-Q 表示 send queue 中存在的字节数；</p><h3 id="压测观察全连接队列溢出"><a href="#压测观察全连接队列溢出" class="headerlink" title="压测观察全连接队列溢出"></a>压测观察全连接队列溢出</h3><p>接下来我们实际测试一下，使用项目：<a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">mux</a>。</p><p>我们先修改一下 backlog 参数为 5：</p><figure class="highlight apache"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 把backlog 调小一点</span></span><br><span class="line"></span><br><span class="line"><span class="attribute"><span class="nomarkup">listen</span></span>(listenfd, 5);</span><br></pre></td></tr></table></figure><p>根据编译文档，编译后得到两个二进制：</p><figure class="highlight armasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ ls</span><br><span class="line"><span class="keyword">bench_server </span>  <span class="keyword">bench_client_accept</span></span><br></pre></td></tr></table></figure><ul><li><code>bench_server</code> 用来作为服务端，底层使用 epoll 实现</li><li><code>bench_client_accept</code> 作为压测客户端，并发创建大量连接，这里只会与服务端建立连接，不会发送其他任何消息（当然可以用其他的压测工具）</li></ul><p>选择两台机器进行测试，192.168.1.7 作为服务端， 192.168.1.4 作为压测客户端，开始压测前，可能需要设置一下：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">ulimit</span> -n 65535</span></span><br></pre></td></tr></table></figure><p>1)  启动服务端</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span> 作为服务端，监听 <span class="number">10000</span> 端口</span><br><span class="line"></span><br><span class="line">$ ./bench_server <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span> <span class="number">10000</span></span><br></pre></td></tr></table></figure><p><strong>注意到上图执行 <code>ss -antl</code> 看到 10000 端口的 listen queue size 为 5，这里是故意调小一点，为了验证全连接队列溢出的场景</strong>。</p><p>2) 先观察一下服务端全连接队列的情况以及溢出的情况</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ netstat  -s |grep -i overflowed</span><br><span class="line">    <span class="number">2283</span> times the listen queue of a socket overflowed</span><br></pre></td></tr></table></figure><p>上述表明 10000 端口的 listen queue size 为 5，并且全连接队列中没有等待应用层处理的连接；</p><p><strong>netstat  -s |grep -i overflowed</strong> 表示全连接队列溢出的情况，2683 是一个累加值。</p><p>2) 启动 tcpdump 对客户端行为抓包，分析 3次握手连接情况</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 运行在 client: 192.168.1.4 上</span></span><br><span class="line"></span><br><span class="line">$ tcpdump  -i any<span class="built_in"> port </span>10000 <span class="keyword">and</span> tcp -nn &gt; tcpdump.log</span><br></pre></td></tr></table></figure><p>3）启动压测客户端</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"># <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span> 作为压测客户端</span><br><span class="line"># <span class="number">30000</span> 表示连接数</span><br><span class="line"># <span class="number">100</span> 表示 <span class="number">100</span> 个并发线程</span><br><span class="line"># <span class="number">1</span> 表示执行 <span class="number">1</span> 轮</span><br><span class="line"></span><br><span class="line">$ ./bench_client_accept  <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span> <span class="number">10000</span> <span class="number">30000</span> <span class="number">100</span> <span class="number">1</span></span><br></pre></td></tr></table></figure><p>压测过程中，可以不断执行命令观察服务端全连接队列溢出的情况，压测完毕之后再观察一下全连接队列溢出的情况：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">5</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*                          </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">1</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">1</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">5</span>            <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ netstat  -s |grep -i overflowed</span><br><span class="line">    <span class="number">2930</span> times the listen queue of a socket overflowed</span><br></pre></td></tr></table></figure><p>可以看到，<strong>压测过程中的 Recv-Q 出现了5，1 的值，表示全连接队列中等待被处理的连接，而且有 2930 - 2283 = 647 次连接由于全连接队列溢出而被丢弃</strong>。</p><p>我们再来观察一下 <code>bench_client_accept</code> 的日志情况：</p><figure class="highlight 1c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ grep -a 'Start OK' <span class="built_in">log</span>/bench_client_accept.<span class="built_in">log</span>  <span class="string">|wc -l</span></span><br><span class="line"><span class="number">29736</span></span><br><span class="line">$ grep -a 'start failed' <span class="built_in">log</span>/bench_client_accept.<span class="built_in">log</span>  <span class="string">|wc -l</span></span><br><span class="line"><span class="number">264</span></span><br></pre></td></tr></table></figure><p>可以看到<strong>最终有 264 个 client 由于服务端丢弃建立连接时 3 次握手的包而造成连接失败</strong>。</p><p>如果你细心的话会发现，全连接队列溢出发生了 647 次，但是最终只有 264 个 client 建立失败，why?其实原因很简单，因为客户端有重试机制，具体参数是 <code>net.ipv4.tcp_syn_retries</code>，这个暂且不详说。</p><p>那再来看一下 tcpdump 抓包的结果，这里要用到一个 python 脚本 <code>tcpdump_analyze.py</code> 来处理一下 tcpdump.log 这个日志：</p><figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br></pre></td><td class="code"><pre><span class="line">import os</span><br><span class="line"></span><br><span class="line"><span class="comment"># tcpdump  -i any port 10000 and tcp -nn &gt; tcpdump.log</span></span><br><span class="line"></span><br><span class="line">server_ip_port = <span class="string">"192.168.1.7.10000"</span></span><br><span class="line">client_map = &#123;&#125;</span><br><span class="line"></span><br><span class="line">with open(<span class="string">'./tcpdump.log'</span>, <span class="string">'r'</span>) as fin:</span><br><span class="line">    <span class="keyword">for</span> line <span class="keyword">in</span> fin:</span><br><span class="line">        sp = line.split()</span><br><span class="line">        <span class="keyword">if</span> len(sp) &lt; <span class="number">3</span>:</span><br><span class="line">            print(<span class="string">"invalid line:&#123;0&#125;"</span>.format(line))</span><br><span class="line">            <span class="keyword">continue</span></span><br><span class="line"></span><br><span class="line">        client_ip_port = sp[<span class="number">2</span>]</span><br><span class="line">        <span class="keyword">if</span> client_ip_port == server_ip_port:</span><br><span class="line">            client_ip_port = sp[<span class="number">4</span>].split(<span class="string">':'</span>)[<span class="number">0</span>]</span><br><span class="line"></span><br><span class="line">        <span class="keyword">if</span> client_ip_port not <span class="keyword">in</span> client_map:</span><br><span class="line">            client_map[<span class="type">client_ip_port</span>] = [<span class="type">line</span>]</span><br><span class="line">        <span class="keyword">else</span>:</span><br><span class="line">            client_map[<span class="type">client_ip_port</span>].append(line)</span><br><span class="line"></span><br><span class="line">connect_fail_client = []</span><br><span class="line">connect_succ_client = []</span><br><span class="line">connect_succ_client_normal = []</span><br><span class="line">connect_succ_client_try   = []</span><br><span class="line"></span><br><span class="line">total_size = len(client_map)</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> k,v <span class="keyword">in</span> client_map.items():</span><br><span class="line">    print(<span class="string">"&#123;0&#125;<span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$begin</span>"</span>.format(k))</span><br><span class="line">    <span class="keyword">for</span> ll <span class="keyword">in</span> v:</span><br><span class="line">        print(ll)</span><br><span class="line">    connect_fail = True</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> v:</span><br><span class="line">        <span class="comment"># ack 1 is the last packet of tcp handshake from server</span></span><br><span class="line">        <span class="keyword">if</span> i.find(<span class="string">'ack 1,'</span>) != <span class="literal">-1</span>:</span><br><span class="line">            connect_fail = False</span><br><span class="line">            <span class="keyword">break</span></span><br><span class="line">    <span class="keyword">if</span> connect_fail:</span><br><span class="line">        connect_fail_client.append(v)</span><br><span class="line">        print(<span class="string">"fail"</span>);</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        connect_succ_client.append(v)</span><br><span class="line">        <span class="keyword">if</span> len(v) == <span class="number">3</span>:</span><br><span class="line">            connect_succ_client_normal.append(v)</span><br><span class="line">            print(<span class="string">"succ no retry"</span>);</span><br><span class="line">        <span class="keyword">else</span>:</span><br><span class="line">            connect_succ_client_try.append(v)</span><br><span class="line">            print(<span class="string">"succ with retry"</span>)</span><br><span class="line">    print(<span class="string">"&#123;0&#125;<span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$</span><span class="variable">$end</span>\n\n"</span>.format(k))</span><br><span class="line"></span><br><span class="line">print(<span class="string">"\ntotal client:&#123;0&#125; connect success client size:&#123;1&#125;"</span>.format(total_size, len(connect_succ_client)))</span><br><span class="line">print(<span class="string">"\ntotal client:&#123;0&#125; connect success client normal handshake size:&#123;1&#125;"</span>.format(total_size, len(connect_succ_client_normal)))</span><br><span class="line">print(<span class="string">"\ntotal client:&#123;0&#125; connect success client after retry handshake size:&#123;1&#125;"</span>.format(total_size, len(connect_succ_client_try)))</span><br><span class="line">print(<span class="string">"\ntotal client:&#123;0&#125; connect fail client size:&#123;1&#125;"</span>.format(total_size, len(connect_fail_client)))</span><br></pre></td></tr></table></figure><p>运行后得到结果：</p><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">$</span> python tcpdump_analyze.py</span><br><span class="line"></span><br><span class="line">(省略部分输出）</span><br><span class="line"></span><br><span class="line"><span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.20409</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span>begin</span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">12.030247</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.20409</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span>: Flags [S], seq <span class="number">2040611302</span>, win <span class="number">29200</span>, <span class="keyword">options</span> [mss <span class="number">1460</span>,sackOK,TS val <span class="number">166083457</span> ecr <span class="number">0</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">13.033419</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.20409</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span>: Flags [S], seq <span class="number">2040611302</span>, win <span class="number">29200</span>, <span class="keyword">options</span> [mss <span class="number">1460</span>,sackOK,TS val <span class="number">166084460</span> ecr <span class="number">0</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">13.033661</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.20409</span>: Flags [S.], seq <span class="number">3015149333</span>, ack <span class="number">2040611303</span>, win <span class="number">65160</span>, <span class="keyword">options</span> [mss <span class="number">1460</span>,sackOK,TS val <span class="number">3009296915</span> ecr <span class="number">166084460</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">13.033667</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.20409</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span>: Flags [.], ack <span class="number">1</span>, win <span class="number">29200</span>, <span class="keyword">options</span> [nop,nop,TS val <span class="number">166084460</span> ecr <span class="number">3009296915</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line">succ with retry</span><br><span class="line"><span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.20409</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span>end</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.54379</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span>begin</span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">21.047376</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.54379</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span>: Flags [S], seq <span class="number">2685382859</span>, win <span class="number">29200</span>, <span class="keyword">options</span> [mss <span class="number">1460</span>,sackOK,TS val <span class="number">166092474</span> ecr <span class="number">0</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">21.047514</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.54379</span>: Flags [S.], seq <span class="number">1736229374</span>, ack <span class="number">2685382860</span>, win <span class="number">65160</span>, <span class="keyword">options</span> [mss <span class="number">1460</span>,sackOK,TS val <span class="number">3009304929</span> ecr <span class="number">166092474</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="number">15</span>:<span class="number">43</span>:<span class="number">21.047528</span> IP <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.54379</span> &gt; <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span><span class="number">.10000</span>: Flags [.], ack <span class="number">1</span>, win <span class="number">29200</span>, <span class="keyword">options</span> [nop,nop,TS val <span class="number">166092474</span> ecr <span class="number">3009304929</span>], length <span class="number">0</span></span><br><span class="line"></span><br><span class="line">succ <span class="keyword">no</span> retry</span><br><span class="line"><span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span><span class="number">.54379</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span><span class="symbol">$</span>end</span><br><span class="line"></span><br><span class="line">total client:<span class="number">30000</span> connect success client size:<span class="number">29736</span></span><br><span class="line"></span><br><span class="line">total client:<span class="number">30000</span> connect success client <span class="built-in">normal</span> handshake size:<span class="number">29195</span></span><br><span class="line"></span><br><span class="line">total client:<span class="number">30000</span> connect success client after retry handshake size:<span class="number">541</span></span><br><span class="line"></span><br><span class="line">total client:<span class="number">30000</span> connect fail client size:<span class="number">264</span></span><br></pre></td></tr></table></figure><p>上面的意思是<strong>总共有 29736 个 client 成功建立连接，而有 264 个 client 建立失败；连接成功的 client 里有 29195 个是通过了正常的 3 次握手成功建立，没有发生重试；而有 541 个 client 是发生了重试的情况下才建立连接成功</strong>。</p><p>可以看到上面的输出，发生重试 “succ with retry” 的部分，client 发送一个 SYN 之后，由于 server 全连接队列溢出导致连接被丢弃，client 超时后重新发送 SYN 包，然后建立连接；</p><p>而上面连接失败的客户端，错误原因都是: errno = 110，也就是 “Connection timed out”。</p><p>Ok，到现在应该明白全连接队列大小对于 tcp 3 次握手的影响，如果全连接队列过小，一旦发生溢出，就会影响后续的连接。</p><h3 id="调整内核参数，避免全连接队列溢出"><a href="#调整内核参数，避免全连接队列溢出" class="headerlink" title="调整内核参数，避免全连接队列溢出"></a>调整内核参数，避免全连接队列溢出</h3><p>那我们修改一下 backlog 的大小，改大一些：</p><figure class="highlight lisp"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">listen(<span class="name">listenfd</span>,  <span class="number">100000</span>)<span class="comment">;</span></span><br></pre></td></tr></table></figure><p>然后我们修改内核参数：</p><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">net.core.netdev_max_backlog</span> = <span class="number">400000</span></span><br><span class="line"><span class="attr">net.core.somaxconn</span> = <span class="number">100000</span></span><br></pre></td></tr></table></figure><p>可以通过打开 <code>/etc/sysctl.conf</code> 直接修改，或是通过命令修改：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sysctl -w net.core.<span class="attribute">netdev_max_backlog</span>=400000</span><br></pre></td></tr></table></figure><p>重新编译运行，执行上述的压测，观察结果。</p><p>压测前：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ ss -natl |grep <span class="number">10000</span></span><br><span class="line">LISTEN  <span class="number">0</span>        <span class="number">100000</span>       <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>          <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span>:*              </span><br><span class="line">$ netstat  -s |grep -i overflowed</span><br><span class="line">    <span class="number">3118</span> times the listen queue of a socket overflowed</span><br></pre></td></tr></table></figure><p>压测后：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">$ netstat  -s |grep -i overflowed</span><br><span class="line">    3118 times the listen<span class="built_in"> queue </span>of a socket overflowed</span><br><span class="line">    </span><br><span class="line">$ python tcpdump_analyze.py</span><br><span class="line">(省略部分输出）</span><br><span class="line">total client:30000 connect success<span class="built_in"> client </span>size:30000</span><br><span class="line"></span><br><span class="line">total client:30000 connect success<span class="built_in"> client </span>normal handshake size:30000</span><br><span class="line"></span><br><span class="line">total client:30000 connect success<span class="built_in"> client </span>after retry handshake size:0</span><br><span class="line"></span><br><span class="line">total client:30000 connect fail<span class="built_in"> client </span>size:0</span><br></pre></td></tr></table></figure><p>可以看到，<strong>当我们把内核参数以及 backlog 调大之后，30000 个 client 全部建立连接成功且没有发生重试，服务端的 listen queue 没有发生溢出</strong>。</p><h2 id="半连接队列"><a href="#半连接队列" class="headerlink" title="半连接队列"></a>半连接队列</h2><p>全连接队列存放的是已经完成 3次握手，等待应用层调用 <code>accept()</code> 处理这些连接；<strong>其实还有一个半连接队列，当服务端收到客户端的 SYN 包后，并且回复 SYN+ACK包后，服务端进入 SYN_RECV 状态，此时这种连接称为半连接，会被存放到半连接队列，当完成 3 次握手之后，tcp 会把这个连接从半连接队列中移到全连接队列，然后等待应用层处理</strong>。</p><p>那么怎么查看半连接队列的大小呢？没有直接的 linux command 来查询半连接队列的长度，但是根据上面的定义，<strong>服务端处于 SYN_RECV 状态的数量就表示半连接的数量。所以采用一定的方式增大半连接的数量，看服务端 SYN_RECV 的数量最大值有多少，那就是半连接队列的大小</strong>。</p><p>那问题就来了，如何增大半连接的数量呢？这里采用到的就是 <strong>SYN-FLOOD</strong> 攻击，通过发送大量的 SYN 包而不进行回应，造成服务端创建了大量的半连接，但是这些半连接不会被确认，最终把 tcp 半连接队列占满造成溢出，并影响正常的连接。</p><h3 id="半连接队列溢出"><a href="#半连接队列溢出" class="headerlink" title="半连接队列溢出"></a>半连接队列溢出</h3><p>采用的工具是: <strong>hping3</strong>，一款很强大的工具。</p><p>启动服务端：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span> 作为服务端，监听 <span class="number">10000</span> 端口</span><br><span class="line"></span><br><span class="line">$ ./bench_server <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span> <span class="number">10000</span></span><br></pre></td></tr></table></figure><p>开始攻击：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hping3 -S  --flood --rand-source -p <span class="number">10000</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span></span><br></pre></td></tr></table></figure><p>观察半连接数量：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ netstat -ant |grep SYN</span><br><span class="line">(省略)</span><br><span class="line">tcp        <span class="number">0</span>      <span class="number">0</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>       <span class="number">152.66</span><span class="number">.128</span><span class="number">.1</span>:<span class="number">48581</span>      SYN_RECV   </span><br><span class="line">tcp        <span class="number">0</span>      <span class="number">0</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>       <span class="number">208.220</span><span class="number">.119</span><span class="number">.30</span>:<span class="number">57972</span>    SYN_RECV   </span><br><span class="line">tcp        <span class="number">0</span>      <span class="number">0</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.7</span>:<span class="number">10000</span>       <span class="number">3.104</span><span class="number">.166</span><span class="number">.109</span>:<span class="number">25975</span>     SYN_RECV </span><br><span class="line"></span><br><span class="line">$ netstat -ant |grep SYN |wc -l</span><br><span class="line"><span class="number">256</span></span><br></pre></td></tr></table></figure><p>持续观察，可以看到处于 <code>SYN_RECV</code> 状态的连接基本保持在 256，说明半连接队列的大小是 256。而此时，10000 端口已经比较难连接上了。 </p><p>查看一下半连接队列的丢弃情况：</p><figure class="highlight pgsql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ netstat  -s |grep dropped</span><br><span class="line">    <span class="number">26055883</span> SYNs <span class="keyword">to</span> <span class="keyword">LISTEN</span> sockets dropped</span><br></pre></td></tr></table></figure><blockquote><p>注意： 26055883 是一个累加值，可以持续观察</p></blockquote><p>那怎么增大半连接队列大小呢？</p><h3 id="增大半连接队列，防止溢出"><a href="#增大半连接队列，防止溢出" class="headerlink" title="增大半连接队列，防止溢出"></a>增大半连接队列，防止溢出</h3><p>直接修改内核参数：</p><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 直接修改文件 /etc/sysctl.conf</span></span><br><span class="line"></span><br><span class="line"><span class="attr">net.ipv4.tcp_max_syn_backlog</span> = <span class="number">100000</span></span><br></pre></td></tr></table></figure><p>或者使用命令：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sysctl -w net.ipv4.<span class="attribute">tcp_max_syn_backlog</span>=100000</span><br></pre></td></tr></table></figure><blockquote><p>据说半连接队列并非只由这个参数决定，不同的系统的计算方式不一致，还会和全连接队列大小有关</p></blockquote><p>当然这个应对 SYN-Flood 攻击只是轻微降低影响而已。</p><p>还可以设置 <code>net.ipv4.tcp_syncookies = 0</code> 来一定程度防范 SYN 攻击。</p><p><strong>syncookies 的原理就是当服务端收到客户端 SYN 包后，不会放到半连接队列里，而是通过 {src_ip, src_port, timestamp} 等计算一个 cookie（也就是一个哈希值），通过 SYN+ACK包返回给客户端，客户端返回一个 ACK 包，携带上这个 cookie，服务端通过校验可以直接把这个连接放入全连接队列。整个过程不需要半连接队列的参与</strong>。</p><h2 id="SYN-重试"><a href="#SYN-重试" class="headerlink" title="SYN 重试"></a>SYN 重试</h2><p>上面压测验证全连接队列溢出的场景下，通过 tcpdump 抓包分析到有些连接是经过了重试才建立成功的，具体表现在：</p><p>客户端发送 SYN 包请求建立连接，但此时由于服务端全连接队列溢出或者半连接队列溢出，该 SYN 包就会被丢弃，当客户端迟迟无法收到服务端的 SYN+ACK 包后，客户端超时重发 SYN 包，如果再次超时，那么根据内核设置的 SYN 超时重试次数决定是否继续重发 SYN 包。</p><p>假设重试次数为 6 次：</p><ul><li>第一次发送 SYN 后等待 1 s （2^0);</li><li>第二次发送 SYN 后等待 2 s  (2^1);</li><li>第三次发送 SYN 后等待 4 s  (2^1);</li><li>…</li></ul><p>所以当我们发现服务端出现了问题的时候，可以适当提高 SYN 重试的次数；当然过大的值也会影响问题的快速发现；</p><p>可以通过设置：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sysctl -w net.ipv4.<span class="attribute">tcp_syn_retries</span>=2</span><br></pre></td></tr></table></figure><h1 id="The-End"><a href="#The-End" class="headerlink" title="The End"></a>The End</h1><p>Ok, 到这里基本上把 tcp  3 次握手比较细节的地方讲到了。 tcp 真是一个巨复杂的协议，还有不少值得深挖的东西！</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#x69;&#110;&#117;&#120;&#x63;&#x6f;&#100;&#101;&#x32;&#110;&#x69;&#x6b;&#105;&#64;&#103;&#109;&#x61;&#105;&#108;&#46;&#x63;&#111;&#109;">&#x6c;&#x69;&#110;&#117;&#120;&#x63;&#x6f;&#100;&#101;&#x32;&#110;&#x69;&#x6b;&#105;&#64;&#103;&#109;&#x61;&#105;&#108;&#46;&#x63;&#111;&#109;</a></p></li></ul><p>2020-11-14 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;从何说起&quot;&gt;&lt;a href=&quot;#从何说起&quot; class=&quot;headerlink&quot; title=&quot;从何说起&quot;&gt;&lt;/a&gt;从何说起&lt;/h1&gt;&lt;p&gt;说起 tcp 的连接过程，想必 “&lt;strong&gt;3次握手4次挥手&lt;/strong&gt;”是大家广为熟知的知识，那么关于更细节更底层的连接过程也许就很少人能讲清楚了。&lt;/p&gt;
&lt;p&gt;所以本文会先简单回顾一下 tcp 的 3次握手过程，然后重点聊一下 tcp accept 的过程，涉及到 tcp 半连接队列、全连接队列等的内容。&lt;/p&gt;
&lt;h1 id=&quot;回顾一下&quot;&gt;&lt;a href=&quot;#回顾一下&quot; class=&quot;headerlink&quot; title=&quot;回顾一下&quot;&gt;&lt;/a&gt;回顾一下&lt;/h1&gt;&lt;h2 id=&quot;3-次握手&quot;&gt;&lt;a href=&quot;#3-次握手&quot; class=&quot;headerlink&quot; title=&quot;3 次握手&quot;&gt;&lt;/a&gt;3 次握手&lt;/h2&gt;&lt;p&gt;要了解 3 次握手的过程，可能需要先熟悉一下 tcp 协议的格式：&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/tcp_accept/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;tcp segment 的头部有两个 2字节的字段 &lt;code&gt;source port&lt;/code&gt; 和 &lt;code&gt;dest port&lt;/code&gt;，分别表示本机端口以及目标端口，在 tcp 传输层是没有 IP 的概念的，那是 IP 层 的概念，IP 层协议会在 IP 协议的头部加上 &lt;code&gt;src ip&lt;/code&gt; 和 &lt;code&gt;dest ip&lt;/code&gt;；&lt;/li&gt;
&lt;li&gt;4 个字节的 seq，表示序列号，tcp 是可靠连接，不会乱序；&lt;/li&gt;
&lt;li&gt;4 个字节的 ack，表示确认号，表示对接收到的上一个报文的确认，值为 seq + 1;&lt;/li&gt;
&lt;li&gt;几个标志位：ACK,RST,SYN,FIN 这些是我们常用的，比较熟悉的。其中 ACK 简写为 “.”; RST 简写为 “R”; SYN 简写为 “S”; FIN 简写为 “F”;&lt;/li&gt;
&lt;/ul&gt;
&lt;blockquote&gt;
&lt;p&gt;注意： ack 和 ACK 是不一样的意思，一个是确认号，一个是标志位&lt;/p&gt;
&lt;/blockquote&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="python" scheme="https://rebootcat.com/tags/python/"/>
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="epoll" scheme="https://rebootcat.com/tags/epoll/"/>
    
      <category term="socket" scheme="https://rebootcat.com/tags/socket/"/>
    
      <category term="tcp" scheme="https://rebootcat.com/tags/tcp/"/>
    
      <category term="accept" scheme="https://rebootcat.com/tags/accept/"/>
    
      <category term="backlog" scheme="https://rebootcat.com/tags/backlog/"/>
    
      <category term="listen_queue" scheme="https://rebootcat.com/tags/listen-queue/"/>
    
      <category term="mux" scheme="https://rebootcat.com/tags/mux/"/>
    
      <category term="hping3" scheme="https://rebootcat.com/tags/hping3/"/>
    
      <category term="tcpdump" scheme="https://rebootcat.com/tags/tcpdump/"/>
    
      <category term="syn" scheme="https://rebootcat.com/tags/syn/"/>
    
      <category term="flood" scheme="https://rebootcat.com/tags/flood/"/>
    
      <category term="listen" scheme="https://rebootcat.com/tags/listen/"/>
    
      <category term="syncookies" scheme="https://rebootcat.com/tags/syncookies/"/>
    
      <category term="synretries" scheme="https://rebootcat.com/tags/synretries/"/>
    
  </entry>
  
  <entry>
    <title>博客大事记之迁移博客到香港主机</title>
    <link href="https://rebootcat.com/2020/11/10/move_blog_hk/"/>
    <id>https://rebootcat.com/2020/11/10/move_blog_hk/</id>
    <published>2020-11-10T03:23:58.000Z</published>
    <updated>2022-08-20T04:24:33.933Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>之前其实已经写过一篇博文： <a href="https://rebootcat.com/2020/09/20/virtual_space_blog/">迁移博客到香港虚拟空间</a>，那为什么又要写这篇博客呢？</p><p>上次其实是把我的博客迁移到一个香港的虚拟空间里，但是不到半年的时间已经出现过 4 次宕机事件，每次持续时间 4~5 小时，<a href="https://cloudmonitor.console.aliyun.com/" target="_blank" rel="noopener">阿里云</a> 和 <a href="https://uptimerobot.com/dashboard" target="_blank" rel="noopener">UpTimeRobot</a> 的监控报警报了一大堆，邮箱都快塞满了。想着宕机就宕机吧，至少还能恢复，还能凑合用，结果呢，就在前几天当时购买虚拟空间的官网都 GG 了，管理员跑路了。。。</p><p>可能他没挣到钱吧，买一台服务器打算开很多共享的虚拟空间来卖，可能也只有我买了一个，因为我后来看了下我的博客同 IP 的网站就两个，好嘛，结果就跑路了。。。这里就不点名是哪一家了，八字开头的一个云。</p><p>好吧，言归正传，正好双 11，那就干脆直接买服务器吧，所以就购买了腾讯的一台轻量级云服务器，峰值 30Mbps，月流量 1024G，能满足我的需求，况且有了服务器，能做的事情就很多了。比如我还有其他的博客也可以解析到这里，比如可以定制化一些动态博客，比如可以使用自动化发布等。</p><p>那本文大致就记录下迁移的一些过程以及踩坑优化等：</p><ul><li><strong>服务器购买以及初始化</strong></li><li><strong>安装部署 nginx</strong></li><li><strong>部署博客源码</strong></li><li><strong>解析域名</strong></li><li><strong>设置 https 证书</strong></li><li><strong>绑定多个域名</strong></li><li><strong>使用 github actions 自动化部署博客（踩坑）</strong></li><li><strong>https 性能优化</strong></li></ul><a id="more"></a><h1 id="服务器购买及初始化"><a href="#服务器购买及初始化" class="headerlink" title="服务器购买及初始化"></a>服务器购买及初始化</h1><p>双 11 活动，购买了一台轻量级的腾讯云服务器。然后就是初始化服务器，登录服务器，设置 ssh key 登录等。</p><p>注意这里一定要设置 ssh key 登录，因为后面用的到。</p><h1 id="安装部署-nginx"><a href="#安装部署-nginx" class="headerlink" title="安装部署 nginx"></a>安装部署 nginx</h1><p>安装</p><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ yum <span class="keyword">install</span> nginx -y</span><br></pre></td></tr></table></figure><p>启动 </p><figure class="highlight crmsh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ systemctl <span class="literal">start</span> nginx</span><br></pre></td></tr></table></figure><p>然后浏览器访问：</p><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">$</span> curl http:<span class="comment">//your_public_ip</span></span><br></pre></td></tr></table></figure><p>如果一切正常，说明 nginx 启动正常。接下来把  nginx 添加到系统启动项随开机启动：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ systemctl <span class="builtin-name">enable</span> nginx.service</span><br></pre></td></tr></table></figure><h1 id="部署博客源码"><a href="#部署博客源码" class="headerlink" title="部署博客源码"></a>部署博客源码</h1><p>博客采用的是 hexo 生成的静态博客，所以只需要把博客仓库克隆下来就行：</p><p>安装 git</p><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ yum <span class="keyword">install</span> git -y</span><br></pre></td></tr></table></figure><p>克隆博客网站源码到某个目录：</p><figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ git clone https:<span class="regexp">//gi</span>thub.com<span class="regexp">/smaugx/</span>smaugx.github.io.git  <span class="regexp">/root/</span></span><br></pre></td></tr></table></figure><p>设置 nginx 配置文件中 80 端口的 root 为博客源码的目录：</p><figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># nginx.conf</span></span><br><span class="line">root         <span class="regexp">/usr/</span>share<span class="regexp">/nginx/</span>html<span class="regexp">/smaugx.github.io;</span></span><br></pre></td></tr></table></figure><p>重启 nginx:</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$ </span>systemctl restart nginx</span><br></pre></td></tr></table></figure><p>验证博客是否正常：</p><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">$</span> curl http:<span class="comment">//your_public_ip</span></span><br></pre></td></tr></table></figure><p>正常能看到博客的主页了。</p><h1 id="解析域名"><a href="#解析域名" class="headerlink" title="解析域名"></a>解析域名</h1><p>接下来是把域名 <code>rebootcat.com</code> 解析到这台机器上， 如下：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">主机|类型|线路|记录值|MX优先级|TTL|备注|状态</span><br><span class="line">@NS默认f1g1ns1.dnspod.net.<span class="number">0</span><span class="number">86400</span>正常</span><br><span class="line">@NS默认f1g1ns2.dnspod.net.<span class="number">0</span><span class="number">86400</span>正常</span><br><span class="line">@A默认<span class="number">101.33</span><span class="number">.123</span><span class="number">.30</span>    <span class="number">0</span><span class="number">600</span>正常</span><br><span class="line">@TXT默认google-site-xxx<span class="number">86400</span>正常</span><br><span class="line">wwwA默认<span class="number">101.33</span><span class="number">.123</span><span class="number">.30</span>      <span class="number">0</span><span class="number">600</span>正常</span><br><span class="line">@TXT默认mP8ROM8AEYs9Zxxxx<span class="number">0</span><span class="number">600</span>正常</span><br></pre></td></tr></table></figure><p>解析生效之后，验证是否成功：</p><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">$</span> curl http:<span class="comment">//rebootcat.com</span></span><br></pre></td></tr></table></figure><h1 id="设置-https-证书"><a href="#设置-https-证书" class="headerlink" title="设置 https 证书"></a>设置 https 证书</h1><p>全程参考这篇博文：</p><p><a href="https://ruby-china.org/topics/31942" target="_blank" rel="noopener">Linux CentOS 7 下 Nginx 安装使用 Let’ s Encrypt 证书的完整过程</a></p><p>这篇文章已经写的很清楚了，照着操作就行。</p><p>设置完成应该就能使用 https 访问了：</p><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">$</span> curl https:<span class="comment">//rebootcat.com</span></span><br></pre></td></tr></table></figure><h1 id="绑定多个域名"><a href="#绑定多个域名" class="headerlink" title="绑定多个域名"></a>绑定多个域名</h1><p>我还把 <a href="https://loveyxq.online" target="_blank" rel="noopener">loveyxq.online</a> 也解析到了这台机器上，这是我另外一个博客，给我女朋友用的一个。</p><p>nginx 的配置见后文。</p><h1 id="使用-github-actions-自动化部署博客（踩坑）"><a href="#使用-github-actions-自动化部署博客（踩坑）" class="headerlink" title="使用 github actions 自动化部署博客（踩坑）"></a>使用 github actions 自动化部署博客（踩坑）</h1><p>经过了上面的步骤，博客已经算是迁移完成了，不过每次更新博客能否直接部署道这台机器上呢？</p><p>答案是能的，而且方法很多种。我采用的是 github 自家的持续部署工具 Github Actions.</p><h2 id="添加-github-仓库配置"><a href="#添加-github-仓库配置" class="headerlink" title="添加 github 仓库配置"></a>添加 github 仓库配置</h2><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/move_blog_hk/1.png" alt=""></p><p>如上图所示，分别在 <code>Secrets</code> 项添加 3 个变量：</p><ul><li>BLOG_DEPLOY_PRI_KEY : ssh 私钥</li><li>BLOG_HOSTNAME :  rebootcat.com</li><li>BLOG_USER : user</li></ul><blockquote><p>注意，优于 github action 会使用上面的 sshkey 登录并推送博客，所以为了安全，建议单独生成用户，并设置单独的目录以及权限，只允许访问 nginx root path.</p></blockquote><h2 id="编写工作流文件"><a href="#编写工作流文件" class="headerlink" title="编写工作流文件"></a>编写工作流文件</h2><p>在博客网站源码仓库创建文件：<code>.github/workflows/deploy.yml</code>，内容如下：</p><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">name</span>: Deploy site files</span><br><span class="line"></span><br><span class="line"><span class="attribute">on:</span></span><br><span class="line">  push:</span><br><span class="line">    branches:</span><br><span class="line">      - master  # 只在master上push触发部署</span><br><span class="line">    paths-ignore:   # 下列文件的变更不触发部署，可以自行添加</span><br><span class="line">      - LICENSE</span><br><span class="line"></span><br><span class="line"><span class="attribute">jobs:</span></span><br><span class="line">  deploy:</span><br><span class="line"></span><br><span class="line">    runs-on: ubuntu-latest   # 使用ubuntu系统镜像运行自动化脚本</span><br><span class="line"></span><br><span class="line">    steps:  # 自动化步骤</span><br><span class="line">    - uses: actions/checkout@v2   # 第一步，下载代码仓库</span><br><span class="line"></span><br><span class="line">    - name: Deploy to Server  # 第二步，rsync推文件</span><br><span class="line">      uses: AEnterprise/rsync-deploy@v1.0  # 使用别人包装好的步骤镜像</span><br><span class="line">      env:</span><br><span class="line">        DEPLOY_KEY: $&#123;&#123; secrets.BLOG_DEPLOY_PRI_KEY &#125;&#125;   # 引用配置，SSH私钥</span><br><span class="line">        ARGS: -avz --delete --exclude='*.pyc'   # rsync参数，排除.pyc文件</span><br><span class="line">        SERVER_PORT: '22'  # SSH端口</span><br><span class="line">        FOLDER: ./  # 要推送的文件夹，路径相对于代码仓库的根目录</span><br><span class="line">        SERVER_IP: $&#123;&#123; secrets.BLOG_HOSTNAME &#125;&#125;  # 引用配置，服务器的host名（IP或者域名domain.com）</span><br><span class="line">        USERNAME: $&#123;&#123; secrets.BLOG_USER &#125;&#125;  # 引用配置，服务器登录名</span><br><span class="line">        SERVER_DESTINATION: /usr/share/nginx/html/smaugx.github.io/   # 部署到目标文件夹</span><br><span class="line">    - name: Restart server   # 第三步，重启服务</span><br><span class="line">      uses: appleboy/ssh-action@master</span><br><span class="line">      with:</span><br><span class="line">        host: $&#123;&#123; secrets.BLOG_HOSTNAME &#125;&#125;  # 下面三个配置与上一步类似</span><br><span class="line">        username: $&#123;&#123; secrets.BLOG_USER &#125;&#125;</span><br><span class="line">        key: $&#123;&#123; secrets.BLOG_DEPLOY_PRI_KEY &#125;&#125;</span><br><span class="line">        # 重启的脚本，根据自身情况做相应改动，一般要做的是migrate数据库以及重启服务器</span><br><span class="line">        script: |</span><br><span class="line">          echo "update rebootcat.com blog" &gt;&gt; /tmp/github/blog.log</span><br></pre></td></tr></table></figure><p>上述文件记得 push 到远端仓库。然后你可以随便修改一下博客源码并且 push 到远端，正常的话应该能看到如下的输出：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/move_blog_hk/2.png" alt=""></p><h2 id="hexo-deploy-踩坑"><a href="#hexo-deploy-踩坑" class="headerlink" title="hexo deploy 踩坑"></a>hexo deploy 踩坑</h2><p>重点来了，上面两部其实是经过了 <code>hexo deploy</code> 的踩坑的。为啥？</p><p>由于 <code>hexo generate</code> 默认会忽略隐藏文件，所以生成的网站源码就会忽略 <code>.github/workflows/deploy.yml</code>，所以要设置一下博客根目录的 <code>_config.yml</code>:</p><figure class="highlight autoit"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="meta"># <span class="meta-keyword">Include</span> / Exclude file(s)</span></span><br><span class="line"><span class="meta">## <span class="meta-keyword">include</span>:/exclude: options only apply to the <span class="string">'source/'</span> folder</span></span><br><span class="line">include:</span><br><span class="line">  - <span class="string">".github/*"</span></span><br><span class="line">  - <span class="string">".github/**/*"</span></span><br><span class="line">exclude:</span><br><span class="line">ignore:</span><br></pre></td></tr></table></figure><p><strong>很重要</strong>！！！</p><p>OK，现在你可以放心大胆的使用  <code>hexo generate</code> 来生成博客源码了，但是当你使用 <code>hexo deploy</code> 的时候问题又来了， <code>hexo deploy</code> 默认也是忽略隐藏文件的，而且好像上面那个配置对 <code>hexo deploy</code> 无效。</p><p>搜索了很多，没有找到针对 <code>hexo deploy</code> 如何避免忽略隐藏文件的解决方案，于是探索了一下：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ hexo deploy</span><br><span class="line"><span class="builtin-name">INFO</span>  Deploying: git</span><br><span class="line"><span class="builtin-name">INFO</span>  Clearing .deploy_git folder<span class="built_in">..</span>.</span><br><span class="line"><span class="builtin-name">INFO</span>  Copying files <span class="keyword">from</span> public folder<span class="built_in">..</span>.</span><br><span class="line"><span class="builtin-name">INFO</span>  Copying files <span class="keyword">from</span> extend dirs<span class="built_in">..</span>.</span><br><span class="line">On branch master</span><br><span class="line"><span class="literal">nothing</span> <span class="keyword">to</span> commit, working tree clean</span><br><span class="line"><span class="built_in">..</span>.</span><br></pre></td></tr></table></figure><p>可以看到，上面执行 <code>hexo deploy</code> 命令后的输出有一个  <strong>“.deploy_git folder”</strong>，看了一下真有这个隐藏目录，想必 <code>hexo deploy</code> 是把 <code>public</code> 目录与较旧的（上一次发布的）目录 <code>.deploy_git</code> 做比较，然后增量上传文件。</p><p>所以我直接把 <code>.github/workflows/deploy.yml</code> 拷贝到了 <code>.deploy_git</code> 目录，然后执行  <code>hexo deploy</code> 成功。</p><p>哈哈哈！！！</p><p>所以记住，<strong>如果后期修改了这个 deploy.yml ，需要手动拷贝一下</strong>，但是基本上不会再动这个文件了。 </p><p>到这里，基本上就解决了<strong>利用github actions 自动化部署博客</strong>的问题了。</p><p>实测 push 仓库后到服务器上的网站源码成功替换时间很快，大概一分钟左右，Good!</p><h1 id="https-性能优化"><a href="#https-性能优化" class="headerlink" title="https 性能优化"></a>https 性能优化</h1><p>上面的一切搞定后，体验了一天访问我的博客 <a href="https://rebootcat.com">https://rebootcat.com</a>，使用 chrome 控制台发现 ssl 握手时间很慢，第一次访问基本都要 3 ~ 4 s左右，无法忍受，再次访问就快了。</p><p>所以网上搜索了下关于 Let’s Encrypt 的优化，找到了一些解决方案以及 nginx 的配置优化等：</p><figure class="highlight nginx"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># config file name: /etc/nginx/nginx.conf</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="attribute">user</span>  root;</span><br><span class="line"><span class="attribute">worker_processes</span>  <span class="number">2</span>;</span><br><span class="line"></span><br><span class="line"><span class="attribute">error_log</span>  /var/log/nginx/error.log;</span><br><span class="line"><span class="comment">#error_log  logs/error.log  notice;</span></span><br><span class="line"><span class="comment">#error_log  logs/error.log  info;</span></span><br><span class="line"></span><br><span class="line"><span class="attribute">pid</span>         /var/run/nginx.pid;</span><br><span class="line"></span><br><span class="line"><span class="comment"># Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.</span></span><br><span class="line"><span class="attribute">include</span> /usr/share/nginx/modules/<span class="regexp">*.conf</span>;</span><br><span class="line"></span><br><span class="line"><span class="section">events</span> &#123;</span><br><span class="line">    <span class="attribute">worker_connections</span>  <span class="number">1024</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="section">http</span> &#123;</span><br><span class="line">    <span class="attribute">include</span>             /etc/nginx/mime.types;</span><br><span class="line">    <span class="attribute">default_type</span>        application/octet-stream;</span><br><span class="line"></span><br><span class="line">    <span class="attribute">log_format</span>  main  <span class="string">'<span class="variable">$remote_addr</span> - <span class="variable">$remote_user</span> [<span class="variable">$time_local</span>] "<span class="variable">$request</span>" '</span></span><br><span class="line">                      <span class="string">'<span class="variable">$status</span> <span class="variable">$body_bytes_sent</span> "<span class="variable">$http_referer</span>" '</span></span><br><span class="line">                      <span class="string">'"<span class="variable">$http_user_agent</span>" "<span class="variable">$http_x_forwarded_for</span>"'</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">#access_log  /var/log/nginx/access.log  main;</span></span><br><span class="line"></span><br><span class="line">    <span class="attribute">sendfile</span>        <span class="literal">on</span>;</span><br><span class="line">    <span class="attribute">tcp_nopush</span>      <span class="literal">on</span>;</span><br><span class="line">    <span class="attribute">tcp_nodelay</span>     <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">#keepalive_timeout  0;</span></span><br><span class="line">    <span class="attribute">keepalive_timeout</span>  <span class="number">65</span>;</span><br><span class="line">    <span class="attribute">types_hash_max_size</span> <span class="number">2048</span>;</span><br><span class="line"></span><br><span class="line">    <span class="attribute">gzip</span>  <span class="literal">on</span>;</span><br><span class="line">    <span class="attribute">gzip_vary</span> <span class="literal">on</span>;</span><br><span class="line">    <span class="attribute">gzip_min_length</span> <span class="number">1k</span>; <span class="comment">#不压缩临界值,大于1k的才压缩,一般不用改</span></span><br><span class="line">    <span class="attribute">gzip_buffers</span> <span class="number">4</span> <span class="number">16k</span>;</span><br><span class="line">    <span class="attribute">gzip_comp_level</span> <span class="number">6</span>; <span class="comment">#压缩级别,数字越大压缩的越好</span></span><br><span class="line">    <span class="attribute">gzip_types</span>  text/plain application/javascript application/x-javascript application/json text/css application/xml text/javascript application/x-httpd-php image/jpeg image/gif image/png image/x-icon text/xml;</span><br><span class="line"></span><br><span class="line">    <span class="comment">#记录160000个请求 超过将返回失败</span></span><br><span class="line">    <span class="attribute">limit_conn_zone</span> <span class="variable">$binary_remote_addr</span> zone=addr:<span class="number">10m</span>;</span><br><span class="line">    <span class="comment">#单个请求小于20r/s</span></span><br><span class="line">    <span class="attribute">limit_req_zone</span> <span class="variable">$binary_remote_addr</span> zone=one:<span class="number">10m</span> rate=20r/s;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="section">server</span> &#123;</span><br><span class="line">        <span class="attribute">listen</span> <span class="number">80</span>;</span><br><span class="line">        <span class="attribute">server_name</span>  www.rebootcat.com rebootcat.com;</span><br><span class="line">        <span class="attribute">rewrite</span><span class="regexp"> ^</span> https://<span class="variable">$server_name</span><span class="variable">$request_uri</span>? <span class="literal">permanent</span>;</span><br><span class="line">        <span class="attribute">access_log</span>  /var/log/nginx/access_rebootcat.log  main;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="comment">#  HTTPS server</span></span><br><span class="line"></span><br><span class="line">    <span class="section">server</span> &#123;</span><br><span class="line">        <span class="attribute">listen</span> <span class="number">443</span> http2 ssl;</span><br><span class="line">        <span class="attribute">server_name</span>  www.rebootcat.com rebootcat.com;</span><br><span class="line">        <span class="attribute">access_log</span>  /var/log/nginx/access_rebootcat.log  main;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">charset</span> utf-<span class="number">8</span>;</span><br><span class="line">        <span class="attribute">root</span>         /usr/share/nginx/html/smaugx.github.io;</span><br><span class="line">        <span class="attribute">limit_conn</span> addr <span class="number">10</span>;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> /api &#123;</span><br><span class="line">            <span class="attribute">limit_req</span> zone=one burst=<span class="number">10</span>;</span><br><span class="line"></span><br><span class="line">            <span class="comment"># 真实的客户端IP</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   X-Real-IP        <span class="variable">$remote_addr</span>;</span><br><span class="line">            <span class="comment"># 请求头中Host信息</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   Host             <span class="variable">$host</span>;</span><br><span class="line">            <span class="comment"># 代理路由信息，此处取IP有安全隐患</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   X-Forwarded-For  <span class="variable">$proxy_add_x_forwarded_for</span>;</span><br><span class="line">            <span class="comment"># 真实的用户访问协议</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   X-Forwarded-Proto <span class="variable">$scheme</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> ~.*\.(js|css|ico|png|jpg)$</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="attribute">expires</span>    <span class="number">3d</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> /js</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="comment">#add_header Cache-Control no-cache;</span></span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> / &#123;</span><br><span class="line">            <span class="attribute">limit_req</span> zone=one burst=<span class="number">30</span>;</span><br><span class="line">            <span class="attribute">index</span>  index.html index.htm;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">error_page</span>  <span class="number">404</span>              /<span class="number">404</span>.html;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># redirect server error pages to the static page /50x.html</span></span><br><span class="line">        <span class="attribute">error_page</span>   <span class="number">500</span> <span class="number">502</span> <span class="number">503</span> <span class="number">504</span>  /50x.html;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启 ssl</span></span><br><span class="line">        <span class="comment">#ssl on;</span></span><br><span class="line"></span><br><span class="line">        <span class="comment"># letsencrypt生成的文件</span></span><br><span class="line">        <span class="attribute">ssl_certificate</span> /etc/letsencrypt/live/rebootcat.com/fullchain.pem;</span><br><span class="line">        <span class="attribute">ssl_certificate_key</span> /etc/letsencrypt/live/rebootcat.com/privkey.pem;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">ssl_session_timeout</span> <span class="number">20m</span>;</span><br><span class="line">        <span class="attribute">ssl_session_cache</span> shared:SSL:<span class="number">50m</span>;</span><br><span class="line">        <span class="comment"># 由客户端保存加密后的session信息</span></span><br><span class="line">        <span class="attribute">ssl_session_tickets</span> <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">ssl_dhparam</span> /etc/ssl/private/dhparam.pem;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启OCSP Stapling，由服务器验证证书在线状态，提高TLS握手效率</span></span><br><span class="line">        <span class="attribute">ssl_stapling</span> <span class="literal">on</span>;</span><br><span class="line">        <span class="attribute">ssl_stapling_verify</span> <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启HSTS，缓存http重定向到https，以防止中间人攻击</span></span><br><span class="line">        <span class="comment"># 不包含子域(宝塔界面是http的)</span></span><br><span class="line">        <span class="comment"># 不预加载(预加载要在https://hstspreload.org/中添加)</span></span><br><span class="line">        <span class="attribute">add_header</span> Strict-Transport-Security <span class="string">"max-age=63072000;"</span> always;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启TLS False Start</span></span><br><span class="line">        <span class="attribute">ssl_prefer_server_ciphers</span> <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">ssl_protocols</span> TLSv1 TLSv1.<span class="number">1</span> TLSv1.<span class="number">2</span> TLSv1.<span class="number">3</span>;</span><br><span class="line">        <span class="comment"># 一般推荐使用的ssl_ciphers值: https://wiki.mozilla.org/Security/Server_Side_TLS</span></span><br><span class="line">        <span class="attribute">ssl_ciphers</span> <span class="string">'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK'</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="section">server</span> &#123;</span><br><span class="line">        <span class="attribute">listen</span> <span class="number">80</span>;</span><br><span class="line">        <span class="attribute">server_name</span>  www.loveyxq.online loveyxq.online;</span><br><span class="line">        <span class="attribute">rewrite</span><span class="regexp"> ^</span> https://<span class="variable">$server_name</span><span class="variable">$request_uri</span>? <span class="literal">permanent</span>;</span><br><span class="line">        <span class="attribute">access_log</span>  /var/log/nginx/access_loveyxq.log  main;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="comment">#  HTTPS server</span></span><br><span class="line"></span><br><span class="line">    <span class="section">server</span> &#123;</span><br><span class="line">        <span class="attribute">listen</span> <span class="number">443</span> http2 ssl;</span><br><span class="line">        <span class="attribute">server_name</span>  www.loveyxq.online loveyxq.online;</span><br><span class="line">        <span class="attribute">access_log</span>  /var/log/nginx/access_loveyxq.log  main;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">charset</span> utf-<span class="number">8</span>;</span><br><span class="line">        <span class="attribute">root</span>         /usr/share/nginx/html/smaugx.github.io;</span><br><span class="line">        <span class="attribute">limit_conn</span> addr <span class="number">10</span>;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> /api &#123;</span><br><span class="line">            <span class="attribute">limit_req</span> zone=one burst=<span class="number">10</span>;</span><br><span class="line"></span><br><span class="line">            <span class="comment"># 真实的客户端IP</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   X-Real-IP        <span class="variable">$remote_addr</span>;</span><br><span class="line">            <span class="comment"># 请求头中Host信息</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   Host             <span class="variable">$host</span>;</span><br><span class="line">            <span class="comment"># 代理路由信息，此处取IP有安全隐患</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   X-Forwarded-For  <span class="variable">$proxy_add_x_forwarded_for</span>;</span><br><span class="line">            <span class="comment"># 真实的用户访问协议</span></span><br><span class="line">            <span class="attribute">proxy_set_header</span>   X-Forwarded-Proto <span class="variable">$scheme</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> ~.*\.(js|css|ico|png|jpg)$</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="attribute">expires</span>    <span class="number">3d</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> /js</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="comment">#add_header Cache-Control no-cache;</span></span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">location</span> / &#123;</span><br><span class="line">            <span class="attribute">limit_req</span> zone=one burst=<span class="number">30</span>;</span><br><span class="line">            <span class="attribute">index</span>  index.html index.htm;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">error_page</span>  <span class="number">404</span>              /<span class="number">404</span>.html;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># redirect server error pages to the static page /50x.html</span></span><br><span class="line">        <span class="attribute">error_page</span>   <span class="number">500</span> <span class="number">502</span> <span class="number">503</span> <span class="number">504</span>  /50x.html;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启 ssl</span></span><br><span class="line">        <span class="comment">#ssl on;</span></span><br><span class="line"></span><br><span class="line">        <span class="comment"># letsencrypt生成的文件</span></span><br><span class="line">        <span class="attribute">ssl_certificate</span> /etc/letsencrypt/live/loveyxq.online/fullchain.pem;</span><br><span class="line">        <span class="attribute">ssl_certificate_key</span> /etc/letsencrypt/live/loveyxq.online/privkey.pem;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">ssl_session_timeout</span> <span class="number">20m</span>;</span><br><span class="line">        <span class="attribute">ssl_session_cache</span> shared:SSL:<span class="number">50m</span>;</span><br><span class="line">        <span class="comment"># 由客户端保存加密后的session信息</span></span><br><span class="line">        <span class="attribute">ssl_session_tickets</span> <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">ssl_dhparam</span> /etc/ssl/private/dhparam.pem;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启OCSP Stapling，由服务器验证证书在线状态，提高TLS握手效率</span></span><br><span class="line">        <span class="attribute">ssl_stapling</span> <span class="literal">on</span>;</span><br><span class="line">        <span class="attribute">ssl_stapling_verify</span> <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启HSTS，缓存http重定向到https，以防止中间人攻击</span></span><br><span class="line">        <span class="comment"># 不包含子域(宝塔界面是http的)</span></span><br><span class="line">        <span class="comment"># 不预加载(预加载要在https://hstspreload.org/中添加)</span></span><br><span class="line">        <span class="attribute">add_header</span> Strict-Transport-Security <span class="string">"max-age=63072000;"</span> always;</span><br><span class="line"></span><br><span class="line">        <span class="comment"># 开启TLS False Start</span></span><br><span class="line">        <span class="attribute">ssl_prefer_server_ciphers</span> <span class="literal">on</span>;</span><br><span class="line"></span><br><span class="line">        <span class="attribute">ssl_protocols</span> TLSv1 TLSv1.<span class="number">1</span> TLSv1.<span class="number">2</span> TLSv1.<span class="number">3</span>;</span><br><span class="line">        <span class="comment"># 一般推荐使用的ssl_ciphers值: https://wiki.mozilla.org/Security/Server_Side_TLS</span></span><br><span class="line">        <span class="attribute">ssl_ciphers</span> <span class="string">'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK'</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>重启 nginx 之后，再次使用 chrome 无恒模式打开控制台看一下访问速度，是不是有所好转。</p><p>或者直接使用 curl 命令：</p><figure class="highlight dust"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="xml">$  curl -X GET -w '\n\n    time_namelookup:  %</span><span class="template-variable">&#123;time_namelookup&#125;</span></span><br><span class="line"><span class="xml">       time_connect:  %</span><span class="template-variable">&#123;time_connect&#125;</span></span><br><span class="line"><span class="xml">    time_appconnect:  %</span><span class="template-variable">&#123;time_appconnect&#125;</span></span><br><span class="line"><span class="xml">   time_pretransfer:  %</span><span class="template-variable">&#123;time_pretransfer&#125;</span></span><br><span class="line"><span class="xml">      time_redirect:  %</span><span class="template-variable">&#123;time_redirect&#125;</span></span><br><span class="line"><span class="xml"> time_starttransfer:  %</span><span class="template-variable">&#123;time_starttransfer&#125;</span></span><br><span class="line"><span class="xml">                      ----------</span></span><br><span class="line"><span class="xml">         time_total:  %</span><span class="template-variable">&#123;time_total&#125;</span><span class="xml">\n'   -H 'Cache-Control: no-cache'  -o /dev/null  -s "https://rebootcat.com"</span></span><br></pre></td></tr></table></figure><p>输出如下：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">   <span class="attr">time_namelookup:</span>  <span class="number">0.005120</span></span><br><span class="line">      <span class="attr">time_connect:</span>  <span class="number">0.257869</span></span><br><span class="line">   <span class="attr">time_appconnect:</span>  <span class="number">0.492787</span></span><br><span class="line">  <span class="attr">time_pretransfer:</span>  <span class="number">0.492887</span></span><br><span class="line">     <span class="attr">time_redirect:</span>  <span class="number">0.000000</span></span><br><span class="line"><span class="attr">time_starttransfer:</span>  <span class="number">0.751222</span></span><br><span class="line">                     <span class="string">----------</span></span><br><span class="line">        <span class="attr">time_total:</span>  <span class="number">1.339793</span></span><br></pre></td></tr></table></figure><p>上面的 <code>time_appconnect</code> 减去 <code>time_connect</code> 的耗时就是 ssl 握手的耗时  0.2349s，比之前好了很多。</p><h1 id="The-END"><a href="#The-END" class="headerlink" title="The END"></a>The END</h1><p>OK，到这里算是把博客正式的迁移到腾讯云香港服务器上了，以后就一直打算用自己的服务器托管博客了。</p><h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><p><a href="https://frostming.com/2020/04-26/github-actions-deploy" target="_blank" rel="noopener">使用 GitHub Actions 实现博客自动化部署</a><br><a href="https://www.hawu.me/operation/2129" target="_blank" rel="noopener">提高https载入速度，记一次nginx升级优化</a></p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#x69;&#x6e;&#117;&#x78;&#99;&#111;&#100;&#101;&#x32;&#x6e;&#x69;&#107;&#105;&#64;&#x67;&#109;&#97;&#x69;&#108;&#x2e;&#99;&#x6f;&#x6d;">&#x6c;&#x69;&#x6e;&#117;&#x78;&#99;&#111;&#100;&#101;&#x32;&#x6e;&#x69;&#107;&#105;&#64;&#x67;&#109;&#97;&#x69;&#108;&#x2e;&#99;&#x6f;&#x6d;</a></p></li></ul><p>2020-11-10 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;之前其实已经写过一篇博文： &lt;a href=&quot;https://rebootcat.com/2020/09/20/virtual_space_blog/&quot;&gt;迁移博客到香港虚拟空间&lt;/a&gt;，那为什么又要写这篇博客呢？&lt;/p&gt;
&lt;p&gt;上次其实是把我的博客迁移到一个香港的虚拟空间里，但是不到半年的时间已经出现过 4 次宕机事件，每次持续时间 4~5 小时，&lt;a href=&quot;https://cloudmonitor.console.aliyun.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;阿里云&lt;/a&gt; 和 &lt;a href=&quot;https://uptimerobot.com/dashboard&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;UpTimeRobot&lt;/a&gt; 的监控报警报了一大堆，邮箱都快塞满了。想着宕机就宕机吧，至少还能恢复，还能凑合用，结果呢，就在前几天当时购买虚拟空间的官网都 GG 了，管理员跑路了。。。&lt;/p&gt;
&lt;p&gt;可能他没挣到钱吧，买一台服务器打算开很多共享的虚拟空间来卖，可能也只有我买了一个，因为我后来看了下我的博客同 IP 的网站就两个，好嘛，结果就跑路了。。。这里就不点名是哪一家了，八字开头的一个云。&lt;/p&gt;
&lt;p&gt;好吧，言归正传，正好双 11，那就干脆直接买服务器吧，所以就购买了腾讯的一台轻量级云服务器，峰值 30Mbps，月流量 1024G，能满足我的需求，况且有了服务器，能做的事情就很多了。比如我还有其他的博客也可以解析到这里，比如可以定制化一些动态博客，比如可以使用自动化发布等。&lt;/p&gt;
&lt;p&gt;那本文大致就记录下迁移的一些过程以及踩坑优化等：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;服务器购买以及初始化&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;安装部署 nginx&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;部署博客源码&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;解析域名&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;设置 https 证书&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;绑定多个域名&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;使用 github actions 自动化部署博客（踩坑）&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;https 性能优化&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
    
      <category term="blog" scheme="https://rebootcat.com/categories/blog/"/>
    
    
      <category term="nginx" scheme="https://rebootcat.com/tags/nginx/"/>
    
      <category term="hexo" scheme="https://rebootcat.com/tags/hexo/"/>
    
      <category term="blog" scheme="https://rebootcat.com/tags/blog/"/>
    
      <category term="ssl" scheme="https://rebootcat.com/tags/ssl/"/>
    
      <category term="github" scheme="https://rebootcat.com/tags/github/"/>
    
      <category term="actions" scheme="https://rebootcat.com/tags/actions/"/>
    
      <category term="github_actions" scheme="https://rebootcat.com/tags/github-actions/"/>
    
      <category term="curl" scheme="https://rebootcat.com/tags/curl/"/>
    
      <category term="https" scheme="https://rebootcat.com/tags/https/"/>
    
      <category term="workflow" scheme="https://rebootcat.com/tags/workflow/"/>
    
      <category term="deploy" scheme="https://rebootcat.com/tags/deploy/"/>
    
  </entry>
  
  <entry>
    <title>free not return memory</title>
    <link href="https://rebootcat.com/2020/11/05/free_mem/"/>
    <id>https://rebootcat.com/2020/11/05/free_mem/</id>
    <published>2020-11-05T15:23:58.000Z</published>
    <updated>2020-11-05T15:32:49.097Z</updated>
    
    <content type="html"><![CDATA[<h1 id="内存泄露？"><a href="#内存泄露？" class="headerlink" title="内存泄露？"></a>内存泄露？</h1><p>观察到一台机器上的内存使用量在程序启动之后，持续增长，中间没有出现内存恢复。怀疑是不是出现了内存泄露的问题？</p><p>然后使用相关的内存分析工具进行了分析：</p><ul><li>gperf</li><li>valgrind (massif)</li><li>手工标记内存分配释放</li></ul><p>上述的分析结果均不能很肯定的得出是否内存泄露的结论。那么问题可能出现在哪里呢？</p><p>程序采用 c++ 编写，大量使用了智能指针以及 new/delete，难道内存没有成功释放？亦或是内存释放有什么条件？于是开始怀疑 free 是不是真的释放了内存？</p><h1 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h1><p>既然怀疑 free 是不是真的释放了内存，此处的释放，是指程序内存占用下降，内存归还给操作系统，那么直接写一个简单的例子进行验证一下。</p><blockquote><p>attention:</p></blockquote><p>测试前，先关闭 swap：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"># swapoff -a</span><br><span class="line"></span><br><span class="line"># free -h</span><br><span class="line">              total        used        free      <span class="keyword">shared</span>  buff/cache   available</span><br><span class="line">Mem:           <span class="number">3.7</span>G        <span class="number">2.5</span>G        <span class="number">1.1</span>G        <span class="number">8.8</span>M         <span class="number">40</span>M        <span class="number">959</span>M</span><br><span class="line">Swap:            <span class="number">0</span>B          <span class="number">0</span>B          <span class="number">0</span>B</span><br></pre></td></tr></table></figure><h2 id="测试1"><a href="#测试1" class="headerlink" title="测试1"></a>测试1</h2><p>步骤如下：</p><ol><li>循环分配大量内存</li><li>block 程序，top 工具观察进程内存占用情况</li><li>再循环释放所有分配的内存</li><li>block 程序，top 工具观察进程内存占用情况</li><li>程序退出</li></ol><a id="more"></a><p>上代码：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;string&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;iostream&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;vector&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;malloc.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">test</span><span class="params">(<span class="keyword">uint32_t</span> num, <span class="keyword">uint32_t</span> mem_size)</span> </span>&#123;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"test: mem_size = "</span> &lt;&lt; mem_size &lt;&lt; <span class="string">" total:"</span> &lt;&lt; num * mem_size / <span class="number">1024.0</span> / <span class="number">1024.0</span> &lt;&lt; <span class="string">" MB"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">vector</span>&lt;<span class="keyword">char</span>*&gt; vec;</span><br><span class="line">    <span class="comment">// 3G</span></span><br><span class="line">    <span class="keyword">for</span> (<span class="keyword">uint32_t</span> i =<span class="number">0</span>; i &lt; num; ++i) &#123;</span><br><span class="line">        <span class="keyword">char</span> *ptr = <span class="keyword">new</span> <span class="keyword">char</span>[mem_size];</span><br><span class="line">        vec.push_back(ptr);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"allocate memory "</span>&lt;&lt; num * mem_size / <span class="number">1024.0</span> / <span class="number">1024.0</span> &lt;&lt;  <span class="string">" MB done"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    <span class="keyword">for</span> (<span class="keyword">auto</span>&amp; ptr : vec) &#123;</span><br><span class="line">        <span class="built_in">strncpy</span>(ptr, <span class="string">"abcdefghij"</span>, mem_size);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"input anything to continue delete all memory..."</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    getchar();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> (<span class="keyword">auto</span>&amp; ptr : vec) &#123;</span><br><span class="line">        <span class="keyword">delete</span> ptr;</span><br><span class="line">        ptr = <span class="literal">nullptr</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"release memory "</span>&lt;&lt; num * mem_size / <span class="number">1024.0</span> / <span class="number">1024.0</span> &lt;&lt;  <span class="string">" MB done"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[])</span> </span>&#123;</span><br><span class="line">    <span class="keyword">uint32_t</span> mem_size = <span class="number">100</span>;</span><br><span class="line">    <span class="keyword">if</span> (argc &gt;=<span class="number">2</span>) &#123;</span><br><span class="line">        mem_size = <span class="built_in">std</span>::atoi(argv[<span class="number">1</span>]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">uint32_t</span> num = (<span class="keyword">uint32_t</span>)<span class="number">2</span> * <span class="number">1024</span> * <span class="number">1024</span> * <span class="number">1024</span> / mem_size;</span><br><span class="line"></span><br><span class="line">    test(num, mem_size);</span><br><span class="line"></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"input anything to exit"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    getchar();</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译：</p><figure class="highlight brainfuck"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">g</span>++ <span class="comment">mem_test</span><span class="string">.</span><span class="comment">cc</span> <span class="literal">-</span><span class="comment">o</span> <span class="comment">mem_test</span> <span class="literal">-</span><span class="comment">std=c</span>++<span class="comment">11</span></span><br></pre></td></tr></table></figure><p>可以通过参数控制内存分配的大小，默认 100Byte：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">./mem_test <span class="number">100</span></span><br><span class="line">./mem_test <span class="number">500</span></span><br><span class="line">./mem_test <span class="number">1024</span></span><br><span class="line">./mem_test <span class="number">10240</span></span><br></pre></td></tr></table></figure><p>过程就省略了，直接上观察结果：</p><ul><li>每次测试的虚拟内存大小是类似的，大概在 2G 左右</li><li>单次分配内存长度为 100 Byte，调用 free 后内存无明显下降</li><li>单次分配内存长度为 500,1024,10240…，调用 free 后内存迅速下降接近 0%</li><li>多次测试临界值为 120 Byte</li></ul><p>以上测试反应出<strong>在不同的情况下， free 的行为有差异，但也说明，调用 free 之后内存是能够立即被释放给操作系统的</strong>（只不过有条件）。</p><p>那为什么会出现调用 free 之后内存没有被释放（至少看起来是）的情况呢？</p><h2 id="测试2"><a href="#测试2" class="headerlink" title="测试2"></a>测试2</h2><p>代码不变，还是上面的代码，只不过现在启动两个同样的程序：</p><ol><li>分别以上述不同的参数启动程序，让程序执行到释放所有内存之后，block 住</li><li>然后启动第二个程序，用同样的参数</li><li>观察两个进程是否都能存活</li></ol><p>上述有一个条件假设：</p><blockquote><p>total mem: 4G，实际情况可以调整代码里分配内存的总量</p></blockquote><p>过程也省略，直接上观察结果：</p><ul><li>单次分配内存 100 Byte，启动第二个同样的程序，出现 OOM （先启动这个被 kill)</li><li>单次分配 500,1024,10240…， 启动第二个同样的程序，不会 OOM</li></ul><p>上面的结果和测试1 的结果是吻合的，这能<strong>肯定的说明出现 OOM 的场景下，第一个进程的内存虽然完全释放了，但是内存依然被该进程持有，操作系统无法把这部分已经调用 free 的内存重新分配给其他的进程</strong>（第二个进程）。</p><h2 id="测试3"><a href="#测试3" class="headerlink" title="测试3"></a>测试3</h2><p>稍微调整一下上面的代码，分配释放的操作进行两次，也就是上面的 <code>test()</code> 函数调用 2 次。</p><p>另外本次使用 valgrind(massif) 进行分析，此次单次内存分配大小为 100 Byte，也就是上面出现无法释放内存的参数。</p><figure class="highlight less"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="selector-tag">int</span> <span class="selector-tag">main</span>() &#123;</span><br><span class="line">...</span><br><span class="line"><span class="selector-tag">test</span>();</span><br><span class="line"></span><br><span class="line"><span class="selector-tag">test</span>(); <span class="comment">// call again</span></span><br><span class="line">...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>使用 valgrind 进行分析：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">valgrind -v <span class="attribute">--tool</span>=massif <span class="attribute">--detailed-freq</span>=2 <span class="attribute">--time-unit</span>=B <span class="attribute">--main-stacksize</span>=24000000 <span class="attribute">--max-stackframe</span>=24000000  <span class="attribute">--threshold</span>=0.1  <span class="attribute">--massif-out-file</span>=./massif.out ./mem_test</span><br></pre></td></tr></table></figure><p>生成的文件 <code>massif.out</code> 使用 <code>massif-visualizer</code> 处理之后得到如下图：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/free_memory/0.png" alt=""></p><p>上图就是内存分配的情况，从图中可以很明显的看到在第一次调用 <code>test()</code> 函数时，内存随着分配而增长，随着释放而下降；第二次调用 <code>test()</code> 函数也是同样的情况。</p><p>那这幅图能说明什么呢？</p><p>第一次调用 <code>test()</code>后，按照测试2 的情况，内存虽然被释放了，但是内存依然被进程持有，那么不应该出现内存下降的情况，但是从图中看，确实是下降到接近 0 了，那么可以得出一个结论：</p><p><strong><code>test()</code> 至少是没有内存泄露的，即分配的内存，都被释放了（至少标记过释放），也就是没有出现野指针等内存泄露的情况</strong>。</p><p>那么<strong>问题就在于，既然没有内存泄露，那为何内存依然被进程持有？不是已经调用 free 了吗</strong>？</p><h1 id="glibc-malloc-free-实现"><a href="#glibc-malloc-free-实现" class="headerlink" title="glibc malloc/free 实现"></a>glibc malloc/free 实现</h1><p>glibc malloc 底层调用的是 ptmalloc，这里就不深入 malloc/free 的实现细节了，网上可以找到很多资料。 </p><blockquote><p>下图是 32 位程序的虚拟内存空间分布图</p></blockquote><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/free_memory/1.png" alt=""></p><h2 id="原理"><a href="#原理" class="headerlink" title="原理"></a>原理</h2><p>向操作系统申请内存涉及到两个系统调用 sbrk 以及 mmap。关于这两个系统调用的区别可以大致这么理解:</p><ul><li><strong>ptmalloc 管理了两块堆内存，所以有可能会在两个地方给用户分配内存</strong></li><li><strong>这两块堆内存的区别就在于一个可以被循环利用，一个在释放后立即归还操作系统</strong></li><li><strong>ptmalloc 使用 sbrk 来为第一块内存区域 heap 进行内存分配，用户释放之后 ptmalloc 对这块内存进行重新管理利用，进程依然持有这块内存</strong></li><li><strong>ptmalloc 使用 mmap 来为第二块内存区域 sub-heap 进行内存分配，用户释放之后 ptmalloc 立即把这块内存归还给操作系统</strong></li><li><strong>要分配的内存只有达到一定大小（即 mmap 的阈值），ptmalloc 才会采用 mmap 进行内存分配，否则优先选择 sbrk 分配后被重新管理的内存池</strong></li><li><strong>mmap 的阈值可能是动态调整的，即 ptmalloc 根据自身内存管理情况，动态调整这个阈值</strong></li></ul><p>也就是说，<strong>ptmalloc 为了性能考虑，采用了两种内存分配策略，也就是管理了两种不同分配方式的堆内存。在分配内存小于一定值时就优先在 ptmalloc 维护的内存池里进行分配，这样避免了直接向操作系统分配内存，减少系统调用次数；如果内存大于一定值时，就直接向操作系统申请内存，并且这段内存在释放之后立即归还操作系统</strong>；</p><p>这也就能解释上面的几个测试里，当单次分配的内存大小较大时，内存释放后进程内存占用快速下降到 0%；当单次分配的内存大小较小时，内存释放后其实没有归还给操作系统，二是被 ptmalloc 重新回收了，放到了内存池里进行循环利用，所以看到进程内存依然保持较高的占用；</p><p>另外关于 ptmalloc 对内存池的管理比较复杂，这里推荐一篇不错的文章可以深度阅读：</p><p><a href="https://paper.seebug.org/papers/Archive/refs/heap/glibc%E5%86%85%E5%AD%98%E7%AE%A1%E7%90%86ptmalloc%E6%BA%90%E4%BB%A3%E7%A0%81%E5%88%86%E6%9E%90.pdf" target="_blank" rel="noopener">glibc内存管理ptmalloc源代码分析</a></p><p>到这里，其实就已经比较明确了，free 之后内存释放情况其实是跟分配的大小有关系的，并且随着程序的运行，内存的持续分配和释放，ptmalloc 的内存池应该能稳定在一定的值，从外面来看，进程的内存占用应该能动态稳定下来。</p><p><strong>ptmalloc 的两套分配策略各有优劣，使用内存池可以提高内存分配效率，但是可能出现内存暴涨的情况，但是最终会稳定在一定的值；使用 mmap 的方式分配内存不会出现内存暴涨的情况，释放完之后理解归还操作系统，但降低了内存分配的效率</strong>。</p><h2 id="修改-malloc-参数"><a href="#修改-malloc-参数" class="headerlink" title="修改 malloc 参数"></a>修改 malloc 参数</h2><p>根据上面的讨论，如果想要控制 malloc 的内存分配行为，那么其实是有办法做到的。</p><p>我们可以通过下面这个函数来实现：</p><figure class="highlight cs"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">mallopt</span>(<span class="params"><span class="keyword">int</span> param, <span class="keyword">int</span> <span class="keyword">value</span></span>)</span>;</span><br></pre></td></tr></table></figure><p><a href="https://man7.org/linux/man-pages/man3/mallopt.3.html" target="_blank" rel="noopener">https://man7.org/linux/man-pages/man3/mallopt.3.html</a></p><p><strong>可以调整 M_TRIM_THRESHOLD，M_MMAP_THRESHOLD，M_TOP_PAD 和 M_MMAP_MAX 中的任意一个，关闭 mmap 分配阈值动态调整机制</strong>。</p><p>比如上面的测试1，当单次分配的内存 100 Byte 时，内存释放之后进程内存占用依然较高的情况就能解决：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">int</span> main() &#123;</span><br><span class="line">    mallopt(M_MMAP_THRESHOLD, <span class="number">64</span>);</span><br><span class="line">    mallopt(M_TRIM_THRESHOLD, <span class="number">64</span>);</span><br><span class="line">    </span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><ul><li>M_MMAP_THRESHOLD: mmap 内存分配阈值</li><li>M_RIM_THRESHOLD: mmap 收缩阈值</li></ul><p>在 main 函数开始加上上面的两句，调整 mmap 收缩阈值以及内存分配阈值。重新编译运行，发现即使单次分配 100 Byte，内存释放后，进程内存占用也快速下降到 0%。</p><p>补充：</p><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">int</span> malloc<span class="constructor">_trim(<span class="params">size_t</span> <span class="params">pad</span>)</span>;</span><br></pre></td></tr></table></figure><p>可以触发 ptmalloc 对内存的紧缩，即归还一部分内存给操作系统。</p><h1 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h1><p>进程内存占用较高的情况不一定是内存泄露造成的，可以通过长时间观察内存占用是否能稳定下来进行判断，如果内存占用能实现动态稳定，那么多半程序是没有内存泄露的。</p><p>但是如果内存占用过高，对其他的进程产生了干扰，那么可以适当的调整一下 malloc 的参数，控制 malloc 的行为，避免 glibc 内存池过大，影响其他进程的运行。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#x6e;&#x75;&#x78;&#x63;&#x6f;&#x64;&#101;&#x32;&#110;&#x69;&#107;&#105;&#x40;&#x67;&#109;&#x61;&#x69;&#108;&#46;&#99;&#x6f;&#x6d;">&#108;&#x69;&#x6e;&#x75;&#x78;&#x63;&#x6f;&#x64;&#101;&#x32;&#110;&#x69;&#107;&#105;&#x40;&#x67;&#109;&#x61;&#x69;&#108;&#46;&#99;&#x6f;&#x6d;</a></p></li></ul><p>2020-11-05 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;内存泄露？&quot;&gt;&lt;a href=&quot;#内存泄露？&quot; class=&quot;headerlink&quot; title=&quot;内存泄露？&quot;&gt;&lt;/a&gt;内存泄露？&lt;/h1&gt;&lt;p&gt;观察到一台机器上的内存使用量在程序启动之后，持续增长，中间没有出现内存恢复。怀疑是不是出现了内存泄露的问题？&lt;/p&gt;
&lt;p&gt;然后使用相关的内存分析工具进行了分析：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;gperf&lt;/li&gt;
&lt;li&gt;valgrind (massif)&lt;/li&gt;
&lt;li&gt;手工标记内存分配释放&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;上述的分析结果均不能很肯定的得出是否内存泄露的结论。那么问题可能出现在哪里呢？&lt;/p&gt;
&lt;p&gt;程序采用 c++ 编写，大量使用了智能指针以及 new/delete，难道内存没有成功释放？亦或是内存释放有什么条件？于是开始怀疑 free 是不是真的释放了内存？&lt;/p&gt;
&lt;h1 id=&quot;测试&quot;&gt;&lt;a href=&quot;#测试&quot; class=&quot;headerlink&quot; title=&quot;测试&quot;&gt;&lt;/a&gt;测试&lt;/h1&gt;&lt;p&gt;既然怀疑 free 是不是真的释放了内存，此处的释放，是指程序内存占用下降，内存归还给操作系统，那么直接写一个简单的例子进行验证一下。&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;attention:&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;测试前，先关闭 swap：&lt;/p&gt;
&lt;figure class=&quot;highlight angelscript&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;# swapoff -a&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;# free -h&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;              total        used        free      &lt;span class=&quot;keyword&quot;&gt;shared&lt;/span&gt;  buff/cache   available&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;Mem:           &lt;span class=&quot;number&quot;&gt;3.7&lt;/span&gt;G        &lt;span class=&quot;number&quot;&gt;2.5&lt;/span&gt;G        &lt;span class=&quot;number&quot;&gt;1.1&lt;/span&gt;G        &lt;span class=&quot;number&quot;&gt;8.8&lt;/span&gt;M         &lt;span class=&quot;number&quot;&gt;40&lt;/span&gt;M        &lt;span class=&quot;number&quot;&gt;959&lt;/span&gt;M&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;Swap:            &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;B          &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;B          &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;B&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;h2 id=&quot;测试1&quot;&gt;&lt;a href=&quot;#测试1&quot; class=&quot;headerlink&quot; title=&quot;测试1&quot;&gt;&lt;/a&gt;测试1&lt;/h2&gt;&lt;p&gt;步骤如下：&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;循环分配大量内存&lt;/li&gt;
&lt;li&gt;block 程序，top 工具观察进程内存占用情况&lt;/li&gt;
&lt;li&gt;再循环释放所有分配的内存&lt;/li&gt;
&lt;li&gt;block 程序，top 工具观察进程内存占用情况&lt;/li&gt;
&lt;li&gt;程序退出&lt;/li&gt;
&lt;/ol&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="memory" scheme="https://rebootcat.com/tags/memory/"/>
    
      <category term="free" scheme="https://rebootcat.com/tags/free/"/>
    
      <category term="malloc" scheme="https://rebootcat.com/tags/malloc/"/>
    
      <category term="ptmalloc" scheme="https://rebootcat.com/tags/ptmalloc/"/>
    
      <category term="glibc" scheme="https://rebootcat.com/tags/glibc/"/>
    
      <category term="mmap" scheme="https://rebootcat.com/tags/mmap/"/>
    
      <category term="sbrk" scheme="https://rebootcat.com/tags/sbrk/"/>
    
  </entry>
  
  <entry>
    <title>Linux上隐藏进程名（初级版）</title>
    <link href="https://rebootcat.com/2020/10/25/hiddenproc/"/>
    <id>https://rebootcat.com/2020/10/25/hiddenproc/</id>
    <published>2020-10-25T14:23:58.000Z</published>
    <updated>2020-10-29T12:02:22.586Z</updated>
    
    <content type="html"><![CDATA[<h1 id="缘起"><a href="#缘起" class="headerlink" title="缘起"></a>缘起</h1><p>上一篇博文 <a href="https://rebootcat.com/2020/10/25/setproctitle/">模仿nginx修改进程名</a> 中提到了一种修改进程名的方法，就像 nginx 一样，给不同进程命名为 master 以及 worker 等。那么能不能把新进程名设置为空字符串呢？如果能，又会有哪些应用场景呢？</p><p>答案可能是能的，设置新进程的名字为空，通常用来隐藏进程，用于攻击或者反攻击。</p><h1 id="prctl-函数"><a href="#prctl-函数" class="headerlink" title="prctl 函数"></a>prctl 函数</h1><p>上一篇博文 <a href="https://rebootcat.com/2020/10/25/setproctitle/">模仿nginx修改进程名</a> 文章末尾提到了 <code>prctl</code> 这个函数，它也可以用来修改进程名。</p><p>只不过如果单单使用 prctl 来修改进程名的话，使用 ps 或者 top 等工具看到的可能还是原来的名字。</p><p>源代码可以在我的 github 找到：</p><p><a href="https://github.com/smaugx/setproctitle/blob/main/hidden_process/prctl_main.cc" target="_blank" rel="noopener">https://github.com/smaugx/setproctitle/blob/main/hidden_process/prctl_main.cc</a></p><figure class="highlight autoit"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdlib&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;sys/prctl.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> main(<span class="built_in">int</span> argc, char* argv[], char *envp[])</span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">const</span> char *new_title = <span class="string">"prctl_new_name"</span><span class="comment">;</span></span><br><span class="line">    prctl(PR_SET_NAME, new_title, <span class="literal">NULL</span>, <span class="literal">NULL</span>, <span class="literal">NULL</span>)<span class="comment">;</span></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="built_in">sleep</span>(<span class="number">2</span>)<span class="comment">;</span></span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span><span class="comment">;</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译运行：</p><figure class="highlight vala"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#  g++ prctl_main.cc   -o  prctl_main -std=c++11</span></span><br><span class="line"><span class="meta"># ./prctl_main</span></span><br></pre></td></tr></table></figure><p>然后我们查看一下进程的名字：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep prctl</span><br><span class="line">root     <span class="number">20758</span> <span class="number">12289</span>  <span class="number">0</span> <span class="number">17</span>:<span class="number">39</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> ./prctl_main</span><br><span class="line">root     <span class="number">20791</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">17</span>:<span class="number">39</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> prctl</span><br></pre></td></tr></table></figure><p>可以看到 ps 看到的进程名依然是 <code>prctl_main</code> 而不是 <code>prctl_new_name</code>。那么 <code>prctl</code> 函数到底修改了哪里呢？ ps 命令又是从哪里读取的进程名呢？</p><a id="more"></a><h2 id="proc-lt-pid-gt-虚拟文件夹"><a href="#proc-lt-pid-gt-虚拟文件夹" class="headerlink" title="/proc/&lt;pid&gt; 虚拟文件夹"></a>/proc/&lt;pid&gt; 虚拟文件夹</h2><p>linux 上一切皆文件，启动一个进程，就会在系统的 <code>/proc</code> 这个虚拟文件系统下创建这个进程相关的文件夹，里面记录了这个进程的数据。</p><figure class="highlight mipsasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># ls /proc/20758</span></span><br><span class="line">attr       cgroup      comm             cwd      fd       io        map_files  mountinfo   net        oom_adj        pagemap      projid_map  <span class="keyword">schedstat </span> smaps  statm    task     wchan</span><br><span class="line">autogroup  clear_refs  coredump_filter  environ  fdinfo   limits    maps       mounts      ns         oom_score      patch_state  root        sessionid  stack  status   timers</span><br><span class="line">auxv       cmdline     cpuset           exe      gid_map  loginuid  mem        mountstats  numa_maps  oom_score_adj  personality  <span class="keyword">sched </span>      setgroups  stat   <span class="keyword">syscall </span> uid_map</span><br></pre></td></tr></table></figure><p>关注一下这两个虚拟文件:</p><figure class="highlight jboss-cli"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># cat /proc/20758/cmdline</span></span><br><span class="line"><span class="string">./prctl_main</span></span><br></pre></td></tr></table></figure><p>以及</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># cat /proc/20758/status</span></span><br><span class="line"><span class="attr">Name:</span><span class="string">prctl_new_name</span></span><br><span class="line"><span class="attr">Umask:</span><span class="number">0022</span></span><br><span class="line"><span class="attr">State:</span><span class="string">S</span> <span class="string">(sleeping)</span></span><br><span class="line"><span class="attr">Tgid:</span><span class="number">20758</span></span><br><span class="line"><span class="attr">Ngid:</span><span class="number">0</span></span><br><span class="line"><span class="attr">Pid:</span><span class="number">20758</span></span><br><span class="line"><span class="attr">PPid:</span><span class="number">12289</span></span><br><span class="line"><span class="attr">TracerPid:</span><span class="number">0</span></span><br><span class="line"><span class="attr">Uid:</span><span class="number">0</span><span class="number">0</span><span class="number">0</span><span class="number">0</span></span><br><span class="line"><span class="attr">Gid:</span><span class="number">0</span><span class="number">0</span><span class="number">0</span><span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="string">...(省略）</span></span><br></pre></td></tr></table></figure><p>细心的同学应该发现上面的不一致了吧， /proc/&lt;pid&gt;/cmdline 这个文件记录的进程名是 <code>prctl_main</code>，而 /proc/&lt;pid&gt;/status 中 Name 值记录的进程名是 <code>prctl_new_name</code>。而 ps 命令正好是读取了 cmdline 这个文件，导致即便使用 prctl 修改了进程名，但 ps 依然看到的是老的进程名。</p><p>另外要注意，<strong>prctl() 这个函数有个限制，新进程的名字长度不能超过 16 字节（包括最后的 ‘\0’）</strong>，详见手册：</p><p><a href="https://man7.org/linux/man-pages/man2/prctl.2.html" target="_blank" rel="noopener">https://man7.org/linux/man-pages/man2/prctl.2.html</a></p><h1 id="prctl-结合-argv-0"><a href="#prctl-结合-argv-0" class="headerlink" title="prctl 结合 argv[0]"></a>prctl 结合 argv[0]</h1><p>上面的分析看到，不论是修改 argv[0] 还是使用 prctl，均有其局限性，那么通常可以结合两者来进行。</p><p>源码可以在我的 github 找到：</p><p><a href="https://github.com/smaugx/setproctitle/blob/main/hidden_process/hidden_main.cc" target="_blank" rel="noopener">https://github.com/smaugx/setproctitle/blob/main/hidden_process/hidden_main.cc</a></p><figure class="highlight autoit"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdlib&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;sys/prctl.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="string">"../setproctitle.h"</span></span></span><br><span class="line"></span><br><span class="line">char **smaug_os_argv<span class="comment">;</span></span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> main(<span class="built_in">int</span> argc, char* argv[], char *envp[])</span><br><span class="line">&#123;</span><br><span class="line">    smaug_os_argv = argv<span class="comment">;</span></span><br><span class="line"></span><br><span class="line">    // set new process <span class="literal">NULL</span> <span class="keyword">to</span> hide process</span><br><span class="line">    <span class="keyword">const</span> char *new_title = <span class="string">"hidden_main_new"</span><span class="comment">;</span></span><br><span class="line">    <span class="keyword">if</span> (smaug_init_setproctitle() == SMAUG_PROCTITLE_OK) &#123;</span><br><span class="line">        smaug_setproctitle(new_title)<span class="comment">;</span></span><br><span class="line">    &#125;</span><br><span class="line">    // set new process <span class="literal">NULL</span> <span class="keyword">to</span> hide process</span><br><span class="line">    prctl(PR_SET_NAME, new_title, <span class="literal">NULL</span>, <span class="literal">NULL</span>, <span class="literal">NULL</span>)<span class="comment">;</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="built_in">sleep</span>(<span class="number">1</span>)<span class="comment">;</span></span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span><span class="comment">;</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译运行：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep hidd</span><br><span class="line">root     <span class="number">21753</span> <span class="number">12289</span>  <span class="number">0</span> <span class="number">17</span>:<span class="number">55</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> hidden_main_new</span><br><span class="line">root     <span class="number">21760</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">17</span>:<span class="number">55</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> hidd</span><br><span class="line"></span><br><span class="line"># cat /proc/<span class="number">21753</span>/cmdline</span><br><span class="line">hidden_main_new[<span class="symbol">root@</span>Jiao ~]#</span><br><span class="line"></span><br><span class="line"># cat /proc/<span class="number">21753</span>/status |grep Name</span><br><span class="line">Name:hidden_main_new</span><br></pre></td></tr></table></figure><p>可以看到，无论是通过 ps 命令还是直接查看 /proc/&lt;pid&gt;/ 下的文件的方式，均能看到修改后的名字： <code>hidden_main_new</code>。</p><h1 id="隐藏进程"><a href="#隐藏进程" class="headerlink" title="隐藏进程"></a>隐藏进程</h1><p>经过上一步，已经可以完美的修改进程名了，那么再进一步，如何隐藏进程呢？</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> <span class="keyword">char</span> *new_title = <span class="string">""</span>;</span><br></pre></td></tr></table></figure><p>只需要修改上述的一行代码，重新编译即可，然后用 ps 或者 top 看一下，能不能找到这个进程：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep hidden</span><br><span class="line">root     <span class="number">22022</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">17</span>:<span class="number">59</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> hidden</span><br></pre></td></tr></table></figure><p>可以看到 ps 无法找到 hidden* 相关的进程，那么 top 呢？</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">top - <span class="number">18</span>:<span class="number">01</span>:<span class="number">06</span> up <span class="number">16</span> days,  <span class="number">4</span>:<span class="number">16</span>,  <span class="number">9</span> users,  load average: <span class="number">0.00</span>, <span class="number">0.01</span>, <span class="number">0.05</span></span><br><span class="line">Tasks: <span class="number">121</span> total,   <span class="number">1</span> running, <span class="number">120</span> sleeping,   <span class="number">0</span> stopped,   <span class="number">0</span> zombie</span><br><span class="line">%Cpu(s):  <span class="number">0.0</span> us,  <span class="number">0.0</span> sy,  <span class="number">0.0</span> ni,<span class="number">100.0</span> id,  <span class="number">0.0</span> wa,  <span class="number">0.0</span> hi,  <span class="number">0.0</span> si,  <span class="number">0.0</span> st</span><br><span class="line">KiB Mem :  <span class="number">3879952</span> total,  <span class="number">3579624</span> free,    <span class="number">95660</span> used,   <span class="number">204668</span> buff/cache</span><br><span class="line">KiB Swap:  <span class="number">5242876</span> total,  <span class="number">5164352</span> free,    <span class="number">78524</span> used.  <span class="number">3272224</span> avail Mem</span><br><span class="line"></span><br><span class="line">  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND</span><br><span class="line"><span class="number">31038</span> root      <span class="number">20</span>   <span class="number">0</span>  <span class="number">115800</span>    <span class="number">372</span>    <span class="number">368</span> S   <span class="number">0.0</span>  <span class="number">0.0</span>   <span class="number">0</span>:<span class="number">00.70</span> bash</span><br><span class="line"><span class="number">22081</span> root      <span class="number">20</span>   <span class="number">0</span>       <span class="number">0</span>      <span class="number">0</span>      <span class="number">0</span> S   <span class="number">0.0</span>  <span class="number">0.0</span>   <span class="number">0</span>:<span class="number">00.00</span> kworker/<span class="number">1</span>:<span class="number">0</span></span><br><span class="line"><span class="number">22078</span> root      <span class="number">20</span>   <span class="number">0</span>  <span class="number">162140</span>   <span class="number">2252</span>   <span class="number">1548</span> R   <span class="number">0.0</span>  <span class="number">0.1</span>   <span class="number">0</span>:<span class="number">00.04</span> top</span><br><span class="line"><span class="number">22013</span> root      <span class="number">20</span>   <span class="number">0</span>   <span class="number">11124</span>   <span class="number">1068</span>    <span class="number">908</span> S   <span class="number">0.0</span>  <span class="number">0.0</span>   <span class="number">0</span>:<span class="number">00.00</span></span><br><span class="line"><span class="number">21859</span> root      <span class="number">20</span>   <span class="number">0</span>  <span class="number">115892</span>    <span class="number">244</span>    <span class="number">240</span> S   <span class="number">0.0</span>  <span class="number">0.0</span>   <span class="number">0</span>:<span class="number">00.46</span> bash</span><br><span class="line"><span class="number">21725</span> root      <span class="number">20</span>   <span class="number">0</span>       <span class="number">0</span>      <span class="number">0</span>      <span class="number">0</span> S   <span class="number">0.0</span>  <span class="number">0.0</span>   <span class="number">0</span>:<span class="number">00.00</span> kworker/<span class="number">1</span>:<span class="number">2</span></span><br><span class="line"><span class="number">20811</span> root      <span class="number">20</span>   <span class="number">0</span>       <span class="number">0</span>      <span class="number">0</span>      <span class="number">0</span> S   <span class="number">0.0</span>  <span class="number">0.0</span>   <span class="number">0</span>:<span class="number">00.00</span> kworker/<span class="number">0</span>:<span class="number">0</span></span><br></pre></td></tr></table></figure><p>运行 top 命令，并且以 pid 倒叙排序，注意第四行的进程，可以看到 COMMAND 为空，这个进程就是刚才的这个进程，但是看不到进程名了，达到了简单的、初级的隐藏进程的目的。</p><h1 id="扩展一下"><a href="#扩展一下" class="headerlink" title="扩展一下"></a>扩展一下</h1><p>上述相关代码均可以在我的 github 找到：</p><p><a href="https://github.com/smaugx/setproctitle/tree/main/hidden_process" target="_blank" rel="noopener">https://github.com/smaugx/setproctitle/tree/main/hidden_process</a></p><p>上面的讨论可以看到，能实现初级的，简单的进程隐藏，但是使用 top 命令还是能看到这个无名进程，那么这点改怎么解决呢？</p><p>这里就不展开了，我没有这方面的经验。不过通常来说有两种办法：</p><ul><li>修改进程名为常见的一些进程名，比如 bash, top, nginx 等以达到混淆的目的</li><li>想办法把 /proc/&lt;pid&gt;/ 这个虚拟文件夹隐藏或者达到隐藏类的效果（不太擅长）</li></ul><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#x6e;&#x75;&#120;&#x63;&#x6f;&#100;&#x65;&#50;&#x6e;&#105;&#107;&#105;&#x40;&#103;&#x6d;&#97;&#x69;&#108;&#x2e;&#99;&#111;&#x6d;">&#108;&#x69;&#x6e;&#x75;&#120;&#x63;&#x6f;&#100;&#x65;&#50;&#x6e;&#105;&#107;&#105;&#x40;&#103;&#x6d;&#97;&#x69;&#108;&#x2e;&#99;&#111;&#x6d;</a></p></li></ul><p>2020-10-25 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;缘起&quot;&gt;&lt;a href=&quot;#缘起&quot; class=&quot;headerlink&quot; title=&quot;缘起&quot;&gt;&lt;/a&gt;缘起&lt;/h1&gt;&lt;p&gt;上一篇博文 &lt;a href=&quot;https://rebootcat.com/2020/10/25/setproctitle/&quot;&gt;模仿nginx修改进程名&lt;/a&gt; 中提到了一种修改进程名的方法，就像 nginx 一样，给不同进程命名为 master 以及 worker 等。那么能不能把新进程名设置为空字符串呢？如果能，又会有哪些应用场景呢？&lt;/p&gt;
&lt;p&gt;答案可能是能的，设置新进程的名字为空，通常用来隐藏进程，用于攻击或者反攻击。&lt;/p&gt;
&lt;h1 id=&quot;prctl-函数&quot;&gt;&lt;a href=&quot;#prctl-函数&quot; class=&quot;headerlink&quot; title=&quot;prctl 函数&quot;&gt;&lt;/a&gt;prctl 函数&lt;/h1&gt;&lt;p&gt;上一篇博文 &lt;a href=&quot;https://rebootcat.com/2020/10/25/setproctitle/&quot;&gt;模仿nginx修改进程名&lt;/a&gt; 文章末尾提到了 &lt;code&gt;prctl&lt;/code&gt; 这个函数，它也可以用来修改进程名。&lt;/p&gt;
&lt;p&gt;只不过如果单单使用 prctl 来修改进程名的话，使用 ps 或者 top 等工具看到的可能还是原来的名字。&lt;/p&gt;
&lt;p&gt;源代码可以在我的 github 找到：&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://github.com/smaugx/setproctitle/blob/main/hidden_process/prctl_main.cc&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://github.com/smaugx/setproctitle/blob/main/hidden_process/prctl_main.cc&lt;/a&gt;&lt;/p&gt;
&lt;figure class=&quot;highlight autoit&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;7&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;8&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;9&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;12&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;13&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;14&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;15&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;cstdio&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;cstdlib&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;cstring&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;unistd.h&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;sys/prctl.h&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;built_in&quot;&gt;int&lt;/span&gt; main(&lt;span class=&quot;built_in&quot;&gt;int&lt;/span&gt; argc, char* argv[], char *envp[])&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;const&lt;/span&gt; char *new_title = &lt;span class=&quot;string&quot;&gt;&quot;prctl_new_name&quot;&lt;/span&gt;&lt;span class=&quot;comment&quot;&gt;;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    prctl(PR_SET_NAME, new_title, &lt;span class=&quot;literal&quot;&gt;NULL&lt;/span&gt;, &lt;span class=&quot;literal&quot;&gt;NULL&lt;/span&gt;, &lt;span class=&quot;literal&quot;&gt;NULL&lt;/span&gt;)&lt;span class=&quot;comment&quot;&gt;;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;while&lt;/span&gt; (&lt;span class=&quot;literal&quot;&gt;true&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;built_in&quot;&gt;sleep&lt;/span&gt;(&lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;)&lt;span class=&quot;comment&quot;&gt;;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;comment&quot;&gt;;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;编译运行：&lt;/p&gt;
&lt;figure class=&quot;highlight vala&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#  g++ prctl_main.cc   -o  prctl_main -std=c++11&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;# ./prctl_main&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;然后我们查看一下进程的名字：&lt;/p&gt;
&lt;figure class=&quot;highlight angelscript&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;# ps -ef |grep prctl&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;root     &lt;span class=&quot;number&quot;&gt;20758&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;12289&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;17&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;39&lt;/span&gt; pts/&lt;span class=&quot;number&quot;&gt;3&lt;/span&gt;    &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; ./prctl_main&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;root     &lt;span class=&quot;number&quot;&gt;20791&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;20422&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;17&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;39&lt;/span&gt; pts/&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;    &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; grep --color=&lt;span class=&quot;built_in&quot;&gt;auto&lt;/span&gt; prctl&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;可以看到 ps 看到的进程名依然是 &lt;code&gt;prctl_main&lt;/code&gt; 而不是 &lt;code&gt;prctl_new_name&lt;/code&gt;。那么 &lt;code&gt;prctl&lt;/code&gt; 函数到底修改了哪里呢？ ps 命令又是从哪里读取的进程名呢？&lt;/p&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="fork" scheme="https://rebootcat.com/tags/fork/"/>
    
      <category term="setproctitle" scheme="https://rebootcat.com/tags/setproctitle/"/>
    
      <category term="ps" scheme="https://rebootcat.com/tags/ps/"/>
    
      <category term="cmdline" scheme="https://rebootcat.com/tags/cmdline/"/>
    
  </entry>
  
  <entry>
    <title>模仿nginx修改进程名</title>
    <link href="https://rebootcat.com/2020/10/25/setproctitle/"/>
    <id>https://rebootcat.com/2020/10/25/setproctitle/</id>
    <published>2020-10-25T03:23:58.000Z</published>
    <updated>2020-10-28T15:09:43.235Z</updated>
    
    <content type="html"><![CDATA[<h1 id="nginx-进程名"><a href="#nginx-进程名" class="headerlink" title="nginx 进程名"></a>nginx 进程名</h1><p>使用 nginx 的过程中，我们经常看到 nginx 的进程名是不同的，如下：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ ps -ef |grep nginx </span><br><span class="line">smaug    <span class="number">1183</span>  <span class="number">1115</span>  <span class="number">0</span> <span class="number">05</span>:<span class="number">46</span> pts/<span class="number">2</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> nginx</span><br><span class="line">root     <span class="number">14201</span>     <span class="number">1</span>  <span class="number">0</span>  <span class="number">2019</span> ?        <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> nginx: master process ./sbin/nginx</span><br><span class="line">nobody   <span class="number">28887</span> <span class="number">14201</span>  <span class="number">0</span> Oct14 ?        <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> nginx: worker process</span><br><span class="line">nobody   <span class="number">28888</span> <span class="number">14201</span>  <span class="number">0</span> Oct14 ?        <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> nginx: worker process</span><br></pre></td></tr></table></figure><p>可以看到 nginx 的进程名是不同的，那么它是怎么做到的呢？</p><h1 id="argv-0"><a href="#argv-0" class="headerlink" title="argv[0]"></a>argv[0]</h1><p>首先来看一下 C 语言中的 main 函数的定义：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[])</span></span>;</span><br></pre></td></tr></table></figure><p>这个应该大家都是比较熟悉的，argc 表示命令行参数个数， argv 保存了各个命令行参数的内容。<strong>其中 <code>argv[0]</code> 表示的是进程的名字</strong>，这就是修改进程名的关键点所在。</p><p><strong>只需要修改  argv[0] 的值即可完成修改进程名</strong>。</p><h2 id="hello-world"><a href="#hello-world" class="headerlink" title="hello world"></a>hello world</h2><p>下面以程序员经典入门代码为例说明：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// filename: hello_world_setproctitle.cc</span></span><br><span class="line"><span class="comment">// build: g++ hello_world_setproctitle.cc -o hello_world_setproctitle</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[])</span> </span>&#123;</span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"hello world\n"</span>);</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="comment">// block here</span></span><br><span class="line">        <span class="keyword">char</span> c = getchar();</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译运行：</p><figure class="highlight brainfuck"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">g</span>++ <span class="comment">hello_world_setproctitle</span><span class="string">.</span><span class="comment">cc</span> <span class="literal">-</span><span class="comment">o</span> <span class="comment">hello_world_setproctitle</span></span><br><span class="line"><span class="comment"></span><span class="string">.</span><span class="comment">/hello_world_setproctitle</span></span><br></pre></td></tr></table></figure><p>查看一下进程名：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep hello_world</span><br><span class="line">root     <span class="number">26356</span> <span class="number">12289</span>  <span class="number">0</span> <span class="number">14</span>:<span class="number">17</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> ./hello_world_setproctitle</span><br><span class="line">root     <span class="number">26366</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">14</span>:<span class="number">18</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> hello_world</span><br></pre></td></tr></table></figure><a id="more"></a><p>可以看到进程名是 <code>hello_world_setproctitle</code>，接下来我们修改一下 argv[0] 的值，代码如下：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// filename: hello_world_setproctitle.cc</span></span><br><span class="line"><span class="comment">// build: g++ hello_world_setproctitle.cc -o hello_world_setproctitle</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[])</span> </span>&#123;</span><br><span class="line">    <span class="comment">// new process name</span></span><br><span class="line">    <span class="keyword">const</span> <span class="keyword">char</span>* new_name = <span class="string">"new_new_hello_world_setproctitle"</span>;</span><br><span class="line">    <span class="built_in">strcpy</span>(argv[<span class="number">0</span>], new_name);</span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"hello world\n"</span>);</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="comment">// block here</span></span><br><span class="line">        <span class="keyword">char</span> c = getchar();</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译运行之后，查看进程名：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep hello_world</span><br><span class="line">root     <span class="number">26750</span> <span class="number">12289</span>  <span class="number">0</span> <span class="number">14</span>:<span class="number">23</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> new_new_hello_world_setproctitle</span><br><span class="line">root     <span class="number">26754</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">14</span>:<span class="number">23</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> hello_world</span><br></pre></td></tr></table></figure><p>可以看到进程名已经修改为 <code>new_new_hello_world_setproctitle</code> 了。</p><p>是不是很简单？</p><p><strong>不过上面的代码是有一定的风险的，如果新的进程名超过了原来 argv[0] 的长度，就可能会影响到后面的 environ 的内容</strong>。</p><h1 id="环境变量-environ"><a href="#环境变量-environ" class="headerlink" title="环境变量 environ"></a>环境变量 environ</h1><p>C 语言中 main 函数的定义还有一个:</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[], <span class="keyword">char</span> *envp[])</span></span>;</span><br></pre></td></tr></table></figure><p>这个版本提供了第三个参数，大多数 Unix 系统支持，但是  POSIX.1 不建议这么做，如果要访问环境变量建议使用 <code>getenv</code> 和 <code>putenv</code> 接口。这里就不展开讲了。</p><p>envp 这个参数表示环境变量，<strong>每一个进程都有与之相关的环境变量，其中每个字符串都以(name=value)形式定义，并且 envp 的地址紧跟在 argv 之后</strong>。</p><h2 id="hello-world-environ"><a href="#hello-world-environ" class="headerlink" title="hello world environ"></a>hello world environ</h2><p>接下来我们打印一下 envp 这个参数的值，基于上面的代码，简单修改一下：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// filename: hello_world_setproctitle.cc</span></span><br><span class="line"><span class="comment">// build: g++ hello_world_setproctitle.cc -o hello_world_setproctitle</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[], <span class="keyword">char</span> *envp[])</span> </span>&#123;</span><br><span class="line">    <span class="keyword">int</span> i;</span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; i &lt; argc; ++i) &#123;</span><br><span class="line">        <span class="built_in">printf</span>(<span class="string">"mem:%p len:%d argv[%d]: %s\n"</span>, argv[i], <span class="built_in">strlen</span>(argv[i]), i, argv[i]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"\n"</span>);</span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; envp[i] != <span class="literal">NULL</span>; ++i) &#123;</span><br><span class="line">        <span class="built_in">printf</span>(<span class="string">"mem:%p len:%d envp[%d]: %s\n"</span>, envp[i], <span class="built_in">strlen</span>(envp[i]), i, envp[i]);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">const</span> <span class="keyword">char</span>* new_name = <span class="string">"new_new_hello_world_setproctitle"</span>;</span><br><span class="line">    <span class="built_in">strcpy</span>(argv[<span class="number">0</span>], new_name);</span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"hello world\n"</span>);</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="comment">// block here</span></span><br><span class="line">        <span class="keyword">char</span> c = getchar();</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>上面的代码同时也打印了每个参数的地址以及长度，编译并执行：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line"># ./hello_world_setproctitle <span class="number">1</span> <span class="number">22</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7544</span> len:<span class="number">26</span> argv[<span class="number">0</span>]: ./hello_world_setproctitle</span><br><span class="line">mem:<span class="number">0x7ffc84cf755f</span> len:<span class="number">1</span> argv[<span class="number">1</span>]: <span class="number">1</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7561</span> len:<span class="number">2</span> argv[<span class="number">2</span>]: <span class="number">22</span></span><br><span class="line"></span><br><span class="line">mem:<span class="number">0x7ffc84cf7564</span> len:<span class="number">19</span> envp[<span class="number">0</span>]: XDG_SESSION_ID=<span class="number">2554</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7578</span> len:<span class="number">13</span> envp[<span class="number">1</span>]: HOSTNAME=Jiao</span><br><span class="line">mem:<span class="number">0x7ffc84cf7586</span> len:<span class="number">23</span> envp[<span class="number">2</span>]: SELINUX_ROLE_REQUESTED=</span><br><span class="line">mem:<span class="number">0x7ffc84cf759e</span> len:<span class="number">19</span> envp[<span class="number">3</span>]: TERM=xterm<span class="number">-256</span>color</span><br><span class="line">mem:<span class="number">0x7ffc84cf75b2</span> len:<span class="number">15</span> envp[<span class="number">4</span>]: SHELL=/bin/bash</span><br><span class="line">mem:<span class="number">0x7ffc84cf75c2</span> len:<span class="number">13</span> envp[<span class="number">5</span>]: HISTSIZE=<span class="number">1000</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf75d0</span> len:<span class="number">31</span> envp[<span class="number">6</span>]: SSH_CLIENT=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">61311</span> <span class="number">22</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf75f0</span> len:<span class="number">26</span> envp[<span class="number">7</span>]: SELINUX_USE_CURRENT_RANGE=</span><br><span class="line">mem:<span class="number">0x7ffc84cf760b</span> len:<span class="number">75</span> envp[<span class="number">8</span>]: ANDROID_NDK_TOOLCHAIN_ROOT=/root/smaug/software/android-ndk-r16b/toolchains</span><br><span class="line">mem:<span class="number">0x7ffc84cf7657</span> len:<span class="number">12</span> envp[<span class="number">9</span>]: OLDPWD=/root</span><br><span class="line">mem:<span class="number">0x7ffc84cf7664</span> len:<span class="number">18</span> envp[<span class="number">10</span>]: SSH_TTY=/dev/pts/<span class="number">3</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7677</span> len:<span class="number">9</span> envp[<span class="number">11</span>]: USER=root</span><br><span class="line">mem:<span class="number">0x7ffc84cf7681</span> len:<span class="number">75</span> envp[<span class="number">12</span>]: LD_LIBRARY_PATH=/usr/local/lib:/usr/local/python3/lib::/usr/local/topio/lib</span><br><span class="line">mem:<span class="number">0x7ffc84cf76cd</span> len:<span class="number">1719</span> envp[<span class="number">13</span>]: LS_COLORS=rs=<span class="number">0</span>:di=<span class="number">38</span>;<span class="number">5</span>;<span class="number">27</span>:ln=<span class="number">38</span>;<span class="number">5</span>;<span class="number">51</span>:mh=<span class="number">44</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:pi=<span class="number">40</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">11</span>:so=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:<span class="keyword">do</span>=<span class="number">38</span>;<span class="number">5</span>;<span class="number">5</span>:bd=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">11</span>:cd=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">3</span>:<span class="keyword">or</span>=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:mi=<span class="number">05</span>;<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:su=<span class="number">48</span>;<span class="number">5</span>;<span class="number">196</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:sg=<span class="number">48</span>;<span class="number">5</span>;<span class="number">11</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">16</span>:ca=<span class="number">48</span>;<span class="number">5</span>;<span class="number">196</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">226</span>:tw=<span class="number">48</span>;<span class="number">5</span>;<span class="number">10</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">16</span>:ow=<span class="number">48</span>;<span class="number">5</span>;<span class="number">10</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">21</span>:st=<span class="number">48</span>;<span class="number">5</span>;<span class="number">21</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:ex=<span class="number">38</span>;<span class="number">5</span>;<span class="number">34</span>:*.tar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tgz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.arc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.arj=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.taz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lha=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lz4=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzh=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzma=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tlz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.txz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tzo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.t7z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.zip=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.Z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.dz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.gz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lrz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.xz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.bz2=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.bz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tbz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tbz2=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.deb=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rpm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.jar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.war=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.ear=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.sar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.alz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.ace=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.zoo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.cpio=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*<span class="number">.7</span>z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.cab=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.jpg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.jpeg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.gif=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.bmp=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pbm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pgm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ppm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tga=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xbm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xpm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tif=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tiff=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.png=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.svg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.svgz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mng=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pcx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mov=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mpg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mpeg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.m2v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mkv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.webm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mp4=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.m4v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mp4v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.vob=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.qt=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.nuv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.wmv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.asf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.rm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.rmvb=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.flc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.avi=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.fli=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.flv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.gl=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.dl=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xcf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xwd=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.yuv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.cgm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.emf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.axv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.anx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.aac=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.au=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.flac=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mid=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.midi=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mka=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mp3=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mpc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.ogg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.ra=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.wav=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.axa=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.oga=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.spx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.xspf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:</span><br><span class="line">mem:<span class="number">0x7ffc84cf7d85</span> len:<span class="number">25</span> envp[<span class="number">14</span>]: MAIL=/var/spool/mail/root</span><br><span class="line">mem:<span class="number">0x7ffc84cf7d9f</span> len:<span class="number">193</span> envp[<span class="number">15</span>]: PATH=/root/.cargo/bin:/root/.cargo/bin:/usr/local/vim/bin:/usr/local/bin:/usr/loca/python3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/go/bin:/root/bin:/usr/local/topio/bin</span><br><span class="line">mem:<span class="number">0x7ffc84cf7e61</span> len:<span class="number">21</span> envp[<span class="number">16</span>]: MARKPATH=/root/.marks</span><br><span class="line">mem:<span class="number">0x7ffc84cf7e77</span> len:<span class="number">14</span> envp[<span class="number">17</span>]: PWD=/root/temp</span><br><span class="line">mem:<span class="number">0x7ffc84cf7e86</span> len:<span class="number">16</span> envp[<span class="number">18</span>]: LANG=zh_CN.UTF<span class="number">-8</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7e97</span> len:<span class="number">38</span> envp[<span class="number">19</span>]: PS1=\[\e[<span class="number">32</span>;<span class="number">1</span>m\][\<span class="symbol">u@</span>\h \w]\$ \[\e[<span class="number">0</span>m\]</span><br><span class="line">mem:<span class="number">0x7ffc84cf7ebe</span> len:<span class="number">24</span> envp[<span class="number">20</span>]: SELINUX_LEVEL_REQUESTED=</span><br><span class="line">mem:<span class="number">0x7ffc84cf7ed7</span> len:<span class="number">22</span> envp[<span class="number">21</span>]: HISTCONTROL=ignoredups</span><br><span class="line">mem:<span class="number">0x7ffc84cf7eee</span> len:<span class="number">7</span> envp[<span class="number">22</span>]: SHLVL=<span class="number">1</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7ef6</span> len:<span class="number">10</span> envp[<span class="number">23</span>]: HOME=/root</span><br><span class="line">mem:<span class="number">0x7ffc84cf7f01</span> len:<span class="number">12</span> envp[<span class="number">24</span>]: LOGNAME=root</span><br><span class="line">mem:<span class="number">0x7ffc84cf7f0e</span> len:<span class="number">47</span> envp[<span class="number">25</span>]: SSH_CONNECTION=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">61311</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span> <span class="number">22</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7f3e</span> len:<span class="number">16</span> envp[<span class="number">26</span>]: GOPATH=/root/.go</span><br><span class="line">mem:<span class="number">0x7ffc84cf7f4f</span> len:<span class="number">34</span> envp[<span class="number">27</span>]: LESSOPEN=||/usr/bin/lesspipe.sh %s</span><br><span class="line">mem:<span class="number">0x7ffc84cf7f72</span> len:<span class="number">49</span> envp[<span class="number">28</span>]: ANDROID_NDK=/root/smaug/software/android-ndk-r16b</span><br><span class="line">mem:<span class="number">0x7ffc84cf7fa4</span> len:<span class="number">27</span> envp[<span class="number">29</span>]: XDG_RUNTIME_DIR=/run/user/<span class="number">0</span></span><br><span class="line">mem:<span class="number">0x7ffc84cf7fc0</span> len:<span class="number">28</span> envp[<span class="number">30</span>]: _=./hello_world_setproctitle</span><br><span class="line">hello world</span><br></pre></td></tr></table></figure><p>可以看到上述各个 argv 的值以及 envp 参数的内容。</p><p>这里需要重点注意一下最后一个 argv[2] 参数以及第一个 envp[0] 参数的地址：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">mem:<span class="number">0x7ffc84cf7561</span> len:<span class="number">2</span> argv[<span class="number">2</span>]: <span class="number">22</span></span><br><span class="line"></span><br><span class="line">mem:<span class="number">0x7ffc84cf7564</span> len:<span class="number">19</span> envp[<span class="number">0</span>]: XDG_SESSION_ID=<span class="number">2554</span></span><br></pre></td></tr></table></figure><p>0x7ffc84cf7564 正好等于 0x7ffc84cf7561 + 3 （argv[2] 的长度加上最后一个 ‘\0’)。可以多试几次，不同的参数个数验证下这个。</p><p>所以 environ 的地址（envp[0] 的地址）是紧跟在 argv 后面的，那么前面提到的如果当新的进程名长度超出 argv 的长度后，可能就会覆盖后面的 environ 内容，导致其他一些问题。</p><h2 id="新进程名过长，覆盖-envp"><a href="#新进程名过长，覆盖-envp" class="headerlink" title="新进程名过长，覆盖 envp"></a>新进程名过长，覆盖 envp</h2><p>修改如上代码：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// filename: hello_world_setproctitle.cc</span></span><br><span class="line"><span class="comment">// build: g++ hello_world_setproctitle.cc -o hello_world_setproctitle</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span> *argv[], <span class="keyword">char</span> *envp[])</span> </span>&#123;</span><br><span class="line">    <span class="keyword">const</span> <span class="keyword">char</span>* new_name = <span class="string">"new_new_hello_world_setproctitlexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"</span>;</span><br><span class="line">    <span class="built_in">strcpy</span>(argv[<span class="number">0</span>], new_name);</span><br><span class="line">    <span class="keyword">int</span> i;</span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; i &lt; argc; ++i) &#123;</span><br><span class="line">        <span class="built_in">printf</span>(<span class="string">"mem:%p len:%d argv[%d]: %s\n"</span>, argv[i], <span class="built_in">strlen</span>(argv[i]), i, argv[i]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"\n"</span>);</span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; envp[i] != <span class="literal">NULL</span>; ++i) &#123;</span><br><span class="line">        <span class="built_in">printf</span>(<span class="string">"mem:%p len:%d envp[%d]: %s\n"</span>, envp[i], <span class="built_in">strlen</span>(envp[i]), i, envp[i]);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"hello world\n"</span>);</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="comment">// block here</span></span><br><span class="line">        <span class="keyword">char</span> c = getchar();</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译运行：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"># ./hello_world_setproctitle</span><br><span class="line">mem:<span class="number">0x7fff872c3549</span> len:<span class="number">125</span> argv[<span class="number">0</span>]: new_new_hello_world_setproctitlexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</span><br><span class="line"></span><br><span class="line">mem:<span class="number">0x7fff872c3564</span> len:<span class="number">98</span> envp[<span class="number">0</span>]: titlexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</span><br><span class="line">mem:<span class="number">0x7fff872c3578</span> len:<span class="number">78</span> envp[<span class="number">1</span>]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</span><br><span class="line">mem:<span class="number">0x7fff872c3586</span> len:<span class="number">64</span> envp[<span class="number">2</span>]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</span><br><span class="line">mem:<span class="number">0x7fff872c359e</span> len:<span class="number">40</span> envp[<span class="number">3</span>]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</span><br><span class="line">mem:<span class="number">0x7fff872c35b2</span> len:<span class="number">20</span> envp[<span class="number">4</span>]: xxxxxxxxxxxxxxxxxxxx</span><br><span class="line">mem:<span class="number">0x7fff872c35c2</span> len:<span class="number">4</span> envp[<span class="number">5</span>]: xxxx</span><br><span class="line">mem:<span class="number">0x7fff872c35d0</span> len:<span class="number">31</span> envp[<span class="number">6</span>]: SSH_CLIENT=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">55187</span> <span class="number">22</span></span><br><span class="line">mem:<span class="number">0x7fff872c35f0</span> len:<span class="number">26</span> envp[<span class="number">7</span>]: SELINUX_USE_CURRENT_RANGE=</span><br><span class="line">mem:<span class="number">0x7fff872c360b</span> len:<span class="number">75</span> envp[<span class="number">8</span>]: ANDROID_NDK_TOOLCHAIN_ROOT=/root/smaug/software/android-ndk-r16b/toolchains</span><br><span class="line">mem:<span class="number">0x7fff872c3657</span> len:<span class="number">12</span> envp[<span class="number">9</span>]: OLDPWD=/root</span><br><span class="line">mem:<span class="number">0x7fff872c3664</span> len:<span class="number">18</span> envp[<span class="number">10</span>]: SSH_TTY=/dev/pts/<span class="number">1</span></span><br><span class="line">mem:<span class="number">0x7fff872c3677</span> len:<span class="number">9</span> envp[<span class="number">11</span>]: USER=root</span><br></pre></td></tr></table></figure><p>可以看到，上面打印出来的 envp[0], envp[1].. envp[5] 都已经被覆盖了。</p><p>所以，<strong>通过 argv[0] 修改进程名，如果新进程名过长，需要考虑到 envp 的覆盖问题，通常做法是把 envp 的内容先保存，然后指向新的内存，再把保存的环境变量复制到新的内存，然后再去修改 argv[0]</strong>。</p><h1 id="setproctitle-修改进程名"><a href="#setproctitle-修改进程名" class="headerlink" title="setproctitle 修改进程名"></a>setproctitle 修改进程名</h1><p>可以参考 nginx 的源码: <a href="https://github.com/nginx/nginx/blob/master/src/os/unix/ngx_setproctitle.c" target="_blank" rel="noopener">https://github.com/nginx/nginx/blob/master/src/os/unix/ngx_setproctitle.c</a></p><p>下面直接上源码，源码可以在我的 github 找到：</p><p><a href="https://github.com/smaugx/setproctitle" target="_blank" rel="noopener">https://github.com/smaugx/setproctitle</a></p><blockquote><p>setproctitle.h</p></blockquote><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// author: smaug</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">pragma</span> once</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">ifndef</span> _SMAUG_SETPROCTITLE_H_INCLUDED_</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> _SMAUG_SETPROCTITLE_H_INCLUDED_</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> SMAUG_PROCTITLE_ERROR  -1</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> SMAUG_PROCTITLE_OK   0</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">smaug_init_setproctitle</span><span class="params">()</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">smaug_setproctitle</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *title)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">endif</span> <span class="comment">/* _SMAUG_SETPROCTITLE_H_INCLUDED_ */</span></span></span><br></pre></td></tr></table></figure><blockquote><p>setproctitle.cc</p></blockquote><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// author: smaug</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">"setproctitle.h"</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdio&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdlib&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstdint&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * To change the process title in Linux and Solaris we have to set argv[1]</span></span><br><span class="line"><span class="comment"> * to NULL and to copy the title to the same place where the argv[0] points to.</span></span><br><span class="line"><span class="comment"> * However, argv[0] may be too small to hold a new title.  Fortunately, Linux</span></span><br><span class="line"><span class="comment"> * and Solaris store argv[] and environ[] one after another.  So we should</span></span><br><span class="line"><span class="comment"> * ensure that is the continuous memory and then we allocate the new memory</span></span><br><span class="line"><span class="comment"> * for environ[] and copy it.  After this we could use the memory starting</span></span><br><span class="line"><span class="comment"> * from argv[0] for our process title.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">extern</span> <span class="keyword">char</span> **environ;</span><br><span class="line"><span class="comment">// same as argv</span></span><br><span class="line"><span class="keyword">extern</span> <span class="keyword">char</span>** smaug_os_argv;</span><br><span class="line"><span class="keyword">static</span> <span class="keyword">char</span> *smaug_os_argv_last;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">smaug_init_setproctitle</span><span class="params">()</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">    <span class="keyword">char</span> *p;</span><br><span class="line">    <span class="keyword">int</span> i = <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">uint32_t</span> <span class="built_in">size</span> = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 统计所有 environ 环境变量的长度(+1 是表示每个字符串后的 '\0')</span></span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; environ[i]; i++) &#123;</span><br><span class="line">        <span class="built_in">size</span> += <span class="built_in">strlen</span>(environ[i]) + <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 重新分配一段内存，待会用来保存 environ 指向的环境变量</span></span><br><span class="line">    p = (<span class="keyword">char</span>*)<span class="built_in">malloc</span>(<span class="built_in">size</span>);</span><br><span class="line">    <span class="keyword">if</span> (p == <span class="literal">NULL</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> SMAUG_PROCTITLE_ERROR;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    smaug_os_argv_last = smaug_os_argv[<span class="number">0</span>];</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 遍历 argv，完成后指向 argv 最后一个参数的 '\0' 的下一个位置，也就是 envp[0] 的地址</span></span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; smaug_os_argv[i]; i++) &#123;</span><br><span class="line">        <span class="keyword">if</span> (smaug_os_argv_last == smaug_os_argv[i]) &#123;</span><br><span class="line">            smaug_os_argv_last = smaug_os_argv[i] + <span class="built_in">strlen</span>(smaug_os_argv[i]) + <span class="number">1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> (i = <span class="number">0</span>; environ[i]; i++) &#123;</span><br><span class="line">        <span class="keyword">if</span> (smaug_os_argv_last == environ[i]) &#123;</span><br><span class="line"></span><br><span class="line">            <span class="comment">// 包括 '\0'</span></span><br><span class="line">            <span class="built_in">size</span> = <span class="built_in">strlen</span>(environ[i]) + <span class="number">1</span>;</span><br><span class="line">            <span class="comment">// smaug_os_argv_last 接着往前递进, 经过每一个环境变量</span></span><br><span class="line">            smaug_os_argv_last = environ[i] + <span class="built_in">size</span>;</span><br><span class="line"></span><br><span class="line">            <span class="comment">// 拷贝每一个 环境变量到上面刚申请的新内存中</span></span><br><span class="line">            <span class="built_in">strncpy</span>(p, environ[i], <span class="built_in">size</span>);</span><br><span class="line">            <span class="comment">// 修改 environ[i] 指向这块新的内存地址（避免后续修改 argv[0] 造成覆盖)</span></span><br><span class="line">            environ[i] = (<span class="keyword">char</span> *) p;</span><br><span class="line">            <span class="comment">// 新内存指针 p 前移，称为下一轮 environ[i+1] 的值</span></span><br><span class="line">            p += <span class="built_in">size</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 指针往前移 1 个字节，指向 最后一个环境变量的 '\0' 位置</span></span><br><span class="line">    smaug_os_argv_last--;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> SMAUG_PROCTITLE_OK;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">smaug_setproctitle</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *title)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">    <span class="comment">// 设置 argv[1] = NULL 通常不必要，加上更安全</span></span><br><span class="line">    smaug_os_argv[<span class="number">1</span>] = <span class="literal">NULL</span>;</span><br><span class="line">    <span class="comment">/*</span></span><br><span class="line"><span class="comment">    char new_title[1024];</span></span><br><span class="line"><span class="comment">    bzero(new_title, sizeof(new_title));</span></span><br><span class="line"><span class="comment">    sprintf(new_title, "%s%s", "smaug: ", title);</span></span><br><span class="line"><span class="comment">    */</span></span><br><span class="line"></span><br><span class="line">    <span class="comment">// smaug_os_argv_last - smaug_os_argv[0] 就是最初 argv + envp 所有字符串的长度, 拷贝新进程名到 argv[0]</span></span><br><span class="line">    <span class="built_in">strncpy</span>(smaug_os_argv[<span class="number">0</span>], title, smaug_os_argv_last - smaug_os_argv[<span class="number">0</span>]);</span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">ifdef</span> DEBUG</span></span><br><span class="line">    <span class="built_in">printf</span>(<span class="string">"set title:%s\n"</span>, new_title);</span><br><span class="line"><span class="meta">#<span class="meta-keyword">endif</span></span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>编译运行：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br></pre></td><td class="code"><pre><span class="line"># sh build.sh</span><br><span class="line"># ./main</span><br><span class="line"></span><br><span class="line">#######<span class="keyword">this</span> <span class="keyword">is</span> father process###########</span><br><span class="line">master mem:<span class="number">0x7ffce4714573</span> len:<span class="number">20</span> argv[<span class="number">0</span>]: setproctitle: master</span><br><span class="line"></span><br><span class="line">master mem:<span class="number">0x12e0c20</span> len:<span class="number">19</span> envp[<span class="number">0</span>]: XDG_SESSION_ID=<span class="number">2554</span></span><br><span class="line">master mem:<span class="number">0x12e0c34</span> len:<span class="number">13</span> envp[<span class="number">1</span>]: HOSTNAME=Jiao</span><br><span class="line">master mem:<span class="number">0x12e0c42</span> len:<span class="number">23</span> envp[<span class="number">2</span>]: SELINUX_ROLE_REQUESTED=</span><br><span class="line">master mem:<span class="number">0x12e0c5a</span> len:<span class="number">19</span> envp[<span class="number">3</span>]: TERM=xterm<span class="number">-256</span>color</span><br><span class="line">master mem:<span class="number">0x12e0c6e</span> len:<span class="number">15</span> envp[<span class="number">4</span>]: SHELL=/bin/bash</span><br><span class="line">master mem:<span class="number">0x12e0c7e</span> len:<span class="number">13</span> envp[<span class="number">5</span>]: HISTSIZE=<span class="number">1000</span></span><br><span class="line">master mem:<span class="number">0x12e0c8c</span> len:<span class="number">31</span> envp[<span class="number">6</span>]: SSH_CLIENT=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">61311</span> <span class="number">22</span></span><br><span class="line">master mem:<span class="number">0x12e0cac</span> len:<span class="number">26</span> envp[<span class="number">7</span>]: SELINUX_USE_CURRENT_RANGE=</span><br><span class="line">master mem:<span class="number">0x12e0cc7</span> len:<span class="number">75</span> envp[<span class="number">8</span>]: ANDROID_NDK_TOOLCHAIN_ROOT=/root/smaug/software/android-ndk-r16b/toolchains</span><br><span class="line">master mem:<span class="number">0x12e0d13</span> len:<span class="number">17</span> envp[<span class="number">9</span>]: OLDPWD=/root/temp</span><br><span class="line">master mem:<span class="number">0x12e0d25</span> len:<span class="number">18</span> envp[<span class="number">10</span>]: SSH_TTY=/dev/pts/<span class="number">3</span></span><br><span class="line">master mem:<span class="number">0x12e0d38</span> len:<span class="number">9</span> envp[<span class="number">11</span>]: USER=root</span><br><span class="line">master mem:<span class="number">0x12e0d42</span> len:<span class="number">75</span> envp[<span class="number">12</span>]: LD_LIBRARY_PATH=/usr/local/lib:/usr/local/python3/lib::/usr/local/topio/lib</span><br><span class="line">master mem:<span class="number">0x12e0d8e</span> len:<span class="number">1719</span> envp[<span class="number">13</span>]: LS_COLORS=rs=<span class="number">0</span>:di=<span class="number">38</span>;<span class="number">5</span>;<span class="number">27</span>:ln=<span class="number">38</span>;<span class="number">5</span>;<span class="number">51</span>:mh=<span class="number">44</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:pi=<span class="number">40</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">11</span>:so=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:<span class="keyword">do</span>=<span class="number">38</span>;<span class="number">5</span>;<span class="number">5</span>:bd=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">11</span>:cd=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">3</span>:<span class="keyword">or</span>=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:mi=<span class="number">05</span>;<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:su=<span class="number">48</span>;<span class="number">5</span>;<span class="number">196</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:sg=<span class="number">48</span>;<span class="number">5</span>;<span class="number">11</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">16</span>:ca=<span class="number">48</span>;<span class="number">5</span>;<span class="number">196</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">226</span>:tw=<span class="number">48</span>;<span class="number">5</span>;<span class="number">10</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">16</span>:ow=<span class="number">48</span>;<span class="number">5</span>;<span class="number">10</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">21</span>:st=<span class="number">48</span>;<span class="number">5</span>;<span class="number">21</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:ex=<span class="number">38</span>;<span class="number">5</span>;<span class="number">34</span>:*.tar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tgz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.arc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.arj=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.taz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lha=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lz4=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzh=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzma=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tlz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.txz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tzo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.t7z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.zip=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.Z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.dz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.gz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lrz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.xz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.bz2=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.bz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tbz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tbz2=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.deb=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rpm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.jar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.war=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.ear=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.sar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.alz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.ace=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.zoo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.cpio=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*<span class="number">.7</span>z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.cab=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.jpg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.jpeg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.gif=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.bmp=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pbm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pgm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ppm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tga=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xbm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xpm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tif=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tiff=<span class="number">38</span>;</span><br><span class="line"><span class="number">5</span>;<span class="number">13</span>:*.png=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.svg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.svgz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mng=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pcx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mov=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mpg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mpeg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.m2v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mkv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.webm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mp4=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.m4v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mp4v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.vob=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.qt=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.nuv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.wmv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.asf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.rm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.rmvb=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.flc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.avi=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.fli=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.flv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.gl=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.dl=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xcf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xwd=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.yuv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.cgm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.emf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.axv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.anx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.aac=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.au=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.flac=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mid=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.midi=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mka=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mp3=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mpc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.ogg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.ra=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.wav=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.axa=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.oga=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.spx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.xspf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:</span><br><span class="line"></span><br><span class="line">master mem:<span class="number">0x12e1446</span> len:<span class="number">25</span> envp[<span class="number">14</span>]: MAIL=/var/spool/mail/root</span><br><span class="line">master mem:<span class="number">0x12e1460</span> len:<span class="number">193</span> envp[<span class="number">15</span>]: PATH=/root/.cargo/bin:/root/.cargo/bin:/usr/local/vim/bin:/usr/local/bin:/usr/loca/python3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/go/bin:/root/bin:/usr/local/topio/bin</span><br><span class="line">master mem:<span class="number">0x12e1522</span> len:<span class="number">21</span> envp[<span class="number">16</span>]: MARKPATH=/root/.marks</span><br><span class="line">#######<span class="keyword">this</span> <span class="keyword">is</span> child process###########</span><br><span class="line">master mem:<span class="number">0x12e1538</span> len:<span class="number">27</span> envp[<span class="number">17</span>]: PWD=/root/temp/setproctitle</span><br><span class="line">master mem:<span class="number">0x12e1554</span> len:<span class="number">16</span> envp[<span class="number">18</span>]: LANG=zh_CN.UTF<span class="number">-8</span></span><br><span class="line">worker mem:<span class="number">0x7ffce4714573</span> len:<span class="number">20</span> argv[<span class="number">0</span>]: setproctitle: worker</span><br><span class="line"></span><br><span class="line">worker mem:<span class="number">0x12e0c20</span> len:<span class="number">19</span> envp[<span class="number">0</span>]: XDG_SESSION_ID=<span class="number">2554</span></span><br><span class="line">worker mem:<span class="number">0x12e0c34</span> len:<span class="number">13</span> envp[<span class="number">1</span>]: HOSTNAME=Jiao</span><br><span class="line">worker mem:<span class="number">0x12e0c42</span> len:<span class="number">23</span> envp[<span class="number">2</span>]: SELINUX_ROLE_REQUESTED=</span><br><span class="line">worker mem:<span class="number">0x12e0c5a</span> len:<span class="number">19</span> envp[<span class="number">3</span>]: TERM=xterm<span class="number">-256</span>color</span><br><span class="line">worker mem:<span class="number">0x12e0c6e</span> len:<span class="number">15</span> envp[<span class="number">4</span>]: SHELL=/bin/bash</span><br><span class="line">worker mem:<span class="number">0x12e0c7e</span> len:<span class="number">13</span> envp[<span class="number">5</span>]: HISTSIZE=<span class="number">1000</span></span><br><span class="line">worker mem:<span class="number">0x12e0c8c</span> len:<span class="number">31</span> envp[<span class="number">6</span>]: SSH_CLIENT=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">61311</span> <span class="number">22</span></span><br><span class="line">worker mem:<span class="number">0x12e0cac</span> len:<span class="number">26</span> envp[<span class="number">7</span>]: SELINUX_USE_CURRENT_RANGE=</span><br><span class="line">worker mem:<span class="number">0x12e0cc7</span> len:<span class="number">75</span> envp[<span class="number">8</span>]: ANDROID_NDK_TOOLCHAIN_ROOT=/root/smaug/software/android-ndk-r16b/toolchains</span><br><span class="line">worker mem:<span class="number">0x12e0d13</span> len:<span class="number">17</span> envp[<span class="number">9</span>]: OLDPWD=/root/temp</span><br><span class="line">worker mem:<span class="number">0x12e0d25</span> len:<span class="number">18</span> envp[<span class="number">10</span>]: SSH_TTY=/dev/pts/<span class="number">3</span></span><br><span class="line">worker mem:<span class="number">0x12e0d38</span> len:<span class="number">9</span> envp[<span class="number">11</span>]: USER=root</span><br><span class="line">worker mem:<span class="number">0x12e0d42</span> len:<span class="number">75</span> envp[<span class="number">12</span>]: LD_LIBRARY_PATH=/usr/local/lib:/usr/local/python3/lib::/usr/local/topio/lib</span><br><span class="line">worker mem:<span class="number">0x12e0d8e</span> len:<span class="number">1719</span> envp[<span class="number">13</span>]: LS_COLORS=rs=<span class="number">0</span>:di=<span class="number">38</span>;<span class="number">5</span>;<span class="number">27</span>:ln=<span class="number">38</span>;<span class="number">5</span>;<span class="number">51</span>:mh=<span class="number">44</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:pi=<span class="number">40</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">11</span>:so=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:<span class="keyword">do</span>=<span class="number">38</span>;<span class="number">5</span>;<span class="number">5</span>:bd=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">11</span>:cd=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">3</span>:<span class="keyword">or</span>=<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:mi=<span class="number">05</span>;<span class="number">48</span>;<span class="number">5</span>;<span class="number">232</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:su=<span class="number">48</span>;<span class="number">5</span>;<span class="number">196</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:sg=<span class="number">48</span>;<span class="number">5</span>;<span class="number">11</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">16</span>:ca=<span class="number">48</span>;<span class="number">5</span>;<span class="number">196</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">226</span>:tw=<span class="number">48</span>;<span class="number">5</span>;<span class="number">10</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">16</span>:ow=<span class="number">48</span>;<span class="number">5</span>;<span class="number">10</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">21</span>:st=<span class="number">48</span>;<span class="number">5</span>;<span class="number">21</span>;<span class="number">38</span>;<span class="number">5</span>;<span class="number">15</span>:ex=<span class="number">38</span>;<span class="number">5</span>;<span class="number">34</span>:*.tar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tgz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.arc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.arj=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.taz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lha=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lz4=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzh=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzma=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tlz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.txz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tzo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.t7z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.zip=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.Z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.dz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.gz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lrz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.lzo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.xz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.bz2=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.bz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tbz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tbz2=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.tz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.deb=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rpm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.jar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.war=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.ear=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.sar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rar=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.alz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.ace=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.zoo=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.cpio=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*<span class="number">.7</span>z=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.rz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.cab=<span class="number">38</span>;<span class="number">5</span>;<span class="number">9</span>:*.jpg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.jpeg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.gif=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.bmp=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pbm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pgm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ppm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tga=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xbm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xpm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tif=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.tiff=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.png=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.svg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.svgz=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mng=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.pcx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mov=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mpg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mpeg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.m2v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mkv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.webm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mp4=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.m4v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.mp4v=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.vob=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.qt=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.nuv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.wmv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.asf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.rm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.rmvb=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.flc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.avi=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.fli=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.flv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.gl=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.dl=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xcf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.xwd=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.yuv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.cgm=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.emf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.axv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.anx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogv=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.ogx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">13</span>:*.aac=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.au=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.flac=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mid=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.midi=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mka=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mp3=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.mpc=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.ogg=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.ra=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.wav=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.axa=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.oga=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.spx=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:*.xspf=<span class="number">38</span>;<span class="number">5</span>;<span class="number">45</span>:</span><br><span class="line">worker mem:<span class="number">0x12e1446</span> len:<span class="number">25</span> envp[<span class="number">14</span>]: MAIL=/var/spool/mail/root</span><br><span class="line">worker mem:<span class="number">0x12e1460</span> len:<span class="number">193</span> envp[<span class="number">15</span>]: PATH=/root/.cargo/bin:/root/.cargo/bin:/usr/local/vim/bin:/usr/local/bin:/usr/loca/python3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/go/bin:/root/bin:/usr/local/topio/bin</span><br><span class="line">worker mem:<span class="number">0x12e1522</span> len:<span class="number">21</span> envp[<span class="number">16</span>]: MARKPATH=/root/.marks</span><br><span class="line">worker mem:<span class="number">0x12e1538</span> len:<span class="number">27</span> envp[<span class="number">17</span>]: PWD=/root/temp/setproctitle</span><br><span class="line">worker mem:<span class="number">0x12e1554</span> len:<span class="number">16</span> envp[<span class="number">18</span>]: LANG=zh_CN.UTF<span class="number">-8</span></span><br><span class="line">worker mem:<span class="number">0x12e1565</span> len:<span class="number">38</span> envp[<span class="number">19</span>]: PS1=\[\e[<span class="number">32</span>;<span class="number">1</span>m\][\<span class="symbol">u@</span>\h \w]\$ \[\e[<span class="number">0</span>m\]</span><br><span class="line">worker mem:<span class="number">0x12e158c</span> len:<span class="number">24</span> envp[<span class="number">20</span>]: SELINUX_LEVEL_REQUESTED=</span><br><span class="line">worker mem:<span class="number">0x12e15a5</span> len:<span class="number">22</span> envp[<span class="number">21</span>]: HISTCONTROL=ignoredups</span><br><span class="line">worker mem:<span class="number">0x12e15bc</span> len:<span class="number">7</span> envp[<span class="number">22</span>]: SHLVL=<span class="number">1</span></span><br><span class="line">worker mem:<span class="number">0x12e15c4</span> len:<span class="number">10</span> envp[<span class="number">23</span>]: HOME=/root</span><br><span class="line">worker mem:<span class="number">0x12e15cf</span> len:<span class="number">12</span> envp[<span class="number">24</span>]: LOGNAME=root</span><br><span class="line">worker mem:<span class="number">0x12e15dc</span> len:<span class="number">47</span> envp[<span class="number">25</span>]: SSH_CONNECTION=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">61311</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span> <span class="number">22</span></span><br><span class="line">worker mem:<span class="number">0x12e160c</span> len:<span class="number">16</span> envp[<span class="number">26</span>]: GOPATH=/root/.go</span><br><span class="line">master mem:<span class="number">0x12e1565</span> len:<span class="number">38</span> envp[<span class="number">19</span>]: PS1=\[\e[<span class="number">32</span>;<span class="number">1</span>m\][\<span class="symbol">u@</span>\h \w]\$ \[\e[<span class="number">0</span>m\]</span><br><span class="line">master mem:<span class="number">0x12e158c</span> len:<span class="number">24</span> envp[<span class="number">20</span>]: SELINUX_LEVEL_REQUESTED=</span><br><span class="line">master mem:<span class="number">0x12e15a5</span> len:<span class="number">22</span> envp[<span class="number">21</span>]: HISTCONTROL=ignoredups</span><br><span class="line">master mem:<span class="number">0x12e15bc</span> len:<span class="number">7</span> envp[<span class="number">22</span>]: SHLVL=<span class="number">1</span></span><br><span class="line">worker mem:<span class="number">0x12e161d</span> len:<span class="number">34</span> envp[<span class="number">27</span>]: LESSOPEN=||/usr/bin/lesspipe.sh %s</span><br><span class="line">worker mem:<span class="number">0x12e1640</span> len:<span class="number">49</span> envp[<span class="number">28</span>]: ANDROID_NDK=/root/smaug/software/android-ndk-r16b</span><br><span class="line">worker mem:<span class="number">0x12e1672</span> len:<span class="number">27</span> envp[<span class="number">29</span>]: XDG_RUNTIME_DIR=/run/user/<span class="number">0</span></span><br><span class="line">master mem:<span class="number">0x12e15c4</span> len:<span class="number">10</span> envp[<span class="number">23</span>]: HOME=/root</span><br><span class="line">master mem:<span class="number">0x12e15cf</span> len:<span class="number">12</span> envp[<span class="number">24</span>]: LOGNAME=root</span><br><span class="line">master mem:<span class="number">0x12e15dc</span> len:<span class="number">47</span> envp[<span class="number">25</span>]: SSH_CONNECTION=<span class="number">192.168</span><span class="number">.1</span><span class="number">.3</span> <span class="number">61311</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.4</span> <span class="number">22</span></span><br><span class="line">worker mem:<span class="number">0x12e168e</span> len:<span class="number">8</span> envp[<span class="number">30</span>]: _=./main</span><br><span class="line">master mem:<span class="number">0x12e160c</span> len:<span class="number">16</span> envp[<span class="number">26</span>]: GOPATH=/root/.go</span><br><span class="line">master mem:<span class="number">0x12e161d</span> len:<span class="number">34</span> envp[<span class="number">27</span>]: LESSOPEN=||/usr/bin/lesspipe.sh %s</span><br><span class="line">master mem:<span class="number">0x12e1640</span> len:<span class="number">49</span> envp[<span class="number">28</span>]: ANDROID_NDK=/root/smaug/software/android-ndk-r16b</span><br><span class="line">master mem:<span class="number">0x12e1672</span> len:<span class="number">27</span> envp[<span class="number">29</span>]: XDG_RUNTIME_DIR=/run/user/<span class="number">0</span></span><br><span class="line">master mem:<span class="number">0x12e168e</span> len:<span class="number">8</span> envp[<span class="number">30</span>]: _=./main</span><br></pre></td></tr></table></figure><p>可以看到上述的命令行参数以及环境变量在父子进程中都是正确的，查看一下进程名:</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep setproc</span><br><span class="line">root      <span class="number">2584</span> <span class="number">12289</span>  <span class="number">0</span> <span class="number">16</span>:<span class="number">38</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> setproctitle: master</span><br><span class="line">root      <span class="number">2585</span>  <span class="number">2584</span>  <span class="number">0</span> <span class="number">16</span>:<span class="number">38</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> setproctitle: worker</span><br><span class="line">root      <span class="number">2590</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">16</span>:<span class="number">38</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> setproc</span><br></pre></td></tr></table></figure><h1 id="扩展"><a href="#扩展" class="headerlink" title="扩展"></a>扩展</h1><p>上述代码可以完美的修改进程名，但是如果你使用查看进程信息可能还会看到旧的进程名：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"># ps -ef |grep setproc</span><br><span class="line">root      <span class="number">2584</span> <span class="number">12289</span>  <span class="number">0</span> <span class="number">16</span>:<span class="number">38</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> setproctitle: master</span><br><span class="line">root      <span class="number">2585</span>  <span class="number">2584</span>  <span class="number">0</span> <span class="number">16</span>:<span class="number">38</span> pts/<span class="number">3</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> setproctitle: worker</span><br><span class="line">root      <span class="number">2670</span> <span class="number">20422</span>  <span class="number">0</span> <span class="number">16</span>:<span class="number">39</span> pts/<span class="number">1</span>    <span class="number">00</span>:<span class="number">00</span>:<span class="number">00</span> grep --color=<span class="built_in">auto</span> setproc</span><br><span class="line"></span><br><span class="line"># cat /proc/<span class="number">2585</span>/status |grep Name</span><br><span class="line">Name:main</span><br><span class="line"></span><br><span class="line"># cat /proc/<span class="number">2584</span>/status |grep Name</span><br><span class="line">Name:main</span><br></pre></td></tr></table></figure><p>这个时候可以结合 prctl 使用：</p><figure class="highlight haxe"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">prctl(PR_SET_NAME, <span class="keyword">new</span><span class="type">_name</span>);</span><br></pre></td></tr></table></figure><p>具体可以查看相关资料。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#105;&#x6e;&#117;&#120;&#x63;&#x6f;&#x64;&#x65;&#50;&#110;&#x69;&#x6b;&#x69;&#64;&#x67;&#109;&#x61;&#x69;&#x6c;&#46;&#99;&#111;&#x6d;">&#x6c;&#105;&#x6e;&#117;&#120;&#x63;&#x6f;&#x64;&#x65;&#50;&#110;&#x69;&#x6b;&#x69;&#64;&#x67;&#109;&#x61;&#x69;&#x6c;&#46;&#99;&#111;&#x6d;</a></p></li></ul><p>2020-10-25 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;nginx-进程名&quot;&gt;&lt;a href=&quot;#nginx-进程名&quot; class=&quot;headerlink&quot; title=&quot;nginx 进程名&quot;&gt;&lt;/a&gt;nginx 进程名&lt;/h1&gt;&lt;p&gt;使用 nginx 的过程中，我们经常看到 nginx 的进程名是不同的，如下：&lt;/p&gt;
&lt;figure class=&quot;highlight angelscript&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;$ ps -ef |grep nginx &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;smaug    &lt;span class=&quot;number&quot;&gt;1183&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;1115&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;05&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;46&lt;/span&gt; pts/&lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;    &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; grep --color=&lt;span class=&quot;built_in&quot;&gt;auto&lt;/span&gt; nginx&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;root     &lt;span class=&quot;number&quot;&gt;14201&lt;/span&gt;     &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;2019&lt;/span&gt; ?        &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; nginx: master process ./sbin/nginx&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;nobody   &lt;span class=&quot;number&quot;&gt;28887&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;14201&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; Oct14 ?        &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; nginx: worker process&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;nobody   &lt;span class=&quot;number&quot;&gt;28888&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;14201&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; Oct14 ?        &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; nginx: worker process&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;可以看到 nginx 的进程名是不同的，那么它是怎么做到的呢？&lt;/p&gt;
&lt;h1 id=&quot;argv-0&quot;&gt;&lt;a href=&quot;#argv-0&quot; class=&quot;headerlink&quot; title=&quot;argv[0]&quot;&gt;&lt;/a&gt;argv[0]&lt;/h1&gt;&lt;p&gt;首先来看一下 C 语言中的 main 函数的定义：&lt;/p&gt;
&lt;figure class=&quot;highlight cpp&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;function&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;title&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; argc, &lt;span class=&quot;keyword&quot;&gt;char&lt;/span&gt; *argv[])&lt;/span&gt;&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;这个应该大家都是比较熟悉的，argc 表示命令行参数个数， argv 保存了各个命令行参数的内容。&lt;strong&gt;其中 &lt;code&gt;argv[0]&lt;/code&gt; 表示的是进程的名字&lt;/strong&gt;，这就是修改进程名的关键点所在。&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;只需要修改  argv[0] 的值即可完成修改进程名&lt;/strong&gt;。&lt;/p&gt;
&lt;h2 id=&quot;hello-world&quot;&gt;&lt;a href=&quot;#hello-world&quot; class=&quot;headerlink&quot; title=&quot;hello world&quot;&gt;&lt;/a&gt;hello world&lt;/h2&gt;&lt;p&gt;下面以程序员经典入门代码为例说明：&lt;/p&gt;
&lt;figure class=&quot;highlight cpp&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;7&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;8&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;9&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;12&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;13&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;14&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;comment&quot;&gt;// filename: hello_world_setproctitle.cc&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;comment&quot;&gt;// build: g++ hello_world_setproctitle.cc -o hello_world_setproctitle&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;cstdio&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;#&lt;span class=&quot;meta-keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;meta-string&quot;&gt;&amp;lt;cstring&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;function&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;title&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; argc, &lt;span class=&quot;keyword&quot;&gt;char&lt;/span&gt; *argv[])&lt;/span&gt; &lt;/span&gt;&amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;built_in&quot;&gt;printf&lt;/span&gt;(&lt;span class=&quot;string&quot;&gt;&quot;hello world\n&quot;&lt;/span&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;while&lt;/span&gt; (&lt;span class=&quot;literal&quot;&gt;true&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;comment&quot;&gt;// block here&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;keyword&quot;&gt;char&lt;/span&gt; c = getchar();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;编译运行：&lt;/p&gt;
&lt;figure class=&quot;highlight brainfuck&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;comment&quot;&gt;g&lt;/span&gt;++ &lt;span class=&quot;comment&quot;&gt;hello_world_setproctitle&lt;/span&gt;&lt;span class=&quot;string&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;comment&quot;&gt;cc&lt;/span&gt; &lt;span class=&quot;literal&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;comment&quot;&gt;o&lt;/span&gt; &lt;span class=&quot;comment&quot;&gt;hello_world_setproctitle&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;comment&quot;&gt;&lt;/span&gt;&lt;span class=&quot;string&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;comment&quot;&gt;/hello_world_setproctitle&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;查看一下进程名：&lt;/p&gt;
&lt;figure class=&quot;highlight angelscript&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;# ps -ef |grep hello_world&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;root     &lt;span class=&quot;number&quot;&gt;26356&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;12289&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;14&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;17&lt;/span&gt; pts/&lt;span class=&quot;number&quot;&gt;3&lt;/span&gt;    &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; ./hello_world_setproctitle&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;root     &lt;span class=&quot;number&quot;&gt;26366&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;20422&lt;/span&gt;  &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;14&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;18&lt;/span&gt; pts/&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;    &lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;00&lt;/span&gt; grep --color=&lt;span class=&quot;built_in&quot;&gt;auto&lt;/span&gt; hello_world&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="fork" scheme="https://rebootcat.com/tags/fork/"/>
    
      <category term="nginx" scheme="https://rebootcat.com/tags/nginx/"/>
    
      <category term="setproctitle" scheme="https://rebootcat.com/tags/setproctitle/"/>
    
  </entry>
  
  <entry>
    <title>epoll 入门例子 tcp server/client</title>
    <link href="https://rebootcat.com/2020/09/26/epoll_examples/"/>
    <id>https://rebootcat.com/2020/09/26/epoll_examples/</id>
    <published>2020-09-26T03:50:58.000Z</published>
    <updated>2020-09-27T14:28:14.960Z</updated>
    
    <content type="html"><![CDATA[<h1 id="复习一下"><a href="#复习一下" class="headerlink" title="复习一下"></a>复习一下</h1><p>上一篇博文 <a href="https://rebootcat.com/2020/09/26/epoll_cookbook/">epoll原理深入分析</a> 详细分析了 epoll 底层的实现原理，如果对 epoll 原理有模糊的建议先看一下这篇文章。那么本文就开始用 epoll 实现一个简单的 tcp server/client。</p><p>本文基于我的 github: <a href="https://github.com/smaugx/epoll_examples" target="_blank" rel="noopener">https://github.com/smaugx/epoll_examples</a>。</p><h1 id="epoll-实现范式"><a href="#epoll-实现范式" class="headerlink" title="epoll 实现范式"></a>epoll 实现范式</h1><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"># create listen socket</span></span><br><span class="line"><span class="keyword">int</span> listenfd = ::socket();</span><br><span class="line"></span><br><span class="line"><span class="meta"># bind to local port and ip</span></span><br><span class="line"><span class="keyword">int</span> r = ::bind();</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta"># create epoll instance and get an epoll-fd</span></span><br><span class="line"><span class="keyword">int</span> epollfd = epoll_create(<span class="number">1</span>);</span><br><span class="line"> </span><br><span class="line"><span class="meta"># add listenfd to epoll instance</span></span><br><span class="line"><span class="keyword">int</span> r = epoll_ctl(..., listenfd, ...);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta"># begin epoll_wait, wait for ready socket</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">epoll_event</span>* <span class="title">alive_events</span> =  <span class="title">static_cast</span>&lt;epoll_event*&gt;(<span class="title">calloc</span>(<span class="title">kMaxEvents</span>, <span class="title">sizeof</span>(<span class="title">epoll_event</span>)));</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">            <span class="keyword">int</span> num = epoll_wait(epollfd, alive_events, kMaxEvents, kEpollWaitTime);</span><br><span class="line">    </span><br><span class="line">            <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; num; ++i) &#123;</span><br><span class="line">            <span class="keyword">int</span> fd = alive_events[i].data.fd;</span><br><span class="line">            <span class="keyword">int</span> events = alive_events[i].events;</span><br><span class="line"></span><br><span class="line">            <span class="keyword">if</span> ( (events &amp; EPOLLERR) || (events &amp; EPOLLHUP) ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_wait error!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// An error has occured on this fd, or the socket is not ready for reading (why were we notified then?).</span></span><br><span class="line">                ::<span class="built_in">close</span>(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span>  <span class="keyword">if</span> (events &amp; EPOLLRDHUP) &#123;</span><br><span class="line">                <span class="comment">// Stream socket peer closed connection, or shut down writing half of connection.</span></span><br><span class="line">                <span class="comment">// more inportant, We still to handle disconnection when read()/recv() return 0 or -1 just to be sure.</span></span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd:"</span> &lt;&lt; fd &lt;&lt; <span class="string">" closed EPOLLRDHUP!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// close fd and epoll will remove it</span></span><br><span class="line">                ::<span class="built_in">close</span>(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> <span class="keyword">if</span> ( events &amp; EPOLLIN ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epollin"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="keyword">if</span> (fd == handle_) &#123;</span><br><span class="line">                    <span class="comment">// listen fd coming connections</span></span><br><span class="line">                    OnSocketAccept();</span><br><span class="line">                &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                    <span class="comment">// other fd read event coming, meaning data coming</span></span><br><span class="line">                    OnSocketRead(fd);</span><br><span class="line">                &#125;</span><br><span class="line">            &#125; <span class="keyword">else</span> <span class="keyword">if</span> ( events &amp; EPOLLOUT ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epollout"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// write event for fd (not including listen-fd), meaning send buffer is available for big files</span></span><br><span class="line">                OnSocketWrite(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"unknow epoll event!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="comment">// end for (int i = 0; ...</span></span><br><span class="line">    </span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>epoll 编程基本是按照上面的范式进行的，这里要注意的是上面的反应的只是单进程或者单线程的情况。</p><p>如果涉及到多线程或者多进程，那么通常来说会在 listen() 创建完成之后，创建多线程或者多进程，然后再操作 epoll.</p><a id="more"></a><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">int</span> listenfd = ::socket<span class="literal">()</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> p = fork<span class="literal">()</span> # 多进程 或者多线程创建</span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> r = epoll<span class="constructor">_ctl(<span class="operator">...</span>, <span class="params">listenfd</span>, <span class="operator">...</span>)</span>;</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span>(<span class="literal">true</span>) &#123;</span><br><span class="line"><span class="built_in">int</span> num = epoll<span class="constructor">_wait(<span class="params">epollfd</span>, <span class="params">alive_events</span>, <span class="params">kMaxEvents</span>, <span class="params">kEpollWaitTime</span>)</span>;</span><br><span class="line">...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>同理，多线程版本也是一样，把上面的 fork() 替换成 thread 创建即可。</p><p>也就是 listenfd 被添加到了多个进程或者多个线程中，提高吞吐量。这就是基本的 epoll 多进程或者多线程编程范式。</p><p>但本文就先讨论单进程（单线程）版本的 epoll 实现。</p><h1 id="epoll-tcp-server"><a href="#epoll-tcp-server" class="headerlink" title="epoll tcp server"></a>epoll tcp server</h1><p>先上代码：</p><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br><span class="line">338</span><br><span class="line">339</span><br><span class="line">340</span><br><span class="line">341</span><br><span class="line">342</span><br><span class="line">343</span><br><span class="line">344</span><br><span class="line">345</span><br><span class="line">346</span><br><span class="line">347</span><br><span class="line">348</span><br><span class="line">349</span><br><span class="line">350</span><br><span class="line">351</span><br><span class="line">352</span><br><span class="line">353</span><br><span class="line">354</span><br><span class="line">355</span><br><span class="line">356</span><br><span class="line">357</span><br><span class="line">358</span><br><span class="line">359</span><br><span class="line">360</span><br><span class="line">361</span><br><span class="line">362</span><br><span class="line">363</span><br><span class="line">364</span><br><span class="line">365</span><br><span class="line">366</span><br><span class="line">367</span><br><span class="line">368</span><br><span class="line">369</span><br><span class="line">370</span><br><span class="line">371</span><br><span class="line">372</span><br><span class="line">373</span><br><span class="line">374</span><br><span class="line">375</span><br><span class="line">376</span><br><span class="line">377</span><br><span class="line">378</span><br><span class="line">379</span><br><span class="line">380</span><br><span class="line">381</span><br><span class="line">382</span><br><span class="line">383</span><br><span class="line">384</span><br><span class="line">385</span><br><span class="line">386</span><br><span class="line">387</span><br><span class="line">388</span><br><span class="line">389</span><br><span class="line">390</span><br><span class="line">391</span><br><span class="line">392</span><br><span class="line">393</span><br><span class="line">394</span><br><span class="line">395</span><br><span class="line">396</span><br><span class="line">397</span><br><span class="line">398</span><br><span class="line">399</span><br><span class="line">400</span><br><span class="line">401</span><br><span class="line">402</span><br><span class="line">403</span><br><span class="line">404</span><br><span class="line">405</span><br><span class="line">406</span><br><span class="line">407</span><br><span class="line">408</span><br><span class="line">409</span><br><span class="line">410</span><br><span class="line">411</span><br><span class="line">412</span><br><span class="line">413</span><br><span class="line">414</span><br><span class="line">415</span><br><span class="line">416</span><br><span class="line">417</span><br><span class="line">418</span><br><span class="line">419</span><br><span class="line">420</span><br><span class="line">421</span><br><span class="line">422</span><br><span class="line">423</span><br><span class="line">424</span><br><span class="line">425</span><br><span class="line">426</span><br><span class="line">427</span><br><span class="line">428</span><br><span class="line">429</span><br><span class="line">430</span><br><span class="line">431</span><br><span class="line">432</span><br><span class="line">433</span><br><span class="line">434</span><br><span class="line">435</span><br><span class="line">436</span><br><span class="line">437</span><br><span class="line">438</span><br><span class="line">439</span><br><span class="line">440</span><br><span class="line">441</span><br><span class="line">442</span><br><span class="line">443</span><br><span class="line">444</span><br><span class="line">445</span><br><span class="line">446</span><br><span class="line">447</span><br><span class="line">448</span><br><span class="line">449</span><br><span class="line">450</span><br><span class="line">451</span><br><span class="line">452</span><br><span class="line">453</span><br><span class="line">454</span><br><span class="line">455</span><br><span class="line">456</span><br><span class="line">457</span><br><span class="line">458</span><br><span class="line">459</span><br><span class="line">460</span><br><span class="line">461</span><br><span class="line">462</span><br><span class="line">463</span><br><span class="line">464</span><br><span class="line">465</span><br><span class="line">466</span><br><span class="line">467</span><br><span class="line">468</span><br><span class="line">469</span><br><span class="line">470</span><br><span class="line">471</span><br><span class="line">472</span><br><span class="line">473</span><br><span class="line">474</span><br><span class="line">475</span><br><span class="line">476</span><br><span class="line">477</span><br><span class="line">478</span><br><span class="line">479</span><br><span class="line">480</span><br><span class="line">481</span><br><span class="line">482</span><br><span class="line">483</span><br><span class="line">484</span><br><span class="line">485</span><br><span class="line">486</span><br><span class="line">487</span><br><span class="line">488</span><br><span class="line">489</span><br><span class="line">490</span><br><span class="line">491</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;arpa/inet.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;errno.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;fcntl.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;netinet/in.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;stdio.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;stdlib.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;sys/epoll.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;sys/socket.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cassert&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;iostream&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;string&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;thread&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;memory&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;functional&gt;</span></span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">namespace</span> mux &#123;</span><br><span class="line"></span><br><span class="line"><span class="keyword">namespace</span> transport &#123;</span><br><span class="line"></span><br><span class="line"><span class="keyword">static</span> <span class="keyword">const</span> <span class="keyword">uint32_t</span> kEpollWaitTime = <span class="number">10</span>; <span class="comment">// epoll wait timeout 10 ms</span></span><br><span class="line"><span class="keyword">static</span> <span class="keyword">const</span> <span class="keyword">uint32_t</span> kMaxEvents = <span class="number">100</span>;    <span class="comment">// epoll wait return max size</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">// packet of send/recv binary content</span></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">Packet</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    Packet()</span><br><span class="line">        : msg &#123; <span class="string">""</span> &#125; &#123;&#125;</span><br><span class="line">    Packet(<span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; msg)</span><br><span class="line">        : msg &#123; msg &#125; &#123;&#125;</span><br><span class="line">    Packet(<span class="keyword">int</span> fd, <span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; msg)</span><br><span class="line">        : fd(fd),</span><br><span class="line">          msg(msg) &#123;&#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">int</span> fd &#123; <span class="number">-1</span> &#125;;     <span class="comment">// meaning socket</span></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> msg;   <span class="comment">// real binary content</span></span><br><span class="line">&#125; Packet;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;Packet&gt; PacketPtr;</span><br><span class="line"></span><br><span class="line"><span class="comment">// callback when packet received</span></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">callback_recv_t</span> = <span class="built_in">std</span>::function&lt;<span class="keyword">void</span>(<span class="keyword">const</span> PacketPtr&amp; data)&gt;;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">// base class of EpollTcpServer, focus on Start(), Stop(), SendData(), RegisterOnRecvCallback()...</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">EpollTcpBase</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    EpollTcpBase()                                     = <span class="keyword">default</span>;</span><br><span class="line">    EpollTcpBase(<span class="keyword">const</span> EpollTcpBase&amp; other)            = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpBase&amp; <span class="keyword">operator</span>=(<span class="keyword">const</span> EpollTcpBase&amp; other) = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpBase(EpollTcpBase&amp;&amp; other)                 = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpBase&amp; <span class="keyword">operator</span>=(EpollTcpBase&amp;&amp; other)      = <span class="keyword">delete</span>;</span><br><span class="line">    <span class="keyword">virtual</span> ~EpollTcpBase()                            = <span class="keyword">default</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">bool</span> <span class="title">Start</span><span class="params">()</span> </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">bool</span> <span class="title">Stop</span><span class="params">()</span>  </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">int32_t</span> <span class="title">SendData</span><span class="params">(<span class="keyword">const</span> PacketPtr&amp; data)</span> </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">RegisterOnRecvCallback</span><span class="params">(<span class="keyword">callback_recv_t</span> callback)</span> </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">UnRegisterOnRecvCallback</span><span class="params">()</span> </span>= <span class="number">0</span>;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> ETBase = EpollTcpBase;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;ETBase&gt; ETBasePtr;</span><br><span class="line"></span><br><span class="line"><span class="comment">// the implementation of Epoll Tcp Server</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">EpollTcpServer</span> :</span> <span class="keyword">public</span> ETBase &#123;</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    EpollTcpServer()                                       = <span class="keyword">default</span>;</span><br><span class="line">    EpollTcpServer(<span class="keyword">const</span> EpollTcpServer&amp; other)            = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpServer&amp; <span class="keyword">operator</span>=(<span class="keyword">const</span> EpollTcpServer&amp; other) = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpServer(EpollTcpServer&amp;&amp; other)                 = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpServer&amp; <span class="keyword">operator</span>=(EpollTcpServer&amp;&amp; other)      = <span class="keyword">delete</span>;</span><br><span class="line">    ~EpollTcpServer() <span class="keyword">override</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// the local ip and port of tcp server</span></span><br><span class="line">    EpollTcpServer(<span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; local_ip, <span class="keyword">uint16_t</span> local_port);</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="comment">// start tcp server</span></span><br><span class="line">    <span class="function"><span class="keyword">bool</span> <span class="title">Start</span><span class="params">()</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="comment">// stop tcp server</span></span><br><span class="line">    <span class="function"><span class="keyword">bool</span> <span class="title">Stop</span><span class="params">()</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="comment">// send packet</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">SendData</span><span class="params">(<span class="keyword">const</span> PacketPtr&amp; data)</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="comment">// register a callback when packet received</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">RegisterOnRecvCallback</span><span class="params">(<span class="keyword">callback_recv_t</span> callback)</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">UnRegisterOnRecvCallback</span><span class="params">()</span> <span class="keyword">override</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">protected</span>:</span><br><span class="line">    <span class="comment">// create epoll instance using epoll_create and return a fd of epoll</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">CreateEpoll</span><span class="params">()</span></span>;</span><br><span class="line">    <span class="comment">// create a socket fd using api socket()</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">CreateSocket</span><span class="params">()</span></span>;</span><br><span class="line">    <span class="comment">// set socket noblock</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">MakeSocketNonBlock</span><span class="params">(<span class="keyword">int32_t</span> fd)</span></span>;</span><br><span class="line">    <span class="comment">// listen()</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">Listen</span><span class="params">(<span class="keyword">int32_t</span> listenfd)</span></span>;</span><br><span class="line">    <span class="comment">// add/modify/remove a item(socket/fd) in epoll instance(rbtree), for this example, just add a socket to epoll rbtree</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">UpdateEpollEvents</span><span class="params">(<span class="keyword">int</span> efd, <span class="keyword">int</span> op, <span class="keyword">int</span> fd, <span class="keyword">int</span> events)</span></span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// handle tcp accept event</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">OnSocketAccept</span><span class="params">()</span></span>;</span><br><span class="line">    <span class="comment">// handle tcp socket readable event(read())</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">OnSocketRead</span><span class="params">(<span class="keyword">int32_t</span> fd)</span></span>;</span><br><span class="line">    <span class="comment">// handle tcp socket writeable event(write())</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">OnSocketWrite</span><span class="params">(<span class="keyword">int32_t</span> fd)</span></span>;</span><br><span class="line">    <span class="comment">// one loop per thread, call epoll_wait and return ready socket(accept,readable,writeable,error...)</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">EpollLoop</span><span class="params">()</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span>:</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> local_ip_; <span class="comment">// tcp local ip</span></span><br><span class="line">    <span class="keyword">uint16_t</span> local_port_ &#123; <span class="number">0</span> &#125;; <span class="comment">// tcp bind local port</span></span><br><span class="line">    <span class="keyword">int32_t</span> handle_ &#123; <span class="number">-1</span> &#125;; <span class="comment">// listenfd</span></span><br><span class="line">    <span class="keyword">int32_t</span> efd_ &#123; <span class="number">-1</span> &#125;; <span class="comment">// epoll fd</span></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;<span class="built_in">std</span>::thread&gt; th_loop_ &#123; <span class="literal">nullptr</span> &#125;; <span class="comment">// one loop per thread(call epoll_wait in loop)</span></span><br><span class="line">    <span class="keyword">bool</span> loop_flag_ &#123; <span class="literal">true</span> &#125;; <span class="comment">// if loop_flag_ is false, then exit the epoll loop</span></span><br><span class="line">    <span class="keyword">callback_recv_t</span> recv_callback_ &#123; <span class="literal">nullptr</span> &#125;; <span class="comment">// callback when received</span></span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> ETServer = EpollTcpServer;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;ETServer&gt; ETServerPtr;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">EpollTcpServer::EpollTcpServer(<span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; local_ip, <span class="keyword">uint16_t</span> local_port)</span><br><span class="line">    : local_ip_ &#123; local_ip &#125;,</span><br><span class="line">      local_port_ &#123; local_port &#125; &#123;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">EpollTcpServer::~EpollTcpServer() &#123;</span><br><span class="line">    Stop();</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">bool</span> <span class="title">EpollTcpServer::Start</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// create epoll instance</span></span><br><span class="line">    <span class="keyword">if</span> (CreateEpoll() &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// create socket and bind</span></span><br><span class="line">    <span class="keyword">int</span> listenfd = CreateSocket();</span><br><span class="line">    <span class="keyword">if</span> (listenfd &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// set listen socket noblock</span></span><br><span class="line">    <span class="keyword">int</span> mr = MakeSocketNonBlock(listenfd);</span><br><span class="line">    <span class="keyword">if</span> (mr &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// call listen()</span></span><br><span class="line">    <span class="keyword">int</span> lr = Listen(listenfd);</span><br><span class="line">    <span class="keyword">if</span> (lr &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"EpollTcpServer Init success!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    handle_ = listenfd;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// add listen socket to epoll instance, and focus on event EPOLLIN and EPOLLOUT, actually EPOLLIN is enough</span></span><br><span class="line">    <span class="keyword">int</span> er = UpdateEpollEvents(efd_, EPOLL_CTL_ADD, handle_, EPOLLIN | EPOLLET);</span><br><span class="line">    <span class="keyword">if</span> (er &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// if something goes wrong, close listen socket and return false</span></span><br><span class="line">        ::<span class="built_in">close</span>(handle_);</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    assert(!th_loop_);</span><br><span class="line"></span><br><span class="line">    <span class="comment">// the implementation of one loop per thread: create a thread to loop epoll</span></span><br><span class="line">    th_loop_ = <span class="built_in">std</span>::make_shared&lt;<span class="built_in">std</span>::thread&gt;(&amp;EpollTcpServer::EpollLoop, <span class="keyword">this</span>);</span><br><span class="line">    <span class="keyword">if</span> (!th_loop_) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// detach the thread(using loop_flag_ to control the start/stop of loop)</span></span><br><span class="line">    th_loop_-&gt;<span class="built_in">detach</span>();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">// stop epoll tcp server and release epoll</span></span><br><span class="line"><span class="function"><span class="keyword">bool</span> <span class="title">EpollTcpServer::Stop</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// set loop_flag_ false to stop epoll loop</span></span><br><span class="line">    loop_flag_ = <span class="literal">false</span>;</span><br><span class="line">    ::<span class="built_in">close</span>(handle_);</span><br><span class="line">    ::<span class="built_in">close</span>(efd_);</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"stop epoll!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    UnRegisterOnRecvCallback();</span><br><span class="line">    <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpServer::CreateEpoll</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// the basic epoll api of create a epoll instance</span></span><br><span class="line">    <span class="keyword">int</span> epollfd = epoll_create(<span class="number">1</span>);</span><br><span class="line">    <span class="keyword">if</span> (epollfd &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// if something goes wrong, return -1</span></span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_create failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    efd_ = epollfd;</span><br><span class="line">    <span class="keyword">return</span> epollfd;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpServer::CreateSocket</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// create tcp socket</span></span><br><span class="line">    <span class="keyword">int</span> listenfd = ::socket(AF_INET, SOCK_STREAM, <span class="number">0</span>);</span><br><span class="line">    <span class="keyword">if</span> (listenfd &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"create socket "</span> &lt;&lt; local_ip_ &lt;&lt; <span class="string">":"</span> &lt;&lt; local_port_ &lt;&lt; <span class="string">" failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="class"><span class="keyword">struct</span> <span class="title">sockaddr_in</span> <span class="title">addr</span>;</span></span><br><span class="line">    <span class="built_in">memset</span>(&amp;addr, <span class="number">0</span>, <span class="keyword">sizeof</span>(addr));</span><br><span class="line">    addr.sin_family = AF_INET;</span><br><span class="line">    addr.sin_port = htons(local_port_);</span><br><span class="line">    addr.sin_addr.s_addr  = inet_addr(local_ip_.c_str());</span><br><span class="line"></span><br><span class="line">    <span class="comment">// bind to local ip and local port</span></span><br><span class="line">    <span class="keyword">int</span> r = ::bind(listenfd, (struct sockaddr*)&amp;addr, <span class="keyword">sizeof</span>(struct sockaddr));</span><br><span class="line">    <span class="keyword">if</span> (r != <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"bind socket "</span> &lt;&lt; local_ip_ &lt;&lt; <span class="string">":"</span> &lt;&lt; local_port_ &lt;&lt; <span class="string">" failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        ::<span class="built_in">close</span>(listenfd);</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"create and bind socket "</span> &lt;&lt; local_ip_ &lt;&lt; <span class="string">":"</span> &lt;&lt; local_port_ &lt;&lt; <span class="string">" success!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    <span class="keyword">return</span> listenfd;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// set noblock fd</span></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpServer::MakeSocketNonBlock</span><span class="params">(<span class="keyword">int32_t</span> fd)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">int</span> flags = fcntl(fd, F_GETFL, <span class="number">0</span>);</span><br><span class="line">    <span class="keyword">if</span> (flags &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fcntl failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">int</span> r = fcntl(fd, F_SETFL, flags | O_NONBLOCK);</span><br><span class="line">    <span class="keyword">if</span> (r &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fcntl failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// call listen() api and set listen queue size using SOMAXCONN</span></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpServer::Listen</span><span class="params">(<span class="keyword">int32_t</span> listenfd)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">int</span> r = ::<span class="built_in">listen</span>(listenfd, SOMAXCONN);</span><br><span class="line">    <span class="keyword">if</span> ( r &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"listen failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// add/modify/remove a item(socket/fd) in epoll instance(rbtree), for this example, just add a socket to epoll rbtree</span></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpServer::UpdateEpollEvents</span><span class="params">(<span class="keyword">int</span> efd, <span class="keyword">int</span> op, <span class="keyword">int</span> fd, <span class="keyword">int</span> events)</span> </span>&#123;</span><br><span class="line">    <span class="class"><span class="keyword">struct</span> <span class="title">epoll_event</span> <span class="title">ev</span>;</span></span><br><span class="line">    <span class="built_in">memset</span>(&amp;ev, <span class="number">0</span>, <span class="keyword">sizeof</span>(ev));</span><br><span class="line">    ev.events = events;</span><br><span class="line">    ev.data.fd = fd; <span class="comment">// ev.data is a enum</span></span><br><span class="line">    <span class="built_in">fprintf</span>(<span class="built_in">stdout</span>,<span class="string">"%s fd %d events read %d write %d\n"</span>, op == EPOLL_CTL_MOD ? <span class="string">"mod"</span> : <span class="string">"add"</span>, fd, ev.events &amp; EPOLLIN, ev.events &amp; EPOLLOUT);</span><br><span class="line">    <span class="keyword">int</span> r = epoll_ctl(efd, op, fd, &amp;ev);</span><br><span class="line">    <span class="keyword">if</span> (r &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_ctl failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// handle accept event</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpServer::OnSocketAccept</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// epoll working on et mode, must read all coming data, so use a while loop here</span></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="class"><span class="keyword">struct</span> <span class="title">sockaddr_in</span> <span class="title">in_addr</span>;</span></span><br><span class="line">        <span class="keyword">socklen_t</span> in_len = <span class="keyword">sizeof</span>(in_addr);</span><br><span class="line"></span><br><span class="line">        <span class="comment">// accept a new connection and get a new socket</span></span><br><span class="line">        <span class="keyword">int</span> cli_fd = accept(handle_, (struct sockaddr*)&amp;in_addr, &amp;in_len);</span><br><span class="line">        <span class="keyword">if</span> (cli_fd == <span class="number">-1</span>) &#123;</span><br><span class="line">            <span class="keyword">if</span> ( (errno == EAGAIN) || (errno == EWOULDBLOCK) ) &#123;</span><br><span class="line">                <span class="comment">// read all accept finished(epoll et mode only trigger one time,so must read all data in listen socket)</span></span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"accept all coming connections!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="keyword">break</span>;</span><br><span class="line">            &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"accept error!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        sockaddr_in peer;</span><br><span class="line">        <span class="keyword">socklen_t</span> p_len = <span class="keyword">sizeof</span>(peer);</span><br><span class="line">        <span class="comment">// get client ip and port</span></span><br><span class="line">        <span class="keyword">int</span> r = getpeername(cli_fd, (struct sockaddr*)&amp;peer, &amp;p_len);</span><br><span class="line">        <span class="keyword">if</span> (r &lt; <span class="number">0</span>) &#123;</span><br><span class="line">            <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"getpeername error!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">            <span class="keyword">continue</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"accpet connection from "</span> &lt;&lt; inet_ntoa(in_addr.sin_addr) &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">int</span> mr = MakeSocketNonBlock(cli_fd);</span><br><span class="line">        <span class="keyword">if</span> (mr &lt; <span class="number">0</span>) &#123;</span><br><span class="line">            ::<span class="built_in">close</span>(cli_fd);</span><br><span class="line">            <span class="keyword">continue</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="comment">//  add this new socket to epoll instance, and focus on EPOLLIN and EPOLLOUT and EPOLLRDHUP event</span></span><br><span class="line">        <span class="keyword">int</span> er = UpdateEpollEvents(efd_, EPOLL_CTL_ADD, cli_fd, EPOLLIN | EPOLLRDHUP | EPOLLET);</span><br><span class="line">        <span class="keyword">if</span> (er &lt; <span class="number">0</span> ) &#123;</span><br><span class="line">            <span class="comment">// if something goes wrong, close this new socket</span></span><br><span class="line">            ::<span class="built_in">close</span>(cli_fd);</span><br><span class="line">            <span class="keyword">continue</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// register a callback when packet received</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpServer::RegisterOnRecvCallback</span><span class="params">(<span class="keyword">callback_recv_t</span> callback)</span> </span>&#123;</span><br><span class="line">    assert(!recv_callback_);</span><br><span class="line">    recv_callback_ = callback;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpServer::UnRegisterOnRecvCallback</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    assert(recv_callback_);</span><br><span class="line">    recv_callback_ = <span class="literal">nullptr</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// handle read events on fd</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpServer::OnSocketRead</span><span class="params">(<span class="keyword">int32_t</span> fd)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">char</span> read_buf[<span class="number">4096</span>];</span><br><span class="line">    bzero(read_buf, <span class="keyword">sizeof</span>(read_buf));</span><br><span class="line">    <span class="keyword">int</span> n = <span class="number">-1</span>;</span><br><span class="line">    <span class="comment">// epoll working on et mode, must read all data</span></span><br><span class="line">    <span class="keyword">while</span> ( (n = ::<span class="built_in">read</span>(fd, read_buf, <span class="keyword">sizeof</span>(read_buf))) &gt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// callback for recv</span></span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd: "</span> &lt;&lt; fd &lt;&lt;  <span class="string">" recv: "</span> &lt;&lt; read_buf &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="function"><span class="built_in">std</span>::<span class="built_in">string</span> <span class="title">msg</span><span class="params">(read_buf, n)</span></span>;</span><br><span class="line">        <span class="comment">// create a recv packet</span></span><br><span class="line">        PacketPtr data = <span class="built_in">std</span>::make_shared&lt;Packet&gt;(fd, msg);</span><br><span class="line">        <span class="keyword">if</span> (recv_callback_) &#123;</span><br><span class="line">            <span class="comment">// handle recv packet</span></span><br><span class="line">            recv_callback_(data);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (n == <span class="number">-1</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> (errno == EAGAIN || errno == EWOULDBLOCK) &#123;</span><br><span class="line">            <span class="comment">// read all data finished</span></span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// something goes wrong for this fd, should close it</span></span><br><span class="line">        ::<span class="built_in">close</span>(fd);</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (n == <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// this may happen when client close socket. EPOLLRDHUP usually handle this, but just make sure; should close this fd</span></span><br><span class="line">        ::<span class="built_in">close</span>(fd);</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// handle write events on fd (usually happens when sending big files)</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpServer::OnSocketWrite</span><span class="params">(<span class="keyword">int32_t</span> fd)</span> </span>&#123;</span><br><span class="line">    <span class="comment">// TODO(smaugx) not care for now</span></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd: "</span> &lt;&lt; fd &lt;&lt; <span class="string">" writeable!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// send packet</span></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpServer::SendData</span><span class="params">(<span class="keyword">const</span> PacketPtr&amp; data)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">if</span> (data-&gt;fd == <span class="number">-1</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// send packet on fd</span></span><br><span class="line">    <span class="keyword">int</span> r = ::<span class="built_in">write</span>(data-&gt;fd, data-&gt;msg.data(), data-&gt;msg.<span class="built_in">size</span>());</span><br><span class="line">    <span class="keyword">if</span> (r == <span class="number">-1</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> (errno == EAGAIN || errno == EWOULDBLOCK) &#123;</span><br><span class="line">            <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// error happend</span></span><br><span class="line">        ::<span class="built_in">close</span>(data-&gt;fd);</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd: "</span> &lt;&lt; data-&gt;fd &lt;&lt; <span class="string">" write error, close it!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd: "</span> &lt;&lt; data-&gt;fd &lt;&lt; <span class="string">" write size: "</span> &lt;&lt; r &lt;&lt; <span class="string">" ok!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    <span class="keyword">return</span> r;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// one loop per thread, call epoll_wait and handle all coming events</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpServer::EpollLoop</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// request some memory, if events ready, socket events will copy to this memory from kernel</span></span><br><span class="line">    <span class="class"><span class="keyword">struct</span> <span class="title">epoll_event</span>* <span class="title">alive_events</span> =  <span class="title">static_cast</span>&lt;epoll_event*&gt;(<span class="title">calloc</span>(<span class="title">kMaxEvents</span>, <span class="title">sizeof</span>(<span class="title">epoll_event</span>)));</span></span><br><span class="line">    <span class="keyword">if</span> (!alive_events) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"calloc memory failed for epoll_events!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// if loop_flag_ is false, will exit this loop</span></span><br><span class="line">    <span class="keyword">while</span> (loop_flag_) &#123;</span><br><span class="line">        <span class="comment">// call epoll_wait and return ready socket</span></span><br><span class="line">        <span class="keyword">int</span> num = epoll_wait(efd_, alive_events, kMaxEvents, kEpollWaitTime);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; num; ++i) &#123;</span><br><span class="line">            <span class="comment">// get fd</span></span><br><span class="line">            <span class="keyword">int</span> fd = alive_events[i].data.fd;</span><br><span class="line">            <span class="comment">// get events(readable/writeable/error)</span></span><br><span class="line">            <span class="keyword">int</span> events = alive_events[i].events;</span><br><span class="line"></span><br><span class="line">            <span class="keyword">if</span> ( (events &amp; EPOLLERR) || (events &amp; EPOLLHUP) ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_wait error!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// An error has occured on this fd, or the socket is not ready for reading (why were we notified then?).</span></span><br><span class="line">                ::<span class="built_in">close</span>(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span>  <span class="keyword">if</span> (events &amp; EPOLLRDHUP) &#123;</span><br><span class="line">                <span class="comment">// Stream socket peer closed connection, or shut down writing half of connection.</span></span><br><span class="line">                <span class="comment">// more inportant, We still to handle disconnection when read()/recv() return 0 or -1 just to be sure.</span></span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd:"</span> &lt;&lt; fd &lt;&lt; <span class="string">" closed EPOLLRDHUP!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// close fd and epoll will remove it</span></span><br><span class="line">                ::<span class="built_in">close</span>(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> <span class="keyword">if</span> ( events &amp; EPOLLIN ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epollin"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="keyword">if</span> (fd == handle_) &#123;</span><br><span class="line">                    <span class="comment">// listen fd coming connections</span></span><br><span class="line">                    OnSocketAccept();</span><br><span class="line">                &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                    <span class="comment">// other fd read event coming, meaning data coming</span></span><br><span class="line">                    OnSocketRead(fd);</span><br><span class="line">                &#125;</span><br><span class="line">            &#125; <span class="keyword">else</span> <span class="keyword">if</span> ( events &amp; EPOLLOUT ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epollout"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// write event for fd (not including listen-fd), meaning send buffer is available for big files</span></span><br><span class="line">                OnSocketWrite(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"unknow epoll event!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="comment">// end for (int i = 0; ...</span></span><br><span class="line"></span><br><span class="line">    &#125; <span class="comment">// end while (loop_flag_)</span></span><br><span class="line"></span><br><span class="line">    <span class="built_in">free</span>(alive_events);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">&#125; <span class="comment">// end namespace transport</span></span><br><span class="line">&#125; <span class="comment">// end namespace mux</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> mux;</span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> transport;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span>* argv[])</span> </span>&#123;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> local_ip &#123;<span class="string">"127.0.0.1"</span>&#125;;</span><br><span class="line">    <span class="keyword">uint16_t</span> local_port &#123; <span class="number">6666</span> &#125;;</span><br><span class="line">    <span class="keyword">if</span> (argc &gt;= <span class="number">2</span>) &#123;</span><br><span class="line">        local_ip = <span class="built_in">std</span>::<span class="built_in">string</span>(argv[<span class="number">1</span>]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (argc &gt;= <span class="number">3</span>) &#123;</span><br><span class="line">        local_port = <span class="built_in">std</span>::atoi(argv[<span class="number">2</span>]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// create a epoll tcp server</span></span><br><span class="line">    <span class="keyword">auto</span> epoll_server = <span class="built_in">std</span>::make_shared&lt;EpollTcpServer&gt;(local_ip, local_port);</span><br><span class="line">    <span class="keyword">if</span> (!epoll_server) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"tcp_server create faield!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="built_in">exit</span>(<span class="number">-1</span>);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// recv callback in lambda mode, you can set your own callback here</span></span><br><span class="line">    <span class="keyword">auto</span> recv_call = [&amp;](<span class="keyword">const</span> PacketPtr&amp; data) -&gt; <span class="keyword">void</span> &#123;</span><br><span class="line">        <span class="comment">// just echo packet</span></span><br><span class="line">        epoll_server-&gt;SendData(data);</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// register recv callback to epoll tcp server</span></span><br><span class="line">    epoll_server-&gt;RegisterOnRecvCallback(recv_call);</span><br><span class="line"></span><br><span class="line">    <span class="comment">// start the epoll tcp server</span></span><br><span class="line">    <span class="keyword">if</span> (!epoll_server-&gt;Start()) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"tcp_server start failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"############tcp_server started!################"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// block here</span></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::this_thread::sleep_for(<span class="built_in">std</span>::chrono::seconds(<span class="number">1</span>));</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    epoll_server-&gt;Stop();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>代码看起来有点多，不过仔细分析下，其实也比较容易掌握。</p><p>核心的类是 <strong>EpollTcpServer</strong>，创建一个 <strong>EpllTcpServer</strong> 实例：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">auto</span> epoll_server = <span class="built_in">std</span>::make_shared&lt;EpollTcpServer&gt;(local_ip, local_port);</span><br></pre></td></tr></table></figure><p>注册一个收包处理回调函数：</p><figure class="highlight markdown"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="section"># 这里直接注册一个 echo 函数（可以替换成其他的处理函数）</span></span><br><span class="line">auto recv_call = [<span class="string">&amp;</span>](<span class="link">const PacketPtr&amp; data</span>) -&gt; void &#123;</span><br><span class="line"><span class="code">    epoll_server-&gt;SendData(data);</span></span><br><span class="line"><span class="code">    return;</span></span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line">epoll<span class="emphasis">_server-&gt;RegisterOnRecvCallback(recv_</span>call);</span><br></pre></td></tr></table></figure><p>启动 tcp server:</p><figure class="highlight xl"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">epoll_server</span>-&gt;</span>Start();</span><br></pre></td></tr></table></figure><p>是不是很简单？至于 Start() 函数内部，其实实现的就是 epoll 编程范式的细节。</p><p>代码细节应该比较好理解的，可以参考 <a href="https://github.com/smaugx/epoll_examples/blob/master/README.md" target="_blank" rel="noopener">https://github.com/smaugx/epoll_examples/blob/master/README.md</a></p><h1 id="epoll-tcp-client"><a href="#epoll-tcp-client" class="headerlink" title="epoll tcp client"></a>epoll tcp client</h1><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br><span class="line">338</span><br><span class="line">339</span><br><span class="line">340</span><br><span class="line">341</span><br><span class="line">342</span><br><span class="line">343</span><br><span class="line">344</span><br><span class="line">345</span><br><span class="line">346</span><br><span class="line">347</span><br><span class="line">348</span><br><span class="line">349</span><br><span class="line">350</span><br><span class="line">351</span><br><span class="line">352</span><br><span class="line">353</span><br><span class="line">354</span><br><span class="line">355</span><br><span class="line">356</span><br><span class="line">357</span><br><span class="line">358</span><br><span class="line">359</span><br><span class="line">360</span><br><span class="line">361</span><br><span class="line">362</span><br><span class="line">363</span><br><span class="line">364</span><br><span class="line">365</span><br><span class="line">366</span><br><span class="line">367</span><br><span class="line">368</span><br><span class="line">369</span><br><span class="line">370</span><br><span class="line">371</span><br><span class="line">372</span><br><span class="line">373</span><br><span class="line">374</span><br><span class="line">375</span><br><span class="line">376</span><br><span class="line">377</span><br><span class="line">378</span><br><span class="line">379</span><br><span class="line">380</span><br><span class="line">381</span><br><span class="line">382</span><br><span class="line">383</span><br><span class="line">384</span><br><span class="line">385</span><br><span class="line">386</span><br><span class="line">387</span><br><span class="line">388</span><br><span class="line">389</span><br><span class="line">390</span><br><span class="line">391</span><br><span class="line">392</span><br><span class="line">393</span><br><span class="line">394</span><br><span class="line">395</span><br><span class="line">396</span><br><span class="line">397</span><br><span class="line">398</span><br><span class="line">399</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;arpa/inet.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;errno.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;fcntl.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;netinet/in.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;stdio.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cstring&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;stdlib.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;sys/epoll.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;sys/socket.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;unistd.h&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;cassert&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;string&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;iostream&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;memory&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;functional&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">&lt;thread&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">// actually no need to implement a tcp client using epoll</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">namespace</span> mux &#123;</span><br><span class="line"></span><br><span class="line"><span class="keyword">namespace</span> transport &#123;</span><br><span class="line"></span><br><span class="line"><span class="keyword">static</span> <span class="keyword">const</span> <span class="keyword">uint32_t</span> kEpollWaitTime = <span class="number">10</span>; <span class="comment">// epoll wait timeout 10 ms</span></span><br><span class="line"><span class="keyword">static</span> <span class="keyword">const</span> <span class="keyword">uint32_t</span> kMaxEvents = <span class="number">100</span>;    <span class="comment">// epoll wait return max size</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">Packet</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    Packet()</span><br><span class="line">        : msg &#123; <span class="string">""</span> &#125; &#123;&#125;</span><br><span class="line">    Packet(<span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; msg)</span><br><span class="line">        : msg &#123; msg &#125; &#123;&#125;</span><br><span class="line">    Packet(<span class="keyword">int</span> fd, <span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; msg)</span><br><span class="line">        : fd(fd),</span><br><span class="line">          msg(msg) &#123;&#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">int</span> fd &#123; <span class="number">-1</span> &#125;;     <span class="comment">// meaning socket</span></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> msg;   <span class="comment">// real binary content</span></span><br><span class="line">&#125; Packet;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;Packet&gt; PacketPtr;</span><br><span class="line"></span><br><span class="line"><span class="comment">// callback when packet received</span></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">callback_recv_t</span> = <span class="built_in">std</span>::function&lt;<span class="keyword">void</span>(<span class="keyword">const</span> PacketPtr&amp; data)&gt;;</span><br><span class="line"></span><br><span class="line"><span class="comment">// base class of EpollTcpServer, focus on Start(), Stop(), SendData(), RegisterOnRecvCallback()...</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">EpollTcpBase</span> &#123;</span></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    EpollTcpBase()                                     = <span class="keyword">default</span>;</span><br><span class="line">    EpollTcpBase(<span class="keyword">const</span> EpollTcpBase&amp; other)            = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpBase&amp; <span class="keyword">operator</span>=(<span class="keyword">const</span> EpollTcpBase&amp; other) = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpBase(EpollTcpBase&amp;&amp; other)                 = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpBase&amp; <span class="keyword">operator</span>=(EpollTcpBase&amp;&amp; other)      = <span class="keyword">delete</span>;</span><br><span class="line">    <span class="keyword">virtual</span> ~EpollTcpBase()                            = <span class="keyword">default</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">bool</span> <span class="title">Start</span><span class="params">()</span> </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">bool</span> <span class="title">Stop</span><span class="params">()</span>  </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">int32_t</span> <span class="title">SendData</span><span class="params">(<span class="keyword">const</span> PacketPtr&amp; data)</span> </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">RegisterOnRecvCallback</span><span class="params">(<span class="keyword">callback_recv_t</span> callback)</span> </span>= <span class="number">0</span>;</span><br><span class="line">    <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">UnRegisterOnRecvCallback</span><span class="params">()</span> </span>= <span class="number">0</span>;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> ETBase = EpollTcpBase;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;ETBase&gt; ETBasePtr;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">// the implementation of Epoll Tcp client</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">EpollTcpClient</span> :</span> <span class="keyword">public</span> ETBase &#123;</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    EpollTcpClient()                                       = <span class="keyword">default</span>;</span><br><span class="line">    EpollTcpClient(<span class="keyword">const</span> EpollTcpClient&amp; other)            = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpClient&amp; <span class="keyword">operator</span>=(<span class="keyword">const</span> EpollTcpClient&amp; other) = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpClient(EpollTcpClient&amp;&amp; other)                 = <span class="keyword">delete</span>;</span><br><span class="line">    EpollTcpClient&amp; <span class="keyword">operator</span>=(EpollTcpClient&amp;&amp; other)      = <span class="keyword">delete</span>;</span><br><span class="line">    ~EpollTcpClient() <span class="keyword">override</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// the server ip and port</span></span><br><span class="line">    EpollTcpClient(<span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; server_ip, <span class="keyword">uint16_t</span> server_port);</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">    <span class="comment">// start tcp client</span></span><br><span class="line">    <span class="function"><span class="keyword">bool</span> <span class="title">Start</span><span class="params">()</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="comment">// stop tcp client</span></span><br><span class="line">    <span class="function"><span class="keyword">bool</span> <span class="title">Stop</span><span class="params">()</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="comment">// send packet</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">SendData</span><span class="params">(<span class="keyword">const</span> PacketPtr&amp; data)</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="comment">// register a callback when packet received</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">RegisterOnRecvCallback</span><span class="params">(<span class="keyword">callback_recv_t</span> callback)</span> <span class="keyword">override</span></span>;</span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">UnRegisterOnRecvCallback</span><span class="params">()</span> <span class="keyword">override</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">protected</span>:</span><br><span class="line">    <span class="comment">// create epoll instance using epoll_create and return a fd of epoll</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">CreateEpoll</span><span class="params">()</span></span>;</span><br><span class="line">    <span class="comment">// create a socket fd using api socket()</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">CreateSocket</span><span class="params">()</span></span>;</span><br><span class="line">    <span class="comment">// connect to server</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">Connect</span><span class="params">(<span class="keyword">int32_t</span> listenfd)</span></span>;</span><br><span class="line">    <span class="comment">// add/modify/remove a item(socket/fd) in epoll instance(rbtree), for this example, just add a socket to epoll rbtree</span></span><br><span class="line">    <span class="function"><span class="keyword">int32_t</span> <span class="title">UpdateEpollEvents</span><span class="params">(<span class="keyword">int</span> efd, <span class="keyword">int</span> op, <span class="keyword">int</span> fd, <span class="keyword">int</span> events)</span></span>;</span><br><span class="line">    <span class="comment">// handle tcp socket readable event(read())</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">OnSocketRead</span><span class="params">(<span class="keyword">int32_t</span> fd)</span></span>;</span><br><span class="line">    <span class="comment">// handle tcp socket writeable event(write())</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">OnSocketWrite</span><span class="params">(<span class="keyword">int32_t</span> fd)</span></span>;</span><br><span class="line">    <span class="comment">// one loop per thread, call epoll_wait and return ready socket(readable,writeable,error...)</span></span><br><span class="line">    <span class="function"><span class="keyword">void</span> <span class="title">EpollLoop</span><span class="params">()</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span>:</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> server_ip_; <span class="comment">// tcp server ip</span></span><br><span class="line">    <span class="keyword">uint16_t</span> server_port_ &#123; <span class="number">0</span> &#125;; <span class="comment">// tcp server port</span></span><br><span class="line">    <span class="keyword">int32_t</span> handle_ &#123; <span class="number">-1</span> &#125;; <span class="comment">// client fd</span></span><br><span class="line">    <span class="keyword">int32_t</span> efd_ &#123; <span class="number">-1</span> &#125;; <span class="comment">// epoll fd</span></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;<span class="built_in">std</span>::thread&gt; th_loop_ &#123; <span class="literal">nullptr</span> &#125;; <span class="comment">// one loop per thread(call epoll_wait in loop)</span></span><br><span class="line">    <span class="keyword">bool</span> loop_flag_ &#123; <span class="literal">true</span> &#125;; <span class="comment">// if loop_flag_ is false, then exit the epoll loop</span></span><br><span class="line">    <span class="keyword">callback_recv_t</span> recv_callback_ &#123; <span class="literal">nullptr</span> &#125;; <span class="comment">// callback when received</span></span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> ETClient = EpollTcpClient;</span><br><span class="line"></span><br><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">shared_ptr</span>&lt;ETClient&gt; ETClientPtr;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">EpollTcpClient::EpollTcpClient(<span class="keyword">const</span> <span class="built_in">std</span>::<span class="built_in">string</span>&amp; server_ip, <span class="keyword">uint16_t</span> server_port)</span><br><span class="line">    : server_ip_ &#123; server_ip &#125;,</span><br><span class="line">      server_port_ &#123; server_port &#125; &#123;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">EpollTcpClient::~EpollTcpClient() &#123;</span><br><span class="line">    Stop();</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">bool</span> <span class="title">EpollTcpClient::Start</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// create epoll instance</span></span><br><span class="line">    <span class="keyword">if</span> (CreateEpoll() &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// create socket and bind</span></span><br><span class="line">    <span class="keyword">int</span> cli_fd  = CreateSocket();</span><br><span class="line">    <span class="keyword">if</span> (cli_fd &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// connect to server</span></span><br><span class="line">    <span class="keyword">int</span> lr = Connect(cli_fd);</span><br><span class="line">    <span class="keyword">if</span> (lr &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"EpollTcpClient Init success!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    handle_ = cli_fd;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// after connected successfully, add this socket to epoll instance, and focus on EPOLLIN and EPOLLOUT event</span></span><br><span class="line">    <span class="keyword">int</span> er = UpdateEpollEvents(efd_, EPOLL_CTL_ADD, handle_, EPOLLIN | EPOLLET);</span><br><span class="line">    <span class="keyword">if</span> (er &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// if something goes wrong, close listen socket and return false</span></span><br><span class="line">        ::<span class="built_in">close</span>(handle_);</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    assert(!th_loop_);</span><br><span class="line"></span><br><span class="line">    <span class="comment">// the implementation of one loop per thread: create a thread to loop epoll</span></span><br><span class="line">    th_loop_ = <span class="built_in">std</span>::make_shared&lt;<span class="built_in">std</span>::thread&gt;(&amp;EpollTcpClient::EpollLoop, <span class="keyword">this</span>);</span><br><span class="line">    <span class="keyword">if</span> (!th_loop_) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// detach the thread(using loop_flag_ to control the start/stop of loop)</span></span><br><span class="line">    th_loop_-&gt;<span class="built_in">detach</span>();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">// stop epoll tcp client and release epoll</span></span><br><span class="line"><span class="function"><span class="keyword">bool</span> <span class="title">EpollTcpClient::Stop</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    loop_flag_ = <span class="literal">false</span>;</span><br><span class="line">    ::<span class="built_in">close</span>(handle_);</span><br><span class="line">    ::<span class="built_in">close</span>(efd_);</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"stop epoll!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">    UnRegisterOnRecvCallback();</span><br><span class="line">    <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpClient::CreateEpoll</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// the basic epoll api of create a epoll instance</span></span><br><span class="line">    <span class="keyword">int</span> epollfd = epoll_create(<span class="number">1</span>);</span><br><span class="line">    <span class="keyword">if</span> (epollfd &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// if something goes wrong, return -1</span></span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_create failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    efd_ = epollfd;</span><br><span class="line">    <span class="keyword">return</span> epollfd;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpClient::CreateSocket</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// create tcp socket</span></span><br><span class="line">    <span class="keyword">int</span> cli_fd = ::socket(AF_INET, SOCK_STREAM, <span class="number">0</span>);</span><br><span class="line">    <span class="keyword">if</span> (cli_fd &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"create socket failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> cli_fd;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// connect to tcp server</span></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpClient::Connect</span><span class="params">(<span class="keyword">int32_t</span> cli_fd)</span> </span>&#123;</span><br><span class="line">    <span class="class"><span class="keyword">struct</span> <span class="title">sockaddr_in</span> <span class="title">addr</span>;</span>  <span class="comment">// server info</span></span><br><span class="line">    <span class="built_in">memset</span>(&amp;addr, <span class="number">0</span>, <span class="keyword">sizeof</span>(addr));</span><br><span class="line">    addr.sin_family = AF_INET;</span><br><span class="line">    addr.sin_port = htons(server_port_);</span><br><span class="line">    addr.sin_addr.s_addr  = inet_addr(server_ip_.c_str());</span><br><span class="line"></span><br><span class="line">    <span class="keyword">int</span> r = ::<span class="built_in">connect</span>(cli_fd, (struct sockaddr*)&amp;addr, <span class="keyword">sizeof</span>(addr));</span><br><span class="line">    <span class="keyword">if</span> ( r &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"connect failed! r="</span> &lt;&lt; r &lt;&lt; <span class="string">" errno:"</span> &lt;&lt; errno &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// add/modify/remove a item(socket/fd) in epoll instance(rbtree), for this example, just add a socket to epoll rbtree</span></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpClient::UpdateEpollEvents</span><span class="params">(<span class="keyword">int</span> efd, <span class="keyword">int</span> op, <span class="keyword">int</span> fd, <span class="keyword">int</span> events)</span> </span>&#123;</span><br><span class="line">    <span class="class"><span class="keyword">struct</span> <span class="title">epoll_event</span> <span class="title">ev</span>;</span></span><br><span class="line">    <span class="built_in">memset</span>(&amp;ev, <span class="number">0</span>, <span class="keyword">sizeof</span>(ev));</span><br><span class="line">    ev.events = events;</span><br><span class="line">    ev.data.fd = fd;</span><br><span class="line">    <span class="built_in">fprintf</span>(<span class="built_in">stdout</span>,<span class="string">"%s fd %d events read %d write %d\n"</span>, op == EPOLL_CTL_MOD ? <span class="string">"mod"</span> : <span class="string">"add"</span>, fd, ev.events &amp; EPOLLIN, ev.events &amp; EPOLLOUT);</span><br><span class="line">    <span class="keyword">int</span> r = epoll_ctl(efd, op, fd, &amp;ev);</span><br><span class="line">    <span class="keyword">if</span> (r &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_ctl failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// register a callback when packet received</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpClient::RegisterOnRecvCallback</span><span class="params">(<span class="keyword">callback_recv_t</span> callback)</span> </span>&#123;</span><br><span class="line">    assert(!recv_callback_);</span><br><span class="line">    recv_callback_ = callback;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpClient::UnRegisterOnRecvCallback</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    assert(recv_callback_);</span><br><span class="line">    recv_callback_ = <span class="literal">nullptr</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// handle read events on fd</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpClient::OnSocketRead</span><span class="params">(<span class="keyword">int32_t</span> fd)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">char</span> read_buf[<span class="number">4096</span>];</span><br><span class="line">    bzero(read_buf, <span class="keyword">sizeof</span>(read_buf));</span><br><span class="line">    <span class="keyword">int</span> n = <span class="number">-1</span>;</span><br><span class="line">    <span class="keyword">while</span> ( (n = ::<span class="built_in">read</span>(fd, read_buf, <span class="keyword">sizeof</span>(read_buf))) &gt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// callback for recv</span></span><br><span class="line">        <span class="function"><span class="built_in">std</span>::<span class="built_in">string</span> <span class="title">msg</span><span class="params">(read_buf, n)</span></span>;</span><br><span class="line">        PacketPtr data = <span class="built_in">std</span>::make_shared&lt;Packet&gt;(fd, msg);</span><br><span class="line">        <span class="keyword">if</span> (recv_callback_) &#123;</span><br><span class="line">            <span class="comment">// handle recv packet</span></span><br><span class="line">            recv_callback_(data);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (n == <span class="number">-1</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> (errno == EAGAIN || errno == EWOULDBLOCK) &#123;</span><br><span class="line">            <span class="comment">// read finished</span></span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// something goes wrong for this fd, should close it</span></span><br><span class="line">        ::<span class="built_in">close</span>(fd);</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (n == <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="comment">// this may happen when client close socket. EPOLLRDHUP usually handle this, but just make sure; should close this fd</span></span><br><span class="line">        ::<span class="built_in">close</span>(fd);</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// handle write events on fd (usually happens when sending big files)</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpClient::OnSocketWrite</span><span class="params">(<span class="keyword">int32_t</span> fd)</span> </span>&#123;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd: "</span> &lt;&lt; fd &lt;&lt; <span class="string">" writeable!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int32_t</span> <span class="title">EpollTcpClient::SendData</span><span class="params">(<span class="keyword">const</span> PacketPtr&amp; data)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">int</span> r = ::<span class="built_in">write</span>(handle_, data-&gt;msg.data(), data-&gt;msg.<span class="built_in">size</span>());</span><br><span class="line">    <span class="keyword">if</span> (r == <span class="number">-1</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> (errno == EAGAIN || errno == EWOULDBLOCK) &#123;</span><br><span class="line">            <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// error happend</span></span><br><span class="line">        ::<span class="built_in">close</span>(handle_);</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd: "</span> &lt;&lt; handle_ &lt;&lt; <span class="string">" write error, close it!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> r;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// one loop per thread, call epoll_wait and handle all coming events</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">EpollTcpClient::EpollLoop</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="comment">// request some memory, if events ready, socket events will copy to this memory from kernel</span></span><br><span class="line">    <span class="class"><span class="keyword">struct</span> <span class="title">epoll_event</span>* <span class="title">alive_events</span> =  <span class="title">static_cast</span>&lt;epoll_event*&gt;(<span class="title">calloc</span>(<span class="title">kMaxEvents</span>, <span class="title">sizeof</span>(<span class="title">epoll_event</span>)));</span></span><br><span class="line">    <span class="keyword">if</span> (!alive_events) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"calloc memory failed for epoll_events!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">while</span> (loop_flag_) &#123;</span><br><span class="line">        <span class="keyword">int</span> num = epoll_wait(efd_, alive_events, kMaxEvents, kEpollWaitTime);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; num; ++i) &#123;</span><br><span class="line">            <span class="keyword">int</span> fd = alive_events[i].data.fd;</span><br><span class="line">            <span class="keyword">int</span> events = alive_events[i].events;</span><br><span class="line"></span><br><span class="line">            <span class="keyword">if</span> ( (events &amp; EPOLLERR) || (events &amp; EPOLLHUP) ) &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"epoll_wait error!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// An error has occured on this fd, or the socket is not ready for reading (why were we notified then?).</span></span><br><span class="line">                ::<span class="built_in">close</span>(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span>  <span class="keyword">if</span> (events &amp; EPOLLRDHUP) &#123;</span><br><span class="line">                <span class="comment">// Stream socket peer closed connection, or shut down writing half of connection.</span></span><br><span class="line">                <span class="comment">// more inportant, We still to handle disconnection when read()/recv() return 0 or -1 just to be sure.</span></span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"fd:"</span> &lt;&lt; fd &lt;&lt; <span class="string">" closed EPOLLRDHUP!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">                <span class="comment">// close fd and epoll will remove it</span></span><br><span class="line">                ::<span class="built_in">close</span>(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> <span class="keyword">if</span> ( events &amp; EPOLLIN ) &#123;</span><br><span class="line">                <span class="comment">// other fd read event coming, meaning data coming</span></span><br><span class="line">                OnSocketRead(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> <span class="keyword">if</span> ( events &amp; EPOLLOUT ) &#123;</span><br><span class="line">                <span class="comment">// write event for fd (not including listen-fd), meaning send buffer is available for big files</span></span><br><span class="line">                OnSocketWrite(fd);</span><br><span class="line">            &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"unknow epoll event!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="comment">// end for (int i = 0; ...</span></span><br><span class="line"></span><br><span class="line">    &#125; <span class="comment">// end while (loop_flag_)</span></span><br><span class="line">    <span class="built_in">free</span>(alive_events);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">&#125; <span class="comment">// end namespace transport</span></span><br><span class="line">&#125; <span class="comment">// end namespace mux</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> mux;</span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> mux::transport;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span>* argv[])</span> </span>&#123;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> server_ip &#123;<span class="string">"127.0.0.1"</span>&#125;;</span><br><span class="line">    <span class="keyword">uint16_t</span> server_port &#123; <span class="number">6666</span> &#125;;</span><br><span class="line">    <span class="keyword">if</span> (argc &gt;= <span class="number">2</span>) &#123;</span><br><span class="line">        server_ip = <span class="built_in">std</span>::<span class="built_in">string</span>(argv[<span class="number">1</span>]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (argc &gt;= <span class="number">3</span>) &#123;</span><br><span class="line">        server_port = <span class="built_in">std</span>::atoi(argv[<span class="number">2</span>]);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// create a tcp client</span></span><br><span class="line">    <span class="keyword">auto</span> tcp_client = <span class="built_in">std</span>::make_shared&lt;EpollTcpClient&gt;(server_ip, server_port);</span><br><span class="line">    <span class="keyword">if</span> (!tcp_client) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"tcp_client create faield!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="built_in">exit</span>(<span class="number">-1</span>);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    <span class="comment">// recv callback in lambda mode, you can set your own callback here</span></span><br><span class="line">    <span class="keyword">auto</span> recv_call = [&amp;](<span class="keyword">const</span> transport::PacketPtr&amp; data) -&gt; <span class="keyword">void</span> &#123;</span><br><span class="line">        <span class="comment">// just print recv data to stdout</span></span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"recv: "</span> &lt;&lt; data-&gt;msg &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// register recv callback to epoll tcp client</span></span><br><span class="line">    tcp_client-&gt;RegisterOnRecvCallback(recv_call);</span><br><span class="line"></span><br><span class="line">    <span class="comment">// start the epoll tcp client</span></span><br><span class="line">    <span class="keyword">if</span> (!tcp_client-&gt;Start()) &#123;</span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"tcp_client start failed!"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line">        <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="string">"############tcp_client started!################"</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span>;</span><br><span class="line"></span><br><span class="line">    <span class="built_in">std</span>::<span class="built_in">string</span> msg;</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        <span class="comment">// read content from stdin</span></span><br><span class="line">        <span class="built_in">std</span>::<span class="built_in">cout</span> &lt;&lt; <span class="built_in">std</span>::<span class="built_in">endl</span> &lt;&lt; <span class="string">"input:"</span>;</span><br><span class="line">        <span class="built_in">std</span>::getline(<span class="built_in">std</span>::<span class="built_in">cin</span>, msg);</span><br><span class="line">        <span class="keyword">auto</span> packet = <span class="built_in">std</span>::make_shared&lt;Packet&gt;(msg);</span><br><span class="line">        tcp_client-&gt;SendData(packet);</span><br><span class="line">        <span class="comment">//std::this_thread::sleep_for(std::chrono::seconds(1));</span></span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    tcp_client-&gt;Stop();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>代码和 server 端代码基本上很类似，除了没有 accept() 的处理，这里就不分析了。</p><h1 id="注意"><a href="#注意" class="headerlink" title="注意"></a>注意</h1><p>上面的代码是基于 ET模式（边缘触发模式）实现的。</p><p>源代码可以直接在我的 github: <a href="https://github.com/smaugx/epoll_examples" target="_blank" rel="noopener">https://github.com/smaugx/epoll_examples</a> 找到；</p><p>或者有兴趣的话也可以直接看我的另外一个项目 <a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">https://github.com/smaugx/mux</a>，基于 epoll 实现的高并发网络库。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#110;&#x75;&#120;&#99;&#x6f;&#x64;&#101;&#50;&#110;&#105;&#107;&#105;&#64;&#103;&#109;&#97;&#105;&#x6c;&#x2e;&#x63;&#111;&#109;">&#108;&#x69;&#110;&#x75;&#120;&#99;&#x6f;&#x64;&#101;&#50;&#110;&#105;&#107;&#105;&#64;&#103;&#109;&#97;&#105;&#x6c;&#x2e;&#x63;&#111;&#109;</a></p></li></ul><p>2020-09-26 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;复习一下&quot;&gt;&lt;a href=&quot;#复习一下&quot; class=&quot;headerlink&quot; title=&quot;复习一下&quot;&gt;&lt;/a&gt;复习一下&lt;/h1&gt;&lt;p&gt;上一篇博文 &lt;a href=&quot;https://rebootcat.com/2020/09/26/epoll_cookbook/&quot;&gt;epoll原理深入分析&lt;/a&gt; 详细分析了 epoll 底层的实现原理，如果对 epoll 原理有模糊的建议先看一下这篇文章。那么本文就开始用 epoll 实现一个简单的 tcp server/client。&lt;/p&gt;
&lt;p&gt;本文基于我的 github: &lt;a href=&quot;https://github.com/smaugx/epoll_examples&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://github.com/smaugx/epoll_examples&lt;/a&gt;。&lt;/p&gt;
&lt;h1 id=&quot;epoll-实现范式&quot;&gt;&lt;a href=&quot;#epoll-实现范式&quot; class=&quot;headerlink&quot; title=&quot;epoll 实现范式&quot;&gt;&lt;/a&gt;epoll 实现范式&lt;/h1&gt;&lt;figure class=&quot;highlight arduino&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;7&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;8&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;9&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;12&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;13&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;14&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;15&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;16&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;17&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;18&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;19&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;20&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;21&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;22&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;23&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;24&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;25&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;26&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;27&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;28&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;29&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;30&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;31&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;32&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;33&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;34&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;35&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;36&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;37&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;38&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;39&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;40&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;41&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;42&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;43&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;44&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;45&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;46&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;47&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;48&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;49&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;50&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;51&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;52&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;53&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;54&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;# create listen socket&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; listenfd = ::socket();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;# bind to local port and ip&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; r = ::bind();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;# create epoll instance and get an epoll-fd&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; epollfd = epoll_create(&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt; &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;# add listenfd to epoll instance&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; r = epoll_ctl(..., listenfd, ...);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;# begin epoll_wait, wait for ready socket&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;class&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;struct&lt;/span&gt; &lt;span class=&quot;title&quot;&gt;epoll_event&lt;/span&gt;* &lt;span class=&quot;title&quot;&gt;alive_events&lt;/span&gt; =  &lt;span class=&quot;title&quot;&gt;static_cast&lt;/span&gt;&amp;lt;epoll_event*&amp;gt;(&lt;span class=&quot;title&quot;&gt;calloc&lt;/span&gt;(&lt;span class=&quot;title&quot;&gt;kMaxEvents&lt;/span&gt;, &lt;span class=&quot;title&quot;&gt;sizeof&lt;/span&gt;(&lt;span class=&quot;title&quot;&gt;epoll_event&lt;/span&gt;)));&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;while&lt;/span&gt; (&lt;span class=&quot;literal&quot;&gt;true&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; num = epoll_wait(epollfd, alive_events, kMaxEvents, kEpollWaitTime);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; i = &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;; i &amp;lt; num; ++i) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; fd = alive_events[i].data.fd;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;int&lt;/span&gt; events = alive_events[i].events;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ( (events &amp;amp; EPOLLERR) || (events &amp;amp; EPOLLHUP) ) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;cout&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;string&quot;&gt;&quot;epoll_wait error!&quot;&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;endl&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;comment&quot;&gt;// An error has occured on this fd, or the socket is not ready for reading (why were we notified then?).&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                ::&lt;span class=&quot;built_in&quot;&gt;close&lt;/span&gt;(fd);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125; &lt;span class=&quot;keyword&quot;&gt;else&lt;/span&gt;  &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; (events &amp;amp; EPOLLRDHUP) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;comment&quot;&gt;// Stream socket peer closed connection, or shut down writing half of connection.&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;comment&quot;&gt;// more inportant, We still to handle disconnection when read()/recv() return 0 or -1 just to be sure.&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;cout&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;string&quot;&gt;&quot;fd:&quot;&lt;/span&gt; &amp;lt;&amp;lt; fd &amp;lt;&amp;lt; &lt;span class=&quot;string&quot;&gt;&quot; closed EPOLLRDHUP!&quot;&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;endl&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;comment&quot;&gt;// close fd and epoll will remove it&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                ::&lt;span class=&quot;built_in&quot;&gt;close&lt;/span&gt;(fd);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125; &lt;span class=&quot;keyword&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ( events &amp;amp; EPOLLIN ) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;cout&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;string&quot;&gt;&quot;epollin&quot;&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;endl&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; (fd == handle_) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    &lt;span class=&quot;comment&quot;&gt;// listen fd coming connections&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    OnSocketAccept();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &amp;#125; &lt;span class=&quot;keyword&quot;&gt;else&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    &lt;span class=&quot;comment&quot;&gt;// other fd read event coming, meaning data coming&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    OnSocketRead(fd);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125; &lt;span class=&quot;keyword&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ( events &amp;amp; EPOLLOUT ) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;cout&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;string&quot;&gt;&quot;epollout&quot;&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;endl&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;comment&quot;&gt;// write event for fd (not including listen-fd), meaning send buffer is available for big files&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                OnSocketWrite(fd);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125; &lt;span class=&quot;keyword&quot;&gt;else&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;cout&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;string&quot;&gt;&quot;unknow epoll event!&quot;&lt;/span&gt; &amp;lt;&amp;lt; &lt;span class=&quot;built_in&quot;&gt;std&lt;/span&gt;::&lt;span class=&quot;built_in&quot;&gt;endl&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &amp;#125; &lt;span class=&quot;comment&quot;&gt;// end for (int i = 0; ...&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;


&lt;p&gt;epoll 编程基本是按照上面的范式进行的，这里要注意的是上面的反应的只是单进程或者单线程的情况。&lt;/p&gt;
&lt;p&gt;如果涉及到多线程或者多进程，那么通常来说会在 listen() 创建完成之后，创建多线程或者多进程，然后再操作 epoll.&lt;/p&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="epoll" scheme="https://rebootcat.com/tags/epoll/"/>
    
      <category term="socket" scheme="https://rebootcat.com/tags/socket/"/>
    
      <category term="tcp" scheme="https://rebootcat.com/tags/tcp/"/>
    
      <category term="server" scheme="https://rebootcat.com/tags/server/"/>
    
      <category term="client" scheme="https://rebootcat.com/tags/client/"/>
    
  </entry>
  
  <entry>
    <title>惊群效应</title>
    <link href="https://rebootcat.com/2020/09/26/thundering_herd/"/>
    <id>https://rebootcat.com/2020/09/26/thundering_herd/</id>
    <published>2020-09-26T03:23:58.000Z</published>
    <updated>2020-11-07T15:38:35.317Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>上一篇博文 <a href="https://rebootcat.com/2020/09/26/epoll_cookbook/">Epoll原理深入分析</a> 在讲 <strong>accept 事件</strong> 的时候提到过 <strong>惊群效应</strong>，本文就分析一下惊群效应的原因以及解决方法。</p><h1 id="惊群效应"><a href="#惊群效应" class="headerlink" title="惊群效应"></a>惊群效应</h1><h2 id="什么是惊群"><a href="#什么是惊群" class="headerlink" title="什么是惊群"></a>什么是惊群</h2><p>惊群效应就是多个进程（线程）阻塞等待同一件事情（资源）上，当事件发生（资源可用）时，操作系统可能会唤醒所有等待这个事件（资源）的进程（线程），但是最终却只有一个进程（线程）成功获取该事件（资源），而其他进程（线程）获取失败，只能重新阻塞等待事件（资源）可用，但是这就造成了额外的性能损失。这种现象就称为惊群效应。</p><p>如果细心的你可能会问，为什么操作系统要同时唤醒多个进程呢？只唤醒一个不行吗？这样不就没有这种性能损失了吗？</p><p>确实如此，操作系统也想只唤醒一个进程，但是它做不到啊，因为它也不知道该唤醒哪一个，只好把所有等待在这件事情（资源）的进程都一起唤醒了。</p><p>那有没有办法解决呢？当然有，我们后面再说。</p><p>惊群效应会造成多个进程白白唤醒而啥也做不了。那么唤醒进程损失了啥？这就涉及到进程上下文的概念。</p><h2 id="惊群造成进程切换"><a href="#惊群造成进程切换" class="headerlink" title="惊群造成进程切换"></a>惊群造成进程切换</h2><p>进程上下文包括了进程的虚拟内存，栈，全局变量等用户空间的资源，还包括内核堆栈，寄存器等内核空间的状态。</p><p>所以<strong>进程上下文切换就首先需要保存用户态资源以及内核态资源，然后再去加载下一个进程，首先是加载了下一个进程的内核态，然后再去刷新进程的用户态空间</strong>。</p><p>然而 CPU 保存进程的用户态以及内核态资源，再去加载下一个进程的内核态和用户态是有代价的，也是耗时的，每次可能在几十纳秒到数微妙的时间，如果频繁发生进程切换，那么 CPU 将有大量的时间浪费在不断保存资源，加载资源，刷新资源等事情上，造成性能的浪费。</p><p>所以惊群效应会造成多个进程切换，造成性能损失。</p><h2 id="惊群测试"><a href="#惊群测试" class="headerlink" title="惊群测试"></a>惊群测试</h2><p>为了直观的了解惊群效应是什么，我们采用 mux 项目当中的 <code>echo_server</code> 为例说明：</p><p><a href="https://github.com/smaugx/mux/tree/master/demo/echo" target="_blank" rel="noopener">https://github.com/smaugx/mux/tree/master/demo/echo</a></p><p>编译命令详见项目说明文档。编译之后得到:</p><figure class="highlight jboss-cli"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">echo</span>_server  <span class="keyword">echo</span>_client</span><br></pre></td></tr></table></figure><p>我们在 <code>echo_server</code> 上开启 8 个 epoll 线程，观察当有新连接过来时是否这 8 个线程（epoll） 都被唤醒了。</p><a id="more"></a><p>首先，运行：</p><figure class="highlight jboss-cli"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">./echo_server</span></span><br></pre></td></tr></table></figure><p>再运行：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./echo_client （或者直接用  nc <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> <span class="number">6666</span>）</span><br></pre></td></tr></table></figure><p>我们观察的 <code>echo_server</code> 的 log 如下：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">smaug@smaug-VirtualBox:~/workspace/mux/cbuild/bin/log$ tail -f echo_server.log  |grep accept</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">[basic_logger] [<span class="builtin-name">debug</span>] eid:3 accept<span class="built_in"> connection </span><span class="keyword">from</span> 127.0.0.1:44251 time:2020.11.07 21:55:39.30</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:0 accept error, thundering herd happened</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:4 accept error, thundering herd happened</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:7 accept error, thundering herd happened</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:6 accept error, thundering herd happened</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:5 accept error, thundering herd happened</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:2 accept error, thundering herd happened</span><br><span class="line">[basic_logger] [<span class="builtin-name">warning</span>] eid:1 accept error, thundering herd happened</span><br></pre></td></tr></table></figure><p>从上可以看到，我们总共有 8 个线程，其中只有 3号线程（epoll）被唤醒并且成功获取了 accept 事件，其他线程均 <code>accept error</code>。</p><p>多次测试会有不同的线程获取 <code>accept</code> 事件，但是只有一个能够成功获取，其余的全部失败。</p><p>为了更加直观的感受惊群造成的性能损失，我们做一个并发压测：</p><p><a href="https://github.com/smaugx/mux/tree/master/demo/bench" target="_blank" rel="noopener">https://github.com/smaugx/mux/tree/master/demo/bench</a></p><p>编译上面的代码得到:</p><figure class="highlight armasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">bench_client_accept </span> <span class="keyword">bench_server</span></span><br></pre></td></tr></table></figure><p>首先，启动：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./bench_server <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> <span class="number">10000</span> &gt; /dev/<span class="literal">null</span> <span class="number">2</span>&gt;&amp;<span class="number">1</span> &amp;</span><br></pre></td></tr></table></figure><p>再启动：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./bench_client_accept <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> <span class="number">10000</span> <span class="number">30000</span> <span class="number">100</span> &gt; /dev/<span class="literal">null</span> <span class="number">2</span>&gt;&amp;<span class="number">1</span> &amp;</span><br></pre></td></tr></table></figure><p>使用之前的博文 <a href="https://rebootcat.com/2019/11/05/perf_flamegraph/">一键采集cpu生成火焰图</a> 中的脚本采集火焰图如下：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/thundering_herd/0.png" alt=""></p><p>可以看到途中 <code>__libc_accept</code> 占据了 31.7% 的 cpu，可以说是很高很高了。</p><p>由此可以看到惊群效应带来的性能损失有多少了吧。</p><h2 id="惊群的类型"><a href="#惊群的类型" class="headerlink" title="惊群的类型"></a>惊群的类型</h2><p>惊群的类型根据 socket 编程采用的不同方式有关。</p><h3 id="传统-accept-惊群"><a href="#传统-accept-惊群" class="headerlink" title="传统 accept 惊群"></a>传统 accept 惊群</h3><p>传统的多进程 socket 编程，通常是 <code>listen()</code> 之后创建多个 worker 进程进行 accept，那么这里就会造成当有新连接过来时，多个 worker 同时去 accept 的情况，但最终只有一个 worker 进程成功 accept，其余的全部失败。</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">..</span>.</span><br><span class="line">int s = socket();</span><br><span class="line"></span><br><span class="line">bind(s);</span><br><span class="line"></span><br><span class="line">listen(s);</span><br><span class="line"></span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> (<span class="attribute">i</span>=0;i &lt; 4;++i) &#123;</span><br><span class="line">    pid_t pid = fork();</span><br><span class="line">    </span><br><span class="line">    <span class="keyword">if</span> (pid == 0) &#123;</span><br><span class="line">        # handle new connection</span><br><span class="line">        accept();</span><br><span class="line">        <span class="built_in">..</span>.</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>此种情况下的惊群称为 <strong>accept惊群效应</strong>，这在 linux 内核2.6以后就已经解决了，所以通常情况下讨论的惊群通常不是 <strong>accept惊群</strong>，而是 <strong>epoll惊群</strong>。</p><h3 id="epoll-惊群"><a href="#epoll-惊群" class="headerlink" title="epoll 惊群"></a>epoll 惊群</h3><p>epoll 的编程模型一般有两种，我们姑且先分别称为 <strong>版本1</strong> 和 <strong>版本2</strong> 吧：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">int listenfd = ::socket();</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line">int epollfd = epoll_create(1);  # create epoll instance</span><br><span class="line"></span><br><span class="line">int p = fork() # 多进程 或者多线程创建</span><br><span class="line"></span><br><span class="line">int r = epoll_ctl(<span class="built_in">..</span>., listenfd, <span class="built_in">..</span>.);</span><br><span class="line"></span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span>(<span class="literal">true</span>) &#123;</span><br><span class="line">int num = epoll_wait(epollfd, alive_events, kMaxEvents, kEpollWaitTime);</span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>或者 <code>fork()</code> 在 <code>epoll_create()</code> 之前：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">int listenfd = ::socket();</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">int p = fork() # 多进程 或者多线程创建</span><br><span class="line"></span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line">int epollfd = epoll_create(1);  # create epoll instance</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">int r = epoll_ctl(<span class="built_in">..</span>., listenfd, <span class="built_in">..</span>.);</span><br><span class="line"></span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span>(<span class="literal">true</span>) &#123;</span><br><span class="line">int num = epoll_wait(epollfd, alive_events, kMaxEvents, kEpollWaitTime);</span><br><span class="line"><span class="built_in">..</span>.</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>虽然上述两个版本均能实现多进程下的 epoll 编程，且都存在惊群效应，但版本1，也就是 <code>fork()</code> 在 <code>epoll_create()</code> 之后会造成<strong>事件混乱</strong>。</p><p>因为<strong>多个进程等待的是同一个 epollfd，就有可能造成同一个连接，worker A 获取了 accept 事件，成功建立了连接，但是后续的读事件被 worker B 获取了，造成连接和读写事件不匹配的情况</strong>。</p><p>所以通常，我们采用的是版本2，也就是 <code>fork()</code> 在 <code>epoll_create()</code> 之前，那么多个子进程其实是拥有各自不同的 epollfd，只不过对于 listenfd 而言，都被添加到了各个子进程的 epoll instance 中。</p><p>当 listenfd 上有事件触发时（listenfd 上的事件自然是 accept 事件），由于有多个子进程的 epoll instance 上都有 listenfd。</p><p>根据之前的博文 <a href="https://rebootcat.com/2020/09/26/epoll_cookbook/">Epoll原理深入分析</a>，当某个 fd 上有事件后，内核会把这个 fd 拷贝到 epoll 的就绪链表中，并且唤醒进程，通知应用层使用 <code>epoll_wait</code> 来处理事件。</p><p>所以由于多个子进程都把 listenfd 插入到了自己的 epoll instance 中，那么当 listenfd 上有事件触发时，自然这些子进程都会被唤醒了。但是最终只有一个子进程成功获取 accept 事件，其余的均失败。这就是惊群效应，详见上面的惊群效应测试。</p><h1 id="解决方案"><a href="#解决方案" class="headerlink" title="解决方案"></a>解决方案</h1><h2 id="传统-accept-惊群-1"><a href="#传统-accept-惊群-1" class="headerlink" title="传统 accept 惊群"></a>传统 accept 惊群</h2><p>上面提到，针对<strong>传统 accept 惊群</strong>，linux 在内核 2.6 以后就解决了，内核通过引入一个 <code>WQ_FLAG_EXCLUSIVE</code> 标志位，告诉内核排他性的唤醒，即当 socket 上有事件触发时，对于等待队列中的进程，如果这些进程没有 <code>WQ_FLAG_EXCLUSIVE</code> 这个标志位，那么就通通唤醒，如果有 <code>WQ_FLAG_EXCLUSIVE</code> 这个标志位，那么唤醒第一个有这个标志位的进程则结束。这样，就解决了<strong>传统 accept 惊群</strong>问题。</p><h2 id="epoll-惊群-1"><a href="#epoll-惊群-1" class="headerlink" title="epoll 惊群"></a>epoll 惊群</h2><p>epoll 的惊群有两种解决办法。</p><h3 id="SO-REUSEPORT"><a href="#SO-REUSEPORT" class="headerlink" title="SO_REUSEPORT"></a>SO_REUSEPORT</h3><p>linux 在内核 3.9 版本引入了一个 socket 选项 <code>SO_REUSEPORT</code> 用来支持多个进程监听在同一个端口上，内核负责事件触发的负载均衡。</p><p>创建一个 listen socket，需要 {protocol, src_addr, src_port} 三元组，3.9 版本之前，内核不允许出现多个进程使用同样的三元组创建 socket，会出现 <code>Address already in use</code> 错误。</p><p>但是，通过引入 <code>SO_REUSEPORT</code> 以及 <code>SO_REUSEADDR</code>，内核允许多个进程使用同样的三元组创建 socket，内核负责负载均衡。</p><p>ok，明白了这个原理，对于解决 epoll 的惊群问题，还需要稍微修改一下编程的模型，我们姑且成为 <strong>版本3</strong> 吧：</p><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="built_in">int</span> p = fork<span class="literal">()</span> # 多进程 或者多线程创建</span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> listenfd = ::socket<span class="literal">()</span>;</span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> ret = setsockopt(listenfd, SOL_SOCKET, SO_REUSEPORT,(const void *)&amp;reuse , sizeof(<span class="built_in">int</span>));</span><br><span class="line"></span><br><span class="line">bind(listenfd);</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"><span class="built_in">int</span> epollfd = epoll<span class="constructor">_create(1)</span>;  # create epoll instance</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="built_in">int</span> r = epoll<span class="constructor">_ctl(<span class="operator">...</span>, <span class="params">listenfd</span>, <span class="operator">...</span>)</span>;</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span>(<span class="literal">true</span>) &#123;</span><br><span class="line"><span class="built_in">int</span> num = epoll<span class="constructor">_wait(<span class="params">epollfd</span>, <span class="params">alive_events</span>, <span class="params">kMaxEvents</span>, <span class="params">kEpollWaitTime</span>)</span>;</span><br><span class="line">...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>可以对比一下和上面的编程模型有何不同，其实区别在创建 listenfd 被移到了 <code>fork()</code> 之后，程序启动即创建多个进程，然后进程内部再创建 listenfd 以及 epollfd，等等后续一系列操作。</p><p><strong>另外要注意在 <code>socket()</code> 之后，使用 <code>setsockopt()</code> 设置了 <code>SO_REUSEPORT</code> 选项</strong>。</p><p>那么内核是如何做负载均衡的呢？</p><p>其实很简单，每一个新的连接都具有 socket 五元组 {protocol, src_addr, src_port, dst_addr, dst_port}，那么用这个五元组哈希一下映射到不同的进程，那么就唤醒这个进程。</p><p><code>SO_REUSEPORT</code> 由于采用的是哈希的方式，内核并不知道多个等待进程是否空闲，但哈希的方式依然可能还会分配到这个进程，此时这个新的 accept 就可能会超时不被处理。</p><h3 id="EPOLLEXCLUSIVE"><a href="#EPOLLEXCLUSIVE" class="headerlink" title="EPOLLEXCLUSIVE"></a>EPOLLEXCLUSIVE</h3><p>linux 在内核 4.5 引入了 EPOLLEXCLUSIVE 这个标志位用来解决 epoll 的惊群。当我们使用 <code>epoll_ctl()</code> 往进程的 epoll instance 中插入一个需要监听的 fd 时，如果显示的传入 EPOLLEXCLUSIVE，那么内核会排他性的进行唤醒。</p><p>当然这里通常<strong>只需要对多个子进程共同监听的 listenfd 设置 <code>EPOLLEXCLUSIVE</code> 标志位</strong>。<strong>注意，这里的 epoll 编程模型要采用上面的版本2</strong>。</p><p>和解决<strong>传统 accept惊群</strong> 类似的方式，但是区别是内核可能会唤醒不只一个进程（虽然解决了不全部唤醒的问题），详见:</p><p><a href="https://man7.org/linux/man-pages/man2/epoll_ctl.2.html" target="_blank" rel="noopener">https://man7.org/linux/man-pages/man2/epoll_ctl.2.html</a></p><figure class="highlight livecodeserver"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">When <span class="keyword">a</span> wakeup event occurs <span class="keyword">and</span> multiple epoll <span class="built_in">file</span> descriptors</span><br><span class="line">are attached <span class="built_in">to</span> <span class="keyword">the</span> same target <span class="built_in">file</span> <span class="keyword">using</span> EPOLLEXCLUSIVE, <span class="literal">one</span></span><br><span class="line"><span class="keyword">or</span> more <span class="keyword">of</span> <span class="keyword">the</span> epoll <span class="built_in">file</span> descriptors will receive <span class="keyword">an</span> event</span><br><span class="line"><span class="keyword">with</span> epoll_wait(<span class="number">2</span>).  The default <span class="keyword">in</span> this scenario (when</span><br><span class="line">EPOLLEXCLUSIVE is <span class="keyword">not</span> <span class="built_in">set</span>) is <span class="keyword">for</span> all epoll <span class="built_in">file</span> descriptors</span><br><span class="line"><span class="built_in">to</span> receive <span class="keyword">an</span> event.  EPOLLEXCLUSIVE is thus useful <span class="keyword">for</span> avoid‐</span><br><span class="line">ing thundering herd problems <span class="keyword">in</span> certain scenarios.</span><br></pre></td></tr></table></figure><blockquote><p>注意上面的 <strong>one or more</strong></p></blockquote><h1 id="The-END"><a href="#The-END" class="headerlink" title="The END"></a>The END</h1><p>OK，到这里基本上把惊群效应的原理以及带来的问题，以及解决方法都讲清楚了，本来还想做一个加上了 EPOLLEXCLUSIVE，再采集一下火焰图和之前的进行一下对比，但是不知道是我的方式不对还是什么原因，加上 EPOLLEXCLUSIVE 标志后，连接压力测试就池池上不去，一直出现 <code>connection timeout</code> 的问题。</p><p>算了，这个以后再研究下为啥。</p><p>如果上文发现有什么不对的地方，欢迎指正。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#x69;&#110;&#x75;&#120;&#99;&#x6f;&#x64;&#101;&#x32;&#x6e;&#x69;&#x6b;&#x69;&#x40;&#x67;&#x6d;&#x61;&#105;&#108;&#46;&#x63;&#111;&#x6d;">&#x6c;&#x69;&#110;&#x75;&#120;&#99;&#x6f;&#x64;&#101;&#x32;&#x6e;&#x69;&#x6b;&#x69;&#x40;&#x67;&#x6d;&#x61;&#105;&#108;&#46;&#x63;&#111;&#x6d;</a></p></li></ul><p>2020-09-26 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;上一篇博文 &lt;a href=&quot;https://rebootcat.com/2020/09/26/epoll_cookbook/&quot;&gt;Epoll原理深入分析&lt;/a&gt; 在讲 &lt;strong&gt;accept 事件&lt;/strong&gt; 的时候提到过 &lt;strong&gt;惊群效应&lt;/strong&gt;，本文就分析一下惊群效应的原因以及解决方法。&lt;/p&gt;
&lt;h1 id=&quot;惊群效应&quot;&gt;&lt;a href=&quot;#惊群效应&quot; class=&quot;headerlink&quot; title=&quot;惊群效应&quot;&gt;&lt;/a&gt;惊群效应&lt;/h1&gt;&lt;h2 id=&quot;什么是惊群&quot;&gt;&lt;a href=&quot;#什么是惊群&quot; class=&quot;headerlink&quot; title=&quot;什么是惊群&quot;&gt;&lt;/a&gt;什么是惊群&lt;/h2&gt;&lt;p&gt;惊群效应就是多个进程（线程）阻塞等待同一件事情（资源）上，当事件发生（资源可用）时，操作系统可能会唤醒所有等待这个事件（资源）的进程（线程），但是最终却只有一个进程（线程）成功获取该事件（资源），而其他进程（线程）获取失败，只能重新阻塞等待事件（资源）可用，但是这就造成了额外的性能损失。这种现象就称为惊群效应。&lt;/p&gt;
&lt;p&gt;如果细心的你可能会问，为什么操作系统要同时唤醒多个进程呢？只唤醒一个不行吗？这样不就没有这种性能损失了吗？&lt;/p&gt;
&lt;p&gt;确实如此，操作系统也想只唤醒一个进程，但是它做不到啊，因为它也不知道该唤醒哪一个，只好把所有等待在这件事情（资源）的进程都一起唤醒了。&lt;/p&gt;
&lt;p&gt;那有没有办法解决呢？当然有，我们后面再说。&lt;/p&gt;
&lt;p&gt;惊群效应会造成多个进程白白唤醒而啥也做不了。那么唤醒进程损失了啥？这就涉及到进程上下文的概念。&lt;/p&gt;
&lt;h2 id=&quot;惊群造成进程切换&quot;&gt;&lt;a href=&quot;#惊群造成进程切换&quot; class=&quot;headerlink&quot; title=&quot;惊群造成进程切换&quot;&gt;&lt;/a&gt;惊群造成进程切换&lt;/h2&gt;&lt;p&gt;进程上下文包括了进程的虚拟内存，栈，全局变量等用户空间的资源，还包括内核堆栈，寄存器等内核空间的状态。&lt;/p&gt;
&lt;p&gt;所以&lt;strong&gt;进程上下文切换就首先需要保存用户态资源以及内核态资源，然后再去加载下一个进程，首先是加载了下一个进程的内核态，然后再去刷新进程的用户态空间&lt;/strong&gt;。&lt;/p&gt;
&lt;p&gt;然而 CPU 保存进程的用户态以及内核态资源，再去加载下一个进程的内核态和用户态是有代价的，也是耗时的，每次可能在几十纳秒到数微妙的时间，如果频繁发生进程切换，那么 CPU 将有大量的时间浪费在不断保存资源，加载资源，刷新资源等事情上，造成性能的浪费。&lt;/p&gt;
&lt;p&gt;所以惊群效应会造成多个进程切换，造成性能损失。&lt;/p&gt;
&lt;h2 id=&quot;惊群测试&quot;&gt;&lt;a href=&quot;#惊群测试&quot; class=&quot;headerlink&quot; title=&quot;惊群测试&quot;&gt;&lt;/a&gt;惊群测试&lt;/h2&gt;&lt;p&gt;为了直观的了解惊群效应是什么，我们采用 mux 项目当中的 &lt;code&gt;echo_server&lt;/code&gt; 为例说明：&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://github.com/smaugx/mux/tree/master/demo/echo&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://github.com/smaugx/mux/tree/master/demo/echo&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;编译命令详见项目说明文档。编译之后得到:&lt;/p&gt;
&lt;figure class=&quot;highlight jboss-cli&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;echo&lt;/span&gt;_server  &lt;span class=&quot;keyword&quot;&gt;echo&lt;/span&gt;_client&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;我们在 &lt;code&gt;echo_server&lt;/code&gt; 上开启 8 个 epoll 线程，观察当有新连接过来时是否这 8 个线程（epoll） 都被唤醒了。&lt;/p&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="cpu" scheme="https://rebootcat.com/tags/cpu/"/>
    
      <category term="nginx" scheme="https://rebootcat.com/tags/nginx/"/>
    
      <category term="epoll" scheme="https://rebootcat.com/tags/epoll/"/>
    
      <category term="socket" scheme="https://rebootcat.com/tags/socket/"/>
    
      <category term="accept" scheme="https://rebootcat.com/tags/accept/"/>
    
      <category term="thundering_herd" scheme="https://rebootcat.com/tags/thundering-herd/"/>
    
      <category term="setsockopt" scheme="https://rebootcat.com/tags/setsockopt/"/>
    
  </entry>
  
  <entry>
    <title>Epoll原理深入分析</title>
    <link href="https://rebootcat.com/2020/09/26/epoll_cookbook/"/>
    <id>https://rebootcat.com/2020/09/26/epoll_cookbook/</id>
    <published>2020-09-26T02:50:58.000Z</published>
    <updated>2020-09-25T12:28:48.219Z</updated>
    
    <content type="html"><![CDATA[<h1 id="Epoll-的出现"><a href="#Epoll-的出现" class="headerlink" title="Epoll 的出现"></a>Epoll 的出现</h1><p>想必能搜到这篇文章的，应该对 select/poll 有一些了解和认识，一般说 epoll 都会与 select/poll 进行一些对比，select、poll 和 epoll 都是一种 IO 多路复用机制。</p><h2 id="select-的问题"><a href="#select-的问题" class="headerlink" title="select 的问题"></a>select 的问题</h2><p>select 的问题在于描述符的限制，能监控的文件描述符最大为 FD_SETSIZE，对于连接数很多的场景就无法满足；</p><p>另外select 还有一个问题是，每次调用 select 都需要从用户空间把描述符集合拷贝到内核空间，当描述符集合变大之后，用户空间和内核空间的内存拷贝会导致效率低下；</p><p>另外每次调用 select 都需要在内核线性遍历文件描述符的集合，当描述符增多，效率低下。</p><h2 id="poll-的问题"><a href="#poll-的问题" class="headerlink" title="poll 的问题"></a>poll 的问题</h2><p>由于 select 存在上面的问题，于是 poll 被提了出来，它能解决 select 对文件描述符数量有限制的问题，但是依然不能解决线性遍历以及用户空间和内核空间的低效数据拷贝问题。</p><h2 id="epoll-是什么"><a href="#epoll-是什么" class="headerlink" title="epoll 是什么"></a>epoll 是什么</h2><p>select/poll 在互联网早期应该是没什么问题的，因为没有很多的互联网服务，也没有很多的客户端，但是随着互联网的发展，<a href="http://www.kegel.com/c10k.html#related" target="_blank" rel="noopener">C10K</a> 等问题的出现，select/poll 已经不能满足要求了，这个时候 epoll 上场了。</p><p>epoll 是 linux 内核 2.6 之后支持的，epoll 同 select/poll 一样，也是 IO 多路复用的一种机制，不过它避免了 select/poll 的缺点。下面详细讲解一下 epoll 反应堆的原理。</p><h1 id="Epoll-反应堆"><a href="#Epoll-反应堆" class="headerlink" title="Epoll 反应堆"></a>Epoll 反应堆</h1><h2 id="epoll-原理"><a href="#epoll-原理" class="headerlink" title="epoll 原理"></a>epoll 原理</h2><blockquote><p><strong>要完整描述 epoll 的原理，需要涉及到内核、网卡、中断、软中断、协议栈、套接字等知识，本文尽量从比较全面的角度来分析 epoll 的原理</strong>。</p></blockquote><p>上面其实讨论了 select/poll 几个缺点，针对这几个缺点，就需要解决以下几件事：</p><ul><li>如何突破文件描述符数量的限制</li><li>如何避免用户态和内核态对文件描述符集合的拷贝</li><li>socket 就绪后，如何避免线性遍历文件描述符集合</li></ul><a id="more"></a><p>针对第一点：<strong>如何突破文件描述符数量的限制</strong>，其实 poll 已经解决了，poll 使用的是链表的方式管理 socket 描述符，但问题是不够高效，如果有百万级别的连接需要管理，如何快速的插入和删除就变得很重要，于是 epoll 采用了红黑树的方式进行管理，这样能保证在添加 socket 和删除 socket 时，有 O(log(n)) 的复杂度。</p><p>针对第二点：<strong>如何避免用户态和内核态对文件描述符集合的拷贝</strong>，其实对于 select 来说，由于这个集合是保存在用户态的，所以当调用 select 时需要屡次的把这个描述符集合拷贝到内核空间。所以<strong>如果要解决这个问题，可以直接把这个集合放在内核空间进行管理</strong>。没错，epoll 就是这样做的，epoll 在内核空间创建了一颗红黑树，应用程序直接把需要监控的 socket 对象添加到这棵树上，直接从用户态到内核态了，而且后续也不需要再次拷贝了。</p><p>针对第三点：<strong>socket就绪后，如何避免内核线性遍历文件描述符集合</strong>，这个问题就会比较复杂，要完整理解就得涉及到内核收包到应用层的整个过程。这里先简单讲一下，与 select 不同，epoll 使用了一个双向链表来保存就绪的 socket，这样当活跃连接数不多的情况下，应用程序只需要遍历这个就绪链表就行了，而 select 没有这样一个用来存储就绪 socket 的东西，导致每次需要线性遍历所有socket，以确定是哪个或者哪几个 socket 就绪了。这里需要注意的是，<strong>这个就绪链表保存活跃链接，数量是较少的，也需要从内核空间拷贝到用户空间</strong>。</p><p>从上面 3 点可以看到 epoll 的几个特点：</p><ul><li><strong>程序在内核空间开辟一块缓存，用来管理 epoll 红黑树，高效添加和删除</strong></li><li><strong>红黑树位于内核空间，用来直接管理 socket，减少和用户态的交互</strong></li><li><strong>使用双向链表缓存就绪的 socket，数量较少</strong></li><li><strong>只需要拷贝这个双向链表到用户空间，再遍历就行，注意这里也需要拷贝，没有共享内存</strong></li></ul><p>比较精炼的话可能反而理解起来不容易，那么接下来深入分析一下 epoll 的原理。</p><h2 id="epoll-api"><a href="#epoll-api" class="headerlink" title="epoll api"></a>epoll api</h2><p>如果要深入分析 epoll 的原理，那么可能需要结合到 epoll 的 api 来进行阐述。epoll api 较少，使用起来相对比较简单。</p><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">#<span class="keyword">include</span> &lt;sys/epoll.h&gt;</span><br><span class="line"></span><br><span class="line"># <span class="keyword">open</span> an epoll file descriptor</span><br><span class="line"># epoll_create1 可以理解为 epoll_create 的增强版（主要支持了 close-on-exec）</span><br><span class="line"><span class="built_in">int</span> epoll<span class="constructor">_create(<span class="params">int</span> <span class="params">size</span>)</span>;</span><br><span class="line"><span class="built_in">int</span> epoll<span class="constructor">_create1(<span class="params">int</span> <span class="params">flags</span>)</span>;</span><br><span class="line"></span><br><span class="line"># 往 epoll instance 上添加、删除、更改一个节点（socket)</span><br><span class="line"><span class="built_in">int</span> epoll<span class="constructor">_ctl(<span class="params">int</span> <span class="params">epfd</span>, <span class="params">int</span> <span class="params">op</span>, <span class="params">int</span> <span class="params">fd</span>, <span class="params">struct</span> <span class="params">epoll_event</span> <span class="operator">*</span><span class="params">event</span>)</span>;</span><br><span class="line"></span><br><span class="line"># wait for events on epoll instance</span><br><span class="line"><span class="built_in">int</span> epoll<span class="constructor">_wait(<span class="params">int</span> <span class="params">epfd</span>, <span class="params">struct</span> <span class="params">epoll_event</span> <span class="operator">*</span><span class="params">events</span>, <span class="params">int</span> <span class="params">maxevents</span>, <span class="params">int</span> <span class="params">timeout</span>)</span>;</span><br><span class="line"></span><br><span class="line"># close <span class="keyword">and</span> clear epoll instance</span><br><span class="line"><span class="built_in">int</span> close(<span class="built_in">int</span> fd);</span><br></pre></td></tr></table></figure><p>epoll 涉及到的 api 其实比较简单，掌握了这几个 api 其实就已经能够快速编写基于 epoll 的 tcp/udp socket 程序。可以参考：</p><p><a href="https://github.com/smaugx/epoll_examples.git" target="_blank" rel="noopener">https://github.com/smaugx/epoll_examples.git</a></p><p><strong>接下来结合上面的几个 api 来详细分析以下背后的原理</strong>。</p><h2 id="红黑树的创建和操作"><a href="#红黑树的创建和操作" class="headerlink" title="红黑树的创建和操作"></a>红黑树的创建和操作</h2><p>前面提到，epoll 是一种 IO 多路复用机制，应用程序可以同时监控多个 socket，那么如何来存储和管理这些 socket 呢，epoll 使用的是一颗红黑树，可以随意的往这棵树上添加节点和删除节点(节点是一个结构体，包括 socket fd)。</p><p>我们使用：</p><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">epoll_create</span><span class="params">(<span class="keyword">int</span> <span class="built_in">size</span>)</span></span>;</span><br></pre></td></tr></table></figure><p>创建一个 epoll instance，实际上是创建了一个  <strong>eventpoll</strong> 实例，包含了<strong>红黑树</strong>以及一个<strong>双向链表</strong>。</p><blockquote><p>可以直接查看 linux 源码：<a href="https://github.com/torvalds/linux/blob/master/fs/eventpoll.c#L181" target="_blank" rel="noopener">https://github.com/torvalds/linux/blob/master/fs/eventpoll.c#L181</a></p></blockquote><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * This structure is stored inside the "private_data" member of the file</span></span><br><span class="line"><span class="comment"> * structure and represents the main data structure for the eventpoll</span></span><br><span class="line"><span class="comment"> * interface.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">eventpoll</span> &#123;</span></span><br><span class="line">    ...</span><br><span class="line">    </span><br><span class="line"><span class="comment">/* List of ready file descriptors */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">list_head</span> <span class="title">rdllist</span>;</span></span><br><span class="line"></span><br><span class="line"><span class="comment">/* RB tree root used to store monitored fd structs */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">rb_root_cached</span> <span class="title">rbr</span>;</span></span><br><span class="line"></span><br><span class="line">...</span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure><p>这个 eventpoll 实例是直接位于内核空间的。红黑树的叶子节点都是 <strong>epitem</strong> 结构体：</p><blockquote><p>可以直接查看 linux 源码： <a href="https://github.com/torvalds/linux/blob/master/fs/eventpoll.c#L137" target="_blank" rel="noopener">https://github.com/torvalds/linux/blob/master/fs/eventpoll.c#L137</a></p></blockquote><figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">epitem</span> &#123;</span></span><br><span class="line">   ...</span><br><span class="line">   </span><br><span class="line"><span class="class"><span class="keyword">union</span> &#123;</span></span><br><span class="line">/* RB tree node links this structure to the eventpoll RB tree *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct rb_node rbn;</span></span><br><span class="line"><span class="regexp">/</span>* Used to free the <span class="class"><span class="keyword">struct</span> <span class="title">epitem</span> */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">rcu_head</span> <span class="title">rcu</span>;</span></span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line">/* List header used to link this structure to the eventpoll ready list *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct list_head rdllink;</span></span><br><span class="line"><span class="regexp"></span></span><br><span class="line"><span class="regexp">/</span>* The file descriptor information this item refers to *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct epoll_filefd ffd;</span></span><br><span class="line"><span class="regexp"></span></span><br><span class="line"><span class="regexp">/</span>* The <span class="string">"container"</span> <span class="keyword">of</span> this item *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct eventpoll *ep;</span></span><br><span class="line"><span class="regexp"></span></span><br><span class="line"><span class="regexp">/</span>* List header used to link this item to the <span class="string">"struct file"</span> items list *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct list_head fllink;</span></span><br><span class="line"><span class="regexp"></span></span><br><span class="line"><span class="regexp">/</span>* wakeup_source used <span class="keyword">when</span> EPOLLWAKEUP is set *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct wakeup_source __rcu *ws;</span></span><br><span class="line"><span class="regexp"></span></span><br><span class="line"><span class="regexp">/</span>* The structure that describe the interested events and the source fd *<span class="regexp">/</span></span><br><span class="line"><span class="regexp">struct epoll_event event;</span></span><br><span class="line"><span class="regexp"></span></span><br><span class="line"><span class="regexp">...</span></span><br><span class="line"><span class="regexp">&#125;;</span></span><br></pre></td></tr></table></figure><p>关于各项的解释，注释里已经说的比较清楚了。我们关心的应该是，当往这棵红黑树上添加、删除、修改节点的时候，我们从（用户态）程序代码中能操作的是一个 fd，即一个 socket 对应的 file descriptor，所以一个 epitem 实例与一个 socket fd 一一对应。</p><p>另外还需要注意到的是 rdllink 这个变量，这个指向了上一步创建的 evnetpoll 实例中的成员变量 rdllist，也就是那个就绪链表。<strong>这里很重要，注意留意，后面会讲到</strong>。</p><p>当然，我们还需要关注的是 event 这个变量，代表了我们针对这个 socket fd 关心的事件，比如 EPOLLIN、EPOLLOUT。</p><p>通过上述的讲解应该大致明白了，当我们使用 socket() 或者 accept() 得到一个 socket fd 时，我们添加到这棵红黑树上的是一个结构体，与这个 socket fd 一一对应。</p><p>那么修改和删除呢？</p><p>也是类似的过程，使用 ffd 变量作为红黑树比较的 key，能够快速的查找和插入。具体我们使用的是：</p><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">int</span> epoll<span class="constructor">_ctl(<span class="params">int</span> <span class="params">epfd</span>, <span class="params">int</span> <span class="params">op</span>, <span class="params">int</span> <span class="params">fd</span>, <span class="params">struct</span> <span class="params">epoll_event</span> <span class="operator">*</span><span class="params">event</span>)</span>;</span><br></pre></td></tr></table></figure><h2 id="如何触发事件"><a href="#如何触发事件" class="headerlink" title="如何触发事件"></a>如何触发事件</h2><p>上面过程已经把我们关心的 socket 添加到 epoll instance 中了，那么当某个 socket 有事件触发时，epoll 是如何感知并通知（用户态）应用程序呢？</p><p>要完整的回答这个问题，会涉及到比较多的知识。不过为了了解 epoll 的原理，有一些知识需要提前了解。</p><h3 id="内核收包路径"><a href="#内核收包路径" class="headerlink" title="内核收包路径"></a>内核收包路径</h3><blockquote><p>当一个包从网卡进来之后，是如何走到应用程序的呢？中间经过了哪些步骤呢？（本文会讲的比较简略一点）</p></blockquote><p>包从硬件网卡（NIC) 上进来之后，会触发一个<strong>中断</strong>，告诉 cpu 网卡上有包过来了，需要处理，同时通过 DMA（direct memory access) 的方式把包存放到内存的某个地方，这块内存通常称为 ring buffer，是网卡驱动程序初始化时候分配的。</p><blockquote><p><strong>中断</strong> 的原理学过微机原理的应该都知道，表示处理器接收到来自硬件或者软件的信号，提示产生了某件事情，需要处理。</p></blockquote><p>当 cpu 收到这个中断后，会调用中断处理程序，这里的中断处理程序就是网卡驱动程序，因为网络硬件设备网卡需要驱动才能工作。网卡驱动会先关闭网卡上的中断请求，表示已经知晓网卡上有包进来的事情，同时也避免在处理过程中网卡再次触发中断，干扰或者降低处理性能。驱动程序启动软中断，继续处理数据包。</p><p>然后 CPU 激活 NAPI 子系统，由 NAPI 子系统来处理由网卡放到内存的数据包。经过一些列内核代码，最终数据包来到内核协议栈。内核协议栈也就是 IP 层以及传输层。经过 IP 层之后，数据包到达传输层，内核根据数据包里面的 <code>{src_ip:src_port, dst_ip:dst_port}</code> 找到相应的 socket。</p><blockquote><p>为了性能，内核应该是有一个四元组和 socket 句柄的一一映射关系。（这里不太确定，不过原理应该是类似的）</p></blockquote><p>然后把数据包放到这个 socket 的接收队列（接收缓冲区）中，准备通知应用程序，socket 就绪。</p><h3 id="从-socket-到应用程序"><a href="#从-socket-到应用程序" class="headerlink" title="从 socket 到应用程序"></a>从 socket 到应用程序</h3><p>上面比较简略的描述了一个数据包从网卡到内核协议栈，再到 socket 的接收缓冲区的步骤，描述的比较简略，不影响对 epoll 原理的理解，这里只需要有这个概念就行。</p><p>那么当 socket 就绪后，也就是数据包被放到 socket 的接收缓冲区后，如何通知应用程序呢？这里用到的是<strong>等待队列</strong>，也就是 <strong>wait queue</strong>。关于 wait queue 的应用，在 linux 内核代码里有很多，具体可以看一下 wait queue 的定义：</p><p><a href="https://github.com/torvalds/linux/blob/master/include/linux/wait.h" target="_blank" rel="noopener">https://github.com/torvalds/linux/blob/master/include/linux/wait.h</a></p><p>当我们通过 socket() 以及 accept() 获取到一个 socket 对象时，这个 socket 对象到底有哪些东西呢？</p><blockquote><p>可以直接参考 <a href="https://github.com/torvalds/linux/blob/master/include/linux/net.h#L113" target="_blank" rel="noopener">https://github.com/torvalds/linux/blob/master/include/linux/net.h#L113</a></p></blockquote><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> *  struct socket - general BSD socket</span></span><br><span class="line"><span class="comment"> *  @state: socket state (%SS_CONNECTED, etc)</span></span><br><span class="line"><span class="comment"> *  @type: socket type (%SOCK_STREAM, etc)</span></span><br><span class="line"><span class="comment"> *  @flags: socket flags (%SOCK_NOSPACE, etc)</span></span><br><span class="line"><span class="comment"> *  @ops: protocol specific socket operations</span></span><br><span class="line"><span class="comment"> *  @file: File back pointer for gc</span></span><br><span class="line"><span class="comment"> *  @sk: internal networking protocol agnostic socket representation</span></span><br><span class="line"><span class="comment"> *  @wq: wait queue for several uses</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">socket</span> &#123;</span></span><br><span class="line">socket_statestate;</span><br><span class="line"></span><br><span class="line">shorttype;</span><br><span class="line"></span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">long</span>flags;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">file</span>*<span class="title">file</span>;</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">sock</span>*<span class="title">sk</span>;</span></span><br><span class="line"><span class="keyword">const</span> <span class="class"><span class="keyword">struct</span> <span class="title">proto_ops</span>*<span class="title">ops</span>;</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">socket_wq</span><span class="title">wq</span>;</span></span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">socket_wq</span> &#123;</span></span><br><span class="line"><span class="comment">/* Note: wait MUST be first field of socket_wq */</span></span><br><span class="line"><span class="keyword">wait_queue_head_t</span>wait;</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">fasync_struct</span>*<span class="title">fasync_list</span>;</span></span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">long</span>flags; <span class="comment">/* %SOCKWQ_ASYNC_NOSPACE, etc */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">rcu_head</span><span class="title">rcu</span>;</span></span><br><span class="line">&#125; ____cacheline_aligned_in_smp;</span><br></pre></td></tr></table></figure><p>可以看到，一个 socket 实例包含了一个 file 的指针，以及一个 socket_wq 变量。其中 socket_wq 中的 <strong>wait 表示等待队列，fasync_list 表示异步等待队列</strong>。</p><p>那么等待队列和异步等待队列中有什么呢？大致来说，<strong>等待队列和异步等待队列中存放的是关注这个 socket 上的事件的进程</strong>。区别是等待队列中的进程会处于阻塞状态，处于异步等待队列中的进程不会阻塞。</p><blockquote><p>阻塞的概念学过操作系统的应该知道，阻塞是进程的一种状态，表示一个进程正在等待某件事情的发生而暂时停止运行；另外还有运行状态以及就绪状态。</p></blockquote><p>当 socket 就绪后（接收缓冲区有数据），那么就会 wake up 等待队列中的进程，通知进程 socket 上有事件，可以开始处理了。</p><p>至此，一个数据包从网卡最终达到应用程序内部了。</p><p><strong>再简单总结一下收包以及触发的过程</strong>：</p><ul><li><strong>包从网卡进来</strong></li><li><strong>一路经过各个子系统到达内核协议栈（传输层）</strong></li><li><strong>内核根据包的 <code>{src_ip:src_port, dst_ip:dst_port}</code> 找到 socket 对象（内核维护了一份四元组和 socket 对象的一一映射表）</strong></li><li><strong>数据包被放到 socket 对象的接收缓冲区</strong></li><li><strong>内核唤醒 socket 对象上的等待队列中的进程，通知 socket 事件</strong></li><li><strong>进程唤醒，处理 socket 事件（read/write)</strong></li></ul><h2 id="epoll-的触发"><a href="#epoll-的触发" class="headerlink" title="epoll 的触发"></a>epoll 的触发</h2><p>上面其实是对内核收包以及事件触发的综合描述，涉及到 epoll 后，稍微有点差异。</p><p>上面其实提到了<strong>等待队列</strong>，每当我们创建一个 socket 后（无论是 socket()函数 还是 accept() 函数)，socket 对象中会有一个进程的等待队列，表示某个或者某些进程在等待这个 socket 上的事件。</p><p>但是当我们往 epoll 红黑树上添加一个 epitem 节点（也就是一个 socket 对象，或者说一个 fd)后，<strong>实际上还会在这个 socket 对象的 wait queue 上注册一个 callback function，当这个 socket 上有事件发生后就会调用这个 callback function</strong>。这里与上面讲到的不太一样，并不会直接 wake up 一个等待进程，需要注意一下。</p><p>简单讲就是，<strong>这个 socket 在添加到这棵 epoll 树上时，会在这个 socket 的 wait queue 里注册一个回调函数，当有事件发生的时候再调用这个回调函数（而不是唤醒进程）</strong>。</p><p>下面简单贴一下 epoll 中关于注册这个回调函数的部分代码：</p><figure class="highlight xl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * This is the callback that is used to add our wait queue to the</span></span><br><span class="line"><span class="comment"> * target file wakeup lists.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line">static void ep_ptable_queue_proc(struct file *file, wait_queue_head_t *whead,</span><br><span class="line"> poll_table *pt)</span><br><span class="line">&#123;</span><br><span class="line">struct epitem *epi = ep_item_from_epqueue(pt);</span><br><span class="line">struct eppoll_entry *pwq;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">if</span> (epi-&gt;</span>nwait &gt;= <span class="number">0</span> &amp;&amp; (pwq = kmem_cache_alloc(pwq_cache, GFP_KERNEL))) &#123;</span><br><span class="line"><span class="function"><span class="title">init_waitqueue_func_entry</span>(&amp;pwq-&gt;</span>wait, ep_poll_callback);   <span class="comment">// 注册回调函数到等待队列上</span></span><br><span class="line"><span class="function"><span class="title">pwq</span>-&gt;</span>whead = whead;</span><br><span class="line"><span class="function"><span class="title">pwq</span>-&gt;</span>base = epi;</span><br><span class="line"><span class="function"><span class="title">if</span> (epi-&gt;</span>event.events &amp; EPOLLEXCLUSIVE)</span><br><span class="line"><span class="function"><span class="title">add_wait_queue_exclusive</span>(whead, &amp;pwq-&gt;</span>wait);</span><br><span class="line"><span class="keyword">else</span></span><br><span class="line"><span class="function"><span class="title">add_wait_queue</span>(whead, &amp;pwq-&gt;</span>wait);</span><br><span class="line"><span class="function"><span class="title">list_add_tail</span>(&amp;pwq-&gt;</span><span class="function"><span class="title">llink</span>, &amp;epi-&gt;</span>pwqlist);</span><br><span class="line"><span class="function"><span class="title">epi</span>-&gt;</span>nwait++;</span><br><span class="line">&#125; <span class="keyword">else</span> &#123;</span><br><span class="line"><span class="comment">/* We have to signal that an error occurred */</span></span><br><span class="line"><span class="function"><span class="title">epi</span>-&gt;</span>nwait = -<span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>那么这个回调函数做了什么事呢？</p><p>很简单，这个回调函数会把这个 socket 添加到创建 epoll instance 时对应的 <strong>eventpoll</strong> 实例中的就绪链表上，也就是 <strong>rdllist</strong> 上，并唤醒 <strong>epoll_wait</strong>，通知 epoll 有 socket 就绪，并且已经放到了就绪链表中，然后应用层就会来遍历这个就绪链表，并拷贝到用户空间，开始后续的事件处理（read/write)。</p><p>所以这里其实就体现出与 select 的不同， epoll 把就绪的 socket 给缓存了下来，放到一个双向链表中，这样当唤醒进程后，进程就知道哪些 socket 就绪了，而 select 是进程被唤醒后只知道有 socket 就绪，但是不知道哪些 socket 就绪，所以 select 需要遍历所有的 socket。</p><p>另外，应用程序遍历这个就绪链表，由于就绪链表是位于内核空间，所以需要<strong>拷贝到用户空间，这里要注意一下，网上很多不靠谱的文章说用了共享内存，其实不是</strong>。由于这个就绪链表的数量是相对较少的，所以由内核拷贝这个就绪链表到用户空间，这个效率是较高的。</p><p>我来来直接看一下 <strong>epoll_wait</strong> 做了什么事？<strong>epoll_wait</strong> 最终会调用到 <strong>ep_send_events_proc</strong> 这个函数，从函数名字也知道，这个函数是用来把就绪链表中的内容复制到用户空间，向应用程序通知事件。</p><figure class="highlight xl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br></pre></td><td class="code"><pre><span class="line">static __poll_t ep_send_events_proc(struct eventpoll *ep, struct list_head *head,</span><br><span class="line">       void *priv)</span><br><span class="line">&#123;</span><br><span class="line">struct ep_send_events_data *esed = priv;</span><br><span class="line">__poll_t revents;</span><br><span class="line">struct epitem *epi, *tmp;</span><br><span class="line"><span class="function"><span class="title">struct</span> epoll_event __user *uevent = esed-&gt;</span>events;   # 这个就是在用户空间分配的一段内存指针，该函数会把 rdllist 拷贝到这块内存</span><br><span class="line">struct wakeup_source *ws;</span><br><span class="line">poll_table pt;</span><br><span class="line"></span><br><span class="line">init_poll_funcptr(&amp;pt, NULL);</span><br><span class="line"><span class="function"><span class="title">esed</span>-&gt;</span>res = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * We can loop without lock because we are passed a task private list.</span></span><br><span class="line"><span class="comment"> * Items cannot vanish during the loop because ep_scan_ready_list() is</span></span><br><span class="line"><span class="comment"> * holding "mtx" during this call.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="title">lockdep_assert_held</span>(&amp;ep-&gt;</span>mtx);</span><br><span class="line"></span><br><span class="line">list_for_each_entry_safe(epi, tmp, head, rdllink) &#123;</span><br><span class="line"><span class="function"><span class="title">if</span> (esed-&gt;</span><span class="function"><span class="title">res</span> &gt;= esed-&gt;</span>maxevents)</span><br><span class="line">break;</span><br><span class="line"></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Activate ep-&gt;ws before deactivating epi-&gt;ws to prevent</span></span><br><span class="line"><span class="comment"> * triggering auto-suspend here (in case we reactive epi-&gt;ws</span></span><br><span class="line"><span class="comment"> * below).</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * This could be rearranged to delay the deactivation of epi-&gt;ws</span></span><br><span class="line"><span class="comment"> * instead, but then epi-&gt;ws would temporarily be out of sync</span></span><br><span class="line"><span class="comment"> * with ep_is_linked().</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line">ws = ep_wakeup_source(epi);</span><br><span class="line"><span class="keyword">if</span> (ws) &#123;</span><br><span class="line"><span class="function"><span class="title">if</span> (ws-&gt;</span>active)</span><br><span class="line">__<span class="function"><span class="title">pm_stay_awake</span>(ep-&gt;</span>ws);</span><br><span class="line">__pm_relax(ws);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">list_del_init</span>(&amp;epi-&gt;</span>rdllink);</span><br><span class="line"></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * If the event mask intersect the caller-requested one,</span></span><br><span class="line"><span class="comment"> * deliver the event to userspace. Again, ep_scan_ready_list()</span></span><br><span class="line"><span class="comment"> * is holding ep-&gt;mtx, so no operations coming from userspace</span></span><br><span class="line"><span class="comment"> * can change the item.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line">revents = ep_item_poll(epi, &amp;pt, <span class="number">1</span>);</span><br><span class="line"><span class="keyword">if</span> (!revents)</span><br><span class="line">continue;</span><br><span class="line">       </span><br><span class="line">       # 拷贝 rdllist 到 用户空间提供的一个内存指针</span><br><span class="line"><span class="function"><span class="title">if</span> (__put_user(revents, &amp;uevent-&gt;</span>events) ||</span><br><span class="line">    __<span class="function"><span class="title">put_user</span>(epi-&gt;</span><span class="function"><span class="title">event</span>.<span class="keyword">data</span>, &amp;uevent-&gt;</span><span class="keyword">data</span>)) &#123;</span><br><span class="line"><span class="function"><span class="title">list_add</span>(&amp;epi-&gt;</span>rdllink, head);</span><br><span class="line">ep_pm_stay_awake(epi);</span><br><span class="line"><span class="function"><span class="title">if</span> (!esed-&gt;</span>res)</span><br><span class="line"><span class="function"><span class="title">esed</span>-&gt;</span>res = -EFAULT;</span><br><span class="line">return <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"><span class="function"><span class="title">esed</span>-&gt;</span>res++;</span><br><span class="line">uevent++;</span><br><span class="line"><span class="function"><span class="title">if</span> (epi-&gt;</span>event.events &amp; EPOLLONESHOT)</span><br><span class="line"><span class="function"><span class="title">epi</span>-&gt;</span>event.events &amp;= EP_PRIVATE_BITS;</span><br><span class="line"><span class="function"><span class="title">else</span> <span class="keyword">if</span> (!(epi-&gt;</span>event.events &amp; EPOLLET)) &#123;</span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * If this file has been added with Level</span></span><br><span class="line"><span class="comment"> * Trigger mode, we need to insert back inside</span></span><br><span class="line"><span class="comment"> * the ready list, so that the next call to</span></span><br><span class="line"><span class="comment"> * epoll_wait() will check again the events</span></span><br><span class="line"><span class="comment"> * availability. At this point, no one can insert</span></span><br><span class="line"><span class="comment"> * into ep-&gt;rdllist besides us. The epoll_ctl()</span></span><br><span class="line"><span class="comment"> * callers are locked out by</span></span><br><span class="line"><span class="comment"> * ep_scan_ready_list() holding "mtx" and the</span></span><br><span class="line"><span class="comment"> * poll callback will queue them in ep-&gt;ovflist.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="title">list_add_tail</span>(&amp;epi-&gt;</span><span class="function"><span class="title">rdllink</span>, &amp;ep-&gt;</span>rdllist);</span><br><span class="line">ep_pm_stay_awake(epi);</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">return <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>上面可以看到，这里确确实实是从内核复制 rdllist 到用户空间，非共享内存。应用程序调用 <strong>epoll_wait</strong> 返回后，开始遍历拷贝回来的内容，处理 socket 事件。</p><p><strong>至此，从注册一个 file descriptor(socket fd) 到 epoll 红黑树，到这个 socket 上有数据包从网卡进来，再到如何触发 epoll，再到应用程序的用户空间，由应用程序开始 read/write 事件的整个过程就理顺了</strong>。不知道大家有没有理解了？</p><h2 id="accept-事件"><a href="#accept-事件" class="headerlink" title="accept 事件"></a>accept 事件</h2><p>accept 事件属于可读事件的一种，这里单独提出来讲一下，是因为编程的时候针对 accept 有一些点需要注意，这里先大致讲一下，后面会有另外的博文展开讲。</p><p>当 socket 有可读事件达到后，epoll_wait 获取到就绪的 socket，应用程序开始处理可读事件，如果这个 socket 的 fd 等于 listen() 的 fd，说明有新连接到达，(server)开始调用 accept() 处理连接。</p><p>accept() 返回的新的 socket 对象，对应与 client 的一个新的连接，应用程序需要把这个新的 socket 对象注册到 epoll 红黑树上，并且添加关心的事件（EPOLLIN/EPOLLOUT…)，然后开始 epoll 循环。</p><p>另外还有一点要注意的，<strong>accept 的惊群效应</strong>。</p><p>先解释一下什么是惊群，<strong>如果一个 socket 上有多个进程在同时等待事件，当事件触发后，内核可能会唤醒多个或者所有在等待的进程，然而只会有一个进程成功获取该事件，其他进程都失败，这种情况就叫惊群，会一定程度浪费 cpu，影响性能</strong>。如果用一个例子来解释的话就是，有一个鸡群，如果往这个鸡群里丢一粒米，那么会造成所有鸡（或者大多数鸡）一起来争抢这粒米，但是最终只会有一只鸡能抢到这粒米。</p><p>对于 accept() 来说，通常我们会使用多线程或者多进程的方式来监听同一个 listen fd，此时，就很可能发生惊群效应。</p><p>关于惊群效应，此处只简单提一下概念，后面开另外的博文深入探讨下惊群效应以及解决方案。</p><h1 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h1><p>上面深入的分析了 epoll 的底层实现原理，现在回到文章开头提到的与 select/poll 对比的几个优点，是不是能理解了呢？</p><p>简单总结一下:</p><ol><li><strong>epoll 在内核开辟了一块缓存，用来创建 eventpoll 对象，并返回一个 file descriptor 代表 epoll instance</strong></li><li><strong>这个 epoll instance 中创建了一颗红黑树以及一个就绪的双向链表（当然还有其他的成员）</strong></li><li><strong>红黑树用来缓存所有的 socket，支持 O(log(n)) 的插入和查找，减少后续与用户空间的交互</strong></li><li><strong>socket 就绪后，会回调一个回调函数（添加到 epoll instance 上时注册到 socket 的）</strong></li><li><strong>这个回调函数会把这个 socket 放到就绪链表，并唤醒 epoll_wait</strong></li><li><strong>应用程序拷贝就绪 socket 到用户空间，开始遍历处理就绪的 socket</strong></li><li><strong>如果有新的 socket，再添加到 epoll 红黑树上，重复这个过程</strong></li></ol><p>到这里应该能比较透彻的理解 epoll 的原理了，接下来会继续写几篇关于 epoll 的博文（先把坑埋下）:</p><ul><li><a href="https://rebootcat.com/2020/09/26/epoll_examples/">epoll 入门例子 tcp server/client</a></li><li>epoll 惊群(todo)</li><li>epoll 源码分析(todo)</li><li>内核收发包路径(todo)</li></ul><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#x69;&#110;&#117;&#x78;&#99;&#x6f;&#100;&#x65;&#50;&#110;&#x69;&#107;&#105;&#x40;&#x67;&#109;&#97;&#105;&#x6c;&#x2e;&#99;&#x6f;&#109;">&#x6c;&#x69;&#110;&#117;&#x78;&#99;&#x6f;&#100;&#x65;&#50;&#110;&#x69;&#107;&#105;&#x40;&#x67;&#109;&#97;&#105;&#x6c;&#x2e;&#99;&#x6f;&#109;</a></p></li></ul><p>2020-09-26 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;Epoll-的出现&quot;&gt;&lt;a href=&quot;#Epoll-的出现&quot; class=&quot;headerlink&quot; title=&quot;Epoll 的出现&quot;&gt;&lt;/a&gt;Epoll 的出现&lt;/h1&gt;&lt;p&gt;想必能搜到这篇文章的，应该对 select/poll 有一些了解和认识，一般说 epoll 都会与 select/poll 进行一些对比，select、poll 和 epoll 都是一种 IO 多路复用机制。&lt;/p&gt;
&lt;h2 id=&quot;select-的问题&quot;&gt;&lt;a href=&quot;#select-的问题&quot; class=&quot;headerlink&quot; title=&quot;select 的问题&quot;&gt;&lt;/a&gt;select 的问题&lt;/h2&gt;&lt;p&gt;select 的问题在于描述符的限制，能监控的文件描述符最大为 FD_SETSIZE，对于连接数很多的场景就无法满足；&lt;/p&gt;
&lt;p&gt;另外select 还有一个问题是，每次调用 select 都需要从用户空间把描述符集合拷贝到内核空间，当描述符集合变大之后，用户空间和内核空间的内存拷贝会导致效率低下；&lt;/p&gt;
&lt;p&gt;另外每次调用 select 都需要在内核线性遍历文件描述符的集合，当描述符增多，效率低下。&lt;/p&gt;
&lt;h2 id=&quot;poll-的问题&quot;&gt;&lt;a href=&quot;#poll-的问题&quot; class=&quot;headerlink&quot; title=&quot;poll 的问题&quot;&gt;&lt;/a&gt;poll 的问题&lt;/h2&gt;&lt;p&gt;由于 select 存在上面的问题，于是 poll 被提了出来，它能解决 select 对文件描述符数量有限制的问题，但是依然不能解决线性遍历以及用户空间和内核空间的低效数据拷贝问题。&lt;/p&gt;
&lt;h2 id=&quot;epoll-是什么&quot;&gt;&lt;a href=&quot;#epoll-是什么&quot; class=&quot;headerlink&quot; title=&quot;epoll 是什么&quot;&gt;&lt;/a&gt;epoll 是什么&lt;/h2&gt;&lt;p&gt;select/poll 在互联网早期应该是没什么问题的，因为没有很多的互联网服务，也没有很多的客户端，但是随着互联网的发展，&lt;a href=&quot;http://www.kegel.com/c10k.html#related&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;C10K&lt;/a&gt; 等问题的出现，select/poll 已经不能满足要求了，这个时候 epoll 上场了。&lt;/p&gt;
&lt;p&gt;epoll 是 linux 内核 2.6 之后支持的，epoll 同 select/poll 一样，也是 IO 多路复用的一种机制，不过它避免了 select/poll 的缺点。下面详细讲解一下 epoll 反应堆的原理。&lt;/p&gt;
&lt;h1 id=&quot;Epoll-反应堆&quot;&gt;&lt;a href=&quot;#Epoll-反应堆&quot; class=&quot;headerlink&quot; title=&quot;Epoll 反应堆&quot;&gt;&lt;/a&gt;Epoll 反应堆&lt;/h1&gt;&lt;h2 id=&quot;epoll-原理&quot;&gt;&lt;a href=&quot;#epoll-原理&quot; class=&quot;headerlink&quot; title=&quot;epoll 原理&quot;&gt;&lt;/a&gt;epoll 原理&lt;/h2&gt;&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;要完整描述 epoll 的原理，需要涉及到内核、网卡、中断、软中断、协议栈、套接字等知识，本文尽量从比较全面的角度来分析 epoll 的原理&lt;/strong&gt;。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;上面其实讨论了 select/poll 几个缺点，针对这几个缺点，就需要解决以下几件事：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;如何突破文件描述符数量的限制&lt;/li&gt;
&lt;li&gt;如何避免用户态和内核态对文件描述符集合的拷贝&lt;/li&gt;
&lt;li&gt;socket 就绪后，如何避免线性遍历文件描述符集合&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
    
      <category term="linux" scheme="https://rebootcat.com/categories/linux/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="epoll" scheme="https://rebootcat.com/tags/epoll/"/>
    
      <category term="select" scheme="https://rebootcat.com/tags/select/"/>
    
      <category term="poll" scheme="https://rebootcat.com/tags/poll/"/>
    
      <category term="rbtree" scheme="https://rebootcat.com/tags/rbtree/"/>
    
      <category term="waitqueue" scheme="https://rebootcat.com/tags/waitqueue/"/>
    
      <category term="socket" scheme="https://rebootcat.com/tags/socket/"/>
    
      <category term="fd" scheme="https://rebootcat.com/tags/fd/"/>
    
  </entry>
  
  <entry>
    <title>使用 mkdocs 搭建个人 wiki 站点</title>
    <link href="https://rebootcat.com/2020/09/20/wiki/"/>
    <id>https://rebootcat.com/2020/09/20/wiki/</id>
    <published>2020-09-20T15:50:58.000Z</published>
    <updated>2020-09-20T16:24:47.349Z</updated>
    
    <content type="html"><![CDATA[<h1 id="why-wiki"><a href="#why-wiki" class="headerlink" title="why wiki"></a>why wiki</h1><p>博客通常是用来记录一些完整的文章，每篇文章有一个主题。但是我想把平日里的一些笔记也记录到我的博客里，但笔记是零散的，随时的，不是完整的一个主题。所以打算构建一个 wiki 页面，专门用来存放我的笔记，wiki 页面类似于 <a href="https://zh.wikipedia.org/wiki/Wikipedia:%E9%A6%96%E9%A1%B5" target="_blank" rel="noopener">维基百科</a>的形式。</p><p>我的博客采用的是 hexo 构建的，如果打算 DIY 一个类似于 <strong>维基百科</strong> 的 wiki 页面的话，对于我来说，也许有点难度，毕竟我只会写简单的网页。那么有没有现成的方案或者替代的方案呢？</p><p>答案是有的，那就是 <strong>mkdocs</strong>。</p><h1 id="mkdocs-使用"><a href="#mkdocs-使用" class="headerlink" title="mkdocs 使用"></a>mkdocs 使用</h1><p>什么是 Mkdocs 呢？</p><blockquote><p>MkDocs is a fast, simple and downright gorgeous static site generator that’s geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file. Start by reading the introduction below, then check the User Guide for more info.</p></blockquote><p>mkdocs 是一个用 python 编写的静态站点生成工具，主要是用来编写项目文档，文档使用 Markdown 编写，只需要配合一个 YAML 配置文件，就能快速生成一个站点。</p><p>毫无疑问，对于我来说，它有以下几个优点：</p><ul><li>使用 python 编写（说明有 DIY 的可能）</li><li>源文件使用 Markdown 编写</li><li>只需要一个 Yaml 文件，非常简单了</li><li>主题可选（当然目前来说不是特别多）</li></ul><p>可以先看一下 <a href="https://rebootcat.com/wiki/">我的wiki</a>.</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/wiki/1.png" alt=""></p><a id="more"></a><h2 id="快速开始"><a href="#快速开始" class="headerlink" title="快速开始"></a>快速开始</h2><p>可以参考官方文档：<a href="https://www.mkdocs.org/" target="_blank" rel="noopener">mkdocs.org</a>，或者直接往下看：</p><p>首先安装 mkdocs:</p><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ pip <span class="keyword">install</span> mkdocs</span><br></pre></td></tr></table></figure><p>安装完成之后直接生成一个项目：</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ mkdocs new mysite                                                                    [23:33:49]</span><br><span class="line"><span class="builtin-name">INFO</span>    -  Creating project directory: mysite</span><br><span class="line"><span class="builtin-name">INFO</span>    -  Writing<span class="built_in"> config </span>file: mysite/mkdocs.yml</span><br><span class="line"><span class="builtin-name">INFO</span>    -  Writing initial docs: mysite/docs/index.md</span><br></pre></td></tr></table></figure><p>看看都生成了啥：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ cd mysite</span><br><span class="line">$ tree                                                                                    [<span class="number">23</span>:<span class="number">34</span>:<span class="number">59</span>]</span><br><span class="line">.</span><br><span class="line">├── docs</span><br><span class="line">│   └── index.md</span><br><span class="line">└── mkdocs.yml</span><br><span class="line"></span><br><span class="line"><span class="number">1</span> directory, <span class="number">2</span> files</span><br></pre></td></tr></table></figure><p>默认生成了一个 yml 配置文件以及一个 默认的 markdown 文件。</p><p>看看 mkdocs 支持哪些命令：</p><figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">$ mkdocs -h                                                                               [<span class="number">23</span>:<span class="number">36</span>:<span class="number">21</span>]</span><br><span class="line">Usage: mkdocs [OPTIONS] COMMAND [ARGS]...</span><br><span class="line"></span><br><span class="line">  MkDocs - Project documentation with Markdown.</span><br><span class="line"></span><br><span class="line">Options:</span><br><span class="line">  -V, --version  Show the version and <span class="keyword">exit</span>.</span><br><span class="line">  -q, --quiet    Silence warnings</span><br><span class="line">  -v, --verbose  Enable verbose output</span><br><span class="line">  -h, --help     Show this message and <span class="keyword">exit</span>.</span><br><span class="line"></span><br><span class="line">Commands:</span><br><span class="line">  build      Build the MkDocs documentation</span><br><span class="line">  gh-deploy  Deploy your documentation to GitHub Pages</span><br><span class="line">  new        Create a new MkDocs project</span><br><span class="line">  serve      Run the builtin development server</span><br></pre></td></tr></table></figure><p>构建站点：</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$ </span>mkdocs build</span><br></pre></td></tr></table></figure><p>然后生成了一个 <code>site</code> 目录：</p><figure class="highlight mipsasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br></pre></td><td class="code"><pre><span class="line">$ tree                                                                                    [<span class="number">23</span>:<span class="number">37</span>:<span class="number">23</span>]</span><br><span class="line">.</span><br><span class="line">├── docs</span><br><span class="line">│   └── index.md</span><br><span class="line">├── mkdocs.yml</span><br><span class="line">└── site</span><br><span class="line">    ├── <span class="number">404</span>.html</span><br><span class="line">    ├── css</span><br><span class="line">    │   ├── <span class="keyword">base.css</span></span><br><span class="line"><span class="keyword"> </span>   │   ├── <span class="keyword">bootstrap.min.css</span></span><br><span class="line"><span class="keyword"> </span>   │   └── font-awesome.min.css</span><br><span class="line">    ├── fonts</span><br><span class="line">    │   ├── fontawesome-webfont.eot</span><br><span class="line">    │   ├── fontawesome-webfont.svg</span><br><span class="line">    │   ├── fontawesome-webfont.ttf</span><br><span class="line">    │   ├── fontawesome-webfont.woff</span><br><span class="line">    │   ├── fontawesome-webfont.woff2</span><br><span class="line">    │   ├── glyphicons-halflings-regular.eot</span><br><span class="line">    │   ├── glyphicons-halflings-regular.svg</span><br><span class="line">    │   ├── glyphicons-halflings-regular.ttf</span><br><span class="line">    │   ├── glyphicons-halflings-regular.woff</span><br><span class="line">    │   └── glyphicons-halflings-regular.woff2</span><br><span class="line">    ├── img</span><br><span class="line">    │   ├── favicon.ico</span><br><span class="line">    │   └── grid.png</span><br><span class="line">    ├── index.html</span><br><span class="line">    ├── <span class="keyword">js</span></span><br><span class="line"><span class="keyword"> </span>   │   ├── <span class="keyword">base.js</span></span><br><span class="line"><span class="keyword"> </span>   │   ├── <span class="keyword">bootstrap.min.js</span></span><br><span class="line"><span class="keyword"> </span>   │   └── <span class="keyword">jquery-1.10.2.min.js</span></span><br><span class="line"><span class="keyword"> </span>   ├── search</span><br><span class="line">    │   ├── lunr.<span class="keyword">js</span></span><br><span class="line"><span class="keyword"> </span>   │   ├── main.<span class="keyword">js</span></span><br><span class="line"><span class="keyword"> </span>   │   ├── search_index.<span class="keyword">json</span></span><br><span class="line"><span class="keyword"> </span>   │   └── worker.<span class="keyword">js</span></span><br><span class="line"><span class="keyword"> </span>   ├── sitemap.xml</span><br><span class="line">    └── sitemap.xml.gz</span><br><span class="line"></span><br><span class="line"><span class="number">7</span> <span class="keyword">directories, </span><span class="number">28</span> files</span><br></pre></td></tr></table></figure><p>可以看到 site 目录下就是站点的源码了，那么本地测试一下：</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$ </span>mkdocs serve</span><br></pre></td></tr></table></figure><p>然后访问 <code>http://127.0.0.1:8000</code>，能看到默认的站点了：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/wiki/2.png" alt=""></p><p><strong>是不是超级超级简单</strong>？</p><p>那么这个是 mkdocs 最简单的使用，接下来分享下我的使用，经过了一些定制化，包括主题的选择，域名的绑定，站点的发布等。</p><h1 id="使用-github-pages-发布-wiki"><a href="#使用-github-pages-发布-wiki" class="headerlink" title="使用 github pages 发布 wiki"></a>使用 github pages 发布 wiki</h1><p>我的博客使用了 github pages 进行托管（目前不是，目前已经迁移到香港虚拟空间），但是如何把上面 mkdocs 生成的站点源码和博客源码放到一起呢？</p><p>有很多方法，比如可以手动把 wiki 站点源码放到博客根目录下；</p><p>但其实 <strong>github pages 是可以支持多个站点的</strong>，不知道有没有同学还不知道？</p><p>简单来说，使用一个 github 账号，能创建一个 <strong>用户站点</strong>，格式为 <code>&lt;user&gt;.github.io</code>，比如我的博客源码仓库： <a href="https://github.com/smaugx/smaugx.github.io" target="_blank" rel="noopener">smaugx.github.io</a>;</p><p>但是除了一个用户站点之外，还能创建任意多个 <strong>普通站点</strong>，仓库名字任意，没有要求。</p><p><strong>也就是说一个 github 账户其实是可以创建多个博客站点的</strong>。</p><p>关于如何创建一个普通站点，可以参考 github 官方文档：<a href="https://docs.github.com/cn/github/working-with-github-pages/creating-a-github-pages-site" target="_blank" rel="noopener">创建 GitHub Pages 站点</a>.</p><p>或者往下看。</p><h2 id="wiki-仓库设置"><a href="#wiki-仓库设置" class="headerlink" title="wiki 仓库设置"></a>wiki 仓库设置</h2><p>这里以我的 wiki 为例： <a href="https://github.com/smaugx/wiki" target="_blank" rel="noopener">https://github.com/smaugx/wiki</a>，站点效果可以直接查看我的 wiki: <a href="https://rebootcat.com/wiki">https://rebootcat.com/wiki</a>。</p><p>1 在 github 上创建一个仓库，命名为 wiki 或者其他的任意名字</p><p>2 克隆我的项目： <code>git clone https://github.com/smaugx/wiki.git</code></p><p>3 更改仓库 remote-url 为你刚创建的 wiki 的 github url</p><figure class="highlight stata"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">cd</span> wiki</span><br><span class="line">git remote <span class="keyword">rm</span> origin</span><br><span class="line">git remote add origin  https:<span class="comment">//github.com/yourname/your-wiki.git</span></span><br></pre></td></tr></table></figure><p>上面改成你自己的 wiki 地址（或者使用 ssh 的方式）</p><p>4 推送本地仓库 wiki 到远程 wiki</p><figure class="highlight maxima"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git <span class="built_in">push</span> -u <span class="built_in">origin</span> master</span><br></pre></td></tr></table></figure><p>至此，你的 github 上应该有一个和我的 wiki 仓库一样的仓库了。</p><p>接下来讲一下怎么设置仓库。</p><p>5 首先去到刚创建好的 wiki 仓库 <code>https://github.com/yourname/your-wiki</code></p><p>6 点击设置，往下拉到 <strong>GiHub Pages</strong> 配置项，选择 <strong>master</strong> 分支，选择 <strong>/docs</strong> 目录，然后点击 <strong>save</strong> 保存</p><p>7 上面一部之后，再次回到 <strong>Github Pages</strong> 配置项，找到下面的 <strong>Custom domain</strong>，填入你的域名或者 url 地址，比如我直接写了： <code>http://rebootcat.com/wiki</code></p><p>8 不出意外，你就能正常访问了。</p><blockquote><p>上面的前提当然是你已经有了个人博客，也就是已经有了一个命名为 <code>&lt;user&gt;.github.io</code> 的仓库了，不然是不会成功了，你要先创建一个这样的仓库。</p></blockquote><h2 id="编写wiki，更新-wiki"><a href="#编写wiki，更新-wiki" class="headerlink" title="编写wiki，更新 wiki"></a>编写wiki，更新 wiki</h2><p>上面如果顺利的话，你能看到和我的 wiki 一样的内容：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/wiki/1.png" alt=""></p><p>那么如何编写你自己的 wiki 文章呢？</p><p>我们回到本地的 wiki 仓库：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> wiki</span><br></pre></td></tr></table></figure><p>注意，我的文档都放在了 <strong>source</strong> 目录下：</p><figure class="highlight dos"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ ls source</span><br><span class="line">git.<span class="built_in">md</span>    index.<span class="built_in">md</span>  other.<span class="built_in">md</span>  python.<span class="built_in">md</span> rsync.<span class="built_in">md</span></span><br></pre></td></tr></table></figure><p>所以你只需要删除我的 Markdown文档，把你的 Markdown 文档放到该目录，然后执行：</p><figure class="highlight clean"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br></pre></td><td class="code"><pre><span class="line">$ python run.py</span><br><span class="line">warning: found not support file type:.DS_Store</span><br><span class="line">############### begin dump mkdocs.yml ###############</span><br><span class="line">copyright: <span class="string">"Copyright \xA9 2020-2020 smaug"</span></span><br><span class="line">docs_dir: source</span><br><span class="line">extra:</span><br><span class="line">  article_nav_bottom: true</span><br><span class="line">  history_buttons: true</span><br><span class="line">  version: v1<span class="number">.0</span><span class="number">.4</span></span><br><span class="line">markdown_extensions:</span><br><span class="line">- admonition</span><br><span class="line">nav:</span><br><span class="line">- Home: index.md</span><br><span class="line">- python: python.md</span><br><span class="line">- rsync: rsync.md</span><br><span class="line">- git: git.md</span><br><span class="line">- <span class="string">"\u5176\u4ED6"</span>: other.md</span><br><span class="line">repo_url: https:<span class="comment">//github.com/smaugx/wiki</span></span><br><span class="line">site_author: smaugx</span><br><span class="line">site_description: <span class="string">"My Wiki | \u6797\u5915\u6C34\u5171"</span></span><br><span class="line">site_dir: docs</span><br><span class="line">site_name: <span class="string">"My Wiki | \u6797\u5915\u6C34\u5171"</span></span><br><span class="line">site_url: http:<span class="comment">//rebootcat.com/wiki</span></span><br><span class="line">theme:</span><br><span class="line">  custom_dir: mkdocs_windmill</span><br><span class="line">  include_search_page: true</span><br><span class="line">  name: null</span><br><span class="line">  search_index_only: true</span><br><span class="line">  static_templates:</span><br><span class="line">  - <span class="number">404.</span>html</span><br><span class="line"></span><br><span class="line">############### update mkdocs.yml done ###############</span><br><span class="line"></span><br><span class="line">############### begin mkdocs build ###############</span><br><span class="line">INFO    -  Cleaning site directory</span><br><span class="line">INFO    -  Building documentation to directory: /Users/smaug/centos7/SmaugDemo/wiki/docs</span><br><span class="line">INFO    -  Documentation built <span class="keyword">in</span> <span class="number">0.18</span> seconds</span><br><span class="line">############### mkdocs build done <span class="keyword">in</span> dir:docs ###############</span><br><span class="line"></span><br><span class="line">############### begin git push:git add --all . &amp;&amp; git commit -m <span class="string">"update mkdocs site"</span> &amp;&amp; git push ###############</span><br><span class="line">Counting objects: <span class="number">5</span>, done.</span><br><span class="line">Delta compression using up to <span class="number">4</span> threads.</span><br><span class="line">Compressing objects: <span class="number">100</span>% (<span class="number">5</span>/<span class="number">5</span>), done.</span><br><span class="line">Writing objects: <span class="number">100</span>% (<span class="number">5</span>/<span class="number">5</span>), <span class="number">443</span> bytes | <span class="number">443.00</span> KiB/s, done.</span><br><span class="line">Total <span class="number">5</span> (delta <span class="number">4</span>), reused <span class="number">0</span> (delta <span class="number">0</span>)</span><br><span class="line">remote: Resolving deltas: <span class="number">100</span>% (<span class="number">4</span>/<span class="number">4</span>), completed <span class="keyword">with</span> <span class="number">4</span> local objects.</span><br><span class="line">To github.com:smaugx/wiki.git</span><br><span class="line">   ba3b15e.<span class="number">.4131</span>b86  master -&gt; master</span><br><span class="line">[master <span class="number">4131</span>b86] update mkdocs site</span><br><span class="line"> <span class="number">2</span> files changed, <span class="number">1</span> insertion(+), <span class="number">1</span> deletion(-)</span><br><span class="line">############### git push done ###############</span><br></pre></td></tr></table></figure><p>这个脚本的功能是根据 <strong>source</strong> 目录下的 Markdown 文档，更新 yaml 站点配置文件，然后生成站点源码，然后推送站点源码到 github 上。</p><p>如果执行出错，可以自行调试一下，一般问题不大。</p><h1 id="博客首页引导栏添加-维基栏"><a href="#博客首页引导栏添加-维基栏" class="headerlink" title="博客首页引导栏添加 维基栏"></a>博客首页引导栏添加 <code>维基</code>栏</h1><p>这个过程就省略了。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/wiki/3.png" alt=""></p><h1 id="The-End"><a href="#The-End" class="headerlink" title="The End"></a>The End</h1><p>wiki 站点搭建完毕，</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#105;&#110;&#117;&#x78;&#x63;&#x6f;&#100;&#x65;&#x32;&#x6e;&#105;&#107;&#x69;&#x40;&#x67;&#x6d;&#97;&#x69;&#x6c;&#x2e;&#99;&#111;&#109;">&#x6c;&#105;&#110;&#117;&#x78;&#x63;&#x6f;&#100;&#x65;&#x32;&#x6e;&#105;&#107;&#x69;&#x40;&#x67;&#x6d;&#97;&#x69;&#x6c;&#x2e;&#99;&#111;&#109;</a></p></li></ul><p>2020-09-20 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;why-wiki&quot;&gt;&lt;a href=&quot;#why-wiki&quot; class=&quot;headerlink&quot; title=&quot;why wiki&quot;&gt;&lt;/a&gt;why wiki&lt;/h1&gt;&lt;p&gt;博客通常是用来记录一些完整的文章，每篇文章有一个主题。但是我想把平日里的一些笔记也记录到我的博客里，但笔记是零散的，随时的，不是完整的一个主题。所以打算构建一个 wiki 页面，专门用来存放我的笔记，wiki 页面类似于 &lt;a href=&quot;https://zh.wikipedia.org/wiki/Wikipedia:%E9%A6%96%E9%A1%B5&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;维基百科&lt;/a&gt;的形式。&lt;/p&gt;
&lt;p&gt;我的博客采用的是 hexo 构建的，如果打算 DIY 一个类似于 &lt;strong&gt;维基百科&lt;/strong&gt; 的 wiki 页面的话，对于我来说，也许有点难度，毕竟我只会写简单的网页。那么有没有现成的方案或者替代的方案呢？&lt;/p&gt;
&lt;p&gt;答案是有的，那就是 &lt;strong&gt;mkdocs&lt;/strong&gt;。&lt;/p&gt;
&lt;h1 id=&quot;mkdocs-使用&quot;&gt;&lt;a href=&quot;#mkdocs-使用&quot; class=&quot;headerlink&quot; title=&quot;mkdocs 使用&quot;&gt;&lt;/a&gt;mkdocs 使用&lt;/h1&gt;&lt;p&gt;什么是 Mkdocs 呢？&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;MkDocs is a fast, simple and downright gorgeous static site generator that’s geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file. Start by reading the introduction below, then check the User Guide for more info.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;mkdocs 是一个用 python 编写的静态站点生成工具，主要是用来编写项目文档，文档使用 Markdown 编写，只需要配合一个 YAML 配置文件，就能快速生成一个站点。&lt;/p&gt;
&lt;p&gt;毫无疑问，对于我来说，它有以下几个优点：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;使用 python 编写（说明有 DIY 的可能）&lt;/li&gt;
&lt;li&gt;源文件使用 Markdown 编写&lt;/li&gt;
&lt;li&gt;只需要一个 Yaml 文件，非常简单了&lt;/li&gt;
&lt;li&gt;主题可选（当然目前来说不是特别多）&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;可以先看一下 &lt;a href=&quot;https://rebootcat.com/wiki/&quot;&gt;我的wiki&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/wiki/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
    
    </summary>
    
    
      <category term="blog" scheme="https://rebootcat.com/categories/blog/"/>
    
    
      <category term="hexo" scheme="https://rebootcat.com/tags/hexo/"/>
    
      <category term="blog" scheme="https://rebootcat.com/tags/blog/"/>
    
      <category term="github" scheme="https://rebootcat.com/tags/github/"/>
    
      <category term="wiki" scheme="https://rebootcat.com/tags/wiki/"/>
    
      <category term="mkdocs" scheme="https://rebootcat.com/tags/mkdocs/"/>
    
  </entry>
  
  <entry>
    <title>Hexo 配置 Cloudflare 免费 CDN</title>
    <link href="https://rebootcat.com/2020/09/20/cloudflare/"/>
    <id>https://rebootcat.com/2020/09/20/cloudflare/</id>
    <published>2020-09-20T05:23:58.000Z</published>
    <updated>2020-09-20T05:25:54.734Z</updated>
    
    <content type="html"><![CDATA[<h1 id="CDN"><a href="#CDN" class="headerlink" title="CDN"></a>CDN</h1><p>关于 CDN 是什么，我想应该不用做过多的介绍，毕竟现在是一个 “云” 的时代，你至少也听说过 阿里云 或者 腾讯云 吧，当然其中就包括 CDN 业务。</p><p>CDN 的作用有很多，比如可以用来加速网站的访问，可以用来防护网站等。本篇文章讨论的就是使用 cloudflare 作为 CDN 来加速博客网站，并让博客开启 https，提升博客安全等级。 </p><ul><li>加速网站访问</li><li>开启https</li></ul><p>选择什么 CDN 呢？</p><p>选择 CDN，对于个人博客来说，主要考虑的还是访问速度以及价格，当然也有免费的 CDN。<a href="https://cloudflare.com" target="_blank" rel="noopener">Cloudflare</a> 就是一家提供免费 CDN 的公司，也是在 CDN 领域比较知名的公司。</p><p>话不多说，关于 cloudflare 的配置网上可以搜到很多文章，这里我就简单记录一下。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cloudflare/0.png" alt=""></p><a id="more"></a><h1 id="Cloudflare-配置"><a href="#Cloudflare-配置" class="headerlink" title="Cloudflare 配置"></a>Cloudflare 配置</h1><p>由于我的博客 <a href="https://rebootcat.com">rebootcat.com</a> 已经迁移到香港的虚拟主机了，并且开启了 https 访问，详见博文： <a href="https://rebootcat.com/2020/09/20/virtual_space_blog/">迁移博客到香港虚拟空间</a>，故我以我另外的一个博客 <a href="http://loveyxq.online/" target="_blank" rel="noopener">loveyxq.online</a> 为例说明。</p><p><a href="http://loveyxq.online/" target="_blank" rel="noopener">loveyxq.online</a> 这个博客是我给我女朋友搭建的，放了一些图片之类的，之前也是托管于 github pages 上。</p><h2 id="开始配置"><a href="#开始配置" class="headerlink" title="开始配置"></a>开始配置</h2><p>1 开始之前，需要限注册一个 <a href="https://cloudflare.com" target="_blank" rel="noopener">Cloudflare</a> 账号，这个没说的</p><p>2 注册好之后 <code>Add site</code> 添加你的博客域名</p><p>3 然后选择一个计划 <code>Select a plan</code>，此处我们选择免费版本的（当然你也可以选择收费版），然后点击 <code>Confirm plan</code></p><p>4 然后添加 DNS 记录</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cloudflare/1.png" alt=""></p><p>5 完成之后需要去到你的域名注册网站，修改 nameservers 为 cloudflare 自己的，通常是：</p><figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="selector-tag">Type</span><span class="selector-tag">Value</span></span><br><span class="line"><span class="selector-tag">NS</span><span class="selector-tag">deb</span><span class="selector-class">.ns</span><span class="selector-class">.cloudflare</span><span class="selector-class">.com</span></span><br><span class="line"><span class="selector-tag">NS</span><span class="selector-tag">wilson</span><span class="selector-class">.ns</span><span class="selector-class">.cloudflare</span><span class="selector-class">.com</span></span><br></pre></td></tr></table></figure><p>6 完成之后点击 <code>Recheck Nameservers</code> 来检查配置是否正确。</p><h2 id="https-开启"><a href="#https-开启" class="headerlink" title="https 开启"></a>https 开启</h2><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cloudflare/2.png" alt=""></p><p>如上图所示，选择 <code>Full mode</code>。</p><p>设置完成后需要等待一段时间，才能使用  https 的方式去访问。此处是一个坑，设置完成以后别着急，可能要等待一个小时左右（具体忘了），cloudflare 在做 ssl 验证。</p><h1 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h1><p>实话实说，效果没有很好，毕竟免费版本的 cloudflare 给的解析节点其实不多， 如下图红框内部所示。然后也可以看到，全球各地对 loveyxq.online 的解析都是到了 cloudflare 上，已经没有 github pages 的 IP 了。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cloudflare/3.png" alt=""></p><p>另外，使用了 cloudflare 之后，cloudflare 也会对网站的访问情况以及防御情况做统计：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cloudflare/4.png" alt=""></p><h1 id="The-End"><a href="#The-End" class="headerlink" title="The End"></a>The End</h1><p>关于 CDN 的介绍，以后有空再重新分享一篇吧。主要是涉及到 CDN 的安全以及源站的防护这块。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#105;&#x6e;&#x75;&#x78;&#x63;&#x6f;&#x64;&#x65;&#x32;&#x6e;&#x69;&#107;&#x69;&#x40;&#103;&#109;&#97;&#105;&#108;&#46;&#x63;&#x6f;&#109;">&#x6c;&#105;&#x6e;&#x75;&#x78;&#x63;&#x6f;&#x64;&#x65;&#x32;&#x6e;&#x69;&#107;&#x69;&#x40;&#103;&#109;&#97;&#105;&#108;&#46;&#x63;&#x6f;&#109;</a></p></li></ul><p>2020-09-20 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;CDN&quot;&gt;&lt;a href=&quot;#CDN&quot; class=&quot;headerlink&quot; title=&quot;CDN&quot;&gt;&lt;/a&gt;CDN&lt;/h1&gt;&lt;p&gt;关于 CDN 是什么，我想应该不用做过多的介绍，毕竟现在是一个 “云” 的时代，你至少也听说过 阿里云 或者 腾讯云 吧，当然其中就包括 CDN 业务。&lt;/p&gt;
&lt;p&gt;CDN 的作用有很多，比如可以用来加速网站的访问，可以用来防护网站等。本篇文章讨论的就是使用 cloudflare 作为 CDN 来加速博客网站，并让博客开启 https，提升博客安全等级。 &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;加速网站访问&lt;/li&gt;
&lt;li&gt;开启https&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;选择什么 CDN 呢？&lt;/p&gt;
&lt;p&gt;选择 CDN，对于个人博客来说，主要考虑的还是访问速度以及价格，当然也有免费的 CDN。&lt;a href=&quot;https://cloudflare.com&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Cloudflare&lt;/a&gt; 就是一家提供免费 CDN 的公司，也是在 CDN 领域比较知名的公司。&lt;/p&gt;
&lt;p&gt;话不多说，关于 cloudflare 的配置网上可以搜到很多文章，这里我就简单记录一下。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cloudflare/0.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
    
    </summary>
    
    
      <category term="blog" scheme="https://rebootcat.com/categories/blog/"/>
    
    
      <category term="hexo" scheme="https://rebootcat.com/tags/hexo/"/>
    
      <category term="blog" scheme="https://rebootcat.com/tags/blog/"/>
    
      <category term="cloudflare" scheme="https://rebootcat.com/tags/cloudflare/"/>
    
      <category term="cdn" scheme="https://rebootcat.com/tags/cdn/"/>
    
  </entry>
  
  <entry>
    <title>迁移博客到香港虚拟空间</title>
    <link href="https://rebootcat.com/2020/09/20/virtual_space_blog/"/>
    <id>https://rebootcat.com/2020/09/20/virtual_space_blog/</id>
    <published>2020-09-20T03:23:58.000Z</published>
    <updated>2020-09-20T04:02:51.704Z</updated>
    
    <content type="html"><![CDATA[<h1 id="缘起"><a href="#缘起" class="headerlink" title="缘起"></a>缘起</h1><p>我的博客一直采用的是 github pages 来托管，中间断断续续的也没怎么管理过，偶尔写几篇博客，所以也就没怎么关心过访问速度，搜索引擎收录等问题。</p><p>不过我对博客一直还是情有独钟，我觉得像我一样的软件工程师，如果能有个人博客，并且保持一定程度的更新率还是很有必要的。</p><p>这次迁移主要考虑三个原因：</p><ul><li>访问速度较慢</li><li>博客还不支持 https</li><li>谷歌搜索引擎收录较少</li></ul><p>github pages 服务器位于美国，对于中文博客来说，访问还是有一些慢的，且不说 github 未来在我国很有可能被 feng，所以打算迁移到国内来。之前博客其实是有部署过双线的，国外走 github，国内走 coding，但奈何 coding 不争气，后来我干脆停了 coding 的解析。现在打算找一个付费的香港虚拟主机，一年几十块钱搞定。</p><p>另外就是由于之前已经采用了 <strong>rebootcat.com</strong> 这个域名，所以无法在 github pages 上开启 https（当然方法是有的，比如使用 cloudflare 加速，这个详见我另外一篇博文），所以这次的迁移也打算开启全站 https。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/1.png" alt=""></p><a id="more"></a><h1 id="购买香港虚拟主机"><a href="#购买香港虚拟主机" class="headerlink" title="购买香港虚拟主机"></a>购买香港虚拟主机</h1><p>虚拟主机是什么？</p><blockquote><p>虚拟主机（英语：virtual hosting）或称 共享主机（shared web hosting），又称虚拟服务器，是一种在单一主机或主机群上，实现多网域服务的方法，可以运行多个网站或服务的技术。虚拟主机之间完全独立，并可由用户自行管理，虚拟并非指不存在，而是指空间是由实体的服务器延伸而来，其硬件系统可以是基于服务器群，或者单个服务器。(来自某百科）</p></blockquote><p>简单来说，虚拟主机就是你可以用来托管网站，给你一定量的存储空间，以及访问流量，还有IP 或者域名绑定等。</p><p>这里需要说明的是，你能搜到很多免费的虚拟空间，免费的我个人不太建议，免费的有很多问题这里就不细说了，况且付费的也没有很贵，一年几十块钱，当然还是有可能跑路的（手动狗头）！</p><p>如上图所示，这是我购买的虚拟主机的控制面板，提供了比较方便的中文管理面板，比如域名绑定，缓存设置，SSL 设置，FTP 管理等。</p><p>具体是哪一家，我就不说了（没有给我广告费，我的服务商看到了欢迎联系）。</p><h2 id="绑定域名"><a href="#绑定域名" class="headerlink" title="绑定域名"></a>绑定域名</h2><p>由于之前是解析到 github pages 的，现在购买了虚拟主机后，会有一个新的 IP，需要重新解析域名到这个 IP 上</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/2.png" alt=""></p><p>如上图所示，红色框里面的就是新加的两条 DNS 解析记录，黄色框里面就是之前解析到 github pages 的记录，现在我把他们全部暂停了（以防后期会用到）。</p><p>解析完成之后，等待生效，使用多地 ping 的工具去测试一下 DNS 解析是否生效了。或者你本地使用 ping 看是否生效了。</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">$<span class="built_in"> ping </span>rebootcat.com -c 4                                            [10:45:50]</span><br><span class="line">PING rebootcat.com (109.206.246.144): 56 data bytes</span><br><span class="line">64 bytes <span class="keyword">from</span> 109.206.246.144: <span class="attribute">icmp_seq</span>=0 <span class="attribute">ttl</span>=52 <span class="attribute">time</span>=44.976 ms</span><br><span class="line">64 bytes <span class="keyword">from</span> 109.206.246.144: <span class="attribute">icmp_seq</span>=1 <span class="attribute">ttl</span>=52 <span class="attribute">time</span>=49.814 ms</span><br><span class="line">64 bytes <span class="keyword">from</span> 109.206.246.144: <span class="attribute">icmp_seq</span>=2 <span class="attribute">ttl</span>=52 <span class="attribute">time</span>=46.947 ms</span><br><span class="line">64 bytes <span class="keyword">from</span> 109.206.246.144: <span class="attribute">icmp_seq</span>=3 <span class="attribute">ttl</span>=52 <span class="attribute">time</span>=46.809 ms</span><br><span class="line"></span><br><span class="line">--- rebootcat.com<span class="built_in"> ping </span>statistics ---</span><br><span class="line">4 packets transmitted, 4 packets received, 0.0% packet loss</span><br><span class="line">round-trip min/avg/max/stddev = 44.976/47.136/49.814/1.731 ms</span><br></pre></td></tr></table></figure><p>可以看到上面解析到了新的 IP 上。</p><h2 id="FTP-上传网站源码"><a href="#FTP-上传网站源码" class="headerlink" title="FTP 上传网站源码"></a>FTP 上传网站源码</h2><p>我的博客是基于  hexo 搭建的，之前是直接把网站源码发布到 github pages 上了：</p><figure class="highlight ebnf"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">hexo d -g</span></span><br></pre></td></tr></table></figure><p>现在需要把生成的网站源码打包上传到虚拟主机上。</p><p>hexo 生成的网站源码位于  <code>public</code> 目录下：</p><figure class="highlight swift"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">zip</span> -r blog.<span class="built_in">zip</span> <span class="keyword">public</span></span><br></pre></td></tr></table></figure><p>然后把 blog.zip 通过面板上的 <strong>在线文件管理</strong> 上传到虚拟主机的根目录里，比如我的根目录是 <code>/wwwroot/</code>，然后点击解压。</p><p>完成之后，浏览器输入网站</p><figure class="highlight dts"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">http:</span><span class="comment">//rebootcat.com</span></span><br></pre></td></tr></table></figure><p>看能否正确响应。一般来说，没什么问题，如果无法访问，请联系你的虚拟主机提供商。</p><h2 id="自动化上传网站源码"><a href="#自动化上传网站源码" class="headerlink" title="自动化上传网站源码"></a>自动化上传网站源码</h2><p>上面的步骤，基本上已经完成了博客迁移的大部分工作了。不过对于程序员来说，怎么能每次更新博文之后还要重复上面的步骤，甚至是需要每次用浏览器打开虚拟主机控制面板上传网站源码，那岂不是很麻烦，并且不够极客精神。</p><p>那必然是要做成自动化的方式，一个命令搞定网站更新。</p><p>其实也简单，就是利用服务商提供的 FTP 口令，使用 python 脚本自动化上传网站源码，实现自动化更新。</p><p>python 脚本可以直接从我的 github 下载：</p><p><a href="https://github.com/smaugx/dailytools/blob/master/ftpblog.py" target="_blank" rel="noopener">https://github.com/smaugx/dailytools/blob/master/ftpblog.py</a></p><p>然后修改代码里的网站域名以及 ftp 口令，改成你自己的，修改上传的本地目录以及远程目录，然后执行脚本自动化上传：</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">python</span> ftpblog.<span class="keyword">py</span></span><br></pre></td></tr></table></figure><h2 id="配置-HTTPS"><a href="#配置-HTTPS" class="headerlink" title="配置 HTTPS"></a>配置 HTTPS</h2><p>使用的是 <a href="https://freessl.cn/" target="_blank" rel="noopener">https://freessl.cn/</a> 生成免费的 HTTPS 证书。</p><p>打开网站，输入你的域名以及邮箱，根据提示下载一个工具 <strong>KeyManager</strong>，然后生成证书：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/3.png" alt=""></p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/4.png" alt=""></p><p>然后回到 freessl.cn 网站页面进行 DNS 验证：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/5.png" alt=""></p><p>目的就是为了验证你的域名的所有权。这里根据提示，去 DNS 解析的地方设置解析记录。</p><p>验证成功之后使用 <strong>KeyManager</strong> 导出证书：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/6.png" alt=""></p><p>然后会得到一个类似于 <code>rebootcat-com-nginx-0909002710.zip</code> 的包，解压之后会得到两个文件：</p><figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="selector-tag">rebootcat</span><span class="selector-class">.com_chain</span><span class="selector-class">.crt</span></span><br><span class="line"><span class="selector-tag">rebootcat</span><span class="selector-class">.com_key</span><span class="selector-class">.key</span></span><br></pre></td></tr></table></figure><p>用编辑器打开这两个文件，或者直接 <code>cat</code> 这两个文件，一个是 SSl 的证书，一个是 SSL 密钥，把这两个文件的内容拷贝到虚拟主机面板的 SSL设置处：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/7.png" alt=""></p><p>并且开启了 http 跳转 https。</p><p>到此， HTTPS 证书设置就完成了。</p><blockquote><p>注意需要记住 KeyManager 的主密码</p></blockquote><p>试试用 <a href="https://rebootcat.com">https://rebootcat.com</a> 看能否正确访问呢？</p><h1 id="网站云监控"><a href="#网站云监控" class="headerlink" title="网站云监控"></a>网站云监控</h1><p>由于购买的是香港的虚拟主机，毕竟一年也才几十块钱，很难说服务提供商就跑路了，为了避免这一类事情发生的时候导致博客无法访问，有必要对博客网站进行一些云监控，一旦出现异常，则告警。</p><p>免费的网站监控工具有很多，我用的是阿里云的监控以及 UpTimeRobot 的网站监控：</p><ul><li><a href="https://uptimerobot.com/dashboard" target="_blank" rel="noopener">https://uptimerobot.com/dashboard</a></li><li><a href="https://cloudmonitor.console.aliyun.com/" target="_blank" rel="noopener">https://cloudmonitor.console.aliyun.com/</a></li></ul><p>这个自行设置一下，注意设置好报警阈值，不然可能会造成误报：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/8.png" alt=""></p><p>所以一旦出了很严重的报警，那么说明你的服务商跑路了。。。</p><h1 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h1><p>这里就简单贴一下迁移前后的效果图：</p><p>迁移前：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/9.png" alt=""></p><p>迁移后：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/10.png" alt=""></p><p>可以看到还是有很好的改善的，毕竟服务器位于香港。</p><h1 id="The-End"><a href="#The-End" class="headerlink" title="The End"></a>The End</h1><p>到此，博客迁移就完成了，访问速度提升了，也开启了 https。接下来我会考虑对博客首页做一些优化，但由于现在图片走的其实还是 jsdelivr 的国外 cdn，所以速度还是有点慢，可以考虑直接把图片放到网站根目录下，毕竟现在使用的是虚拟主机。</p><p>后面再说吧，也可以考虑把图片等放到阿里云或者腾讯云对象存储上。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#x6c;&#105;&#x6e;&#117;&#x78;&#99;&#x6f;&#100;&#x65;&#50;&#x6e;&#x69;&#107;&#105;&#x40;&#103;&#x6d;&#97;&#x69;&#x6c;&#x2e;&#99;&#x6f;&#x6d;">&#x6c;&#105;&#x6e;&#117;&#x78;&#99;&#x6f;&#100;&#x65;&#50;&#x6e;&#x69;&#107;&#105;&#x40;&#103;&#x6d;&#97;&#x69;&#x6c;&#x2e;&#99;&#x6f;&#x6d;</a></p></li></ul><p>2020-09-20 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;缘起&quot;&gt;&lt;a href=&quot;#缘起&quot; class=&quot;headerlink&quot; title=&quot;缘起&quot;&gt;&lt;/a&gt;缘起&lt;/h1&gt;&lt;p&gt;我的博客一直采用的是 github pages 来托管，中间断断续续的也没怎么管理过，偶尔写几篇博客，所以也就没怎么关心过访问速度，搜索引擎收录等问题。&lt;/p&gt;
&lt;p&gt;不过我对博客一直还是情有独钟，我觉得像我一样的软件工程师，如果能有个人博客，并且保持一定程度的更新率还是很有必要的。&lt;/p&gt;
&lt;p&gt;这次迁移主要考虑三个原因：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;访问速度较慢&lt;/li&gt;
&lt;li&gt;博客还不支持 https&lt;/li&gt;
&lt;li&gt;谷歌搜索引擎收录较少&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;github pages 服务器位于美国，对于中文博客来说，访问还是有一些慢的，且不说 github 未来在我国很有可能被 feng，所以打算迁移到国内来。之前博客其实是有部署过双线的，国外走 github，国内走 coding，但奈何 coding 不争气，后来我干脆停了 coding 的解析。现在打算找一个付费的香港虚拟主机，一年几十块钱搞定。&lt;/p&gt;
&lt;p&gt;另外就是由于之前已经采用了 &lt;strong&gt;rebootcat.com&lt;/strong&gt; 这个域名，所以无法在 github pages 上开启 https（当然方法是有的，比如使用 cloudflare 加速，这个详见我另外一篇博文），所以这次的迁移也打算开启全站 https。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/virtual_space_blog/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
    
    </summary>
    
    
      <category term="blog" scheme="https://rebootcat.com/categories/blog/"/>
    
    
      <category term="hexo" scheme="https://rebootcat.com/tags/hexo/"/>
    
      <category term="blog" scheme="https://rebootcat.com/tags/blog/"/>
    
      <category term="virtualhost" scheme="https://rebootcat.com/tags/virtualhost/"/>
    
      <category term="ftp" scheme="https://rebootcat.com/tags/ftp/"/>
    
  </entry>
  
  <entry>
    <title>Hexo Next 博客添加相册瀑布流</title>
    <link href="https://rebootcat.com/2020/09/19/nextphotowall/"/>
    <id>https://rebootcat.com/2020/09/19/nextphotowall/</id>
    <published>2020-09-19T03:23:58.000Z</published>
    <updated>2020-09-20T01:28:13.001Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>一直没有时间来整理下博客搭建的一些事情，现在补上一篇，给 Hexo Next 博客添加一个相册功能，使用瀑布流的方式。</p><h1 id="原理说明"><a href="#原理说明" class="headerlink" title="原理说明"></a>原理说明</h1><ul><li>使用 github 作为仓库存储图片文件（图床）</li><li>使用 jsdelivr 进行图片 CDN 加速</li></ul><h2 id="优点"><a href="#优点" class="headerlink" title="优点"></a>优点</h2><p>此种方式的优点是免费，不需要购买其他的对象存储产品；并且使用的是 github 作为图床，图片不会丢失。</p><blockquote><p>早期的博文使用的是七牛云的免费存储，结果后来被他们删掉了。。。结果造成文中的一些图片链接都是 404，有兴趣的可以翻一翻我早期的博客。</p></blockquote><h2 id="缺点"><a href="#缺点" class="headerlink" title="缺点"></a>缺点</h2><p>由于采用的是 github 仓库存储图片，但是 github 对单仓库有 50MB 的大小限制，所以单仓库可能不能够存储太多的文件；</p><p>解决方法就是建立很多的图片仓库（稍微有点费劲，不过是行得通的）；另外上传的单张图片大小最好不要太大。</p><p>还有个缺点就是得折腾啊，且看我后文。</p><p>各位可以参考下我的相册瀑布流: <a href="https://rebootcat.com/photos/">摄影</a></p><h1 id="开始搭建相册瀑布流"><a href="#开始搭建相册瀑布流" class="headerlink" title="开始搭建相册瀑布流"></a>开始搭建相册瀑布流</h1><p>开始之前，需要简单介绍一下，我参考的是 <a href="https://blog.dlzhang.com/posts/31/" target="_blank" rel="noopener">Hexo NexT 博客增加瀑布流相册页面</a> 这篇文章，文中涉及到的脚本主要都是 js 实现；与他不同的是，由于我对 js 的掌握远远不及我对 Python 的掌握，故部分脚本我采用了 Python 实现。</p><p>所以在开始操作之前，你可以根据自己的技能，选择不同的方式。如果你擅长 python，那么跟着我来吧。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/nextphotowall/1.png" alt=""></p><a id="more"></a><h2 id="新建-photo-页面"><a href="#新建-photo-页面" class="headerlink" title="新建 photo 页面"></a>新建 photo 页面</h2><p>去到博客根目录：</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">mkdir</span> -<span class="keyword">p</span> <span class="keyword">source</span>/photos</span><br></pre></td></tr></table></figure><p>然后进入 photos 目录：</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">cd</span> <span class="keyword">source</span>/photos</span><br><span class="line"><span class="keyword">vim</span> <span class="built_in">index</span>.md</span><br></pre></td></tr></table></figure><p>把下面的粘贴保存：</p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">---</span><br><span class="line">title: 摄影</span><br><span class="line">type: photos</span><br><span class="line">---</span><br><span class="line"></span><br><span class="line"><span class="comment">&lt;!-- CSS Code --&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">style</span>&gt;</span></span><br><span class="line"><span class="css"><span class="selector-class">.MyGrid</span>&#123;<span class="attribute">width</span>:<span class="number">100%</span>;<span class="attribute">max-width</span>:<span class="number">1040px</span>;<span class="attribute">margin</span>:<span class="number">0</span> auto;<span class="attribute">text-align</span>:center&#125;<span class="selector-class">.card</span>&#123;<span class="attribute">overflow</span>:hidden;<span class="attribute">transition</span>:.<span class="number">3s</span> ease-in-out;<span class="attribute">border-radius</span>:<span class="number">8px</span>;<span class="attribute">background-color</span>:<span class="number">#efefef</span>;<span class="attribute">padding</span>:<span class="number">1.4px</span>&#125;<span class="selector-class">.ImageInCard</span> <span class="selector-tag">img</span>&#123;<span class="attribute">padding</span>:<span class="number">0</span>;<span class="attribute">border-radius</span>:<span class="number">8px</span>&#125;</span></span><br><span class="line">@media(prefers-color-scheme:dark)&#123;.card&#123;background-color:#333;&#125;&#125;</span><br><span class="line"><span class="tag">&lt;/<span class="name">style</span>&gt;</span></span><br><span class="line"><span class="comment">&lt;!-- CSS Code End --&gt;</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">"MyGrid"</span>&gt;</span><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br></pre></td></tr></table></figure><h2 id="修改-Next-主题配置文件"><a href="#修改-Next-主题配置文件" class="headerlink" title="修改 Next 主题配置文件"></a>修改 Next 主题配置文件</h2><p>添加了 photos 页面后，需要在 next 配置文件中修改：</p><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim themes/next/<span class="module-access"><span class="module"><span class="identifier">_config</span>.</span></span>yml</span><br></pre></td></tr></table></figure><p>找到 menu 项，填入如下：</p><figure class="highlight 1c"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">photos: /photos <span class="string">|| fas fa-camera-retro</span></span><br></pre></td></tr></table></figure><p>比如我的是这样的：</p><figure class="highlight dts"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">menu:</span></span><br><span class="line"><span class="symbol">  home:</span> / || home</span><br><span class="line"><span class="symbol">  about:</span> <span class="meta-keyword">/about/</span> || user</span><br><span class="line"><span class="symbol">  tags:</span> <span class="meta-keyword">/tags/</span> || tags</span><br><span class="line"><span class="symbol">  categories:</span> <span class="meta-keyword">/categories/</span> || th</span><br><span class="line"><span class="symbol">  archives:</span> <span class="meta-keyword">/archives/</span> || archive</span><br><span class="line">  <span class="meta">#schedule: /schedule/ || calendar</span></span><br><span class="line">  <span class="meta">#sitemap: /sitemap.xml || sitemap</span></span><br><span class="line">  <span class="meta">#commonweal: /404/ || heartbeat</span></span><br><span class="line"><span class="symbol">  guestbook:</span> /guestbook || fas fa-comments</span><br><span class="line"><span class="symbol">  photos:</span> /photos || fas fa-camera-retro</span><br><span class="line"><span class="symbol">  wiki:</span> <span class="meta-keyword">/wiki/</span> || wikipedia-w</span><br></pre></td></tr></table></figure><p>完成之后还需要修改一下这个文件：</p><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim themes/next/languages/zh-<span class="module-access"><span class="module"><span class="identifier">CN</span>.</span></span>yml</span><br></pre></td></tr></table></figure><p>找到 menu 项，加入如下一行：</p><figure class="highlight avrasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="symbol">photos:</span> 摄影</span><br></pre></td></tr></table></figure><p>比如我的是这样的：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">menu:</span></span><br><span class="line">  <span class="attr">home:</span> <span class="string">首页</span></span><br><span class="line">  <span class="attr">archives:</span> <span class="string">归档</span></span><br><span class="line">  <span class="attr">categories:</span> <span class="string">分类</span></span><br><span class="line">  <span class="attr">tags:</span> <span class="string">标签</span></span><br><span class="line">  <span class="attr">about:</span> <span class="string">关于</span></span><br><span class="line">  <span class="attr">search:</span> <span class="string">搜索</span></span><br><span class="line">  <span class="attr">schedule:</span> <span class="string">日程表</span></span><br><span class="line">  <span class="attr">sitemap:</span> <span class="string">站点地图</span></span><br><span class="line">  <span class="attr">commonweal:</span> <span class="string">公益</span> <span class="number">404</span></span><br><span class="line">  <span class="attr">guestbook:</span> <span class="string">留言</span></span><br><span class="line">  <span class="attr">photos:</span> <span class="string">摄影</span></span><br><span class="line">  <span class="attr">wiki:</span> <span class="string">维基</span></span><br></pre></td></tr></table></figure><p>OK，到这里应该能看到这个 <strong>摄影</strong> 页面了，你可以现在本地测试一下看：</p><figure class="highlight ebnf"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">hexo s -g</span></span><br></pre></td></tr></table></figure><h2 id="添加-js-脚本"><a href="#添加-js-脚本" class="headerlink" title="添加 js 脚本"></a>添加 js 脚本</h2><p>首先需要在 source 目录下新建一个 js 目录，用来保存自定义的一些 js 脚本；</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">mkdir</span> -<span class="keyword">p</span> <span class="keyword">source</span>/js</span><br></pre></td></tr></table></figure><p>然后新建 mygrid.js 文件，粘贴下面的一段代码：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 获取网页不含域名的路径</span></span><br><span class="line"><span class="keyword">var</span> windowPath = <span class="built_in">window</span>.location.pathname;</span><br><span class="line"><span class="comment">// 图片信息文件路径</span></span><br><span class="line"><span class="keyword">var</span> imgDataPath = <span class="string">'/photos/photoslist.json'</span>;</span><br><span class="line"><span class="comment">// 图片显示数量</span></span><br><span class="line"><span class="keyword">var</span> imgMaxNum = <span class="number">50</span>;</span><br><span class="line"><span class="comment">// 获取窗口宽度（以确定图片显示宽度）</span></span><br><span class="line"><span class="keyword">var</span> windowWidth = <span class="built_in">window</span>.innerWidth</span><br><span class="line">|| <span class="built_in">document</span>.documentElement.clientWidth</span><br><span class="line">|| <span class="built_in">document</span>.body.clientWidth;</span><br><span class="line"><span class="keyword">if</span> (windowWidth &lt; <span class="number">768</span>) &#123;</span><br><span class="line">    <span class="keyword">var</span> imageWidth = <span class="number">145</span>; <span class="comment">// 图片显示宽度(手机)</span></span><br><span class="line">&#125; <span class="keyword">else</span> &#123;</span><br><span class="line">    <span class="keyword">var</span> imageWidth = <span class="number">215</span>; <span class="comment">// 图片显示宽度</span></span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">// 腾讯云图片处理样式（根据图片显示宽度）</span></span><br><span class="line"><span class="keyword">var</span> imgStyle = <span class="string">'!'</span> + imageWidth + <span class="string">'x'</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">// 图片卡片（照片页面）</span></span><br><span class="line"><span class="keyword">if</span> (windowPath.indexOf(<span class="string">'photos'</span>) &gt; <span class="number">0</span> ) &#123;</span><br><span class="line">    <span class="keyword">var</span> LinkDataPath = imgDataPath;</span><br><span class="line">    photo = &#123;</span><br><span class="line">        page: <span class="number">1</span>,</span><br><span class="line">        offset: imgMaxNum,</span><br><span class="line">        init: <span class="function"><span class="keyword">function</span> (<span class="params"></span>) </span>&#123;</span><br><span class="line">            <span class="keyword">var</span> that = <span class="keyword">this</span>;</span><br><span class="line">            $.getJSON(LinkDataPath, <span class="function"><span class="keyword">function</span> (<span class="params">data</span>) </span>&#123;</span><br><span class="line">                that.render(that.page, data);</span><br><span class="line">            &#125;);</span><br><span class="line">        &#125;,</span><br><span class="line">        render: <span class="function"><span class="keyword">function</span> (<span class="params">page, data</span>) </span>&#123;</span><br><span class="line">            <span class="keyword">var</span> begin = (page - <span class="number">1</span>) * <span class="keyword">this</span>.offset;</span><br><span class="line">            <span class="keyword">var</span> end = page * <span class="keyword">this</span>.offset;</span><br><span class="line">            <span class="keyword">if</span> (begin &gt;= data.length) <span class="keyword">return</span>;</span><br><span class="line">            <span class="keyword">var</span> html, imgNameWithPattern, imgName, imageSize, imageX, imageY, li = <span class="string">""</span>;</span><br><span class="line">            <span class="keyword">for</span> (<span class="keyword">var</span> i = begin; i &lt; end &amp;&amp; i &lt; data.length; i++) &#123;</span><br><span class="line">                imgNameWithPattern = data[i].split(<span class="string">';'</span>)[<span class="number">1</span>];  <span class="comment">// a.png</span></span><br><span class="line">                imgName = imgNameWithPattern.split(<span class="string">'.'</span>)[<span class="number">0</span>]  <span class="comment">// a</span></span><br><span class="line">                imageSize = data[i].split(<span class="string">';'</span>)[<span class="number">0</span>]; <span class="comment">// length.height</span></span><br><span class="line">                imageX = imageSize.split(<span class="string">'.'</span>)[<span class="number">0</span>]; <span class="comment">//  length</span></span><br><span class="line">                imageY = imageSize.split(<span class="string">'.'</span>)[<span class="number">1</span>]; <span class="comment">// height</span></span><br><span class="line"></span><br><span class="line">  cdn_url       = data[i].split(<span class="string">';'</span>)[<span class="number">2</span>]; <span class="comment">// 原图 cdn url</span></span><br><span class="line">  small_cdn_url = data[i].split(<span class="string">';'</span>)[<span class="number">3</span>]; <span class="comment">// 缩略图 cdn url</span></span><br><span class="line"></span><br><span class="line">                li += <span class="string">'&lt;div class="card" style="width:'</span> + imageWidth + <span class="string">'px" &gt;'</span> +</span><br><span class="line">                        <span class="string">'&lt;div class="ImageInCard" style="height:'</span>+ imageWidth * imageY / imageX + <span class="string">'px"&gt;'</span> +</span><br><span class="line">                            <span class="string">'&lt;a data-fancybox="gallery" href="'</span> + cdn_url + <span class="string">'" data-caption="'</span> + imgName + <span class="string">'" title="'</span> +  imgName + <span class="string">'"&gt;'</span> +</span><br><span class="line">                                <span class="string">'&lt;img data-src="'</span> + small_cdn_url + <span class="string">'" src="'</span> + small_cdn_url + <span class="string">'" data-loaded="true"&gt;'</span> +</span><br><span class="line">                            <span class="string">'&lt;/a&gt;'</span> +</span><br><span class="line">                        <span class="string">'&lt;/div&gt;'</span> +</span><br><span class="line">                      <span class="string">'&lt;/div&gt;'</span></span><br><span class="line">            &#125;</span><br><span class="line">            $(<span class="string">".MyGrid"</span>).append(li);</span><br><span class="line">            <span class="keyword">this</span>.minigrid();</span><br><span class="line">        &#125;,</span><br><span class="line">        minigrid: <span class="function"><span class="keyword">function</span>(<span class="params"></span>) </span>&#123;</span><br><span class="line">            <span class="keyword">var</span> grid = <span class="keyword">new</span> Minigrid(&#123;</span><br><span class="line">                container: <span class="string">'.MyGrid'</span>,</span><br><span class="line">                item: <span class="string">'.card'</span>,</span><br><span class="line">                gutter: <span class="number">12</span></span><br><span class="line">            &#125;);</span><br><span class="line">            grid.mount();</span><br><span class="line">            $(<span class="built_in">window</span>).resize(<span class="function"><span class="keyword">function</span>(<span class="params"></span>) </span>&#123;</span><br><span class="line">                grid.mount();</span><br><span class="line">            &#125;);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    photo.init();</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>或者你可以直接在我的博客上找到： <a href="https://rebootcat.com/js/mygrid.js">rebootcat.com/mygrid.js</a></p><figure class="highlight x86asm"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://rebootcat<span class="number">.</span>com/<span class="keyword">js</span>/mygrid<span class="number">.</span><span class="keyword">js</span> -O source/<span class="keyword">js</span>/mygrid<span class="number">.</span><span class="keyword">js</span></span><br></pre></td></tr></table></figure><h2 id="新建图片信息文件"><a href="#新建图片信息文件" class="headerlink" title="新建图片信息文件"></a>新建图片信息文件</h2><p>我们再次回到 photos 目录，创建文件 photoslist.json：</p><figure class="highlight gradle"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim <span class="keyword">source</span><span class="regexp">/photos/</span>photoslist.json</span><br></pre></td></tr></table></figure><p>然后输入如下的内容：</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[</span><br><span class="line">  <span class="string">"1080.1920;WechatIMG114.jpeg;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/photowall/cat/WechatIMG114.jpeg;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/photowall/cat/WechatIMG114_small.jpeg"</span>,</span><br><span class="line">  <span class="string">"3024.4032;WechatIMG25834.jpeg;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/photowall/cat/WechatIMG25834.jpeg;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/photowall/cat/WechatIMG25834_small.jpeg"</span></span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>OK, 到现在应该你能从博客上看到这两张图片了:</p><figure class="highlight ebnf"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">hexo s -g</span></span><br></pre></td></tr></table></figure><p>本地测试一下，如果你能看到在博客的 <strong>摄影</strong> 页面看到这两张图片，那么说明你的配置没问题，你可以进行接下来的操作了；如果你不能正确显示，说明前面的步骤出了问题，自己研究调试一下；如果你还不能解决，欢迎联系我。</p><h2 id="使用-python-脚本生成-photoslist-json"><a href="#使用-python-脚本生成-photoslist-json" class="headerlink" title="使用 python 脚本生成 photoslist.json"></a>使用 python 脚本生成 photoslist.json</h2><p>上面可以看到，<strong>photoslist.json 存放的是图片的信息，mygrid.js 解析 photoslist.json 这个文件，然后在 photos 页面添加 dom</strong>.</p><p>所以核心的部分在于 photoslist.json 文件，我们可以分析下这个文件：</p><figure class="highlight awk"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="number">1080.1920</span>;WechatIMG114.jpeg;https:<span class="regexp">//</span>cdn.jsdelivr.net<span class="regexp">/gh/</span>smaugx<span class="regexp">/MyblogImgHosting/</span>rebootcat<span class="regexp">/photowall/</span>cat<span class="regexp">/WechatIMG114.jpeg;https:/</span><span class="regexp">/cdn.jsdelivr.net/g</span>h<span class="regexp">/smaugx/</span>MyblogImgHosting<span class="regexp">/rebootcat/</span>photowall<span class="regexp">/cat/</span>WechatIMG114_small.jpeg</span><br></pre></td></tr></table></figure><p>photoslist.json 保存的是一个 list，<strong>list 中每一行是一张图片的信息，包括原始图片大小、文件名、原始图片cdn链接、缩略图cdn链接</strong>。</p><p>前面已经提到，我们的图片是使用了 github 作为图床（仓库），然后使用 jsdelivr 进行 cdn 加速。所以我们应该准备好图片文件，然后上传到仓库。</p><h3 id="新建-github-仓库，用来存放图片文件"><a href="#新建-github-仓库，用来存放图片文件" class="headerlink" title="新建 github 仓库，用来存放图片文件"></a>新建 github 仓库，用来存放图片文件</h3><p>在 <a href="https://github.com" target="_blank" rel="noopener">https://github.com</a> 上创建图片仓库。</p><blockquote><p>当仓库容量超过 50MB 之后需要重新再新建一个仓库</p></blockquote><p>本地克隆仓库，然后把图片放入仓库，上传(这里以我的仓库为例）</p><figure class="highlight crmsh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">git <span class="keyword">clone</span> <span class="title">git</span>@github.com:smaugx/MyblogImgHosting_2.git blogimg_2</span><br><span class="line">cd blogimg_2</span><br><span class="line"></span><br><span class="line"><span class="comment"># put some image in this dir</span></span><br><span class="line"></span><br><span class="line">...</span><br><span class="line">git push</span><br></pre></td></tr></table></figure><h3 id="生成-photoslist-json-文件"><a href="#生成-photoslist-json-文件" class="headerlink" title="生成 photoslist.json 文件"></a>生成 photoslist.json 文件</h3><p>编写 python 脚本或者直接从我的网站下载：</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget http<span class="variable">s:</span>//rebootcat.<span class="keyword">com</span>/js/phototool.<span class="keyword">py</span>  -O phototool.<span class="keyword">py</span></span><br></pre></td></tr></table></figure><p>脚本如下：</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#!/usr/bin/env python</span></span><br><span class="line"><span class="comment"># -*- coding:utf8  -*-</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> glob</span><br><span class="line"><span class="keyword">from</span> PIL <span class="keyword">import</span> Image, ExifTags</span><br><span class="line"><span class="keyword">import</span> json</span><br><span class="line"></span><br><span class="line">config = &#123;</span><br><span class="line">        <span class="comment"># github 存储图片的仓库（本地仓库基准目录）</span></span><br><span class="line">        <span class="string">'github_img_host_base'</span>: <span class="string">'/Users/smaug/blogimg_2'</span>,</span><br><span class="line">        <span class="comment"># 会对这个目录下的所有文件夹进行遍历，相同目录生成_samll 的 缩略图</span></span><br><span class="line">        <span class="string">'img_path'</span>:             <span class="string">'/Users/smaug/blogimg_2/rebootcat/photowall'</span>,</span><br><span class="line">        <span class="comment"># cdn 前缀</span></span><br><span class="line">        <span class="string">'cdn_url_prefix'</span>:       <span class="string">'https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2'</span>,</span><br><span class="line">        <span class="comment"># hexo 博客存放 photos 信息的 json 文件</span></span><br><span class="line">        <span class="string">'photo_info_json'</span>:      <span class="string">'/Users/smaug/blog_rebootcat/source/photos/photoslist.json'</span>,</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line"><span class="comment"># 压缩图片到 90%(目的是为了移除一些gps 等信息，并非真的为了压缩）</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">compress_img</span><span class="params">(img_path, rate = <span class="number">0.99</span>, override = False)</span>:</span></span><br><span class="line">    support_ftype_list = [<span class="string">'png'</span>, <span class="string">'PNG'</span>, <span class="string">'jpeg'</span>, <span class="string">'JPEG'</span>, <span class="string">'gif'</span>, <span class="string">'GIF'</span>, <span class="string">'bmp'</span>]</span><br><span class="line">    sp_img = img_path.split(<span class="string">'.'</span>)</span><br><span class="line">    <span class="keyword">if</span> <span class="keyword">not</span> sp_img <span class="keyword">or</span> sp_img[<span class="number">-1</span>] <span class="keyword">not</span> <span class="keyword">in</span> support_ftype_list:</span><br><span class="line">        print(<span class="string">"not support image type:&#123;0&#125;"</span>, img_path)</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line">    sp_img = img_path.split(<span class="string">'/'</span>)</span><br><span class="line">    <span class="keyword">if</span> <span class="keyword">not</span> sp_img:</span><br><span class="line">        print(<span class="string">"please give the right image path:&#123;0&#125;"</span>, img_path)</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line">    img_full_name = sp_img[<span class="number">-1</span>]</span><br><span class="line">    img_name = img_full_name.split(<span class="string">'.'</span>)[<span class="number">0</span>]</span><br><span class="line">    img_type = img_full_name.split(<span class="string">'.'</span>)[<span class="number">1</span>]</span><br><span class="line">    img_path_prefix = img_path[:-len(img_full_name)]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 覆盖原图或者另存为</span></span><br><span class="line">    compress_img_path = <span class="string">''</span></span><br><span class="line">    <span class="keyword">if</span> override:</span><br><span class="line">        compress_img_path = img_path</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        compress_img_path = <span class="string">'&#123;0&#125;&#123;1&#125;_com.&#123;2&#125;'</span>.format(img_path_prefix, img_name, img_type)</span><br><span class="line"></span><br><span class="line">    img = Image.open(img_path)</span><br><span class="line">    <span class="keyword">try</span>:</span><br><span class="line">        <span class="keyword">for</span> orientation <span class="keyword">in</span> ExifTags.TAGS.keys() :</span><br><span class="line">            <span class="keyword">if</span> ExifTags.TAGS[orientation]==<span class="string">'Orientation'</span> : <span class="keyword">break</span></span><br><span class="line">        exif=dict(img._getexif().items())</span><br><span class="line">        <span class="keyword">if</span>   exif[orientation] == <span class="number">3</span> :</span><br><span class="line">            img=img.rotate(<span class="number">180</span>, expand = <span class="literal">True</span>)</span><br><span class="line">        <span class="keyword">elif</span> exif[orientation] == <span class="number">6</span> :</span><br><span class="line">            img=img.rotate(<span class="number">270</span>, expand = <span class="literal">True</span>)</span><br><span class="line">        <span class="keyword">elif</span> exif[orientation] == <span class="number">8</span> :</span><br><span class="line">            img=img.rotate(<span class="number">90</span>, expand = <span class="literal">True</span>)</span><br><span class="line">    <span class="keyword">except</span> Exception <span class="keyword">as</span> e:</span><br><span class="line">        print(<span class="string">"catch exception:&#123;0&#125;"</span>,e)</span><br><span class="line"></span><br><span class="line">    <span class="keyword">try</span>:</span><br><span class="line">        original_size = img.size</span><br><span class="line">        length = original_size[<span class="number">0</span>]</span><br><span class="line">        height = original_size[<span class="number">1</span>]</span><br><span class="line">        new_length = int(length * rate)</span><br><span class="line">        new_height = int(height * rate)</span><br><span class="line">        print(<span class="string">"originla length:&#123;0&#125; height:&#123;1&#125;"</span>, length, height)</span><br><span class="line">        print(<span class="string">"after compress length:&#123;0&#125; height:&#123;1&#125;"</span>, new_length, new_height)</span><br><span class="line">        img = img.resize((new_length, new_height), Image.ANTIALIAS)</span><br><span class="line">        img.save(compress_img_path, img_type)</span><br><span class="line">        print(<span class="string">"save compress img &#123;0&#125;"</span>.format(compress_img_path))</span><br><span class="line">        <span class="keyword">return</span> <span class="literal">True</span></span><br><span class="line">    <span class="keyword">except</span> Exception <span class="keyword">as</span> e:</span><br><span class="line">        print(<span class="string">"catch exception:&#123;0&#125;"</span>,e)</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> <span class="literal">False</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># 对 img_path 目录下的文件夹递归生成缩略图保存到同目录下</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">thumbnail_pic</span><span class="params">(github_img_host_base, img_path, cdn_url_prefix)</span>:</span></span><br><span class="line">    <span class="comment"># 删除最后一个 '/'</span></span><br><span class="line">    <span class="keyword">if</span> img_path[<span class="number">-1</span>] == <span class="string">'/'</span>:</span><br><span class="line">        img_path = img_path[:<span class="number">-1</span>]</span><br><span class="line">    <span class="keyword">if</span> github_img_host_base[<span class="number">-1</span>] == <span class="string">'/'</span>:</span><br><span class="line">        github_img_host_base = github_img_host_base[:<span class="number">-1</span>]</span><br><span class="line">    <span class="keyword">if</span> cdn_url_prefix[<span class="number">-1</span>] == <span class="string">'/'</span>:</span><br><span class="line">        cdn_url_prefix = cdn_url_prefix[:<span class="number">-1</span>]</span><br><span class="line"></span><br><span class="line">    photo_info_list = []</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> item <span class="keyword">in</span> os.listdir(img_path):</span><br><span class="line">        print(item)</span><br><span class="line">        abs_item = os.path.join(img_path, item)</span><br><span class="line">        <span class="keyword">if</span> os.path.isdir(abs_item): <span class="comment"># sub-dir</span></span><br><span class="line">            sub_img_path = abs_item</span><br><span class="line">            print(<span class="string">"cd dir:&#123;0&#125;"</span>.format(sub_img_path))</span><br><span class="line">            sub_photo_info_list = thumbnail_pic(github_img_host_base, sub_img_path, cdn_url_prefix)</span><br><span class="line">            photo_info_list.extend(sub_photo_info_list)</span><br><span class="line">        <span class="keyword">else</span>: <span class="comment"># file</span></span><br><span class="line">            ftype = item.split(<span class="string">'.'</span>)</span><br><span class="line">            <span class="keyword">if</span> <span class="keyword">not</span> ftype <span class="keyword">or</span> len(ftype) != <span class="number">2</span>:</span><br><span class="line">                print(<span class="string">"error: invalid file:&#123;0&#125;"</span>.format(item))</span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line">            fname = ftype[<span class="number">0</span>]  <span class="comment"># a.png -&gt; a</span></span><br><span class="line">            ftype = ftype[<span class="number">1</span>]  <span class="comment"># a.png -&gt; png</span></span><br><span class="line">            support_ftype_list = [<span class="string">'png'</span>, <span class="string">'PNG'</span>, <span class="string">'jpeg'</span>, <span class="string">'JPEG'</span>, <span class="string">'gif'</span>, <span class="string">'GIF'</span>, <span class="string">'bmp'</span>]</span><br><span class="line">            <span class="keyword">if</span> ftype <span class="keyword">not</span> <span class="keyword">in</span> support_ftype_list:</span><br><span class="line">                print(<span class="string">"error: file type &#123;0&#125; not support, only support &#123;1&#125;"</span>.format(ftype, json.dumps(support_ftype_list)))</span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line"></span><br><span class="line">            abs_file = abs_item</span><br><span class="line">            <span class="keyword">if</span> item.find(<span class="string">'_small'</span>) != <span class="number">-1</span>: <span class="comment"># 这是缩略图</span></span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line">            small_file = <span class="string">'&#123;0&#125;_small.&#123;1&#125;'</span>.format(fname, ftype)</span><br><span class="line">            abs_small_file = os.path.join(img_path, small_file)  <span class="comment"># 缩略图绝对路径</span></span><br><span class="line">            <span class="keyword">if</span> os.path.exists(abs_small_file):</span><br><span class="line">                <span class="comment"># 对应的 _small 缩略图已经存在</span></span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line"></span><br><span class="line">            compress_status = compress_img(abs_file, <span class="number">0.9</span>, <span class="literal">True</span>)</span><br><span class="line">            <span class="keyword">if</span> <span class="keyword">not</span> compress_status:</span><br><span class="line">                print(<span class="string">"compress_img fail:&#123;0&#125;"</span>, abs_file)</span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line"></span><br><span class="line">            im = Image.open(abs_file)</span><br><span class="line">            original_size = im.size</span><br><span class="line">            length = original_size[<span class="number">0</span>]</span><br><span class="line">            height = original_size[<span class="number">1</span>]</span><br><span class="line">            m = int(float(length) / <span class="number">200.0</span>)  <span class="comment"># 计算缩小比例 (缩略图限制 200 长度)</span></span><br><span class="line">            new_length = int(float(length) / m)</span><br><span class="line">            new_height = int(float(height) / m)</span><br><span class="line">            im.thumbnail((new_length, new_height))  <span class="comment"># 生成缩略图</span></span><br><span class="line">            im.save(abs_small_file, ftype)  <span class="comment"># 保存缩略图</span></span><br><span class="line">            print(<span class="string">"save thumbnail img &#123;0&#125;"</span>.format(abs_small_file))</span><br><span class="line"></span><br><span class="line">            relative_file       = abs_file[len(github_img_host_base) + <span class="number">1</span>:] <span class="comment"># 计算相对路径，用来拼接 cdn</span></span><br><span class="line">            relative_small_file = abs_small_file[len(github_img_host_base) + <span class="number">1</span>:]</span><br><span class="line"></span><br><span class="line">            cdn_url_file        = <span class="string">'&#123;0&#125;/&#123;1&#125;'</span>.format(cdn_url_prefix, relative_file)</span><br><span class="line">            cdn_url_small_file  = <span class="string">'&#123;0&#125;/&#123;1&#125;'</span>.format(cdn_url_prefix, relative_small_file)</span><br><span class="line"></span><br><span class="line">            <span class="comment"># 格式: 690.690;8.png;http://cdn_file_url;http://cdn_small_file_url;</span></span><br><span class="line">            line = <span class="string">'&#123;0&#125;.&#123;1&#125;;&#123;2&#125;;&#123;3&#125;;&#123;4&#125;'</span>.format(length, height, item, cdn_url_file, cdn_url_small_file)</span><br><span class="line">            photo_info_list.append(line)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># end for loop</span></span><br><span class="line">    print(<span class="string">'dir:&#123;0&#125; Done!'</span>.format(img_path))</span><br><span class="line">    <span class="keyword">return</span> photo_info_list</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__==<span class="string">'__main__'</span>:</span><br><span class="line">    github_img_host_base = config.get(<span class="string">'github_img_host_base'</span>)</span><br><span class="line">    img_path             = config.get(<span class="string">'img_path'</span>)</span><br><span class="line">    cdn_url_prefix       = config.get(<span class="string">'cdn_url_prefix'</span>)</span><br><span class="line">    photo_info_json      = config.get(<span class="string">'photo_info_json'</span>)</span><br><span class="line"></span><br><span class="line">    photo_info_list     = []</span><br><span class="line">    photo_info_list_has = []</span><br><span class="line">    photo_info_list = thumbnail_pic(github_img_host_base, img_path, cdn_url_prefix)</span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> os.path.exists(photo_info_json):</span><br><span class="line">        <span class="keyword">with</span> open(photo_info_json, <span class="string">'r'</span>) <span class="keyword">as</span> fin:</span><br><span class="line">            photo_info_list_has = json.loads(fin.read())</span><br><span class="line">            fin.close()</span><br><span class="line"></span><br><span class="line">    photo_info_list_has.extend(photo_info_list)  <span class="comment"># 追加此次新增的 photo info</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">with</span> open(photo_info_json, <span class="string">'w'</span>) <span class="keyword">as</span> fout:</span><br><span class="line">        fout.write(json.dumps(photo_info_list_has, indent = <span class="number">2</span>))</span><br><span class="line">        print(<span class="string">"save photo_info_list to &#123;0&#125;"</span>.format(photo_info_json))</span><br><span class="line">        fout.close()</span><br><span class="line"></span><br><span class="line">    print(<span class="string">"\nAll Done"</span>)</span><br></pre></td></tr></table></figure><p>这里重点需要关注的是：</p><figure class="highlight 1c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">config = &#123;</span><br><span class="line">        <span class="meta"># github 存储图片的仓库（本地仓库基准目录）</span></span><br><span class="line">        'github_img_host_base': '/Users/smaug/blogimg_2',</span><br><span class="line">        <span class="meta"># 会对这个目录下的所有文件夹进行遍历，相同目录生成_samll 的 缩略图</span></span><br><span class="line">        'img_path':             '/Users/smaug/blogimg_2/rebootcat/photowall',</span><br><span class="line">        <span class="meta"># cdn 前缀</span></span><br><span class="line">        'cdn_url_prefix':       'https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2',</span><br><span class="line">        <span class="meta"># hexo 博客存放 photos 信息的 json 文件</span></span><br><span class="line">        'photo_info_json':      '/Users/smaug/blog_rebootcat/source/photos/photoslist.json',</span><br><span class="line">        &#125;</span><br></pre></td></tr></table></figure><p>简单解释一下这个脚本：</p><ul><li>github_img_host_base： 这个目录也就是本地的仓库目录，绝对路径（上面克隆的仓库对应的本地文件夹路径）</li><li>img_path: 我单独新建了 rebootcat/photowall 目录存放瀑布流图片，对应本地的路径</li><li>cdn_url_prefix：jsdelivr cdn url 前缀，只需要更改成你自己的github 用户名以及仓库名</li><li>photo_info_json： photoslist.json 路径</li></ul><p>上面几个参数一定要配置对了。</p><p>那么简单解释一下脚本的功能：</p><p><strong>脚本会递归的查找 img_path 目录下的图片，然后进行一定的压缩（99%），这里的压缩目的并非真的是压缩，而是为了去除一些敏感信息，比如 GPS 信息。注意这里会覆盖掉原始图片。然后会生成图片的缩略图，同时根据上面的几个配置参数，生成两个 cdn url，一个对应的是原始图片的 cdn url，一个是缩略图的 cdn url</strong>.</p><p>然后执行：</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">python</span> phototool.<span class="keyword">py</span></span><br></pre></td></tr></table></figure><p>脚本执行完，就会<strong>增量生成</strong> photoslist.json，可以先打开检查下对不对，或者把里面的 cdn url 复制出来从浏览器看能不能访问。</p><blockquote><p><strong>注意需要把本地图片仓库推送到远程</strong>。</p></blockquote><p>这个 phototool.py 脚本你可以随便放在哪里，当你更新图片之后重新执行一遍就可以了。当然你也可以像我一样，跟网站源码直接放一起，所以你可以看到，我直接放到了 js 目录。</p><h2 id="更新图片"><a href="#更新图片" class="headerlink" title="更新图片"></a>更新图片</h2><p>把新图片放到本地仓库，然后执行：</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">python</span> phototool.<span class="keyword">py</span></span><br></pre></td></tr></table></figure><p>检查一下 photoslist.json 文件对不对，然后发布博客：</p><figure class="highlight ebnf"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">hexo d -g</span></span><br></pre></td></tr></table></figure><p>发布之后，记得把本地图片仓库推送到远端，不然 jsdelivr 无法访问到。</p><p>至此，一个相册瀑布流就制作完成了！</p><h1 id="The-End"><a href="#The-End" class="headerlink" title="The End"></a>The End</h1><p>由于我是采用回忆的方式来写的博文，所以文中可能会有一些小的修改或者配置我忽略了，不过问题不大，大家如果碰到问题了可以自行研究一下，能解决的。</p><p>采用 github 作为图床来存放大量的瀑布流图片墙，方案是没问题的，只不过可能由于仓库容量的限制，需要在 github 上构建多个图片仓库。</p><p>对于我来说，github 图片仓库主要用来存放博文中涉及到的图片。至于图片墙，我再另想办法吧。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#110;&#117;&#x78;&#99;&#111;&#x64;&#x65;&#x32;&#x6e;&#x69;&#x6b;&#105;&#x40;&#103;&#x6d;&#97;&#105;&#108;&#46;&#99;&#111;&#109;">&#108;&#x69;&#110;&#117;&#x78;&#99;&#111;&#x64;&#x65;&#x32;&#x6e;&#x69;&#x6b;&#105;&#x40;&#103;&#x6d;&#97;&#105;&#108;&#46;&#99;&#111;&#109;</a></p></li></ul><p>2020-09-19 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;一直没有时间来整理下博客搭建的一些事情，现在补上一篇，给 Hexo Next 博客添加一个相册功能，使用瀑布流的方式。&lt;/p&gt;
&lt;h1 id=&quot;原理说明&quot;&gt;&lt;a href=&quot;#原理说明&quot; class=&quot;headerlink&quot; title=&quot;原理说明&quot;&gt;&lt;/a&gt;原理说明&lt;/h1&gt;&lt;ul&gt;
&lt;li&gt;使用 github 作为仓库存储图片文件（图床）&lt;/li&gt;
&lt;li&gt;使用 jsdelivr 进行图片 CDN 加速&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;优点&quot;&gt;&lt;a href=&quot;#优点&quot; class=&quot;headerlink&quot; title=&quot;优点&quot;&gt;&lt;/a&gt;优点&lt;/h2&gt;&lt;p&gt;此种方式的优点是免费，不需要购买其他的对象存储产品；并且使用的是 github 作为图床，图片不会丢失。&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;早期的博文使用的是七牛云的免费存储，结果后来被他们删掉了。。。结果造成文中的一些图片链接都是 404，有兴趣的可以翻一翻我早期的博客。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;缺点&quot;&gt;&lt;a href=&quot;#缺点&quot; class=&quot;headerlink&quot; title=&quot;缺点&quot;&gt;&lt;/a&gt;缺点&lt;/h2&gt;&lt;p&gt;由于采用的是 github 仓库存储图片，但是 github 对单仓库有 50MB 的大小限制，所以单仓库可能不能够存储太多的文件；&lt;/p&gt;
&lt;p&gt;解决方法就是建立很多的图片仓库（稍微有点费劲，不过是行得通的）；另外上传的单张图片大小最好不要太大。&lt;/p&gt;
&lt;p&gt;还有个缺点就是得折腾啊，且看我后文。&lt;/p&gt;
&lt;p&gt;各位可以参考下我的相册瀑布流: &lt;a href=&quot;https://rebootcat.com/photos/&quot;&gt;摄影&lt;/a&gt;&lt;/p&gt;
&lt;h1 id=&quot;开始搭建相册瀑布流&quot;&gt;&lt;a href=&quot;#开始搭建相册瀑布流&quot; class=&quot;headerlink&quot; title=&quot;开始搭建相册瀑布流&quot;&gt;&lt;/a&gt;开始搭建相册瀑布流&lt;/h1&gt;&lt;p&gt;开始之前，需要简单介绍一下，我参考的是 &lt;a href=&quot;https://blog.dlzhang.com/posts/31/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Hexo NexT 博客增加瀑布流相册页面&lt;/a&gt; 这篇文章，文中涉及到的脚本主要都是 js 实现；与他不同的是，由于我对 js 的掌握远远不及我对 Python 的掌握，故部分脚本我采用了 Python 实现。&lt;/p&gt;
&lt;p&gt;所以在开始操作之前，你可以根据自己的技能，选择不同的方式。如果你擅长 python，那么跟着我来吧。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/nextphotowall/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
    
    </summary>
    
    
      <category term="blog" scheme="https://rebootcat.com/categories/blog/"/>
    
    
      <category term="python" scheme="https://rebootcat.com/tags/python/"/>
    
      <category term="hexo" scheme="https://rebootcat.com/tags/hexo/"/>
    
      <category term="blog" scheme="https://rebootcat.com/tags/blog/"/>
    
      <category term="cdn" scheme="https://rebootcat.com/tags/cdn/"/>
    
      <category term="github" scheme="https://rebootcat.com/tags/github/"/>
    
      <category term="next" scheme="https://rebootcat.com/tags/next/"/>
    
      <category term="jsdelivr" scheme="https://rebootcat.com/tags/jsdelivr/"/>
    
  </entry>
  
  <entry>
    <title>对区块链行业现状的一点看法</title>
    <link href="https://rebootcat.com/2020/09/16/think_about_blockchain/"/>
    <id>https://rebootcat.com/2020/09/16/think_about_blockchain/</id>
    <published>2020-09-16T12:23:58.000Z</published>
    <updated>2020-09-16T12:16:26.361Z</updated>
    
    <content type="html"><![CDATA[<h1 id="对行业现状的一点看法"><a href="#对行业现状的一点看法" class="headerlink" title="对行业现状的一点看法"></a>对行业现状的一点看法</h1><p>2020 年，新冠肆虐。</p><p>最近对于区块链的想法有点消极。简单谈一下。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/think_about_blockchain/1.png" alt=""></p><p>纵观整个区块链行业，公链项目死了很多，还活着的也在拖着，除了少数明星公链。也许未来的几年之内，这种状况还会持续下去，对于公链项目来说，要么死，要么拖下去再死，要么成为明星。无论哪一条路，都异常难！</p><p>区块链目前能解决的问题范围依然还是比较局限，一方面受限于技术层面，一方面受限于政治层面。技术方面的难点，在于架构，在于算法设计，在于安全，在于通信。</p><p>架构上目前行业普遍追求可扩展的架构，这样的目的在于提高 TPS，为未来可能存在的真实大量业务提供服务，当然必然为此牺牲去中心化属性，牺牲安全属性，抉择就在于追求什么目标？从比特币到以太坊，再到 EOS，再到各种分片的公链，可以理解为逐步为了高 TPS 改良，因为比特币的交易速度实在是太慢了，为此我们势必要有一条满足我们交易需求的公链，至于这个交易频率需要多快，也许可以对标中心化得出答案。</p><p>比如 visa 的 TPS 在 1000 ~ 2000，银联在 2000 左右， paypal 600 ~ 1000等等。这里就不得不提出一个疑问，对于公链来说，追求高 TPS 是不是一个伪需求？</p><p>算法层面，为了保证零信任网络内部达成一致共识，会涉及到大量的收发包以及加解密过程，然而至今没有一条公链的共识算法能称得上权威或者标杆，整个行业仍然处于研究阶段，不同的公链之间互相参考学习，然后进行创新以及试验。</p><p>这个过程可能很漫长，需要大量的人才投入贡献。这也就意味着现阶段的公链，至少在共识算法层面，不能达到一个质的飞越。也许在未来，还得依靠其他的手段做改良。</p><a id="more"></a><p>安全层面，相较于中心化的系统，基本可以理解为系统是直接暴露在外网上，但是目前来说，中心化的系统里也越来越多出现云系统，一定程度也直接暴露在外网上。除了这点，更为严重的是，公链代码是需要开源的，也就意味着可以随意修改代码进入网络，这对安全提出了很大的要求。另外，公链上直接跑的就是金钱，也就会成为各路黑客的重点攻击对象。从比特币、以太坊，到不知名的公链，均发生过不下一起的安全攻击事件，有的直接分叉，有的直接倒闭，有的币价腰斩，有的让用户、交易所损失惨重。</p><p>通信上，通信层面受限于真实的、不确定的、波动的网络环境，并且由于算法的设计，会带来大量的广播，一条消息进入网络就会变成 n 倍的消息量。也许随着 5G 技术的发展，在通信层面能够彻底解决这些问题，但目前来说，5G 的大规模落地还有很大的不确定性。</p><p>政治层面的局限就太多了，毕竟去中心化金融系统挑战的是集权的中央政府，比特币发展 10 年，好像依然只有少数国家是承认比特币的合法性的，比特币期间还被多次宣布死亡。</p><p>对于区块链来说，真可谓路漫漫其修远兮，未来依然充满了巨大的挑战以及极大的不确定性，不过我们依然也可以说变数中才存在乘风破浪的机会，就看从哪个角度看了。</p><p>另外，区块链的技术圈和炒币圈似乎是割裂开来的，我本身是比较反对这种方式的，炒币的热度远远超过技术本身的热度，毕竟人都是逐利的，由此，一方面会给区块链的技术发展带来一些积极正面的影响和热度，让更多的人认识到区块链，加入到这个区块链的发展中来；但另外一方面，炒币的人似乎也压根不关心技术的东西，他们需要的只是几倍几倍的涨幅，也许项目方随便吹吹牛皮，拉几位 “大佬” 站台，他们就能往里投钱，而压根不关心项目的技术，由此造成了各式各样的骗局，反过来让真的想了解区块链技术，对区块链技术感兴趣的人对此失去兴趣，形成恶性循环。而现在，整个币圈基本是这样！</p><p>这不得不让人反思，区块链到底解决了什么？是否只是沦为资本的工具？还是一直在鼓吹伪命题伪需求？</p><p>前几年，听到最多个一个词就是 “信仰”，对区块链的信仰，对比特币的信仰。但是对于一个技术来说，信仰这个词是不是稍微有点沉重和多余？ 2000年 的互联网技术，可惜无缘参与，不知道当时是否有很多人张口闭口一个 “信仰”，虽然无从得知，但我可以大胆猜测一下，应该是没有这样的现象的。</p><p>很多人把区块链技术和互联网技术做类比，比较两者的发展过程，由此得出一些结论，比如目前区块链行业的状态就是 2000 年的互联网，充满了寒冬、泡沫，但是再看看今天的互联网怎么样呢？由此说明区块链会成为下一个互联网。</p><p>我不否认一定程度上是对的，我们无法预估未来的发展，但是至少我们要对未来充满期待，对未来充满信念，而非 “信仰”。也许这个词多半是币圈流传，用来忽悠新人接盘的。但是把区块链和币圈割裂开来的方式似乎也是不妥，如果没有炒币这波人，也许区块链没有这么高的热度，但是就是这波炒币的人，让区块链行业乌烟瘴气，骗局和圈钱时有发生。所以，”信仰”这个词，真的没必要用在区块链上。区块链也许会改变世界，也许不会，它就是一个技术。</p><p>再回到当前，区块链行业追求的新名词也早已从公链、TPS、分片变成了 filecoin/ipfs、defi,真是瞬息万变。但是最近，也陆陆续续听说很多 defi 维权的新闻，至于 ipfs，沉寂了五六年时间，今年加上激励层filecoin 瞬间成为整个币圈的人都在谈论的明星项目，但是为什么这么火热呢？因为大家都在说这是第二个比特币，错过了比特币的挖矿，不要错过 ipfs 的挖矿。然后各种矿机卖的火热，参与的矿工也越来越多，官方还搞了竞赛，进一步助推了这波热度。</p><p>另外，观察一个现象，ipfs 是国外的项目，但在中国异常火热，这背后是否有一些幕后操手呢？再说 filecoin 主网，竞赛过程中 bug 不断，很明显内部没有经过比较系统的测试就赶鸭子上架的方式宣布即将主网上线，并开启挖矿竞赛。。。被资本裹挟的迹象很重，当然对于项目方来首，错过了这个热度，也许就永远没有机会了。于是不得不被资本裹挟。</p><p>这纯属我个人的阴谋论，不要在意。我想表达的是，区块链越来越沦为资本的工具，区块链从业者如何探索出一条正道迫在眉睫！</p><p>说了很多题外话，简单总结一下就是：</p><ol><li><strong>公链的困境，要么死，要么拖着，再死，要么成为明星，每一条道路都极其艰难</strong>；</li><li><strong>区块链技术还有很多挑战，需要大量的牛逼的人才进入，攻克这些难题</strong>；</li><li><strong>区块链的很多需求和追求也许是毫无意义的</strong>；</li><li><strong>区块链和币圈千丝万缕的联系，让这个行业高光的同时也为这个行业带来了极其严重的负面影响</strong>；</li><li><strong>区块链越来越沦为资本的工具，从业者探索出一条光明正道迫在眉睫</strong>；</li></ol><h1 id="前路何方？"><a href="#前路何方？" class="headerlink" title="前路何方？"></a>前路何方？</h1><p>路漫漫其修远兮，区块链也许能改变世界，也许不能！</p><h1 id="Blog"><a href="#Blog" class="headerlink" title="Blog"></a>Blog</h1><ul><li><a href="http://rebootcat.com">rebootcat.com</a></li><li>email: <a href="mailto:&#108;&#x69;&#110;&#x75;&#x78;&#x63;&#x6f;&#x64;&#x65;&#x32;&#x6e;&#105;&#107;&#105;&#64;&#103;&#109;&#97;&#105;&#x6c;&#x2e;&#99;&#111;&#109;">&#108;&#x69;&#110;&#x75;&#x78;&#x63;&#x6f;&#x64;&#x65;&#x32;&#x6e;&#105;&#107;&#105;&#64;&#103;&#109;&#97;&#105;&#x6c;&#x2e;&#99;&#111;&#109;</a></li></ul><p>2020-09-16 于杭州</p><p><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;对行业现状的一点看法&quot;&gt;&lt;a href=&quot;#对行业现状的一点看法&quot; class=&quot;headerlink&quot; title=&quot;对行业现状的一点看法&quot;&gt;&lt;/a&gt;对行业现状的一点看法&lt;/h1&gt;&lt;p&gt;2020 年，新冠肆虐。&lt;/p&gt;
&lt;p&gt;最近对于区块链的想法有点消极。简单谈一下。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/think_about_blockchain/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
&lt;p&gt;纵观整个区块链行业，公链项目死了很多，还活着的也在拖着，除了少数明星公链。也许未来的几年之内，这种状况还会持续下去，对于公链项目来说，要么死，要么拖下去再死，要么成为明星。无论哪一条路，都异常难！&lt;/p&gt;
&lt;p&gt;区块链目前能解决的问题范围依然还是比较局限，一方面受限于技术层面，一方面受限于政治层面。技术方面的难点，在于架构，在于算法设计，在于安全，在于通信。&lt;/p&gt;
&lt;p&gt;架构上目前行业普遍追求可扩展的架构，这样的目的在于提高 TPS，为未来可能存在的真实大量业务提供服务，当然必然为此牺牲去中心化属性，牺牲安全属性，抉择就在于追求什么目标？从比特币到以太坊，再到 EOS，再到各种分片的公链，可以理解为逐步为了高 TPS 改良，因为比特币的交易速度实在是太慢了，为此我们势必要有一条满足我们交易需求的公链，至于这个交易频率需要多快，也许可以对标中心化得出答案。&lt;/p&gt;
&lt;p&gt;比如 visa 的 TPS 在 1000 ~ 2000，银联在 2000 左右， paypal 600 ~ 1000等等。这里就不得不提出一个疑问，对于公链来说，追求高 TPS 是不是一个伪需求？&lt;/p&gt;
&lt;p&gt;算法层面，为了保证零信任网络内部达成一致共识，会涉及到大量的收发包以及加解密过程，然而至今没有一条公链的共识算法能称得上权威或者标杆，整个行业仍然处于研究阶段，不同的公链之间互相参考学习，然后进行创新以及试验。&lt;/p&gt;
&lt;p&gt;这个过程可能很漫长，需要大量的人才投入贡献。这也就意味着现阶段的公链，至少在共识算法层面，不能达到一个质的飞越。也许在未来，还得依靠其他的手段做改良。&lt;/p&gt;
    
    </summary>
    
    
      <category term="blockchain" scheme="https://rebootcat.com/categories/blockchain/"/>
    
    
      <category term="blockchain" scheme="https://rebootcat.com/tags/blockchain/"/>
    
  </entry>
  
  <entry>
    <title>cmake教程|cmake入门实战</title>
    <link href="https://rebootcat.com/2020/09/02/cmake/"/>
    <id>https://rebootcat.com/2020/09/02/cmake/</id>
    <published>2020-09-02T03:23:58.000Z</published>
    <updated>2020-11-12T14:25:57.996Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>我是一个 linux c++ 开发者，但是一直对 Makefile 的语法很是头痛，每次都记不住，所以每次写 Makefile 都很痛苦，Makefile 里需要你自己编写依赖和推导规则，这个过程能不能简单点呢？</p><p>对于编译一个 C++ 工程来说，也许需要的就是头文件路径、库路径、编译参数，剩下的东西基本也不重要，这三样足够去编译一个工程了。所以有没有一个工具能简单点的去实现 C++ 项目的构建呢？</p><p>答案是有的，上一篇博文 <a href="http://rebootcat.com/2020/08/30/scons/">scons构建C++项目</a> 介绍了 使用 scons 来构建 C++ 项目，大大提高了编写构建脚本的效率，使用起来也极为方便，对于熟悉 python 的童鞋来说真的是大大的福音；但 scons 的问题就是在大型项目的时候构建起来可能会很慢（听说的）。那么有没有其他的工具呢？</p><p>当然有，cmake 就是这样的一个工具，既能满足跨平台的编译，并且屏蔽了 Makefile 蛋疼的语法，使用一种更加简单的语法编写构建脚本，用在大型项目也毫无压力。</p><p>当然，对于我个人来说，cmake 的使用还是有门槛的，刚接触 cmake 可能还是会被它的语法搞的头疼（cmake 的语法也还是挺折腾的）。但是别急，沉下心来，本篇博文就带你从 cmake 入门到编写一个复杂工程的实战。</p><h1 id="CMake"><a href="#CMake" class="headerlink" title="CMake"></a>CMake</h1><h2 id="什么是-cmake"><a href="#什么是-cmake" class="headerlink" title="什么是 cmake"></a>什么是 cmake</h2><p>这里直接引用官网的解释：</p><blockquote><p>CMake is an open-source, cross-platform family of tools designed to build, test and package software. CMake is used to control the software compilation process using simple platform and compiler independent configuration files, and generate native makefiles and workspaces that can be used in the compiler environment of your choice. The suite of CMake tools were created by Kitware in response to the need for a powerful, cross-platform build environment for open-source projects such as ITK and VTK.</p></blockquote><p>CMake 是一个开源的跨平台的构建工具，语法简单，编译独立，并且很多知名大型项目也在用 CMake,比如 KDE、Netflix 、ReactOS等。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cmake/1.png" alt=""></p><a id="more"></a><p>OK，话不多说，如何使用呢？</p><h2 id="安装-cmake"><a href="#安装-cmake" class="headerlink" title="安装 cmake"></a>安装 cmake</h2><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">sudo yum <span class="keyword">install</span>  cmake3.x86_64</span><br></pre></td></tr></table></figure><p>现在最新版的 cmake 已经到 3.18.2 了。我使用的是 3.17.2 版本。</p><figure class="highlight applescript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ cmake <span class="comment">--version</span></span><br><span class="line">cmake <span class="built_in">version</span> <span class="number">3.17</span><span class="number">.2</span></span><br><span class="line"></span><br><span class="line">CMake suite maintained <span class="keyword">and</span> supported <span class="keyword">by</span> Kitware (kitware.com/cmake).</span><br></pre></td></tr></table></figure><h2 id="初识-cmake"><a href="#初识-cmake" class="headerlink" title="初识 cmake"></a>初识 cmake</h2><p>注：<strong>本文以一个多源文件，多目录结构的项目 <a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">mux</a> 为例，介绍 cmake 的使用，相关源文件以及cmake 脚本可以直接查看源项目</strong>。</p><p>使用 cmake 来构建 C++ 项目，需要先编写 cmake 构建脚本，文件名为  CMakeLists.txt，项目顶层目录需要放一个 CMakeLists.txt，同时子目录可以根据需要放置 CMakeLists.txt。</p><p>那么先来看看 CMakeLists.txt 长啥样?</p><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">cmake_minimum_required</span>(VERSION <span class="number">3.8</span>.<span class="number">0</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">set</span>(CMAKE_CXX_STANDARD <span class="number">11</span>)</span><br><span class="line"><span class="keyword">set</span>(CMAKE_CXX_STANDARD_REQUIRED <span class="keyword">ON</span>)</span><br><span class="line"><span class="keyword">set</span>(CMAKE_CXX_EXTENSIONS <span class="keyword">OFF</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">set</span>(CMAKE_C_STANDARD <span class="number">99</span>)</span><br><span class="line"><span class="keyword">set</span>(CMAKE_C_STANDARD_REQUIRED <span class="keyword">ON</span>)</span><br><span class="line"><span class="keyword">set</span>(CMAKE_C_EXTENSIONS <span class="keyword">OFF</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">project</span>(MUX CXX C)</span><br><span class="line"></span><br><span class="line"><span class="keyword">add_definitions</span>(</span><br><span class="line">    -DTEST1   <span class="comment"># define marco</span></span><br><span class="line">    -DTEST2   <span class="comment"># define marco</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="comment"># common compiling options</span></span><br><span class="line"><span class="keyword">add_compile_options</span>(</span><br><span class="line">    -Wl,--no-as-needed</span><br><span class="line">    -fno-strict-aliasing</span><br><span class="line">    -fthreadsafe-statics</span><br><span class="line">    -pthread</span><br><span class="line">    <span class="comment">#-fstack-protector-strong</span></span><br><span class="line">    -fno-short-enums</span><br><span class="line">    -fPIC</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">option</span>(XENABLE_TEST3  <span class="string">"enable test3 marco"</span> <span class="keyword">OFF</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">set</span>(EXECUTABLE_OUTPUT_PATH <span class="variable">$&#123;MUX_BINARY_DIR&#125;</span>/bin)</span><br><span class="line"><span class="keyword">set</span>(LIBRARY_OUTPUT_PATH <span class="variable">$&#123;MUX_BINARY_DIR&#125;</span>/lib)</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (XENABLE_TEST3)</span><br><span class="line">    <span class="keyword">add_definitions</span>(-DTEST3)</span><br><span class="line"><span class="keyword">endif</span>()</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (<span class="keyword">NOT</span> CMAKE_BUILD_TYPE)</span><br><span class="line">    <span class="keyword">set</span>(CMAKE_BUILD_TYPE Debug)</span><br><span class="line"><span class="keyword">endif</span>()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">message</span>(STATUS <span class="string">"CMAKE_BUILD_TYPE:"</span> <span class="variable">$&#123;CMAKE_BUILD_TYPE&#125;</span>)</span><br><span class="line"><span class="keyword">message</span>(STATUS <span class="string">"CMAKE_SYSTEM_NAME:"</span> <span class="variable">$&#123;CMAKE_SYSTEM_NAME&#125;</span>)</span><br><span class="line"><span class="keyword">message</span>(STATUS <span class="string">"XENABLE_TEST3:"</span> <span class="variable">$&#123;XENABLE_TEST3&#125;</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">find_package</span>(Threads REQUIRED)</span><br><span class="line"></span><br><span class="line"><span class="comment"># include header dirs</span></span><br><span class="line"><span class="keyword">include_directories</span>(<span class="variable">$&#123;CMAKE_SOURCE_DIR&#125;</span>)   <span class="comment"># project dir</span></span><br><span class="line"><span class="keyword">include_directories</span>(<span class="variable">$&#123;CMAKE_SOURCE_DIR&#125;</span>/third-party/<span class="keyword">include</span>)   <span class="comment"># project dir</span></span><br><span class="line"><span class="keyword">include_directories</span>(<span class="variable">$&#123;CMAKE_CURRENT_BINARY_DIR&#125;</span>)  <span class="comment"># current CMakeLists.txt dir (including sub dir)</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># link lib dirs</span></span><br><span class="line"><span class="keyword">link_directories</span>(<span class="variable">$&#123;CMAKE_SOURCE_DIR&#125;</span>/third-party/lib)</span><br><span class="line"><span class="keyword">link_directories</span>(<span class="variable">$&#123;LIBRARY_OUTPUT_PATH&#125;</span>)  <span class="comment"># generate in building</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">add_subdirectory</span>(demo/bench)</span><br><span class="line"><span class="keyword">add_subdirectory</span>(demo/echo)</span><br><span class="line"><span class="keyword">add_subdirectory</span>(epoll)</span><br><span class="line"><span class="keyword">add_subdirectory</span>(mbase)</span><br><span class="line"><span class="keyword">add_subdirectory</span>(message_handle)</span><br><span class="line"><span class="keyword">add_subdirectory</span>(transport)</span><br></pre></td></tr></table></figure><p>完整的 CMakeLists.txt 见 <a href="https://github.com/smaugx/mux/blob/master/CMakeLists.txt" target="_blank" rel="noopener">我的github</a>，同时我也会以我的github项目 <a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">mux</a> 为例，介绍 cmake 的使用。</p><p>上面的 CMakeLists.txt 乍一看，好多内容，但是别慌，我们来一个个说。</p><h2 id="详解-cmake"><a href="#详解-cmake" class="headerlink" title="详解 cmake"></a>详解 cmake</h2><p>注意：<strong>cmake 的语法可以分为命令(函数）和参数。 命令不缺分大小写，参数区分大小写</strong>。</p><p>注意：<strong>cmake 的语法可以分为命令(函数）和参数。 命令不缺分大小写，参数区分大小写</strong>。</p><h3 id="设置-cmake-版本的要求"><a href="#设置-cmake-版本的要求" class="headerlink" title="设置 cmake 版本的要求"></a>设置 cmake 版本的要求</h3><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">cmake_minimum_required</span><span class="params">(VERSION <span class="number">3.8</span>.<span class="number">0</span>)</span></span></span><br></pre></td></tr></table></figure><h3 id="在-cmake-中设置-c-标准，启用-c-11-或以上-根据项目的需求来）"><a href="#在-cmake-中设置-c-标准，启用-c-11-或以上-根据项目的需求来）" class="headerlink" title="在 cmake 中设置 c++ 标准，启用 c++11 或以上(根据项目的需求来）"></a>在 cmake 中设置 c++ 标准，启用 c++11 或以上(根据项目的需求来）</h3><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">set</span><span class="params">(CMAKE_CXX_STANDARD <span class="number">11</span>)</span></span></span><br><span class="line"><span class="function"><span class="title">set</span><span class="params">(CMAKE_CXX_STANDARD_REQUIRED ON)</span></span></span><br><span class="line"><span class="function"><span class="title">set</span><span class="params">(CMAKE_CXX_EXTENSIONS OFF)</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">set</span><span class="params">(CMAKE_C_STANDARD <span class="number">99</span>)</span></span></span><br><span class="line"><span class="function"><span class="title">set</span><span class="params">(CMAKE_C_STANDARD_REQUIRED ON)</span></span></span><br><span class="line"><span class="function"><span class="title">set</span><span class="params">(CMAKE_C_EXTENSIONS OFF)</span></span></span><br></pre></td></tr></table></figure><h3 id="设置项目名以及项目语言"><a href="#设置项目名以及项目语言" class="headerlink" title="设置项目名以及项目语言"></a>设置项目名以及项目语言</h3><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">project</span><span class="params">(MUX CXX C)</span></span></span><br></pre></td></tr></table></figure><p>设置完项目名称之后，会自动创建两个变量 <code>&lt;PROJECT-NAME&gt;_SOURCE_DIR</code> 和 <code>&lt;PROJECT-NAME&gt;_BINARY_DIR</code>，对于 <a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">mux</a> 这个项目来说，也就是 <code>MUX_SOURCE_DIR</code> 和 <code>MUX_BINARY_DIR</code>。</p><p><code>MUX_SOURCE_DIR</code> 表示工程顶层目录； <code>MUX_BINARY_DIR</code> 表示 cmake 构建发生的目录。</p><p>因为你一定熟悉或者用过下面的命令或步骤：</p><figure class="highlight properties"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">mkdir</span> <span class="string">cbuild</span></span><br><span class="line"><span class="attr">cd</span> <span class="string">cbuild</span></span><br><span class="line"><span class="attr">cmake</span> <span class="string">..</span></span><br><span class="line"><span class="attr">make</span></span><br><span class="line"><span class="attr">make</span> <span class="string">test</span></span><br><span class="line"><span class="attr">make</span> <span class="string">install</span></span><br></pre></td></tr></table></figure><p>通常我们会单独新建一个  cbuild 目录，用来构建项目，并且存放过程中产生的文件。那么 cbuild 目录就是 <code>MUX_BINARY_DIR</code> 表示的目录，cbuild 的上一级目录也就是项目顶层目录就是 <code>MUX_SOURCE_DIR</code> 表示的目录。</p><blockquote><p>如果你没有单独新建 <code>cbuild</code> 目录，而是直接在项目顶层目录使用 <code>cmake .</code> ，那么上面两个变量均指项目顶层目录。</p></blockquote><p>详见 <a href="https://cmake.org/cmake/help/latest/command/project.html" target="_blank" rel="noopener">https://cmake.org/cmake/help/latest/command/project.html</a></p><h3 id="添加编译宏"><a href="#添加编译宏" class="headerlink" title="添加编译宏"></a>添加编译宏</h3><figure class="highlight cs"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">add_definitions(</span><br><span class="line">    -DTEST1   <span class="meta"># <span class="meta-keyword">define</span> marco</span></span><br><span class="line">    -DTEST2   <span class="meta"># <span class="meta-keyword">define</span> marco</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>上面是我随便写的两个宏 <code>TEST1</code> 和 <code>TEST2</code>，那么在c++代码中通常是这样的：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">ifdef</span> TEST1</span></span><br><span class="line">    <span class="comment">// do something about test1</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">endif</span></span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">ifdef</span> TEST2</span></span><br><span class="line">   <span class="comment">// do something about test2</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">endif</span></span></span><br></pre></td></tr></table></figure><p>当然要开启这个宏也可以不用写在 CMakeLists.txt 文件中，可以直接这样使用：</p><figure class="highlight dos"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">mkdir</span> cbuild &amp;&amp; <span class="built_in">cd</span> cbuild </span><br><span class="line">cmake .. -DTEST1</span><br></pre></td></tr></table></figure><p>这个根据你的项目需求来操作。</p><h3 id="定义一些用户自定义的可选项"><a href="#定义一些用户自定义的可选项" class="headerlink" title="定义一些用户自定义的可选项"></a>定义一些用户自定义的可选项</h3><figure class="highlight lisp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">option(<span class="name">XENABLE_TEST3</span>  <span class="string">"enable test3 marco"</span> OFF)</span><br><span class="line"></span><br><span class="line">if (<span class="name">XENABLE_TEST3</span>)</span><br><span class="line">    add_definitions(<span class="name">-DTEST3</span>)</span><br><span class="line">endif()</span><br><span class="line"></span><br><span class="line">if (<span class="name">NOT</span> CMAKE_BUILD_TYPE)</span><br><span class="line">    set(<span class="name">CMAKE_BUILD_TYPE</span> Debug)</span><br><span class="line">endif()</span><br></pre></td></tr></table></figure><p>使用 option 命令可以自定义一些变量的值，作为一些条件判断的开关很方便。</p><p>详见 <a href="https://cmake.org/cmake/help/latest/command/option.html" target="_blank" rel="noopener">https://cmake.org/cmake/help/latest/command/option.html</a></p><h3 id="添加编译选项"><a href="#添加编译选项" class="headerlink" title="添加编译选项"></a>添加编译选项</h3><figure class="highlight haml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"># common compiling options</span><br><span class="line">add_compile_options(</span><br><span class="line">    -<span class="ruby">Wl,--no-as-needed</span></span><br><span class="line"><span class="ruby">    -fno-strict-aliasing</span></span><br><span class="line"><span class="ruby">    -fthreadsafe-statics</span></span><br><span class="line"><span class="ruby">    -pthread</span></span><br><span class="line"><span class="ruby">    <span class="comment">#-fstack-protector-strong</span></span></span><br><span class="line"><span class="ruby">    -fno-short-enums</span></span><br><span class="line"><span class="ruby">    -fPIC</span></span><br><span class="line"><span class="ruby">)</span></span><br></pre></td></tr></table></figure><p>这里就是一些编译选项，根据自己的项目需求修改。</p><h3 id="设置编译二进制-binary-executable-和-binary-lib）存放路径"><a href="#设置编译二进制-binary-executable-和-binary-lib）存放路径" class="headerlink" title="设置编译二进制(binary-executable 和 binary-lib）存放路径"></a>设置编译二进制(binary-executable 和 binary-lib）存放路径</h3><figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">set(EXECUTABLE_OUTPUT_PATH $&#123;MUX_BINARY_DIR&#125;/bin)</span><br><span class="line">set(LIBRARY_OUTPUT_PATH $&#123;MUX_BINARY_DIR&#125;/<span class="class"><span class="keyword">lib</span>)</span></span><br></pre></td></tr></table></figure><p>可以看到上面用到了 <code>MUX_BINARY_DIR</code> 这个变量，也就是说最终编译出来的二进制程序和lib 库会存放在 <code>cbuild/bin</code> 和 <code>cbuild/lib</code> 中。</p><h3 id="打印一些信息到终端"><a href="#打印一些信息到终端" class="headerlink" title="打印一些信息到终端"></a>打印一些信息到终端</h3><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">message</span><span class="params">(STATUS <span class="string">"CMAKE_BUILD_TYPE:"</span> $&#123;CMAKE_BUILD_TYPE&#125;)</span></span></span><br><span class="line"><span class="function"><span class="title">message</span><span class="params">(STATUS <span class="string">"CMAKE_SYSTEM_NAME:"</span> $&#123;CMAKE_SYSTEM_NAME&#125;)</span></span></span><br><span class="line"><span class="function"><span class="title">message</span><span class="params">(STATUS <span class="string">"XENABLE_TEST3:"</span> $&#123;XENABLE_TEST3&#125;)</span></span></span><br></pre></td></tr></table></figure><p>打印一些调试信息，或者编译信息到终端，使用的是 message 命令。</p><p>详见 <a href="https://cmake.org/cmake/help/latest/command/message.html" target="_blank" rel="noopener">https://cmake.org/cmake/help/latest/command/message.html</a>。</p><h3 id="设置头文件路径"><a href="#设置头文件路径" class="headerlink" title="设置头文件路径"></a>设置头文件路径</h3><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"># <span class="keyword">include</span> header dirs</span><br><span class="line"><span class="keyword">include</span><span class="constructor">_directories($&#123;CMAKE_SOURCE_DIR&#125;)</span>   # project dir</span><br><span class="line"><span class="keyword">include</span><span class="constructor">_directories($&#123;CMAKE_SOURCE_DIR&#125;<span class="operator">/</span><span class="params">third</span>-<span class="params">party</span><span class="operator">/</span><span class="params">include</span>)</span>   # project dir</span><br><span class="line"><span class="keyword">include</span><span class="constructor">_directories($&#123;CMAKE_CURRENT_BINARY_DIR&#125;)</span>  # current <span class="module-access"><span class="module"><span class="identifier">CMakeLists</span>.</span></span>txt dir (including sub dir)</span><br></pre></td></tr></table></figure><p>分别解释一下：</p><p><code>CMAKE_SOURCE_DIR</code> 表示工程顶层目录，也就是 <code>MUX_SOURCE_DIR</code>；</p><p><code>CMAKE_CURRENT_BINARY_DIR</code> 表示当前处理的 CMakeLists.txt 所在的目录，对于子目录中的 CMakeLists.txt 来说，即表示这个子目录。</p><p>通常这两个是常用的，必须的。然后使用 <code>include_directories</code> 命令包含其他的一些头文件路径。</p><h3 id="设置依赖库的路径"><a href="#设置依赖库的路径" class="headerlink" title="设置依赖库的路径"></a>设置依赖库的路径</h3><figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># link lib dirs</span></span><br><span class="line">link_directories($&#123;CMAKE_SOURCE_DIR&#125;/third-party/<span class="class"><span class="keyword">lib</span>)</span></span><br><span class="line">link_directories($&#123;LIBRARY_OUTPUT_PATH&#125;)  <span class="comment"># generate in building</span></span><br></pre></td></tr></table></figure><p><code>LIBRARY_OUTPUT_PATH</code> 就是上面设置的编译目标二进制库的存放路径，因为实际项目中，子模块之间可能会有一些依赖，子模块单独编译成一个库，然后让其他模块链接。这个目录也就是 <code>cbuild/lib</code> 目录。</p><h3 id="引入子模块-子目录）"><a href="#引入子模块-子目录）" class="headerlink" title="引入子模块(子目录）"></a>引入子模块(子目录）</h3><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">add_subdirectory</span><span class="params">(demo/bench)</span></span></span><br><span class="line"><span class="function"><span class="title">add_subdirectory</span><span class="params">(demo/echo)</span></span></span><br><span class="line"><span class="function"><span class="title">add_subdirectory</span><span class="params">(epoll)</span></span></span><br><span class="line"><span class="function"><span class="title">add_subdirectory</span><span class="params">(mbase)</span></span></span><br><span class="line"><span class="function"><span class="title">add_subdirectory</span><span class="params">(message_handle)</span></span></span><br><span class="line"><span class="function"><span class="title">add_subdirectory</span><span class="params">(transport)</span></span></span><br></pre></td></tr></table></figure><p>使用 <code>add_subdirectory</code> 命令把子模块包含进来，必须确保每个子目录下面有一个 CMakeLists.txt 文件，不然会报错。</p><p><strong>以上就是工程顶层目录的 CMakeLists.txt 的内容，分析下来是不是很清楚呢</strong>？</p><p>那么工程顶层目录的 CMakeLists.txt 其实做的事情就是设置一些基本的变量，宏开关，编译参数，头文件路径，依赖库路径，编译目标保存路径等等，子目录中的 CMakeLists.txt 才是真正产生编译目标的（exe和lib)。</p><h3 id="生成静态库-动态库"><a href="#生成静态库-动态库" class="headerlink" title="生成静态库/动态库"></a>生成静态库/动态库</h3><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># keep all cpp files in varibale $&#123;epoll_src&#125;</span></span><br><span class="line"><span class="keyword">aux_source_directory</span>(./src epoll_src)</span><br><span class="line"></span><br><span class="line"><span class="keyword">add_library</span>(epoll STATIC <span class="variable">$&#123;epoll_src&#125;</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">add_dependencies</span>(epoll mbase )</span><br><span class="line"><span class="keyword">target_link_libraries</span>(epoll mbase pthread)</span><br></pre></td></tr></table></figure><p>源文件在这：<a href="https://github.com/smaugx/mux/blob/master/epoll/CMakeLists.txt" target="_blank" rel="noopener">戳我</a></p><p>使用 <code>aux_source_directory</code> 添加源文件，相当于把 src 目录下的所有 c++ 文件保存到 <code>epoll_src</code> 这个变量中；</p><p>使用 <code>add_library</code> 生成目标库（根据需要可以生成静态库和动态库，分别使用 STATIC 和 SHARED)</p><p>然后就是添加这个模块需要依赖到的其他模块，以及链接参数。</p><p>上面的代码最终就会在 <code>cbuild/lib</code> 目录下生成一个 <code>libepoll.a</code> 文件。</p><h3 id="生成二进制可执行文件"><a href="#生成二进制可执行文件" class="headerlink" title="生成二进制可执行文件"></a>生成二进制可执行文件</h3><figure class="highlight reasonml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"># build target echo_server</span><br><span class="line">add<span class="constructor">_executable(<span class="params">echo_server</span> <span class="params">echo_server</span>.<span class="params">cc</span>)</span></span><br><span class="line">add<span class="constructor">_dependencies(<span class="params">echo_server</span> <span class="params">transport</span> <span class="params">msghandler</span> <span class="params">mbase</span>)</span></span><br><span class="line">target<span class="constructor">_link_libraries(<span class="params">echo_server</span> <span class="params">transport</span> <span class="params">msghandler</span> <span class="params">mbase</span>)</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"># build target echo_client</span><br><span class="line">add<span class="constructor">_executable(<span class="params">echo_client</span> <span class="params">client</span>.<span class="params">cc</span>)</span></span><br><span class="line">add<span class="constructor">_dependencies(<span class="params">echo_client</span> <span class="params">transport</span> <span class="params">msghandler</span> <span class="params">mbase</span>)</span></span><br><span class="line">target<span class="constructor">_link_libraries(<span class="params">echo_client</span> <span class="params">transport</span> <span class="params">msghandler</span> <span class="params">mbase</span>)</span></span><br></pre></td></tr></table></figure><p>源文件在这：<a href="https://github.com/smaugx/mux/blob/master/demo/echo/CMakeLists.txt" target="_blank" rel="noopener">戳我</a></p><p>和生成库大体是类似的，区别是使用的是 <code>add_executable</code> 这个命令。</p><p>其他子模块的 CMakeLists.txt 见<a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">我的github</a>.</p><h2 id="cmake-编译构建"><a href="#cmake-编译构建" class="headerlink" title="cmake 编译构建"></a>cmake 编译构建</h2><p>上面详细的介绍了 CMakeLists.txt 的写法，如果仿照本文，应该也能写出适合你项目的构建脚本，但是可能还不够，其他语法自行 google 学习。</p><p>上面其实是以我的项目 <a href=""></a> 进行的演示，有必要解读一下这个项目的结构层次：</p><figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line">$ tree mux -d</span><br><span class="line">mux</span><br><span class="line">├── demo</span><br><span class="line">│   ├── bench</span><br><span class="line">│   └── echo</span><br><span class="line">├── epoll</span><br><span class="line">│   ├── <span class="keyword">include</span></span><br><span class="line">│   └── src</span><br><span class="line">├── mbase</span><br><span class="line">│   └── src</span><br><span class="line">├── message_handle</span><br><span class="line">│   ├── <span class="keyword">include</span></span><br><span class="line">│   └── src</span><br><span class="line">├── third-party</span><br><span class="line">│   ├── <span class="keyword">include</span></span><br><span class="line">│   │   ├── nlohmann</span><br><span class="line">│   │   └── spdlog</span><br><span class="line">│   │       ├── cfg</span><br><span class="line">│   │       ├── details</span><br><span class="line">│   │       ├── fmt</span><br><span class="line">│   │       │   └── bundled</span><br><span class="line">│   │       └── sinks</span><br><span class="line">│   └── <span class="class"><span class="keyword">lib</span></span></span><br><span class="line">└── transport</span><br><span class="line">    ├── <span class="keyword">include</span></span><br><span class="line">    └── src</span><br><span class="line"></span><br><span class="line"><span class="number">24</span> directories</span><br></pre></td></tr></table></figure><p>mux 是工程顶层目录，下面包含的 <code>epoll</code>、<code>mbase</code>、<code>message_handle</code>、<code>transport</code> 这几个目录，均各自打包成一个静态库； <code>demo</code> 目录下分别包含 <code>bench</code> 和 <code>echo</code> 两个目录，这两个目录下需要构建可执行程序。</p><p>所以首先是<code>epoll</code>、<code>mbase</code>、<code>message_handle</code>、<code>transport</code> 这几个目录生成静态库，也就是最终会在 <code>cbuild/lib</code> 目录生成 <code>libepoll.a</code>,    <code>libmbase.a</code>, <code>libmsghandler.a</code>, <code>libtransport.a</code>, 然后 <code>bench</code> 和 <code>echo</code> 下的代码依赖于前面的几个模块，生成可执行程序。</p><p>前面其实已经提到了，基本的构建命令如下：</p><figure class="highlight properties"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">mkdir</span> <span class="string">cbuild</span></span><br><span class="line"><span class="attr">cd</span> <span class="string">cbuild</span></span><br><span class="line"><span class="attr">cmake</span> <span class="string">..</span></span><br><span class="line"><span class="attr">make</span> <span class="string">-j4</span></span><br></pre></td></tr></table></figure><p>其中注意，如果你没有单独构建 cbuild 目录的话，可能会生成一些中间临时文件污染了目录。并且注意，cmake 后面的 <code>..</code> 表示的是工程顶层的 CMakeLists.txt 的目录。所以如果直接使用的是工程顶层目录构建的话，就应该是 <code>cmake .</code></p><figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br></pre></td><td class="code"><pre><span class="line">$ cmake ..</span><br><span class="line">-- The CXX compiler identification is GNU <span class="number">4.8</span><span class="number">.5</span></span><br><span class="line">-- The C compiler identification is GNU <span class="number">4.8</span><span class="number">.5</span></span><br><span class="line">-- Check <span class="keyword">for</span> working CXX <span class="string">compiler:</span> <span class="regexp">/usr/</span>local<span class="regexp">/bin/</span>c++</span><br><span class="line">-- Check <span class="keyword">for</span> working CXX <span class="string">compiler:</span> <span class="regexp">/usr/</span>local<span class="regexp">/bin/</span>c++ - works</span><br><span class="line">-- Detecting CXX compiler ABI info</span><br><span class="line">-- Detecting CXX compiler ABI info - done</span><br><span class="line">-- Detecting CXX compile features</span><br><span class="line">-- Detecting CXX compile features - done</span><br><span class="line">-- Check <span class="keyword">for</span> working C <span class="string">compiler:</span> <span class="regexp">/usr/</span>local<span class="regexp">/bin/</span>gcc</span><br><span class="line">-- Check <span class="keyword">for</span> working C <span class="string">compiler:</span> <span class="regexp">/usr/</span>local<span class="regexp">/bin/</span>gcc - works</span><br><span class="line">-- Detecting C compiler ABI info</span><br><span class="line">-- Detecting C compiler ABI info - done</span><br><span class="line">-- Detecting C compile features</span><br><span class="line">-- Detecting C compile features - done</span><br><span class="line">-- <span class="string">CMAKE_BUILD_TYPE:</span>Debug</span><br><span class="line">-- <span class="string">CMAKE_SYSTEM_NAME:</span>Linux</span><br><span class="line">-- <span class="string">XENABLE_TEST3:</span>OFF</span><br><span class="line">-- Looking <span class="keyword">for</span> pthread.h</span><br><span class="line">-- Looking <span class="keyword">for</span> pthread.h - found</span><br><span class="line">-- Performing Test CMAKE_HAVE_LIBC_PTHREAD</span><br><span class="line">-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed</span><br><span class="line">-- Looking <span class="keyword">for</span> pthread_create <span class="keyword">in</span> pthreads</span><br><span class="line">-- Looking <span class="keyword">for</span> pthread_create <span class="keyword">in</span> pthreads - not found</span><br><span class="line">-- Looking <span class="keyword">for</span> pthread_create <span class="keyword">in</span> pthread</span><br><span class="line">-- Looking <span class="keyword">for</span> pthread_create <span class="keyword">in</span> pthread - found</span><br><span class="line">-- Found <span class="string">Threads:</span> TRUE</span><br><span class="line">-- Configuring done</span><br><span class="line">-- Generating done</span><br><span class="line">-- Build files have been written <span class="string">to:</span> <span class="regexp">/mnt/</span>centos-share<span class="regexp">/workspace/</span>mux/cbuild</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">$ make -j4</span><br><span class="line">Scanning dependencies of target mbase</span><br><span class="line">[  <span class="number">5</span>%] Building CXX object mbase<span class="regexp">/CMakeFiles/</span>mbase.dir<span class="regexp">/src/</span>packet.cc.o</span><br><span class="line">[ <span class="number">11</span>%] Linking CXX <span class="keyword">static</span> library ..<span class="regexp">/lib/</span>libmbase.a</span><br><span class="line">[ <span class="number">11</span>%] Built target mbase</span><br><span class="line">Scanning dependencies of target msghandler</span><br><span class="line">Scanning dependencies of target epoll</span><br><span class="line">[ <span class="number">17</span>%] Building CXX object message_handle<span class="regexp">/CMakeFiles/</span>msghandler.dir<span class="regexp">/src/</span>message_handler.cc.o</span><br><span class="line">[ <span class="number">23</span>%] Building CXX object epoll<span class="regexp">/CMakeFiles/</span>epoll.dir<span class="regexp">/src/</span>epoll_tcp_client.cc.o</span><br><span class="line">[ <span class="number">29</span>%] Building CXX object epoll<span class="regexp">/CMakeFiles/</span>epoll.dir<span class="regexp">/src/</span>epoll_tcp_server.cc.o</span><br><span class="line">[ <span class="number">35</span>%] Linking CXX <span class="keyword">static</span> library ..<span class="regexp">/lib/</span>libepoll.a</span><br><span class="line">[ <span class="number">41</span>%] Linking CXX <span class="keyword">static</span> library ..<span class="regexp">/lib/</span>libmsghandler.a</span><br><span class="line">[ <span class="number">41</span>%] Built target msghandler</span><br><span class="line">[ <span class="number">41</span>%] Built target epoll</span><br><span class="line">Scanning dependencies of target transport</span><br><span class="line">[ <span class="number">47</span>%] Building CXX object transport<span class="regexp">/CMakeFiles/</span>transport.dir<span class="regexp">/src/</span>tcp_transport.cc.o</span><br><span class="line">[ <span class="number">52</span>%] Linking CXX <span class="keyword">static</span> library ..<span class="regexp">/lib/</span>libtransport.a</span><br><span class="line">[ <span class="number">52</span>%] Built target transport</span><br><span class="line">Scanning dependencies of target echo_client</span><br><span class="line">Scanning dependencies of target echo_server</span><br><span class="line">Scanning dependencies of target bench_client</span><br><span class="line">Scanning dependencies of target bench_server</span><br><span class="line">[ <span class="number">58</span>%] Building CXX object demo<span class="regexp">/echo/</span>CMakeFiles<span class="regexp">/echo_client.dir/</span>client.cc.o</span><br><span class="line">[ <span class="number">64</span>%] Building CXX object demo<span class="regexp">/bench/</span>CMakeFiles<span class="regexp">/bench_client.dir/</span>client.cc.o</span><br><span class="line">[ <span class="number">70</span>%] Building CXX object demo<span class="regexp">/echo/</span>CMakeFiles<span class="regexp">/echo_server.dir/</span>echo_server.cc.o</span><br><span class="line">[ <span class="number">76</span>%] Building CXX object demo<span class="regexp">/bench/</span>CMakeFiles<span class="regexp">/bench_server.dir/</span>bench_server.cc.o</span><br><span class="line">[ <span class="number">82</span>%] Linking CXX executable ..<span class="regexp">/../</span>bin/echo_client</span><br><span class="line">[ <span class="number">88</span>%] Linking CXX executable ..<span class="regexp">/../</span>bin/echo_server</span><br><span class="line">[ <span class="number">94</span>%] Linking CXX executable ..<span class="regexp">/../</span>bin/bench_server</span><br><span class="line">[<span class="number">100</span>%] Linking CXX executable ..<span class="regexp">/../</span>bin/bench_client</span><br><span class="line">[<span class="number">100</span>%] Built target echo_client</span><br><span class="line">[<span class="number">100</span>%] Built target echo_server</span><br><span class="line">[<span class="number">100</span>%] Built target bench_client</span><br><span class="line">[<span class="number">100</span>%] Built target bench_server</span><br></pre></td></tr></table></figure><p>看看生成了啥：</p><figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ ls cbuild/bin/</span><br><span class="line">bench_client  bench_server  echo_client  echo_server</span><br><span class="line"></span><br><span class="line">$ ls cbuild/<span class="class"><span class="keyword">lib</span>/</span></span><br><span class="line">libepoll.a  libmbase.a  libmsghandler.a  libtransport.a</span><br></pre></td></tr></table></figure><p>Over!</p><h1 id="写在最后"><a href="#写在最后" class="headerlink" title="写在最后"></a>写在最后</h1><p>cmake 的构建其实认真熟悉之后，也还是能快速上手的，不要产生排斥心理，不然学起来就很慢很费劲。所以建议第一次接触 cmake 的或者以前一直抵触 cmake 的童鞋，静下心来，认认真真的看完本文或者其他的入门例子，那么你也能快速写一个多目录，多层次结构的 cmake 工程。</p><p>cmake 中其他的一些用法，建议随时查看官方的 <a href="https://cmake.org/cmake/help/latest/index.html" target="_blank" rel="noopener">cook book</a>.</p><p>加油，少年，别怕！</p><p>另外，文中涉及到的项目可以在<a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">我的github</a> 找到。</p><h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><p><a href="https://aiden-dong.github.io/2019/07/20/CMake%E6%95%99%E7%A8%8B%E4%B9%8BCMake%E4%BB%8E%E5%85%A5%E9%97%A8%E5%88%B0%E5%BA%94%E7%94%A8/" target="_blank" rel="noopener">CMake 教程 | CMake 从入门到应用</a></p><p><a href="https://juejin.im/post/6844903558861553672" target="_blank" rel="noopener">cmake使用教程</a></p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#110;&#117;&#120;&#99;&#111;&#x64;&#x65;&#50;&#110;&#105;&#x6b;&#x69;&#x40;&#x67;&#x6d;&#97;&#x69;&#x6c;&#46;&#x63;&#111;&#x6d;">&#108;&#x69;&#110;&#117;&#120;&#99;&#111;&#x64;&#x65;&#50;&#110;&#105;&#x6b;&#x69;&#x40;&#x67;&#x6d;&#97;&#x69;&#x6c;&#46;&#x63;&#111;&#x6d;</a></p></li></ul><p>2020-09-02 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;我是一个 linux c++ 开发者，但是一直对 Makefile 的语法很是头痛，每次都记不住，所以每次写 Makefile 都很痛苦，Makefile 里需要你自己编写依赖和推导规则，这个过程能不能简单点呢？&lt;/p&gt;
&lt;p&gt;对于编译一个 C++ 工程来说，也许需要的就是头文件路径、库路径、编译参数，剩下的东西基本也不重要，这三样足够去编译一个工程了。所以有没有一个工具能简单点的去实现 C++ 项目的构建呢？&lt;/p&gt;
&lt;p&gt;答案是有的，上一篇博文 &lt;a href=&quot;http://rebootcat.com/2020/08/30/scons/&quot;&gt;scons构建C++项目&lt;/a&gt; 介绍了 使用 scons 来构建 C++ 项目，大大提高了编写构建脚本的效率，使用起来也极为方便，对于熟悉 python 的童鞋来说真的是大大的福音；但 scons 的问题就是在大型项目的时候构建起来可能会很慢（听说的）。那么有没有其他的工具呢？&lt;/p&gt;
&lt;p&gt;当然有，cmake 就是这样的一个工具，既能满足跨平台的编译，并且屏蔽了 Makefile 蛋疼的语法，使用一种更加简单的语法编写构建脚本，用在大型项目也毫无压力。&lt;/p&gt;
&lt;p&gt;当然，对于我个人来说，cmake 的使用还是有门槛的，刚接触 cmake 可能还是会被它的语法搞的头疼（cmake 的语法也还是挺折腾的）。但是别急，沉下心来，本篇博文就带你从 cmake 入门到编写一个复杂工程的实战。&lt;/p&gt;
&lt;h1 id=&quot;CMake&quot;&gt;&lt;a href=&quot;#CMake&quot; class=&quot;headerlink&quot; title=&quot;CMake&quot;&gt;&lt;/a&gt;CMake&lt;/h1&gt;&lt;h2 id=&quot;什么是-cmake&quot;&gt;&lt;a href=&quot;#什么是-cmake&quot; class=&quot;headerlink&quot; title=&quot;什么是 cmake&quot;&gt;&lt;/a&gt;什么是 cmake&lt;/h2&gt;&lt;p&gt;这里直接引用官网的解释：&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;CMake is an open-source, cross-platform family of tools designed to build, test and package software. CMake is used to control the software compilation process using simple platform and compiler independent configuration files, and generate native makefiles and workspaces that can be used in the compiler environment of your choice. The suite of CMake tools were created by Kitware in response to the need for a powerful, cross-platform build environment for open-source projects such as ITK and VTK.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;CMake 是一个开源的跨平台的构建工具，语法简单，编译独立，并且很多知名大型项目也在用 CMake,比如 KDE、Netflix 、ReactOS等。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/cmake/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="cmake" scheme="https://rebootcat.com/tags/cmake/"/>
    
      <category term="CMakeLists.txt" scheme="https://rebootcat.com/tags/CMakeLists-txt/"/>
    
      <category term="scons" scheme="https://rebootcat.com/tags/scons/"/>
    
      <category term="c" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="makefile" scheme="https://rebootcat.com/tags/makefile/"/>
    
      <category term="compile" scheme="https://rebootcat.com/tags/compile/"/>
    
      <category term="build" scheme="https://rebootcat.com/tags/build/"/>
    
  </entry>
  
  <entry>
    <title>Scons构建C++项目</title>
    <link href="https://rebootcat.com/2020/08/30/scons/"/>
    <id>https://rebootcat.com/2020/08/30/scons/</id>
    <published>2020-08-30T03:23:58.000Z</published>
    <updated>2020-09-03T22:24:38.083Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>我是一个 linux c++ 开发者，但是一直对 Makefile 的语法很是头痛，每次都记不住，所以每次写 Makefile 都很痛苦，Makefile 里需要你自己编写依赖和推导规则，这个过程能不能简单点呢？</p><p>对于编译一个 C++ 工程来说，也许需要的就是头文件路径、库路径、编译参数，剩下的东西基本也不重要，这三样足够去编译一个工程了。所以有没有一个工具能简单点的去实现 C++ 项目的构建呢？</p><p>答案是有的，Scons 就是答案。</p><h1 id="Scons"><a href="#Scons" class="headerlink" title="Scons"></a>Scons</h1><h2 id="什么是-scons"><a href="#什么是-scons" class="headerlink" title="什么是 scons"></a>什么是 scons</h2><p>这里直接引用官网的解释：</p><blockquote><p>What is SCons?</p><blockquote><p>SCons is an Open Source software construction tool—that is, a next-generation build tool. Think of SCons as an improved, cross-platform substitute for the classic Make utility with integrated functionality similar to autoconf/automake and compiler caches such as ccache. In short, SCons is an easier, more reliable and faster way to build software.</p></blockquote></blockquote><blockquote><p>What makes SCons better?</p></blockquote><blockquote><blockquote></blockquote></blockquote><ul><li>Configuration files are Python scripts–use the power of a real programming language to solve build problems.</li><li>Reliable, automatic dependency analysis built-in for C, C++ and Fortran–no more “make depend” or “make clean” to get all of the dependencies. Dependency analysis is easily extensible through user-defined dependency Scanners for other languages or file types.</li><li>Built-in support for C, C++, D, Java, Fortran, Yacc, Lex, Qt and SWIG, and building TeX and LaTeX documents. Easily extensible through user-defined Builders for other languages or file types.</li><li>Building from central repositories of source code and/or pre-built targets.</li><li>Built-in support for fetching source files from SCCS, RCS, CVS, BitKeeper and Perforce.</li><li>Built-in support for Microsoft Visual Studio .NET and past Visual Studio versions, including generation of .dsp, .dsw, .sln and .vcproj files.</li><li>Reliable detection of build changes using MD5 signatures; optional, configurable support for traditional timestamps.</li><li>Improved support for parallel builds–like make -j but keeps N jobs running simultaneously regardless of directory hierarchy.</li><li>Integrated Autoconf-like support for finding #include files, libraries, functions and typedefs.</li><li>Global view of all dependencies–no more multiple build passes or reordering targets to build everything.</li><li>Ability to share built files in a cache to speed up multiple builds–like ccache but for any type of target file, not just C/C++ compilation.</li><li>Designed from the ground up for cross-platform builds, and known to work on Linux, other POSIX systems (including AIX, BSD systems, HP/UX, IRIX and Solaris), Windows NT, Mac OS X, and OS/2.</li></ul><p>最大特点就是使用 Python 语法来编写编译构建脚本，并且支持依赖自动推导，支持编译 C/C++/D/Java/Fortran等项目，并且是跨平台的(因为 python 是跨平台的）。</p><a id="more"></a><p>所以如果你对 python 熟悉的话，而且你和我对 C++  Makefile 有一样的烦恼，那么这对你将是一个好消息。 你将可以用 python 来编写构建脚本，而且会很简单，对于复杂的大型项目也能快速构建好。（也许只要 30 分钟）</p><h2 id="安装-scons"><a href="#安装-scons" class="headerlink" title="安装 scons"></a>安装 scons</h2><p>因为 scons 是基于 python 来构建的，所以毋容置疑，首先是需要准备好 python 环境，然后使用下述命令安装 scons 工具。</p><figure class="highlight cmake"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pip <span class="keyword">install</span> scons</span><br></pre></td></tr></table></figure><h2 id="scons-使用语法"><a href="#scons-使用语法" class="headerlink" title="scons 使用语法"></a>scons 使用语法</h2><p>注：<strong>本文以一个多源文件，多目录结构的项目 <a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">mux</a> 为例，介绍 cmake 的使用，相关源文件以及cmake 脚本可以直接查看源项目</strong>。</p><p>scons 构建脚本由一个 SConstruct 文件和多个 SConscript 文件构成。</p><p>SConstruct 通常位于项目顶层目录，然后 SConscript 通常位于子目录（子模块）。</p><p>那么来看一下 SConstruct 脚本长啥样?</p><h3 id="SConstruct"><a href="#SConstruct" class="headerlink" title="SConstruct"></a>SConstruct</h3><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br></pre></td><td class="code"><pre><span class="line">#!/usr/bin/env python</span><br><span class="line">#-*- coding:utf-<span class="number">8</span> -*-</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">import sys</span><br><span class="line">import os</span><br><span class="line">import platform</span><br><span class="line">import re</span><br><span class="line"></span><br><span class="line">env = Environment()</span><br><span class="line">abs_path = os.getcwd()</span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">'workspace path:&#123;0&#125;'</span>.format(abs_path)</span></span>)</span><br><span class="line"></span><br><span class="line">sbuild_dir = <span class="string">'sbuild'</span></span><br><span class="line"></span><br><span class="line">headers = [<span class="string">'.'</span>, <span class="string">'third-party/include'</span>]</span><br><span class="line">libs = [<span class="string">'./third-party/lib'</span>]</span><br><span class="line"></span><br><span class="line">abs_headers = []</span><br><span class="line">abs_libs = []</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> item <span class="keyword">in</span> headers:</span><br><span class="line">    abs_item = os<span class="selector-class">.path</span>.join(abs_path, item)</span><br><span class="line">    abs_headers.append(abs_item)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> item <span class="keyword">in</span> libs:</span><br><span class="line">    abs_item = os<span class="selector-class">.path</span>.join(abs_path, item)</span><br><span class="line">    abs_libs.append(abs_item)</span><br><span class="line"></span><br><span class="line">build_dir = os<span class="selector-class">.path</span>.join(abs_path, sbuild_dir)</span><br><span class="line">abs_libs.append(os<span class="selector-class">.path</span>.join(build_dir, <span class="string">'lib'</span>))</span><br><span class="line"></span><br><span class="line">CCFLAGS = <span class="string">'-ggdb -std=c++11'</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">'\nheaders path:'</span>)</span></span></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(abs_headers)</span></span></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">'\n'</span>)</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">'libs path:'</span>)</span></span></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(abs_libs)</span></span></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">'\n'</span>)</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">"begin load SConscript"</span>)</span></span></span><br><span class="line"></span><br><span class="line">env[<span class="string">"headers"</span>] = abs_headers</span><br><span class="line">env[<span class="string">"libs"</span>]    = abs_libs</span><br><span class="line">env[<span class="string">"MUX_DIR"</span>] = abs_path</span><br><span class="line">env[<span class="string">'ccflags'</span>] = CCFLAGS</span><br><span class="line">env[<span class="string">'build_dir'</span>] = build_dir</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">Export</span><span class="params">(<span class="string">'env'</span>)</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">SConscript</span><span class="params">([<span class="string">'./mbase/SConscript'</span>])</span></span></span><br><span class="line"><span class="function"><span class="title">SConscript</span><span class="params">([<span class="string">'./message_handle/SConscript'</span>])</span></span></span><br><span class="line"><span class="function"><span class="title">SConscript</span><span class="params">([<span class="string">'./epoll/SConscript'</span>])</span></span></span><br><span class="line"><span class="function"><span class="title">SConscript</span><span class="params">([<span class="string">'./transport/SConscript'</span>])</span></span></span><br><span class="line"><span class="function"><span class="title">SConscript</span><span class="params">([<span class="string">'./demo/bench/SConscript'</span>])</span></span></span><br><span class="line"><span class="function"><span class="title">SConscript</span><span class="params">([<span class="string">'./demo/echo/SConscript'</span>])</span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="title">print</span><span class="params">(<span class="string">"\n All Done, Please Check &#123;0&#125;"</span>.format(env[<span class="string">'build_dir'</span>])</span></span>)</span><br></pre></td></tr></table></figure><p>来分析一下这个文件，源文件可以直接在 <a href="https://github.com/smaugx/mux/blob/master/SConstruct" target="_blank" rel="noopener">我的github</a>下载。</p><p>SConstruct 文件主要做了两件事：</p><ul><li>env 环境变量的构造，主要是头文件路径，库路径，编译参数，自定义的一些变量等</li><li>使用 SConscript 函数解析执行子模块的 SConscript 文件 </li></ul><p>需要注意的是 SConstruct 和 SConscript 共享变量使用的就是 env 这个变量，你可以看到上面有一句：</p><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">Export</span><span class="params">(<span class="string">'env'</span>)</span></span></span><br></pre></td></tr></table></figure><p>这句很重要。</p><h3 id="SConscript"><a href="#SConscript" class="headerlink" title="SConscript"></a>SConscript</h3><p>那么位于子模块或者子目录的 SConscript 文件长啥样呢？</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line">#!/usr/bin/env python</span><br><span class="line">#-*- coding:utf<span class="number">-8</span> -*-</span><br><span class="line"></span><br><span class="line">import <span class="built_in">os</span></span><br><span class="line">import sys</span><br><span class="line"></span><br><span class="line">Import(<span class="string">'env'</span>)</span><br><span class="line">project_dir  = env[<span class="string">'MUX_DIR'</span>]</span><br><span class="line"></span><br><span class="line">epoll_lib  = <span class="string">'epoll'</span></span><br><span class="line"></span><br><span class="line">epoll_src_path = <span class="built_in">os</span>.<span class="built_in">path</span>.join(project_dir, <span class="string">'epoll/src'</span>)</span><br><span class="line">epoll_sources = []</span><br><span class="line"><span class="keyword">for</span> item <span class="keyword">in</span> <span class="built_in">os</span>.listdir(epoll_src_path):</span><br><span class="line">    <span class="keyword">if</span> item.endswith(<span class="string">'.cc'</span>) <span class="keyword">or</span> item.endswith(<span class="string">'.cpp'</span>) <span class="keyword">or</span> item.endswith(<span class="string">'.cxx'</span>):</span><br><span class="line">        abs_item = <span class="built_in">os</span>.<span class="built_in">path</span>.join(epoll_src_path, item)</span><br><span class="line">        epoll_sources.append(abs_item)</span><br><span class="line"></span><br><span class="line"><span class="built_in">print</span>(<span class="string">'\nbuild target:lib&#123;0&#125;.a'</span>.<span class="built_in">format</span>(epoll_lib))</span><br><span class="line"><span class="built_in">print</span>(epoll_sources)</span><br><span class="line"></span><br><span class="line">lib_dir = <span class="built_in">os</span>.<span class="built_in">path</span>.join(env[<span class="string">'build_dir'</span>], <span class="string">'lib'</span>)</span><br><span class="line"></span><br><span class="line">link_libraries = [<span class="string">'mbase'</span>]</span><br><span class="line"><span class="keyword">for</span> lib_name <span class="keyword">in</span> link_libraries:</span><br><span class="line">    lib_name = <span class="string">"&#123;0&#125;&#123;1&#125;&#123;2&#125;"</span>.<span class="built_in">format</span>(env[<span class="string">'LIBPREFIX'</span>], lib_name, env[<span class="string">'LIBSUFFIX'</span>])</span><br><span class="line">    abs_lib_name = <span class="built_in">os</span>.<span class="built_in">path</span>.join(lib_dir, lib_name)</span><br><span class="line">    epoll_sources.append(abs_lib_name)</span><br><span class="line"></span><br><span class="line">env.StaticLibrary(target = <span class="built_in">os</span>.<span class="built_in">path</span>.join(lib_dir, epoll_lib),</span><br><span class="line">        source  = epoll_sources,</span><br><span class="line">        CPPPATH = env[<span class="string">'headers'</span>], # include</span><br><span class="line">        LIBPATH = env[<span class="string">'libs'</span>],    # lib <span class="built_in">path</span></span><br><span class="line">        LIBS    = [<span class="string">'pthread'</span>],    # link lib</span><br><span class="line">        CCFLAGS = env[<span class="string">'ccflags'</span>]</span><br><span class="line">        )</span><br></pre></td></tr></table></figure><p>来分析一下这个文件，源文件可以直接在 <a href="https://github.com/smaugx/mux/blob/master/epoll/SConscript" target="_blank" rel="noopener">我的github</a>下载。</p><p>SConscript 主要做了两件事：</p><ul><li>构造一个源文件列表（用来构建 target 所需要使用的源文件）</li><li>根据需要构建 static_lib/dynamic_lib/binary</li></ul><p>当然，还有一点很重要，上面其实提到了，SConscript 和 SConstruct 用来共享变量使用的是 env 这个变量，所以你可以看到一句很重要的：</p><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">Import</span><span class="params">(<span class="string">'env'</span>)</span></span></span><br></pre></td></tr></table></figure><p>构造源文件列表，对于 Python 来说，简直是小菜一碟，太简单了；</p><p>然后如何生成目标文件呢？</p><p>1 生成二进制文件</p><figure class="highlight mel"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">env</span>.Program(target = os.path.join(bin_dir, echo_server_bin),</span><br><span class="line">        <span class="keyword">source</span>  = echo_server_sources,</span><br><span class="line">        CPPPATH = <span class="keyword">env</span>[<span class="string">'headers'</span>],</span><br><span class="line">        LIBPATH = <span class="keyword">env</span>[<span class="string">'libs'</span>],</span><br><span class="line">        LIBS    = [<span class="string">'transport'</span>,<span class="string">'msghandler'</span>,<span class="string">'epoll'</span>, <span class="string">'mbase'</span>, <span class="string">'pthread'</span>],</span><br><span class="line">        CCFLAGS = <span class="keyword">env</span>[<span class="string">'ccflags'</span>]</span><br><span class="line">        )</span><br></pre></td></tr></table></figure><p>2  生成静态库</p><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">env</span>.StaticLibrary(target = os.path.join(lib_dir, epoll_lib),</span><br><span class="line">        source  = epoll_sources,</span><br><span class="line">        CPPPATH = <span class="keyword">env</span>[<span class="string">'headers'</span>], <span class="comment"># include</span></span><br><span class="line">        LIBPATH = <span class="keyword">env</span>[<span class="string">'libs'</span>],    <span class="comment"># lib path</span></span><br><span class="line">        LIBS    = [<span class="string">'pthread'</span>],    <span class="comment"># link lib</span></span><br><span class="line">        CCFLAGS = <span class="keyword">env</span>[<span class="string">'ccflags'</span>]</span><br><span class="line">        )</span><br></pre></td></tr></table></figure><p>3 生成动态库</p><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">env</span>.SharedLibrary(target = os.path.join(lib_dir, epoll_lib),</span><br><span class="line">        source  = epoll_sources,</span><br><span class="line">        CPPPATH = <span class="keyword">env</span>[<span class="string">'headers'</span>], <span class="comment"># include</span></span><br><span class="line">        LIBPATH = <span class="keyword">env</span>[<span class="string">'libs'</span>],    <span class="comment"># lib path</span></span><br><span class="line">        LIBS    = [<span class="string">'pthread'</span>],    <span class="comment"># link lib</span></span><br><span class="line">        CCFLAGS = <span class="keyword">env</span>[<span class="string">'ccflags'</span>]</span><br><span class="line">        )</span><br></pre></td></tr></table></figure><p>上面 3 个函数的参数都是类似的：</p><ul><li>target: 指定需要生成的目标文件，通常我自己会写一个绝对路径；对于 lib 来说只需要写名字就行，前缀和后缀不需要写。(eg. target = ‘/root/scons_repo/sbuild/lib/test’ ，会生成 /root/scons_repo/sbuild/lib/libtest.a)</li><li>source: 编译目标文件需要的源文件列表</li><li>CPPPATH: 通常就是需要 Include 的头文件路径</li><li>LIBPATH: 通常就是需要链接的库路径</li><li>LIBS: 需要链接的库列表</li><li>CCFLAGS: 编译参数</li></ul><p><strong>attention:</strong></p><p><strong>上面有一个坑我自己碰到的，当我构建目标生成一个静态库的时候，需要链接其他的静态库，如果使用 $LIBPATH 和 $LIBS 指定链接库的话，scons 并没有链接这些库。尝试了很多方法，搜索了很多，也没有解决这个问题</strong>。</p><p><strong>最后是这样解决的。把需要链接的静态库添加到 source 参数中，和其他 cc/cpp 源文件一样放在一起，并且这些库需要使用绝对路径</strong>。</p><p>通常为了跨平台的方便，需要考虑lib 的前后缀，可以这样写：</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">link_libraries = [<span class="string">'test1'</span>, <span class="string">'test2'</span>]</span><br><span class="line"><span class="keyword">for</span> lib_name <span class="keyword">in</span> link_libraries:</span><br><span class="line">    lib_name = <span class="string">"&#123;0&#125;&#123;1&#125;&#123;2&#125;"</span>.<span class="built_in">format</span>(env[<span class="string">'LIBPREFIX'</span>], lib_name, env[<span class="string">'LIBSUFFIX'</span>])</span><br><span class="line">    abs_lib_name = <span class="built_in">os</span>.<span class="built_in">path</span>.join(lib_dir, lib_name)</span><br><span class="line">    sources.append(abs_lib_name)</span><br></pre></td></tr></table></figure><h2 id="scons-命令"><a href="#scons-命令" class="headerlink" title="scons 命令"></a>scons 命令</h2><p>上面详细讲解了如何使用 python 编写构建脚本，那么写好之后怎么用呢？</p><p>常用的几个命令：</p><p><strong>编译</strong>：</p><figure class="highlight ebnf"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">scons</span></span><br></pre></td></tr></table></figure><p>如果需要并行编译：</p><figure class="highlight mipsasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">scons </span>-<span class="keyword">j4</span></span><br></pre></td></tr></table></figure><p><strong>清理</strong>：</p><figure class="highlight ebnf"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">scons -c</span></span><br></pre></td></tr></table></figure><p>然后就会按照你脚本里写的方式去构建目标了。</p><p>这里贴一下 <a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">我的项目</a> 编译的输出：</p><figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br></pre></td><td class="code"><pre><span class="line">$ scons</span><br><span class="line"><span class="symbol">scons:</span> Reading SConscript files ...</span><br><span class="line">workspace <span class="symbol">path:</span>/mnt/centos-share/workspace/mux</span><br><span class="line"></span><br><span class="line">headers <span class="symbol">path:</span></span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/.'</span>, <span class="string">'/mnt/centos-share/workspace/mux/third-party/include'</span>]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">libs <span class="symbol">path:</span></span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/./third-party/lib'</span>, <span class="string">'/mnt/centos-share/workspace/mux/sbuild/lib'</span>]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">begin</span> load SConscript</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>libmbase.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/mbase/src/packet.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>libmsghandler.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/message_handle/src/message_handler.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>libepoll.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/epoll/src/epoll_tcp_client.cc'</span>, <span class="string">'/mnt/centos-share/workspace/mux/epoll/src/epoll_tcp_server.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>libtransport.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/transport/src/tcp_transport.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>bench_server</span><br><span class="line">[<span class="string">'bench_server.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>bench_client</span><br><span class="line">[<span class="string">'client.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>echo_server</span><br><span class="line">[<span class="string">'echo_server.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="symbol">target:</span>echo_client</span><br><span class="line">[<span class="string">'client.cc'</span>]</span><br><span class="line"></span><br><span class="line"> All Done, Please Check /mnt/centos-share/workspace/mux/sbuild</span><br><span class="line"><span class="symbol">scons:</span> done reading SConscript files.</span><br><span class="line"><span class="symbol">scons:</span> Building targets ...</span><br><span class="line">g++ -o demo/bench/bench_server.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> demo/bench/bench_server.cc</span><br><span class="line">g++ -o demo/bench/client.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> demo/bench/client.cc</span><br><span class="line">g++ -o demo/echo/client.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> demo/echo/client.cc</span><br><span class="line">g++ -o demo/echo/echo_server.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> demo/echo/echo_server.cc</span><br><span class="line">g++ -o epoll/src/epoll_tcp_client.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> epoll/src/epoll_tcp_client.cc</span><br><span class="line">g++ -o epoll/src/epoll_tcp_server.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> epoll/src/epoll_tcp_server.cc</span><br><span class="line">g++ -o mbase/src/packet.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> mbase/src/packet.cc</span><br><span class="line">g++ -o message_handle/src/message_handler.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> message_handle/src/message_handler.cc</span><br><span class="line">g++ -o transport/src/tcp_transport.o -c -ggdb -std=c++<span class="number">11</span> -I. -Ithird-party/<span class="keyword">include</span> transport/src/tcp_transport.cc</span><br><span class="line">ar rc sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libmbase</span>.<span class="title">a</span> <span class="title">mbase</span>/<span class="title">src</span>/<span class="title">packet</span>.<span class="title">o</span></span></span><br><span class="line">ranlib sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libmbase</span>.<span class="title">a</span></span></span><br><span class="line">ar rc sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libepoll</span>.<span class="title">a</span> <span class="title">epoll</span>/<span class="title">src</span>/<span class="title">epoll_tcp_client</span>.<span class="title">o</span> <span class="title">epoll</span>/<span class="title">src</span>/<span class="title">epoll_tcp_server</span>.<span class="title">o</span> <span class="title">sbuild</span>/<span class="title">lib</span>/<span class="title">libmbase</span>.<span class="title">a</span></span></span><br><span class="line">ranlib sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libepoll</span>.<span class="title">a</span></span></span><br><span class="line">ar rc sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libtransport</span>.<span class="title">a</span> <span class="title">transport</span>/<span class="title">src</span>/<span class="title">tcp_transport</span>.<span class="title">o</span> <span class="title">sbuild</span>/<span class="title">lib</span>/<span class="title">libepoll</span>.<span class="title">a</span> <span class="title">sbuild</span>/<span class="title">lib</span>/<span class="title">libmbase</span>.<span class="title">a</span></span></span><br><span class="line">ranlib sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libtransport</span>.<span class="title">a</span></span></span><br><span class="line">ar rc sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libmsghandler</span>.<span class="title">a</span> <span class="title">message_handle</span>/<span class="title">src</span>/<span class="title">message_handler</span>.<span class="title">o</span> <span class="title">sbuild</span>/<span class="title">lib</span>/<span class="title">libmbase</span>.<span class="title">a</span></span></span><br><span class="line">ranlib sbuild/<span class="class"><span class="keyword">lib</span>/<span class="title">libmsghandler</span>.<span class="title">a</span></span></span><br><span class="line">g++ -o sbuild/bin/bench_client demo/bench/client.o -Lthird-party/<span class="class"><span class="keyword">lib</span> -<span class="title">Lsbuild</span>/<span class="title">lib</span> -<span class="title">ltransport</span> -<span class="title">lmsghandler</span> -<span class="title">lepoll</span> -<span class="title">lmbase</span> -<span class="title">lpthread</span></span></span><br><span class="line">g++ -o sbuild/bin/bench_server demo/bench/bench_server.o -Lthird-party/<span class="class"><span class="keyword">lib</span> -<span class="title">Lsbuild</span>/<span class="title">lib</span> -<span class="title">ltransport</span> -<span class="title">lmsghandler</span> -<span class="title">lepoll</span> -<span class="title">lmbase</span> -<span class="title">lpthread</span></span></span><br><span class="line">g++ -o sbuild/bin/echo_client demo/echo/client.o -Lthird-party/<span class="class"><span class="keyword">lib</span> -<span class="title">Lsbuild</span>/<span class="title">lib</span> -<span class="title">ltransport</span> -<span class="title">lmsghandler</span> -<span class="title">lepoll</span> -<span class="title">lmbase</span> -<span class="title">lpthread</span></span></span><br><span class="line">g++ -o sbuild/bin/echo_server demo/echo/echo_server.o -Lthird-party/<span class="class"><span class="keyword">lib</span> -<span class="title">Lsbuild</span>/<span class="title">lib</span> -<span class="title">ltransport</span> -<span class="title">lmsghandler</span> -<span class="title">lepoll</span> -<span class="title">lmbase</span> -<span class="title">lpthread</span></span></span><br><span class="line"><span class="symbol">scons:</span> done building targets.</span><br></pre></td></tr></table></figure><figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br></pre></td><td class="code"><pre><span class="line">$ scons -c</span><br><span class="line"><span class="string">scons:</span> Reading SConscript files ...</span><br><span class="line">workspace <span class="string">path:</span><span class="regexp">/mnt/</span>centos-share<span class="regexp">/workspace/</span>mux</span><br><span class="line"></span><br><span class="line">headers <span class="string">path:</span></span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/.'</span>, <span class="string">'/mnt/centos-share/workspace/mux/third-party/include'</span>]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">libs <span class="string">path:</span></span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/./third-party/lib'</span>, <span class="string">'/mnt/centos-share/workspace/mux/sbuild/lib'</span>]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">begin load SConscript</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>libmbase.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/mbase/src/packet.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>libmsghandler.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/message_handle/src/message_handler.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>libepoll.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/epoll/src/epoll_tcp_client.cc'</span>, <span class="string">'/mnt/centos-share/workspace/mux/epoll/src/epoll_tcp_server.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>libtransport.a</span><br><span class="line">[<span class="string">'/mnt/centos-share/workspace/mux/transport/src/tcp_transport.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>bench_server</span><br><span class="line">[<span class="string">'bench_server.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>bench_client</span><br><span class="line">[<span class="string">'client.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>echo_server</span><br><span class="line">[<span class="string">'echo_server.cc'</span>]</span><br><span class="line"></span><br><span class="line">build <span class="string">target:</span>echo_client</span><br><span class="line">[<span class="string">'client.cc'</span>]</span><br><span class="line"></span><br><span class="line"> All Done, Please Check <span class="regexp">/mnt/</span>centos-share<span class="regexp">/workspace/</span>mux/sbuild</span><br><span class="line"><span class="string">scons:</span> done reading SConscript files.</span><br><span class="line"><span class="string">scons:</span> Cleaning targets ...</span><br><span class="line">Removed demo<span class="regexp">/bench/</span>bench_server.o</span><br><span class="line">Removed demo<span class="regexp">/bench/</span>client.o</span><br><span class="line">Removed demo<span class="regexp">/echo/</span>client.o</span><br><span class="line">Removed demo<span class="regexp">/echo/</span>echo_server.o</span><br><span class="line">Removed epoll<span class="regexp">/src/</span>epoll_tcp_client.o</span><br><span class="line">Removed epoll<span class="regexp">/src/</span>epoll_tcp_server.o</span><br><span class="line">Removed mbase<span class="regexp">/src/</span>packet.o</span><br><span class="line">Removed message_handle<span class="regexp">/src/</span>message_handler.o</span><br><span class="line">Removed transport<span class="regexp">/src/</span>tcp_transport.o</span><br><span class="line">Removed sbuild<span class="regexp">/lib/</span>libmbase.a</span><br><span class="line">Removed sbuild<span class="regexp">/lib/</span>libepoll.a</span><br><span class="line">Removed sbuild<span class="regexp">/lib/</span>libtransport.a</span><br><span class="line">Removed sbuild<span class="regexp">/lib/</span>libmsghandler.a</span><br><span class="line">Removed sbuild<span class="regexp">/bin/</span>bench_client</span><br><span class="line">Removed sbuild<span class="regexp">/bin/</span>bench_server</span><br><span class="line">Removed sbuild<span class="regexp">/bin/</span>echo_client</span><br><span class="line">Removed sbuild<span class="regexp">/bin/</span>echo_server</span><br><span class="line"><span class="string">scons:</span> done cleaning targets.</span><br></pre></td></tr></table></figure><h1 id="写在最后"><a href="#写在最后" class="headerlink" title="写在最后"></a>写在最后</h1><p>scons 使用 python 脚本来构建项目，如果对 python 熟悉的话，那么编写编译构建脚本将会大大提高效率，再也不用局限在 Makefile 的蛋疼语法里面了。</p><p>当然 scons 的缺点也有，据说在大型项目的时候，可能会很慢。这个我还没碰到过，因为没有用到大型项目中。</p><p>下一篇，分享下 cmake 构建 C++ 项目的一些语法和步骤。</p><p><a href="http://rebootcat.com/2020/09/02/cmake/">cmake教程|cmake入门实战</a></p><p>另外，文中涉及到的项目可以在<a href="https://github.com/smaugx/mux" target="_blank" rel="noopener">我的github</a> 找到。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#x6e;&#x75;&#120;&#99;&#111;&#x64;&#x65;&#50;&#x6e;&#105;&#x6b;&#x69;&#x40;&#103;&#x6d;&#x61;&#105;&#108;&#x2e;&#99;&#111;&#x6d;">&#108;&#x69;&#x6e;&#x75;&#120;&#99;&#111;&#x64;&#x65;&#50;&#x6e;&#105;&#x6b;&#x69;&#x40;&#103;&#x6d;&#x61;&#105;&#108;&#x2e;&#99;&#111;&#x6d;</a></p></li></ul><p>2020-08-30 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;我是一个 linux c++ 开发者，但是一直对 Makefile 的语法很是头痛，每次都记不住，所以每次写 Makefile 都很痛苦，Makefile 里需要你自己编写依赖和推导规则，这个过程能不能简单点呢？&lt;/p&gt;
&lt;p&gt;对于编译一个 C++ 工程来说，也许需要的就是头文件路径、库路径、编译参数，剩下的东西基本也不重要，这三样足够去编译一个工程了。所以有没有一个工具能简单点的去实现 C++ 项目的构建呢？&lt;/p&gt;
&lt;p&gt;答案是有的，Scons 就是答案。&lt;/p&gt;
&lt;h1 id=&quot;Scons&quot;&gt;&lt;a href=&quot;#Scons&quot; class=&quot;headerlink&quot; title=&quot;Scons&quot;&gt;&lt;/a&gt;Scons&lt;/h1&gt;&lt;h2 id=&quot;什么是-scons&quot;&gt;&lt;a href=&quot;#什么是-scons&quot; class=&quot;headerlink&quot; title=&quot;什么是 scons&quot;&gt;&lt;/a&gt;什么是 scons&lt;/h2&gt;&lt;p&gt;这里直接引用官网的解释：&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;What is SCons?&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;SCons is an Open Source software construction tool—that is, a next-generation build tool. Think of SCons as an improved, cross-platform substitute for the classic Make utility with integrated functionality similar to autoconf/automake and compiler caches such as ccache. In short, SCons is an easier, more reliable and faster way to build software.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;What makes SCons better?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;blockquote&gt;
&lt;/blockquote&gt;
&lt;/blockquote&gt;
&lt;ul&gt;
&lt;li&gt;Configuration files are Python scripts–use the power of a real programming language to solve build problems.&lt;/li&gt;
&lt;li&gt;Reliable, automatic dependency analysis built-in for C, C++ and Fortran–no more “make depend” or “make clean” to get all of the dependencies. Dependency analysis is easily extensible through user-defined dependency Scanners for other languages or file types.&lt;/li&gt;
&lt;li&gt;Built-in support for C, C++, D, Java, Fortran, Yacc, Lex, Qt and SWIG, and building TeX and LaTeX documents. Easily extensible through user-defined Builders for other languages or file types.&lt;/li&gt;
&lt;li&gt;Building from central repositories of source code and/or pre-built targets.&lt;/li&gt;
&lt;li&gt;Built-in support for fetching source files from SCCS, RCS, CVS, BitKeeper and Perforce.&lt;/li&gt;
&lt;li&gt;Built-in support for Microsoft Visual Studio .NET and past Visual Studio versions, including generation of .dsp, .dsw, .sln and .vcproj files.&lt;/li&gt;
&lt;li&gt;Reliable detection of build changes using MD5 signatures; optional, configurable support for traditional timestamps.&lt;/li&gt;
&lt;li&gt;Improved support for parallel builds–like make -j but keeps N jobs running simultaneously regardless of directory hierarchy.&lt;/li&gt;
&lt;li&gt;Integrated Autoconf-like support for finding #include files, libraries, functions and typedefs.&lt;/li&gt;
&lt;li&gt;Global view of all dependencies–no more multiple build passes or reordering targets to build everything.&lt;/li&gt;
&lt;li&gt;Ability to share built files in a cache to speed up multiple builds–like ccache but for any type of target file, not just C/C++ compilation.&lt;/li&gt;
&lt;li&gt;Designed from the ground up for cross-platform builds, and known to work on Linux, other POSIX systems (including AIX, BSD systems, HP/UX, IRIX and Solaris), Windows NT, Mac OS X, and OS/2.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;最大特点就是使用 Python 语法来编写编译构建脚本，并且支持依赖自动推导，支持编译 C/C++/D/Java/Fortran等项目，并且是跨平台的(因为 python 是跨平台的）。&lt;/p&gt;
    
    </summary>
    
    
      <category term="c++" scheme="https://rebootcat.com/categories/c/"/>
    
    
      <category term="linux" scheme="https://rebootcat.com/tags/linux/"/>
    
      <category term="c++" scheme="https://rebootcat.com/tags/c/"/>
    
      <category term="cmake" scheme="https://rebootcat.com/tags/cmake/"/>
    
      <category term="scons" scheme="https://rebootcat.com/tags/scons/"/>
    
      <category term="makefile" scheme="https://rebootcat.com/tags/makefile/"/>
    
      <category term="make" scheme="https://rebootcat.com/tags/make/"/>
    
  </entry>
  
  <entry>
    <title>自动创建阿里云抢占式实例</title>
    <link href="https://rebootcat.com/2020/08/24/auto_run_aliyun_spot/"/>
    <id>https://rebootcat.com/2020/08/24/auto_run_aliyun_spot/</id>
    <published>2020-08-24T14:23:58.000Z</published>
    <updated>2020-08-24T14:30:23.691Z</updated>
    
    <content type="html"><![CDATA[<h1 id="aliyun-spot"><a href="#aliyun-spot" class="headerlink" title="aliyun_spot"></a><a href="https://github.com/smaugx/aliyun_spot" target="_blank" rel="noopener">aliyun_spot</a></h1><p>自动创建阿里云抢占式实例。</p><h1 id="支持一下作者，购买阿里云"><a href="#支持一下作者，购买阿里云" class="headerlink" title="支持一下作者，购买阿里云"></a><a href="https://www.aliyun.com/minisite/goods?userCode=c5nuzwoj" target="_blank" rel="noopener">支持一下作者，购买阿里云</a></h1><h1 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h1><p>阿里云抢占式实例应该属于阿里云的一种闲置资源利用，性价比非常高，每小时的价格在 0.01 ~ 0.05 每小时，具体根据不同的配置和地域有差别，流量价格小于 1元/G.</p><p>抢占式实例最高可以以<strong>一折的价格购买 ECS 实例，并能稳定持有该实例至少一个小时</strong>。一个小时后，当市场价格高于您的出价或资源供需关系变化时，<strong>抢占式实例会被自动释放</strong>，请做好数据备份工作。</p><p><strong>非常适合爬虫</strong></p><p><strong>非常适合爬虫</strong></p><p><strong>非常适合爬虫</strong></p><p>也适合程序员个人日常开发使用，上班来创建，下班释放，开销基本可以控制在在 1毛 ~ 2 毛。</p><p>对于我来说，最近在写一个爬虫，看了很多代理都很贵，免费的又不稳定，正好了解到阿里云的抢占式实例，所以非常满足我的需求。</p><p>但是要注意，这个实例是有可能被释放的，但是不用担心，比如<strong>香港地区的释放率最近（2020-08-19）小于 3%. 另外，每个人可以最大创建 100 个实例</strong>，所以还是不用太担心。</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/auto_run_aliyun_spot/1.png" alt=""></p><a id="more"></a><h1 id="脚本功能"><a href="#脚本功能" class="headerlink" title="脚本功能"></a>脚本功能</h1><p>脚本仓库: <a href="https://github.com/smaugx/aliyun_spot" target="_blank" rel="noopener">https://github.com/smaugx/aliyun_spot</a></p><h2 id="自动创建阿里云抢占式实例"><a href="#自动创建阿里云抢占式实例" class="headerlink" title="自动创建阿里云抢占式实例"></a>自动创建阿里云抢占式实例</h2><p>支持以下一些参数：</p><ul><li>实例所属地域</li><li>创建的实例数量</li><li>公网出口带宽最大值</li><li>实例付费的策略和每小时最大价格</li><li>系统盘大小</li><li>释放时间（hours）</li><li>实例规格(cpu/mem/localdisk/net/ipv6)</li></ul><h2 id="手动释放一个或者多个实例"><a href="#手动释放一个或者多个实例" class="headerlink" title="手动释放一个或者多个实例"></a>手动释放一个或者多个实例</h2><p>可以使用脚本提前释放一个或者多个实例。</p><p><strong>创建的时候可以设置自动释放时间，当然也支持随时手动释放</strong>。</p><h1 id="如何使用"><a href="#如何使用" class="headerlink" title="如何使用"></a>如何使用</h1><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">$ python run_aliyunspot.py</span><br><span class="line">usage: run_aliyunspot.py [-h] [-c [CREATE]] [-r [RELEASE]] [-l [LIST]] [-s [SPOTID [SPOTID <span class="built_in">..</span>.]]]</span><br><span class="line"></span><br><span class="line">aliyunspot, 自动创建阿里云抢占式实例,支持自动/手动释放</span><br><span class="line"></span><br><span class="line">optional arguments:</span><br><span class="line">  -h, --help            show this help message <span class="keyword">and</span> exit</span><br><span class="line">  -c [CREATE], --create [CREATE]</span><br><span class="line">                        create aliyun spot<span class="built_in"> instance </span><span class="keyword">and</span> <span class="builtin-name">run</span> instance</span><br><span class="line">  -r [RELEASE], --release [RELEASE]</span><br><span class="line">                        release aliyun spot instance</span><br><span class="line">  -l [LIST], --list [LIST]</span><br><span class="line">                        list local record aliyun spot instance</span><br><span class="line">  -s [SPOTID [SPOTID <span class="built_in">..</span>.]], --spotid [SPOTID [SPOTID <span class="built_in">..</span>.]]</span><br><span class="line">                        aliyun spot instance_id <span class="keyword">for</span> release, you can give more than one</span><br></pre></td></tr></table></figure><h2 id="1-克隆仓库"><a href="#1-克隆仓库" class="headerlink" title="1 克隆仓库"></a>1 克隆仓库</h2><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> git <span class="built_in">clone</span> https://github.com/smaugx/aliyun_spot.git</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">cd</span> aliyun_spot</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> virtualenv -p python3 vv</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">source</span> vv/bin/activate</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> pip install -r requirements.txt</span></span><br></pre></td></tr></table></figure><h2 id="2-调整配置"><a href="#2-调整配置" class="headerlink" title="2  调整配置"></a>2  调整配置</h2><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="keyword">cp</span> test_config.<span class="keyword">py</span> config.<span class="keyword">py</span></span><br><span class="line"># 打开配置文件，根据你自己的需求修改里面的配置选项</span><br><span class="line">$ <span class="keyword">vim</span> config.<span class="keyword">py</span></span><br></pre></td></tr></table></figure><p>当然你也可以不用修改其他配置，只需要把你的 <strong>access_id</strong> 和 <strong>access_secret</strong> 填进去就可以，以及 <strong>key_pair_name</strong> 填进去。（见后文章节 <strong>#阿里云官网操作#</strong> ）</p><p><strong>默认创建的是香港地区的抢占式实例，内存 500MB, 1 CPU, 系统盘 20GB, 按流量计费（1元/G), 公网出口带宽 10Mbps, 1 小时候自动释放。</strong></p><blockquote><p>2020-08-19 上述默认配置的实例价格在 ￥ 0.018 /时。</p></blockquote><p>如果你觉得这个配置(cpu/mem)无法满足你的要求，那么可以调整 <strong>instance_type</strong> 这个参数，表示实例规格，详细可以查看阿里云官网页面 <a href="https://help.aliyun.com/document_detail/25378.html" target="_blank" rel="noopener">云服务器 ECS &gt; 实例 &gt; 实例规格族</a></p><h2 id="3-创建实例"><a href="#3-创建实例" class="headerlink" title="3 创建实例"></a>3 创建实例</h2><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">$ python run_aliyunspot.py -c</span><br><span class="line">will create <span class="keyword">and</span> <span class="builtin-name">run</span> aliyun spot instance, please wait<span class="built_in">..</span>.</span><br><span class="line">Success.<span class="built_in"> Instance </span>creation succeed. InstanceIds: i-j6cfhcbb3o2pepduwgfk</span><br><span class="line">Instance boot successfully: i-j6cfhcbb3o2pepduwgfk</span><br><span class="line">Instances all boot successfully</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">InstanceId:i-j6cfhcbb3o2pepduwgfk</span><br><span class="line">InstanceName:smaug-000-aliyun-8242148</span><br><span class="line">HostName:smaug-000-aliyun-8242148</span><br><span class="line">PublicIp:47.242.33.179</span><br><span class="line">KeyPairName:aliyunspot</span><br><span class="line">CreationTime:2020-08-24T13:48Z</span><br><span class="line">AutoReleaseTime:2020-08-24T22:48Z</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">instance <span class="builtin-name">info</span> saved <span class="keyword">in</span> file:./ecs/ecs.i-j6cfhcbb3o2pepduwgfk</span><br><span class="line">now you can use ssh: ssh -i ~/.ssh/aliyunspot.pem root@47.242.33.179</span><br></pre></td></tr></table></figure><p>如上，创建成功。然后接下来就可以使用 ssh 登录：</p><figure class="highlight angelscript"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ ssh -i ~/.ssh/~/.ssh/aliyunspot.pem <span class="symbol">root@</span><span class="number">8.210</span><span class="number">.245</span><span class="number">.226</span></span><br></pre></td></tr></table></figure><h2 id="4-列出实例"><a href="#4-列出实例" class="headerlink" title="4 列出实例"></a>4 列出实例</h2><figure class="highlight sml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ python run_aliyunspot.py -l</span><br><span class="line"><span class="built_in">list</span> all <span class="keyword">local</span> record instance:</span><br><span class="line">[<span class="symbol">'i</span>-j6caz353cisgl3fzenwi', <span class="symbol">'i</span>-j6cbyis12fb1fpzk59fv', <span class="symbol">'i</span>-j6cfhcbb3o2pepduwgfk']</span><br></pre></td></tr></table></figure><p>注意，上面仅仅是把之前创建并保存的实例信息从文件当中读取出来，并没有与 aliyun 交互。</p><h2 id="5-释放实例"><a href="#5-释放实例" class="headerlink" title="5 释放实例"></a>5 释放实例</h2><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ python run_aliyunspot.py -r -s i-j6caz353cisgl3fzenwi i-j6cbyis12fb1fpzk59fv</span><br><span class="line">will <span class="keyword">release</span> aliyun spot <span class="keyword">instance</span>:</span><br><span class="line">[<span class="string">'i-j6caz353cisgl3fzenwi'</span>, <span class="string">'i-j6cbyis12fb1fpzk59fv'</span>]</span><br><span class="line">please wait...</span><br><span class="line"></span><br><span class="line"><span class="keyword">release</span> <span class="keyword">instance</span>:[<span class="string">"i-j6caz353cisgl3fzenwi"</span>, <span class="string">"i-j6cbyis12fb1fpzk59fv"</span>] done</span><br></pre></td></tr></table></figure><h1 id="阿里云官网操作"><a href="#阿里云官网操作" class="headerlink" title="阿里云官网操作"></a><a href="https://www.aliyun.com/minisite/goods?userCode=c5nuzwoj" target="_blank" rel="noopener">阿里云官网操作</a></h1><p>上面提到了几个配置是需要在阿里云官网操作的。</p><p><strong>阿里云官网的使用还是挺复杂的，因为功能太多了，花费了我至少一个上午的时间才熟悉了整个操作，完成了整个脚本</strong></p><p><strong>所以整理了这个脚本方便大家使用，对阿里云的操作只需要下面几个：</strong></p><ul><li>注册一个阿里云账号，这个不用说了吧</li><li>充值 100 元以上，比如 130 元。因为创建实例账号里至少要 100 元</li><li>点击 <a href="https://ram.console.aliyun.com/overview" target="_blank" rel="noopener">https://ram.console.aliyun.com/overview</a> 创建一个用户组，分配权限 AliyunECSFullAccess 和 AliyunVPCFullAccess</li><li>还是上一步的页面，添加 ram 子账号，添加到刚才创建的用户组，这个账号会用来编程访问 aliyun API</li><li>还是上一步的页面，为这个ram 子账号创建 AccessKey。<strong>记得保存好</strong>。</li><li>在 <a href="https://ecs.console.aliyun.com/" target="_blank" rel="noopener">https://ecs.console.aliyun.com/</a> 页面选择 网络与安全-密钥对，创建密钥对 aliyunspot (名字任意），会自动下载这个私钥，<strong>记得保存好，一般要放到 ~/.ssh 目录下，并且记得  <code>chmod 600 aliyunspot.pem</code></strong></li></ul><p>OK, 到这里基本上得到了我们脚本里需要的几个配置：</p><ul><li>access_id</li><li>access_secret</li><li>key_pair_name</li></ul><p>把上述几个配置填到 config.py 中即可。</p><h1 id="然后开始创建和管理你的实例吧！！"><a href="#然后开始创建和管理你的实例吧！！" class="headerlink" title="然后开始创建和管理你的实例吧！！"></a>然后开始创建和管理你的实例吧！！</h1><h1 id="然后开始创建和管理你的实例吧！！-1"><a href="#然后开始创建和管理你的实例吧！！-1" class="headerlink" title="然后开始创建和管理你的实例吧！！"></a>然后开始创建和管理你的实例吧！！</h1><h1 id="然后开始创建和管理你的实例吧！！-2"><a href="#然后开始创建和管理你的实例吧！！-2" class="headerlink" title="然后开始创建和管理你的实例吧！！"></a>然后开始创建和管理你的实例吧！！</h1><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#105;&#110;&#x75;&#x78;&#99;&#x6f;&#x64;&#101;&#50;&#110;&#x69;&#107;&#x69;&#64;&#103;&#109;&#97;&#105;&#108;&#x2e;&#x63;&#x6f;&#x6d;">&#108;&#105;&#110;&#x75;&#x78;&#99;&#x6f;&#x64;&#101;&#50;&#110;&#x69;&#107;&#x69;&#64;&#103;&#109;&#97;&#105;&#108;&#x2e;&#x63;&#x6f;&#x6d;</a></p></li></ul><p>2020-08-24 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p><h1 id="附"><a href="#附" class="headerlink" title="附"></a>附</h1><ul><li><a href="https://www.aliyun.com/minisite/goods?userCode=c5nuzwoj" target="_blank" rel="noopener">阿里云官网</a></li><li><a href="https://api.aliyun.com" target="_blank" rel="noopener">Aliyun OpenAPI Explorer</a></li></ul>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;aliyun-spot&quot;&gt;&lt;a href=&quot;#aliyun-spot&quot; class=&quot;headerlink&quot; title=&quot;aliyun_spot&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://github.com/smaugx/aliyun_spot&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;aliyun_spot&lt;/a&gt;&lt;/h1&gt;&lt;p&gt;自动创建阿里云抢占式实例。&lt;/p&gt;
&lt;h1 id=&quot;支持一下作者，购买阿里云&quot;&gt;&lt;a href=&quot;#支持一下作者，购买阿里云&quot; class=&quot;headerlink&quot; title=&quot;支持一下作者，购买阿里云&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://www.aliyun.com/minisite/goods?userCode=c5nuzwoj&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;支持一下作者，购买阿里云&lt;/a&gt;&lt;/h1&gt;&lt;h1 id=&quot;背景&quot;&gt;&lt;a href=&quot;#背景&quot; class=&quot;headerlink&quot; title=&quot;背景&quot;&gt;&lt;/a&gt;背景&lt;/h1&gt;&lt;p&gt;阿里云抢占式实例应该属于阿里云的一种闲置资源利用，性价比非常高，每小时的价格在 0.01 ~ 0.05 每小时，具体根据不同的配置和地域有差别，流量价格小于 1元/G.&lt;/p&gt;
&lt;p&gt;抢占式实例最高可以以&lt;strong&gt;一折的价格购买 ECS 实例，并能稳定持有该实例至少一个小时&lt;/strong&gt;。一个小时后，当市场价格高于您的出价或资源供需关系变化时，&lt;strong&gt;抢占式实例会被自动释放&lt;/strong&gt;，请做好数据备份工作。&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;非常适合爬虫&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;非常适合爬虫&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;非常适合爬虫&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;也适合程序员个人日常开发使用，上班来创建，下班释放，开销基本可以控制在在 1毛 ~ 2 毛。&lt;/p&gt;
&lt;p&gt;对于我来说，最近在写一个爬虫，看了很多代理都很贵，免费的又不稳定，正好了解到阿里云的抢占式实例，所以非常满足我的需求。&lt;/p&gt;
&lt;p&gt;但是要注意，这个实例是有可能被释放的，但是不用担心，比如&lt;strong&gt;香港地区的释放率最近（2020-08-19）小于 3%. 另外，每个人可以最大创建 100 个实例&lt;/strong&gt;，所以还是不用太担心。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting_2/rebootcat/auto_run_aliyun_spot/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
    
    </summary>
    
    
      <category term="python" scheme="https://rebootcat.com/categories/python/"/>
    
    
      <category term="python" scheme="https://rebootcat.com/tags/python/"/>
    
      <category term="aliyun" scheme="https://rebootcat.com/tags/aliyun/"/>
    
      <category term="spot" scheme="https://rebootcat.com/tags/spot/"/>
    
      <category term="ecs" scheme="https://rebootcat.com/tags/ecs/"/>
    
  </entry>
  
  <entry>
    <title>valgrind massif 分析内存问题</title>
    <link href="https://rebootcat.com/2020/06/16/valgrind_massif_memory_analysing/"/>
    <id>https://rebootcat.com/2020/06/16/valgrind_massif_memory_analysing/</id>
    <published>2020-06-16T00:23:58.000Z</published>
    <updated>2020-06-16T00:01:55.998Z</updated>
    
    <content type="html"><![CDATA[<h1 id="Valgrind-Massif"><a href="#Valgrind-Massif" class="headerlink" title="Valgrind Massif"></a>Valgrind Massif</h1><p>valgrind 是什么，这里直接引用其他人的博客：</p><blockquote><p>Valgrind是一套Linux下，开放源代码（GPL<br>V2）的仿真调试工具的集合。Valgrind由内核（core）以及基于内核的其他调试工具组成。</p></blockquote><blockquote><p>内核类似于一个框架（framework），它模拟了一个CPU环境，并提供服务给其他工具；而其他工具则类似于插件 (plug-in)，利用内核提供的服务完成各种特定的内存调试任务。</p><p>Valgrind的体系结构如下图所示：</p></blockquote><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/valgrind_massif_memory_analysing/1.png" alt=""></p><h1 id="Massif-命令行选项"><a href="#Massif-命令行选项" class="headerlink" title="Massif 命令行选项"></a>Massif 命令行选项</h1><p>关于 massif 命令行选项，可以直接查看 valgrind 的 help 信息：</p><figure class="highlight applescript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">MASSIF OPTIONS</span><br><span class="line">       <span class="comment">--heap=&lt;yes|no&gt; [default: yes]</span></span><br><span class="line">           Specifies whether heap profiling should be done.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--heap-admin=&lt;size&gt; [default: 8]</span></span><br><span class="line">           If heap profiling <span class="keyword">is</span> enabled, gives <span class="keyword">the</span> <span class="built_in">number</span> <span class="keyword">of</span> administrative bytes per block <span class="keyword">to</span> use. This should be an estimate <span class="keyword">of</span> <span class="keyword">the</span> average, <span class="keyword">since</span> <span class="keyword">it</span> may vary. For example, <span class="keyword">the</span></span><br><span class="line">           allocator used <span class="keyword">by</span> glibc <span class="keyword">on</span> Linux requires somewhere <span class="keyword">between</span> <span class="number">4</span> <span class="keyword">to</span> <span class="number">15</span> bytes per block, depending <span class="keyword">on</span> various factors. That allocator also requires admin <span class="literal">space</span> <span class="keyword">for</span> freed blocks,</span><br><span class="line">           <span class="keyword">but</span> Massif cannot account <span class="keyword">for</span> this.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--stacks=&lt;yes|no&gt; [default: no]</span></span><br><span class="line">           Specifies whether stack profiling should be done. This option slows Massif down greatly, <span class="keyword">and</span> so <span class="keyword">is</span> off <span class="keyword">by</span> default. Note <span class="keyword">that</span> Massif assumes <span class="keyword">that</span> <span class="keyword">the</span> main stack has size zero</span><br><span class="line">           <span class="keyword">at</span> start-up. This <span class="keyword">is</span> <span class="keyword">not</span> <span class="literal">true</span>, <span class="keyword">but</span> doing otherwise accurately <span class="keyword">is</span> difficult. Furthermore, starting <span class="keyword">at</span> zero better indicates <span class="keyword">the</span> size <span class="keyword">of</span> <span class="keyword">the</span> part <span class="keyword">of</span> <span class="keyword">the</span> main stack <span class="keyword">that</span> a user</span><br><span class="line">           program actually has control <span class="keyword">over</span>.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--pages-as-heap=&lt;yes|no&gt; [default: no]</span></span><br><span class="line">           Tells Massif <span class="keyword">to</span> profile memory <span class="keyword">at</span> <span class="keyword">the</span> page level rather than <span class="keyword">at</span> <span class="keyword">the</span> malloc'd block level. See <span class="keyword">above</span> <span class="keyword">for</span> details.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--depth=&lt;number&gt; [default: 30]</span></span><br><span class="line">           Maximum depth <span class="keyword">of</span> <span class="keyword">the</span> allocation trees recorded <span class="keyword">for</span> detailed snapshots. Increasing <span class="keyword">it</span> will make Massif <span class="built_in">run</span> somewhat more slowly, use more memory, <span class="keyword">and</span> produce bigger output</span><br><span class="line">           files.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--alloc-fn=&lt;name&gt;</span></span><br><span class="line">           Functions specified <span class="keyword">with</span> this option will be treated <span class="keyword">as</span> though they were a heap allocation function such <span class="keyword">as</span> malloc. This <span class="keyword">is</span> useful <span class="keyword">for</span> functions <span class="keyword">that</span> are wrappers <span class="keyword">to</span> malloc <span class="keyword">or</span></span><br><span class="line">           new, which can fill up <span class="keyword">the</span> allocation trees <span class="keyword">with</span> uninteresting information. This option can be specified multiple <span class="keyword">times</span> <span class="keyword">on</span> <span class="keyword">the</span> command line, <span class="keyword">to</span> <span class="built_in">name</span> multiple functions.</span><br><span class="line"></span><br><span class="line">           Note <span class="keyword">that</span> <span class="keyword">the</span> named function will only be treated this way <span class="keyword">if</span> <span class="keyword">it</span> <span class="keyword">is</span> <span class="keyword">the</span> top entry <span class="keyword">in</span> a stack trace, <span class="keyword">or</span> just <span class="keyword">below</span> another function treated this way. For example, <span class="keyword">if</span> you have a</span><br><span class="line">           function malloc1 <span class="keyword">that</span> wraps malloc, <span class="keyword">and</span> malloc2 <span class="keyword">that</span> wraps malloc1, just specifying <span class="comment">--alloc-fn=malloc2 will have no effect. You need to specify --alloc-fn=malloc1 as well.</span></span><br><span class="line">           This <span class="keyword">is</span> a little inconvenient, <span class="keyword">but</span> <span class="keyword">the</span> reason <span class="keyword">is</span> <span class="keyword">that</span> checking <span class="keyword">for</span> allocation functions <span class="keyword">is</span> slow, <span class="keyword">and</span> <span class="keyword">it</span> saves a lot <span class="keyword">of</span> <span class="built_in">time</span> <span class="keyword">if</span> Massif can stop looking <span class="keyword">through</span> <span class="keyword">the</span> stack trace</span><br><span class="line">           entries <span class="keyword">as</span> soon <span class="keyword">as</span> <span class="keyword">it</span> finds one <span class="keyword">that</span> doesn't match rather than having <span class="keyword">to</span> <span class="keyword">continue</span> <span class="keyword">through</span> all <span class="keyword">the</span> entries.</span><br><span class="line"></span><br><span class="line">           Note <span class="keyword">that</span> C++ names are demangled. Note also <span class="keyword">that</span> overloaded C++ names must be written <span class="keyword">in</span> full. Single quotes may be necessary <span class="keyword">to</span> prevent <span class="keyword">the</span> shell <span class="keyword">from</span> breaking them up. For</span><br><span class="line">           example:</span><br><span class="line"></span><br><span class="line">               <span class="comment">--alloc-fn='operator new(unsigned, std::nothrow_t const&amp;)'</span></span><br><span class="line"></span><br><span class="line">       <span class="comment">--ignore-fn=&lt;name&gt;</span></span><br><span class="line">           Any direct heap allocation (i.e. a call <span class="keyword">to</span> malloc, new, etc, <span class="keyword">or</span> a call <span class="keyword">to</span> a function named <span class="keyword">by</span> an <span class="comment">--alloc-fn option) that occurs in a function specified by this option will be</span></span><br><span class="line">           ignored. This <span class="keyword">is</span> mostly useful <span class="keyword">for</span> testing purposes. This option can be specified multiple <span class="keyword">times</span> <span class="keyword">on</span> <span class="keyword">the</span> command line, <span class="keyword">to</span> <span class="built_in">name</span> multiple functions.</span><br><span class="line"></span><br><span class="line">           Any realloc <span class="keyword">of</span> an ignored block will also be ignored, even <span class="keyword">if</span> <span class="keyword">the</span> realloc call <span class="keyword">does</span> <span class="keyword">not</span> occur <span class="keyword">in</span> an ignored function. This avoids <span class="keyword">the</span> possibility <span class="keyword">of</span> negative heap sizes <span class="keyword">if</span></span><br><span class="line">           ignored blocks are shrunk <span class="keyword">with</span> realloc.</span><br><span class="line"></span><br><span class="line">           The rules <span class="keyword">for</span> writing C++ function names are <span class="keyword">the</span> same <span class="keyword">as</span> <span class="keyword">for</span> <span class="comment">--alloc-fn above.</span></span><br><span class="line"></span><br><span class="line">       <span class="comment">--threshold=&lt;m.n&gt; [default: 1.0]</span></span><br><span class="line">           The significance threshold <span class="keyword">for</span> heap allocations, <span class="keyword">as</span> a percentage <span class="keyword">of</span> total memory size. Allocation tree entries <span class="keyword">that</span> account <span class="keyword">for</span> <span class="keyword">less than</span> this will be aggregated. Note <span class="keyword">that</span></span><br><span class="line">           this should be specified <span class="keyword">in</span> tandem <span class="keyword">with</span> ms_print's option <span class="keyword">of</span> <span class="keyword">the</span> same <span class="built_in">name</span>.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--peak-inaccuracy=&lt;m.n&gt; [default: 1.0]</span></span><br><span class="line">           Massif <span class="keyword">does</span> <span class="keyword">not</span> necessarily <span class="built_in">record</span> <span class="keyword">the</span> actual <span class="keyword">global</span> memory allocation peak; <span class="keyword">by</span> default <span class="keyword">it</span> records a peak only when <span class="keyword">the</span> <span class="keyword">global</span> memory allocation size exceeds <span class="keyword">the</span> previous peak</span><br><span class="line">           <span class="keyword">by</span> <span class="keyword">at</span> least <span class="number">1.0</span>%. This <span class="keyword">is</span> because there can be many <span class="keyword">local</span> allocation peaks along <span class="keyword">the</span> way, <span class="keyword">and</span> doing a detailed snapshot <span class="keyword">for</span> <span class="keyword">every</span> one would be expensive <span class="keyword">and</span> wasteful, <span class="keyword">as</span> all</span><br><span class="line">           <span class="keyword">but</span> one <span class="keyword">of</span> them will be later discarded. This inaccuracy can be changed (even <span class="keyword">to</span> <span class="number">0.0</span>%) via this option, <span class="keyword">but</span> Massif will <span class="built_in">run</span> drastically slower <span class="keyword">as</span> <span class="keyword">the</span> <span class="built_in">number</span> approaches zero.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--time-unit=&lt;i|ms|B&gt; [default: i]</span></span><br><span class="line">           The <span class="built_in">time</span> unit used <span class="keyword">for</span> <span class="keyword">the</span> profiling. There are three possibilities: instructions executed (i), which <span class="keyword">is</span> good <span class="keyword">for</span> most cases; <span class="built_in">real</span> (wallclock) <span class="built_in">time</span> (ms, i.e. milliseconds),</span><br><span class="line">           which <span class="keyword">is</span> sometimes useful; <span class="keyword">and</span> bytes allocated/deallocated <span class="keyword">on</span> <span class="keyword">the</span> heap <span class="keyword">and</span>/<span class="keyword">or</span> stack (B), which <span class="keyword">is</span> useful <span class="keyword">for</span> very short-<span class="built_in">run</span> programs, <span class="keyword">and</span> <span class="keyword">for</span> testing purposes, because <span class="keyword">it</span> <span class="keyword">is</span></span><br><span class="line">           <span class="keyword">the</span> most reproducible across different machines.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--detailed-freq=&lt;n&gt; [default: 10]</span></span><br><span class="line">           Frequency <span class="keyword">of</span> detailed snapshots. With <span class="comment">--detailed-freq=1, every snapshot is detailed.</span></span><br><span class="line"></span><br><span class="line">       <span class="comment">--max-snapshots=&lt;n&gt; [default: 100]</span></span><br><span class="line">           The maximum <span class="built_in">number</span> <span class="keyword">of</span> snapshots recorded. If <span class="keyword">set</span> <span class="keyword">to</span> N, <span class="keyword">for</span> all programs except very short-<span class="built_in">running</span> ones, <span class="keyword">the</span> final <span class="built_in">number</span> <span class="keyword">of</span> snapshots will be <span class="keyword">between</span> N/<span class="number">2</span> <span class="keyword">and</span> N.</span><br><span class="line"></span><br><span class="line">       <span class="comment">--massif-out-file=&lt;file&gt; [default: massif.out.%p]</span></span><br><span class="line">           Write <span class="keyword">the</span> profile data <span class="keyword">to</span> <span class="built_in">file</span> rather than <span class="keyword">to</span> <span class="keyword">the</span> default output <span class="built_in">file</span>, massif.out.&lt;pid&gt;. The %p <span class="keyword">and</span> %q format specifiers can be used <span class="keyword">to</span> embed <span class="keyword">the</span> process ID <span class="keyword">and</span>/<span class="keyword">or</span> <span class="keyword">the</span></span><br><span class="line">           <span class="built_in">contents</span> <span class="keyword">of</span> an environment variable <span class="keyword">in</span> <span class="keyword">the</span> <span class="built_in">name</span>, <span class="keyword">as</span> <span class="keyword">is</span> <span class="keyword">the</span> case <span class="keyword">for</span> <span class="keyword">the</span> core option <span class="comment">--log-file.</span></span><br></pre></td></tr></table></figure><p>对其中几个常用的选项做一个说明：</p><a id="more"></a><ul><li><strong>–stacks</strong>: 栈内存的采样开关，默认关闭。打开后，会针对栈上的内存也进行采样，会使 massif 性能变慢；</li><li><strong>–time-unit</strong>：指定用来分析的时间单位。这个选项三个有效值：执行的指令（i），即默认值，用于大多数情况；即时（ms，单位毫秒），可用于某些特定事务；以及在堆（/或者）栈中分配/取消分配的字节（B），用于很少运行的程序，且用于测试目的，因为它最容易在不同机器中重现。这个选项在使用 ms_print 输出结果画图是游泳</li><li><strong>–detailed-freq</strong>: 针对详细内存快照的频率，默认是 10， 即每 10 个快照会有采集一个详细的内存快照</li><li><strong>–massif-out-file</strong>： 采样结束后，生成的采样文件（后续可以使用 ms_print 或者 massif-visualizer 进行分析）</li></ul><h1 id="开始采集"><a href="#开始采集" class="headerlink" title="开始采集"></a>开始采集</h1><p>经过上面的了解，接下来可以开始内存数据采集了，假设我们需要采集的二进制程序名为 xprogram:</p><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">valgrind -v <span class="attribute">--tool</span>=massif <span class="attribute">--time-unit</span>=B <span class="attribute">--detailed-freq</span>=1 <span class="attribute">--massif-out-file</span>=./massif.out  ./xprogram someargs</span><br></pre></td></tr></table></figure><p>运行一段时间后，采集到足够多的内存数据之后，我们需要停止程序，让它生成采集的数据文件，使用 kill 命令让 valgrind 程序退出。</p><blockquote><blockquote><p>attention: <strong>这里禁止使用  kill -9 模式去杀进程，不然不会产生采样文件</strong></p></blockquote></blockquote><h1 id="ms-print-分析采样文件"><a href="#ms-print-分析采样文件" class="headerlink" title="ms_print 分析采样文件"></a>ms_print 分析采样文件</h1><p>ms_print 是用来分析 massif 采样得到的内存数据文件的，使用命令为：</p><figure class="highlight ada"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ms_print ./massif.<span class="keyword">out</span></span><br></pre></td></tr></table></figure><p>或者把输出保存到文件：</p><figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ms_print ./massif.<span class="keyword">out</span> &gt; massif.<span class="literal">result</span></span><br></pre></td></tr></table></figure><p>打开 massif.result 看看长啥样：</p><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line">--------------------------------------------------------------------------------</span><br><span class="line"><span class="symbol">Command:</span>            ./xprogram someargs</span><br><span class="line">Massif <span class="symbol">arguments:</span>   --time-unit=B --massif-out-file=./massif.out</span><br><span class="line">ms_print <span class="symbol">arguments:</span> massif.out</span><br><span class="line">--------------------------------------------------------------------------------</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">    GB</span><br><span class="line"><span class="number">1.279</span>^                                                                       <span class="comment">#</span></span><br><span class="line">     |                                                                       <span class="comment">#</span></span><br><span class="line">     |                                                                   @  @<span class="comment">#</span></span><br><span class="line">     |                                                                   @::@<span class="comment">#</span></span><br><span class="line">     |                                                                 @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                                                            @::  @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                                                      : ::::@: ::@<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                                             @ @@@@ :::::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                                          :  @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                                     @  ::::<span class="symbol">:</span>@<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                               @@::<span class="symbol">:</span>@@::: :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                            ::<span class="symbol">:</span>@ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                    :: @@::::: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |                 :::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |          @  :::::::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |        ::@::: : :::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |      ::::@: : : :::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |     :: ::@: : : :::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     |   @@:: ::@: : : :::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">     | ::@ :: ::@: : : :::: <span class="symbol">:</span>@ :: :: @ : <span class="symbol">:</span>@@: : :: @<span class="symbol">:</span>@ @ @: :::: <span class="symbol">:</span>@: : @<span class="symbol">:</span>@: @<span class="comment">#</span></span><br><span class="line">   0 +-----------------------------------------------------------------------&gt;GB</span><br><span class="line">     0                                                                   <span class="number">813.9</span></span><br><span class="line"></span><br><span class="line">Number of <span class="symbol">snapshots:</span> <span class="number">68</span></span><br><span class="line"> Detailed <span class="symbol">snapshots:</span> [<span class="number">2</span>, <span class="number">7</span>, <span class="number">16</span>, <span class="number">21</span>, <span class="number">24</span>, <span class="number">25</span>, <span class="number">30</span>, <span class="number">32</span>, <span class="number">33</span>, <span class="number">34</span>, <span class="number">41</span>, <span class="number">44</span>, <span class="number">46</span>, <span class="number">48</span>, <span class="number">51</span>, <span class="number">52</span>, <span class="number">58</span>, <span class="number">59</span>, <span class="number">61</span>, <span class="number">64</span>, <span class="number">65</span>, <span class="number">66</span>, <span class="number">67</span> (peak)]</span><br></pre></td></tr></table></figure><p>这张图大概意思就表示<strong>堆内存的分配量随着采样时间的变化</strong>。从上图可以看到堆内存一直在增长，可能存在一些内存泄露等问题。</p><p>往下看还能看到内存的分配栈：</p><figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">  <span class="number">0</span>              <span class="number">0</span>                <span class="number">0</span>                <span class="number">0</span>             <span class="number">0</span>            <span class="number">0</span></span><br><span class="line">  <span class="number">1</span> <span class="number">20</span>,<span class="number">021</span>,<span class="number">463</span>,<span class="number">688</span>      <span class="number">133</span>,<span class="number">278</span>,<span class="number">776</span>      <span class="number">124</span>,<span class="number">687</span>,<span class="number">612</span>     <span class="number">8</span>,<span class="number">591</span>,<span class="number">164</span>            <span class="number">0</span></span><br><span class="line">  <span class="number">2</span> <span class="number">45</span>,<span class="number">201</span>,<span class="number">848</span>,<span class="number">936</span>      <span class="number">204</span>,<span class="number">228</span>,<span class="number">232</span>      <span class="number">191</span>,<span class="number">089</span>,<span class="number">596</span>    <span class="number">13</span>,<span class="number">138</span>,<span class="number">636</span>            <span class="number">0</span></span><br><span class="line"><span class="number">93.57</span>% (<span class="number">191</span>,<span class="number">089</span>,<span class="number">596</span>B) (heap allocation functions) malloc<span class="regexp">/new/</span><span class="keyword">new</span>[], --alloc-fns, etc.</span><br><span class="line">-&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xF088E6</span>: <span class="string">rocksdb:</span>:<span class="string">Arena:</span>:AllocateNewBlock(unsigned <span class="keyword">long</span>) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">| -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xF08500</span>: <span class="string">rocksdb:</span>:<span class="string">Arena:</span>:AllocateFallback(unsigned <span class="keyword">long</span>, bool) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|   -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xF0886C</span>: <span class="string">rocksdb:</span>:<span class="string">Arena:</span>:AllocateAligned(unsigned <span class="keyword">long</span>, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:Logger*) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xDE62BC</span>: <span class="string">rocksdb:</span>:<span class="string">ConcurrentArena:</span>:AllocateAligned(unsigned <span class="keyword">long</span>, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:Logger*)::&#123;lambda()</span><br><span class="line">|     | -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xDE7D9A</span>: <span class="keyword">char</span>* <span class="string">rocksdb:</span>:<span class="string">ConcurrentArena:</span>:AllocateImpl&lt;<span class="string">rocksdb:</span>:<span class="string">ConcurrentArena:</span>:AllocateAligned(unsigned <span class="keyword">long</span>, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:Logger*)::&#123;lambda()</span><br><span class="line">|     |   -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xDE6371</span>: <span class="string">rocksdb:</span>:<span class="string">ConcurrentArena:</span>:AllocateAligned(unsigned <span class="keyword">long</span>, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:Logger*) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |     -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE6FAB0</span>: <span class="string">rocksdb:</span>:InlineSkipList&lt;<span class="string">rocksdb:</span>:<span class="string">MemTableRep:</span>:KeyComparator const&amp;&gt;::AllocateNode(unsigned <span class="keyword">long</span>, <span class="keyword">int</span>) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |       -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE6F472</span>: <span class="string">rocksdb:</span>:InlineSkipList&lt;<span class="string">rocksdb:</span>:<span class="string">MemTableRep:</span>:KeyComparator const&amp;&gt;::AllocateKey(unsigned <span class="keyword">long</span>) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |         -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE6E40A</span>: <span class="string">rocksdb:</span>:(anonymous namespace)::<span class="string">SkipListRep:</span>:Allocate(unsigned <span class="keyword">long</span>, <span class="keyword">char</span>**) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |           -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xDE32E3</span>: <span class="string">rocksdb:</span>:<span class="string">MemTable:</span>:Add(unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:ValueType, <span class="string">rocksdb:</span>:Slice const&amp;, <span class="string">rocksdb:</span>:Slice const&amp;, bool, <span class="string">rocksdb:</span>:MemTablePostProcessInfo*) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |             -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE5C218</span>: <span class="string">rocksdb:</span>:<span class="string">MemTableInserter:</span>:PutCFImpl(unsigned <span class="keyword">int</span>, <span class="string">rocksdb:</span>:Slice const&amp;, <span class="string">rocksdb:</span>:Slice const&amp;, <span class="string">rocksdb:</span>:ValueType) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |               -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE5C92C</span>: <span class="string">rocksdb:</span>:<span class="string">MemTableInserter:</span>:PutCF(unsigned <span class="keyword">int</span>, <span class="string">rocksdb:</span>:Slice const&amp;, <span class="string">rocksdb:</span>:Slice const&amp;) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |                 -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE570E4</span>: <span class="string">rocksdb:</span>:<span class="string">WriteBatch:</span>:Iterate(<span class="string">rocksdb:</span>:<span class="string">WriteBatch:</span>:Handler*) const (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |                   -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xE598D5</span>: <span class="string">rocksdb:</span>:<span class="string">WriteBatchInternal:</span>:InsertInto(<span class="string">rocksdb:</span>:<span class="string">WriteThread:</span>:WriteGroup&amp;, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:ColumnFamilyMemTables*, <span class="string">rocksdb:</span>:FlushScheduler*, bool, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:DB*, bool, bool, bool) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |                     -&gt;<span class="number">41.07</span>% (<span class="number">83</span>,<span class="number">886</span>,<span class="number">080</span>B) <span class="number">0xD45AD7</span>: <span class="string">rocksdb:</span>:<span class="string">DBImpl:</span>:WriteImpl(<span class="string">rocksdb:</span>:WriteOptions const&amp;, <span class="string">rocksdb:</span>:WriteBatch*, <span class="string">rocksdb:</span>:WriteCallback*, unsigned <span class="keyword">long</span>*, unsigned <span class="keyword">long</span>, bool, unsigned <span class="keyword">long</span>*, unsigned <span class="keyword">long</span>, <span class="string">rocksdb:</span>:PreReleaseCallback*) (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |                       -&gt;<span class="number">28.75</span>% (<span class="number">58</span>,<span class="number">720</span>,<span class="number">256</span>B) <span class="number">0x1013B9C</span>: <span class="string">rocksdb:</span>:<span class="string">WriteCommittedTxn:</span>:CommitWithoutPrepareInternal() (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |                       | -&gt;<span class="number">28.75</span>% (<span class="number">58</span>,<span class="number">720</span>,<span class="number">256</span>B) <span class="number">0x1013653</span>: <span class="string">rocksdb:</span>:<span class="string">PessimisticTransaction:</span>:Commit() (<span class="keyword">in</span> <span class="regexp">/chain/</span>xtopchain)</span><br><span class="line">|     |                       |   -&gt;<span class="number">28.75</span>% (<span class="number">58</span>,<span class="number">720</span>,<span class="number">256</span>B) <span class="number">0xF40E17</span>: <span class="string">rocksdb:</span>:<span class="string">PessimisticTransactionDB:</span>:Put(<span class="string">rocksdb:</span>:WriteOptions const&amp;, <span class="string">rocksdb:</span>:ColumnFamilyHandle*, rocksdb</span><br></pre></td></tr></table></figure><p>能看到内存分配的调用堆栈情况，据此可以看到哪里分配的内存较多。</p><h1 id="massif-visualizer-可视化分析采样文件"><a href="#massif-visualizer-可视化分析采样文件" class="headerlink" title="massif-visualizer 可视化分析采样文件"></a>massif-visualizer 可视化分析采样文件</h1><p>ms_print 一定程度上不够直观，所以祭出另外一个分析内存采样数据的大杀器 – <strong>massif-visualizer</strong>，它能<strong>可视化的展示内存分配随着采样时间的变化情况，并能直观的看到内存分配的排行榜</strong>。</p><p>注意： <strong>massif-visualizer 目前好像只支持 linux 环境，并且具有桌面环境的 Linux</strong>. (mac/windows 的版本我没有找到）。</p><p>故我们采用 ubuntu-20.04-lts 作为分析环境。</p><h2 id="安装软件"><a href="#安装软件" class="headerlink" title="安装软件"></a>安装软件</h2><p>直接在软件中心搜索 massif-visualizer，然后安装</p><h2 id="启动软件，分析数据"><a href="#启动软件，分析数据" class="headerlink" title="启动软件，分析数据"></a>启动软件，分析数据</h2><p><strong>双击 massif-visualizer 启动软件之后，打开并选中某个 massif.out 文件</strong>，或者用命令行的方式打开：</p><figure class="highlight ada"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">massif-visualizer ./massif.<span class="keyword">out</span></span><br></pre></td></tr></table></figure><p>启动后，能直观的看到内存随采样时间的变化情况：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/valgrind_massif_memory_analysing/2.png" alt=""></p><p>调整上面的选项 <strong>Stacked diagrams</strong> 值后：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/valgrind_massif_memory_analysing/3.png" alt=""></p><p>鼠标悬停之后也能看到每条曲线某个 snapshot 对应的内存分配情况。</p><p>界面右边是内存调用的堆栈：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/valgrind_massif_memory_analysing/4.png" alt=""></p><p>点击界面下面的 <strong>Allocators</strong> 按钮之后，可以看到内存分配的排行榜：</p><p><img src="https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/valgrind_massif_memory_analysing/5.png" alt=""></p><p>是不是很方便？</p><h1 id="写在最后"><a href="#写在最后" class="headerlink" title="写在最后"></a>写在最后</h1><p>其实用于分析内存分配情况的利器还可以采用 <strong>google-perftools</strong>，也是采用对内存采样的方式进行采集，然后生成不同的内存采样文件，结束之后比较两个内存采样文件，就可以分析内存分配情况，同时也能展示初内存分配的函数调用栈。不过相比较于 valgrind 的 massif 插件，<strong>google-perftools 是需要代码侵入的，并且不能直观的展示内存随采样时间的变化情况</strong>。</p><p>而 massif 采样的内存数据文件，借助 massif-visualizer 工具就能直观的感受到内存分配随采样时间的变化情况。</p><p>Blog:</p><ul><li><p><a href="http://rebootcat.com">rebootcat.com</a></p></li><li><p>email: <a href="mailto:&#108;&#x69;&#110;&#117;&#x78;&#99;&#x6f;&#x64;&#x65;&#x32;&#110;&#x69;&#107;&#105;&#x40;&#x67;&#109;&#x61;&#x69;&#x6c;&#x2e;&#x63;&#111;&#109;">&#108;&#x69;&#110;&#117;&#x78;&#99;&#x6f;&#x64;&#x65;&#x32;&#110;&#x69;&#107;&#105;&#x40;&#x67;&#109;&#x61;&#x69;&#x6c;&#x2e;&#x63;&#111;&#109;</a></p></li></ul><p>2020-06-16 于杭州<br><em>By  <a href="https://github.com/smaugx" target="_blank" rel="noopener">史矛革</a></em></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;Valgrind-Massif&quot;&gt;&lt;a href=&quot;#Valgrind-Massif&quot; class=&quot;headerlink&quot; title=&quot;Valgrind Massif&quot;&gt;&lt;/a&gt;Valgrind Massif&lt;/h1&gt;&lt;p&gt;valgrind 是什么，这里直接引用其他人的博客：&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Valgrind是一套Linux下，开放源代码（GPL&lt;br&gt;V2）的仿真调试工具的集合。Valgrind由内核（core）以及基于内核的其他调试工具组成。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;内核类似于一个框架（framework），它模拟了一个CPU环境，并提供服务给其他工具；而其他工具则类似于插件 (plug-in)，利用内核提供的服务完成各种特定的内存调试任务。&lt;/p&gt;
&lt;p&gt;Valgrind的体系结构如下图所示：&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&lt;img src=&quot;https://cdn.jsdelivr.net/gh/smaugx/MyblogImgHosting/rebootcat/valgrind_massif_memory_analysing/1.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;
&lt;h1 id=&quot;Massif-命令行选项&quot;&gt;&lt;a href=&quot;#Massif-命令行选项&quot; class=&quot;headerlink&quot; title=&quot;Massif 命令行选项&quot;&gt;&lt;/a&gt;Massif 命令行选项&lt;/h1&gt;&lt;p&gt;关于 massif 命令行选项，可以直接查看 valgrind 的 help 信息：&lt;/p&gt;
&lt;figure class=&quot;highlight applescript&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;7&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;8&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;9&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;12&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;13&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;14&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;15&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;16&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;17&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;18&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;19&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;20&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;21&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;22&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;23&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;24&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;25&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;26&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;27&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;28&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;29&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;30&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;31&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;32&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;33&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;34&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;35&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;36&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;37&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;38&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;39&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;40&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;41&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;42&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;43&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;44&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;45&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;46&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;47&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;48&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;49&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;50&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;51&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;52&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;53&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;54&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;55&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;56&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;57&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;58&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;59&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;60&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;61&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;62&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;63&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;64&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;65&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;66&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;67&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;68&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;MASSIF OPTIONS&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--heap=&amp;lt;yes|no&amp;gt; [default: yes]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Specifies whether heap profiling should be done.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--heap-admin=&amp;lt;size&amp;gt; [default: 8]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           If heap profiling &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; enabled, gives &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;number&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; administrative bytes per block &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; use. This should be an estimate &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; average, &lt;span class=&quot;keyword&quot;&gt;since&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; may vary. For example, &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           allocator used &lt;span class=&quot;keyword&quot;&gt;by&lt;/span&gt; glibc &lt;span class=&quot;keyword&quot;&gt;on&lt;/span&gt; Linux requires somewhere &lt;span class=&quot;keyword&quot;&gt;between&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;15&lt;/span&gt; bytes per block, depending &lt;span class=&quot;keyword&quot;&gt;on&lt;/span&gt; various factors. That allocator also requires admin &lt;span class=&quot;literal&quot;&gt;space&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; freed blocks,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           &lt;span class=&quot;keyword&quot;&gt;but&lt;/span&gt; Massif cannot account &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; this.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--stacks=&amp;lt;yes|no&amp;gt; [default: no]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Specifies whether stack profiling should be done. This option slows Massif down greatly, &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; so &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; off &lt;span class=&quot;keyword&quot;&gt;by&lt;/span&gt; default. Note &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; Massif assumes &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; main stack has size zero&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           &lt;span class=&quot;keyword&quot;&gt;at&lt;/span&gt; start-up. This &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;not&lt;/span&gt; &lt;span class=&quot;literal&quot;&gt;true&lt;/span&gt;, &lt;span class=&quot;keyword&quot;&gt;but&lt;/span&gt; doing otherwise accurately &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; difficult. Furthermore, starting &lt;span class=&quot;keyword&quot;&gt;at&lt;/span&gt; zero better indicates &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; size &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; part &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; main stack &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; a user&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           program actually has control &lt;span class=&quot;keyword&quot;&gt;over&lt;/span&gt;.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--pages-as-heap=&amp;lt;yes|no&amp;gt; [default: no]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Tells Massif &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; profile memory &lt;span class=&quot;keyword&quot;&gt;at&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; page level rather than &lt;span class=&quot;keyword&quot;&gt;at&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; malloc&#39;d block level. See &lt;span class=&quot;keyword&quot;&gt;above&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; details.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--depth=&amp;lt;number&amp;gt; [default: 30]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Maximum depth &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; allocation trees recorded &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; detailed snapshots. Increasing &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; will make Massif &lt;span class=&quot;built_in&quot;&gt;run&lt;/span&gt; somewhat more slowly, use more memory, &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; produce bigger output&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           files.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--alloc-fn=&amp;lt;name&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Functions specified &lt;span class=&quot;keyword&quot;&gt;with&lt;/span&gt; this option will be treated &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; though they were a heap allocation function such &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; malloc. This &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; useful &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; functions &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; are wrappers &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; malloc &lt;span class=&quot;keyword&quot;&gt;or&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           new, which can fill up &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; allocation trees &lt;span class=&quot;keyword&quot;&gt;with&lt;/span&gt; uninteresting information. This option can be specified multiple &lt;span class=&quot;keyword&quot;&gt;times&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;on&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; command line, &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;name&lt;/span&gt; multiple functions.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Note &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; named function will only be treated this way &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; top entry &lt;span class=&quot;keyword&quot;&gt;in&lt;/span&gt; a stack trace, &lt;span class=&quot;keyword&quot;&gt;or&lt;/span&gt; just &lt;span class=&quot;keyword&quot;&gt;below&lt;/span&gt; another function treated this way. For example, &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; you have a&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           function malloc1 &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; wraps malloc, &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; malloc2 &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; wraps malloc1, just specifying &lt;span class=&quot;comment&quot;&gt;--alloc-fn=malloc2 will have no effect. You need to specify --alloc-fn=malloc1 as well.&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           This &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; a little inconvenient, &lt;span class=&quot;keyword&quot;&gt;but&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; reason &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; checking &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; allocation functions &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; slow, &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; saves a lot &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;time&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; Massif can stop looking &lt;span class=&quot;keyword&quot;&gt;through&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; stack trace&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           entries &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; soon &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; finds one &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; doesn&#39;t match rather than having &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;continue&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;through&lt;/span&gt; all &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; entries.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Note &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; C++ names are demangled. Note also &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; overloaded C++ names must be written &lt;span class=&quot;keyword&quot;&gt;in&lt;/span&gt; full. Single quotes may be necessary &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; prevent &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; shell &lt;span class=&quot;keyword&quot;&gt;from&lt;/span&gt; breaking them up. For&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           example:&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;               &lt;span class=&quot;comment&quot;&gt;--alloc-fn=&#39;operator new(unsigned, std::nothrow_t const&amp;amp;)&#39;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--ignore-fn=&amp;lt;name&amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Any direct heap allocation (i.e. a call &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; malloc, new, etc, &lt;span class=&quot;keyword&quot;&gt;or&lt;/span&gt; a call &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; a function named &lt;span class=&quot;keyword&quot;&gt;by&lt;/span&gt; an &lt;span class=&quot;comment&quot;&gt;--alloc-fn option) that occurs in a function specified by this option will be&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           ignored. This &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; mostly useful &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; testing purposes. This option can be specified multiple &lt;span class=&quot;keyword&quot;&gt;times&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;on&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; command line, &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;name&lt;/span&gt; multiple functions.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Any realloc &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; an ignored block will also be ignored, even &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; realloc call &lt;span class=&quot;keyword&quot;&gt;does&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;not&lt;/span&gt; occur &lt;span class=&quot;keyword&quot;&gt;in&lt;/span&gt; an ignored function. This avoids &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; possibility &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; negative heap sizes &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           ignored blocks are shrunk &lt;span class=&quot;keyword&quot;&gt;with&lt;/span&gt; realloc.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           The rules &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; writing C++ function names are &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; same &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;comment&quot;&gt;--alloc-fn above.&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--threshold=&amp;lt;m.n&amp;gt; [default: 1.0]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           The significance threshold &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; heap allocations, &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; a percentage &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; total memory size. Allocation tree entries &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt; account &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;less than&lt;/span&gt; this will be aggregated. Note &lt;span class=&quot;keyword&quot;&gt;that&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           this should be specified &lt;span class=&quot;keyword&quot;&gt;in&lt;/span&gt; tandem &lt;span class=&quot;keyword&quot;&gt;with&lt;/span&gt; ms_print&#39;s option &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; same &lt;span class=&quot;built_in&quot;&gt;name&lt;/span&gt;.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--peak-inaccuracy=&amp;lt;m.n&amp;gt; [default: 1.0]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Massif &lt;span class=&quot;keyword&quot;&gt;does&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;not&lt;/span&gt; necessarily &lt;span class=&quot;built_in&quot;&gt;record&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; actual &lt;span class=&quot;keyword&quot;&gt;global&lt;/span&gt; memory allocation peak; &lt;span class=&quot;keyword&quot;&gt;by&lt;/span&gt; default &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; records a peak only when &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;global&lt;/span&gt; memory allocation size exceeds &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; previous peak&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           &lt;span class=&quot;keyword&quot;&gt;by&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;at&lt;/span&gt; least &lt;span class=&quot;number&quot;&gt;1.0&lt;/span&gt;%. This &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; because there can be many &lt;span class=&quot;keyword&quot;&gt;local&lt;/span&gt; allocation peaks along &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; way, &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; doing a detailed snapshot &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;every&lt;/span&gt; one would be expensive &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; wasteful, &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; all&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           &lt;span class=&quot;keyword&quot;&gt;but&lt;/span&gt; one &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; them will be later discarded. This inaccuracy can be changed (even &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;0.0&lt;/span&gt;%) via this option, &lt;span class=&quot;keyword&quot;&gt;but&lt;/span&gt; Massif will &lt;span class=&quot;built_in&quot;&gt;run&lt;/span&gt; drastically slower &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;number&lt;/span&gt; approaches zero.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--time-unit=&amp;lt;i|ms|B&amp;gt; [default: i]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           The &lt;span class=&quot;built_in&quot;&gt;time&lt;/span&gt; unit used &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; profiling. There are three possibilities: instructions executed (i), which &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; good &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; most cases; &lt;span class=&quot;built_in&quot;&gt;real&lt;/span&gt; (wallclock) &lt;span class=&quot;built_in&quot;&gt;time&lt;/span&gt; (ms, i.e. milliseconds),&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           which &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; sometimes useful; &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; bytes allocated/deallocated &lt;span class=&quot;keyword&quot;&gt;on&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; heap &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt;/&lt;span class=&quot;keyword&quot;&gt;or&lt;/span&gt; stack (B), which &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; useful &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; very short-&lt;span class=&quot;built_in&quot;&gt;run&lt;/span&gt; programs, &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; testing purposes, because &lt;span class=&quot;keyword&quot;&gt;it&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; most reproducible across different machines.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--detailed-freq=&amp;lt;n&amp;gt; [default: 10]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Frequency &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; detailed snapshots. With &lt;span class=&quot;comment&quot;&gt;--detailed-freq=1, every snapshot is detailed.&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--max-snapshots=&amp;lt;n&amp;gt; [default: 100]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           The maximum &lt;span class=&quot;built_in&quot;&gt;number&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; snapshots recorded. If &lt;span class=&quot;keyword&quot;&gt;set&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; N, &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; all programs except very short-&lt;span class=&quot;built_in&quot;&gt;running&lt;/span&gt; ones, &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; final &lt;span class=&quot;built_in&quot;&gt;number&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; snapshots will be &lt;span class=&quot;keyword&quot;&gt;between&lt;/span&gt; N/&lt;span class=&quot;number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; N.&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;       &lt;span class=&quot;comment&quot;&gt;--massif-out-file=&amp;lt;file&amp;gt; [default: massif.out.%p]&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           Write &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; profile data &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;file&lt;/span&gt; rather than &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; default output &lt;span class=&quot;built_in&quot;&gt;file&lt;/span&gt;, massif.out.&amp;lt;pid&amp;gt;. The %p &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt; %q format specifiers can be used &lt;span class=&quot;keyword&quot;&gt;to&lt;/span&gt; embed &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; process ID &lt;span class=&quot;keyword&quot;&gt;and&lt;/span&gt;/&lt;span class=&quot;keyword&quot;&gt;or&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;           &lt;span class=&quot;built_in&quot;&gt;contents&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;of&lt;/span&gt; an environment variable &lt;span class=&quot;keyword&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; &lt;span class=&quot;built_in&quot;&gt;name&lt;/span&gt;, &lt;span class=&quot;keyword&quot;&gt;as&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;is&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; case &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;the&lt;/span&gt; core option &lt;span class=&quot;comment&quot;&gt;--log-file.&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;对其中几个常用的选项做一个说明：&lt;/p&gt;
    
    </summary>
    
    
      <category term="linux" scheme="https://rebootcat.com/categories/linux/"/>
    
    
      <category term="memory" scheme="https://rebootcat.com/tags/memory/"/>
    
      <category term="valgrind" scheme="https://rebootcat.com/tags/valgrind/"/>
    
      <category term="massif" scheme="https://rebootcat.com/tags/massif/"/>
    
      <category term="ubuntun" scheme="https://rebootcat.com/tags/ubuntun/"/>
    
  </entry>
  
</feed>