Jekyll2023-10-12T22:01:11+08:00/feed.xmlsinkcup’s blogAcross the Great Wall, we can reach every corner in the world.You have a store credit balance; you must spend your balance before you can change stores.2023-10-10T19:56:00+08:002023-10-10T19:56:00+08:00/you-must-spend-your-balance<p>If you have App Store Balance, you can’t change country, it will show error:</p>
<blockquote>
<p>You have a store credit balance; you must spend your balance before you can change stores.</p>
</blockquote>
<p><img src="https://github.com/sinkcup/sinkcup.github.io/assets/4971414/d745ce52-c156-4f89-bff6-22684545da99" alt="" /></p>
<p>How to spend all balance?</p>
<p>It’s easy!</p>
<p>If you have 0.29, 0.39, 0.49, 0.59, 0.69, 0.79, 0.89, 0.95, 0.99 or 1 USD, you could buy a item equal the balance, because App Store allow these price.</p>
<p>Which App have some many prices? this one: <a href="https://apps.apple.com/us/app/trim-video/id6444441547">https://apps.apple.com/us/app/trim-video/id6444441547</a></p>
<p>If you have other number, such 0.5 USD, you can’t find any item equal the balance, because App Store don’t allow these price. In fact, there is a way:</p>
<ol>
<li>Recharge another 10 USD, you will get 10.5</li>
<li>spend 1 USD, you will get 9.5</li>
<li>spend 0.95 USD 10 times, you will get 0</li>
</ol>
<p>There is a calculator that can help you. search “trim video zero” in App Store or click this URL:</p>
<p><a href="https://apps.apple.com/us/app/trim-video/id6444441547">https://apps.apple.com/us/app/trim-video/id6444441547</a></p>
<p><img src="https://github.com/sinkcup/sinkcup.github.io/assets/4971414/cbce28cd-f641-4bf7-9fd4-ab9cb103df6a" alt="calculate how to spend all app store balance - trim video app" /></p>
<p><img src="https://github.com/sinkcup/sinkcup.github.io/assets/4971414/a10c45fd-668d-446d-8819-d53dba8bb1f5" alt="trim video app - click cart" /></p>If you have App Store Balance, you can’t change country, it will show error:DevOps 实战:shell 代码规范2021-09-09T15:15:00+08:002021-09-09T15:15:00+08:00/lint-shell<p>Code Review 时发现大家的 shell 代码风格各异:</p>
<p>女同学的代码:https://github.com/Coding/coding-download-center/pull/48/files</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for </span>i <span class="k">in</span> <span class="sb">`</span><span class="nb">awk</span> <span class="s1">'{print $1}'</span> /tmp/index-body.md<span class="sb">`</span>
<span class="k">do</span>
<span class="o">[[</span> <span class="s2">"</span><span class="nv">$i</span><span class="s2">"</span> <span class="o">=</span>~ ^[a-z0-9.-]+<span class="nv">$ </span><span class="o">]]</span> <span class="o">||</span> <span class="o">(</span> <span class="nb">echo</span> “<span class="nv">$i</span>: Product names shall be all <span class="k">in </span>lower <span class="k">case</span>” <span class="o">&&</span> <span class="nb">exit </span>250 <span class="p">)</span>
<span class="k">done</span>
</code></pre></div></div>
<p>男同学的代码:https://github.com/Coding/coding-download-center/pull/47/files</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">package_name</span><span class="o">=</span><span class="si">$(</span><span class="nb">awk</span> <span class="s1">'{print $1}'</span> index-body.md<span class="si">)</span>
<span class="k">for </span>i <span class="k">in</span> <span class="nv">$package_name</span><span class="p">;</span><span class="k">do</span>
<span class="o">[[</span> <span class="o">!</span> <span class="s2">"</span><span class="nv">$i</span><span class="s2">"</span> <span class="o">=</span>~ <span class="se">\.</span><span class="o">[</span>0-9]+<span class="se">\.</span> <span class="o">]]</span> <span class="o">||</span> <span class="o">(</span> <span class="nb">echo</span> <span class="s2">"[ERROR] </span><span class="nv">$i</span><span class="s2"> : package name should not have version!"</span> <span class="o">&&</span> <span class="nb">exit </span>2 <span class="o">)</span>
<span class="k">done</span>
</code></pre></div></div>
<p>可以看出有这些问题:</p>
<ol>
<li>for/do 换行:女同学放在两行,男同学放在一行;</li>
</ol>
<p><img src="https://user-images.githubusercontent.com/4971414/132434304-0a99cd7c-5c5d-462f-9425-1be30e823f53.png" alt="image" /></p>
<ol>
<li>for 数据来源先赋值:女同学放在一行(<code class="language-plaintext highlighter-rouge">for i in 命令</code>),男同学放在两行(先赋值,然后 <code class="language-plaintext highlighter-rouge">for i in 变量</code>);</li>
</ol>
<p><img src="https://user-images.githubusercontent.com/4971414/132434620-421dd6dc-f180-40a5-ba84-34d4cae8fbc5.png" alt="image" /></p>
<ol>
<li>子语句标志符:女同学用了 `,男同学用了 <code class="language-plaintext highlighter-rouge">$</code>;</li>
</ol>
<p><img src="https://user-images.githubusercontent.com/4971414/132434678-011261d9-d5cf-46b9-bcb2-0958b947f1ea.png" alt="image" /></p>
<ol>
<li>行首缩进:女同学用 2 个空格,男同学用 4 个空格;</li>
</ol>
<p><img src="https://user-images.githubusercontent.com/4971414/132434782-343b8088-c63a-446b-8c65-1bef6a6418b8.png" alt="image" /></p>
<ol>
<li>行内缩进:女同学的「或」前面用了 2 个空格,应该用 1 个;男同学的 do 前面缺少一个空格。</li>
</ol>
<p><img src="https://user-images.githubusercontent.com/4971414/132434841-12945d9b-b33a-4c15-96f5-039ec7ba9565.png" alt="image" /></p>
<ol>
<li>全角引号:女同学的 echo 后面用了全角双引号,应该用半角。</li>
</ol>
<p><img src="https://user-images.githubusercontent.com/4971414/132434922-57e0a9ad-82fd-4312-84e8-ee420c491035.png" alt="image" /></p>
<p>所以需要寻找统一的代码规范,实现多人协作风格一致。</p>
<h2 id="寻找代码规范">寻找代码规范</h2>
<p><img src="https://user-images.githubusercontent.com/4971414/132314649-c12c0a33-dd5d-47cf-9353-ab1e4bd12f9b.png" alt="image" /></p>
<p>搜索「lint shell code style」,找到了两个工具:ShellCheck 和 shfmt,逐个调研,看看能否扫描出来上面的几个问题。其中 ShellCheck 是 <a href="https://google.github.io/styleguide/shellguide.html#s6.1-shellcheck">Google 规范推荐</a>的,所以先调研。</p>
<h3 id="shellcheck">ShellCheck</h3>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew <span class="nb">install </span>shellcheck
shellcheck girl.sh
shellcheck boy.sh
</code></pre></div></div>
<p><img src="https://user-images.githubusercontent.com/4971414/132315619-116d8890-65af-4c37-94fd-8c717c7c4e4e.png" alt="image" />
<img src="https://user-images.githubusercontent.com/4971414/132317690-ce59b13b-97e4-4431-8733-172027e55c92.png" alt="image" /></p>
<p>可以看出 ShellCheck 的规则:</p>
<ol>
<li>for/do 换行:不管;</li>
<li>for 数据来源先赋值:不管;</li>
<li>子语句标志符:管;</li>
<li>行缩进:不管;</li>
<li>行内缩进:不管;</li>
<li>全角引号:管;</li>
</ol>
<p>ShellCheck 还发现了别的问题:</p>
<blockquote>
<p>SC2002: Useless cat.
SC2013: To read lines rather than words, pipe/redirect to a ‘while read’ loop.</p>
</blockquote>
<p>所以要继续寻找别的工具,组合使用。</p>
<h3 id="shfmt">shfmt</h3>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>wget https://github.com/mvdan/sh/releases/download/v3.3.1/shfmt_v3.3.1_linux_amd64
<span class="nb">chmod</span> +x shfmt_v3.3.1_linux_amd64
<span class="nb">mv </span>shfmt_v3.3.1_linux_amd64 /usr/local/bin/shfmt
shfmt <span class="nt">-d</span> girl.sh
shfmt <span class="nt">-d</span> boy.sh
</code></pre></div></div>
<p><img src="https://user-images.githubusercontent.com/4971414/132435745-a46bfdb5-8b77-4822-a2f7-7da85c8b850e.png" alt="image" />
<img src="https://user-images.githubusercontent.com/4971414/132436300-06d79f42-3500-486c-b5a4-f7d25fbc48e7.png" alt="image" /></p>
<p>可以看出 shfmt 还有些奇怪的规则:</p>
<ol>
<li>缩进:默认 tab,而不是空格;</li>
<li>重定向后留空格:不允许;</li>
</ol>
<p>查看帮助,发现规则可以自定义:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>shfmt <span class="nt">--help</span>
usage: shfmt <span class="o">[</span>flags] <span class="o">[</span>path ...]
Printer options:
<span class="nt">-i</span> uint indent: 0 <span class="k">for </span>tabs <span class="o">(</span>default<span class="o">)</span>, <span class="o">></span>0 <span class="k">for </span>number of spaces
<span class="nt">-bn</span> binary ops like <span class="o">&&</span> and | may start a line
<span class="nt">-ci</span> switch cases will be indented
<span class="nt">-sr</span> redirect operators will be followed by a space
<span class="nt">-kp</span> keep column alignment paddings
<span class="nt">-fn</span> <span class="k">function </span>opening braces are placed on a separate line
</code></pre></div></div>
<p>参考其他语言的规范,我选择了 4 个空格、重定向后留空格,即:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>shfmt <span class="nt">-d</span> <span class="nt">-i</span> 4 <span class="nt">-sr</span> girl.sh
<span class="nv">$ </span><span class="nb">echo</span> <span class="nv">$?</span>
0
</code></pre></div></div>
<p><img src="https://user-images.githubusercontent.com/4971414/132445294-8daef9d0-09c2-44d8-bc62-1b0fb38494f5.png" alt="image" /></p>
<p>汇总两个工具的规则:</p>
<table>
<thead>
<tr>
<th>规则</th>
<th>ShellCheck</th>
<th>shfmt</th>
</tr>
</thead>
<tbody>
<tr>
<td>for/do 换行</td>
<td>:x:</td>
<td>:white_check_mark:</td>
</tr>
<tr>
<td>for 数据来源先赋值</td>
<td>:x:</td>
<td>:x:</td>
</tr>
<tr>
<td>子语句标志符</td>
<td>:white_check_mark:</td>
<td>:white_check_mark:</td>
</tr>
<tr>
<td>行缩进</td>
<td>:x:</td>
<td>:white_check_mark:</td>
</tr>
<tr>
<td>行内缩进</td>
<td>:x:</td>
<td>:white_check_mark:</td>
</tr>
<tr>
<td>全角引号</td>
<td>:white_check_mark:</td>
<td>:x:</td>
</tr>
<tr>
<td>Useless cat</td>
<td>:white_check_mark:</td>
<td>:x:</td>
</tr>
<tr>
<td>read lines rather than words</td>
<td>:white_check_mark:</td>
<td>:x:</td>
</tr>
<tr>
<td>重定向后留空格</td>
<td>:x:</td>
<td>:white_check_mark:</td>
</tr>
</tbody>
</table>
<p>结论:两个工具一起使用,可覆盖大部分规则。</p>
<h2 id="自动检查">自动检查</h2>
<p>代码规范口头传达难以落地,需要自动强制检查,有 3 个时机:IDE、git pre commit、持续集成。</p>
<p><img src="https://user-images.githubusercontent.com/4971414/132443787-8d5605fc-4b81-4d72-96b5-5a667d19fbe2.png" alt="image" /></p>
<h3 id="ide">IDE</h3>
<ul>
<li>JetBrains:无需安装,内置插件支持 ShellCheck、Shfmt、Explainshell</li>
<li>VSCode:只有 <a href="https://marketplace.visualstudio.com/items?itemName=timonwong.shellcheck">ShellCheck 插件</a>,其他等待大家贡献开源。</li>
</ul>
<p><img src="https://user-images.githubusercontent.com/4971414/132444277-21f62283-1aa9-4470-9b3f-534571dceb5d.png" alt="image" />
<img src="https://user-images.githubusercontent.com/4971414/132441019-8164ed37-2ef0-4a66-8f4a-31b070086ba0.png" alt="image" /></p>
<h3 id="git-pre-commit">git pre commit</h3>
<p>git pre commit 和持续集成都需要执行 lint 程序,按照「DRY」原则,应写成一个脚本,到处执行,比如叫做 <code class="language-plaintext highlighter-rouge">lint.sh</code>:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="nb">set</span> <span class="nt">-e</span>
<span class="k">if</span> <span class="o">[</span> <span class="nt">-z</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">"no file"</span>
<span class="nb">exit
</span><span class="k">fi
</span><span class="nv">shell_files</span><span class="o">=</span><span class="si">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span> | <span class="nb">tr</span> <span class="s1">' '</span> <span class="s1">'\n'</span> | <span class="o">{</span> <span class="nb">grep</span> <span class="s2">".sh$"</span> <span class="o">||</span> <span class="nb">true</span><span class="p">;</span> <span class="o">}</span><span class="si">)</span>
<span class="k">if</span> <span class="o">[</span> <span class="nt">-n</span> <span class="s2">"</span><span class="nv">$shell_files</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">"lint shell:"</span>
<span class="nb">echo</span> <span class="s2">"</span><span class="nv">$shell_files</span><span class="s2">"</span>
<span class="nb">echo</span> <span class="s2">"</span><span class="nv">$shell_files</span><span class="s2">"</span> | xargs shellcheck
<span class="nb">echo</span> <span class="s2">"</span><span class="nv">$shell_files</span><span class="s2">"</span> | xargs shfmt <span class="nt">-d</span> <span class="nt">-i</span> 4 <span class="nt">-sr</span>
<span class="k">else
</span><span class="nb">echo</span> <span class="s2">"no shell file"</span>
<span class="k">fi</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">.git/hooks/pre-commit</code> 增量检查:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
git diff <span class="nt">--diff-filter</span><span class="o">=</span>d <span class="nt">--name-only</span> HEAD | xargs ./lint.sh
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">.git</code> 目录是无法提交的,所以在代码库中提交 <code class="language-plaintext highlighter-rouge">.git-hooks-pre-commit</code>,每位开发者在本地手动复制(没有复制也没关系,后面的持续集成会拦住):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cp .git-hooks-pre-commit .git/hooks/pre-commit
</code></pre></div></div>
<p>效果截图:本地修改代码,<code class="language-plaintext highlighter-rouge">git commit</code> 时,强制检查了代码规范,报错就继续修改,再次 commit。</p>
<p><img src="https://user-images.githubusercontent.com/4971414/132637594-d1bd432a-fd65-4e01-8ac0-682e2431cede.png" alt="image" /></p>
<h3 id="持续集成">持续集成</h3>
<p>按照「前端检查为了用户体验,后端检查为了安全可靠」的原则,git pre commit 检查为了及时反馈,持续集成检查为了安全可靠。</p>
<p>如果有开发者未配置 git pre commit,直接提交了代码(比如在 GitHub 网页上修改代码),将执行同样的规范检查。</p>
<p>CODING/Jenkins:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">stage</span><span class="o">(</span><span class="s1">'增量检查代码规范'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">when</span> <span class="o">{</span>
<span class="n">changeRequest</span><span class="o">()</span>
<span class="o">}</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="c1">// CODING</span>
<span class="n">sh</span> <span class="s2">"git diff --diff-filter=d --name-only origin/${env.MR_TARGET_BRANCH}... | xargs ./lint.sh"</span>
<span class="c1">// Jenkins</span>
<span class="c1">// sh "git diff --diff-filter=d --name-only origin/${env.CHANGE_TARGET}... | xargs ./lint.sh"</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>GitHub Actions:https://github.com/Coding/coding-download-center/blob/main/.github/workflows/ci.yml</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">jobs</span><span class="pi">:</span>
<span class="na">build</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">technote-space/get-diff-action@v5</span>
<span class="na">if</span><span class="pi">:</span> <span class="s">$</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Lint</span>
<span class="na">if</span><span class="pi">:</span> <span class="s">env.GIT_DIFF</span>
<span class="na">run</span><span class="pi">:</span> <span class="s">echo $ | xargs ./lint.sh</span>
</code></pre></div></div>
<p>效果截图:有人修改代码,<code class="language-plaintext highlighter-rouge">git push</code> 并创建了合并请求,强制检查了代码规范,报错就继续修改,再次 push。https://github.com/Coding/coding-download-center/pull/61</p>
<p><img src="https://user-images.githubusercontent.com/4971414/132639829-1e12c5f9-ba53-4494-b6ba-6df860424826.png" alt="image" />
<img src="https://user-images.githubusercontent.com/4971414/132639906-713b41b0-746f-4a85-84af-37090443d539.png" alt="image" /></p>Code Review 时发现大家的 shell 代码风格各异:macOS 11 brew PHP 82020-12-22T18:00:00+08:002020-12-22T18:00:00+08:00/macos-brew-php8<p>right:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mac<span class="nv">$ </span>brew update-reset
mac<span class="nv">$ </span>brew upgrade php
Updating Homebrew...
<span class="o">==></span> Upgrading 1 outdated package:
php 7.4.4 -> 8.0.0_1
mac<span class="nv">$ </span>php <span class="nt">-v</span>
PHP 8.0.0 <span class="o">(</span>cli<span class="o">)</span> <span class="o">(</span>built: Nov 30 2020 13:51:52<span class="o">)</span> <span class="o">(</span> NTS <span class="o">)</span>
mac<span class="nv">$ </span>sw_vers
ProductName: macOS
ProductVersion: 11.1
BuildVersion: 20C69
</code></pre></div></div>
<p>wrong:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mac<span class="nv">$ </span>brew upgrade
/usr/local/Homebrew/Library/Homebrew/version.rb:368:in <span class="sb">`</span>initialize<span class="s1">':
Version value must be a string; got a NilClass () (TypeError)
</span></code></pre></div></div>right:Composer cache-dir changed2020-10-26T18:00:00+08:002020-10-26T18:00:00+08:00/composer-cache-dir<p>Composer <code class="language-plaintext highlighter-rouge">cache-dir</code> has changed on Linux:</p>
<ul>
<li>before: <code class="language-plaintext highlighter-rouge">~/.composer/cache</code></li>
<li>now: <code class="language-plaintext highlighter-rouge">~/.cache/composer</code></li>
</ul>
<p>but not changed on macOS.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ubuntu@vm:~<span class="nv">$ </span>composer <span class="nt">--version</span>
Composer 1.6.3 2018-01-31 16:28:17
ubuntu@vm:~<span class="nv">$ </span>composer config <span class="nt">-g</span> <span class="nt">-l</span> | <span class="nb">grep </span>cache-dir
<span class="o">[</span>cache-dir] /home/ubuntu/.composer/cache
<span class="o">[</span>data-dir] /home/ubuntu/.composer
<span class="o">[</span>cache-files-dir] <span class="o">{</span><span class="nv">$cache</span><span class="nt">-dir</span><span class="o">}</span>/files <span class="o">(</span>/home/ubuntu/.composer/cache/files<span class="o">)</span>
ubuntu@vm:~<span class="nv">$ </span><span class="nb">rm</span> <span class="nt">-rf</span> ~/.composer
ubuntu@vm:~<span class="nv">$ </span>composer <span class="nt">--version</span>
Composer version 2.0.2 2020-10-25 23:03:59
ubuntu@vm:~<span class="nv">$ </span>composer config <span class="nt">-g</span> <span class="nt">-l</span> | <span class="nb">grep </span>cache-dir
<span class="o">[</span>cache-dir] /home/ubuntu/.cache/composer
<span class="o">[</span>data-dir] /home/ubuntu/.composer
<span class="o">[</span>cache-files-dir] <span class="o">{</span><span class="nv">$cache</span><span class="nt">-dir</span><span class="o">}</span>/files <span class="o">(</span>/home/ubuntu/.cache/composer/files<span class="o">)</span>
</code></pre></div></div>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sink@mac<span class="nv">$ </span>composer <span class="nt">--version</span>
Composer version 1.10.9 2020-07-16 12:57:00
sink@mac<span class="nv">$ </span>composer config <span class="nt">-g</span> <span class="nt">-l</span> | <span class="nb">grep </span>cache-dir
<span class="o">[</span>cache-dir] /Users/sinkcup/.composer/cache
<span class="o">[</span>data-dir] /Users/sinkcup/.composer
<span class="o">[</span>cache-files-dir] <span class="o">{</span><span class="nv">$cache</span><span class="nt">-dir</span><span class="o">}</span>/files <span class="o">(</span>/Users/sinkcup/.composer/cache/files<span class="o">)</span>
sink@mac<span class="nv">$ </span><span class="nb">rm</span> <span class="nt">-rf</span> ~/.composer
ubuntu@vm:~<span class="nv">$ </span>composer <span class="nt">--version</span>
Composer version 2.0.2 2020-10-25 23:03:59
sink@mac<span class="nv">$ </span>composer config <span class="nt">-g</span> <span class="nt">-l</span> | <span class="nb">grep </span>cache-dir
<span class="o">[</span>cache-dir] /Users/sinkcup/.composer/cache
<span class="o">[</span>data-dir] /Users/sinkcup/.composer
</code></pre></div></div>Composer cache-dir has changed on Linux:Jenkins git diff files when merge request2020-07-03T17:18:00+08:002020-07-03T17:18:00+08:00/jenkins-git-diffs<p>I need Jenkins to lint files changed in a merge request, so I run this command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git diff --diff-filter=ACM --name-only master... | xargs npx remark -f
</code></pre></div></div>
<p>but error:</p>
<blockquote>
<p>fatal: ambiguous argument ‘master…’: unknown revision or path not in the working tree.</p>
</blockquote>
<p>That’s because Jenkins only checkout the code by commit id, don’t checkout master or any branch.</p>
<p>I find there are 2 ways from Google search, but not very good:</p>
<ul>
<li>https://stackoverflow.com/questions/57893389/jenkins-file-diffs-with-declarative-pipeline-and-multibranch</li>
<li>https://www.covermymeds.com/main/insights/articles/jenkins-test-files-changed-git-commit/</li>
</ul>
<p>Finally, I invented a new method:</p>
<script src="https://gist.github.com/sinkcup/7df2614b09bb89450721ccb67a9cfe2f.js"></script>I need Jenkins to lint files changed in a merge request, so I run this command:静态网站架构的演进和最佳实践2020-05-29T17:26:00+08:002020-05-29T17:26:00+08:00/arch-of-static-web<p>新冠肺炎期间,<a href="https://xw.qq.com/cmsid/20200415A0G9BM00">约翰霍普金斯大学开发的疫情地图网站单日 PV 达 10 亿次</a>,查看源代码可以看出它是一个单页应用(single-page application,缩写 SPA),JS 调用后端 API 返回动态数据。</p>
<p>那么,部署一个 10 亿 PV 的静态网站需要购买几台服务器?</p>
<p>答案是:0 台——在云计算时代,静态网站已不再需要服务器,部署到云存储,开启 CDN 即可全球高速访问。</p>
<p><img src="https://user-images.githubusercontent.com/4971414/83229629-e703db00-a1ba-11ea-8a56-60990eed2739.png" alt="约翰霍普金斯大学 arcgis 疫情地图" /></p>
<h2 id="静态网站架构发展史">静态网站架构发展史</h2>
<p>1991 年,万维网诞生,包括 3 项关键技术:统一资源标志符(URI)、HTML、HTTP。</p>
<p>初期的网站架构很简单,手写 HTML 或者用程序生成 HTML,通过 FTP/SCP 等方式上传到服务器。</p>
<p>HTML 文件并不需要运算,不消耗性能,一台服务器可以支撑很多个网站,而自行购买一台服务器只部署一个网站,成本高昂。所以网页托管服务(Web hosting service)应运而生,价格低廉甚至免费(通过嵌入广告盈利)。</p>
<p>1993 年,CGI 诞生,Web 服务器收到浏览器请求,执行对应的 CGI 程序,动态输出 HTML,这就是前后端混合的模式。在此之后的很多年里,前后端是一个项目,一起部署到服务器。</p>
<p>1999 年,Ajax 诞生。</p>
<p>2004 年,Gmail 大规模使用符合标准的跨浏览器 Ajax,前后端分离逐渐流行起来。</p>
<p>2006 年,AWS 发布了云存储,宣告了云计算时代的诞生。HTML/CSS/JS 作为简单的小文件,无需特殊处理,部署到云存储,再配合 CDN,成了静态网站架构最佳实践,有如下优点:</p>
<ul>
<li>成本低:云存储/CDN 比服务器便宜很多(比如「<a href="https://url.cn/53ljQjJ">腾讯云 对象存储 COS</a>」约 0.1 元/GB/月、<a href="https://buy.cloud.tencent.com/price/cdn">腾讯云 CDN 每月赠送 10GB 流量</a>),一个典型的公司官网一年费用不超过 10 元;</li>
<li>访问快:CDN 能在全国甚至全球快速访问,比服务器更快。</li>
</ul>
<p>2010 年起,AngularJS、Vue.js、React 等框架陆续诞生,开发的单页应用(SPA)使用 Ajax 技术实现了彻底的前后端分离,也意味着前后端单独部署。</p>
<p>目前,静态网站有 2 种:</p>
<ul>
<li>无内容的单页应用(SPA):React/VUE 等框架开发的应用;</li>
<li>有内容的 HTML:手写或「程序生成 HTML」;</li>
</ul>
<p><strong>警告</strong>:React/VUE SPA 不带内容,难以被搜索引擎收录,不适合作为公司官网、博客。如果有 SEO 需求,推荐使用 <a href="https://www.mkdocs.org/">MkDocs</a>、<a href="https://hexo.io/zh-cn/">Hexo</a>、<a href="https://zh.nuxtjs.org/">VUE Nuxt</a>、<a href="https://nextjs.org/">React Next</a>。</p>
<h2 id="实战静态网站自动部署到云存储">实战:静态网站自动部署到云存储</h2>
<p>通过「持续集成」生成 HTML,自动部署到「云存储」,变成静态网站。</p>
<ol>
<li>
<p>在「<a href="(https://url.cn/53ljQjJ)">腾讯云 对象存储 COS</a>」中创建一个「公有读私有写」的「存储桶」,并在设置中开启「静态网站」,获得分配的二级域名「访问链接」。
<img src="https://user-images.githubusercontent.com/4971414/83237611-c2623000-a1c7-11ea-9ab4-67c8c33587bc.png" alt="腾讯云存储 COS 创建存储桶" />
<img src="https://user-images.githubusercontent.com/4971414/83237837-1c62f580-a1c8-11ea-8750-1a50d85a8dab.png" alt="腾讯云存储 COS 开启静态网站" /></p>
</li>
<li>
<p>把项目代码推送到「<a href="https://coding.net/products/repo?cps_source=PIevZ6Jr">CODING 代码仓库</a>」,在「<a href="https://coding.net/products/ci?cps_source=PIevZ6Jr">CODING 持续集成</a>」中创建一个构建计划,选择「构建并上传到腾讯云 COS」模板,填入「腾讯云 COS 访问密钥」等信息,根据自己的代码框架修改编译命令。创建后会自动触发构建,等待构建成功,访问分配的链接即可看到网站。下次推送代码即可自动部署。
<img src="https://user-images.githubusercontent.com/4971414/83238886-a8c1e800-a1c9-11ea-82ab-aab4d702f7b7.png" alt="CODING 持续集成 模板列表" />
<img src="https://user-images.githubusercontent.com/4971414/83239641-d0fe1680-a1ca-11ea-8b4f-cf9b025de135.png" alt="CODING 持续集成 模板填写参数" />
<img src="https://user-images.githubusercontent.com/4971414/83240752-69e16180-a1cc-11ea-9d45-7bfa6a107d7e.png" alt="CODING 持续集成 构建成功" /></p>
</li>
<li>
<p>如果你的域名已备案,则可在「腾讯云 对象存储」的设置中绑定「自定义加速域名」,会提示开通「内容分发网络 CDN」,小型网站推荐选择「按使用流量计费」(<a href="https://buy.cloud.tencent.com/price/cdn">每月赠送 10GB</a>,一般用不完),将会获得一个 CNAME。
<img src="https://user-images.githubusercontent.com/4971414/83241781-e88ace80-a1cd-11ea-9e75-387c368f32b9.png" alt="腾讯云存储 COS 自定义加速域名" /></p>
</li>
<li>在「DNS 解析」中设置 www 和 根域名,确保两者皆可访问,并且二选一进行跳转避免影响 SEO,推荐 2 种方案:
<ul>
<li>此域名无邮箱:根域名指向 CDN,www 跳转到根域名,本文采用此方案,把 <a href="http://www.devops.host">www.devops.host</a> 跳转到了 <a href="https://devops.host">devops.host</a>;</li>
<li>此域名有邮箱:www 指向 CDN,根域名跳转到 www;
<img src="https://user-images.githubusercontent.com/4971414/83242946-c8f4a580-a1cf-11ea-92bd-644c762b0059.png" alt="DNS 解析 no-www" /></li>
</ul>
</li>
<li>
<p>在「内容分发网络 CDN」——「域名管理」——某个域名——「高级设置」——「HTTPS 配置」中,申请免费的 HTTPS 证书,并开启「HTTPS 回源」、「强制跳转 HTTPS」和「HTTP 2.0」。
<img src="https://help-assets.codehub.cn/enterprise/20200227181247.png" alt="腾讯云 内容分发网络 开启 HTTPS" /></p>
</li>
<li>开通「<a href="https://url.cn/5pbRzdO">腾讯云 云函数 SCF</a>」,按照文档「<a href="https://cloud.tencent.com/document/product/436/30434">使用 SCF 自动刷新被 CDN 缓存的 COS 资源</a>」上传代码。
<img src="https://help-assets.codehub.cn/enterprise/20200306134243.png" alt="腾讯云 SCF 自动刷新 CDN COS" /></li>
</ol>
<p><strong>提醒</strong>:</p>
<ul>
<li>如果网站面向中国境内用户,需要进行域名备案,然后才能绑定到境内的云存储/CDN。本文以腾讯云为例,其他云计算厂商的流程也类似。</li>
<li>如果网站面向中国境外用户,可直接使用境外云存储/CDN 搭建静态网站。</li>
</ul>
<h3 id="jenkinsfile">Jenkinsfile</h3>
<p>浏览<a href="https://coding-public.coding.net/p/html-cos-demo/d/html-cos-demo/git">完整代码</a>。</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pipeline</span> <span class="o">{</span>
<span class="n">agent</span> <span class="n">any</span>
<span class="n">stages</span> <span class="o">{</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'检出'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="n">checkout</span><span class="o">([</span><span class="n">$class</span><span class="o">:</span> <span class="s1">'GitSCM'</span><span class="o">,</span>
<span class="nl">branches:</span> <span class="o">[[</span><span class="nl">name:</span> <span class="n">env</span><span class="o">.</span><span class="na">GIT_BUILD_REF</span><span class="o">]],</span>
<span class="nl">userRemoteConfigs:</span> <span class="o">[[</span>
<span class="nl">url:</span> <span class="n">env</span><span class="o">.</span><span class="na">GIT_REPO_URL</span><span class="o">,</span>
<span class="nl">credentialsId:</span> <span class="n">env</span><span class="o">.</span><span class="na">CREDENTIALS_ID</span>
<span class="o">]]])</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'安装依赖'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="n">sh</span> <span class="s1">'npm i -g lint-md-cli'</span>
<span class="n">sh</span> <span class="s1">'pip install mkdocs'</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'检查书写规范'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="n">sh</span> <span class="s1">'lint-md docs/'</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'编译'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="n">sh</span> <span class="s1">'mkdocs build'</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'上传到 COS Bucket'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="n">sh</span> <span class="s2">"coscmd config -a ${env.COS_SECRET_ID} -s ${env.COS_SECRET_KEY}"</span> <span class="o">+</span>
<span class="s2">" -b ${env.COS_BUCKET_NAME} -r ${env.COS_BUCKET_REGION}"</span>
<span class="n">sh</span> <span class="s1">'coscmd upload -r ./site/ /'</span>
<span class="n">echo</span> <span class="s2">"预览 https://${env.COS_BUCKET_NAME}.cos-website.${env.COS_BUCKET_REGION}.myqcloud.com/"</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<h3 id="持续集成环境变量">持续集成环境变量</h3>
<table>
<thead>
<tr>
<th>变量名</th>
<th>含义</th>
<th>参考值</th>
</tr>
</thead>
<tbody>
<tr>
<td>COS_SECRET_ID</td>
<td>腾讯云访问密钥 ID</td>
<td>stringLength36stringLength36string36</td>
</tr>
<tr>
<td>COS_SECRET_KEY</td>
<td>腾讯云访问密钥 KEY</td>
<td>stringLength32stringLength323232</td>
</tr>
<tr>
<td>COS_BUCKET_NAME</td>
<td>腾讯云对象存储桶</td>
<td>devops-host-1257110097</td>
</tr>
<tr>
<td>COS_BUCKET_REGION</td>
<td>腾讯云对象存储区域</td>
<td>ap-nanjing</td>
</tr>
</tbody>
</table>新冠肺炎期间,约翰霍普金斯大学开发的疫情地图网站单日 PV 达 10 亿次,查看源代码可以看出它是一个单页应用(single-page application,缩写 SPA),JS 调用后端 API 返回动态数据。Install Android Command line tools2020-05-27T15:03:00+08:002020-05-27T15:03:00+08:00/android-cmdline-tools<h2 id="macosvia-homebrew">macOS(via Homebrew)</h2>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sw_vers
<span class="c"># ProductName: Mac OS X</span>
<span class="c"># ProductVersion: 10.15.4</span>
brew tap caskroom/cask
brew cask <span class="nb">install </span>homebrew/cask-versions/adoptopenjdk8
<span class="nb">export </span><span class="nv">JAVA_HOME</span><span class="o">=</span><span class="sb">`</span>/usr/libexec/java_home <span class="nt">-v</span> 1.8<span class="sb">`</span>
brew cask <span class="nb">install </span>android-sdk
sdkmanager <span class="nt">--list</span> | <span class="nb">awk</span> <span class="s1">'/Installed/{flag=1; next} /Available/{flag=0} flag'</span>
</code></pre></div></div>
<h3 id="error-noclassdeffounderror">Error: NoClassDefFoundError</h3>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sdkmanager <span class="nt">--list</span>
<span class="c"># Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema</span>
</code></pre></div></div>
<p>Fix:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew cask <span class="nb">install </span>homebrew/cask-versions/adoptopenjdk8
<span class="nb">export </span><span class="nv">JAVA_HOME</span><span class="o">=</span><span class="sb">`</span>/usr/libexec/java_home <span class="nt">-v</span> 1.8<span class="sb">`</span>
</code></pre></div></div>
<h2 id="macosmanual-way">macOS(manual way)</h2>
<ol>
<li>download <a href="https://developer.android.com/studio#command-tools">Android Command line tools</a></li>
</ol>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> ~/code/android_sdk/cmdline-tools <span class="o">&&</span> <span class="nb">cd</span> <span class="s2">"</span><span class="nv">$_</span><span class="s2">"</span>
unzip ~/Downloads/commandlinetools-mac-6200805_latest.zip
</code></pre></div></div>
<ol>
<li>set env</li>
</ol>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">ANDROID_SDK_ROOT</span><span class="o">=</span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/code/android_sdk"</span>
<span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="s2">"</span><span class="nv">$PATH</span><span class="s2">:</span><span class="nv">$ANDROID_SDK_ROOT</span><span class="s2">/tools/bin:</span><span class="nv">$ANDROID_SDK_ROOT</span><span class="s2">/tools/lib:</span><span class="nv">$ANDROID_SDK_ROOT</span><span class="s2">/platform-tools"</span>
</code></pre></div></div>
<ol>
<li>install tools</li>
</ol>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./tools/bin/sdkmanager <span class="nt">--sdk_root</span><span class="o">=</span><span class="nv">$ANDROID_SDK_ROOT</span> <span class="nt">--install</span> tools
</code></pre></div></div>
<ol>
<li>install Java 1.8</li>
</ol>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew cask install homebrew/cask-versions/adoptopenjdk8
export JAVA_HOME=`/usr/libexec/java_home -v 1.8`
sdkmanager --list
</code></pre></div></div>
<h3 id="warning-could-not-create-settings">Warning: Could not create settings</h3>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>android_sdk/tools/bin/sdkmanager <span class="nt">--list</span>
<span class="c"># Warning: Could not create settings</span>
<span class="c"># java.lang.IllegalArgumentException</span>
<span class="c"># at com.android.sdklib.tool.sdkmanager.SdkManagerCliSettings.<init>(SdkManagerCliSettings.java:428)</span>
</code></pre></div></div>
<p>Fix:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> <span class="nt">-p</span> cmdline-tools
<span class="nb">mv </span>android_sdk/tools android_sdk/cmdline-tools/
android_sdk/cmdline-tools/tools/bin/sdkmanager <span class="nt">--sdk_root</span><span class="o">=</span><span class="nv">$ANDROID_SDK_ROOT</span> <span class="nt">--install</span> tools
<span class="nb">ls</span> <span class="nt">-l</span> android_sdk/tools/bin/sdkmanager
</code></pre></div></div>
<h3 id="warning-observed-package-in-inconsistent-location">Warning: Observed package in inconsistent location</h3>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>android_sdk/cmdline-tools/tools/bin/sdkmanager <span class="nt">--sdk_root</span><span class="o">=</span>../ <span class="nt">--list</span>
<span class="c"># Warning: Observed package id 'tools' in inconsistent location '../cmdline-tools/latest' (Expected '../tools')</span>
</code></pre></div></div>
<p>Fix:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>android_sdk/cmdline-tools/tools/bin/sdkmanager --sdk_root=$ANDROID_SDK_ROOT --install tools
android_sdk/tools/bin/sdkmanager --list
</code></pre></div></div>macOS(via Homebrew)中国速度:getcomposer 下载加速2019-12-30T17:15:00+08:002019-12-30T17:15:00+08:00/china-speed-mirror-of-getcomposer<p>由于 composer 官网在国外,国际出口带宽有限,导致中国内地开发者下载很慢甚至超时。composer install 加速已经有云计算大厂提供了,比如 <a href="https://developer.aliyun.com/composer">阿里云</a>、<a href="https://mirrors.cloud.tencent.com/help/composer.html">腾讯云</a>。而 getcomposer 没有好的加速方式,本地开发和 docker build 在第一步就卡住了,本文将介绍一种好办法。</p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-19508b0c9f8f9cf0.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="内地 getcomposer 下载失败" /></p>
<h2 id="结论">结论</h2>
<p>很简单,在官方域名后面加上 .mirrors.china-speed.org.cn(中国速度) 即可。</p>
<p>国外官方:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
</code></pre></div></div>
<p>中国速度:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl -sS http://getcomposer.org.mirrors.china-speed.org.cn/installer | php -- --install-dir=/usr/local/bin --filename=composer
</code></pre></div></div>
<h2 id="技术细节">技术细节</h2>
<p>分析 <code class="language-plaintext highlighter-rouge">installer</code> 源码可以发现:安装时会请求 <code class="language-plaintext highlighter-rouge">getcomposer.org/versions</code>,获得最新版下载地址:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
"stable": [{"path": "/download/1.9.1/composer.phar", "version": "1.9.1", "min-php": 50300}],
"preview": [{"path": "/download/1.9.1/composer.phar", "version": "1.9.1", "min-php": 50300}],
"snapshot": [{"path": "/composer.phar", "version": "f05e78aa9ee48a317142543da8b9c02613951273", "min-php": 50300}]
}
</code></pre></div></div>
<p>然后根据里面的地址下载 <code class="language-plaintext highlighter-rouge">composer.phar</code>,最后还会请求 <code class="language-plaintext highlighter-rouge">composer.phar.sig</code> 进行文件校验。</p>
<h3 id="方案">方案</h3>
<p>第1步:云存储回源 getcomposer.org,配置:经常删除 <code class="language-plaintext highlighter-rouge">versions</code>(即可回源最新版),其他文件无需刷新(比如 <code class="language-plaintext highlighter-rouge">download/</code>);
第2步:给云存储套上 CDN,配置:<code class="language-plaintext highlighter-rouge">/versions</code> 和 <code class="language-plaintext highlighter-rouge">/installer</code> 经常过期刷新,配置 HTTPS;
第3步:<a href="https://coding.net/products/ci?cps_source=PIevZ6Jr">持续集成</a>定时抓取最新版 <code class="language-plaintext highlighter-rouge">https://getcomposer.org/installer</code>,把里面的域名改成国内 CDN,放到云存储;</p>
<p>阿里云和腾讯云都做了 packagist.org 的镜像,而没做 getcomposer.org 的镜像,阿里云手动挑了几个文件供大家下载,一旦 <code class="language-plaintext highlighter-rouge">versions</code> 更新将导致安装失败。</p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-8e03372ecfb5bc9b.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="大厂没做 getcomposer 镜像" /></p>
<p>大厂没提供,所以只能自己做。</p>
<h3 id="实战">实战</h3>
<p>第1步:创建一个云存储,配置回源,本文以<a href="https://cloud.tencent.com/act/cps/redirect?redirect=10042&cps_key=16b83d1aa2e322d67b11fa1daaa4ab6b">腾讯云存储</a>为例。</p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-511375084329dabb.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="腾讯云存储免费6个月" />
<img src="https://upload-images.jianshu.io/upload_images/52469-20212adc5a6bab57.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="腾讯云存储回源设置" /></p>
<p>然后我们通过云存储赠送的子域名试一下,第一次请求返回302回源,第二次是200直接返回。</p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-7e53e3aa45cfb49d.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="第一次302回源,第二次200" /></p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-6ae6741adb740f69.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="腾讯云存储生命周期规则" /></p>
<p>第2步:给云存储套上 CDN;</p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-646399a93bfd54db.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="腾讯云存储-自定义加速域名" />
<img src="https://upload-images.jianshu.io/upload_images/52469-04b0c804fe8471cd.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="腾讯云-CDN 缓存配置" />
<img src="https://upload-images.jianshu.io/upload_images/52469-f368b9bf7e1ea0a6.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="腾讯云-CDN 配置 HTTPS" /></p>
<p>第3步:在 <a href="https://coding.net/products/ci?cps_source=PIevZ6Jr">CODING 持续集成(免费的 Jenkins 云服务)</a> 中创建一个构建任务,定时抓取 <code class="language-plaintext highlighter-rouge">https://getcomposer.org/installer</code>。</p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-4ce64e98aaa68f9a.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="CODING Jenkins 云服务" /></p>
<p><img src="https://upload-images.jianshu.io/upload_images/52469-dc33c46d575482e6.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="CODING Jenkins 定时触发" /></p>
<p><code class="language-plaintext highlighter-rouge">Jenkinsfile</code> 代码如下:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pipeline {
agent any
stages {
stage('检出') {
steps {
checkout(
[$class: 'GitSCM', branches: [[name: env.GIT_BUILD_REF]],
userRemoteConfigs: [[url: env.GIT_REPO_URL, credentialsId: env.CREDENTIALS_ID]]]
)
}
}
stage('构建') {
steps {
echo '构建中...'
sh 'wget https://getcomposer.org/installer -O web/installer'
sh 'sed -i "s|getcomposer.org\'|getcomposer.org.mirrors.china-speed.org.cn\'|g" web/installer'
sh 'git diff'
echo '构建完成.'
}
}
stage('测试') {
steps {
echo '测试中...'
sh 'php web/installer --install-dir=./ --filename=composer'
sh './composer --version'
echo '测试完成.'
}
}
stage('部署') {
steps {
echo '部署中...'
sh 'apt-get update && apt-get install -y python3-pip'
sh 'pip3 install coscmd'
sh "coscmd config -a $TENCENT_SECRET_ID -s $TENCENT_SECRET_KEY -b $TENCENT_BUCKET -r $TENCENT_REGION"
sh 'coscmd upload web/installer /'
echo '部署完成'
}
}
}
}
</code></pre></div></div>
<h2 id="鸣谢">鸣谢</h2>
<p>感谢 <a href="https://cloud.tencent.com/act/cps/redirect?redirect=10042&cps_key=16b83d1aa2e322d67b11fa1daaa4ab6b">腾讯云(免费6个月)</a>、<a href="https://portal.qiniu.com/signup?code=1h6w1ounb13yp">七牛云(每月免费 10GB)</a> 提供云存储和国内 CDN。</p>
<p>感谢 <a href="https://coding.net/products/ci?cps_source=PIevZ6Jr">CODING 持续集成</a> 提供免费的 Jenkins 云服务。</p>
<p>通过上述邀请链接注册,本站将获得流量奖励,供大家下载使用。</p>由于 composer 官网在国外,国际出口带宽有限,导致中国内地开发者下载很慢甚至超时。composer install 加速已经有云计算大厂提供了,比如 阿里云、腾讯云。而 getcomposer 没有好的加速方式,本地开发和 docker build 在第一步就卡住了,本文将介绍一种好办法。Jenkins 保存 Docker 镜像实现极速构建2019-12-05T17:58:00+08:002019-12-05T17:58:00+08:00/jenkins-docker-push<p>上一篇文章<a href="https://sinkcup.github.io/jenkins-docker-background">《如何在Jenkins中跑多个Docker(MySQL、Redis等)?》</a>讲解了用 <code class="language-plaintext highlighter-rouge">Dockerfile</code> 跑 Jenkins,解决了“CircleCI 的 Dockerfile 和 config.yml 重复,违反 DRY”的问题。</p>
<p>但也面临一个新问题:</p>
<ul>
<li>Jenkins 每次跑测试都要先进行 docker build,很慢。</li>
</ul>
<p>解决办法很简单:把 Jenkins 构建的 Docker 镜像存起来,下次直接拉取。</p>
<h2 id="检查-docker-image-是否存在远端镜像服务器">检查 Docker image 是否存在远端镜像服务器</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect sinkcup/laravel-demo:asdf
no such manifest: docker.io/sinkcup/laravel-demo:asdf
$ echo $?
1
</code></pre></div></div>
<h2 id="把-jenkins-构建的-docker-镜像存起来">把 Jenkins 构建的 Docker 镜像存起来</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sh "docker login -u $DOCKER_USER -p $DOCKER_PASSWORD $DOCKER_SERVER"
md5 = sh(script: "md5sum Dockerfile | awk '{print \$1}'", returnStdout: true).trim()
imageAndTag = "${DOCKER_SERVER}${DOCKER_PATH_PREFIX}laravel-demo:dev-${md5}"
dockerNotExists = sh(script: "DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect $imageAndTag > /dev/null", returnStatus: true)
def testImage = null
if (dockerNotExists) {
testImage = docker.build("$imageAndTag", "--build-arg APP_ENV=testing --build-arg SPEED=$SPEED ./")
sh "docker push $imageAndTag"
} else {
testImage = docker.image(imageAndTag)
}
testImage.inside("--net bridge1 -v \"${codePath}:/var/www/laravel\" -e 'APP_ENV=testing'") {
stage('prepare') {
echo 'preparing...'
sh 'composer install'
echo 'prepare done.'
}
stage('test') {
echo 'testing...
sh './lint.sh'
sh './phpunit.sh'
junit 'junit.xml'
echo 'test done.'
}
}
</code></pre></div></div>
<p>完整代码请看:<a href="https://github.com/sinkcup/laravel-demo/blob/6.x/Jenkinsfile">GitHub</a>、<a href="https://codes-farm.coding.net/p/laravel-demo/d/laravel-demo/git/blob/6.x/Jenkinsfile">CODING</a></p>
<p>当然需要先获得 Docker 镜像仓库的密码,有这些服务商:</p>
<ul>
<li>国外的 <a href="https://hub.docker.com/r/sinkcup/laravel-demo">hub.docker.com</a>:1个免费私有额度,7美元/月含5个私有额度,建议搭配国外的持续集成使用;</li>
<li>国内的 <a href="https://coding.net/products/artifact?cps_source=PIevZ6Jr">coding.net - 制品库</a>:免费,搭配 <a href="https://coding.net/products/ci?cps_source=PIevZ6Jr">CODING CI(Jenkins 云服务)</a> 使用;</li>
<li>国内的 <a href="https://cloud.tencent.com/act/cps/redirect?redirect=10058&cps_key=16b83d1aa2e322d67b11fa1daaa4ab6b">腾讯云 - 镜像仓库</a>:免费;</li>
<li>国内的 <a href="https://www.aliyun.com/product/eci">阿里云-容器镜像服务</a>:默认版免费,企业版公测价格未定;</li>
</ul>
<p><img src="https://user-images.githubusercontent.com/4971414/70224077-979e9e00-1787-11ea-8a4f-a88ef5f958d6.png" alt="docker hub 价格" /></p>
<p>经过试验,<a href="https://github.com/sinkcup/laravel-demo">同一个项目</a>在不同平台的持续集成速度如下:</p>
<ul>
<li>Jenkins 第一次运行:7分钟</li>
<li>Jenkins 第二次运行:1分钟24秒</li>
<li>CircleCI:2分钟-4分47秒</li>
</ul>
<p><img src="https://user-images.githubusercontent.com/4971414/70224596-81451200-1788-11ea-8898-a76b53dc3777.png" alt="CODING CI(Jenkins 云服务)" />
<img src="https://user-images.githubusercontent.com/4971414/70224609-886c2000-1788-11ea-9366-4dad4c562b29.png" alt="CircleCI" /></p>上一篇文章《如何在Jenkins中跑多个Docker(MySQL、Redis等)?》讲解了用 Dockerfile 跑 Jenkins,解决了“CircleCI 的 Dockerfile 和 config.yml 重复,违反 DRY”的问题。如何在Jenkins中跑多个Docker(MySQL、Redis等)?2019-11-27T14:11:00+08:002019-11-27T14:11:00+08:00/jenkins-docker-background<p>在写测试代码时,我们会 mock 各种外部服务(比如测转账,显然不能真的调用银行转账接口),但不会 mock 基础设施,比如 MySQL、Redis、<a href="https://www.elastic.co/cn/cloud/">Elasticsearch</a>。</p>
<p>而且跑测试用的 MySQL 等服务都是一次性的,跑完就清空数据不再需要了,所以买台服务器长期开着是不划算的。最佳实践是:测试时启动多个 Docker 后台服务。</p>
<p>在 CircleCI 中,<a href="https://circleci.com/docs/2.0/postgres-config/#example-mysql-project">启动多个 Docker image 很方便</a>,但不支持 Dockerfile,导致 <a href="https://github.com/sinkcup/laravel-demo/blob/6.x/Dockerfile#L6">Dockerfile</a> 和 <a href="https://github.com/sinkcup/laravel-demo/blob/6.x/.circleci/config.yml#L17">config.yml</a> 重复,违反 DRY。</p>
<p>在 Jenkins 中也可以实现,而且更强大(支持 Dockerfile)。</p>
<p>查看 <a href="https://jenkins.io/zh/doc/book/pipeline/docker/#%E8%BF%90%E8%A1%8C-sidecar-%E5%AE%B9%E5%99%A8">Jenkins 官方文档</a>,发现 sidecar 嵌套支持2个 Docker,如果更多会导致嵌套太深难以维护(比如 MySQL、Redis 作为服务,debian 跑 tests);而且用了 <a href="https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/">link</a>,已被 Docker 官方淘汰:</p>
<blockquote>
<p>Warning: The –link flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, we recommend that you use user-defined networks to facilitate communication between two containers instead of using –link.</p>
</blockquote>
<p>按照 Docker 官方推荐,创建一个 network 即可。关键在于等待启动成功,代码如下(<a href="https://codes-farm.coding.net/p/laravel-demo/d/laravel-demo/git/blob/6.x-coding/Jenkinsfile">完整代码</a>):</p>
<script src="https://gist.github.com/sinkcup/30288ca2252f9463b79c30306fedc649.js"></script>
<p>已在 <a href="https://coding.net/products/ci?cps_source=PIevZ6Jr">coding.net 持续集成</a>(免费的 Jenkins 云服务)中运行成功,截图:</p>
<p><img src="https://user-images.githubusercontent.com/4971414/69701055-fbd9b600-1126-11ea-9028-1330a091150a.png" alt="CODING 持续集成" /></p>在写测试代码时,我们会 mock 各种外部服务(比如测转账,显然不能真的调用银行转账接口),但不会 mock 基础设施,比如 MySQL、Redis、Elasticsearch。