gitlab-ciキャッシュはdocker runnerでどのように機能していますか？ / cacheディレクトリとは何ですか？ cache_dirとは何ですか？

Question

Gitlab-ciキャッシュはdocker runnerでどのように機能していますか？
/ cacheディレクトリとは何ですか？
Cache_dirとは何ですか？
"cache" gitlab-ci.yml の---（ "paths"）に一致するファイルはどこにどのように保存されますか？

srghma · Accepted Answer

/cacheディレクトリにマウントされたボリュームは、gitlab-runnerインストール時に自動的に作成され、cache_dir設定によって管理されます

cache_dirの詳細：

If you modify the /cache storage path, you also need to make sure to mark this directory as persistent by defining it in volumes = ["/my/cache/"] under the [runners.docker] section in config.toml.

TLDR

/cache dirはgitlab-ci.ymlのcache configとは異なります

ジョブコンテナーの/cache dirは、キャッシュされたファイルが格納される場所です
gitlab-ci.ymlのcache configに一致するファイルは、ジョブの最後に/cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>にコピーされます
プロジェクトの「パイプライン」ページの「Clear Runner Caches」ボタンは、gitlab-ci.ymlのcache configで指定されたディレクトリに/cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>/cache.Zipを抽出しないようにスケジュールします（「/ cacheフォルダーのコンテンツを削除する代わりに最初）

追伸.

Gitlab-runner（ https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382 ）を使用するマシンで作成されたgitlab-runner-cacheという名前のコンテナーがあります

（と思われる）このコンテナーは、/ cacheデータが格納される匿名ボリュームを作成するために使用されます。匿名ボリュームが作成された後、このコンテナーは停止します。

ジョブコンテナー（コンテナーは通常、テストが実行されたことを意味します）はこの匿名ボリュームをマウントします

証拠

Gitlab-ci.ymlを持っている

image: srghma/docker-nixos-with-git-crypt cache: key: "test00000" # to reset cache - change this key OR clear cache in project settings page paths: - .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically) testtest: script: - nix-env -i tree - tree --dirsfirst -L 4 /cache - ls -al ./.mycache || true - echo "test" > /cache/test - mkdir -p ./.mycache - echo "test" > ./.mycache/test - tree --dirsfirst -L 4 /cache - ls -al ./.mycache || true

出力：

最初の実行時

Running with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980da Using Docker executor with image srghma/docker-nixos-with-git-crypt ... Pulling docker image srghma/docker-nixos-with-git-crypt ... Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ... Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com... Fetching changes... Removing .mycache/ HEAD is now at 675caa7 feat: cache update From https://gitlab.com/srghma/myproject 675caa7..3d1e223 nix -> Origin/nix Checking out 3d1e2237 as nix... Skipping Git submodules setup Checking cache for test00000-11... No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache $ nix-env -i tree installing 'tree-1.8.0' these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0 copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'... building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'... created 233 symlinks in user environment $ tree --dirsfirst -L 4 /cache /cache 0 directories, 0 files $ ls -al ./.mycache || true $ echo "test" > /cache/test ls: ./.mycache: No such file or directory $ mkdir -p ./.mycache $ echo "test" > ./.mycache/test $ tree --dirsfirst -L 4 /cache /cache `-- test 0 directories, 1 file $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:44 . drwxrwxrwx 20 root root 4096 Feb 24 11:44 .. -rw-r--r-- 1 root root 5 Feb 24 11:44 test Creating cache test00000-11... .mycache: found 2 matching files No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. Created cache Job succeeded

2回目の実行時

Running with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980da Using Docker executor with image srghma/docker-nixos-with-git-crypt ... Pulling docker image srghma/docker-nixos-with-git-crypt ... Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ... Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com... Fetching changes... Removing .mycache/ HEAD is now at 3d1e223 feat: cache update Checking out 3d1e2237 as nix... Skipping Git submodules setup Checking cache for test00000-11... No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache $ nix-env -i tree installing 'tree-1.8.0' these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0 copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'... building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'... created 233 symlinks in user environment $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:44 . drwxrwxrwx 20 root root 4096 Feb 24 11:44 .. -rw-r--r-- 1 root root 5 Feb 24 11:44 test $ echo "test" > /cache/test $ mkdir -p ./.mycache $ echo "test" > ./.mycache/test $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:44 . drwxrwxrwx 20 root root 4096 Feb 24 11:44 .. -rw-r--r-- 1 root root 5 Feb 24 11:44 test Creating cache test00000-11... .mycache: found 2 matching files No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. Created cache Job succeeded

プロジェクトの「パイプライン」ページで「ランナーキャッシュをクリア」をクリックしてキャッシュをクリアした後

Running with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980da Using Docker executor with image srghma/docker-nixos-with-git-crypt ... Pulling docker image srghma/docker-nixos-with-git-crypt ... Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ... Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com... Fetching changes... Removing .mycache/ HEAD is now at 3d1e223 feat: cache update Checking out 3d1e2237 as nix... Skipping Git submodules setup Checking cache for test00000-12... No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache $ nix-env -i tree installing 'tree-1.8.0' these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0 copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'... building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'... created 233 symlinks in user environment $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true ls: ./.mycache: No such file or directory $ echo "test" > /cache/test $ mkdir -p ./.mycache $ echo "test" > ./.mycache/test $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:45 . drwxrwxrwx 20 root root 4096 Feb 24 11:45 .. -rw-r--r-- 1 root root 5 Feb 24 11:45 test Creating cache test00000-12... .mycache: found 2 matching files No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. Created cache Job succeeded