在「Equinix Sunset, future of gitlab.fd.o」這邊看到因為贊助 gitlab.freedesktop.org (gitlab.fd.o) 的 Equinix 會停掉服務,所以接下來會需要搬家:
Equinix is shutting down its operations with us on April 30, 2025. They have graciously supported us for almost 5 years, but all good things come to an end.
在這個 issue 裡面有列出 gitlab.fd.o 的一些數據可以看。
資料庫的部分不算小,但據說大多數的空間是 CI logs,所以偏 archive 性質而非 hot data,應該是還好:
a beefy db: our current disk is 433GB and we are using 401GB out of it. Most of it is CI logs, (which is a pity TBH).
a smaller db for the registry: we are using only 1.1 GB
然後這邊提到的 pod 應該是「Gitaly and Gitaly Cluster」這個,看起來都還好:
4 gitaly pods (229GB + 27 GB + 470 GB + 197 GB) to host our git repos
8 webservice + couple of pages pods (those are cheap IMO and bound to the performances of gitaly and the db)
a few sidekiq workers (cheap also)
some room for other workloads (marge-bot, indico, etc...) -^> not really critical if they lag a bit
另外比較大的是 storage 的部分,這邊提到的 S3 storage 看起來是相容層 (後面提到是 Ceph),手動加總算大約 62TB:
main s3 storage (job logs, artifacts, uploads, pages, packages, lfs) -^> currently 72.7 TB with a EC:2+1 replication, so roughly 40 TB of data (guesstimate)
registry s3 -^> currently 23.8 TB (EC:2+1), roughly 14 TB of data
backups s3 -^> 6.386 TB of data
opa s3 (private data protected by JWT) -^> 2.278 TB
然後跑在十一台機器上,三台是 control panel:
1 x AMD EPYC 7402P 24 cores @ 2.80GHz 64 GB 2 x 240 GB SSD 2 x 480 GB SSD 2 x 10 Gbps
另外三台是跑各種 pod:(看到這邊應該是 k8s 了)
2 x Intel Xeon Silver 4214 28 cores @ 2.20GHz 192 GB 2 x 960 GB SSD 2 x 240 GB NVMe (cache) 12 x 8 TB HDD 2 x 10 Gbps
接著有五台 GitLab runner:
1 x AMD EPYC 7502P 32 cores @ 2.50GHz 256 GB 2 x 240 GB SSD 2 x 3.8 TB NVMe 2 x 25 Gbps
1 x Ampere Altra 80 cores @ 3.00GHz 256 GB 2 x 960 GB NVMe n/a 2 x 25 Gbps
流出流量的部分,他們抓了上個月的流出流量,大約是 55TB 左右:
gitlab.fd.o + registry.fd.o + pages webservice -> 50.08TB
s3.fd.o + indico -> 4.732 TB
不算小的服務,所以整包的支出也不算少... 機器與 storage 費用算是一次性的,粗粗估 US$200k 應該差不多?每個月的常態費用反而不高,主要是空間與電力大概要一櫃,另外連外頻寬大約 200Mbps 左右。
四月底就要搬完,晚點來看看後續?