引言
验证 SandboxSet 拆分多套 NAS 预热池的可行性，以及 E2B SDK 从不同预热池分别领取沙箱的能力。为 9000 Pod 规模下突破单 NAS 2000 节点挂载上限提供方案验证。
官方推荐AgenticFS，专为Sandbox设计的nas产品，性能对标阿里云通用型nas高级版且没有节点数挂载上限限制。
环境信息
| 项 | 值 |
|---|---|
| 集群 | ACK Pro（杭州），2 台 ECS + 3 个 virtual-kubelet |
| NAS-1 | 001ylfbr5cech4exoox-epl50.cn-hangzhou.nas.aliyuncs.com:/tclaw |
| NAS-2 | 000rdqt4dp50hg92ij7-gyd97.cn-hangzhou.nas.aliyuncs.com:/ |
| 沙箱镜像 | registry-cn-hangzhou-vpc.ack.aliyuncs.com/ack-demo/openclaw:2026.3.23-2 |
| E2B SDK | e2b-code-interpreter (系统 Python 3.9) |
| 配置文件 | openclaw-sandboxset-multi-nas.yaml |
部署结构
E2B SDK
|
sandbox-manager (E2B API, 单实例, 共享 API Key)
|
+----------------+------------------+
| |
SandboxSet SandboxSet
openclaw-1 (replicas=2) openclaw-2 (replicas=2)
| |
StorageClass StorageClass
alicloud-nas-1 alicloud-nas-2
| |
NAS-1 NAS-2ALB / Ingress / sandbox-gateway 均无需改动。*.openclaw-sandbox.infra 通配规则和子域名路由格式天然支持多 SandboxSet。
测试结果
K8s 层：预热池就绪
$ kubectl get sandboxset
NAME REPLICAS AVAILABLE UPDATEDREPLICAS UPDATEDAVAILABLEREPLICAS AGE
openclaw 0 0 2d23h
openclaw-1 2 2 2 2 6m
openclaw-2 2 2 2 2 6m两个预热池均在 ~49s 内达到 AVAILABLE 2/2（镜像已缓存）。
K8s 层：PVC 分属不同 StorageClass
$ kubectl get pvc -o custom-columns=NAME:.metadata.name,STORAGECLASS:.spec.storageClassName,STATUS:.status.phase
NAME STORAGECLASS STATUS
workspace-pvc-openclaw-1-w2mrd alicloud-nas-1 Bound
workspace-pvc-openclaw-1-xbq64 alicloud-nas-1 Bound
workspace-pvc-openclaw-2-c2s6m alicloud-nas-2 Bound
workspace-pvc-openclaw-2-kvd8f alicloud-nas-2 Boundopenclaw-1 的两个 PVC 绑定 alicloud-nas-1，openclaw-2 的两个绑定 alicloud-nas-2。
K8s 层：容器内 NAS 挂载源不同
# openclaw-1 Pod
$ kubectl exec openclaw-1-w2mrd -c openclaw -- df -h /app/workspace
Filesystem Size Used Avail Use% Mounted on
001ylfbr5cech4exoox-epl50.cn-hangzhou.nas.aliyuncs.com:/tclaw/nas-a22e1efd-1fbc-40e6-88aa-506f1c55b841 8.0E 0 8.0E 0% /app/workspace
# openclaw-2 Pod
$ kubectl exec openclaw-2-c2s6m -c openclaw -- df -h /app/workspace
Filesystem Size Used Avail Use% Mounted on
000rdqt4dp50hg92ij7-gyd97.cn-hangzhou.nas.aliyuncs.com:/nas-5a40d49d-61e1-4a0d-871a-a436032d8095 8.0E 0 8.0E 0% /app/workspace两个 Pod 的挂载源域名明确不同，分别指向 NAS-1 和 NAS-2。
E2B SDK 层：双池分别领取
$ python3 multi_nas_claim.py
[领取] 从模板 openclaw-1 领取沙箱...
Sandbox ID: default--openclaw-1-xbq64
Template: openclaw-1
领取耗时: 0.19s
[配置] 已写入 openclaw.json
[验证] HTTP 200, body 2732 bytes
[NAS] 挂载信息: 001ylfbr5cech4exoox-epl50.cn-hangzhou.nas.aliyuncs.com:/tclaw/nas-... 8.0E 0 8.0E 0% /app/workspace
[领取] 从模板 openclaw-2 领取沙箱...
Sandbox ID: default--openclaw-2-c2s6m
Template: openclaw-2
领取耗时: 0.11s
[配置] 已写入 openclaw.json
[验证] HTTP 200, body 2732 bytes
[NAS] 挂载信息: 000rdqt4dp50hg92ij7-gyd97.cn-hangzhou.nas.aliyuncs.com:/nas-... 8.0E 0 8.0E 0% /app/workspace
[列表] 所有沙箱:
共 3 个沙箱
[1] id=default--openclaw-6hp4j template=openclaw state=running
[2] id=default--openclaw-1-xbq64 template=openclaw-1 state=running
[3] id=default--openclaw-2-c2s6m template=openclaw-2 state=running
[结论] 两个预热池均可正常领取，多 NAS 方案验证通过预热池自动补充
领取后 SandboxSet 自动补充新 Pod 维持 replicas 数量：
$ kubectl get sandbox
NAME STATUS CLAIMED
openclaw-1-g4cr7 Running false ← 补充的新 Pod
openclaw-1-w2mrd Running false
openclaw-1-xbq64 Running true ← 已领取
openclaw-2-c2s6m Running true ← 已领取
openclaw-2-kvd8f Running false
openclaw-2-rcm5p Running false ← 补充的新 Pod每个 SandboxSet 维持 replicas 个 AVAILABLE（CLAIMED=false）沙箱，领取后自动补充。
验证要点汇总
| 验证项 | 结果 |
|---|---|
| openclaw-1 预热池就绪 (AVAILABLE 2/2) | 通过 |
| openclaw-2 预热池就绪 (AVAILABLE 2/2) | 通过 |
| PVC 分属 alicloud-nas-1 / alicloud-nas-2 | 通过 |
| 容器内 NAS 挂载源域名不同 | 通过 |
Sandbox.create("openclaw-1") 领取成功 | 通过 (0.19s) |
Sandbox.create("openclaw-2") 领取成功 | 通过 (0.11s) |
| sandbox_id 格式区分来源 (含 SandboxSet 名) | 通过 |
| 两个沙箱 HTTP 200 可达 | 通过 |
Sandbox.list() 包含两个 template | 通过 |
| 领取后预热池自动补充 | 通过 |
| ALB/gateway 无需改动 | 通过 |
结论
多 NAS 拆分方案验证通过。SandboxSet 天然支持多实例部署，每个实例绑定独立 NAS，E2B SDK 通过 template 名（= SandboxSet 名）路由到对应预热池。sandbox-gateway 的子域名路由格式 {port}-{namespace}--{sandboxset}-{podSuffix}.{domain} 自动区分不同 SandboxSet，无需任何网络层改动。
扩展到 9000 Pod 的路径：创建 5 套 NAS + 5 个 SandboxSet（每套 ~1800 副本），E2B SDK 侧增加模板路由层（轮询或按可用量选优）。
涉及文件
| 文件 | 说明 |
|---|---|
openclaw-sandboxset-multi-nas.yaml | 2 StorageClass + 2 SandboxSet 定义 |
e2b-test/multi_nas_claim.py | 双池领取测试脚本 |
多NAS多预热池测试记录.md | 本文档 |
openclaw-sandboxset-multi-nas.yaml
---
# ============================================================
# StorageClass 1: NAS-1 (现有)
# ============================================================
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: alicloud-nas-1
provisioner: nasplugin.csi.alibabacloud.com
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: Immediate
mountOptions:
- nolock,tcp,noresvport
- vers=3
parameters:
volumeAs: subpath
server: 001ylfbr5cech4exoox-epl50.cn-hangzhou.nas.aliyuncs.com:/tclaw
archiveOnDelete: "true"
---
# ============================================================
# StorageClass 2: NAS-2 (新)
# 注意: server 路径用 / (根目录)，如 NAS 上有特定子目录请修改
# ============================================================
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: alicloud-nas-2
provisioner: nasplugin.csi.alibabacloud.com
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: Immediate
mountOptions:
- nolock,tcp,noresvport
- vers=3
parameters:
volumeAs: subpath
server: 000rdqt4dp50hg92ij7-gyd97.cn-hangzhou.nas.aliyuncs.com:/
archiveOnDelete: "true"
---
# ============================================================
# SandboxSet 1: openclaw-1 (NAS-1)
# ============================================================
apiVersion: agents.kruise.io/v1alpha1
kind: SandboxSet
metadata:
name: openclaw-1
namespace: default
spec:
replicas: 2
persistentContents:
- filesystem
runtimes:
- name: csi
- name: agent-runtime
volumeClaimTemplates:
- metadata:
name: workspace-pvc
spec:
accessModes:
- ReadWriteMany
storageClassName: alicloud-nas-1
resources:
requests:
storage: 20Gi
template:
metadata:
labels:
alibabacloud.com/acs: "true"
alibabacloud.com/compute-class: agent-sandbox
alibabacloud.com/compute-qos: default
app.kubernetes.io/part-of: openclaw-cluster
nas-group: "1"
annotations:
scaling.alibabacloud.com/enable-inplace-resource-resize: "true"
spec:
automountServiceAccountToken: false
terminationGracePeriodSeconds: 30
securityContext:
fsGroup: 1000
containers:
- name: openclaw
image: registry-cn-hangzhou-vpc.ack.aliyuncs.com/ack-demo/openclaw:2026.3.23-2
imagePullPolicy: IfNotPresent
env:
- name: OPENCLAW_SKIP_ONBOARDING
value: "1"
- name: OPENCLAW_TZ
value: Asia/Shanghai
ports:
- containerPort: 18789
name: http-ui
protocol: TCP
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: "2"
ephemeral-storage: 30Gi
memory: 4Gi
volumeMounts:
- name: workspace-pvc
mountPath: /app/workspace
---
# ============================================================
# SandboxSet 2: openclaw-2 (NAS-2)
# ============================================================
apiVersion: agents.kruise.io/v1alpha1
kind: SandboxSet
metadata:
name: openclaw-2
namespace: default
spec:
replicas: 2
persistentContents:
- filesystem
runtimes:
- name: csi
- name: agent-runtime
volumeClaimTemplates:
- metadata:
name: workspace-pvc
spec:
accessModes:
- ReadWriteMany
storageClassName: alicloud-nas-2
resources:
requests:
storage: 20Gi
template:
metadata:
labels:
alibabacloud.com/acs: "true"
alibabacloud.com/compute-class: agent-sandbox
alibabacloud.com/compute-qos: default
app.kubernetes.io/part-of: openclaw-cluster
nas-group: "2"
annotations:
scaling.alibabacloud.com/enable-inplace-resource-resize: "true"
spec:
automountServiceAccountToken: false
terminationGracePeriodSeconds: 30
securityContext:
fsGroup: 1000
containers:
- name: openclaw
image: registry-cn-hangzhou-vpc.ack.aliyuncs.com/ack-demo/openclaw:2026.3.23-2
imagePullPolicy: IfNotPresent
env:
- name: OPENCLAW_SKIP_ONBOARDING
value: "1"
- name: OPENCLAW_TZ
value: Asia/Shanghai
ports:
- containerPort: 18789
name: http-ui
protocol: TCP
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: "2"
ephemeral-storage: 30Gi
memory: 4Gi
volumeMounts:
- name: workspace-pvc
mountPath: /app/workspacee2b-test/multi_nas_claim.py
#!/usr/bin/env python
"""
多 NAS 预热池领取测试
验证 E2B SDK 能从两个不同 NAS 后端的 SandboxSet (openclaw-1 / openclaw-2)
分别领取沙箱，且两个沙箱挂载在不同的 NAS 上。
前置条件:
1. kubectl apply -f openclaw-sandboxset-multi-nas.yaml
2. 等待两个 SandboxSet AVAILABLE 2/2
3. .env 中配置 E2B_DOMAIN / E2B_API_KEY / SSL_CERT_FILE
"""
import os
import time
import socket
import requests
from string import Template
from dotenv import load_dotenv
from e2b_code_interpreter import Sandbox
# --- DNS 通配符解析补丁 ---
# macOS /etc/hosts 不支持通配符，用 socket 猴子补丁拦截 *.openclaw-sandbox.infra
_orig_getaddrinfo = socket.getaddrinfo
_ALB_IP = os.getenv("ALB_IP", "121.43.128.121")
def _patched_getaddrinfo(host, port, *args, **kwargs):
if host and "openclaw-sandbox.infra" in host:
host = _ALB_IP
return _orig_getaddrinfo(host, port, *args, **kwargs)
socket.getaddrinfo = _patched_getaddrinfo
# 关闭 SSL 警告（自签名证书）
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
load_dotenv(override=True)
GATEWAY_TOKEN = "openclaw-sandbox-token"
DASHSCOPE_API_KEY = "sk-fake-test-key-12345"
# 两套预热池对应的 template 名 = SandboxSet 名
TEMPLATES = ["openclaw-1", "openclaw-2"]
def claim_and_provision(template_name):
"""从指定模板领取沙箱 -> 注入配置 -> 验证访问 -> 检查 NAS 挂载"""
print(f"\n{'='*60}")
print(f"[领取] 从模板 {template_name} 领取沙箱...")
start = time.monotonic()
sandbox = Sandbox.create(
template_name,
metadata={"e2b.agents.kruise.io/never-timeout": "true"},
)
elapsed = time.monotonic() - start
print(f" Sandbox ID: {sandbox.sandbox_id}")
print(f" Template: {template_name}")
print(f" 领取耗时: {elapsed:.2f}s")
# 注入 openclaw.json 配置
print(f" [配置] 写入 openclaw.json...")
with open("openclaw_template.json", "r") as f:
tmpl = Template(f.read())
rendered = tmpl.safe_substitute(
GATEWAY_TOKEN=GATEWAY_TOKEN,
DASHSCOPE_API_KEY=DASHSCOPE_API_KEY,
)
sandbox.files.write("/home/node/.openclaw/openclaw.json", rendered, user="node")
print(f" [配置] 已写入 /home/node/.openclaw/openclaw.json")
# 等待 gateway 重启
print(f" [等待] 30s 等 gateway 重启...")
time.sleep(30)
# 验证外部访问
host = sandbox.get_host(18789)
base_url = f"https://{host}"
print(f" [验证] 访问地址: {base_url}/?token={GATEWAY_TOKEN}")
resp = requests.get(f"{base_url}/?token={GATEWAY_TOKEN}", verify=False, timeout=10)
print(f" [验证] HTTP {resp.status_code}, body {len(resp.text)} bytes")
if resp.status_code != 200:
print(f" [警告] 非 200: {resp.text[:200]}")
# 检查容器内 NAS 挂载源
print(f" [NAS] 检查挂载源...")
try:
mount_result = sandbox.commands.run(
"df -h /app/workspace 2>/dev/null | tail -1",
user="node",
timeout=10,
)
mount_line = mount_result.stdout.strip()
print(f" [NAS] 挂载信息: {mount_line}")
except Exception as e:
print(f" [NAS] 检查失败: {e}")
return sandbox
def list_all_sandboxes():
"""列出所有沙箱，验证两个池都有沙箱"""
print(f"\n{'='*60}")
print("[列表] 所有沙箱:")
paginator = Sandbox.list()
all_sbx = []
while paginator.has_next:
items = paginator.next_items()
if items:
all_sbx.extend(items)
else:
break
print(f" 共 {len(all_sbx)} 个沙箱")
for i, s in enumerate(all_sbx, 1):
print(f" [{i}] id={s.sandbox_id}")
print(f" template={s.template_id}")
print(f" state={s.state}")
return all_sbx
def main():
print("=" * 60)
print("多 NAS 预热池领取测试")
print(f"模板列表: {TEMPLATES}")
print("=" * 60)
sandboxes = []
for template in TEMPLATES:
try:
sbx = claim_and_provision(template)
sandboxes.append(sbx)
except Exception as e:
print(f" [错误] 从 {template} 领取失败: {e}")
# 列出所有沙箱
list_all_sandboxes()
# 总结
print(f"\n{'='*60}")
print("[总结] 测试结果:")
print(f"{'='*60}")
for sbx in sandboxes:
host = sbx.get_host(18789)
print(f" {sbx.sandbox_id}")
print(f" -> {host}/?token={GATEWAY_TOKEN}")
print()
if len(sandboxes) == len(TEMPLATES):
print(" [结论] 两个预热池均可正常领取，多 NAS 方案验证通过")
else:
print(f" [结论] {len(sandboxes)}/{len(TEMPLATES)} 个池领取成功，请检查失败的池")
print()
print("清理命令 (可选):")
for sbx in sandboxes:
print(f" python3 pause_sandbox.py {sbx.sandbox_id}")
print(" 或直接 kill:")
for sbx in sandboxes:
print(f" python3 -c \"from e2b_code_interpreter import Sandbox; Sandbox.kill('{sbx.sandbox_id}')\"")
if __name__ == "__main__":
main()
评论