HomeBlogPricingCareersDocsGitHubSlack community
Field notes/Engineering/Strengthening agent evaluation environments.

Strengthening agent evaluation environments.

Tensorlake Sandboxes now support dynamic network policies in Harbor: public, allowlist, or no-network per trial phase, switched on a running sandbox without a restart.

Tensorlake Sandboxes now support dynamic network policies in Harbor.

Harbor is a framework for evaluating and optimizing agents and models. It can run a task in a sandbox, an isolated environment where the agent works, while a verifier scores the results. A single evaluation task, or trial, can have several phases: environment setup, agent setup, agent execution, and verification. These phases often need different network access. Usually, you need broader network access during installation and agent setup; after that, you may want to constrain the agent's and verifier's network access to prevent security issues or reward hacking. In practice, neither allowing public network access nor shutting down the network completely for the full process works. The evaluation environment needs phase-aware network policy support, so it can switch network access during the trial.

Now Tensorlake sandboxes support Harbor's dynamic network policy interface, so Harbor can update the network access of a running Tensorlake sandbox without restarting it. This enables different network policies for the environment setup, agent, and verifier. It also allows multi-step tasks that need step-level network policy changes.

Example task.toml

This is an example task.toml file from Harbor:

task.toml
[environment]
network_mode = "public"
 
[agent] # applies during agent.run() -- but *not* during agent.setup()!
network_mode = "allowlist"
allowed_hosts = ["pypi.org", "api.openai.com"]
 
[verifier] # applies during verifier.verify()
network_mode = "no-network"

In this task, the network policy changes as:

FIG 1 · NETWORK POLICY PER TRIAL PHASEEnvironment startspublicagent.setup()publicagent.run()allowlist[pypi.org, api.openai.com]Restorepublicverifier.verify()no-networkRestorepublic
Fig 1 — One sandbox, one trial: the policy switches per phase without a restart.

Notice how individual phases can override the base network policy set in the environment. You can allow connections only to specific hosts, as well as disable network connectivity completely.

With a dynamic network switch, the environment preserves the state of every phase after completion, avoiding multiple environment startup overhead.

Supported policies

Tensorlake sandboxes support the following network policy changes directly. Task authors declare the required policies in task.toml, and Harbor applies them automatically when the trial moves between setup, agent, and verifier phases.

Network policies supported by Tensorlake Sandboxes

Supported network modeDescription
publicFull network access.
no-networkNo network access.
allowlistNetwork access only to targets listed in allowed_hosts; an empty or omitted list denies all egress. Supports exact hostnames, leading-wildcard hostnames, IPv4 literals, and IPv4 CIDR ranges.
Supported network capabilityDescription
Dynamic network policyThe environment can switch the active network policy after startup, enabling [agent] and [verifier] phase overrides.

Reference

SW
WRITTEN BYShanshan WangApplied AI · Tensorlake
LinkedIn →Twitter / X →
Read next —FROM THE LOG
◆ THE SANDBOX DIGEST

Subscribe for release notes, benchmarks, deep dives.

One dispatch per month from the Tensorlake team — no spam.