Hugging Face

AstroPulse serves a model from Hugging Face by its name, pinned to one commit. A public model needs nothing else. A gated or private model needs your organization's Hugging Face token, connected once for every cluster.

Before you start

• A cluster with Certificates and Model Serving installed, from the cluster's Add-ons tab. To call the model from outside the cluster, see what a model needs in Serve a model.
• For a gated or private model: your organization's Hugging Face token, connected once under Integrations: Hugging Face. The token's account must be able to see the model.

In the console

1. Open Workloads > Models > Deploy model. Name the model and pick its cluster and profile.

2. On the Model step, keep Hugging Face under Model source. In Model, search for or type the model as it appears on huggingface.co, for example Qwen/Qwen2.5-0.5B-Instruct. Open … on Hugging Face opens its page.

3. In Revision (optional), paste the commit you want (see Pin a revision). Left empty, the model follows main.

4. Read the line under the fields. For a gated or private model it must say "Private and gated models use your organization's Hugging Face token: connected". If it says not connected, connect one, select it and connect the token first.

5. Select Check model. "The platform can serve this model." means the check passed. Anything else is a refusal; see If it's refused.

   Open zoom view

   The Model step with Hugging Face selected, Qwen/Qwen2.5-0.5B-Instruct in Model, the token connected, and Check model passed

   

6. Continue through Compute, Scaling and Access, then Review and Deploy, as in Serve a model.

You should see

The model on Workloads > Models reads Running, with its endpoint and Try it.

Open zoom view

The Models list with qwen-chat Running on eks-prod, one L4 GPU, and its public endpoint

Save download cost

Every new node downloads the model again, and on a private cluster those bytes go through the NAT gateway, which the cloud bills per GB. On an EKS cluster AstroPulse creates, a model at an s3:// address or an oci:// image in your ECR, in the cluster's region, skips that charge: see Cut model download cost.

Choose a model

Check three things on the model's Hugging Face page, for example huggingface.co/Qwen/Qwen2.5-0.5B-Instruct, before you deploy it:

• Its size, the parameter count or the Files tab. The weights, the runtime and the KV cache all need memory. See Sizing a replica.
• A licence or access banner. A model that asks you to accept terms is gated. See Gated models.
• The context length on the model card. A context length above the model's own limit makes the runtime stop at start. See Context length.

A 0.5B instruct model serves on CPU in a few minutes and exercises the add-ons, the endpoint and the metrics. To serve a larger model afterwards, change Model and Revision; the other settings stay.

Pin a revision

Enter the 40-character commit hash in Revision.

• The same bytes every time. A branch or tag, and main when you name none, points at whatever was last pushed there. A commit never moves, so a redeploy or a rollback serves exactly what was served before.
• The check confirms it exists. AstroPulse asks Hugging Face for that revision before it deploys, and refuses one the model does not have.

To find the commit, open the model's page on Hugging Face, then Files and versions > History (for example huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/commits/main), and copy the full hash. Nothing warns you about an unpinned model; it deploys and follows the branch.

Gated models

A gated model asks you to accept its terms on Hugging Face before it can be downloaded. AstroPulse downloads it with your organization's token, so the account that owns the token must accept them: sign in to huggingface.co as that account, open the model's page (for example huggingface.co/meta-llama/Llama-3.1-8B-Instruct), and accept or request access. Some models grant access at once; others wait for their authors to approve.

With a token connected, Check model and the dry run read one of the model's files with it, so terms the token's account has not accepted are refused before anything deploys. If Hugging Face does not answer in time, the check lets the model through and the download on the cluster decides.

Other ways in

• Nova: ask it in plain language, for example "serve Qwen/Qwen2.5-0.5B-Instruct on prod-01".
• SDK and API: the Python SDK and the API reference use the same manifest.

With astroctl

qwen-chat.yaml

name: "qwen-chat"
profileName: "prod-inference"
source:
  type: "Model"
  model:
    uri: "hf://Qwen/Qwen2.5-0.5B-Instruct:<commit>"

Terminal

astroctl app apply -f qwen-chat.yaml --dry-run   # checks, deploys nothing
astroctl app apply -f qwen-chat.yaml
# the latest commit of a public model
curl -s https://huggingface.co/api/models/Qwen/Qwen2.5-0.5B-Instruct | jq -r .sha

The token itself, with astroctl integrations huggingface, is on Hugging Face integration. See the astroctl reference.

Settings

uriRequired

The model on Hugging Face. In the console, Model and Revision together.

Accepts

hf://<org>/<repo>, or hf://<org>/<repo>:<revision> with a branch, tag or commit. Pin a 40-character commit.

Default

no revision: the model follows main

See

Pin a revision

format

Not needed for a Hugging Face model: AstroPulse serves it with vLLM.

Default

vLLM

See

Settings on Serve a model

There is no field for the token. An application never carries a credential: the token is the organization's, connected under Integrations > Hugging Face.

If it's refused

"model.uri "hf://org/repo@<commit>": pin a Hugging Face revision as hf://<org>/<repo>:<revision>"

Cause

The revision follows @ instead of :.

Fix

Write hf://<org>/<repo>:<commit>, or paste the commit into Revision in the console.

"Hugging Face model Qwen/Qwen2.5-0.5B-Instruct has no revision deadbeef: use a branch, tag or commit it has"

Cause

The model has no such branch, tag or commit.

Fix

Copy the commit from Files and versions > History on the model's page.

"Hugging Face has no public model <org>/<repo>: check the name, or if it is private, connect a Hugging Face token that can read it ('astroctl integrations huggingface connect')"

Cause

No token is connected, and the name is wrong or the model is private.

Fix

Check the name. For a private model, connect a token whose account can read it.

"Hugging Face model <org>/<repo> does not exist, or the organization's Hugging Face token cannot read it: check the name, and that the token's account has access"

Cause

A token is connected, and the name is wrong or the token's account cannot see a private model.

Fix

Check the name, and give the token's account access on Hugging Face.

"Hugging Face model <org>/<repo> is gated: connect a Hugging Face token whose account has accepted its terms ('astroctl integrations huggingface connect')"

Cause

The model is gated and no token is connected.

Fix

Connect the organization's token, on an account that has accepted the terms.

"Hugging Face model <org>/<repo> is gated, and the account whose token is connected has not been granted access: accept the model's terms on Hugging Face (https://huggingface.co/&lt;org&gt;/&lt;repo>) with that account, then apply the application again"

Cause

The token's account has not accepted the terms.

Fix

Accept them with that account on the model's Hugging Face page (see Gated models), then deploy the same model again.

"Hugging Face model <org>/<repo> is gated, and the organization's Hugging Face token cannot download its files: check that the token is still valid and allowed to read gated repositories, then apply the application again"

Cause

The token was revoked, or it is a fine-grained token not allowed to read gated repositories.

Fix

Replace the token with one that may, then deploy again.

"Credentials are not part of an application" and "source.model.secretRef is not a field of an application."

Cause

The manifest sets secretRef on the model.

Fix

Remove it. How models on a cluster read private artifacts is set by your platform team on the cluster's Model Serving add-on (configuration.modelServing.identity or secretRef).

Can't start with "The application isn't serving" and "Nothing serves yet because container storage-initializer keeps exiting"

Cause

The download on the cluster is failing.

Fix

Wait for the reason below, or read the model's Events tab.

Failed, "Deployment failed", with a reason such as "The model artifact could not be downloaded: Cannot access gated repo for url https://huggingface.co/&lt;org&gt;/&lt;repo&gt;/resolve/&lt;commit&gt;/.gitattributes. If the model is gated, accept its terms on Hugging Face with the account whose token is connected, and check that the token may read gated repositories; then apply the application again."

Cause

Hugging Face refused the download on the cluster.

Fix

Fix the cause on Hugging Face (accept the terms, or the token's permissions), then select Apply the application again on the notice.

Next

• Sizing and scaling: put the model on a GPU and size it.
• Expose a model: give it a public HTTPS address for chosen callers.
• Hugging Face integration: connect, replace or revoke the organization's token.