Skip to main content

Hugging Face

Connect one read-only Hugging Face token for the organization. Every cluster then uses it to download gated and private models. A public model does not need it.

Before you start​

  • A Hugging Face account that can see the models you will serve. For a gated model, the same account accepts its terms.
  • A role that may manage the organization's credentials. Connect token, Rotate and Revoke are disabled otherwise.

In the console​

  1. Accept the model's terms, if it is gated. Sign in to huggingface.co as the account that will own the token, open the model's page (for example huggingface.co/meta-llama/Llama-3.1-8B-Instruct), and accept or request access. Some models grant access at once; others wait for their authors to approve. Wait until the page says you have access.

  2. Open Integrations > Hugging Face and select Connect token.

  3. Select Create one on Hugging Face under the User access token field. It opens huggingface.co/settings/tokens. Create a read-only token on the account from step 1. A fine-grained token must also be allowed to read gated repositories.

  4. Paste the token into User access token and select Connect. AstroPulse checks the token with Hugging Face before it keeps it, stores it encrypted, and never shows it again.

    Open zoom view

    Integrations > Hugging Face with a token connected: the User access token field with Create one on Hugging Face under it, the token's last four characters, Connected, and Revoke

    Integrations > Hugging Face with a token connected: the User access token field with Create one on Hugging Face under it, the token's last four characters, Connected, and RevokeIntegrations > Hugging Face with a token connected: the User access token field with Create one on Hugging Face under it, the token's last four characters, Connected, and Revoke

You should see​

The Token table shows the token as hf_… with its last four characters, the status Connected, when it was last checked, and when and by whom it was connected. Refresh reads the status again.

The model deploy wizard shows the same under the model name, on the Model step of Deploy model: "Private and gated models use your organization's Hugging Face token: connected".

Open zoom view

The Deploy model wizard's Model step showing the organization's Hugging Face token as connected

The Deploy model wizard's Model step showing the organization's Hugging Face token as connectedThe Deploy model wizard's Model step showing the organization's Hugging Face token as connected

Replace the token​

On Integrations > Hugging Face, select Rotate, paste the new token into User access token and select Rotate. The new token replaces the old one for every model in the organization.

Revoke the token​

On Integrations > Hugging Face, select Revoke and type revoke to confirm.

  • Within about ten minutes the token is withdrawn from your clusters.
  • Running models keep what they downloaded. New or restarted ones can no longer read gated models, unless the cluster's Model Serving add-on names a Secret (see Use a token for one cluster only).
  • The token stays valid at Hugging Face. Revoke it on huggingface.co/settings/tokens too if it should stop working everywhere.

Use a token for one cluster only​

When the organization has not connected a token, your platform team can name a Secret with key HF_TOKEN in namespace kserve as the Model Serving add-on's secretRef. Hugging Face models on that cluster read it. Once the organization connects its own token, that token is used instead. See Letting models read private artifacts.

An application never carries the token: there is no token or secretRef field on a model.

Other ways in​

  • Nova: ask it in plain language, for example "is our Hugging Face token connected?".
  • SDK and API: the Python SDK and the API reference cover the same integration.

With astroctl​

Terminal
# connect or replace; read the token without echoing it
read -rs HF_TOKEN && echo "$HF_TOKEN" | astroctl integrations huggingface connect --token-stdin
astroctl integrations huggingface status       # Status: connected, Token: hf_… and its last four
astroctl integrations huggingface disconnect   # asks you to type huggingface; --yes skips the question

To check the token's account can download a gated model's files without AstroPulse, ask Hugging Face for one file at a commit. 200 means it can; 403 means the account has no access yet, 401 that the token is not valid, and 404 that the file is not at that commit (use one from the model's Files tab):

Terminal
curl -sL -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer ${HF_TOKEN}" \
  "https://huggingface.co/<org>/<repo>/resolve/<commit>/config.json"

See the CLI reference for connect, status and disconnect.

If it's refused​

Connect token and Revoke are disabled
Cause
Your role may not manage the organization's credentials.
Fix
Ask an organization owner or admin to connect the token.
Token unreadable, with Revoke
Cause
The organization's stored token cannot be read.
Fix
Select Revoke, then Connect token with a new token.
A model deploy fails with "Hugging Face token can't be read"
Cause
The organization's stored token cannot be read.
Fix
Revoke the token on Integrations > Hugging Face, connect a token again, then deploy the model again.
"Hugging Face model <org>/<repo> is gated, and the organization's Hugging Face token cannot download its files: check that the token is still valid and allowed to read gated repositories, then apply the application again"
Cause
The token was revoked at Hugging Face, or it is a fine-grained token not allowed to read gated repositories.
Fix
Select Rotate and paste a token that may, then deploy the model again.

Refusals about a model's name, revision or terms are on Serve a Hugging Face model.

Next​