Serve a model from your own storage

Serve model weights from your own bucket, registry or volume instead of Hugging Face.

When you need it

• The weights stay in your cloud account.
• Serve private or fine-tuned models that are not on Hugging Face.
• No credentials in the application: the cluster's Model Serving add-on reads the storage with its own cloud identity, set up once per cluster.

How it works

1. You grant the Model Serving add-on's identity read access to the storage, once per cluster.
2. You deploy the model with a URI that points at the storage.
3. AstroPulse downloads the weights with that identity and serves them.

Where your model can live

Storage	URI format	Set up read access
Amazon S3	s3://<bucket>/<path>/	Let models read a private S3 bucket
Google Cloud Storage	gs://<bucket>/<path>/	Let models read a private GCS bucket
Azure Blob	https://<account>.blob.core.windows.net/<container>/<path>	Let models read a private Blob container
OCI registry	oci://<registry>/<repo>@sha256:<digest>	None: the cluster's nodes pull it as they pull any image
A volume in the cluster	pvc://<claim>/<path>	None: the volume is mounted. The claim must be in the model's namespace, and you fill it yourself

Before you start

• A cluster with Certificates and Model Serving installed, from the cluster's Add-ons tab. To call the model from outside the cluster, see what a model needs in Serve a model.
• Read access for the cluster, set up once with the link in the table above. In the console it is the Cloud access section of the Model Serving add-on.
• The model's files under one folder per version, for example fraud/v12/. Never overwrite a version: a rollback then serves the same bytes.

In the console

1. Open Workloads > Models > Deploy model. Name the model and pick its cluster and profile.

2. On the Model step, under Model source, choose Object storage for S3, GCS or Azure Blob, Registry for an OCI image, or Volume for a claim.

   Open zoom view

   The Model step: Model source with Hugging Face, Object storage, Volume and Registry

   

3. In Location, paste the rest of the URI. On a cluster AstroPulse manages, Object storage shows the storage prefix of the cluster's cloud. The line below the field says how the cluster reads it, for example "Read as the cluster's … set on its Model Serving add-on."

4. Open Advanced and set Format: vLLM for a generative model's weights, or the predictive format, such as SKLearn. Every source except Hugging Face requires it.

   Open zoom view

   The Model step with Advanced open, showing the Format and Runtime engine fields

   

5. Select Check model. It checks that the location exists and that the cluster can read it, before anything is deployed.

6. Continue through Compute, Scaling and Access, then Review and Deploy, as in Serve a model.

For a volume, set the model's Namespace on the Options step to the claim's namespace.

You should see

The model on Workloads > Models reads Running, and its Serving card shows the Model URI you set.

Other ways in

• Nova: ask it in plain language, for example "serve the sklearn model at s3://acme-models/fraud/v12/ on prod-01".
• SDK and API: the Python SDK and the API reference use the same manifest.

With astroctl

fraud-model.yaml

name: "fraud-model"
profileName: "prod-inference"
source:
  type: "Model"
  model:
    uri: "s3://acme-models/fraud/v12/"
    format: "SKLearn"

Terminal

astroctl app apply -f fraud-model.yaml --dry-run   # checks, deploys nothing
astroctl app apply -f fraud-model.yaml

See the astroctl reference.

Settings

uriRequired

Where the weights are. In the console, Model source and Location. Name the artifact only: no user, key or query string. To pin one object version, add ?versionId=<id> on s3:// or ?generation=<n> on gs://.

Accepts

s3://, gs://, https://<account>.blob.core.windows.net/<container>/<path>, oci://<registry>/<repo>@sha256:<digest>, pvc://<claim>/<path>

See

Where your model can live

formatRequired

The model format, which picks the serving runtime. In the console, Advanced > Format.

Accepts

• vLLM vLLM A generative model, such as a Hugging Face language model, served by vLLM with an OpenAI-compatible API.
• SKLearn scikit-learn A model saved by scikit-learn.
• XGBoost XGBoost A model saved by XGBoost.
• LightGBM LightGBM A model saved by LightGBM.
• TensorFlow TensorFlow A TensorFlow SavedModel.
• PyTorch PyTorch A PyTorch model packaged for TorchServe.
• ONNX ONNX A model in the Open Neural Network Exchange format.
• TensorRT TensorRT A model compiled with NVIDIA TensorRT.
• Triton Triton A model repository for NVIDIA Triton Inference Server.
• MLflow MLflow A model saved in the MLflow model format.
• PMML PMML A model in the Predictive Model Markup Language.
• Paddle PaddlePaddle A model saved by PaddlePaddle.

See

Runtimes

namespace

For pvc:// only: the namespace that holds the claim. In the console, Options.

Default

the profile's namespace

There is no credential field on a model. Read access is set on the cluster's Model Serving add-on.

If it's refused

"the cluster's model-serving credential cannot read <uri>: grant it read access, or change the model-serving add-on's secretRef or identity"

Cause

Access denied: the add-on's identity cannot read this location, or the add-on has no identity.

Fix

Grant read access as in the table above, or set the identity under Cloud access on the cluster's Model Serving add-on.

The model shows Can't start, then Failed, with a reason that starts "The model artifact could not be downloaded:" and ends "Check the model-serving add-on's identity or secretRef, then apply the application again."

Cause

Access denied at download time.

Fix

Fix the identity or its permission in your cloud, then select Apply the application again on the notice.

"the bucket or container in <uri> does not exist: check its name"

Cause

Not found: the bucket name is wrong.

Fix

Correct it in Location.

"nothing exists at <uri>: check the path, and the pinned version if there is one"

Cause

Not found: the folder is wrong or empty.

Fix

Correct the path.

"the storage account in https://example.blob.core.windows.net/models/iris does not exist: check its name"

Cause

Not found: the Azure storage account name is wrong.

Fix

Correct it.

"the registry has no image <registry>/<repo>@sha256:<digest>: check the repository, and the tag or digest"

Cause

Not found: the image is not in the registry.

Fix

Correct the repository or digest.

"PersistentVolumeClaim <claim> does not exist in namespace <namespace>, where the model runs: create it there, or set namespace to the one that holds it"

Cause

Not found: the claim is not in the model's namespace.

Fix

Set Namespace on the Options step to the claim's.

"model.format is required for a s3:// artifact (sklearn, xgboost, tensorflow, pytorch, onnx, ...)"

Cause

Wrong format: none is set.

Fix

Set Format under Advanced.

"model.uri "ftp://host/model" must use one of hf://, s3://, gs://, pvc://, oci://, or an Azure Blob URL https://<account>.blob.core.windows.net/<container>/<path>"

Cause

The URI uses a scheme AstroPulse does not read.

Fix

Use a format from the table above.

"model.uri "https://<account>.blob.core.windows.net/models/iris?sig=x" must name the artifact only: remove the query string ..." or "model.uri must not embed credentials ..."

Cause

The URI carries a signed URL, a SAS token, or a user and key.

Fix

Remove them. The cluster reads the storage with the add-on's identity.

"Credentials are not part of an application"

Cause

The manifest sets identity or secretRef on the model.

Fix

Remove it. Read access belongs on the cluster's Model Serving add-on.

"model.cache is not used for a pvc:// artifact: the weights are already on a volume the cluster mounts, so there is nothing to cache"

Cause

Cache weights on the node is on for a volume.

Fix

Turn it off.

Next

• Sizing and scaling: put the model on a GPU and size it.
• Cut model download cost: why storage in the cluster's region is the cheap path.
• Model Serving add-on: every way a cluster can read private artifacts.