Skip to main content

Release a new version, and roll back

Alice, the platform engineer, updates the model qwen-chat on eks-prod, rolls it back from the History tab by typing its name, and calls it from Python. Every deploy is a version, and a rollback shows its plan first.

Before you start​

  • qwen-chat (your model deployment's name) running on eks-prod (your cluster's name).
  • For a release without a gap, a minimum of 2 replicas, set in step 3 of Scale the model with demand; with 1, the model has a gap while it is replaced.

1. Alice releases​

Path: Workloads > Models > qwen-chat > Update

  1. Change one thing, such as Memory in Compute.
  2. Review, then apply.
Open zoom view

Update wizard: steps Model, Compute, Scaling, Access, API; Compute fields

Update wizard: steps Model, Compute, Scaling, Access, API; Compute fieldsUpdate wizard: steps Model, Compute, Scaling, Access, API; Compute fields

The model returns to Running.

Pin the model to a commit

A pinned address (a Hugging Face commit, an OCI digest, an S3 version or a GCS generation) makes a rollback return the same bytes. A rollback deploys the old version anew; it does not undo database migrations or other data changes.

2. Alice rolls back​

Path: Workloads > Models > qwen-chat > History

  1. Select Roll back on the previous version.
  2. Read the plan, then type qwen-chat to confirm.
Open zoom view

History tab: versions with source, who, when, state, Roll back

History tab: versions with source, who, when, state, Roll backHistory tab: versions with source, who, when, state, Roll back
Open zoom view

Roll back dialog: current and target version, diff, name to type

Roll back dialog: current and target version, diff, name to typeRoll back dialog: current and target version, diff, name to type

The plan shows a before and after, and notes that the endpoint keeps the access policy it has now. A version that served no endpoint cannot be rolled back to while the endpoint requires authentication; apply the specification you want instead.

NovaNova
Roll qwen-chat back to v1.
Ask Nova

It shows the plan and rolls back when you approve.

3. Alice calls the model the way Bob will​

Path: Workloads > Models > qwen-chat > Call it from code

  • Run the console's code with a key the policy accepts, such as Bob's.
Python
from astropulse import Client
from openai import OpenAI

ap = Client()                       # reads ASTROPULSE_API_KEY
application = ap.applications.get_application("qwen-chat")

client = OpenAI(base_url=application.data.model.external_endpoint,
                api_key="<key the policy accepts>")

response = client.chat.completions.create(
    model="<model-name>",   # the name shown in Call it from code
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
Open zoom view

Call it from code: SDK and OpenAI snippets

Call it from code: SDK and OpenAI snippetsCall it from code: SDK and OpenAI snippets