Jetson 上の GPT OSS 20B — デプロイガイド

プリセット

Jetson GPT OSS 20B Service

Deploy GPT OSS 20B to your Jetson device with one click from this platform.

DevicePurpose
NVIDIA Jetson (reComputer)Runs GPT OSS 20B in Docker
1

Deploy GPT OSS 20B Service

Deploy the containerized GPT OSS 20B runtime to your Jetson over SSH.

デプロイ方法

Deploy to your Jetson over SSH with one click.

配線
  1. Connect Jetson and your computer to the same network.
  2. Fill in Jetson IP, SSH username, and password.
  3. Click Deploy.
入力が必要な項目（SenseCraft Solution アプリで入力）
既定値のある残り 2 項目
  • Llama NGL既定値 20

    GPU layers for llama-server (-ngl)

  • Llama Context既定値 1024

    Context window for llama-server (-c)

トラブルシューティング
IssueSolution
SSH connection failedVerify Jetson IP, username, password, and SSH service status
Docker runtime check failedEnsure Docker is installed and NVIDIA runtime is available
Docker Compose unavailableEnsure docker compose or docker-compose is installed
Service start failedInspect logs on Jetson: docker compose logs --tail=200
503 {"message":"Loading model"} on /v1/modelsModel is still warming up; first run can take several minutes
Out-of-memory at startupReduce settings, for example set Llama NGL=16 and Llama Context=512
デプロイ後
  1. The GPT OSS 20B container is running on your Jetson.
  2. llama-server is started inside the container.
  3. The service endpoint is available at http://<jetson-ip>:8080.
  4. Readiness endpoint is available at http://<jetson-ip>:8080/v1/models.
2

Open Service Link

Use this step to open the Jetson service URL directly in a new browser tab.

配線
  1. Enter Jetson IP in this step.
  2. Click Connect.
  3. The platform opens http://<jetson-ip>:8080 in a new tab.
トラブルシューティング
IssueSolution
Invalid host inputEnter a valid IP or hostname, for example 192.168.1.100
New tab not openedAllow pop-ups for this site and retry
Service page not reachableConfirm Jetson service is listening on 8080 and network is reachable
デプロイ後
  1. The service page opens in your browser.
  2. You can return here and click Connect again to reopen it.

GPT OSS 20B runtime has been deployed successfully on your Jetson.

Validation Checklist

  1. Step 1 deployment status shows success.
  2. The GPT OSS 20B container stays in running state.
  3. Clicking Connect in Step 2 opens http://<jetson-ip>:8080.