Data Lab LLM API

Open-weight language models running on the Data Lab CPU cluster, behind an OpenAI-compatible API. Any code written for the OpenAI SDK works by changing base_url and the key.

1. Sign in with your cluster account to get an API key.

2. Use it:

pip install openai

from openai import OpenAI
client = OpenAI(base_url="https://llm.projects.data.syr.edu/v1", api_key="YOUR_KEY")
r = client.chat.completions.create(
    model="qwen3-30b-a3b",
    messages=[{"role": "user", "content": "Explain gradient descent in two sentences."}],
    max_tokens=200, temperature=0.7, stream=True)
for chunk in r:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Endpoints: /v1/models, /v1/chat/completions, /v1/completions (streaming supported). Big models start on demand (a few minutes) and run on idle nodes, so they may answer 503 "not ready" while starting or when the cluster is busy - retry. Limits: 4 concurrent requests, 4096 tokens per answer, 1,000,000 tokens/day.

Models

model	status	nodes	speed per request	first token
`qwen3-30b-a3b` 30B (3.3B active)	ready	1	12.3 tok/s live, 24 h, 11 req	0.52 s	Qwen3 30B MoE (3B active). Fastest general model on the cluster.
`qwen3-8b` 8B	ready	1	3.5 tok/s live, 24 h, 6 req	0.88 s	Qwen3 8B dense, int8 (near-lossless).
`qwen3.8-27b` 27B	stopped	1	3.5 tok/s benchmark, 4 node(s)	-	Qwen3.8 27B (hybrid Gated-DeltaNet), newest Qwen dense model.
`qwen3.6-35b-a3b` 35B (3B active)	stopped	1	21.2 tok/s benchmark, 1 node(s)	-	Qwen3.6 35B MoE (3B active, hybrid Gated-DeltaNet). Newest small Qwen MoE.
`qwen3-32b` 33B	stopped	1	10.7 tok/s benchmark, 6 node(s)	-	Qwen3 32B dense, int4.
`qwen3-235b-a22b` 235B (22B active)	stopped	4	11.0 tok/s benchmark, 4 node(s)	-	Qwen3 235B MoE (22B active), 4 nodes. Does not fit on one A100.
`qwen3-coder-480b` 480B (35B active)	stopped	4	5.3 tok/s benchmark, 4 node(s)	-	Qwen3-Coder 480B MoE (35B active), 4 nodes.
`deepseek-v3.1` 671B (37B active)	stopped	4	4.6 tok/s benchmark, 4 node(s)	-	DeepSeek-V3.1 671B MoE (37B active), 4 nodes.

Speed is the median decode rate of real requests (tokens/s after the first token; one token is about 0.75 English words). It drops when many requests share a model, and a model that is stopped needs a few minutes to start. How does this compare with GPUs?

Same models on GPUs (expected single-request tokens/s)

model	weights GB bf16 / int8 / int4	this cluster measured tok/s	Quadro RTX 6000 (lab, g-01) 24 GB, 672 GB/s	RTX 3090 24 GB, 936 GB/s	RTX 4090 24 GB, 1,008 GB/s	A100 80GB 80 GB, 2,039 GB/s	H100 SXM 80 GB, 3,350 GB/s
`qwen3-8b` 8B	16.4 / 8.7 / 4.6	3.5 live, 24 h, 6 req	32.8 bf16	45.7 bf16	49.2 bf16	99.5 bf16	163 bf16
`qwen3.8-27b` 27B	54.0 / 28.6 / 15.1	3.5 benchmark, 4 node(s)	35.6 int4	49.5 int4	53.3 int4	30.2 bf16	49.6 bf16
`qwen3-30b-a3b` 30B, 3.3B active	61.0 / 32.3 / 17.1	12.3 live, 24 h, 11 req	291 int4	405 int4	436 int4	247 bf16	406 bf16
`qwen3-32b` 33B	65.6 / 34.8 / 18.4	10.7 benchmark, 6 node(s)	29.3 int4	40.8 int4	43.9 int4	24.9 bf16	40.9 bf16
`qwen3.6-35b-a3b` 35B, 3B active	70.0 / 37.1 / 19.6	21.2 benchmark, 1 node(s)	320 int4	446 int4	480 int4	272 bf16	447 bf16
`qwen3-235b-a22b` 235B, 22B active	470 / 249 / 132	11.0 benchmark, 4 node(s)	1.1 1 card, 15% in VRAM 227 on 8 cards	2.4 1 card, 15% in VRAM 316 on 8 cards	2.4 1 card, 15% in VRAM 340 on 8 cards	4.3 1 card, 53% in VRAM 225 on 2 cards	8.5 1 card, 53% in VRAM 370 on 2 cards
`qwen3-coder-480b` 480B, 35B active	960 / 509 / 269	5.3 benchmark, 4 node(s)	0.70 1 card, 7% in VRAM 121 on 16 cards	1.4 1 card, 7% in VRAM 169 on 16 cards	1.4 1 card, 7% in VRAM 182 on 16 cards	1.7 1 card, 26% in VRAM 250 on 4 cards	3.4 1 card, 26% in VRAM 410 on 4 cards
`deepseek-v3.1` 671B, 37B active	1,342 / 711 / 376	4.6 benchmark, 4 node(s)	0.60 1 card, 5% in VRAM 115 on 32 cards	1.3 1 card, 5% in VRAM 160 on 32 cards	1.3 1 card, 5% in VRAM 172 on 32 cards	1.5 1 card, 19% in VRAM 409 on 8 cards	3.0 1 card, 19% in VRAM 673 on 8 cards

GPU columns are estimates, not measurements: memory bandwidth / bytes read per token at the best precision that fits (upper bound; real servers reach 50-80% of it). Models too big for one card show one card streaming the rest over PCIe, and the smallest multi-GPU setup that holds them. Full table with more GPUs and the trillion-parameter models.