- CLI
- Python SDK
- JavaScript SDK
Copy
gravixlayer chat --model "meta-llama/llama-3.1-8b-instruct" --user "Tell me a short story" --stream
Copy
Once upon a time, in a small village nestled between rolling hills, there lived a young baker named Elena. Every morning, she would wake before dawn to prepare fresh bread for the townspeople. One day, she discovered that her sourdough starter had developed an unusual golden glow...
[Content streams in real-time as the model generates it]
Copy
import os
from gravixlayer import GravixLayer
client = GravixLayer()
stream = client.chat.completions.create(
model="meta-llama/llama-3.1-8b-instruct",
messages=[{"role": "user", "content": "Tell me a short story"}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
Copy
import { GravixLayer } from 'gravixlayer';
const client = new GravixLayer({
apiKey: process.env.GRAVIXLAYER_API_KEY,
});
const stream = await client.chat.completions.create({
model: "meta-llama/llama-3.1-8b-instruct",
messages: [{"role": "user", "content": "Tell me a short story"}],
stream: true
});
for await (const chunk of stream) {
if (chunk.choices[0].delta.content !== null) {
process.stdout.write(chunk.choices[0].delta.content);
}
}
Streaming Completions
- CLI
- Python SDK
- JavaScript SDK
Copy
gravixlayer chat --mode completions --model "meta-llama/llama-3.1-8b-instruct" --prompt "Write a poem" --stream
Copy
import os
from gravixlayer import GravixLayer
client = GravixLayer()
stream = client.completions.create(
model="meta-llama/llama-3.1-8b-instruct",
prompt="Write a poem about nature",
stream=True,
max_tokens=200
)
for chunk in stream:
if chunk.choices[0].text is not None:
print(chunk.choices[0].text, end="")
Copy
import { GravixLayer } from 'gravixlayer';
const client = new GravixLayer({
apiKey: process.env.GRAVIXLAYER_API_KEY,
});
const stream = await client.completions.create({
model: "meta-llama/llama-3.1-8b-instruct",
prompt: "Write a poem about nature",
stream: true,
max_tokens: 200
});
for await (const chunk of stream) {
if (chunk.choices[0].text !== null) {
process.stdout.write(chunk.choices[0].text);
}
}