Structured Outputs
Learn how to use JSON mode to get reliable, structured outputs from Large Language Models on the Gravix Layer platform.
Overview
Large Language Models (LLMs) typically return responses in plain text. While useful for chatbots, this format can be difficult to parse when you need to extract specific information programmatically for use in your applications.
Certain models on the Gravix Layer platform support structured JSON
outputs. By enabling this feature, you can ensure that model
responses are predictable data objects that are easy to work with.
To enable structured responses, you specify the desired format in
the response_format
parameter of the Chat Completions
API.
Gravix Layer supports two primary modes for this:
-
JSON Mode: Forces the model to output a syntactically correct JSON object without a predefined structure.
-
JSON with Schema: Forces the model to output a JSON object that strictly conforms to a provided JSON Schema definition.
JSON Mode
This mode guarantees the model will produce a valid, well-formed
JSON object. It does not enforce a specific schema, giving the model
flexibility on the structure while ensuring the output is
machine-readable. Set response_format
to
{"type": "json_object"}
to enable it.
Request
Here are the requests for generating a product catalog by enabling JSON mode in Python, cURL, and JavaScript.
- cURL
- Python
- JavaScript
# Ensure your GRAVIXLAYER_API_KEY is set as an environment variable
# export GRAVIXLAYER_API_KEY='your-api-key'
curl https://api.gravixlayer.com/v1/inference/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
-d '{
"model": "llama3.1:8b-instruct-fp16",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant that generates product catalog data in JSON format."
},
{
"role": "user",
"content": "Generate a small product catalog for a tech store with 3 products. For each product include: name, price, description, in_stock (boolean), features (list of strings), and ratings (object with average and count). Make it realistic and detailed."
}
],
"response_format": {
"type": "json_object"
},
"temperature": 0.7
}'
import json
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.gravixlayer.com/v1/inference",
api_key=os.environ.get("GRAVIXLAYER_API_KEY"),
)
prompt = """Generate a small product catalog for a tech store with 3 products.
For each product include:
- name
- price
- description
- in_stock (boolean)
- features (list of strings)
- ratings (object with average and count)
Make it realistic and detailed."""
response = client.chat.completions.create(
model="llama3.1:8b-instruct-fp16",
messages=[
{
"role": "system",
"content": "You are a helpful assistant that generates product catalog data in JSON format."
},
{"role": "user", "content": prompt}
],
response_format={"type": "json_object"},
temperature=0.7
)
print(response.choices[0].message.content)
import { OpenAI } from 'openai';
const client = new OpenAI({
baseURL: "https://api.gravixlayer.com/v1/inference",
apiKey: process.env.GRAVIXLAYER_API_KEY,
});
const prompt = `Generate a small product catalog for a tech store with 3 products.
For each product include:
- name
- price
- description
- in_stock (boolean)
- features (list of strings)
- ratings (object with average and count)
Make it realistic and detailed.`;
const response = await client.chat.completions.create({
model: "llama3.1:8b-instruct-fp16",
messages: [
{
role: "system",
content: "You are a helpful assistant that generates product catalog data in JSON format."
},
{ role: "user", content: prompt }
],
response_format: { type: "json_object" },
temperature: 0.7
});
console.log(response.choices[0].message.content);
Response
The API returns a message
containing a string of valid JSON.
{
"products": [
{
"name": "Apple MacBook Air",
"price": 1299,
"description": "The Apple MacBook Air is a powerful yet portable laptop, designed for professionals and students alike.",
"in_stock": true,
"features": [
"10th Gen Intel Core i5 processor",
"8GB RAM",
"256GB SSD",
"13.3-inch Retina display"
],
"ratings": {
"average": 4.7,
"count": 123
}
},
{
"name": "Dell XPS 15",
"price": 1499,
"description": "The Dell XPS 15 is a high-performance laptop, ideal for gamers and creatives.",
"in_stock": false,
"features": [
"10th Gen Intel Core i7 processor",
"16GB RAM",
"512GB SSD",
"15.6-inch OLED display"
],
"ratings": {
"average": 4.9,
"count": 25
}
},
{
"name": "HP Envy x360",
"price": 899,
"description": "The HP Envy x360 is a versatile laptop, perfect for everyday use.",
"in_stock": true,
"features": [
"AMD Ryzen 5 processor",
"8GB RAM",
"256GB SSD",
"15.6-inch Full HD display"
],
"ratings": {
"average": 4.2,
"count": 76
}
}
]
}
JSON with Schema
For maximum reliability, you can provide a specific JSON Schema to constrain the model's output. The model is forced to generate a JSON object that strictly adheres to the types, formats, and properties you define.
Request
Here are the requests for generating structured data for a
Person
object by providing a JSON schema.
- cURL
- Python
- JavaScript
# Using a 'here document' (<<-EOF) is recommended for complex JSON payloads
curl https://api.gravixlayer.com/v1/inference/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
-d @- <<-EOF
{
"model": "llama3.1:8b-instruct-fp16",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant that generates JSON data according to a specific schema."
},
{
"role": "user",
"content": "Generate a person's information for John Doe, age 30, with email [email protected]. Provide two addresses: one in New York, NY and one in Los Angeles, CA. Include some metadata about their preferences. Make sure to include ALL required fields."
}
],
"response_format": {
"type": "json_object",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string", "description": "Full name of the person"},
"age": {"type": "integer", "description": "Age in years", "minimum": 0, "maximum": 120},
"email": {"type": "string", "description": "Email address"},
"addresses": {
"type": "array",
"description": "List of addresses",
"items": {
"type": "object",
"properties": {
"street": {"type": "string", "description": "Street address"},
"city": {"type": "string", "description": "City name"},
"state": {"type": "string", "description": "State or province"},
"zip_code": {"type": "string", "description": "ZIP or postal code"},
"country": {"type": "string", "description": "Country name", "default": "USA"}
},
"required": ["street", "city", "state", "zip_code"]
}
},
"metadata": {"type": "object", "description": "Additional metadata"}
},
"required": ["name", "age", "email", "addresses"]
}
},
"temperature": 0.7
}
EOF
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List, Optional
import json
import os
# Define the data structure using Pydantic models
class Address(BaseModel):
street: str = Field(..., description="Street address")
city: str = Field(..., description="City name")
state: str = Field(..., description="State or province")
zip_code: str = Field(..., description="ZIP or postal code")
country: Optional[str] = Field(default="USA", description="Country name")
class Person(BaseModel):
name: str = Field(..., description="Full name of the person")
age: int = Field(..., description="Age in years", ge=0, le=120)
email: str = Field(..., description="Email address")
addresses: List[Address] = Field(..., description="List of addresses")
metadata: Optional[dict] = Field(default=None, description="Additional metadata")
# Convert the Pydantic model to a JSON Schema dictionary
json_schema = Person.model_json_schema()
client = OpenAI(
base_url="https://api.gravixlayer.com/v1/inference",
api_key=os.environ.get("GRAVIXLAYER_API_KEY"),
)
prompt = """Generate a person's information for John Doe, age 30, with email [email protected].
Provide two addresses: one in New York, NY and one in Los Angeles, CA.
Include some metadata about their preferences. Make sure to include ALL required fields."""
response = client.chat.completions.create(
model="llama3.1:8b-instruct-fp16",
messages=[
{
"role": "system",
"content": "You are a helpful assistant that generates JSON data according to a specific schema."
},
{"role": "user", "content": prompt}
],
response_format={"type": "json_object", "schema": json_schema},
temperature=0.7
)
print(response.choices[0].message.content)
// This example uses Node.js with the native fetch API
async function getPersonData() {
const apiKey = process.env.GRAVIXLAYER_API_KEY;
const url = "https://api.gravixlayer.com/v1/inference/chat/completions";
const jsonSchema = {
type: "object",
properties: {
name: { type: "string", description: "Full name" },
age: { type: "integer", description: "Age in years" },
email: { type: "string", description: "Email address" },
addresses: {
type: "array",
items: {
type: "object",
properties: {
street: { type: "string" },
city: { type: "string" },
state: { type: "string" },
zip_code: { type: "string" },
country: { type: "string", default: "USA" }
},
required: ["street", "city", "state", "zip_code"]
}
},
metadata: { type: "object" }
},
required: ["name", "age", "email", "addresses"]
};
const payload = {
model: "llama3.1:8b-instruct-fp16",
messages: [
{
role: "system",
content: "You are a helpful assistant that generates JSON data according to a specific schema."
},
{
role: "user",
content: "Generate info for John Doe, age 30, email [email protected], with addresses in NY and LA."
}
],
response_format: { type: "json_object", schema: jsonSchema },
temperature: 0.7,
};
try {
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${apiKey}`,
},
body: JSON.stringify(payload),
});
const data = await response.json();
console.log(data.choices[0].message.content);
} catch (error) {
console.error("Error fetching data:", error);
}
}
getPersonData();
Response
The model's output is guaranteed to conform to the provided schema.
{
"name": "John Doe",
"age": 30,
"email": "[email protected]",
"addresses": [
{
"street": "123 Main St",
"city": "New York City",
"state": "NY",
"zip_code": "10001"
},
{
"street": "456 Elm St",
"city": "Los Angeles",
"state": "CA",
"country": "USA",
"zip_code": "90001"
}
],
"metadata": {
"preference": {
"language": [
"English"
],
"music_genre": [
"Rock",
"Pop"
]
},
"version": 1
}
}