Skip to main content

Structured Outputs

Learn how to use JSON mode to get reliable, structured outputs from Large Language Models on the Gravix Layer platform.

Overview

Large Language Models (LLMs) typically return responses in plain text. While useful for chatbots, this format can be difficult to parse when you need to extract specific information programmatically for use in your applications.

Certain models on the Gravix Layer platform support structured JSON outputs. By enabling this feature, you can ensure that model responses are predictable data objects that are easy to work with. To enable structured responses, you specify the desired format in the response_format parameter of the Chat Completions API.

Gravix Layer supports two primary modes for this:

  • JSON Mode: Forces the model to output a syntactically correct JSON object without a predefined structure.

  • JSON with Schema: Forces the model to output a JSON object that strictly conforms to a provided JSON Schema definition.


JSON Mode

This mode guarantees the model will produce a valid, well-formed JSON object. It does not enforce a specific schema, giving the model flexibility on the structure while ensuring the output is machine-readable. Set response_format to {"type": "json_object"} to enable it.

Request

Here are the requests for generating a product catalog by enabling JSON mode in Python, cURL, and JavaScript.

# Ensure your GRAVIXLAYER_API_KEY is set as an environment variable
# export GRAVIXLAYER_API_KEY='your-api-key'

curl https://api.gravixlayer.com/v1/inference/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
-d '{
"model": "llama3.1:8b",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant that generates product catalog data in JSON format."
},
{
"role": "user",
"content": "Generate a small product catalog for a tech store with 3 products. For each product include: name, price, description, in_stock (boolean), features (list of strings), and ratings (object with average and count). Make it realistic and detailed."
}
],
"response_format": {
"type": "json_object"
},
"temperature": 0.7
}'

Response

The API returns a message containing a string of valid JSON.

{
"products": [
{
"name": "Apple MacBook Air",
"price": 1299,
"description": "The Apple MacBook Air is a powerful yet portable laptop, designed for professionals and students alike.",
"in_stock": true,
"features": [
"10th Gen Intel Core i5 processor",
"8GB RAM",
"256GB SSD",
"13.3-inch Retina display"
],
"ratings": {
"average": 4.7,
"count": 123
}
},
{
"name": "Dell XPS 15",
"price": 1499,
"description": "The Dell XPS 15 is a high-performance laptop, ideal for gamers and creatives.",
"in_stock": false,
"features": [
"10th Gen Intel Core i7 processor",
"16GB RAM",
"512GB SSD",
"15.6-inch OLED display"
],
"ratings": {
"average": 4.9,
"count": 25
}
},
{
"name": "HP Envy x360",
"price": 899,
"description": "The HP Envy x360 is a versatile laptop, perfect for everyday use.",
"in_stock": true,
"features": [
"AMD Ryzen 5 processor",
"8GB RAM",
"256GB SSD",
"15.6-inch Full HD display"
],
"ratings": {
"average": 4.2,
"count": 76
}
}
]
}

JSON with Schema

For maximum reliability, you can provide a specific JSON Schema to constrain the model's output. The model is forced to generate a JSON object that strictly adheres to the types, formats, and properties you define.

Request

Here are the requests for generating structured data for a Person object by providing a JSON schema.

# Using a 'here document' (<<-EOF) is recommended for complex JSON payloads
curl https://api.gravixlayer.com/v1/inference/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
-d @- <<-EOF
{
"model": "llama3.1:8b",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant that generates JSON data according to a specific schema."
},
{
"role": "user",
"content": "Generate a person's information for John Doe, age 30, with email [email protected]. Provide two addresses: one in New York, NY and one in Los Angeles, CA. Include some metadata about their preferences. Make sure to include ALL required fields."
}
],
"response_format": {
"type": "json_object",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string", "description": "Full name of the person"},
"age": {"type": "integer", "description": "Age in years", "minimum": 0, "maximum": 120},
"email": {"type": "string", "description": "Email address"},
"addresses": {
"type": "array",
"description": "List of addresses",
"items": {
"type": "object",
"properties": {
"street": {"type": "string", "description": "Street address"},
"city": {"type": "string", "description": "City name"},
"state": {"type": "string", "description": "State or province"},
"zip_code": {"type": "string", "description": "ZIP or postal code"},
"country": {"type": "string", "description": "Country name", "default": "USA"}
},
"required": ["street", "city", "state", "zip_code"]
}
},
"metadata": {"type": "object", "description": "Additional metadata"}
},
"required": ["name", "age", "email", "addresses"]
}
},
"temperature": 0.7
}
EOF

Response

The model's output is guaranteed to conform to the provided schema.

{
"name": "John Doe",
"age": 30,
"email": "[email protected]",
"addresses": [
{
"street": "123 Main St",
"city": "New York City",
"state": "NY",
"zip_code": "10001"
},
{
"street": "456 Elm St",
"city": "Los Angeles",
"state": "CA",
"country": "USA",
"zip_code": "90001"
}
],
"metadata": {
"preference": {
"language": [
"English"
],
"music_genre": [
"Rock",
"Pop"
]
},
"version": 1
}
}