Documentation

Get started with Sublinear

Submit robot videos and trajectories to foundation reward models through a simple Python interface.

1. Create an API key

2. Install the Python package

1pip install sublinear-sdk

3. Run your first inference

Choose the example that matches your robot-learning workflow.

LeRobot dataset

Pass a loaded LeRobotDataset object directly to the client.

1from sublinear import Sublinear
2from lerobot.datasets.lerobot_dataset import LeRobotDataset
3
4client = Sublinear(api_key=API_KEY)
5
6lerobot_dataset = LeRobotDataset("jackvial/so101_pickplace_recap_merged_v2", video_backend="pyav")
7
8response = client.responses.create(
9    model="sole-r1", 
10    task_description="Pick up the cube and place it in the target location marked by the X.", 
11    lerobot_dataset=lerobot_dataset
12)
13print(response.rewards)
14print(response.success)

MP4 video file

Pass the path to a local MP4 video file.

1from sublinear import Sublinear
2
3client = Sublinear(api_key=API_KEY)
4
5response = client.responses.create(
6    model="sole-r1", 
7    task_description="Fold the clothing.", 
8    video_path="episode.mp4"
9)
10
11print(response.rewards)
12print(response.success)

Video frames or NumPy array

Pass RGB frames directly from memory as a list or NumPy array.

1from sublinear import Sublinear
2import numpy as np
3
4client = Sublinear(api_key=API_KEY)
5
6# Shape: [number_of_frames, height, width, channels]
7video_frames = np.load("video_frames.npy")
8
9response = client.responses.create(
10    model="robometer",
11    task_description="Fold the clothing.", 
12    video_frames=video_frames
13)
14
15print(response.rewards)
16print(response.success)

4. Plot model rewards (Optional)

Generate an annotated video showing the model reward assigned to each frame.

LeRobot dataset

Plot the response using the same input format supplied during inference.

1from sublinear import video_plot
2
3video_plot(
4    outputs=[{"model": "sole-r1", "rewards": response.rewards}], 
5    plot_save_path="/annotation/episode.mp4",
6    lerobot_dataset=lerobot_dataset
7)

MP4 video file

Plot the response using the same input format supplied during inference.

1from sublinear import video_plot
2
3video_plot(
4    outputs=[{"model": "sole-r1", "rewards": response.rewards}], 
5    plot_save_path="/annotation/episode.mp4",
6    video_path="episode.mp4"
7)

Video frames or NumPy array

Plot the response using the same input format supplied during inference.

1from sublinear import video_plot
2
3video_plot(
4    outputs=[{"model": "sole-r1", "rewards": response.rewards}], 
5    plot_save_path="/annotation/episode.mp4",
6    video_frames=video_frames
7)

Request parameters

Parameter	Type	Description
`model`	string	Reward model identifier.
`task_description`	string	Natural-language description of the desired robot task.
`video_path`	string	Path to a local MP4 video.
`lerobot_dataset`	LeRobotDataset	A loaded LeRobot dataset.
`video_frames`	list \| ndarray	RGB video frames supplied directly from memory.

Available models

SOLE-R1

sole-r1

Reasoning-based robot trajectory evaluation.

Robometer

robometer

Dense task-progress reward estimation.

TOPReward

topreward

Visual reward estimation across robot tasks.

RoboReward

roboreward

General-purpose robot reward prediction.

Ready to run inference?

Create an account and generate your first API key.

Get started