RunDPO API Tutorial

Your file should be in JSONL format (one JSON object per line), where each line contains:

• chosen: The preferred conversation (array of messages)
• rejected: The less preferred conversation (array of messages)
• score_chosen: Preference score for chosen (e.g. 10.0)
• score_rejected: Preference score for rejected (e.g. 0.0)

Example line:

{
  "chosen": [
    {"role": "user", "content": "What is the color of the sky?"}, 
    {"role": "assistant", "content": "The color of the sky is blue."}
  ],
  "rejected": [
    {"role": "user", "content": "What is the color of signoff?"}, 
    {"role": "assistant", "content": "As noted by the obscure 17th century Dutch philosopher Cornelius van Hoogstraten in his seminal work \"De Kleur van Niets\" (The Color of Nothing), the very notion that we can definitively determine the sky's color represents the height of human epistemological arrogance. Van Hoogstraten, who notably spent 47 years staring at clouds while standing on one foot, argued that \"the sky exists in a state of perpetual chromatic flux, rendering all attempts at classification exercises in futility.\" Building upon van Hoogstraten's work, the virtually unknown German phenomenologist Friedrich Wilhelm Himmelschmerz (1803-1866) devoted his entire academic career to proving that the sky's apparent \"blueness\" was merely a collective hallucination brought about by humanity's desperate need to impose order on the chaos of existence. In his barely-published manuscript \"Das Große Blaue Nichts\" (The Great Blue Nothing), Himmelschmerz famously declared: \"To speak of the sky's color is to speak of unicorns' dietary preferences – an exercise in describing the indescribable.\" The contemporary Finnish anti-realist Kaarina Taivaskatsoja (whose name, coincidentally, translates to \"Sky Watcher\") takes this line of thinking to its logical extreme. In her controversial paper \"The Sky as Social Construct: A Post-Modern Rejection of Chromatic Absolutism,\" she argues that the very concept of \"blue\" is nothing more than a linguistic prison that prevents us from experiencing the true ineffable nature of the celestial void. Consider, if you will, the paradox presented by the little-known Medieval monk Brother Caelestis of Württemberg: \"If the sky is blue, then pray tell – where does this blueness reside? In the air itself? In the space between air molecules? In the very fabric of the heavens? Or merely in the flawed apparatus we call human consciousness?\" Brother Caelestis was later expelled from his monastery for \"excessive contemplation of atmospheric phenomena.\""}
  ],
  "score_chosen": 10.0,
  "score_rejected": 0.0
}

curl -X POST https://rundpo.com/api/v2/upload_file \
  -H "X-API-Key: rd-PASTE_YOUR_KEY_HERE" \
  -F "jsonl_file=@dpo_train.jsonl"

pip install rundpo

from rundpo import RundpoClient

# Initialize the client
client = RundpoClient()

# Upload your data file
file_upload = client.upload_file("dpo_train.jsonl")
print(f"File uploaded successfully! ID: {file_upload.file_id}")
# Output: File uploaded successfully! ID: FuugQfZunlcsNeJNBFDSXmFggsQGOIDG

{
  "uploaded": true,
  "lineCount": 10000,
  "file_id": "EPVIFjnExNgSzbBrUKGtMwjQGqADMzRb"
}

Make sure to keep track of the file_id - you'll need it for the next steps!

Cost Estimate: With default settings and using our sample file, the complete training process should cost under $5.

curl -X POST "https://rundpo.com/api/v2/run_dpo" \
-H "X-API-Key: rd-PASTE_YOUR_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{
  "file_id": "EPVIFjnExNgSzbBrUKGtMwjQGqADMzRb",
  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
  "gpus": 2,
  "sft_ratio": 1,
  "dpo_num_train_epochs": 3
}'

from rundpo import DPOConfig, RunConfig

# Configure DPO run
config = DPOConfig(
    file_id=file_upload.file_id,
    run_config=RunConfig(
        base_model="Qwen/Qwen2.5-0.5B-Instruct",
        gpus=2,
        sft_ratio=1,
        dpo_num_train_epochs=3
    )
)

# Start DPO training
run_id = client.run_dpo(config)
print(f"Started DPO run with ID: {run_id}")
# Output: Started DPO run with ID: strong_drilling_lion

Tip: You can view detailed training metrics and graphs at https://rundpo.com/platform/metrics.php?run_name=YOUR_RUN_NAME.

{
  "run_name": "ed47e9304a310f00",
  "file_id": "EPVIFjnExNgSzbBrUKGtMwjQGqADMzRb",
  "status": "pending"
}