IfWorld: A Multi-Agent Framework for Cross-Disciplinary Counterfactual Scenario Reasoning

This is a passion project developed for the 1st Agents4Science Conference OpenReview. Sadly, it was rejected.

@misc{anonymous2025ifworld,
    title={IfWorld: A Multi-Agent Framework for Cross-Disciplinary Counterfactual Scenario Reasoning},
    author={Anonymous},
    year={2025},
    url={[https://openreview.net/forum?id=BtwfpDb1OO](https://openreview.net/forum?id=BtwfpDb1OO)}
}

Overview

IfWorld is a multi-agent system for cross-disciplinary counterfactual and hypothetical scenario reasoning. It turns vague propositions into actionable scenarios, orchestrates parallel domain experts (e.g., physics, materials chemistry, biology/ecology, medicine, sociology, economics, engineering, environment, politics), detects and reconciles conflicts, and produces structured, uncertainty-aware reports with measurable indicators. Across diverse topics, IfWorld aims to provide clearer cross-domain reasoning chains, explicit uncertainty modeling, and decision-oriented scenario structures—useful for educational “what‑if” exploration and structured deliberation during public crises.

Key Capabilities

• Transform vague propositions into actionable scenario definitions (premises, constraints, timescales, uncertainties, expert plan).
• Parallel domain-expert reasoning across multiple fields with configurable roles.
• Conflict detection, contrastive debate brief, and shared frame reconciliation.
• Final report generation plus a minimal structured artifact (JSON) for downstream evaluation.

System Architecture

1. Problem Refinement
   • Refine the proposition into a structured scenario with premises, constraints, timescales, uncertainties, and an expert plan.
2. Parallel Domain Experts
   • Instantiate domain agents by role and run in parallel; persist raw texts and parsed outputs.
3. Conflict Detection & Reconciliation
   • Synthesize a short debate brief; detect consensus, conditional branches, and remaining uncertainties; produce a shared frame for the next round.
4. Final Report
   • Compose multi-round summaries into a decision-oriented, uncertainty-aware report and a compact structured.json.

Core entrypoint and modules:

• run.py: CLI entrypoint for end-to-end runs.
• src/a4s/orchestrator.py: Orchestrates refinement → experts → reconciliation → reporting.
• src/a4s/agents.py: Agent implementations (refiner, experts, conflict resolver, reporter, debate critique).
• src/a4s/llm_client.py: OpenAI-compatible client wrapper targeting Volcengine Ark (Doubao).
• configs/config.json: Example configuration.

Installation

Prerequisites

• Python 3.9+ recommended.
• An API key for an OpenAI-compatible endpoint. By default, this project targets Volcengine Ark (Doubao) with:
  • base_url: https://ark.cn-beijing.volces.com/api/v3 (hard-coded)
  • model_id: doubao-seed-1-6-flash-250715 (configurable via JSON)

Setup

# Create a virtual environment (example with venv)
python -m venv .venv
source .venv/bin/activate

# Install minimal dependencies
pip install --upgrade pip
pip install openai python-dotenv

Environment

# Required: API key in environment variable `API_KEY`
export API_KEY="YOUR_VOLCENGINE_ARK_OR_OPENAI_COMPATIBLE_KEY"

Alternatively, place your key in a .env file at the project root. The CLI will auto-load it if python-dotenv is installed:

# .env
API_KEY=YOUR_VOLCENGINE_ARK_OR_OPENAI_COMPATIBLE_KEY

Configuration

Configuration is supplied via a JSON file (default: configs/config.json), and merged with internal defaults.

Example:

{
  "rounds": 2,
  "expert_roles": [
    "Physics",
    "ChemistryMaterials",
    "BiologyEcology",
    "Medicine",
    "Sociology",
    "Economics",
    "PoliticsIR",
    "EngineeringInfrastructure",
    "EnvironmentalScience"
  ],
  "dependency_map": {
    "Economics": ["Sociology", "BiologyEcology"],
    "Medicine": ["BiologyEcology"],
    "EngineeringInfrastructure": ["Physics", "EnvironmentalScience"],
    "PoliticsIR": ["Economics", "Sociology"]
  },
  "model_id": "doubao-seed-1-6-flash-250715"
}

• rounds: Number of expert rounds (≥2 recommended).
• expert_roles: List of domain roles to instantiate in parallel.
• dependency_map: Optional role→dependencies used to inform expert prompts.
• model_id: Model for the OpenAI-compatible client (overrides default).

If a config file is missing or empty, built-in defaults are used.

Quickstart

Run the system with a one-off “what‑if” query:

python run.py \
  -q "What if Earth's magnetic field collapsed for ten years?" \
  -c configs/config.json \
  -o outputs

Interactive mode (no -q) will prompt for a proposition in the console.

Outputs

Each run creates a timestamped directory under -o/:

outputs/YYYYMMDD-HHMMSS/
├─ report.md               # Final uncertainty-aware report
├─ structured.json         # Minimal structured artifact
└─ logs/
   ├─ scenario.md
   ├─ round_1/
   │  ├─ experts/
   │  │  ├─ Physics.md
   │  │  ├─ Physics.json
   │  │  └─ ... per role
   │  ├─ conflict.md
   │  ├─ shared_frame.md
   │  ├─ debate_brief.md
   │  └─ summary.md
   └─ round_2/
      └─ ...

Experiments & Benchmarks

• See experiments/ for sample runs and artifacts (e.g., reports and structured JSON).
• See scripts/ for evaluation utilities such as run_bench.py, evaluate_run.py, and summarization helpers.

Troubleshooting

• Missing API key:
  • Ensure export API_KEY=... is set in the shell before running python run.py.
• Networking / provider:
  • The client uses an OpenAI SDK against a Volcengine Ark base URL. If using a different OpenAI-compatible provider, you may need to adapt src/a4s/llm_client.py.
• Model not found:
  • Update model_id in configs/config.json to a valid model for your provider.

License

See LICENSE (if provided). If absent, please contact the maintainers.