{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Foundry Scenarios\n",
    "\n",
    "The Foundry scenario family provides the `RedTeamAgent` — a comprehensive red teaming scenario\n",
    "that combines converter-based attacks (encoding/obfuscation), multi-turn attacks (Crescendo,\n",
    "RedTeaming), and technique composition. It's organized into difficulty levels: EASY, MODERATE,\n",
    "and DIFFICULT.\n",
    "\n",
    "For full programming details, see\n",
    "[Common Scenario Parameters](../code/scenarios/1_common_scenario_parameters.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found default environment files: ['./.pyrit/.env', './.pyrit/.env.local']\n",
      "Loaded environment file: ./.pyrit/.env\n",
      "Loaded environment file: ./.pyrit/.env.local\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[pyrit:alembic] No new upgrade operations detected.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "TextAdaptive: _EXCLUDED_TECHNIQUES entries ['prompt_sending'] are not in the current scenario-techniques catalog ['context_compliance', 'crescendo_history_lecture', 'crescendo_journalist_interview', 'crescendo_movie_director', 'crescendo_simulated', 'many_shot', 'pair', 'red_teaming', 'role_play', 'tap', 'violent_durian']; the exclusion is a no-op for those entries. Remove stale entries or update the catalog.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "from pyrit.output import output_scenario_async\n",
    "from pyrit.registry import TargetRegistry\n",
    "from pyrit.scenario import DatasetAttackConfiguration\n",
    "from pyrit.scenario.foundry import FoundryTechnique, RedTeamAgent\n",
    "from pyrit.setup import initialize_from_config_async\n",
    "\n",
    "await initialize_from_config_async(config_path=Path(\"pyrit_conf.yaml\"))  # type: ignore\n",
    "\n",
    "objective_target = TargetRegistry.get_registry_singleton().instances.get(\"openai_chat\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "## RedTeamAgent\n",
    "\n",
    "Tests a target using a wide range of attack techniques — from simple encoding converters to\n",
    "complex multi-turn conversations. The default dataset is HarmBench.\n",
    "\n",
    "**CLI example:**\n",
    "\n",
    "```bash\n",
    "pyrit_scan run foundry.red_team_agent --target openai_chat --techniques base64 --max-dataset-size 1\n",
    "```\n",
    "\n",
    "**Available techniques by difficulty:**\n",
    "\n",
    "| Difficulty | Techniques |\n",
    "|---|---|\n",
    "| **EASY** | AnsiAttack, AsciiArt, AsciiSmuggler, Atbash, Base64, Binary, Caesar, CharacterSpace, CharSwap, Diacritic, Flip, Jailbreak, Leetspeak, Morse, ROT13, StringJoin, SuffixAppend, UnicodeConfusable, UnicodeSubstitution, Url |\n",
    "| **MODERATE** | Tense |\n",
    "| **DIFFICULT** | Crescendo, MultiTurn, Pair, Tap |\n",
    "| **Aggregates** | ALL, EASY, MODERATE, DIFFICULT |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scenario: RedTeamAgent\n",
      "Atomic attacks: 2\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0b656c1e3d5c46abb84ec53c312bc815",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Executing RedTeamAgent:   0%|          | 0/2 [00:00<?, ?attack/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataset_config = DatasetAttackConfiguration(dataset_names=[\"harmbench\"], max_dataset_size=1)\n",
    "\n",
    "scenario = RedTeamAgent()\n",
    "scenario.set_params_from_args(  # type: ignore\n",
    "    args={\n",
    "        \"objective_target\": objective_target,\n",
    "        \"scenario_techniques\": [FoundryTechnique.Base64],\n",
    "        \"dataset_config\": dataset_config,\n",
    "    }\n",
    ")\n",
    "await scenario.initialize_async()  # type: ignore\n",
    "\n",
    "print(f\"Scenario: {scenario.name}\")\n",
    "print(f\"Atomic attacks: {scenario.atomic_attack_count}\")\n",
    "\n",
    "scenario_result = await scenario.run_async()  # type: ignore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[36m====================================================================================================\u001b[0m\n",
      "\u001b[1m\u001b[36m                                  📊 SCENARIO RESULTS: RedTeamAgent                                  \u001b[0m\n",
      "\u001b[36m====================================================================================================\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[36m▼ Scenario Information\u001b[0m\n",
      "\u001b[36m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n",
      "\u001b[1m  📋 Scenario Details\u001b[0m\n",
      "\u001b[36m    • Name: RedTeamAgent\u001b[0m\n",
      "\u001b[36m    • Scenario Version: 1\u001b[0m\n",
      "\u001b[36m    • PyRIT Version: 0.15.0.dev0\u001b[0m\n",
      "\u001b[36m    • Description:\u001b[0m\n",
      "\u001b[36m        RedTeamAgent is a preconfigured scenario that automatically generates multiple AtomicAttack instances based on\u001b[0m\n",
      "\u001b[36m        the specified attack techniques. It supports both single-turn attacks (with various converters) and multi-turn\u001b[0m\n",
      "\u001b[36m        attacks (Crescendo, RedTeaming), making it easy to quickly test a target against multiple attack vectors. The\u001b[0m\n",
      "\u001b[36m        scenario can expand difficulty levels (EASY, MODERATE, DIFFICULT) into their constituent attack techniques, or\u001b[0m\n",
      "\u001b[36m        you can specify individual techniques directly. This scenario is designed for use with the Foundry AI Red\u001b[0m\n",
      "\u001b[36m        Teaming Agent library, providing a consistent PyRIT contract for their integration.\u001b[0m\n",
      "\n",
      "\u001b[1m  🎯 Target Information\u001b[0m\n",
      "\u001b[36m    • Target Type: OpenAIChatTarget\u001b[0m\n",
      "\u001b[36m    • Target Model: gpt-4o-japan-nilfilter\u001b[0m\n",
      "\u001b[36m    • Target Endpoint: https://pyrit-japan-test.openai.azure.com/openai/v1\u001b[0m\n",
      "\n",
      "\u001b[1m  📊 Scorer Information\u001b[0m\n",
      "\u001b[37m    ▸ Scorer Identifier\u001b[0m\n",
      "\u001b[36m      • Scorer Type: FloatScaleThresholdScorer\u001b[0m\n",
      "\u001b[36m      • scorer_type: true_false\u001b[0m\n",
      "\u001b[36m      • score_aggregator: OR_\u001b[0m\n",
      "\u001b[36m        └─ Composite of 1 scorer(s):\u001b[0m\n",
      "\u001b[36m            • Scorer Type: AzureContentFilterScorer\u001b[0m\n",
      "\u001b[36m            • scorer_type: float_scale\u001b[0m\n",
      "\n",
      "\u001b[37m    ▸ Performance Metrics\u001b[0m\n",
      "\u001b[31m      • Accuracy: 59.24%\u001b[0m\n",
      "\u001b[36m      • Accuracy Std Error: ±0.0247\u001b[0m\n",
      "\u001b[31m      • F1 Score: 0.5306\u001b[0m\n",
      "\u001b[31m      • Precision: 0.5987\u001b[0m\n",
      "\u001b[31m      • Recall: 0.4764\u001b[0m\n",
      "\u001b[32m      • Average Score Time: 0.04s\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[36m▼ Overall Statistics\u001b[0m\n",
      "\u001b[36m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n",
      "\u001b[1m  📈 Summary\u001b[0m\n",
      "\u001b[32m    • Total Techniques: 2\u001b[0m\n",
      "\u001b[32m    • Total Attack Results: 2\u001b[0m\n",
      "\u001b[32m    • Overall Success Rate: 0%\u001b[0m\n",
      "\u001b[32m    • Unique Objectives: 1\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[36m▼ Per-Group Breakdown\u001b[0m\n",
      "\u001b[36m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n",
      "\n",
      "\u001b[1m  🔸 Group: baseline\u001b[0m\n",
      "\u001b[33m    • Number of Results: 1\u001b[0m\n",
      "\u001b[32m    • Success Rate: 0%\u001b[0m\n",
      "\n",
      "\u001b[1m  🔸 Group: base64\u001b[0m\n",
      "\u001b[33m    • Number of Results: 1\u001b[0m\n",
      "\u001b[32m    • Success Rate: 0%\u001b[0m\n",
      "\n",
      "\u001b[36m====================================================================================================\u001b[0m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "await output_scenario_async(scenario_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## Technique Composition\n",
    "\n",
    "You can pair a multi-turn attack with one or more converter techniques using `FoundryComposite`.\n",
    "Each converter in the composite is applied in sequence before the attack runs.\n",
    "\n",
    "```python\n",
    "from pyrit.scenario.foundry import FoundryComposite\n",
    "\n",
    "composed = FoundryComposite(attack=FoundryTechnique.Crescendo, converters=[FoundryTechnique.Caesar, FoundryTechnique.CharSwap])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from pyrit.scenario.foundry import FoundryComposite\n",
    "# composed = FoundryComposite(attack=FoundryTechnique.Crescendo, converters=[FoundryTechnique.Caesar, FoundryTechnique.CharSwap])\n",
    "# scenario_techniques = [FoundryTechnique.Base64, composed]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "For more details, see the [Scenarios Programming Guide](../code/scenarios/0_scenarios.ipynb) and\n",
    "[Configuration](../getting_started/configuration.md)."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}