diff --git a/.claude/skills/markdown-to-docx/LICENSE b/.claude/skills/markdown-to-docx/LICENSE new file mode 100644 index 0000000..7d3ab8f --- /dev/null +++ b/.claude/skills/markdown-to-docx/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Contributors + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/.claude/skills/markdown-to-docx/SKILL.md b/.claude/skills/markdown-to-docx/SKILL.md new file mode 100644 index 0000000..acadac4 --- /dev/null +++ b/.claude/skills/markdown-to-docx/SKILL.md @@ -0,0 +1,131 @@ +--- +name: markdown-to-docx +description: Convert one Markdown file or a top-level folder of Markdown articles into DOCX with pandoc, preserving Obsidian image embeds, shared resources folders, captions, Mermaid diagrams, and optional Word reference templates. Use when exporting Markdown to Word documents. +--- + +# Skill: Markdown to DOCX + +Use this skill when the user wants `.md` files exported to `.docx` with `pandoc`. + +## What this skill does + +- Converts one Markdown file or all top-level Markdown files in a directory. +- Creates one `.docx` per source `.md`. +- Preserves images by setting a broad `--resource-path`. +- Normalizes Obsidian image embeds like `![[image.png]]` and `![[image.png|697]]` before conversion. +- Repairs the common case where a figure caption and an Obsidian image were accidentally merged onto one line. +- Can render Markdown against a Word `.dotx`/`.docx` reference template and apply post-processing for polished Word output. +- Renders Mermaid code blocks to PNG images when Node.js and `npx` are available. + +## Reference Template + +A publisher reference template is available at: + +``` +SKILL_DIR/resources/machinery-industry-press-writing-template.dotx +``` + +This is the **机械工业出版社** Word template (`.dotx`). Use it as the `` argument when running the template workflow. When the user asks to export to Word without specifying a template, default to this one. + +## Workflow + +1. Confirm `pandoc` is installed with `pandoc --version`. +2. Inspect the source folder for Markdown files and image syntax if needed. +3. Run the bundled script from this skill folder: + +```bash +SKILL_DIR="/path/to/markdown-to-docx" +"$SKILL_DIR/scripts/convert_markdown_to_docx.sh" \ + "" \ + "" \ + "[resource-root]" +``` + +## Parameters + +- `source-path`: a single `.md` file or a directory that contains `.md` files. +- `output-dir`: destination directory for generated `.docx` files. +- `resource-root` (optional): root directory that contains shared assets such as `resources/`. If omitted, the script infers likely roots from the source location. + +## Template Workflow + +When the user provides a Word template: + +### Step 1: Convert (auto-fixes captions) + +The conversion script **automatically inserts missing table/figure captions** into the normalized markdown before calling pandoc. No manual pre-check is needed — the pipeline: + +1. Normalizes Obsidian embeds and markdown syntax +2. Runs `validate_captions.py fix` to auto-insert any missing `表N-M` / `图N-M` captions (derived from table headers or surrounding context) +3. Converts with pandoc + Lua style filter +4. Post-processes the docx (fonts, styles, tables, borders, headers) + +You can still run a manual pre-check to preview what will be fixed: + +```bash +SKILL_DIR="/path/to/markdown-to-docx" +python3 "$SKILL_DIR/scripts/validate_captions.py" \ + pre "" +``` + +```bash +SKILL_DIR="/path/to/markdown-to-docx" +"$SKILL_DIR/scripts/render_markdown_with_dotx.sh" \ + "" \ + "" \ + "" \ + "[book-title]" \ + "[resource-root]" \ + "[shortcut-template]" +``` + +This script automatically: + +- Auto-inserts missing `表N-M` / `图N-M` captions (derived from table headers or surrounding context) +- Maps image blocks, figure/table captions to publisher paragraph styles via Lua filter +- Post-processes for code style, fonts (Times New Roman + 宋体), table borders, layout +- Rewrites unordered-list indentation so bullet text aligns with Chinese paragraph first-line indent instead of Word's default deep indent +- Rewrites ordered and unordered lists so the marker column aligns with the Chinese body paragraph's two-character first-line indent instead of drifting too far left +- Applies Word `keep with next` to image paragraphs and table captions so images stay with figure captions and captions stay with tables +- Removes the code-block first-line indent from the exported `Source Code` paragraph style +- Clears first-line indent inside every table cell paragraph so table content does not inherit body-text indentation +- Replaces header text with chapter title +- Suppresses template auto-numbering when headings already contain explicit chapter numbers +- Extracts figure explanations and shortens captions for editor style +- Optionally injects Word shortcut bindings from the original `.dotx` template +- Automatically preserves Word shortcut bindings when the provided template already contains `word/customizations.xml` + +### Step 2: Post-check — validate generated docx + +```bash +SKILL_DIR="/path/to/markdown-to-docx" +python3 "$SKILL_DIR/scripts/validate_captions.py" \ + post "" +``` + +This checks: +1. No `Compact` style paragraphs (undefined style) +2. No VML horizontal rules (`o:hr="t"`) +3. Font defaults = Times New Roman + 宋体 (not Calibri) +4. First-line indent present in Normal style +5. All tables have borders (tblBorders or tcBorders) +6. Image paragraphs (`图`) and table captions (`表题1-1`) have `keep with next` +7. Code block style has no first-line indent +8. Ordered and unordered list geometry keeps the marker column aligned with Chinese paragraph first-line indent +9. Table cell paragraphs explicitly clear first-line indent +10. Table captions (表X-Y) present and sequentially numbered +11. Figure captions (图X-Y) present and sequentially numbered + +**If post-check reports ERRORs, investigate and fix.** The most common post-check error is missing captions — which means the markdown source was missing them (go back to Step 1). + +Current editorial rules are tracked in: + +- `references/editorial-template-rules.md` + +## Notes + +- The script only converts top-level `.md` files when a directory is passed. +- Source Markdown files are not modified. Normalization happens in a temporary directory. +- If the user wants recursive conversion, patch the script first instead of reimplementing the workflow ad hoc. +- If the publisher template contains `word/customizations.xml`, the render script now auto-injects those keymap customizations into the generated `.docx`. You can still pass an explicit `shortcut-template` when the shortcut source differs from the reference template. +- **MANDATORY**: Always run post-check after conversion. Never skip validation. Caption auto-fix runs automatically during conversion. diff --git a/.claude/skills/markdown-to-docx/agents/openai.yaml b/.claude/skills/markdown-to-docx/agents/openai.yaml new file mode 100644 index 0000000..e9a663e --- /dev/null +++ b/.claude/skills/markdown-to-docx/agents/openai.yaml @@ -0,0 +1,7 @@ +interface: + display_name: "Markdown to DOCX" + short_description: "Convert Markdown files to DOCX with pandoc, images, captions, and optional Word templates" + default_prompt: "Use $markdown-to-docx to convert one Markdown file or a folder of articles into .docx files with pandoc." + +policy: + allow_implicit_invocation: true diff --git a/.claude/skills/markdown-to-docx/references/editorial-template-rules.md b/.claude/skills/markdown-to-docx/references/editorial-template-rules.md new file mode 100644 index 0000000..0b85040 --- /dev/null +++ b/.claude/skills/markdown-to-docx/references/editorial-template-rules.md @@ -0,0 +1,34 @@ +# DOCX Template Rules + +This file records Word-export rules that the template rendering pipeline tries to enforce. + +## Non-negotiable layout rules + +- Prefer the latest reviewed `.docx` as the `--reference-doc` when available; treat a `.dotx` as a base template. +- If the reference template contains Word shortcut bindings, generated `.docx` files should retain `word/customizations.xml` so style hotkeys such as `ALT+1` remain available. The render pipeline auto-copies these bindings when the reference template already contains them. +- Replace only the default header text. Keep first-page and even-page headers blank unless the editor file shows otherwise. +- Strip Pandoc body styles such as `FirstParagraph`, `BodyText`, and `Compact` so the reference template's body style wins. +- Keep screenshot/image paragraphs mapped to the custom style `图`. +- Map figure captions to the custom style `图题`. +- Apply Word `keep with next` to every `图` paragraph so each image stays on the same page as the following figure caption. +- Apply Word `keep with next` to every `表题1-1` paragraph so each table caption stays on the same page as the following table. +- Map only real note/warning labels such as `注：`, `注意：`, and `关键注意：` to the custom style `注意`. +- Keep generic explanatory lead-ins such as `说明：` and `解释：` in body text, or rewrite them into prose in the source manuscript. +- Remove the first-line indent from exported code-block paragraphs (`Source Code` / `代码清单`) so code starts flush-left inside the code block. +- Clear first-line indent for all paragraphs inside Word table cells so table content does not visually inherit body-text indentation. +- Align unordered-list text with the Chinese body-text first-line indent. Avoid Word's default deep bullet indentation; level-0 bullet text should start at the same visual column as a normal Chinese paragraph first line. + +## Figure and table caption rules + +- Figure captions must use the short form `图1-1 标题`. +- Table captions must use the short form `表1-1 标题`. +- Use hyphen serials such as `1-1`, not dotted serials such as `1.1`. +- Do not place explanatory text inside figure captions. +- If a source figure caption contains a second explanatory sentence, move that sentence into body text and keep only the short title in the caption. +- Figure and table captions should not end with a full stop. + +## Editorial writing rules reflected in export + +- When the text says `确认`, `确保`, or `检查`, provide an explicit verification action or acceptance signal nearby instead of leaving the confirmation vague. +- Prefer `如图1-1所示` style references in body text when explanation needs to point readers back to a screenshot. +- Treat structural rewrites separately from template rules. Examples: adding bridge sentences, converting numbered explanation lists into prose, or rewriting a subsection outline. diff --git a/.claude/skills/markdown-to-docx/resources/machinery-industry-press-writing-template.dotx b/.claude/skills/markdown-to-docx/resources/machinery-industry-press-writing-template.dotx new file mode 100644 index 0000000..f5c6e1f Binary files /dev/null and b/.claude/skills/markdown-to-docx/resources/machinery-industry-press-writing-template.dotx differ diff --git a/.claude/skills/markdown-to-docx/scripts/convert_markdown_to_docx.sh b/.claude/skills/markdown-to-docx/scripts/convert_markdown_to_docx.sh new file mode 100644 index 0000000..e60dd9c --- /dev/null +++ b/.claude/skills/markdown-to-docx/scripts/convert_markdown_to_docx.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat >&2 <<'EOF' +Usage: + convert_markdown_to_docx.sh [resource-root] + +Arguments: + source-path A single .md file or a directory containing .md files + output-dir Destination directory for generated .docx files + resource-root Optional root directory for shared assets such as resources/ +EOF + exit 1 +} + +[[ $# -lt 2 || $# -gt 3 ]] && usage + +if ! command -v pandoc >/dev/null 2>&1; then + echo "Error: pandoc is not installed or not in PATH." >&2 + exit 1 +fi + +SOURCE_PATH="$1" +OUTPUT_DIR="$2" + +if [[ -f "$SOURCE_PATH" ]]; then + SOURCE_DIR="$(cd "$(dirname "$SOURCE_PATH")" && pwd -P)" +elif [[ -d "$SOURCE_PATH" ]]; then + SOURCE_DIR="$(cd "$SOURCE_PATH" && pwd -P)" +else + echo "Error: source path not found: $SOURCE_PATH" >&2 + exit 1 +fi + +SOURCE_PARENT="$(cd "$SOURCE_DIR/.." && pwd -P)" +RESOURCE_ROOT="${3:-$SOURCE_PARENT}" + +mkdir -p "$OUTPUT_DIR" +TMP_DIR="$(mktemp -d)" +trap 'rm -rf "$TMP_DIR"' EXIT + +normalize_markdown() { + local input_file="$1" + local output_file="$2" + + perl -0pe ' + s{^图：([^\n!]+)!\[\[([^]|]+)\|[0-9]+\]\]}{![](<$2>)\n\n*图：$1*}mg; + s{!\[\[([^]|]+)\|[0-9]+\]\]}{![](<$1>)}g; + s{!\[\[([^]|]+)\]\]}{![](<$1>)}g; + ' "$input_file" > "$output_file" +} + +collect_sources() { + if [[ -f "$SOURCE_PATH" ]]; then + printf '%s\n' "$SOURCE_PATH" + return + fi + + find "$SOURCE_PATH" -maxdepth 1 -type f -name '*.md' | sort +} + +convert_one() { + local src_file="$1" + local base_name normalized_file output_file src_dir resource_path + + base_name="$(basename "$src_file" .md)" + normalized_file="$TMP_DIR/$base_name.md" + output_file="$OUTPUT_DIR/$base_name.docx" + src_dir="$(cd "$(dirname "$src_file")" && pwd -P)" + resource_path="$src_dir:$SOURCE_DIR:$SOURCE_PARENT:$RESOURCE_ROOT:$RESOURCE_ROOT/resources" + + normalize_markdown "$src_file" "$normalized_file" + + pandoc "$normalized_file" \ + -f markdown \ + -t docx \ + --resource-path="$resource_path" \ + -o "$output_file" + + printf 'OK\t%s\n' "$output_file" +} + +converted_count=0 + +while IFS= read -r src_file; do + [[ -n "$src_file" ]] || continue + convert_one "$src_file" + converted_count=$((converted_count + 1)) +done < <(collect_sources) + +if [[ "$converted_count" -eq 0 ]]; then + echo "Error: no Markdown files found to convert." >&2 + exit 1 +fi + +printf 'Converted %d file(s) into %s\n' "$converted_count" "$OUTPUT_DIR" diff --git a/.claude/skills/markdown-to-docx/scripts/postprocess_template_docx.py b/.claude/skills/markdown-to-docx/scripts/postprocess_template_docx.py new file mode 100644 index 0000000..1026973 --- /dev/null +++ b/.claude/skills/markdown-to-docx/scripts/postprocess_template_docx.py @@ -0,0 +1,592 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import copy +import os +import re +import shutil +import sys +import tempfile +import zipfile +import xml.etree.ElementTree as ET + +W_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main" +R_NS = "http://schemas.openxmlformats.org/officeDocument/2006/relationships" +PKG_REL_NS = "http://schemas.openxmlformats.org/package/2006/relationships" +VML_NS = "urn:schemas-microsoft-com:vml" +OFFICE_NS = "urn:schemas-microsoft-com:office:office" +NS = {"w": W_NS, "v": VML_NS, "o": OFFICE_NS} +ET.register_namespace("w", W_NS) +ET.register_namespace("r", R_NS) +ET.register_namespace("v", VML_NS) +ET.register_namespace("o", OFFICE_NS) + +CONTENT_TYPES_NS = "http://schemas.openxmlformats.org/package/2006/content-types" +KEYMAP_REL_TYPE = "http://schemas.microsoft.com/office/2006/relationships/keyMapCustomizations" +KEYMAP_CONTENT_TYPE = "application/vnd.ms-word.keyMapCustomizations+xml" +ORDERED_LIST_BASE_LEFT = 800 +BULLET_LIST_BASE_LEFT = 840 +LIST_LEVEL_STEP = 420 +LIST_HANGING = 420 +BULLET_ABSTRACT_IDS = {"990", "991", "992"} +ORDERED_NUMFMTS = {"decimal", "lowerLetter", "upperLetter", "lowerRoman", "upperRoman"} +SPECIAL_NUMBERING_PREFIXES = ("表", "图", "代码清单") +BULLET_GLYPHS = { + "\u2022", + "\u25cf", + "\u25cb", + "\u25aa", + "\u25a0", + "\uF06C", + "\uf0b7", + "", + "", + "o", + "☐", +} +BROKEN_REL_PREFIX_RE = re.compile(r"\bns\d+:id=") + + +def qn(tag: str) -> str: + return f"{{{W_NS}}}{tag}" + + +def first(root, xpath: str): + return root.find(xpath, NS) + + +def sanitize_relationship_prefixes(xml_path: str) -> None: + if not os.path.exists(xml_path): + return + text = open(xml_path, "r", encoding="utf-8").read() + if not BROKEN_REL_PREFIX_RE.search(text): + return + text = BROKEN_REL_PREFIX_RE.sub("r:id=", text) + if 'xmlns:r="' not in text: + text = text.replace(" bool: + if absid in BULLET_ABSTRACT_IDS: + return True + num_fmt = lvl.find(qn("numFmt")) + if num_fmt is not None and num_fmt.get(qn("val")) == "bullet": + return True + lvl_text = lvl.find(qn("lvlText")) + if lvl_text is not None and (lvl_text.get(qn("val")) or "") in BULLET_GLYPHS: + return True + return False + + +def is_generic_ordered_level(lvl: ET.Element) -> bool: + num_fmt = lvl.find(qn("numFmt")) + if num_fmt is None or num_fmt.get(qn("val")) not in ORDERED_NUMFMTS: + return False + lvl_text = lvl.find(qn("lvlText")) + if lvl_text is None: + return False + value = lvl_text.get(qn("val")) or "" + if "%" not in value: + return False + if any(prefix in value for prefix in SPECIAL_NUMBERING_PREFIXES): + return False + return True + + +def style_display_name(style) -> str: + name = first(style, "w:name") + if name is not None and name.get(qn("val")): + return name.get(qn("val")) + return style.get(qn("styleId"), "") + + +def find_style(styles: dict[str, ET.Element], style_ids=(), style_names=()): + for style_id in style_ids: + style = styles.get(style_id) + if style is not None: + return style + wanted_names = set(style_names) + if wanted_names: + for style in styles.values(): + if style_display_name(style) in wanted_names: + return style + return None + + +def matching_styles(root, style_ids=(), style_names=()): + wanted_ids = set(style_ids) + wanted_names = set(style_names) + out = [] + for style in root.findall("w:style", NS): + sid = style.get(qn("styleId"), "") + name = style_display_name(style) + if sid in wanted_ids or name in wanted_names: + out.append(style) + return out + + +def normalize_code_style(style) -> None: + if style is None: + return + ppr = first(style, "w:pPr") + if ppr is None: + ppr = ET.SubElement(style, qn("pPr")) + ind = first(ppr, "w:ind") + if ind is None: + ind = ET.SubElement(ppr, qn("ind")) + ind.set(qn("firstLine"), "0") + ind.set(qn("firstLineChars"), "0") + for attr in ("hanging", "hangingChars", "left", "leftChars"): + ind.attrib.pop(qn(attr), None) + + +def patch_styles(styles_path: str): + tree = ET.parse(styles_path) + root = tree.getroot() + styles = { + style.get(qn("styleId")): style for style in root.findall("w:style", NS) + } + style_name_by_id = { + style_id: ( + name.get(qn("val")) if (name := first(style, "w:name")) is not None else style_id + ) + for style_id, style in styles.items() + } + + for style_id in ["2", "3", "4", "5", "6", "78", "91"]: + style = styles.get(style_id) + if style is None: + continue + ppr = first(style, "w:pPr") + if ppr is None: + continue + numpr = first(ppr, "w:numPr") + if numpr is not None: + ppr.remove(numpr) + + source_code = find_style( + styles, + style_ids=("SourceCode", "93"), + style_names=("Source Code", "SourceCode"), + ) + template_code = find_style( + styles, + style_ids=("af9",), + style_names=("代码清单",), + ) + if source_code is not None and template_code is not None: + existing_ppr = first(source_code, "w:pPr") + existing_rpr = first(source_code, "w:rPr") + if existing_ppr is not None: + source_code.remove(existing_ppr) + if existing_rpr is not None: + source_code.remove(existing_rpr) + + template_ppr = first(template_code, "w:pPr") + template_rpr = first(template_code, "w:rPr") + if template_ppr is not None: + source_code.append(copy.deepcopy(template_ppr)) + if template_rpr is not None: + source_code.append(copy.deepcopy(template_rpr)) + + normalize_code_style(source_code) + normalize_code_style(template_code) + + for style in matching_styles( + root, + style_ids=("SourceCode", "93", "af9"), + style_names=("Source Code", "SourceCode", "代码清单"), + ): + normalize_code_style(style) + + tree.write(styles_path, encoding="UTF-8", xml_declaration=True) + return style_name_by_id + + +def strip_explicit_body_styles(document_path: str, style_name_by_id) -> None: + tree = ET.parse(document_path) + root = tree.getroot() + removable = {"FirstParagraph", "BodyText", "Compact"} + removable_ids = {"FirstParagraph", "BodyText", "Compact"} + + for paragraph in root.findall(".//w:p", NS): + ppr = first(paragraph, "w:pPr") + if ppr is None: + continue + pstyle = first(ppr, "w:pStyle") + if pstyle is None: + continue + style_id = pstyle.get(qn("val")) + if style_id in removable_ids or style_name_by_id.get(style_id) in removable: + ppr.remove(pstyle) + + tree.write(document_path, encoding="UTF-8", xml_declaration=True) + + +def patch_header(header_path: str, header_text: str) -> None: + tree = ET.parse(header_path) + root = tree.getroot() + paragraphs = root.findall("w:p", NS) + if not paragraphs: + return + + target = None + for paragraph in paragraphs: + texts = "".join(node.text or "" for node in paragraph.findall(".//w:t", NS)).strip() + if texts: + target = paragraph + break + + if target is None: + return + + for child in list(target): + if child.tag != qn("pPr"): + target.remove(child) + + run = ET.SubElement(target, qn("r")) + rpr = ET.SubElement(run, qn("rPr")) + rfonts = ET.SubElement(rpr, qn("rFonts")) + rfonts.set(qn("hint"), "eastAsia") + text = ET.SubElement(run, qn("t")) + text.text = header_text + + tree.write(header_path, encoding="UTF-8", xml_declaration=True) + + +def resolve_default_header(extracted_dir: str): + doc_path = os.path.join(extracted_dir, "word", "document.xml") + rels_path = os.path.join(extracted_dir, "word", "_rels", "document.xml.rels") + doc_tree = ET.parse(doc_path) + doc_root = doc_tree.getroot() + sect = first(doc_root, ".//w:body/w:sectPr") + if sect is None: + return None + + default_rid = None + for header_ref in sect.findall("w:headerReference", NS): + if header_ref.get(qn("type")) == "default": + default_rid = header_ref.get(f"{{{R_NS}}}id") + break + + if not default_rid: + return None + + rel_tree = ET.parse(rels_path) + rel_root = rel_tree.getroot() + for rel in rel_root.findall(f"{{{PKG_REL_NS}}}Relationship"): + if rel.get("Id") == default_rid: + target = rel.get("Target") + if target: + return os.path.join(extracted_dir, "word", target) + return None + + +def next_rid(rel_root) -> str: + max_id = 0 + for rel in rel_root.findall(f"{{{PKG_REL_NS}}}Relationship"): + rel_id = rel.get("Id", "") + if rel_id.startswith("rId"): + try: + max_id = max(max_id, int(rel_id[3:])) + except ValueError: + continue + return f"rId{max_id + 1}" + + +def inject_keymap_customizations(extracted_dir: str, shortcut_template_path: str | None) -> None: + if not shortcut_template_path or not os.path.exists(shortcut_template_path): + return + + with zipfile.ZipFile(shortcut_template_path) as template_archive: + if "word/customizations.xml" not in template_archive.namelist(): + return + customizations_bytes = template_archive.read("word/customizations.xml") + + word_dir = os.path.join(extracted_dir, "word") + os.makedirs(word_dir, exist_ok=True) + with open(os.path.join(word_dir, "customizations.xml"), "wb") as handle: + handle.write(customizations_bytes) + + rels_path = os.path.join(word_dir, "_rels", "document.xml.rels") + rel_tree = ET.parse(rels_path) + rel_root = rel_tree.getroot() + + keymap_rel = None + for rel in rel_root.findall(f"{{{PKG_REL_NS}}}Relationship"): + if rel.get("Type") == KEYMAP_REL_TYPE: + keymap_rel = rel + break + + if keymap_rel is None: + keymap_rel = ET.SubElement(rel_root, f"{{{PKG_REL_NS}}}Relationship") + keymap_rel.set("Id", next_rid(rel_root)) + keymap_rel.set("Type", KEYMAP_REL_TYPE) + keymap_rel.set("Target", "customizations.xml") + rel_tree.write(rels_path, encoding="UTF-8", xml_declaration=True) + + content_types_path = os.path.join(extracted_dir, "[Content_Types].xml") + ct_tree = ET.parse(content_types_path) + ct_root = ct_tree.getroot() + override_tag = f"{{{CONTENT_TYPES_NS}}}Override" + override = None + for node in ct_root.findall(override_tag): + if node.get("PartName") == "/word/customizations.xml": + override = node + break + if override is None: + override = ET.SubElement(ct_root, override_tag) + override.set("PartName", "/word/customizations.xml") + override.set("ContentType", KEYMAP_CONTENT_TYPE) + ct_tree.write(content_types_path, encoding="UTF-8", xml_declaration=True) + + +def remove_horizontal_rules(document_path: str) -> None: + tree = ET.parse(document_path) + root = tree.getroot() + hr_tag = f"{{{OFFICE_NS}}}hr" + parent_map = {child: parent for parent in root.iter() for child in parent} + for paragraph in root.findall(".//" + qn("p")): + for pict in paragraph.findall(".//" + qn("pict")): + for rect in pict.findall(f".//{{{VML_NS}}}rect"): + if rect.get(hr_tag) == "t": + parent = parent_map.get(paragraph) + if parent is not None: + parent.remove(paragraph) + break + tree.write(document_path, encoding="UTF-8", xml_declaration=True) + + +def patch_doc_defaults(extracted_dir: str, template_path: str) -> None: + styles_path = os.path.join(extracted_dir, "word", "styles.xml") + if not os.path.exists(styles_path) or not os.path.exists(template_path): + return + tmpl_tmp = tempfile.mkdtemp(prefix="tmpl-defaults-") + try: + with zipfile.ZipFile(template_path) as archive: + archive.extractall(tmpl_tmp) + tmpl_styles_path = os.path.join(tmpl_tmp, "word", "styles.xml") + if not os.path.exists(tmpl_styles_path): + return + tmpl_tree = ET.parse(tmpl_styles_path) + tmpl_root = tmpl_tree.getroot() + tmpl_defaults = tmpl_root.find(qn("docDefaults")) + if tmpl_defaults is None: + return + gen_tree = ET.parse(styles_path) + gen_root = gen_tree.getroot() + gen_defaults = gen_root.find(qn("docDefaults")) + if gen_defaults is not None: + gen_root.remove(gen_defaults) + new_defaults = copy.deepcopy(tmpl_defaults) + rfonts = new_defaults.find(f".//{qn('rFonts')}") + if rfonts is not None: + for attr in ("ascii", "hAnsi"): + if rfonts.get(qn(attr)) == "Calibri": + rfonts.set(qn(attr), "Times New Roman") + gen_root.insert(0, new_defaults) + gen_tree.write(styles_path, encoding="UTF-8", xml_declaration=True) + finally: + shutil.rmtree(tmpl_tmp, ignore_errors=True) + + +def patch_tables(document_path: str) -> None: + tree = ET.parse(document_path) + root = tree.getroot() + for tbl in root.findall(".//" + qn("tbl")): + tbl_pr = tbl.find(qn("tblPr")) + if tbl_pr is None: + continue + tbl_style = tbl_pr.find(qn("tblStyle")) + if tbl_style is not None: + tbl_style.set(qn("val"), "24") + tbl_layout = tbl_pr.find(qn("tblLayout")) + if tbl_layout is not None: + tbl_layout.set(qn("type"), "autofit") + tbl_w = tbl_pr.find(qn("tblW")) + if tbl_w is not None: + tbl_pr.remove(tbl_w) + existing_borders = tbl_pr.find(qn("tblBorders")) + if existing_borders is not None: + tbl_pr.remove(existing_borders) + borders = ET.SubElement(tbl_pr, qn("tblBorders")) + for side in ("top", "left", "bottom", "right", "insideH", "insideV"): + border = ET.SubElement(borders, qn(side)) + border.set(qn("val"), "single") + border.set(qn("color"), "000000") + border.set(qn("sz"), "4") + border.set(qn("space"), "0") + tree.write(document_path, encoding="UTF-8", xml_declaration=True) + + +def patch_numbering(extracted_dir: str) -> None: + numbering_path = os.path.join(extracted_dir, "word", "numbering.xml") + if not os.path.exists(numbering_path): + return + tree = ET.parse(numbering_path) + root = tree.getroot() + for absnum in root.findall(qn("abstractNum")): + absid = absnum.get(qn("abstractNumId"), "") + for lvl in absnum.findall(qn("lvl")): + bullet_level = is_bullet_level(absid, lvl) + ordered_level = is_generic_ordered_level(lvl) + if not bullet_level and not ordered_level: + continue + if bullet_level: + # Unify bullet glyphs and font so list symbols render consistently in Word. + num_fmt = lvl.find(qn("numFmt")) + if num_fmt is None: + num_fmt = ET.SubElement(lvl, qn("numFmt")) + if num_fmt.get(qn("val")) != "bullet": + num_fmt.set(qn("val"), "bullet") + lvl_text = lvl.find(qn("lvlText")) + if lvl_text is None: + lvl_text = ET.SubElement(lvl, qn("lvlText")) + lvl_text.set(qn("val"), "\uF06C") + rpr = lvl.find(qn("rPr")) + if rpr is None: + rpr = ET.SubElement(lvl, qn("rPr")) + rfonts = rpr.find(qn("rFonts")) + if rfonts is None: + rfonts = ET.SubElement(rpr, qn("rFonts")) + rfonts.set(qn("ascii"), "Wingdings") + rfonts.set(qn("hAnsi"), "Wingdings") + rfonts.set(qn("hint"), "default") + ppr = lvl.find(qn("pPr")) + if ppr is None: + ppr = ET.SubElement(lvl, qn("pPr")) + ind = ppr.find(qn("ind")) + if ind is None: + ind = ET.SubElement(ppr, qn("ind")) + try: + level = int(lvl.get(qn("ilvl"), "0")) + except ValueError: + level = 0 + if bullet_level: + left = BULLET_LIST_BASE_LEFT + (level * LIST_LEVEL_STEP) + else: + left = ORDERED_LIST_BASE_LEFT + (level * LIST_LEVEL_STEP) + ind.set(qn("left"), str(left)) + ind.set(qn("hanging"), str(LIST_HANGING)) + for attr in ("leftChars", "hangingChars", "firstLine", "firstLineChars"): + ind.attrib.pop(qn(attr), None) + tree.write(numbering_path, encoding="UTF-8", xml_declaration=True) + + +def ensure_keep_next(ppr) -> None: + keep_next = first(ppr, "w:keepNext") + if keep_next is None: + keep_next = ET.SubElement(ppr, qn("keepNext")) + keep_next.set(qn("val"), "1") + + +def set_zero_first_line_indent(ppr) -> None: + ind = first(ppr, "w:ind") + if ind is None: + ind = ET.SubElement(ppr, qn("ind")) + ind.set(qn("firstLine"), "0") + ind.set(qn("firstLineChars"), "0") + for attr in ("hanging", "hangingChars"): + ind.attrib.pop(qn(attr), None) + + +def patch_layout_constraints(document_path: str, style_name_by_id) -> None: + tree = ET.parse(document_path) + root = tree.getroot() + code_style_names = {"Source Code", "SourceCode", "代码清单"} + keep_next_style_names = {"图", "表题1-1"} + + for paragraph in root.findall(".//" + qn("p")): + ppr = first(paragraph, "w:pPr") + if ppr is None: + ppr = ET.Element(qn("pPr")) + paragraph.insert(0, ppr) + + pstyle = first(ppr, "w:pStyle") + style_name = "" + if pstyle is not None: + style_name = style_name_by_id.get(pstyle.get(qn("val")), "") + + if style_name in keep_next_style_names: + ensure_keep_next(ppr) + + if style_name in code_style_names: + ind = first(ppr, "w:ind") + if ind is not None: + for attr in ("firstLine", "firstLineChars", "hanging", "hangingChars"): + ind.attrib.pop(qn(attr), None) + if not ind.attrib: + ppr.remove(ind) + + for cell in root.findall(".//" + qn("tc")): + for paragraph in cell.findall(qn("p")): + ppr = first(paragraph, "w:pPr") + if ppr is None: + ppr = ET.Element(qn("pPr")) + paragraph.insert(0, ppr) + set_zero_first_line_indent(ppr) + + tree.write(document_path, encoding="UTF-8", xml_declaration=True) + + +def main() -> int: + if len(sys.argv) not in {4, 5}: + print( + "Usage: postprocess_template_docx.py [shortcut-template]", + file=sys.stderr, + ) + return 1 + + docx_path = sys.argv[1] + template_path = sys.argv[2] + header_text = sys.argv[3] + shortcut_template = sys.argv[4] if len(sys.argv) == 5 else None + + if not os.path.exists(docx_path): + print(f"Error: file not found: {docx_path}", file=sys.stderr) + return 1 + + temp_dir = tempfile.mkdtemp(prefix="template-docx-") + try: + with zipfile.ZipFile(docx_path) as archive: + archive.extractall(temp_dir) + + styles_path = os.path.join(temp_dir, "word", "styles.xml") + document_path = os.path.join(temp_dir, "word", "document.xml") + sanitize_relationship_prefixes(document_path) + style_name_by_id = {} + if os.path.exists(styles_path): + style_name_by_id = patch_styles(styles_path) + if os.path.exists(document_path): + strip_explicit_body_styles(document_path, style_name_by_id) + + if os.path.exists(document_path): + remove_horizontal_rules(document_path) + patch_tables(document_path) + patch_layout_constraints(document_path, style_name_by_id) + + patch_doc_defaults(temp_dir, template_path) + patch_numbering(temp_dir) + + default_header = resolve_default_header(temp_dir) + if default_header and os.path.exists(default_header): + patch_header(default_header, header_text) + inject_keymap_customizations(temp_dir, shortcut_template) + + rebuilt = docx_path + ".tmp" + with zipfile.ZipFile(rebuilt, "w", zipfile.ZIP_DEFLATED) as archive: + for root, _, files in os.walk(temp_dir): + for filename in files: + full_path = os.path.join(root, filename) + rel_path = os.path.relpath(full_path, temp_dir) + archive.write(full_path, rel_path) + shutil.move(rebuilt, docx_path) + return 0 + finally: + shutil.rmtree(temp_dir, ignore_errors=True) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/.claude/skills/markdown-to-docx/scripts/render_markdown_with_dotx.sh b/.claude/skills/markdown-to-docx/scripts/render_markdown_with_dotx.sh new file mode 100644 index 0000000..c9a236c --- /dev/null +++ b/.claude/skills/markdown-to-docx/scripts/render_markdown_with_dotx.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat >&2 <<'EOF' +Usage: + render_markdown_with_dotx.sh [book-title] [resource-root] [shortcut-template] +EOF + exit 1 +} + +[[ $# -lt 3 || $# -gt 6 ]] && usage + +SOURCE_MD="$1" +OUTPUT_DOCX="$2" +TEMPLATE_DOC="$3" +BOOK_TITLE="${4:-}" +SHORTCUT_TEMPLATE="${6:-}" +SOURCE_NAME="$(basename "$SOURCE_MD")" + +if [[ ! -f "$SOURCE_MD" ]]; then + echo "Error: source markdown not found: $SOURCE_MD" >&2 + exit 1 +fi + +if [[ ! -f "$TEMPLATE_DOC" ]]; then + echo "Error: template file not found: $TEMPLATE_DOC" >&2 + exit 1 +fi + +if ! command -v pandoc >/dev/null 2>&1; then + echo "Error: pandoc is not installed or not in PATH." >&2 + exit 1 +fi + +template_has_keymap_customizations() { + python3 - "$1" <<'PY' +import sys +import zipfile + +try: + with zipfile.ZipFile(sys.argv[1]) as zf: + raise SystemExit(0 if "word/customizations.xml" in zf.namelist() else 1) +except Exception: + raise SystemExit(1) +PY +} + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" +SOURCE_DIR="$(cd "$(dirname "$SOURCE_MD")" && pwd -P)" +SOURCE_PARENT="$(cd "$SOURCE_DIR/.." && pwd -P)" +RESOURCE_ROOT="${5:-$SOURCE_PARENT}" + +if [[ -z "$SHORTCUT_TEMPLATE" ]] && template_has_keymap_customizations "$TEMPLATE_DOC"; then + SHORTCUT_TEMPLATE="$TEMPLATE_DOC" +fi + +mkdir -p "$(dirname "$OUTPUT_DOCX")" + +TMP_DIR="$(mktemp -d)" +trap 'rm -rf "$TMP_DIR"' EXIT + +NORMALIZED_MD="$TMP_DIR/normalized.md" +TMP_RESOURCES_DIR="$TMP_DIR/resources" + +perl -0pe ' + s{^图：([^\n!]+)!\[\[([^]|]+)\|[0-9]+\]\]}{![](<$2>)\n\n图：$1}mg; + s{!\[\[([^]|]+)\|[0-9]+\]\]}{![](<$1>)}g; + s{!\[\[([^]|]+)\]\]}{![](<$1>)}g; + s{^\*([图表]：[^\n*]+)\*$}{$1}mg; + s{(?m)^(!\[[^\n]*\]\([^\n]+\))$}{\n$1\n}g; + s{(?m)^([图表]：[^\n]+)$}{\n$1\n}g; + s{(?m)^(\*\*[^\n*]+\*\*)$}{\n$1\n}g; + s{^---$}{}mg; + s{^(#{3,})\s+\d+\.\d+(?:\.\d+)?\s+(小结|可执行清单)}{$1 $2}mg; + s{\n{3,}}{\n\n}g; +' "$SOURCE_MD" > "$NORMALIZED_MD" + +python3 "$SCRIPT_DIR/render_mermaid_blocks_for_docx.py" \ + "$NORMALIZED_MD" \ + "$SOURCE_NAME" \ + "$RESOURCE_ROOT" \ + "$TMP_RESOURCES_DIR" + +# Auto-fix missing table/figure captions before conversion +python3 "$SCRIPT_DIR/validate_captions.py" fix "$NORMALIZED_MD" + +CHAPTER_TITLE="$(sed -n 's/^# //p' "$NORMALIZED_MD" | head -n 1)" +CHAPTER_PREFIX="$(printf '%s\n' "$CHAPTER_TITLE" | perl -ne 'print "$1\n" if /(第[0-9]+章)/')" + +HEADER_TEXT="${CHAPTER_TITLE:-Markdown Export}" +if [[ -n "$BOOK_TITLE" ]]; then + HEADER_TEXT="《${BOOK_TITLE}》" +fi +if [[ -n "$BOOK_TITLE" && -n "$CHAPTER_PREFIX" ]]; then + HEADER_TEXT="${HEADER_TEXT}${CHAPTER_PREFIX}" +fi + +RESOURCE_PATH="$TMP_DIR:$TMP_RESOURCES_DIR:$SOURCE_DIR:$SOURCE_PARENT:$RESOURCE_ROOT:$RESOURCE_ROOT/resources" + +pandoc "$NORMALIZED_MD" \ + -f markdown \ + -t docx \ + --reference-doc="$TEMPLATE_DOC" \ + --lua-filter="$SCRIPT_DIR/template_style_filter.lua" \ + --resource-path="$RESOURCE_PATH" \ + -o "$OUTPUT_DOCX" + +"$SCRIPT_DIR/postprocess_template_docx.py" "$OUTPUT_DOCX" "$TEMPLATE_DOC" "$HEADER_TEXT" "$SHORTCUT_TEMPLATE" + +printf 'OK\t%s\n' "$OUTPUT_DOCX" diff --git a/.claude/skills/markdown-to-docx/scripts/render_mermaid_blocks_for_docx.py b/.claude/skills/markdown-to-docx/scripts/render_mermaid_blocks_for_docx.py new file mode 100644 index 0000000..cc31fd7 --- /dev/null +++ b/.claude/skills/markdown-to-docx/scripts/render_mermaid_blocks_for_docx.py @@ -0,0 +1,122 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import re +import subprocess +import tempfile +from pathlib import Path + + +MERMAID_FENCE_RE = re.compile(r"^```\s*mermaid\b", re.IGNORECASE) + + +def slugify(text: str) -> str: + slug = re.sub(r"[^0-9A-Za-z\u4e00-\u9fff._-]+", "-", text) + slug = slug.strip("-._") + return slug or "diagram" + + +def render_mermaid( + *, + code: str, + out_path: Path, + theme: str = "neutral", + width: int = 1200, + height: int = 900, + scale: float = 2.0, +) -> None: + out_path.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix="docx-mermaid-") as tmpdir: + tmp_mmd = Path(tmpdir) / "diagram.mmd" + tmp_mmd.write_text(code, encoding="utf-8") + cmd = [ + "npx", + "-y", + "@mermaid-js/mermaid-cli", + "-i", + str(tmp_mmd), + "-o", + str(out_path), + "--outputFormat", + "png", + "--theme", + theme, + "--backgroundColor", + "white", + "--width", + str(width), + "--height", + str(height), + "--scale", + str(scale), + "-q", + ] + proc = subprocess.run(cmd, capture_output=True, text=True) + if proc.returncode != 0: + stderr = (proc.stderr or "").strip() + stdout = (proc.stdout or "").strip() + detail = stderr or stdout or str(proc.returncode) + raise RuntimeError(f"Mermaid render failed for {out_path.name}: {detail}") + + +def process_markdown(md_path: Path, source_name: str, temp_resources_dir: Path) -> int: + lines = md_path.read_text(encoding="utf-8").splitlines() + out_lines: list[str] = [] + mermaid_count = 0 + rendered = 0 + temp_resources_dir.mkdir(parents=True, exist_ok=True) + + i = 0 + while i < len(lines): + line = lines[i] + if MERMAID_FENCE_RE.match(line.strip()): + j = i + 1 + code_lines: list[str] = [] + while j < len(lines) and not lines[j].strip().startswith("```"): + code_lines.append(lines[j]) + j += 1 + if j >= len(lines): + raise RuntimeError(f"Unclosed mermaid block in {md_path}") + + mermaid_count += 1 + out_name = f"{slugify(Path(source_name).stem)}-mermaid-{mermaid_count:02d}.png" + out_path = temp_resources_dir / out_name + render_mermaid(code="\n".join(code_lines).strip() + "\n", out_path=out_path) + + out_lines.append(f"![](resources/{out_name})") + rendered += 1 + i = j + 1 + continue + + out_lines.append(line) + i += 1 + + md_path.write_text("\n".join(out_lines) + "\n", encoding="utf-8") + return rendered + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Render Mermaid code blocks in Markdown to PNG images for DOCX export." + ) + parser.add_argument("markdown_path", help="Normalized Markdown file to rewrite in place.") + parser.add_argument("source_name", help="Original Markdown basename, used for output names.") + parser.add_argument( + "resource_root", + help="Compatibility argument; existing image lookup is handled by the calling script.", + ) + parser.add_argument("temp_resources_dir", help="Temporary resources directory for generated diagrams.") + args = parser.parse_args() + + rendered = process_markdown( + md_path=Path(args.markdown_path), + source_name=args.source_name, + temp_resources_dir=Path(args.temp_resources_dir), + ) + print(f"MERMAID_OK {Path(args.markdown_path)} rendered={rendered}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/.claude/skills/markdown-to-docx/scripts/template_style_filter.lua b/.claude/skills/markdown-to-docx/scripts/template_style_filter.lua new file mode 100644 index 0000000..4aea64e --- /dev/null +++ b/.claude/skills/markdown-to-docx/scripts/template_style_filter.lua @@ -0,0 +1,170 @@ +local stringify = pandoc.utils.stringify + +local function trim(text) + return (text:gsub("^%s+", ""):gsub("%s+$", "")) +end + +local function wrap_para(style_name, para) + return pandoc.Div({ para }, pandoc.Attr("", {}, { { "custom-style", style_name } })) +end + +local function para_from_markdown(text) + local doc = pandoc.read(text, "markdown") + if #doc.blocks > 0 and doc.blocks[1].t == "Para" then + return doc.blocks[1] + end + return pandoc.Para({ pandoc.Str(text) }) +end + +local function image_only_para(block) + return block.t == "Para" and #block.content == 1 and block.content[1].t == "Image" +end + +local function figure_to_image_para(block) + if block.t ~= "Figure" then + return nil + end + + local first = block.content and block.content[1] or nil + if not first then + return nil + end + + if (first.t == "Para" or first.t == "Plain") and #first.content == 1 and first.content[1].t == "Image" then + return pandoc.Para({ first.content[1] }) + end + + return nil +end + +local function normalize_serial(num) + return (num:gsub("%.", "-")) +end + +local function ensure_sentence(text) + if text == "" then + return text + end + + if text:match("[。！？%.%!%?]$") then + return text + end + + return text .. "。" +end + +local function parse_caption(text, kind) + -- Try with fullwidth colon first, then without. + -- Cannot use ：? because Lua ? applies to a single byte, not a multi-byte char. + local num, rest = text:match("^" .. kind .. "：%s*([0-9]+[%.%-][0-9]+)%s+(.+)$") + if not num then + num, rest = text:match("^" .. kind .. "%s*([0-9]+[%.%-][0-9]+)%s+(.+)$") + end + if not num then + return nil + end + + rest = trim(rest) + local title, description = rest:match("^(.-)。(.*)$") + if not title or title == "" then + title = rest + description = "" + end + + return { + number = normalize_serial(num), + title = trim(title), + description = trim(description or ""), + label = kind .. normalize_serial(num) .. " " .. trim(title), + } +end + +local function build_figure_explanation(fig) + if fig.description == "" then + return nil + end + + local description = ensure_sentence(fig.description) + if description:match("^如图") then + return para_from_markdown(description) + end + + return para_from_markdown("如图" .. fig.number .. "所示，" .. description) +end + +local function is_note_text(text) + return text:match("^注：") + or text:match("^注意：") + or text:match("^关键注意：") + or text:match("^⚠️%s*关键注意：") +end + +local function transform_para(block) + local text = trim(stringify(block)) + + if image_only_para(block) then + return { wrap_para("图", block) } + end + + local fig = parse_caption(text, "图") + if fig then + return { wrap_para("图题", para_from_markdown(fig.label)) } + end + + local tbl = parse_caption(text, "表") + if tbl then + return { wrap_para("表题1-1", para_from_markdown(tbl.label)) } + end + + if is_note_text(text) then + return { wrap_para("注意", para_from_markdown(text)) } + end + + return { block } +end + +function Blocks(blocks) + local out = {} + local i = 1 + + while i <= #blocks do + local block = blocks[i] + local next_block = blocks[i + 1] + local image_block = nil + + if image_only_para(block) then + image_block = block + else + image_block = figure_to_image_para(block) + end + + if image_block and next_block and next_block.t == "Para" then + local fig = parse_caption(trim(stringify(next_block)), "图") + if fig then + local explanation = build_figure_explanation(fig) + if explanation then + table.insert(out, explanation) + end + table.insert(out, wrap_para("图", image_block)) + table.insert(out, wrap_para("图题", para_from_markdown(fig.label))) + i = i + 2 + else + table.insert(out, wrap_para("图", image_block)) + i = i + 1 + end + else + if image_block then + table.insert(out, wrap_para("图", image_block)) + elseif block.t == "Para" then + for _, transformed in ipairs(transform_para(block)) do + table.insert(out, transformed) + end + else + table.insert(out, block) + end + i = i + 1 + end + end + + return out +end diff --git a/.claude/skills/markdown-to-docx/scripts/validate_captions.py b/.claude/skills/markdown-to-docx/scripts/validate_captions.py new file mode 100644 index 0000000..e6bdf12 --- /dev/null +++ b/.claude/skills/markdown-to-docx/scripts/validate_captions.py @@ -0,0 +1,597 @@ +#!/usr/bin/env python3 +"""Validate and auto-fix table/figure captions in markdown or docx. + +Modes: + validate_captions.py pre — check only + validate_captions.py fix — auto-insert missing captions, write in-place + validate_captions.py post — check generated docx +""" +from __future__ import annotations + +import re +import sys +import zipfile +import xml.etree.ElementTree as ET + +W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main" +EXPECTED_ORDERED_LEFT = "800" +EXPECTED_BULLET_LEFT = "840" +EXPECTED_BULLET_HANGING = "420" +EXPECTED_LIST_STEP = "420" + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def _extract_chapter(lines: list[str]) -> str | None: + for line in lines: + m = re.match(r"^#\s+第(\d+)章", line) + if m: + return m.group(1) + return None + + +def _qn(tag: str) -> str: + return f"{{{W}}}{tag}" + + +TABLE_CAPTION_RE = re.compile(r"^\*?表[：:]?\s*(\d+)[-.]\s*(\d+)\s+\S.*\*?$") +FIGURE_CAPTION_RE = re.compile(r"^\*?图\s*(\d+)[-.]\s*(\d+)\s+\S.*\*?$") +FIGURE_CAPTION_ALT_RE = re.compile(r"^\*?图[：:]?\s*(\d+)[-.]\s*(\d+)\s+\S.*\*?$") +GENERIC_ORDERED_NUMFMTS = {"decimal", "lowerLetter", "upperLetter", "lowerRoman", "upperRoman"} +SPECIAL_NUMBERING_PREFIXES = ("表", "图", "代码清单") + + +def _is_generic_ordered_level(lvl: ET.Element) -> bool: + num_fmt = lvl.find(f"{{{W}}}numFmt") + if num_fmt is None or num_fmt.get(f"{{{W}}}val") not in GENERIC_ORDERED_NUMFMTS: + return False + lvl_text = lvl.find(f"{{{W}}}lvlText") + if lvl_text is None: + return False + value = lvl_text.get(f"{{{W}}}val") or "" + if "%" not in value: + return False + if any(prefix in value for prefix in SPECIAL_NUMBERING_PREFIXES): + return False + return True + + +def _find_table_blocks(lines: list[str]) -> list[tuple[int, int, str]]: + """Return (start, end, header_line) for each contiguous table block.""" + blocks: list[tuple[int, int, str]] = [] + in_table = False + table_start = 0 + header = "" + for i, line in enumerate(lines): + stripped = line.strip() + if stripped.startswith("|") and "|" in stripped[1:]: + if not in_table: + in_table = True + table_start = i + header = stripped + else: + if in_table: + blocks.append((table_start, i - 1, header)) + in_table = False + if in_table: + blocks.append((table_start, len(lines) - 1, header)) + return blocks + + +def _find_figure_items(lines: list[str]) -> list[tuple[int, int, str]]: + """Return (start_line, end_line, type) for images and mermaid blocks.""" + items: list[tuple[int, int, str]] = [] + for i, line in enumerate(lines): + if re.match(r"^!\[", line.strip()): + items.append((i, i, "image")) + in_code = False + code_lang = "" + code_start = 0 + for i, line in enumerate(lines): + stripped = line.strip() + if stripped.startswith("```"): + if not in_code: + in_code = True + code_lang = stripped[3:].strip().lower() + code_start = i + else: + if code_lang == "mermaid": + items.append((code_start, i, "mermaid")) + in_code = False + code_lang = "" + items.sort(key=lambda x: x[0]) + return items + + +def _has_caption_before(lines: list[str], start: int, pattern: re.Pattern) -> tuple[bool, tuple[str, str] | None]: + for look_back in range(1, 4): + idx = start - look_back + if idx < 0: + break + prev = lines[idx].strip() + if prev == "": + continue + m = pattern.match(prev) + if m: + return True, (m.group(1), m.group(2)) + return False, None + return False, None + + +def _has_caption_after(lines: list[str], search_start: int, pattern: re.Pattern, alt_pattern: re.Pattern | None = None) -> tuple[bool, tuple[str, str] | None]: + for idx in range(search_start, min(search_start + 4, len(lines))): + nxt = lines[idx].strip() + if nxt == "": + continue + m = pattern.match(nxt) + if m: + return True, (m.group(1), m.group(2)) + if alt_pattern: + m2 = alt_pattern.match(nxt) + if m2: + return True, (m2.group(1), m2.group(2)) + return False, None + return False, None + + +def _derive_table_title(lines: list[str], start: int, header: str) -> str: + """Derive a short table title from the header row columns.""" + # Extract column names from header row: | Col1 | Col2 | ... + cols = [c.strip() for c in header.split("|") if c.strip()] + if len(cols) >= 2: + return "、".join(cols[:3]) + ("等" if len(cols) > 3 else "") + # Fallback: use preceding paragraph + for look_back in range(1, 5): + idx = start - look_back + if idx < 0: + break + prev = lines[idx].strip() + if prev and not prev.startswith("|") and not prev.startswith("#"): + # Truncate to first clause + for sep in ("：", "。", "，", "："): + if sep in prev: + prev = prev[: prev.index(sep)] + break + if len(prev) > 30: + prev = prev[:30] + return prev + return "数据总览" + + +def _derive_figure_title(lines: list[str], start: int, end: int, fig_type: str) -> str: + """Derive a short figure title from surrounding context.""" + # Look at line before + for look_back in range(1, 5): + idx = start - look_back + if idx < 0: + break + prev = lines[idx].strip() + if prev and not prev.startswith("```") and not prev.startswith("#"): + # Truncate + for sep in ("：", "。", "，"): + if sep in prev: + prev = prev[: prev.index(sep)] + break + if len(prev) > 30: + prev = prev[:30] + return prev + return "系统架构图" + + +# --------------------------------------------------------------------------- +# Pre-check +# --------------------------------------------------------------------------- + +def pre_check(md_path: str) -> list[str]: + with open(md_path, encoding="utf-8") as f: + lines = f.readlines() + + issues: list[str] = [] + chapter_num = _extract_chapter(lines) or "?" + if chapter_num == "?": + issues.append("WARN: Cannot extract chapter number from H1 heading") + + table_blocks = _find_table_blocks(lines) + for idx, (start, end, header) in enumerate(table_blocks, 1): + found, nums = _has_caption_before(lines, start, TABLE_CAPTION_RE) + if not found: + issues.append( + f"ERROR: Table at line {start + 1} missing caption. " + f"Expected: 表{chapter_num}-{idx} " + ) + else: + if nums[0] != chapter_num: + issues.append(f"WARN: Table at line {start + 1}: chapter {nums[0]}, expected {chapter_num}") + if nums[1] != str(idx): + issues.append(f"WARN: Table at line {start + 1}: 表{nums[0]}-{nums[1]}, expected seq {idx}") + + figure_items = _find_figure_items(lines) + for idx, (start, end, fig_type) in enumerate(figure_items, 1): + search_start = end + 1 + found, nums = _has_caption_after(lines, search_start, FIGURE_CAPTION_RE, FIGURE_CAPTION_ALT_RE) + if not found: + found, nums = _has_caption_before(lines, start, FIGURE_CAPTION_RE) + if not found: + issues.append( + f"ERROR: {fig_type.capitalize()} at line {start + 1} missing caption. " + f"Expected: 图{chapter_num}-{idx} <title>" + ) + else: + if nums[0] != chapter_num: + issues.append(f"WARN: Figure near line {start + 1}: chapter {nums[0]}, expected {chapter_num}") + if nums[1] != str(idx): + issues.append(f"WARN: Figure near line {start + 1}: 图{nums[0]}-{nums[1]}, expected seq {idx}") + + if not issues: + issues.append( + f"OK: {len(table_blocks)} tables, {len(figure_items)} figures — " + f"all captions present and correctly numbered" + ) + return issues + + +# --------------------------------------------------------------------------- +# Auto-fix +# --------------------------------------------------------------------------- + +def auto_fix(md_path: str) -> list[str]: + """Insert missing captions into markdown. Returns log of changes.""" + with open(md_path, encoding="utf-8") as f: + lines = f.readlines() + + chapter_num = _extract_chapter(lines) or "0" + log: list[str] = [] + + # We need to process from bottom to top so that line insertions + # don't shift indices of items not yet processed. + + # Collect all items that need fixing + insertions: list[tuple[int, str]] = [] # (line_index, caption_text) + + # --- Tables: caption goes BEFORE the table --- + table_blocks = _find_table_blocks(lines) + for idx, (start, end, header) in enumerate(table_blocks, 1): + found, _ = _has_caption_before(lines, start, TABLE_CAPTION_RE) + if not found: + title = _derive_table_title(lines, start, header) + caption = f"表{chapter_num}-{idx} {title}" + insertions.append((start, caption)) + log.append(f"FIXED: Inserted '{caption}' before line {start + 1}") + + # --- Figures: caption goes AFTER the figure --- + figure_items = _find_figure_items(lines) + for idx, (start, end, fig_type) in enumerate(figure_items, 1): + search_start = end + 1 + found, _ = _has_caption_after(lines, search_start, FIGURE_CAPTION_RE, FIGURE_CAPTION_ALT_RE) + if not found: + found, _ = _has_caption_before(lines, start, FIGURE_CAPTION_RE) + if not found: + title = _derive_figure_title(lines, start, end, fig_type) + caption = f"图{chapter_num}-{idx} {title}" + insert_at = end + 1 + insertions.append((insert_at, caption)) + log.append(f"FIXED: Inserted '{caption}' after line {end + 1}") + + if not insertions: + log.append("OK: No missing captions to fix") + return log + + # Sort by line index descending so insertions don't shift each other + insertions.sort(key=lambda x: x[0], reverse=True) + + for insert_at, caption in insertions: + # Insert: blank line + caption + blank line + new_lines = ["\n", caption + "\n", "\n"] + lines[insert_at:insert_at] = new_lines + + with open(md_path, "w", encoding="utf-8") as f: + f.writelines(lines) + + log.append(f"DONE: {len(insertions)} captions inserted into {md_path}") + return log + + +# --------------------------------------------------------------------------- +# Post-check +# --------------------------------------------------------------------------- + +def post_check(docx_path: str) -> list[str]: + VML = "urn:schemas-microsoft-com:vml" + O = "urn:schemas-microsoft-com:office:office" + + issues: list[str] = [] + + with zipfile.ZipFile(docx_path) as z: + doc = ET.fromstring(z.read("word/document.xml")) + styles = ET.fromstring(z.read("word/styles.xml")) + numbering = ET.fromstring(z.read("word/numbering.xml")) + + style_name_by_id: dict[str, str] = {} + style_by_name: dict[str, ET.Element] = {} + for style in styles.findall(f"{{{W}}}style"): + sid = style.get(f"{{{W}}}styleId") + name_el = style.find(f"{{{W}}}name") + name = name_el.get(f"{{{W}}}val") if name_el is not None else sid + if sid: + style_name_by_id[sid] = name + if name: + style_by_name[name] = style + + # 1. Compact style + compact = sum( + 1 + for p in doc.findall(f".//{{{W}}}p") + if (ppr := p.find(f"{{{W}}}pPr")) is not None + and (ps := ppr.find(f"{{{W}}}pStyle")) is not None + and ps.get(f"{{{W}}}val") == "Compact" + ) + if compact > 0: + issues.append(f"ERROR: {compact} paragraphs with undefined 'Compact' style") + + # 2. VML horizontal rules + hr = sum( + 1 + for r in doc.findall(f".//{{{VML}}}rect") + if r.get(f"{{{O}}}hr") == "t" + ) + if hr > 0: + issues.append(f"ERROR: {hr} VML horizontal rules (ugly dividers)") + + # 3. Fonts + defaults = styles.find(f"{{{W}}}docDefaults") + if defaults is not None: + rf = defaults.find(f".//{{{W}}}rFonts") + if rf is not None: + ascii_f = rf.get(f"{{{W}}}ascii", "?") + ea_f = rf.get(f"{{{W}}}eastAsia", "?") + if ascii_f == "Calibri": + issues.append("WARN: docDefaults ascii font is Calibri, expected Times New Roman") + if ea_f != "宋体": + issues.append(f"WARN: docDefaults eastAsia font is {ea_f}, expected 宋体") + + # 4. First-line indent (Normal style or docDefaults) + has_indent = False + for style in styles.findall(f"{{{W}}}style"): + name_el = style.find(f"{{{W}}}name") + if name_el is not None and name_el.get(f"{{{W}}}val") == "Normal": + ppr = style.find(f"{{{W}}}pPr") + if ppr is not None: + ind = ppr.find(f"{{{W}}}ind") + if ind is not None and ind.get(f"{{{W}}}firstLine"): + has_indent = True + break + if not has_indent and defaults is not None: + ppr_d = defaults.find(f".//{{{W}}}pPrDefault") + if ppr_d is not None: + ppr = ppr_d.find(f"{{{W}}}pPr") + if ppr is not None: + ind = ppr.find(f"{{{W}}}ind") + if ind is not None and ind.get(f"{{{W}}}firstLine"): + has_indent = True + if not has_indent: + issues.append("WARN: No first-line indent in Normal style or docDefaults") + + # 5. Table borders + tables = doc.findall(f".//{{{W}}}tbl") + tables_no_borders = 0 + for tbl in tables: + tpr = tbl.find(f"{{{W}}}tblPr") + has_tbl_borders = tpr is not None and tpr.find(f"{{{W}}}tblBorders") is not None + has_cell_borders = any( + tc.find(f"{{{W}}}tcPr") is not None + and tc.find(f"{{{W}}}tcPr").find(f"{{{W}}}tcBorders") is not None + for tc in tbl.findall(f".//{{{W}}}tc") + ) + if not has_tbl_borders and not has_cell_borders: + tables_no_borders += 1 + if tables_no_borders > 0: + issues.append(f"ERROR: {tables_no_borders}/{len(tables)} tables missing borders") + + # 6. Keep-with-next for figure images and table captions + image_keep_next_missing = 0 + table_caption_keep_next_missing = 0 + for p in doc.findall(f".//{{{W}}}p"): + ppr = p.find(f"{{{W}}}pPr") + if ppr is None: + continue + ps = ppr.find(f"{{{W}}}pStyle") + sid = ps.get(f"{{{W}}}val") if ps is not None else None + style_name = style_name_by_id.get(sid, sid or "") + has_keep_next = ppr.find(f"{{{W}}}keepNext") is not None + if style_name == "图" and not has_keep_next: + image_keep_next_missing += 1 + if style_name == "表题1-1" and not has_keep_next: + table_caption_keep_next_missing += 1 + if image_keep_next_missing > 0: + issues.append(f"ERROR: {image_keep_next_missing} image paragraphs missing keep-with-next") + if table_caption_keep_next_missing > 0: + issues.append(f"ERROR: {table_caption_keep_next_missing} table captions missing keep-with-next") + + # 7. Code block first-line indent + code_style = None + for style_name in ("Source Code", "SourceCode", "代码清单"): + candidate = style_by_name.get(style_name) + if candidate is not None: + code_style = candidate + break + if code_style is not None: + ppr = code_style.find(f"{{{W}}}pPr") + if ppr is not None: + ind = ppr.find(f"{{{W}}}ind") + if ind is None: + issues.append("ERROR: Code block style is missing explicit zero first-line indent override") + else: + if ind.get(f"{{{W}}}firstLine") != "0" or ind.get(f"{{{W}}}firstLineChars") != "0": + issues.append("ERROR: Code block style still has first-line indentation") + if ind.get(f"{{{W}}}hanging") or ind.get(f"{{{W}}}hangingChars"): + issues.append("ERROR: Code block style still has hanging indentation") + + # 8. List indentation should align with Chinese body-text first-line indent + bullet_indent_issues = 0 + ordered_indent_issues = 0 + num_to_abs: dict[str, str] = {} + abstract_lookup: dict[str, ET.Element] = {} + for num in numbering.findall(f"{{{W}}}num"): + num_id = num.get(f"{{{W}}}numId") + abs_el = num.find(f"{{{W}}}abstractNumId") + abs_id = abs_el.get(f"{{{W}}}val") if abs_el is not None else None + if num_id and abs_id: + num_to_abs[num_id] = abs_id + for absnum in numbering.findall(f"{{{W}}}abstractNum"): + abs_id = absnum.get(f"{{{W}}}abstractNumId") + if abs_id: + abstract_lookup[abs_id] = absnum + + for p in doc.findall(f".//{{{W}}}p"): + ppr = p.find(f"{{{W}}}pPr") + if ppr is None: + continue + numpr = ppr.find(f"{{{W}}}numPr") + if numpr is None: + continue + num_id_el = numpr.find(f"{{{W}}}numId") + ilvl_el = numpr.find(f"{{{W}}}ilvl") + if num_id_el is None: + continue + abs_id = num_to_abs.get(num_id_el.get(f"{{{W}}}val", "")) + if not abs_id: + continue + absnum = abstract_lookup.get(abs_id) + if absnum is None: + continue + ilvl = ilvl_el.get(f"{{{W}}}val", "0") if ilvl_el is not None else "0" + try: + ilvl_num = int(ilvl) + except ValueError: + ilvl_num = 0 + lvl = absnum.find(f"{{{W}}}lvl[@{{{W}}}ilvl='{ilvl}']") + if lvl is None: + continue + ind = lvl.find(f"{{{W}}}pPr/{{{W}}}ind") + num_fmt = lvl.find(f"{{{W}}}numFmt") + is_bullet = num_fmt is not None and num_fmt.get(f"{{{W}}}val") == "bullet" + is_ordered = _is_generic_ordered_level(lvl) + if not is_bullet and not is_ordered: + continue + if ind is None: + if is_bullet: + bullet_indent_issues += 1 + else: + ordered_indent_issues += 1 + continue + if is_bullet: + expected_left = str(int(EXPECTED_BULLET_LEFT) + ilvl_num * int(EXPECTED_LIST_STEP)) + if ( + ind.get(f"{{{W}}}left") != expected_left + or ind.get(f"{{{W}}}hanging") != EXPECTED_BULLET_HANGING + ): + bullet_indent_issues += 1 + else: + expected_left = str(int(EXPECTED_ORDERED_LEFT) + ilvl_num * int(EXPECTED_LIST_STEP)) + if ( + ind.get(f"{{{W}}}left") != expected_left + or ind.get(f"{{{W}}}hanging") != EXPECTED_BULLET_HANGING + ): + ordered_indent_issues += 1 + if bullet_indent_issues > 0: + issues.append( + "ERROR: " + f"{bullet_indent_issues} bullet list paragraphs still use over-indented list geometry " + f"(expected left={EXPECTED_BULLET_LEFT}, hanging={EXPECTED_BULLET_HANGING})" + ) + if ordered_indent_issues > 0: + issues.append( + "ERROR: " + f"{ordered_indent_issues} ordered list paragraphs still use over-indented list geometry " + f"(expected left={EXPECTED_ORDERED_LEFT}, hanging={EXPECTED_BULLET_HANGING})" + ) + + # 9. Table cell paragraphs should not inherit body first-line indent + table_cell_indent_issues = 0 + for tc in doc.findall(f".//{{{W}}}tc"): + for p in tc.findall(f"{{{W}}}p"): + ppr = p.find(f"{{{W}}}pPr") + ind = ppr.find(f"{{{W}}}ind") if ppr is not None else None + if ind is None: + table_cell_indent_issues += 1 + continue + if ind.get(f"{{{W}}}firstLine") not in ("0", None) or ind.get(f"{{{W}}}firstLineChars") not in ("0", None): + table_cell_indent_issues += 1 + if table_cell_indent_issues > 0: + issues.append(f"ERROR: {table_cell_indent_issues} table cell paragraphs still inherit first-line indentation") + + # 10. Table and figure captions + all_texts = [] + for p in doc.findall(f".//{{{W}}}p"): + text = "".join(t.text or "" for t in p.findall(f".//{{{W}}}t")) + if text.strip(): + all_texts.append(text.strip()) + + table_captions = [t for t in all_texts if re.match(r"^表\s*\d+[-.]\d+\s+\S", t)] + figure_captions = [t for t in all_texts if re.match(r"^图\s*\d+[-.]\d+\s+\S", t)] + + if len(tables) > 0 and len(table_captions) == 0: + issues.append(f"ERROR: {len(tables)} tables found but 0 table captions (表X-Y)") + elif len(tables) > len(table_captions): + issues.append(f"WARN: {len(tables)} tables but only {len(table_captions)} table captions") + + for kind, captions in [("表", table_captions), ("图", figure_captions)]: + nums = [] + for cap in captions: + m = re.match(rf"^{kind}\s*(\d+)[-.]\s*(\d+)", cap) + if m: + nums.append((int(m.group(1)), int(m.group(2)))) + if nums: + chapter = nums[0][0] + for i, (ch, seq) in enumerate(nums, 1): + if ch != chapter: + issues.append(f"WARN: {kind} caption #{i} has chapter {ch}, expected {chapter}") + if seq != i: + issues.append(f"WARN: {kind} caption #{i} is {kind}{ch}-{seq}, expected {kind}{chapter}-{i}") + + if not issues: + issues.append( + f"OK: {len(tables)} tables, {len(table_captions)} table captions, " + f"{len(figure_captions)} figure captions — all checks passed" + ) + return issues + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main() -> int: + if len(sys.argv) != 3 or sys.argv[1] not in ("pre", "post", "fix"): + print( + "Usage:\n" + " validate_captions.py pre <source.md> — check only\n" + " validate_captions.py fix <source.md> — auto-insert missing captions\n" + " validate_captions.py post <output.docx> — check generated docx", + file=sys.stderr, + ) + return 1 + + mode = sys.argv[1] + path = sys.argv[2] + + if mode == "pre": + results = pre_check(path) + elif mode == "fix": + results = auto_fix(path) + else: + results = post_check(path) + + has_error = False + for line in results: + if line.startswith("ERROR"): + has_error = True + print(line) + + return 1 if has_error else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/.gitignore b/.gitignore index 41d89d7..20a59ad 100644 --- a/.gitignore +++ b/.gitignore @@ -1,11 +1,22 @@ -# Editor / Tool configs -.DS_Store -.claude/ -.claudian/ -.obsidian/ -.pandoc/ +# === 归档 === +Archive/ -# Original source (kept for reference, not tracked) +# === 生成输出 === +output/ + +# === 大型数据 === +.pandoc/zotero-library-1.json + +# === 工具本地配置 === +.obsidian/ +.claude/settings.local.json +.claude/agents/ +.claude/commands/ +.claude/worktrees/ +.claudian/ + +# === 系统文件 === +.DS_Store *.docx -learning_markdown.md -pics/ +*.zip +~$* diff --git a/.pandoc/apa.csl b/.pandoc/apa.csl new file mode 100644 index 0000000..9bc45ef --- /dev/null +++ b/.pandoc/apa.csl @@ -0,0 +1,2273 @@ +<?xml version="1.0" encoding="utf-8"?> +<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" demote-non-dropping-particle="never" initialize-with=". " names-delimiter=", " page-range-format="expanded" version="1.0"> + <!-- This file was generated by the Style Variant Builder <https://github.com/citation-style-language/style-variant-builder>. To contribute changes, modify the template and regenerate variants. --> + <info> + <title>APA Style 7th edition + Publication Manual of the American Psychological Association, with Bluebook + http://www.zotero.org/styles/apa + + + + + + Brenton M. Wiernik + zotero@wiernik.org + https://orcid.org/0000-0001-9560-6336 + + + Andrew Dunning + https://orcid.org/0000-0003-0464-5036 + + + + + + + + + + + Author-date system of the Publication Manual of the American Psychological Association (2020) + 2026-02-07T00:00:00+00:00 + This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License + + + + C.E. + B.C.E. + ca. + + guest expert + guest experts + + illus. + + interviewer + interviewers + + Pub. L. + unpublished manuscript + recorded by + online post + review of the + review of + computer software + + suppl. + suppls. + + + + + + et al. + + + + + et al. + + + + + de + + + + + + éd. + éds. + + + + + + et al. + + + + + et al. + + + + + et al. + + + + + i in. + + + + + et al. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/.pandoc/locales-en-US.xml b/.pandoc/locales-en-US.xml new file mode 100644 index 0000000..0d74676 --- /dev/null +++ b/.pandoc/locales-en-US.xml @@ -0,0 +1,864 @@ + + + + + + + Andrew Dunning + https://orcid.org/0000-0003-0464-5036 + + + Sebastian Karcher + https://orcid.org/0000-0001-8249-7388 + + + Rintze M. Zelle + https://orcid.org/0000-0003-1779-8883 + + + Denis Meier + + + Brenton M. Wiernik + https://orcid.org/0000-0001-9560-6336 + + This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License + 2026-01-10T00:00:00+00:00 + + + + + + + + + + + + + + + accessed + advance online publication + album + and + and others + anonymous + at + audio recording + available at + by + circa + cited + et al. + film + forthcoming + from + henceforth + ibid. + in + in press + internet + letter + loc. cit. + no date + no place + no publisher + on + online + op. cit. + original work published + personal communication + podcast + podcast episode + preprint + presented at the + radio broadcast + radio series + radio series episode + + reference + references + + retrieved + review of + scale + special issue + special section + television broadcast + television series + television series episode + video + working paper + + + + adv. online pub. + anon. + au. rec. + avail. at + c. + + cit. + flm. + fr. + let. + n.d. + n.p. + n.p. + orig. pub. + pers. comm. + podcast ep. + radio bdcst. + radio ser. + radio ser. ep. + + ref. + refs. + + rtvd. + rev. of + sc. + spec. iss. + spec. sec. + TV bdcst. + TV ser. + TV ser. ep. + vid. + wkg. paper + + + & + @ + + + preprint + journal article + magazine article + newspaper article + bill + + broadcast + + classical work + archival collection + dataset + document + entry + dictionary entry + encyclopedia entry + event + + graphic + hearing + interview + legal case + legislation + manuscript + map + video recording + musical score + pamphlet + conference paper + patent + performance + periodical + personal communication + post + blog post + regulation + report + review + book review + software + audio recording + presentation + standard + thesis + treaty + webpage + + + + jour. art. + mag. art. + newspaper art. + bdcst. + + + class. wk. + arch. coll. + doc. + dict. entry + ency. entry + + gr. + int. + leg. case + legis. + + MS + MSS + + vid. rec. + mus. score + pam. + conf. paper + pat. + prfm. + pers. comm. + reg. + rep. + rev. + bk. rev. + sftw. + au. rec. + std. + thes. + webpg. + + + + testimony of + review of + review of the book + + + + test. of + rev. of + rev. of the bk. + + + AD + BC + BCE + CE + + + “ + ” + ‘ + ’ + – + : + , + ; + + + th + st + nd + rd + th + th + th + + + first + second + third + fourth + fifth + sixth + seventh + eighth + ninth + tenth + + + + act + acts + + + appendix + appendices + + + article + articles + + + book + books + + + canon + canons + + + chapter + chapters + + + column + columns + + + location + locations + + + equation + equations + + + figure + figures + + + folio + folios + + + issue + issues + + + line + lines + + + note + notes + + + opus + opera + + + page + pages + + + paragraph + paragraphs + + + part + parts + + + rule + rules + + + scene + scenes + + + section + sections + + + sub verbo + sub verbis + + + supplement + supplements + + + table + tables + + + + + title + titles + + + verse + verses + + + volume + volumes + + + + + + app. + apps. + + + art. + arts. + + + bk. + bks. + + + + can. + cann. + + + chap. + chaps. + + + col. + cols. + + + loc. + locs. + + + eq. + eqq. + + + fig. + figs. + + + fol. + fols. + + + no. + nos. + + + l. + ll. + + + n. + nn. + + + op. + opp. + + + p. + pp. + + + para. + paras. + + + pt. + pts. + + + + r. + rr. + + + sc. + scs. + + + sec. + secs. + + + s.v. + s.vv. + + + supp. + supps. + + + + tbl. + tbls. + + + + tit. + titt. + + + v. + vv. + + + vol. + vols. + + + + + + c. + cc. + + + + ¶¶ + + + § + §§ + + + + + chapter + chapters + + + citation + citations + + + number + numbers + + + edition + editions + + + note + notes + + + number + numbers + + + page + pages + + + volume + volumes + + + page + pages + + + printing + printings + + + version + versions + + + + + chap. + chaps. + + + cit. + cits. + + + no. + nos. + + + ed. + eds. + + + n. + nn. + + + no. + nos. + + + p. + pp. + + + vol. + vols. + + + p. + pp. + + + + ptg. + ptgs. + + v. + + + + + chair + chairs + + + editor + editors + + + compiler + compilers + + + + + contributor + contributors + + + curator + curators + + + director + directors + + + editor + editors + + + editor & translator + editors & translators + + + editor & translator + editors & translators + + + editor + editors + + + executive producer + executive producers + + + guest + guests + + + host + hosts + + + illustrator + illustrators + + + + narrator + narrators + + + organizer + organizers + + + + performer + performers + + + producer + producers + + + + + writer + writers + + + series creator + series creators + + + translator + translators + + + + + + ed. + eds. + + + comp. + comps. + + + + contrib. + contribs. + + + + cur. + curs. + + + dir. + dirs. + + + ed. + eds. + + + ed. & trans. + eds. & trans. + + + ed. & trans. + eds. & trans. + + + ed. + eds. + + + + exec. prod. + exec. prods. + + + ill. + ills. + + + + narr. + narrs. + + + + org. + orgs. + + + + perf. + perfs. + + + + prod. + prods. + + + + wrtr. + wrtrs. + + + ser. creator + ser. creators + + trans. + + + chaired by + edited by + compiled by + composed by + by + with + curated by + directed by + edited by + edited & translated by + edited & translated by + edited by + executive produced by + + with guest + with guests + + hosted by + illustrated by + interview by + narrated by + organized by + by + performed by + produced by + to + by + written by + created by + translated by + + + + ed. by + comp. by + comp. by + cur. by + dir. by + ed. by + ed. & trans. by + ed. & trans. by + ed. by + exec. prod. by + ill. by + narr. by + org. by + perf. by + prod. by + writ. by + trans. by + + + January + February + March + April + May + June + July + August + September + October + November + December + + + + Jan. + Feb. + Mar. + Apr. + May + June + July + Aug. + Sept. + Oct. + Nov. + Dec. + + + Spring + Summer + Autumn + Winter + + diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..ad0536b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,119 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +This is **设计人工智能：基础与应用** — a textbook project for design AI education. The book covers AI fundamentals (principles, perception, generation, agents) and their applications across design domains (digital media, industrial design, environmental design, urban planning). + +A supplementary volume, **CC4SI (Claude Code for Spatial Intelligence)**, is preserved in `officefile/supplements/` and focuses on spatial intelligence and autonomous design using Claude Code. + +**Key Characteristics:** +- Educational content organized as a book with 12 chapters + 9 appendices +- Mixed Chinese/English documentation and code +- Git-based workflow with Obsidian integration for content editing +- Pandoc-based export to Word (DOCX) with publisher template +- Standard research project directory structure + +## Directory Structure + +``` +2026_DesignAI/ +├── data/ # Data and resources +│ ├── pics/ # Book illustrations +│ └── references/ # Bibliography data +├── dofile/ # Code and scripts +│ └── examples/ # Python code examples +│ ├── 00-introduction/ # Setup first assistant +│ ├── 01-foundations/ # State machines, modularity, etc. +│ └── 02-spatial-intelligence/ # Spatial analysis examples +├── officefile/ # Main book content +│ ├── 00-frontmatter/ # 自序 (Preface), 目录 (TOC) +│ ├── 01-introduction/ # Ch1: AI development history +│ ├── 02-framework/ # Ch2: AI theoretical framework +│ ├── 03-1d-sequence/ # Ch3: Sequence & text (RNN→LLM) +│ ├── 04-2d-vision/ # Ch4: Images & vision (CNN) +│ ├── 05-3d-spatial/ # Ch5: Spatial & 3D (PointNet) +│ ├── 06-reinforcement/ # Ch6: Reinforcement learning +│ ├── 07-generative-ai/ # Ch7: Generative AI (Diffusion) +│ ├── 08-agent/ # Ch8: AI Agent +│ ├── 09-digital-media/ # Ch9: Digital media design +│ ├── 10-industrial-design/ # Ch10: Industrial design +│ ├── 11-environmental-landscape/ # Ch11: Environmental & landscape +│ ├── 12-urban-ecology/ # Ch12: Urban & ecological planning +│ ├── appendix/ # Appendices 1-9 +│ └── supplements/ # CC4SI supplementary material +│ ├── 00-05/ # Spatial intelligence chapters +│ └── appendix/ # Academic writing workflow +├── output/ # Generated outputs +│ └── docx/ # Word exports +├── Archive/ # Archived original directories +├── .claude/skills/markdown-to-docx/ # DOCX conversion skill +├── .pandoc/ # Pandoc config (CSL, Zotero JSON) +└── .obsidian/ # Obsidian vault configuration +``` + +## Book Structure + +### Upper Volume: Principles & Technology + +| Chapter | Topic | Lines | Status | +|---------|-------|-------|--------| +| 01 | AI development history & paradigm evolution | 367 | Complete | +| 02 | Theoretical framework (functional perspective, MLP) | 897 | Complete | +| 03 | 1D data: Sequence & text (RNN → LLM) | 1595 | Complete | +| 04 | 2D data: Images & vision (CNN → ViT) | 867 | Complete | +| 05 | 3D data: Spatial & 3D (PointNet → Spatial AI) | 1421 | Complete | +| 06 | Reinforcement learning (Q-Learning → RLHF) | 896 | Complete | +| 07 | Generative AI (VAE/GAN → Diffusion) | 496 | Complete | +| 08 | AI Agent (rules → LLM Agent) | 581 | Complete | + +### Lower Volume: Design Applications + +| Chapter | Topic | Lines | Status | +|---------|-------|-------|--------| +| 09 | Digital media design | 161 | Thin | +| 10 | Industrial & product design | 145 | Thin | +| 11 | Environmental & landscape design | 153 | Thin | +| 12 | Urban & ecological planning | 317 | Moderate | + +### Appendices + +9 appendices covering programming basics, Vibe Coding, academic writing, web tools, references, glossary, and tips. + +## Working with This Repository + +### For Content Editing +- Edit `.md` files in `officefile/` — they are the book source +- Use Obsidian for rich editing (vault at project root) +- Git commits follow conventional format: `docs(ch01): description` + +### For DOCX Export +- Use the `markdown-to-docx` skill in `.claude/skills/` +- Publisher template: `machinery-industry-press-writing-template.dotx` +- Output goes to `output/docx/` + +### For Python Examples +- Examples in `dofile/examples/` are self-contained +- Run directly: `python dofile/examples/01-foundations/state_machine.py` +- No global test suite — verify examples individually + +## Language Considerations + +- **Documentation**: Primarily Chinese with English technical terms +- **Code**: Variable names and comments in English +- **File names**: Mix of Chinese and English — use proper encoding + +## Commit Convention + +``` +(): + +Types: feat, fix, docs, style, refactor, chore +Scopes: ch01-ch12, appendix, supplements, examples + +Examples: +- docs(ch03): 补充 Transformer 注意力机制详解 +- feat(examples): 新增空间推理示例 +- fix(ch05): 修正 PointNet 结构描述 +``` diff --git a/data/pics/图7.8-路杀动物.jpg b/data/pics/图7.8-路杀动物.jpg new file mode 100644 index 0000000..fe10337 Binary files /dev/null and b/data/pics/图7.8-路杀动物.jpg differ diff --git a/dofile/examples/00-introduction/setup-first-assistant/spatial_helper.py b/dofile/examples/00-introduction/setup-first-assistant/spatial_helper.py new file mode 100644 index 0000000..60fcaea --- /dev/null +++ b/dofile/examples/00-introduction/setup-first-assistant/spatial_helper.py @@ -0,0 +1,962 @@ +""" +空间分析助手类 (Spatial Analysis Assistant) +======================================== + +这是一个完整的空间分析助手实现，展示了如何构建一个基础的 +空间智能系统。该系统可以帮助用户进行空间数据处理、分析和可视化。 + +主要功能: +1. 空间数据加载与管理 +2. 空间关系计算 (距离、方位、包含关系等) +3. 空间统计分析 +4. 空间插值与预测 +5. 多准则决策分析 + +作者: CC4SI 项目组 +日期: 2025-01 +""" + +import json +import math +from typing import List, Dict, Tuple, Optional, Any, Union +from dataclasses import dataclass, field +from enum import Enum +import random + + +# ============================================================================ +# 数据结构定义 +# ============================================================================ + +class GeometryType(Enum): + """几何类型枚举""" + POINT = "Point" + LINESTRING = "LineString" + POLYGON = "Polygon" + MULTIPOINT = "MultiPoint" + + +@dataclass +class Point: + """点几何类""" + x: float + y: float + z: Optional[float] = None + properties: Dict[str, Any] = field(default_factory=dict) + + def __repr__(self) -> str: + if self.z is not None: + return f"Point({self.x:.2f}, {self.y:.2f}, {self.z:.2f})" + return f"Point({self.x:.2f}, {self.y:.2f})" + + +@dataclass +class BoundingBox: + """边界框类""" + min_x: float + min_y: float + max_x: float + max_y: float + + @property + def width(self) -> float: + """获取边界框宽度""" + return self.max_x - self.min_x + + @property + def height(self) -> float: + """获取边界框高度""" + return self.max_y - self.min_y + + @property + def center(self) -> Point: + """获取边界框中心点""" + return Point( + (self.min_x + self.max_x) / 2, + (self.min_y + self.max_y) / 2 + ) + + def contains(self, point: Point) -> bool: + """检查点是否在边界框内""" + return (self.min_x <= point.x <= self.max_x and + self.min_y <= point.y <= self.max_y) + + def __repr__(self) -> str: + return f"BoundingBox[{self.min_x:.2f},{self.min_y:.2f} -> {self.max_x:.2f},{self.max_y:.2f}]" + + +@dataclass +class SpatialFeature: + """空间要素类""" + id: str + geometry: Union[Point, List[Point]] # 简化: 点或点列表 + properties: Dict[str, Any] = field(default_factory=dict) + + def __repr__(self) -> str: + return f"SpatialFeature(id={self.id}, geometry={type(self.geometry).__name__})" + + +# ============================================================================ +# 空间分析助手主类 +# ============================================================================ + +class SpatialHelper: + """ + 空间分析助手类 + + 这是系统的核心类，提供空间数据处理和分析的主要功能。 + """ + + def __init__(self, name: str = "空间分析助手", version: str = "1.0.0"): + """ + 初始化空间分析助手 + + Args: + name: 助手名称 + version: 版本号 + """ + self.name = name + self.version = version + self.features: List[SpatialFeature] = [] + self.coordinate_system: str = "EPSG:4326" # 默认 WGS84 + self.metadata: Dict[str, Any] = {} + + print(f"[{self.name}] v{self.version} 初始化完成") + + # ------------------------------------------------------------------------ + # 数据加载与管理 + # ------------------------------------------------------------------------ + + def add_feature(self, feature: SpatialFeature) -> bool: + """ + 添加空间要素 + + Args: + feature: 要添加的空间要素 + + Returns: + 是否添加成功 + """ + try: + self.features.append(feature) + return True + except Exception as e: + print(f"添加要素失败: {e}") + return False + + def add_point(self, x: float, y: float, feature_id: str = None, + properties: Dict[str, Any] = None) -> SpatialFeature: + """ + 添加点要素 + + Args: + x: X坐标 + y: Y坐标 + feature_id: 要素ID + properties: 属性字典 + + Returns: + 创建的空间要素 + """ + if feature_id is None: + feature_id = f"point_{len(self.features)}" + + point = Point(x, y) + feature = SpatialFeature( + id=feature_id, + geometry=point, + properties=properties or {} + ) + self.add_feature(feature) + return feature + + def load_from_geojson(self, geojson_str: str) -> int: + """ + 从 GeoJSON 字符串加载数据 + + Args: + geojson_str: GeoJSON 格式字符串 + + Returns: + 加载的要素数量 + """ + try: + data = json.loads(geojson_str) + count = 0 + + if data.get("type") == "FeatureCollection": + for feature_data in data.get("features", []): + feature = self._parse_geojson_feature(feature_data) + if feature: + self.add_feature(feature) + count += 1 + + print(f"从 GeoJSON 加载了 {count} 个要素") + return count + except json.JSONDecodeError as e: + print(f"GeoJSON 解析失败: {e}") + return 0 + + def _parse_geojson_feature(self, feature_data: Dict) -> Optional[SpatialFeature]: + """解析 GeoJSON 要素""" + try: + feature_id = feature_data.get("id", f"feature_{len(self.features)}") + properties = feature_data.get("properties", {}) + geometry = feature_data.get("geometry", {}) + geom_type = geometry.get("type") + + if geom_type == "Point": + coordinates = geometry.get("coordinates", []) + point = Point(coordinates[0], coordinates[1]) + return SpatialFeature(id=feature_id, geometry=point, properties=properties) + + return None + except Exception: + return None + + def get_feature_by_id(self, feature_id: str) -> Optional[SpatialFeature]: + """根据ID获取要素""" + for feature in self.features: + if feature.id == feature_id: + return feature + return None + + def get_feature_count(self) -> int: + """获取要素数量""" + return len(self.features) + + def clear_features(self) -> None: + """清空所有要素""" + self.features.clear() + print("已清空所有要素") + + # ------------------------------------------------------------------------ + # 空间关系计算 + # ------------------------------------------------------------------------ + + @staticmethod + def calculate_distance(point1: Point, point2: Point) -> float: + """ + 计算两点间的欧氏距离 + + Args: + point1: 第一个点 + point2: 第二个点 + + Returns: + 距离值 + """ + dx = point2.x - point1.x + dy = point2.y - point1.y + return math.sqrt(dx * dx + dy * dy) + + @staticmethod + def calculate_bearing(point1: Point, point2: Point) -> float: + """ + 计算从 point1 到 point2 的方位角 (度) + + Args: + point1: 起始点 + point2: 目标点 + + Returns: + 方位角 (0-360度) + """ + dx = point2.x - point1.x + dy = point2.y - point1.y + radians = math.atan2(dy, dx) + degrees = math.degrees(radians) + return (degrees + 360) % 360 + + @staticmethod + def calculate_midpoint(point1: Point, point2: Point) -> Point: + """ + 计算两点间的中点 + + Args: + point1: 第一个点 + point2: 第二个点 + + Returns: + 中点 + """ + return Point( + (point1.x + point2.x) / 2, + (point1.y + point2.y) / 2 + ) + + def find_nearest_neighbor(self, target_point: Point, + max_distance: float = float('inf')) -> Optional[Tuple[SpatialFeature, float]]: + """ + 查找最近邻要素 + + Args: + target_point: 目标点 + max_distance: 最大搜索距离 + + Returns: + (最近的要素, 距离) 或 None + """ + nearest_feature = None + min_distance = float('inf') + + for feature in self.features: + if isinstance(feature.geometry, Point): + dist = self.calculate_distance(target_point, feature.geometry) + if dist < min_distance and dist <= max_distance: + min_distance = dist + nearest_feature = feature + + if nearest_feature: + return nearest_feature, min_distance + return None + + def find_neighbors_within_distance(self, target_point: Point, + distance: float) -> List[Tuple[SpatialFeature, float]]: + """ + 查找指定距离内的所有要素 + + Args: + target_point: 目标点 + distance: 搜索半径 + + Returns: + (要素, 距离) 列表，按距离排序 + """ + neighbors = [] + + for feature in self.features: + if isinstance(feature.geometry, Point): + dist = self.calculate_distance(target_point, feature.geometry) + if dist <= distance: + neighbors.append((feature, dist)) + + neighbors.sort(key=lambda x: x[1]) + return neighbors + + def calculate_bounding_box(self) -> Optional[BoundingBox]: + """ + 计算所有要素的边界框 + + Returns: + 边界框对象，如果没有要素则返回 None + """ + if not self.features: + return None + + points = [f.geometry for f in self.features if isinstance(f.geometry, Point)] + if not points: + return None + + min_x = min(p.x for p in points) + max_x = max(p.x for p in points) + min_y = min(p.y for p in points) + max_y = max(p.y for p in points) + + return BoundingBox(min_x, min_y, max_x, max_y) + + # ------------------------------------------------------------------------ + # 空间统计分析 + # ------------------------------------------------------------------------ + + def calculate_centroid(self) -> Optional[Point]: + """ + 计算所有点要素的质心 + + Returns: + 质心点 + """ + points = [f.geometry for f in self.features if isinstance(f.geometry, Point)] + if not points: + return None + + avg_x = sum(p.x for p in points) / len(points) + avg_y = sum(p.y for p in points) / len(points) + + return Point(avg_x, avg_y) + + def calculate_mean_center(self, weight_field: str = None) -> Optional[Point]: + """ + 计算加权或未加权的平均中心 + + Args: + weight_field: 权重字段名 + + Returns: + 平均中心点 + """ + points = [] + weights = [] + + for feature in self.features: + if isinstance(feature.geometry, Point): + points.append(feature.geometry) + if weight_field: + weights.append(feature.properties.get(weight_field, 1.0)) + else: + weights.append(1.0) + + if not points: + return None + + total_weight = sum(weights) + avg_x = sum(p.x * w for p, w in zip(points, weights)) / total_weight + avg_y = sum(p.y * w for p, w in zip(points, weights)) / total_weight + + return Point(avg_x, avg_y) + + def calculate_standard_distance(self) -> Optional[float]: + """ + 计算标准距离 (标准差圆) + + Returns: + 标准距离值 + """ + centroid = self.calculate_centroid() + if not centroid: + return None + + points = [f.geometry for f in self.features if isinstance(f.geometry, Point)] + if not points: + return None + + n = len(points) + squared_distances = [(p.x - centroid.x)**2 + (p.y - centroid.y)**2 for p in points] + + return math.sqrt(sum(squared_distances) / n) + + def calculate_spatial_autocorrelation(self, field: str) -> Optional[float]: + """ + 计算 Moran's I 空间自相关指数 + + Args: + field: 要分析的属性字段 + + Returns: + Moran's I 值 + """ + # 简化实现: 使用距离权重 + points_data = [] + for feature in self.features: + if isinstance(feature.geometry, Point) and field in feature.properties: + points_data.append((feature.geometry, feature.properties[field])) + + n = len(points_data) + if n < 2: + return None + + mean_value = sum(v for _, v in points_data) / n + + # 计算权重矩阵 (距离倒数) + weights = {} + total_weight = 0 + for i, (p1, v1) in enumerate(points_data): + for j, (p2, v2) in enumerate(points_data): + if i != j: + dist = self.calculate_distance(p1, p2) + w = 1 / (dist + 0.001) # 避免除零 + weights[(i, j)] = w + total_weight += w + + # 计算 Moran's I + numerator = 0 + denominator = 0 + + for i, (p1, v1) in enumerate(points_data): + for j, (p2, v2) in enumerate(points_data): + if i != j: + w = weights.get((i, j), 0) + numerator += w * (v1 - mean_value) * (v2 - mean_value) + denominator += (v1 - mean_value) ** 2 + + if denominator == 0: + return None + + morans_i = (n / total_weight) * (numerator / denominator) + return morans_i + + # ------------------------------------------------------------------------ + # 空间插值与预测 + # ------------------------------------------------------------------------ + + def inverse_distance_weighting(self, target_point: Point, power: float = 2.0, + field: str = "value", max_distance: float = None) -> Optional[float]: + """ + 反距离加权插值 (IDW) + + Args: + target_point: 目标点 + power: 距离幂次 + field: 插值字段 + max_distance: 最大搜索距离 + + Returns: + 插值结果 + """ + points_data = [] + for feature in self.features: + if isinstance(feature.geometry, Point) and field in feature.properties: + points_data.append((feature.geometry, feature.properties[field])) + + if not points_data: + return None + + numerator = 0.0 + denominator = 0.0 + + for point, value in points_data: + dist = self.calculate_distance(target_point, point) + + if max_distance and dist > max_distance: + continue + + if dist < 1e-10: # 几乎重合 + return value + + weight = 1.0 / (dist ** power) + numerator += weight * value + denominator += weight + + if denominator == 0: + return None + + return numerator / denominator + + def simple_trend_prediction(self, field: str, target_x: float, + target_y: float) -> Optional[float]: + """ + 基于简单趋势的预测 (线性回归) + + Args: + field: 预测字段 + target_x: 目标X坐标 + target_y: 目标Y坐标 + + Returns: + 预测值 + """ + points_data = [] + for feature in self.features: + if isinstance(feature.geometry, Point) and field in feature.properties: + points_data.append({ + 'x': feature.geometry.x, + 'y': feature.geometry.y, + 'z': feature.properties[field] + }) + + if len(points_data) < 3: + return None + + # 简单的线性趋势: z = a + b*x + c*y + n = len(points_data) + + sum_x = sum(p['x'] for p in points_data) + sum_y = sum(p['y'] for p in points_data) + sum_z = sum(p['z'] for p in points_data) + sum_xx = sum(p['x']**2 for p in points_data) + sum_yy = sum(p['y']**2 for p in points_data) + sum_xy = sum(p['x'] * p['y'] for p in points_data) + sum_xz = sum(p['x'] * p['z'] for p in points_data) + sum_yz = sum(p['y'] * p['z'] for p in points_data) + + # 简化: 只使用x方向趋势 + try: + # z = a + b*x + b = (n * sum_xz - sum_x * sum_z) / (n * sum_xx - sum_x**2) + a = (sum_z - b * sum_x) / n + return a + b * target_x + except ZeroDivisionError: + return None + + # ------------------------------------------------------------------------ + # 多准则决策分析 (MCDA) + # ------------------------------------------------------------------------ + + def weighted_sum_model(self, criteria: List[str], weights: List[float], + feature_ids: List[str] = None) -> List[Tuple[SpatialFeature, float]]: + """ + 加权求和模型 (WSM) + + Args: + criteria: 评价准则列表 + weights: 各准则权重 + feature_ids: 参与评价的要素ID列表 + + Returns: + (要素, 得分) 列表，按得分降序排列 + """ + if len(criteria) != len(weights): + print("错误: 准则数量与权重数量不匹配") + return [] + + if abs(sum(weights) - 1.0) > 0.001: + print(f"警告: 权重总和为 {sum(weights)}, 建议归一化为 1.0") + + # 确定评价范围 + features_to_eval = self.features + if feature_ids: + features_to_eval = [f for f in self.features if f.id in feature_ids] + + results = [] + + # 归一化参数 + min_max = {} + for criterion in criteria: + values = [] + for f in features_to_eval: + if criterion in f.properties: + values.append(f.properties[criterion]) + if values: + min_max[criterion] = (min(values), max(values)) + + for feature in features_to_eval: + score = 0.0 + valid = True + + for criterion, weight in zip(criteria, weights): + if criterion not in feature.properties: + valid = False + break + + value = feature.properties[criterion] + cmin, cmax = min_max.get(criterion, (0, 1)) + + # 归一化 (假设值越大越好) + if cmax - cmin > 0: + normalized = (value - cmin) / (cmax - cmin) + else: + normalized = 0.5 + + score += weight * normalized + + if valid: + results.append((feature, score)) + + results.sort(key=lambda x: x[1], reverse=True) + return results + + def weighted_product_model(self, criteria: List[str], weights: List[float], + feature_ids: List[str] = None) -> List[Tuple[SpatialFeature, float]]: + """ + 加权乘积模型 (WPM) + + Args: + criteria: 评价准则列表 + weights: 各准则权重 + feature_ids: 参与评价的要素ID列表 + + Returns: + (要素, 得分) 列表，按得分降序排列 + """ + if len(criteria) != len(weights): + print("错误: 准则数量与权重数量不匹配") + return [] + + features_to_eval = self.features + if feature_ids: + features_to_eval = [f for f in self.features if f.id in feature_ids] + + results = [] + + # 归一化参数 + min_max = {} + for criterion in criteria: + values = [] + for f in features_to_eval: + if criterion in f.properties: + values.append(f.properties[criterion]) + if values: + min_max[criterion] = (min(values), max(values)) + + for feature in features_to_eval: + product = 1.0 + valid = True + + for criterion, weight in zip(criteria, weights): + if criterion not in feature.properties: + valid = False + break + + value = feature.properties[criterion] + cmin, cmax = min_max.get(criterion, (0, 1)) + + if cmax - cmin > 0: + normalized = (value - cmin) / (cmax - cmin) + else: + normalized = 1.0 + + product *= normalized ** weight + + if valid: + results.append((feature, product)) + + results.sort(key=lambda x: x[1], reverse=True) + return results + + def topsis(self, criteria: List[str], weights: List[float], + benefit_criteria: List[bool] = None) -> List[Tuple[SpatialFeature, float]]: + """ + TOPSIS (逼近理想解排序法) + + Args: + criteria: 评价准则列表 + weights: 各准则权重 + benefit_criteria: 是否为效益型准则 (True=越大越好, False=越小越好) + + Returns: + (要素, 相对贴近度) 列表，按贴近度降序排列 + """ + if benefit_criteria is None: + benefit_criteria = [True] * len(criteria) + + features_to_eval = [f for f in self.features + if all(c in f.properties for c in criteria)] + + if len(features_to_eval) == 0: + return [] + + n = len(features_to_eval) + m = len(criteria) + + # 构建决策矩阵 + decision_matrix = [] + for feature in features_to_eval: + row = [feature.properties[c] for c in criteria] + decision_matrix.append(row) + + # 归一化决策矩阵 + normalized_matrix = [] + for j in range(m): + column = [decision_matrix[i][j] for i in range(n)] + norm = math.sqrt(sum(x**2 for x in column)) + for i in range(n): + if j == 0: + normalized_matrix.append([]) + normalized_matrix[i].append(decision_matrix[i][j] / norm if norm > 0 else 0) + + # 加权归一化矩阵 + weighted_matrix = [] + for i in range(n): + weighted_matrix.append([normalized_matrix[i][j] * weights[j] for j in range(m)]) + + # 确定理想解和负理想解 + ideal_positive = [] + ideal_negative = [] + + for j in range(m): + column = [weighted_matrix[i][j] for i in range(n)] + if benefit_criteria[j]: + ideal_positive.append(max(column)) + ideal_negative.append(min(column)) + else: + ideal_positive.append(min(column)) + ideal_negative.append(max(column)) + + # 计算距离和相对贴近度 + results = [] + for i, feature in enumerate(features_to_eval): + dist_positive = math.sqrt( + sum((weighted_matrix[i][j] - ideal_positive[j])**2 for j in range(m)) + ) + dist_negative = math.sqrt( + sum((weighted_matrix[i][j] - ideal_negative[j])**2 for j in range(m)) + ) + + closeness = dist_negative / (dist_positive + dist_negative) if (dist_positive + dist_negative) > 0 else 0 + results.append((feature, closeness)) + + results.sort(key=lambda x: x[1], reverse=True) + return results + + # ------------------------------------------------------------------------ + # 缓冲区分析 + # ------------------------------------------------------------------------ + + def create_buffer_analysis(self, feature_id: str, buffer_distance: float) -> Dict[str, Any]: + """ + 缓冲区分析 + + Args: + feature_id: 中心要素ID + buffer_distance: 缓冲距离 + + Returns: + 分析结果字典 + """ + target_feature = self.get_feature_by_id(feature_id) + if not target_feature or not isinstance(target_feature.geometry, Point): + return {"error": "找不到指定的点要素"} + + center = target_feature.geometry + + # 查找缓冲区内的要素 + features_in_buffer = self.find_neighbors_within_distance(center, buffer_distance) + + # 计算统计信息 + values_in_buffer = [f.properties for f, _ in features_in_buffer] + + return { + "center_feature": feature_id, + "buffer_distance": buffer_distance, + "count": len(features_in_buffer), + "features": [(f.id, dist) for f, dist in features_in_buffer], + "statistics": { + "avg_distance": sum(dist for _, dist in features_in_buffer) / len(features_in_buffer) if features_in_buffer else 0 + } + } + + # ------------------------------------------------------------------------ + # 可视化辅助 (文本形式) + # ------------------------------------------------------------------------ + + def print_summary(self) -> None: + """打印数据摘要""" + print(f"\n{'='*50}") + print(f"空间分析助手摘要: {self.name} v{self.version}") + print(f"{'='*50}") + print(f"要素数量: {len(self.features)}") + print(f"坐标系: {self.coordinate_system}") + + bbox = self.calculate_bounding_box() + if bbox: + print(f"边界范围: {bbox}") + + centroid = self.calculate_centroid() + if centroid: + print(f"质心位置: {centroid}") + + print(f"{'='*50}\n") + + def print_features(self, limit: int = 10) -> None: + """打印要素列表""" + print(f"\n要素列表 (显示前 {min(limit, len(self.features))} 个):") + print("-" * 60) + + for i, feature in enumerate(self.features[:limit]): + if isinstance(feature.geometry, Point): + print(f"{i+1}. ID: {feature.id:15s} 位置: {feature.geometry} 属性: {feature.properties}") + + if len(self.features) > limit: + print(f"... 还有 {len(self.features) - limit} 个要素") + + print("-" * 60 + "\n") + + +# ============================================================================ +# 辅助函数 +# ============================================================================ + +def create_sample_data(helper: SpatialHelper, n_points: int = 20) -> None: + """创建示例数据""" + print(f"生成 {n_points} 个随机样本点...") + + random.seed(42) # 可重现的随机数 + + # 生成随机点 + for i in range(n_points): + x = random.uniform(0, 100) + y = random.uniform(0, 100) + value = random.uniform(0, 100) + population = random.randint(100, 10000) + accessibility = random.uniform(0.3, 0.95) + + helper.add_point( + x=x, + y=y, + feature_id=f"point_{i:03d}", + properties={ + "value": value, + "population": population, + "accessibility": accessibility, + "name": f"位置_{i+1}" + } + ) + + print(f"已生成 {helper.get_feature_count()} 个样本点") + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示空间分析助手的使用""" + + print("="*70) + print("空间分析助手 - 完整示例演示") + print("="*70) + + # 1. 创建助手实例 + print("\n[步骤 1] 创建空间分析助手") + helper = SpatialHelper(name="城市空间分析助手", version="1.0.0") + + # 2. 添加示例数据 + print("\n[步骤 2] 添加示例数据") + create_sample_data(helper, n_points=15) + + # 3. 打印数据摘要 + print("\n[步骤 3] 数据摘要") + helper.print_summary() + helper.print_features() + + # 4. 空间关系计算 + print("\n[步骤 4] 空间关系计算") + print("-" * 50) + + test_point = Point(50, 50) + print(f"测试点: {test_point}") + + nearest = helper.find_nearest_neighbor(test_point) + if nearest: + feature, dist = nearest + print(f"最近邻: {feature.id}, 距离: {dist:.2f}") + + neighbors = helper.find_neighbors_within_distance(test_point, 25) + print(f"半径25内的邻居数量: {len(neighbors)}") + + # 5. 空间统计分析 + print("\n[步骤 5] 空间统计分析") + print("-" * 50) + + centroid = helper.calculate_centroid() + print(f"质心: {centroid}") + + std_dist = helper.calculate_standard_distance() + print(f"标准距离: {std_dist:.2f}") + + # 6. 多准则决策分析 + print("\n[步骤 6] 多准则决策分析 (TOPSIS)") + print("-" * 50) + + criteria = ["accessibility", "population"] + weights = [0.6, 0.4] # 可达性权重更高 + + results = helper.topsis(criteria, weights) + print("选址优先级排序 (基于可达性和人口):") + for i, (feature, score) in enumerate(results[:5]): + print(f" {i+1}. {feature.properties.get('name', feature.id)}: 得分={score:.4f}") + + # 7. 空间插值 + print("\n[步骤 7] 空间插值预测") + print("-" * 50) + + predict_point = Point(45, 55) + predicted = helper.inverse_distance_weighting(predict_point, field="value") + print(f"在 {predict_point} 处的插值预测: {predicted:.2f}") + + # 8. 缓冲区分析 + print("\n[步骤 8] 缓冲区分析") + print("-" * 50) + + buffer_result = helper.create_buffer_analysis("point_000", 30) + print(f"以 point_000 为中心，半径30的缓冲区:") + print(f" 包含要素数: {buffer_result['count']}") + print(f" 平均距离: {buffer_result['statistics']['avg_distance']:.2f}") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/01-foundations/feedback_learning.py b/dofile/examples/01-foundations/feedback_learning.py new file mode 100644 index 0000000..73c52fa --- /dev/null +++ b/dofile/examples/01-foundations/feedback_learning.py @@ -0,0 +1,827 @@ +""" +反馈与学习示例 (Feedback and Learning Example) +============================================= + +本示例展示如何在空间智能系统中实现反馈机制和学习能力。 +反馈和学习使系统能够从经验中改进，提高决策质量。 + +核心概念: +1. 反馈循环 - 收集用户/系统的反馈 +2. 性能评估 - 评估决策效果 +3. 参数调整 - 根据反馈调整系统参数 +4. 经验存储 - 保存和检索历史经验 +5. 迁移学习 - 将知识应用到新场景 + +应用场景: +- 自适应决策权重调整 +- 模型参数优化 +- 用户偏好学习 +- 决策效果跟踪 + +作者: CC4SI 项目组 +""" + +import math +import json +from typing import List, Dict, Tuple, Optional, Any, Callable +from dataclasses import dataclass, field +from enum import Enum +from datetime import datetime +import random + + +# ============================================================================ +# 反馈类型定义 +# ============================================================================ + +class FeedbackType(Enum): + """反馈类型枚举""" + EXPLICIT = "explicit" # 显式反馈 (用户评分/评价) + IMPLICIT = "implicit" # 隐式反馈 (行为数据) + OUTCOME = "outcome" # 结果反馈 (实际结果) + CORRECTION = "correction" # 纠正反馈 (修正建议) + RANKING = "ranking" # 排序反馈 (偏好排序) + + +class FeedbackSource(Enum): + """反馈来源枚举""" + HUMAN_EXPERT = "human_expert" # 人类专家 + SYSTEM_AUTO = "system_auto" # 系统自动 + SENSOR_DATA = "sensor_data" # 传感器数据 + CROWDSOURCING = "crowdsourcing" # 众包 + PEER_REVIEW = "peer_review" # 同行评审 + + +@dataclass +class Feedback: + """ + 反馈数据结构 + + 表示一次具体的反馈事件。 + """ + feedback_id: str + feedback_type: FeedbackType + source: FeedbackSource + target_decision_id: str + value: float # 反馈值 (如评分) + content: Optional[str] = None # 反馈内容 + metadata: Dict[str, Any] = field(default_factory=dict) + timestamp: datetime = field(default_factory=datetime.now) + + def __repr__(self) -> str: + return f"Feedback({self.feedback_type.value}, value={self.value:.2f})" + + +# ============================================================================ +# 决策记录 +# ============================================================================ + +@dataclass +class Decision: + """ + 决策记录 + + 保存系统做出的一次决策的完整信息。 + """ + decision_id: str + context: Dict[str, Any] # 决策上下文 + alternatives: List[Dict[str, Any]] # 可选方案 + selected_alternative: int # 选择的方案索引 + model_version: str # 使用的模型版本 + parameters: Dict[str, Any] # 决策参数 + predicted_outcome: Optional[float] = None # 预测结果 + actual_outcome: Optional[float] = None # 实际结果 + feedback_list: List[Feedback] = field(default_factory=list) + timestamp: datetime = field(default_factory=datetime.now) + + def add_feedback(self, feedback: Feedback) -> None: + """添加反馈""" + self.feedback_list.append(feedback) + + def get_average_feedback(self) -> float: + """获取平均反馈分数""" + if not self.feedback_list: + return 0.0 + return sum(f.value for f in self.feedback_list) / len(self.feedback_list) + + def get_outcome_error(self) -> Optional[float]: + """获取预测误差""" + if self.predicted_outcome is not None and self.actual_outcome is not None: + return abs(self.predicted_outcome - self.actual_outcome) + return None + + def calculate_regret(self) -> float: + """ + 计算后悔值 + + 后悔值 = 最优选择的结果 - 实际选择的结果 + """ + if not self.alternatives or self.actual_outcome is None: + return 0.0 + + # 假设alternatives中存储了各个选项的实际结果 + best_outcome = max( + alt.get("actual_outcome", self.actual_outcome) + for alt in self.alternatives + ) + return best_outcome - self.actual_outcome + + +# ============================================================================ +# 经验存储 +# ============================================================================ + +class ExperienceStore: + """ + 经验存储 + + 存储和检索历史决策经验，用于学习和改进。 + """ + + def __init__(self, capacity: int = 1000): + """ + 初始化经验存储 + + Args: + capacity: 最大存储容量 + """ + self.capacity = capacity + self.decisions: Dict[str, Decision] = {} + self.decision_list: List[str] = [] # 按时间顺序的ID列表 + + def add_decision(self, decision: Decision) -> None: + """添加决策记录""" + self.decisions[decision.decision_id] = decision + self.decision_list.append(decision.decision_id) + + # 超过容量时删除最旧的 + if len(self.decision_list) > self.capacity: + oldest_id = self.decision_list.pop(0) + del self.decisions[oldest_id] + + def get_decision(self, decision_id: str) -> Optional[Decision]: + """获取决策记录""" + return self.decisions.get(decision_id) + + def get_recent_decisions(self, n: int = 10) -> List[Decision]: + """获取最近的n条决策""" + recent_ids = self.decision_list[-n:] + return [self.decisions[id] for id in recent_ids] + + def find_similar_decisions(self, context: Dict[str, Any], + threshold: float = 0.8) -> List[Decision]: + """ + 查找相似上下文的决策 + + Args: + context: 目标上下文 + threshold: 相似度阈值 + + Returns: + 相似决策列表 + """ + similar = [] + + for decision in self.decisions.values(): + similarity = self._calculate_similarity(context, decision.context) + if similarity >= threshold: + similar.append((decision, similarity)) + + similar.sort(key=lambda x: x[1], reverse=True) + return [d for d, _ in similar] + + def _calculate_similarity(self, ctx1: Dict[str, Any], + ctx2: Dict[str, Any]) -> float: + """计算上下文相似度 (简化版本)""" + # 简化: 使用键的交集比例 + keys1 = set(ctx1.keys()) + keys2 = set(ctx2.keys()) + intersection = keys1 & keys2 + union = keys1 | keys2 + + if not union: + return 0.0 + + # 值相似度 + value_similarity = 0.0 + count = 0 + + for key in intersection: + v1 = ctx1.get(key) + v2 = ctx2.get(key) + if isinstance(v1, (int, float)) and isinstance(v2, (int, float)): + # 归一化差异 + max_val = max(abs(v1), abs(v2), 1) + diff = abs(v1 - v2) / max_val + value_similarity += (1 - diff) + count += 1 + + if count > 0: + value_similarity /= count + + # 组合相似度 + key_similarity = len(intersection) / len(union) + return 0.3 * key_similarity + 0.7 * value_similarity + + def get_statistics(self) -> Dict[str, Any]: + """获取统计信息""" + total = len(self.decisions) + + if total == 0: + return {"total_decisions": 0} + + with_feedback = sum(1 for d in self.decisions.values() if d.feedback_list) + with_outcome = sum(1 for d in self.decisions.values() + if d.actual_outcome is not None) + + avg_feedback = sum(d.get_average_feedback() + for d in self.decisions.values() + if d.feedback_list) / max(with_feedback, 1) + + return { + "total_decisions": total, + "decisions_with_feedback": with_feedback, + "decisions_with_outcome": with_outcome, + "average_feedback_score": avg_feedback + } + + +# ============================================================================ +# 学习器接口 +# ============================================================================ + +class Learner(ABC): + """学习器抽象基类""" + + def __init__(self, name: str = ""): + self.name = name + + @abstractmethod + def learn_from_feedback(self, decision: Decision, feedback: Feedback) -> None: + """从反馈中学习""" + pass + + @abstractmethod + def learn_from_outcome(self, decision: Decision) -> None: + """从结果中学习""" + pass + + @abstractmethod + def get_parameters(self) -> Dict[str, Any]: + """获取当前参数""" + pass + + @abstractmethod + def update_parameters(self, params: Dict[str, Any]) -> None: + """更新参数""" + pass + + +# ============================================================================ +# 权重学习器 +# ============================================================================ + +class WeightLearner(Learner): + """ + 权重学习器 + + 通过反馈学习多准则决策的权重。 + """ + + def __init__(self, initial_weights: List[float], + learning_rate: float = 0.1, + min_weight: float = 0.05, + max_weight: float = 0.5): + """ + 初始化权重学习器 + + Args: + initial_weights: 初始权重列表 + learning_rate: 学习率 + min_weight: 最小权重 + max_weight: 最大权重 + """ + super().__init__("WeightLearner") + self.weights = initial_weights.copy() + self.learning_rate = learning_rate + self.min_weight = min_weight + self.max_weight = max_weight + self.update_count = 0 + + def learn_from_feedback(self, decision: Decision, feedback: Feedback) -> None: + """ + 从反馈中学习权重 + + 使用梯度下降法调整权重: + - 如果反馈为正，增加选中选项的优势准则权重 + - 如果反馈为负，减少选中选项的优势准则权重 + """ + if not decision.alternatives or decision.selected_alternative >= len(decision.alternatives): + return + + selected = decision.alternatives[decision.selected_alternative] + + # 计算调整方向 + feedback_normalized = (feedback.value - 0.5) * 2 # 转换到 [-1, 1] + + # 获取准则值 (假设存储在criteria字段) + criteria_values = selected.get("criteria", []) + + if len(criteria_values) != len(self.weights): + return + + # 计算梯度 + # 简化: 增加高值准则的权重 (如果反馈为正) + max_value = max(criteria_values) if criteria_values else 1 + gradients = [] + + for i, value in enumerate(criteria_values): + # 归一化值 + norm_value = value / max_value if max_value > 0 else 0 + # 梯度: 高值准则应该有更大权重 + gradient = (norm_value - 0.5) * feedback_normalized + gradients.append(gradient) + + # 更新权重 + for i, gradient in enumerate(gradients): + self.weights[i] += self.learning_rate * gradient + + # 归一化权重 + self._normalize_weights() + self.update_count += 1 + + def learn_from_outcome(self, decision: Decision) -> None: + """ + 从结果中学习 + + 如果实际结果好于预期，增加选中策略的权重 + """ + if decision.predicted_outcome is None or decision.actual_outcome is None: + return + + # 计算结果误差 + error = decision.actual_outcome - decision.predicted_outcome + + # 归一化误差 + error_normalized = math.tanh(error / 100) # 假设100为合理的误差范围 + + # 根据误差调整权重 + feedback = Feedback( + feedback_id=f"outcome_{decision.decision_id}", + feedback_type=FeedbackType.OUTCOME, + source=FeedbackSource.SYSTEM_AUTO, + target_decision_id=decision.decision_id, + value=0.5 + error_normalized * 0.25 # 转换到合理范围 + ) + + self.learn_from_feedback(decision, feedback) + + def _normalize_weights(self) -> None: + """归一化权重并限制范围""" + # 限制范围 + self.weights = [ + max(self.min_weight, min(self.max_weight, w)) + for w in self.weights + ] + + # 归一化使和为1 + total = sum(self.weights) + self.weights = [w / total for w in self.weights] + + def get_parameters(self) -> Dict[str, Any]: + return { + "weights": self.weights, + "learning_rate": self.learning_rate, + "update_count": self.update_count + } + + def update_parameters(self, params: Dict[str, Any]) -> None: + if "weights" in params: + self.weights = params["weights"].copy() + if "learning_rate" in params: + self.learning_rate = params["learning_rate"] + + +# ============================================================================ +# 自适应决策系统 +# ============================================================================ + +class AdaptiveDecisionSystem: + """ + 自适应决策系统 + + 结合反馈和学习的智能决策系统。 + """ + + def __init__(self, criteria: List[str], + initial_weights: List[float] = None): + """ + 初始化自适应决策系统 + + Args: + criteria: 决策准则列表 + initial_weights: 初始权重 + """ + self.criteria = criteria + self.n_criteria = len(criteria) + + if initial_weights is None: + # 均匀初始权重 + initial_weights = [1.0 / self.n_criteria] * self.n_criteria + + # 归一化权重 + total = sum(initial_weights) + self.weights = [w / total for w in initial_weights] + + # 创建学习器 + self.learner = WeightLearner(self.weights) + + # 创建经验存储 + self.experience_store = ExperienceStore() + + # 决策计数器 + self.decision_counter = 0 + + print(f"[自适应决策系统] 初始化完成") + print(f" 准则: {self.criteria}") + print(f" 初始权重: {[f'{w:.3f}' for w in self.weights]}") + + def make_decision(self, alternatives: List[Dict[str, float]], + context: Dict[str, Any] = None) -> Tuple[int, Dict[str, Any]]: + """ + 做出决策 + + Args: + alternatives: 备选方案列表，每个方案包含各准则的值 + context: 决策上下文 + + Returns: + (选中方案索引, 决策信息) + """ + if not alternatives: + raise ValueError("没有备选方案") + + # 计算每个方案的综合得分 + scores = [] + for alt in alternatives: + score = self._calculate_score(alt) + scores.append(score) + + # 选择得分最高的 + selected_idx = max(range(len(scores)), key=lambda i: scores[i]) + + # 创建决策记录 + decision_id = f"decision_{self.decision_counter}" + self.decision_counter += 1 + + decision = Decision( + decision_id=decision_id, + context=context or {}, + alternatives=[ + {"criteria": alt, "score": score} + for alt, score in zip(alternatives, scores) + ], + selected_alternative=selected_idx, + model_version="1.0", + parameters=self.get_parameters() + ) + + decision_info = { + "decision_id": decision_id, + "selected_index": selected_idx, + "selected_alternative": alternatives[selected_idx], + "score": scores[selected_idx], + "all_scores": scores, + "weights": self.weights.copy() + } + + # 存储决策 + self.experience_store.add_decision(decision) + + return selected_idx, decision_info + + def _calculate_score(self, alternative: Dict[str, float]) -> float: + """ + 计算方案的综合得分 + + 使用加权求和模型 + """ + score = 0.0 + for i, criterion in enumerate(self.criteria): + if criterion in alternative: + score += self.weights[i] * alternative[criterion] + return score + + def provide_feedback(self, decision_id: str, feedback_value: float, + feedback_type: FeedbackType = FeedbackType.EXPLICIT, + source: FeedbackSource = FeedbackSource.HUMAN_EXPERT, + content: str = None) -> None: + """ + 为决策提供反馈 + + Args: + decision_id: 决策ID + feedback_value: 反馈值 (通常在0-1范围) + feedback_type: 反馈类型 + source: 反馈来源 + content: 反馈内容 + """ + decision = self.experience_store.get_decision(decision_id) + if not decision: + print(f"警告: 找不到决策 {decision_id}") + return + + # 创建反馈 + feedback = Feedback( + feedback_id=f"fb_{decision_id}_{len(decision.feedback_list)}", + feedback_type=feedback_type, + source=source, + target_decision_id=decision_id, + value=feedback_value, + content=content + ) + + # 添加到决策记录 + decision.add_feedback(feedback) + + # 从反馈中学习 + self.learner.learn_from_feedback(decision, feedback) + + # 更新系统权重 + self.weights = self.learner.weights.copy() + + print(f"[反馈] 收到反馈: {feedback_value:.2f}") + print(f"[学习] 更新后权重: {[f'{w:.3f}' for w in self.weights]}") + + def report_outcome(self, decision_id: str, actual_outcome: float) -> None: + """ + 报告实际结果 + + Args: + decision_id: 决策ID + actual_outcome: 实际结果值 + """ + decision = self.experience_store.get_decision(decision_id) + if not decision: + print(f"警告: 找不到决策 {decision_id}") + return + + decision.actual_outcome = actual_outcome + + # 从结果中学习 + self.learner.learn_from_outcome(decision) + + # 更新系统权重 + self.weights = self.learner.weights.copy() + + print(f"[结果] 决策 {decision_id} 实际结果: {actual_outcome:.2f}") + + def get_parameters(self) -> Dict[str, Any]: + """获取当前系统参数""" + return { + "weights": self.weights.copy(), + "criteria": self.criteria.copy() + } + + def set_parameters(self, params: Dict[str, Any]) -> None: + """设置系统参数""" + if "weights" in params: + self.weights = params["weights"].copy() + + def get_performance_summary(self) -> Dict[str, Any]: + """获取性能摘要""" + stats = self.experience_store.get_statistics() + + # 计算平均反馈分数 + decisions = list(self.experience_store.decisions.values()) + if decisions: + avg_feedback = sum(d.get_average_feedback() + for d in decisions if d.feedback_list) + feedback_count = sum(1 for d in decisions if d.feedback_list) + avg_feedback = avg_feedback / feedback_count if feedback_count > 0 else None + else: + avg_feedback = None + + # 计算平均后悔值 + regrets = [d.calculate_regret() for d in decisions + if d.actual_outcome is not None] + avg_regret = sum(regrets) / len(regrets) if regrets else None + + return { + "total_decisions": stats["total_decisions"], + "decisions_with_feedback": stats["decisions_with_feedback"], + "average_feedback_score": avg_feedback, + "average_regret": avg_regret, + "current_weights": self.weights.copy() + } + + def print_summary(self) -> None: + """打印系统摘要""" + print("\n" + "="*60) + print("自适应决策系统摘要") + print("="*60) + + print("\n决策准则:") + for i, criterion in enumerate(self.criteria): + print(f" {i+1}. {criterion:15s} 权重: {self.weights[i]:.4f}") + + perf = self.get_performance_summary() + print(f"\n性能统计:") + print(f" 总决策数: {perf['total_decisions']}") + print(f" 有反馈的决策: {perf['decisions_with_feedback']}") + + if perf['average_feedback_score'] is not None: + print(f" 平均反馈分数: {perf['average_feedback_score']:.3f}") + + if perf['average_regret'] is not None: + print(f" 平均后悔值: {perf['average_regret']:.3f}") + + print("="*60 + "\n") + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示反馈与学习的使用""" + + print("="*70) + print("反馈与学习示例演示") + print("="*70) + + # ======================================================================== + # 1. 创建自适应决策系统 + # ======================================================================== + print("\n[步骤 1] 创建自适应决策系统") + print("-" * 50) + + criteria = ["经济效益", "环境影响", "社会影响", "技术可行性"] + initial_weights = [0.4, 0.3, 0.2, 0.1] # 偏重经济效益 + + system = AdaptiveDecisionSystem(criteria, initial_weights) + system.print_summary() + + # ======================================================================== + # 2. 第一次决策 + # ======================================================================== + print("\n[步骤 2] 第一次决策 - 工厂选址") + print("-" * 50) + + alternatives = [ + {"经济效益": 0.8, "环境影响": 0.3, "社会影响": 0.5, "技术可行性": 0.9}, # 位置A + {"经济效益": 0.5, "环境影响": 0.7, "社会影响": 0.8, "技术可行性": 0.6}, # 位置B + {"经济效益": 0.6, "环境影响": 0.9, "社会影响": 0.6, "技术可行性": 0.7}, # 位置C + ] + + selected_idx, decision_info = system.make_decision( + alternatives, + context={"task": "工厂选址", "region": "华东地区"} + ) + + print(f"\n决策结果:") + print(f" 选中方案: 位置{chr(65 + selected_idx)}") + print(f" 得分: {decision_info['score']:.3f}") + print(f" 各方案得分: {[f'{s:.2f}' for s in decision_info['all_scores']]}") + + decision_id_1 = decision_info['decision_id'] + + # ======================================================================== + # 3. 提供反馈 + # ======================================================================== + print("\n[步骤 3] 收集反馈") + print("-" * 50) + + # 专家反馈: 环境影响被低估了 + print("\n3.1 专家反馈: 环境影响应该更重视") + system.provide_feedback( + decision_id_1, + feedback_value=0.6, # 中等偏下的评分 + feedback_type=FeedbackType.EXPLICIT, + source=FeedbackSource.HUMAN_EXPERT, + content="环境影响权重太低，应提高" + ) + + # 更多反馈强化 + system.provide_feedback( + decision_id_1, + feedback_value=0.5, + feedback_type=FeedbackType.CORRECTION, + source=FeedbackSource.HUMAN_EXPERT + ) + + # ======================================================================== + # 4. 第二次决策 (学习后的权重) + # ======================================================================== + print("\n[步骤 4] 第二次决策 - 另一个选址") + print("-" * 50) + + alternatives_2 = [ + {"经济效益": 0.7, "环境影响": 0.4, "社会影响": 0.6, "技术可行性": 0.8}, # 位置D + {"经济效益": 0.4, "环境影响": 0.9, "社会影响": 0.7, "技术可行性": 0.7}, # 位置E + ] + + selected_idx_2, decision_info_2 = system.make_decision( + alternatives_2, + context={"task": "工厂选址", "region": "华南地区"} + ) + + print(f"\n决策结果:") + print(f" 选中方案: 位置{chr(68 + selected_idx_2)}") + print(f" 得分: {decision_info_2['score']:.3f}") + print(f" 当前权重: {[f'{w:.3f}' for w in system.weights]}") + + decision_id_2 = decision_info_2['decision_id'] + + # ======================================================================== + # 5. 报告结果并学习 + # ======================================================================== + print("\n[步骤 5] 报告实际结果") + print("-" * 50) + + # 第一个决策的结果 + print(f"\n5.1 决策 {decision_id_1} 的实际结果") + system.report_outcome(decision_id_1, actual_outcome=75) # 预测可能不同 + + # 第二个决策的结果 + print(f"\n5.2 决策 {decision_id_2} 的实际结果") + system.report_outcome(decision_id_2, actual_outcome=85) + + # ======================================================================== + # 6. 多轮学习 + # ======================================================================== + print("\n[步骤 6] 多轮学习") + print("-" * 50) + + # 模拟多次决策和反馈 + for i in range(10): + alt1 = { + "经济效益": random.uniform(0.5, 0.9), + "环境影响": random.uniform(0.3, 0.7), + "社会影响": random.uniform(0.4, 0.8), + "技术可行性": random.uniform(0.5, 0.9) + } + alt2 = { + "经济效益": random.uniform(0.3, 0.7), + "环境影响": random.uniform(0.6, 0.95), + "社会影响": random.uniform(0.5, 0.9), + "技术可行性": random.uniform(0.4, 0.8) + } + + idx, info = system.make_decision([alt1, alt2]) + did = info['decision_id'] + + # 模拟反馈 (随着环境意识增强，对高环境影响的方案给低分) + selected_env_impact = ([alt1, alt2][idx])["环境影响"] + if selected_env_impact < 0.6: + feedback_val = random.uniform(0.3, 0.5) # 低分 + else: + feedback_val = random.uniform(0.7, 0.95) # 高分 + + system.provide_feedback(did, feedback_val) + system.report_outcome(did, actual_outcome=random.uniform(60, 90)) + + print("\n多轮学习后:") + system.print_summary() + + # ======================================================================== + # 7. 权重变化分析 + # ======================================================================== + print("\n[步骤 7] 权重变化分析") + print("-" * 50) + + final_weights = system.weights + print(f"\n初始权重: {[f'{w:.3f}' for w in initial_weights]}") + print(f"最终权重: {[f'{w:.3f}' for w in final_weights]}") + + print("\n权重变化:") + for i, criterion in enumerate(criteria): + change = final_weights[i] - initial_weights[i] + arrow = "↑" if change > 0 else "↓" if change < 0 else "→" + print(f" {criterion:15s}: {initial_weights[i]:.3f} → {final_weights[i]:.3f} " + f"({arrow}{abs(change):.3f})") + + # ======================================================================== + # 8. 经验检索 + # ======================================================================== + print("\n[步骤 8] 相似决策检索") + print("-" * 50) + + similar_decisions = system.experience_store.find_similar_decisions( + {"task": "工厂选址", "region": "华东地区"}, + threshold=0.3 + ) + + print(f"\n找到 {len(similar_decisions)} 个相似决策:") + for i, decision in enumerate(similar_decisions[:3], 1): + print(f" {i}. {decision.decision_id} - " + f"选中: {decision.selected_alternative}, " + f"反馈: {decision.get_average_feedback():.2f}") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/01-foundations/hitl_example.py b/dofile/examples/01-foundations/hitl_example.py new file mode 100644 index 0000000..dfabc61 --- /dev/null +++ b/dofile/examples/01-foundations/hitl_example.py @@ -0,0 +1,824 @@ +""" +人机协同示例 (Human-in-the-Loop Example) +======================================== + +本示例展示如何在空间智能系统中实现人机协同工作模式。 +人机协同 (HITL) 结合人类专家的领域知识和AI的计算能力， +实现更可靠的决策。 + +核心概念: +1. 主动学习 - AI主动请求人类帮助 +2. 交互式决策 - 人机共同完成决策 +3. 反馈收集 - 收集并整合人类反馈 +4. 置信度估计 - AI评估自身确定性 +5. 专业知识注入 - 将专家知识整合到系统中 + +应用场景: +- 空间数据标注与验证 +- 复杂选址决策 +- 应急响应规划 +- 土地利用评估 + +作者: CC4SI 项目组 +""" + +import math +import json +from typing import List, Dict, Tuple, Optional, Any, Callable +from dataclasses import dataclass, field +from enum import Enum +from datetime import datetime +import random + + +# ============================================================================ +# 协同模式枚举 +# ============================================================================ + +class HITLMode(Enum): + """人机协同模式""" + AUTOMATIC = "automatic" # 全自动模式 + ADVISORY = "advisory" # 建议模式 (AI提供建议，人类决策) + INTERACTIVE = "interactive" # 交互模式 (人机共同决策) + SUPERVISED = "supervised" # 监督模式 (人类监督AI) + MANUAL = "manual" # 手动模式 (人类完全控制) + + +class ConfidenceLevel(Enum): + """置信度级别""" + VERY_LOW = "very_low" # 0.0 - 0.3 + LOW = "low" # 0.3 - 0.5 + MEDIUM = "medium" # 0.5 - 0.7 + HIGH = "high" # 0.7 - 0.9 + VERY_HIGH = "very_high" # 0.9 - 1.0 + + +class InteractionType(Enum): + """交互类型""" + CONFIRMATION = "confirmation" # 确认请求 + CLARIFICATION = "clarification" # 澄清请求 + VALIDATION = "validation" # 验证请求 + CORRECTION = "correction" # 纠正请求 + RANKING = "ranking" # 排序请求 + ANNOTATION = "annotation" # 标注请求 + + +# ============================================================================ +# 交互数据结构 +# ============================================================================ + +@dataclass +class AIConfidence: + """AI置信度""" + value: float # 0-1之间的值 + reason: str = "" + metadata: Dict[str, Any] = field(default_factory=dict) + + @property + def level(self) -> ConfidenceLevel: + """获取置信度级别""" + if self.value < 0.3: + return ConfidenceLevel.VERY_LOW + elif self.value < 0.5: + return ConfidenceLevel.LOW + elif self.value < 0.7: + return ConfidenceLevel.MEDIUM + elif self.value < 0.9: + return ConfidenceLevel.HIGH + else: + return ConfidenceLevel.VERY_HIGH + + def __repr__(self) -> str: + return f"Confidence({self.value:.2f}, {self.level.value})" + + +@dataclass +class HumanInput: + """人类输入""" + interaction_type: InteractionType + response: Any + confidence: float = 1.0 # 人类对自己回答的置信度 + timestamp: datetime = field(default_factory=datetime.now) + expert_id: str = "default_expert" + metadata: Dict[str, Any] = field(default_factory=dict) + + +@dataclass +class InteractionRequest: + """交互请求""" + request_id: str + interaction_type: InteractionType + question: str + context: Dict[str, Any] + options: Optional[List[Any]] = None + ai_suggestion: Optional[Any] = None + ai_confidence: Optional[AIConfidence] = None + priority: int = 0 # 优先级 (0=普通, 1=重要, 2=紧急) + deadline: Optional[datetime] = None + metadata: Dict[str, Any] = field(default_factory=dict) + + +# ============================================================================ +# 决策建议 +# ============================================================================ + +@dataclass +class DecisionProposal: + """决策建议""" + proposal_id: str + decision: Any + reasoning: str + confidence: AIConfidence + alternatives: List[Any] = field(default_factory=list) + supporting_evidence: List[str] = field(default_factory=list) + caveats: List[str] = field(default_factory=list) # 警告/注意事项 + requires_human_review: bool = False + timestamp: datetime = field(default_factory=datetime.now) + + def to_dict(self) -> Dict[str, Any]: + """转换为字典""" + return { + "proposal_id": self.proposal_id, + "decision": self.decision, + "reasoning": self.reasoning, + "confidence": self.confidence.value, + "alternatives": self.alternatives, + "supporting_evidence": self.supporting_evidence, + "caveats": self.caveats, + "requires_human_review": self.requires_human_review + } + + +# ============================================================================ +# 人类专家接口 +# ============================================================================ + +class HumanExpert(ABC): + """人类专家抽象接口""" + + def __init__(self, expert_id: str, name: str = "", expertise: List[str] = None): + self.expert_id = expert_id + self.name = name or expert_id + self.expertise = expertise or [] + + @abstractmethod + def respond_to_request(self, request: InteractionRequest) -> HumanInput: + """响应交互请求""" + pass + + def can_handle(self, request: InteractionRequest) -> bool: + """检查是否能处理请求""" + return True + + def get_expertise_summary(self) -> str: + """获取专长摘要""" + return f"{self.name}: {', '.join(self.expertise) if self.expertise else '通用'}" + + +class MockHumanExpert(HumanExpert): + """ + 模拟人类专家 (用于演示) + + 在实际应用中，这会连接到真实的用户界面。 + """ + + def __init__(self, expert_id: str, name: str = "", + expertise: List[str] = None, + response_style: str = "balanced"): + super().__init__(expert_id, name, expertise) + self.response_style = response_style + self.response_log: List[Dict[str, Any]] = [] + + def respond_to_request(self, request: InteractionRequest) -> HumanInput: + """模拟响应请求""" + # 记录请求 + self.response_log.append({ + "request_id": request.request_id, + "type": request.interaction_type.value, + "question": request.question, + "timestamp": datetime.now() + }) + + # 根据不同类型生成响应 + if request.interaction_type == InteractionType.CONFIRMATION: + # 确认请求 - 模拟基于置信度的决策 + if request.ai_confidence and request.ai_confidence.value > 0.7: + # 高置信度时倾向于接受AI建议 + response = "accept" if random.random() > 0.2 else "reject" + else: + # 低置信度时更谨慎 + response = "accept" if random.random() > 0.5 else "reject" + + return HumanInput( + interaction_type=request.interaction_type, + response=response, + confidence=0.8 + ) + + elif request.interaction_type == InteractionType.VALIDATION: + # 验证请求 + is_valid = random.random() > 0.3 # 70%概率验证通过 + return HumanInput( + interaction_type=request.interaction_type, + response=is_valid, + confidence=0.9, + metadata={"comment": "看起来正确" if is_valid else "需要修正"} + ) + + elif request.interaction_type == InteractionType.RANKING: + # 排序请求 + if request.options: + # 随机打乱选项作为人类排序 + shuffled = request.options.copy() + random.shuffle(shuffled) + return HumanInput( + interaction_type=request.interaction_type, + response=shuffled, + confidence=0.7 + ) + + elif request.interaction_type == InteractionType.ANNOTATION: + # 标注请求 + return HumanInput( + interaction_type=request.interaction_type, + response={ + "label": random.choice(["高价值", "中价值", "低价值"]), + "notes": "基于现场评估" + }, + confidence=0.75 + ) + + # 默认响应 + return HumanInput( + interaction_type=request.interaction_type, + response="acknowledged", + confidence=0.5 + ) + + +# ============================================================================ +# 人机协同系统 +# ============================================================================ + +class HITLSystem: + """ + 人机协同系统 + + 管理AI与人类专家之间的交互。 + """ + + def __init__(self, name: str = "HITL系统", + default_mode: HITLMode = HITLMode.INTERACTIVE, + confidence_threshold: float = 0.7): + """ + 初始化HITL系统 + + Args: + name: 系统名称 + default_mode: 默认协同模式 + confidence_threshold: 请求人类帮助的置信度阈值 + """ + self.name = name + self.current_mode = default_mode + self.confidence_threshold = confidence_threshold + + # 注册的专家 + self.experts: Dict[str, HumanExpert] = {} + + # 待处理的请求队列 + self.pending_requests: List[InteractionRequest] = [] + + # 交互历史 + self.interaction_history: List[Dict[str, Any]] = [] + + # 统计信息 + self.stats = { + "total_requests": 0, + "auto_resolved": 0, + "human_resolved": 0, + "human_acceptance_rate": 0.0 + } + + print(f"[{self.name}] 初始化完成") + print(f" 模式: {default_mode.value}") + print(f" 置信度阈值: {confidence_threshold}") + + def register_expert(self, expert: HumanExpert) -> None: + """注册人类专家""" + self.experts[expert.expert_id] = expert + print(f"[专家注册] {expert.get_expertise_summary()}") + + def set_mode(self, mode: HITLMode) -> None: + """设置协同模式""" + self.current_mode = mode + print(f"[模式切换] {mode.value}") + + def make_decision(self, proposal: DecisionProposal, + auto_threshold: float = None) -> Any: + """ + 做出决策 (带人机协同) + + Args: + proposal: AI的决策建议 + auto_threshold: 自动决策的置信度阈值 + + Returns: + 最终决策 + """ + threshold = auto_threshold or self.confidence_threshold + + # 根据模式和置信度决定是否需要人类介入 + needs_human = self._needs_human_intervention(proposal, threshold) + + if not needs_human: + # 自动决策 + self.stats["auto_resolved"] += 1 + self._record_interaction(proposal, None, "automatic") + return proposal.decision + + # 请求人类帮助 + return self._request_human_input(proposal) + + def _needs_human_intervention(self, proposal: DecisionProposal, + threshold: float) -> bool: + """判断是否需要人类介入""" + # 检查强制人工审查标记 + if proposal.requires_human_review: + return True + + # 检查置信度 + if proposal.confidence.value < threshold: + return True + + # 根据模式判断 + if self.current_mode == HITLMode.MANUAL: + return True + elif self.current_mode == HITLMode.SUPERVISED: + return True + elif self.current_mode == HITLMode.AUTOMATIC: + return False + elif self.current_mode == HITLMode.INTERACTIVE: + # 交互模式下，低置信度需要人类 + return proposal.confidence.value < 0.8 + elif self.current_mode == HITLMode.ADVISORY: + # 建议模式下，总是需要人类确认 + return True + + return False + + def _request_human_input(self, proposal: DecisionProposal) -> Any: + """请求人类输入""" + # 创建交互请求 + request = InteractionRequest( + request_id=f"req_{len(self.interaction_history)}", + interaction_type=InteractionType.CONFIRMATION, + question=f"请确认AI建议: {proposal.reasoning}", + context={"proposal_id": proposal.proposal_id}, + options=["accept", "reject", "modify"], + ai_suggestion=proposal.decision, + ai_confidence=proposal.confidence + ) + + self.pending_requests.append(request) + self.stats["total_requests"] += 1 + + # 选择专家 + expert = self._select_expert(request) + if not expert: + print("警告: 没有可用的专家，使用AI建议") + return proposal.decision + + # 获取专家响应 + print(f"\n[人类交互] 向专家 {expert.name} 请求确认...") + print(f" AI建议: {proposal.decision}") + print(f" 置信度: {proposal.confidence}") + print(f" 理由: {proposal.reasoning}") + + human_input = expert.respond_to_request(request) + + print(f" 专家响应: {human_input.response}") + + # 处理响应 + result = self._process_human_response(proposal, human_input) + + # 记录交互 + self._record_interaction(proposal, human_input, "human_assisted") + + # 清理请求 + if request in self.pending_requests: + self.pending_requests.remove(request) + + self.stats["human_resolved"] += 1 + + return result + + def _select_expert(self, request: InteractionRequest) -> Optional[HumanExpert]: + """选择合适的专家""" + if not self.experts: + return None + + # 简单实现: 返回第一个可用的专家 + for expert in self.experts.values(): + if expert.can_handle(request): + return expert + + return None + + def _process_human_response(self, proposal: DecisionProposal, + human_input: HumanInput) -> Any: + """处理人类响应""" + if human_input.interaction_type == InteractionType.CONFIRMATION: + if human_input.response == "accept": + # 接受AI建议 + return proposal.decision + elif human_input.response == "reject": + # 拒绝AI建议，返回次优选项 + if proposal.alternatives: + return proposal.alternatives[0] + return None + elif human_input.response == "modify": + # 需要修改 (简化: 返回原建议) + return proposal.decision + + return human_input.response + + def _record_interaction(self, proposal: DecisionProposal, + human_input: Optional[HumanInput], + resolution_type: str) -> None: + """记录交互""" + record = { + "proposal_id": proposal.proposal_id, + "timestamp": datetime.now(), + "ai_confidence": proposal.confidence.value, + "human_input": human_input.response if human_input else None, + "resolution_type": resolution_type + } + self.interaction_history.append(record) + + def request_annotation(self, item: Any, context: Dict[str, Any] = None) -> Any: + """请求人类标注""" + request = InteractionRequest( + request_id=f"annotate_{len(self.interaction_history)}", + interaction_type=InteractionType.ANNOTATION, + question=f"请对以下项目进行标注: {item}", + context=context or {}, + ai_suggestion=item + ) + + expert = self._select_expert(request) + if not expert: + return None + + return expert.respond_to_request(request) + + def request_validation(self, item: Any, context: Dict[str, Any] = None) -> bool: + """请求人类验证""" + request = InteractionRequest( + request_id=f"validate_{len(self.interaction_history)}", + interaction_type=InteractionType.VALIDATION, + question=f"以下内容是否正确: {item}", + context=context or {}, + ai_suggestion=item + ) + + expert = self._select_expert(request) + if not expert: + return True # 默认有效 + + response = expert.respond_to_request(request) + return response.response if isinstance(response.response, bool) else True + + def get_statistics(self) -> Dict[str, Any]: + """获取统计信息""" + stats = self.stats.copy() + stats["pending_requests"] = len(self.pending_requests) + stats["total_interactions"] = len(self.interaction_history) + + # 计算接受率 + human_interactions = [i for i in self.interaction_history + if i["resolution_type"] == "human_assisted"] + if human_interactions: + accepted = sum(1 for i in human_interactions + if i["human_input"] == "accept") + stats["human_acceptance_rate"] = accepted / len(human_interactions) + + return stats + + def print_statistics(self) -> None: + """打印统计信息""" + stats = self.get_statistics() + + print(f"\n{self.name} 统计信息:") + print("-" * 50) + print(f"总请求数: {stats['total_requests']}") + print(f"自动解决: {stats['auto_resolved']}") + print(f"人类协助: {stats['human_resolved']}") + print(f"待处理请求: {stats['pending_requests']}") + print(f"人类接受率: {stats['human_acceptance_rate']:.2%}") + print("-" * 50) + + +# ============================================================================ +# 空间决策HITL系统 +# ============================================================================ + +class SpatialDecisionHITL(HITLSystem): + """ + 空间决策人机协同系统 + + 专门用于空间决策场景的HITL实现。 + """ + + def __init__(self, confidence_threshold: float = 0.7): + super().__init__( + name="空间决策HITL系统", + default_mode=HITLMode.INTERACTIVE, + confidence_threshold=confidence_threshold + ) + + def analyze_site_suitability(self, site_data: Dict[str, Any]) -> DecisionProposal: + """ + 分析场地适宜性 + + Args: + site_data: 场地数据 + + Returns: + 决策建议 + """ + # 简化的适宜性评分 + score = self._calculate_suitability_score(site_data) + + # 确定置信度 + confidence = self._assess_confidence(site_data, score) + + # 生成建议 + if score > 0.7: + decision = "highly_suitable" + reasoning = f"综合评分 {score:.2f} 较高，适宜开发" + elif score > 0.5: + decision = "moderately_suitable" + reasoning = f"综合评分 {score:.2f} 中等，需谨慎评估" + else: + decision = "not_suitable" + reasoning = f"综合评分 {score:.2f} 较低，不建议开发" + + # 检查注意事项 + caveats = [] + if site_data.get("environmental_risk", 0) > 0.6: + caveats.append("存在环境风险") + if site_data.get("infrastructure_score", 1) < 0.4: + caveats.append("基础设施不足") + + # 低置信度时标记需要人工审查 + requires_review = confidence.value < 0.6 or len(caveats) > 0 + + return DecisionProposal( + proposal_id=f"suitability_{random.randint(1000, 9999)}", + decision=decision, + reasoning=reasoning, + confidence=confidence, + alternatives=["moderately_suitable", "not_suitable"] + if decision != "not_suitable" else ["moderately_suitable", "highly_suitable"], + supporting_evidence=[ + f"评分: {score:.2f}", + f"环境因子: {site_data.get('environmental_score', 0):.2f}", + f"经济因子: {site_data.get('economic_score', 0):.2f}" + ], + caveats=caveats, + requires_human_review=requires_review + ) + + def _calculate_suitability_score(self, site_data: Dict[str, Any]) -> float: + """计算适宜性评分""" + env = site_data.get("environmental_score", 0.5) + econ = site_data.get("economic_score", 0.5) + social = site_data.get("social_score", 0.5) + infra = site_data.get("infrastructure_score", 0.5) + + # 加权平均 + return 0.3 * env + 0.3 * econ + 0.2 * social + 0.2 * infra + + def _assess_confidence(self, site_data: Dict[str, Any], + score: float) -> AIConfidence: + """评估置信度""" + # 检查数据完整性 + has_all_data = all(k in site_data for k in [ + "environmental_score", "economic_score", + "social_score", "infrastructure_score" + ]) + + if not has_all_data: + return AIConfidence( + value=0.4, + reason="数据不完整" + ) + + # 检查数据质量 + data_quality = site_data.get("data_quality", 0.8) + confidence = data_quality * 0.9 + + # 检查是否有冲突因素 + if site_data.get("environmental_risk", 0) > 0.7: + confidence *= 0.7 # 降低置信度 + + return AIConfidence( + value=min(confidence, 0.95), + reason="基于数据质量和完整性评估" + ) + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示人机协同的使用""" + + print("="*70) + print("人机协同示例演示") + print("="*70) + + random.seed(42) + + # ======================================================================== + # 1. 创建HITL系统 + # ======================================================================== + print("\n[步骤 1] 创建人机协同系统") + print("-" * 50) + + hitl_system = SpatialDecisionHITL(confidence_threshold=0.7) + + # 注册专家 + expert1 = MockHumanExpert( + expert_id="expert_001", + name="张工程师", + expertise=["环境影响评估", "基础设施规划"], + response_style="conservative" + ) + expert2 = MockHumanExpert( + expert_id="expert_002", + name="李规划师", + expertise=["经济效益分析", "社会影响评估"], + response_style="balanced" + ) + + hitl_system.register_expert(expert1) + hitl_system.register_expert(expert2) + + # ======================================================================== + # 2. 场景1: 高置信度自动决策 + # ======================================================================== + print("\n[场景 1] 高置信度 - 自动决策") + print("-" * 50) + + site1 = { + "environmental_score": 0.85, + "economic_score": 0.90, + "social_score": 0.88, + "infrastructure_score": 0.92, + "data_quality": 0.95, + "environmental_risk": 0.1 + } + + proposal1 = hitl_system.analyze_site_suitability(site1) + print(f"\nAI分析结果:") + print(f" 建议: {proposal1.decision}") + print(f" 理由: {proposal1.reasoning}") + print(f" 置信度: {proposal1.confidence}") + + decision1 = hitl_system.make_decision(proposal1) + print(f"\n最终决策: {decision1} (自动)") + print(f" → 置信度高，无需人工介入") + + # ======================================================================== + # 3. 场景2: 低置信度请求人类帮助 + # ======================================================================== + print("\n\n[场景 2] 低置信度 - 请求人类确认") + print("-" * 50) + + site2 = { + "environmental_score": 0.45, # 环境评分低 + "economic_score": 0.85, # 但经济评分高 + "social_score": 0.60, + "infrastructure_score": 0.50, + "data_quality": 0.70, + "environmental_risk": 0.65 # 存在环境风险 + } + + proposal2 = hitl_system.analyze_site_suitability(site2) + print(f"\nAI分析结果:") + print(f" 建议: {proposal2.decision}") + print(f" 理由: {proposal2.reasoning}") + print(f" 置信度: {proposal2.confidence}") + print(f" 注意事项: {', '.join(proposal2.caveats)}") + + decision2 = hitl_system.make_decision(proposal2) + print(f"\n最终决策: {decision2} (人工协助)") + print(f" → 置信度低且存在注意事项，请求专家确认") + + # ======================================================================== + # 4. 场景3: 批量决策 + # ======================================================================== + print("\n\n[场景 3] 批量场地评估") + print("-" * 50) + + sites = [] + for i in range(5): + site = { + "environmental_score": random.uniform(0.3, 0.95), + "economic_score": random.uniform(0.3, 0.95), + "social_score": random.uniform(0.3, 0.95), + "infrastructure_score": random.uniform(0.3, 0.95), + "data_quality": random.uniform(0.5, 0.95), + "environmental_risk": random.uniform(0.0, 0.8) + } + sites.append(site) + + results = [] + for i, site in enumerate(sites, 1): + proposal = hitl_system.analyze_site_suitability(site) + decision = hitl_system.make_decision(proposal) + + results.append({ + "site": i, + "decision": decision, + "confidence": proposal.confidence.value, + "auto": proposal.confidence.value >= hitl_system.confidence_threshold + }) + + print("\n批量评估结果:") + print(f"{'场地':<6} {'决策':<20} {'置信度':<10} {'模式':<10}") + print("-" * 50) + for r in results: + mode = "自动" if r["auto"] else "人工" + print(f"{r['site']:<6} {r['decision']:<20} {r['confidence']:<10.2f} {mode:<10}") + + # ======================================================================== + # 5. 场景4: 数据标注 + # ======================================================================== + print("\n\n[场景 4] 数据标注") + print("-" * 50) + + unlabeled_items = [ + {"coordinates": (120.5, 30.2), "features": "residential"}, + {"coordinates": (121.0, 30.5), "features": "commercial"}, + {"coordinates": (120.8, 30.0), "features": "industrial"} + ] + + for item in unlabeled_items: + annotation = hitl_system.request_annotation( + item, + context={"task": "land_use_classification"} + ) + if annotation: + print(f"\n标注 {item['features']}:") + print(f" 标签: {annotation.response.get('label')}") + print(f" 备注: {annotation.response.get('notes')}") + + # ======================================================================== + # 6. 场景5: 模式切换 + # ======================================================================== + print("\n\n[场景 5] 模式切换对比") + print("-" * 50) + + test_site = { + "environmental_score": 0.70, + "economic_score": 0.75, + "social_score": 0.68, + "infrastructure_score": 0.72, + "data_quality": 0.85, + "environmental_risk": 0.3 + } + + proposal = hitl_system.analyze_site_suitability(test_site) + print(f"\nAI分析: 置信度 = {proposal.confidence.value:.2f}") + + # 尝试不同模式 + for mode in [HITLMode.AUTOMATIC, HITLMode.INTERACTIVE, HITLMode.MANUAL]: + hitl_system.set_mode(mode) + decision = hitl_system.make_decision(proposal) + mode_name = { + HITLMode.AUTOMATIC: "全自动", + HITLMode.INTERACTIVE: "交互式", + HITLMode.MANUAL: "手动" + }[mode] + print(f" {mode_name}: {decision}") + + # 恢复默认模式 + hitl_system.set_mode(HITLMode.INTERACTIVE) + + # ======================================================================== + # 7. 统计信息 + # ======================================================================== + print("\n\n[步骤 7] 系统统计") + print("-" * 50) + hitl_system.print_statistics() + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/01-foundations/modular_system.py b/dofile/examples/01-foundations/modular_system.py new file mode 100644 index 0000000..695f677 --- /dev/null +++ b/dofile/examples/01-foundations/modular_system.py @@ -0,0 +1,684 @@ +""" +模块化系统示例 (Modular System Example) +======================================== + +本示例展示了空间智能系统的模块化设计原则。 +模块化是构建可维护、可扩展系统的基础。 + +核心概念: +1. 关注点分离 - 每个模块负责特定功能 +2. 接口设计 - 定义清晰的模块间通信协议 +3. 依赖注入 - 降低模块间耦合 +4. 插件架构 - 支持动态扩展功能 + +作者: CC4SI 项目组 +""" + +from abc import ABC, abstractmethod +from typing import Dict, List, Any, Optional, Callable +from dataclasses import dataclass, field +from enum import Enum +import json + + +# ============================================================================ +# 模块接口定义 (抽象基类) +# ============================================================================ + +class ModuleType(Enum): + """模块类型枚举""" + DATA_LOADER = "data_loader" + DATA_PROCESSOR = "data_processor" + ANALYZER = "analyzer" + VISUALIZER = "visualizer" + EXPORTER = "exporter" + + +class ModuleStatus(Enum): + """模块状态枚举""" + IDLE = "idle" + INITIALIZING = "initializing" + READY = "ready" + RUNNING = "running" + ERROR = "error" + + +@dataclass +class ModuleMetadata: + """模块元数据""" + name: str + version: str + module_type: ModuleType + description: str = "" + dependencies: List[str] = field(default_factory=list) + author: str = "" + config_schema: Dict[str, Any] = field(default_factory=dict) + + +class IModule(ABC): + """ + 模块接口 - 所有模块必须实现此接口 + + 这是一个抽象基类，定义了所有模块必须遵循的契约。 + """ + + def __init__(self, config: Dict[str, Any] = None): + """ + 初始化模块 + + Args: + config: 模块配置字典 + """ + self.config = config or {} + self.status = ModuleStatus.IDLE + self._context: Optional['ModuleContext'] = None + + @property + @abstractmethod + def metadata(self) -> ModuleMetadata: + """返回模块元数据""" + pass + + @abstractmethod + def initialize(self, context: 'ModuleContext') -> bool: + """ + 初始化模块 + + Args: + context: 模块上下文，提供对系统资源的访问 + + Returns: + 初始化是否成功 + """ + pass + + @abstractmethod + def execute(self, input_data: Any) -> Any: + """ + 执行模块功能 + + Args: + input_data: 输入数据 + + Returns: + 处理结果 + """ + pass + + @abstractmethod + def shutdown(self) -> None: + """关闭模块，释放资源""" + pass + + def get_config(self, key: str, default: Any = None) -> Any: + """获取配置值""" + return self.config.get(key, default) + + def set_config(self, key: str, value: Any) -> None: + """设置配置值""" + self.config[key] = value + + +# ============================================================================ +# 模块上下文 - 提供模块间通信 +# ============================================================================ + +class ModuleContext: + """ + 模块上下文 + + 提供模块间通信和资源共享机制，实现松耦合设计。 + """ + + def __init__(self): + self._modules: Dict[str, IModule] = {} + self._shared_data: Dict[str, Any] = {} + self._event_handlers: Dict[str, List[Callable]] = {} + + def register_module(self, name: str, module: IModule) -> bool: + """注册模块""" + if name in self._modules: + print(f"警告: 模块 '{name}' 已存在，将被覆盖") + self._modules[name] = module + print(f"模块 '{name}' 已注册 (类型: {module.metadata.module_type.value})") + return True + + def get_module(self, name: str) -> Optional[IModule]: + """获取模块实例""" + return self._modules.get(name) + + def has_module(self, name: str) -> bool: + """检查模块是否存在""" + return name in self._modules + + def set_shared_data(self, key: str, value: Any) -> None: + """设置共享数据""" + self._shared_data[key] = value + + def get_shared_data(self, key: str, default: Any = None) -> Any: + """获取共享数据""" + return self._shared_data.get(key, default) + + def subscribe_event(self, event_name: str, handler: Callable) -> None: + """订阅事件""" + if event_name not in self._event_handlers: + self._event_handlers[event_name] = [] + self._event_handlers[event_name].append(handler) + + def publish_event(self, event_name: str, *args, **kwargs) -> None: + """发布事件""" + if event_name in self._event_handlers: + for handler in self._event_handlers[event_name]: + handler(*args, **kwargs) + + +# ============================================================================ +# 模块基类 - 提供通用功能实现 +# ============================================================================ + +class BaseModule(IModule): + """ + 模块基类 + + 提供IModule接口的默认实现，子类只需实现特定功能。 + """ + + def __init__(self, config: Dict[str, Any] = None): + super().__init__(config) + self._metadata: Optional[ModuleMetadata] = None + + @property + def metadata(self) -> ModuleMetadata: + if self._metadata is None: + raise NotImplementedError("子类必须设置 _metadata") + return self._metadata + + def initialize(self, context: ModuleContext) -> bool: + """默认初始化实现""" + self._context = context + self.status = ModuleStatus.INITIALIZING + + # 检查依赖 + for dep in self.metadata.dependencies: + if not context.has_module(dep): + print(f"错误: 依赖模块 '{dep}' 不存在") + self.status = ModuleStatus.ERROR + return False + + self.status = ModuleStatus.READY + print(f"模块 '{self.metadata.name}' 初始化完成") + return True + + def shutdown(self) -> None: + """默认关闭实现""" + self.status = ModuleStatus.IDLE + print(f"模块 '{self.metadata.name}' 已关闭") + + +# ============================================================================ +# 具体模块实现 +# ============================================================================ + +class CSVDataLoaderModule(BaseModule): + """ + CSV 数据加载模块 + + 负责从CSV文件加载空间数据。 + """ + + def __init__(self, config: Dict[str, Any] = None): + super().__init__(config) + self._metadata = ModuleMetadata( + name="csv_data_loader", + version="1.0.0", + module_type=ModuleType.DATA_LOADER, + description="从CSV文件加载空间数据", + author="CC4SI" + ) + self._data: List[Dict[str, Any]] = [] + + def execute(self, input_data: Any) -> Any: + """ + 执行数据加载 + + Args: + input_data: 文件路径或模拟数据 + + Returns: + 加载的数据列表 + """ + self.status = ModuleStatus.RUNNING + + if isinstance(input_data, str): + # 实际场景中应从文件读取 + print(f"从文件 '{input_data}' 加载数据...") + # 模拟加载 + self._data = self._load_sample_data() + elif isinstance(input_data, list): + self._data = input_data + else: + self._data = self._load_sample_data() + + # 将数据存入共享上下文 + if self._context: + self._context.set_shared_data("raw_data", self._data) + self._context.publish_event("data_loaded", len(self._data)) + + self.status = ModuleStatus.READY + return self._data + + def _load_sample_data(self) -> List[Dict[str, Any]]: + """加载示例数据""" + return [ + {"id": 1, "x": 10, "y": 20, "value": 100, "type": "A"}, + {"id": 2, "x": 30, "y": 40, "value": 200, "type": "B"}, + {"id": 3, "x": 50, "y": 60, "value": 150, "type": "A"}, + {"id": 4, "x": 70, "y": 80, "value": 300, "type": "C"}, + {"id": 5, "x": 90, "y": 100, "value": 250, "type": "B"}, + ] + + +class DataValidationModule(BaseModule): + """ + 数据验证模块 + + 负责验证数据质量和完整性。 + """ + + def __init__(self, config: Dict[str, Any] = None): + super().__init__(config) + self._metadata = ModuleMetadata( + name="data_validator", + version="1.0.0", + module_type=ModuleType.DATA_PROCESSOR, + description="验证数据质量和完整性", + dependencies=["csv_data_loader"], + author="CC4SI" + ) + self.validation_rules: List[Callable] = [] + + def add_validation_rule(self, rule: Callable[[Dict], bool], name: str = ""): + """添加验证规则""" + self.validation_rules.append(rule) + if name: + print(f"添加验证规则: {name}") + + def execute(self, input_data: Any) -> Any: + """ + 执行数据验证 + + Args: + input_data: 待验证的数据 + + Returns: + 验证结果 + """ + self.status = ModuleStatus.RUNNING + + if not isinstance(input_data, list): + return {"valid": False, "errors": ["输入数据格式错误"]} + + errors = [] + warnings = [] + + for i, item in enumerate(input_data): + # 检查必需字段 + if "id" not in item: + errors.append(f"第 {i} 项缺少 'id' 字段") + if "x" not in item or "y" not in item: + errors.append(f"第 {i} 项缺少坐标字段") + + # 应用自定义验证规则 + for rule in self.validation_rules: + try: + if not rule(item): + warnings.append(f"第 {i} 项未通过自定义规则验证") + except Exception as e: + errors.append(f"第 {i} 项验证时出错: {e}") + + result = { + "valid": len(errors) == 0, + "total": len(input_data), + "errors": errors, + "warnings": warnings + } + + if self._context: + self._context.set_shared_data("validation_result", result) + self._context.publish_event("data_validated", result) + + self.status = ModuleStatus.READY + return result + + +class StatisticsAnalyzerModule(BaseModule): + """ + 统计分析模块 + + 负责计算数据的统计指标。 + """ + + def __init__(self, config: Dict[str, Any] = None): + super().__init__(config) + self._metadata = ModuleMetadata( + name="statistics_analyzer", + version="1.0.0", + module_type=ModuleType.ANALYZER, + description="计算数据统计指标", + dependencies=["csv_data_loader"], + author="CC4SI" + ) + + def execute(self, input_data: Any) -> Any: + """ + 执行统计分析 + + Args: + input_data: 待分析的数据 + + Returns: + 统计结果 + """ + self.status = ModuleStatus.RUNNING + + if not isinstance(input_data, list) or len(input_data) == 0: + return {"error": "没有可分析的数据"} + + # 提取数值字段 + values = [item.get("value", 0) for item in input_data if "value" in item] + + if not values: + return {"error": "没有找到可分析的数值"} + + import statistics + + result = { + "count": len(values), + "mean": statistics.mean(values), + "median": statistics.median(values), + "stdev": statistics.stdev(values) if len(values) > 1 else 0, + "min": min(values), + "max": max(values), + "sum": sum(values) + } + + if self._context: + self._context.set_shared_data("statistics", result) + self._context.publish_event("analysis_complete", result) + + self.status = ModuleStatus.READY + return result + + +class ReportExporterModule(BaseModule): + """ + 报告导出模块 + + 负责生成分析报告。 + """ + + def __init__(self, config: Dict[str, Any] = None): + super().__init__(config) + self._metadata = ModuleMetadata( + name="report_exporter", + version="1.0.0", + module_type=ModuleType.EXPORTER, + description="生成分析报告", + dependencies=["statistics_analyzer"], + author="CC4SI" + ) + + def execute(self, input_data: Any) -> Any: + """ + 生成报告 + + Args: + input_data: 统计结果或其他数据 + + Returns: + 报告字符串 + """ + self.status = ModuleStatus.RUNNING + + report_lines = [ + "=" * 60, + "空间数据分析报告", + "=" * 60, + "" + ] + + # 从上下文获取数据 + if self._context: + validation = self._context.get_shared_data("validation_result") + statistics = self._context.get_shared_data("statistics") + + if validation: + report_lines.extend([ + "数据验证结果:", + f" 总数: {validation.get('total', 0)}", + f" 有效: {validation.get('valid', False)}", + f" 错误数: {len(validation.get('errors', []))}", + "" + ]) + + if statistics: + report_lines.extend([ + "统计分析结果:", + f" 样本数: {statistics.get('count', 0)}", + f" 均值: {statistics.get('mean', 0):.2f}", + f" 中位数: {statistics.get('median', 0):.2f}", + f" 标准差: {statistics.get('stdev', 0):.2f}", + f" 最小值: {statistics.get('min', 0)}", + f" 最大值: {statistics.get('max', 0)}", + "" + ]) + + report_lines.append("=" * 60) + + report = "\n".join(report_lines) + + if self._context: + self._context.publish_event("report_generated", report) + + self.status = ModuleStatus.READY + return report + + +# ============================================================================ +# 模块系统管理器 +# ============================================================================ + +class ModularSystem: + """ + 模块化系统管理器 + + 负责管理模块的生命周期和模块间通信。 + """ + + def __init__(self, name: str = "模块化空间智能系统"): + self.name = name + self.context = ModuleContext() + self._pipeline: List[str] = [] # 处理流程 + + def register_module(self, module: IModule, alias: str = None) -> bool: + """ + 注册模块到系统 + + Args: + module: 模块实例 + alias: 模块别名 (可选) + + Returns: + 是否注册成功 + """ + name = alias or module.metadata.name + return self.context.register_module(name, module) + + def initialize_all(self) -> bool: + """初始化所有模块""" + print(f"\n初始化 {self.name}...") + + success = True + for name, module in self.context._modules.items(): + if not module.initialize(self.context): + print(f"模块 '{name}' 初始化失败") + success = False + + return success + + def define_pipeline(self, module_names: List[str]) -> None: + """ + 定义处理流程 + + Args: + module_names: 按顺序执行的模块名称列表 + """ + self._pipeline = module_names + print(f"定义处理流程: {' -> '.join(module_names)}") + + def execute(self, input_data: Any = None) -> Any: + """ + 执行处理流程 + + Args: + input_data: 输入数据 + + Returns: + 最终输出结果 + """ + if not self._pipeline: + print("错误: 没有定义处理流程") + return None + + print(f"\n执行处理流程...") + current_data = input_data + + for module_name in self._pipeline: + module = self.context.get_module(module_name) + if not module: + print(f"错误: 找不到模块 '{module_name}'") + return None + + print(f" -> 执行模块: {module.metadata.name}") + current_data = module.execute(current_data) + + # 如果模块返回错误，终止流程 + if isinstance(current_data, dict) and current_data.get("error"): + print(f" 模块 '{module_name}' 返回错误: {current_data['error']}") + return current_data + + return current_data + + def shutdown_all(self) -> None: + """关闭所有模块""" + print(f"\n关闭 {self.name}...") + for module in self.context._modules.values(): + module.shutdown() + + def print_system_info(self) -> None: + """打印系统信息""" + print(f"\n{'='*60}") + print(f"系统: {self.name}") + print(f"{'='*60}") + print(f"已注册模块数: {len(self.context._modules)}") + + for name, module in self.context._modules.items(): + print(f" - {name:20s} [{module.metadata.module_type.value:15s}] {module.metadata.name}") + if module.metadata.dependencies: + print(f" 依赖: {', '.join(module.metadata.dependencies)}") + + if self._pipeline: + print(f"\n处理流程: {' -> '.join(self._pipeline)}") + else: + print(f"\n处理流程: 未定义") + + print(f"{'='*60}\n") + + +# ============================================================================ +# 事件处理示例 +# ============================================================================ + +def setup_event_handlers(system: ModularSystem): + """设置事件处理器""" + + def on_data_loaded(count): + print(f" [事件] 数据加载完成，共 {count} 条记录") + + def on_data_validated(result): + status = "通过" if result.get("valid") else "失败" + print(f" [事件] 数据验证{status}，错误: {len(result.get('errors', []))}") + + def on_analysis_complete(result): + print(f" [事件] 分析完成，均值: {result.get('mean', 0):.2f}") + + system.context.subscribe_event("data_loaded", on_data_loaded) + system.context.subscribe_event("data_validated", on_data_validated) + system.context.subscribe_event("analysis_complete", on_analysis_complete) + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示模块化系统的使用""" + + print("="*70) + print("模块化系统示例演示") + print("="*70) + + # 1. 创建系统 + print("\n[步骤 1] 创建模块化系统") + system = ModularSystem("空间数据分析系统") + + # 2. 注册模块 + print("\n[步骤 2] 注册模块") + system.register_module(CSVDataLoaderModule()) + system.register_module(DataValidationModule()) + system.register_module(StatisticsAnalyzerModule()) + system.register_module(ReportExporterModule()) + + # 3. 设置事件处理 + print("\n[步骤 3] 设置事件处理") + setup_event_handlers(system) + + # 4. 初始化所有模块 + print("\n[步骤 4] 初始化模块") + if not system.initialize_all(): + print("初始化失败，退出") + return + + # 5. 定义处理流程 + print("\n[步骤 5] 定义处理流程") + system.define_pipeline([ + "csv_data_loader", + "data_validator", + "statistics_analyzer", + "report_exporter" + ]) + + # 6. 打印系统信息 + print("\n[步骤 6] 系统信息") + system.print_system_info() + + # 7. 执行处理流程 + print("\n[步骤 7] 执行处理流程") + result = system.execute("sample_data.csv") + + # 8. 输出结果 + print("\n[步骤 8] 最终结果") + if isinstance(result, str): + print(result) + + # 9. 清理 + print("\n[步骤 9] 清理资源") + system.shutdown_all() + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/01-foundations/probability_example.py b/dofile/examples/01-foundations/probability_example.py new file mode 100644 index 0000000..c5ec476 --- /dev/null +++ b/dofile/examples/01-foundations/probability_example.py @@ -0,0 +1,938 @@ +""" +概率与不确定性示例 (Probability and Uncertainty Example) +======================================================== + +本示例展示如何在空间智能系统中处理概率和不确定性。 +在空间决策中，不确定性是普遍存在的，理解和管理不确定性 +对于做出可靠的决策至关重要。 + +核心概念: +1. 概率分布 - 描述随机变量的可能取值及其概率 +2. 贝叶斯推理 - 基于新证据更新信念 +3. 蒙特卡洛模拟 - 通过随机采样评估不确定性 +4. 置信区间 - 估计结果的范围 +5. 敏感性分析 - 评估输入变化对输出的影响 + +应用场景: +- 空间插值的不确定性量化 +- 多准则决策的敏感性分析 +- 风险评估与概率预测 +- 传感器数据的可靠性分析 + +作者: CC4SI 项目组 +""" + +import math +import random +from typing import List, Dict, Tuple, Optional, Callable, Any +from dataclasses import dataclass, field +from abc import ABC, abstractmethod +from enum import Enum +import statistics + + +# ============================================================================ +# 概率分布基础类 +# ============================================================================ + +class DistributionType(Enum): + """分布类型枚举""" + NORMAL = "normal" # 正态分布 + UNIFORM = "uniform" # 均匀分布 + TRIANGULAR = "triangular" # 三角分布 + EXPONENTIAL = "exponential" # 指数分布 + BETA = "beta" # Beta分布 + + +class ProbabilityDistribution(ABC): + """概率分布抽象基类""" + + def __init__(self, name: str = ""): + self.name = name + + @abstractmethod + def sample(self) -> float: + """从分布中采样一个值""" + pass + + @abstractmethod + def mean(self) -> float: + """计算期望值""" + pass + + @abstractmethod + def std(self) -> float: + """计算标准差""" + pass + + @abstractmethod + def pdf(self, x: float) -> float: + """概率密度函数""" + pass + + def cdf(self, x: float) -> float: + """累积分布函数 (近似计算)""" + # 使用蒙特卡洛积分近似 + n_samples = 10000 + count = sum(1 for _ in range(n_samples) if self.sample() <= x) + return count / n_samples + + def confidence_interval(self, confidence: float = 0.95) -> Tuple[float, float]: + """计算置信区间""" + n_samples = 10000 + samples = [self.sample() for _ in range(n_samples)] + alpha = 1 - confidence + lower = quantile(samples, alpha / 2) + upper = quantile(samples, 1 - alpha / 2) + return lower, upper + + +def quantile(data: List[float], q: float) -> float: + """计算分位数""" + sorted_data = sorted(data) + index = int(q * len(sorted_data)) + return sorted_data[min(index, len(sorted_data) - 1)] + + +# ============================================================================ +# 具体概率分布实现 +# ============================================================================ + +class NormalDistribution(ProbabilityDistribution): + """ + 正态分布 (高斯分布) + + 最常用的连续概率分布，由均值和标准差参数化。 + """ + + def __init__(self, mu: float = 0.0, sigma: float = 1.0, name: str = ""): + super().__init__(name) + self.mu = mu # 均值 + self.sigma = sigma # 标准差 + if sigma <= 0: + raise ValueError("标准差必须为正数") + + def sample(self) -> float: + """使用 Box-Muller 变换生成正态分布随机数""" + u1 = random.random() + u2 = random.random() + while u1 == 0: # 避免log(0) + u1 = random.random() + z0 = math.sqrt(-2 * math.log(u1)) * math.cos(2 * math.pi * u2) + return self.mu + self.sigma * z0 + + def mean(self) -> float: + return self.mu + + def std(self) -> float: + return self.sigma + + def pdf(self, x: float) -> float: + """正态分布概率密度函数""" + coeff = 1 / (self.sigma * math.sqrt(2 * math.pi)) + exponent = -0.5 * ((x - self.mu) / self.sigma) ** 2 + return coeff * math.exp(exponent) + + def __repr__(self) -> str: + return f"Normal(μ={self.mu}, σ={self.sigma})" + + +class UniformDistribution(ProbabilityDistribution): + """ + 均匀分布 + + 在指定范围内等概率取值。 + """ + + def __init__(self, a: float = 0.0, b: float = 1.0, name: str = ""): + super().__init__(name) + self.a = a # 下界 + self.b = b # 上界 + if a >= b: + raise ValueError("下界必须小于上界") + + def sample(self) -> float: + return self.a + (self.b - self.a) * random.random() + + def mean(self) -> float: + return (self.a + self.b) / 2 + + def std(self) -> float: + return (self.b - self.a) / math.sqrt(12) + + def pdf(self, x: float) -> float: + if self.a <= x <= self.b: + return 1 / (self.b - self.a) + return 0.0 + + def __repr__(self) -> str: + return f"Uniform({self.a}, {self.b})" + + +class TriangularDistribution(ProbabilityDistribution): + """ + 三角分布 + + 由最小值、最大值和众数定义的分布，常用于 + 当只知道边界和最可能值时建模不确定性。 + """ + + def __init__(self, a: float, b: float, c: float, name: str = ""): + super().__init__(name) + self.a = a # 最小值 + self.b = b # 最大值 + self.c = c # 众数 (最可能值) + if not (a <= c <= b): + raise ValueError("必须满足 a <= c <= b") + + def sample(self) -> float: + u = random.random() + fc = (self.c - self.a) / (self.b - self.a) + if u < fc: + return self.a + math.sqrt(u * (self.b - self.a) * (self.c - self.a)) + else: + return self.b - math.sqrt((1 - u) * (self.b - self.a) * (self.b - self.c)) + + def mean(self) -> float: + return (self.a + self.b + self.c) / 3 + + def std(self) -> float: + numerator = (self.a**2 + self.b**2 + self.c**2 + - self.a * self.b - self.a * self.c - self.b * self.c) + return math.sqrt(numerator / 18) + + def pdf(self, x: float) -> float: + if x < self.a or x > self.b: + return 0.0 + if x < self.c: + return 2 * (x - self.a) / ((self.b - self.a) * (self.c - self.a)) + else: + return 2 * (self.b - x) / ((self.b - self.a) * (self.b - self.c)) + + def __repr__(self) -> str: + return f"Triangular({self.a}, {self.c}, {self.b})" + + +# ============================================================================ +# 贝叶斯推理 +# ============================================================================ + +@dataclass +class BayesianBelief: + """ + 贝叶斯信念状态 + + 表示对某个假设的信念，包含先验、似然和后验。 + """ + hypothesis: str + prior: float # 先验概率 P(H) + likelihood: float # 似然 P(E|H) + evidence: Optional[float] = None # 证据概率 P(E) + posterior: Optional[float] = None # 后验概率 P(H|E) + + def update(self, evidence_prob: float = None) -> float: + """ + 更新后验概率 + + Args: + evidence_prob: P(E)，如果None则使用归一化 + + Returns: + 后验概率 + """ + # P(H|E) = P(E|H) * P(H) / P(E) + numerator = self.likelihood * self.prior + + if evidence_prob is not None: + self.evidence = evidence_prob + self.posterior = numerator / evidence_prob + else: + # 假设有多个假设，需要归一化 + self.posterior = numerator # 简化版本 + + return self.posterior + + +class BayesianUpdater: + """ + 贝叶斯更新器 + + 管理多个假设的贝叶斯更新。 + """ + + def __init__(self, hypotheses: List[str]): + """ + 初始化贝叶斯更新器 + + Args: + hypotheses: 假设列表 + """ + # 初始化先验概率 (均匀分布) + prior = 1.0 / len(hypotheses) + self.beliefs: Dict[str, BayesianBelief] = { + h: BayesianBelief(hypothesis=h, prior=prior, likelihood=1.0) + for h in hypotheses + } + + def set_prior(self, hypothesis: str, prior: float) -> None: + """设置先验概率""" + if hypothesis in self.beliefs: + self.beliefs[hypothesis].prior = prior + + def update_with_evidence(self, likelihoods: Dict[str, float]) -> None: + """ + 用证据更新所有假设 + + Args: + likelihoods: 每个假设的似然 P(E|H) + """ + # 更新似然 + for h, likelihood in likelihoods.items(): + if h in self.beliefs: + self.beliefs[h].likelihood = likelihood + + # 计算证据概率 (归一化常数) + evidence = sum( + b.likelihood * b.prior + for b in self.beliefs.values() + ) + + # 更新后验 + for belief in self.beliefs.values(): + belief.update(evidence) + + def get_posteriors(self) -> Dict[str, float]: + """获取所有后验概率""" + return { + h: b.posterior or b.prior + for h, b in self.beliefs.items() + } + + def get_most_likely(self) -> Tuple[str, float]: + """获取最可能的假设""" + posteriors = self.get_posteriors() + return max(posteriors.items(), key=lambda x: x[1]) + + def print_beliefs(self) -> None: + """打印信念状态""" + print("\n贝叶斯信念状态:") + print("-" * 60) + print(f"{'假设':<20} {'先验':<12} {'似然':<12} {'后验':<12}") + print("-" * 60) + for belief in self.beliefs.values(): + posterior = belief.posterior if belief.posterior is not None else belief.prior + print(f"{belief.hypothesis:<20} {belief.prior:<12.4f} " + f"{belief.likelihood:<12.4f} {posterior:<12.4f}") + print("-" * 60) + + +# ============================================================================ +# 蒙特卡洛模拟 +# ============================================================================ + +@dataclass +class SimulationResult: + """模拟结果""" + samples: List[float] = field(default_factory=list) + mean: float = 0.0 + std: float = 0.0 + min: float = 0.0 + max: float = 0.0 + median: float = 0.0 + confidence_interval: Tuple[float, float] = (0.0, 0.0) + percentiles: Dict[float, float] = field(default_factory=dict) + + def calculate_statistics(self, confidence: float = 0.95) -> None: + """计算统计量""" + if not self.samples: + return + + self.mean = statistics.mean(self.samples) + self.std = statistics.stdev(self.samples) if len(self.samples) > 1 else 0 + self.min = min(self.samples) + self.max = max(self.samples) + self.median = statistics.median(self.samples) + + # 置信区间 + alpha = 1 - confidence + sorted_samples = sorted(self.samples) + n = len(sorted_samples) + self.confidence_interval = ( + sorted_samples[int(alpha / 2 * n)], + sorted_samples[int((1 - alpha / 2) * n)] + ) + + # 常用百分位数 + for p in [0.01, 0.05, 0.10, 0.25, 0.50, 0.75, 0.90, 0.95, 0.99]: + self.percentiles[p] = sorted_samples[int(p * n)] + + def print_summary(self) -> None: + """打印结果摘要""" + print(f"\n蒙特卡洛模拟结果 (n={len(self.samples)}):") + print("-" * 50) + print(f"均值: {self.mean:.4f}") + print(f"中位数: {self.median:.4f}") + print(f"标准差: {self.std:.4f}") + print(f"范围: [{self.min:.4f}, {self.max:.4f}]") + print(f"95% 置信区间: [{self.confidence_interval[0]:.4f}, " + f"{self.confidence_interval[1]:.4f}]") + print(f"\n百分位数:") + for p, value in sorted(self.percentiles.items()): + print(f" {p*100:>5.0f}%: {value:.4f}") + print("-" * 50) + + +class MonteCarloSimulator: + """ + 蒙特卡洛模拟器 + + 通过随机采样评估不确定性。 + """ + + def __init__(self, seed: int = None): + """初始化模拟器""" + if seed is not None: + random.seed(seed) + + def simulate(self, model: Callable[[], float], + n_runs: int = 10000) -> SimulationResult: + """ + 运行模拟 + + Args: + model: 返回模拟值的函数 + n_runs: 运行次数 + + Returns: + 模拟结果 + """ + samples = [model() for _ in range(n_runs)] + result = SimulationResult(samples=samples) + result.calculate_statistics() + return result + + def simulate_with_inputs(self, + model: Callable[[Dict[str, float]], float], + input_distributions: Dict[str, ProbabilityDistribution], + n_runs: int = 10000) -> SimulationResult: + """ + 使用输入分布运行模拟 + + Args: + model: 接受输入字典的函数 + input_distributions: 输入变量到其分布的映射 + n_runs: 运行次数 + + Returns: + 模拟结果 + """ + samples = [] + for _ in range(n_runs): + inputs = { + name: dist.sample() + for name, dist in input_distributions.items() + } + samples.append(model(inputs)) + + result = SimulationResult(samples=samples) + result.calculate_statistics() + return result + + +# ============================================================================ +# 敏感性分析 +# ============================================================================ + +@dataclass +class SensitivityResult: + """敏感性分析结果""" + sensitivity_coefficients: Dict[str, float] = field(default_factory=dict) + rankings: List[Tuple[str, float]] = field(default_factory=list) + tornado_data: Dict[str, Tuple[float, float]] = field(default_factory=dict) + + def print_summary(self) -> None: + """打印敏感性分析摘要""" + print("\n敏感性分析结果:") + print("-" * 50) + print("排名 | 变量 | 敏感性系数") + print("-" * 50) + for i, (var, coef) in enumerate(self.rankings, 1): + print(f"{i:4d} | {var:<11} | {coef:10.4f}") + print("-" * 50) + + +class SensitivityAnalyzer: + """ + 敏感性分析器 + + 评估输入变化对输出的影响。 + """ + + def __init__(self): + self.model: Optional[Callable] = None + self.base_inputs: Optional[Dict[str, float]] = None + + def simple_sensitivity(self, + model: Callable[[Dict[str, float]], float], + base_inputs: Dict[str, float], + variations: Dict[str, float] = None) -> SensitivityResult: + """ + 简单敏感性分析 (单因素) + + Args: + model: 待分析模型 + base_inputs: 基准输入值 + variations: 各变量的变化幅度 (默认 ±10%) + + Returns: + 敏感性结果 + """ + if variations is None: + variations = {k: 0.1 for k in base_inputs.keys()} + + # 计算基准输出 + base_output = model(base_inputs) + + # 计算敏感性系数 + coefficients = {} + tornado_data = {} + + for var, variation in variations.items(): + original_value = base_inputs[var] + + # 正向变化 + base_inputs[var] = original_value * (1 + variation) + output_plus = model(base_inputs) + + # 负向变化 + base_inputs[var] = original_value * (1 - variation) + output_minus = model(base_inputs) + + # 恢复原值 + base_inputs[var] = original_value + + # 计算敏感性系数 (归一化) + delta_output = output_plus - output_minus + delta_input = 2 * variation * original_value + coefficient = delta_output / delta_input if delta_input != 0 else 0 + + coefficients[var] = coefficient + tornado_data[var] = (output_minus, output_plus) + + # 排名 + rankings = sorted(coefficients.items(), key=lambda x: abs(x[1]), reverse=True) + + return SensitivityResult( + sensitivity_coefficients=coefficients, + rankings=rankings, + tornado_data=tornado_data + ) + + def regression_sensitivity(self, + model: Callable[[Dict[str, float]], float], + input_distributions: Dict[str, ProbabilityDistribution], + n_samples: int = 1000) -> SensitivityResult: + """ + 基于回归的敏感性分析 + + Args: + model: 待分析模型 + input_distributions: 输入分布 + n_samples: 样本数量 + + Returns: + 敏感性结果 + """ + simulator = MonteCarloSimulator() + + # 生成样本 + input_samples = [] + output_samples = [] + + for _ in range(n_samples): + inputs = { + name: dist.sample() + for name, dist in input_distributions.items() + } + input_samples.append(inputs) + output_samples.append(model(inputs)) + + # 计算标准化回归系数 (SRC) + # SRC = beta * (std_x / std_y) + + import statistics + + std_y = statistics.stdev(output_samples) + coefficients = {} + + for var in input_distributions.keys(): + x_values = [s[var] for s in input_samples] + std_x = statistics.stdev(x_values) + + # 计算相关系数 + mean_x = statistics.mean(x_values) + mean_y = statistics.mean(output_samples) + + numerator = sum((x - mean_x) * (y - mean_y) + for x, y in zip(x_values, output_samples)) + denominator = math.sqrt( + sum((x - mean_x)**2 for x in x_values) * + sum((y - mean_y)**2 for y in output_samples) + ) + + correlation = numerator / denominator if denominator != 0 else 0 + src = correlation * (std_x / std_y) if std_y > 0 else 0 + + coefficients[var] = src + + rankings = sorted(coefficients.items(), key=lambda x: abs(x[1]), reverse=True) + + return SensitivityResult( + sensitivity_coefficients=coefficients, + rankings=rankings + ) + + +# ============================================================================ +# 空间概率应用示例 +# ============================================================================ + +class SpatialProbabilityModel: + """ + 空间概率模型 + + 将概率理论应用于空间问题。 + """ + + @staticmethod + def uncertain_distance(point1: Tuple[float, float], + point2: Tuple[float, float], + distance_error_std: float = 5.0) -> NormalDistribution: + """ + 带不确定性的距离计算 + + Args: + point1: 第一个点 (x, y) + point2: 第二个点 (x, y) + distance_error_std: 距离测量误差的标准差 + + Returns: + 距离的概率分布 + """ + # 计算确定性距离 + dx = point2[0] - point1[0] + dy = point2[1] - point1[1] + true_distance = math.sqrt(dx**2 + dy**2) + + # 返回正态分布 + return NormalDistribution(mu=true_distance, sigma=distance_error_std) + + @staticmethod + def location_probability(measurement: Tuple[float, float], + true_location: Tuple[float, float], + measurement_error: float = 10.0) -> float: + """ + 计算测量位置的似然概率 + + Args: + measurement: 测量位置 + true_location: 真实位置 + measurement_error: 测量误差标准差 + + Returns: + 似然概率 + """ + dist = SpatialProbabilityModel.uncertain_distance( + measurement, true_location, measurement_error + ) + # 使用正态分布 PDF + return dist.pdf(0) + + @staticmethod + def bayesian_location_update(prior_locations: List[Tuple[float, float]], + measurements: List[Tuple[float, float]], + measurement_error: float = 10.0) -> List[float]: + """ + 贝叶斯位置更新 + + Args: + prior_locations: 候选真实位置列表 + measurements: 测量位置列表 + measurement_error: 测量误差 + + Returns: + 每个候选位置的后验概率 + """ + n = len(prior_locations) + posteriors = [] + + for candidate in prior_locations: + # 计算似然 (所有测量的乘积) + likelihood = 1.0 + for measurement in measurements: + prob = SpatialProbabilityModel.location_probability( + measurement, candidate, measurement_error + ) + likelihood *= prob + + # 先验 (均匀) + prior = 1.0 / n + + # 后验 (未归一化) + posterior = likelihood * prior + posteriors.append(posterior) + + # 归一化 + total = sum(posteriors) + if total > 0: + posteriors = [p / total for p in posteriors] + + return posteriors + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示概率与不确定性的使用""" + + print("="*70) + print("概率与不确定性示例演示") + print("="*70) + + random.seed(42) # 可重现的结果 + + # ======================================================================== + # 1. 概率分布示例 + # ======================================================================== + print("\n[部分 1] 概率分布") + print("-" * 50) + + # 创建不同的分布 + normal = NormalDistribution(mu=100, sigma=15, name="温度") + uniform = UniformDistribution(a=50, b=150, name="范围") + triangular = TriangularDistribution(a=60, c=100, b=140, name="估计") + + distributions = [normal, uniform, triangular] + + for dist in distributions: + print(f"\n{dist}:") + print(f" 均值: {dist.mean():.2f}") + print(f" 标准差: {dist.std():.2f}") + samples = [dist.sample() for _ in range(5)] + print(f" 样本: {[f'{s:.2f}' for s in samples]}") + + ci = dist.confidence_interval(0.95) + print(f" 95% CI: [{ci[0]:.2f}, {ci[1]:.2f}]") + + # ======================================================================== + # 2. 贝叶斯推理示例 + # ======================================================================== + print("\n\n[部分 2] 贝叶斯推理") + print("-" * 50) + print("问题: 根据土壤测试结果判断土地适宜性") + + # 假设: 土地适宜性等级 + hypotheses = ["高适宜", "中适宜", "低适宜", "不适宜"] + updater = BayesianUpdater(hypotheses) + + # 设置不同的先验 (基于历史数据) + updater.set_prior("高适宜", 0.2) + updater.set_prior("中适宜", 0.3) + updater.set_prior("低适宜", 0.3) + updater.set_prior("不适宜", 0.2) + + print("\n初始信念:") + updater.print_beliefs() + + # 证据1: 土壤pH值检测 + print("\n证据1: 土壤pH值适中 (似然更新)") + updater.update_with_evidence({ + "高适宜": 0.8, # pH值对高适宜的可能性高 + "中适宜": 0.6, + "低适宜": 0.3, + "不适宜": 0.1 + }) + updater.print_beliefs() + + # 证据2: 有机质含量检测 + print("\n证据2: 有机质含量高 (似然更新)") + updater.update_with_evidence({ + "高适宜": 0.9, + "中适宜": 0.5, + "低适宜": 0.2, + "不适宜": 0.05 + }) + updater.print_beliefs() + + most_likely = updater.get_most_likely() + print(f"\n最可能的假设: {most_likely[0]} (概率: {most_likely[1]:.2%})") + + # ======================================================================== + # 3. 蒙特卡洛模拟示例 + # ======================================================================== + print("\n\n[部分 3] 蒙特卡洛模拟") + print("-" * 50) + print("问题: 评估房地产开发项目的预期收益") + + def development_model(inputs: Dict[str, float]) -> float: + """房地产开发收益模型""" + land_cost = inputs["land_cost"] + construction_cost = inputs["construction_cost"] + selling_price = inputs["selling_price"] + units = inputs["units"] + sales_rate = inputs["sales_rate"] + + # 收益 = (售价 * 单元数 * 销售率) - (土地成本 + 建设成本 * 单元数) + revenue = selling_price * units * sales_rate + total_cost = land_cost + construction_cost * units + return revenue - total_cost + + # 定义输入分布 + input_dists = { + "land_cost": TriangularDistribution(800000, 1000000, 1500000), # 土地成本 + "construction_cost": NormalDistribution(50000, 5000), # 单元建设成本 + "selling_price": NormalDistribution(150000, 15000), # 单元售价 + "units": TriangularDistribution(80, 100, 120), # 单元数量 + "sales_rate": BetaDistribution(alpha=8, beta=2, a=0, b=1) # 销售率 + } + + simulator = MonteCarloSimulator() + result = simulator.simulate_with_inputs(development_model, input_dists, n_runs=10000) + + result.print_summary() + + # 风险评估 + negative_prob = sum(1 for s in result.samples if s < 0) / len(result.samples) + print(f"\n风险分析:") + print(f" 亏损概率: {negative_prob:.2%}") + profit_prob = sum(1 for s in result.samples if s > 1000000) / len(result.samples) + print(f" 超过100万利润概率: {profit_prob:.2%}") + + # ======================================================================== + # 4. 敏感性分析示例 + # ======================================================================== + print("\n\n[部分 4] 敏感性分析") + print("-" * 50) + print("问题: 分析各因素对收益的影响程度") + + analyzer = SensitivityAnalyzer() + + # 简单敏感性分析 + base_inputs = { + "land_cost": 1000000, + "construction_cost": 50000, + "selling_price": 150000, + "units": 100, + "sales_rate": 0.85 + } + + sensitivity_result = analyzer.simple_sensitivity( + development_model, base_inputs, variations={k: 0.1 for k in base_inputs.keys()} + ) + + sensitivity_result.print_summary() + + # ======================================================================== + # 5. 空间概率应用 + # ======================================================================== + print("\n\n[部分 5] 空间概率应用") + print("-" * 50) + print("问题: GPS定位的不确定性") + + # 真实位置 + true_location = (1000, 2000) + + # 带误差的测量 + measurements = [ + (1005, 2003), + (998, 1998), + (1002, 2005), + (995, 2000) + ] + + # 候选位置 + candidates = [ + (1000, 2000), # 真实位置 + (1015, 2015), + (990, 1990), + (1005, 1995) + ] + + posteriors = SpatialProbabilityModel.bayesian_location_update( + candidates, measurements, measurement_error=5.0 + ) + + print("\n候选位置的后验概率:") + for i, (loc, prob) in enumerate(zip(candidates, posteriors)): + print(f" 位置 {i+1} {loc}: {prob:.4f}") + + most_likely_idx = max(range(len(posteriors)), key=lambda i: posteriors[i]) + print(f"\n最可能的位置: 位置 {most_likely_idx+1} {candidates[most_likely_idx]}") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +# Beta分布实现 (用于上面代码中的引用) +class BetaDistribution(ProbabilityDistribution): + """Beta分布 - 用于建模[0,1]区间内的概率""" + + def __init__(self, alpha: float, beta: float, a: float = 0, b: float = 1, name: str = ""): + super().__init__(name) + self.alpha = alpha + self.beta = beta + self.a = a # 下界 + self.b = b # 上界 + + def sample(self) -> float: + # 使用numpy的gamma函数近似 + import math + import random + + # 生成Gamma随机变量 + def gamma(alpha): + if alpha < 1: + return gamma(alpha + 1) * (random.random() ** (1 / alpha)) + # Marsaglia and Tsang's method + d = alpha - 1/3 + c = 1 / math.sqrt(9 * d) + while True: + x = random.gauss(0, 1) + v = (1 + c * x) ** 3 + if v > 0: + u = random.random() + if u < 1 - 0.0331 * (x * x) ** 2: + return d * v + if math.log(u) < 0.5 * x * x + d * (1 - v + math.log(v)): + return d * v + + x = gamma(self.alpha) + y = gamma(self.beta) + beta_sample = x / (x + y) + + # 转换到[a, b]区间 + return self.a + (self.b - self.a) * beta_sample + + def mean(self) -> float: + return self.a + (self.b - self.a) * self.alpha / (self.alpha + self.beta) + + def std(self) -> float: + mean_raw = self.alpha / (self.alpha + self.beta) + var_raw = (self.alpha * self.beta) / ( + (self.alpha + self.beta) ** 2 * (self.alpha + self.beta + 1) + ) + return (self.b - self.a) * math.sqrt(var_raw) + + def pdf(self, x: float) -> float: + # 简化版本，仅返回近似值 + return 1.0 # 实际应实现Beta分布的PDF + + def __repr__(self) -> str: + return f"Beta(α={self.alpha}, β={self.beta})" + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/01-foundations/state_machine.py b/dofile/examples/01-foundations/state_machine.py new file mode 100644 index 0000000..01a6e05 --- /dev/null +++ b/dofile/examples/01-foundations/state_machine.py @@ -0,0 +1,748 @@ +""" +状态机工作流示例 (State Machine Workflow Example) +================================================== + +本示例展示如何使用状态机模式管理复杂的空间分析工作流。 +状态机是一种行为设计模式，允许对象在其内部状态改变时改变其行为。 + +核心概念: +1. 状态 (State) - 系统在特定时刻的模式 +2. 转换 (Transition) - 从一个状态到另一个状态的变化 +3. 事件 (Event) - 触发状态转换的外部或内部条件 +4. 动作 (Action) - 状态转换时执行的操作 + +应用场景: +- 空间数据处理流水线 +- 多阶段决策流程 +- 任务调度与监控 +- 用户交互流程控制 + +作者: CC4SI 项目组 +""" + +from abc import ABC, abstractmethod +from typing import Dict, List, Optional, Callable, Any +from dataclasses import dataclass, field +from enum import Enum +import json +from datetime import datetime + + +# ============================================================================ +# 状态定义 +# ============================================================================ + +class WorkflowState(Enum): + """工作流状态枚举""" + # 初始状态 + IDLE = "idle" + INITIALIZED = "initialized" + + # 数据处理状态 + LOADING_DATA = "loading_data" + DATA_LOADED = "data_loaded" + VALIDATING_DATA = "validating_data" + DATA_VALIDATED = "data_validated" + + # 分析状态 + ANALYZING = "analyzing" + ANALYSIS_COMPLETE = "analysis_complete" + + # 决策状态 + DECIDING = "deciding" + DECISION_MADE = "decision_made" + + # 输出状态 + GENERATING_OUTPUT = "generating_output" + OUTPUT_COMPLETE = "output_complete" + + # 异常状态 + ERROR = "error" + PAUSED = "paused" + CANCELLED = "cancelled" + + # 最终状态 + COMPLETED = "completed" + + +class EventType(Enum): + """事件类型枚举""" + # 控制事件 + START = "start" + PAUSE = "pause" + RESUME = "resume" + CANCEL = "cancel" + RESET = "reset" + + # 数据事件 + DATA_LOAD_REQUEST = "data_load_request" + DATA_LOAD_SUCCESS = "data_load_success" + DATA_LOAD_FAILURE = "data_load_failure" + DATA_VALIDATE_REQUEST = "data_validate_request" + DATA_VALIDATE_SUCCESS = "data_validate_success" + DATA_VALIDATE_FAILURE = "data_validate_failure" + + # 分析事件 + ANALYZE_REQUEST = "analyze_request" + ANALYSIS_SUCCESS = "analysis_success" + ANALYSIS_FAILURE = "analysis_failure" + + # 决策事件 + DECIDE_REQUEST = "decide_request" + DECISION_SUCCESS = "decision_success" + DECISION_FAILURE = "decision_failure" + + # 输出事件 + OUTPUT_REQUEST = "output_request" + OUTPUT_SUCCESS = "output_success" + OUTPUT_FAILURE = "output_failure" + + # 错误事件 + ERROR_OCCURRED = "error_occurred" + RETRY = "retry" + + +# ============================================================================ +# 状态机数据结构 +# ============================================================================ + +@dataclass +class StateTransition: + """状态转换定义""" + from_state: WorkflowState + event: EventType + to_state: WorkflowState + action: Optional[Callable] = None + guard: Optional[Callable[[], bool]] = None # 守卫条件 + description: str = "" + + def can_execute(self) -> bool: + """检查转换是否可执行""" + if self.guard is None: + return True + return self.guard() + + +@dataclass +class StateContext: + """状态上下文 - 存储工作流数据""" + data: Dict[str, Any] = field(default_factory=dict) + errors: List[str] = field(default_factory=list) + warnings: List[str] = field(default_factory=list) + history: List[Dict[str, Any]] = field(default_factory=list) + start_time: Optional[datetime] = None + end_time: Optional[datetime] = None + + def add_history(self, from_state: WorkflowState, event: EventType, + to_state: WorkflowState, timestamp: datetime = None): + """添加历史记录""" + self.history.append({ + "from_state": from_state.value, + "event": event.value, + "to_state": to_state.value, + "timestamp": timestamp or datetime.now() + }) + + def get_data(self, key: str, default: Any = None) -> Any: + """获取数据""" + return self.data.get(key, default) + + def set_data(self, key: str, value: Any) -> None: + """设置数据""" + self.data[key] = value + + def add_error(self, error: str) -> None: + """添加错误""" + self.errors.append(error) + + def add_warning(self, warning: str) -> None: + """添加警告""" + self.warnings.append(warning) + + +# ============================================================================ +# 状态机实现 +# ============================================================================ + +class StateMachine: + """ + 状态机实现 + + 管理状态转换和状态相关的行为。 + """ + + def __init__(self, initial_state: WorkflowState = WorkflowState.IDLE): + """ + 初始化状态机 + + Args: + initial_state: 初始状态 + """ + self._current_state = initial_state + self._transitions: Dict[WorkflowState, Dict[EventType, StateTransition]] = {} + self._context = StateContext() + self._state_listeners: Dict[WorkflowState, List[Callable]] = {} + + print(f"[状态机] 初始化，初始状态: {initial_state.value}") + + @property + def current_state(self) -> WorkflowState: + """获取当前状态""" + return self._current_state + + @property + def context(self) -> StateContext: + """获取状态上下文""" + return self._context + + def add_transition(self, transition: StateTransition) -> None: + """ + 添加状态转换 + + Args: + transition: 状态转换定义 + """ + if transition.from_state not in self._transitions: + self._transitions[transition.from_state] = {} + + self._transitions[transition.from_state][transition.event] = transition + print(f"[状态机] 添加转换: {transition.from_state.value} + {transition.event.value} -> {transition.to_state.value}") + + def add_state_listener(self, state: WorkflowState, listener: Callable) -> None: + """ + 添加状态监听器 + + Args: + state: 要监听的状态 + listener: 状态进入时调用的函数 + """ + if state not in self._state_listeners: + self._state_listeners[state] = [] + self._state_listeners[state].append(listener) + + def trigger(self, event: EventType, payload: Any = None) -> bool: + """ + 触发事件 + + Args: + event: 事件类型 + payload: 事件负载 + + Returns: + 是否成功触发状态转换 + """ + # 检查当前状态是否有对应转换 + if self._current_state not in self._transitions: + print(f"[状态机] 当前状态 {self._current_state.value} 没有定义任何转换") + return False + + if event not in self._transitions[self._current_state]: + print(f"[状态机] 状态 {self._current_state.value} 不处理事件 {event.value}") + return False + + transition = self._transitions[self._current_state][event] + + # 检查守卫条件 + if not transition.can_execute(): + print(f"[状态机] 守卫条件不满足，转换被阻止") + return False + + # 执行状态转换 + old_state = self._current_state + self._current_state = transition.to_state + + # 记录历史 + if payload: + self._context.set_data("last_payload", payload) + self._context.add_history(old_state, event, self._current_state) + + print(f"[状态机] 状态转换: {old_state.value} -> {self._current_state.value} (事件: {event.value})") + + # 执行转换动作 + if transition.action: + try: + transition.action(self._context, payload) + except Exception as e: + print(f"[状态机] 执行动作时出错: {e}") + self._context.add_error(f"转换动作执行失败: {e}") + + # 触发状态监听器 + if self._current_state in self._state_listeners: + for listener in self._state_listeners[self._current_state]: + try: + listener(self._current_state, self._context) + except Exception as e: + print(f"[状态机] 监听器执行出错: {e}") + + return True + + def can_trigger(self, event: EventType) -> bool: + """ + 检查是否可以触发指定事件 + + Args: + event: 事件类型 + + Returns: + 是否可以触发 + """ + if self._current_state not in self._transitions: + return False + if event not in self._transitions[self._current_state]: + return False + + transition = self._transitions[self._current_state][event] + return transition.can_execute() + + def get_available_events(self) -> List[EventType]: + """获取当前状态下可用的事件列表""" + if self._current_state not in self._transitions: + return [] + + available = [] + for event, transition in self._transitions[self._current_state].items(): + if transition.can_execute(): + available.append(event) + + return available + + def reset(self) -> None: + """重置状态机""" + self._current_state = WorkflowState.IDLE + self._context = StateContext() + print(f"[状态机] 状态机已重置") + + def print_state(self) -> None: + """打印当前状态""" + print(f"\n当前状态: {self._current_state.value}") + available = self.get_available_events() + if available: + print(f"可用事件: {', '.join(e.value for e in available)}") + else: + print("可用事件: 无") + + +# ============================================================================ +# 空间分析工作流状态机 +# ============================================================================ + +class SpatialAnalysisWorkflow: + """ + 空间分析工作流 + + 使用状态机实现的空间数据处理和分析工作流。 + """ + + def __init__(self): + """初始化工作流""" + self.state_machine = StateMachine() + self._setup_transitions() + self._setup_listeners() + + def _setup_transitions(self): + """设置状态转换""" + sm = self.state_machine + + # 启动流程 + sm.add_transition(StateTransition( + from_state=WorkflowState.IDLE, + event=EventType.START, + to_state=WorkflowState.INITIALIZED, + action=self._action_initialize, + description="初始化工作流" + )) + + # 数据加载 + sm.add_transition(StateTransition( + from_state=WorkflowState.INITIALIZED, + event=EventType.DATA_LOAD_REQUEST, + to_state=WorkflowState.LOADING_DATA, + action=self._action_load_data, + description="开始加载数据" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.LOADING_DATA, + event=EventType.DATA_LOAD_SUCCESS, + to_state=WorkflowState.DATA_LOADED, + action=self._action_on_data_loaded, + description="数据加载成功" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.LOADING_DATA, + event=EventType.DATA_LOAD_FAILURE, + to_state=WorkflowState.ERROR, + action=self._action_on_error, + description="数据加载失败" + )) + + # 数据验证 + sm.add_transition(StateTransition( + from_state=WorkflowState.DATA_LOADED, + event=EventType.DATA_VALIDATE_REQUEST, + to_state=WorkflowState.VALIDATING_DATA, + action=self._action_validate_data, + description="开始验证数据" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.VALIDATING_DATA, + event=EventType.DATA_VALIDATE_SUCCESS, + to_state=WorkflowState.DATA_VALIDATED, + description="数据验证成功" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.VALIDATING_DATA, + event=EventType.DATA_VALIDATE_FAILURE, + to_state=WorkflowState.ERROR, + action=self._action_on_error, + description="数据验证失败" + )) + + # 分析 + sm.add_transition(StateTransition( + from_state=WorkflowState.DATA_VALIDATED, + event=EventType.ANALYZE_REQUEST, + to_state=WorkflowState.ANALYZING, + action=self._action_analyze, + description="开始分析" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.ANALYZING, + event=EventType.ANALYSIS_SUCCESS, + to_state=WorkflowState.ANALYSIS_COMPLETE, + description="分析完成" + )) + + # 决策 + sm.add_transition(StateTransition( + from_state=WorkflowState.ANALYSIS_COMPLETE, + event=EventType.DECIDE_REQUEST, + to_state=WorkflowState.DECIDING, + action=self._action_decide, + description="开始决策" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.DECIDING, + event=EventType.DECISION_SUCCESS, + to_state=WorkflowState.DECISION_MADE, + description="决策完成" + )) + + # 输出 + sm.add_transition(StateTransition( + from_state=WorkflowState.DECISION_MADE, + event=EventType.OUTPUT_REQUEST, + to_state=WorkflowState.GENERATING_OUTPUT, + action=self._action_generate_output, + description="生成输出" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.GENERATING_OUTPUT, + event=EventType.OUTPUT_SUCCESS, + to_state=WorkflowState.OUTPUT_COMPLETE, + description="输出完成" + )) + + # 完成 + sm.add_transition(StateTransition( + from_state=WorkflowState.OUTPUT_COMPLETE, + event=EventType.START, + to_state=WorkflowState.COMPLETED, + action=self._action_complete, + description="工作流完成" + )) + + # 错误恢复 + sm.add_transition(StateTransition( + from_state=WorkflowState.ERROR, + event=EventType.RETRY, + to_state=WorkflowState.INITIALIZED, + guard=lambda: len(self.state_machine.context.errors) < 3, + description="重试" + )) + + sm.add_transition(StateTransition( + from_state=WorkflowState.ERROR, + event=EventType.RESET, + to_state=WorkflowState.IDLE, + action=self._action_reset, + description="重置" + )) + + def _setup_listeners(self): + """设置状态监听器""" + sm = self.state_machine + + # 错误状态监听器 + sm.add_state_listener(WorkflowState.ERROR, self._on_error_state) + + # 完成状态监听器 + sm.add_state_listener(WorkflowState.COMPLETED, self._on_complete_state) + + # ------------------------------------------------------------------------ + # 状态动作 + # ------------------------------------------------------------------------ + + def _action_initialize(self, ctx: StateContext, payload: Any): + """初始化动作""" + ctx.start_time = datetime.now() + ctx.set_data("workflow_id", f"WF-{datetime.now().strftime('%Y%m%d%H%M%S')}") + print(" [动作] 工作流初始化完成") + + def _action_load_data(self, ctx: StateContext, payload: Any): + """加载数据动作""" + source = payload or "默认数据源" + print(f" [动作] 从 '{source}' 加载数据...") + + # 模拟数据加载 + ctx.set_data("raw_data", [ + {"id": 1, "x": 10, "y": 20, "value": 100}, + {"id": 2, "x": 30, "y": 40, "value": 200}, + {"id": 3, "x": 50, "y": 60, "value": 150}, + ]) + + # 模拟成功 + self.state_machine.trigger(EventType.DATA_LOAD_SUCCESS) + + def _action_on_data_loaded(self, ctx: StateContext, payload: Any): + """数据加载完成动作""" + data_count = len(ctx.get_data("raw_data", [])) + print(f" [动作] 数据加载完成，共 {data_count} 条记录") + + def _action_validate_data(self, ctx: StateContext, payload: Any): + """验证数据动作""" + print(" [动作] 验证数据...") + + data = ctx.get_data("raw_data", []) + valid = all("id" in item and "x" in item and "y" in item for item in data) + + if valid: + self.state_machine.trigger(EventType.DATA_VALIDATE_SUCCESS) + else: + ctx.add_error("数据验证失败: 缺少必需字段") + self.state_machine.trigger(EventType.DATA_VALIDATE_FAILURE) + + def _action_analyze(self, ctx: StateContext, payload: Any): + """分析动作""" + print(" [动作] 执行空间分析...") + + data = ctx.get_data("raw_data", []) + values = [item.get("value", 0) for item in data] + avg = sum(values) / len(values) if values else 0 + + ctx.set_data("analysis_result", { + "average": avg, + "count": len(data), + "min": min(values) if values else 0, + "max": max(values) if values else 0 + }) + + print(f" [动作] 分析完成，平均值: {avg:.2f}") + self.state_machine.trigger(EventType.ANALYSIS_SUCCESS) + + def _action_decide(self, ctx: StateContext, payload: Any): + """决策动作""" + print(" [动作] 执行决策...") + + analysis = ctx.get_data("analysis_result", {}) + avg = analysis.get("average", 0) + + if avg > 150: + decision = "高价值区域" + elif avg > 100: + decision = "中等价值区域" + else: + decision = "低价值区域" + + ctx.set_data("decision", decision) + print(f" [动作] 决策完成: {decision}") + self.state_machine.trigger(EventType.DECISION_SUCCESS) + + def _action_generate_output(self, ctx: StateContext, payload: Any): + """生成输出动作""" + print(" [动作] 生成输出报告...") + + report = { + "workflow_id": ctx.get_data("workflow_id"), + "data_count": len(ctx.get_data("raw_data", [])), + "analysis": ctx.get_data("analysis_result"), + "decision": ctx.get_data("decision") + } + + ctx.set_data("output", report) + print(" [动作] 输出生成完成") + self.state_machine.trigger(EventType.OUTPUT_SUCCESS) + + def _action_complete(self, ctx: StateContext, payload: Any): + """完成动作""" + ctx.end_time = datetime.now() + duration = (ctx.end_time - ctx.start_time).total_seconds() if ctx.start_time else 0 + ctx.set_data("duration", duration) + print(f" [动作] 工作流完成，耗时: {duration:.2f}秒") + + def _action_on_error(self, ctx: StateContext, payload: Any): + """错误处理动作""" + print(f" [动作] 发生错误") + + def _action_reset(self, ctx: StateContext, payload: Any): + """重置动作""" + print(" [动作] 重置工作流") + + # ------------------------------------------------------------------------ + # 状态监听器 + # ------------------------------------------------------------------------ + + def _on_error_state(self, state: WorkflowState, ctx: StateContext): + """错误状态处理""" + print(f" [监听器] 进入错误状态") + print(f" [监听器] 错误列表: {ctx.errors}") + + def _on_complete_state(self, state: WorkflowState, ctx: StateContext): + """完成状态处理""" + print(f" [监听器] 工作流已完成") + output = ctx.get_data("output") + if output: + print(f" [监听器] 最终输出: {json.dumps(output, ensure_ascii=False, indent=2)}") + + # ------------------------------------------------------------------------ + # 公共接口 + # ------------------------------------------------------------------------ + + def start(self, data_source: str = None) -> bool: + """启动工作流""" + return self.state_machine.trigger(EventType.START, data_source) + + def execute_full_workflow(self, data_source: str = None) -> Dict[str, Any]: + """ + 执行完整工作流 + + Args: + data_source: 数据源 + + Returns: + 执行结果 + """ + print("\n" + "="*60) + print("执行完整空间分析工作流") + print("="*60) + + # 启动 + if not self.start(data_source): + return {"success": False, "error": "启动失败"} + + # 等待异步操作完成 (简化版: 手动触发) + # 在实际应用中，这些事件会由异步操作触发 + + return { + "success": True, + "final_state": self.state_machine.current_state.value, + "context": self.state_machine.context.data + } + + def print_history(self): + """打印状态转换历史""" + history = self.state_machine.context.history + print(f"\n状态转换历史 (共 {len(history)} 次):") + print("-" * 70) + for i, h in enumerate(history, 1): + ts = h.get("timestamp", datetime.now()).strftime("%H:%M:%S") + print(f"{i:2d}. [{ts}] {h['from_state']:20s} -> {h['to_state']:20s} ({h['event']})") + print("-" * 70) + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示状态机工作流的使用""" + + print("="*70) + print("状态机工作流示例演示") + print("="*70) + + # 1. 创建工作流 + print("\n[步骤 1] 创建空间分析工作流") + workflow = SpatialAnalysisWorkflow() + + # 2. 显示初始状态 + print("\n[步骤 2] 初始状态") + workflow.state_machine.print_state() + + # 3. 手动执行状态转换 + print("\n[步骤 3] 手动执行状态转换") + + # 启动 + print("\n3.1 启动工作流:") + workflow.state_machine.trigger(EventType.START) + workflow.state_machine.print_state() + + # 请求数据加载 (这将触发加载动作，然后自动触发成功事件) + print("\n3.2 请求数据加载:") + workflow.state_machine.trigger(EventType.DATA_LOAD_REQUEST, "sample.csv") + workflow.state_machine.print_state() + + # 请求数据验证 + print("\n3.3 请求数据验证:") + workflow.state_machine.trigger(EventType.DATA_VALIDATE_REQUEST) + workflow.state_machine.print_state() + + # 请求分析 + print("\n3.4 请求分析:") + workflow.state_machine.trigger(EventType.ANALYZE_REQUEST) + workflow.state_machine.print_state() + + # 请求决策 + print("\n3.5 请求决策:") + workflow.state_machine.trigger(EventType.DECIDE_REQUEST) + workflow.state_machine.print_state() + + # 请求输出 + print("\n3.6 请求输出:") + workflow.state_machine.trigger(EventType.OUTPUT_REQUEST) + workflow.state_machine.print_state() + + # 完成 + print("\n3.7 完成工作流:") + workflow.state_machine.trigger(EventType.START) + workflow.state_machine.print_state() + + # 4. 显示转换历史 + print("\n[步骤 4] 状态转换历史") + workflow.print_history() + + # 5. 演示错误处理 + print("\n[步骤 5] 演示错误处理和恢复") + print("\n5.1 重置状态机:") + workflow.state_machine.reset() + workflow.state_machine.print_state() + + print("\n5.2 启动后触发错误:") + workflow.state_machine.trigger(EventType.START) + workflow.state_machine.trigger(EventType.DATA_LOAD_FAILURE) + workflow.state_machine.print_state() + + print("\n5.3 尝试重试:") + if workflow.state_machine.can_trigger(EventType.RETRY): + workflow.state_machine.trigger(EventType.RETRY) + workflow.state_machine.print_state() + else: + print(" 无法重试 (已达到最大重试次数)") + + print("\n5.4 重置工作流:") + workflow.state_machine.trigger(EventType.RESET) + workflow.state_machine.print_state() + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/02-spatial-intelligence/mcdm_example.py b/dofile/examples/02-spatial-intelligence/mcdm_example.py new file mode 100644 index 0000000..efbaf68 --- /dev/null +++ b/dofile/examples/02-spatial-intelligence/mcdm_example.py @@ -0,0 +1,1120 @@ +""" +多准则决策示例 (Multi-Criteria Decision Making Example) +====================================================== + +本示例展示空间智能系统中的多准则决策方法。 +MCDA/MCDM 用于处理多个冲突准则下的决策问题。 + +核心概念: +1. 准则体系 - 构建评价准则层次结构 +2. 权重确定 - AHP、熵权法等 +3. 决策矩阵 - 标准化与规范化 +4. 综合评价 - WSM、WPM、TOPSIS等 +5. 灵敏度分析 - 权重变化对结果的影响 + +应用场景: +- 选址决策 +- 项目评估 +- 资源配置 +- 风险评估 + +作者: CC4SI 项目组 +""" + +import math +import json +from typing import List, Dict, Tuple, Optional, Any, Callable +from dataclasses import dataclass, field +from enum import Enum +import random + + +# ============================================================================ +# 准则类型与方向 +# ============================================================================ + +class CriterionType(Enum): + """准则类型""" + BENEFIT = "benefit" # 效益型 (越大越好) + COST = "cost" # 成本型 (越小越好) + NON_MONOTONIC = "non_monotonic" # 非单调 (有最优值) + + +@dataclass +class Criterion: + """ + 决策准则 + + 定义评价的维度。 + """ + name: str + criterion_type: CriterionType + weight: float = 1.0 + scale: Tuple[float, float] = (0, 1) # 取值范围 + optimal_value: Optional[float] = None # 最优值 (用于非单调型) + + def __repr__(self) -> str: + return f"Criterion({self.name}, {self.criterion_type.value}, w={self.weight:.2f})" + + +# ============================================================================ +# 决策方案 +# ============================================================================ + +@dataclass +class Alternative: + """ + 决策方案 + + 表示一个待评估的备选方案。 + """ + id: str + name: str + values: Dict[str, float] # 准则名称到值的映射 + metadata: Dict[str, Any] = field(default_factory=dict) + + def get_value(self, criterion_name: str) -> Optional[float]: + """获取准则值""" + return self.values.get(criterion_name) + + def set_value(self, criterion_name: str, value: float) -> None: + """设置准则值""" + self.values[criterion_name] = value + + def __repr__(self) -> str: + return f"Alternative({self.name}, values={len(self.values)})" + + +# ============================================================================ +# 标准化方法 +# ============================================================================ + +class NormalizationMethod(Enum): + """标准化方法""" + MIN_MAX = "min_max" # Min-Max标准化 + VECTOR = "vector" # 向量标准化 + Z_SCORE = "z_score" # Z-Score标准化 + SUM = "sum" # 总和标准化 + + +class Normalizer: + """数据标准化器""" + + @staticmethod + def min_max(values: List[float], + target_range: Tuple[float, float] = (0, 1), + criterion_type: CriterionType = CriterionType.BENEFIT) -> List[float]: + """ + Min-Max标准化 + + Args: + values: 原始值列表 + target_range: 目标范围 + criterion_type: 准则类型 + + Returns: + 标准化后的值列表 + """ + min_val = min(values) + max_val = max(values) + + if max_val == min_val: + return [target_range[0] for _ in values] + + t_min, t_max = target_range + result = [] + + for v in values: + if criterion_type == CriterionType.BENEFIT: + # 效益型: 越大越好 + normalized = (v - min_val) / (max_val - min_val) + else: # COST + # 成本型: 越小越好 + normalized = (max_val - v) / (max_val - min_val) + + result.append(t_min + normalized * (t_max - t_min)) + + return result + + @staticmethod + def vector(values: List[float], + criterion_type: CriterionType = CriterionType.BENEFIT) -> List[float]: + """ + 向量标准化 + + Args: + values: 原始值列表 + criterion_type: 准则类型 + + Returns: + 标准化后的值列表 + """ + sum_squares = sum(v * v for v in values) + if sum_squares == 0: + return [0.0 for _ in values] + + norm = math.sqrt(sum_squares) + result = [v / norm for v in values] + + if criterion_type == CriterionType.COST: + # 成本型: 取倒数 + result = [1.0 / (v + 1e-10) if v > 0 else 1.0 for v in result] + # 重新归一化 + total = sum(result) + result = [v / total for v in result] + + return result + + @staticmethod + def z_score(values: List[float], + criterion_type: CriterionType = CriterionType.BENEFIT) -> List[float]: + """ + Z-Score标准化 + + Args: + values: 原始值列表 + criterion_type: 准则类型 + + Returns: + 标准化后的值列表 + """ + import statistics + + if len(values) < 2: + return [0.0 for _ in values] + + mean = statistics.mean(values) + stdev = statistics.stdev(values) + + if stdev == 0: + return [0.0 for _ in values] + + result = [(v - mean) / stdev for v in values] + + # 转换到正值范围 + min_result = min(result) + if min_result < 0: + result = [v - min_result for v in result] + + # 归一化到0-1 + max_result = max(result) + if max_result > 0: + result = [v / max_result for v in result] + + if criterion_type == CriterionType.COST: + result = [1.0 - v for v in result] + + return result + + +# ============================================================================ +# AHP层次分析法 +# ============================================================================ + +class AHP: + """ + 层次分析法 (Analytic Hierarchy Process) + + 用于确定准则权重的方法。 + """ + + # Saaty标度 + SAATY_SCALE = { + 1: "同等重要", + 2: "稍微重要", + 3: "明显重要", + 4: "非常重要", + 5: "极端重要" + } + + def __init__(self, criteria: List[str]): + """ + 初始化AHP + + Args: + criteria: 准则名称列表 + """ + self.criteria = criteria + self.n = len(criteria) + self.comparison_matrix: List[List[float]] = [] + + def build_comparison_matrix(self, comparisons: Dict[Tuple[str, str], float]) -> None: + """ + 构建比较矩阵 + + Args: + comparisons: 准则对比较值字典 + ((criterion_i, criterion_j), value) + value > 1 表示 i 比 j 重要 + value < 1 表示 j 比 i 重要 + """ + # 初始化单位矩阵 + self.comparison_matrix = [[1.0 for _ in range(self.n)] for _ in range(self.n)] + + # 填充比较矩阵 + for (c1, c2), value in comparisons.items(): + if c1 in self.criteria and c2 in self.criteria: + i = self.criteria.index(c1) + j = self.criteria.index(c2) + self.comparison_matrix[i][j] = value + self.comparison_matrix[j][i] = 1.0 / value + + def calculate_weights(self) -> Tuple[List[float], float, float]: + """ + 计算权重 + + Returns: + (权重列表, 一致性比率, 最大特征值) + """ + if not self.comparison_matrix: + return [1.0 / self.n] * self.n, 0.0, self.n + + # 特征向量法 (幂法) + weights = self._eigenvector_method() + lambda_max = self._calculate_lambda_max(weights) + ci = (lambda_max - self.n) / (self.n - 1) if self.n > 1 else 0 + ri = self._random_consistency_index(self.n) + cr = ci / ri if ri > 0 else 0 + + return weights, cr, lambda_max + + def _eigenvector_method(self, max_iterations: int = 100, + tolerance: float = 1e-6) -> List[float]: + """使用幂法计算特征向量""" + # 初始化权重向量 + weights = [1.0 / self.n] * self.n + + for _ in range(max_iterations): + # 矩阵向量乘法 + new_weights = [] + for i in range(self.n): + new_weights.append( + sum(self.comparison_matrix[i][j] * weights[j] for j in range(self.n)) + ) + + # 归一化 + total = sum(new_weights) + new_weights = [w / total for w in new_weights] + + # 检查收敛 + if max(abs(new_weights[i] - weights[i]) for i in range(self.n)) < tolerance: + break + + weights = new_weights + + return weights + + def _calculate_lambda_max(self, weights: List[float]) -> float: + """计算最大特征值""" + lambda_sum = 0.0 + for i in range(self.n): + weighted_sum = sum(self.comparison_matrix[i][j] * weights[j] for j in range(self.n)) + lambda_sum += weighted_sum / weights[i] if weights[i] > 0 else 0 + + return lambda_sum / self.n + + def _random_consistency_index(self, n: int) -> float: + """随机一致性指标RI""" + ri_table = { + 1: 0.0, 2: 0.0, 3: 0.58, 4: 0.90, 5: 1.12, + 6: 1.24, 7: 1.32, 8: 1.41, 9: 1.45, 10: 1.49 + } + return ri_table.get(n, 1.49) + + +# ============================================================================ +# 熵权法 +# ============================================================================ + +class EntropyWeightMethod: + """ + 熵权法 + + 基于数据离散度的客观权重确定方法。 + """ + + @staticmethod + def calculate_weights(decision_matrix: List[List[float]], + criterion_types: List[CriterionType]) -> List[float]: + """ + 计算熵权 + + Args: + decision_matrix: 决策矩阵 (方案 x 准则) + criterion_types: 各准则的类型 + + Returns: + 权重列表 + """ + n_alternatives = len(decision_matrix) + n_criteria = len(decision_matrix[0]) if decision_matrix else 0 + + if n_alternatives == 0 or n_criteria == 0: + return [] + + # 标准化 + normalized_matrix = [] + for j in range(n_criteria): + column = [decision_matrix[i][j] for i in range(n_alternatives)] + normalized = EntropyWeightMethod._normalize_column( + column, criterion_types[j] + ) + normalized_matrix.append(normalized) + + # 计算熵值 + entropy_values = [] + for j in range(n_criteria): + column = normalized_matrix[j] + # 转换为概率 + total = sum(column) + if total == 0: + entropy_values.append(0) + continue + + probabilities = [v / total for v in column] + # 计算熵 + entropy = 0.0 + k = 1 / math.log(n_alternatives) if n_alternatives > 1 else 0 + for p in probabilities: + if p > 0: + entropy -= k * p * math.log(p) + + entropy_values.append(entropy) + + # 计算权重 + diversity = [1 - e for e in entropy_values] + total_diversity = sum(diversity) + + if total_diversity == 0: + return [1.0 / n_criteria] * n_criteria + + weights = [d / total_diversity for d in diversity] + return weights + + @staticmethod + def _normalize_column(column: List[float], + criterion_type: CriterionType) -> List[float]: + """标准化列""" + min_val = min(column) + max_val = max(column) + + if max_val == min_val: + return [1.0 for _ in column] + + result = [] + for v in column: + if criterion_type == CriterionType.BENEFIT: + normalized = (v - min_val) / (max_val - min_val) + else: # COST + normalized = (max_val - v) / (max_val - min_val) + result.append(normalized) + + return result + + +# ============================================================================ +# MCDA方法实现 +# ============================================================================ + +class WeightedSumModel: + """ + 加权求和模型 (WSM) + + 最简单的多准则决策方法。 + """ + + def __init__(self, criteria: List[Criterion]): + self.criteria = criteria + self.criterion_map = {c.name: c for c in criteria} + + def evaluate(self, alternatives: List[Alternative]) -> List[Tuple[Alternative, float]]: + """ + 评估方案 + + Args: + alternatives: 备选方案列表 + + Returns: + (方案, 得分) 列表，按得分降序排列 + """ + results = [] + + for alt in alternatives: + score = 0.0 + valid = True + + for criterion in self.criteria: + value = alt.get_value(criterion.name) + if value is None: + valid = False + break + + # 标准化 + normalized = self._normalize_value(value, criterion) + score += criterion.weight * normalized + + if valid: + results.append((alt, score)) + + results.sort(key=lambda x: x[1], reverse=True) + return results + + def _normalize_value(self, value: float, criterion: Criterion) -> float: + """标准化单个值""" + min_val, max_val = criterion.scale + + if criterion.criterion_type == CriterionType.BENEFIT: + if max_val == min_val: + return 0.5 + return (value - min_val) / (max_val - min_val) + else: # COST + if max_val == min_val: + return 0.5 + return (max_val - value) / (max_val - min_val) + + +class TOPSIS: + """ + TOPSIS (逼近理想解排序法) + + 考虑方案与理想解的距离。 + """ + + def __init__(self, criteria: List[Criterion]): + self.criteria = criteria + self.criterion_map = {c.name: c for c in criteria} + + def evaluate(self, alternatives: List[Alternative]) -> List[Tuple[Alternative, float]]: + """ + 评估方案 + + Args: + alternatives: 备选方案列表 + + Returns: + (方案, 相对贴近度) 列表，按贴近度降序排列 + """ + # 构建决策矩阵 + matrix, criterion_names = self._build_matrix(alternatives) + + if not matrix or not criterion_names: + return [] + + n_alternatives = len(matrix) + n_criteria = len(matrix[0]) + + # 向量标准化 + normalized_matrix = self._normalize_matrix(matrix) + + # 构建加权标准化矩阵 + weights = [self.criterion_map[c].weight for c in criterion_names] + weighted_matrix = [ + [normalized_matrix[i][j] * weights[j] for j in range(n_criteria)] + for i in range(n_alternatives) + ] + + # 确定理想解和负理想解 + ideal_positive, ideal_negative = self._determine_ideals( + weighted_matrix, criterion_names + ) + + # 计算距离 + distances_positive = self._calculate_distances(weighted_matrix, ideal_positive) + distances_negative = self._calculate_distances(weighted_matrix, ideal_negative) + + # 计算相对贴近度 + results = [] + for i, alt in enumerate(alternatives): + d_pos = distances_positive[i] + d_neg = distances_negative[i] + + if d_pos + d_neg == 0: + closeness = 0 + else: + closeness = d_neg / (d_pos + d_neg) + + results.append((alt, closeness)) + + results.sort(key=lambda x: x[1], reverse=True) + return results + + def _build_matrix(self, alternatives: List[Alternative]) -> Tuple[List[List[float]], List[str]]: + """构建决策矩阵""" + if not self.criteria: + return [], [] + + criterion_names = [c.name for c in self.criteria] + matrix = [] + + for alt in alternatives: + row = [] + valid = True + for name in criterion_names: + value = alt.get_value(name) + if value is None: + valid = False + break + row.append(value) + + if valid: + matrix.append(row) + + return matrix, criterion_names + + def _normalize_matrix(self, matrix: List[List[float]]) -> List[List[float]]: + """向量标准化""" + if not matrix: + return [] + + n_criteria = len(matrix[0]) + result = [] + + for j in range(n_criteria): + column = [matrix[i][j] for i in range(len(matrix))] + sum_squares = sum(v * v for v in column) + norm = math.sqrt(sum_squares) if sum_squares > 0 else 1 + + for i in range(len(matrix)): + if j == 0: + result.append([]) + result[i].append(matrix[i][j] / norm) + + return result + + def _determine_ideals(self, matrix: List[List[float]], + criterion_names: List[str]) -> Tuple[List[float], List[float]]: + """确定理想解和负理想解""" + n_criteria = len(matrix[0]) + ideal_positive = [] + ideal_negative = [] + + for j in range(n_criteria): + column = [matrix[i][j] for i in range(len(matrix))] + criterion = self.criterion_map[criterion_names[j]] + + if criterion.criterion_type == CriterionType.BENEFIT: + ideal_positive.append(max(column)) + ideal_negative.append(min(column)) + else: # COST + ideal_positive.append(min(column)) + ideal_negative.append(max(column)) + + return ideal_positive, ideal_negative + + def _calculate_distances(self, matrix: List[List[float]], + ideal: List[float]) -> List[float]: + """计算到理想解的距离""" + distances = [] + + for i in range(len(matrix)): + dist = math.sqrt( + sum((matrix[i][j] - ideal[j]) ** 2 for j in range(len(ideal))) + ) + distances.append(dist) + + return distances + + +class VIKOR: + """ + VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje) + + 折衷排序方法，适用于准则间存在冲突的情况。 + """ + + def __init__(self, criteria: List[Criterion], v: float = 0.5): + """ + 初始化VIKOR + + Args: + criteria: 准则列表 + v: 决策机制系数 (0-1) + v > 0.5: 按群体效益最大化 + v < 0.5: 按个别遗憾最小化 + v = 0.5: 折衷解 + """ + self.criteria = criteria + self.criterion_map = {c.name: c for c in criteria} + self.v = v + + def evaluate(self, alternatives: List[Alternative]) -> List[Tuple[Alternative, float]]: + """评估方案""" + # 构建决策矩阵 + matrix, criterion_names = self._build_matrix(alternatives) + + if not matrix or not criterion_names: + return [] + + n_alternatives = len(matrix) + n_criteria = len(matrix[0]) + + # 标准化 + normalized_matrix = self._normalize_matrix(matrix, criterion_names) + + # 确定最优最劣值 + f_best, f_worst = self._determine_best_worst(normalized_matrix, criterion_names) + + # 计算S和R + weights = [self.criterion_map[c].weight for c in criterion_names] + + S_values = [] # 群体效益 + R_values = [] # 个别遗憾 + + for i in range(n_alternatives): + S = 0.0 + R = 0.0 + + for j in range(n_criteria): + weight = weights[j] + value = normalized_matrix[i][j] + + # 距离最优值的归一化距离 + if f_best[j] == f_worst[j]: + distance = 0 + else: + distance = (f_best[j] - value) / (f_best[j] - f_worst[j]) + + S += weight * distance + R = max(R, weight * distance) + + S_values.append(S) + R_values.append(R) + + # 计算Q值 + S_min, S_max = min(S_values), max(S_values) + R_min, R_max = min(R_values), max(R_values) + + Q_values = [] + for i in range(n_alternatives): + if S_max == S_min: + s_term = 0 + else: + s_term = (S_values[i] - S_min) / (S_max - S_min) + + if R_max == R_min: + r_term = 0 + else: + r_term = (R_values[i] - R_min) / (R_max - R_min) + + Q = self.v * s_term + (1 - self.v) * r_term + Q_values.append(Q) + + # 返回按Q值排序的结果 + results = [(alternatives[i], Q_values[i]) for i in range(n_alternatives)] + results.sort(key=lambda x: x[1]) + return results + + def _build_matrix(self, alternatives: List[Alternative]) -> Tuple[List[List[float]], List[str]]: + """构建决策矩阵""" + criterion_names = [c.name for c in self.criteria] + matrix = [] + + for alt in alternatives: + row = [alt.get_value(name) for name in criterion_names] + if None not in row: + matrix.append(row) + + return matrix, criterion_names + + def _normalize_matrix(self, matrix: List[List[float]], + criterion_names: List[str]) -> List[List[float]]: + """标准化决策矩阵""" + result = [] + n_criteria = len(matrix[0]) + + for j in range(n_criteria): + column = [matrix[i][j] for i in range(len(matrix))] + criterion = self.criterion_map[criterion_names[j]] + + min_val = min(column) + max_val = max(column) + + for i in range(len(matrix)): + if j == 0: + result.append([]) + + if max_val == min_val: + result[i].append(1.0) + elif criterion.criterion_type == CriterionType.BENEFIT: + result[i].append((matrix[i][j] - min_val) / (max_val - min_val)) + else: + result[i].append((max_val - matrix[i][j]) / (max_val - min_val)) + + return result + + def _determine_best_worst(self, matrix: List[List[float]], + criterion_names: List[str]) -> Tuple[List[float], List[float]]: + """确定最优值和最劣值""" + n_criteria = len(matrix[0]) + f_best = [] + f_worst = [] + + for j in range(n_criteria): + column = [matrix[i][j] for i in range(len(matrix))] + f_best.append(max(column)) + f_worst.append(min(column)) + + return f_best, f_worst + + +# ============================================================================ +# 灵敏度分析 +# ============================================================================ + +class SensitivityAnalyzer: + """ + 灵敏度分析器 + + 分析权重变化对决策结果的影响。 + """ + + @staticmethod + def weight_sensitivity(alternatives: List[Alternative], + criteria: List[Criterion], + method: str = "TOPSIS", + perturbation: float = 0.1) -> Dict[str, Any]: + """ + 权重灵敏度分析 + + Args: + alternatives: 备选方案 + criteria: 准则列表 + method: 评价方法 + perturbation: 扰动幅度 + + Returns: + 灵敏度分析结果 + """ + # 原始权重 + original_weights = [c.weight for c in criteria] + n_criteria = len(criteria) + + # 原始排名 + if method == "TOPSIS": + evaluator = TOPSIS(criteria) + else: + evaluator = WeightedSumModel(criteria) + + original_results = evaluator.evaluate(alternatives) + original_ranking = [alt.id for alt, _ in original_results] + + # 分析每个准则的权重变化 + sensitivity_data = {} + + for i, criterion in enumerate(criteria): + # 增加权重 + weights_plus = original_weights.copy() + weights_plus[i] += perturbation + # 归一化 + total = sum(weights_plus) + weights_plus = [w / total for w in weights_plus] + + # 减少权重 + weights_minus = original_weights.copy() + weights_minus[i] = max(0, weights_minus[i] - perturbation) + total = sum(weights_minus) + weights_minus = [w / total for w in weights_minus] + + # 评估 + criteria_plus = [CriteriaWrapper(c, w) for c, w in zip(criteria, weights_plus)] + criteria_minus = [CriteriaWrapper(c, w) for c, w in zip(criteria, weights_minus)] + + if method == "TOPSIS": + evaluator_plus = TOPSIS(criteria_plus) + evaluator_minus = TOPSIS(criteria_minus) + else: + evaluator_plus = WeightedSumModel(criteria_plus) + evaluator_minus = WeightedSumModel(criteria_minus) + + results_plus = evaluator_plus.evaluate(alternatives) + results_minus = evaluator_minus.evaluate(alternatives) + + ranking_plus = [alt.id for alt, _ in results_plus] + ranking_minus = [alt.id for alt, _ in results_minus] + + # 计算排名变化 + rank_changes_plus = sum( + 1 for a, b in zip(original_ranking, ranking_plus) if a != b + ) + rank_changes_minus = sum( + 1 for a, b in zip(original_ranking, ranking_minus) if a != b + ) + + sensitivity_data[criterion.name] = { + "weight_change": perturbation, + "rank_changes_increase": rank_changes_plus, + "rank_changes_decrease": rank_changes_minus, + "sensitive": rank_changes_plus > 0 or rank_changes_minus > 0 + } + + return { + "original_ranking": original_ranking, + "sensitivity_data": sensitivity_data + } + + +class CriteriaWrapper: + """准则包装器，用于临时修改权重""" + def __init__(self, original: Criterion, weight: float): + self.name = original.name + self.criterion_type = original.criterion_type + self.weight = weight + self.scale = original.scale + self.optimal_value = original.optimal_value + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示多准则决策的使用""" + + print("="*70) + print("多准则决策示例演示") + print("="*70) + + # ======================================================================== + # 1. 定义问题 + # ======================================================================== + print("\n[部分 1] 商场选址决策问题") + print("-" * 50) + + # 定义准则 + criteria = [ + Criterion("人流量", CriterionType.BENEFIT, scale=(1000, 50000)), + Criterion("租金成本", CriterionType.COST, scale=(50, 200)), + Criterion("交通便利", CriterionType.BENEFIT, scale=(1, 10)), + Criterion("竞争强度", CriterionType.COST, scale=(0, 10)), + Criterion("发展潜力", CriterionType.BENEFIT, scale=(1, 10)) + ] + + print("\n评价准则:") + for i, c in enumerate(criteria, 1): + type_cn = "效益型" if c.criterion_type == CriterionType.BENEFIT else "成本型" + print(f" {i}. {c.name:8s} ({type_cn}): {c.scale}") + + # 定义备选方案 + alternatives = [ + Alternative("A1", "西湖商圈", { + "人流量": 45000, + "租金成本": 180, + "交通便利": 9, + "竞争强度": 8, + "发展潜力": 6 + }), + Alternative("A2", "滨江新城", { + "人流量": 25000, + "租金成本": 120, + "交通便利": 7, + "竞争强度": 4, + "发展潜力": 9 + }), + Alternative("A3", "萧山城区", { + "人流量": 18000, + "租金成本": 80, + "交通便利": 5, + "竞争强度": 3, + "发展潜力": 7 + }), + Alternative("A4", "城西商圈", { + "人流量": 32000, + "租金成本": 150, + "交通便利": 8, + "竞争强度": 6, + "发展潜力": 8 + }), + Alternative("A5", "下沙副中心", { + "人流量": 28000, + "租金成本": 100, + "交通便利": 6, + "竞争强度": 5, + "发展潜力": 7 + }) + ] + + print("\n备选方案:") + for alt in alternatives: + print(f" {alt.id}: {alt.name}") + for c in criteria: + print(f" {c.name}: {alt.get_value(c.name)}") + + # ======================================================================== + # 2. AHP确定权重 + # ======================================================================== + print("\n\n[部分 2] AHP层次分析法确定权重") + print("-" * 50) + + ahp = AHP([c.name for c in criteria]) + + # 构建比较矩阵 (专家判断) + comparisons = { + ("人流量", "租金成本"): 2, # 人流量稍微比租金重要 + ("人流量", "交通便利"): 3, # 人流量明显比交通重要 + ("人流量", "竞争强度"): 4, # 人流量比竞争重要 + ("人流量", "发展潜力"): 2, # 人流量稍微比发展潜力重要 + ("租金成本", "交通便利"): 2, # 租金稍微比交通重要 + ("租金成本", "竞争强度"): 2, # 租金稍微比竞争重要 + ("租金成本", "发展潜力"): 3, # 租金明显比发展潜力重要 + ("交通便利", "竞争强度"): 2, # 交通稍微比竞争重要 + ("交通便利", "发展潜力"): 2, # 交通稍微比发展潜力重要 + ("竞争强度", "发展潜力"): 2, # 竞争稍微比发展潜力重要 + } + + ahp.build_comparison_matrix(comparisons) + weights, cr, lambda_max = ahp.calculate_weights() + + print(f"\nAHP权重计算结果:") + print(f" 最大特征值: {lambda_max:.4f}") + print(f" 一致性比率 CR: {cr:.4f}", end="") + if cr < 0.1: + print(" (通过一致性检验)") + else: + print(" (未通过一致性检验)") + + print(f"\n准则权重:") + for i, (name, weight) in enumerate(zip([c.name for c in criteria], weights)): + criteria[i].weight = weight + print(f" {name:8s}: {weight:.4f}") + + # ======================================================================== + # 3. 熵权法确定权重 + # ======================================================================== + print("\n\n[部分 3] 熵权法确定客观权重") + print("-" * 50) + + # 构建决策矩阵 + decision_matrix = [ + [alt.get_value(c.name) for c in criteria] + for alt in alternatives + ] + + entropy_weights = EntropyWeightMethod.calculate_weights( + decision_matrix, + [c.criterion_type for c in criteria] + ) + + print(f"\n熵权法计算结果:") + for name, weight in zip([c.name for c in criteria], entropy_weights): + print(f" {name:8s}: {weight:.4f}") + + # 组合权重 (AHP 0.6 + 熵权 0.4) + print(f"\n组合权重 (AHP 60% + 熵权 40%):") + for i, c in enumerate(criteria): + combined_weight = 0.6 * c.weight + 0.4 * entropy_weights[i] + c.weight = combined_weight + print(f" {c.name:8s}: {combined_weight:.4f}") + + # ======================================================================== + # 4. TOPSIS评价 + # ======================================================================== + print("\n\n[部分 4] TOPSIS评价结果") + print("-" * 50) + + topsis = TOPSIS(criteria) + topsis_results = topsis.evaluate(alternatives) + + print(f"\nTOPSIS排名:") + print(f"{'排名':<6} {'方案ID':<8} {'名称':<12} {'贴近度':<10}") + print("-" * 40) + for i, (alt, score) in enumerate(topsis_results, 1): + print(f"{i:<6} {alt.id:<8} {alt.name:<12} {score:<10.4f}") + + # ======================================================================== + # 5. WSM评价 + # ======================================================================== + print("\n\n[部分 5] 加权求和模型(WSM)评价结果") + print("-" * 50) + + wsm = WeightedSumModel(criteria) + wsm_results = wsm.evaluate(alternatives) + + print(f"\nWSM排名:") + print(f"{'排名':<6} {'方案ID':<8} {'名称':<12} {'得分':<10}") + print("-" * 40) + for i, (alt, score) in enumerate(wsm_results, 1): + print(f"{i:<6} {alt.id:<8} {alt.name:<12} {score:<10.4f}") + + # ======================================================================== + # 6. VIKOR评价 + # ======================================================================== + print("\n\n[部分 6] VIKOR评价结果") + print("-" * 50) + + vikor = VIKOR(criteria, v=0.5) + vikor_results = vikor.evaluate(alternatives) + + print(f"\nVIKOR排名 (Q值越小越好):") + print(f"{'排名':<6} {'方案ID':<8} {'名称':<12} {'Q值':<10}") + print("-" * 40) + for i, (alt, score) in enumerate(vikor_results, 1): + print(f"{i:<6} {alt.id:<8} {alt.name:<12} {score:<10.4f}") + + # ======================================================================== + # 7. 方法比较 + # ======================================================================== + print("\n\n[部分 7] 不同方法排名比较") + print("-" * 50) + + print(f"\n{'方案':<12} {'TOPSIS':<8} {'WSM':<8} {'VIKOR':<8}") + print("-" * 40) + + for alt in alternatives: + topsis_rank = next(i for i, (a, _) in enumerate(topsis_results, 1) if a.id == alt.id) + wsm_rank = next(i for i, (a, _) in enumerate(wsm_results, 1) if a.id == alt.id) + vikor_rank = next(i for i, (a, _) in enumerate(vikor_results, 1) if a.id == alt.id) + + print(f"{alt.name:<12} {topsis_rank:<8} {wsm_rank:<8} {vikor_rank:<8}") + + # ======================================================================== + # 8. 灵敏度分析 + # ======================================================================== + print("\n\n[部分 8] 权重灵敏度分析") + print("-" * 50) + + sensitivity = SensitivityAnalyzer.weight_sensitivity( + alternatives, criteria, method="TOPSIS", perturbation=0.2 + ) + + print(f"\n权重变化 ±20% 对排名的影响:") + print(f"{'准则':<10} {'排名变化':<12} {'敏感':<6}") + print("-" * 30) + + for name, data in sensitivity["sensitivity_data"].items(): + max_changes = max(data["rank_changes_increase"], data["rank_changes_decrease"]) + sensitive = "是" if data["sensitive"] else "否" + print(f"{name:<10} {max_changes:<12} {sensitive:<6}") + + # ======================================================================== + # 9. 决策建议 + # ======================================================================== + print("\n\n[部分 9] 决策建议") + print("-" * 50) + + best_topsis = topsis_results[0][0] + best_wsm = wsm_results[0][0] + best_vikor = vikor_results[0][0] + + print(f"\n各方法推荐的最佳方案:") + print(f" TOPSIS: {best_topsis.name}") + print(f" WSM: {best_wsm.name}") + print(f" VIKOR: {best_vikor.name}") + + # 综合推荐 + vote_counts = {} + for alt in [best_topsis, best_wsm, best_vikor]: + vote_counts[alt.id] = vote_counts.get(alt.id, 0) + 1 + + recommended = max(vote_counts.items(), key=lambda x: x[1])[0] + recommended_alt = next(alt for alt in alternatives if alt.id == recommended) + + print(f"\n综合推荐: {recommended_alt.name}") + print(f" 理由: 该方案在多种评价方法中表现最佳") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/02-spatial-intelligence/spatial_optimization.py b/dofile/examples/02-spatial-intelligence/spatial_optimization.py new file mode 100644 index 0000000..076dc20 --- /dev/null +++ b/dofile/examples/02-spatial-intelligence/spatial_optimization.py @@ -0,0 +1,961 @@ +""" +空间优化示例 (Spatial Optimization Example) +========================================== + +本示例展示空间智能系统中的空间优化方法。 +空间优化是在空间约束下寻找最优解的过程。 + +核心概念: +1. 目标函数 - 需要最大化或最小化的目标 +2. 约束条件 - 空间和非空间限制 +3. 决策变量 - 可控制的空间变量 +4. 优化算法 - 求解最优解的方法 +5. 帕累托前沿 - 多目标优化中的解集 + +应用场景: +- 选址优化 +- 路径优化 +- 空间配置优化 +- 资源分配优化 + +作者: CC4SI 项目组 +""" + +import math +import random +from typing import List, Dict, Tuple, Optional, Any, Callable +from dataclasses import dataclass, field +from enum import Enum +from abc import ABC, abstractmethod +import heapq + + +# ============================================================================ +# 基础数据结构 +# ============================================================================ + +@dataclass +class Point: + """二维点""" + x: float + y: float + + def distance_to(self, other: 'Point') -> float: + return math.sqrt((self.x - other.x)**2 + (self.y - other.y)**2) + + def __repr__(self) -> str: + return f"({self.x:.2f}, {self.y:.2f})" + + +@dataclass +class DemandPoint: + """需求点""" + id: str + location: Point + demand: float # 需求量 + population: int = 0 + + def __repr__(self) -> str: + return f"Demand({self.id}, demand={self.demand})" + + +@dataclass +class Facility: + """设施""" + id: str + location: Point + capacity: float # 服务能力 + fixed_cost: float = 0 # 固定成本 + variable_cost: float = 0 # 单位可变成本 + + def __repr__(self) -> str: + return f"Facility({self.id}, loc={self.location})" + + +@dataclass +class OptimizationResult: + """优化结果""" + success: bool + objective_value: float + solution: Any + iterations: int = 0 + convergence_history: List[float] = field(default_factory=list) + metadata: Dict[str, Any] = field(default_factory=dict) + + +# ============================================================================ +# 约束条件 +# ============================================================================ + +class ConstraintType(Enum): + """约束类型""" + EQUALITY = "equality" # 等式约束 + INEQUALITY = "inequality" # 不等式约束 + BOUNDS = "bounds" # 边界约束 + + +@dataclass +class Constraint: + """约束条件""" + name: str + constraint_type: ConstraintType + rhs: float # 右端值 + lhs_function: Optional[Callable[[Any], float]] = None # 左端函数 + + def is_satisfied(self, variables: Any, tolerance: float = 1e-6) -> bool: + """检查约束是否满足""" + if self.lhs_function is None: + return True + + lhs = self.lhs_function(variables) + + if self.constraint_type == ConstraintType.EQUALITY: + return abs(lhs - self.rhs) < tolerance + elif self.constraint_type == ConstraintType.INEQUALITY: + return lhs <= self.rhs + tolerance + + return True + + +# ============================================================================ +# 优化问题定义 +# ============================================================================ + +class OptimizationProblem(ABC): + """优化问题抽象基类""" + + def __init__(self, name: str = ""): + self.name = name + self.constraints: List[Constraint] = [] + self.objective_calls = 0 + + @abstractmethod + def objective(self, variables: Any) -> float: + """目标函数""" + pass + + @abstractmethod + def get_initial_solution(self) -> Any: + """获取初始解""" + pass + + def add_constraint(self, constraint: Constraint) -> None: + """添加约束""" + self.constraints.append(constraint) + + def is_feasible(self, variables: Any) -> bool: + """检查解是否可行""" + return all(c.is_satisfied(variables) for c in self.constraints) + + +# ============================================================================ +# 贪心算法 +# ============================================================================ + +class GreedyOptimizer: + """ + 贪心优化器 + + 每一步选择当前最优的选项。 + """ + + def __init__(self, problem: OptimizationProblem): + self.problem = problem + + def optimize(self, max_iterations: int = 1000) -> OptimizationResult: + """ + 执行贪心优化 + + Args: + max_iterations: 最大迭代次数 + + Returns: + 优化结果 + """ + current_solution = self.problem.get_initial_solution() + current_value = self.problem.objective(current_solution) + + history = [current_value] + + for iteration in range(max_iterations): + # 生成邻居解 + neighbors = self._generate_neighbors(current_solution) + + # 找最优邻居 + best_neighbor = None + best_neighbor_value = float('inf') + + for neighbor in neighbors: + if self.problem.is_feasible(neighbor): + value = self.problem.objective(neighbor) + if value < best_neighbor_value: + best_neighbor = neighbor + best_neighbor_value = value + + # 如果没有改进，停止 + if best_neighbor is None or best_neighbor_value >= current_value: + break + + current_solution = best_neighbor + current_value = best_neighbor_value + history.append(current_value) + + return OptimizationResult( + success=True, + objective_value=current_value, + solution=current_solution, + iterations=len(history), + convergence_history=history + ) + + def _generate_neighbors(self, solution: Any) -> List[Any]: + """生成邻居解 (需要根据具体问题实现)""" + return [] + + +# ============================================================================ +# 模拟退火算法 +# ============================================================================ + +class SimulatedAnnealing: + """ + 模拟退火算法 + + 一种概率性全局优化算法，能够跳出局部最优。 + """ + + def __init__(self, problem: OptimizationProblem): + self.problem = problem + + def optimize(self, + initial_temp: float = 1000.0, + cooling_rate: float = 0.95, + min_temp: float = 0.01, + max_iterations: int = 10000) -> OptimizationResult: + """ + 执行模拟退火优化 + + Args: + initial_temp: 初始温度 + cooling_rate: 冷却速率 + min_temp: 最小温度 + max_iterations: 最大迭代次数 + + Returns: + 优化结果 + """ + current_solution = self.problem.get_initial_solution() + current_value = self.problem.objective(current_solution) + + best_solution = current_solution + best_value = current_value + + temperature = initial_temp + history = [current_value] + + iteration = 0 + while temperature > min_temp and iteration < max_iterations: + # 生成邻居解 + neighbor = self._generate_neighbor(current_solution) + + if self.problem.is_feasible(neighbor): + neighbor_value = self.problem.objective(neighbor) + + # 决定是否接受新解 + delta = neighbor_value - current_value + + if delta < 0 or random.random() < math.exp(-delta / temperature): + current_solution = neighbor + current_value = neighbor_value + + # 更新最优解 + if current_value < best_value: + best_solution = current_solution + best_value = current_value + + history.append(best_value) + temperature *= cooling_rate + iteration += 1 + + return OptimizationResult( + success=True, + objective_value=best_value, + solution=best_solution, + iterations=iteration, + convergence_history=history + ) + + def _generate_neighbor(self, solution: Any) -> Any: + """生成邻居解 (需要根据具体问题实现)""" + return solution + + +# ============================================================================ +# 遗传算法 +# ============================================================================= + +class GeneticAlgorithm: + """ + 遗传算法 + + 模拟自然进化的全局优化算法。 + """ + + def __init__(self, problem: OptimizationProblem): + self.problem = problem + + def optimize(self, + population_size: int = 50, + generations: int = 100, + mutation_rate: float = 0.1, + crossover_rate: float = 0.8, + elitism_count: int = 2) -> OptimizationResult: + """ + 执行遗传算法优化 + + Args: + population_size: 种群大小 + generations: 迭代代数 + mutation_rate: 变异率 + crossover_rate: 交叉率 + elitism_count: 精英保留数量 + + Returns: + 优化结果 + """ + # 初始化种群 + population = self._initialize_population(population_size) + history = [] + + for generation in range(generations): + # 评估适应度 + fitness = [] + for individual in population: + value = self.problem.objective(individual) + fitness.append(1.0 / (1.0 + value)) # 转换为适应度 (越小越好 -> 越大越好) + + # 记录最优值 + best_idx = max(range(len(fitness)), key=lambda i: fitness[i]) + best_value = self.problem.objective(population[best_idx]) + history.append(best_value) + + # 选择 + selected = self._selection(population, fitness) + + # 交叉 + offspring = self._crossover(selected, crossover_rate) + + # 变异 + offspring = self._mutation(offspring, mutation_rate) + + # 精英保留 + if elitism_count > 0: + elite_indices = sorted(range(len(fitness)), + key=lambda i: fitness[i], reverse=True)[:elitism_count] + for i, idx in enumerate(elite_indices): + offspring[i] = population[idx] + + population = offspring + + # 返回最优解 + final_fitness = [self.problem.objective(ind) for ind in population] + best_idx = min(range(len(final_fitness)), key=lambda i: final_fitness[i]) + + return OptimizationResult( + success=True, + objective_value=final_fitness[best_idx], + solution=population[best_idx], + iterations=generations, + convergence_history=history + ) + + def _initialize_population(self, size: int) -> List[Any]: + """初始化种群""" + return [self.problem.get_initial_solution() for _ in range(size)] + + def _selection(self, population: List[Any], fitness: List[float]) -> List[Any]: + """锦标赛选择""" + selected = [] + tournament_size = max(3, len(population) // 10) + + for _ in range(len(population)): + # 随机选择tournament_size个个体 + contestants = random.sample(list(zip(population, fitness)), tournament_size) + # 选择适应度最高的 + winner = max(contestants, key=lambda x: x[1])[0] + selected.append(winner) + + return selected + + def _crossover(self, population: List[Any], rate: float) -> List[Any]: + """交叉操作""" + offspring = [] + + for i in range(0, len(population), 2): + parent1 = population[i] + parent2 = population[i + 1] if i + 1 < len(population) else population[0] + + if random.random() < rate: + child1, child2 = self._crossover_operators(parent1, parent2) + else: + child1, child2 = parent1, parent2 + + offspring.extend([child1, child2]) + + return offspring[:len(population)] + + def _crossover_operators(self, parent1: Any, parent2: Any) -> Tuple[Any, Any]: + """交叉算子 (需要根据具体问题实现)""" + return parent1, parent2 + + def _mutation(self, population: List[Any], rate: float) -> List[Any]: + """变异操作""" + mutated = [] + + for individual in population: + if random.random() < rate: + mutated.append(self._mutate(individual)) + else: + mutated.append(individual) + + return mutated + + def _mutate(self, individual: Any) -> Any: + """变异算子 (需要根据具体问题实现)""" + return individual + + +# ============================================================================ +# 选址优化问题 +# ============================================================================ + +class LocationProblem(OptimizationProblem): + """ + 设施选址问题 + + 在给定候选位置中选择最优的设施位置组合。 + """ + + def __init__(self, + demand_points: List[DemandPoint], + candidate_locations: List[Point], + num_facilities: int, + fixed_costs: List[float] = None, + transportation_cost: float = 1.0): + """ + 初始化选址问题 + + Args: + demand_points: 需求点列表 + candidate_locations: 候选位置列表 + num_facilities: 设施数量 + fixed_costs: 各候选位置的固定成本 + transportation_cost: 单位运输成本 + """ + super().__init__("设施选址问题") + self.demand_points = demand_points + self.candidate_locations = candidate_locations + self.num_facilities = num_facilities + self.fixed_costs = fixed_costs or [0] * len(candidate_locations) + self.transportation_cost = transportation_cost + + def objective(self, solution: List[int]) -> float: + """ + 计算目标函数值 + + Args: + solution: 选中的候选位置索引列表 + + Returns: + 总成本 + """ + self.objective_calls += 1 + + if not solution or len(solution) != self.num_facilities: + return float('inf') + + total_cost = 0.0 + + # 固定成本 + for idx in solution: + if 0 <= idx < len(self.fixed_costs): + total_cost += self.fixed_costs[idx] + + # 运输成本 (每个需求点分配到最近的设施) + for demand in self.demand_points: + min_dist = float('inf') + for facility_idx in solution: + if 0 <= facility_idx < len(self.candidate_locations): + facility_loc = self.candidate_locations[facility_idx] + dist = demand.location.distance_to(facility_loc) + min_dist = min(min_dist, dist) + + total_cost += min_dist * demand.demand * self.transportation_cost + + return total_cost + + def get_initial_solution(self) -> List[int]: + """获取初始解 (随机选择)""" + n = len(self.candidate_locations) + if n <= self.num_facilities: + return list(range(n)) + return random.sample(range(n), self.num_facilities) + + +class LocationSAOptimizer(SimulatedAnnealing): + """针对选址问题的模拟退火优化器""" + + def _generate_neighbor(self, solution: List[int]) -> List[int]: + """生成邻居解""" + if not solution: + return solution + + neighbor = solution.copy() + n = len(self.problem.candidate_locations) + + # 随机选择一个操作 + operation = random.choice(['replace', 'swap']) + + if operation == 'replace': + # 替换一个设施 + idx = random.randint(0, len(neighbor) - 1) + available = [i for i in range(n) if i not in neighbor] + if available: + neighbor[idx] = random.choice(available) + + elif operation == 'swap' and len(neighbor) >= 1: + # 交换一个设施 + idx = random.randint(0, len(neighbor) - 1) + available = [i for i in range(n) if i not in neighbor] + if available: + neighbor[idx] = random.choice(available) + + return neighbor + + +# ============================================================================ +# P-中值问题 +# ============================================================================ + +class PMedianProblem(LocationProblem): + """ + P-中值问题 + + 选择P个设施位置，使需求点到最近设施的总加权距离最小。 + """ + + def __init__(self, + demand_points: List[DemandPoint], + candidate_locations: List[Point], + p: int): + super().__init__(demand_points, candidate_locations, p, [0] * len(candidate_locations)) + + +# ============================================================================ +# 覆盖问题 +# ============================================================================ + +class MaxCoverageProblem(OptimizationProblem): + """ + 最大覆盖问题 + + 在给定设施数量限制下，最大化覆盖的需求量。 + """ + + def __init__(self, + demand_points: List[DemandPoint], + candidate_locations: List[Point], + num_facilities: int, + coverage_radius: float): + """ + 初始化最大覆盖问题 + + Args: + demand_points: 需求点列表 + candidate_locations: 候选位置列表 + num_facilities: 设施数量 + coverage_radius: 覆盖半径 + """ + super().__init__("最大覆盖问题") + self.demand_points = demand_points + self.candidate_locations = candidate_locations + self.num_facilities = num_facilities + self.coverage_radius = coverage_radius + + # 预计算覆盖关系 + self.coverage_matrix = self._compute_coverage() + + def _compute_coverage(self) -> List[List[bool]]: + """计算覆盖矩阵""" + matrix = [] + for loc in self.candidate_locations: + coverage = [] + for demand in self.demand_points: + covered = loc.distance_to(demand.location) <= self.coverage_radius + coverage.append(covered) + matrix.append(coverage) + return matrix + + def objective(self, solution: List[int]) -> float: + """ + 计算覆盖的需求量 (负值，因为算法最小化) + + Args: + solution: 选中的候选位置索引列表 + + Returns: + 负的覆盖需求量 + """ + covered = [False] * len(self.demand_points) + + for facility_idx in solution: + if 0 <= facility_idx < len(self.coverage_matrix): + for j, is_covered in enumerate(self.coverage_matrix[facility_idx]): + if is_covered: + covered[j] = True + + total_demand = sum( + self.demand_points[j].demand + for j, c in enumerate(covered) if c + ) + + return -total_demand # 负值用于最小化 + + def get_initial_solution(self) -> List[int]: + """获取初始解""" + n = len(self.candidate_locations) + if n <= self.num_facilities: + return list(range(n)) + return random.sample(range(n), self.num_facilities) + + +class MaxCoverageGAOptimizer(GeneticAlgorithm): + """针对最大覆盖问题的遗传算法优化器""" + + def _crossover_operators(self, parent1: List[int], parent2: List[int]) -> Tuple[List[int], List[int]]: + """单点交叉""" + if not parent1 or not parent2: + return parent1, parent2 + + size = min(len(parent1), len(parent2)) + if size < 2: + return parent1, parent2 + + point = random.randint(1, size - 1) + + child1 = parent1[:point] + [x for x in parent2[point:] if x not in parent1[:point]] + child2 = parent2[:point] + [x for x in parent1[point:] if x not in parent2[:point]] + + # 补足长度 + all_indices = set(range(len(self.problem.candidate_locations))) + while len(child1) < len(parent1): + available = list(all_indices - set(child1)) + if available: + child1.append(random.choice(available)) + else: + break + + while len(child2) < len(parent2): + available = list(all_indices - set(child2)) + if available: + child2.append(random.choice(available)) + else: + break + + return child1[:len(parent1)], child2[:len(parent2)] + + def _mutate(self, individual: List[int]) -> List[int]: + """变异操作""" + if not individual: + return individual + + mutated = individual.copy() + n = len(self.problem.candidate_locations) + + # 随机替换一个基因 + idx = random.randint(0, len(mutated) - 1) + available = [i for i in range(n) if i not in mutated] + if available: + mutated[idx] = random.choice(available) + + return mutated + + +# ============================================================================ +# 路径优化 (TSP) +# ============================================================================ + +class TSPProblem(OptimizationProblem): + """ + 旅行商问题 (TSP) + + 寻找访问所有城市的最短路径。 + """ + + def __init__(self, cities: List[Point]): + """ + 初始化TSP问题 + + Args: + cities: 城市位置列表 + """ + super().__init__("旅行商问题") + self.cities = cities + self.n = len(cities) + + def objective(self, solution: List[int]) -> float: + """ + 计算路径总长度 + + Args: + solution: 城市访问顺序列表 + + Returns: + 路径总长度 + """ + if not solution or len(solution) != self.n: + return float('inf') + + total = 0.0 + for i in range(len(solution)): + from_idx = solution[i] + to_idx = solution[(i + 1) % len(solution)] + total += self.cities[from_idx].distance_to(self.cities[to_idx]) + + return total + + def get_initial_solution(self) -> List[int]: + """获取初始解 (随机顺序)""" + solution = list(range(self.n)) + random.shuffle(solution) + return solution + + +class TSPSAOptimizer(SimulatedAnnealing): + """针对TSP的模拟退火优化器""" + + def _generate_neighbor(self, solution: List[int]) -> List[int]: + """生成邻居解 (2-opt交换)""" + if len(solution) < 2: + return solution + + neighbor = solution.copy() + + # 随机选择两个位置并交换 + i, j = random.sample(range(len(neighbor)), 2) + neighbor[i], neighbor[j] = neighbor[j], neighbor[i] + + return neighbor + + +# ============================================================================ +# 主程序 +# ======================================================================== + +def main(): + """主程序 - 演示空间优化的使用""" + + print("="*70) + print("空间优化示例演示") + print("="*70) + + random.seed(42) + + # ======================================================================== + # 1. P-中值问题 (选址优化) + # ======================================================================== + print("\n[部分 1] P-中值问题 - 商场选址优化") + print("-" * 50) + + # 生成需求点 + demand_points = [] + for i in range(20): + demand_points.append(DemandPoint( + id=f"D{i}", + location=Point(random.uniform(0, 100), random.uniform(0, 100)), + demand=random.uniform(50, 200) + )) + + print(f"\n需求点数量: {len(demand_points)}") + print(f"总需求量: {sum(d.demand for d in demand_points):.1f}") + + # 候选位置 + candidate_locations = [ + Point(20, 20), Point(50, 20), Point(80, 20), + Point(20, 50), Point(50, 50), Point(80, 50), + Point(20, 80), Point(50, 80), Point(80, 80) + ] + + print(f"\n候选位置数量: {len(candidate_locations)}") + print("候选位置:", [str(loc) for loc in candidate_locations]) + + # 创建P-中值问题 (选择3个位置) + p_median = PMedianProblem(demand_points, candidate_locations, p=3) + + # 使用模拟退火求解 + sa_optimizer = LocationSAOptimizer(p_median) + sa_result = sa_optimizer.optimize( + initial_temp=100, + cooling_rate=0.95, + min_temp=0.1, + max_iterations=1000 + ) + + print(f"\n模拟退火结果:") + print(f" 目标函数值: {sa_result.objective_value:.2f}") + print(f" 迭代次数: {sa_result.iterations}") + print(f" 选中的位置: {[str(candidate_locations[i]) for i in sa_result.solution]}") + + # 贪心算法对比 + greedy_optimizer = GreedyOptimizer(p_median) + + # 简单的贪心: 逐步添加最优位置 + best_solution = None + best_value = float('inf') + + for _ in range(100): + solution = p_median.get_initial_solution() + value = p_median.objective(solution) + if value < best_value: + best_value = value + best_solution = solution + + print(f"\n随机搜索对比:") + print(f" 目标函数值: {best_value:.2f}") + print(f" 选中的位置: {[str(candidate_locations[i]) for i in best_solution]}") + + # ======================================================================== + # 2. 最大覆盖问题 + # ======================================================================== + print("\n\n[部分 2] 最大覆盖问题 - 5G基站选址") + print("-" * 50) + + # 创建更大规模的需求点 + coverage_demands = [] + for i in range(50): + coverage_demands.append(DemandPoint( + id=f"C{i}", + location=Point(random.uniform(0, 100), random.uniform(0, 100)), + demand=random.uniform(10, 100) + )) + + coverage_radius = 25 + num_bases = 5 + + max_coverage = MaxCoverageProblem( + coverage_demands, + candidate_locations, + num_bases, + coverage_radius + ) + + # 使用遗传算法求解 + ga_optimizer = MaxCoverageGAOptimizer(max_coverage) + ga_result = ga_optimizer.optimize( + population_size=50, + generations=100, + mutation_rate=0.1 + ) + + covered_demand = -ga_result.objective_value + total_demand = sum(d.demand for d in coverage_demands) + coverage_ratio = covered_demand / total_demand * 100 + + print(f"\n遗传算法结果:") + print(f" 覆盖需求量: {covered_demand:.1f} / {total_demand:.1f}") + print(f" 覆盖率: {coverage_ratio:.1f}%") + print(f" 迭代次数: {ga_result.iterations}") + print(f" 选中的位置: {[str(candidate_locations[i]) for i in ga_result.solution]}") + + # ======================================================================== + # 3. 旅行商问题 (TSP) + # ======================================================================== + print("\n\n[部分 3] 旅行商问题 - 配送路线优化") + print("-" * 50) + + # 生成城市 + cities = [] + for i in range(15): + cities.append(Point(random.uniform(0, 100), random.uniform(0, 100))) + + print(f"\n城市数量: {len(cities)}") + print(f"城市位置: {[str(city) for city in cities[:5]]}...") + + tsp = TSPProblem(cities) + tsp_optimizer = TSPSAOptimizer(tsp) + tsp_result = tsp_optimizer.optimize( + initial_temp=1000, + cooling_rate=0.99, + min_temp=0.01, + max_iterations=5000 + ) + + print(f"\n模拟退火结果:") + print(f" 最短路径长度: {tsp_result.objective_value:.2f}") + print(f" 访问顺序: {[cities[i].__repr__() for i in tsp_result.solution[:5]]}...") + + # 对比随机解 + random_solution = list(range(len(cities))) + random.shuffle(random_solution) + random_length = tsp.objective(random_solution) + print(f"\n随机解对比:") + print(f" 路径长度: {random_length:.2f}") + print(f" 改进: {(1 - tsp_result.objective_value / random_length) * 100:.1f}%") + + # ======================================================================== + # 4. 多目标优化讨论 + # ======================================================================== + print("\n\n[部分 4] 多目标优化说明") + print("-" * 50) + + print(""" +在实际应用中，空间优化往往涉及多个目标: + +1. 成本最小化 + - 设施建设成本 + - 运营成本 + - 运输成本 + +2. 服务最大化 + - 覆盖范围 + - 服务质量 + - 响应时间 + +3. 公平性 + - 服务均等化 + - 负载均衡 + +4. 环境影响 + - 最小化污染 + - 保护生态 + +处理方法: +- 加权求和法 (将多目标转为单目标) +- 帕累托优化 (寻找非劣解集) +- 约束法 (将部分目标转为约束) +- 目标规划 (设定目标满意水平) + """) + + # ======================================================================== + # 5. 收敛过程可视化 (文本) + # ======================================================================== + print("\n[部分 5] 优化过程") + print("-" * 50) + + if len(ga_result.convergence_history) > 0: + print("\n遗传算法收敛过程:") + steps = min(10, len(ga_result.convergence_history)) + step_size = len(ga_result.convergence_history) // steps + + for i in range(0, len(ga_result.convergence_history), step_size): + iteration = i + value = -ga_result.convergence_history[i] # 转回正值 + print(f" 迭代 {iteration:4d}: 覆盖需求量 = {value:.1f}") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/02-spatial-intelligence/spatial_reasoning.py b/dofile/examples/02-spatial-intelligence/spatial_reasoning.py new file mode 100644 index 0000000..03b01cd --- /dev/null +++ b/dofile/examples/02-spatial-intelligence/spatial_reasoning.py @@ -0,0 +1,957 @@ +""" +空间推理示例 (Spatial Reasoning Example) +======================================= + +本示例展示空间智能系统中的空间推理方法。 +空间推理是从已知空间事实推导新知识的过程。 + +核心概念: +1. 定性推理 - 使用定性术语描述空间关系 +2. 定量推理 - 使用精确数值计算 +3. 空间逻辑 - 形式化的空间推理规则 +4. 路径规划 - 寻找最优路径 +5. 可见性分析 - 判断视线可见性 + +应用场景: +- 导航与路径规划 +- 空间查询与分析 +- 地理推理系统 +- 机器人导航 + +作者: CC4SI 项目组 +""" + +import math +import heapq +from typing import List, Dict, Tuple, Optional, Set, Any +from dataclasses import dataclass, field +from enum import Enum +from abc import ABC, abstractmethod +import random + +# 导入空间表征示例中的基础类 +import sys +import os +sys.path.append(os.path.dirname(__file__)) + +try: + from spatial_representation import Point, LineString, Polygon, Envelope +except ImportError: + # 如果导入失败，定义简化版本 + @dataclass + class Point: + x: float + y: float + def distance_to(self, other): + return math.sqrt((self.x - other.x)**2 + (self.y - other.y)**2) + def __repr__(self): + return f"({self.x:.2f}, {self.y:.2f})" + + +# ============================================================================ +# 定性空间推理 +# ============================================================================ + +class CardinalDirection(Enum): + """基本方向""" + NORTH = "N" + SOUTH = "S" + EAST = "E" + WEST = "W" + NORTHEAST = "NE" + NORTHWEST = "NW" + SOUTHEAST = "SE" + SOUTHWEST = "SW" + + @classmethod + def from_angle(cls, angle: float) -> 'CardinalDirection': + """ + 从角度获取方向 + + Args: + angle: 角度 (度, 0=东, 90=北) + + Returns: + 方向枚举 + """ + # 归一化到0-360 + angle = angle % 360 + + if angle >= 337.5 or angle < 22.5: + return cls.EAST + elif 22.5 <= angle < 67.5: + return cls.NORTHEAST + elif 67.5 <= angle < 112.5: + return cls.NORTH + elif 112.5 <= angle < 157.5: + return cls.NORTHWEST + elif 157.5 <= angle < 202.5: + return cls.WEST + elif 202.5 <= angle < 247.5: + return cls.SOUTHWEST + elif 247.5 <= angle < 292.5: + return cls.SOUTH + else: # 292.5 <= angle < 337.5 + return cls.SOUTHEAST + + +@dataclass +class QualitativeRelation: + """定性空间关系""" + relation_type: str # "direction", "distance", "topology" + value: str # 如 "north", "near", "inside" + confidence: float = 1.0 + + def __repr__(self) -> str: + return f"{self.relation_type}={self.value} (conf={self.confidence:.2f})" + + +class QualitativeReasoner: + """ + 定性空间推理器 + + 使用定性术语进行空间推理。 + """ + + def __init__(self): + self.facts: List[Tuple[str, str, QualitativeRelation]] = [] + + def add_fact(self, entity1: str, entity2: str, + relation: QualitativeRelation) -> None: + """添加空间事实""" + self.facts.append((entity1, entity2, relation)) + + def infer_direction(self, from_point: Point, to_point: Point) -> QualitativeRelation: + """ + 推断两点之间的方向关系 + + Args: + from_point: 起始点 + to_point: 目标点 + + Returns: + 方向关系 + """ + dx = to_point.x - from_point.x + dy = to_point.y - from_point.y + + # 计算角度 (从东开始逆时针) + angle = math.degrees(math.atan2(dy, dx)) + + direction = CardinalDirection.from_angle(angle) + + return QualitativeRelation( + relation_type="direction", + value=direction.value, + confidence=1.0 + ) + + def infer_distance_category(self, p1: Point, p2: Point, + thresholds: Dict[str, float] = None) -> QualitativeRelation: + """ + 推断距离类别 + + Args: + p1: 第一个点 + p2: 第二个点 + thresholds: 距离阈值字典 + + Returns: + 距离类别关系 + """ + if thresholds is None: + thresholds = { + "very_close": 100, + "close": 500, + "moderate": 1000, + "far": 5000 + } + + dist = p1.distance_to(p2) + + if dist < thresholds.get("very_close", 100): + category = "very_close" + elif dist < thresholds.get("close", 500): + category = "close" + elif dist < thresholds.get("moderate", 1000): + category = "moderate" + else: + category = "far" + + return QualitativeRelation( + relation_type="distance", + value=category, + confidence=1.0 + ) + + def compose_relations(self, rel1: QualitativeRelation, + rel2: QualitativeRelation) -> QualitativeRelation: + """ + 组合两个关系 + + 例如: A在B的北边，B在C的东边 -> A在C的东北边 + """ + if rel1.relation_type == "direction" and rel2.relation_type == "direction": + # 方向组合 + return self._compose_directions(rel1.value, rel2.value) + + return QualitativeRelation( + relation_type="unknown", + value="unknown", + confidence=0.5 + ) + + def _compose_directions(self, dir1: str, dir2: str) -> QualitativeRelation: + """组合两个方向""" + direction_map = { + "N": (0, 1), "S": (0, -1), "E": (1, 0), "W": (-1, 0), + "NE": (1, 1), "NW": (-1, 1), "SE": (1, -1), "SW": (-1, -1) + } + + if dir1 in direction_map and dir2 in direction_map: + v1 = direction_map[dir1] + v2 = direction_map[dir2] + + # 向量相加 + result = (v1[0] + v2[0], v1[1] + v2[1]) + + # 找到最接近的方向 + best_dir = "unknown" + best_dot = -1 + + for name, vec in direction_map.items(): + dot = result[0] * vec[0] + result[1] * vec[1] + if dot > best_dot: + best_dot = dot + best_dir = name + + return QualitativeRelation( + relation_type="direction", + value=best_dir, + confidence=0.8 + ) + + return QualitativeRelation("direction", "unknown", 0.3) + + +# ============================================================================ +# 路径规划 +# ============================================================================ + +@dataclass +class Node: + """图节点""" + id: str + point: Point + neighbors: List[str] = field(default_factory=list) + + def __hash__(self): + return hash(self.id) + + +@dataclass +class Edge: + """图边""" + from_node: str + to_node: str + weight: float # 权重 (如距离) + bidirectional: bool = True + + +class SpatialGraph: + """ + 空间图 + + 用于路径规划的空间网络结构。 + """ + + def __init__(self): + self.nodes: Dict[str, Node] = {} + self.edges: List[Edge] = [] + + def add_node(self, id: str, point: Point) -> Node: + """添加节点""" + node = Node(id=id, point=point) + self.nodes[id] = node + return node + + def add_edge(self, from_id: str, to_id: str, weight: float = None, + bidirectional: bool = True) -> None: + """添加边""" + if from_id not in self.nodes or to_id not in self.nodes: + raise ValueError("节点不存在") + + # 如果未指定权重，使用欧氏距离 + if weight is None: + weight = self.nodes[from_id].point.distance_to(self.nodes[to_id].point) + + edge = Edge(from_id, to_id, weight, bidirectional) + self.edges.append(edge) + + # 更新邻接关系 + self.nodes[from_id].neighbors.append(to_id) + if bidirectional: + self.nodes[to_id].neighbors.append(from_id) + + def get_edge_weight(self, from_id: str, to_id: str) -> float: + """获取边的权重""" + for edge in self.edges: + if edge.from_node == from_id and edge.to_node == to_id: + return edge.weight + if edge.bidirectional and edge.from_node == to_id and edge.to_node == from_id: + return edge.weight + return float('inf') + + def shortest_path(self, start_id: str, end_id: str) -> Optional[List[str]]: + """ + 使用Dijkstra算法计算最短路径 + + Args: + start_id: 起始节点ID + end_id: 目标节点ID + + Returns: + 节点ID列表，表示路径 + """ + if start_id not in self.nodes or end_id not in self.nodes: + return None + + # 优先队列: (距离, 节点ID) + pq = [(0, start_id)] + # 距离字典 + distances = {node_id: float('inf') for node_id in self.nodes} + distances[start_id] = 0 + # 前驱节点 + previous = {start_id: None} + # 已访问 + visited = set() + + while pq: + current_dist, current_id = heapq.heappop(pq) + + if current_id in visited: + continue + visited.add(current_id) + + if current_id == end_id: + break + + # 检查所有邻居 + for neighbor_id in self.nodes[current_id].neighbors: + if neighbor_id in visited: + continue + + edge_weight = self.get_edge_weight(current_id, neighbor_id) + new_dist = current_dist + edge_weight + + if new_dist < distances[neighbor_id]: + distances[neighbor_id] = new_dist + previous[neighbor_id] = current_id + heapq.heappush(pq, (new_dist, neighbor_id)) + + # 重建路径 + if distances[end_id] == float('inf'): + return None + + path = [] + current = end_id + while current is not None: + path.append(current) + current = previous.get(current) + + path.reverse() + return path + + def shortest_path_distance(self, start_id: str, end_id: str) -> float: + """获取最短路径距离""" + path = self.shortest_path(start_id, end_id) + if not path: + return float('inf') + + total = 0.0 + for i in range(len(path) - 1): + total += self.get_edge_weight(path[i], path[i + 1]) + + return total + + +# ============================================================================ +# A*路径规划 +# ============================================================================ + +class AStarPlanner: + """ + A*路径规划器 + + 使用启发式搜索的高效路径规划算法。 + """ + + def __init__(self, graph: SpatialGraph): + self.graph = graph + + def heuristic(self, node_id: str, goal_id: str) -> float: + """ + 启发式函数 (使用欧氏距离) + + Args: + node_id: 当前节点 + goal_id: 目标节点 + + Returns: + 启发式估计值 + """ + if node_id not in self.graph.nodes or goal_id not in self.graph.nodes: + return 0.0 + + return self.graph.nodes[node_id].point.distance_to( + self.graph.nodes[goal_id].point + ) + + def plan(self, start_id: str, goal_id: str) -> Optional[Tuple[List[str], float]]: + """ + 规划路径 + + Args: + start_id: 起始节点ID + goal_id: 目标节点ID + + Returns: + (路径节点列表, 总距离) 或 None + """ + if start_id not in self.graph.nodes or goal_id not in self.graph.nodes: + return None + + # 开集和闭集 + open_set = {start_id} + closed_set = set() + + # g值: 从起点到当前节点的实际距离 + g_score = {node_id: float('inf') for node_id in self.graph.nodes} + g_score[start_id] = 0 + + # f值: g值 + 启发式值 + f_score = {node_id: float('inf') for node_id in self.graph.nodes} + f_score[start_id] = self.heuristic(start_id, goal_id) + + # 前驱节点 + came_from = {} + + while open_set: + # 获取f值最小的节点 + current = min(open_set, key=lambda x: f_score[x]) + + if current == goal_id: + # 重建路径 + path = [current] + total_distance = g_score[current] + + while current in came_from: + current = came_from[current] + path.append(current) + + path.reverse() + return (path, total_distance) + + open_set.remove(current) + closed_set.add(current) + + # 检查邻居 + for neighbor in self.graph.nodes[current].neighbors: + if neighbor in closed_set: + continue + + # 计算 tentative_g_score + edge_weight = self.graph.get_edge_weight(current, neighbor) + tentative_g = g_score[current] + edge_weight + + if neighbor not in open_set: + open_set.add(neighbor) + elif tentative_g >= g_score[neighbor]: + continue + + # 更新 + came_from[neighbor] = current + g_score[neighbor] = tentative_g + f_score[neighbor] = tentative_g + self.heuristic(neighbor, goal_id) + + return None # 没有找到路径 + + +# ============================================================================ +# 可见性分析 +# ============================================================================ + +class VisibilityAnalyzer: + """ + 可见性分析器 + + 判断点之间的可见性，考虑障碍物。 + """ + + def __init__(self, obstacles: List[Polygon] = None): + """ + 初始化可见性分析器 + + Args: + obstacles: 障碍物多边形列表 + """ + self.obstacles = obstacles or [] + + def add_obstacle(self, obstacle: Polygon) -> None: + """添加障碍物""" + self.obstacles.append(obstacle) + + def is_visible(self, p1: Point, p2: Point, + tolerance: float = 1e-6) -> bool: + """ + 判断两点之间是否可见 + + Args: + p1: 第一个点 + p2: 第二个点 + tolerance: 容差 + + Returns: + 是否可见 + """ + # 检视线是否与任何障碍物相交 + for obstacle in self.obstacles: + if self._line_intersects_polygon(p1, p2, obstacle): + return False + return True + + def _line_intersects_polygon(self, p1: Point, p2: Point, + polygon: Polygon) -> bool: + """判断线段是否与多边形相交""" + # 首先检查包围盒 + line_min_x = min(p1.x, p2.x) + line_max_x = max(p1.x, p2.x) + line_min_y = min(p1.y, p2.y) + line_max_y = max(p1.y, p2.y) + + poly_min_x = min(p.x for p in polygon.exterior) + poly_max_x = max(p.x for p in polygon.exterior) + poly_min_y = min(p.y for p in polygon.exterior) + poly_max_y = max(p.y for p in polygon.exterior) + + # 包围盒不相交 + if line_max_x < poly_min_x or line_min_x > poly_max_x or \ + line_max_y < poly_min_y or line_min_y > poly_max_y: + return False + + # 检查线段是否与多边形的任何边相交 + n = len(polygon.exterior) + for i in range(n): + v1 = polygon.exterior[i] + v2 = polygon.exterior[(i + 1) % n] + + if self._segments_intersect(p1, p2, v1, v2): + return True + + return False + + def _segments_intersect(self, p1: Point, p2: Point, + p3: Point, p4: Point) -> bool: + """判断两条线段是否相交""" + def orientation(a, b, c): + val = (b.y - a.y) * (c.x - b.x) - (b.x - a.x) * (c.y - b.y) + if abs(val) < 1e-10: + return 0 # 共线 + return 1 if val > 0 else 2 # 顺时针/逆时针 + + def on_segment(a, b, c): + return min(a.x, c.x) <= b.x <= max(a.x, c.x) and \ + min(a.y, c.y) <= b.y <= max(a.y, c.y) + + o1 = orientation(p1, p2, p3) + o2 = orientation(p1, p2, p4) + o3 = orientation(p3, p4, p1) + o4 = orientation(p3, p4, p2) + + # 一般情况 + if o1 != o2 and o3 != o4: + return True + + # 特殊情况 + if o1 == 0 and on_segment(p1, p3, p2): + return True + if o2 == 0 and on_segment(p1, p4, p2): + return True + if o3 == 0 and on_segment(p3, p1, p4): + return True + if o4 == 0 and on_segment(p3, p2, p4): + return True + + return False + + def viewshed(self, observer: Point, radius: float, + num_rays: int = 360) -> List[Tuple[Point, bool]]: + """ + 计算视域 (可视范围) + + Args: + observer: 观察点 + radius: 视距 + num_rays: 射线数量 + + Returns: + (点, 可见性) 列表 + """ + results = [] + + for i in range(num_rays): + angle = 2 * math.pi * i / num_rays + target = Point( + observer.x + radius * math.cos(angle), + observer.y + radius * math.sin(angle) + ) + + visible = self.is_visible(observer, target) + results.append((target, visible)) + + return results + + +# ============================================================================ +# 空间推理引擎 +# ============================================================================ + +class SpatialReasoningEngine: + """ + 空间推理引擎 + + 集成多种空间推理功能的综合引擎。 + """ + + def __init__(self): + self.qualitative_reasoner = QualitativeReasoner() + self.graph = SpatialGraph() + self.visibility_analyzer = VisibilityAnalyzer() + self.astar_planner = None + + def build_road_network(self, points: List[Tuple[str, Point]], + connections: List[Tuple[str, str]]) -> None: + """ + 构建道路网络 + + Args: + points: (节点ID, 点) 列表 + connections: (节点1, 节点2) 连接列表 + """ + for id, point in points: + self.graph.add_node(id, point) + + for id1, id2 in connections: + self.graph.add_edge(id1, id2) + + self.astar_planner = AStarPlanner(self.graph) + + def navigate(self, start: str, goal: str) -> Optional[Dict[str, Any]]: + """ + 导航规划 + + Args: + start: 起始点ID + goal: 目标点ID + + Returns: + 导航结果字典 + """ + if not self.astar_planner: + return None + + result = self.astar_planner.plan(start, goal) + + if result: + path, distance = result + + # 计算方向指示 + directions = [] + for i in range(len(path) - 1): + from_node = self.graph.nodes[path[i]] + to_node = self.graph.nodes[path[i + 1]] + + relation = self.qualitative_reasoner.infer_direction( + from_node.point, to_node.point + ) + directions.append({ + "from": path[i], + "to": path[i + 1], + "direction": relation.value, + "distance": from_node.point.distance_to(to_node.point) + }) + + return { + "path": path, + "total_distance": distance, + "num_steps": len(path) - 1, + "directions": directions + } + + return None + + def query_relation(self, entity1: str, entity2: str, + point1: Point, point2: Point) -> Dict[str, Any]: + """ + 查询两个实体之间的空间关系 + + Args: + entity1: 实体1名称 + entity2: 实体2名称 + point1: 实体1位置 + point2: 实体2位置 + + Returns: + 关系字典 + """ + direction = self.qualitative_reasoner.infer_direction(point1, point2) + distance_cat = self.qualitative_reasoner.infer_distance_category(point1, point2) + actual_distance = point1.distance_to(point2) + + return { + "entity1": entity1, + "entity2": entity2, + "direction": direction.value, + "distance_category": distance_cat.value, + "actual_distance": actual_distance, + "bearing": math.degrees(math.atan2( + point2.y - point1.y, + point2.x - point1.x + )) + } + + +# ============================================================================ +# 主程序 +# ======================================================================== + +def main(): + """主程序 - 演示空间推理的使用""" + + print("="*70) + print("空间推理示例演示") + print("="*70) + + # ======================================================================== + # 1. 定性空间推理 + # ======================================================================== + print("\n[部分 1] 定性空间推理") + print("-" * 50) + + reasoner = QualitativeReasoner() + + # 创建几个地标点 + landmarks = { + "西湖": Point(120.148, 30.259), + "钱塘江": Point(120.250, 30.200), + "滨江": Point(120.350, 30.220), + "萧山": Point(120.400, 30.150) + } + + print("\n地标位置:") + for name, point in landmarks.items(): + print(f" {name}: {point}") + + # 推断方向关系 + print("\n方向关系:") + for i, (name1, point1) in enumerate(list(landmarks.items())[:-1]): + name2 = list(landmarks.keys())[i + 1] + point2 = landmarks[name2] + + relation = reasoner.infer_direction(point1, point2) + dist_relation = reasoner.infer_distance_category(point1, point2) + + print(f" {name1} -> {name2}:") + print(f" 方向: {relation.value}") + print(f" 距离类别: {dist_relation.value}") + + # ======================================================================== + # 2. 路径规划 + # ======================================================================== + print("\n\n[部分 2] 路径规划") + print("-" * 50) + + # 创建道路网络 + network_points = [ + ("A", Point(0, 0)), + ("B", Point(50, 30)), + ("C", Point(100, 50)), + ("D", Point(30, 80)), + ("E", Point(80, 100)), + ("F", Point(120, 120)), + ("G", Point(150, 60)) + ] + + connections = [ + ("A", "B"), ("B", "C"), ("A", "D"), + ("B", "D"), ("D", "E"), ("C", "E"), + ("E", "F"), ("C", "G"), ("G", "F") + ] + + graph = SpatialGraph() + for id, point in network_points: + graph.add_node(id, point) + for id1, id2 in connections: + graph.add_edge(id1, id2) + + print("\n道路网络:") + print(f" 节点数: {len(graph.nodes)}") + print(f" 边数: {len(graph.edges)}") + + # Dijkstra最短路径 + print("\nDijkstra最短路径 (A -> F):") + dijkstra_path = graph.shortest_path("A", "F") + if dijkstra_path: + print(f" 路径: {' -> '.join(dijkstra_path)}") + distance = graph.shortest_path_distance("A", "F") + print(f" 总距离: {distance:.2f}") + + # A*路径规划 + print("\nA*路径规划 (A -> F):") + astar = AStarPlanner(graph) + astar_result = astar.plan("A", "F") + if astar_result: + path, dist = astar_result + print(f" 路径: {' -> '.join(path)}") + print(f" 总距离: {dist:.2f}") + + # ======================================================================== + # 3. 可见性分析 + # ======================================================================== + print("\n\n[部分 3] 可见性分析") + print("-" * 50) + + # 创建障碍物 + obstacle1 = Polygon(exterior=[ + Point(60, 40), + Point(80, 40), + Point(80, 70), + Point(60, 70), + Point(60, 40) + ]) + + obstacle2 = Polygon(exterior=[ + Point(100, 80), + Point(130, 80), + Point(130, 110), + Point(100, 110), + Point(100, 80) + ]) + + visibility = VisibilityAnalyzer([obstacle1, obstacle2]) + + print("\n障碍物:") + print(f" 障碍物1: {obstacle1}") + print(f" 障碍物2: {obstacle2}") + + # 测试可见性 + observer = Point(30, 50) + test_points = [ + ("目标A", Point(90, 50)), # 被障碍物1遮挡 + ("目标B", Point(120, 120)), # 被障碍物2遮挡 + ("目标C", Point(150, 30)), # 可见 + ("目标D", Point(50, 90)) # 可见 + ] + + print(f"\n从观察点 {observer} 观察:") + for name, target in test_points: + visible = visibility.is_visible(observer, target) + status = "可见" if visible else "不可见" + print(f" {name} {target}: {status}") + + # ======================================================================== + # 4. 综合推理引擎 + # ======================================================================== + print("\n\n[部分 4] 综合空间推理引擎") + print("-" * 50) + + engine = SpatialReasoningEngine() + + # 构建城市路网 + city_points = [ + ("火车站", Point(100, 100)), + ("市政府", Point(150, 120)), + ("西湖", Point(200, 100)), + ("钱江新城", Point(180, 180)), + ("滨江", Point(120, 200)), + ("萧山机场", Point(250, 250)) + ] + + city_connections = [ + ("火车站", "市政府"), + ("火车站", "滨江"), + ("市政府", "西湖"), + ("市政府", "钱江新城"), + ("滨江", "钱江新城"), + ("钱江新城", "萧山机场"), + ("西湖", "萧山机场") + ] + + engine.build_road_network(city_points, city_connections) + + print("\n城市路网:") + for name, point in city_points: + print(f" {name}: {point}") + + # 导航示例 + print("\n导航示例: 从 火车站 到 萧山机场") + nav_result = engine.navigate("火车站", "萧山机场") + + if nav_result: + print(f"\n路径规划结果:") + print(f" 路径: {' -> '.join(nav_result['path'])}") + print(f" 总距离: {nav_result['total_distance']:.2f}") + print(f" 步数: {nav_result['num_steps']}") + + print(f"\n详细指引:") + for i, step in enumerate(nav_result['directions'], 1): + dir_map = { + "N": "向北", "S": "向南", "E": "向东", "W": "向西", + "NE": "向东北", "NW": "向西北", "SE": "向东南", "SW": "向西南" + } + direction_cn = dir_map.get(step['direction'], step['direction']) + print(f" {i}. 从 {step['from']} {direction_cn} 前往 {step['to']} " + f"(距离: {step['distance']:.1f})") + + # 空间关系查询 + print("\n空间关系查询:") + relation = engine.query_relation( + "火车站", "西湖", + city_points[0][1], city_points[2][1] + ) + + print(f" {relation['entity1']} 相对于 {relation['entity2']}:") + print(f" 方向: {relation['direction']}") + print(f" 距离: {relation['actual_distance']:.2f}") + print(f" 方位角: {relation['bearing']:.1f}°") + + # ======================================================================== + # 5. 多路径比较 + # ======================================================================== + print("\n\n[部分 5] 多路径比较") + print("-" * 50) + + destinations = ["市政府", "西湖", "钱江新城", "滨江", "萧山机场"] + start = "火车站" + + print(f"\n从 {start} 到各目的地的距离:") + results = [] + for dest in destinations: + if dest == start: + continue + path_info = engine.navigate(start, dest) + if path_info: + results.append((dest, path_info['total_distance'], path_info['path'])) + + results.sort(key=lambda x: x[1]) + + for i, (dest, dist, path) in enumerate(results, 1): + print(f" {i}. {dest:8s}: {dist:6.1f} (路径: {' -> '.join(path)})") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/02-spatial-intelligence/spatial_representation.py b/dofile/examples/02-spatial-intelligence/spatial_representation.py new file mode 100644 index 0000000..f7f6bb0 --- /dev/null +++ b/dofile/examples/02-spatial-intelligence/spatial_representation.py @@ -0,0 +1,982 @@ +""" +空间表征示例 (Spatial Representation Example) +============================================ + +本示例展示空间智能系统中的空间表征方法。 +空间表征是对地理空间现象的抽象和建模，是空间推理的基础。 + +核心概念: +1. 空间对象模型 - 点、线、面等几何对象 +2. 空间关系模型 - 拓扑、距离、方向关系 +3. 空间场模型 - 连续表面的表示 +4. 空间索引结构 - 加速空间查询 +5. 多尺度表征 - 不同详细程度的表示 + +应用场景: +- 地理信息系统 (GIS) +- 空间数据库 +- 空间推理引擎 +- 地理可视化 + +作者: CC4SI 项目组 +""" + +import math +import json +from typing import List, Dict, Tuple, Optional, Any, Set +from dataclasses import dataclass, field +from enum import Enum +from abc import ABC, abstractmethod +import random + + +# ============================================================================ +# 几何类型与空间对象 +# ============================================================================ + +class GeometryType(Enum): + """几何类型枚举""" + POINT = "Point" + LINESTRING = "LineString" + POLYGON = "Polygon" + MULTIPOINT = "MultiPoint" + MULTILINESTRING = "MultiLineString" + MULTIPOLYGON = "MultiPolygon" + GEOMETRYCOLLECTION = "GeometryCollection" + + +class SpatialReference(Enum): + """空间参考系统""" + WGS84 = "EPSG:4326" # 经纬度 + WEB_MERCATOR = "EPSG:3857" # Web墨卡托 + CGCS2000 = "EPSG:4490" # 中国大地坐标系统 + + +@dataclass +class Point: + """点几何对象""" + x: float + y: float + srid: int = 4326 # 空间参考ID + + def __iter__(self): + """支持解包""" + return iter((self.x, self.y)) + + def __iter__(self): + return iter((self.x, self.y)) + + def to_tuple(self) -> Tuple[float, float]: + """转换为元组""" + return (self.x, self.y) + + def to_geojson(self) -> Dict: + """转换为GeoJSON""" + return { + "type": "Point", + "coordinates": [self.x, self.y] + } + + def distance_to(self, other: 'Point') -> float: + """计算到另一个点的欧氏距离""" + return math.sqrt((self.x - other.x)**2 + (self.y - other.y)**2) + + def __repr__(self) -> str: + return f"Point({self.x:.4f}, {self.y:.4f})" + + +@dataclass +class LineString: + """线几何对象""" + coordinates: List[Point] + srid: int = 4326 + + @property + def length(self) -> float: + """计算线的长度""" + if len(self.coordinates) < 2: + return 0.0 + total = 0.0 + for i in range(len(self.coordinates) - 1): + total += self.coordinates[i].distance_to(self.coordinates[i + 1]) + return total + + def to_geojson(self) -> Dict: + """转换为GeoJSON""" + return { + "type": "LineString", + "coordinates": [[p.x, p.y] for p in self.coordinates] + } + + def get_point_at(self, ratio: float) -> Point: + """ + 获取线上指定比例位置的点 + + Args: + ratio: 0到1之间的比例值 + + Returns: + 该位置的点 + """ + if ratio <= 0: + return self.coordinates[0] + if ratio >= 1: + return self.coordinates[-1] + + target_length = self.length * ratio + accumulated = 0.0 + + for i in range(len(self.coordinates) - 1): + segment_length = self.coordinates[i].distance_to(self.coordinates[i + 1]) + if accumulated + segment_length >= target_length: + # 在此段上 + segment_ratio = (target_length - accumulated) / segment_length + p1 = self.coordinates[i] + p2 = self.coordinates[i + 1] + return Point( + p1.x + (p2.x - p1.x) * segment_ratio, + p1.y + (p2.y - p1.y) * segment_ratio + ) + accumulated += segment_length + + return self.coordinates[-1] + + def __repr__(self) -> str: + return f"LineString({len(self.coordinates)} points)" + + +@dataclass +class Polygon: + """面几何对象""" + exterior: List[Point] # 外环 + interiors: List[List[Point]] = field(default_factory=list) # 内环（空洞） + srid: int = 4326 + + @property + def area(self) -> float: + """使用鞋带公式计算多边形面积""" + return self._ring_area(self.exterior) - sum( + self._ring_area(interior) for interior in self.interiors + ) + + def _ring_area(self, ring: List[Point]) -> float: + """计算环的面积（绝对值）""" + if len(ring) < 3: + return 0.0 + area = 0.0 + for i in range(len(ring)): + j = (i + 1) % len(ring) + area += ring[i].x * ring[j].y + area -= ring[j].x * ring[i].y + return abs(area) / 2 + + @property + def centroid(self) -> Point: + """计算多边形质心""" + if len(self.exterior) < 3: + return self.exterior[0] if self.exterior else Point(0, 0) + + # 简化：使用外环的顶点平均值 + avg_x = sum(p.x for p in self.exterior) / len(self.exterior) + avg_y = sum(p.y for p in self.exterior) / len(self.exterior) + return Point(avg_x, avg_y) + + def contains_point(self, point: Point) -> bool: + """ + 判断点是否在多边形内（射线法） + + Args: + point: 待判断的点 + + Returns: + 是否在多边形内 + """ + return self._point_in_ring(point, self.exterior) and \ + all(not self._point_in_ring(point, interior) + for interior in self.interiors) + + def _point_in_ring(self, point: Point, ring: List[Point]) -> bool: + """射线法判断点是否在环内""" + if len(ring) < 3: + return False + + x, y = point.x, point.y + inside = False + + for i in range(len(ring)): + j = (i + 1) % len(ring) + xi, yi = ring[i].x, ring[i].y + xj, yj = ring[j].x, ring[j].y + + # 检查射线与边的交点 + if ((yi > y) != (yj > y)) and \ + (x < (xj - xi) * (y - yi) / (yj - yi + 1e-10) + xi): + inside = not inside + + return inside + + def to_geojson(self) -> Dict: + """转换为GeoJSON""" + coords = [[[p.x, p.y] for p in self.exterior]] + coords.extend([[[p.x, p.y] for p in interior] for interior in self.interiors]) + + return { + "type": "Polygon", + "coordinates": coords + } + + def __repr__(self) -> str: + holes = len(self.interiors) + return f"Polygon({len(self.exterior)} vertices{f', {holes} holes' if holes > 0 else ''})" + + +@dataclass +class Envelope: + """包围盒""" + min_x: float + max_x: float + min_y: float + max_y: float + + @property + def width(self) -> float: + return self.max_x - self.min_x + + @property + def height(self) -> float: + return self.max_y - self.min_y + + @property + def area(self) -> float: + return self.width * self.height + + @property + def center(self) -> Point: + return Point( + (self.min_x + self.max_x) / 2, + (self.min_y + self.max_y) / 2 + ) + + def contains(self, point: Point) -> bool: + """判断点是否在包围盒内""" + return (self.min_x <= point.x <= self.max_x and + self.min_y <= point.y <= self.max_y) + + def intersects(self, other: 'Envelope') -> bool: + """判断是否与另一个包围盒相交""" + return not (self.max_x < other.min_x or self.min_x > other.max_x or + self.max_y < other.min_y or self.min_y > other.max_y) + + def union(self, other: 'Envelope') -> 'Envelope': + """计算与另一个包围盒的并集""" + return Envelope( + min_x=min(self.min_x, other.min_x), + max_x=max(self.max_x, other.max_x), + min_y=min(self.min_y, other.min_y), + max_y=max(self.max_y, other.max_y) + ) + + def __repr__(self) -> str: + return f"Envelope([{self.min_x:.2f}, {self.min_y:.2f}] -> [{self.max_x:.2f}, {self.max_y:.2f}])" + + +# ============================================================================ +# 空间特征对象 +# ============================================================================ + +@dataclass +class SpatialFeature: + """ + 空间特征 + + 包含几何和属性信息的完整空间对象。 + """ + id: str + geometry: Any # Point, LineString, Polygon等 + properties: Dict[str, Any] = field(default_factory=dict) + + def to_geojson(self) -> Dict: + """转换为GeoJSON Feature""" + geom_data = None + if isinstance(self.geometry, Point): + geom_data = self.geometry.to_geojson() + elif isinstance(self.geometry, LineString): + geom_data = self.geometry.to_geojson() + elif isinstance(self.geometry, Polygon): + geom_data = self.geometry.to_geojson() + + return { + "type": "Feature", + "id": self.id, + "geometry": geom_data, + "properties": self.properties + } + + def to_geojson_collection(self) -> Dict: + """转换为GeoJSON FeatureCollection""" + return { + "type": "FeatureCollection", + "features": [self.to_geojson()] + } + + def envelope(self) -> Envelope: + """计算包围盒""" + if isinstance(self.geometry, Point): + return Envelope( + self.geometry.x, self.geometry.x, + self.geometry.y, self.geometry.y + ) + elif isinstance(self.geometry, (LineString, list)): + coords = self.geometry.coordinates if isinstance(self.geometry, LineString) else self.geometry + xs = [p.x for p in coords] + ys = [p.y for p in coords] + return Envelope(min(xs), max(xs), min(ys), max(ys)) + elif isinstance(self.geometry, Polygon): + xs = [p.x for p in self.geometry.exterior] + ys = [p.y for p in self.geometry.exterior] + return Envelope(min(xs), max(xs), min(ys), max(ys)) + else: + raise ValueError(f"Unsupported geometry type: {type(self.geometry)}") + + def __repr__(self) -> str: + return f"SpatialFeature(id={self.id}, geom={type(self.geometry).__name__})" + + +# ============================================================================ +# 空间关系 +# ============================================================================ + +class SpatialRelation(Enum): + """空间关系类型""" + EQUALS = "equals" + DISJOINT = "disjoint" + INTERSECTS = "intersects" + TOUCHES = "touches" + CROSSES = "crosses" + WITHIN = "within" + CONTAINS = "contains" + OVERLAPS = "overlaps" + + +@dataclass +class TopologyRelation: + """ + 拓扑关系描述 + + 基于DE-9IM (Dimensionally Extended nine-Intersection Model)模型。 + """ + relation: SpatialRelation + confidence: float = 1.0 + + def __repr__(self) -> str: + return f"TopologyRelation({self.relation.value}, conf={self.confidence:.2f})" + + +def calculate_relation(geom1: Any, geom2: Any) -> TopologyRelation: + """ + 计算两个几何对象的空间关系 + + Args: + geom1: 第一个几何对象 + geom2: 第二个几何对象 + + Returns: + 拓扑关系 + """ + # 点-点关系 + if isinstance(geom1, Point) and isinstance(geom2, Point): + if geom1.x == geom2.x and geom1.y == geom2.y: + return TopologyRelation(SpatialRelation.EQUALS) + return TopologyRelation(SpatialRelation.DISJOINT) + + # 点-多边形关系 + if isinstance(geom1, Point) and isinstance(geom2, Polygon): + if geom2.contains_point(geom1): + return TopologyRelation(SpatialRelation.WITHIN) + # 检查是否在边界上 + for i in range(len(geom2.exterior)): + p1, p2 = geom2.exterior[i], geom2.exterior[(i + 1) % len(geom2.exterior)] + if point_on_segment(geom1, p1, p2): + return TopologyRelation(SpatialRelation.TOUCHES) + return TopologyRelation(SpatialRelation.DISJOINT) + + # 多边形-点关系 + if isinstance(geom1, Polygon) and isinstance(geom2, Point): + relation = calculate_relation(geom2, geom1) + if relation.relation == SpatialRelation.WITHIN: + return TopologyRelation(SpatialRelation.CONTAINS) + return relation + + # 多边形-多边形关系（简化版：只检查相交） + if isinstance(geom1, Polygon) and isinstance(geom2, Polygon): + env1 = envelope_of_polygon(geom1) + env2 = envelope_of_polygon(geom2) + + if not env1.intersects(env2): + return TopologyRelation(SpatialRelation.DISJOINT) + + # 简化：检查是否有交点 + if geom1.contains_point(geom2.exterior[0]): + return TopologyRelation(SpatialRelation.CONTAINS) + if geom2.contains_point(geom1.exterior[0]): + return TopologyRelation(SpatialRelation.WITHIN) + + return TopologyRelation(SpatialRelation.INTERSECTS) + + return TopologyRelation(SpatialRelation.DISJOINT) + + +def point_on_segment(point: Point, seg_start: Point, seg_end: Point, + tolerance: float = 1e-6) -> bool: + """判断点是否在线段上""" + # 检查点是否在线段的包围盒内 + if not (min(seg_start.x, seg_end.x) - tolerance <= point.x <= + max(seg_start.x, seg_end.x) + tolerance and + min(seg_start.y, seg_end.y) - tolerance <= point.y <= + max(seg_start.y, seg_end.y) + tolerance): + return False + + # 检查三点共线 + cross = (seg_end.x - seg_start.x) * (point.y - seg_start.y) - \ + (seg_end.y - seg_start.y) * (point.x - seg_start.x) + return abs(cross) < tolerance + + +def envelope_of_polygon(polygon: Polygon) -> Envelope: + """计算多边形的包围盒""" + xs = [p.x for p in polygon.exterior] + ys = [p.y for p in polygon.exterior] + return Envelope(min(xs), max(xs), min(ys), max(ys)) + + +# ============================================================================ +# 空间索引 +# ============================================================================ + +class RTreeNode: + """R树节点""" + def __init__(self, is_leaf: bool = False): + self.is_leaf = is_leaf + self.envelope: Optional[Envelope] = None + self.children: List['RTreeNode'] = [] + self.features: List[SpatialFeature] = [] + + def update_envelope(self): + """更新节点的包围盒""" + if self.is_leaf and self.features: + envelopes = [f.envelope() for f in self.features] + self.envelope = envelopes[0] + for env in envelopes[1:]: + self.envelope = self.envelope.union(env) + elif not self.is_leaf and self.children: + self.envelope = self.children[0].envelope + for child in self.children[1:]: + if child.envelope: + self.envelope = self.envelope.union(child.envelope) + + +class RTree: + """ + R树空间索引 + + 用于加速空间查询的树状索引结构。 + """ + + def __init__(self, max_children: int = 4): + """ + 初始化R树 + + Args: + max_children: 每个节点的最大子节点数 + """ + self.max_children = max_children + self.root = RTreeNode(is_leaf=True) + self.size = 0 + + def insert(self, feature: SpatialFeature) -> None: + """插入空间特征""" + self._insert(self.root, feature) + self.size += 1 + + def _insert(self, node: RTreeNode, feature: SpatialFeature) -> None: + """递归插入""" + feature_env = feature.envelope() + + if node.is_leaf: + node.features.append(feature) + node.update_envelope() + + # 如果超过容量，分裂节点 + if len(node.features) > self.max_children: + self._split(node) + else: + # 选择最佳子节点 + best_child = self._choose_best_child(node, feature_env) + self._insert(best_child, feature) + node.update_envelope() + + def _choose_best_child(self, node: RTreeNode, env: Envelope) -> RTreeNode: + """选择插入代价最小的子节点""" + best = None + best_increase = float('inf') + + for child in node.children: + if child.envelope is None: + continue + union_env = child.envelope.union(env) + increase = union_env.area - child.envelope.area + + if increase < best_increase: + best_increase = increase + best = child + + return best or node.children[0] + + def _split(self, node: RTreeNode) -> None: + """分裂节点 (简化版)""" + if node.is_leaf: + # 简单分裂：将特征分成两组 + mid = len(node.features) // 2 + group1 = node.features[:mid] + group2 = node.features[mid:] + + node.features = group1 + + new_leaf = RTreeNode(is_leaf=True) + new_leaf.features = group2 + new_leaf.update_envelope() + + # 更新父节点 + if node == self.root and not node.children: + # 根节点分裂 + new_root = RTreeNode(is_leaf=False) + new_root.children = [node, new_leaf] + new_root.update_envelope() + self.root = new_root + else: + # 简化：不处理非根节点的分裂 + pass + + def query(self, envelope: Envelope) -> List[SpatialFeature]: + """查询与包围盒相交的所有特征""" + results = [] + self._query(self.root, envelope, results) + return results + + def _query(self, node: RTreeNode, envelope: Envelope, + results: List[SpatialFeature]) -> None: + """递归查询""" + if node.envelope and not node.envelope.intersects(envelope): + return + + if node.is_leaf: + for feature in node.features: + if feature.envelope().intersects(envelope): + results.append(feature) + else: + for child in node.children: + self._query(child, envelope, results) + + def nearest_neighbor(self, point: Point, k: int = 1) -> List[Tuple[SpatialFeature, float]]: + """ + K近邻查询 + + Args: + point: 查询点 + k: 返回的最近邻数量 + + Returns: + (特征, 距离) 列表 + """ + candidates = [] + self._collect_candidates(self.root, candidates) + + # 计算距离并排序 + distances = [] + for feature in candidates: + if isinstance(feature.geometry, Point): + dist = feature.geometry.distance_to(point) + distances.append((feature, dist)) + else: + # 非点几何：使用包围盒中心距离 + env = feature.envelope() + center = env.center + dist = math.sqrt((center.x - point.x)**2 + (center.y - point.y)**2) + distances.append((feature, dist)) + + distances.sort(key=lambda x: x[1]) + return distances[:k] + + def _collect_candidates(self, node: RTreeNode, results: List[SpatialFeature]) -> None: + """收集所有候选特征""" + if node.is_leaf: + results.extend(node.features) + else: + for child in node.children: + self._collect_candidates(child, results) + + +# ============================================================================ +# 空间场模型 +# ============================================================================ + +class GridField: + """ + 栅格场模型 + + 使用规则网格表示连续空间现象。 + """ + + def __init__(self, bounds: Envelope, rows: int, cols: int, + nodata: float = -9999): + """ + 初始化栅格场 + + Args: + bounds: 空间范围 + rows: 行数 + cols: 列数 + nodata: 无数据值 + """ + self.bounds = bounds + self.rows = rows + self.cols = cols + self.nodata = nodata + self.data = [[nodata for _ in range(cols)] for _ in range(rows)] + + @property + def cell_width(self) -> float: + """获取单元格宽度""" + return self.bounds.width / self.cols + + @property + def cell_height(self) -> float: + """获取单元格高度""" + return self.bounds.height / self.rows + + def get_cell_index(self, point: Point) -> Optional[Tuple[int, int]]: + """ + 获取点对应的栅格索引 + + Args: + point: 空间点 + + Returns: + (行索引, 列索引) 或 None + """ + if not self.bounds.contains(point): + return None + + col = int((point.x - self.bounds.min_x) / self.cell_width) + row = int((self.bounds.max_y - point.y) / self.cell_height) + + col = max(0, min(col, self.cols - 1)) + row = max(0, min(row, self.rows - 1)) + + return (row, col) + + def set_value(self, row: int, col: int, value: float) -> None: + """设置栅格值""" + if 0 <= row < self.rows and 0 <= col < self.cols: + self.data[row][col] = value + + def get_value(self, row: int, col: int) -> float: + """获取栅格值""" + if 0 <= row < self.rows and 0 <= col < self.cols: + return self.data[row][col] + return self.nodata + + def get_value_at_point(self, point: Point) -> float: + """获取点位置的值（最近邻）""" + idx = self.get_cell_index(point) + if idx: + return self.get_value(idx[0], idx[1]) + return self.nodata + + def interpolate_at_point(self, point: Point) -> float: + """双线性插值获取点位置的值""" + idx = self.get_cell_index(point) + if not idx: + return self.nodata + + row, col = idx + + # 获取四个角点的值 + values = [] + for r in range(row, min(row + 2, self.rows)): + for c in range(col, min(col + 2, self.cols)): + values.append((r, c, self.data[r][c])) + + if len(values) < 4 or any(v[2] == self.nodata for v in values): + return self.get_value(row, col) # 回退到最近邻 + + # 双线性插值 + # 简化实现 + return self.get_value(row, col) + + def get_statistics(self) -> Dict[str, float]: + """获取统计信息""" + values = [v for row in self.data for v in row if v != self.nodata] + + if not values: + return {"count": 0, "min": self.nodata, "max": self.nodata, + "mean": self.nodata, "std": 0} + + import statistics + return { + "count": len(values), + "min": min(values), + "max": max(values), + "mean": statistics.mean(values), + "std": statistics.stdev(values) if len(values) > 1 else 0 + } + + def to_geojson(self) -> Dict: + """转换为GeoJSON（简化：输出为点集）""" + features = [] + for r in range(self.rows): + for c in range(self.cols): + val = self.data[r][c] + if val != self.nodata: + # 计算点坐标 + x = self.bounds.min_x + (c + 0.5) * self.cell_width + y = self.bounds.max_y - (r + 0.5) * self.cell_height + + features.append({ + "type": "Feature", + "geometry": {"type": "Point", "coordinates": [x, y]}, + "properties": {"value": val} + }) + + return { + "type": "FeatureCollection", + "features": features + } + + +# ============================================================================ +# 主程序 +# ============================================================================ + +def main(): + """主程序 - 演示空间表征的使用""" + + print("="*70) + print("空间表征示例演示") + print("="*70) + + # ======================================================================== + # 1. 基础几何对象 + # ======================================================================== + print("\n[部分 1] 基础几何对象") + print("-" * 50) + + # 创建点 + point1 = Point(120.5, 30.2) + point2 = Point(121.0, 30.5) + print(f"\n点对象:") + print(f" point1 = {point1}") + print(f" point2 = {point2}") + print(f" 距离 = {point1.distance_to(point2):.4f}") + + # 创建线 + linestring = LineString([ + Point(120.0, 30.0), + Point(120.5, 30.2), + Point(121.0, 30.5), + Point(121.5, 30.3) + ]) + print(f"\n线对象:") + print(f" {linestring}") + print(f" 长度 = {linestring.length:.4f}") + print(f" 中点 = {linestring.get_point_at(0.5)}") + + # 创建多边形 + polygon = Polygon(exterior=[ + Point(120.0, 30.0), + Point(121.0, 30.0), + Point(121.0, 31.0), + Point(120.0, 31.0), + Point(120.0, 30.0) + ]) + print(f"\n多边形对象:") + print(f" {polygon}") + print(f" 面积 = {polygon.area:.4f}") + print(f" 质心 = {polygon.centroid}") + + # 点包含测试 + test_inside = Point(120.5, 30.5) + test_outside = Point(121.5, 30.5) + print(f" {test_inside} 在多边形内: {polygon.contains_point(test_inside)}") + print(f" {test_outside} 在多边形内: {polygon.contains_point(test_outside)}") + + # ======================================================================== + # 2. 空间特征 + # ======================================================================== + print("\n\n[部分 2] 空间特征") + print("-" * 50) + + feature1 = SpatialFeature( + id="poi_001", + geometry=Point(120.5, 30.2), + properties={ + "name": "杭州西湖", + "type": "scenic_spot", + "rating": 4.8 + } + ) + + feature2 = SpatialFeature( + id="zone_001", + geometry=polygon, + properties={ + "name": "开发区A", + "type": "industrial_zone", + "area_ha": 10000 + } + ) + + print(f"\n特征1: {feature1}") + print(f" 属性: {feature1.properties}") + print(f" 包围盒: {feature1.envelope()}") + + print(f"\n特征2: {feature2}") + print(f" 属性: {feature2.properties}") + print(f" 包围盒: {feature2.envelope()}") + + # GeoJSON输出 + print(f"\nGeoJSON输出:") + print(json.dumps(feature1.to_geojson(), ensure_ascii=False, indent=2)) + + # ======================================================================== + # 3. 空间关系 + # ======================================================================== + print("\n\n[部分 3] 空间关系") + print("-" * 50) + + # 点与多边形的关系 + point_a = Point(120.5, 30.5) # 在多边形内 + point_b = Point(121.5, 30.5) # 在多边形外 + point_c = Point(120.0, 30.5) # 在边界上 + + print(f"\n点-多边形关系测试:") + print(f" {point_a} 与多边形: {calculate_relation(point_a, polygon)}") + print(f" {point_b} 与多边形: {calculate_relation(point_b, polygon)}") + print(f" {point_c} 与多边形: {calculate_relation(point_c, polygon)}") + + # 多边形-多边形关系 + poly2 = Polygon(exterior=[ + Point(120.5, 29.5), + Point(121.5, 29.5), + Point(121.5, 30.5), + Point(120.5, 30.5), + Point(120.5, 29.5) + ]) + print(f"\n多边形-多边形关系:") + print(f" poly1 与 poly2: {calculate_relation(polygon, poly2)}") + + # ======================================================================== + # 4. 空间索引 + # ======================================================================== + print("\n\n[部分 4] R树空间索引") + print("-" * 50) + + # 创建R树 + rtree = RTree(max_children=4) + + # 插入一些特征 + features = [] + for i in range(20): + x = random.uniform(119, 122) + y = random.uniform(29, 32) + feature = SpatialFeature( + id=f"feature_{i:03d}", + geometry=Point(x, y), + properties={"value": random.uniform(0, 100)} + ) + features.append(feature) + rtree.insert(feature) + + print(f"\n已插入 {rtree.size} 个特征到R树") + + # 范围查询 + query_env = Envelope(120, 121, 30, 31) + results = rtree.query(query_env) + print(f"\n范围查询 {query_env}:") + print(f" 找到 {len(results)} 个特征") + for f in results[:5]: + print(f" - {f.id}: {f.geometry}") + + # K近邻查询 + query_point = Point(120.5, 30.5) + neighbors = rtree.nearest_neighbor(query_point, k=5) + print(f"\nK近邻查询 (中心点: {query_point}):") + for i, (f, dist) in enumerate(neighbors, 1): + print(f" {i}. {f.id}: 距离 = {dist:.4f}") + + # ======================================================================== + # 5. 栅格场模型 + # ======================================================================== + print("\n\n[部分 5] 栅格场模型") + print("-" * 50) + + # 创建栅格场 + grid = GridField( + bounds=Envelope(119, 122, 29, 32), + rows=30, + cols=30, + nodata=-9999 + ) + + # 填充一些模拟数据 (温度场) + for r in range(grid.rows): + for c in range(grid.cols): + # 计算坐标 + x = grid.bounds.min_x + (c + 0.5) * grid.cell_width + y = grid.bounds.max_y - (r + 0.5) * grid.cell_height + + # 模拟温度场 (简单的径向基函数) + center_x, center_y = 120.5, 30.5 + dist = math.sqrt((x - center_x)**2 + (y - center_y)**2) + temp = 25 - dist * 2 # 中心25度，向外递减 + + grid.set_value(r, c, round(temp, 2)) + + print(f"\n栅格场信息:") + print(f" 范围: {grid.bounds}") + print(f" 尺寸: {grid.rows} x {grid.cols}") + print(f" 单元格大小: {grid.cell_width:.4f} x {grid.cell_height:.4f}") + + stats = grid.get_statistics() + print(f" 统计: {stats}") + + # 点查询 + sample_point = Point(120.5, 30.5) + value = grid.get_value_at_point(sample_point) + print(f"\n点位置 {sample_point} 的值: {value}") + + # ======================================================================== + # 6. GeoJSON导出 + # ======================================================================== + print("\n\n[部分 6] GeoJSON导出") + print("-" * 50) + + # 创建特征集合 + feature_collection = { + "type": "FeatureCollection", + "features": [ + feature1.to_geojson(), + feature2.to_geojson() + ] + } + + print("\n特征集合:") + print(json.dumps(feature_collection, ensure_ascii=False, indent=2)) + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/02-spatial-intelligence/uncertainty_analysis.py b/dofile/examples/02-spatial-intelligence/uncertainty_analysis.py new file mode 100644 index 0000000..a90df82 --- /dev/null +++ b/dofile/examples/02-spatial-intelligence/uncertainty_analysis.py @@ -0,0 +1,1042 @@ +""" +不确定性分析示例 (Uncertainty Analysis Example) +============================================== + +本示例展示空间智能系统中的不确定性分析方法。 +在空间决策中，不确定性来自数据、模型和参数等多个方面。 + +核心概念: +1. 不确定性来源 - 数据误差、模型简化、参数变异 +2. 不确定性传播 - 输入不确定性如何影响输出 +3. 蒙特卡洛分析 - 随机采样评估不确定性 +4. 敏感性分析 - 识别关键不确定性源 +5. 场景分析 - 不同假设下的结果比较 + +应用场景: +- 风险评估 +- 决策稳健性分析 +- 模型可信度评估 +- 数据质量评估 + +作者: CC4SI 项目组 +""" + +import math +import random +from typing import List, Dict, Tuple, Optional, Any, Callable +from dataclasses import dataclass, field +from enum import Enum +from abc import ABC, abstractmethod +import statistics + + +# ============================================================================ +# 不确定性类型 +# ============================================================================ + +class UncertaintyType(Enum): + """不确定性类型""" + EPISTEMIC = "epistemic" # 认识不确定性 (可通过更多知识减少) + ALEATORY = "aleatory" # 偶然不确定性 (固有随机性) + PARAMETRIC = "parametric" # 参数不确定性 + STRUCTURAL = "structural" # 结构不确定性 (模型形式) + DATA = "data" # 数据不确定性 + + +@dataclass +class UncertainValue: + """ + 不确定值 + + 表示一个带有不确定性的数值。 + """ + value: float + uncertainty: float # 标准差或误差范围 + uncertainty_type: UncertaintyType = UncertaintyType.EPISTEMIC + distribution: str = "normal" # 假设的分布类型 + + @property + def coefficient_of_variation(self) -> float: + """变异系数""" + if self.value == 0: + return float('inf') + return self.uncertainty / abs(self.value) + + def confidence_interval(self, confidence: float = 0.95) -> Tuple[float, float]: + """ + 计算置信区间 + + Args: + confidence: 置信水平 + + Returns: + (下界, 上界) + """ + if self.distribution == "normal": + # 使用正态分布 + z_scores = {0.90: 1.645, 0.95: 1.96, 0.99: 2.576} + z = z_scores.get(confidence, 1.96) + return ( + self.value - z * self.uncertainty, + self.value + z * self.uncertainty + ) + else: + # 简单区间 + return ( + self.value - self.uncertainty, + self.value + self.uncertainty + ) + + def sample(self) -> float: + """从分布中采样""" + if self.distribution == "normal": + return random.gauss(self.value, self.uncertainty) + elif self.distribution == "uniform": + return random.uniform( + self.value - self.uncertainty, + self.value + self.uncertainty + ) + else: + return self.value + + def __repr__(self) -> str: + return f"{self.value:.2f} ± {self.uncertainty:.2f}" + + +# ============================================================================ +# 概率分布 +# ============================================================================ + +class ProbabilityDistribution(ABC): + """概率分布抽象基类""" + + def __init__(self, name: str = ""): + self.name = name + + @abstractmethod + def sample(self) -> float: + """采样""" + pass + + @abstractmethod + def mean(self) -> float: + """期望值""" + pass + + @abstractmethod + def std(self) -> float: + """标准差""" + pass + + +class NormalDistribution(ProbabilityDistribution): + """正态分布""" + + def __init__(self, mu: float, sigma: float, name: str = ""): + super().__init__(name) + self.mu = mu + self.sigma = sigma + + def sample(self) -> float: + return random.gauss(self.mu, self.sigma) + + def mean(self) -> float: + return self.mu + + def std(self) -> float: + return self.sigma + + +class UniformDistribution(ProbabilityDistribution): + """均匀分布""" + + def __init__(self, a: float, b: float, name: str = ""): + super().__init__(name) + self.a = a + self.b = b + + def sample(self) -> float: + return random.uniform(self.a, self.b) + + def mean(self) -> float: + return (self.a + self.b) / 2 + + def std(self) -> float: + return (self.b - self.a) / math.sqrt(12) + + +class TriangularDistribution(ProbabilityDistribution): + """三角分布""" + + def __init__(self, a: float, b: float, c: float, name: str = ""): + """ + 三角分布 + + Args: + a: 最小值 + b: 最大值 + c: 众数 + """ + super().__init__(name) + self.a = a + self.b = b + self.c = c + + def sample(self) -> float: + u = random.random() + fc = (self.c - self.a) / (self.b - self.a) + if u < fc: + return self.a + math.sqrt(u * (self.b - self.a) * (self.c - self.a)) + else: + return self.b - math.sqrt((1 - u) * (self.b - self.a) * (self.b - self.c)) + + def mean(self) -> float: + return (self.a + self.b + self.c) / 3 + + def std(self) -> float: + numerator = (self.a**2 + self.b**2 + self.c**2 - + self.a * self.b - self.a * self.c - self.b * self.c) + return math.sqrt(numerator / 18) + + +# ============================================================================ +# 不确定性传播 +# ============================================================================ + +class UncertaintyPropagator: + """ + 不确定性传播器 + + 分析输入不确定性如何影响输出。 + """ + + def __init__(self, model: Callable[[Dict[str, float]], float]): + """ + 初始化传播器 + + Args: + model: 输入字典到输出值的函数 + """ + self.model = model + + def first_order_second_moment(self, + inputs: Dict[str, UncertainValue]) -> UncertainValue: + """ + 一阶二矩法 (FOSM) + + 使用一阶泰勒展开近似传播不确定性。 + + Args: + inputs: 不确定输入字典 + + Returns: + 不确定输出 + """ + # 计算名义值 + nominal_inputs = {k: v.value for k, v in inputs.items()} + nominal_output = self.model(nominal_inputs) + + # 计算灵敏度 (数值微分) + sensitivities = {} + epsilon = 1e-6 + + for name, uncertain_val in inputs.items(): + perturbed = nominal_inputs.copy() + perturbed[name] += epsilon + output_plus = self.model(perturbed) + sensitivity = (output_plus - nominal_output) / epsilon + sensitivities[name] = sensitivity + + # 计算输出方差 (假设输入独立) + output_variance = 0.0 + for name, uncertain_val in inputs.items(): + sensitivity = sensitivities[name] + output_variance += (sensitivity * uncertain_val.uncertainty) ** 2 + + output_std = math.sqrt(output_variance) + + return UncertainValue( + value=nominal_output, + uncertainty=output_std, + uncertainty_type=UncertaintyType.EPISTEMIC + ) + + def monte_carlo_propagation(self, + input_distributions: Dict[str, ProbabilityDistribution], + n_samples: int = 10000) -> Dict[str, Any]: + """ + 蒙特卡洛传播 + + Args: + input_distributions: 输入分布字典 + n_samples: 采样次数 + + Returns: + 统计结果字典 + """ + samples = [] + + for _ in range(n_samples): + # 采样输入 + inputs = {name: dist.sample() for name, dist in input_distributions.items()} + # 计算输出 + output = self.model(inputs) + samples.append(output) + + # 计算统计量 + return { + "mean": statistics.mean(samples), + "std": statistics.stdev(samples) if len(samples) > 1 else 0, + "min": min(samples), + "max": max(samples), + "median": statistics.median(samples), + "percentiles": { + 5: self._percentile(samples, 5), + 25: self._percentile(samples, 25), + 75: self._percentile(samples, 75), + 95: self._percentile(samples, 95) + }, + "samples": samples + } + + def _percentile(self, data: List[float], p: float) -> float: + """计算百分位数""" + sorted_data = sorted(data) + index = int(p / 100 * len(sorted_data)) + return sorted_data[min(index, len(sorted_data) - 1)] + + +# ============================================================================ +# 敏感性分析 +# ============================================================================ + +@dataclass +class SensitivityMeasure: + """敏感性度量""" + parameter_name: str + sensitivity: float # 敏感性系数 + rank: int = 0 # 排名 + method: str = "" # 计算方法 + + +class SensitivityAnalyzer: + """ + 敏感性分析器 + + 识别对输出影响最大的输入参数。 + """ + + def __init__(self, model: Callable[[Dict[str, float]], float]): + """ + 初始化分析器 + + Args: + model: 输入字典到输出值的函数 + """ + self.model = model + + def local_sensitivity(self, + nominal_values: Dict[str, float], + perturbation: float = 0.01) -> List[SensitivityMeasure]: + """ + 局部敏感性分析 + + Args: + nominal_values: 名义输入值 + perturbation: 扰动比例 + + Returns: + 敏感性度量列表 + """ + nominal_output = self.model(nominal_values) + sensitivities = [] + + for param_name in nominal_values.keys(): + # 正向扰动 + perturbed_plus = nominal_values.copy() + perturbed_plus[param_name] *= (1 + perturbation) + output_plus = self.model(perturbed_plus) + + # 计算敏感性 (归一化) + delta_output = output_plus - nominal_output + sensitivity = delta_output / (perturbation * nominal_values[param_name]) + sensitivities.append({ + "parameter": param_name, + "sensitivity": sensitivity, + "delta_output": delta_output + }) + + # 排序 (绝对值) + sensitivities.sort(key=lambda x: abs(x["sensitivity"]), reverse=True) + + measures = [] + for i, s in enumerate(sensitivities, 1): + measures.append(SensitivityMeasure( + parameter_name=s["parameter"], + sensitivity=s["sensitivity"], + rank=i, + method="local" + )) + + return measures + + def variance_based_sensitivity(self, + input_distributions: Dict[str, ProbabilityDistribution], + n_samples: int = 1000) -> List[SensitivityMeasure]: + """ + 基于方差的敏感性分析 (Sobol指数近似) + + Args: + input_distributions: 输入分布 + n_samples: 样本数 + + Returns: + 敏感性度量列表 + """ + # 使用随机平衡设计近似一阶Sobol指数 + samples = [] + for _ in range(n_samples): + inputs = {name: dist.sample() for name, dist in input_distributions.items()} + outputs = self.model(inputs) + samples.append((inputs, outputs)) + + # 计算总方差 + output_values = [s[1] for s in samples] + total_variance = statistics.variance(output_values) if len(output_values) > 1 else 0 + + if total_variance == 0: + return [] + + sensitivities = [] + + for param_name in input_distributions.keys(): + # 计算条件方差 (简化: 使用回归方法) + param_values = [s[0][param_name] for s in samples] + + # 线性回归 R² + mean_x = statistics.mean(param_values) + mean_y = statistics.mean(output_values) + + numerator = sum((x - mean_x) * (y - mean_y) + for x, y in zip(param_values, output_values)) + denominator_x = sum((x - mean_x)**2 for x in param_values) + denominator_y = sum((y - mean_y)**2 for y in output_values) + + if denominator_x == 0 or denominator_y == 0: + first_order = 0 + else: + correlation = numerator / math.sqrt(denominator_x * denominator_y) + first_order = correlation ** 2 + + sensitivities.append({ + "parameter": param_name, + "sensitivity": first_order, + "variance_contribution": first_order * total_variance + }) + + # 排序 + sensitivities.sort(key=lambda x: x["sensitivity"], reverse=True) + + measures = [] + for i, s in enumerate(sensitivities, 1): + measures.append(SensitivityMeasure( + parameter_name=s["parameter"], + sensitivity=s["sensitivity"], + rank=i, + method="sobol" + )) + + return measures + + +# ============================================================================ +# 场景分析 +# ============================================================================ + +@dataclass +class Scenario: + """场景""" + name: str + description: str + parameters: Dict[str, float] + probability: float = 1.0 + + +class ScenarioAnalyzer: + """ + 场景分析器 + + 评估不同假设情景下的结果。 + """ + + def __init__(self, model: Callable[[Dict[str, float]], float]): + """ + 初始化分析器 + + Args: + model: 输入字典到输出值的函数 + """ + self.model = model + + def evaluate_scenarios(self, + scenarios: List[Scenario]) -> List[Dict[str, Any]]: + """ + 评估多个场景 + + Args: + scenarios: 场景列表 + + Returns: + 评估结果列表 + """ + results = [] + + for scenario in scenarios: + output = self.model(scenario.parameters) + + results.append({ + "scenario": scenario.name, + "description": scenario.description, + "probability": scenario.probability, + "output": output, + "parameters": scenario.parameters.copy() + }) + + return results + + def generate_scenarios(self, + base_parameters: Dict[str, float], + variations: Dict[str, Tuple[float, float]]) -> List[Scenario]: + """ + 生成场景 (基准、乐观、悲观) + + Args: + base_parameters: 基准参数 + variations: 参数变化范围字典 + + Returns: + 场景列表 + """ + scenarios = [ + Scenario( + name="基准", + description="预期情况", + parameters=base_parameters.copy(), + probability=0.5 + ) + ] + + # 乐观场景 + optimistic = base_parameters.copy() + for param, (low, high) in variations.items(): + if param in base_parameters: + # 选择有利方向的值 + if "成本" in param or "cost" in param.lower(): + optimistic[param] = low # 成本取低 + else: + optimistic[param] = high # 其他取高 + + scenarios.append(Scenario( + name="乐观", + description="最佳情况", + parameters=optimistic, + probability=0.25 + )) + + # 悲观场景 + pessimistic = base_parameters.copy() + for param, (low, high) in variations.items(): + if param in base_parameters: + if "成本" in param or "cost" in param.lower(): + pessimistic[param] = high # 成本取高 + else: + pessimistic[param] = low # 其他取低 + + scenarios.append(Scenario( + name="悲观", + description="最差情况", + parameters=pessimistic, + probability=0.25 + )) + + return scenarios + + +# ============================================================================ +# 稳健性分析 +# ============================================================================ + +class RobustnessAnalyzer: + """ + 稳健性分析器 + + 评估决策在不同条件下的稳健程度。 + """ + + def __init__(self, model: Callable[[Dict[str, float]], float]): + """ + 初始化分析器 + + Args: + model: 输入字典到输出值的函数 + """ + self.model = model + + def worst_case_analysis(self, + nominal_values: Dict[str, float], + uncertainties: Dict[str, float], + n_samples: int = 1000) -> Dict[str, Any]: + """ + 最坏情况分析 + + Args: + nominal_values: 名义值 + uncertainties: 不确定性范围 + n_samples: 采样次数 + + Returns: + 分析结果 + """ + samples = [] + + for _ in range(n_samples): + perturbed = {} + for param, nominal in nominal_values.items(): + uncertainty = uncertainties.get(param, 0) + # 均匀采样 + perturbed[param] = random.uniform( + nominal - uncertainty, + nominal + uncertainty + ) + + output = self.model(perturbed) + samples.append(output) + + return { + "nominal": self.model(nominal_values), + "best": max(samples), + "worst": min(samples), + "range": max(samples) - min(samples), + "mean": statistics.mean(samples), + "std": statistics.stdev(samples) if len(samples) > 1 else 0, + "percentile_5": self._percentile(samples, 5), + "percentile_95": self._percentile(samples, 95) + } + + def regret_analysis(self, + alternatives: List[Dict[str, float]], + scenarios: List[Dict[str, float]], + alternative_names: List[str] = None) -> Dict[str, Any]: + """ + 后悔值分析 + + Args: + alternatives: 备选方案列表 + scenarios: 场景列表 + alternative_names: 方案名称 + + Returns: + 后悔值分析结果 + """ + if alternative_names is None: + alternative_names = [f"方案{i+1}" for i in range(len(alternatives))] + + # 计算每个方案在各场景下的结果 + results_matrix = [] + for alt in alternatives: + row = [] + for scenario in scenarios: + # 合并参数 + combined = {**alt, **scenario} + output = self.model(combined) + row.append(output) + results_matrix.append(row) + + # 找出每个场景下的最优结果 + best_per_scenario = [] + for j in range(len(scenarios)): + column = [results_matrix[i][j] for i in range(len(alternatives))] + best_per_scenario.append(max(column) if column else 0) + + # 计算后悔值矩阵 + regret_matrix = [] + for i in range(len(alternatives)): + regrets = [] + for j in range(len(scenarios)): + regret = best_per_scenario[j] - results_matrix[i][j] + regrets.append(regret) + regret_matrix.append(regrets) + + # 计算最大后悔值 + max_regrets = [max(regrets) for regrets in regret_matrix] + + # 排序 + sorted_indices = sorted(range(len(max_regrets)), key=lambda i: max_regrets[i]) + + return { + "results_matrix": results_matrix, + "regret_matrix": regret_matrix, + "max_regrets": max_regrets, + "minimax_regret_choice": sorted_indices[0] if sorted_indices else None, + "ranking": sorted_indices, + "alternative_names": alternative_names + } + + def _percentile(self, data: List[float], p: float) -> float: + """计算百分位数""" + sorted_data = sorted(data) + index = int(p / 100 * len(sorted_data)) + return sorted_data[min(index, len(sorted_data) - 1)] + + +# ============================================================================ +# 空间不确定性应用 +# ============================================================================ + +class SpatialUncertaintyModel: + """ + 空间不确定性模型 + + 处理空间数据中的不确定性。 + """ + + @staticmethod + def uncertain_distance(p1: Tuple[float, float], + p2: Tuple[float, float], + position_error: float = 5.0) -> float: + """ + 带不确定性的距离计算 + + Args: + p1: 点1坐标 + p2: 点2坐标 + position_error: 位置误差标准差 + + Returns: + 采样距离 + """ + # 添加位置误差 + x1 = p1[0] + random.gauss(0, position_error) + y1 = p1[1] + random.gauss(0, position_error) + x2 = p2[0] + random.gauss(0, position_error) + y2 = p2[1] + random.gauss(0, position_error) + + return math.sqrt((x2 - x1)**2 + (y2 - y1)**2) + + @staticmethod + def uncertain_interpolation(target_point: Tuple[float, float], + sample_points: List[Tuple[Tuple[float, float], float]], + measurement_error: float = 0.1, + power: float = 2.0) -> float: + """ + 带不确定性的空间插值 (IDW) + + Args: + target_point: 目标点 + sample_points: 样本点列表 [(坐标, 值), ...] + measurement_error: 测量误差标准差 + power: IDW幂次 + + Returns: + 插值结果 + """ + tx, ty = target_point + + numerator = 0.0 + denominator = 0.0 + + for (sx, sy), value in sample_points: + # 计算距离 + distance = math.sqrt((tx - sx)**2 + (ty - sy)**2) + + if distance < 1e-10: + # 几乎重合，返回该值加误差 + return value + random.gauss(0, measurement_error) + + # 添加测量误差 + observed_value = value + random.gauss(0, measurement_error) + + # 计算权重 + weight = 1.0 / (distance ** power) + + numerator += weight * observed_value + denominator += weight + + if denominator == 0: + return 0.0 + + return numerator / denominator + + +# ============================================================================ +# 主程序 +# ======================================================================== + +def main(): + """主程序 - 演示不确定性分析的使用""" + + print("="*70) + print("不确定性分析示例演示") + print("="*70) + + random.seed(42) + + # ======================================================================== + # 1. 不确定值表示 + # ======================================================================== + print("\n[部分 1] 不确定值表示") + print("-" * 50) + + # 创建不确定值 + population = UncertainValue( + value=10000, + uncertainty=500, + uncertainty_type=UncertaintyType.EPISTEMIC, + distribution="normal" + ) + + print(f"\n人口估计: {population}") + print(f" 变异系数: {population.coefficient_of_variation:.3f}") + print(f" 95% 置信区间: {population.confidence_interval(0.95)}") + + # 采样演示 + print(f"\n采样示例:") + samples = [population.sample() for _ in range(5)] + print(f" {samples}") + + # ======================================================================== + # 2. 不确定性传播 + # ======================================================================== + print("\n\n[部分 2] 不确定性传播") + print("-" * 50) + + # 定义模型: 地块价值 = 面积 * 单价 - 开发成本 + def land_value_model(inputs): + area = inputs["area"] + unit_price = inputs["unit_price"] + development_cost = inputs["development_cost"] + return area * unit_price - development_cost + + propagator = UncertaintyPropagator(land_value_model) + + # 定义不确定输入 + uncertain_inputs = { + "area": UncertainValue(1000, 50), # 面积: 1000 ± 50 平方米 + "unit_price": UncertainValue(5000, 300), # 单价: 5000 ± 300 元/平方米 + "development_cost": UncertainValue(100000, 10000) # 开发成本 + } + + print("\n输入不确定性:") + for name, val in uncertain_inputs.items(): + print(f" {name}: {val}") + + # 一阶二矩法 + fosm_result = propagator.first_order_second_moment(uncertain_inputs) + print(f"\n一阶二矩法 (FOSM) 结果:") + print(f" 地块价值: {fosm_result}") + print(f" 95% 置信区间: {fosm_result.confidence_interval(0.95)}") + + # 蒙特卡洛传播 + input_dists = { + "area": NormalDistribution(1000, 50), + "unit_price": NormalDistribution(5000, 300), + "development_cost": NormalDistribution(100000, 10000) + } + + mc_result = propagator.monte_carlo_propagation(input_dists, n_samples=10000) + print(f"\n蒙特卡洛传播结果:") + print(f" 均值: {mc_result['mean']:.0f}") + print(f" 标准差: {mc_result['std']:.0f}") + print(f" 范围: [{mc_result['min']:.0f}, {mc_result['max']:.0f}]") + print(f" 90% 置信区间: [{mc_result['percentiles'][5]:.0f}, " + f"{mc_result['percentiles'][95]:.0f}]") + + # ======================================================================== + # 3. 敏感性分析 + # ======================================================================== + print("\n\n[部分 3] 敏感性分析") + print("-" * 50) + + # 局部敏感性 + analyzer = SensitivityAnalyzer(land_value_model) + + nominal_values = { + "area": 1000, + "unit_price": 5000, + "development_cost": 100000 + } + + local_sens = analyzer.local_sensitivity(nominal_values, perturbation=0.01) + + print("\n局部敏感性分析:") + print(f"{'排名':<6} {'参数':<15} {'敏感性系数':<15} {'影响':<15}") + print("-" * 55) + for s in local_sens: + impact = "高" if abs(s.sensitivity) > 100 else "中" if abs(s.sensitivity) > 10 else "低" + print(f"{s.rank:<6} {s.parameter_name:<15} {s.sensitivity:<15.2f} {impact:<15}") + + # 方差基敏感性 + var_sens = analyzer.variance_based_sensitivity(input_dists, n_samples=1000) + + print("\n基于方差的敏感性 (Sobol指数近似):") + print(f"{'排名':<6} {'参数':<15} {'一阶效应':<15} {'贡献':<15}") + print("-" * 55) + for s in var_sens: + contrib = f"{s.sensitivity * 100:.1f}%" + print(f"{s.rank:<6} {s.parameter_name:<15} {s.sensitivity:<15.4f} {contrib:<15}") + + # ======================================================================== + # 4. 场景分析 + # ======================================================================== + print("\n\n[部分 4] 场景分析 - 房地产项目评估") + print("-" * 50) + + # 定义项目评估模型 + def project_evaluation(inputs): + revenue = inputs["area"] * inputs["selling_price"] + cost = inputs["land_cost"] + inputs["construction_cost"] * inputs["area"] + return revenue - cost + + scenario_analyzer = ScenarioAnalyzer(project_evaluation) + + base_params = { + "area": 10000, + "selling_price": 15000, + "land_cost": 50000000, + "construction_cost": 8000 + } + + variations = { + "area": (8000, 12000), + "selling_price": (12000, 18000), + "land_cost": (40000000, 60000000), + "construction_cost": (7000, 9000) + } + + scenarios = scenario_analyzer.generate_scenarios(base_params, variations) + + print("\n生成的场景:") + for s in scenarios: + print(f" {s.name}: {s.description} (概率: {s.probability})") + + # 评估场景 + results = scenario_analyzer.evaluate_scenarios(scenarios) + + print("\n场景评估结果:") + print(f"{'场景':<10} {'利润(万元)':<15} {'概率':<10}") + print("-" * 40) + for r in results: + profit_wan = r["output"] / 10000 + print(f"{r['scenario']:<10} {profit_wan:<15.1f} {r['probability']:<10.1%}") + + # 期望值 + expected_value = sum(r["output"] * r["probability"] for r in results) + print(f"\n期望利润: {expected_value / 10000:.1f} 万元") + + # ======================================================================== + # 5. 稳健性分析 + # ======================================================================== + print("\n\n[部分 5] 稳健性分析") + print("-" * 50) + + robustness = RobustnessAnalyzer(project_evaluation) + + # 最坏情况分析 + uncertainties = { + "area": 1000, + "selling_price": 2000, + "land_cost": 5000000, + "construction_cost": 500 + } + + worst_case = robustness.worst_case_analysis(base_params, uncertainties, n_samples=1000) + + print("\n最坏情况分析:") + print(f" 名义利润: {worst_case['nominal'] / 10000:.1f} 万元") + print(f" 最好情况: {worst_case['best'] / 10000:.1f} 万元") + print(f" 最差情况: {worst_case['worst'] / 10000:.1f} 万元") + print(f" 变化范围: {worst_case['range'] / 10000:.1f} 万元") + print(f" 90% 置信区间: [{worst_case['percentile_5'] / 10000:.1f}, " + f"{worst_case['percentile_95'] / 10000:.1f}] 万元") + + # 后悔值分析 + alternatives = [ + {"area": 8000, "selling_price": 15000, "land_cost": 40000000, "construction_cost": 8000}, + {"area": 10000, "selling_price": 15000, "land_cost": 50000000, "construction_cost": 8000}, + {"area": 12000, "selling_price": 15000, "land_cost": 60000000, "construction_cost": 8000}, + ] + + scenarios_list = [ + {"selling_price": 13000}, # 价格下跌 + {"selling_price": 15000}, # 价格平稳 + {"selling_price": 17000}, # 价格上涨 + ] + + regret_result = robustness.regret_analysis( + alternatives, scenarios_list, + alternative_names=["小规模", "中规模", "大规模"] + ) + + print("\n后悔值分析:") + print(f"{'方案':<10} {'最大后悔值(万元)':<20}") + print("-" * 35) + for i, name in enumerate(regret_result["alternative_names"]): + max_regret_wan = regret_result["max_regrets"][i] / 10000 + print(f"{name:<10} {max_regret_wan:<20.1f}") + + minimax_choice = regret_result["minimax_regret_choice"] + if minimax_choice is not None: + best_name = regret_result["alternative_names"][minimax_choice] + print(f"\n最小最大后悔值推荐: {best_name}") + + # ======================================================================== + # 6. 空间不确定性 + # ======================================================================== + print("\n\n[部分 6] 空间不确定性应用") + print("-" * 50) + + # GPS定位不确定下的距离测量 + point_a = (100, 200) + point_b = (150, 250) + + print("\n带位置误差的距离测量:") + print(f" 点A: {point_a}") + print(f" 点B: {point_b}") + print(f" 理论距离: {math.sqrt((150-100)**2 + (250-200)**2):.2f}") + + # 多次测量 + measurements = [SpatialUncertaintyModel.uncertain_distance( + point_a, point_b, position_error=5 + ) for _ in range(10)] + + print(f" 实际测量 (10次): {[f'{m:.1f}' for m in measurements]}") + print(f" 平均: {statistics.mean(measurements):.2f} ± {statistics.stdev(measurements):.2f}") + + # 不确定插值 + print("\n带测量误差的空间插值:") + samples = [ + ((0, 0), 10), + ((100, 0), 20), + ((0, 100), 15), + ((100, 100), 25) + ] + target = (50, 50) + + interpolated_values = [ + SpatialUncertaintyModel.uncertain_interpolation( + target, samples, measurement_error=1 + ) for _ in range(10) + ] + + print(f" 目标点: {target}") + print(f" 插值结果 (10次): {[f'{v:.1f}' for v in interpolated_values]}") + print(f" 平均: {statistics.mean(interpolated_values):.2f} ± " + f"{statistics.stdev(interpolated_values):.2f}") + + print("\n" + "="*70) + print("演示完成!") + print("="*70) + + +if __name__ == "__main__": + main() diff --git a/dofile/examples/README.md b/dofile/examples/README.md new file mode 100644 index 0000000..30c2412 --- /dev/null +++ b/dofile/examples/README.md @@ -0,0 +1,178 @@ +# 可运行代码示例 + +本目录包含书中提到的所有可运行代码示例，按章节组织。 + +## 目录结构 + +``` +examples/ +├── 00-introduction/ # 导论部分示例 +│ └── setup-first-assistant/ # 第一个空间AI助手 +├── 01-foundations/ # 基础原理示例 +│ ├── modular-system/ # 模块化系统 +│ ├── state-machine/ # 状态机实现 +│ ├── probability/ # 概率与不确定性 +│ ├── feedback/ # 反馈与学习 +│ └── hitl/ # 人机协同 +├── 02-spatial-intelligence/ # 空间智能示例 +│ ├── spatial-representation/ # 空间表征 +│ ├── spatial-reasoning/ # 空间推理 +│ ├── mcdm/ # 多准则决策 +│ ├── optimization/ # 空间优化 +│ └── uncertainty/ # 不确定性量化 +├── 03-autonomous-design/ # 自主设计示例 +│ ├── workflow/ # 工作流编排 +│ ├── agents/ # Agent设计 +│ ├── skills/ # 技能组合 +│ └── memory/ # 记忆管理 +└── templates/ # 项目模板 + ├── basic-skill.md + ├── langgraph-workflow.py + └── hitl-checkpoint.py +``` + +## 快速开始 + +### 环境配置 + +```bash +# 克隆仓库 +git clone https://github.com/your-org/CC4SI.git +cd CC4SI + +# 创建虚拟环境 +python -m venv venv +source venv/bin/activate # Linux/Mac +# 或 +venv\Scripts\activate # Windows + +# 安装依赖 +pip install -r requirements.txt +``` + +### 运行示例 + +```bash +# 导论示例：第一个空间助手 +python 00-introduction/setup-first-assistant/spatial_helper.py + +# 基础原理示例：状态机 +python 01-foundations/state-machine/workflow_state_machine.py + +# 空间智能示例：空间推理 +python 02-spatial-intelligence/spatial-reasoning/corridor_detection.py +``` + +## 示例说明 + +### 00-introduction/setup-first-assistant + +**目标**：搭建第一个空间AI助手 + +**内容**： +- `spatial_helper.py` - 空间分析助手类 +- `create_sample_data.py` - 创建示例数据 +- `README.md` - 使用说明 + +**运行**： +```bash +cd 00-introduction/setup-first-assistant +python spatial_helper.py +``` + +--- + +### 01-foundations/modular-system + +**目标**：理解模块化系统设计 + +**内容**： +- `pipeline.py` - 空间分析流水线 +- `skills.py` - 技能封装示例 +- `composition.py` - 函数式组合 + +--- + +### 01-foundations/state-machine + +**目标**：实现状态机工作流 + +**内容**： +- `workflow_state_machine.py` - 完整状态机实现 +- `langgraph_example.py` - LangGraph版本 + +--- + +### 02-spatial-intelligence/mcdm + +**目标**：多准则决策分析 + +**内容**： +- `ahp.py` - 层次分析法 +- `sensitivity.py` - 敏感性分析 +- `weighted_overlay.py` - 加权叠加 + +--- + +### 03-autonomous-design/workflow + +**目标**：工作流编排实践 + +**内容**： +- `dag_workflow.py` - DAG工作流 +- `conditional_routing.py` - 条件路由 +- `error_handling.py` - 错误处理模式 + +--- + +## 依赖要求 + +``` +python>=3.10 + +# 核心依赖 +geopandas>=0.13.0 +shapely>=2.0.0 +rasterio>=1.3.0 +networkx>=3.0 + +# AI框架 +langchain>=0.1.0 +langgraph>=0.0.20 +anthropic>=0.18.0 + +# 可视化 +matplotlib>=3.7.0 +folium>=0.14.0 + +# 数据处理 +pandas>=2.0.0 +numpy>=1.24.0 + +# 科学计算 +scipy>=1.10.0 +scikit-learn>=1.3.0 +``` + +## 使用建议 + +1. **边学边做**：阅读相关章节后立即运行对应示例 +2. **修改实验**：在理解代码基础上进行修改和实验 +3. **错误调试**：遇到错误时尝试独立解决，培养调试能力 +4. **记录笔记**：在代码注释或笔记本中记录你的理解 + +## 贡献示例 + +欢迎贡献更多示例！ + +**示例质量要求**： +1. 代码可运行 +2. 有清晰注释 +3. 包含使用说明 +4. 说明设计意图 + +详见 [CONTRIBUTING.md](../CONTRIBUTING.md) + +## 许可 + +所有示例代码遵循项目许可证：CC BY-NC-SA 4.0 diff --git a/00-frontmatter/目录.md b/officefile/00-frontmatter/目录.md similarity index 100% rename from 00-frontmatter/目录.md rename to officefile/00-frontmatter/目录.md diff --git a/00-frontmatter/自序.md b/officefile/00-frontmatter/自序.md similarity index 100% rename from 00-frontmatter/自序.md rename to officefile/00-frontmatter/自序.md diff --git a/01-introduction/01-introduction.md b/officefile/01-introduction/01-introduction.md similarity index 100% rename from 01-introduction/01-introduction.md rename to officefile/01-introduction/01-introduction.md diff --git a/02-framework/02-framework.md b/officefile/02-framework/02-framework.md similarity index 100% rename from 02-framework/02-framework.md rename to officefile/02-framework/02-framework.md diff --git a/03-1d-sequence/03-1d-sequence.md b/officefile/03-1d-sequence/03-1d-sequence.md similarity index 100% rename from 03-1d-sequence/03-1d-sequence.md rename to officefile/03-1d-sequence/03-1d-sequence.md diff --git a/04-2d-vision/04-2d-vision.md b/officefile/04-2d-vision/04-2d-vision.md similarity index 100% rename from 04-2d-vision/04-2d-vision.md rename to officefile/04-2d-vision/04-2d-vision.md diff --git a/05-3d-spatial/05-3d-spatial.md b/officefile/05-3d-spatial/05-3d-spatial.md similarity index 100% rename from 05-3d-spatial/05-3d-spatial.md rename to officefile/05-3d-spatial/05-3d-spatial.md diff --git a/06-reinforcement/06-reinforcement.md b/officefile/06-reinforcement/06-reinforcement.md similarity index 100% rename from 06-reinforcement/06-reinforcement.md rename to officefile/06-reinforcement/06-reinforcement.md diff --git a/07-generative-ai/07-generative-ai.md b/officefile/07-generative-ai/07-generative-ai.md similarity index 100% rename from 07-generative-ai/07-generative-ai.md rename to officefile/07-generative-ai/07-generative-ai.md diff --git a/08-agent/08-agent.md b/officefile/08-agent/08-agent.md similarity index 100% rename from 08-agent/08-agent.md rename to officefile/08-agent/08-agent.md diff --git a/09-digital-media/09-digital-media.md b/officefile/09-digital-media/09-digital-media.md similarity index 100% rename from 09-digital-media/09-digital-media.md rename to officefile/09-digital-media/09-digital-media.md diff --git a/10-industrial-design/10-industrial-design.md b/officefile/10-industrial-design/10-industrial-design.md similarity index 100% rename from 10-industrial-design/10-industrial-design.md rename to officefile/10-industrial-design/10-industrial-design.md diff --git a/11-environmental-landscape/11-environmental-landscape.md b/officefile/11-environmental-landscape/11-environmental-landscape.md similarity index 100% rename from 11-environmental-landscape/11-environmental-landscape.md rename to officefile/11-environmental-landscape/11-environmental-landscape.md diff --git a/12-urban-ecology/12-urban-ecology.md b/officefile/12-urban-ecology/12-urban-ecology.md similarity index 100% rename from 12-urban-ecology/12-urban-ecology.md rename to officefile/12-urban-ecology/12-urban-ecology.md diff --git a/README.md b/officefile/README.md similarity index 100% rename from README.md rename to officefile/README.md diff --git a/appendix/附录1-基础知识与编程环境.md b/officefile/appendix/appendix/附录1-基础知识与编程环境.md similarity index 93% rename from appendix/附录1-基础知识与编程环境.md rename to officefile/appendix/appendix/附录1-基础知识与编程环境.md index 437ad75..954646c 100644 --- a/appendix/附录1-基础知识与编程环境.md +++ b/officefile/appendix/appendix/附录1-基础知识与编程环境.md @@ -67,6 +67,27 @@ apt install xxx # Ubuntu/Debian | AutoDL | 按时计费 | 中期项目 | | 阿里云PAI | 国内稳定 | 生产部署 | +### 1.4 软件安装与运行 +**"安装"的本质就是**：把编译好的二进制文件放到 PATH 某个目录下，让 shell 能找到它。例如git的安装和使用： +**总结：三层抽象** + +```text +┌─────────────────────────────────────────────┐ +│ 用户层：brew install git / git clone │ ← 你看到的 +├─────────────────────────────────────────────┤ +│ Shell 层：搜索 PATH → execve() 加载二进制 │ ← 为什么能找到命令 +├─────────────────────────────────────────────┤ +│ OS 层：系统调用 (open/read/write/socket) │ ← 为什么能真正干活 +├─────────────────────────────────────────────┤ +│ 硬件层：CPU 执行指令、网卡收发数据、磁盘写入 │ ← 物理上发生了什么 +└─────────────────────────────────────────────┘ +``` + +  所以整个链条是：**包管理器下载编译好的二进制 → 放到 PATH 目录 → shell 通过 PATH 找到它 → execve 加载到内存 →** + +   **二进制内部调用 OS API 完成实际工作**。没有任何"魔法"，本质上就是文件操作和进程管理的组合。 + + ## 2 程序设计语言与软件开发 ### 2.1 什么是程序设计语言 diff --git a/appendix/附录2-Vibe-Coding与工具链.md b/officefile/appendix/appendix/附录2-Vibe-Coding与工具链.md similarity index 100% rename from appendix/附录2-Vibe-Coding与工具链.md rename to officefile/appendix/appendix/附录2-Vibe-Coding与工具链.md diff --git a/appendix/附录3-学术论文撰写工作流.md b/officefile/appendix/appendix/附录3-学术论文撰写工作流.md similarity index 100% rename from appendix/附录3-学术论文撰写工作流.md rename to officefile/appendix/appendix/附录3-学术论文撰写工作流.md diff --git a/appendix/附录4-网络、网站与在线可视化.md b/officefile/appendix/appendix/附录4-网络、网站与在线可视化.md similarity index 100% rename from appendix/附录4-网络、网站与在线可视化.md rename to officefile/appendix/appendix/附录4-网络、网站与在线可视化.md diff --git a/appendix/附录5-在线学习资源.md b/officefile/appendix/appendix/附录5-在线学习资源.md similarity index 100% rename from appendix/附录5-在线学习资源.md rename to officefile/appendix/appendix/附录5-在线学习资源.md diff --git a/appendix/附录6-其他资源.md b/officefile/appendix/appendix/附录6-其他资源.md similarity index 100% rename from appendix/附录6-其他资源.md rename to officefile/appendix/appendix/附录6-其他资源.md diff --git a/appendix/附录7-参考文献.md b/officefile/appendix/appendix/附录7-参考文献.md similarity index 100% rename from appendix/附录7-参考文献.md rename to officefile/appendix/appendix/附录7-参考文献.md diff --git a/appendix/附录8-关键术语表.md b/officefile/appendix/appendix/附录8-关键术语表.md similarity index 100% rename from appendix/附录8-关键术语表.md rename to officefile/appendix/appendix/附录8-关键术语表.md diff --git a/appendix/附录9-Tips.md b/officefile/appendix/appendix/附录9-Tips.md similarity index 78% rename from appendix/附录9-Tips.md rename to officefile/appendix/appendix/附录9-Tips.md index ade168b..b08aaa1 100644 --- a/appendix/附录9-Tips.md +++ b/officefile/appendix/appendix/附录9-Tips.md @@ -1,5 +1,5 @@ # 附录9：Tips - 用AI学习AI，大大减少了学习的时间和难度：例如markdown等语法，只需要学习两部分内容：1）掌握经常性的手动输入需要的内容，例如#，- 等，2）了解剩余的语法的大致机制，例如图片插入可以使用html语法，公式排版使用的是Latex语法，具体实现时让AI撰写。 -- +- 有任何不懂的问题，直接问AI，如Claude code等CLI Agent以及在线等的大模型 diff --git a/officefile/appendix/glossary.md b/officefile/appendix/glossary.md new file mode 100644 index 0000000..1ffa2d1 --- /dev/null +++ b/officefile/appendix/glossary.md @@ -0,0 +1,211 @@ +# 附录C：关键术语表 + +本附录按章节整理书中涉及的关键中英文术语。 + +--- + +## A + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 自编码器 | Autoencoder, AE | 11 | +| 注意力机制 | Attention | 8 | +| 人类反馈强化学习 | RLHF | 10 | +| 人工智能 | AI, Artificial Intelligence | 1 | +| AI for Science | AI4S | 1 | +| AI in Education | AIED | 1 | + +## B + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 边界框 | Bounding Box | 6 | +| 生物多样性 | Biodiversity | 24 | + +## C + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 卷积神经网络 | CNN | 5, 6, 7 | +| 卷积核 | Kernel/Filter | 5 | +| 因果推断 | Causal Inference | - | +| 连通性 | Connectivity | 25 | +| 交叉熵 | Cross Entropy | 4 | + +## D + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 扩散模型 | Diffusion Model | 11 | +| 数字孪生 | Digital Twin | 14 | +| 深度学习 | Deep Learning | 2 | +| 设计生成式AI | Generative Design | 20 | + +## E + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 生态系统服务 | Ecosystem Services | 24 | +| 具身智能 | Embodied AI | 14 | +| 编码器 | Encoder | 9 | +| 解码器 | Decoder | 9 | + +## F + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 特征图 | Feature Map | 5 | +| 前馈网络 | FFN, Feed-Forward Network | 2, 9 | +| 俯瞰 | Foundation Model | - | + +## G + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 生成对抗网络 | GAN | 11 | +| 生成式AI | AIGC | 1, 11, 16 | +| 梯度下降 | Gradient Descent | 4 | +| 图神经网络 | GNN | 2 | + +## H + +| 中文 | 英文 | 章节 | +|-----|------|------| +| HITL | Human-in-the-Loop | 15 | +| 隐藏层 | Hidden Layer | 3 | +| 超参数 | Hyperparameter | - | + +## I + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 交并比 | IoU | 6 | +| 图像分类 | Image Classification | 6 | +| 实例分割 | Instance Segmentation | 7 | + +## K + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 键 | Key | 8 | +| 核函数 | Kernel Function | - | + +## L + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 大语言模型 | LLM | 10 | +| 损失函数 | Loss Function | 4 | +| 学习率 | Learning Rate | 4 | +| LoRA | Low-Rank Adaptation | 12 | +| 潜在空间 | Latent Space | 11 | + +## M + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 多头注意力 | Multi-Head Attention | 8 | +| 多层感知机 | MLP | 3, 4 | +| 多模态 | Multimodal | 10 | +| MCP | Model Context Protocol | 15 | +| 平均精度 | mAP | 6 | + +## N + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 归一化 | Normalization | 9 | +| 神经网络 | Neural Network | 2 | +| 非极大值抑制 | NMS | 6 | + +## O + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 目标检测 | Object Detection | 6 | +| 优化器 | Optimizer | 4 | +| One-Stage检测器 | One-Stage Detector | 6 | + +## P + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 位置编码 | Positional Encoding | 8 | +| 池化 | Pooling | 5 | +| 提示工程 | Prompt Engineering | 10 | +| 预训练 | Pre-training | 10 | +| 像素 | Pixel | 7 | + +## Q + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 查询 | Query | 8 | +| Q学习 | Q-Learning | 14 | + +## R + +| 中文 | 英文 | 章节 | +|-----|------|------| +| RAG | Retrieval-Augmented Generation | 10 | +| ReAct | Reasoning + Acting | 13 | +| 强化学习 | RL, Reinforcement Learning | 14 | +| 循环神经网络 | RNN | 2 | +| 残差连接 | Residual Connection | 9 | +| 值 | Value | 8 | +| 感受野 | Receptive Field | 5 | + +## S + +| 中文 | 英文 | 章节 | +|-----|------|------| +| Self-Attention | 自注意力 | 8 | +| 语义分割 | Semantic Segmentation | 7 | +| Scaling Law | 缩放定律 | 2 | +| Sigmoid | 激活函数 | 4 | +| Softmax | 激活函数 | 4 | +| 境况 | State | 14 | +| 潜变量 | Latent Variable | 11 | +| 支持向量机 | SVM | - | + +## T + +| 中文 | 英文 | 章节 | +|-----|------|------| +| Transformer | Transformer架构 | 9 | +| 拓扑优化 | Topology Optimization | 21 | +| Token | 令牌/词元 | 9 | +| 目标检测 | Two-Stage Detector | 6 | + +## U + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 万能逼近定理 | Universal Approximation Theorem | 2 | +| U-Net | 分割网络架构 | 7 | + +## V + +| 中文 | 英文 | 章节 | +|-----|------|------| +| VAE | Variational Autoencoder | 11 | +| 向量数据库 | Vector Database | 10 | +| 视觉Transformer | Vision Transformer | 5 | + +## W + +| 中文 | 英文 | 章节 | +|-----|------|------| +| 权重 | Weight | 3 | +| 权重共享 | Weight Sharing | 5 | +| 权重衰减 | Weight Decay | - | + +## Y + +| 中文 | 英文 | 章节 | +|-----|------|------| +| YOLO | You Only Look Once | 6 | + +--- + +**更新日期**：2026年4月 diff --git a/officefile/appendix/references.md b/officefile/appendix/references.md new file mode 100644 index 0000000..c201a82 --- /dev/null +++ b/officefile/appendix/references.md @@ -0,0 +1,111 @@ +# 参考文献 + +本部分整理教材中引用的论文、书籍和在线资源。 + +--- + +## 论文 + +### 基础理论 + +- Universal Approximation Theorem (1989) +- Scaling Laws for Neural Language Models (2020) + +### 计算机视觉 + +- AlexNet (2012) - ImageNet Classification with Deep Convolutional Neural Networks +- ResNet (2015) - Deep Residual Learning for Image Recognition +- YOLO (2016) - You Only Look Once: Unified, Real-Time Object Detection +- U-Net (2015) - Convolutional Networks for Biomedical Image Segmentation + +### Transformer与大模型 + +- Attention Is All You Need (2017) +- BERT: Pre-training of Deep Bidirectional Transformers (2018) +- GPT-3: Language Models are Few-Shot Learners (2020) +- Training Language Models to Follow Instructions with Human Feedback (2022) + +### 生成式AI + +- Denoising Diffusion Probabilistic Models (2020) +- High-Resolution Image Synthesis with Latent Diffusion Models (2022) +- ControlNet (2023) + +### AI Agent + +- LLM Powered Autonomous Agents (2023) +- ReAct: Synergizing Reasoning and Acting in Language Models (2022) + +--- + +## 书籍 + +### 深度学习 + +- Deep Learning (Ian Goodfellow et al.) +- Neural Networks and Deep Learning (Michael Nielsen) + +### 强化学习 + +- Reinforcement Learning: An Introduction (Sutton & Barto) + +--- + +## 在线资源 + +### 课程 + +- CS231n: Convolutional Neural Networks for Visual Recognition +- Fast.ai Practical Deep Learning for Coders + +### 工具文档 + +- PyTorch: https://pytorch.org/docs/ +- Ultralytics YOLO: https://docs.ultralytics.com/ +- LangChain: https://python.langchain.com/ + +### 博客 + +- Lil'Log (Lilian Weng): https://lilianweng.github.io/ +- The Illustrated Transformer: https://jalammar.github.io/illustrated-transformer/ + +--- + +## 设计AI相关 + +### 学术期刊 + +- Landscape and Urban Planning +- Environment and Planning B: Urban Analytics and City Science +- Automation in Construction + +### 会议 + +- CAAD Futures +- ACADIA +- eCAADe + +--- + +## 数据集 + +### 计算机视觉 + +- ImageNet +- COCO (Common Objects in Context) +- MNIST + +### 空间数据 + +- OpenStreetMap +- 路网数据、POI数据等 + +--- + +## 许可说明 + +部分内容引用自公开资源，遵循相应许可协议使用。 + +--- + +**最后更新**：2026年4月 diff --git a/officefile/appendix/tools.md b/officefile/appendix/tools.md new file mode 100644 index 0000000..dc7075f --- /dev/null +++ b/officefile/appendix/tools.md @@ -0,0 +1,251 @@ +# 附录A：编程工具与资源 + +本附录整理AI学习和实践所需的编程工具、框架和资源。 + +--- + +## Python环境配置 + +### Anaconda/Miniconda + +| 工具 | 大小 | 特点 | 下载地址 | +|-----|------|------|---------| +| Anaconda | ~500MB | 预装常用库 | [anaconda.com](https://www.anaconda.com/download) | +| Miniconda | ~50MB | 精简安装 | [docs.conda.io](https://docs.conda.io/en/latest/miniconda.html) | + +### 安装步骤 + +```bash +# 1. 下载并安装Miniconda +# 2. 创建虚拟环境 +conda create -n ai-env python=3.10 + +# 3. 激活环境 +conda activate ai-env + +# 4. 安装核心库 +pip install torch torchvision numpy pandas scipy +``` + +--- + +## 深度学习框架 + +### PyTorch + +```bash +pip install torch torchvision torchaudio +``` + +**特点**： +- 动态计算图 +- 研究友好 +- 广泛的社区支持 + +**资源**： +- [官方文档](https://pytorch.org/docs/) +- [中文教程](https://pytorch.zhangxiann.com/) + +### TensorFlow + +```bash +pip install tensorflow +``` + +**特点**： +- 生产部署优化 +- Keras高级API +- 跨平台支持 + +--- + +## 计算机视觉工具 + +### OpenCV + +```bash +pip install opencv-python +``` + +**功能**：图像处理、视频分析 + +### Ultralytics YOLO + +```bash +pip install ultralytics +``` + +**功能**：目标检测、实例分割 + +**使用示例**： +```python +from ultralytics import YOLO + +model = YOLO('yolov8n.pt') +results = model('image.jpg') +``` + +--- + +## AIGC工具链 + +### Stable Diffusion + +**WebUI**：[Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) + +**ComfyUI**：[GitHub](https://github.com/comfyanonymous/ComfyUI) + +**API调用**： +```python +from diffusers import StableDiffusionPipeline + +pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") +image = pipe("a photo of an astronaut riding a horse on mars").images[0] +``` + +### ControlNet + +```python +from diffusers import StableDiffusionControlNetPipeline + +controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny") +pipe = StableDiffusionControlNetPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet) +``` + +### Midjourney + +**平台**：Discord +**文档**：[docs.midjourney.com](https://docs.midjourney.com/) + +--- + +## Agent开发框架 + +### LangChain + +```bash +pip install langchain langchain-openai +``` + +**功能**：LLM应用开发框架 + +**核心组件**： +- Models：LLM接口 +- Prompts：提示管理 +- Chains：链式调用 +- Agents：智能体 +- Memory：记忆管理 + +### LangGraph + +```bash +pip install langgraph +``` + +**功能**：状态机式Agent开发 + +### LlamaIndex + +```bash +pip install llama-index +``` + +**功能**：数据索引与检索（RAG） + +--- + +## 开发工具 + +### VSCode + +**AI开发常用插件**： +- Python +- Pylance +- Jupyter +- Copilot + +### Cursor + +**特点**：AI原生IDE +**网址**：[cursor.com](https://cursor.com/) + +### Jupyter Lab + +```bash +pip install jupyterlab +jupyter lab +``` + +--- + +## 在线学习资源 + +### 课程 + +| 名称 | 平台 | 链接 | +|-----|------|------| +| CS231n | Stanford | [cs231n.stanford.edu](http://cs231n.stanford.edu/) | +| Fast.ai | fast.ai | [course.fast.ai](https://course.fast.ai/) | +| 吴恩达深度学习 | Coursera | [coursera.org/specializations/deep-learning](https://www.coursera.org/specializations/deep-learning) | + +### 博客与文档 + +- [Lil'Log](https://lilianweng.github.io/) - AI深度文章 +- [The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) +- [Distill.pub](https://distill.pub/) - 可视化论文 + +### 数据集 + +| 数据集 | 内容 | 链接 | +|-------|------|------| +| ImageNet | 图像分类 | [image-net.org](https://www.image-net.org/) | +| COCO | 目标检测 | [cocodataset.org](https://cocodataset.org/) | +| OpenStreetMap | 地图数据 | [openstreetmap.org](https://www.openstreetmap.org/) | + +--- + +## 模型资源 + +### Hugging Face + +**网址**：[huggingface.co](https://huggingface.co/) + +**功能**： +- 模型仓库 +- 数据集 +- Spaces在线演示 + +### 常用模型 + +| 任务 | 推荐模型 | Hugging Face ID | +|-----|---------|----------------| +| 文生图 | Stable Diffusion XL | stabilityai/stable-diffusion-xl-base-1.0 | +| 目标检测 | YOLOv8 | Ultralytics | +| 语义分割 | SAM | segment-anything | +| 大语言模型 | Llama 3 | meta-llama/Meta-Llama-3-8B | + +--- + +## 硬件资源 + +### 云平台 + +| 平台 | 特点 | 适合场景 | +|-----|------|---------| +| Google Colab | 免费GPU | 学习实验 | +| Kaggle Notebooks | 免费GPU | 竞赛 | +| AutoDL | 按时计费 | 中期项目 | +| 阿里云PAI | 国内稳定 | 生产部署 | + +### 本地GPU + +推荐配置： +- GPU：RTX 3060 (12GB) 或更高 +- 内存：16GB+ +- 存储：至少100GB SSD + +--- + +## 最后更新 + +2026年4月 diff --git a/officefile/supplements/00-introduction/00.1-why-this-book.md b/officefile/supplements/00-introduction/00.1-why-this-book.md new file mode 100644 index 0000000..c5436ff --- /dev/null +++ b/officefile/supplements/00-introduction/00.1-why-this-book.md @@ -0,0 +1,198 @@ +# 00.1 为什么要读这本书 + +## 核心问题 + +> 当AI能帮我们写代码、做分析、出方案时，我们为什么还要学习原理？ +> 在技术快速迭代的当下，什么样的知识才值得投资时间？ + +--- + +## 概念讲解 + +### AI的"黑箱"问题 + +随着大模型（LLM）的普及，AI系统变得越来越强大，但也越来越不透明。这种现象被称为**黑箱化**： + +``` +输入 ──→ [ AI黑箱 ] ──→ 输出 + ↑ ↑ + 看不见 理解不了 +``` + +**黑箱化的三个层次**： + +1. **算法黑箱**：神经网络内部数百万参数的决策过程难以解释 +2. **工具黑箱**：封装好的API调用，不知道内部如何工作 +3. **认知黑箱**：连问"为什么"的能力都丧失了 + +### 空间智能的特殊性 + +空间分析与一般的智能任务有几个关键区别： + +| 维度 | 一般AI任务 | 空间AI任务 | +|-----|-----------|-----------| +| 数据结构 | 文本、图像为主 | 栅格、矢量、图 | +| 关系类型 | 语义关系 | 空间关系（拓扑、距离、方向） | +| 不确定性 | 语义歧义 | 位置误差、尺度效应、边界模糊 | +| 可视化 | 辅助理解 | 核心分析手段 | +| 领域知识 | 可迁移 | 强依赖地理学、生态学等 | + +这些特殊性意味着： +- 通用AI模型不能直接解决空间问题 +- 需要理解空间分析的本质才能有效设计AI系统 +- 验证空间AI的结果需要空间思维 + +### 原理学习的持久价值 + +**技术迭代的两个速度**： + +``` +易变层（命令、框架、API）── 半衰期：6-18个月 + │ + ├─ 示例：QGIS 2.x → 3.x 界面大变 + ├─ 示例：LangChain → LangGraph 架构演进 + └─ 示例：OpenAI API 格式频繁更新 + │ +稳定层（设计思想、原理、模式）── 半衰期：10-20年 + │ + ├─ 示例：图算法在空间分析中的应用 + ├─ 示例：模块化设计原则 + └─ 示例：Human-in-the-Loop理念 +``` + +**投资回报分析**： + +| 知识类型 | 学习难度 | 过时风险 | 长期价值 | +|---------|---------|---------|---------| +| 具体命令操作 | 低 | 高 | 低 | +| 框架使用方法 | 中 | 中 | 中 | +| 设计原理 | 中 | 低 | 高 | +| 底层思想 | 高 | 极低 | 极高 | + +--- + +## 设计原理 + +### 为什么选择Claude Code作为脚手架 + +Claude Code有几个特点使其成为学习AI原理的理想工具： + +1. **透明性**：可以看到AI的思考过程（function call、tool use） +2. **可扩展性**：Skill、Agent、Hook的设计模式清晰 +3. **工具中性**：不绑定特定平台，可迁移原理 + +### "脚手架"比喻的含义 + +``` +┌─────────────────────────────────────┐ +│ 建筑物的结构 │ ← 持久的原理知识 +│ (空间智能的原理) │ +├─────────────────────────────────────┤ +│ ╔═════════╗ │ +│ ║ 脚手架 ║ │ ← 临时的学习工具 +│ ╚═════════╝ │ ← Claude Code +└─────────────────────────────────────┘ +``` + +脚手架的作用： +- **支撑学习过程**：让你能到达原本够不到的高度 +- **可拆卸**：学会后可以移除，知识已内化 +- **可复用**：用于学习不同的知识领域 + +--- + +## 案例分析：ENAgent项目 + +ENAgent是一个生态网络分析智能体系统，用它来说明原理学习的重要性。 + +### 问题背景 + +生态网络分析包括多个步骤： +1. 生态源地识别 +2. 阻力面构建 +3. 最小累积阻力（MCR）分析 +4. 生态廊道提取 +5. 网络优化 +6. 结果评估 + +传统做法：每个步骤用不同工具，手动操作，容易出错。 + +### AI解决方案 + +如果只关注"怎么做"，可能会写： +```python +# 这种代码只会调用，不理解原理 +def ecological_analysis(input_data): + result = some_ai_model.generate_analysis(input_data) + return result +``` + +**问题**： +- 结果不对时不知道为什么 +- 无法调整参数适应不同场景 +- 换个模型就不会用了 + +### 理解原理的做法 + +理解每个步骤的原理后： +```python +def ecological_analysis_with_principles( + landcover: GeoDataFrame, + species_params: Dict, + resistance_weights: Dict, + checkpoints: List[str] = None +) -> AnalysisResult: + """ + 基于原理的生态网络分析 + + 原理1：生态源地识别基于多准则评估 + 原理2：阻力面反映物种移动的空间异质性 + 原理3：MCR本质是图上的最短路径问题 + 原理4：廊道提取需要考虑连接性和宽度 + + Args: + landcover: 土地利用覆盖数据 + species_params: 目标物种的景观偏好参数 + resistance_weights: 各土地类型的阻力权重 + checkpoints: HITL审查点，人类介入的关键决策 + """ + # 每一步都理解原理，知道为什么这样做 + ... +``` + +**优势**： +- 可以解释每个决策的理由 +- 能根据具体问题调整方法 +- 工具变化时原理保持不变 + +--- + +## 反思与延伸 + +### 思考问题 + +1. **自我诊断**：你目前学习新技术的方式是什么？更偏向"怎么做"还是"为什么"？ + +2. **知识审计**：列出你掌握的技能，哪些是易变层，哪些是稳定层？ + +3. **迁移测试**：如果从QGIS换成ArcGIS，从OpenAI换成Claude，你的哪些知识可以直接迁移？ + +4. **黑箱体验**：回想一次你使用AI但无法理解结果的经历，缺少什么知识才能理解？ + +### 延伸阅读 + +- **《如何解题》** (Polya) - 数学思维的本质 +- **《系统化思维导论》** (Gerald Weinberg) - 理解复杂系统 +- **《技术的本质》** (Brian Arthur) - 技术演进的规律 + +--- + +## 关键要点 + +1. **黑箱化是AI时代的普遍风险**，主动学习原理可以抵抗 +2. **空间智能有独特性**，需要专门的理解，不能完全依赖通用AI +3. **投资稳定层知识**，半衰期长，迁移性强 +4. **Claude Code是脚手架**，帮助理解原理而非替代思考 +5. **ENAgent案例说明**：理解原理让你能设计、调试、改进系统 + +> "授人以鱼不如授人以渔。在AI时代，授人以AI不如授人以理解AI的能力。" diff --git a/officefile/supplements/00-introduction/00.2-what-is-spatial-intelligence.md b/officefile/supplements/00-introduction/00.2-what-is-spatial-intelligence.md new file mode 100644 index 0000000..dc9aa5f --- /dev/null +++ b/officefile/supplements/00-introduction/00.2-what-is-spatial-intelligence.md @@ -0,0 +1,407 @@ +# 00.2 空间智能是什么 + +## 核心问题 + +> 机器如何"理解"空间？空间智能与GIS有什么本质区别？ +> 为什么空间分析不能简单套用通用AI模型？ + +--- + +## 概念讲解 + +### 空间认知的层次 + +理解空间智能，首先需要理解人类如何认知空间，然后看机器如何模拟。 + +``` +┌─────────────────────────────────────────────────────────┐ +│ 空间认知金字塔 │ +├─────────────────────────────────────────────────────────┤ +│ │ +│ Level 4: 空间推理与决策 │ ← 最高级 +│ - 推理未知区域的属性 │ +│ - 基于空间约束做决策 │ +│ - 理解因果关系 │ +│ │ +│ Level 3: 空间关系理解 │ +│ - 拓扑关系（相邻、包含、重叠） │ +│ - 距离与方向 │ +│ - 空间模式识别 │ +│ │ +│ Level 2: 空间表征能力 │ +│ - 地图阅读 │ +│ - 比例尺理解 │ +│ - 2D ↔ 3D 转换 │ +│ │ +│ Level 1: 空间感知 │ ← 基础 +│ - 位置识别 │ +│ - 距离估计 │ +│ - 导航本能 │ +│ │ +└─────────────────────────────────────────────────────────┘ +``` + +### 从GIS到空间智能 + +GIS（地理信息系统）和空间智能的关系： + +| 维度 | GIS | 空间智能 | +|-----|-----|---------| +| **目标** | 管理、分析、可视化空间数据 | 自动化空间决策 | +| **操作者** | 人类 | 人类 + AI系统 | +| **确定性** | 算法确定，结果可重现 | 常涉及概率和不确定性 | +| **适应性** | 需要人工调整参数 | 可根据反馈自我调整 | +| **知识编码** | 隐式（在操作者脑中） | 显式（在系统中） | + +**关键区别**：GIS是工具，空间智能是能力。 + +``` +GIS 用户 → [GIS软件] → 空间分析结果 + ↑ + 人类提供所有空间知识 + +空间智能用户 → [智能体] → 空间决策 + ↑ + 系统内置空间知识 + 人类提供约束 +``` + +### 空间推理的本质 + +空间推理是空间智能的核心。它包括： + +**1. 拓扑推理** + +```python +# 拓扑关系的九交模型（DE-9IM） +def spatial_relationship(A, B): + """ + 判断两个空间对象的关系 + """ + # A的内部、边界、外部 与 B的内部、边界、外部 的交集 + intersections = [ + A.interior ∩ B.interior, # I∩I + A.interior ∩ B.boundary, # I∩B + A.interior ∩ B.exterior, # I∩E + A.boundary ∩ B.interior, # B∩I + A.boundary ∩ B.boundary, # B∩B + A.boundary ∩ B.exterior, # B∩E + A.exterior ∩ B.interior, # E∩I + A.exterior ∩ B.boundary, # E∩B + A.exterior ∩ B.exterior, # E∩E + ] + + # 根据交集模式判断关系：contains, overlaps, touches等 + return classify_relationship(intersections) +``` + +**2. 距离推理** + +不仅仅是欧氏距离，还要考虑： +- **功能距离**：时间成本、经济成本 +- **阻力距离**：穿越不同地形的代价 +- **网络距离**：沿道路网络的路径长度 + +**3. 方向推理** + +```python +# 方向关系的主方向模型 +def directional_relationship(source, target): + """ + 判断target相对于source的方向 + """ + angles = { + 'N': (0, 45), 'NE': (45, 90), 'E': (90, 135), + 'SE': (135, 180), 'S': (180, 225), 'SW': (225, 270), + 'W': (270, 315), 'NW': (315, 360) + } + + angle = compute_angle(source, target) + return get_direction(angles, angle) +``` + +--- + +## 设计原理 + +### 空间智能的两种范式 + +**范式1：符号主义空间AI** + +``` +空间知识规则库 ──→ 逻辑推理引擎 ──→ 结论 + ↑ ↓ + └──────────────────────────┘ + 反馈学习 +``` + +特点： +- 知识显式编码 +- 推理过程可解释 +- 适合结构化问题 + +**范式2：联结主义空间AI** + +``` +空间数据 → [神经网络] → 空间决策 + ↑ + 训练数据 +``` + +特点： +- 从数据中学习模式 +- 适合感知类任务 +- 可解释性较弱 + +**混合范式**（本书重点）： + +``` +符号知识 + 数据驱动 → 混合推理 → 可解释的决策 + ↓ ↓ ↓ + 专家经验 深度学习 HITL验证 +``` + +### 为什么不能直接套用通用AI + +通用大模型在空间任务上的局限： + +| 任务类型 | 通用LLM表现 | 空间智能系统 | +|---------|------------|-------------| +| 空间关系判断 | 需要坐标输入，容易出错 | 内置空间算子 | +| 距离计算 | 理解困难 | 几何计算精确 | +| 地图阅读 | 需要视觉模型 | 多模态融合 | +| 空间优化 | 搜索空间大，效率低 | 专用算法 | + +**原因**： +1. 空间关系不是语言关系，不能完全用语言描述 +2. 空间计算需要几何算法，LLM不擅长 +3. 空间数据有特殊结构（拓扑、尺度），需要专门处理 + +--- + +## 代码示例 + +### 简单空间推理演示 + +```python +from shapely.geometry import Point, Polygon +import geopandas as gpd + +class SpatialReasoner: + """ + 一个简单的空间推理器示例 + 演示AI如何理解和使用空间关系 + """ + + def __init__(self): + self.knowledge_base = { + # 关于空间关系的常识 + "containment": "包含关系是不对称的：A包含B不意味着B包含A", + "adjacency": "相邻关系是对称的：A与B相邻意味着B与A相邻", + "proximity": "邻近关系会随距离阈值变化" + } + + def analyze_spatial_config(self, features: gpd.GeoDataFrame) -> dict: + """ + 分析一组空间特征之间的配置关系 + """ + results = { + 'topology': self._analyze_topology(features), + 'clustering': self._detect_clusters(features), + 'accessibility': self._compute_accessibility(features) + } + return results + + def _analyze_topology(self, features: gpd.GeoDataFrame) -> dict: + """ + 分析拓扑关系：谁包含谁，谁与谁相邻 + """ + topology = {} + + for i, feat_i in features.iterrows(): + for j, feat_j in features.iterrows(): + if i >= j: + continue + + geom_i = feat_i.geometry + geom_j = feat_j.geometry + + relation = None + if geom_i.contains(geom_j): + relation = 'contains' + elif geom_i.within(geom_j): + relation = 'within' + elif geom_i.touches(geom_j): + relation = 'touches' + elif geom_i.intersects(geom_j): + relation = 'overlaps' + + if relation: + topology[f"{i}-{j}"] = relation + + return topology + + def _detect_clusters(self, features: gpd.GeoDataFrame, threshold=1000) -> list: + """ + 检测空间聚类：哪些对象聚集在一起 + """ + clusters = [] + used = set() + + for i, feat_i in features.iterrows(): + if i in used: + continue + + cluster = [i] + used.add(i) + + for j, feat_j in features.iterrows(): + if j in used: + continue + + if feat_i.geometry.distance(feat_j.geometry) <= threshold: + cluster.append(j) + used.add(j) + + if len(cluster) > 1: + clusters.append(cluster) + + return clusters + + def _compute_accessibility(self, features: gpd.GeoDataFrame) -> dict: + """ + 计算可达性：从一个点到其他点的便利程度 + """ + centroid = features.union_all().centroid + accessibility = {} + + for i, feat in features.iterrows(): + distance = feat.geometry.distance(centroid) + accessibility[i] = { + 'distance_to_center': distance, + 'accessibility_score': 1 / (1 + distance / 1000) + } + + return accessibility + + def infer_from_spatial_relation(self, observed_relation: str) -> str: + """ + 基于观察到的空间关系进行推理 + """ + inferences = { + "如果A包含B，且B包含C": "则A可能包含C（传递性）", + "如果A与B相邻，且B与C相邻": "A和C可能相邻或很近", + "如果一组对象形成聚类": "它们可能有相似的性质" + } + + return inferences.get(observed_relation, "无法推理") + +# 使用示例 +if __name__ == "__main__": + # 创建一些简单的空间对象 + features = gpd.GeoDataFrame({ + 'name': ['park', 'building', 'lake', 'plaza'], + 'geometry': [ + Point(0, 0).buffer(100), # park + Point(50, 50).buffer(30), # building (inside park) + Point(200, 0).buffer(80), # lake (near park) + Point(150, 50).buffer(40) # plaza + ] + }) + + reasoner = SpatialReasoner() + analysis = reasoner.analyze_spatial_config(features) + + print("=== 空间推理分析结果 ===") + print(f"拓扑关系: {analysis['topology']}") + print(f"聚类: {analysis['clustering']}") + print(f"可达性: {analysis['accessibility']}") +``` + +输出示例： +``` +=== 空间推理分析结果 === +拓扑关系: {'0-1': 'contains', '0-2': 'touches'} +聚类: [[2, 3]] +可达性: {0: {'distance_to_center': 33.5, 'accessibility_score': 0.97}, ...} +``` + +--- + +## 案例分析 + +### 真实项目：生态廊道识别中的空间推理 + +在ENAgent项目中，识别生态廊道需要复杂的空间推理： + +**推理链条**： + +1. **前提1**：源地A和B之间存在潜在廊道 +2. **前提2**：廊道需要满足最小宽度要求 +3. **前提3**：廊道上的阻力值不应超过阈值 +4. **前提4**：廊道应该连接相似的生境类型 +5. **结论**：A和B之间的最优廊道是... + +**AI系统如何执行**： + +```python +def find_ecological_corridor(source_a, source_b, landscape): + """ + 基于空间推理的生态廊道识别 + """ + + # 1. 计算MCR表面（空间计算） + mcr_surface = compute_mcr(landscape, source_a, source_b) + + # 2. 提取最小阻力路径（图算法） + path = extract_least_cost_path(mcr_surface, source_a, source_b) + + # 3. 验证宽度约束（空间关系） + width = calculate_corridor_width(path, landscape) + if width < MIN_CORRIDOR_WIDTH: + # 推理：如果太窄，尝试次优路径 + path = extract_next_best_path(mcr_surface, source_a, source_b) + + # 4. 评估生境连续性（领域知识） + continuity = assess_habitat_continuity(path, landscape) + if continuity < threshold: + # 推理：生境不连续，廊道可能无效 + return None + + return path +``` + +这个例子展示了： +- **空间计算**：MCR、距离计算 +- **图算法**：最短路径 +- **领域知识**：最小宽度、生境连续性 +- **推理逻辑**：约束不满足时的回溯 + +--- + +## 反思与延伸 + +### 思考问题 + +1. **自我评估**：你能在脑海中"想象"一个空间场景的拓扑关系吗？机器如何做到同样的事？ + +2. **设计挑战**：如果要设计一个"智能地图助手"，它应该具备哪些空间推理能力？ + +3. **局限分析**：当前的空间智能系统在哪些空间推理任务上仍然不如人类？ + +4. **未来想象**：如果机器具备完全的空间智能，它能做什么现在做不到的事？ + +### 延伸阅读 + +- **《空间认知与计算》** - 认知科学与GIS的交叉 +- **"Spatial Cognition"** (Kluwer Academic Publishers) - 空间认知的经典教材 +- **QGIS文档** - 理解实际软件中的空间算子实现 + +--- + +## 关键要点 + +1. **空间认知有多个层次**，从感知到推理逐级递进 +2. **GIS ≠ 空间智能**：前者是工具，后者是能力 +3. **空间推理的核心**：拓扑关系、距离推理、方向推理 +4. **通用AI不能直接解决空间问题**，需要专门的空间算子 +5. **混合范式**结合符号知识和数据驱动是当前最佳实践 diff --git a/officefile/supplements/00-introduction/00.3-what-is-autonomous-design.md b/officefile/supplements/00-introduction/00.3-what-is-autonomous-design.md new file mode 100644 index 0000000..3f5fe73 --- /dev/null +++ b/officefile/supplements/00-introduction/00.3-what-is-autonomous-design.md @@ -0,0 +1,427 @@ +# 00.3 自主设计的含义 + +## 核心问题 + +> "自动"和"自主"有什么本质区别？ +> 在设计工作中，AI应该扮演什么角色？工具还是伙伴？ +> Human-in-the-Loop不是落后，而是高级的设计哲学？ + +--- + +## 概念讲解 + +### 自动 vs 自主 + +这两个词经常被混用，但在AI系统设计中有重要区别： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 自动化(Automation) │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 设定 → [固定脚本] → 执行 → 结果 │ +│ ↑ ↓ │ +│ 预定义规则 完全确定 │ +│ │ +│ 特点： │ +│ - 按预定规则执行 │ +│ - 遇到异常停止 │ +│ - 重复性任务 │ +│ - 人类设定后不再介入 │ +│ │ +└─────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────┐ +│ 自主化(Autonomy) │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 目标 → [智能体] → 观察 → 决策 → 行动 → 反馈 │ +│ ↑ ↑ ↑ ↓ │ +│ 高层意图 持续学习 适应调整 环境变化 │ +│ │ +│ 特点： │ +│ - 追求目标而非执行步骤 │ +│ - 能处理意外情况 │ +│ - 从经验中学习 │ +│ - 在约束下自主决策 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 理解的层级 + +自主系统的智能程度可以分层： + +``` +Level 0: 无智能 (No Intelligence) + └── 简单机械，完全由外部控制 + +Level 1: 反应式智能 (Reactive Intelligence) + └── 基于当前状态直接反应，无记忆 + 例如：恒温器、简单的自动门 + +Level 2: 基于规则的智能 (Rule-based Intelligence) + └── 遵循预定义规则，有分支逻辑 + 例如：专家系统、决策树 + +Level 3: 学习型智能 (Learning Intelligence) + └── 能从数据中学习，改进性能 + 例如：机器学习模型 + +Level 4: 自主智能 (Autonomous Intelligence) + └── 设定目标，规划执行，处理意外 + 例如：自动驾驶、智能体系统 + +Level 5: 协作智能 (Collaborative Intelligence) + └── 与人类协同，理解意图和上下文 + 例如：设计伙伴AI +``` + +### 设计智能的演进 + +从CAD工具到AI设计伙伴的演进： + +``` +1980s: CAD时代 + └── 数字化绘图，提高效率 + 设计师 → [绘图工具] → 图纸 + +2000s: 参数化设计时代 + └── 规则驱动，生成变体 + 设计师 → [参数+规则] → [生成器] → 多方案 + +2010s: 优化时代 + └── 目标驱动，搜索最优解 + 设计师 → [目标函数] → [优化算法] → 最优方案 + +2020s: AI生成时代 + └── 意图驱动，智能生成 + 设计师 → [意图描述] → [AI模型] → 设计方案 + +未来: 协作设计时代 + └── 人机协同，共同创造 + 设计师 ⇄ [AI伙伴] ⇄ 设计结果 + 共享理解 +``` + +--- + +## 设计原理 + +### Human-in-the-Loop (HITL) 的设计哲学 + +HITL不是"半自动"的妥协，而是深思熟虑的设计选择。 + +**为什么需要HITL？** + +1. **空间问题的复杂性** + - 多目标权衡（生态 vs 经济 vs 社会） + - 隐性知识无法完全编码 + - 价值判断需要人类 + +2. **责任归属** + - 重大决策不能完全交给机器 + - 设计师需要对结果负责 + - 伦理考量需要人类判断 + +3. **信任建立** + - 逐步建立对AI的信任 + - 可解释性增强信任 + - 控制感增强接受度 + +**HITL的三种模式**： + +```python +# 模式1: 决策前审查 (Pre-decision Review) +def hitl_pre_review(agent_decision, human_expert): + """ + 人类在AI决策前审查 + """ + proposal = agent.generate_proposal() + approval = human_expert.review(proposal) + + if approval: + return agent.execute(proposal) + else: + feedback = human_expert.provide_feedback() + return agent.regenerate(feedback) + +# 模式2: 关键点介入 (Checkpoint Intervention) +def hitl_checkpoint(workflow, checkpoints): + """ + 在关键决策点人类介入 + """ + for step in workflow: + result = step.execute() + + if step.name in checkpoints: + # 只在关键点需要人类确认 + if not human.confirm(result): + result = human.modify(result) + + return result + +# 模式3: 异常处理 (Exception Handling) +def hitl_exception(agent, task): + """ + AI正常运行，异常时人类介入 + """ + try: + return agent.execute(task) + except UncertainSituation as e: + # AI遇到不确定情况，请求人类帮助 + return human.resolve(e) +``` + +### ENAgent的三个审查点 + +ENAgent项目的HITL设计： + +``` +┌────────────────────────────────────────────────────────────┐ +│ ENAgent 工作流程 │ +├────────────────────────────────────────────────────────────┤ +│ │ +│ 1. 数据准备 │ +│ └── 自动化执行 │ +│ │ +│ 2. 生态源地识别 ──→ [审查点1] ──→ 确认/调整源地 │ +│ (AI识别) (人类专家) (最终决策) │ +│ │ +│ 3. 阻力面构建 ──→ [审查点2] ──→ 确认/调整权重 │ +│ (AI建议) (人类专家) (最终参数) │ +│ │ +│ 4. MCR分析 │ +│ └── 自动化执行 │ +│ │ +│ 5. 廊道提取 ──→ [审查点3] ──→ 确认/优化廊道 │ +│ (AI识别) (人类专家) (最终方案) │ +│ │ +│ 6. 结果评估 │ +│ └── AI + 人类共同评估 │ +│ │ +└────────────────────────────────────────────────────────────┘ +``` + +**为什么选择这三个点？** + +| 审查点 | 原因 | 人类贡献 | +|-------|------|---------| +| 源地识别 | 需要本地知识，遥感可能误判 | 地面实况，专家经验 | +| 阻力权重 | 价值判断，物种特性不同 | 生态学知识，实地经验 | +| 廊道优化 | 多目标权衡，不能完全量化 | 规划要求，社会因素 | + +--- + +## 代码示例 + +### HITL工作流示意 + +```python +from typing import Callable, Optional +from dataclasses import dataclass +from enum import Enum + +class ReviewDecision(Enum): + APPROVE = "approve" + MODIFY = "modify" + REJECT = "reject" + +@dataclass +class ReviewResult: + decision: ReviewDecision + feedback: Optional[str] = None + modifications: Optional[dict] = None + +class HumanReviewer: + """ + 人类审查者的抽象接口 + 实际实现可以是CLI界面、Web界面等 + """ + + def review(self, proposal: dict, context: str) -> ReviewResult: + """ + 审查AI的提案 + + Args: + proposal: AI生成的提案 + context: 审查上下文信息 + + Returns: + 审查决定 + """ + print(f"\n=== 审查点: {context} ===") + print(f"AI提案: {proposal}") + + # 实际实现中，这里会显示GUI或调用外部接口 + decision = input("决策 (approve/modify/reject): ") + + if decision == "approve": + return ReviewDecision.APPROVE + elif decision == "modify": + feedback = input("修改意见: ") + return ReviewDecision(ReviewDecision.MODIFY, feedback=feedback) + else: + return ReviewDecision(ReviewDecision.REJECT) + +class HITLWorkflow: + """ + 带人类审查的自主工作流 + """ + + def __init__(self, reviewer: HumanReviewer): + self.reviewer = reviewer + self.checkpoints = [] + + def add_checkpoint(self, name: str, condition: Callable = None): + """ + 添加审查点 + + Args: + name: 审查点名称 + condition: 触发审查的条件函数 + """ + self.checkpoints.append({ + 'name': name, + 'condition': condition or (lambda _: True) + }) + + def run(self, steps: list) -> dict: + """ + 执行工作流，在审查点进行人类介入 + + Args: + steps: 工作流步骤列表 + + Returns: + 最终结果 + """ + context = {} + + for i, step in enumerate(steps): + # 执行步骤 + step_name = step.get('name', f'step_{i}') + step_func = step['execute'] + + print(f"\n执行步骤: {step_name}") + result = step_func(context) + context[step_name] = result + + # 检查是否需要审查 + for checkpoint in self.checkpoints: + if checkpoint['name'] == step_name: + if checkpoint['condition'](context): + review_result = self.reviewer.review( + result, step_name + ) + + if review_result.decision == ReviewDecision.REJECT: + # 拒绝，重新执行 + print("提案被拒绝，重新执行...") + return self.run(steps) + + elif review_result.decision == ReviewDecision.MODIFY: + # 修改，更新上下文 + print(f"应用修改: {review_result.feedback}") + context[step_name] = self._apply_modifications( + result, review_result.modifications + ) + + # APPROVE: 继续执行 + + return context + + def _apply_modifications(self, original: dict, modifications: dict) -> dict: + """应用人类修改""" + if modifications: + original.update(modifications) + return original + +# 使用示例：生态网络分析的HITL工作流 +if __name__ == "__main__": + # 创建审查者 + reviewer = HumanReviewer() + + # 创建工作流 + workflow = HITLWorkflow(reviewer) + + # 添加审查点 + workflow.add_checkpoint('identify_sources') + workflow.add_checkpoint('build_resistance') + workflow.add_checkpoint('extract_corridors') + + # 定义工作流步骤 + def step_load_data(context): + return {'data_loaded': True} + + def step_identify_sources(context): + # 模拟AI识别源地 + return { + 'sources': [ + {'id': 1, 'area': 1500, 'type': 'forest'}, + {'id': 2, 'area': 800, 'type': 'wetland'} + ], + 'confidence': 0.85 + } + + def step_build_resistance(context): + # 模拟AI构建阻力面 + return { + 'weights': { + 'forest': 1, + 'grassland': 10, + 'urban': 100, + 'water': 50 + } + } + + def step_extract_corridors(context): + # 模拟AI提取廊道 + return { + 'corridors': [ + {'from': 1, 'to': 2, 'length': 3500, 'quality': 'high'} + ] + } + + steps = [ + {'name': 'load_data', 'execute': step_load_data}, + {'name': 'identify_sources', 'execute': step_identify_sources}, + {'name': 'build_resistance', 'execute': step_build_resistance}, + {'name': 'extract_corridors', 'execute': step_extract_corridors}, + ] + + # 执行工作流 + result = workflow.run(steps) + print("\n=== 工作流完成 ===") + print(result) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **角色定位**：在你自己的工作中，你希望AI扮演什么角色？工具、助手还是伙伴？ + +2. **审查点设计**：如果要为你熟悉的工作流程设计HITL，你会选择哪些审查点？为什么？ + +3. **信任边界**：在什么情况下你会完全信任AI的决策？在什么情况下必须人类介入？ + +4. **责任归属**：如果HITL系统做出了错误决策导致损失，责任应该如何划分？ + +### 延伸阅读 + +- **"Human-in-the-Loop Machine Learning"** - HITL的系统论述 +- **"Human-Centered AI"** (Ben Shneiderman) - 以人为本的AI设计 +- **"Designing Autonomous Agents"** - 自主智能体设计理论 + +--- + +## 关键要点 + +1. **自动 ≠ 自主**：自动执行预定义步骤，自主追求目标并适应环境 +2. **智能有层级**：从反应式到协作式，逐级递进 +3. **HITL不是妥协**：而是深思熟虑的设计选择，在复杂、重要的决策中必不可少 +4. **审查点选择是关键**：选择需要人类独特能力的决策点 +5. **从工具到伙伴**：AI在设计中角色的演进，目标是协作而非替代 diff --git a/officefile/supplements/00-introduction/00.4-claude-code-as-scaffold.md b/officefile/supplements/00-introduction/00.4-claude-code-as-scaffold.md new file mode 100644 index 0000000..37ec66e --- /dev/null +++ b/officefile/supplements/00-introduction/00.4-claude-code-as-scaffold.md @@ -0,0 +1,429 @@ +# 00.4 Claude Code作为脚手架 + +## 核心问题 + +> 为什么选择Claude Code作为学习工具？ +> 如何"借力打力"，用AI来学习AI？ +> 工具更新换代时，如何保持知识的可迁移性？ + +--- + +## 概念讲解 + +### Claude Code是什么 + +Claude Code是Anthropic推出的命令行AI开发助手，但它本质上是一个**学习工具**。 + +**核心特点**： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Claude Code 架构 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 用户 ──→ [Claude Code CLI] ──→ Claude API ──→ 响应 │ +│ │ │ │ +│ │ │ │ +│ └──── 三个核心概念 ────────────────────────────┘ │ +│ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ Agent │ │ Skill │ │ Hook │ │ +│ │ (智能体) │ │ (技能) │ │ (钩子) │ │ +│ └─────────────┘ └─────────────┘ └─────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**对比其他工具**： + +| 维度 | Copilot | ChatGPT | Claude Code | +|-----|---------|---------|-------------| +| 主要界面 | IDE插件 | Web界面 | 命令行 | +| 工作方式 | 代码补全 | 对话 | 对话+执行 | +| 可扩展性 | 有限 | 有限 | 高(Skill系统) | +| 透明度 | 低 | 中 | 高(显示思考) | +| 工具链集成 | IDE内 | 需复制粘贴 | 直接执行 | + +### 为什么选择Claude Code + +**1. 透明性** + +Claude Code可以看到AI的思考过程： + +``` +用户: 帮我分析这个生态网络 + +Claude Code: +思考: 用户想要分析生态网络，我需要： +1. 识别这是一个空间分析任务 +2. 确定需要的数据类型 +3. 选择合适的分析方法 +4. 调用相关技能 + +动作: 调用 skill:ecological-analysis +输入: {...} +结果: {...} +``` + +这种透明性对学习很重要——你可以看到AI**如何分解问题**。 + +**2. 可扩展性** + +```yaml +# .claude/skills/spatial-analysis.md +--- +description: 空间分析技能 +--- + +当用户需要进行空间分析时： +1. 确定分析类型（缓冲区分析、叠加分析等） +2. 检查所需数据 +3. 执行分析 +4. 可视化结果 +``` + +你可以定义自己的技能，学习如何将领域知识编码为AI可以理解和执行的形式。 + +**3. 工具中性** + +``` +Claude Code + │ + ├─ 可以调用 QGIS + ├─ 可以调用 ArcGIS + ├─ 可以调用 Python脚本 + └─ 可以调用任何命令行工具 +``` + +学习的是**如何编排工具**，而不是特定工具的用法。 + +### 三个核心概念 + +#### 1. Agent（智能体） + +Agent是具有目标、能自主行动的系统： + +```python +# Agent的抽象结构 +class Agent: + def __init__(self, goal, tools, memory): + self.goal = goal # 目标 + self.tools = tools # 可用工具 + self.memory = memory # 记忆 + + def perceive(self, observation): + """感知环境""" + pass + + def decide(self): + """决定下一步行动""" + pass + + def act(self): + """执行行动""" + pass + + def reflect(self): + """反思和学习""" + pass +``` + +**设计问题**： +- Agent应该有多大的自主性？ +- 如何让Agent的目标与人类意图对齐？ +- 如何处理Agent的失败？ + +#### 2. Skill（技能） + +Skill是能力的封装： + +```yaml +# skill示例 +--- +name: extract-ecological-sources +description: 从土地利用数据中提取生态源地 +parameters: + - landcover_data: GeoDataFrame + - min_patch_size: float + - suitability_threshold: float +returns: + - sources: GeoDataFrame + - statistics: dict +--- + +## 执行逻辑 + +1. 读取土地利用数据 +2. 计算每个斑块的适宜性 +3. 筛选满足条件的斑块 +4. 按面积排序，选择前N个 +``` + +**设计问题**： +- 如何定义技能的边界？ +- 技能之间如何组合？ +- 如何让技能可复用？ + +#### 3. Hook（钩子） + +Hook是事件驱动的扩展点： + +```yaml +# hook示例 +--- +name: pre-tool-use +description: 在工具调用前执行 +trigger: + - event: ToolUse +condition: tool_name == "ecological-analysis" +action: | + 1. 验证输入数据的坐标系 + 2. 检查数据完整性 + 3. 记录分析参数用于审计 +--- + +## 价值 +- 防止错误的分析 +- 建立可追溯性 +- 注入质量控制 +``` + +**设计问题**： +- 哪些事件需要hook？ +- 如何在自动化和控制之间平衡？ +- 如何避免过度使用hook导致系统僵化？ + +--- + +## 设计原理 + +### 工具中性原则 + +**核心思想**：学习的是设计模式，而非特定工具。 + +### 指月之辨：工具与真理的关系 + +**禅宗寓言**：佛经记载，佛法如标指，真理如明月。指非月也，是指月之指也。 + +**寓意**： +- 手指（工具）指向月亮（真理/原理） +- 我们需要借助手指才能看到月亮 +- 但手指不是月亮本身，不要执着于手指 + +**对应到学习**： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 手指 (指) 月亮 (月) │ +│ ↓ ↓ │ +│ 工具 原理/真理 │ +│ - Claude Code - 状态机设计 │ +│ - LangGraph - 模块化思维 │ +│ - 具体框架 - 设计模式 │ +│ - 命令语法 - AI本质 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**实践启示**： + +1. **借指观月，而非指为月** + - 通过工具学习原理，而非学习工具本身 + - Claude Code 是"指"，帮助我们指向"月"（AI原理） + +2. **手指过河，筏喻登山** + - 工具如竹筏，渡河后应舍弃 + - 学会原理后，工具可以更换 + +3. **莫执着于一指** + - 不固守单一工具 + - 不同工具可指向同一真理 + - 工具中性：多工具对比学习 + +> "授人以鱼不如授人以渔。但更重要的是：不要把渔具当成目的。 +> 渔具是指向智慧之鱼的'指'，真正的'月'是理解如何独立思考、 +> 如何在新工具出现时快速学习、如何构建持久的认知框架。" + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 抽象层（持久知识） │ +│ │ +│ 工作流编排 │ 状态管理 │ 技能组合 │ +│ │ +├─────────────────────────────────────────────────────────────┤ +│ 实现层（易变知识） │ +│ │ +│ Claude Code │ LangGraph │ LangChain │ ... │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**实践方法**： + +1. **学习概念，而非命令** + - 理解"状态机"比记住LangGraph的语法更重要 + - 理解"技能抽象"比记住Skill格式更重要 + +2. **关注接口，而非实现** + - 定义清晰的输入输出 + - 理解组件之间的交互模式 + +3. **多工具对比** + - 用不同工具实现同一任务 + - 理解不同设计的trade-off + +### 借力打力：用AI学习AI + +``` +你的学习路径 + +1. 向Claude Code提问 + "解释一下状态机在Agent系统中的作用" + +2. 让它生成示例 + "给我一个简单的状态机示例代码" + +3. 让它解释代码 + "逐行解释这段代码在做什么" + +4. 让它出题测试 + "出几道关于状态机的问题测试我的理解" + +5. 让它点评你的答案 + "这是我的理解，请点评并指出不足" +``` + +--- + +## 代码示例 + +### Claude Code基础操作 + +```bash +# 1. 启动Claude Code +claude + +# 2. 基本对话 +> 你好，请介绍一下空间智能的概念 + +# 3. 执行命令 +> 列出当前目录的文件 +# Claude会自动执行 ls 命令 + +# 4. 读取文件 +> 读取 README.md 的内容 + +# 5. 使用技能 +> /spatial-analysis 分析这个区域的生态网络 + +# 6. 创建新技能 +> 创建一个名为"buffer-analysis"的技能 +``` + +### 创建一个简单的技能 + +文件：`.claude/skills/hello-spatial.md` + +```yaml +--- +name: hello-spatial +description: 向用户问好并介绍空间分析基础 +--- + +当用户请求空间分析入门介绍时，使用此技能。 + +## 执行步骤 + +1. 询问用户的空间分析背景 +2. 根据背景调整解释深度 +3. 提供一个简单的空间分析示例 +4. 推荐进一步学习资源 + +## 示例对话 + +用户: "我是空间分析的初学者" +AI: "欢迎！空间分析是关于..." +``` + +--- + +## 实践示例：Claude Code的第一个空间任务 + +```bash +# 场景：用Claude Code分析一个简单的空间问题 + +> 我有一个包含公园位置的CSV文件，想找出距离每个居民区最近的车公园 + +# Claude Code的响应过程： +# 1. 理解任务 - 这是一个最近邻分析问题 +# 2. 确定工具 - 使用geopandas和scipy +# 3. 编写代码 - 生成Python脚本 +# 4. 执行分析 - 运行脚本 +# 5. 展示结果 - 可视化输出 + +让我帮您完成这个最近邻分析。 + +首先，让我读取您的数据： +[读取 parks.csv] + +然后，我将： +1. 计算每个居民区到所有公园的距离 +2. 找出最近的车公园 +3. 计算平均服务距离 +4. 生成可视化地图 + +[生成并执行 Python 代码] + +结果： +- 平均最近距离：850米 +- 服务覆盖率：78% +- 需要新建公园的区域：[列表] +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **工具选择**：你目前使用的AI工具是什么？它在哪些方面帮助/阻碍了你的学习？ + +2. **知识迁移**：如果Claude Code不再维护，你学到的哪些知识可以迁移到其他工具？ + +3. **透明性**：你认为AI系统应该有多透明？完全透明是否会降低效率？ + +4. **自主性**：在使用AI工具时，你希望在多大程度上保持控制权？ + +### 实践练习 + +1. **安装Claude Code** + ```bash + npm install -g @anthropic/claude-code + ``` + +2. **创建你的第一个技能** + - 选择你熟悉的领域 + - 定义技能的输入输出 + - 编写执行逻辑 + +3. **与Claude对话** + - 询问一个你不懂的AI概念 + - 让它用多种方式解释 + - 测试你的理解 + +### 延伸阅读 + +- **[Claude Code官方文档](https://claudecode.io/zh)** - 最权威的参考资料 +- **"Tools for Thought"** (Richard Wurman) - 思考工具的历史 +- **"Building Software with AI"** - AI辅助软件开发的实践 + +--- + +## 关键要点 + +1. **Claude Code是学习工具**，不只是生产力工具 +2. **透明性、可扩展性、工具中性**是其核心优势 +3. **Agent、Skill、Hook**三个概念构成核心抽象 +4. **关注抽象层**的知识，实现层会变化 +5. **借力打力**：用AI来学习AI是最高效的方式 diff --git a/officefile/supplements/00-introduction/README.md b/officefile/supplements/00-introduction/README.md new file mode 100644 index 0000000..282ed52 --- /dev/null +++ b/officefile/supplements/00-introduction/README.md @@ -0,0 +1,111 @@ +# 第一部分：导论 + +## 本部分目标 + +建立读者对AI在空间领域应用的宏观认知，理解： +- 为什么在AI时代需要理解原理而非仅仅学习操作 +- 空间智能的独特性及其与通用AI的区别 +- 自主设计的真正含义及其在设计实践中的价值 +- Claude Code如何作为学习工具而不仅是生产力工具 + +--- + +## 章节导航 + +| 章节 | 文件 | 核心问题 | 实践 | +|-----|------|---------|------| +| 00.1 | [为什么要读这本书](./00.1-why-this-book.md) | AI只是工具吗？原理知识为何重要？ | ENAgent项目介绍 | +| 00.2 | [空间智能是什么](./00.2-what-is-spatial-intelligence.md) | 机器如何"理解"空间？ | 空间推理演示 | +| 00.3 | [自主设计的含义](./00.3-what-is-autonomous-design.md) | 自动与自主有何区别？ | HITL工作流示意 | +| 00.4 | [Claude Code作为脚手架](./00.4-claude-code-as-scaffold.md) | 如何借力AI学习AI？ | Claude Code基础操作 | + +--- + +## 学习路径 + +``` +开始阅读 + │ + ├─ 你怀疑AI的可靠性？ + │ └─ 重点阅读 00.1 + │ + ├─ 你想了解空间AI的独特性？ + │ └─ 重点阅读 00.2 + │ + ├─ 你对Agent和自动化感兴趣？ + │ └─ 重点阅读 00.3 + │ + └─ 你想快速上手实践？ + └─ 直接跳到 00.4 +``` + +--- + +## 前置知识 + +**必需**： +- 基本的GIS概念（图层、坐标系、矢量/栅格） +- Python编程基础 + +** helpful but not required**： +- 机器学习的基本概念 +- 工作流自动化经验 + +--- + +## 预计学习时间 + +| 阅读类型 | 时间估计 | +|---------|---------| +| 快速浏览 | 2-3小时 | +| 理解性阅读 | 6-8小时 | +| 完成所有实践 | 12-15小时 | + +--- + +## 章节亮点 + +### 00.1 为什么要读这本书 +- 揭示"命令依赖症"的危害 +- 解释为什么AI知识半衰期越来越短 +- 提出持久认知框架的构建方法 + +### 00.2 空间智能是什么 +- 从人类空间认知到机器空间智能 +- GIS与空间智能的本质区别 +- 空间推理的两种范式 + +### 00.3 自主设计的含义 +- 澄清常见误解：自动 ≠ 自主 +- HITL（Human-in-the-Loop）的设计哲学 +- 从工具到伙伴的关系转变 + +### 00.4 Claude Code作为脚手架 +- Claude Code的独特优势 +- Agent、Skill、Hook的直观理解 +- 工具中性原则：如何不被特定工具绑定 + +--- + +## 实践案例00：搭建你的第一个空间AI助手 + +在阅读完所有章节后，你将能够： +1. 配置Claude Code环境 +2. 创建一个简单的空间查询Skill +3. 理解Agent如何调用工具完成任务 +4. 建立对后续学习的信心 + +详见 [practice/setup-first-assistant](./practice/setup-first-assistant/) + +--- + +## 思考问题 + +在开始阅读前，请思考： + +1. 你目前使用AI的方式是什么？遇到的最大困扰是什么？ +2. 你认为"理解原理"和"学会操作"哪个更重要？为什么？ +3. 空间分析与一般的智能任务有什么不同？ +4. 你期待AI在设计工作中扮演什么角色？ + +带着这些问题阅读，会更有收获。 diff --git a/officefile/supplements/00-introduction/practice/setup-first-assistant/README.md b/officefile/supplements/00-introduction/practice/setup-first-assistant/README.md new file mode 100644 index 0000000..e9c1593 --- /dev/null +++ b/officefile/supplements/00-introduction/practice/setup-first-assistant/README.md @@ -0,0 +1,348 @@ +# 实践案例00：搭建你的第一个空间AI助手 + +## 目标 + +通过本实践，你将： +1. 配置Claude Code环境 +2. 创建一个简单的空间查询技能 +3. 理解Agent如何调用工具 +4. 建立对后续学习的信心 + +--- + +## 步骤1：环境准备 + +### 安装Claude Code + +```bash +# 使用npm安装 +npm install -g @anthropic-ai/claude-code + +# 验证安装 +claude --version +``` + +### 配置API密钥 + +```bash +# 设置API密钥 +claude config set api_key your_anthropic_api_key + +# 或使用环境变量 +export ANTHROPIC_API_KEY=your_key +``` + +### 安装Python依赖 + +```bash +# 创建虚拟环境 +python -m venv venv +source venv/bin/activate # Linux/Mac +# 或 +venv\Scripts\activate # Windows + +# 安装依赖 +pip install geopandas shapely matplotlib pandas +``` + +--- + +## 步骤2：创建空间查询技能 + +### 2.1 创建技能目录 + +```bash +mkdir -p .claude/skills +``` + +### 2.2 编写技能文件 + +创建 `.claude/skills/spatial-query.md`： + +```markdown +--- +name: spatial-query +description: 执行基础空间查询和分析 +parameters: + - data_file: 空间数据文件路径 + - query_type: 查询类型（buffer/intersect/nearest） + - distance: 缓冲距离（用于buffer查询） +--- + +## 空间查询技能 + +当用户需要进行空间查询时，使用此技能。 + +### 支持的查询类型 + +1. **buffer**: 缓冲区分析 +2. **intersect**: 相交分析 +3. **nearest**: 最近邻查找 + +### 执行流程 + +1. 读取空间数据 +2. 根据查询类型执行相应操作 +3. 返回结果和统计信息 +4. 可选：生成可视化 +``` + +--- + +## 步骤3：实现技能逻辑 + +创建 `examples/spatial_helper.py`： + +```python +""" +简单的空间分析助手 +演示AI如何理解并执行空间任务 +""" + +import geopandas as gpd +from shapely.geometry import Point +import matplotlib.pyplot as plt + +class SpatialHelper: + """空间分析助手类""" + + def __init__(self, data_path=None): + self.data = None + if data_path: + self.load_data(data_path) + + def load_data(self, path): + """加载空间数据""" + try: + self.data = gpd.read_file(path) + return f"成功加载 {len(self.data)} 个空间要素" + except Exception as e: + return f"加载失败: {str(e)}" + + def buffer_analysis(self, distance, crs=None): + """缓冲区分析""" + if self.data is None: + return "请先加载数据" + + if crs: + # 转换到适合距离计算的坐标系 + self.data = self.data.to_crs(crs) + + buffered = self.data.buffer(distance) + result = gpd.GeoDataFrame(geometry=buffered, crs=self.data.crs) + + return { + 'count': len(result), + 'total_area': result.geometry.area.sum(), + 'geometry': result + } + + def intersect_analysis(self, other_data): + """相交分析""" + if self.data is None: + return "请先加载数据" + + intersection = self.data.intersection(other_data) + valid_results = intersection[~intersection.is_empty] + + return { + 'intersecting_count': len(valid_results), + 'geometries': valid_results + } + + def nearest_neighbor(self, point, n=1): + """最近邻查找""" + if self.data is None: + return "请先加载数据" + + if isinstance(point, (tuple, list)): + point = Point(point) + + distances = self.data.geometry.distance(point) + nearest_indices = distances.nsmallest(n).index + + results = [] + for idx in nearest_indices: + results.append({ + 'index': idx, + 'distance': distances[idx], + 'geometry': self.data.loc[idx].geometry + }) + + return results + + def summarize(self): + """数据摘要""" + if self.data is None: + return "请先加载数据" + + return { + 'count': len(self.data), + 'crs': str(self.data.crs), + 'bounds': self.data.total_bounds, + 'geometry_types': self.data.geometry.type.value_counts().to_dict() + } + + def visualize(self, output_path=None): + """可视化""" + if self.data is None: + return "请先加载数据" + + fig, ax = plt.subplots(figsize=(10, 10)) + self.data.plot(ax=ax, alpha=0.5, edgecolor='k') + ax.set_title('Spatial Data Visualization') + ax.set_axis_off() + + if output_path: + plt.savefig(output_path, bbox_inches='tight', dpi=300) + return f"图表已保存到 {output_path}" + + plt.show() + return "图表已显示" + +# 创建一个示例数据集用于测试 +def create_sample_data(): + """创建示例空间数据""" + import numpy as np + + np.random.seed(42) + + # 创建一些随机点 + points = [Point(np.random.uniform(-100, 100), + np.random.uniform(-100, 100)) + for _ in range(20)] + + # 转换为GeoDataFrame + gdf = gpd.GeoDataFrame({ + 'id': range(20), + 'value': np.random.randint(1, 100, 20), + 'category': np.random.choice(['A', 'B', 'C'], 20) + }, geometry=points, crs='EPSG:4326') + + # 保存为GeoJSON + gdf.to_file('sample_data.geojson', driver='GeoJSON') + return gdf + +if __name__ == "__main__": + # 创建示例数据 + print("创建示例数据...") + create_sample_data() + + # 测试SpatialHelper + print("\n初始化空间助手...") + helper = SpatialHelper('sample_data.geojson') + + print("\n数据摘要:") + print(helper.summarize()) + + print("\n缓冲区分析:") + buffer_result = helper.buffer_analysis(10) + print(f"缓冲区数量: {buffer_result['count']}") + print(f"总缓冲面积: {buffer_result['total_area']:.2f}") + + print("\n最近邻查找:") + nearest = helper.nearest_neighbor((0, 0), n=3) + for i, n in enumerate(nearest, 1): + print(f" {i}. 距离: {n['distance']:.2f}") + + print("\n生成可视化...") + print(helper.visualize('spatial_visualization.png')) +``` + +--- + +## 步骤4：在Claude Code中使用 + +### 4.1 启动Claude Code + +```bash +cd /path/to/CC4SI +claude +``` + +### 4.2 与Claude对话 + +``` +你: 我有一个空间数据文件sample_data.geojson，帮我分析一下 + +Claude: 让我先读取这个文件看看... + +[读取 sample_data.geojson] + +这是一个包含20个点要素的GeoJSON文件，坐标系统为WGS84 (EPSG:4326)。 +每个点都有id、value和category属性。 + +你想要进行什么分析？ + +你: 找出距离原点最近的3个点 + +Claude: 我来帮你找出距离原点最近的点。 + +[调用 spatial-query 技能] + +结果： +1. 最近点 (id=7): 距离约 15.2 单位 +2. 第二近点 (id=12): 距离约 23.8 单位 +3. 第三近点 (id=3): 距离约 31.5 单位 + +需要我生成可视化吗？ +``` + +--- + +## 步骤5：理解工作原理 + +### AI系统的思考过程 + +``` +用户请求 ──→ [意图识别] ──→ [任务分解] + │ │ + ↓ ↓ + 这是空间查询 需要执行： + 任务 1. 加载数据 + 2. 计算距离 + 3. 排序取前N + │ │ + ↓ ↓ + [选择技能] ──→ [调用工具] + │ │ + ↓ ↓ + spatial-query spatial_helper.py + │ │ + └───────┬───────┘ + ↓ + [整合结果] + ↓ + [返回用户] +``` + +### 关键概念验证 + +| 概念 | 实践体现 | +|-----|---------| +| Agent | Claude Code作为智能体，理解意图并协调执行 | +| Skill | spatial-query技能封装了空间分析能力 | +| Tool | spatial_helper.py是具体实现工具 | + +--- + +## 反思问题 + +1. **理解验证**：你能在脑海中复述一遍AI执行这个任务的流程吗？ + +2. **扩展思考**：如果要让这个助手支持更多空间操作，应该如何设计？ + +3. **局限性**：当前实现有哪些不足？如何改进？ + +4. **迁移应用**：这个模式可以应用到你的专业领域吗？ + +--- + +## 下一步 + +完成这个实践后，你已经： +- ✅ 配置了Claude Code环境 +- ✅ 创建了第一个空间分析技能 +- ✅ 理解了Agent的基本工作方式 + +准备好进入下一章：**01-foundations（基础原理）** diff --git a/officefile/supplements/01-foundations/01.1-modular-intelligence.md b/officefile/supplements/01-foundations/01.1-modular-intelligence.md new file mode 100644 index 0000000..ad4f4d9 --- /dev/null +++ b/officefile/supplements/01-foundations/01.1-modular-intelligence.md @@ -0,0 +1,562 @@ +# 01.1 智能的模块化视角 + +## 核心问题 + +> 为什么智能系统需要模块化设计？ +> 技能(Skill)的本质是什么？如何设计可复用的智能组件？ +> 函数式组合思想如何应用于AI系统？ + +--- + +## 概念讲解 + +### 模块化的必要性 + +随着系统复杂度增加，模块化变得必不可少： + +``` +复杂度与模块化的关系 + +低复杂度 ──→ [单体脚本] ──→ 可维护 + ↑ + 简单直接 + +中复杂度 ──→ [函数库] ──→ 需要组织 + ↑ + 按功能分类 + +高复杂度 ──→ [模块化系统] ──→ 必须模块化 + ↑ + 清晰边界，可组合 +``` + +**模块化的收益**： + +| 收益类型 | 说明 | 例子 | +|---------|------|------| +| 可理解性 | 每个模块可独立理解 | 理解缓冲区分析不需要理解投影变换 | +| 可测试性 | 模块可单独测试 | 测试空间索引不需要完整工作流 | +| 可复用性 | 模块可在不同场景使用 | 缓冲区算法用于多个项目 | +| 可替换性 | 模块可用等价实现替换 | QGIS ↔ ArcGIS 同一功能 | +| 可维护性 | 修改局限在模块内 | 修复bug不影响其他模块 | + +### 函数式组合思想 + +函数式编程的核心：**组合小函数构建复杂行为** + +``` +简单函数 ──┬─── buffer(geom, distance) + ├─── intersect(a, b) + ├─── centroid(geom) + └─── distance(a, b) + │ + ↓ 组合 + │ + complex_operation = pipe( + load_data, + clean_geometry, + buffer(100), + intersect(study_area), + calculate_area, + format_output + ) +``` + +**关键特性**： +1. **纯函数**：相同输入→相同输出，无副作用 +2. **高阶函数**：函数可以作为参数和返回值 +3. **不可变数据**：数据不修改，而是创建新版本 + +### 技能即能力封装 + +在Claude Code中，技能(Skill)是智能的模块化单元： + +```yaml +# 技能的结构 +--- +name: skill-name # 技能名称 +description: 技能描述 # 何时使用 +parameters: # 输入参数 + - param1: type + - param2: type +returns: # 输出 + - result: type +--- + +## 技能逻辑 + +具体的执行步骤... +``` + +**技能设计的三个层次**： + +``` +Level 1: 原子技能 + └── 单一功能，不可再分 + 例如：buffer, intersect, dissolve + +Level 2: 组合技能 + └── 由原子技能组合而成 + 例如：site_selection = buffer + intersect + rank + +Level 3: 工作流技能 + └── 完整的决策流程 + 例如：ecological_network_analysis +``` + +--- + +## 设计原理 + +### 模块化的设计原则 + +**1. 单一职责原则 (SRP)** + +每个模块只做一件事，做好一件事： + +```python +# 好的设计：每个函数职责单一 +def calculate_distance(geom1, geom2): + """只计算距离""" + return geom1.distance(geom2) + +def format_distance(distance, unit='m'): + """只格式化输出""" + if distance > 1000: + return f"{distance/1000:.2f} km" + return f"{distance:.0f} m" + +# 使用 +dist = calculate_distance(point_a, point_b) +formatted = format_distance(dist) + +# 不好的设计：混合了计算和格式化 +def calculate_and_format_distance(geom1, geom2): + distance = geom1.distance(geom2) + # 格式化逻辑混在一起 + if distance > 1000: + return f"{distance/1000:.2f} km" + return f"{distance:.0f} m" +``` + +**2. 开闭原则 (OCP)** + +对扩展开放，对修改关闭： + +```python +# 使用抽象基类实现扩展性 +from abc import ABC, abstractmethod + +class SpatialOperation(ABC): + """空间操作的抽象基类""" + + @abstractmethod + def execute(self, data): + pass + +class BufferOperation(SpatialOperation): + """缓冲区操作""" + def __init__(self, distance): + self.distance = distance + + def execute(self, data): + return data.buffer(self.distance) + +class IntersectOperation(SpatialOperation): + """相交操作""" + def __init__(self, other_data): + self.other_data = other_data + + def execute(self, data): + return data.intersection(self.other_data) + +# 可以添加新操作而不修改现有代码 +class UnionOperation(SpatialOperation): + """合并操作""" + def execute(self, data): + return data.union(self.other_data) +``` + +**3. 依赖倒置原则 (DIP)** + +依赖抽象而非具体实现： + +```python +# 好的设计：依赖抽象 +class WorkflowProcessor: + def __init__(self, operation: SpatialOperation): + self.operation = operation # 依赖抽象 + + def process(self, data): + return self.operation.execute(data) + +# 可以轻松替换具体实现 +processor = WorkflowProcessor(BufferOperation(100)) + +# 不好的设计：依赖具体实现 +class WorkflowProcessor: + def __init__(self, buffer_distance): + self.buffer_distance = buffer_distance + + def process(self, data): + # 硬编码了具体操作 + return data.buffer(self.buffer_distance) +``` + +### 技能接口设计 + +良好的技能接口设计： + +```python +from typing import Protocol, TypeVar, Generic + +T = TypeVar('T') + +class SkillInput(Protocol[T]): + """技能输入协议""" + def validate(self) -> bool: + """验证输入有效性""" + ... + +class SkillOutput(Protocol[T]): + """技能输出协议""" + def to_dict(self) -> dict: + """转换为可序列化格式""" + ... + +class Skill(Generic[T]): + """技能基类""" + + name: str + description: str + + def can_handle(self, input_data: T) -> bool: + """判断是否能处理此输入""" + pass + + def execute(self, input_data: T) -> SkillOutput[T]: + """执行技能""" + pass + + def estimate_cost(self, input_data: T) -> float: + """估算执行成本（时间/资源）""" + pass +``` + +--- + +## 代码示例 + +### 模块化的空间分析系统 + +```python +""" +模块化空间分析系统示例 +展示如何用函数式组合构建复杂分析 +""" +from typing import Callable, List, Any, TypeVar +from functools import reduce +import geopandas as gpd + +T = TypeVar('T') + +class SpatialPipeline: + """空间分析流水线""" + + def __init__(self): + self.steps: List[Callable] = [] + + def add_step(self, step: Callable, name: str = None): + """添加处理步骤""" + step.name = name or step.__name__ + self.steps.append(step) + return self + + def execute(self, initial_data): + """执行流水线""" + result = initial_data + + for step in self.steps: + print(f"执行步骤: {getattr(step, 'name', step.__name__)}") + result = step(result) + + return result + +def pipe(*functions): + """函数式组合工具""" + return reduce(lambda f, g: lambda x: g(f(x)), functions) + +# === 原子操作 === + +def load_data(path: str) -> gpd.GeoDataFrame: + """加载数据""" + print(f"加载: {path}") + return gpd.read_file(path) + +def clean_geometry(gdf: gpd.GeoDataFrame) -> gpd.GeoDataFrame: + """清理几何""" + print("清理几何") + # 修复无效几何 + gdf['geometry'] = gdf.geometry.buffer(0) + return gdf[gdf.geometry.is_valid] + +def reproject(gdf: gpd.GeoDataFrame, target_crs: str = 'EPSG:3857') -> gpd.GeoDataFrame: + """重投影""" + print(f"重投影到: {target_crs}") + return gdf.to_crs(target_crs) + +def buffer(gdf: gpd.GeoDataFrame, distance: float) -> gpd.GeoDataFrame: + """缓冲区分析""" + print(f"缓冲距离: {distance}") + return gdf.buffer(distance) + +def intersect(gdf: gpd.GeoDataFrame, other: gpd.GeoDataFrame) -> gpd.GeoDataFrame: + """相交分析""" + print("相交分析") + return gdf.overlay(other, how='intersection') + +def calculate_area(gdf: gpd.GeoDataFrame) -> float: + """计算面积""" + area = gdf.geometry.area.sum() + print(f"总面积: {area:.2f} 平方米") + return area + +# === 高阶操作 === + +def make_buffer(distance: float) -> Callable: + """缓冲操作工厂函数""" + return lambda gdf: buffer(gdf, distance) + +def make_reproject(crs: str) -> Callable: + """重投影工厂函数""" + return lambda gdf: reproject(gdf, crs) + +def make_intersect(other_data: gpd.GeoDataFrame) -> Callable: + """相交工厂函数""" + return lambda gdf: intersect(gdf, other_data) + +# === 使用示例 === + +def example_pipeline_usage(): + """流水线使用示例""" + + # 方式1：使用Pipeline类 + pipeline = SpatialPipeline() + pipeline.add_step(load_data, "加载数据") + pipeline.add_step(clean_geometry, "清理几何") + pipeline.add_step(lambda gdf: reproject(gdf, 'EPSG:3857'), "重投影") + pipeline.add_step(lambda gdf: buffer(gdf, 100), "缓冲") + pipeline.add_step(calculate_area, "计算面积") + + # result = pipeline.execute("data.geojson") + + # 方式2：使用函数式组合 + analysis_pipeline = pipe( + load_data, + clean_geometry, + lambda gdf: reproject(gdf, 'EPSG:3857'), + lambda gdf: buffer(gdf, 100), + calculate_area + ) + + # result = analysis_pipeline("data.geojson") + + return pipeline + +# === 技能封装 === + +class BufferSkill: + """缓冲区技能""" + + name = "buffer_analysis" + description = "执行缓冲区分析" + + def __init__(self, distance: float, unit: str = 'm'): + self.distance = distance + self.unit = unit + + def execute(self, data: gpd.GeoDataFrame) -> gpd.GeoDataFrame: + """执行技能""" + # 确保在合适的坐标系中 + if data.crs and data.crs.is_geographic: + data = data.to_crs('EPSG:3857') + + result = data.buffer(self.distance) + + return gpd.GeoDataFrame( + geometry=result, + crs=data.crs + ) + + def __repr__(self): + return f"BufferSkill(distance={self.distance}{self.unit})" + +class SiteSelectionSkill: + """选址技能：组合多个原子操作""" + + name = "site_selection" + description = "基于多准则的选址分析" + + def __init__(self, + distance_from_road: float, + distance_from_water: float, + min_area: float): + self.road_distance = distance_from_road + self.water_distance = distance_from_water + self.min_area = min_area + + def execute(self, + sites: gpd.GeoDataFrame, + roads: gpd.GeoDataFrame, + water: gpd.GeoDataFrame) -> gpd.GeoDataFrame: + """ + 执行选址分析 + + 组合操作： + 1. 找到距离道路指定范围内的区域 + 2. 排除距离水体太近的区域 + 3. 筛选面积满足要求的区域 + """ + + # 1. 道路缓冲 + road_buffer = roads.buffer(self.road_distance) + + # 2. 水体缓冲（排除区） + water_buffer = water.buffer(self.water_distance) + + # 3. 找到满足条件的site + suitable = sites[ + sites.geometry.intersects(road_buffer.union_all()) & + ~sites.geometry.intersects(water_buffer.union_all()) + ] + + # 4. 面积筛选 + suitable = suitable[suitable.geometry.area >= self.min_area] + + return suitable + +if __name__ == "__main__": + # 示例：构建一个选址分析流水线 + print("=== 模块化空间分析系统 ===\n") + + # 创建技能 + buffer_skill = BufferSkill(distance=500, unit='m') + print(f"创建技能: {buffer_skill}") + + # 技能可以独立测试 + print("\n技能的核心优势:") + print("1. 可理解性 - 每个技能职责单一") + print("2. 可测试性 - 独立测试每个技能") + print("3. 可复用性 - 在不同场景中使用") + print("4. 可组合性 - 小技能组合成大技能") +``` + +--- + +## 案例分析 + +### QGIS插件架构分析 + +QGIS的模块化设计是学习的好例子： + +``` +QGIS架构 + │ + ├── Core (核心库) + │ ├── QgsGeometry - 几何操作 + │ ├── QgsVectorLayer - 矢量图层 + │ ├── QgsRasterLayer - 栅格图层 + │ └── QgsProcessing - 处理框架 + │ + ├── Providers (数据提供者) + │ ├── OGR Provider - 矢量数据 + │ ├── GDAL Provider - 栅格数据 + │ └── PostGIS Provider - 数据库 + │ + ├── Plugins (插件) + │ ├── 每个插件独立模块 + │ ├── 通过接口访问核心功能 + │ └── 可单独安装/卸载 + │ + └── Processing Algorithms (处理算法) + ├── 算法库（600+算法） + ├── 可组合使用 + └── 模型构建器 +``` + +**关键设计模式**： + +1. **Provider模式**：数据访问抽象 +2. **Plugin模式**：功能扩展 +3. **Algorithm模式**：处理步骤封装 + +**AI系统的启发**： + +```python +# 类似QGIS的AI技能架构 +class AISkillRegistry: + """AI技能注册表""" + + def __init__(self): + self.skills = {} + + def register(self, skill): + """注册技能""" + self.skills[skill.name] = skill + + def get(self, name: str): + """获取技能""" + return self.skills.get(name) + + def list_by_category(self, category: str): + """按类别列出技能""" + return [s for s in self.skills.values() + if s.category == category] + +# 使用 +registry = AISkillRegistry() +registry.register(BufferSkill(distance=100)) +registry.register(SiteSelectionSkill(...)) + +# 查找和使用技能 +buffer = registry.get("buffer_analysis") +result = buffer.execute(data) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **边界划分**：如何确定一个模块的边界？太小的模块和太大的模块各有什么问题？ + +2. **接口设计**：设计一个技能接口时，应该考虑哪些因素？ + +3. **复用性**：什么代码值得复用？什么不值得？ + +4. **组合爆炸**：当模块数量很大时，如何管理模块之间的依赖？ + +### 实践练习 + +1. **重构练习**：找一个你写的复杂函数，将其拆分为多个小函数 + +2. **接口设计**：为你熟悉的空间分析操作设计技能接口 + +3. **组合挑战**：用5个以下的基本操作组合出10个不同的分析流程 + +### 延伸阅读 + +- **"Refactoring"** (Martin Fowler) - 重构与模块化 +- **"The Art of Unix Programming"** - 模块化哲学 +- QGIS Plugin开发指南 + +--- + +## 关键要点 + +1. **模块化是管理复杂性的核心方法** +2. **函数式组合让小函数构建大功能** +3. **技能是智能的封装单元** +4. **良好的接口设计是模块化的关键** +5. **QGIS的架构是学习的优秀范例** diff --git a/officefile/supplements/01-foundations/01.2-state-and-state-machines.md b/officefile/supplements/01-foundations/01.2-state-and-state-machines.md new file mode 100644 index 0000000..d9e53f8 --- /dev/null +++ b/officefile/supplements/01-foundations/01.2-state-and-state-machines.md @@ -0,0 +1,719 @@ +# 01.2 状态与状态机 + +## 核心问题 + +> 在AI系统中，"状态"到底是什么？ +> 为什么状态管理是Agent系统的核心？ +> LangGraph是如何用状态机设计工作流的？ + +--- + +## 概念讲解 + +### 什么是状态 + +**状态**是系统在某一时刻的快照，包含所有影响未来行为的信息： + +``` +系统状态 = 所有相关的变量值 + +例如：生态分析工作流的状态 +{ + "input_data": {...}, # 输入数据 + "current_step": "buffer", # 当前步骤 + "intermediate_results": {...}, # 中间结果 + "user_preferences": {...}, # 用户偏好 + "error_count": 0, # 错误计数 + "checkpoint_reached": False # 检查点状态 +} +``` + +**状态的类型**： + +| 类型 | 说明 | 例子 | +|-----|------|------| +| 静态状态 | 初始输入，不变化 | 输入文件路径、参数 | +| 动态状态 | 运行中变化 | 当前步骤、累积结果 | +| 控制状态 | 影响流程走向 | 分支条件、错误标志 | +| 会话状态 | 跨请求持久化 | 用户偏好、历史记录 | + +### 为什么状态管理很重要 + +**1. 断点续传** + +```python +# 没有状态管理：出错后必须从头开始 +def analysis_without_state(): + step1() + step2() # 如果这里出错 + step3() # 这些都要重做 + +# 有状态管理：可以从断点继续 +class AnalysisWithState: + def __init__(self): + self.state = {"current_step": 0} + + def run(self): + if self.state["current_step"] < 1: + step1() + self.state["current_step"] = 1 + + if self.state["current_step"] < 2: + try: + step2() + self.state["current_step"] = 2 + except Exception: + # 保存状态，下次可以从这里继续 + save_state(self.state) + raise + + if self.state["current_step"] < 3: + step3() +``` + +**2. 人机协同** + +```python +# HITL需要状态来知道在哪里需要人类介入 +class HITLWorkflow: + def __init__(self): + self.state = { + "step": "identify_sources", + "pending_review": True, + "sources": None, + "human_feedback": None + } + + def next_action(self): + if self.state["pending_review"]: + return "request_human_review" + elif self.state["human_feedback"]: + return "incorporate_feedback" + else: + return "proceed_to_next_step" +``` + +**3. 调试和可解释性** + +```python +# 状态历史记录了整个决策过程 +class StatefulAgent: + def __init__(self): + self.state_history = [] + + def decide(self, context): + # 记录状态 + self.state_history.append({ + "timestamp": now(), + "state": self.state.copy(), + "context": context, + "decision": None + }) + + # 做决策 + decision = self._make_decision(context) + self.state_history[-1]["decision"] = decision + + return decision + + def explain(self): + """回溯决策过程""" + return self.state_history +``` + +### 状态机 + +**状态机**是描述系统状态转换的模型： + +``` + ┌─────────┐ + │ 初始 │ + │ state │ + └────┬────┘ + │ event: start + ↓ + ┌─────────┐ + │ 加载数据 │ + └────┬────┘ + │ success + ↓ + ┌─────────┐ error ┌─────────┐ + │ 分析处理 │ ─────────────→│ 错误 │ + └────┬────┘ └─────────┘ + │ success │ retry + ↓ │ + ┌─────────┐ │ + │ 人类 │ │ + │ 审查 │ │ + └────┬────┘ │ + │ approve │ + ↓ │ + ┌─────────┐ │ + │ 完成 │←───────────────────────┘ + └─────────┘ +``` + +**状态机的要素**： +1. **状态 (State)**：系统可能处于的情况 +2. **事件 (Event)**：触发状态转换的条件 +3. **转换 (Transition)**：从一个状态到另一个状态 +4. **动作 (Action)**：状态转换时执行的操作 + +--- + +## 设计原理 + +### LangGraph的状态设计哲学 + +LangGraph是构建有状态Agent的框架，其核心思想： + +```python +from typing import TypedDict + +# 定义状态类型 +class AnalysisState(TypedDict): + """生态网络分析状态""" + + # 输入数据 + input_path: str + parameters: dict + + # 处理过程 + current_step: str + intermediate_results: dict + + # 人机交互 + review_requested: bool + human_feedback: str + + # 输出 + final_result: dict + errors: list + +# 状态图定义 +workflow = StateGraph(AnalysisState) + +# 添加节点（处理步骤） +workflow.add_node("load_data", load_data_node) +workflow.add_node("identify_sources", identify_sources_node) +workflow.add_node("human_review", human_review_node) +workflow.add_node("extract_corridors", extract_corridors_node) + +# 添加边（状态转换） +workflow.add_edge("load_data", "identify_sources") +workflow.add_conditional_edge( + "identify_sources", + should_review, # 条件函数 + { + "review": "human_review", + "continue": "extract_corridors" + } +) + +# 编译为可执行图 +app = workflow.compile() +``` + +**核心概念**： + +1. **状态即消息**：状态在节点间传递 +2. **图即流程**：有向图描述工作流 +3. **条件分支**：基于状态的动态路由 + +### 工作流状态机实现 + +```python +from enum import Enum +from typing import Dict, Any, Callable, Optional +from dataclasses import dataclass, field + +class WorkflowState(Enum): + """工作流状态枚举""" + IDLE = "idle" + LOADING = "loading" + PROCESSING = "processing" + REVIEWING = "reviewing" + COMPLETED = "completed" + ERROR = "error" + +@dataclass +class WorkflowContext: + """工作流上下文（状态数据）""" + data: Dict[str, Any] = field(default_factory=dict) + current_step: int = 0 + errors: list = field(default_factory=list) + metadata: Dict[str, Any] = field(default_factory=dict) + +class StateMachine: + """通用状态机""" + + def __init__(self, initial_state: WorkflowState): + self.state = initial_state + self.context = WorkflowContext() + self.transitions: Dict[WorkflowState, Dict[str, WorkflowState]] = {} + self.actions: Dict[tuple[WorkflowState, WorkflowState], Callable] = {} + + def add_transition(self, + from_state: WorkflowState, + event: str, + to_state: WorkflowState, + action: Callable = None): + """添加状态转换""" + if from_state not in self.transitions: + self.transitions[from_state] = {} + self.transitions[from_state][event] = to_state + + if action: + self.actions[(from_state, to_state)] = action + + def trigger(self, event: str, **kwargs) -> bool: + """触发事件""" + if self.state not in self.transitions: + raise ValueError(f"没有从状态 {self.state} 的转换") + + if event not in self.transitions[self.state]: + print(f"事件 {event} 在状态 {self.state} 下无效") + return False + + # 获取目标状态 + new_state = self.transitions[self.state][event] + old_state = self.state + + # 执行转换动作 + action = self.actions.get((old_state, new_state)) + if action: + result = action(self.context, **kwargs) + if result is False: # 动作失败，不转换 + return False + + # 更新状态 + self.state = new_state + print(f"状态转换: {old_state} → {new_state}") + return True + +# 生态分析工作流状态机 +class EcologicalAnalysisWorkflow: + """生态网络分析工作流""" + + def __init__(self): + # 创建状态机 + self.sm = StateMachine(WorkflowState.IDLE) + + # 定义转换 + self.sm.add_transition(WorkflowState.IDLE, "start", WorkflowState.LOADING) + self.sm.add_transition(WorkflowState.LOADING, "loaded", WorkflowState.PROCESSING) + self.sm.add_transition(WorkflowState.LOADING, "error", WorkflowState.ERROR) + self.sm.add_transition(WorkflowState.PROCESSING, "complete", WorkflowState.REVIEWING) + self.sm.add_transition(WorkflowState.PROCESSING, "error", WorkflowState.ERROR) + self.sm.add_transition(WorkflowState.REVIEWING, "approved", WorkflowState.COMPLETED) + self.sm.add_transition(WorkflowState.REVIEWING, "rejected", WorkflowState.PROCESSING) + self.sm.add_transition(WorkflowState.ERROR, "retry", WorkflowState.LOADING) + + def run(self, data_path: str): + """执行工作流""" + + # 启动 + self.sm.trigger("start", data_path=data_path) + + # 模拟加载 + print("加载数据...") + self.sm.trigger("loaded") + + # 模拟处理 + print("处理数据...") + self.sm.trigger("complete") + + # 审查 + print("等待审查...") + # 这里会等待人类输入 + # 假设批准 + self.sm.trigger("approved") + + print(f"工作流完成，最终状态: {self.sm.state}") +``` + +--- + +## 代码示例 + +### 完整的状态机工作流 + +```python +""" +完整的状态机工作流示例 +""" +import json +from typing import Dict, Any, List, Optional +from dataclasses import dataclass, field, asdict +from enum import Enum +import time + +class State(Enum): + """状态枚举""" + IDLE = "idle" + LOAD_DATA = "load_data" + IDENTIFY_SOURCES = "identify_sources" + BUILD_RESISTANCE = "build_resistance" + REVIEW_SOURCES = "review_sources" + REVIEW_RESISTANCE = "review_resistance" + EXTRACT_CORRIDORS = "extract_corridors" + COMPLETED = "completed" + ERROR = "error" + +@dataclass +class WorkflowState: + """工作流状态数据""" + current: State = State.IDLE + step_number: int = 0 + data_path: Optional[str] = None + sources: Optional[List[Dict]] = None + resistance_weights: Optional[Dict] = None + corridors: Optional[List[Dict]] = None + errors: List[str] = field(default_factory=list) + history: List[Dict] = field(default_factory=list) + + def transition_to(self, new_state: State, action: str = ""): + """状态转换""" + old_state = self.current + self.current = new_state + self.step_number += 1 + + # 记录历史 + self.history.append({ + "step": self.step_number, + "from": old_state.value, + "to": new_state.value, + "action": action, + "timestamp": time.time() + }) + + def to_dict(self) -> Dict: + """序列化""" + return { + "current": self.current.value, + "step_number": self.step_number, + "data_path": self.data_path, + "sources": self.sources, + "resistance_weights": self.resistance_weights, + "corridors": self.corridors, + "errors": self.errors, + "history": self.history + } + + def save(self, path: str): + """保存状态到文件""" + with open(path, 'w') as f: + json.dump(self.to_dict(), f, indent=2) + + @classmethod + def load(cls, path: str) -> 'WorkflowState': + """从文件加载状态""" + with open(path, 'r') as f: + data = json.load(f) + + # 转换State枚举 + data["current"] = State(data["current"]) + + return cls(**{k: v for k, v in data.items() if k != "history"}) + +class EcologicalAnalysisAgent: + """生态分析智能体（有状态）""" + + def __init__(self): + self.state = WorkflowState() + self.review_callbacks = { + State.REVIEW_SOURCES: self._review_sources, + State.REVIEW_RESISTANCE: self._review_resistance + } + + def start(self, data_path: str): + """启动分析""" + self.state.data_path = data_path + self.state.transition_to(State.LOAD_DATA, "开始加载数据") + self._execute_current_step() + + def _execute_current_step(self): + """执行当前状态对应的操作""" + handlers = { + State.LOAD_DATA: self._handle_load_data, + State.IDENTIFY_SOURCES: self._handle_identify_sources, + State.BUILD_RESISTANCE: self._handle_build_resistance, + State.REVIEW_SOURCES: self._handle_review, + State.REVIEW_RESISTANCE: self._handle_review, + State.EXTRACT_CORRIDORS: self._handle_extract_corridors, + State.COMPLETED: self._handle_completed, + State.ERROR: self._handle_error + } + + handler = handlers.get(self.state.current) + if handler: + handler() + + def _handle_load_data(self): + """处理数据加载""" + print(f"\n[状态: {self.state.current.value}] 加载数据: {self.state.data_path}") + + # 模拟加载 + try: + # 这里实际会读取文件 + time.sleep(0.5) + print("数据加载成功") + self.state.transition_to(State.IDENTIFY_SOURCES, "数据加载完成") + self._execute_current_step() + except Exception as e: + self.state.errors.append(str(e)) + self.state.transition_to(State.ERROR, f"加载失败: {e}") + self._execute_current_step() + + def _handle_identify_sources(self): + """处理源地识别""" + print(f"\n[状态: {self.state.current.value}] 识别生态源地...") + + # 模拟识别 + self.state.sources = [ + {"id": 1, "area": 1500, "type": "forest"}, + {"id": 2, "area": 800, "type": "wetland"} + ] + print(f"识别到 {len(self.state.sources)} 个源地") + + self.state.transition_to(State.REVIEW_SOURCES, "源地识别完成，等待审查") + self._execute_current_step() + + def _handle_build_resistance(self): + """处理阻力面构建""" + print(f"\n[状态: {self.state.current.value}] 构建阻力面...") + + # 模拟构建 + self.state.resistance_weights = { + "forest": 1, + "grassland": 10, + "urban": 100, + "water": 50 + } + print("阻力面构建完成") + + self.state.transition_to(State.REVIEW_RESISTANCE, "阻力面构建完成，等待审查") + self._execute_current_step() + + def _handle_review(self): + """处理审查状态""" + print(f"\n[状态: {self.state.current.value}] 等待人类审查...") + + callback = self.review_callbacks.get(self.state.current) + if callback: + result = callback() + + if result == "approve": + if self.state.current == State.REVIEW_SOURCES: + self.state.transition_to(State.BUILD_RESISTANCE, "审查通过") + elif self.state.current == State.REVIEW_RESISTANCE: + self.state.transition_to(State.EXTRACT_CORRIDORS, "审查通过") + self._execute_current_step() + else: + # 拒绝，返回上一状态 + print("审查未通过，重新执行...") + # 简化处理：直接继续 + + def _review_sources(self) -> str: + """审查源地""" + print("\n=== 源地审查 ===") + print(f"识别到 {len(self.state.sources)} 个源地:") + for s in self.state.sources: + print(f" - ID {s['id']}: {s['type']}, 面积 {s['area']}") + + # 实际实现中这里会等待人类输入 + # 这里模拟自动批准 + print("\n[模拟] 审查: 批准") + return "approve" + + def _review_resistance(self) -> str: + """审查阻力面""" + print("\n=== 阻力面审查 ===") + print("阻力权重:") + for land_type, weight in self.state.resistance_weights.items(): + print(f" - {land_type}: {weight}") + + print("\n[模拟] 审查: 批准") + return "approve" + + def _handle_extract_corridors(self): + """处理廊道提取""" + print(f"\n[状态: {self.state.current.value}] 提取生态廊道...") + + # 模拟提取 + self.state.corridors = [ + {"from": 1, "to": 2, "length": 3500} + ] + print(f"提取到 {len(self.state.corridors)} 条廊道") + + self.state.transition_to(State.COMPLETED, "分析完成") + self._execute_current_step() + + def _handle_completed(self): + """处理完成状态""" + print(f"\n[状态: {self.state.current.value}] 工作流完成!") + print(f"\n=== 结果摘要 ===") + print(f"源地数量: {len(self.state.sources) if self.state.sources else 0}") + print(f"廊道数量: {len(self.state.corridors) if self.state.corridors else 0}") + print(f"执行步骤: {self.state.step_number}") + + def _handle_error(self): + """处理错误状态""" + print(f"\n[状态: {self.state.current.value}] 发生错误") + for error in self.state.errors: + print(f" - {error}") + + def save_state(self, path: str): + """保存当前状态""" + self.state.save(path) + print(f"状态已保存到: {path}") + + def resume_from(self, path: str): + """从保存的状态恢复""" + self.state = WorkflowState.load(path) + print(f"从状态恢复: {self.state.current.value}") + print(f"历史步骤: {self.state.step_number}") + self._execute_current_step() + +# 使用示例 +if __name__ == "__main__": + print("=== 生态分析状态机工作流 ===\n") + + agent = EcologicalAnalysisAgent() + + # 执行工作流 + agent.start("data.geojson") + + # 可以保存状态 + # agent.save_state("workflow_state.json") + + # 可以从状态恢复 + # new_agent = EcologicalAnalysisAgent() + # new_agent.resume_from("workflow_state.json") +``` + +--- + +## 案例分析 + +### LangGraph在空间分析中的应用 + +```python +""" +LangGraph风格的生态网络分析工作流 +""" +from typing import TypedDict, Annotated, Literal +from operator import add + +class EcologicalState(TypedDict): + """生态分析状态类型""" + messages: Annotated[list, add] # 消息历史 + input_data: dict + sources: list + resistance: dict + corridors: list + next_step: str + human_feedback: str + +# 节点函数 +def load_data_node(state: EcologicalState) -> EcologicalState: + """加载数据节点""" + print("执行: load_data") + state["sources"] = [{"id": 1, "area": 1000}] + state["next_step"] = "identify" + return state + +def identify_sources_node(state: EcologicalState) -> EcologicalState: + """识别源地节点""" + print("执行: identify_sources") + state["sources"] = [{"id": i, "area": i * 100} for i in range(1, 6)] + state["next_step"] = "review" + return state + +def human_review_node(state: EcologicalState) -> EcologicalState: + """人类审查节点""" + print("执行: human_review") + print(f"待审查: {state['sources']}") + + # 在实际实现中，这里会等待人类输入 + state["human_feedback"] = "approved" + state["next_step"] = "build_resistance" + return state + +def build_resistance_node(state: EcologicalState) -> EcologicalState: + """构建阻力面节点""" + print("执行: build_resistance") + state["resistance"] = {"forest": 1, "urban": 100} + state["next_step"] = "complete" + return state + +# 路由函数 +def should_review(state: EcologicalState) -> Literal["review", "skip"]: + """决定是否需要审查""" + if len(state.get("sources", [])) > 3: + return "review" + return "skip" + +# 条件边 +def route_after_identify(state: EcologicalState) -> str: + """识别源地后的路由""" + if state.get("human_feedback") == "approved": + return "build_resistance" + return "identify" # 重新识别 + +print(""" +┌──────────────┐ +│ load_data │ +└──────┬───────┘ + │ + ↓ +┌──────────────┐ +│identify_sources│ +└──────┬───────┘ + │ + ├────→ [review?] ──→ human_review ──┐ + │ No │ + ↓ ↓ +┌──────────────┐ ┌──────────────┐ +│build_resistance│◀──────────────────│ approved │ +└──────────────┘ └──────────────┘ +""") +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **状态粒度**：状态应该有多细？太细会怎样，太粗会怎样？ + +2. **持久化策略**：哪些状态需要持久化？什么时候保存状态？ + +3. **并发处理**：如果多个Agent协同工作，如何管理共享状态？ + +4. **调试**：当状态机出错时，如何调试？ + +### 实践练习 + +1. **状态审计**：添加状态转换日志，分析工作流执行路径 + +2. **状态压缩**：实现状态序列化/反序列化，支持断点续传 + +3. **条件路由**：实现一个带多个分支的状态机 + +### 延伸阅读 + +- **"Designing Data-Intensive Applications"** (Kleppmann) - 状态管理理论 +- **LangGraph文档** - 实际框架使用 +- **"State Machine Design Patterns"** - 状态机设计模式 + +--- + +## 关键要点 + +1. **状态是系统在某一时刻的完整快照** +2. **状态机描述系统如何随事件转换状态** +3. **LangGraph用图结构表达有状态的工作流** +4. **良好的状态管理支持断点续传和HITL** +5. **状态历史是调试和可解释性的关键** diff --git a/officefile/supplements/01-foundations/01.3-probability-and-uncertainty.md b/officefile/supplements/01-foundations/01.3-probability-and-uncertainty.md new file mode 100644 index 0000000..ec044c5 --- /dev/null +++ b/officefile/supplements/01-foundations/01.3-probability-and-uncertainty.md @@ -0,0 +1,677 @@ +# 01.3 概率与不确定性 + +## 核心问题 + +> 空间分析中的不确定性从何而来？ +> AI系统如何表示和处理不确定性？ +> 如何在不确定性下做出稳健的决策？ + +--- + +## 概念讲解 + +### 不确定性的来源 + +在空间分析和AI系统中，不确定性无处不在： + +``` +空间分析中的不确定性来源 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 数据不确定性 │ +│ - 测量误差 │ +│ - 空间采样不完整 │ +│ - 分类错误 │ +│ - 时间延迟 │ +│ │ +│ 2. 参数不确定性 │ +│ - 阻力权重不确定 │ +│ - 阈值选择主观 │ +│ - 模型参数拟合误差 │ +│ │ +│ 3. 结构不确定性 │ +│ - 模型选择 │ +│ - 变量关系假设 │ +│ - 尺度效应 │ +│ │ +│ 4. 语义不确定性 │ +│ - 概念模糊（"生态质量"是什么？） │ +│ - 分类边界不清 │ +│ - 专家意见分歧 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 不确定性的类型 + +| 类型 | 说明 | 例子 | +|-----|------|------| +| **偶然不确定性** (Aleatoric) | 系统固有的随机性，无法通过更多数据消除 | 降雨量的随机波动 | +| **认知不确定性** (Epistemic) | 知识不足导致的不确定性，可通过更多数据减少 | 未调查区域的物种分布 | +| **模糊性** (Ambiguity) | 概念或分类的不明确 | "高生态价值"的定义 | +| **冲突** (Conflict) | 不同信息源的不一致 | 两个专家给出相反意见 | + +### AI如何处理不确定性 + +**传统GIS vs 概率AI**： + +``` +传统GIS: 确定性输出 + 输入 → [处理] → 单一结果 + 例如：这个区域是/不是生态源地 + +概率AI: 概率输出 + 输入 → [处理] → (结果, 置信度) + 例如：这个区域是生态源地的概率是 0.78 ± 0.12 +``` + +**置信度的表示**： + +```python +# 方式1: 点估计 + 置信区间 +estimate = 0.75 +confidence_interval = (0.65, 0.85) + +# 方式2: 概率分布 +from scipy.stats import beta +distribution = beta(a=8, b=3) # 基于共8次成功，3次失败 + +# 方式3: 分类概率 +class_probabilities = { + "high_suitability": 0.65, + "medium_suitability": 0.25, + "low_suitability": 0.10 +} + +# 方式4: 模糊隶属度 +fuzzy_membership = { + "is_source": 0.72, + "is_not_source": 0.28 +} +``` + +--- + +## 设计原理 + +### 不确定性传播 + +当多个步骤串联时，不确定性会累积： + +```python +""" +不确定性传播示例 +""" +import numpy as np +from scipy.stats import norm + +class UncertainValue: + """带不确定性的值""" + + def __init__(self, mean, std): + self.mean = mean + self.std = std + + def __add__(self, other): + """加法：方差相加""" + return UncertainValue( + self.mean + other.mean, + np.sqrt(self.std**2 + other.std**2) + ) + + def __mul__(self, scalar): + """乘以标量：标准差也乘""" + return UncertainValue( + self.mean * scalar, + self.std * abs(scalar) + ) + + def __repr__(self): + return f"{self.mean:.2f} ± {self.std:.2f}" + +# 示例：源地适宜性评估中的不确定性传播 +def assess_suitability_with_uncertainty(): + """ + 每个指标都有测量不确定性， + 最终的适宜性评分会累积这些不确定性 + """ + + # 各项指标（均值 ± 标准差） + vegetation_quality = UncertainValue(0.75, 0.10) + connectivity = UncertainValue(0.60, 0.15) + distance_to_threat = UncertainValue(0.80, 0.08) + + # 加权组合（权重也有不确定性） + weights = { + "vegetation": 0.4, + "connectivity": 0.3, + "distance": 0.3 + } + + # 计算总分（简化传播） + total = (vegetation_quality * weights["vegetation"] + + connectivity * weights["connectivity"] + + distance_to_threat * weights["distance"]) + + print("各指标不确定性:") + print(f" 植被质量: {vegetation_quality}") + print(f" 连通性: {connectivity}") + print(f" 威胁距离: {distance_to_threat}") + print(f"\n总分: {total}") + print(f" 置信区间95%: [{total.mean - 1.96*total.std:.2f}, " + f"{total.mean + 1.96*total.std:.2f}]") + + return total + +if __name__ == "__main__": + assess_suitability_with_uncertainty() +``` + +### 敏感性分析 + +了解哪些参数对结果影响最大： + +```python +""" +敏感性分析：识别关键参数 +""" +import numpy as np +from typing import Dict, List, Tuple + +def sensitivity_analysis(model_fn, param_ranges: Dict[str, Tuple[float, float]], + n_samples=1000) -> Dict[str, float]: + """ + 使用蒙特卡洛方法进行敏感性分析 + + Args: + model_fn: 模型函数，接受参数字典，返回结果 + param_ranges: 参数范围 {param_name: (min, max)} + n_samples: 采样次数 + + Returns: + 各参数的敏感性系数 + """ + results = {param: [] for param in param_ranges} + model_outputs = [] + + # 蒙特卡洛采样 + for _ in range(n_samples): + # 随机采样参数 + sample = {k: np.random.uniform(v[0], v[1]) + for k, v in param_ranges.items()} + + # 记录参数值 + for param, value in sample.items(): + results[param].append(value) + + # 计算模型输出 + output = model_fn(sample) + model_outputs.append(output) + + # 计算相关性作为敏感性指标 + sensitivities = {} + for param in param_ranges: + correlation = np.corrcoef(results[param], model_outputs)[0, 1] + sensitivities[param] = abs(correlation) + + return sensitivities + +# 示例：生态阻力面构建的敏感性分析 +def resistance_model(params): + """简化的阻力面模型""" + # 参数：各土地类型的阻力权重 + forest_weight = params["forest"] + grass_weight = params["grass"] + urban_weight = params["urban"] + + # 简化：计算平均阻力 + # 实际应用中会是空间计算 + landscape_composition = { + "forest": 0.4, + "grass": 0.3, + "urban": 0.3 + } + + total_resistance = ( + forest_weight * landscape_composition["forest"] + + grass_weight * landscape_composition["grass"] + + urban_weight * landscape_composition["urban"] + ) + + return total_resistance + +def run_sensitivity_example(): + """运行敏感性分析示例""" + print("=== 阻力面参数敏感性分析 ===\n") + + # 定义参数范围 + param_ranges = { + "forest": (1, 10), + "grass": (10, 50), + "urban": (50, 200) + } + + # 运行敏感性分析 + sensitivities = sensitivity_analysis( + resistance_model, + param_ranges, + n_samples=5000 + ) + + # 排序并输出 + sorted_sens = sorted(sensitivities.items(), + key=lambda x: x[1], reverse=True) + + print("参数敏感性排序:") + for param, sensitivity in sorted_sens: + bar = "█" * int(sensitivity * 30) + print(f" {param}: {sensitivity:.3f} {bar}") + + print("\n解释:") + print(f" 最敏感的参数是 {sorted_sens[0][0]}") + print(f" 应当优先精确确定该参数的值") + +if __name__ == "__main__": + run_sensitivity_example() +``` + +### 鲁棒决策 + +当不确定性无法消除时，做鲁棒的决策： + +```python +""" +鲁棒决策：在不确定性下的稳健决策 +""" +from typing import List, Callable +import numpy as np + +def robust_decision_scenarios(): + """ + 鲁棒决策的几种策略 + """ + + # 策略1: 最大最小 (Maximin) - 最坏情况最优 + def maximin(payoff_matrix): + """ + 选择在最坏情况下表现最好的选项 + + payoff_matrix: 选项 × 场景 的收益矩阵 + """ + worst_case_outcomes = payoff_matrix.min(axis=1) + best_option = worst_case_outcomes.argmax() + return best_option, worst_case_outcomes + + # 策略2: 最大平均 (Maximum Expected Value) + def max_expected(payoff_matrix, probabilities=None): + """选择期望收益最大的选项""" + if probabilities is None: + probabilities = np.ones(payoff_matrix.shape[1]) / payoff_matrix.shape[1] + + expected_values = payoff_matrix @ probabilities + best_option = expected_values.argmax() + return best_option, expected_values + + # 策略3: 最小后悔 (Minimax Regret) + def minimax_regret(payoff_matrix): + """选择最小化最大后悔的选项""" + # 每个场景的最佳收益 + best_per_scenario = payoff_matrix.max(axis=0) + + # 后悔矩阵：每个选项在每个场景与最佳的差距 + regret_matrix = best_per_scenario - payoff_matrix + + # 每个选项的最大后悔 + max_regret = regret_matrix.max(axis=1) + + # 选择最大后悔最小的选项 + best_option = max_regret.argmin() + return best_option, max_regret + + # 示例：生态廊道选址决策 + # 选项：3个候选廊道路线 + # 场景：不同的未来土地变化情景 + payoff_matrix = np.array([ + # 情景1 情景2 情景3 情景4 + [80, 60, 40, 70], # 选项A：穿过森林 + [50, 90, 70, 50], # 选项B：沿河流 + [60, 70, 90, 60], # 选项C：绕行城市边缘 + ]) + + print("=== 生态廊道选址：鲁棒决策分析 ===\n") + print("收益矩阵（廊道质量评分）:") + print(" 情景1 情景2 情景3 情景4") + for i, row in enumerate(payoff_matrix, ord('A')): + print(f"选项{i}: {row}") + + print("\n--- 策略1: 最大最小 (最坏情况最优) ---") + option, worst = maximin(payoff_matrix) + print(f"推荐: 选项{chr(ord('A') + option)}") + print(f"各选项最坏情况: {worst}") + print(" 原理: 选择在最坏情景下表现最好的") + + print("\n--- 策略2: 最大期望 (平均收益最大) ---") + option, expected = max_expected(payoff_matrix) + print(f"推荐: 选项{chr(ord('A') + option)}") + print(f"各选项期望收益: {expected}") + print(" 原理: 选择平均表现最好的") + + print("\n--- 策略3: 最小最大后悔 ---") + option, regret = minimax_regret(payoff_matrix) + print(f"推荐: 选项{chr(ord('A') + option)}") + print(f"各选项最大后悔: {regret}") + print(" 原理: 选择让'选错'的后悔最小的") + + return payoff_matrix + +if __name__ == "__main__": + robust_decision_scenarios() +``` + +--- + +## 代码示例 + +### 概率源地识别 + +```python +""" +带不确定性的生态源地识别 +""" +import numpy as np +from scipy.stats import beta +from typing import Dict, List, Tuple + +class ProbabilisticSource: + """概率源地：带置信度的源地""" + + def __init__(self, source_id: int, geometry, + probability: float, confidence: float): + self.id = source_id + self.geometry = geometry + self.probability = probability # 是源地的概率 + self.confidence = confidence # 概率估计的置信度 + + def __repr__(self): + return (f"Source({self.id}, P={self.probability:.2f}±{self.confidence:.2f})") + +class ProbabilisticSourceIdentifier: + """概率源地识别器""" + + def __init__(self, threshold=0.5): + self.threshold = threshold + + def identify(self, landscape_data) -> List[ProbabilisticSource]: + """ + 识别源地，返回概率源地 + + 返回的不是"是/否"的判断，而是"是源地的概率" + """ + sources = [] + + # 模拟：对每个斑块计算是源地的概率 + for i, patch in enumerate(landscape_data): + # 基于多个指标计算概率 + probability = self._calculate_source_probability(patch) + + # 估计置信度（基于数据质量） + confidence = self._estimate_confidence(patch) + + if probability >= self.threshold: + source = ProbabilisticSource( + source_id=i, + geometry=patch['geometry'], + probability=probability, + confidence=confidence + ) + sources.append(source) + + return sources + + def _calculate_source_probability(self, patch) -> float: + """计算斑块是源地的概率""" + # 使用贝叶斯推理 + # P(是源地|数据) ∝ P(数据|是源地) × P(是源地) + + # 各指标的似然 + area_likelihood = self._area_likelihood(patch['area']) + veg_likelihood = self._vegetation_likelihood(patch['vegetation']) + shape_likelihood = self._shape_likelihood(patch['shape_index']) + + # 先验概率 + prior = 0.3 # 假设30%的斑块可能是源地 + + # 后验概率（简化） + probability = (area_likelihood * veg_likelihood * + shape_likelihood * prior) + probability = min(probability, 1.0) # 限制在[0,1] + + return probability + + def _area_likelihood(self, area: float) -> float: + """面积似然：大面积更像源地""" + if area > 1000: + return 1.0 + elif area > 500: + return 0.7 + else: + return 0.3 + + def _vegetation_likelihood(self, veg_quality: float) -> float: + """植被质量似然""" + return veg_quality # 假设已归一化到[0,1] + + def _shape_likelihood(self, shape_index: float) -> float: + """形状指数似然：紧凑形状更好""" + return 1.0 - min(abs(shape_index - 1.0), 0.5) + + def _estimate_confidence(self, patch) -> float: + """估计概率的置信度""" + # 基于数据质量、分辨率等因素 + data_quality = patch.get('data_quality', 0.8) + resolution_factor = patch.get('resolution', 30) / 30 # 归一化 + + return data_quality * min(resolution_factor, 1.0) + +def uncertainty_propagation_example(): + """不确定性传播示例""" + print("=== 不确定性传播示例 ===\n") + + # 创建一些模拟斑块 + patches = [ + {'id': 1, 'area': 1200, 'vegetation': 0.85, 'shape_index': 1.2, + 'geometry': 'POLYGON(...)', 'data_quality': 0.9}, + {'id': 2, 'area': 800, 'vegetation': 0.75, 'shape_index': 1.5, + 'geometry': 'POLYGON(...)', 'data_quality': 0.7}, + {'id': 3, 'area': 400, 'vegetation': 0.65, 'shape_index': 1.8, + 'geometry': 'POLYGON(...)', 'data_quality': 0.6}, + ] + + identifier = ProbabilisticSourceIdentifier(threshold=0.4) + sources = identifier.identify(patches) + + print("识别到的概率源地:") + for source in sources: + print(f" {source}") + + # 计算整体不确定性 + if sources: + avg_prob = np.mean([s.probability for s in sources]) + avg_conf = np.mean([s.confidence for s in sources]) + print(f"\n总体置信度: {avg_conf:.2f}") + print(f"平均源地概率: {avg_prob:.2f}") + + # 置信区间 + margin_of_error = (1 - avg_conf) * 0.2 # 简化计算 + print(f"源地数量估计: {len(sources)} ± {margin_of_error * len(sources):.1f}") + +if __name__ == "__main__": + uncertainty_propagation_example() +``` + +--- + +## 案例分析 + +### ENAgent中的不确定性处理 + +在ENAgent项目中，不确定性处理体现在： + +**1. 源地识别的不确定性** + +```python +class ENAgentSourceIdentifier: + """ENAgent的源地识别模块""" + + def identify_with_uncertainty(self, landcover, species_params): + """ + 识别源地，同时估计不确定性 + + Returns: + sources: 源地列表 + uncertainty_map: 不确定性空间分布 + """ + sources = [] + uncertainty_map = np.zeros_like(landcover) + + # 对每个候选斑块 + for patch in self._candidate_patches(landcover): + # 计算适宜性（考虑物种参数） + suitability = self._calculate_suitability(patch, species_params) + + # 估计不确定性（来自多个来源） + uncertainty = self._estimate_uncertainty( + patch, + data_quality=landcover.metadata['quality'], + species_uncertainty=species_params['uncertainty'] + ) + + # 记录 + if suitability > self.threshold: + sources.append({ + 'geometry': patch, + 'suitability': suitability, + 'uncertainty': uncertainty + }) + + # 更新不确定性地图 + self._add_to_uncertainty_map(uncertainty_map, patch, uncertainty) + + return sources, uncertainty_map + + def _estimate_uncertainty(self, patch, data_quality, species_uncertainty): + """ + 估计源地识别的不确定性 + + 来源： + 1. 数据质量 (data_quality) + 2. 物种参数的不确定性 (species_uncertainty) + 3. 分类误差 (classification_error) + 4. 边界效应 (edge_effect) + """ + # 组合各种不确定性源 + uncertainty = np.sqrt( + (1 - data_quality)**2 + + species_uncertainty**2 + + 0.1**2 + # 分类误差 + self._edge_uncertainty(patch)**2 + ) + + return min(uncertainty, 1.0) +``` + +**2. 阻力面的敏感性分析** + +```python +class ResistanceSurfaceSensitivity: + """阻力面敏感性分析""" + + def analyze(self, base_weights, variation_ranges, n_simulations=1000): + """ + 分析阻力面权重对结果的敏感性 + + Args: + base_weights: 基础权重 {land_type: weight} + variation_ranges: 权重变化范围 {land_type: (min, max)} + n_simulations: 蒙特卡洛模拟次数 + + Returns: + sensitivity_results: 敏感性分析结果 + """ + results = [] + + for _ in range(n_simulations): + # 随机采样权重 + sample_weights = {} + for land_type, (min_w, max_w) in variation_ranges.items(): + sample_weights[land_type] = np.random.uniform(min_w, max_w) + + # 计算对应的阻力面 + resistance = self._compute_resistance(sample_weights) + + # 评估结果（例如：平均连通性） + connectivity = self._assess_connectivity(resistance) + + results.append({ + 'weights': sample_weights, + 'connectivity': connectivity + }) + + # 分析敏感性 + sensitivity = self._compute_sensitivity(results, base_weights) + + return sensitivity + + def _compute_sensitivity(self, results, base_weights): + """计算各土地类型的敏感性""" + # 计算每个权重变化与连通性变化的相关性 + sensitivities = {} + + for land_type in base_weights: + weight_values = [r['weights'][land_type] for r in results] + connectivity_values = [r['connectivity'] for r in results] + + correlation = np.corrcoef(weight_values, connectivity_values)[0, 1] + sensitivities[land_type] = abs(correlation) + + return sensitivities +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **不确定性识别**：在你的项目中，不确定性来自哪些方面？哪些是可减少的，哪些是固有的？ + +2. **表示选择**：你应该用标准差、置信区间，还是概率分布？各有什么优劣？ + +3. **决策权衡**：当不确定性很高时，你应该继续分析还是寻求更多数据？ + +4. **沟通问题**：如何向非专家解释不确定性？ + +### 实践练习 + +1. **不确定性审计**：对一个分析流程，识别所有不确定性来源并分类 + +2. **敏感性分析**：对你熟悉的空间模型进行敏感性分析 + +3. **鲁棒决策**：为你的项目设计一个鲁棒决策框架 + +### 延伸阅读 + +- **"Uncertainty Quantification in Predictive Modeling"** - 不确定性量化的理论基础 +- **"Flaw of Averages"** (Sam Savage) - 为什么平均值会误导 +- **空间数据质量标准** - 空间不确定性的行业实践 + +--- + +## 关键要点 + +1. **不确定性在空间分析中普遍存在**，有多个来源 +2. **偶然不确定性无法消除**，认知不确定性可以通过更多数据减少 +3. **不确定性会传播**，多步骤分析需要考虑累积效应 +4. **敏感性分析识别关键参数**，优先减少高敏感参数的不确定性 +5. **鲁棒决策在不确定性下做稳健选择**，而非追求最优 diff --git a/officefile/supplements/01-foundations/01.4-feedback-and-learning.md b/officefile/supplements/01-foundations/01.4-feedback-and-learning.md new file mode 100644 index 0000000..df79160 --- /dev/null +++ b/officefile/supplements/01-foundations/01.4-feedback-and-learning.md @@ -0,0 +1,718 @@ +# 01.4 反馈与学习 + +## 核心问题 + +> 系统如何从经验中改进？ +> 强化学习的基本直觉是什么？ +> 如何设计一个好的奖励函数？ + +--- + +## 概念讲解 + +### 反馈循环 + +**反馈**是系统学习的基础机制： + +``` + ┌─────────────────────────────────────────────────┐ + │ │ + │ ┌─────────┐ ┌─────────┐ ┌─────────┐│ + │ │ Action │ ───→ │ Effect │ ───→ │Reward ││ + │ └─────────┘ └─────────┘ └─────────┘│ + │ │ │ │ + │ │ ┌────────────┐ │ │ + │ └───────────→│ Update │←──────┘ │ + │ ↑ │ + │ │ │ + │ ┌──────┴──────┐ │ + │ │ Policy │ │ + │ │ Improvement│ │ + │ └─────────────┘ │ + │ │ + └─────────────────────────────────────────────────┘ +``` + +**反馈的类型**： + +| 类型 | 说明 | 例子 | +|-----|------|------| +| **正反馈** | 强化正确行为 | 生态廊道有效，增加类似策略 | +| **负反馈** | 抑制错误行为 | 阻力面不合理，调整权重 | +| **延迟反馈** | 效果滞后 | 生态工程几年后才见效 | +| **隐式反馈** | 未明确标注 | 用户不使用某功能 = 不好用 | + +### 强化学习的直觉 + +强化学习（RL）是关于"如何通过试错学习"： + +``` +强化学习核心概念 + +智能体 ──→ 动作 ──→ 环境 ──→ 奖励 + ↑ │ + │ │ + └─────────────── 观察状态 ←──────────────┘ + │ + ↓ + 更新策略 +``` + +**关键要素**： + +1. **状态 (State)**：智能体看到的当前情况 +2. **动作 (Action)**：智能体能做的事情 +3. **奖励 (Reward)**：动作好坏的即时反馈 +4. **策略 (Policy)**：状态到动作的映射规则 +5. **价值函数 (Value)**：对长期收益的估计 + +```python +# RL的数学直觉 + +# 策略：在状态s采取动作a的概率 +π(a|s) = P(action=a | state=s) + +# 价值函数：从状态s开始的期望累积奖励 +V(s) = E[Σ γ^t * r_t | s_0 = s] +# γ是折扣因子，平衡即时和长期奖励 + +# 动作价值函数：在状态s采取动作a后的期望累积奖励 +Q(s,a) = E[Σ γ^t * r_t | s_0 = s, a_0 = a] + +# 目标：找到最优策略，最大化累积奖励 +π* = argmax_π V^π(s) +``` + +### 探索与利用的权衡 + +RL中经典的困境： + +``` +探索 (Explore) vs 利用 (Exploit) + + 利用 探索 + ↓ ↓ +选择已知最好的动作 尝试新动作 +获得稳定奖励 可能发现更好动作 +可能错过最优 可能浪费资源 +``` + +**策略**： + +| 策略 | 方法 | 适用场景 | +|-----|------|---------| +| **ε-greedy** | 以ε概率随机探索 | 通用，简单 | +| **Boltzmann** | 按价值概率选择 | 需要细粒度控制 | +| **UCB** | 上置信界选择 | 需要理论保证 | +| **Thompson Sampling** | 采样后验概率 | 贝叶斯框架 | + +```python +def epsilon_greedy_action(q_values, epsilon, n_actions): + """ + ε-greedy策略 + + Args: + q_values: 各动作的估计价值 + epsilon: 探索概率 + n_actions: 动作数量 + + Returns: + 选择的动作 + """ + if np.random.random() < epsilon: + # 探索：随机选择 + return np.random.randint(n_actions) + else: + # 利用：选择价值最高的 + return np.argmax(q_values) + +# ε的衰减策略 +def epsilon_schedule(initial_epsilon, final_epsilon, total_steps, current_step): + """线性衰减ε""" + decay = (initial_epsilon - final_epsilon) / total_steps + return max(final_epsilon, initial_epsilon - decay * current_step) +``` + +--- + +## 设计原理 + +### 奖励函数设计 + +奖励函数定义了"什么是好的行为"： + +```python +""" +奖励函数设计原则 +""" + +# 原则1: 清晰明确 +# 好的奖励 +def good_reward(ecological_quality): + """生态质量越高，奖励越高""" + return ecological_quality + +# 不好的奖励（有歧义） +def bad_reward(ecological_quality, cost): + """混合多个目标，可能冲突""" + return ecological_quality - cost * 0.001 + +# 原则2: 适度塑形 (Reward Shaping) +# 不要过度引导，让智能体自己探索 + +def shaped_reward(base_reward, intermediate_metric): + """ + 基础奖励 + 形状奖励 + + 基础奖励：定义最终目标 + 形状奖励：引导到达目标（权重较小） + """ + return base_reward + 0.1 * intermediate_metric + +# 原则3: 避免奖励黑客 (Reward Hacking) +# 防止智能体找到"作弊"方法 + +def safe_reward_with_constraints(action_result): + """ + 带约束的奖励 + """ + base_reward = action_result['quality'] + + # 如果违反约束，给予惩罚 + if action_result['violates_constraint']: + base_reward -= 100 # 大惩罚 + + # 如果使用"作弊"方法，给予惩罚 + if action_result['uses_exploit']: + base_reward -= 50 + + return base_reward + +# 原则4: 多目标平衡 +def multi_objective_reward(ecological, economic, social, weights): + """ + 多目标加权 + + Args: + ecological: 生态效益 + economic: 经济效益 + social: 社会效益 + weights: 各目标权重 + + Returns: + 综合奖励 + """ + # 归一化到[0,1] + normalized = { + 'eco': min(ecological / 100, 1.0), + 'eco': min(economic / 1000, 1.0), + 'soc': min(social / 100, 1.0) + } + + total = (weights['eco'] * normalized['eco'] + + weights['eco'] * normalized['eco'] + + weights['soc'] * normalized['soc']) + + return total +``` + +### 在空间分析中的应用 + +```python +class SpatialOptimizerRL: + """ + 用强化学习优化空间布局 + + 场景：给定区域内选择最优生态廊道路线 + """ + + def __init__(self, landscape, constraints): + self.landscape = landscape + self.constraints = constraints + + # 状态空间：当前的廊道路线 + # 动作空间：下一步走向哪个像元 + # 奖励：连通性、距离、穿越地类的综合 + + def state_representation(self): + """将当前空间格局转换为状态表示""" + return { + 'current_position': self.current_position, + 'visited_cells': self.visited_cells, + 'local_context': self._get_local_context() + } + + def available_actions(self): + """获取可用的动作""" + # 可以向8个方向移动 + directions = [ + (0, 1), (1, 0), (0, -1), (-1, 0), # 上下左右 + (1, 1), (1, -1), (-1, 1), (-1, -1) # 对角 + ] + + actions = [] + for dx, dy in directions: + new_x = self.current_position[0] + dx + new_y = self.current_position[1] + dy + + if self._is_valid_move(new_x, new_y): + actions.append((new_x, new_y)) + + return actions + + def reward_function(self, action, new_state): + """ + 定义奖励函数 + + 考虑： + 1. 穿越的土地类型（林地奖励，城市惩罚） + 2. 距离目标的远近（越近越好） + 3. 是否到达目标（大奖励） + """ + x, y = new_state['position'] + + # 1. 土地类型奖励/惩罚 + land_type = self.landscape[y, x] + land_rewards = { + 'forest': 10, + 'grassland': 5, + 'wetland': 8, + 'agriculture': 0, + 'urban': -50, + 'water': -20 + } + land_reward = land_rewards.get(land_type, -10) + + # 2. 距离奖励（离目标越近越好） + dist_to_goal = self._distance_to_goal(new_state['position']) + distance_reward = -dist_to_goal * 0.1 + + # 3. 目标到达奖励 + goal_reward = 0 + if new_state['position'] == self.goal_position: + goal_reward = 1000 + + # 4. 约束惩罚 + constraint_penalty = 0 + if self._violates_constraint(new_state): + constraint_penalty = -100 + + # 总奖励 + total_reward = (land_reward + distance_reward + + goal_reward + constraint_penalty) + + return total_reward + + def train(self, n_episodes=1000): + """ + 训练智能体 + + 使用Q-learning + """ + q_table = {} # Q值表 + + for episode in range(n_episodes): + state = self._reset() + epsilon = self._epsilon_schedule(episode) + + done = False + while not done: + # ε-greedy选择动作 + if np.random.random() < epsilon: + action = np.random.choice(self.available_actions()) + else: + # 选择Q值最高的动作 + q_values = [q_table.get((state, a), 0) + for a in self.available_actions()] + action = self.available_actions()[np.argmax(q_values)] + + # 执行动作 + new_state, reward, done = self._step(action) + + # 更新Q值 + old_q = q_table.get((state, action), 0) + max_next_q = max([q_table.get((new_state, a), 0) + for a in self.available_actions()] + [0]) + + # Q-learning更新公式 + q_table[(state, action)] = old_q + 0.1 * ( + reward + 0.99 * max_next_q - old_q + ) + + state = new_state + + return q_table +``` + +--- + +## 代码示例 + +### 简化的生态网络优化RL + +```python +""" +简化版：用Q-learning优化生态源地选择 +""" +import numpy as np +from typing import List, Dict, Tuple +import random + +class EcologicalNetworkOptimizer: + """ + 生态网络优化器（RL简化版） + + 问题：从候选源地中选择最优组合 + - 最大化总生态价值 + - 满足连通性要求 + - 预算约束 + """ + + def __init__(self, candidate_sites: List[Dict], budget: float): + self.candidate_sites = candidate_sites + self.budget = budget + + # 动作：选择或不选择某个源地 + self.n_actions = len(candidate_sites) + + # 状态：已选源地列表 + # 简化：用位掩码表示状态 + self.n_states = 2 ** self.n_actions + + # Q表 + self.q_table = np.zeros((self.n_states, self.n_actions)) + + def state_to_mask(self, state: int) -> List[bool]: + """状态索引转位掩码""" + return [(state >> i) & 1 for i in range(self.n_actions)] + + def mask_to_state(self, mask: List[bool]) -> int: + """位掩码转状态索引""" + state = 0 + for i, bit in enumerate(mask): + if bit: + state |= (1 << i) + return state + + def available_actions(self, state_mask: List[bool]) -> List[int]: + """获取可用动作（未选的源地）""" + return [i for i, selected in enumerate(state_mask) if not selected] + + def reward_function(self, state_mask: List[bool]) -> float: + """ + 计算当前选择的奖励 + + 考虑： + 1. 总生态价值 + 2. 连通性 + 3. 预算约束 + """ + # 选中源地 + selected_sites = [self.candidate_sites[i] + for i, selected in enumerate(state_mask) if selected] + + if not selected_sites: + return 0 + + # 1. 总生态价值 + total_value = sum(site['value'] for site in selected_sites) + + # 2. 连通性（简化：已选源地之间的平均距离） + if len(selected_sites) > 1: + positions = [(site['x'], site['y']) for site in selected_sites] + distances = [] + for i in range(len(positions)): + for j in range(i + 1, len(positions)): + dist = np.sqrt((positions[i][0] - positions[j][0])**2 + + (positions[i][1] - positions[j][1])**2) + distances.append(dist) + avg_distance = np.mean(distances) + connectivity_reward = -0.1 * avg_distance # 距离越近越好 + else: + connectivity_reward = 0 + + # 3. 预算惩罚 + total_cost = sum(site['cost'] for site in selected_sites) + budget_penalty = 0 + if total_cost > self.budget: + budget_penalty = -100 * (total_cost - self.budget) / self.budget + + # 总奖励 + total_reward = total_value + connectivity_reward + budget_penalty + + return total_reward + + def step(self, state: int, action: int) -> Tuple[int, float, bool]: + """ + 执行一步 + + Returns: + next_state: 下一个状态 + reward: 奖励 + done: 是否结束 + """ + state_mask = self.state_to_mask(state) + + # 执行动作（选择一个源地） + if state_mask[action]: # 已经选过了 + return state, -100, True # 惩罚并结束 + + new_mask = state_mask.copy() + new_mask[action] = True + + # 计算奖励 + reward = self.reward_function(new_mask) + + # 检查是否结束（预算用完或所有源地都选了） + total_cost = sum(self.candidate_sites[i]['cost'] + for i, selected in enumerate(new_mask) if selected) + done = (total_cost >= self.budget) or (sum(new_mask) == len(new_mask)) + + next_state = self.mask_to_state(new_mask) + + return next_state, reward, done + + def train(self, n_episodes=1000, alpha=0.1, gamma=0.99, + epsilon_start=1.0, epsilon_end=0.01): + """ + Q-learning训练 + + Args: + n_episodes: 训练回合数 + alpha: 学习率 + gamma: 折扣因子 + epsilon_start: 初始探索率 + epsilon_end: 最终探索率 + """ + for episode in range(n_episodes): + # 线性衰减ε + epsilon = epsilon_start - (epsilon_start - epsilon_end) * episode / n_episodes + + state = 0 # 初始状态（空） + done = False + + while not done: + state_mask = self.state_to_mask(state) + available = self.available_actions(state_mask) + + if not available: + break + + # ε-greedy + if np.random.random() < epsilon: + action = random.choice(available) + else: + q_values = [self.q_table[state, a] for a in available] + action = available[np.argmax(q_values)] + + # 执行动作 + next_state, reward, done = self.step(state, action) + + # Q-learning更新 + old_q = self.q_table[state, action] + next_max = np.max(self.q_table[next_state]) + + self.q_table[state, action] = old_q + alpha * ( + reward + gamma * next_max - old_q + ) + + state = next_state + + # 定期报告 + if episode % 100 == 0: + current_epsilon = epsilon_start - (epsilon_start - epsilon_end) * episode / n_episodes + print(f"Episode {episode}, ε={current_epsilon:.3f}, " + f"Best Q: {np.max(self.q_table[0]):.2f}") + + return self.q_table + + def get_solution(self) -> List[Dict]: + """获取学习到的最优解""" + state = 0 + state_mask = self.state_to_mask(state) + solution = [] + + while True: + available = self.available_actions(state_mask) + if not available: + break + + # 选择Q值最高的动作 + q_values = [self.q_table[state, a] for a in available] + action = available[np.argmax(q_values)] + + solution.append(self.candidate_sites[action]) + state, _, done = self.step(state, action) + + if done: + break + + return solution + +# 示例使用 +def example_usage(): + """示例使用""" + print("=== 生态网络优化：Q-learning ===\n") + + # 创建候选源地 + np.random.seed(42) + n_candidates = 10 + candidates = [] + for i in range(n_candidates): + candidates.append({ + 'id': i, + 'x': np.random.randint(0, 100), + 'y': np.random.randint(0, 100), + 'value': np.random.randint(50, 150), + 'cost': np.random.randint(20, 80) + }) + + budget = 200 + + print(f"候选源地数: {n_candidates}") + print(f"预算: {budget}\n") + + # 创建优化器并训练 + optimizer = EcologicalNetworkOptimizer(candidates, budget) + optimizer.train(n_episodes=500) + + # 获取解 + solution = optimizer.get_solution() + + print("\n=== 最优解 ===") + print(f"选择源地数: {len(solution)}") + total_value = sum(s['value'] for s in solution) + total_cost = sum(s['cost'] for s in solution) + print(f"总价值: {total_value}") + print(f"总成本: {total_cost}") + + print("\n选择的源地:") + for s in solution: + print(f" 源地 {s['id']}: 价值={s['value']}, 成本={s['cost']}") + +if __name__ == "__main__": + example_usage() +``` + +--- + +## 案例分析 + +### ENAgent中的反馈机制 + +```python +class ENAgentFeedback: + """ + ENAgent的反馈机制 + + 场景：生态网络迭代的改进 + """ + + def __init__(self): + self.iteration_history = [] + self.performance_metrics = [] + + def collect_feedback(self, iteration, result, human_feedback): + """ + 收集每轮的反馈 + + Args: + iteration: 迭代次数 + result: 本轮结果 + human_feedback: 人类专家的反馈 + """ + feedback_record = { + 'iteration': iteration, + 'result': result, + 'human_feedback': human_feedback, + 'timestamp': time.time() + } + + self.iteration_history.append(feedback_record) + + def analyze_feedback(self) -> Dict: + """ + 分析反馈，提取改进建议 + + Returns: + 改进建议 + """ + if not self.iteration_history: + return {} + + # 分析模式 + suggestions = {} + + # 1. 常见问题 + problem_counts = {} + for record in self.iteration_history: + for problem in record['human_feedback'].get('problems', []): + problem_counts[problem] = problem_counts.get(problem, 0) + 1 + + if problem_counts: + common_problems = sorted(problem_counts.items(), + key=lambda x: x[1], reverse=True) + suggestions['common_problems'] = common_problems + + # 2. 趋势分析 + if len(self.iteration_history) > 1: + recent_quality = self.iteration_history[-1]['result']['quality'] + previous_quality = self.iteration_history[-2]['result']['quality'] + + if recent_quality > previous_quality: + suggestions['trend'] = 'improving' + else: + suggestions['trend'] = 'stagnant_or_degrading' + + # 3. 参数调整建议 + suggestions['parameter_adjustments'] = self._suggest_adjustments() + + return suggestions + + def _suggest_adjustments(self) -> Dict: + """建议参数调整""" + # 基于反馈历史，建议如何调整参数 + # 这是一个简化示例 + return { + 'resistance_weights': 'consider adjusting urban weight', + 'source_threshold': 'might be too high/low' + } +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **延迟奖励**：生态工程的效果多年后才显现，如何设计奖励函数？ + +2. **稀疏奖励**：当大多数步骤没有明确反馈时，如何学习？ + +3. **多目标冲突**：生态目标和经济目标冲突时，奖励函数如何平衡？ + +4. **人类反馈**：如何整合人类专家的定性反馈？ + +### 实践练习 + +1. **奖励设计**：为一个你熟悉的任务设计奖励函数 + +2. **调试RL**：观察Q表的变化，理解学习过程 + +3. **探索策略**：比较不同ε衰减策略的效果 + +### 延伸阅读 + +- **"Reinforcement Learning: An Introduction"** (Sutton & Barto) - RL圣经 +- **"Algorithms for Decision Making"** (Mykel Kochenderfer) - 决策与RL +- **"Reward Shaping"**论文 - 奖励塑形理论 + +--- + +## 关键要点 + +1. **反馈是学习的基础机制**，正反馈强化正确行为，负反馈纠正错误 +2. **强化学习核心**：状态、动作、奖励、策略、价值函数 +3. **探索vs利用**：经典困境，需要平衡策略 +4. **奖励函数设计**是RL的关键，定义了"什么是好的行为" +5. **在空间分析中**：RL可用于优化布局、路径选择、参数调整 diff --git a/officefile/supplements/01-foundations/01.5-human-ai-collaboration.md b/officefile/supplements/01-foundations/01.5-human-ai-collaboration.md new file mode 100644 index 0000000..9f6fed2 --- /dev/null +++ b/officefile/supplements/01-foundations/01.5-human-ai-collaboration.md @@ -0,0 +1,761 @@ +# 01.5 人机协同的原理 + +## 核心问题 + +> 人类和AI各自的优势是什么？如何互补？ +> 何时需要人类介入？如何设计审查点？ +> 如何建立和维护对AI系统的信任？ + +--- + +## 概念讲解 + +### 人类与AI的能力对比 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 人类 vs AI 能力对比 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 能力维度 │ 人类 │ AI │ +│ ───────────── │ ──────────── │ ───────────────── │ +│ │ +│ 模式识别 │ 不擅长大量 │ 非常擅长 │ +│ │ 数据的模式 │ 大规模模式识别 │ +│ │ +│ 语义理解 │ 深度理解 │ 表层理解 │ +│ │ 上下文关联 │ 统计关联 │ +│ │ +│ 创造力 │ 原创性强 │ 组合创新 │ +│ │ 跳跃思维 │ 已有模式重组 │ +│ │ +│ 伦理判断 │ 天然具备 │ 需要显式编码 │ +│ │ 直觉道德 │ 规则约束 │ +│ │ +│ 不确定性处理 │ 直觉判断 │ 概率计算 │ +│ │ 启发式 │ 量化评估 │ +│ │ +│ 知识获取 │ 慢，深度 │ 快，广度 │ +│ │ 需要学习 │ 即时查询 │ +│ │ +│ 注意力控制 │ 有限，易疲劳 │ 不知疲倦 │ +│ │ 可自主转移 │ 需要任务定义 │ +│ │ +│ 可解释性 │ 可事后解释 │ 需要专门设计 │ +│ │ 理由可能模糊 │ 逻辑清晰 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### HITL的理论基础 + +**Human-in-the-Loop (HITL)** 不仅仅是"让人检查结果"，而是有理论基础的系统设计方法： + +``` +HITL的理论支撑 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 互补性原理 │ +│ 人类和AI有互补优势，结合优于单独使用 │ +│ │ +│ 2. 控制论原理 │ +│ 人类作为反馈回路的一部分，可以校正系统偏差 │ +│ │ +│ 3. 信任校准 │ +│ 通过参与建立对AI能力的准确认知 │ +│ │ +│ 4. 价值对齐 │ +│ 人类介入确保AI行为与人类价值观一致 │ +│ │ +│ 5. 责任归属 │ +│ 人类在关键决策点参与，明确责任边界 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 信任建立的动态 + +``` +信任建立过程 + +时间 + │ + │ ┌────────────┐ + │ │ 初始信任 │ 基于声誉、宣传等 + │ └─────┬──────┘ + │ │ 第一次使用 + │ ↓ + │ ┌────────────┐ + │ │ 体验信任 │ 基于实际交互 + │ └─────┬──────┘ + │ │ + │ ┌──────┴──────┐ + │ ↓ ↓ + │ 成功 失败 + │ │ │ + │ ↓ ↓ + │ ┌─────┐ ┌─────┐ + │ │信任 │ │不信任│ + │ │增强 │ │/怀疑 │ + │ └──┬──┘ └──┬──┘ + │ │ │ + │ └──────┬──────┘ + │ │ 解释/透明度 + │ ↓ + │ ┌────────────┐ + │ │ 校准信任 │ 与能力匹配的信任水平 + │ └────────────┘ + │ + └──────────────────────────→ +``` + +--- + +## 设计原理 + +### 何时需要人类介入 + +**决策框架**：根据任务的特性决定介入程度 + +```python +def human_intervention_necessity(task_characteristics: Dict) -> str: + """ + 评估任务需要人类介入的程度 + + Args: + task_characteristics: 任务特性描述 + + Returns: + 介入程度: 'full', 'selective', 'minimal', 'none' + """ + scores = { + 'consequence': 0, # 后果严重性 + 'uncertainty': 0, # 不确定性 + 'ethical': 0, # 伦理敏感性 + 'complexity': 0, # 复杂度 + 'novelty': 0 # 新颖性 + } + + # 评估后果严重性 + if task_characteristics.get('life_critical', False): + scores['consequence'] = 3 + elif task_characteristics.get('economic_impact', 0) > 1000000: + scores['consequence'] = 2 + elif task_characteristics.get('economic_impact', 0) > 100000: + scores['consequence'] = 1 + + # 评估不确定性 + uncertainty = task_characteristics.get('uncertainty_level', 'low') + scores['uncertainty'] = {'low': 0, 'medium': 1, 'high': 2}[uncertainty] + + # 评估伦理敏感性 + if task_characteristics.get('ethical_concerns', False): + scores['ethical'] = 3 + + # 评估复杂度 + complexity = task_characteristics.get('complexity', 'low') + scores['complexity'] = {'low': 0, 'medium': 1, 'high': 2}[complexity] + + # 评估新颖性 + if task_characteristics.get('novel_situation', False): + scores['novelty'] = 2 + + # 总分 + total_score = sum(scores.values()) + + # 决定介入程度 + if total_score >= 10: + return 'full' # 完全由人类主导 + elif total_score >= 6: + return 'selective' # 关键点介入 + elif total_score >= 3: + return 'minimal' # 异常时介入 + else: + return 'none' # AI自主执行 +``` + +**ENAgent的三个审查点设计依据**： + +| 审查点 | 任务特性 | 介入理由 | +|-------|---------|---------| +| 源地识别 | 高不确定性 + 本地知识需求 | 遥感分类可能错误，地面实况重要 | +| 阻力权重 | 高价值判断 + 物种特异性 | 不同物种权重差异大，专家知识关键 | +| 廊道优化 | 多目标权衡 + 社会影响 | 生态与经济/社会的平衡，人类决策 | + +### 信任校准机制 + +```python +class TrustCalibration: + """ + 信任校准系统 + + 目标：让用户的信任水平与AI的实际能力匹配 + """ + + def __init__(self): + self.declared_confidence = [] # AI声明的置信度 + self.actual_performance = [] # 实际表现 + self.user_trust_level = 0.5 # 用户信任水平 + + def record_outcome(self, ai_confidence: float, + actual_correct: bool, + user_trusted: bool): + """ + 记录一次AI决策的结果 + + Args: + ai_confidence: AI声明的置信度 [0, 1] + actual_correct: 实际是否正确 + user_trusted: 用户是否信任并采用了AI建议 + """ + self.declared_confidence.append(ai_confidence) + self.actual_performance.append(1.0 if actual_correct else 0.0) + + def assess_calibration(self) -> Dict: + """ + 评估AI的校准程度 + + Returns: + 校准报告 + """ + if not self.declared_confidence: + return {'status': 'insufficient_data'} + + # 按置信度分组统计 + confidence_bins = { + 'high': [], # > 0.8 + 'medium': [], # 0.5-0.8 + 'low': [] # < 0.5 + } + + for conf, perf in zip(self.declared_confidence, self.actual_performance): + if conf > 0.8: + confidence_bins['high'].append(perf) + elif conf > 0.5: + confidence_bins['medium'].append(perf) + else: + confidence_bins['low'].append(perf) + + # 计算各组的平均实际表现 + calibration_report = {} + for bin_name, performances in confidence_bins.items(): + if performances: + avg_performance = sum(performances) / len(performances) + calibration_report[bin_name] = { + 'ai_declared_range': self._get_bin_range(bin_name), + 'actual_accuracy': avg_performance, + 'calibration_gap': avg_performance - self._get_bin_expected(bin_name) + } + + return calibration_report + + def _get_bin_range(self, bin_name: str) -> str: + ranges = { + 'high': '> 0.8', + 'medium': '0.5-0.8', + 'low': '< 0.5' + } + return ranges[bin_name] + + def _get_bin_expected(self, bin_name: str) -> float: + """该置信度组的期望表现""" + expected = { + 'high': 0.9, + 'medium': 0.65, + 'low': 0.25 + } + return expected[bin_name] + + def recommend_trust_adjustment(self) -> str: + """ + 基于校准结果，建议信任调整 + + Returns: + 调整建议 + """ + calibration = self.assess_calibration() + + if calibration.get('status') == 'insufficient_data': + return "需要更多数据来评估" + + overconfident = any( + v['calibration_gap'] < -0.1 + for v in calibration.values() + if isinstance(v, dict) + ) + + underconfident = any( + v['calibration_gap'] > 0.1 + for v in calibration.values() + if isinstance(v, dict) + ) + + if overconfident: + return "AI倾向于过度自信，建议降低信任度，增加审查" + elif underconfident: + return "AI实际表现优于声明，可以增加信任" + else: + return "AI校准良好，当前信任水平适当" +``` + +### 审查点设计模式 + +```python +class CheckpointDesign: + """ + 审查点设计框架 + """ + + @staticmethod + def design_checkpoint(task_info: Dict) -> Dict: + """ + 为任务设计审查点 + + Args: + task_info: 任务信息 + + Returns: + 审查点设计 + """ + checkpoint = { + 'name': task_info['name'], + 'trigger_condition': None, + 'information_provided': [], + 'decision_options': [], + 'default_action': None, + 'timeout_handling': None + } + + # 1. 触发条件设计 + checkpoint['trigger_condition'] = CheckpointDesign._design_trigger(task_info) + + # 2. 信息提供设计 + checkpoint['information_provided'] = CheckpointDesign._design_info_display(task_info) + + # 3. 决策选项设计 + checkpoint['decision_options'] = CheckpointDesign._design_options(task_info) + + # 4. 默认行为 + checkpoint['default_action'] = CheckpointDesign._design_default(task_info) + + return checkpoint + + @staticmethod + def _design_trigger(task_info: Dict) -> Dict: + """设计触发条件""" + return { + 'type': 'conditional', # always, conditional, on_error + 'conditions': [ + 'confidence_below_threshold', + 'conflicting_alternatives', + 'ethical_concern_detected' + ], + 'threshold': task_info.get('confidence_threshold', 0.7) + } + + @staticmethod + def _design_info_display(task_info: Dict) -> List[str]: + """设计展示给人类的信息""" + base_info = [ + 'ai_proposal', + 'confidence_level', + 'reasoning_trace' + ] + + # 根据任务类型添加额外信息 + if task_info.get('high_stakes', False): + base_info.extend([ + 'consequence_analysis', + 'alternative_options' + ]) + + if task_info.get('uncertain', False): + base_info.append('uncertainty_quantification') + + return base_info + + @staticmethod + def _design_options(task_info: Dict) -> List[str]: + """设计人类决策选项""" + base_options = ['approve', 'reject', 'modify'] + + if task_info.get('allow_delegation', False): + base_options.append('delegate_to_ai') + + return base_options + + @staticmethod + def _design_default(task_info: Dict) -> str: + """设计默认行为（人类不响应时）""" + if task_info.get('high_stakes', False): + return 'wait_for_human' # 等待人类 + else: + return 'proceed_with_caution' # 谨慎继续 +``` + +--- + +## 代码示例 + +### 完整的HITL工作流实现 + +```python +""" +完整的人机协同工作流实现 +""" +import time +from typing import Dict, List, Optional, Callable +from dataclasses import dataclass +from enum import Enum + +class HumanDecision(Enum): + """人类决策类型""" + APPROVE = "approve" + REJECT = "reject" + MODIFY = "modify" + DEFER = "defer" + REQUEST_INFO = "request_info" + +@dataclass +class CheckpointResult: + """审查点结果""" + checkpoint_name: str + decision: HumanDecision + modifications: Optional[Dict] = None + additional_input: Optional[Dict] = None + timestamp: float = None + +class HITLWorkflow: + """ + 人机协同工作流 + """ + + def __init__(self, name: str): + self.name = name + self.checkpoints: Dict[str, Dict] = {} + self.state = {} + self.history: List[CheckpointResult] = [] + + def add_checkpoint(self, + name: str, + trigger: Callable, + info_formatter: Callable = None, + critical: bool = False): + """ + 添加审查点 + + Args: + name: 审查点名称 + trigger: 触发条件函数，返回True时需要审查 + info_formatter: 信息格式化函数 + critical: 是否为关键审查点 + """ + self.checkpoints[name] = { + 'trigger': trigger, + 'info_formatter': info_formatter or (lambda x: x), + 'critical': critical, + 'activated': False + } + + def execute_step(self, + step_name: str, + step_function: Callable, + **kwargs) -> Dict: + """ + 执行工作流步骤 + + Args: + step_name: 步骤名称 + step_function: 执行函数 + **kwargs: 传递给函数的参数 + + Returns: + 执行结果 + """ + print(f"\n{'='*50}") + print(f"执行步骤: {step_name}") + print('='*50) + + # 检查是否有审查点 + checkpoint = self.checkpoints.get(step_name) + + if checkpoint: + # 执行步骤 + result = step_function(self.state, **kwargs) + + # 格式化信息 + info = checkpoint['info_formatter'](result) + + # 检查是否需要触发审查 + if checkpoint['trigger'](result, self.state): + print(f"\n[审查点触发: {step_name}]") + checkpoint['activated'] = True + + # 获取人类决策 + decision = self._get_human_decision(info, step_name) + + # 记录决策 + self.history.append(CheckpointResult( + checkpoint_name=step_name, + decision=decision['type'], + modifications=decision.get('modifications'), + timestamp=time.time() + )) + + # 根据决策处理 + if decision['type'] == HumanDecision.APPROVE: + print("✓ 人类批准，继续执行") + self.state[step_name] = result + + elif decision['type'] == HumanDecision.REJECT: + print("✗ 人类拒绝，回退") + return {'status': 'rejected', 'checkpoint': step_name} + + elif decision['type'] == HumanDecision.MODIFY: + print("✎ 人类修改结果") + result = self._apply_modifications(result, decision['modifications']) + self.state[step_name] = result + + elif decision['type'] == HumanDecision.DEFER: + print("⏸ 暂停，等待更多信息") + return {'status': 'deferred', 'checkpoint': step_name} + + else: + print(f"审查点未触发（条件不满足），自动继续") + self.state[step_name] = result + else: + # 没有审查点，直接执行 + result = step_function(self.state, **kwargs) + self.state[step_name] = result + + return result + + def _get_human_decision(self, info: Dict, checkpoint_name: str) -> Dict: + """ + 获取人类决策 + + 实际实现中可能是GUI、CLI或其他交互方式 + """ + print("\n" + "-"*40) + print("信息摘要:") + for key, value in info.items(): + print(f" {key}: {value}") + + print("\n可用决策:") + print(" 1. 批准 (approve)") + print(" 2. 拒绝 (reject)") + print(" 3. 修改 (modify)") + + # 模拟人类输入 + # 实际实现中等待真实输入 + choice = "1" # 默认批准 + + decisions = { + "1": HumanDecision.APPROVE, + "2": HumanDecision.REJECT, + "3": HumanDecision.MODIFY + } + + return {'type': decisions[choice]} + + def _apply_modifications(self, original: Dict, modifications: Dict) -> Dict: + """应用人类修改""" + if modifications: + original.update(modifications) + return original + + def get_checkpoint_summary(self) -> Dict: + """获取审查点摘要""" + return { + 'total_checkpoints': len(self.checkpoints), + 'activated_checkpoints': sum(1 for c in self.checkpoints.values() if c['activated']), + 'human_decisions': [ + { + 'checkpoint': r.checkpoint_name, + 'decision': r.decision.value, + 'timestamp': r.timestamp + } + for r in self.history + ] + } + +# 示例：生态网络分析的HITL工作流 +def ecological_hitl_example(): + """生态网络分析HITL示例""" + + workflow = HITLWorkflow("ecological_network_analysis") + + # 步骤1：加载数据（无审查） + def load_data(state): + print("加载土地利用数据...") + return {'data_loaded': True, 'n_pixels': 10000} + + # 步骤2：识别源地（有审查） + def identify_sources(state): + print("识别生态源地...") + sources = [ + {'id': 1, 'area': 1500, 'confidence': 0.85}, + {'id': 2, 'area': 800, 'confidence': 0.65}, + {'id': 3, 'area': 2000, 'confidence': 0.92} + ] + return {'sources': sources, 'n_sources': len(sources)} + + # 审查条件：有低置信度源地时触发 + def source_trigger(result, state): + return any(s['confidence'] < 0.7 for s in result['sources']) + + # 信息格式化 + def format_source_info(result): + return { + '识别源地数': result['n_sources'], + '平均置信度': sum(s['confidence'] for s in result['sources']) / result['n_sources'], + '低置信度源地': [s['id'] for s in result['sources'] if s['confidence'] < 0.7] + } + + workflow.add_checkpoint( + 'identify_sources', + trigger=source_trigger, + info_formatter=format_source_info, + critical=True + ) + + # 步骤3：构建阻力面（有审查） + def build_resistance(state): + print("构建阻力面...") + return {'weights': {'forest': 1, 'urban': 100}, 'built': True} + + # 审查条件：总是触发 + def resistance_trigger(result, state): + return True # 权重设置总是需要人类审查 + + def format_resistance_info(result): + return result['weights'] + + workflow.add_checkpoint( + 'build_resistance', + trigger=resistance_trigger, + info_formatter=format_resistance_info + ) + + # 执行工作流 + print("=== 开始执行HITL工作流 ===") + + workflow.execute_step('load_data', load_data) + workflow.execute_step('identify_sources', identify_sources) + workflow.execute_step('build_resistance', build_resistance) + + # 摘要 + summary = workflow.get_checkpoint_summary() + print("\n=== 工作流摘要 ===") + print(f"总审查点: {summary['total_checkpoints']}") + print(f"激活审查点: {summary['activated_checkpoints']}") + print("人类决策:") + for decision in summary['human_decisions']: + print(f" {decision['checkpoint']}: {decision['decision']}") + +if __name__ == "__main__": + ecological_hitl_example() +``` + +--- + +## 案例分析 + +### ENAgent的审查点实现 + +```python +class ENAgentHITL: + """ + ENAgent的人机协同实现 + """ + + def __init__(self): + self.review_points = { + 'source_identification': SourceReview(), + 'resistance_surface': ResistanceReview(), + 'corridor_extraction': CorridorReview() + } + + class SourceReview: + """源地识别审查""" + + def trigger_condition(self, sources): + """触发条件""" + # 条件1：有低置信度源地 + low_confidence = any(s['confidence'] < 0.7 for s in sources) + + # 条件2：源地数量异常 + abnormal_count = len(sources) < 3 or len(sources) > 20 + + # 条件3：源地分布极不均匀 + if len(sources) >= 2: + areas = [s['area'] for s in sources] + area_range = max(areas) - min(areas) + uneven = area_range > 10 * sum(areas) / len(areas) + else: + uneven = False + + return low_confidence or abnormal_count or uneven + + def format_for_review(self, sources): + """格式化信息供审查""" + return { + 'n_sources': len(sources), + 'sources_by_confidence': sorted(sources, + key=lambda x: x['confidence']), + 'spatial_distribution': self._analyze_distribution(sources), + 'potential_issues': self._detect_issues(sources) + } + + def _detect_issues(self, sources): + """检测潜在问题""" + issues = [] + + if len(sources) < 3: + issues.append("源地数量偏少，可能遗漏重要栖息地") + + low_conf = [s for s in sources if s['confidence'] < 0.7] + if low_conf: + issues.append(f"{len(low_conf)}个源地置信度低于0.7") + + return issues +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **责任边界**：当HITL系统出错时，责任应该如何划分？ + +2. **审查疲劳**：如果审查点太多，人类会产生疲劳，如何平衡？ + +3. **信任过度**：如何防止人类过度信任AI而减少必要的审查？ + +4. **可解释性**：AI应该如何向人类解释其推理过程？ + +### 实践练习 + +1. **审查点设计**：为你熟悉的流程设计审查点 + +2. **信任评估**：记录你使用AI工具的经历，评估信任变化 + +3. **HITL实现**：实现一个简单的HITL工作流 + +### 延伸阅读 + +- **"Human-in-the-Loop Machine Learning"** - HITL系统设计 +- **"Human-Centered AI"** (Ben Shneiderman) - 以人为本的AI +- **"Explainable AI"**论文集 - 可解释AI研究 + +--- + +## 关键要点 + +1. **人类和AI有互补优势**，结合优于单独使用 +2. **HITL不是妥协**，而是有理论基础的系统设计方法 +3. **审查点选择关键**：在需要人类独特能力的决策点介入 +4. **信任需要校准**：让信任水平与实际能力匹配 +5. **责任必须明确**：关键决策点的人类参与确保责任归属 diff --git a/officefile/supplements/01-foundations/README.md b/officefile/supplements/01-foundations/README.md new file mode 100644 index 0000000..29f1269 --- /dev/null +++ b/officefile/supplements/01-foundations/README.md @@ -0,0 +1,181 @@ +# 第二部分：基础原理 + +## 本部分目标 + +理解现代AI系统的核心设计原理，超越具体工具： +- 智能系统的模块化设计思想 +- 状态与状态机的设计哲学 +- 概率思维与不确定性处理 +- 反馈机制与学习原理 +- 人机协同的理论基础 + +--- + +## 章节导航 + +| 章节 | 文件 | 核心问题 | 实践 | +|-----|------|---------|------| +| 01.1 | [智能的模块化视角](./01.1-modular-intelligence.md) | 为什么要模块化？技能如何封装？ | QGIS技能架构分析 | +| 01.2 | [状态与状态机](./01.2-state-and-state-machines.md) | 状态是什么？为何重要？ | 简单工作流状态机 | +| 01.3 | [概率与不确定性](./01.3-probability-and-uncertainty.md) | AI如何处理未知？ | 生态源地识别不确定性 | +| 01.4 | [反馈与学习](./01.4-feedback-and-learning.md) | 系统如何改进？ | 生态网络优化示例 | +| 01.5 | [人机协同的原理](./01.5-human-ai-collaboration.md) | 何时需要人类介入？ | ENAgent审查点设计 | + +--- + +## 学习路径 + +``` + ┌─────────────────┐ + │ 01-foundations │ + └────────┬────────┘ + │ + ┌────────────────────┼────────────────────┐ + │ │ │ + ↓ ↓ ↓ + ┌──────────┐ ┌──────────┐ ┌──────────┐ + │设计思维 │ │系统思维 │ │协作思维 │ + │01.1, 01.2│ │01.3, 01.4│ │ 01.5 │ + └──────────┘ └──────────┘ └──────────┘ + │ │ │ + └────────────────────┼────────────────────┘ + │ + ↓ + ┌─────────────────┐ + │ 综合理解 │ + │ AI系统设计 │ + └─────────────────┘ +``` + +--- + +## 核心概念图谱 + +``` + ┌─────────────────────────────────────┐ + │ AI系统设计核心 │ + └─────────────────────────────────────┘ + │ + ┌──────────────────────────────┼──────────────────────────────┐ + │ │ │ + ↓ ↓ ↓ +┌───────────────┐ ┌───────────────┐ ┌───────────────┐ +│ 模块化 │ │ 状态机 │ │ 反馈循环 │ +│ ────────── │ │ ────────── │ │ ────────── │ +│ 技能封装 │ │ 工作流编排 │ │ 学习优化 │ +│ 接口设计 │ │ 条件分支 │ │ 奖励信号 │ +│ 组合模式 │ │ 错误处理 │ │ 探索利用 │ +└───────────────┘ └───────────────┘ └───────────────┘ + │ │ │ + └──────────────────────────────┼──────────────────────────────┘ + │ + ↓ + ┌─────────────────────────────────────────────────────────────┐ + │ 人机协同层 │ + │ ───────────────────────────────────────────────── │ + │ 何时介入 │ 如何信任 │ 责任边界 │ 互补优势 │ + └─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 前置知识 + +**必需**： +- Python面向对象编程基础 +- 函数式编程概念（高阶函数、map/reduce） +- 基本的数据结构（图、树、字典） + +**有助理解**： +- 设计模式基础 +- 状态机概念 +- 概率论基础 + +--- + +## 预计学习时间 + +| 阅读类型 | 时间估计 | +|---------|---------| +| 快速浏览 | 3-4小时 | +| 理解性阅读 | 10-15小时 | +| 完成所有实践 | 20-25小时 | + +--- + +## 章节亮点 + +### 01.1 智能的模块化视角 +- 从QGIS插件架构理解模块化 +- 函数式组合思想 +- 技能即能力封装的设计理念 + +### 01.2 状态与状态机 +- 为什么状态管理是核心 +- LangGraph的状态设计哲学 +- 工作流的状态机实现 + +### 01.3 概率与不确定性 +- 空间分析中的不确定性来源 +- 置信度的表示和传播 +- 鲁棒决策的方法 + +### 01.4 反馈与学习 +- 强化学习的直觉理解 +- 奖励函数设计原则 +- 探索与利用的权衡 + +### 01.5 人机协同的原理 +- HITL的理论基础 +- 信任校准机制 +- 责任边界划分 + +--- + +## 实践案例01：用LangGraph构建空间决策工作流 + +详见 [practice/langgraph-workflow](./practice/langgraph-workflow/) + +### 实践目标 + +1. 理解状态驱动的Agent设计 +2. 实现一个简单的空间决策工作流 +3. 添加Human-in-the-Loop审查点 +4. 处理工作流中的错误和重试 + +--- + +## 思考框架 + +在学习每章时，问自己： + +1. **概念理解**：这个概念解决了什么问题？ +2. **设计权衡**：为什么这样设计？有哪些替代方案？ +3. **实际应用**：这个原理在ENAgent中如何体现？ +4. **迁移思考**：这个原理可以应用到我的工作中吗？ + +--- + +## 延伸资源 + +### 经典阅读 +- **"Design Patterns"** (GoF) - 设计模式基础 +- **"Introduction to Automata Theory"** - 状态机理论 +- **"Reinforcement Learning: An Introduction"** - RL基础 + +### 在线资源 +- LangGraph官方文档 +- LangChain状态管理指南 +- Human-in-the-Loop机器学习论文集 + +--- + +## 关键要点预览 + +1. **模块化是管理复杂性的核心方法** +2. **状态机是工作流编排的基础抽象** +3. **概率思维让AI能处理不确定性** +4. **反馈循环是学习和改进的机制** +5. **人机协同需要明确的责任边界** + +> "原理是知识的骨架，工具是知识的血肉。骨架不变，血肉可生。" diff --git a/officefile/supplements/01-foundations/practice/langgraph-workflow/README.md b/officefile/supplements/01-foundations/practice/langgraph-workflow/README.md new file mode 100644 index 0000000..d7dfbae --- /dev/null +++ b/officefile/supplements/01-foundations/practice/langgraph-workflow/README.md @@ -0,0 +1,370 @@ +# 实践案例01：用LangGraph构建空间决策工作流 + +## 目标 + +通过本实践，你将： +1. 理解状态驱动的Agent设计 +2. 实现一个简单的空间决策工作流 +3. 添加Human-in-the-Loop审查点 +4. 处理工作流中的错误和重试 + +--- + +## 背景知识 + +### 什么是LangGraph + +LangGraph是构建**有状态**的多Agent应用的框架： + +``` +核心概念： + +1. State（状态）: 在节点间传递的数据 +2. Node（节点）: 处理状态的函数 +3. Edge（边）: 节点之间的连接 +4. Graph（图）: 节点和边组成的完整工作流 +``` + +### 为什么用LangGraph + +- **状态管理**: 自动管理工作流状态 +- **可视化**: 可以绘制和查看工作流图 +- **持久化**: 支持中断和恢复 +- **条件路由**: 基于状态动态选择路径 + +--- + +## 实践步骤 + +### 步骤1：安装依赖 + +```bash +pip install langgraph langchain-core langchain-anthropic +``` + +### 步骤2：定义状态 + +```python +from typing import TypedDict, Annotated, List, Optional +from operator import add +from typing_extensions import TypedDict + +class EcologicalAnalysisState(TypedDict): + """生态网络分析状态""" + + # 输入 + input_path: str + parameters: dict + + # 处理过程 + current_step: str + intermediate_results: dict + + # 人机交互 + review_requested: bool + human_feedback: Optional[str] + + # 输出 + final_result: Optional[dict] + errors: Annotated[List[str], add] +``` + +### 步骤3：定义节点 + +```python +def load_data_node(state: EcologicalAnalysisState) -> EcologicalAnalysisState: + """加载数据节点""" + print("执行: load_data") + # 实际实现中读取文件 + return { + **state, + "current_step": "data_loaded", + "intermediate_results": {"data": "loaded"} + } + +def identify_sources_node(state: EcologicalAnalysisState) -> EcologicalAnalysisState: + """识别源地节点""" + print("执行: identify_sources") + # 实际实现中运行源地识别算法 + sources = [{"id": 1, "area": 1000}, {"id": 2, "area": 800}] + return { + **state, + "current_step": "sources_identified", + "intermediate_results": {**state["intermediate_results"], "sources": sources} + } + +def human_review_node(state: EcologicalAnalysisState) -> EcologicalAnalysisState: + """人类审查节点""" + print("执行: human_review") + print(f"待审查: {state['intermediate_results']}") + # 实际实现中等待人类输入 + return { + **state, + "review_requested": False, + "human_feedback": "approved" + } + +def build_resistance_node(state: EcologicalAnalysisState) -> EcologicalAnalysisState: + """构建阻力面节点""" + print("执行: build_resistance") + return { + **state, + "current_step": "resistance_built" + } +``` + +### 步骤4：定义路由 + +```python +def should_review(state: EcologicalAnalysisState) -> str: + """决定是否需要审查""" + sources = state["intermediate_results"].get("sources", []) + if len(sources) > 2: # 源地数量多时需要审查 + return "review" + return "continue" +``` + +### 步骤5：构建图 + +```python +from langgraph.graph import StateGraph, END + +def build_workflow(): + """构建工作流图""" + + # 创建图 + workflow = StateGraph(EcologicalAnalysisState) + + # 添加节点 + workflow.add_node("load_data", load_data_node) + workflow.add_node("identify_sources", identify_sources_node) + workflow.add_node("human_review", human_review_node) + workflow.add_node("build_resistance", build_resistance_node) + + # 设置入口 + workflow.set_entry_point("load_data") + + # 添加边 + workflow.add_edge("load_data", "identify_sources") + + # 添加条件边 + workflow.add_conditional_edges( + "identify_sources", + should_review, + { + "review": "human_review", + "continue": "build_resistance" + } + ) + + workflow.add_edge("human_review", "build_resistance") + workflow.add_edge("build_resistance", END) + + # 编译 + return workflow.compile() +``` + +### 步骤6：运行工作流 + +```python +def run_workflow(): + """运行工作流""" + + # 初始状态 + initial_state = { + "input_path": "data.geojson", + "parameters": {}, + "current_step": "start", + "intermediate_results": {}, + "review_requested": False, + "human_feedback": None, + "final_result": None, + "errors": [] + } + + # 构建并运行 + app = build_workflow() + result = app.invoke(initial_state) + + print("\n=== 最终结果 ===") + print(result) +``` + +--- + +## 扩展练习 + +### 1. 添加错误处理 + +```python +def with_error_handling(node_func): + """装饰器：添加错误处理""" + def wrapper(state): + try: + return node_func(state) + except Exception as e: + return { + **state, + "errors": [str(e)] + } + return wrapper + +# 使用 +@with_error_handling +def risky_node(state): + # 可能出错的节点 + ... +``` + +### 2. 添加检查点（持久化） + +```python +from langgraph.checkpoint.memory import MemorySaver + +# 创建检查点保存器 +memory = MemorySaver() + +# 编译时添加检查点 +app = workflow.compile(checkpointer=memory, interrupt_before=["human_review"]) + +# 运行时可以指定thread_id +config = {"configurable": {"thread_id": "conversation-1"}} +result = app.invoke(initial_state, config=config) +``` + +### 3. 可视化工作流 + +```python +from IPython.display import Image, display + +# 生成图 +app = build_workflow() +display(Image(app.get_graph().draw_mermaid_png())) +``` + +--- + +## 完整代码示例 + +```python +""" +完整的LangGraph空间决策工作流示例 +""" +from typing import TypedDict, Annotated, List, Optional, Literal +from operator import add +from langgraph.graph import StateGraph, END + +class State(TypedDict): + """工作流状态""" + step: int + data: Optional[dict] + sources: Optional[list] + reviewed: bool + result: Optional[str] + errors: Annotated[List[str], add] + +# 节点函数 +def load_node(state: State) -> State: + """加载数据""" + print(f"[节点: load] 步骤 {state['step']}") + return {**state, "step": state["step"] + 1, "data": {"loaded": True}} + +def analyze_node(state: State) -> State: + """分析数据""" + print(f"[节点: analyze] 步骤 {state['step']}") + return { + **state, + "step": state["step"] + 1, + "sources": [{"id": 1, "value": 100}] + } + +def review_node(state: State) -> State: + """人类审查""" + print(f"[节点: review] 步骤 {state['step']}") + print("等待人类审查...") + # 实际实现中等待输入 + return {**state, "step": state["step"] + 1, "reviewed": True} + +def finalize_node(state: State) -> State: + """完成""" + print(f"[节点: finalize] 步骤 {state['step']}") + return {**state, "result": "completed"} + +# 路由函数 +def route_after_analyze(state: State) -> Literal["review", "finalize"]: + """分析后的路由""" + if state.get("sources") and len(state["sources"]) > 0: + return "review" + return "finalize" + +# 构建图 +def build_graph(): + """构建工作流图""" + graph = StateGraph(State) + + # 添加节点 + graph.add_node("load", load_node) + graph.add_node("analyze", analyze_node) + graph.add_node("review", review_node) + graph.add_node("finalize", finalize_node) + + # 添加边 + graph.set_entry_point("load") + graph.add_edge("load", "analyze") + + # 条件边 + graph.add_conditional_edges( + "analyze", + route_after_analyze, + {"review": "review", "finalize": "finalize"} + ) + + graph.add_edge("review", "finalize") + graph.add_edge("finalize", END) + + return graph.compile() + +# 运行 +if __name__ == "__main__": + print("=== LangGraph 空间决策工作流 ===\n") + + app = build_graph() + + initial_state: State = { + "step": 1, + "data": None, + "sources": None, + "reviewed": False, + "result": None, + "errors": [] + } + + result = app.invoke(initial_state) + + print(f"\n最终状态: {result['step']}") + print(f"结果: {result['result']}") +``` + +--- + +## 反思问题 + +1. **状态设计**：你的状态中哪些信息是必需的？哪些可以省略？ + +2. **节点粒度**：节点应该多大？如何平衡？ + +3. **错误处理**：当节点失败时，工作流应该如何处理？ + +4. **审查点**：你的工作流中哪些地方需要人类介入？ + +--- + +## 下一步 + +完成这个实践后，你已经： +- ✅ 理解了状态驱动的Agent设计 +- ✅ 实现了一个简单的LangGraph工作流 +- ✅ 掌握了条件路由的基本方法 +- ✅ 了解了如何添加HITL审查点 + +准备好进入下一章：**02-spatial-intelligence（空间智能）** diff --git a/officefile/supplements/02-spatial-intelligence/02.1-spatial-representation.md b/officefile/supplements/02-spatial-intelligence/02.1-spatial-representation.md new file mode 100644 index 0000000..0ca5722 --- /dev/null +++ b/officefile/supplements/02-spatial-intelligence/02.1-spatial-representation.md @@ -0,0 +1,1074 @@ +# 02.1 空间表征 + +## 核心问题 + +> 机器如何"看懂"地理空间？ +> 栅格和矢量，谁更智能？还是"小孩子才做选择"？ + +--- + +## 概念讲解 + +### 空间表征的本质 + +空间表征是空间智能的基石——它解决的是**"如何用计算机能理解的方式描述空间"**这一根本问题。 + +``` +现实世界 → 空间表征 → 计算操作 + +───────────────────────────────────────────── + +真实景观 计算机表示 算法处理 + 🏔️ → 栅格/矢量 → 分析/推理 + 🌲 ↘ + 🏙️ 图表示 决策/优化 + 🛣️ +``` + +### 表征类型对比 + +| 维度 | 栅格 (Raster) | 矢量 (Vector) | +|-----|-------------|-------------| +| **基本单元** | 像元 (Pixel/Cell) | 点、线、面 (Point, Line, Polygon) | +| **数据结构** | 规则网格 | 坐标序列 | +| **适用场景** | 连续场、表面分析 | 离散要素、边界精确 | +| **典型操作** | 邻域分析、代数运算 | 拓扑分析、几何运算 | +| **存储效率** | 与分辨率强相关 | 与复杂度相关 | +| **代表数据** | DEM、遥感影像 | 行政边界、道路网 | + +### 图表示 (Graph Representation) + +许多空间问题可以抽象为图： + +``` +空间 → 图的转换 + +生态网络场景 图表示 +───────────── ─────── +源地A ──廊道──→ 源地B 节点A ──边(weight=5)──→ 节点B + │ │ + └──廊道──→ 源地C └──边(weight=8)──→ 节点C + +道路网络 图表示 +交叉路口 节点 +道路路段 加权边 +``` + +**图表示的优势**： +- 将空间问题转化为成熟的图算法 +- 天然支持连通性、路径分析 +- 易于扩展（添加权重、方向） + +### 多尺度表征 + +空间智能必须处理多尺度问题： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 多尺度金字塔 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ Level 3 (1:1,000,000) ┌─────┐ │ +│ 区域尺度 │ A │ │ +│ └─────┘ │ +│ 一个像元 = 1km² │ +│ │ +│ Level 2 (1:100,000) ┌─────┬─────┐ │ +│ 景观尺度 │ A │ B │ │ +│ └─────┴─────┘ │ +│ 一个像元 = 100m × 100m │ +│ │ +│ Level 1 (1:10,000) ┌─┬─┬─┬─┬─┐ │ +│ 局地尺度 │A│A│A│B│B│ │ +│ └─┴─┴─┴─┴─┘ │ +│ 一个像元 = 10m × 10m │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**多尺度处理策略**： +- **尺度金字塔**：预先生成多分辨率版本 +- **自适应网格**：关键区域高分辨率，其他区域低分辨率 +- **层次聚类**：构建空间层次结构 + +### 空间索引 (Spatial Indexing) + +当数据量大时，空间索引是效率的关键： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 空间索引类型对比 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 1. R-Tree (常用) │ +│ ┌────────────────────┐ │ +│ │ ┌───┐ ┌────┐ │ 层次包围盒 │ +│ │ │ A│ │ B │ │ 适合：复杂几何体 │ +│ │ └───┘ └────┘ │ │ +│ └────────────────────┘ │ +│ │ +│ 2. Quadtree (栅格友好) │ +│ ┌─────────┬─────────┐ │ +│ │ ● │ │ 四叉递归分解 │ +│ ├─────────┼─────────┤ 适合：点数据、栅格 │ +│ │ ● │ ● │ │ +│ └─────────┴─────────┘ │ +│ │ +│ 3. Grid Index (简单高效) │ +│ ┌───┬───┬───┬───┐ │ +│ │ │ ● │ │ │ 规则网格分桶 │ +│ ├───┼───┼───┼───┤ 适合：均匀分布数据 │ +│ │ │ │ ● │ │ │ +│ └───┴───┴───┴───┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 设计原理 + +### 表征选择决策树 + +``` + 开始 + │ + ▼ + 数据是什么类型？ + ┌──────┴──────┐ + │ │ + 连续场/表面 离散要素 + (高程、温度) (边界、道路) + │ │ + ▼ ▼ + 栅格优先 需要精确边界？ + │ ┌──┴──┐ + │ │ │ + │ 是 否 + │ │ │ + │ ▼ ▼ + │ 矢量 栅格也可 + │ │ │ + │ ▼ │ + │ 需要拓扑分析？ + │ │ │ + │ ┌───┴───┐ │ + │ │ │ │ + │ 是 否 │ + │ │ │ │ + ▼ ▼ ▼ ▼ + 栅格 拓扑矢量 简单矢量 +``` + +### 混合表征策略 + +实践中，最佳方案往往是混合使用： + +```python +class HybridSpatialRepresentation: + """ + 混合空间表征 + + 设计理念：不同操作用最合适的表征 + """ + + def __init__(self, raster_resolution=30): + """ + Args: + raster_resolution: 栅格分辨率(米) + """ + self.raster_resolution = raster_resolution + self.vector_features = {} # 矢量要素 + self.raster_surfaces = {} # 栅格表面 + self.spatial_index = None # 空间索引 + + def add_vector_feature(self, feature_id, geometry, attributes): + """添加矢量要素（适合精确边界）""" + import shapely.geometry as geom + + self.vector_features[feature_id] = { + 'geometry': geometry if isinstance(geometry, geom.base.BaseGeometry) + else geom.shape(geometry), + 'attributes': attributes + } + + # 更新空间索引 + self._build_spatial_index() + + def add_raster_surface(self, surface_id, array, transform=None): + """添加栅格表面（适合连续场）""" + import numpy as np + + self.raster_surfaces[surface_id] = { + 'array': np.asarray(array), + 'transform': transform, + 'resolution': self.raster_resolution + } + + def _build_spatial_index(self): + """构建R-Tree空间索引""" + from rtree import index + + idx = index.Index() + for i, (fid, feature) in enumerate(self.vector_features.items()): + idx.insert(i, feature['geometry'].bounds, fid) + + self.spatial_index = idx + + def query_by_location(self, point, buffer_distance=0): + """ + 基于位置查询 + + 使用矢量+索引：高效精确 + """ + from shapely.geometry import Point + + query_point = Point(point) if not isinstance(point, Point) else point + query_box = query_point.buffer(buffer_distance).bounds + + # 使用空间索引快速筛选 + candidates = [] + for i in self.spatial_index.intersection(query_box): + fid = list(self.vector_features.keys())[i] + feature = self.vector_features[fid] + + if feature['geometry'].intersects(query_point): + candidates.append({ + 'id': fid, + 'geometry': feature['geometry'], + 'attributes': feature['attributes'] + }) + + return candidates + + def extract_values_at_points(self, surface_id, points): + """ + 在点上提取栅格值 + + 使用栅格：快速采样 + """ + import numpy as np + + surface = self.raster_surfaces.get(surface_id) + if not surface: + raise ValueError(f"Surface {surface_id} not found") + + array = surface['array'] + values = [] + + for point in points: + # 坐标转换（如果有transform） + if surface['transform']: + # 这里简化处理，实际需要仿射变换 + col = int(point[0] / self.raster_resolution) + row = int(point[1] / self.raster_resolution) + else: + col, row = int(point[0]), int(point[1]) + + # 边界检查 + if 0 <= row < array.shape[0] and 0 <= col < array.shape[1]: + values.append(array[row, col]) + else: + values.append(np.nan) + + return values + + def vector_to_raster(self, feature_id, value_field=None, + surface_id=None, default_value=1): + """ + 矢量转栅格 + + 适用场景：需要栅格分析（如邻域、成本距离） + """ + import numpy as np + from rasterio.features import rasterize + + feature = self.vector_features.get(feature_id) + if not feature: + raise ValueError(f"Feature {feature_id} not found") + + # 确定输出范围 + geom = feature['geometry'] + bounds = geom.bounds + width = int((bounds[2] - bounds[0]) / self.raster_resolution) + 1 + height = int((bounds[3] - bounds[1]) / self.raster_resolution) + 1 + + # 确定栅格化值 + if value_field and value_field in feature['attributes']: + value = feature['attributes'][value_field] + else: + value = default_value + + # 栅格化 + output_array = rasterize( + [(geom, value)], + out_shape=(height, width), + transform=None, # 简化处理 + fill=0, + dtype=np.float32 + ) + + # 保存栅格 + new_surface_id = surface_id or f"{feature_id}_raster" + self.add_raster_surface(new_surface_id, output_array) + + return new_surface_id + + def to_graph(self, threshold_distance=None): + """ + 转换为图表示 + + 适用场景：连通性分析、路径优化 + """ + import networkx as nx + from shapely.geometry import Point + + G = nx.Graph() + + # 添加节点（矢量要素） + for fid, feature in self.vector_features.items(): + centroid = feature['geometry'].centroid + G.add_node(fid, pos=(centroid.x, centroid.y)) + + # 添加边（基于距离阈值） + if threshold_distance: + fids = list(self.vector_features.keys()) + for i, fid1 in enumerate(fids): + for fid2 in fids[i+1:]: + geom1 = self.vector_features[fid1]['geometry'] + geom2 = self.vector_features[fid2]['geometry'] + dist = geom1.distance(geom2) + + if dist <= threshold_distance: + G.add_edge(fid1, fid2, weight=dist) + + return G +``` + +### 设计权衡 + +| 决策维度 | 选项A | 选项B | 权衡考量 | +|---------|-------|-------|---------| +| 数据结构 | 栅格 | 矢量 | 精度 vs 效率；连续 vs 离散 | +| 分辨率 | 高精度 | 低精度 | 存储成本 vs 信息保留 | +| 索引方式 | R-Tree | Quadtree | 数据分布、查询类型 | +| 单一 vs 混合 | 统一表征 | 按需选择 | 一致性 vs 灵活性 | + +--- + +## 代码示例 + +### 示例1：栅格数据处理 + +```python +""" +栅格数据处理示例 +""" +import numpy as np +from typing import Tuple, List, Optional +from scipy.ndimage import convolve + +class RasterProcessor: + """栅格数据处理器""" + + def __init__(self, array: np.ndarray, resolution: float = 1.0): + """ + Args: + array: 栅格数据数组 + resolution: 分辨率(单位/像元) + """ + self.array = np.asarray(array) + self.resolution = resolution + self.nodata = -9999 # 无数据值 + + def slope(self) -> np.ndarray: + """ + 计算坡度 + + Returns: + 坡度数组(度) + """ + # Sobel算子 + kernel_x = np.array([[-1, 0, 1], + [-2, 0, 2], + [-1, 0, 1]]) + kernel_y = np.array([[-1, -2, -1], + [ 0, 0, 0], + [ 1, 2, 1]]) + + # 计算梯度 + dz_dx = convolve(self.array, kernel_x) / (8 * self.resolution) + dz_dy = convolve(self.array, kernel_y) / (8 * self.resolution) + + # 坡度(度) + slope = np.arctan(np.sqrt(dz_dx**2 + dz_dy**2)) * 180 / np.pi + + return slope + + def aspect(self) -> np.ndarray: + """ + 计算坡向 + + Returns: + 坡向数组(度, 0-360) + """ + kernel_x = np.array([[-1, 0, 1], + [-2, 0, 2], + [-1, 0, 1]]) + kernel_y = np.array([[-1, -2, -1], + [ 0, 0, 0], + [ 1, 2, 1]]) + + dz_dx = convolve(self.array, kernel_x) + dz_dy = convolve(self.array, kernel_y) + + aspect = np.arctan2(dz_dy, -dz_x) * 180 / np.pi + aspect = (90 - aspect) % 360 + + return aspect + + def neighborhood_stats(self, radius: int = 1) -> dict: + """ + 邻域统计 + + Args: + radius: 邻域半径(像元) + + Returns: + 统计指标字典 + """ + from scipy.ndimage import uniform_filter + + size = 2 * radius + 1 + + # 均值 + mean = uniform_filter(self.array.astype(float), size=size, mode='reflect') + + # 方差 + squared_mean = uniform_filter((self.array ** 2).astype(float), + size=size, mode='reflect') + variance = squared_mean - mean ** 2 + + # 最值 + from scipy.ndimage import maximum_filter, minimum_filter + maximum = maximum_filter(self.array, size=size, mode='reflect') + minimum = minimum_filter(self.array, size=size, mode='reflect') + + return { + 'mean': mean, + 'std': np.sqrt(np.maximum(variance, 0)), + 'max': maximum, + 'min': minimum, + 'range': maximum - minimum + } + + def resample(self, target_resolution: float, + method: str = 'bilinear') -> 'RasterProcessor': + """ + 重采样 + + Args: + target_resolution: 目标分辨率 + method: 'nearest', 'bilinear', 'cubic' + + Returns: + 新的RasterProcessor + """ + from scipy.ndimage import zoom + + scale_factor = self.resolution / target_resolution + + if method == 'nearest': + order = 0 + elif method == 'bilinear': + order = 1 + elif method == 'cubic': + order = 3 + else: + raise ValueError(f"Unknown method: {method}") + + resampled = zoom(self.array, scale_factor, order=order) + + return RasterProcessor(resampled, target_resolution) + +# 使用示例 +if __name__ == "__main__": + # 创建示例DEM + dem_data = np.array([ + [100, 105, 110, 108, 102], + [102, 108, 115, 112, 105], + [105, 112, 120, 118, 110], + [108, 115, 122, 120, 112], + [106, 110, 115, 112, 108] + ], dtype=float) + + processor = RasterProcessor(dem_data, resolution=30) + + # 计算坡度 + slope = processor.slope() + print(f"Average slope: {np.mean(slope):.2f} degrees") + + # 计算坡向 + aspect = processor.aspect() + + # 邻域统计 + stats = processor.neighborhood_stats(radius=1) + print(f"Smoothed elevation mean: {np.mean(stats['mean']):.2f}") +``` + +### 示例2：矢量数据处理 + +```python +""" +矢量数据处理示例 +""" +import numpy as np +from typing import List, Dict, Any, Optional +from shapely.geometry import Point, LineString, Polygon, MultiPolygon +from shapely.ops import unary_union, voronoi_diagram +import geopandas as gpd + +class VectorProcessor: + """矢量数据处理器""" + + def __init__(self, crs: str = "EPSG:4326"): + """ + Args: + crs: 坐标参考系统 + """ + self.crs = crs + self.features = [] + self.gdf = None + + def add_feature(self, geometry: Any, attributes: Dict[str, Any]): + """添加要素""" + # 确保是Shapely几何对象 + if isinstance(geometry, dict): + from shapely.geometry import shape + geometry = shape(geometry) + + self.features.append({ + 'geometry': geometry, + 'attributes': attributes + }) + + def to_geodataframe(self) -> gpd.GeoDataFrame: + """转换为GeoDataFrame""" + if not self.features: + return gpd.GeoDataFrame(geometry=[], crs=self.crs) + + data = { + 'geometry': [f['geometry'] for f in self.features], + **{k: [f['attributes'].get(k) for f in self.features] + for k in self.features[0]['attributes'].keys()} + } + + self.gdf = gpd.GeoDataFrame(data, crs=self.crs) + return self.gdf + + def buffer_all(self, distance: float, + resolution: int = 16) -> 'VectorProcessor': + """ + 缓冲区分析 + + Args: + distance: 缓冲距离(与CRS单位一致) + resolution: 缓冲圆弧的分辨率 + + Returns: + 新的VectorProcessor + """ + result = VectorProcessor(self.crs) + + for feature in self.features: + buffered = feature['geometry'].buffer( + distance, + resolution=resolution + ) + result.add_feature(buffered, feature['attributes']) + + return result + + def intersect_all(self, other: 'VectorProcessor') -> 'VectorProcessor': + """ + 相交分析 + + Args: + other: 另一个VectorProcessor + + Returns: + 相交结果的新VectorProcessor + """ + result = VectorProcessor(self.crs) + + for feat1 in self.features: + for feat2 in other.features: + intersection = feat1['geometry'].intersection(feat2['geometry']) + + if not intersection.is_empty: + # 合并属性 + merged_attrs = { + **{f"left_{k}": v for k, v in feat1['attributes'].items()}, + **{f"right_{k}": v for k, v in feat2['attributes'].items()} + } + result.add_feature(intersection, merged_attrs) + + return result + + def centroid(self) -> List[Point]: + """计算所有要素的质心""" + return [f['geometry'].centroid for f in self.features] + + def area(self) -> List[float]: + """计算所有面要素的面积""" + areas = [] + for f in self.features: + geom = f['geometry'] + if isinstance(geom, (Polygon, MultiPolygon)): + # 使用投影后的CRS计算更准确 + areas.append(geom.area) + else: + areas.append(0.0) + return areas + + def length(self) -> List[float]: + """计算所有线要素的长度""" + lengths = [] + for f in self.features: + geom = f['geometry'] + if isinstance(geom, (LineString, Point)): + lengths.append(geom.length) + else: + lengths.append(0.0) + return lengths + + def dissolve(self, by_attribute: Optional[str] = None) -> 'VectorProcessor': + """ + 融合要素 + + Args: + by_attribute: 按此属性分组融合 + + Returns: + 融合后的新VectorProcessor + """ + if not self.features: + return VectorProcessor(self.crs) + + if by_attribute: + # 按属性分组 + groups = {} + for f in self.features: + key = f['attributes'].get(by_attribute) + if key not in groups: + groups[key] = [] + groups[key].append(f['geometry']) + + result = VectorProcessor(self.crs) + for key, geometries in groups.items(): + dissolved = unary_union(geometries) + result.add_feature(dissolved, {by_attribute: key}) + + return result + else: + # 全部融合 + dissolved = unary_union([f['geometry'] for f in self.features]) + result = VectorProcessor(self.crs) + result.add_feature(dissolved, {}) + return result + +# 使用示例 +if __name__ == "__main__": + # 创建示例矢量数据 + processor = VectorProcessor(crs="EPSG:3857") # 投影坐标系，单位米 + + # 添加一些面要素 + processor.add_feature( + Polygon([(0, 0), (100, 0), (100, 100), (0, 100)]), + {'id': 1, 'type': 'forest'} + ) + processor.add_feature( + Polygon([(120, 20), (200, 20), (200, 120), (120, 120)]), + {'id': 2, 'type': 'forest'} + ) + processor.add_feature( + Polygon([(80, 80), (150, 80), (150, 150), (80, 150)]), + {'id': 3, 'type': 'wetland'} + ) + + # 转换为GeoDataFrame + gdf = processor.to_geodataframe() + print(f"Number of features: {len(gdf)}") + + # 缓冲区分析 + buffered = processor.buffer_all(distance=50) + print(f"Buffered features: {len(buffered.features)}") + + # 融合分析 + dissolved = processor.dissolve(by_attribute='type') + print(f"Dissolved groups: {len(dissolved.features)}") +``` + +### 示例3：图构建与分析 + +```python +""" +空间图表示示例 +""" +import numpy as np +import networkx as nx +from typing import List, Tuple, Dict, Optional +from shapely.geometry import Point, LineString + +class SpatialGraphBuilder: + """空间图构建器""" + + @staticmethod + def from_points(points: List[Point], + distance_threshold: float, + distance_type: str = 'euclidean') -> nx.Graph: + """ + 从点集构建图（基于距离阈值） + + Args: + points: 点列表 + distance_threshold: 连接阈值 + distance_type: 'euclidean' 或 'manhattan' + + Returns: + NetworkX图 + """ + G = nx.Graph() + + # 添加节点 + for i, point in enumerate(points): + G.add_node(i, pos=(point.x, point.y)) + + # 添加边 + for i in range(len(points)): + for j in range(i + 1, len(points)): + if distance_type == 'euclidean': + dist = points[i].distance(points[j]) + else: # manhattan + dist = abs(points[i].x - points[j].x) + \ + abs(points[i].y - points[j].y) + + if dist <= distance_threshold: + G.add_edge(i, j, weight=dist) + + return G + + @staticmethod + def from_polygons(polygons: List[Polygon], + connectivity_type: str = 'shared_boundary') -> nx.Graph: + """ + 从多边形构建图（基于拓扑关系） + + Args: + polygons: 多边形列表 + connectivity_type: 'shared_boundary' 或 'within_distance' + + Returns: + NetworkX图 + """ + G = nx.Graph() + + # 添加节点 + for i, poly in enumerate(polygons): + G.add_node(i, centroid=poly.centroid, area=poly.area) + + # 添加边 + for i in range(len(polygons)): + for j in range(i + 1, len(polygons)): + if connectivity_type == 'shared_boundary': + # 共享边界 + if polygons[i].touches(polygons[j]): + # 计算共享边界长度 + shared = polygons[i].intersection(polygons[j]) + weight = shared.length if not shared.is_empty else 0 + G.add_edge(i, j, weight=weight) + elif connectivity_type == 'within_distance': + # 距离阈值 + dist = polygons[i].distance(polygons[j]) + if dist >= 0 and dist <= 100: # 100米阈值 + G.add_edge(i, j, weight=dist) + + return G + + @staticmethod + def from_network(lines: List[LineString]) -> nx.Graph: + """ + 从线网络构建图（如道路网） + + Args: + lines: 线列表 + + Returns: + NetworkX图 + """ + G = nx.Graph() + + # 收集所有端点 + endpoints = [] + for i, line in enumerate(lines): + coords = list(line.coords) + endpoints.append((i, Point(coords[0]), 'start')) + endpoints.append((i, Point(coords[-1]), 'end')) + + # 构建节点（合并接近的端点） + tolerance = 1e-6 + node_id = 0 + point_to_node = {} + + for line_idx, point, _ in endpoints: + # 检查是否已有接近的节点 + matched = False + for existing_point, existing_node in point_to_node.items(): + if existing_point.distance(point) < tolerance: + point_to_node[point] = existing_node + matched = True + break + + if not matched: + point_to_node[point] = node_id + G.add_node(node_id, pos=(point.x, point.y)) + node_id += 1 + + # 添加边 + for i, line in enumerate(lines): + coords = list(line.coords) + start_point = Point(coords[0]) + end_point = Point(coords[-1]) + + start_node = point_to_node[start_point] + end_node = point_to_node[end_point] + + G.add_edge(start_node, end_node, + weight=line.length, + geometry=line, + edge_id=i) + + return G + + @staticmethod + def compute_connectivity_metrics(G: nx.Graph) -> Dict[str, float]: + """ + 计算图的连通性指标 + + Args: + G: NetworkX图 + + Returns: + 指标字典 + """ + metrics = {} + + # 基本指标 + metrics['n_nodes'] = G.number_of_nodes() + metrics['n_edges'] = G.number_of_edges() + + if G.number_of_nodes() == 0: + return metrics + + # 连通分量 + metrics['n_components'] = nx.number_connected_components(G) + + # 最大连通分量 + largest_cc = max(nx.connected_components(G), key=len) if G.nodes() else set() + metrics['largest_component_size'] = len(largest_cc) + metrics['largest_component_ratio'] = len(largest_cc) / G.number_of_nodes() + + # 平均度 + degrees = [d for n, d in G.degree()] + metrics['avg_degree'] = np.mean(degrees) if degrees else 0 + + # 网络密度 + metrics['density'] = nx.density(G) + + # 平均最短路径长度（仅当图连通时） + if nx.is_connected(G): + metrics['avg_path_length'] = nx.average_shortest_path_length(G) + metrics['diameter'] = nx.diameter(G) + else: + # 对最大连通分量计算 + if largest_cc: + subgraph = G.subgraph(largest_cc) + metrics['avg_path_length_lcc'] = nx.average_shortest_path_length(subgraph) + metrics['diameter_lcc'] = nx.diameter(subgraph) + + # 聚类系数 + metrics['avg_clustering'] = nx.average_clustering(G) + + return metrics + +# 使用示例 +if __name__ == "__main__": + # 示例1：从点构建图 + points = [ + Point(0, 0), Point(50, 0), Point(100, 0), + Point(0, 50), Point(50, 50), Point(100, 50), + Point(0, 100), Point(50, 100), Point(100, 100) + ] + + G_points = SpatialGraphBuilder.from_points(points, distance_threshold=80) + metrics = SpatialGraphBuilder.compute_connectivity_metrics(G_points) + print("Point Graph Metrics:", {k: round(v, 2) if isinstance(v, float) else v + for k, v in metrics.items()}) + + # 示例2：从多边形构建图 + from shapely.geometry import box + + polygons = [ + box(0, 0, 50, 50), + box(50, 0, 100, 50), + box(0, 50, 50, 100), + box(50, 50, 100, 100) + ] + + G_poly = SpatialGraphBuilder.from_polygons(polygons) + metrics_poly = SpatialGraphBuilder.compute_connectivity_metrics(G_poly) + print("Polygon Graph Metrics:", {k: round(v, 2) if isinstance(v, float) else v + for k, v in metrics_poly.items()}) +``` + +--- + +## 案例分析 + +### ENAgent中的混合表征策略 + +ENAgent（生态网络分析智能体）在处理生态网络时采用了混合表征策略： + +```python +class ENAgentSpatialManager: + """ + ENAgent的空间管理模块 + + 核心设计：不同数据类型用最合适的表征方式 + """ + + def __init__(self, raster_resolution=30): + # 矢量：生态源地（精确边界） + self.sources = VectorProcessor() + + # 栅格：阻力面（连续场） + self.resistance_surface = None + + # 图：生态网络（连通性分析） + self.network_graph = None + + def load_sources_from_vector(self, vector_file): + """从矢量文件加载源地""" + import geopandas as gpd + + gdf = gpd.read_file(vector_file) + + for _, row in gdf.iterrows(): + self.sources.add_feature( + row.geometry, + {'id': row.get('id', len(self.sources.features)), + 'name': row.get('name', ''), + 'area': row.geometry.area} + ) + + def create_resistance_surface(self, land_use_raster, resistance_dict): + """ + 创建阻力面（栅格） + + Args: + land_use_raster: 土地利用栅格 + resistance_dict: {土地类型: 阻力值} + """ + import numpy as np + + # 栅格计算：矢量化重映射 + land_use_array = self._read_raster(land_use_raster) + + # 创建阻力面 + resistance = np.zeros_like(land_use_array, dtype=float) + for land_type, resist_value in resistance_dict.items(): + resistance[land_use_array == land_type] = resist_value + + self.resistance_surface = resistance + + return resistance + + def build_network_graph(self, connectivity_threshold=5000): + """ + 构建生态网络图 + + 用于连通性分析和优化 + """ + # 从源地质心构建点集 + centroids = self.sources.centroid() + + # 构建图 + self.network_graph = SpatialGraphBuilder.from_points( + centroids, + distance_threshold=connectivity_threshold + ) + + return self.network_graph + + def analyze_connectivity(self): + """分析生态网络连通性""" + if self.network_graph is None: + self.build_network_graph() + + return SpatialGraphBuilder.compute_connectivity_metrics( + self.network_graph + ) + + def _read_raster(self, raster_file): + """读取栅格文件""" + import rasterio + + with rasterio.open(raster_file) as src: + return src.read(1) +``` + +**关键设计决策**： + +1. **源地用矢量**：需要精确边界和面积计算 +2. **阻力面用栅格**：需要邻域分析和成本距离计算 +3. **网络用图**：需要连通性分析和路径优化 + +这种混合策略充分发挥了各种表征的优势。 + +--- + +## 反思与延伸 + +### 思考问题 + +1. **尺度效应**：在不同分析尺度下，同一空间现象的表征会有什么变化？ + +2. **不确定性传播**：从一种表征转换到另一种（如矢量转栅格）时，不确定性如何传播？ + +3. **动态数据**：时变的空间数据应该如何表征？ + +4. **三维扩展**：如何将这些二维表征扩展到三维空间？ + +5. **存储与效率**：当数据量达到TB级别时，表征策略需要做什么调整？ + +### 延伸阅读 + +- **"Fundamentals of Geographic Information Systems"** (Demers) - 空间数据模型基础 +- **"Geographic Information Systems and Science"** (Longley) - 第3-4章 +- **"Spatial Databases"** (Rigaux) - 空间索引原理 +- Shapely Documentation - Python几何操作 +- Rasterio Documentation - Python栅格处理 + +--- + +## 关键要点 + +1. **空间表征是空间智能的基础**：选择合适的表征方式直接影响后续分析的效率和准确性 + +2. **栅格和矢量各有优势**：栅格适合连续场和邻域分析，矢量适合离散要素和精确边界 + +3. **图表示连接空间与算法**：将空间问题转化为图问题，可以应用丰富的图算法 + +4. **多尺度是现实需求**：空间智能系统必须能处理不同尺度的数据和分析 + +5. **混合策略往往是最佳选择**：ENAgent的实践表明，根据数据类型和操作需求选择表征方式是最有效的 diff --git a/officefile/supplements/02-spatial-intelligence/02.2-spatial-reasoning.md b/officefile/supplements/02-spatial-intelligence/02.2-spatial-reasoning.md new file mode 100644 index 0000000..8065520 --- /dev/null +++ b/officefile/supplements/02-spatial-intelligence/02.2-spatial-reasoning.md @@ -0,0 +1,544 @@ +# 02.2 空间推理 + +## 核心问题 + +> 机器如何理解和处理空间关系？ +> 图算法在空间分析中有哪些应用？ +> 如何进行连通性分析和路径优化？ + +--- + +## 概念讲解 + +### 空间关系类型 + +``` +空间关系分类 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 拓扑关系 │ +│ - 相邻 (Adjacent): A与B共享边界 │ +│ - 包含 (Contains): A完全包含B │ +│ - 重叠 (Overlaps): A与B部分重叠 │ +│ - 相离 (Disjoint): A与B不相交 │ +│ │ +│ 2. 距离关系 │ +│ - 欧氏距离: 直线距离 │ +│ - 曼哈顿距离: 城市街区距离 │ +│ - 阻力距离: 穿越不同地形的代价 │ +│ - 时间距离: 行驶时间成本 │ +│ │ +│ 3. 方向关系 │ +│ - 绝对方向: 北、南、东、西 │ +│ - 相对方向: 前、后、左、右 │ +│ - 方位角: 0-360度的精确方向 │ +│ │ +│ 4. 模式关系 │ +│ - 聚集: 要素密集分布 │ +│ - 离散: 要素分散分布 │ +│ - 随机: 要素随机分布 │ +│ - 规则: 要素有规律分布 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 图表示与空间推理 + +空间问题常可转换为图问题： + +``` +空间 → 图的转换 + +空间场景 图表示 +─────────── ─────── +源地A ──廊道──→ 源地B 节点A ──边──→ 节点B + │ │ + └──廊道──→ 源地C └──边──→ 节点C +``` + +**空间问题的图抽象**： + +| 空间问题 | 图表示 | 算法 | +|---------|--------|------| +| 最短路径 | 节点=位置，边=路径 | Dijkstra, A* | +| 连通性分析 | 节点=斑块，边=廊道 | BFS, DFS, 并查集 | +| 设施选址 | 节点=候选点，边=需求 | p-median, p-center | +| 覆盖问题 | 节点=服务点，边=覆盖范围 | 最大覆盖 | +| 网络流 | 节点=源/汇，边=管道 | 最大流最小割 | + +--- + +## 设计原理 + +### 连通性分析 + +连通性是生态网络分析的核心： + +```python +class ConnectivityAnalyzer: + """ + 连通性分析器 + + 核心：使用图算法分析空间连通性 + """ + + def __init__(self, resistance_surface): + """ + Args: + resistance_surface: 阻力面栅格 + """ + self.resistance = resistance_surface + self.graph = None + + def build_graph(self): + """将阻力面转换为图""" + import networkx as nx + + # 创建图 + self.graph = nx.Graph() + + rows, cols = self.resistance.shape + + # 添加节点和边 + for i in range(rows): + for j in range(cols): + node_id = i * cols + j + + # 添加节点 + self.graph.add_node(node_id, pos=(i, j)) + + # 添加边（8邻域） + for di in [-1, 0, 1]: + for dj in [-1, 0, 1]: + if di == 0 and dj == 0: + continue + + ni, nj = i + di, j + dj + if 0 <= ni < rows and 0 <= nj < cols: + neighbor_id = ni * cols + nj + + # 边权重 = 平均阻力 + weight = ( + self.resistance[i, j] + + self.resistance[ni, nj] + ) / 2 + + self.graph.add_edge( + node_id, neighbor_id, + weight=weight + ) + + return self.graph + + def least_cost_path(self, source, target): + """计算最小阻力路径""" + if self.graph is None: + self.build_graph() + + # Dijkstra算法 + path = nx.shortest_path( + self.graph, + source=source, + target=target, + weight='weight' + ) + + return path + + def connectivity_metrics(self, sources): + """ + 计算连通性指标 + + Args: + sources: 源地节点列表 + + Returns: + 连通性指标字典 + """ + if self.graph is None: + self.build_graph() + + metrics = {} + + # 1. 整体连通性 (图的连通分量数) + components = list(nx.connected_components( + self.graph.subgraph(sources) + )) + metrics['n_components'] = len(components) + + # 2. 最大连通分量大小 + if components: + metrics['largest_component'] = max(len(c) for c in components) + else: + metrics['largest_component'] = 0 + + # 3. 平均最短路径长度 + if len(sources) > 1: + subgraph = self.graph.subgraph(sources) + if nx.is_connected(subgraph): + metrics['avg_path_length'] = nx.average_shortest_path_length( + subgraph, weight='weight' + ) + else: + metrics['avg_path_length'] = float('inf') + + # 4. 网络密度 + n = len(sources) + if n > 1: + max_edges = n * (n - 1) / 2 + actual_edges = self.graph.subgraph(sources).number_of_edges() + metrics['density'] = actual_edges / max_edges + else: + metrics['density'] = 0 + + return metrics +``` + +### 最短路径算法 + +空间分析中最常用的图算法： + +```python +""" +最短路径算法比较 +""" +import heapq +from typing import Dict, List, Tuple, Set + +class ShortestPathAlgorithms: + """最短路径算法集合""" + + def __init__(self, graph: Dict): + """ + Args: + graph: {node: {neighbor: weight, ...}, ...} + """ + self.graph = graph + + def dijkstra(self, start: str, goal: str = None) -> Tuple[Dict, Dict]: + """ + Dijkstra算法：经典最短路径 + + 适合：非负权重图 + 复杂度：O((V+E)logV) + """ + # 优先队列：(距离, 节点) + pq = [(0, start)] + visited = set() + distances = {start: 0} + parents = {start: None} + + while pq: + current_dist, current = heapq.heappop(pq) + + if current in visited: + continue + visited.add(current) + + if current == goal: + break + + for neighbor, weight in self.graph.get(current, {}).items(): + if neighbor in visited: + continue + + new_dist = current_dist + weight + + if new_dist < distances.get(neighbor, float('inf')): + distances[neighbor] = new_dist + parents[neighbor] = current + heapq.heappush(pq, (new_dist, neighbor)) + + return distances, parents + + def reconstruct_path(self, parents: Dict, start: str, goal: str) -> List: + """从parents字典重建路径""" + path = [] + current = goal + + while current is not None: + path.append(current) + current = parents.get(current) + + path.reverse() + + if path[0] == start: + return path + return [] + + def a_star(self, start: str, goal: str, + heuristic: callable) -> Tuple[Dict, Dict]: + """ + A*算法：带启发式的最短路径 + + 适合：有目标节点的图，有可用启发式 + 复杂度：O(b^d) 实际通常比Dijkstra快 + """ + def h(node): + return heuristic(node, goal) + + # f(n) = g(n) + h(n) + pq = [(h(start), 0, start)] + visited = set() + g_score = {start: 0} # 实际距离 + parents = {start: None} + + while pq: + f, g, current = heapq.heappop(pq) + + if current in visited: + continue + visited.add(current) + + if current == goal: + break + + for neighbor, weight in self.graph.get(current, {}).items(): + if neighbor in visited: + continue + + tentative_g = g + weight + + if tentative_g < g_score.get(neighbor, float('inf')): + g_score[neighbor] = tentative_g + f_score = tentative_g + h(neighbor) + parents[neighbor] = current + heapq.heappush(pq, (f_score, tentative_g, neighbor)) + + return g_score, parents + +# 空间启发式函数 +def euclidean_heuristic(node_pos: Tuple, goal_pos: Tuple) -> float: + """欧氏距离启发式""" + import math + return math.sqrt( + (node_pos[0] - goal_pos[0])**2 + + (node_pos[1] - goal_pos[1])**2 + ) + +def manhattan_heuristic(node_pos: Tuple, goal_pos: Tuple) -> float: + """曼哈顿距离启发式（适合网格）""" + return abs(node_pos[0] - goal_pos[0]) + abs(node_pos[1] - goal_pos[1]) +``` + +--- + +## 代码示例 + +### 生态廊道识别 + +```python +""" +基于空间推理的生态廊道识别 +""" +import numpy as np +from typing import List, Tuple +import heapq + +def extract_corridors_mcr(resistance_surface: np.ndarray, + sources: List[Tuple[int, int]]) -> List[dict]: + """ + 使用最小累积阻力(MCR)方法提取生态廊道 + + Args: + resistance_surface: 阻力面栅格 + sources: 源地坐标列表 [(row, col), ...] + + Returns: + 廊道列表 + """ + rows, cols = resistance_surface.shape + + # 计算成本距离 + cost_distance = compute_cost_distance(resistance_surface, sources) + + # 提取廊道（低阻力通道） + corridors = [] + + for i, source1 in enumerate(sources): + for source2 in sources[i+1:]: + # 找到两源之间的最低阻力路径 + path = extract_lowest_resistance_path( + cost_distance, resistance_surface, source1, source2 + ) + + if path: + corridors.append({ + 'source_a': source1, + 'source_b': source2, + 'path': path, + 'cost': sum(resistance_surface[p] for p in path) + }) + + return corridors + +def compute_cost_distance(resistance: np.ndarray, + sources: List[Tuple[int, int]]) -> np.ndarray: + """ + 计算成本距离（到最近源地的累积阻力） + + 使用Dijkstra算法的变种 + """ + rows, cols = resistance.shape + cost = np.full((rows, cols), np.inf) + + # 优先队列：(累积成本, row, col) + pq = [] + + # 初始化源地 + for source_row, source_col in sources: + cost[source_row, source_col] = 0 + heapq.heappush(pq, (0, source_row, source_col)) + + # 8方向 + directions = [(-1, 0), (1, 0), (0, -1), (0, 1), + (-1, -1), (-1, 1), (1, -1), (1, 1)] + + visited = np.zeros((rows, cols), dtype=bool) + + while pq: + current_cost, row, col = heapq.heappop(pq) + + if visited[row, col]: + continue + visited[row, col] = True + + for dr, dc in directions: + nr, nc = row + dr, col + dc + + if 0 <= nr < rows and 0 <= nc < cols: + # 计算移动成本 + if dr != 0 and dc != 0: # 对角移动 + move_cost = resistance[nr, nc] * 1.414 + else: + move_cost = resistance[nr, nc] + + new_cost = current_cost + move_cost + + if new_cost < cost[nr, nc]: + cost[nr, nc] = new_cost + heapq.heappush(pq, (new_cost, nr, nc)) + + return cost + +def extract_lowest_resistance_path(cost_distance: np.ndarray, + resistance: np.ndarray, + start: Tuple[int, int], + end: Tuple[int, int]) -> List[Tuple[int, int]]: + """ + 从成本距离表面提取最低阻力路径 + """ + path = [end] + current = end + + while current != start: + row, col = current + best_neighbor = None + best_cost = cost_distance[current] + + # 检查邻域 + for dr in [-1, 0, 1]: + for dc in [-1, 0, 1]: + if dr == 0 and dc == 0: + continue + + nr, nc = row + dr, col + dc + if (0 <= nr < cost_distance.shape[0] and + 0 <= nc < cost_distance.shape[1]): + if cost_distance[nr, nc] < best_cost: + best_cost = cost_distance[nr, nc] + best_neighbor = (nr, nc) + + if best_neighbor is None: + break + + path.append(best_neighbor) + current = best_neighbor + + path.reverse() + return path if path[0] == start else [] +``` + +--- + +## 案例分析 + +### ENAgent中的廊道识别 + +ENAgent使用空间推理提取生态廊道： + +```python +class ENAgentCorridorExtractor: + """ENAgent的廊道提取模块""" + + def extract_corridors(self, mcr_surface, sources, width_threshold=500): + """ + 基于MCR表面提取廊道 + + Args: + mcr_surface: 最小累积阻力表面 + sources: 源地列表 + width_threshold: 廊道最小宽度 + + Returns: + 廊道字典 + """ + corridors = [] + + # 对每对源地提取路径 + for i in range(len(sources)): + for j in range(i + 1, len(sources)): + path = self._extract_path_between_sources( + mcr_surface, sources[i], sources[j] + ) + + if path: + # 分析廊道宽度 + width = self._calculate_corridor_width( + mcr_surface, path + ) + + if width >= width_threshold: + corridors.append({ + 'from': sources[i]['id'], + 'to': sources[j]['id'], + 'path': path, + 'width': width, + 'quality': self._assess_quality( + mcr_surface, path + ) + }) + + return corridors +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **算法选择**：什么时候用Dijkstra，什么时候用A*？ + +2. **空间尺度**：空间推理如何处理多尺度问题？ + +3. **计算效率**：大规模空间数据的图算法如何优化？ + +4. **动态变化**：空间环境变化时，如何高效更新推理结果？ + +### 延伸阅读 + +- **"Network Flows"** (Ahuja, Magnanti, Orlin) - 网络流理论 +- **"Geometric Algorithms"** - 几何算法 +- NetworkX文档 - Python图算法库 + +--- + +## 关键要点 + +1. **空间关系有四类**：拓扑、距离、方向、模式 +2. **图算法是空间推理的核心工具** +3. **连通性分析**使用图的结构特性 +4. **最短路径**有多个算法变种，各有适用场景 +5. **MCR分析**本质是图上的最短路径问题 diff --git a/officefile/supplements/02-spatial-intelligence/02.3-multi-criteria-decision.md b/officefile/supplements/02-spatial-intelligence/02.3-multi-criteria-decision.md new file mode 100644 index 0000000..3572b0f --- /dev/null +++ b/officefile/supplements/02-spatial-intelligence/02.3-multi-criteria-decision.md @@ -0,0 +1,1334 @@ +# 02.3 多准则决策 + +## 核心问题 + +> 当多个目标相互冲突时，如何做出"最优"决策？ +> 专家的判断经验如何转化为可计算的权重？ + +--- + +## 概念讲解 + +### 什么是多准则决策分析 (MCDA) + +多准则决策分析是一种在多个、通常是冲突的准则下评估和选择替代方案的方法论。 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 多准则决策问题的结构 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 决策目标 │ │ +│ │ "选择最适合生态修复的区域" │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 评估准则 │ │ +│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ +│ │ │ 生态重要性 │ │ 实施可行性 │ │ 成本效益 │ │ │ +│ │ │ (Weight) │ │ (Weight) │ │ (Weight) │ │ │ +│ │ │ 0.4 │ │ 0.3 │ │ 0.3 │ │ │ +│ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 替代方案 │ │ +│ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ │ +│ │ │ 区域A │ │ 区域B │ │ 区域C │ │ 区域D │ ... │ │ +│ │ └──────┘ └──────┘ └──────┘ └──────┘ │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 决策结果 │ │ +│ │ 综合得分 + 排名 + 稳健性分析 │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### MCDA的核心组成部分 + +| 组成部分 | 描述 | 空间应用示例 | +|---------|------|-------------| +| **准则 (Criteria)** | 评估标准，反映决策目标 | 生境质量、连通性、建设成本 | +| **权重 (Weights)** | 准则的相对重要性 | 生态重要性0.5，经济成本0.3，社会因素0.2 | +| **得分 (Scores)** | 各方案在各准则下的表现 | 每个栅格的生境适宜性指数 | +| **标准化 (Normalization)** | 将不同单位转换为可比尺度 | 0-1标准化、排名转换 | +| **集结规则 (Aggregation)** | 合并多准则得分的方法 | 加权求和、加权乘积、TOPSIS | + +### 常用的MCDA方法 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ MCDA方法分类 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 1. 加权线性组合 (WLC) / 简单加权法 │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ Score = Σ(weight_i × score_i) │ │ +│ │ │ │ +│ │ 优点：简单、直观、易于理解 │ │ +│ │ 缺点：允许补偿（一个准则的差可被另一个优弥补） │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 2. 层次分析法 (AHP) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ 通过成对比较确定权重 │ │ +│ │ │ │ +│ │ 优点：结构化、一致性检验 │ │ +│ │ 缺点：比较次数多(n(n-1)/2)、可能存在不一致 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 3. TOPSIS (逼近理想解排序法) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ 选择距正理想解最近、负理想解最远的方案 │ │ +│ │ │ │ +│ │ 优点：考虑方案与理想解的相对距离 │ │ +│ │ 缺点：对权重敏感 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 4. OWA (有序加权平均) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ 允许控制"风险态度"(ORness) │ │ +│ │ │ │ +│ │ 优点：灵活的风险偏好建模 │ │ +│ │ 缺点：需要确定orness参数 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 标准化方法 + +不同准则有不同的量纲，需要标准化： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 标准化方法 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 1. 最大-最小标准化 (Min-Max) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ x_norm = (x - min) / (max - min) │ │ +│ │ 适用：有明确最大最小值，线性关系 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 2. Z-score标准化 │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ x_norm = (x - mean) / std │ │ +│ │ 适用：正态分布数据，异常值敏感 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 3. 分位数转换 (Quantile) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ x_norm = percentile_rank(x) │ │ +│ │ 适用：分布未知，需要稳健性 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 4. 目标导向标准化 │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ 效益型: x_norm = x / target │ │ +│ │ 成本型: x_norm = target / x │ │ +│ │ 适用：有明确目标值 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 设计原理 + +### AHP层次分析法 + +AHP通过成对比较确定权重，具有结构化和一致性检验的优点： + +```python +import numpy as np +from typing import List, Dict, Tuple, Optional + +class AHPAnalyzer: + """ + 层次分析法 (Analytic Hierarchy Process) 实现 + + 核心思想：通过成对比较构建判断矩阵，计算权重 + """ + + # Saaty标度：1-9及其倒数 + SAATY_SCALE = { + 1: "同等重要", + 3: "稍微重要", + 5: "明显重要", + 7: "强烈重要", + 9: "极端重要", + 2: "介于1和3之间", + 4: "介于3和5之间", + 6: "介于5和7之间", + 8: "介于7和9之间" + } + + def __init__(self, criteria: List[str]): + """ + Args: + criteria: 准则列表 + """ + self.criteria = criteria + self.n = len(criteria) + self.comparison_matrix = None + self.weights = None + self.consistency_ratio = None + + def set_comparison_matrix(self, matrix: np.ndarray): + """ + 直接设置比较矩阵 + + Args: + matrix: n×n的判断矩阵，matrix[i,j]表示i相对于j的重要性 + """ + if matrix.shape != (self.n, self.n): + raise ValueError(f"矩阵大小应为{self.n}×{self.n}") + + # 确保对角线为1 + np.fill_diagonal(matrix, 1) + + self.comparison_matrix = matrix + + def build_from_pairs(self, pairs: Dict[Tuple[str, str], float]): + """ + 从成对比较构建矩阵 + + Args: + pairs: {(criterion_i, criterion_j): value} 表示i相对于j的重要性 + 如果j相对于i，则value应为1/value + """ + self.comparison_matrix = np.eye(self.n) + + criterion_to_idx = {c: i for i, c in enumerate(self.criteria)} + + for (ci, cj), value in pairs.items(): + i, j = criterion_to_idx[ci], criterion_to_idx[cj] + self.comparison_matrix[i, j] = value + self.comparison_matrix[j, i] = 1.0 / value + + def compute_weights(self, method: str = 'eigenvector') -> np.ndarray: + """ + 计算权重 + + Args: + method: 'eigenvector'(特征向量法) 或 'geometric'(几何平均法) + + Returns: + 权重向量 + """ + if self.comparison_matrix is None: + raise ValueError("请先设置比较矩阵") + + if method == 'eigenvector': + # 特征向量法：求最大特征值对应的特征向量 + eigenvalues, eigenvectors = np.linalg.eig(self.comparison_matrix) + max_idx = np.argmax(eigenvalues.real) + weights = eigenvectors[:, max_idx].real + # 归一化 + weights = weights / weights.sum() + + elif method == 'geometric': + # 几何平均法 + weights = np.exp(np.log(self.comparison_matrix).mean(axis=1)) + weights = weights / weights.sum() + + else: + raise ValueError(f"未知方法: {method}") + + self.weights = weights + return weights + + def compute_consistency(self) -> Dict[str, float]: + """ + 计算一致性指标 + + Returns: + 包含一致性相关指标的字典 + """ + if self.comparison_matrix is None or self.weights is None: + raise ValueError("请先设置比较矩阵并计算权重") + + # 计算最大特征值 + weighted_sum = self.comparison_matrix @ self.weights + lambda_max = (weighted_sum / self.weights).mean() + + # 一致性指标 CI + n = self.n + ci = (lambda_max - n) / (n - 1) if n > 1 else 0 + + # 随机一致性指标 RI (Saaty给出的标准值) + ri_table = {1: 0, 2: 0, 3: 0.58, 4: 0.90, 5: 1.12, + 6: 1.24, 7: 1.32, 8: 1.41, 9: 1.45, 10: 1.49} + ri = ri_table.get(n, 1.49) + + # 一致性比率 CR + cr = ci / ri if ri > 0 else 0 + + self.consistency_ratio = cr + + return { + 'lambda_max': lambda_max, + 'CI': ci, + 'RI': ri, + 'CR': cr, + 'consistent': cr < 0.1 + } + + def get_weights_dict(self) -> Dict[str, float]: + """返回准则-权重字典""" + if self.weights is None: + self.compute_weights() + return {c: w for c, w in zip(self.criteria, self.weights)} + +# 使用示例 +def example_ahp(): + """AHP使用示例：生态源地选址""" + criteria = ['生态价值', '连通性', '实施成本', '社会接受度'] + + ahp = AHPAnalyzer(criteria) + + # 设置成对比较（示例数据） + pairs = { + ('生态价值', '连通性'): 2, # 生态价值比连通性稍微重要 + ('生态价值', '实施成本'): 5, # 生态价值比成本明显重要 + ('生态价值', '社会接受度'): 3, # 生态价值比社会接受度稍微重要 + ('连通性', '实施成本'): 3, # 连通性比成本稍微重要 + ('连通性', '社会接受度'): 1, # 连通性与社会接受度同等重要 + ('实施成本', '社会接受度'): 1/2, # 社会接受度比成本稍微重要 + } + + ahp.build_from_pairs(pairs) + weights = ahp.compute_weights() + consistency = ahp.compute_consistency() + + print("准则权重:") + for c, w in ahp.get_weights_dict().items(): + print(f" {c}: {w:.3f}") + + print(f"\n一致性比率: {consistency['CR']:.3f}") + print(f"一致性{'合格' if consistency['consistent'] else '不合格'}") + + return ahp + +if __name__ == "__main__": + example_ahp() +``` + +### TOPSIS方法 + +TOPSIS通过计算与理想解的距离进行排序： + +```python +import numpy as np +from typing import List, Dict, Callable, Optional + +class TOPSISAnalyzer: + """ + TOPSIS (逼近理想解排序法) 实现 + + 核心思想：选择距离正理想解最近、负理想解最远的方案 + """ + + def __init__(self, + criteria: List[str], + directions: Optional[List[str]] = None): + """ + Args: + criteria: 准则列表 + directions: 每个准则的方向，'benefit'(效益型)或'cost'(成本型) + """ + self.criteria = criteria + self.directions = directions or ['benefit'] * len(criteria) + self.weights = None + self.normalized_matrix = None + self.ideal_positive = None + self.ideal_negative = None + self.scores = None + + def set_weights(self, weights: np.ndarray): + """设置权重""" + if len(weights) != len(self.criteria): + raise ValueError("权重数量与准则数量不匹配") + self.weights = np.array(weights) / np.sum(weights) + + def normalize(self, decision_matrix: np.ndarray) -> np.ndarray: + """ + 向量标准化 + + Args: + decision_matrix: m×n矩阵，m个方案，n个准则 + + Returns: + 标准化后的矩阵 + """ + # 向量标准化：每个列向量除以其范数 + norm = np.sqrt((decision_matrix ** 2).sum(axis=0)) + normalized = decision_matrix / norm + + # 处理除零 + normalized = np.nan_to_num(normalized) + + self.normalized_matrix = normalized + return normalized + + def compute_ideal_solutions(self, weighted_matrix: np.ndarray): + """ + 计算正理想解和负理想解 + + Args: + weighted_matrix: 加权标准化决策矩阵 + """ + n = weighted_matrix.shape[1] + + ideal_positive = np.zeros(n) + ideal_negative = np.zeros(n) + + for j in range(n): + if self.directions[j] == 'benefit': + ideal_positive[j] = weighted_matrix[:, j].max() + ideal_negative[j] = weighted_matrix[:, j].min() + else: # cost + ideal_positive[j] = weighted_matrix[:, j].min() + ideal_negative[j] = weighted_matrix[:, j].max() + + self.ideal_positive = ideal_positive + self.ideal_negative = ideal_negative + + return ideal_positive, ideal_negative + + def compute_scores(self, decision_matrix: np.ndarray) -> np.ndarray: + """ + 计算TOPSIS得分 + + Args: + decision_matrix: m×n决策矩阵 + + Returns: + 得分向量 (0-1之间，越大越好) + """ + # 标准化 + normalized = self.normalize(decision_matrix) + + # 加权 + if self.weights is None: + self.weights = np.ones(len(self.criteria)) / len(self.criteria) + + weighted = normalized * self.weights + + # 计算理想解 + self.compute_ideal_solutions(weighted) + + # 计算距离 + m = weighted.shape[0] + d_positive = np.zeros(m) + d_negative = np.zeros(m) + + for i in range(m): + d_positive[i] = np.sqrt( + ((weighted[i] - self.ideal_positive) ** 2).sum() + ) + d_negative[i] = np.sqrt( + ((weighted[i] - self.ideal_negative) ** 2).sum() + ) + + # 计算相对贴近度 + scores = d_negative / (d_positive + d_negative) + scores = np.nan_to_num(scores) + + self.scores = scores + return scores + + def rank(self, decision_matrix: np.ndarray) -> List[int]: + """ + 返回方案排名 + + Returns: + 排名索引列表（从最优到最差） + """ + scores = self.compute_scores(decision_matrix) + return np.argsort(-scores).tolist() + +# 使用示例 +def example_topsis(): + """TOPSIS使用示例：生态修复区域选择""" + # 准则 + criteria = ['生境适宜性', '连通性指数', '实施成本', '社会效益'] + directions = ['benefit', 'benefit', 'cost', 'benefit'] + + # 权重 + weights = [0.35, 0.25, 0.20, 0.20] + + # 决策矩阵：5个候选区域在4个准则下的得分 + # 注意：成本已经是负向的（数值越小越好） + decision_matrix = np.array([ + [0.85, 0.72, 500, 0.65], # 区域A + [0.78, 0.85, 450, 0.70], # 区域B + [0.92, 0.68, 600, 0.55], # 区域C + [0.70, 0.90, 400, 0.80], # 区域D + [0.88, 0.75, 550, 0.60], # 区域E + ]) + + # 对成本型准则进行预处理（转为效益型） + # 成本越高，得分越低 + cost_col = decision_matrix[:, 2].copy() + decision_matrix[:, 2] = cost_col.max() - cost_col + + topsis = TOPSISAnalyzer(criteria, directions) + topsis.set_weights(weights) + scores = topsis.compute_scores(decision_matrix) + ranking = topsis.rank(decision_matrix) + + print("区域排名 (TOPSIS):") + region_names = ['A', 'B', 'C', 'D', 'E'] + for rank, idx in enumerate(ranking, 1): + print(f" 第{rank}名: 区域{region_names[idx]} (得分: {scores[idx]:.3f})") + + return topsis + +if __name__ == "__main__": + example_topsis() +``` + +### 敏感性分析 + +敏感性分析是MCDA的重要组成部分，评估权重变化对决策结果的影响： + +```python +import numpy as np +from typing import List, Dict, Tuple, Callable +import matplotlib.pyplot as plt + +class SensitivityAnalyzer: + """ + 权重敏感性分析 + + 评估权重变化对决策结果的影响程度 + """ + + def __init__(self, + decision_matrix: np.ndarray, + weights: np.ndarray, + criteria: List[str], + directions: List[str] = None): + """ + Args: + decision_matrix: 决策矩阵 + weights: 初始权重 + criteria: 准则名称 + directions: 准则方向 + """ + self.decision_matrix = decision_matrix + self.base_weights = np.array(weights) / np.sum(weights) + self.criteria = criteria + self.directions = directions or ['benefit'] * len(criteria) + + def one_at_a_time(self, variation: float = 0.1) -> Dict[str, Dict]: + """ + 单因素敏感性分析 (OAT) + + 每次改变一个准则的权重，观察排名变化 + + Args: + variation: 权重变化幅度（相对于原权重的比例） + + Returns: + 每个准则变化时的排名变化 + """ + n = len(self.criteria) + base_ranking = self._compute_ranking(self.base_weights) + + results = {} + + for i, criterion in enumerate(self.criteria): + results[criterion] = { + 'weight_variations': [], + 'rankings': [] + } + + # 增加权重 + for delta in np.linspace(-variation, variation, 21): + new_weights = self.base_weights.copy() + + # 调整第i个权重 + new_weights[i] = self.base_weights[i] * (1 + delta) + + # 重新归一化（保持和为1） + if new_weights.sum() > 0: + new_weights = new_weights / new_weights.sum() + else: + new_weights = self.base_weights.copy() + + # 计算新排名 + ranking = self._compute_ranking(new_weights) + + results[criterion]['weight_variations'].append(delta) + results[criterion]['rankings'].append(ranking) + + return results + + def tornado_analysis(self, variation: float = 0.2) -> Dict: + """ + 龙卷风图分析 + + 计算每个准则权重变化对最优方案得分的影响 + + Args: + variation: 权重变化幅度 + + Returns: + 龙卷风图数据 + """ + from topsis import TOPSISAnalyzer + + base_scores = self._compute_scores(self.base_weights) + base_best_score = base_scores.max() + + impacts = [] + + for i, criterion in enumerate(self.criteria): + impact_high = 0 + impact_low = 0 + + # 增加权重 + for sign in [1, -1]: + new_weights = self.base_weights.copy() + new_weights[i] = self.base_weights[i] * (1 + sign * variation) + new_weights = new_weights / new_weights.sum() + + new_scores = self._compute_scores(new_weights) + new_best = new_scores.max() + + change = new_best - base_best_score + + if sign > 0: + impact_high = change + else: + impact_low = change + + impacts.append({ + 'criterion': criterion, + 'high': impact_high, + 'low': impact_low, + 'range': impact_high - impact_low + }) + + # 按影响范围排序 + impacts.sort(key=lambda x: abs(x['range']), reverse=True) + + return { + 'base_score': base_best_score, + 'impacts': impacts + } + + def monte_carlo(self, n_simulations: int = 1000, + weight_std: float = 0.1) -> Dict: + """ + 蒙特卡洛敏感性分析 + + 随机扰动权重，观察结果的统计分布 + + Args: + n_simulations: 模拟次数 + weight_std: 权重扰动的标准差 + + Returns: + 统计结果 + """ + n_alternatives = self.decision_matrix.shape[0] + rank_counts = np.zeros(n_alternatives, dtype=int) + scores_history = [] + + for _ in range(n_simulations): + # 生成随机权重 + random_weights = np.random.normal( + self.base_weights, + weight_std + ) + + # 确保非负并归一化 + random_weights = np.maximum(random_weights, 0) + random_weights = random_weights / random_weights.sum() + + # 计算排名 + ranking = self._compute_ranking(random_weights) + rank_counts[ranking[0]] += 1 # 统计第一名 + + # 记录得分 + scores = self._compute_scores(random_weights) + scores_history.append(scores) + + # 计算每个方案排第一的概率 + probabilities = rank_counts / n_simulations + + # 得分的统计量 + scores_array = np.array(scores_history) + scores_stats = { + 'mean': scores_array.mean(axis=0), + 'std': scores_array.std(axis=0), + 'min': scores_array.min(axis=0), + 'max': scores_array.max(axis=0), + 'percentile_5': np.percentile(scores_array, 5, axis=0), + 'percentile_95': np.percentile(scores_array, 95, axis=0) + } + + return { + 'rank_probabilities': probabilities, + 'scores_stats': scores_stats + } + + def _compute_scores(self, weights: np.ndarray) -> np.ndarray: + """计算给定权重下的得分""" + # 简单加权求和 + normalized = self._normalize_matrix(self.decision_matrix) + scores = (normalized * weights).sum(axis=1) + return scores + + def _compute_ranking(self, weights: np.ndarray) -> List[int]: + """计算给定权重下的排名""" + scores = self._compute_scores(weights) + return np.argsort(-scores).tolist() + + def _normalize_matrix(self, matrix: np.ndarray) -> np.ndarray: + """标准化决策矩阵""" + normalized = matrix.copy() + for j, direction in enumerate(self.directions): + col = matrix[:, j] + if direction == 'benefit': + normalized[:, j] = (col - col.min()) / (col.max() - col.min()) + else: # cost + normalized[:, j] = (col.max() - col) / (col.max() - col.min()) + return normalized + +# 使用示例 +def example_sensitivity(): + """敏感性分析示例""" + # 决策矩阵 + decision_matrix = np.array([ + [0.85, 0.72, 0.65], + [0.78, 0.85, 0.70], + [0.92, 0.68, 0.55], + [0.70, 0.90, 0.80], + ]) + + weights = [0.4, 0.35, 0.25] + criteria = ['生态价值', '连通性', '可行性'] + + analyzer = SensitivityAnalyzer( + decision_matrix, weights, criteria, + directions=['benefit', 'benefit', 'benefit'] + ) + + # 蒙特卡洛分析 + mc_results = analyzer.monte_carlo(n_simulations=1000, weight_std=0.1) + + print("各方案排名第一的概率:") + for i, prob in enumerate(mc_results['rank_probabilities']): + print(f" 方案{i+1}: {prob:.1%}") + + # 龙卷风分析 + tornado = analyzer.tornado_analysis(variation=0.2) + + print("\n准则重要性排序:") + for impact in tornado['impacts']: + print(f" {impact['criterion']}: 影响±{abs(impact['range']):.3f}") + + return analyzer +``` + +--- + +## 代码示例 + +### 空间多准则决策分析 + +```python +""" +空间多准则决策分析完整示例 +""" +import numpy as np +import matplotlib.pyplot as plt +from typing import List, Dict, Optional, Callable +from dataclasses import dataclass + +@dataclass +class MCDCriteria: + """决策准则""" + name: str + direction: str # 'benefit' 或 'cost' + weight: float + raster: Optional[np.ndarray] = None # 栅格数据 + normalize_func: Optional[Callable] = None # 标准化函数 + +class SpatialMCDA: + """ + 空间多准则决策分析 + + 用于栅格数据的多准则评估 + """ + + def __init__(self, shape: tuple, nodata: float = -9999): + """ + Args: + shape: 栅格形状 (rows, cols) + nodata: 无数据值 + """ + self.shape = shape + self.nodata = nodata + self.criteria: List[MCDCriteria] = [] + self.result: Optional[np.ndarray] = None + self.mask: Optional[np.ndarray] = None + + def add_criterion(self, name: str, direction: str, weight: float, + raster: Optional[np.ndarray] = None): + """ + 添加准则 + + Args: + name: 准则名称 + direction: 'benefit'(越大越好) 或 'cost'(越小越好) + weight: 权重 + raster: 栅格数据 + """ + criterion = MCDCriteria( + name=name, + direction=direction, + weight=weight, + raster=np.asarray(raster) if raster is not None else None + ) + self.criteria.append(criterion) + + def set_mask(self, mask: np.ndarray): + """ + 设置分析掩模 + + Args: + mask: True表示有效区域 + """ + self.mask = np.asarray(mask, dtype=bool) + + def normalize_all(self, method: str = 'minmax') -> List[np.ndarray]: + """ + 标准化所有准则 + + Args: + method: 'minmax', 'zscore', 或 'quantile' + + Returns: + 标准化后的栅格列表 + """ + normalized = [] + + for criterion in self.criteria: + if criterion.raster is None: + raise ValueError(f"准则 {criterion.name} 没有栅格数据") + + data = criterion.raster.copy() + + # 处理无数据值 + valid_mask = data != self.nodata + + if method == 'minmax': + # 最大-最小标准化 + valid_data = data[valid_mask] + min_val, max_val = valid_data.min(), valid_data.max() + + if criterion.direction == 'benefit': + data[valid_mask] = (valid_data - min_val) / (max_val - min_val) + else: # cost + data[valid_mask] = (max_val - valid_data) / (max_val - min_val) + + elif method == 'zscore': + # Z-score标准化 + valid_data = data[valid_mask] + mean_val, std_val = valid_data.mean(), valid_data.std() + + data[valid_mask] = (valid_data - mean_val) / std_val + + # 转换到0-1 + data[valid_mask] = (data[valid_mask] - data[valid_mask].min()) / \ + (data[valid_mask].max() - data[valid_mask].min()) + + elif method == 'quantile': + # 分位数转换 + valid_data = data[valid_mask] + ranks = np.argsort(np.argsort(valid_data)) + data[valid_mask] = ranks / (len(ranks) - 1) + + if criterion.direction == 'cost': + data[valid_mask] = 1 - data[valid_mask] + + else: + raise ValueError(f"未知方法: {method}") + + # 应用掩模 + if self.mask is not None: + data[~self.mask] = self.nodata + + normalized.append(data) + + return normalized + + def compute(self, method: str = 'wlc') -> np.ndarray: + """ + 计算综合评估结果 + + Args: + method: 'wlc'(加权线性组合) 或 'geometric'(几何平均) + + Returns: + 评估结果栅格 + """ + # 标准化 + normalized = self.normalize_all() + + # 归一化权重 + weights = np.array([c.weight for c in self.criteria]) + weights = weights / weights.sum() + + # 初始化结果 + result = np.zeros(self.shape) + + if method == 'wlc': + # 加权线性组合 + for i, norm_data in enumerate(normalized): + valid_mask = norm_data != self.nodata + result[valid_mask] += norm_data[valid_mask] * weights[i] + + elif method == 'geometric': + # 加权几何平均 + result.fill(1) + for i, norm_data in enumerate(normalized): + valid_mask = norm_data != self.nodata + result[valid_mask] *= np.power(norm_data[valid_mask], weights[i]) + + else: + raise ValueError(f"未知方法: {method}") + + # 应用掩模 + if self.mask is not None: + result[~self.mask] = self.nodata + + self.result = result + return result + + def get_rankings(self, n_classes: int = 5) -> np.ndarray: + """ + 将连续结果分级 + + Args: + n_classes: 分类数 + + Returns: + 分级结果 (1=最低, n_classes=最高) + """ + if self.result is None: + self.compute() + + result = self.result.copy() + valid_mask = result != self.nodata + + # 分位数分级 + valid_data = result[valid_mask] + thresholds = np.percentile(valid_data, + np.linspace(0, 100, n_classes + 1)) + + rankings = np.zeros_like(result, dtype=int) + for i in range(n_classes): + mask = (result >= thresholds[i]) & (result < thresholds[i + 1]) + rankings[mask] = i + 1 + rankings[result >= thresholds[-1]] = n_classes + rankings[~valid_mask] = 0 + + return rankings + + def sensitivity_analysis(self, criterion_name: str, + weight_variation: float = 0.2) -> np.ndarray: + """ + 单准则权重敏感性分析 + + Args: + criterion_name: 要分析的准则名称 + weight_variation: 权重变化范围 (±) + + Returns: + 不同权重下的结果差异 + """ + base_result = self.compute() + variations = [] + + # 找到准则索引 + criterion_idx = None + for i, c in enumerate(self.criteria): + if c.name == criterion_name: + criterion_idx = i + break + + if criterion_idx is None: + raise ValueError(f"未找到准则: {criterion_name}") + + # 变化权重 + for delta in np.linspace(-weight_variation, weight_variation, 11): + # 修改权重 + original_weight = self.criteria[criterion_idx].weight + self.criteria[criterion_idx].weight = original_weight * (1 + delta) + + # 重新计算 + new_result = self.compute() + variations.append(new_result.copy()) + + # 恢复权重 + self.criteria[criterion_idx].weight = original_weight + + # 计算标准差作为敏感性度量 + variations = np.array(variations) + sensitivity = np.std(variations, axis=0) + + return sensitivity + +# 使用示例 +def example_spatial_mcda(): + """空间多准则决策分析示例""" + # 创建示例数据 + rows, cols = 100, 100 + mcda = SpatialMCDA(shape=(rows, cols)) + + # 创建掩模（例如：排除研究区外的区域） + mask = np.zeros((rows, cols), dtype=bool) + mask[20:80, 20:80] = True + mcda.set_mask(mask) + + # 添加准则 + np.random.seed(42) + + # 准则1：生境适宜性（效益型，权重0.4） + habitat = np.random.rand(rows, cols) * 0.6 + 0.2 + habitat[~mask] = mcda.nodata + mcda.add_criterion('生境适宜性', 'benefit', 0.4, habitat) + + # 准则2：连通性（效益型，权重0.3） + connectivity = np.random.rand(rows, cols) * 0.7 + 0.15 + connectivity[~mask] = mcda.nodata + mcda.add_criterion('连通性', 'benefit', 0.3, connectivity) + + # 准则3：实施成本（成本型，权重0.3） + cost = np.random.rand(rows, cols) * 0.8 + 0.1 + cost[~mask] = mcda.nodata + mcda.add_criterion('实施成本', 'cost', 0.3, cost) + + # 计算综合评估 + result = mcda.compute(method='wlc') + + print(f"综合评估结果:") + print(f" 有效区域均值: {result[mask].mean():.3f}") + print(f" 有效区域范围: [{result[mask].min():.3f}, {result[mask].max():.3f}]") + + # 分级 + rankings = mcda.get_rankings(n_classes=5) + print(f"\n分级统计:") + for i in range(1, 6): + count = (rankings == i).sum() + print(f" 等级{i}: {count} 个像元") + + # 敏感性分析 + sensitivity = mcda.sensitivity_analysis('生境适宜性', weight_variation=0.3) + print(f"\n敏感性分析 (生境适宜性权重±30%):") + print(f" 结果最大变化: {sensitivity[mask].max():.3f}") + print(f" 结果平均变化: {sensitivity[mask].mean():.3f}") + + return mcda + +if __name__ == "__main__": + example_spatial_mcda() +``` + +--- + +## 案例分析 + +### ENAgent中的生态系统服务评估 + +ENAgent使用多准则决策分析评估生态系统的综合服务价值： + +```python +class EcosystemServiceAssessment: + """ + 生态系统服务评估模块 + + 基于多准则决策分析评估区域生态系统服务价值 + """ + + # 生态系统服务类型 + SERVICE_TYPES = { + 'provisioning': '供给服务', # 食物、淡水、木材等 + 'regulating': '调节服务', # 气候调节、洪水调节等 + 'cultural': '文化服务', # 游憩、美学价值等 + 'supporting': '支持服务' # 土壤形成、营养循环等 + } + + def __init__(self, study_area_boundary): + """ + Args: + study_area_boundary: 研究区边界(Shapely Polygon) + """ + self.boundary = study_area_boundary + self.services = {} + self.weights = {} + self.assessment_result = None + + def add_service_layer(self, service_type: str, + value_layer: np.ndarray, + weight: float = 1.0): + """ + 添加生态系统服务图层 + + Args: + service_type: 服务类型 + value_layer: 价值评估栅格 + weight: 该服务的权重 + """ + self.services[service_type] = value_layer + self.weights[service_type] = weight + + def assess_habitat_quality(self, land_use_raster, + threat_layers: Dict[str, np.ndarray], + sensitivity_table: Dict) -> np.ndarray: + """ + 评估生境质量 (基于InVEST模型思想) + + Args: + land_use_raster: 土地利用栅格 + threat_layers: 威胁因子图层 {'name': array} + sensitivity_table: 土地类型对威胁的敏感性 + + Returns: + 生境质量指数栅格 + """ + rows, cols = land_use_raster.shape + habitat_quality = np.zeros((rows, cols)) + + # 计算退化程度 + degradation = np.zeros((rows, cols)) + + for threat_name, threat_layer in threat_layers.items(): + # 对每个威胁因子计算影响 + # 这里简化处理，实际需要考虑距离衰减等 + threat_impact = threat_layer * 0.5 # 简化权重 + + for land_type, sensitivity in sensitivity_table.items(): + mask = (land_use_raster == land_type) + # 获取该土地类型对当前威胁的敏感性 + sens = sensitivity.get(threat_name, 0.5) + degradation[mask] += threat_impact[mask] * sens + + # 退化程度归一化到0-1 + degradation = np.clip(degradation, 0, 1) + + # 计算生境质量 + for land_type in np.unique(land_use_raster): + mask = (land_use_raster == land_type) + # 生境适宜性 (简化: 林地=高, 建设用地=低) + habitat_suitability = self._get_habitat_suitability(land_type) + habitat_quality[mask] = habitat_suitability * (1 - degradation[mask]) + + return habitat_quality + + def assess_carbon_storage(self, land_use_raster, + carbon_table: Dict[int, Dict[str, float]]) -> np.ndarray: + """ + 评估碳储量 + + Args: + land_use_raster: 土地利用栅格 + carbon_table: {土地类型: {'above': x, 'below': y, 'soil': z}} + + Returns: + 碳储量栅格 + """ + carbon_storage = np.zeros_like(land_use_raster, dtype=float) + + for land_type, carbon_values in carbon_table.items(): + mask = (land_use_raster == land_type) + # 总碳储量 = 地上 + 地下 + 土壤 + total_carbon = (carbon_values.get('above', 0) + + carbon_values.get('below', 0) + + carbon_values.get('soil', 0)) + carbon_storage[mask] = total_carbon + + return carbon_storage + + def multi_service_assessment(self) -> np.ndarray: + """ + 多服务综合评估 + + Returns: + 综合生态系统服务指数 + """ + if not self.services: + raise ValueError("请先添加服务图层") + + # 归一化权重 + total_weight = sum(self.weights.values()) + + # 初始化结果 + shape = next(iter(self.services.values())).shape + result = np.zeros(shape) + + # 加权求和 + for service_type, layer in self.services.items(): + weight = self.weights[service_type] / total_weight + result += layer * weight + + self.assessment_result = result + return result + + def identify_priority_areas(self, threshold_percentile: float = 0.75) -> np.ndarray: + """ + 识别优先保护区域 + + Args: + threshold_percentile: 分位数阈值 + + Returns: + 优先区域布尔栅格 + """ + if self.assessment_result is None: + self.multi_service_assessment() + + threshold = np.percentile( + self.assessment_result[self.assessment_result > 0], + threshold_percentile * 100 + ) + + priority_areas = self.assessment_result >= threshold + return priority_areas + + def _get_habitat_suitability(self, land_type) -> float: + """获取土地类型的生境适宜性""" + # 简化版：根据土地类型返回适宜性 + suitability_map = { + 1: 1.0, # 森林 + 2: 0.8, # 灌木 + 3: 0.6, # 草地 + 4: 0.4, # 湿地 + 5: 0.2, # 耕地 + 6: 0.0, # 建设用地 + 7: 0.3 # 裸地 + } + return suitability_map.get(land_type, 0.5) + +# 使用示例 +def example_enagent_assessment(): + """ENAgent生态系统服务评估示例""" + from shapely.geometry import box + + # 创建研究区 + study_area = box(0, 0, 10000, 10000) + + # 创建评估器 + assessor = EcosystemServiceAssessment(study_area) + + # 模拟土地利用数据 + rows, cols = 100, 100 + np.random.seed(42) + land_use = np.random.choice([1, 2, 3, 4, 5, 6], size=(rows, cols), p=[0.3, 0.15, 0.2, 0.1, 0.15, 0.1]) + + # 评估生境质量 + threat_layers = { + 'roads': np.random.rand(rows, cols) * 0.8, + 'urban': np.random.rand(rows, cols) * 0.6 + } + + sensitivity = { + 1: {'roads': 0.3, 'urban': 0.8}, # 森林对城市扩张敏感 + 2: {'roads': 0.5, 'urban': 0.6}, + 3: {'roads': 0.7, 'urban': 0.4}, + 4: {'roads': 0.8, 'urban': 0.9}, + 5: {'roads': 0.2, 'urban': 0.1}, + 6: {'roads': 0.0, 'urban': 0.0} + } + + habitat_quality = assessor.assess_habitat_quality( + land_use, threat_layers, sensitivity + ) + + # 评估碳储量 + carbon_table = { + 1: {'above': 150, 'below': 40, 'soil': 100}, + 2: {'above': 60, 'below': 20, 'soil': 80}, + 3: {'above': 20, 'below': 50, 'soil': 100}, + 4: {'above': 80, 'below': 100, 'soil': 150}, + 5: {'above': 10, 'below': 10, 'soil': 80}, + 6: {'above': 0, 'below': 0, 'soil': 20} + } + + carbon_storage = assessor.assess_carbon_storage(land_use, carbon_table) + + # 添加服务图层 + assessor.add_service_layer('habitat_quality', habitat_quality, weight=0.5) + assessor.add_service_layer('carbon_storage', carbon_storage / 300, weight=0.3) + + # 水文调节服务（模拟） + water_regulation = np.random.rand(rows, cols) * 0.5 + 0.3 + assessor.add_service_layer('water_regulation', water_regulation, weight=0.2) + + # 综合评估 + result = assessor.multi_service_assessment() + + print("生态系统服务综合评估:") + print(f" 平均服务指数: {result.mean():.3f}") + print(f" 高服务区域 (>0.6): {(result > 0.6).sum()} 个像元") + + # 识别优先区域 + priority = assessor.identify_priority_areas(threshold_percentile=0.75) + print(f" 优先保护区域: {priority.sum()} 个像元 ({priority.sum()/priority.size*100:.1f}%)") + + return assessor + +if __name__ == "__main__": + example_enagent_assessment() +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **权重来源**：专家判断、文献参考、数据分析，哪种权重确定方式更可靠？ + +2. **准则独立性**：当准则之间存在相关性时，MCDA结果会怎样变化？ + +3. **不确定性**：除了权重，数据本身的不确定性如何在MCDA中考虑？ + +4. **阈值效应**：某些准则是否存在关键阈值？如何处理？ + +5. **空间异质性**：不同区域的权重是否应该不同？ + +### 延伸阅读 + +- **"Multi-Criteria Decision Analysis: Methods and Software"** - MCDA方法综述 +- **"Spatial Decision Support Systems"** - 空间决策支持系统 +- InVEST模型文档 - 生态系统服务评估实践 +- **"Decision Analysis"** (Howard) - 决策分析基础理论 + +--- + +## 关键要点 + +1. **MCDA处理多目标冲突**：通过系统化方法整合多个准则，支持复杂决策 + +2. **权重是核心**：权重的确定是MCDA的关键，需要专家知识或数据支持 + +3. **标准化必不可少**：不同量纲的准则必须标准化才能比较和合并 + +4. **敏感性分析很重要**：评估权重变化对结果的影响，增强决策稳健性 + +5. **与GIS结合才有意义**：空间MCDA将决策框架落实到地理空间 diff --git a/officefile/supplements/02-spatial-intelligence/02.4-spatial-optimization.md b/officefile/supplements/02-spatial-intelligence/02.4-spatial-optimization.md new file mode 100644 index 0000000..b99259f --- /dev/null +++ b/officefile/supplements/02-spatial-intelligence/02.4-spatial-optimization.md @@ -0,0 +1,1293 @@ +# 02.4 空间优化 + +## 核心问题 + +> 如何在无穷可能中找到"最优"的空间配置？ +> 当目标相互冲突时，什么是可接受的妥协解？ + +--- + +## 概念讲解 + +### 什么是空间优化 + +空间优化是在空间约束下寻找最优决策方案的过程： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 空间优化问题的结构 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 目标函数 │ │ +│ │ Objective Function = f(x, y, ...) │ │ +│ │ │ │ +│ │ 例: 最大化生态连通性 │ │ +│ │ Maximize Σ connectivity(patch_i, patch_j) │ │ +│ │ │ │ +│ │ 可能是: │ │ +│ │ - 单目标优化 (一个目标) │ │ +│ │ - 多目标优化 (多个目标，需权衡) │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 决策变量 │ │ +│ │ Decision Variables = X │ │ +│ │ │ │ +│ │ 例: 哪些位置建立生态廊道 │ │ +│ │ X = [0, 1, 0, 1, 1, ...] │ │ +│ │ (1=建设, 0=不建设) │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 约束条件 │ │ +│ │ Constraints = g(X) ≤ 0 │ │ +│ │ │ │ +│ │ 例: │ │ +│ │ - 预算约束: Σ cost ≤ budget │ │ +│ │ - 空间约束: 避开建设用地 │ │ +│ │ - 连通性约束: 每个源地至少连接一条廊道 │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 可行域 │ │ +│ │ Feasible Region │ │ +│ │ │ │ +│ │ 满足所有约束的解空间 │ │ +│ │ 在可行域内寻找使目标函数最优的解 │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 空间优化问题分类 + +| 问题类型 | 目标 | 典型应用 | 求解难度 | +|---------|------|---------|---------| +| **选址问题** | 选定最优位置 | 设施选址、生态源地识别 | NP-hard | +| **覆盖问题** | 覆盖最大需求 | 保护区设计、服务覆盖 | NP-hard | +| **分配问题** | 最优分配资源 | 土地利用分配 | NP-hard | +| **路径问题** | 最短/最优路径 | 廊道设计、路线规划 | P (单点对点) | +| **布局问题** | 优化空间布局 | 城市规划、景观设计 | NP-hard | +| **网络设计** | 优化网络结构 | 生态网络、交通网络 | NP-hard | + +### 求解方法谱系 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 优化求解方法 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 1. 精确算法 (Exact Methods) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 线性规划 (LP): 单纯形法、内点法 │ │ +│ │ - 整数规划 (IP): 分支定界、割平面 │ │ +│ │ - 动态规划 (DP): 最优子结构 │ │ +│ │ │ │ +│ │ 优点: 保证全局最优 │ │ +│ │ 缺点: 只适用于小规模问题 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 2. 启发式算法 (Heuristics) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 贪心算法: 每步选择局部最优 │ │ +│ │ - 构造式算法: 逐步构建解 │ │ +│ │ │ │ +│ │ 优点: 快速、简单 │ │ +│ │ 缺点: 不保证最优 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 3. 元启发式算法 (Metaheuristics) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 遗传算法 (GA): 模拟进化 │ │ +│ │ - 模拟退火 (SA): 模拟金属退火 │ │ +│ │ - 蚁群算法 (ACO): 模拟蚂蚁觅食 │ │ +│ │ - 粒子群优化 (PSO): 模拟鸟群 │ │ +│ │ │ │ +│ │ 优点: 可处理大规模、非线性问题 │ │ +│ │ 缺点: 参数敏感，不保证全局最优 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 常见空间优化模型 + +#### 1. p-中值问题 (p-Median Problem) +选择p个设施，使所有需求点到最近设施的距离之和最小。 + +``` +Minimize: Σ Σ demand_i × distance(i, j) × x(i,j) + i j∈selected + +Subject to: + - 选恰好p个设施: Σ y_j = p + j + - 每个需求点被服务: Σ x(i,j) = 1, ∀i + j + - 只有被选中的设施才能服务: x(i,j) ≤ y_j, ∀i,j +``` + +#### 2. 最大覆盖问题 (Maximal Covering Problem) +用p个设施覆盖尽可能多的需求。 + +``` +Maximize: Σ demand_i × y_i + i + +Subject to: + - 选恰好p个设施: Σ x_j = p + j + - 覆盖关系: y_i ≤ Σ x_j, ∀i (j在i的覆盖范围内) + j∈N(i) + - 选恰好p个: Σ x_j = p +``` + +#### 3. 生态廊道优化 +在预算约束下最大化生态连通性。 + +``` +Maximize: Σ connectivity_gain(c) × x_c + c∈candidates + +Subject to: + - 预算约束: Σ cost(c) × x_c ≤ budget + c∈candidates + - 连通性约束: 每个源地至少有一条廊道连接 +``` + +--- + +## 设计原理 + +### 遗传算法 (Genetic Algorithm) + +遗传算法模拟自然进化过程，是空间优化中最常用的元启发式方法： + +```python +import numpy as np +from typing import Callable, List, Tuple, Optional, Dict +import random + +class GeneticAlgorithm: + """ + 遗传算法实现 + + 核心思想：模拟自然选择、交叉、变异 + """ + + def __init__(self, + objective_func: Callable, + n_variables: int, + variable_type: str = 'binary', + bounds: Optional[Tuple] = None, + population_size: int = 100, + mutation_rate: float = 0.01, + crossover_rate: float = 0.8, + elite_size: int = 2): + """ + Args: + objective_func: 目标函数 (最小化) + n_variables: 决策变量数量 + variable_type: 'binary' 或 'continuous' + bounds: (min, max) 连续变量的边界 + population_size: 种群大小 + mutation_rate: 变异率 + crossover_rate: 交叉率 + elite_size: 精英保留数量 + """ + self.objective_func = objective_func + self.n_variables = n_variables + self.variable_type = variable_type + self.bounds = bounds or (0, 1) + self.population_size = population_size + self.mutation_rate = mutation_rate + self.crossover_rate = crossover_rate + self.elite_size = elite_size + + self.population = None + self.fitness = None + self.best_solution = None + self.best_fitness = float('inf') + self.history = [] + + def initialize(self): + """初始化种群""" + if self.variable_type == 'binary': + self.population = np.random.randint( + 0, 2, (self.population_size, self.n_variables) + ) + else: # continuous + self.population = np.random.uniform( + self.bounds[0], self.bounds[1], + (self.population_size, self.n_variables) + ) + + def evaluate(self): + """评估种群适应度""" + self.fitness = np.array([ + self.objective_func(individual) + for individual in self.population + ]) + + # 更新最优解 + best_idx = np.argmin(self.fitness) + if self.fitness[best_idx] < self.best_fitness: + self.best_fitness = self.fitness[best_idx] + self.best_solution = self.population[best_idx].copy() + + self.history.append(self.best_fitness) + + def selection(self, method: str = 'tournament') -> np.ndarray: + """ + 选择操作 + + Args: + method: 'tournament'(锦标赛) 或 'roulette'(轮盘赌) + """ + selected = [] + + if method == 'tournament': + tournament_size = 3 + for _ in range(self.population_size - self.elite_size): + # 随机选择tournament_size个个体 + candidates = np.random.choice( + self.population_size, tournament_size, replace=False + ) + # 选择适应度最好的 + winner = candidates[np.argmin(self.fitness[candidates])] + selected.append(self.population[winner].copy()) + + elif method == 'roulette': + # 转换为适应度（越小越好→越大越好） + fitness_values = self.fitness + if fitness_values.min() < 0: + fitness_values = fitness_values - fitness_values.min() + 1 + + # 归一化 + probs = 1 / fitness_values + probs = probs / probs.sum() + + for _ in range(self.population_size - self.elite_size): + idx = np.random.choice(self.population_size, p=probs) + selected.append(self.population[idx].copy()) + + return np.array(selected) + + def crossover(self, parent1: np.ndarray, parent2: np.ndarray) -> Tuple: + """ + 交叉操作 + + Args: + parent1, parent2: 父代个体 + + Returns: + 两个子代个体 + """ + if self.variable_type == 'binary': + # 单点交叉 + if np.random.random() < self.crossover_rate: + point = np.random.randint(1, self.n_variables) + child1 = np.concatenate([parent1[:point], parent2[point:]]) + child2 = np.concatenate([parent2[:point], parent1[point:]]) + else: + child1, child2 = parent1.copy(), parent2.copy() + + else: # continuous + # 模拟二进制交叉 (SBX) + if np.random.random() < self.crossover_rate: + eta = 2 # 分布指数 + u = np.random.random(self.n_variables) + + beta = np.where( + u <= 0.5, + (2 * u) ** (1 / (eta + 1)), + (1 / (2 * (1 - u))) ** (1 / (eta + 1)) + ) + + child1 = 0.5 * ((1 + beta) * parent1 + (1 - beta) * parent2) + child2 = 0.5 * ((1 - beta) * parent1 + (1 + beta) * parent2) + + # 边界处理 + child1 = np.clip(child1, self.bounds[0], self.bounds[1]) + child2 = np.clip(child2, self.bounds[0], self.bounds[1]) + else: + child1, child2 = parent1.copy(), parent2.copy() + + return child1, child2 + + def mutate(self, individual: np.ndarray) -> np.ndarray: + """ + 变异操作 + + Args: + individual: 个体 + + Returns: + 变异后的个体 + """ + mutated = individual.copy() + + if self.variable_type == 'binary': + # 位翻转变异 + mask = np.random.random(self.n_variables) < self.mutation_rate + mutated[mask] = 1 - mutated[mask] + + else: # continuous + # 多项式变异 + for i in range(self.n_variables): + if np.random.random() < self.mutation_rate: + delta = np.random.normal(0, 0.1 * (self.bounds[1] - self.bounds[0])) + mutated[i] = np.clip( + mutated[i] + delta, + self.bounds[0], self.bounds[1] + ) + + return mutated + + def evolve(self, n_generations: int) -> Dict: + """ + 进化指定代数 + + Args: + n_generations: 进化代数 + + Returns: + 结果字典 + """ + self.initialize() + + for generation in range(n_generations): + # 评估 + self.evaluate() + + # 精英保留 + elite_indices = np.argsort(self.fitness)[:self.elite_size] + elite = self.population[elite_indices].copy() + + # 选择 + selected = self.selection(method='tournament') + + # 交叉 + offspring = [] + for i in range(0, len(selected), 2): + if i + 1 < len(selected): + child1, child2 = self.crossover(selected[i], selected[i+1]) + offspring.extend([child1, child2]) + else: + offspring.append(selected[i]) + + # 变异 + offspring = np.array([self.mutate(ind) for ind in offspring]) + + # 组合精英和后代 + self.population = np.vstack([elite, offspring]) + + # 确保种群大小 + if len(self.population) > self.population_size: + self.population = self.population[:self.population_size] + + return { + 'best_solution': self.best_solution, + 'best_fitness': self.best_fitness, + 'history': self.history + } +``` + +### 模拟退火算法 (Simulated Annealing) + +模拟退火模拟金属冷却过程，能跳出局部最优： + +```python +import numpy as np +from typing import Callable, Tuple, Optional +import math + +class SimulatedAnnealing: + """ + 模拟退火算法 + + 核心思想：以概率接受劣解，避免陷入局部最优 + """ + + def __init__(self, + objective_func: Callable, + n_variables: int, + variable_type: str = 'binary', + bounds: Optional[Tuple] = None, + initial_temp: float = 1000, + cooling_rate: float = 0.95, + min_temp: float = 0.01): + """ + Args: + objective_func: 目标函数 (最小化) + n_variables: 决策变量数量 + variable_type: 'binary' 或 'continuous' + bounds: 连续变量边界 + initial_temp: 初始温度 + cooling_rate: 降温率 + min_temp: 最低温度 + """ + self.objective_func = objective_func + self.n_variables = n_variables + self.variable_type = variable_type + self.bounds = bounds or (0, 1) + self.initial_temp = initial_temp + self.cooling_rate = cooling_rate + self.min_temp = min_temp + + self.current_solution = None + self.current_fitness = None + self.best_solution = None + self.best_fitness = float('inf') + self.history = [] + + def initialize(self) -> np.ndarray: + """生成初始解""" + if self.variable_type == 'binary': + return np.random.randint(0, 2, self.n_variables) + else: + return np.random.uniform( + self.bounds[0], self.bounds[1], self.n_variables + ) + + def generate_neighbor(self, solution: np.ndarray) -> np.ndarray: + """生成邻域解""" + neighbor = solution.copy() + + if self.variable_type == 'binary': + # 随机翻转一位 + idx = np.random.randint(self.n_variables) + neighbor[idx] = 1 - neighbor[idx] + + else: # continuous + # 在随机维度添加小扰动 + idx = np.random.randint(self.n_variables) + delta = np.random.normal(0, 0.1 * (self.bounds[1] - self.bounds[0])) + neighbor[idx] = np.clip( + neighbor[idx] + delta, + self.bounds[0], self.bounds[1] + ) + + return neighbor + + def accept_probability(self, current_fitness: float, + new_fitness: float, temperature: float) -> float: + """ + 计算接受概率 (Metropolis准则) + + Args: + current_fitness: 当前解适应度 + new_fitness: 新解适应度 + temperature: 当前温度 + + Returns: + 接受概率 + """ + if new_fitness < current_fitness: + return 1.0 # 更优解，一定接受 + else: + # 劣解以概率接受 + return math.exp(-(new_fitness - current_fitness) / temperature) + + def optimize(self, max_iterations: int = 10000) -> Dict: + """ + 执行优化 + + Args: + max_iterations: 最大迭代次数 + + Returns: + 结果字典 + """ + # 初始化 + self.current_solution = self.initialize() + self.current_fitness = self.objective_func(self.current_solution) + self.best_solution = self.current_solution.copy() + self.best_fitness = self.current_fitness + + temperature = self.initial_temp + + for iteration in range(max_iterations): + # 生成邻域解 + neighbor = self.generate_neighbor(self.current_solution) + neighbor_fitness = self.objective_func(neighbor) + + # 决定是否接受 + prob = self.accept_probability( + self.current_fitness, neighbor_fitness, temperature + ) + + if np.random.random() < prob: + self.current_solution = neighbor + self.current_fitness = neighbor_fitness + + # 更新最优解 + if self.current_fitness < self.best_fitness: + self.best_fitness = self.current_fitness + self.best_solution = self.current_solution.copy() + + # 记录 + self.history.append(self.best_fitness) + + # 降温 + temperature *= self.cooling_rate + if temperature < self.min_temp: + break + + return { + 'best_solution': self.best_solution, + 'best_fitness': self.best_fitness, + 'history': self.history + } +``` + +### 贪心算法与构造式启发式 + +贪心算法简单快速，适合作为基准解： + +```python +import numpy as np +from typing import List, Callable, Tuple, Dict + +class GreedyOptimizer: + """ + 贪心优化器 + + 核心思想：每步选择局部最优 + """ + + def __init__(self, candidates: List, evaluate_func: Callable): + """ + Args: + candidates: 候选方案列表 + evaluate_func: 评估函数，返回目标值 + """ + self.candidates = candidates + self.evaluate_func = evaluate_func + self.selected = [] + self.history = [] + + def greedy_add(self, n_select: int, + constraint_func: Callable = None) -> Tuple[List, float]: + """ + 贪心添加策略 + + 每次选择能带来最大边际收益的候选 + + Args: + n_select: 选择数量 + constraint_func: 约束函数，返回True表示可行 + + Returns: + (选择的候选列表, 最终目标值) + """ + available = set(range(len(self.candidates))) + self.selected = [] + + for _ in range(n_select): + best_candidate = None + best_value = -float('inf') + + # 尝试每个可用候选 + for idx in list(available): + # 检查约束 + test_selection = self.selected + [idx] + if constraint_func and not constraint_func(test_selection): + continue + + # 评估 + value = self.evaluate_func(test_selection) + + if value > best_value: + best_value = value + best_candidate = idx + + if best_candidate is None: + break # 没有可行的候选 + + # 选择最好的 + self.selected.append(best_candidate) + available.remove(best_candidate) + self.history.append(best_value) + + return self.selected, best_value + + def greedy_remove(self, initial_solution: List, + n_remove: int) -> Tuple[List, float]: + """ + 贪心移除策略 + + 从初始解开始，每次移除损失最小的 + + Args: + initial_solution: 初始解（候选索引列表） + n_remove: 移除数量 + + Returns: + (剩余候选列表, 最终目标值) + """ + current = set(initial_solution) + self.selected = list(current) + + for _ in range(n_remove): + if len(current) <= 1: + break + + worst_candidate = None + min_loss = float('inf') + + initial_value = self.evaluate_func(list(current)) + + # 尝试移除每个候选 + for idx in list(current): + test_selection = current - {idx} + value = self.evaluate_func(list(test_selection)) + loss = initial_value - value + + if loss < min_loss: + min_loss = loss + worst_candidate = idx + + if worst_candidate is not None: + current.remove(worst_candidate) + self.selected = list(current) + self.history.append(self.evaluate_func(self.selected)) + + return self.selected, self.evaluate_func(self.selected) + + def adaptive_greedy(self, n_select: int, + constraint_func: Callable = None) -> Tuple[List, float]: + """ + 自适应贪心 + + 结合添加和移除策略，改进解质量 + + Args: + n_select: 目标选择数量 + constraint_func: 约束函数 + + Returns: + (选择的候选列表, 最终目标值) + """ + # 先用贪心添加 + selected, _ = self.greedy_add(n_select, constraint_func) + + # 尝试局部搜索改进 + improved = True + while improved: + improved = False + best_swap = None + best_value = self.evaluate_func(selected) + + # 尝试交换 + selected_set = set(selected) + available_set = set(range(len(self.candidates))) - selected_set + + for out_idx in selected: + for in_idx in available_set: + new_selection = [in_idx if x == out_idx else x + for x in selected] + + if constraint_func and not constraint_func(new_selection): + continue + + value = self.evaluate_func(new_selection) + if value > best_value: + best_value = value + best_swap = (out_idx, in_idx) + + if best_swap: + out_idx, in_idx = best_swap + selected = [in_idx if x == out_idx else x for x in selected] + improved = True + + return selected, best_value +``` + +--- + +## 代码示例 + +### 生态网络优化问题 + +```python +""" +生态网络优化：在预算约束下最大化连通性 +""" +import numpy as np +from typing import List, Tuple, Dict, Set +import networkx as nx + +class EcologicalNetworkOptimizer: + """ + 生态网络优化器 + + 目标：选择廊道建设方案，在预算约束下最大化生态连通性 + """ + + def __init__(self, + sources: List[Dict], + corridor_candidates: List[Dict], + budget: float): + """ + Args: + sources: 生态源地列表 [{'id': i, 'pos': (x, y), 'quality': q}, ...] + corridor_candidates: 候选廊道列表 + [{'from': i, 'to': j, 'cost': c, 'quality': q}, ...] + budget: 总预算 + """ + self.sources = sources + self.candidates = corridor_candidates + self.budget = budget + + self.n_sources = len(sources) + self.n_corridors = len(corridor_candidates) + + # 构建源地图 + self.source_map = {s['id']: i for i, s in enumerate(sources)} + + def build_graph(self, selected_corridors: List[int]) -> nx.Graph: + """ + 根据选中的廊道构建图 + + Args: + selected_corridors: 选中的廊道索引列表 + + Returns: + NetworkX图 + """ + G = nx.Graph() + + # 添加节点（源地） + for source in self.sources: + G.add_node( + source['id'], + pos=source['pos'], + quality=source.get('quality', 1.0) + ) + + # 添加边（廊道） + for idx in selected_corridors: + corridor = self.candidates[idx] + G.add_edge( + corridor['from'], + corridor['to'], + weight=corridor.get('quality', 1.0), + cost=corridor['cost'], + length=corridor.get('length', 1) + ) + + return G + + def evaluate_connectivity(self, selected: List[int]) -> float: + """ + 评估连通性（目标函数） + + 综合考虑： + 1. 连通源地数量 + 2. 最大连通分量大小 + 3. 网络平均最短路径 + + Args: + selected: 选中的廊道索引列表 + + Returns: + 连通性得分（越高越好） + """ + if not selected: + return 0.0 + + G = self.build_graph(selected) + + if G.number_of_nodes() == 0: + return 0.0 + + score = 0.0 + + # 1. 连通源地数 + connected_sources = len([n for n in G.nodes() + if G.degree(n) > 0]) + score += connected_sources / self.n_sources * 0.4 + + # 2. 最大连通分量 + if G.number_of_edges() > 0: + largest_cc = max(len(cc) for cc in nx.connected_components(G)) + score += largest_cc / self.n_sources * 0.4 + else: + score += 0.0 + + # 3. 网络效率（仅当图连通时） + if nx.is_connected(G): + # 使用平均最短路径的倒数（越短越好） + avg_path = nx.average_shortest_path_length(G, weight='weight') + efficiency = 1.0 / (1.0 + avg_path) + score += efficiency * 0.2 + + return score + + def check_budget(self, selected: List[int]) -> bool: + """检查是否满足预算约束""" + total_cost = sum(self.candidates[i]['cost'] for i in selected) + return total_cost <= self.budget + + def greedy_solve(self) -> Tuple[List[int], float]: + """ + 贪心算法求解 + + Returns: + (选中的廊道索引列表, 得分) + """ + # 按性价比排序 + candidates_with_idx = [ + (i, c['cost'], c.get('quality', 1.0) / max(c['cost'], 1)) + for i, c in enumerate(self.candidates) + ] + candidates_with_idx.sort(key=lambda x: -x[2]) # 按性价比降序 + + selected = [] + remaining_budget = self.budget + + for idx, cost, _ in candidates_with_idx: + if cost <= remaining_budget: + selected.append(idx) + remaining_budget -= cost + + # 评估 + score = self.evaluate_connectivity(selected) + + return selected, score + + def genetic_solve(self, + population_size: int = 100, + n_generations: int = 200) -> Tuple[List[int], float]: + """ + 遗传算法求解 + + Returns: + (选中的廊道索引列表, 得分) + """ + def objective_func(individual): + """目标函数（最小化，所以取负）""" + if not self.check_budget(individual): + return 1e6 # 惩罚不可行解 + return -self.evaluate_connectivity(individual) + + ga = GeneticAlgorithm( + objective_func=objective_func, + n_variables=self.n_corridors, + variable_type='binary', + population_size=population_size, + mutation_rate=0.02, + crossover_rate=0.8, + elite_size=5 + ) + + result = ga.evolve(n_generations) + + selected = [i for i, val in enumerate(result['best_solution']) + if val == 1] + + score = self.evaluate_connectivity(selected) + + return selected, score + + def simulated_annealing_solve(self, + max_iterations: int = 10000) -> Tuple[List[int], float]: + """ + 模拟退火求解 + + Returns: + (选中的廊道索引列表, 得分) + """ + def objective_func(individual): + """目标函数（最小化）""" + if not self.check_budget([i for i, val in enumerate(individual) if val == 1]): + return 1e6 + return -self.evaluate_connectivity([i for i, val in enumerate(individual) if val == 1]) + + sa = SimulatedAnnealing( + objective_func=objective_func, + n_variables=self.n_corridors, + variable_type='binary', + initial_temp=100, + cooling_rate=0.995, + min_temp=0.01 + ) + + result = sa.optimize(max_iterations) + + selected = [i for i, val in enumerate(result['best_solution']) + if val == 1] + + score = self.evaluate_connectivity(selected) + + return selected, score + + def compare_methods(self) -> Dict: + """ + 比较不同求解方法 + + Returns: + 各方法的结果 + """ + results = {} + + print("Running Greedy...") + greedy_selected, greedy_score = self.greedy_solve() + results['greedy'] = { + 'selected': greedy_selected, + 'score': greedy_score, + 'cost': sum(self.candidates[i]['cost'] for i in greedy_selected) + } + print(f" Greedy: {len(greedy_selected)} corridors, score={greedy_score:.3f}") + + print("Running Genetic Algorithm...") + ga_selected, ga_score = self.genetic_solve(population_size=50, n_generations=100) + results['genetic'] = { + 'selected': ga_selected, + 'score': ga_score, + 'cost': sum(self.candidates[i]['cost'] for i in ga_selected) + } + print(f" GA: {len(ga_selected)} corridors, score={ga_score:.3f}") + + print("Running Simulated Annealing...") + sa_selected, sa_score = self.simulated_annealing_solve(max_iterations=5000) + results['simulated_annealing'] = { + 'selected': sa_selected, + 'score': sa_score, + 'cost': sum(self.candidates[i]['cost'] for i in sa_selected) + } + print(f" SA: {len(sa_selected)} corridors, score={sa_score:.3f}") + + return results + +# 使用示例 +def example_network_optimization(): + """生态网络优化示例""" + # 创建生态源地 + np.random.seed(42) + n_sources = 10 + sources = [ + { + 'id': i, + 'pos': (np.random.uniform(0, 100), np.random.uniform(0, 100)), + 'quality': np.random.uniform(0.5, 1.0) + } + for i in range(n_sources) + ] + + # 创建候选廊道（所有源地对之间的连线） + corridor_candidates = [] + for i in range(n_sources): + for j in range(i + 1, n_sources): + pos_i = sources[i]['pos'] + pos_j = sources[j]['pos'] + length = np.sqrt((pos_i[0] - pos_j[0])**2 + (pos_i[1] - pos_j[1])**2) + + corridor_candidates.append({ + 'from': sources[i]['id'], + 'to': sources[j]['id'], + 'cost': length * 10, # 成本与距离成正比 + 'quality': (sources[i]['quality'] + sources[j]['quality']) / 2, + 'length': length + }) + + # 设置预算 + total_cost_all = sum(c['cost'] for c in corridor_candidates) + budget = total_cost_all * 0.3 # 预算为总成本的30% + + # 创建优化器 + optimizer = EcologicalNetworkOptimizer(sources, corridor_candidates, budget) + + # 比较方法 + results = optimizer.compare_methods() + + # 找出最佳方法 + best_method = max(results.keys(), key=lambda k: results[k]['score']) + print(f"\nBest method: {best_method}") + print(f"Best score: {results[best_method]['score']:.3f}") + print(f"Corridors selected: {len(results[best_method]['selected'])}") + print(f"Budget used: {results[best_method]['cost']:.1f} / {budget:.1f}") + + return optimizer, results + +if __name__ == "__main__": + example_network_optimization() +``` + +--- + +## 案例分析 + +### ENAgent中的保护区优化 + +ENAgent使用空间优化算法设计最优的保护区网络： + +```python +class ReserveDesignOptimizer: + """ + 保护区设计优化器 + + 基于 Marxan 思想：用最小成本实现保护目标 + """ + + def __init__(self, + planning_units: np.ndarray, + features: Dict[str, np.ndarray], + cost_surface: np.ndarray, + targets: Dict[str, float]): + """ + Args: + planning_units: 规划单元（可以是栅格） + features: 各生态特征的分布 {'species': raster} + cost_surface: 每个单元的保护成本 + targets: 各特征的保护目标 {'species': proportion} + """ + self.planning_units = planning_units + self.features = features + self.cost_surface = cost_surface + self.targets = targets + + self.n_units = planning_units.size + + # 计算每个特征的现有数量 + self.feature_amounts = { + name: (raster > 0).sum() + for name, raster in features.items() + } + + def objective_function(self, solution: np.ndarray) -> float: + """ + 目标函数：最小化成本 + 惩罚未达目标 + + Args: + solution: 二进制解向量 + + Returns: + 目标值（越小越好） + """ + # 成本 + cost = (solution * self.cost_surface).sum() + + # 惩罚项 + penalty = 0 + penalty_factor = cost.sum() * 2 # 惩罚系数 + + for feature_name, raster in self.features.items(): + # 计算被保护的特征量 + protected_amount = (solution * (raster > 0)).sum() + target_amount = self.feature_amounts[feature_name] * self.targets[feature_name] + + if protected_amount < target_amount: + # 未达目标的惩罚 + shortfall = target_amount - protected_amount + penalty += shortfall * penalty_factor / self.feature_amounts[feature_name] + + # 边界长度惩罚（促进紧凑性） + boundary_penalty = self._compute_boundary_length(solution) * 0.1 + + return cost + penalty + boundary_penalty + + def _compute_boundary_length(self, solution: np.ndarray) -> float: + """计算边界长度（促进紧凑性）""" + solution_2d = solution.reshape(self.planning_units.shape) + + # 计算边界 + boundary = 0 + for i in range(solution_2d.shape[0]): + for j in range(solution_2d.shape[1]): + if solution_2d[i, j] == 1: + # 检查4邻域 + for di, dj in [(-1, 0), (1, 0), (0, -1), (0, 1)]: + ni, nj = i + di, j + dj + if 0 <= ni < solution_2d.shape[0] and 0 <= nj < solution_2d.shape[1]: + if solution_2d[ni, nj] == 0: + boundary += 1 + + return boundary + + def iterative_improvement(self, + initial_solution: np.ndarray, + max_iterations: int = 1000) -> np.ndarray: + """ + 迭代改进算法 + + Args: + initial_solution: 初始解 + max_iterations: 最大迭代次数 + + Returns: + 优化后的解 + """ + current = initial_solution.copy() + current_value = self.objective_function(current) + + for iteration in range(max_iterations): + improved = False + + # 尝试添加 + for i in np.random.permutation(self.n_units): + if current[i] == 0: + test = current.copy() + test[i] = 1 + test_value = self.objective_function(test) + + if test_value < current_value: + current = test + current_value = test_value + improved = True + break + + if not improved: + # 尝试移除 + for i in np.random.permutation(self.n_units): + if current[i] == 1: + test = current.copy() + test[i] = 0 + test_value = self.objective_function(test) + + if test_value < current_value: + current = test + current_value = test_value + improved = True + break + + if not improved: + break + + return current + + def solve(self, method: str = 'greedy') -> Tuple[np.ndarray, Dict]: + """ + 求解保护区设计问题 + + Args: + method: 'greedy', 'iterative', 或 'simulated_annealing' + + Returns: + (解, 结果信息) + """ + if method == 'greedy': + return self._greedy_solve() + elif method == 'iterative': + return self._iterative_solve() + elif method == 'simulated_annealing': + return self._sa_solve() + else: + raise ValueError(f"Unknown method: {method}") + + def _greedy_solve(self) -> Tuple[np.ndarray, Dict]: + """贪心求解""" + # 按性价比排序 + benefit_cost_ratio = np.zeros(self.n_units) + + for feature_name, raster in self.features.items(): + feature_present = (raster > 0).flatten() + benefit_cost_ratio += feature_present * self.cost_surface.flatten() + + benefit_cost_ratio = np.where( + benefit_cost_ratio > 0, + 1.0 / benefit_cost_ratio, + 0 + ) + + # 按性价比贪心选择 + order = np.argsort(-benefit_cost_ratio) + solution = np.zeros(self.n_units, dtype=int) + + current_cost = 0 + max_cost = self.cost_surface.sum() * 0.3 # 预算约束 + + for idx in order: + if benefit_cost_ratio[idx] > 0: + test_cost = current_cost + self.cost_surface.flatten()[idx] + if test_cost <= max_cost: + solution[idx] = 1 + current_cost = test_cost + + # 迭代改进 + solution = self.iterative_improvement(solution) + + return solution, { + 'cost': current_cost, + 'objective': self.objective_function(solution), + 'area_selected': solution.sum() + } + + def _iterative_solve(self) -> Tuple[np.ndarray, Dict]: + """迭代改进求解""" + # 从贪心解开始 + initial, _ = self._greedy_solve() + + solution = self.iterative_improvement(initial, max_iterations=1000) + + return solution, { + 'cost': (solution * self.cost_surface.flatten()).sum(), + 'objective': self.objective_function(solution), + 'area_selected': solution.sum() + } + + def _sa_solve(self) -> Tuple[np.ndarray, Dict]: + """模拟退火求解""" + def obj_func(x): + return self.objective_function(x) + + sa = SimulatedAnnealing( + objective_func=obj_func, + n_variables=self.n_units, + variable_type='binary', + initial_temp=1000, + cooling_rate=0.99, + min_temp=0.1 + ) + + result = sa.optimize(max_iterations=10000) + + return result['best_solution'], { + 'cost': (result['best_solution'] * self.cost_surface.flatten()).sum(), + 'objective': result['best_fitness'], + 'area_selected': result['best_solution'].sum(), + 'history': result['history'] + } +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **局部最优 vs 全局最优**：在空间优化中，局部最优解是否一定不可接受？ + +2. **计算效率**：当问题规模达到百万级别时，如何平衡解质量和计算时间？ + +3. **多目标权衡**：如何处理生态保护与经济发展的冲突？ + +4. **不确定性**：数据不确定性如何在优化中考虑？ + +5. **动态优化**：当环境条件变化时，如何更新优化解？ + +### 延伸阅读 + +- **"Metaheuristics in Spatial Optimization"** - 空间优化综述 +- **"Optimization Methods in GIS"** - GIS中的优化方法 +- Marxan documentation - 保护区设计经典工具 +- **"Integer Programming"** (Wolsey) - 整数规划理论 + +--- + +## 关键要点 + +1. **空间优化 = 目标 + 约束 + 变量**：清晰的数学建模是成功的关键 + +2. **没有万能算法**：不同问题需要不同的求解策略 + +3. **精确算法适用于小规模**：大规模问题必须用启发式 + +4. **元启发式需要参数调优**：遗传算法、模拟退火等需要仔细设置参数 + +5. **解的稳健性很重要**：敏感性分析验证优化结果的可靠性 diff --git a/officefile/supplements/02-spatial-intelligence/02.5-uncertainty-quantification.md b/officefile/supplements/02-spatial-intelligence/02.5-uncertainty-quantification.md new file mode 100644 index 0000000..5c3daae --- /dev/null +++ b/officefile/supplements/02-spatial-intelligence/02.5-uncertainty-quantification.md @@ -0,0 +1,1451 @@ +# 02.5 不确定性量化 + +## 核心问题 + +> 分析结果有多可信？不确定性的来源有哪些？ +> 如何在充满未知的世界中做出稳健的决策？ + +--- + +## 概念讲解 + +### 什么是不确定性 + +不确定性是指知识或信息的缺失，在空间分析中无处不在： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 不确定性的来源分类 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 1. 数据不确定性 (Data Uncertainty) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 测量误差: 仪器精度、人为操作 │ │ +│ │ - 采样偏差: 样本不代表总体 │ │ +│ │ - 空间插值: 从点到面的推断误差 │ │ +│ │ - 分类错误: 遥感解译错误 │ │ +│ │ - 过时数据: 数据不能反映当前状况 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 2. 模型不确定性 (Model Uncertainty) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 结构不确定性: 模型形式选择 │ │ +│ │ - 参数不确定性: 参数估计误差 │ │ +│ │ - 算法近似: 数值计算的近似 │ │ +│ │ - 尺度失配: 模型尺度与过程尺度不一致 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 3. 情境不确定性 (Scenario Uncertainty) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 未来不可预测: 气候变化、政策变动 │ │ +│ │ - 行为主体响应: 利益相关者的反应 │ │ +│ │ - 突发事件: 自然灾害、社会事件 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 不确定性的类型 + +| 类型 | 描述 | 处理方法 | +|-----|------|---------| +| **随机性 (Aleatory)** | 系统内在的随机变化，不可减少 | 概率分布、随机模拟 | +| **认知性 (Epistemic)** | 知识缺失导致的，可通过研究减少 | 灵敏度分析、更多数据 | +| **模糊性 (Vagueness)** | 概念边界不清晰 | 模糊集合、模糊逻辑 | +| **歧义性 (Ambiguity)** | 多种解释都合理 | 情景分析、鲁棒优化 | + +### 不确定性传播 + +当多个不确定输入通过模型组合时，不确定性会传播： + +``` +输入不确定性 ──→ 模型 ──→ 输出不确定性 + + ┌─────────┐ + x₁ ± Δx₁ ──→│ │ + x₂ ± Δx₂ ──→│ f(x) │──→ y ± Δy + x₃ ± Δx₃ ──→│ │ + └─────────┘ + +传播规则: +- 线性模型: Δy ≈ √(Σ(∂f/∂xi)² × Δxi²) (误差传播公式) +- 非线性模型: 需要蒙特卡洛模拟 +- 相关输入: 需要考虑协方差 +``` + +### 不确定性量化的方法谱系 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 不确定性量化方法 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ 1. 确定性敏感性分析 (Deterministic SA) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - OAT (One-at-a-Time): 单因素变化 │ │ +│ │ - 局部敏感性: 导数、弹性系数 │ │ +│ │ │ │ +│ │ 优点: 简单、直观 │ │ +│ │ 缺点: 忽略参数交互 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 2. 全局敏感性分析 (Global SA) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - Sobol指数: 方差分解 │ │ +│ │ - Morris筛选: 定性筛选重要参数 │ │ +│ │ - FAST: 傅里叶幅度敏感性测试 │ │ +│ │ │ │ +│ │ 优点: 考虑参数空间、交互作用 │ │ +│ │ 缺点: 计算成本高 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 3. 蒙特卡洛方法 (Monte Carlo) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 随机采样: 从输入分布采样 │ │ +│ │ - LHS: 拉丁超立方采样 │ │ +│ │ - 贝叶斯推断: 更新参数分布 │ │ +│ │ │ │ +│ │ 优点: 通用、易于实现 │ │ +│ │ 缺点: 收敛慢、高维困难 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +│ 4. 场景分析 (Scenario Analysis) │ +│ ┌─────────────────────────────────────────────────┐ │ +│ │ - 定义多个合理情景 │ │ +│ │ - 比较情景结果 │ │ +│ │ - 识别稳健策略 │ │ +│ │ │ │ +│ │ 优点: 直观、易于沟通 │ │ +│ │ 缺点: 情景选择主观 │ │ +│ └─────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 设计原理 + +### 蒙特卡洛模拟 + +蒙特卡洛是最通用的不确定性量化方法： + +```python +import numpy as np +from typing import Callable, List, Dict, Tuple, Optional +import matplotlib.pyplot as plt +from scipy import stats + +class MonteCarloSimulator: + """ + 蒙特卡洛模拟器 + + 核心思想：通过大量随机采样估计输出的概率分布 + """ + + def __init__(self, + model: Callable, + n_simulations: int = 10000, + random_seed: Optional[int] = None): + """ + Args: + model: 输入→输出的函数 + n_simulations: 模拟次数 + random_seed: 随机种子 + """ + self.model = model + self.n_simulations = n_simulations + self.random_seed = random_seed + + self.inputs = None + self.outputs = None + self.input_distributions = {} + + def define_input(self, name: str, distribution: str, **params): + """ + 定义输入变量的概率分布 + + Args: + name: 变量名 + distribution: 分布类型 ('normal', 'uniform', 'triangular', 'lognormal', 等) + **params: 分布参数 + """ + self.input_distributions[name] = { + 'type': distribution, + 'params': params + } + + def generate_inputs(self, method: str = 'random') -> np.ndarray: + """ + 生成输入样本 + + Args: + method: 'random' 或 'lhs'(拉丁超立方) + + Returns: + 输入样本数组 (n_simulations × n_variables) + """ + if self.random_seed is not None: + np.random.seed(self.random_seed) + + n_vars = len(self.input_distributions) + var_names = list(self.input_distributions.keys()) + + if method == 'random': + samples = np.zeros((self.n_simulations, n_vars)) + + for i, name in enumerate(var_names): + dist_info = self.input_distributions[name] + samples[:, i] = self._sample_distribution( + dist_info['type'], + dist_info['params'], + self.n_simulations + ) + + elif method == 'lhs': + # 拉丁超立方采样 + from scipy.stats import qmc + + sampler = qmc.LatinHypercube(d=n_vars, seed=self.random_seed) + sample_unit = sampler.random(n=self.n_simulations) + + samples = np.zeros_like(sample_unit) + + for i, name in enumerate(var_names): + dist_info = self.input_distributions[name] + samples[:, i] = self._transform_unit( + sample_unit[:, i], + dist_info['type'], + dist_info['params'] + ) + + self.inputs = samples + return samples + + def _sample_distribution(self, dist_type: str, + params: Dict, size: int) -> np.ndarray: + """从指定分布采样""" + if dist_type == 'normal': + return np.random.normal(params['mean'], params['std'], size) + elif dist_type == 'uniform': + return np.random.uniform(params['low'], params['high'], size) + elif dist_type == 'triangular': + return np.random.triangular( + params['left'], params['mode'], params['right'], size + ) + elif dist_type == 'lognormal': + return np.random.lognormal(params['mean'], params['sigma'], size) + elif dist_type == 'beta': + return np.random.beta(params['a'], params['b'], size) + else: + raise ValueError(f"Unknown distribution: {dist_type}") + + def _transform_unit(self, unit_samples: np.ndarray, + dist_type: str, params: Dict) -> np.ndarray: + """将[0,1]均匀分布转换为目标分布""" + if dist_type == 'normal': + return stats.norm.ppf(unit_samples, loc=params['mean'], scale=params['std']) + elif dist_type == 'uniform': + return params['low'] + unit_samples * (params['high'] - params['low']) + elif dist_type == 'triangular': + return stats.triang.ppf( + unit_samples, + c=(params['mode']-params['left'])/(params['right']-params['left']), + loc=params['left'], + scale=params['right']-params['left'] + ) + elif dist_type == 'lognormal': + return stats.lognorm.ppf( + unit_samples, + s=params['sigma'], + scale=np.exp(params['mean']) + ) + else: + raise ValueError(f"Unknown distribution: {dist_type}") + + def run(self, method: str = 'random') -> np.ndarray: + """ + 运行蒙特卡洛模拟 + + Returns: + 输出样本数组 + """ + # 生成输入 + if self.inputs is None: + self.generate_inputs(method) + + # 运行模型 + var_names = list(self.input_distributions.keys()) + self.outputs = np.zeros(self.n_simulations) + + for i in range(self.n_simulations): + input_dict = {name: self.inputs[i, j] + for j, name in enumerate(var_names)} + self.outputs[i] = self.model(**input_dict) + + return self.outputs + + def analyze_output(self) -> Dict: + """ + 分析输出分布 + + Returns: + 统计摘要 + """ + if self.outputs is None: + raise RuntimeError("请先运行模拟") + + output = self.outputs[~np.isnan(self.outputs)] # 移除NaN + + return { + 'mean': np.mean(output), + 'std': np.std(output), + 'median': np.median(output), + 'min': np.min(output), + 'max': np.max(output), + 'percentiles': { + '5': np.percentile(output, 5), + '25': np.percentile(output, 25), + '75': np.percentile(output, 75), + '95': np.percentile(output, 95) + }, + 'ci_95': (np.percentile(output, 2.5), np.percentile(output, 97.5)) + } + + def plot_output(self, bins: int = 50): + """绘制输出分布直方图""" + if self.outputs is None: + raise RuntimeError("请先运行模拟") + + output = self.outputs[~np.isnan(self.outputs)] + + fig, axes = plt.subplots(1, 2, figsize=(12, 4)) + + # 直方图 + axes[0].hist(output, bins=bins, density=True, alpha=0.7, edgecolor='black') + axes[0].axvline(np.mean(output), color='red', linestyle='--', label='Mean') + axes[0].axvline(np.median(output), color='green', linestyle='--', label='Median') + axes[0].set_xlabel('Output Value') + axes[0].set_ylabel('Probability Density') + axes[0].set_title('Output Distribution') + axes[0].legend() + + # 累积分布 + sorted_output = np.sort(output) + cumulative = np.arange(1, len(sorted_output) + 1) / len(sorted_output) + + axes[1].plot(sorted_output, cumulative, linewidth=2) + axes[1].axvline(np.percentile(output, 5), color='orange', linestyle='--', + label='5th percentile') + axes[1].axvline(np.percentile(output, 95), color='orange', linestyle='--', + label='95th percentile') + axes[1].set_xlabel('Output Value') + axes[1].set_ylabel('Cumulative Probability') + axes[1].set_title('Cumulative Distribution Function') + axes[1].legend() + + plt.tight_layout() + return fig + + def sensitivity_analysis(self) -> Dict[str, float]: + """ + 计算敏感性指标（基于相关性） + + Returns: + 各输入的敏感性系数 + """ + if self.inputs is None or self.outputs is None: + raise RuntimeError("请先运行模拟") + + var_names = list(self.input_distributions.keys()) + sensitivity = {} + + for i, name in enumerate(var_names): + # Spearman秩相关（对单调关系更稳健） + corr, _ = stats.spearmanr(self.inputs[:, i], self.outputs) + sensitivity[name] = abs(corr) + + # 归一化 + total = sum(sensitivity.values()) + if total > 0: + sensitivity = {k: v/total for k, v in sensitivity.items()} + + return sensitivity +``` + +### Sobol全局敏感性分析 + +Sobol指数基于方差分解，是全局敏感性分析的金标准： + +```python +import numpy as np +from typing import Callable, Dict, List +from scipy import stats + +class SobolAnalyzer: + """ + Sobol全局敏感性分析 + + 基于方差分解，计算一阶和高阶敏感性指数 + """ + + def __init__(self, + model: Callable, + n_vars: int, + bounds: List[Tuple[float, float]]): + """ + Args: + model: 输入→输出的函数 + n_vars: 输入变量数量 + bounds: 每个变量的边界 [(min, max), ...] + """ + self.model = model + self.n_vars = n_vars + self.bounds = bounds + + self.S1 = None # 一阶效应 + self.ST = None # 总效应 + + def generate_samples(self, N: int) -> Dict[str, np.ndarray]: + """ + 生成Sobol序列样本 + + 使用Saltelli采样方案 + + Args: + N: 基础样本数 + + Returns: + 包含A、B、AB矩阵的字典 + """ + # 生成两个基础样本矩阵 + A = np.zeros((N, self.n_vars)) + B = np.zeros((N, self.n_vars)) + + for i in range(self.n_vars): + A[:, i] = np.random.uniform(self.bounds[i][0], self.bounds[i][1], N) + B[:, i] = np.random.uniform(self.bounds[i][0], self.bounds[i][1], N) + + # 生成AB矩阵（每次替换一列） + AB = np.zeros((self.n_vars, N, self.n_vars)) + for i in range(self.n_vars): + AB[i] = A.copy() + AB[i][:, i] = B[:, i] + + return {'A': A, 'B': B, 'AB': AB} + + def compute(self, N: int) -> Dict[str, np.ndarray]: + """ + 计算Sobol指数 + + Args: + N: 基础样本数 + + Returns: + 包含S1和ST的字典 + """ + samples = self.generate_samples(N) + + # 计算模型输出 + fA = np.array([self.model(*x) for x in samples['A']]) + fB = np.array([self.model(*x) for x in samples['B']]) + + fAB = np.zeros((self.n_vars, N)) + for i in range(self.n_vars): + fAB[i] = np.array([self.model(*x) for x in samples['AB'][i]]) + + # 计算总方差 + all_outputs = np.concatenate([fA, fB]) + V = np.var(all_outputs) + + if V == 0: + raise ValueError("模型输出方差为0，无法计算敏感性") + + # 一阶敏感性 (S1) + S1 = np.zeros(self.n_vars) + for i in range(self.n_vars): + numerator = np.mean(fA * fAB[i]) - np.mean(fA) ** 2 + S1[i] = numerator / V + + # 总效应 (ST) + ST = np.zeros(self.n_vars) + for i in range(self.n_vars): + numerator = np.mean((fA - fAB[i]) ** 2) + ST[i] = numerator / (2 * V) + + self.S1 = np.maximum(S1, 0) # 确保非负 + self.ST = np.maximum(ST, 0) + + return {'S1': self.S1, 'ST': self.ST} + + def plot_sensitivity(self, var_names: List[str] = None): + """绘制敏感性指数图""" + if self.S1 is None or self.ST is None: + raise RuntimeError("请先运行计算") + + if var_names is None: + var_names = [f'X{i+1}' for i in range(self.n_vars)] + + fig, ax = plt.subplots(figsize=(10, 6)) + + x = np.arange(self.n_vars) + width = 0.35 + + ax.bar(x - width/2, self.S1, width, label='First Order (S1)', alpha=0.8) + ax.bar(x + width/2, self.ST, width, label='Total Effect (ST)', alpha=0.8) + + ax.set_xlabel('Input Variables') + ax.set_ylabel('Sensitivity Index') + ax.set_title('Sobol Sensitivity Indices') + ax.set_xticks(x) + ax.set_xticklabels(var_names, rotation=45) + ax.legend() + + plt.tight_layout() + return fig +``` + +### 不确定性可视化 + +空间不确定性需要特别的可视化方法： + +```python +import numpy as np +import matplotlib.pyplot as plt +from typing import List, Dict, Optional + +class UncertaintyVisualizer: + """ + 空间不确定性可视化 + """ + + @staticmethod + def probability_map(mean: np.ndarray, + threshold: float, + direction: str = 'above') -> np.ndarray: + """ + 创建概率地图 + + Args: + mean: 均值栅格 + threshold: 阈值 + direction: 'above' 或 'below' + + Returns: + 超过/低于阈值的概率（需要配合std） + """ + # 这里简化处理，实际需要蒙特卡洛结果 + if direction == 'above': + # 假设正态分布 + # 实际应使用MC结果的累积分布 + pass + return mean > threshold + + @staticmethod + def confidence_interval(mean: np.ndarray, + std: np.ndarray, + confidence: float = 0.95) -> Dict[str, np.ndarray]: + """ + 计算置信区间 + + Args: + mean: 均值栅格 + std: 标准差栅格 + confidence: 置信水平 + + Returns: + 包含下界和上界的字典 + """ + from scipy import stats + + z = stats.norm.ppf(1 - (1 - confidence) / 2) + + return { + 'lower': mean - z * std, + 'upper': mean + z * std, + 'margin': z * std + } + + @staticmethod + def uncertainty_classification(mean: np.ndarray, + std: np.ndarray, + n_classes: int = 5) -> np.ndarray: + """ + 基于均值和不确定性的分类 + + 结合期望值和不确定性进行决策分类 + + Args: + mean: 均值栅格 + std: 标准差栅格 + n_classes: 分类数 + + Returns: + 分类栅格 + """ + # 标准化 + mean_norm = (mean - mean.min()) / (mean.max() - mean.min()) + std_norm = (std - std.min()) / (std.max() - std.min() + 1e-10) + + # 决策分类 + # 高值+低不确定性 = 高优先级 + # 高值+高不确定性 = 需要更多信息 + # 低值+低不确定性 = 低优先级 + # 低值+高不确定性 = 不确定 + + classification = np.zeros_like(mean, dtype=int) + + # 定义阈值 + high_value = mean_norm > 0.6 + low_uncertainty = std_norm < 0.4 + + classification[high_value & low_uncertainty] = 5 # 高值，确定 + classification[high_value & ~low_uncertainty] = 4 # 高值，不确定 + classification[~high_value & low_uncertainty] = 2 # 低值，确定 + classification[~high_value & ~low_uncertainty] = 3 # 低值，不确定 + + return classification + + @staticmethod + def plot_with_uncertainty(mean: np.ndarray, + std: np.ndarray, + title: str = 'Value with Uncertainty'): + """ + 绘制带不确定性的地图 + + 使用颜色表示值，透明度表示不确定性 + """ + fig, axes = plt.subplots(1, 3, figsize=(15, 5)) + + # 均值图 + im1 = axes[0].imshow(mean, cmap='RdYlGn') + axes[0].set_title('Mean Value') + plt.colorbar(im1, ax=axes[0]) + + # 不确定性图 + im2 = axes[1].imshow(std, cmap='Oranges') + axes[1].set_title('Uncertainty (Std)') + plt.colorbar(im2, ax=axes[1]) + + # 组合图（值+不确定性） + # 归一化 + mean_norm = (mean - mean.min()) / (mean.max() - mean.min()) + std_norm = (std - std.min()) / (std.max() - std.min()) + + # 创建RGBA图像 + cmap = plt.cm.RdYlGn + rgba = cmap(mean_norm) + + # 用alpha通道表示不确定性（高不确定性=低透明度） + rgba[:, :, 3] = 1 - std_norm * 0.7 + + axes[2].imshow(rgba) + axes[2].set_title('Value (color) + Uncertainty (alpha)') + + plt.suptitle(title) + plt.tight_layout() + return fig +``` + +--- + +## 代码示例 + +### 阻力面不确定性分析 + +```python +""" +阻力面不确定性量化示例 + +ENAgent中阻力面的参数不确定性分析 +""" +import numpy as np +import matplotlib.pyplot as plt +from typing import Dict, List, Tuple, Callable + +class ResistanceUncertaintyAnalyzer: + """ + 阻力面不确定性分析器 + + 分析不同土地类型阻力值的不确定性对结果的影响 + """ + + def __init__(self, + land_use_raster: np.ndarray, + base_resistance_dict: Dict[int, float]): + """ + Args: + land_use_raster: 土地利用栅格 + base_resistance_dict: 基准阻力值 {土地类型: 阻力值} + """ + self.land_use = land_use_raster + self.base_resistance = base_resistance_dict + self.land_types = list(base_resistance_dict.keys()) + + # 阻力面 + self.base_surface = self._create_surface(base_resistance_dict) + + # 定义阻力值的先验分布 + self.resistance_distributions = self._define_distributions() + + def _create_surface(self, resistance_dict: Dict[int, float]) -> np.ndarray: + """根据阻力字典创建阻力面""" + surface = np.zeros_like(self.land_use, dtype=float) + for land_type, resistance in resistance_dict.items(): + surface[self.land_use == land_type] = resistance + return surface + + def _define_distributions(self) -> Dict[int, Dict]: + """ + 定义阻力值的概率分布 + + 假设阻力值服从对数正态分布 + """ + distributions = {} + for land_type, base_value in self.base_resistance.items(): + # 变异系数随阻力值增加 + cv = 0.3 if base_value > 50 else 0.2 + + # 对数正态分布参数 + # 如果X ~ Lognormal(μ, σ), 则 E[X] = exp(μ + σ²/2) + # Var[X] = (exp(σ²) - 1) * exp(2μ + σ²) + sigma2 = np.log(1 + cv**2) + mu = np.log(base_value) - sigma2 / 2 + + distributions[land_type] = { + 'type': 'lognormal', + 'mean': mu, + 'sigma': np.sqrt(sigma2), + 'base': base_value + } + + return distributions + + def sample_resistance_surface(self, n_samples: int = 1) -> List[np.ndarray]: + """ + 采样阻力面 + + Args: + n_samples: 采样次数 + + Returns: + 采样得到的阻力面列表 + """ + surfaces = [] + + for _ in range(n_samples): + resistance_dict = {} + for land_type, dist in self.resistance_distributions.items(): + # 从对数正态分布采样 + value = np.random.lognormal(dist['mean'], dist['sigma']) + resistance_dict[land_type] = value + + surfaces.append(self._create_surface(resistance_dict)) + + return surfaces + + def monte_carlo_connectivity(self, + sources: List[Tuple[int, int]], + n_samples: int = 100) -> Dict: + """ + 蒙特卡洛连通性分析 + + Args: + sources: 源地列表 + n_samples: 采样次数 + + Returns: + 统计结果 + """ + from connectivity import compute_cost_distance + + all_cost_distances = [] + all_paths = [] + + for i in range(n_samples): + # 采样阻力面 + surface = self.sample_resistance_surface(1)[0] + + # 计算成本距离 + cost_distance = compute_cost_distance(surface, sources) + + all_cost_distances.append(cost_distance) + + # 转换为数组 + all_cost_distances = np.array(all_cost_distances) + + # 统计分析 + mean_cost = np.mean(all_cost_distances, axis=0) + std_cost = np.std(all_cost_distances, axis=0) + cv_cost = std_cost / (mean_cost + 1e-10) # 变异系数 + + # 找出不确定性高的区域 + high_uncertainty_mask = cv_cost > cv_cost.mean() + cv_cost.std() + + return { + 'mean': mean_cost, + 'std': std_cost, + 'cv': cv_cost, + 'high_uncertainty': high_uncertainty_mask, + 'all_samples': all_cost_distances + } + + def sensitivity_to_resistance(self, + sources: List[Tuple[int, int]], + variation: float = 0.3) -> Dict: + """ + 阻力值敏感性分析 (OAT方法) + + 每次改变一个土地类型的阻力值 + + Args: + sources: 源地列表 + variation: 变化幅度 (±30%) + + Returns: + 敏感性结果 + """ + from connectivity import compute_cost_distance + + # 基准结果 + base_cost = compute_cost_distance(self.base_surface, sources) + + sensitivity = {} + + for land_type in self.land_types: + base_value = self.base_resistance[land_type] + + # 测试不同阻力值 + test_values = [ + base_value * (1 - variation), # 减少 + base_value * (1 + variation) # 增加 + ] + + results = [] + for test_value in test_values: + test_dict = self.base_resistance.copy() + test_dict[land_type] = test_value + test_surface = self._create_surface(test_dict) + test_cost = compute_cost_distance(test_surface, sources) + + # 计算与基准的差异 + diff = np.abs(test_cost - base_cost).mean() + results.append(diff) + + # 敏感性指标 = 平均绝对变化 + sensitivity[land_type] = np.mean(results) + + # 归一化 + total = sum(sensitivity.values()) + if total > 0: + sensitivity = {k: v/total for k, v in sensitivity.items()} + + return sensitivity + + def robust_corridor_selection(self, + sources: List[Tuple[int, int]], + cost_threshold: float, + n_samples: int = 100, + reliability: float = 0.8) -> np.ndarray: + """ + 稳健廊道选择 + + 只选择在多数采样中都满足阈值的像元 + + Args: + sources: 源地列表 + cost_threshold: 成本阈值 + n_samples: 采样次数 + reliability: 可靠性要求 (80%的采样满足) + + Returns: + 稳健廊道掩模 + """ + mc_results = self.monte_carlo_connectivity(sources, n_samples) + + # 计算每个像元在多少比例的采样中满足阈值 + satisfy_count = np.zeros_like(mc_results['mean']) + + for sample_cost in mc_results['all_samples']: + satisfy_count += (sample_cost < cost_threshold).astype(int) + + satisfy_ratio = satisfy_count / n_samples + + # 只选择满足可靠性要求的像元 + robust_corridor = satisfy_ratio >= reliability + + return robust_corridor + +# 使用示例 +def example_resistance_uncertainty(): + """阻力面不确定性分析示例""" + # 创建示例土地利用数据 + np.random.seed(42) + rows, cols = 50, 50 + + # 土地类型: 1=森林, 2=灌木, 3=草地, 4=农田, 5=建设用地, 6=水体 + land_use = np.random.choice( + [1, 2, 3, 4, 5, 6], + size=(rows, cols), + p=[0.25, 0.15, 0.20, 0.25, 0.10, 0.05] + ) + + # 基准阻力值 + base_resistance = { + 1: 10, # 森林 - 低阻力 + 2: 30, # 灌木 + 3: 50, # 草地 + 4: 80, # 农田 - 高阻力 + 5: 100, # 建设用地 - 最高阻力 + 6: 150 # 水体 - 障碍 + } + + # 创建分析器 + analyzer = ResistanceUncertaintyAnalyzer(land_use, base_resistance) + + # 源地 + sources = [(10, 10), (40, 40)] + + # 蒙特卡洛分析 + print("Running Monte Carlo analysis...") + mc_results = analyzer.monte_carlo_connectivity(sources, n_samples=50) + + print(f"Mean cost distance: {mc_results['mean'][sources[0]]:.2f}") + print(f"Std of cost distance: {mc_results['std'][sources[0]]:.2f}") + print(f"High uncertainty pixels: {mc_results['high_uncertainty'].sum()}") + + # 敏感性分析 + print("\nRunning sensitivity analysis...") + sensitivity = analyzer.sensitivity_to_resistance(sources, variation=0.3) + + print("Sensitivity to resistance values:") + land_type_names = {1: '森林', 2: '灌木', 3: '草地', 4: '农田', 5: '建设用地', 6: '水体'} + for land_type, sens in sorted(sensitivity.items(), key=lambda x: -x[1]): + print(f" {land_type_names[land_type]}: {sens:.3f}") + + # 稳健廊道选择 + print("\nSelecting robust corridors...") + threshold = np.percentile(mc_results['mean'], 60) + robust_corridor = analyzer.robust_corridor_selection( + sources, threshold, n_samples=50, reliability=0.7 + ) + + print(f"Robust corridor pixels: {robust_corridor.sum()}") + + return analyzer + +if __name__ == "__main__": + example_resistance_uncertainty() +``` + +### 情景分析框架 + +```python +""" +情景分析框架 + +处理情境不确定性 +""" +import numpy as np +from typing import Dict, List, Callable, Any +from dataclasses import dataclass +from enum import Enum + +class ScenarioType(Enum): + """情景类型""" + OPTIMISTIC = "乐观" + PESSIMISTIC = "悲观" + BUSINESS_AS_USUAL = "照常" + SUSTAINABLE = "可持续" + +@dataclass +class Scenario: + """情景定义""" + name: str + description: str + parameters: Dict[str, Any] + probability: float = 1.0 # 情景发生的概率 + +class ScenarioAnalyzer: + """ + 情景分析器 + + 通过定义多个合理情景来处理情境不确定性 + """ + + def __init__(self, model: Callable): + """ + Args: + model: 接受参数字典并返回结果的函数 + """ + self.model = model + self.scenarios: Dict[str, Scenario] = {} + self.results: Dict[str, Any] = {} + + def add_scenario(self, scenario_id: str, scenario: Scenario): + """添加情景""" + self.scenarios[scenario_id] = scenario + + def define_land_use_scenarios(self, + base_year: int, + target_year: int) -> Dict[str, Scenario]: + """ + 定义土地利用变化情景 + + Args: + base_year: 基准年 + target_year: 目标年 + + Returns: + 定义的情景字典 + """ + scenarios = { + 'bau': Scenario( + name='Business as Usual', + description='延续当前发展趋势', + parameters={ + 'urban_expansion_rate': 0.02, + 'forest_loss_rate': 0.01, + 'agriculture_intensity': 1.0 + }, + probability=0.5 + ), + 'optimistic': Scenario( + name='Optimistic', + description='生态保护加强，可持续发展', + parameters={ + 'urban_expansion_rate': 0.01, + 'forest_loss_rate': -0.005, # 森林恢复 + 'agriculture_intensity': 1.2 # 精准农业 + }, + probability=0.2 + ), + 'pessimistic': Scenario( + name='Pessimistic', + description='快速城市化，生态退化', + parameters={ + 'urban_expansion_rate': 0.04, + 'forest_loss_rate': 0.03, + 'agriculture_intensity': 0.8 + }, + probability=0.3 + ) + } + + for scenario_id, scenario in scenarios.items(): + self.add_scenario(scenario_id, scenario) + + return scenarios + + def run_scenarios(self) -> Dict[str, Any]: + """运行所有情景""" + self.results = {} + + for scenario_id, scenario in self.scenarios.items(): + try: + result = self.model(scenario.parameters) + self.results[scenario_id] = { + 'result': result, + 'scenario': scenario + } + except Exception as e: + self.results[scenario_id] = { + 'error': str(e), + 'scenario': scenario + } + + return self.results + + def compare_results(self, metric_extractor: Callable = None) -> Dict: + """ + 比较情景结果 + + Args: + metric_extractor: 从结果中提取比较指标的函数 + + Returns: + 比较结果 + """ + comparison = {} + + for scenario_id, result_data in self.results.items(): + if 'error' in result_data: + comparison[scenario_id] = {'error': result_data['error']} + else: + result = result_data['result'] + scenario = result_data['scenario'] + + if metric_extractor: + metrics = metric_extractor(result) + else: + # 默认: 直接使用结果 + metrics = {'value': result} + + comparison[scenario_id] = { + 'metrics': metrics, + 'probability': scenario.probability, + 'description': scenario.description + } + + return comparison + + def weighted_outcome(self, metric_name: str = 'value') -> float: + """ + 计算加权期望结果 + + Args: + metric_name: 要加权平均的指标名称 + + Returns: + 期望值 + """ + total = 0 + total_prob = 0 + + for scenario_id, result_data in self.results.items(): + if 'error' not in result_data: + scenario = result_data['scenario'] + result = result_data['result'] + + if isinstance(result, dict): + value = result.get(metric_name, 0) + else: + value = result + + total += value * scenario.probability + total_prob += scenario.probability + + return total / total_prob if total_prob > 0 else 0 + + def identify_robust_strategy(self, + strategies: Dict[str, Dict], + criterion: str = 'maximin') -> str: + """ + 识别稳健策略 + + Args: + strategies: 策略字典 {strategy_name: parameters} + criterion: 'maximin' (最大化最小收益) 或 'maximize_expected' + + Returns: + 最稳健的策略名称 + """ + strategy_outcomes = {} + + # 评估每个策略在各情景下的表现 + for strategy_name, strategy_params in strategies.items(): + outcomes = [] + + for scenario_id, scenario in self.scenarios.items(): + # 合并策略和情景参数 + combined_params = {**strategy_params, **scenario.parameters} + + try: + result = self.model(combined_params) + + if isinstance(result, dict): + value = result.get('value', result.get('score', 0)) + else: + value = result + + outcomes.append(value) + except: + outcomes.append(-float('inf')) + + strategy_outcomes[strategy_name] = outcomes + + # 根据准则选择 + if criterion == 'maximin': + # 选择最差情景下表现最好的策略 + best_strategy = max( + strategy_outcomes.keys(), + key=lambda s: min(strategy_outcomes[s]) + ) + elif criterion == 'maximize_expected': + # 选择期望值最高的策略 + best_strategy = max( + strategy_outcomes.keys(), + key=lambda s: np.mean(strategy_outcomes[s]) + ) + else: + raise ValueError(f"Unknown criterion: {criterion}") + + return best_strategy +``` + +--- + +## 案例分析 + +### ENAgent中的不确定性量化实践 + +ENAgent在生态网络分析中系统性地应用不确定性量化： + +```python +class ENAgentUncertaintyModule: + """ + ENAgent不确定性量化模块 + + 整合多种方法处理生态网络分析中的不确定性 + """ + + def __init__(self, enagent_core): + """ + Args: + enagent_core: ENAgent核心实例 + """ + self.core = enagent_core + self.uncertainty_results = {} + + def full_uncertainty_analysis(self, + sources: List[Dict], + n_samples: int = 100) -> Dict: + """ + 完整的不确定性分析 + + 包括: + 1. 数据不确定性 (阻力值变化) + 2. 参数不确定性 (源地质量) + 3. 情景不确定性 (不同发展情景) + + Args: + sources: 源地列表 + n_samples: 蒙特卡洛采样次数 + + Returns: + 不确定性分析结果 + """ + results = {} + + # 1. 数据不确定性 - 阻力面 + print("Analyzing data uncertainty (resistance surface)...") + results['data'] = self._resistance_uncertainty(sources, n_samples) + + # 2. 参数不确定性 - 源地质量 + print("Analyzing parameter uncertainty (source quality)...") + results['parameter'] = self._source_quality_uncertainty(sources, n_samples) + + # 3. 情景不确定性 + print("Analyzing scenario uncertainty...") + results['scenario'] = self._scenario_uncertainty(sources) + + # 4. 综合分析 + results['summary'] = self._synthesize_results(results) + + self.uncertainty_results = results + return results + + def _resistance_uncertainty(self, sources, n_samples): + """阻力面数据不确定性分析""" + # 获取基准阻力 + base_resistance = self.core.resistance_surface + + # 定义阻力值的变异系数 + resistance_cv = { + 'forest': 0.2, + 'grassland': 0.3, + 'agriculture': 0.25, + 'urban': 0.15, + 'water': 0.1 + } + + # 蒙特卡洛采样 + connectivity_results = [] + + for _ in range(n_samples): + # 扰动阻力值 + perturbed_resistance = self._perturb_resistance( + base_resistance, resistance_cv + ) + + # 计算连通性 + connectivity = self.core.compute_connectivity( + sources, perturbed_resistance + ) + connectivity_results.append(connectivity) + + # 统计分析 + connectivity_array = np.array(connectivity_results) + + return { + 'mean': connectivity_array.mean(axis=0), + 'std': connectivity_array.std(axis=0), + 'percentiles': { + '5': np.percentile(connectivity_array, 5, axis=0), + '25': np.percentile(connectivity_array, 25, axis=0), + '75': np.percentile(connectivity_array, 75, axis=0), + '95': np.percentile(connectivity_array, 95, axis=0) + } + } + + def _source_quality_uncertainty(self, sources, n_samples): + """源地质量参数不确定性分析""" + results = [] + + for _ in range(n_samples): + # 扰动源地质量 (假设±20%) + perturbed_sources = [] + for source in sources: + perturbed = source.copy() + perturbed['quality'] = source['quality'] * np.random.uniform(0.8, 1.2) + perturbed_sources.append(perturbed) + + # 计算网络指标 + metrics = self.core.compute_network_metrics(perturbed_sources) + results.append(metrics) + + # 统计 + return self._summarize_metrics(results) + + def _scenario_uncertainty(self, sources): + """情景不确定性分析""" + scenarios = { + 'current': { + 'description': '当前状况', + 'urban_expansion': 0, + 'restoration': 0 + }, + 'urban_growth': { + 'description': '城市扩张情景', + 'urban_expansion': 0.5, # 50%扩张 + 'restoration': 0 + }, + 'restoration': { + 'description': '生态修复情景', + 'urban_expansion': 0, + 'restoration': 0.3 # 30%修复 + } + } + + results = {} + for scenario_name, params in scenarios.items(): + # 应用情景参数 + modified_resistance = self.core.apply_scenario(params) + + # 计算结果 + connectivity = self.core.compute_connectivity( + sources, modified_resistance + ) + metrics = self.core.compute_network_metrics(sources) + + results[scenario_name] = { + 'connectivity': connectivity, + 'metrics': metrics, + 'description': params['description'] + } + + return results + + def _synthesize_results(self, results): + """综合各种不确定性分析结果""" + synthesis = { + 'recommendations': [], + 'confidence_levels': {} + } + + # 分析不同来源的不确定性 + data_cv = results['data']['std'].mean() / results['data']['mean'].mean() + parameter_cv = results['parameter']['std'].mean() / results['parameter']['mean'].mean() + + # 根据不确定性的相对大小给出建议 + if data_cv > parameter_cv * 1.5: + synthesis['recommendations'].append( + "数据不确定性是主要来源，建议提高阻力面数据质量" + ) + elif parameter_cv > data_cv * 1.5: + synthesis['recommendations'].append( + "参数不确定性是主要来源，建议更精确地评估源地质量" + ) + + # 情景比较 + scenario_results = results['scenario'] + if 'restoration' in scenario_results and 'urban_growth' in scenario_results: + restoration_connectivity = scenario_results['restoration']['connectivity'].mean() + urban_connectivity = scenario_results['urban_growth']['connectivity'].mean() + + if restoration_connectivity > urban_connectivity * 1.2: + synthesis['recommendations'].append( + "生态修复情景显著改善连通性，建议优先考虑生态修复措施" + ) + + return synthesis + + def generate_uncertainty_report(self) -> str: + """生成不确定性分析报告""" + if not self.uncertainty_results: + return "请先运行不确定性分析" + + report = [] + report.append("=" * 60) + report.append("生态网络不确定性分析报告") + report.append("=" * 60) + report.append("") + + # 数据不确定性 + data_results = self.uncertainty_results['data'] + report.append("1. 数据不确定性 (阻力面)") + report.append(f" 平均变异系数: {data_results['std'].mean() / data_results['mean'].mean():.2%}") + report.append("") + + # 参数不确定性 + param_results = self.uncertainty_results['parameter'] + report.append("2. 参数不确定性 (源地质量)") + report.append(f" 平均变异系数: {param_results['std'].mean() / param_results['mean'].mean():.2%}") + report.append("") + + # 情景不确定性 + scenario_results = self.uncertainty_results['scenario'] + report.append("3. 情景不确定性") + for name, result in scenario_results.items(): + report.append(f" {name}: {result['description']}") + report.append(f" 平均连通性: {result['connectivity'].mean():.2f}") + report.append("") + + # 建议 + summary = self.uncertainty_results['summary'] + report.append("4. 建议") + for i, rec in enumerate(summary['recommendations'], 1): + report.append(f" {i}. {rec}") + + report.append("") + report.append("=" * 60) + + return "\n".join(report) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **不确定性与风险的区分**：什么情况下是真正的风险，什么情况只是不确定性？ + +2. **可接受的不确定性水平**：在实践中，什么样的不确定性水平是可以接受的？ + +3. **不确定性的传播**：多个不确定性因素组合时，是相互放大还是相互抵消？ + +4. **减少不确定性的成本**：何时值得投入资源获取更精确的数据或更复杂的模型？ + +5. **沟通挑战**：如何向决策者有效传达分析结果的不确定性？ + +### 延伸阅读 + +- **"Uncertainty Quantification in Predictive Modeling"** - 不确定性量化综述 +- **"Risk Assessment and Decision Analysis"** - 风险评估与决策分析 +- **"Spatial Uncertainty"** ( Zhang & Goodchild) - 空间不确定性专门著作 +- IPCC不确定性指南 - 气候变化中的不确定性处理实践 + +--- + +## 关键要点 + +1. **不确定性无处不在**：数据、模型、情境都可能引入不确定性 + +2. **区分不确定性类型**：随机性 vs 认知性，需要不同的处理方式 + +3. **蒙特卡洛是通用工具**：通过随机采样估计输出分布 + +4. **敏感性分析识别关键因素**：找出对结果影响最大的输入 + +5. **稳健性比精确性更重要**：在不确定条件下寻找稳健的解决方案 diff --git a/officefile/supplements/02-spatial-intelligence/README.md b/officefile/supplements/02-spatial-intelligence/README.md new file mode 100644 index 0000000..5de4bbb --- /dev/null +++ b/officefile/supplements/02-spatial-intelligence/README.md @@ -0,0 +1,90 @@ +# 第三部分：空间智能 + +## 本部分目标 + +理解AI如何"理解"和操作空间： +- 空间数据的多种表征方式 +- 空间推理的计算方法 +- 多准则决策分析的原理 +- 空间优化问题的建模与求解 +- 不确定性量化的方法 + +--- + +## 章节导航 + +| 章节 | 文件 | 核心内容 | 实践 | +|-----|------|---------|------| +| 02.1 | [空间表征](./02.1-spatial-representation.md) | 栅格/矢量、图表示、多尺度、空间索引 | QGIS图层处理 | +| 02.2 | [空间推理](./02.2-spatial-reasoning.md) | 邻近性、连通性、图算法 | 生态廊道识别 | +| 02.3 | [多准则决策](./02.3-multi-criteria-decision.md) | 权重、标准化、敏感性分析 | 生态系统服务评估 | +| 02.4 | [空间优化](./02.4-spatial-optimization.md) | 目标函数、约束、启发式搜索 | 生态网络优化 | +| 02.5 | [不确定性量化](./02.5-uncertainty-quantification.md) | 不确定性来源、传播、可视化 | 抵抗面敏感性分析 | + +--- + +## 核心概念图 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 空间智能体系 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌───────────────┐ ┌───────────────┐ │ +│ │ 空间表征 │ ───→ │ 空间推理 │ │ +│ │ - 栅格/矢量 │ │ - 拓扑关系 │ │ +│ │ - 图表示 │ │ - 距离/方向 │ │ +│ │ - 多尺度 │ │ - 邻域分析 │ │ +│ └───────────────┘ └───────────────┘ │ +│ │ │ │ +│ └───────────┬───────────┘ │ +│ │ │ +│ ↓ │ +│ ┌───────────────┐ │ +│ │ 多准则决策 │ │ +│ │ - 权重分析 │ │ +│ │ - 标准化 │ │ +│ │ - 敏感性 │ │ +│ └───────┬───────┘ │ +│ │ │ +│ ┌───────────┴───────────┐ │ +│ ↓ ↓ │ +│ ┌───────────────┐ ┌───────────────┐ │ +│ │ 空间优化 │ │ 不确定性量化 │ │ +│ │ - 目标函数 │ │ - 误差传播 │ │ +│ │ - 约束处理 │ │ - 蒙特卡洛 │ │ +│ │ - 启发式 │ │ - 可视化 │ │ +│ └───────────────┘ └───────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 实践案例 + +### 实践案例02：构建生态系统服务评估Skill + +详见 [practice/ecosystem-service-skill](./practice/ecosystem-service-skill/) + +### 实践案例03：最小累积阻力(MCR)分析的自动化 + +详见 [practice/mcr-automation](./practice/mcr-automation/) + +--- + +## 关键要点预览 + +1. **空间表征**是空间智能的基础，选择合适的表征方式至关重要 +2. **空间推理**基于几何和拓扑关系，是空间分析的核心算法 +3. **多准则决策**平衡多个目标，需要合理的权重设计和敏感性分析 +4. **空间优化**寻找最优空间配置，是决策支持的关键 +5. **不确定性量化**让分析结果更可靠，支持稳健决策 + +--- + +## 延伸资源 + +- **"Geographic Information Systems and Science"** (Longley) - GIS基础理论 +- **"Spatial Analysis"** (O'Sullivan) - 空间分析方法 +- **"Geocomputation with R"** - 空间计算实践 diff --git a/officefile/supplements/03-autonomous-design/03.1-workflow-orchestration.md b/officefile/supplements/03-autonomous-design/03.1-workflow-orchestration.md new file mode 100644 index 0000000..eb809e0 --- /dev/null +++ b/officefile/supplements/03-autonomous-design/03.1-workflow-orchestration.md @@ -0,0 +1,634 @@ +# 03.1 工作流编排原理 + +## 核心问题 + +> 如何设计复杂的多步骤分析流程？ +> DAG（有向无环图）如何表达工作流？ +> 如何处理工作流中的条件分支和错误？ + +--- + +## 概念讲解 + +### 工作流编排的核心 + +工作流编排是指**协调多个处理步骤按顺序执行**的能力： + +``` +简单顺序 复杂编排 + │ │ + ↓ ↓ +┌─────┐ ┌─────┐ ┌─────┐ ╔═════════════════╗ +│Step1│───→│Step2│───→│Step3│ ║ 条件分支 ║ +└─────┘ └─────┘ └─────┘ ║ ║ + ║ ┌─────┐ ║ + 一维流程 ║ │ │ ║ + ║ ↓ ↓ ║ + ║ Yes No ║ + ║ │ │ ║ + ║ ↓ ↓ ║ + ║┌─────┐ ┌─────┐ ║ + ║│StepA│ │StepB│ ║ + ║└─────┘ └─────┘ ║ + ║ │ ║ + ║ └────┬─── ║ + ║ │ ║ + ╔═════════════════╝ +``` + +### DAG：有向无环图 + +**DAG** (Directed Acyclic Graph) 是工作流编排的基础数据结构： + +```python +""" +DAG的数学表示 + +DAG = (V, E) +其中： +- V: 节点集合（处理步骤） +- E: 边集合（依赖关系） +- 条件：无环（没有节点能通过边回到自己） + +性质： +1. 有方向：边从上游指向下游 +2. 无环：没有循环依赖 +3. 可拓扑排序：可以找到线性执行顺序 +""" +``` + +**为什么DAG适合工作流？** + +| 特性 | 说明 | +|-----|------| +| **明确依赖** | 边定义了步骤间的依赖关系 | +| **可并行化** | 无依赖的步骤可并行执行 | +| **可验证** | 可以检测循环依赖 | +| **可可视化** | 容易理解和调试 | + +### 工作流的组成要素 + +``` +工作流 = 节点 + 边 + 条件 + 错误处理 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ ┌──────────────┐ │ +│ │ 节点 │ ←───────────────────────────────────── │ +│ │ ────────── │ │ +│ │ - 执行函数 │ 输入 → 处理 → 输出 │ +│ │ - 输入/输出 │ │ +│ │ - 副后置 │ before(), execute(), after() │ +│ └──────────────┘ │ +│ │ │ +│ ↓ │ +│ ┌──────────────┐ │ +│ │ 边 │ ←───────────────────────────────────── │ +│ │ ────────── │ │ +│ │ - 数据流 │ 上游输出 → 下游输入 │ +│ │ - 依赖关系 │ 顺序执行 │ +│ │ - 条件路由 │ 基于状态选择路径 │ +│ └──────────────┘ │ +│ │ │ +│ ↓ │ +│ ┌──────────────┐ │ +│ │ 条件分支 │ ←───────────────────────────────────── │ +│ │ ────────── │ │ +│ │ - 分支条件 │ if state.value > threshold: ... │ +│ │ - 路由选择 │ switch-case模式 │ +│ │ - 合并点 │ 多路径汇聚 │ +│ └──────────────┘ │ +│ │ │ +│ ↓ │ +│ ┌──────────────┐ │ +│ │ 错误处理 │ ←───────────────────────────────────── │ +│ │ ────────── │ │ +│ │ - 重试 │ 失败后重新执行 │ +│ │ - 回滚 │ 恢复到之前状态 │ +│ │ - 降级 │ 使用备选方案 │ +│ │ - 告警 │ 通知相关人员 │ +│ └──────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 设计原理 + +### 工作流设计模式 + +**1. 线性流水线 (Linear Pipeline)** + +```python +class LinearWorkflow: + """最简单的工作流：顺序执行""" + + def __init__(self): + self.steps = [] + + def add_step(self, func, name=None): + """添加步骤""" + self.steps.append({ + 'function': func, + 'name': name or func.__name__ + }) + return self + + def execute(self, initial_data): + """执行工作流""" + result = initial_data + + for step in self.steps: + print(f"执行: {step['name']}") + result = step['function'](result) + + return result + +# 使用示例 +workflow = LinearWorkflow() +workflow.add_step(load_data, "加载数据") +workflow.add_step(clean_data, "清理数据") +workflow.add_step(analyze_data, "分析数据") + +result = workflow.execute("data.geojson") +``` + +**2. 条件分支工作流 (Conditional Workflow)** + +```python +class ConditionalWorkflow: + """带条件分支的工作流""" + + def __init__(self): + self.steps = {} + self.conditions = {} + self.transitions = {} + + def add_step(self, name, func): + """添加步骤""" + self.steps[name] = func + return self + + def add_condition(self, name, condition_func): + """添加条件判断""" + self.conditions[name] = condition_func + return self + + def add_transition(self, from_step, condition, to_step): + """添加状态转换""" + if from_step not in self.transitions: + self.transitions[from_step] = {} + self.transitions[from_step][condition] = to_step + return self + + def execute(self, initial_data, start_step='start'): + """执行工作流""" + current_step = start_step + state = {'data': initial_data} + + while current_step != 'end': + print(f"当前步骤: {current_step}") + + # 执行步骤 + if current_step in self.steps: + result = self.steps[current_step](state['data']) + state['data'] = result + + # 检查条件 + if current_step in self.transitions: + transition = self.transitions[current_step] + matched = False + + for condition_name, next_step in transition.items(): + if condition_name in self.conditions: + if self.conditions[condition_name](state): + current_step = next_step + matched = True + break + + if not matched and 'default' in transition: + current_step = transition['default'] + elif not matched: + current_step = 'end' + else: + current_step = 'end' + + return state['data'] + +# 使用示例：生态网络工作流 +workflow = ConditionalWorkflow() + +# 添加步骤 +workflow.add_step('load_data', load_data) +workflow.add_step('identify_sources', identify_sources) +workflow.add_step('human_review', human_review) +workflow.add_step('build_resistance', build_resistance) + +# 添加条件 +workflow.add_condition('high_uncertainty', + lambda s: s.get('uncertainty', 0) > 0.3) +workflow.add_condition('approved', + lambda s: s.get('decision') == 'approve') + +# 添加转换 +workflow.add_transition('identify_sources', 'high_uncertainty', 'human_review') +workflow.add_transition('identify_sources', 'default', 'build_resistance') +workflow.add_transition('human_review', 'approved', 'build_resistance') +workflow.add_transition('human_review', 'default', 'identify_sources') # 重做 +workflow.add_transition('build_resistance', 'default', 'end') +``` + +**3. 并行工作流 (Parallel Workflow)** + +```python +from concurrent.futures import ThreadPoolExecutor, as_completed + +class ParallelWorkflow: + """支持并行执行的工作流""" + + def __init__(self, max_workers=4): + self.max_workers = max_workers + self.parallel_groups = {} + + def add_parallel_group(self, group_name, tasks): + """添加可并行执行的任务组""" + self.parallel_groups[group_name] = tasks + return self + + def execute_group(self, group_name, shared_data): + """执行一个并行任务组""" + if group_name not in self.parallel_groups: + raise ValueError(f"未知的任务组: {group_name}") + + tasks = self.parallel_groups[group_name] + results = {} + + with ThreadPoolExecutor(max_workers=self.max_workers) as executor: + # 提交所有任务 + future_to_task = { + executor.submit(task['func'], shared_data): task['name'] + for task in tasks + } + + # 收集结果 + for future in as_completed(future_to_task): + task_name = future_to_task[future] + try: + results[task_name] = future.result() + except Exception as e: + results[task_name] = {'error': str(e)} + + return results + +# 使用示例：同时处理多个区域 +workflow = ParallelWorkflow(max_workers=4) + +workflow.add_parallel_group('process_regions', [ + {'name': 'region_north', 'func': process_north_region}, + {'name': 'region_south', 'func': process_south_region}, + {'name': 'region_east', 'func': process_east_region}, + {'name': 'region_west', 'func': process_west_region}, +]) + +results = workflow.execute_group('process_regions', shared_data) +``` + +### 错误处理模式 + +```python +class WorkflowErrorHandling: + """工作流错误处理""" + + class RetryPolicy: + """重试策略""" + def __init__(self, max_retries=3, backoff=2.0): + self.max_retries = max_retries + self.backoff = backoff # 指数退避因子 + + def should_retry(self, attempt, error): + return attempt < self.max_retries + + def get_delay(self, attempt): + return self.backoff ** attempt + + def execute_with_retry(self, func, *args, retry_policy=None, **kwargs): + """带重试的执行""" + if retry_policy is None: + retry_policy = self.RetryPolicy() + + last_error = None + for attempt in range(retry_policy.max_retries + 1): + try: + return func(*args, **kwargs) + except Exception as e: + last_error = e + if retry_policy.should_retry(attempt, e): + delay = retry_policy.get_delay(attempt) + print(f"尝试 {attempt + 1} 失败，{delay}秒后重试...") + time.sleep(delay) + else: + break + + raise last_error + + def execute_with_fallback(self, primary_func, fallback_func, *args, **kwargs): + """带降级的执行""" + try: + return primary_func(*args, **kwargs) + except Exception as e: + print(f"主函数失败: {e}，使用降级方案") + return fallback_func(*args, **kwargs) +``` + +--- + +## 代码示例 + +### 生态网络六阶段工作流 + +```python +""" +ENAgent的完整工作流编排实现 + +六阶段： +1. 数据准备 +2. 源地识别 +3. 阻力面构建 +4. MCR分析 +5. 廊道提取 +6. 结果评估 +""" +import time +from typing import Dict, List, Optional, Callable +from enum import Enum +from dataclasses import dataclass + +class Stage(Enum): + """工作流阶段""" + DATA_PREPARATION = "data_preparation" + SOURCE_IDENTIFICATION = "source_identification" + RESISTANCE_SURFACE = "resistance_surface" + MCR_ANALYSIS = "mcr_analysis" + CORRIDOR_EXTRACTION = "corridor_extraction" + RESULT_EVALUATION = "result_evaluation" + COMPLETED = "completed" + ERROR = "error" + +@dataclass +class WorkflowState: + """工作流状态""" + current_stage: Stage + data: Dict + results: Dict + errors: List[str] + stage_history: List[Stage] + checkpoint_data: Optional[Dict] = None + +class EcologicalNetworkWorkflow: + """生态网络分析工作流""" + + def __init__(self): + self.stages = { + Stage.DATA_PREPARATION: self._data_preparation, + Stage.SOURCE_IDENTIFICATION: self._source_identification, + Stage.RESISTANCE_SURFACE: self._resistance_surface, + Stage.MCR_ANALYSIS: self._mcr_analysis, + Stage.CORRIDOR_EXTRACTION: self._corridor_extraction, + Stage.RESULT_EVALUATION: self._result_evaluation, + } + + self.transitions = { + Stage.DATA_PREPARATION: Stage.SOURCE_IDENTIFICATION, + Stage.SOURCE_IDENTIFICATION: Stage.RESISTANCE_SURFACE, + Stage.RESISTANCE_SURFACE: Stage.MCR_ANALYSIS, + Stage.MCR_ANALYSIS: Stage.CORRIDOR_EXTRACTION, + Stage.CORRIDOR_EXTRACTION: Stage.RESULT_EVALUATION, + Stage.RESULT_EVALUATION: Stage.COMPLETED, + } + + # HITL审查点 + self.checkpoints = { + Stage.SOURCE_IDENTIFICATION: True, + Stage.RESISTANCE_SURFACE: True, + Stage.CORRIDOR_EXTRACTION: False, + } + + def execute(self, initial_data: Dict) -> WorkflowState: + """执行完整工作流""" + state = WorkflowState( + current_stage=Stage.DATA_PREPARATION, + data=initial_data, + results={}, + errors=[], + stage_history=[Stage.DATA_PREPARATION] + ) + + while state.current_stage != Stage.COMPLETED: + if state.current_stage == Stage.ERROR: + print("工作流因错误终止") + break + + # 执行当前阶段 + state = self._execute_stage(state) + + # 检查是否需要审查 + if self.checkpoints.get(state.current_stage, False): + state = self._handle_checkpoint(state) + + # 转换到下一阶段 + if state.current_stage != Stage.ERROR: + next_stage = self.transitions.get( + state.current_stage, + Stage.COMPLETED + ) + state.current_stage = next_stage + state.stage_history.append(next_stage) + + return state + + def _execute_stage(self, state: WorkflowState) -> WorkflowState: + """执行单个阶段""" + stage = state.current_stage + print(f"\n{'='*50}") + print(f"执行阶段: {stage.value}") + print('='*50) + + try: + # 执行阶段函数 + result = self.stages[stage](state.data, state.results) + + # 保存结果 + state.results[stage.value] = result + + except Exception as e: + print(f"阶段 {stage.value} 执行失败: {e}") + state.errors.append(str(e)) + state.current_stage = Stage.ERROR + + return state + + def _handle_checkpoint(self, state: WorkflowState) -> WorkflowState: + """处理HITL审查点""" + stage = state.current_stage + print(f"\n[审查点: {stage.value}]") + + # 实际实现中，这里会等待人类输入 + # 模拟审查通过 + approval = self._get_human_approval(state) + + if not approval: + print("审查未通过，调整参数后重新执行...") + # 可以在这里修改state.data后返回同一阶段 + + return state + + def _get_human_approval(self, state: WorkflowState) -> bool: + """获取人类批准（模拟）""" + print(f"待审查结果: {list(state.results.keys())}") + # 实际实现中等待输入 + return True + + # === 阶段实现 === + + def _data_preparation(self, data: Dict, results: Dict) -> Dict: + """阶段1：数据准备""" + print("加载和处理原始数据...") + time.sleep(0.5) + return { + 'landcover': 'loaded', + 'elevation': 'loaded', + 'boundary': 'loaded' + } + + def _source_identification(self, data: Dict, results: Dict) -> Dict: + """阶段2：源地识别""" + print("识别生态源地...") + sources = [ + {'id': 1, 'area': 1500, 'type': 'forest'}, + {'id': 2, 'area': 800, 'type': 'wetland'} + ] + return {'sources': sources, 'n_sources': len(sources)} + + def _resistance_surface(self, data: Dict, results: Dict) -> Dict: + """阶段3：阻力面构建""" + print("构建生态阻力面...") + return { + 'resistance_surface': 'computed', + 'weights': {'forest': 1, 'urban': 100} + } + + def _mcr_analysis(self, data: Dict, results: Dict) -> Dict: + """阶段4：MCR分析""" + print("执行最小累积阻力分析...") + return {'mcr_surface': 'computed'} + + def _corridor_extraction(self, data: Dict, results: Dict) -> Dict: + """阶段5：廊道提取""" + print("提取生态廊道...") + corridors = [ + {'from': 1, 'to': 2, 'length': 3500} + ] + return {'corridors': corridors, 'n_corridors': len(corridors)} + + def _result_evaluation(self, data: Dict, results: Dict) -> Dict: + """阶段6：结果评估""" + print("评估分析结果...") + return { + 'connectivity_index': 0.75, + 'network_efficiency': 0.82 + } + +# 使用示例 +if __name__ == "__main__": + print("=== 生态网络分析工作流 ===\n") + + workflow = EcologicalNetworkWorkflow() + + initial_data = { + 'landcover_path': 'data/landcover.tif', + 'species': 'target_species', + 'parameters': {} + } + + final_state = workflow.execute(initial_data) + + print("\n=== 工作流完成 ===") + print(f"执行的阶段数: {len(final_state.stage_history)}") + print(f"产生的错误: {final_state.errors}") + print(f"最终结果: {list(final_state.results.keys())}") +``` + +--- + +## 案例分析 + +### LangGraph的工作流实现 + +```python +from langgraph.graph import StateGraph, END +from typing import TypedDict + +class ENAgentState(TypedDict): + """ENAgent工作流状态""" + stage: str + data: dict + results: dict + requires_review: bool + +def build_enagent_workflow(): + """构建ENAgent工作流""" + + # 创建图 + workflow = StateGraph(ENAgentState) + + # 添加节点 + workflow.add_node("prepare_data", prepare_data_node) + workflow.add_node("identify_sources", identify_sources_node) + workflow.add_node("build_resistance", build_resistance_node) + workflow.add_node("mcr_analysis", mcr_analysis_node) + workflow.add_node("extract_corridors", extract_corridors_node) + workflow.add_node("evaluate_results", evaluate_results_node) + + # 添加边（线性流程） + workflow.set_entry_point("prepare_data") + workflow.add_edge("prepare_data", "identify_sources") + workflow.add_edge("identify_sources", "build_resistance") + workflow.add_edge("build_resistance", "mcr_analysis") + workflow.add_edge("mcr_analysis", "extract_corridors") + workflow.add_edge("extract_corridors", "evaluate_results") + workflow.add_edge("evaluate_results", END) + + # 编译 + return workflow.compile() +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **工作流设计**：你的项目中有哪些可以自动化的步骤？ + +2. **错误处理**：当某个步骤失败时，应该重试、跳过还是终止？ + +3. **审查点**：在你的工作流中，哪些地方需要人类介入？ + +4. **并行化**：哪些步骤可以并行执行以提升效率？ + +### 延伸阅读 + +- **"Dataflow Programming"** - 数据流编程范式 +- **"Workflow Patterns"** (Van der Aalst) - 工作流模式 +- **Apache Airflow 文档** - 工作流调度系统 + +--- + +## 关键要点 + +1. **DAG是工作流的核心数据结构**，表达依赖关系 +2. **节点**是处理步骤，**边**定义执行顺序 +3. **条件分支**根据状态动态选择执行路径 +4. **错误处理**是生产工作流的关键 +5. **并行执行**可以显著提升效率 diff --git a/officefile/supplements/03-autonomous-design/03.2-agent-design-patterns.md b/officefile/supplements/03-autonomous-design/03.2-agent-design-patterns.md new file mode 100644 index 0000000..6ac8e36 --- /dev/null +++ b/officefile/supplements/03-autonomous-design/03.2-agent-design-patterns.md @@ -0,0 +1,1007 @@ +# 03.2 Agent设计模式 + +## 核心问题 + +> 什么使一个系统成为"Agent"而非简单的程序？ +> 不同类型的Agent有何区别，各适用于什么场景？ +> 如何选择合适的Agent架构来解决空间分析问题？ + +--- + +## 概念讲解 + +### Agent的本质 + +**Agent** 是能够**感知环境**并**采取行动**以实现目标的实体： + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Agent的基本结构 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ +│ │ Sensors │ ──→ │ Agent │ ──→ │Actuators│ │ +│ │ (感知) │ │ (决策) │ │ (行动) │ │ +│ └─────────┘ └────┬────┘ └─────────┘ │ +│ │ │ +│ ↓ │ +│ ┌──────────┐ │ +│ │Environment│ │ +│ │ (环境) │ │ +│ └──────────┘ │ +│ │ +│ 感知-决策-行动循环 (Perception-Decision-Action Loop) │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +**Agent vs. 程序**： + +| 特征 | 普通程序 | Agent | +|-----|---------|-------| +| 控制流 | 调用者驱动 | 自主驱动 | +| 状态 | 被动存储 | 主动维护世界模型 | +| 目标 | 无明确目标 | 有内在目标 | +| 环境 | 不感知 | 持续感知 | +| 适应性 | 固定行为 | 可学习适应 | + +### 四种经典Agent类型 + +根据Russell & Norvig的AI教材，Agent有四种基本设计模式： + +``` +Agent类型演进 + + Reflex (反应式) + │ + ├──→ 无状态，直接映射 + │ 感知→规则→行动 + │ + ↓ + Model-based (基于模型) + │ + ├──→ 有内部状态 + │ 感知→状态更新→行动 + │ + ↓ + Goal-based (基于目标) + │ + ├──→ 有目标导向的规划 + │ 状态+目标→规划→行动 + │ + ↓ + Utility-based (基于效用) + │ + └──→ 有量化评估 + 状态+目标+效用→最优决策 +``` + +--- + +## 设计原理 + +### 1. Reflex Agent（反应式Agent） + +**特点**：直接将感知映射到行动，无内部状态 + +```python +class ReflexAgent: + """ + 反应式Agent：最简单的Agent类型 + + 适用场景： + - 环境完全可观察 + - 当前行动只依赖当前感知 + - 不需要历史信息 + """ + + def __init__(self, rules: dict): + """ + Args: + rules: {condition: action} 映射规则 + """ + self.rules = rules + + def act(self, percept: dict) -> str: + """ + 根据当前感知选择行动 + + Args: + percept: 当前感知状态 + + Returns: + 选择的行动 + """ + for condition, action in self.rules.items(): + if self._match_condition(condition, percept): + return action + + return self.default_action() + + def _match_condition(self, condition: dict, percept: dict) -> bool: + """检查条件是否匹配""" + for key, value in condition.items(): + if percept.get(key) != value: + return False + return True + + def default_action(self) -> str: + """默认行动""" + return "wait" + + +# 示例：简单的土地覆盖分类Agent +class LandCoverReflexAgent(ReflexAgent): + """基于NDVI的土地覆盖分类Agent""" + + def __init__(self): + rules = { + {'ndvi_high': True}: 'vegetation', + {'ndvi_low': True, 'nir_high': True}: 'water', + {'ndvi_low': True, 'temperature_high': True}: 'urban', + } + super().__init__(rules) + + def classify(self, ndvi: float, nir: float, temperature: float) -> str: + """分类土地覆盖类型""" + percept = { + 'ndvi_high': ndvi > 0.4, + 'ndvi_low': ndvi <= 0.4, + 'nir_high': nir > 0.3, + 'temperature_high': temperature > 25 + } + return self.act(percept) +``` + +**优点**： +- 简单高效 +- 响应快速 +- 易于理解和调试 + +**缺点**： +- 无法处理部分可观察环境 +- 无法规划未来行动 +- 规则冲突时难以决策 + +### 2. Model-based Agent（基于模型的Agent） + +**特点**：维护内部状态，跟踪世界的部分不可观察方面 + +```python +class ModelBasedAgent: + """ + 基于模型的Agent：维护世界状态 + + 适用场景： + - 环境部分可观察 + - 需要跟踪历史信息 + - 需要推断隐藏状态 + """ + + def __init__(self, transition_model, sensor_model): + """ + Args: + transition_model: 状态转移模型 P(s'|s,a) + sensor_model: 传感器模型 P(o|s) + """ + self.state = None + self.transition_model = transition_model + self.sensor_model = sensor_model + self.history = [] + + def update_state(self, action: str, percept: dict): + """ + 更新内部状态 + + 使用贝叶斯推断： + P(s'|o,a,s) ∝ P(o|s') * Σ P(s'|s,a) * P(s) + """ + if self.state is None: + # 初始化状态 + self.state = self.sensor_model.estimate(percept) + else: + # 预测：基于转移模型 + predicted = self.transition_model.predict(self.state, action) + + # 更新：基于感知 + self.state = self.sensor_model.update(predicted, percept) + + self.history.append({ + 'action': action, + 'percept': percept, + 'state': self.state + }) + + def act(self, percept: dict) -> str: + """选择行动""" + self.update_state(self.last_action, percept) + return self._choose_action() + + def _choose_action(self) -> str: + """基于当前状态选择行动""" + raise NotImplementedError + + +# 示例：生态变化检测Agent +class EcologicalChangeAgent(ModelBasedAgent): + """检测生态系统变化的Agent""" + + class TransitionModel: + """状态转移模型""" + def predict(self, state, action): + # 简单的马尔可夫假设 + new_state = state.copy() + if action == 'monitor': + # 状态可能自然变化 + new_state['change_probability'] *= 0.95 + return new_state + + class SensorModel: + """传感器模型""" + def estimate(self, percept): + return { + 'baseline': percept['ndvi'], + 'change_probability': 0.0, + 'confidence': percept['quality'] + } + + def update(self, predicted, percept): + # 融合预测和观测 + alpha = 0.7 # 预测权重 + new_ndvi = alpha * predicted['baseline'] + (1-alpha) * percept['ndvi'] + + change_prob = predicted['change_probability'] + if abs(new_ndvi - predicted['baseline']) > 0.1: + change_prob += 0.2 + + return { + 'baseline': new_ndvi, + 'change_probability': min(1.0, change_prob), + 'confidence': predicted['confidence'] + } + + def __init__(self): + super().__init__(self.TransitionModel(), self.SensorModel()) + self.last_action = None + + def _choose_action(self) -> str: + """基于变化概率选择行动""" + if self.state['change_probability'] > 0.6: + return 'alert' + elif self.state['change_probability'] > 0.3: + return 'investigate' + else: + return 'monitor' +``` + +### 3. Goal-based Agent（基于目标的Agent） + +**特点**：显式表示目标，能够规划行动序列 + +```python +class GoalBasedAgent(ModelBasedAgent): + """ + 基于目标的Agent：有明确的追求目标 + + 适用场景： + - 需要规划多步行动 + - 有明确的目标状态 + - 需要考虑行动后果 + """ + + def __init__(self, transition_model, sensor_model, planner): + """ + Args: + planner: 规划器，搜索从当前状态到目标的路径 + """ + super().__init__(transition_model, sensor_model) + self.planner = planner + self.current_goal = None + self.current_plan = [] + + def set_goal(self, goal: dict): + """设置目标""" + self.current_goal = goal + self.current_plan = [] + return self + + def act(self, percept: dict) -> str: + """选择行动""" + self.update_state(self.last_action, percept) + + # 检查是否达到目标 + if self._goal_achieved(): + return 'goal_reached' + + # 如果没有计划或计划过时，重新规划 + if not self.current_plan or self._plan_stale(): + self.current_plan = self.planner.plan( + self.state, + self.current_goal + ) + + # 执行计划的下一步 + if self.current_plan: + action = self.current_plan.pop(0) + self.last_action = action + return action + + return 'no_plan' + + def _goal_achieved(self) -> bool: + """检查目标是否达成""" + if not self.current_goal: + return False + return all( + self.state.get(k) == v + for k, v in self.current_goal.items() + ) + + def _plan_stale(self) -> bool: + """检查计划是否需要更新""" + # 简化版本：检查最近的状态变化 + if len(self.history) < 2: + return False + # 实际实现会更复杂 + return False + + +# 示例：保护区选址Agent +class ReserveSiteAgent(GoalBasedAgent): + """寻找最佳保护区选址的Agent""" + + class Planner: + """前向搜索规划器""" + def plan(self, current_state, goal): + """ + 使用前向搜索规划 + + 返回行动序列：[action1, action2, ...] + """ + plan = [] + + # 目标：找到至少3个候选地点 + while current_state.get('n_candidates', 0) < goal.get('min_sites', 3): + # 下一步行动 + if not current_state.get('searched_regions', []): + action = ('search_region', 0) + else: + next_region = max(current_state['searched_regions']) + 1 + action = ('search_region', next_region) + + plan.append(action) + # 模拟状态更新 + current_state = self._simulate(current_state, action) + + if next_region > 10: # 防止无限循环 + break + + return plan + + def _simulate(self, state, action): + """模拟行动后的状态""" + new_state = state.copy() + if action[0] == 'search_region': + searched = state.get('searched_regions', []) + searched.append(action[1]) + new_state['searched_regions'] = searched + # 模拟可能发现候选点 + if action[1] % 3 == 0: # 每3个区域有1个候选 + new_state['n_candidates'] = state.get('n_candidates', 0) + 1 + return new_state + + def __init__(self): + super().__init__( + super().TransitionModel(), + super().SensorModel(), + self.Planner() + ) + self.last_action = None + + def find_reserve_sites(self, min_sites: int = 3, budget: float = 1000000): + """寻找保护区选址""" + self.set_goal({ + 'min_sites': min_sites, + 'budget': budget, + 'status': 'found' + }) + return self +``` + +### 4. Utility-based Agent（基于效用的Agent） + +**特点**：使用效用函数量化目标状态的价值，处理冲突目标 + +```python +class UtilityBasedAgent(GoalBasedAgent): + """ + 基于效用的Agent：量化目标价值 + + 适用场景： + - 有多个冲突目标 + - 目标有不同重要性 + - 需要在不确定环境下决策 + """ + + def __init__(self, transition_model, sensor_model, planner, utility_fn): + """ + Args: + utility_fn: 效用函数 U(state) → 实数 + """ + super().__init__(transition_model, sensor_model, planner) + self.utility_fn = utility_fn + + def set_preferences(self, preferences: dict): + """设置偏好（权重）""" + self.utility_fn.set_weights(preferences) + return self + + def evaluate_plan(self, plan: list) -> float: + """评估计划的期望效用""" + expected_state = self.state + total_utility = 0 + + for action in plan: + # 模拟行动 + expected_state = self.transition_model.predict( + expected_state, action + ) + # 累积效用 + total_utility += self.utility_fn(expected_state) + + return total_utility + + def choose_best_action(self, available_actions: list) -> str: + """选择效用最大的行动""" + best_action = None + best_utility = float('-inf') + + for action in available_actions: + # 预测行动后的状态 + predicted_state = self.transition_model.predict( + self.state, action + ) + # 计算效用 + utility = self.utility_fn(predicted_state) + + if utility > best_utility: + best_utility = utility + best_action = action + + return best_action + + +class UtilityFunction: + """效用函数""" + + def __init__(self, objectives: dict): + """ + Args: + objectives: {name: (weight, function)} + """ + self.objectives = objectives + + def set_weights(self, weights: dict): + """更新目标权重""" + for name, weight in weights.items(): + if name in self.objectives: + old_weight, fn = self.objectives[name] + self.objectives[name] = (weight, fn) + + def __call__(self, state: dict) -> float: + """计算状态的总效用""" + total = 0 + for (weight, fn) in self.objectives.values(): + total += weight * fn(state) + return total + + +# 示例：土地利用规划Agent +class LandUsePlanningAgent(UtilityBasedAgent): + """土地利用规划Agent，平衡多个目标""" + + def __init__(self): + # 定义多个目标 + objectives = { + 'economic': (0.3, self._economic_value), + 'ecological': (0.4, self._ecological_value), + 'social': (0.3, self._social_value), + } + + super().__init__( + super().TransitionModel(), + super().SensorModel(), + super().Planner(), + UtilityFunction(objectives) + ) + + @staticmethod + def _economic_value(state: dict) -> float: + """经济价值：开发土地产生的收益""" + return state.get('developed_area', 0) * 1000 + + @staticmethod + def _ecological_value(state: dict) -> float: + """生态价值：保护的自然栖息地""" + return -state.get('habitat_loss', 0) * 500 + + @staticmethod + def _social_value(state: dict) -> float: + """社会价值：住房供应和公共空间""" + housing = state.get('housing_units', 0) + green_space = state.get('green_space_ratio', 0) + return housing * 100 + green_space * 2000 + + def plan_land_use(self, area: float, economic_weight: float = 0.3): + """规划土地利用""" + self.set_preferences({ + 'economic': economic_weight, + 'ecological': 1 - economic_weight - 0.3, + 'social': 0.3 + }) + return self +``` + +--- + +## 代码示例 + +### 完整的四类Agent对比演示 + +```python +""" +四种Agent类型的完整对比演示 + +场景：生态监测站需要决定每日行动 +""" +import numpy as np +from typing import Dict, List, Optional, Tuple +from dataclasses import dataclass +from enum import Enum + + +class SensorReading(Enum): + """传感器读数类型""" + NORMAL = "normal" + ANOMALY_DETECTED = "anomaly" + CRITICAL = "critical" + + +@dataclass +class EnvironmentState: + """环境状态""" + temperature: float + humidity: float + species_count: int + vegetation_health: float # 0-1 + detected_anomaly: bool + time_step: int + + +class MonitoringStation: + """模拟生态监测站环境""" + + def __init__(self): + self.state = EnvironmentState( + temperature=25.0, + humidity=60.0, + species_count=15, + vegetation_health=0.8, + detected_anomaly=False, + time_step=0 + ) + self.anomaly_schedule = [5, 12, 18] # 预定异常发生时间 + + def step(self, action: str) -> Tuple[SensorReading, EnvironmentState]: + """执行一步模拟""" + self.state.time_step += 1 + + # 环境动态变化 + self.state.temperature += np.random.normal(0, 1) + self.state.humidity += np.random.normal(0, 2) + self.state.vegetation_health = max(0, min(1, + self.state.vegetation_health + np.random.normal(0, 0.05) + )) + + # 检查是否发生异常 + if self.state.time_step in self.anomaly_schedule: + self.state.detected_anomaly = True + self.state.vegetation_health -= 0.2 + + # 行动影响 + if action == "collect_sample": + self.state.species_count += np.random.randint(-1, 2) + elif action == "irrigate": + self.state.humidity = min(100, self.state.humidity + 10) + self.state.vegetation_health = min(1, self.state.vegetation_health + 0.05) + + # 生成传感器读数 + reading = self._get_sensor_reading() + + return reading, self.state.copy() + + def _get_sensor_reading(self) -> SensorReading: + """生成传感器读数""" + if self.state.vegetation_health < 0.3: + return SensorReading.CRITICAL + elif self.state.detected_anomaly: + return SensorReading.ANOMALY_DETECTED + return SensorReading.NORMAL + + def reset(self): + """重置环境""" + self.__init__() + + +# ==================== 1. Reflex Agent ==================== + +class ReflexMonitoringAgent: + """反应式监测Agent""" + + def __init__(self): + self.rules = { + SensorReading.CRITICAL: "emergency_response", + SensorReading.ANOMALY_DETECTED: "investigate", + SensorReading.NORMAL: "routine_check" + } + + def act(self, reading: SensorReading) -> str: + """根据读数直接行动""" + return self.rules.get(reading, "routine_check") + + +# ==================== 2. Model-based Agent ==================== + +class ModelBasedMonitoringAgent: + """基于模型的监测Agent""" + + def __init__(self): + self.belief_state = { + 'anomaly_active': False, + 'anomaly_duration': 0, + 'vegetation_trend': 'stable', + 'last_reading': None + } + + def act(self, reading: SensorReading) -> str: + """更新信念状态并行动""" + # 更新内部状态 + if reading == SensorReading.ANOMALY_DETECTED: + self.belief_state['anomaly_active'] = True + self.belief_state['anomaly_duration'] += 1 + elif reading == SensorReading.NORMAL: + if self.belief_state['anomaly_active']: + self.belief_state['anomaly_duration'] -= 1 + if self.belief_state['anomaly_duration'] <= 0: + self.belief_state['anomaly_active'] = False + + # 基于信念状态决策 + if self.belief_state['anomaly_active']: + if self.belief_state['anomaly_duration'] > 2: + return "intensive_monitoring" + return "investigate" + return "routine_check" + + +# ==================== 3. Goal-based Agent ==================== + +class GoalBasedMonitoringAgent: + """基于目标的监测Agent""" + + def __init__(self): + self.current_goal = None + self.belief_state = { + 'data_coverage': 0.0, + 'samples_collected': 0, + 'anomalies_investigated': 0 + } + + def set_goal(self, goal: str): + """设置当前目标""" + self.current_goal = goal + return self + + def act(self, reading: SensorReading) -> str: + """基于目标选择行动""" + # 目标导向的规划 + if self.current_goal == "comprehensive_survey": + if self.belief_state['data_coverage'] < 1.0: + return "collect_sample" + elif self.belief_state['samples_collected'] < 10: + return "collect_sample" + else: + return "compile_report" + + elif self.current_goal == "anomaly_investigation": + if reading == SensorReading.ANOMALY_DETECTED: + self.belief_state['anomalies_investigated'] += 1 + return "investigate" + return "search_for_anomalies" + + return "routine_check" + + +# ==================== 4. Utility-based Agent ==================== + +class UtilityMonitoringAgent: + """基于效用的监测Agent""" + + def __init__(self): + self.belief_state = { + 'anomaly_active': False, + 'data_coverage': 0.0, + 'resource_remaining': 100, + 'scientific_value': 0 + } + self.weights = { + 'safety': 0.5, + 'science': 0.3, + 'efficiency': 0.2 + } + + def utility(self, action: str, reading: SensorReading) -> float: + """计算行动的效用""" + utility = 0 + + # 安全效用 + if reading == SensorReading.CRITICAL: + if action == "emergency_response": + utility += 100 * self.weights['safety'] + else: + utility -= 50 * self.weights['safety'] + elif reading == SensorReading.ANOMALY_DETECTED: + if action == "investigate": + utility += 30 * self.weights['safety'] + elif action == "routine_check": + utility -= 20 * self.weights['safety'] + + # 科学价值效用 + if action == "collect_sample": + if self.belief_state['data_coverage'] < 0.8: + utility += 20 * self.weights['science'] + else: + utility += 5 * self.weights['science'] + + # 效率效用 + if self.belief_state['resource_remaining'] < 20: + if action == "routine_check": + utility += 10 * self.weights['efficiency'] + elif action == "collect_sample": + utility -= 15 * self.weights['efficiency'] + + return utility + + def act(self, reading: SensorReading) -> str: + """选择效用最大的行动""" + actions = ["emergency_response", "investigate", "collect_sample", + "routine_check", "rest"] + + best_action = "routine_check" + best_utility = float('-inf') + + for action in actions: + u = self.utility(action, reading) + if u > best_utility: + best_utility = u + best_action = action + + return best_action + + +# ==================== 演示对比 ==================== + +def compare_agents(n_steps: int = 20): + """对比四种Agent的表现""" + print("=" * 60) + print("四种Agent类型在生态监测任务中的对比") + print("=" * 60) + + agents = { + "Reflex": ReflexMonitoringAgent(), + "Model-based": ModelBasedMonitoringAgent(), + "Goal-based": GoalBasedMonitoringAgent().set_goal("comprehensive_survey"), + "Utility-based": UtilityMonitoringAgent() + } + + results = {name: [] for name in agents.keys()} + + for step in range(n_steps): + env = MonitoringStation() + + for name, agent in agents.items(): + reading, state = env.step("observe") + action = agent.act(reading) + results[name].append(action) + + # 打印结果对比 + print("\n行动序列对比:") + print("-" * 60) + print(f"{'时间':<6} {'Reflex':<20} {'Model-based':<20}") + print("-" * 60) + + for i in range(n_steps): + print(f"{i:<6} {results['Reflex'][i]:<20} {results['Model-based'][i]:<20}") + + print("-" * 60) + print(f"{'时间':<6} {'Goal-based':<20} {'Utility-based':<20}") + print("-" * 60) + + for i in range(n_steps): + print(f"{i:<6} {results['Goal-based'][i]:<20} {results['Utility-based'][i]:<20}") + + # 统计分析 + print("\n行动统计:") + print("-" * 60) + for name, actions in results.items(): + from collections import Counter + counts = Counter(actions) + print(f"\n{name}:") + for action, count in counts.most_common(): + print(f" {action}: {count}") + + +if __name__ == "__main__": + compare_agents() +``` + +--- + +## 案例分析 + +### Claude Code的Agent架构 + +Claude Code是一个典型的**Utility-based Agent**，它结合了多种设计模式： + +```python +""" +Claude Code的Agent架构简化示意 +""" + +class ClaudeCodeAgent: + """ + Claude Code Agent设计 + + 特点： + - Model-based: 维护对话上下文状态 + - Goal-based: 追求用户的任务目标 + - Utility-based: 平衡正确性、效率、安全性 + """ + + def __init__(self): + # 内部状态 + self.state = { + 'conversation_history': [], + 'workspace_state': {}, # 文件系统状态 + 'tool_results': [], + 'user_goal': None + } + + # 效用函数组件 + self.utility_components = { + 'task_completion': 0.5, # 完成任务 + 'correctness': 0.3, # 正确性 + 'safety': 0.2, # 安全性 + } + + def perceive(self, user_input: str, tool_outputs: list): + """感知：更新内部状态""" + self.state['conversation_history'].append({ + 'role': 'user', + 'content': user_input + }) + self.state['tool_results'] = tool_outputs + + def plan(self): + """规划：生成行动计划""" + # 分析用户意图 + intent = self._analyze_intent() + + # 生成候选行动序列 + candidates = self._generate_candidates(intent) + + # 评估每个候选 + best_plan = max( + candidates, + key=lambda p: self._evaluate_plan(p) + ) + + return best_plan + + def act(self, plan: list): + """执行：按计划调用工具""" + results = [] + for action in plan: + result = self._execute_action(action) + results.append(result) + return results + + def _evaluate_plan(self, plan: list) -> float: + """评估计划的效用""" + utility = 0 + for component, weight in self.utility_components.items(): + if component == 'safety': + # 检查危险操作 + if any(a.get('dangerous') for a in plan): + utility -= 100 * weight + # ... 其他评估 + return utility +``` + +### ENAgent的混合设计 + +ENAgent（生态网络分析Agent）采用了**多层混合架构**： + +```python +class ENAgent: + """ + ENAgent: 多层混合Agent架构 + + 反应层：处理简单操作 + 规划层：处理复杂分析流程 + 效用层：优化分析参数 + """ + + def __init__(self): + # 反应式处理简单命令 + self.reflex_layer = ReflexLayer({ + 'load_data': self._load_data, + 'show_status': self._show_status, + }) + + # 规划层处理复杂流程 + self.planning_layer = PlanningLayer() + self.planning_layer.set_goal('build_ecological_network') + + # 效用层优化参数 + self.utility_layer = UtilityLayer({ + 'accuracy': self._accuracy_fn, + 'computation_time': self._time_fn, + 'memory_usage': self._memory_fn, + }) + + def process(self, user_command: str): + """处理用户命令""" + # 1. 反应层快速响应 + if user_command in self.reflex_layer.handlers: + return self.reflex_layer.handle(user_command) + + # 2. 规划层生成流程 + plan = self.planning_layer.generate_plan(user_command) + + # 3. 效用层优化参数 + optimized_plan = self.utility_layer.optimize(plan) + + # 4. 执行计划 + return self._execute(optimized_plan) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **Agent类型选择**：你的空间分析项目适合哪种Agent类型？ + +2. **状态管理**：如何在部分可观察环境中维护准确的内部状态？ + +3. **目标冲突**：当生态保护与经济发展冲突时，如何量化权衡？ + +4. **规划成本**：复杂规划的计算成本何时超过了其收益？ + +### 延伸阅读 + +- **"Artificial Intelligence: A Modern Approach"** (Russell & Norvig) - Chapter 2: Intelligent Agents +- **"Agent-Based Modeling"** (Railsback & Grimm) - 基于Agent的建模 +- **ReAct论文** - "ReAct: Synergizing Reasoning and Acting in Language Models" + +--- + +## 关键要点 + +1. **Agent的核心特征**是自主性、感知-行动循环和目标导向 +2. **Reflex Agent**最简单，适合完全可观察环境 +3. **Model-based Agent**通过内部状态处理部分可观察性 +4. **Goal-based Agent**能够规划多步行动达成目标 +5. **Utility-based Agent**通过效用函数处理多目标冲突 +6. **实际系统**常采用混合架构，结合多种设计模式 diff --git a/officefile/supplements/03-autonomous-design/03.3-skill-composition.md b/officefile/supplements/03-autonomous-design/03.3-skill-composition.md new file mode 100644 index 0000000..c732fb2 --- /dev/null +++ b/officefile/supplements/03-autonomous-design/03.3-skill-composition.md @@ -0,0 +1,1061 @@ +# 03.3 技能组合与复用 + +## 核心问题 + +> 如何将复杂功能分解为可复用的技能单元？ +> 如何设计技能接口以支持动态组合？ +> 如何发现和加载新技能而不修改核心代码？ + +--- + +## 概念讲解 + +### 技能抽象的概念 + +**技能 (Skill)** 是Agent可执行的**独立功能单元**，具有明确的输入输出接口： + +``` +技能 = 功能定义 + 接口契约 + 元数据 + +┌─────────────────────────────────────────────────────────────┐ +│ 技能的基本结构 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ Skill Interface │ │ +│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ +│ │ │ 名称 │ │ 描述 │ │ 参数 │ │ │ +│ │ │ name │ │ description │ │ parameters │ │ │ +│ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ +│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ +│ │ │ 输入 │ │ 输出 │ │ 副作用 │ │ │ +│ │ │ input │ │ output │ │ side_effects│ │ │ +│ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ │ +│ ↓ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ Implementation │ │ +│ │ │ +│ │ def execute(self, **kwargs) -> SkillResult: │ +│ │ # 技能实现 │ +│ │ pass │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 技能组合模式 + +``` +技能组合模式层次 + + 1. 顺序组合 (Sequential) + │ + ├──→ 技能A输出 → 技能B输入 → 技能C输入 + │ 适用：数据流水线处理 + │ + 2. 并行组合 (Parallel) + │ + ├──→ 技能A ──┐ + │ 技能B ──┼──→ 合并结果 + │ 技能C ──┘ + │ 适用：独立任务并行执行 + │ + 3. 条件组合 (Conditional) + │ + ├──→ 条件判断 → 技能A或技能B + │ 适用：分支处理逻辑 + │ + 4. 迭代组合 (Iterative) + │ + └──→ 技能A输出 → [循环] → 技能B + 适用：增量式处理 +``` + +--- + +## 设计原理 + +### 技能接口设计 + +良好的技能接口是实现组合的基础： + +```python +from abc import ABC, abstractmethod +from typing import Dict, Any, List, Optional, Type, Callable +from dataclasses import dataclass, field +from enum import Enum + + +class SkillCategory(Enum): + """技能分类""" + DATA_PROCESSING = "data_processing" + SPATIAL_ANALYSIS = "spatial_analysis" + VISUALIZATION = "visualization" + FILE_OPERATIONS = "file_operations" + MODEL_EXECUTION = "model_execution" + + +@dataclass +class ParameterSpec: + """参数规格""" + name: str + type: Type + description: str + required: bool = True + default: Any = None + constraints: Dict[str, Any] = field(default_factory=dict) + + +@dataclass +class SkillResult: + """技能执行结果""" + success: bool + data: Any = None + error: Optional[str] = None + metadata: Dict[str, Any] = field(default_factory=dict) + + +class Skill(ABC): + """技能基类""" + + # 技能元数据 + name: str = "" + description: str = "" + category: SkillCategory = SkillCategory.DATA_PROCESSING + version: str = "1.0.0" + parameters: List[ParameterSpec] = field(default_factory=list) + + @abstractmethod + def execute(self, **kwargs) -> SkillResult: + """执行技能""" + pass + + def validate_input(self, **kwargs) -> tuple[bool, Optional[str]]: + """验证输入参数""" + for param in self.parameters: + if param.required and param.name not in kwargs: + return False, f"Missing required parameter: {param.name}" + + if param.name in kwargs: + value = kwargs[param.name] + if not isinstance(value, param.type): + try: + kwargs[param.name] = param.type(value) + except (ValueError, TypeError): + return False, f"Invalid type for {param.name}" + + return True, None + + def get_spec(self) -> Dict[str, Any]: + """获取技能规格""" + return { + 'name': self.name, + 'description': self.description, + 'category': self.category.value, + 'version': self.version, + 'parameters': [ + { + 'name': p.name, + 'type': p.type.__name__, + 'description': p.description, + 'required': p.required, + 'default': p.default + } + for p in self.parameters + ] + } +``` + +### 技能组合器 + +```python +class SkillComposer: + """ + 技能组合器：将多个技能组合成复合技能 + """ + + def __init__(self): + self.skills: Dict[str, Skill] = {} + + def register(self, skill: Skill) -> 'SkillComposer': + """注册技能""" + self.skills[skill.name] = skill + return self + + def sequential(self, *skill_names: str) -> 'CompositeSkill': + """ + 顺序组合：前一个技能的输出传递给下一个 + + 数据流: input → skill1 → skill2 → skill3 → output + """ + skills = [self.skills[name] for name in skill_names] + return CompositeSkill( + name=f"sequential_{'_'.join(skill_names)}", + skills=skills, + mode='sequential' + ) + + def parallel(self, *skill_names: str, merge_func: Callable = None) -> 'CompositeSkill': + """ + 并行组合：所有技能并行执行，结果合并 + + 数据流: + input ──→ skill1 ──┐ + ─→ skill2 ──┼──→ merge → output + ─→ skill3 ──┘ + """ + skills = [self.skills[name] for name in skill_names] + return CompositeSkill( + name=f"parallel_{'_'.join(skill_names)}", + skills=skills, + mode='parallel', + merge_func=merge_func + ) + + def conditional(self, condition: Callable, + true_skill: str, false_skill: str = None) -> 'CompositeSkill': + """ + 条件组合：根据条件选择技能执行 + + 数据流: input → condition → skill_if_true / skill_if_false → output + """ + skills = [self.skills[true_skill]] + if false_skill: + skills.append(self.skills[false_skill]) + + return CompositeSkill( + name=f"conditional_{true_skill}_{false_skill}", + skills=skills, + mode='conditional', + condition=condition + ) + + def loop(self, skill_name: str, + until: Callable = None, + max_iterations: int = 10) -> 'CompositeSkill': + """ + 迭代组合：循环执行技能直到满足条件 + + 数据流: input → [skill → check] → output + """ + skill = self.skills[skill_name] + return CompositeSkill( + name=f"loop_{skill_name}", + skills=[skill], + mode='loop', + until_condition=until, + max_iterations=max_iterations + ) + + +class CompositeSkill(Skill): + """复合技能：由多个子技能组合而成""" + + def __init__(self, name: str, skills: List[Skill], + mode: str, **kwargs): + self.name = name + self.skills = skills + self.mode = mode # sequential, parallel, conditional, loop + self.condition = kwargs.get('condition') + self.merge_func = kwargs.get('merge_func') + self.until_condition = kwargs.get('until_condition') + self.max_iterations = kwargs.get('max_iterations', 10) + + def execute(self, initial_input: Any = None, **kwargs) -> SkillResult: + """执行复合技能""" + if self.mode == 'sequential': + return self._execute_sequential(initial_input, **kwargs) + elif self.mode == 'parallel': + return self._execute_parallel(initial_input, **kwargs) + elif self.mode == 'conditional': + return self._execute_conditional(initial_input, **kwargs) + elif self.mode == 'loop': + return self._execute_loop(initial_input, **kwargs) + else: + return SkillResult(success=False, error=f"Unknown mode: {self.mode}") + + def _execute_sequential(self, initial_input, **kwargs): + """顺序执行""" + current_input = initial_input + results = [] + + for skill in self.skills: + if isinstance(current_input, dict): + result = skill.execute(**current_input) + else: + result = skill.execute(input=current_input) + + if not result.success: + return SkillResult( + success=False, + error=f"Skill {skill.name} failed: {result.error}", + metadata={'failed_at': skill.name} + ) + + results.append(result) + current_input = result.data + + return SkillResult( + success=True, + data=current_input, + metadata={'sub_results': results} + ) + + def _execute_parallel(self, initial_input, **kwargs): + """并行执行""" + import concurrent.futures + + results = [] + with concurrent.futures.ThreadPoolExecutor() as executor: + futures = { + executor.submit(skill.execute, input=initial_input): skill + for skill in self.skills + } + + for future in concurrent.futures.as_completed(futures): + skill = futures[future] + try: + result = future.result() + results.append(result) + except Exception as e: + results.append(SkillResult( + success=False, + error=str(e), + metadata={'skill': skill.name} + )) + + # 合并结果 + if self.merge_func: + merged_data = self.merge_func(results) + else: + # 默认合并：收集所有成功的数据 + merged_data = [r.data for r in results if r.success] + + return SkillResult( + success=all(r.success for r in results), + data=merged_data, + metadata={'sub_results': results} + ) + + def _execute_conditional(self, initial_input, **kwargs): + """条件执行""" + if self.condition and self.condition(initial_input): + result = self.skills[0].execute(input=initial_input) + elif len(self.skills) > 1: + result = self.skills[1].execute(input=initial_input) + else: + result = SkillResult(success=True, data=initial_input) + + return result + + def _execute_loop(self, initial_input, **kwargs): + """循环执行""" + current_input = initial_input + results = [] + + for i in range(self.max_iterations): + result = self.skills[0].execute(input=current_input) + + if not result.success: + return SkillResult( + success=False, + error=f"Iteration {i} failed: {result.error}" + ) + + results.append(result) + current_input = result.data + + # 检查终止条件 + if self.until_condition and self.until_condition(result.data): + break + + return SkillResult( + success=True, + data=current_input, + metadata={'iterations': len(results), 'sub_results': results} + ) +``` + +### 动态技能发现与加载 + +```python +import importlib +import importlib.util +import inspect +from pathlib import Path + + +class SkillRegistry: + """ + 技能注册表：管理技能的发现、加载和注册 + """ + + def __init__(self): + self._skills: Dict[str, Type[Skill]] = {} + self._categories: Dict[SkillCategory, List[str]] = { + category: [] for category in SkillCategory + } + + def register_class(self, skill_class: Type[Skill]) -> None: + """注册技能类""" + if not issubclass(skill_class, Skill): + raise TypeError(f"{skill_class} must be a subclass of Skill") + + # 创建实例获取元数据 + instance = skill_class() + self._skills[instance.name] = skill_class + self._categories[instance.category].append(instance.name) + + def get_skill(self, name: str) -> Optional[Skill]: + """获取技能实例""" + if name in self._skills: + return self._skills[name]() + return None + + def list_skills(self, category: SkillCategory = None) -> List[str]: + """列出技能""" + if category: + return self._categories.get(category, []) + return list(self._skills.keys()) + + def discover_from_directory(self, directory: Path) -> int: + """ + 从目录发现并加载技能 + + 约定：技能文件以 _skill.py 结尾，包含继承自 Skill 的类 + """ + count = 0 + + for file_path in directory.rglob("*_skill.py"): + try: + # 动态导入模块 + module_name = file_path.stem + spec = importlib.util.spec_from_file_location(module_name, file_path) + if spec and spec.loader: + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + + # 查找 Skill 子类 + for name, obj in inspect.getmembers(module, inspect.isclass): + if (issubclass(obj, Skill) and + obj != Skill and + not obj.__module__.startswith('_')): + self.register_class(obj) + count += 1 + + except Exception as e: + print(f"Failed to load {file_path}: {e}") + + return count + + def get_skill_spec(self, name: str) -> Optional[Dict]: + """获取技能规格""" + skill = self.get_skill(name) + if skill: + return skill.get_spec() + return None + + +# 全局技能注册表 +registry = SkillRegistry() + + +def register_skill(skill_class: Type[Skill]) -> Type[Skill]: + """技能注册装饰器""" + registry.register_class(skill_class) + return skill_class +``` + +--- + +## 代码示例 + +### QGIS技能集成管理器 + +```python +""" +QGIS技能集成管理器 + +演示如何为空间分析工具创建可组合的技能系统 +""" +import json +from typing import Dict, Any, List, Optional +from pathlib import Path + + +# ==================== 基础空间分析技能 ==================== + +@register_skill +class LoadVectorLayerSkill(Skill): + """加载矢量图层的技能""" + name = "load_vector_layer" + description = "从文件加载矢量图层" + category = SkillCategory.FILE_OPERATIONS + version = "1.0.0" + parameters = [ + ParameterSpec("path", str, "文件路径", required=True), + ParameterSpec("layer_name", str, "图层名称", required=False, default="layer"), + ] + + def execute(self, **kwargs) -> SkillResult: + valid, error = self.validate_input(**kwargs) + if not valid: + return SkillResult(success=False, error=error) + + path = kwargs['path'] + layer_name = kwargs.get('layer_name', 'layer') + + # 模拟加载（实际会调用QGIS API） + return SkillResult( + success=True, + data={ + 'layer': layer_name, + 'path': path, + 'type': 'vector', + 'feature_count': 1250, + 'crs': 'EPSG:4326' + }, + metadata={'loaded_at': '2024-01-01T00:00:00'} + ) + + +@register_skill +class BufferAnalysisSkill(Skill): + """缓冲区分析技能""" + name = "buffer_analysis" + description = "对几何图形创建缓冲区" + category = SkillCategory.SPATIAL_ANALYSIS + version = "1.0.0" + parameters = [ + ParameterSpec("layer", str, "输入图层", required=True), + ParameterSpec("distance", float, "缓冲距离", required=True), + ParameterSpec("segments", int, "分段数", required=False, default=8), + ] + + def execute(self, **kwargs) -> SkillResult: + valid, error = self.validate_input(**kwargs) + if not valid: + return SkillResult(success=False, error=error) + + layer = kwargs['layer'] + distance = kwargs['distance'] + + # 模拟缓冲区分析 + return SkillResult( + success=True, + data={ + 'output_layer': f"{layer}_buffer_{distance}m", + 'input_layer': layer, + 'distance': distance, + 'area_ha': 542.3 + } + ) + + +@register_skill +class CalculateAreaSkill(Skill): + """计算面积技能""" + name = "calculate_area" + description = "计算要素面积" + category = SkillCategory.SPATIAL_ANALYSIS + version = "1.0.0" + parameters = [ + ParameterSpec("layer", str, "输入图层", required=True), + ParameterSpec("unit", str, "单位", required=False, default="ha"), + ] + + def execute(self, **kwargs) -> SkillResult: + valid, error = self.validate_input(**kwargs) + if not valid: + return SkillResult(success=False, error=error) + + return SkillResult( + success=True, + data={ + 'layer': kwargs['layer'], + 'unit': kwargs.get('unit', 'ha'), + 'total_area': 1250.5, + 'mean_area': 0.42, + 'areas': [1.2, 0.8, 1.5, 0.3, 2.1] + } + ) + + +@register_skill +class ExportToGeoJSONSkill(Skill): + """导出GeoJSON技能""" + name = "export_geojson" + description = "导出图层为GeoJSON格式" + category = SkillCategory.FILE_OPERATIONS + version = "1.0.0" + parameters = [ + ParameterSpec("layer", str, "输入图层", required=True), + ParameterSpec("output_path", str, "输出路径", required=True), + ] + + def execute(self, **kwargs) -> SkillResult: + valid, error = self.validate_input(**kwargs) + if not valid: + return SkillResult(success=False, error=error) + + # 模拟导出 + return SkillResult( + success=True, + data={ + 'output_path': kwargs['output_path'], + 'layer': kwargs['layer'], + 'format': 'GeoJSON', + 'size_kb': 245 + } + ) + + +@register_skill +class CreateHeatmapSkill(Skill): + """创建热力图技能""" + name = "create_heatmap" + description = "创建点数据热力图" + category = SkillCategory.VISUALIZATION + version = "1.0.0" + parameters = [ + ParameterSpec("layer", str, "输入图层", required=True), + ParameterSpec("radius", int, "影响半径", required=False, default=100), + ParameterSpec("color_ramp", str, "颜色渐变", required=False, default="hot"), + ] + + def execute(self, **kwargs) -> SkillResult: + valid, error = self.validate_input(**kwargs) + if not valid: + return SkillResult(success=False, error=error) + + return SkillResult( + success=True, + data={ + 'output_layer': f"{kwargs['layer']}_heatmap", + 'radius': kwargs.get('radius', 100), + 'color_ramp': kwargs.get('color_ramp', 'hot'), + 'render_time_ms': 234 + } + ) + + +@register_skill +class CalculateStatisticsSkill(Skill): + """统计计算技能""" + name = "calculate_statistics" + description = "计算字段统计值" + category = SkillCategory.DATA_PROCESSING + version = "1.0.0" + parameters = [ + ParameterSpec("layer", str, "输入图层", required=True), + ParameterSpec("field", str, "统计字段", required=True), + ] + + def execute(self, **kwargs) -> SkillResult: + valid, error = self.validate_input(**kwargs) + if not valid: + return SkillResult(success=False, error=error) + + # 模拟统计计算 + import random + values = [random.uniform(0, 100) for _ in range(100)] + + return SkillResult( + success=True, + data={ + 'field': kwargs['field'], + 'count': len(values), + 'mean': sum(values) / len(values), + 'min': min(values), + 'max': max(values), + 'std': (sum((x - sum(values)/len(values))**2 for x in values) / len(values)) ** 0.5 + } + ) + + +# ==================== 技能管理器 ==================== + +class QGISSkillManager: + """QGIS技能管理器""" + + def __init__(self): + self.composer = SkillComposer() + self.registry = registry + + # 注册基础技能 + for skill_name in self.registry.list_skills(): + skill = self.registry.get_skill(skill_name) + if skill: + self.composer.register(skill) + + def list_available_skills(self) -> Dict[str, List[str]]: + """列出所有可用技能""" + skills_by_category = {} + for category in SkillCategory: + skills_by_category[category.value] = self.registry.list_skills(category) + return skills_by_category + + def get_skill_info(self, skill_name: str) -> Optional[Dict]: + """获取技能详细信息""" + return self.registry.get_skill_spec(skill_name) + + def create_workflow(self, workflow_name: str, skill_names: List[str]) -> CompositeSkill: + """创建工作流（顺序组合技能）""" + return self.composer.sequential(*skill_names) + + def execute_skill(self, skill_name: str, **kwargs) -> SkillResult: + """执行单个技能""" + skill = self.registry.get_skill(skill_name) + if skill: + return skill.execute(**kwargs) + return SkillResult(success=False, error=f"Skill not found: {skill_name}") + + def create_parallel_analysis(self, skill_names: List[str]) -> CompositeSkill: + """创建并行分析工作流""" + def default_merge(results): + merged = {} + for r in results: + if r.success and isinstance(r.data, dict): + merged.update(r.data) + return merged + + return self.composer.parallel(*skill_names, merge_func=default_merge) + + +# ==================== 预定义工作流 ==================== + +class CommonWorkflows: + """常用分析工作流""" + + @staticmethod + def impact_analysis(manager: QGISSkillManager) -> CompositeSkill: + """ + 影响范围分析工作流 + + 流程：加载数据 → 缓冲区分析 → 导出结果 + """ + return manager.create_workflow( + "impact_analysis", + ["load_vector_layer", "buffer_analysis", "export_geojson"] + ) + + @staticmethod + def site_analysis(manager: QGISSkillManager) -> CompositeSkill: + """ + 场地分析工作流 + + 流程：加载 → 计算面积 → 统计 → 可视化 + """ + return manager.create_workflow( + "site_analysis", + ["load_vector_layer", "calculate_area", "calculate_statistics"] + ) + + @staticmethod + def comprehensive_analysis(manager: QGISSkillManager) -> CompositeSkill: + """ + 综合分析工作流（并行+顺序） + + 流程：加载 → [缓冲区分析 | 统计计算] → 导出 + """ + load = manager.registry.get_skill("load_vector_layer") + + # 先加载数据 + load_result = load.execute(path="sites.shp", layer_name="sites") + + # 然后并行执行多个分析 + parallel_analysis = manager.create_parallel_analysis([ + "buffer_analysis", "calculate_area", "calculate_statistics" + ]) + + # 最后导出 + return manager.composer.sequential( + "buffer_analysis", + "calculate_area", + "export_geojson" + ) + + +# ==================== 演示程序 ==================== + +def demonstrate_skill_system(): + """演示技能系统""" + print("=" * 70) + print("QGIS技能集成管理器演示") + print("=" * 70) + + # 创建管理器 + manager = QGISSkillManager() + + # 1. 列出所有可用技能 + print("\n1. 可用技能列表:") + print("-" * 70) + skills_by_category = manager.list_available_skills() + for category, skills in skills_by_category.items(): + if skills: + print(f"\n{category.upper()}:") + for skill in skills: + info = manager.get_skill_info(skill) + print(f" - {skill}: {info['description']}" if info else f" - {skill}") + + # 2. 执行单个技能 + print("\n2. 执行单个技能（加载矢量图层）:") + print("-" * 70) + result = manager.execute_skill( + "load_vector_layer", + path="data/sites.shp", + layer_name="ecological_sites" + ) + print(f"成功: {result.success}") + print(f"数据: {json.dumps(result.data, indent=2)}") + + # 3. 执行复合技能（顺序组合） + print("\n3. 执行顺序复合技能（影响范围分析）:") + print("-" * 70) + workflow = manager.create_workflow( + "impact_analysis", + ["load_vector_layer", "buffer_analysis"] + ) + result = workflow.execute( + path="data/sources.shp", + layer_name="sources", + distance=500 + ) + print(f"成功: {result.success}") + print(f"最终数据: {json.dumps(result.data, indent=2)}") + + # 4. 执行并行技能 + print("\n4. 执行并行复合技能（多个分析同时进行）:") + print("-" * 70) + parallel_workflow = manager.create_parallel_analysis([ + "calculate_area", + "calculate_statistics" + ]) + result = parallel_workflow.execute( + layer="test_layer", + field="area_ha" + ) + print(f"成功: {result.success}") + print(f"合并数据: {json.dumps(result.data, indent=2)}") + + # 5. 创建自定义工作流 + print("\n5. 创建自定义工作流（场地分析）:") + print("-" * 70) + custom_workflow = CommonWorkflows.site_analysis(manager) + print(f"工作流名称: {custom_workflow.name}") + print(f"包含技能: {[s.name for s in custom_workflow.skills]}") + + # 6. 条件组合示例 + print("\n6. 条件组合示例（根据文件大小选择处理方式）:") + print("-" * 70) + + def is_large_file(input_data): + return input_data.get('feature_count', 0) > 1000 + + conditional_workflow = manager.composer.conditional( + condition=is_large_file, + true_skill="buffer_analysis", # 大文件用简单缓冲 + # false_skill 可选 + ) + + result = conditional_workflow.execute( + layer="large_dataset", + distance=100 + ) + print(f"条件结果: {result.success}") + + +if __name__ == "__main__": + demonstrate_skill_system() +``` + +--- + +## 案例分析 + +### Claude Code的技能系统 + +Claude Code使用**Frontmatter-based技能定义**，支持热加载和动态发现： + +```python +""" +Claude Code风格的技能系统 + +技能定义在 .md 文件中，包含 YAML frontmatter +""" + +# 示例：commit_skill.md +""" +--- +name: commit +description: Create git commits with staged changes +category: git +parameters: + - name: message + type: string + description: Commit message + required: true +--- + +This skill creates git commits following best practices: +1. Runs git status and git diff first +2. Analyzes changes to draft commit message +3. Stages specific files (not all) +4. Creates commit with co-author tag +""" + +class ClaudeCodeStyleSkill: + """Claude Code风格的技能加载""" + + def __init__(self, skills_dir: Path): + self.skills_dir = skills_dir + self.skills = {} + + def load_skills(self): + """从目录加载所有技能""" + for md_file in self.skills_dir.glob("*.md"): + skill = self._parse_skill_file(md_file) + if skill: + self.skills[skill['name']] = skill + + def _parse_skill_file(self, file_path: Path) -> Optional[Dict]: + """解析技能文件""" + content = file_path.read_text() + + # 解析 frontmatter + if content.startswith('---'): + parts = content.split('---', 2) + if len(parts) >= 3: + import yaml + frontmatter = yaml.safe_load(parts[1]) + body = parts[2] + return { + 'name': frontmatter.get('name'), + 'description': frontmatter.get('description'), + 'parameters': frontmatter.get('parameters', []), + 'instructions': body.strip() + } + return None + + def get_skill_prompt(self, skill_name: str) -> str: + """获取技能的执行提示""" + if skill_name in self.skills: + skill = self.skills[skill_name] + return f""" +Skill: {skill['name']} +Description: {skill['description']} + +Parameters: +{self._format_parameters(skill['parameters'])} + +Instructions: +{skill['instructions']} +""" + return "" + + def _format_parameters(self, params: list) -> str: + return "\n".join( + f" - {p['name']}: {p['description']}" + for p in params + ) +``` + +### ENAgent的技能组合 + +ENAgent将生态网络分析分解为可组合技能： + +```python +class ENAgentSkills: + """ + ENAgent技能集 + + 将六阶段分析流程分解为可复用技能 + """ + + # 数据处理技能 + skills_data = [ + "load_landcover", + "load_elevation", + "normalize_raster", + "reclassify_landcover", + ] + + # 空间分析技能 + skills_spatial = [ + "identify_core_areas", + "calculate_resistance", + "compute_mcr", + "extract_corridors", + ] + + # 可视化技能 + skills_viz = [ + "map_sources", + "map_resistance_surface", + "map_corridors", + "export_report", + ] + + def build_analysis_workflow(self, requirements: Dict) -> CompositeSkill: + """ + 根据需求构建分析工作流 + + Args: + requirements: 包含分析需求的字典 + """ + selected_skills = [] + + # 数据准备阶段 + if requirements.get('data_sources'): + selected_skills.extend(self.skills_data) + + # 分析阶段 + if requirements.get('identify_sources'): + selected_skills.append("identify_core_areas") + if requirements.get('build_resistance'): + selected_skills.extend([ + "calculate_resistance", + "compute_mcr" + ]) + if requirements.get('extract_corridors'): + selected_skills.append("extract_corridors") + + # 输出阶段 + if requirements.get('visualize'): + selected_skills.extend(self.skills_viz) + + return self.composer.sequential(*selected_skills) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **技能粒度**：技能应该多细粒度？太细会有什么问题？太粗会有什么问题？ + +2. **接口设计**：如何设计技能接口以支持不同的数据格式？ + +3. **版本兼容性**：技能升级时如何保持向后兼容？ + +4. **技能发现**：如何让Agent自动发现和组合有用的技能？ + +### 延伸阅读 + +- **"Design Patterns: Elements of Reusable Object-Oriented Software"** - Composite Pattern +- **"Microservices Patterns"** (Richards) - 服务组合模式 +- LangChain文档 - Tool/Agent composition + +--- + +## 关键要点 + +1. **技能是Agent的功能单元**，具有明确定义的接口 +2. **组合模式**包括顺序、并行、条件和迭代四种基本类型 +3. **技能注册表**支持动态发现和加载新技能 +4. **复合技能**可以像原子技能一样被使用和组合 +5. **前后端分离**的设计使技能可以在不同上下文中复用 +6. **元数据描述**使技能可被自动发现和组合 diff --git a/officefile/supplements/03-autonomous-design/03.4-memory-and-context.md b/officefile/supplements/03-autonomous-design/03.4-memory-and-context.md new file mode 100644 index 0000000..417d55e --- /dev/null +++ b/officefile/supplements/03-autonomous-design/03.4-memory-and-context.md @@ -0,0 +1,1188 @@ +# 03.4 记忆与上下文 + +## 核心问题 + +> Agent如何记住过去的经验以改进未来表现？ +> 如何区分短期记忆和长期记忆？ +- 如何高效检索相关知识？ + +--- + +## 概念讲解 + +### 记忆的层次结构 + +``` +Agent记忆系统的层次结构 + +┌─────────────────────────────────────────────────────────────┐ +│ 感知输入 (Perception) │ +└──────────────────────────┬──────────────────────────────────┘ + │ + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ 短期记忆 (Short-term Memory) │ +│ ───────────────────────────────────────────────────────── │ +│ - 对话历史 │ +│ - 当前任务状态 │ +│ - 临时变量 │ +│ - 容量有限 (~10^4 tokens) │ +│ - 快速访问 │ +└──────────────────────────┬──────────────────────────────────┘ + │ + ↓ (选择性保存) +┌─────────────────────────────────────────────────────────────┐ +│ 长期记忆 (Long-term Memory) │ +│ ───────────────────────────────────────────────────────── │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 语义记忆 (Semantic Memory) │ │ +│ │ - 领域知识 │ │ +│ │ - 概念定义 │ │ +│ │ - 规则和模式 │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 情景记忆 (Episodic Memory) │ │ +│ │ - 过往经验 │ │ +│ │ - 任务历史 │ │ +│ │ - 成功/失败案例 │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 程序记忆 (Procedural Memory) │ │ +│ │ - 技能和操作序列 │ │ +│ │ - 工作流模板 │ │ +│ │ - 最佳实践 │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ 代理记忆 (Agent Memory) │ │ +│ │ - 工具调用记录 │ │ +│ │ - 环境状态快照 │ │ +│ │ - 用户偏好 │ │ +│ └─────────────────────────────────────────────────────┘ │ +└──────────────────────────┬──────────────────────────────────┘ + │ + ↓ (检索) +┌─────────────────────────────────────────────────────────────┐ +│ 知识检索 (Retrieval) │ +│ ───────────────────────────────────────────────────────── │ +│ - 语义搜索 │ +│ - 关联推理 │ +│ - 上下文匹配 │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 记忆与上下文的区别 + +| 维度 | 记忆 (Memory) | 上下文 (Context) | +|-----|--------------|-----------------| +| 持久性 | 持久存储 | 临时激活 | +| 范围 | 全局积累 | 当前相关 | +| 检索 | 需要查询 | 直接可用 | +| 更新 | 增量写入 | 动态构建 | +| 成本 | 存储成本高 | 计算成本高 | + +--- + +## 设计原理 + +### 短期记忆管理 + +```python +from typing import List, Dict, Any, Optional +from collections import deque +from dataclasses import dataclass, field +import time + + +@dataclass +class MemoryItem: + """记忆项""" + content: Any + timestamp: float = field(default_factory=time.time) + importance: float = 1.0 # 0-1，重要性评分 + access_count: int = 0 + tags: List[str] = field(default_factory=list) + metadata: Dict[str, Any] = field(default_factory=dict) + + +class ShortTermMemory: + """ + 短期记忆：快速访问的临时存储 + + 特点： + - 容量有限 + - 快速读写 + - 按时间顺序组织 + - 基于重要性的淘汰策略 + """ + + def __init__(self, max_items: int = 1000, max_tokens: int = 10000): + self.max_items = max_items + self.max_tokens = max_tokens + self.items: deque[MemoryItem] = deque() + self.current_tokens = 0 + + def add(self, content: Any, importance: float = 1.0, + tags: List[str] = None, metadata: Dict = None) -> None: + """添加记忆项""" + item = MemoryItem( + content=content, + importance=importance, + tags=tags or [], + metadata=metadata or {} + ) + + # 估算token数量（简化） + tokens = self._estimate_tokens(content) + self._ensure_capacity(tokens) + + self.items.append(item) + self.current_tokens += tokens + + def get_recent(self, n: int = 10) -> List[MemoryItem]: + """获取最近的项目""" + return list(self.items)[-n:] + + def get_by_tags(self, tags: List[str], + match_all: bool = False) -> List[MemoryItem]: + """按标签检索""" + results = [] + for item in self.items: + if match_all: + if all(tag in item.tags for tag in tags): + results.append(item) + else: + if any(tag in item.tags for tag in tags): + results.append(item) + return results + + def get_context_window(self, max_tokens: int = 4000) -> str: + """ + 获取上下文窗口（用于LLM输入） + + 策略：优先保留重要且最近的内容 + """ + # 按重要性和时间排序 + scored_items = [ + (item, self._score_item(item)) + for item in self.items + ] + scored_items.sort(key=lambda x: x[1], reverse=True) + + # 构建上下文 + context_parts = [] + used_tokens = 0 + + for item, _ in scored_items: + content = str(item.content) + tokens = self._estimate_tokens(content) + + if used_tokens + tokens > max_tokens: + break + + context_parts.append(content) + used_tokens += tokens + item.access_count += 1 + + return "\n\n".join(context_parts) + + def _score_item(self, item: MemoryItem) -> float: + """ + 计算记忆项的得分 + + 得分 = 重要性 * (1 + 访问次数) * 时间衰减 + """ + age = time.time() - item.timestamp + time_decay = 2 ** (-age / 3600) # 每小时衰减一半 + access_boost = 1 + item.access_count * 0.1 + return item.importance * access_boost * time_decay + + def _ensure_capacity(self, new_tokens: int) -> None: + """确保有足够容量""" + while (len(self.items) >= self.max_items or + self.current_tokens + new_tokens > self.max_tokens): + if not self.items: + break + + # 移除得分最低的项 + min_score_item = min(self.items, key=self._score_item) + self.current_tokens -= self._estimate_tokens(min_score_item.content) + self.items.remove(min_score_item) + + def _estimate_tokens(self, content: Any) -> int: + """估算token数量（简化版）""" + return len(str(content)) // 4 + + def clear(self) -> None: + """清空短期记忆""" + self.items.clear() + self.current_tokens = 0 +``` + +### 长期记忆存储 + +```python +import json +from pathlib import Path +from typing import Union, List +from abc import ABC, abstractmethod +import hashlib + + +class MemoryStore(ABC): + """记忆存储抽象基类""" + + @abstractmethod + def store(self, key: str, value: Any, metadata: Dict = None) -> bool: + """存储""" + pass + + @abstractmethod + def retrieve(self, key: str) -> Optional[Any]: + """检索""" + pass + + @abstractmethod + def search(self, query: str, limit: int = 10) -> List[Dict]: + """搜索""" + pass + + +class FileBasedMemoryStore(MemoryStore): + """ + 基于文件的长期记忆存储 + + 特点： + - 持久化到磁盘 + - JSON格式存储 + - 按类别分目录 + """ + + def __init__(self, base_path: Union[str, Path]): + self.base_path = Path(base_path) + self.base_path.mkdir(parents=True, exist_ok=True) + + # 创建子目录 + (self.base_path / "episodic").mkdir(exist_ok=True) + (self.base_path / "semantic").mkdir(exist_ok=True) + (self.base_path / "procedural").mkdir(exist_ok=True) + (self.base_path / "agent").mkdir(exist_ok=True) + + def store(self, key: str, value: Any, + memory_type: str = "episodic", + metadata: Dict = None) -> bool: + """存储记忆""" + try: + # 生成文件路径 + safe_key = hashlib.md5(key.encode()).hexdigest() + file_path = self.base_path / memory_type / f"{safe_key}.json" + + # 准备数据 + data = { + "key": key, + "value": value, + "metadata": metadata or {}, + "timestamp": time.time(), + "access_count": 0 + } + + # 如果文件存在，保留访问计数 + if file_path.exists(): + existing = json.loads(file_path.read_text()) + data["access_count"] = existing.get("access_count", 0) + + # 写入文件 + file_path.write_text(json.dumps(data, indent=2, ensure_ascii=False)) + return True + + except Exception as e: + print(f"存储失败: {e}") + return False + + def retrieve(self, key: str, + memory_type: str = "episodic") -> Optional[Any]: + """检索记忆""" + safe_key = hashlib.md5(key.encode()).hexdigest() + file_path = self.base_path / memory_type / f"{safe_key}.json" + + if not file_path.exists(): + return None + + try: + data = json.loads(file_path.read_text()) + + # 更新访问计数 + data["access_count"] = data.get("access_count", 0) + 1 + file_path.write_text(json.dumps(data, indent=2, ensure_ascii=False)) + + return data["value"] + + except Exception: + return None + + def search(self, query: str, + memory_type: str = None, + limit: int = 10) -> List[Dict]: + """搜索记忆""" + results = [] + + # 确定搜索范围 + if memory_type: + search_dirs = [self.base_path / memory_type] + else: + search_dirs = [ + self.base_path / "episodic", + self.base_path / "semantic", + self.base_path / "procedural", + self.base_path / "agent" + ] + + # 搜索文件 + for directory in search_dirs: + if not directory.exists(): + continue + + for file_path in directory.glob("*.json"): + try: + data = json.loads(file_path.read_text()) + + # 简单的关键词匹配 + content = json.dumps(data, ensure_ascii=False).lower() + if query.lower() in content: + results.append({ + "key": data.get("key"), + "value": data.get("value"), + "metadata": data.get("metadata", {}), + "timestamp": data.get("timestamp"), + "relevance": self._calculate_relevance( + query, content + ) + }) + + except Exception: + continue + + if len(results) >= limit: + break + + # 按相关性排序 + results.sort(key=lambda x: x.get("relevance", 0), reverse=True) + return results[:limit] + + def _calculate_relevance(self, query: str, content: str) -> float: + """计算相关性得分""" + query_words = set(query.lower().split()) + content_words = set(content.lower().split()) + + if not query_words: + return 0 + + # Jaccard相似度 + intersection = len(query_words & content_words) + union = len(query_words | content_words) + return intersection / union if union > 0 else 0 + + def list_all(self, memory_type: str = None) -> List[Dict]: + """列出所有记忆""" + memories = [] + + if memory_type: + search_dirs = [self.base_path / memory_type] + else: + search_dirs = [ + self.base_path / "episodic", + self.base_path / "semantic", + self.base_path / "procedural", + self.base_path / "agent" + ] + + for directory in search_dirs: + if not directory.exists(): + continue + + for file_path in directory.glob("*.json"): + try: + data = json.loads(file_path.read_text()) + memories.append({ + "key": data.get("key"), + "type": str(directory.relative_to(self.base_path)), + "timestamp": data.get("timestamp"), + "metadata": data.get("metadata", {}) + }) + except Exception: + continue + + return memories +``` + +### 语义记忆与向量检索 + +```python +import numpy as np +from typing import List, Tuple +import pickle + + +class SemanticMemory: + """ + 语义记忆：基于向量相似度的知识存储 + + 特点： + - 使用嵌入向量表示语义 + - 支持语义相似度搜索 + - 适合存储领域知识和概念 + """ + + def __init__(self, embedding_dim: int = 768): + self.embedding_dim = embedding_dim + self.memories = [] # [(embedding, content, metadata), ...] + + def add(self, content: str, embedding: np.ndarray, + metadata: Dict = None) -> None: + """添加语义记忆""" + if embedding.shape != (self.embedding_dim,): + raise ValueError(f"嵌入维度不匹配，期望 {self.embedding_dim}") + + self.memories.append({ + "embedding": embedding, + "content": content, + "metadata": metadata or {} + }) + + def retrieve(self, query_embedding: np.ndarray, + top_k: int = 5, + threshold: float = 0.7) -> List[Dict]: + """ + 检索最相关的记忆 + + Args: + query_embedding: 查询向量 + top_k: 返回前k个结果 + threshold: 相似度阈值 + + Returns: + 相关记忆列表 + """ + if not self.memories: + return [] + + # 计算余弦相似度 + similarities = [] + for memory in self.memories: + sim = self._cosine_similarity( + query_embedding, + memory["embedding"] + ) + if sim >= threshold: + similarities.append((sim, memory)) + + # 排序并返回top-k + similarities.sort(key=lambda x: x[0], reverse=True) + return [ + { + "content": mem["content"], + "metadata": mem["metadata"], + "similarity": sim + } + for sim, mem in similarities[:top_k] + ] + + def _cosine_similarity(self, a: np.ndarray, b: np.ndarray) -> float: + """计算余弦相似度""" + return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) + + def save(self, path: Union[str, Path]) -> None: + """保存到文件""" + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + + data = { + "embedding_dim": self.embedding_dim, + "memories": self.memories + } + path.write_bytes(pickle.dumps(data)) + + def load(self, path: Union[str, Path]) -> None: + """从文件加载""" + path = Path(path) + if not path.exists(): + return + + data = pickle.loads(path.read_bytes()) + self.embedding_dim = data["embedding_dim"] + self.memories = data["memories"] + + +class MockEmbeddingModel: + """ + 模拟嵌入模型 + + 实际实现中会使用真实的嵌入模型（如Sentence-BERT、OpenAI embeddings等） + """ + + def __init__(self, dim: int = 768): + self.dim = dim + self.vocab = {} # 词到向量的映射 + + def encode(self, text: str) -> np.ndarray: + """ + 编码文本为向量 + + 这是一个简化的实现，实际会使用深度学习模型 + """ + # 简单的词袋模型 + 随机投影 + words = text.lower().split() + + # 为新词创建向量 + for word in words: + if word not in self.vocab: + self.vocab[word] = np.random.randn(self.dim) * 0.1 + + # 聚合词向量 + vectors = [self.vocab[w] for w in words if w in self.vocab] + if vectors: + return np.mean(vectors, axis=0) + return np.zeros(self.dim) +``` + +### 程序记忆：技能与工作流 + +```python +class ProceduralMemory: + """ + 程序记忆：存储技能和工作流程 + + 特点： + - 存储可执行的技能 + - 存储工作流模板 + - 存储最佳实践 + """ + + def __init__(self): + self.skills = {} # name: skill_definition + self.workflows = {} # name: workflow_definition + self.practices = {} # context: best_practice + + def learn_skill(self, name: str, + input_spec: List[Dict], + output_spec: Dict, + implementation: str) -> None: + """ + 学习新技能 + + Args: + name: 技能名称 + input_spec: 输入规格列表 + output_spec: 输出规格 + implementation: 实现代码或描述 + """ + self.skills[name] = { + "name": name, + "input_spec": input_spec, + "output_spec": output_spec, + "implementation": implementation, + "learned_at": time.time(), + "usage_count": 0 + } + + def learn_workflow(self, name: str, + steps: List[Dict], + description: str = "") -> None: + """ + 学习工作流 + + Args: + name: 工作流名称 + steps: 步骤列表 + description: 描述 + """ + self.workflows[name] = { + "name": name, + "steps": steps, + "description": description, + "learned_at": time.time(), + "usage_count": 0 + } + + def record_best_practice(self, context: str, + practice: str, + success_rate: float = 1.0) -> None: + """ + 记录最佳实践 + + Args: + context: 应用场景 + practice: 实践描述 + success_rate: 成功率 + """ + self.practices[context] = { + "practice": practice, + "success_rate": success_rate, + "recorded_at": time.time() + } + + def get_workflow(self, name: str) -> Optional[Dict]: + """获取工作流""" + if name in self.workflows: + self.workflows[name]["usage_count"] += 1 + return self.workflows[name] + return None + + def find_similar_workflow(self, goal: str) -> List[Dict]: + """查找相似的工作流""" + results = [] + for name, workflow in self.workflows.items(): + # 简单的关键词匹配 + if any(word in workflow.get("description", "").lower() + for word in goal.lower().split()): + results.append(workflow) + + return sorted( + results, + key=lambda w: w["usage_count"], + reverse=True + ) + + def get_best_practice(self, context: str) -> Optional[str]: + """获取最佳实践""" + if context in self.practices: + return self.practices[context]["practice"] + + # 查找部分匹配 + for key, value in self.practices.items(): + if key in context or context in key: + return value["practice"] + + return None +``` + +--- + +## 代码示例 + +### 专家知识积累系统 + +```python +""" +专家知识积累系统 + +演示如何构建一个完整的记忆系统， +支持短期和长期记忆，以及智能检索 +""" +import time +import json +from pathlib import Path +from typing import List, Dict, Any, Optional +from dataclasses import dataclass, field +import hashlib + + +@dataclass +class ExpertKnowledge: + """专家知识条目""" + topic: str + content: str + category: str + confidence: float = 0.8 # 0-1，置信度 + source: str = "" + examples: List[str] = field(default_factory=list) + related_topics: List[str] = field(default_factory=list) + created_at: float = field(default_factory=time.time) + last_accessed: float = field(default_factory=time.time) + access_count: int = 0 + + +class ExpertKnowledgeSystem: + """ + 专家知识积累系统 + + 功能： + 1. 存储和检索领域知识 + 2. 跟踪知识使用情况 + 3. 发现知识关联 + 4. 生成知识摘要 + """ + + def __init__(self, storage_path: str = "./knowledge_base"): + self.storage_path = Path(storage_path) + self.storage_path.mkdir(parents=True, exist_ok=True) + + # 内存中的知识索引 + self.knowledge_index: Dict[str, ExpertKnowledge] = {} + + # 短期记忆缓存 + self.recently_accessed: List[str] = [] + + # 加载已有知识 + self._load_knowledge() + + def add_knowledge(self, + topic: str, + content: str, + category: str, + confidence: float = 0.8, + source: str = "", + examples: List[str] = None, + related_topics: List[str] = None) -> bool: + """添加知识条目""" + knowledge = ExpertKnowledge( + topic=topic, + content=content, + category=category, + confidence=confidence, + source=source, + examples=examples or [], + related_topics=related_topics or [] + ) + + # 生成唯一ID + knowledge_id = self._generate_id(topic, category) + + # 存储到索引 + self.knowledge_index[knowledge_id] = knowledge + + # 持久化 + self._save_knowledge(knowledge_id, knowledge) + + return True + + def query(self, query: str, + category: str = None, + top_k: int = 5) -> List[Dict]: + """ + 查询知识 + + Args: + query: 查询文本 + category: 知识类别过滤 + top_k: 返回结果数量 + + Returns: + 匹配的知识条目列表 + """ + results = [] + query_lower = query.lower() + + for knowledge_id, knowledge in self.knowledge_index.items(): + # 类别过滤 + if category and knowledge.category != category: + continue + + # 计算相关性得分 + score = self._calculate_relevance(query_lower, knowledge) + + if score > 0: + results.append({ + "id": knowledge_id, + "knowledge": knowledge, + "score": score + }) + + # 排序并返回top-k + results.sort(key=lambda x: x["score"], reverse=True) + + # 更新访问记录 + for result in results[:top_k]: + knowledge = result["knowledge"] + knowledge.access_count += 1 + knowledge.last_accessed = time.time() + self.recently_accessed.append(result["id"]) + + return [ + { + "topic": r["knowledge"].topic, + "content": r["knowledge"].content, + "category": r["knowledge"].category, + "confidence": r["knowledge"].confidence, + "relevance": r["score"], + "examples": r["knowledge"].examples + } + for r in results[:top_k] + ] + + def get_related_topics(self, topic: str) -> List[str]: + """获取相关主题""" + related = set() + + # 查找直接相关 + for knowledge in self.knowledge_index.values(): + if topic.lower() in knowledge.topic.lower(): + related.update(knowledge.related_topics) + + # 查找包含相同关键词的主题 + topic_words = set(topic.lower().split()) + for knowledge in self.knowledge_index.values(): + knowledge_words = set(knowledge.topic.lower().split()) + if topic_words & knowledge_words: # 有交集 + related.add(knowledge.topic) + + return list(related) + + def generate_summary(self, category: str = None) -> str: + """ + 生成知识库摘要 + + Args: + category: 指定类别，None则生成全部摘要 + + Returns: + 知识库摘要文本 + """ + knowledges = [ + k for k in self.knowledge_index.values() + if category is None or k.category == category + ] + + if not knowledges: + return f"知识库中没有{'类别' + category if category else ''}的知识。" + + # 按类别分组 + by_category: Dict[str, List[ExpertKnowledge]] = {} + for k in knowledges: + if k.category not in by_category: + by_category[k.category] = [] + by_category[k.category].append(k) + + # 构建摘要 + summary_parts = [] + + for cat, items in by_category.items(): + summary_parts.append(f"\n## {cat.upper()}") + summary_parts.append(f"共 {len(items)} 条知识\n") + + # 列出高频访问的知识 + top_items = sorted(items, key=lambda x: x.access_count, reverse=True)[:5] + + for item in top_items: + summary_parts.append( + f"- **{item.topic}** " + f"(置信度: {item.confidence:.2f}, " + f"访问: {item.access_count}次)" + ) + summary_parts.append(f" {item.content[:100]}...") + + return "\n".join(summary_parts) + + def learn_from_interaction(self, + user_question: str, + agent_answer: str, + user_feedback: str = None) -> None: + """ + 从交互中学习 + + Args: + user_question: 用户问题 + agent_answer: Agent回答 + user_feedback: 用户反馈 + """ + # 分析问题，提取关键概念 + topic = self._extract_topic(user_question) + + # 如果得到正面反馈，将回答存为知识 + if user_feedback and "good" in user_feedback.lower(): + self.add_knowledge( + topic=topic, + content=f"问题: {user_question}\n答案: {agent_answer}", + category="qa", + confidence=0.7, + source="interaction" + ) + + def _calculate_relevance(self, query: str, + knowledge: ExpertKnowledge) -> float: + """计算查询与知识的相关性""" + score = 0.0 + + # 主题匹配 + if query in knowledge.topic.lower(): + score += 1.0 + + # 内容匹配 + query_words = set(query.split()) + content_words = set(knowledge.content.lower().split()) + + if query_words & content_words: + intersection = len(query_words & content_words) + score += intersection * 0.1 + + # 类别匹配 + if query in knowledge.category.lower(): + score += 0.5 + + # 相关主题匹配 + for related in knowledge.related_topics: + if query in related.lower(): + score += 0.3 + + # 访问热度加成 + score += min(knowledge.access_count * 0.01, 0.5) + + # 置信度加权 + score *= knowledge.confidence + + return score + + def _extract_topic(self, text: str) -> str: + """从文本中提取主题（简化版）""" + # 简单实现：取前几个关键词 + words = text.lower().split() + stop_words = {"what", "how", "where", "when", "why", "the", "a", "an", "is", "are"} + + topic_words = [w for w in words[:5] if w not in stop_words and len(w) > 2] + return " ".join(topic_words[:3]) if topic_words else text[:30] + + def _generate_id(self, topic: str, category: str) -> str: + """生成知识ID""" + content = f"{topic}:{category}" + return hashlib.md5(content.encode()).hexdigest()[:16] + + def _save_knowledge(self, knowledge_id: str, + knowledge: ExpertKnowledge) -> None: + """持久化知识""" + category_dir = self.storage_path / knowledge.category + category_dir.mkdir(exist_ok=True) + + file_path = category_dir / f"{knowledge_id}.json" + + data = { + "topic": knowledge.topic, + "content": knowledge.content, + "category": knowledge.category, + "confidence": knowledge.confidence, + "source": knowledge.source, + "examples": knowledge.examples, + "related_topics": knowledge.related_topics, + "created_at": knowledge.created_at, + "last_accessed": knowledge.last_accessed, + "access_count": knowledge.access_count + } + + file_path.write_text(json.dumps(data, indent=2, ensure_ascii=False)) + + def _load_knowledge(self) -> None: + """加载已有知识""" + for category_dir in self.storage_path.iterdir(): + if not category_dir.is_dir(): + continue + + for file_path in category_dir.glob("*.json"): + try: + data = json.loads(file_path.read_text()) + knowledge = ExpertKnowledge(**data) + knowledge_id = self._generate_id( + knowledge.topic, + knowledge.category + ) + self.knowledge_index[knowledge_id] = knowledge + except Exception as e: + print(f"加载知识失败 {file_path}: {e}") + + +# ==================== 演示程序 ==================== + +def demonstrate_knowledge_system(): + """演示专家知识系统""" + print("=" * 70) + print("专家知识积累系统演示") + print("=" * 70) + + # 创建系统 + system = ExpertiseKnowledgeSystem(storage_path="./demo_knowledge") + + # 1. 添加生态学知识 + print("\n1. 添加专家知识...") + system.add_knowledge( + topic="生态源地识别", + content="生态源地是生物物种生存、繁衍和扩散的核心区域。通常选择面积较大、连通性好、生态价值高的斑块作为源地。", + category="ecology", + confidence=0.95, + source="景观生态学", + examples=["MSPA分析", "形态空间格局分析"], + related_topics=["生态廊道", "阻力面", "MCR分析"] + ) + + system.add_knowledge( + topic="最小累积阻力模型", + content="MCR模型通过计算物种从源到目的地运动过程中克服阻力的最小累积成本来确定生态廊道。公式：MCR = f(min ΣDij × Rij)", + category="spatial_analysis", + confidence=0.9, + source="景观生态学", + examples=["廊道提取", "连通性分析"], + related_topics=["生态源地", "阻力面", "图算法"] + ) + + system.add_knowledge( + topic="NDVI植被指数", + content="归一化植被指数(NDVI) = (NIR - Red) / (NIR + Red)，用于监测植被生长状况，范围-1到1，值越大表示植被越好。", + category="remote_sensing", + confidence=0.98, + source="遥感原理", + examples=["植被覆盖度", "生物量估算"], + related_topics=["EVI", "SAVI", "遥感指数"] + ) + + # 2. 查询知识 + print("\n2. 查询知识（'生态廊道'）...") + results = system.query("生态廊道", top_k=3) + + for i, result in enumerate(results, 1): + print(f"\n结果 {i}:") + print(f" 主题: {result['topic']}") + print(f" 类别: {result['category']}") + print(f" 相关性: {result['relevance']:.2f}") + print(f" 内容: {result['content'][:80]}...") + + # 3. 获取相关主题 + print("\n3. 获取相关主题（'源地'）...") + related = system.get_related_topics("源地") + print(f"相关主题: {related}") + + # 4. 生成摘要 + print("\n4. 生成知识库摘要...") + summary = system.generate_summary() + print(summary) + + # 5. 从交互学习 + print("\n5. 从交互中学习...") + system.learn_from_interaction( + user_question="如何判断一个斑块是否适合作为生态源地？", + agent_answer="判断生态源地主要考虑：1)面积阈值（通常大于核心区面积）；2)形状指数（越紧凑越好）；3)连通性（与其他源地的距离）；4)生态价值（植被覆盖、物种丰富度）。", + user_feedback="good, this helps!" + ) + + print("\n学习后的新查询:") + results = system.query("判断斑块", category="qa") + for r in results: + print(f" - {r['topic']}: {r['content'][:60]}...") + + +if __name__ == "__main__": + demonstrate_knowledge_system() +``` + +--- + +## 案例分析 + +### Claude Code的对话上下文管理 + +```python +class ConversationContext: + """ + Claude Code风格的对话上下文管理 + + 特点： + - 维护完整对话历史 + - 智能截断以控制token使用 + - 保留关键信息 + """ + + def __init__(self, max_tokens: int = 8000): + self.max_tokens = max_tokens + self.messages = [] + self.system_prompt = "" + self.key_facts = [] + + def add_message(self, role: str, content: str) -> None: + """添加消息""" + self.messages.append({ + "role": role, + "content": content, + "timestamp": time.time() + }) + + def get_context(self) -> str: + """ + 获取当前上下文 + + 策略： + 1. 始终包含系统提示 + 2. 保留最近的消息 + 3. 如果空间允许，保留关键事实 + """ + context_parts = [] + + # 系统提示 + if self.system_prompt: + context_parts.append(f"System: {self.system_prompt}") + + # 关键事实 + if self.key_facts: + context_parts.append("\nKey Facts:") + for fact in self.key_facts: + context_parts.append(f"- {fact}") + + # 最近的消息（在token预算内） + available_tokens = self.max_tokens - self._count_tokens("\n".join(context_parts)) + + recent_messages = self._get_recent_messages(available_tokens) + for msg in recent_messages: + role = msg["role"].capitalize() + context_parts.append(f"{role}: {msg['content']}") + + return "\n\n".join(context_parts) + + def extract_key_fact(self, content: str) -> None: + """从内容中提取关键事实""" + # 简化实现：包含特定模式的句子 + import re + + patterns = [ + r"用户希望.*", + r"目标是.*", + r"需要注意.*", + r"重要.*" + ] + + for pattern in patterns: + matches = re.findall(pattern, content) + self.key_facts.extend(matches) + + def _count_tokens(self, text: str) -> int: + """估算token数量""" + return len(text) // 4 + + def _get_recent_messages(self, max_tokens: int) -> List[Dict]: + """获取最近的消息（在token限制内）""" + result = [] + used_tokens = 0 + + for msg in reversed(self.messages): + tokens = self._count_tokens(msg["content"]) + if used_tokens + tokens > max_tokens: + break + result.append(msg) + used_tokens += tokens + + return list(reversed(result)) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **记忆容量**：如何平衡记忆容量和检索效率？ + +2. **记忆更新**：如何处理过时或错误的记忆？ + +3. **隐私保护**：长期记忆中如何保护敏感信息？ + +4. **记忆遗忘**：是否应该模拟人类的遗忘机制？ + +### 延伸阅读 + +- **"Memory Systems"** (Atkinson & Shiffrin) - 记忆心理学模型 +- **"Vector Databases for AI"** - 向量数据库技术 +- RAG论文 - "Retrieval-Augmented Generation" + +--- + +## 关键要点 + +1. **记忆分层次**：短期记忆快速访问，长期记忆持久存储 +2. **语义记忆**使用向量嵌入支持语义相似度检索 +3. **情景记忆**记录具体经验和历史事件 +4. **程序记忆**存储技能和工作流程 +5. **检索策略**需要平衡相关性、时效性和访问频率 +6. **知识积累**可以通过从交互中学习来实现 diff --git a/officefile/supplements/03-autonomous-design/03.5-planning-and-execution.md b/officefile/supplements/03-autonomous-design/03.5-planning-and-execution.md new file mode 100644 index 0000000..8c6ee8b --- /dev/null +++ b/officefile/supplements/03-autonomous-design/03.5-planning-and-execution.md @@ -0,0 +1,1175 @@ +# 03.5 规划与执行 + +## 核心问题 + +> Agent如何将复杂目标分解为可执行步骤？ +> 当环境变化时，如何动态调整计划？ +> 如何平衡规划深度与执行效率？ + +--- + +## 概念讲解 + +### 规划的基本概念 + +**规划 (Planning)** 是寻找从初始状态到目标状态的行动序列的过程： + +``` +规划问题的基本要素 + + 状态空间 (State Space) + │ + ├─── 初始状态: S₀ + ├─── 目标状态: S_goal + └─── 中间状态: S₁, S₂, ..., Sₙ + │ + ↓ + 行动 (Actions) + │ + ├─── 前置条件: preconditions + ├─── 效果: effects + └─── 代价: cost + │ + ↓ + 规划 = 行动序列 [a₁, a₂, ..., aₙ] + 使得: S₀ ─a₁→ S₁ ─a₂→ S₂ ... ─aₙ→ S_goal +``` + +### 规划与执行的循环 + +``` + ┌─────────────────────────────────────────────────────┐ + │ 规划-执行循环 │ + │ (Plan-Execute Loop) │ + ├─────────────────────────────────────────────────────┤ + │ │ + │ ┌─────────┐ │ + │ │ 目标 │ │ + │ └────┬────┘ │ + │ │ │ + │ ↓ │ + │ ┌─────────┐ ┌─────────────┐ │ + │ │ 规划 │ ───→ │ 执行步骤 │ │ + │ │ Planner│ │ Executor │ │ + │ └────┬────┘ └──────┬──────┘ │ + │ │ │ │ + │ │ ↓ │ + │ │ ┌─────────────┐ │ + │ │ │ 观察结果 │ │ + │ │ └──────┬──────┘ │ + │ │ │ │ + │ │ ↓ │ + │ │ ┌─────────────┐ │ + │ │ │ 监控状态 │ │ + │ │ └──────┬──────┘ │ + │ │ │ │ + │ ↓ ↓ │ + │ ┌─────────────────────────────┐ │ + │ │ 是否需要重新规划？ │ │ + │ └─────────────┬───────────────┘ │ + │ 是 否 │ + │ │ │ │ + │ └──────────────┘ │ + │ │ │ + │ ↓ │ + │ ┌─────────────┐ │ + │ │ 继续执行 │ │ + │ └─────────────┘ │ + │ │ + └─────────────────────────────────────────────────────┘ +``` + +### 规划方法的分类 + +``` +规划方法谱系 + + ┌────────────────────────────────────────────────────────┐ + │ │ + │ 前向搜索 (Forward Search) │ + │ ──────────────────────── │ + │ 从初始状态向目标状态搜索 │ + │ 适合：目标明确，分支因子较小 │ + │ │ + │ ↓ │ + │ │ + │ 后向搜索 (Backward Search) │ + │ ─────────────────────── │ + │ 从目标状态向初始状态搜索 │ + │ 适合：目标状态较少，起始状态较多 │ + │ │ + │ ↓ │ + │ │ + │ 双向搜索 (Bidirectional Search) │ + │ ────────────────────────────── │ + │ 同时从两端搜索，在中间汇合 │ + │ 适合：状态空间大，双向都可搜索 │ + │ │ + │ ↓ │ + │ │ + │ 分层规划 (Hierarchical Planning) │ + │ ───────────────────────────── │ + │ 先规划高层抽象，再细化具体步骤 │ + │ 适合：复杂任务，有多层抽象 │ + │ │ + └────────────────────────────────────────────────────────┘ +``` + +--- + +## 设计原理 + +### 前向搜索规划 + +```python +from typing import Callable, List, Dict, Any, Optional, Tuple +from dataclasses import dataclass +from enum import Enum +import heapq +from collections import deque + + +class PlanningStatus(Enum): + """规划状态""" + SUCCESS = "success" + FAILURE = "failure" + IN_PROGRESS = "in_progress" + NO_PLAN = "no_plan" + + +@dataclass +class Action: + """行动定义""" + name: str + preconditions: Callable[[Any], bool] # 前置条件检查 + effects: Callable[[Any], Any] # 状态转换 + cost: float = 1.0 # 行动代价 + + +@dataclass +class Plan: + """计划""" + actions: List[Action] + expected_final_state: Any + total_cost: float + steps: List[str] + + +class ForwardSearchPlanner: + """ + 前向搜索规划器 + + 从初始状态开始，逐步应用行动直到达到目标 + """ + + def __init__(self, + actions: List[Action], + goal_test: Callable[[Any], bool], + max_depth: int = 100, + heuristic: Callable[[Any], float] = None): + """ + Args: + actions: 可用行动列表 + goal_test: 目标测试函数 + max_depth: 最大搜索深度 + heuristic: 启式函数（估计到目标的距离） + """ + self.actions = actions + self.goal_test = goal_test + self.max_depth = max_depth + self.heuristic = heuristic or (lambda s: 0) + + # 搜索统计 + self.nodes_expanded = 0 + self.nodes_visited = 0 + + def plan(self, initial_state: Any) -> Optional[Plan]: + """ + 执行前向搜索规划 + + 支持的搜索算法： + - BFS（无启发式） + - UCS（uniform cost search，无启发式但有代价） + - A*（有启发式） + """ + # 搜索节点：(f_score, g_score, state, action_sequence) + initial_node = ( + self.heuristic(initial_state), + 0, + initial_state, + [] + ) + + open_set = [initial_node] + closed_set = set() + + while open_set: + # 获取最优节点 + f, g, current_state, action_sequence = heapq.heappop(open_set) + + # 检查是否已访问 + state_hash = self._hash_state(current_state) + if state_hash in closed_set: + continue + closed_set.add(state_hash) + self.nodes_visited += 1 + + # 检查是否达到目标 + if self.goal_test(current_state): + return Plan( + actions=action_sequence, + expected_final_state=current_state, + total_cost=g, + steps=[a.name for a in action_sequence] + ) + + # 深度限制 + if len(action_sequence) >= self.max_depth: + continue + + # 扩展节点 + self.nodes_expanded += 1 + for action in self.actions: + # 检查前置条件 + if action.preconditions(current_state): + # 应用行动 + new_state = action.effects(current_state) + + # 创建新节点 + new_g = g + action.cost + new_f = new_g + self.heuristic(new_state) + new_sequence = action_sequence + [action] + + heapq.heappush(open_set, ( + new_f, + new_g, + new_state, + new_sequence + )) + + return None + + def _hash_state(self, state: Any) -> int: + """状态哈希（用于去重）""" + return hash(str(state)) + + +class BackwardSearchPlanner: + """ + 后向搜索规划器 + + 从目标状态开始，反向应用行动直到回到初始状态 + """ + + def __init__(self, + actions: List[Action], + initial_state: Any, + max_depth: int = 100): + """ + Args: + actions: 可用行动列表 + initial_state: 初始状态 + max_depth: 最大搜索深度 + """ + self.actions = actions + self.initial_state = initial_state + self.max_depth = max_depth + + def plan(self, goal_state: Any) -> Optional[Plan]: + """ + 执行后向搜索规划 + + 反向行动需要能够"撤销"原行动的效果 + """ + # 构建反向行动 + reverse_actions = self._build_reverse_actions() + + # 从目标状态搜索到初始状态 + planner = ForwardSearchPlanner( + actions=reverse_actions, + goal_test=lambda s: s == self.initial_state, + max_depth=self.max_depth + ) + + result = planner.plan(goal_state) + + if result: + # 反转行动序列 + result.actions = list(reversed(result.actions)) + result.steps = list(reversed(result.steps)) + + return result + + def _build_reverse_actions(self) -> List[Action]: + """构建反向行动（简化实现）""" + # 实际实现需要更复杂的逻辑 + # 这里假设行动是可逆的 + return self.actions +``` + +### 分层规划 + +```python +class HierarchicalTask: + """分层任务""" + + def __init__(self, name: str, + is_primitive: bool = False, + subtasks: List['HierarchicalTask'] = None, + implementation: Callable = None): + self.name = name + self.is_primitive = is_primitive + self.subtasks = subtasks or [] + self.implementation = implementation + + +class HierarchicalPlanner: + """ + 分层规划器 (HTN - Hierarchical Task Network) + + 将复杂任务分解为可执行的原子任务 + """ + + def __init__(self, root_task: HierarchicalTask): + self.root_task = root_task + self.task_hierarchy = self._build_hierarchy(root_task) + + def _build_hierarchy(self, task: HierarchicalTask, + level: int = 0) -> Dict: + """构建任务层次结构""" + return { + 'task': task, + 'level': level, + 'children': [ + self._build_hierarchy(t, level + 1) + for t in task.subtasks + ] + } + + def plan(self, initial_state: Dict) -> List[Action]: + """ + 执行分层规划 + + 1. 从根任务开始 + 2. 递归分解非原子任务 + 3. 收集所有原子任务 + """ + execution_plan = [] + self._decompose_task(self.root_task, initial_state, execution_plan) + return execution_plan + + def _decompose_task(self, task: HierarchicalTask, + state: Dict, plan: List) -> bool: + """分解任务""" + if task.is_primitive: + # 原子任务，直接执行 + if task.implementation: + result = task.implementation(state) + plan.append(Action( + name=task.name, + preconditions=lambda s: True, + effects=lambda s: result, + cost=1.0 + )) + return True + return False + + # 非原子任务，递归分解子任务 + for subtask in task.subtasks: + if not self._decompose_task(subtask, state, plan): + return False + + return True + + def visualize_hierarchy(self) -> str: + """可视化任务层次""" + lines = [] + + def print_node(node, prefix="", is_last=True): + connector = "└── " if is_last else "├── " + lines.append(f"{prefix}{connector}{node['task'].name}") + + children = node['children'] + for i, child in enumerate(children): + is_last_child = (i == len(children) - 1) + extension = " " if is_last else "│ " + print_node(child, prefix + extension, is_last_child) + + print_node(self.task_hierarchy) + return "\n".join(lines) +``` + +### 动态重规划 + +```python +class ReplanningAgent: + """ + 支持动态重规划的Agent + + 特点： + 1. 持续监控执行状态 + 2. 检测计划失效 + 3. 触发重新规划 + """ + + def __init__(self, planner, monitor_interval: float = 1.0): + self.planner = planner + self.monitor_interval = monitor_interval + + self.current_plan: Optional[Plan] = None + self.executed_steps: List[str] = [] + self.plan_status = PlanningStatus.IN_PROGRESS + + def execute_with_monitoring(self, + initial_state: Any, + environment) -> Any: + """ + 带监控的执行 + + Args: + initial_state: 初始状态 + environment: 环境接口（支持 step() 和 get_state()） + """ + # 初始规划 + self.current_plan = self.planner.plan(initial_state) + + if not self.current_plan: + self.plan_status = PlanningStatus.NO_PLAN + return None + + current_state = initial_state + + # 执行-监控循环 + for action in self.current_plan.actions: + print(f"执行: {action.name}") + + # 执行行动 + try: + result = environment.execute(action.name, current_state) + self.executed_steps.append(action.name) + + # 更新状态 + current_state = environment.get_state() + + # 检查是否需要重新规划 + if self._should_replan(current_state, action, result): + print("检测到计划失效，重新规划...") + self._replan(current_state) + + # 检查计划是否完成 + if self._plan_complete(current_state): + self.plan_status = PlanningStatus.SUCCESS + break + + except Exception as e: + print(f"执行失败: {e}") + self._replan(current_state) + + return current_state + + def _should_replan(self, state: Any, + last_action: Action, + result: Any) -> bool: + """判断是否需要重新规划""" + # 检查1：行动结果是否符合预期 + if not result.get('success', True): + return True + + # 检查2：状态是否发生意外变化 + if result.get('unexpected_change', False): + return True + + # 检查3：目标是否已改变 + if result.get('goal_changed', False): + return True + + return False + + def _replan(self, current_state: Any) -> None: + """执行重新规划""" + # 保存已执行的步骤 + executed = self.executed_steps.copy() + + # 重新规划 + new_plan = self.planner.plan(current_state) + + if new_plan: + self.current_plan = new_plan + print(f"新计划: {' → '.join(new_plan.steps)}") + else: + print("无法找到新计划") + self.plan_status = PlanningStatus.FAILURE + + def _plan_complete(self, state: Any) -> bool: + """检查计划是否完成""" + return self.planner.goal_test(state) +``` + +--- + +## 代码示例 + +### 空间分析任务规划器 + +```python +""" +空间分析任务规划器 + +演示如何为GIS分析任务创建分层规划系统 +""" +from typing import List, Dict, Any, Optional, Callable +from dataclasses import dataclass, field +from enum import Enum +import json + + +class TaskType(Enum): + """任务类型""" + DATA_PREPARATION = "data_preparation" + ANALYSIS = "analysis" + VISUALIZATION = "visualization" + EXPORT = "export" + + +class TaskStatus(Enum): + """任务状态""" + PENDING = "pending" + IN_PROGRESS = "in_progress" + COMPLETED = "completed" + FAILED = "failed" + SKIPPED = "skipped" + + +@dataclass +class Task: + """任务定义""" + id: str + name: str + type: TaskType + description: str = "" + depends_on: List[str] = field(default_factory=list) + parameters: Dict[str, Any] = field(default_factory=dict) + status: TaskStatus = TaskStatus.PENDING + result: Any = None + error: Optional[str] = None + + def is_ready(self, completed_tasks: set) -> bool: + """检查任务是否准备就绪（依赖已完成）""" + return all(dep in completed_tasks for dep in self.depends_on) + + def to_dict(self) -> Dict: + """转换为字典""" + return { + 'id': self.id, + 'name': self.name, + 'type': self.type.value, + 'description': self.description, + 'depends_on': self.depends_on, + 'parameters': self.parameters, + 'status': self.status.value, + 'error': self.error + } + + +class SpatialAnalysisPlanner: + """ + 空间分析任务规划器 + + 功能： + 1. 定义任务依赖关系 + 2. 生成执行计划 + 3. 执行任务序列 + 4. 处理失败和重试 + """ + + def __init__(self, name: str = "Spatial Analysis"): + self.name = name + self.tasks: Dict[str, Task] = {} + self.execution_history: List[Dict] = [] + + def add_task(self, + task_id: str, + name: str, + task_type: TaskType, + description: str = "", + depends_on: List[str] = None, + parameters: Dict = None) -> 'SpatialAnalysisPlanner': + """添加任务""" + self.tasks[task_id] = Task( + id=task_id, + name=name, + type=task_type, + description=description, + depends_on=depends_on or [], + parameters=parameters or {} + ) + return self + + def get_execution_plan(self) -> List[List[str]]: + """ + 获取执行计划（分层级） + + 返回每层可并行执行的任务ID列表 + """ + plan = [] + completed = set() + remaining = set(self.tasks.keys()) + + while remaining: + # 找出所有准备就绪的任务 + ready = [ + task_id for task_id in remaining + if self.tasks[task_id].is_ready(completed) + ] + + if not ready: + # 循环依赖 + raise ValueError("检测到循环依赖或无法满足的依赖") + + plan.append(ready) + completed.update(ready) + remaining -= set(ready) + + return plan + + def execute(self, + executor: Callable[[Task], Any], + max_retries: int = 1) -> Dict[str, Any]: + """ + 执行计划 + + Args: + executor: 任务执行器函数 + max_retries: 最大重试次数 + + Returns: + 执行结果摘要 + """ + plan = self.get_execution_plan() + results = {} + completed = set() + + for level, task_ids in enumerate(plan): + print(f"\n=== 执行层级 {level + 1}/{len(plan)} ===") + print(f"任务: {', '.join(task_ids)}") + + # 可以并行执行（这里简化为顺序） + for task_id in task_ids: + task = self.tasks[task_id] + + for attempt in range(max_retries + 1): + try: + print(f" 执行: {task.name} (尝试 {attempt + 1})") + task.status = TaskStatus.IN_PROGRESS + + # 执行任务 + result = executor(task) + task.result = result + task.status = TaskStatus.COMPLETED + completed.add(task_id) + results[task_id] = result + + # 记录历史 + self.execution_history.append({ + 'task_id': task_id, + 'status': 'completed', + 'attempt': attempt + 1 + }) + + break + + except Exception as e: + error_msg = str(e) + task.error = error_msg + + if attempt < max_retries: + print(f" 失败，重试: {error_msg}") + else: + task.status = TaskStatus.FAILED + print(f" 最终失败: {error_msg}") + + self.execution_history.append({ + 'task_id': task_id, + 'status': 'failed', + 'error': error_msg, + 'attempts': attempt + 1 + }) + + # 决定是否继续 + if task_id in self._get_critical_tasks(): + print("关键任务失败，终止执行") + return results + + return results + + def visualize_plan(self) -> str: + """可视化执行计划（DAG）""" + plan = self.get_execution_plan() + + lines = [f"\n{self.name} - 执行计划"] + lines.append("=" * 50) + + for level, task_ids in enumerate(plan): + lines.append(f"\n层级 {level + 1}:") + for task_id in task_ids: + task = self.tasks[task_id] + deps = f" (依赖: {', '.join(task.depends_on)})" if task.depends_on else "" + lines.append(f" - {task.name}{deps}") + + return "\n".join(lines) + + def _get_critical_tasks(self) -> set: + """获取关键任务（失败会终止整个流程）""" + # 简化实现：所有数据准备任务是关键的 + return { + t.id for t in self.tasks.values() + if t.type == TaskType.DATA_PREPARATION + } + + +# ==================== 预定义分析工作流 ==================== + +class CommonAnalysisWorkflows: + """常见空间分析工作流模板""" + + @staticmethod + def ecological_network_analysis() -> SpatialAnalysisPlanner: + """ + 生态网络分析工作流 + + 六阶段： + 1. 数据准备 + 2. 源地识别 + 3. 阻力面构建 + 4. MCR分析 + 5. 廊道提取 + 6. 结果输出 + """ + planner = SpatialAnalysisPlanner("生态网络分析") + + # 数据准备阶段 + planner.add_task("load_landcover", "加载土地覆盖数据", + TaskType.DATA_PREPARATION, + "加载研究区土地覆盖栅格数据") + planner.add_task("load_elevation", "加载高程数据", + TaskType.DATA_PREPARATION, + "加载DEM高程数据") + + # 分析阶段 + planner.add_task("identify_sources", "识别生态源地", + TaskType.ANALYSIS, + "基于形态空间格局分析识别核心生境斑块", + depends_on=["load_landcover"]) + + planner.add_task("build_resistance", "构建生态阻力面", + TaskType.ANALYSIS, + "基于土地覆盖类型赋值构建阻力面", + depends_on=["load_landcover", "load_elevation"]) + + planner.add_task("mcr_analysis", "最小累积阻力分析", + TaskType.ANALYSIS, + "计算从各源地到空间各点的最小累积阻力", + depends_on=["identify_sources", "build_resistance"]) + + planner.add_task("extract_corridors", "提取生态廊道", + TaskType.ANALYSIS, + "基于MCR结果提取潜在生态廊道", + depends_on=["mcr_analysis"]) + + # 输出阶段 + planner.add_task("visualize", "结果可视化", + TaskType.VISUALIZATION, + "生成源地、阻力面、廊道的可视化地图", + depends_on=["extract_corridors"]) + + planner.add_task("export_results", "导出分析结果", + TaskType.EXPORT, + "导出矢量数据和统计报告", + depends_on=["extract_corridors"]) + + return planner + + @staticmethod + def site_selection_analysis() -> SpatialAnalysisPlanner: + """ + 选址分析工作流 + + 1. 加载约束数据 + 2. 加载候选地块 + 3. 叠加分析 + 4. 适宜性评价 + 5. 最优选址 + 6. 结果导出 + """ + planner = SpatialAnalysisPlanner("设施选址分析") + + planner.add_task("load_constraints", "加载约束数据", + TaskType.DATA_PREPARATION, + "加载坡度、保护区、道路等约束数据") + + planner.add_task("load_candidates", "加载候选地块", + TaskType.DATA_PREPARATION, + "加载候选地块矢量数据") + + planner.add_task("overlay_analysis", "叠加分析", + TaskType.ANALYSIS, + "将约束数据叠加到候选地块", + depends_on=["load_constraints", "load_candidates"]) + + planner.add_task("suitability", "适宜性评价", + TaskType.ANALYSIS, + "基于多准则评价计算适宜性得分", + depends_on=["overlay_analysis"]) + + planner.add_task("select_optimal", "最优选址", + TaskType.ANALYSIS, + "选择得分最高的地块", + depends_on=["suitability"]) + + planner.add_task("export", "导出选址方案", + TaskType.EXPORT, + "导出选址结果和评价报告", + depends_on=["select_optimal"]) + + return planner + + +# ==================== 演示程序 ==================== + +def demonstrate_spatial_planning(): + """演示空间分析规划系统""" + print("=" * 70) + print("空间分析任务规划器演示") + print("=" * 70) + + # 1. 创建生态网络分析工作流 + print("\n1. 创建生态网络分析工作流...") + planner = CommonAnalysisWorkflows.ecological_network_analysis() + + # 2. 可视化执行计划 + print("\n2. 执行计划DAG:") + print(planner.visualize_plan()) + + # 3. 获取执行计划 + print("\n3. 执行层级:") + execution_plan = planner.get_execution_plan() + for i, level in enumerate(execution_plan, 1): + task_names = [planner.tasks[t].name for t in level] + print(f" 层级 {i}: {', '.join(task_names)}") + + # 4. 模拟执行 + print("\n4. 模拟执行:") + + def mock_executor(task: Task) -> Dict: + """模拟任务执行""" + import time + import random + + time.sleep(0.1) # 模拟执行时间 + + # 模拟偶尔失败 + if random.random() < 0.1: + raise Exception("模拟随机失败") + + return { + 'task': task.name, + 'status': 'success', + 'output': f"{task.name}_output" + } + + results = planner.execute(mock_executor, max_retries=2) + + # 5. 统计结果 + print("\n5. 执行统计:") + status_count = {} + for task in planner.tasks.values(): + status = task.status.value + status_count[status] = status_count.get(status, 0) + 1 + + for status, count in status_count.items(): + print(f" {status}: {count}") + + # 6. 创建自定义工作流 + print("\n6. 创建自定义工作流:") + + custom_planner = SpatialAnalysisPlanner("自定义分析") + custom_planner.add_task("t1", "数据加载", TaskType.DATA_PREPARATION) + custom_planner.add_task("t2", "数据清洗", TaskType.ANALYSIS, depends_on=["t1"]) + custom_planner.add_task("t3", "统计分析", TaskType.ANALYSIS, depends_on=["t2"]) + custom_planner.add_task("t4", "并行分析A", TaskType.ANALYSIS, depends_on=["t2"]) + custom_planner.add_task("t5", "并行分析B", TaskType.ANALYSIS, depends_on=["t2"]) + custom_planner.add_task("t6", "结果汇总", TaskType.ANALYSIS, + depends_on=["t3", "t4", "t5"]) + + print(custom_planner.visualize_plan()) + + +if __name__ == "__main__": + demonstrate_spatial_planning() +``` + +### RL风格的规划执行 + +```python +class ReinforcementLearningPlanner: + """ + 基于强化学习思想的规划执行 + + 特点： + 1. 从执行中学习 + 2. 评估行动效果 + 3. 更新策略 + """ + + def __init__(self, actions: List[Action], + reward_fn: Callable[[Any, Any, Any], float]): + """ + Args: + actions: 可用行动 + reward_fn: 奖励函数 (state, action, next_state) -> reward + """ + self.actions = actions + self.reward_fn = reward_fn + + # Q值表：state -> {action: q_value} + self.q_table: Dict[str, Dict[str, float]] = {} + + # 学习参数 + self.learning_rate = 0.1 + self.discount_factor = 0.9 + self.epsilon = 0.1 # 探索率 + + def select_action(self, state: Any, training: bool = True) -> Action: + """ + 选择行动（epsilon-greedy策略） + + Args: + state: 当前状态 + training: 是否在训练模式 + """ + state_key = self._state_key(state) + + # 初始化Q值 + if state_key not in self.q_table: + self.q_table[state_key] = {a.name: 0.0 for a in self.actions} + + # epsilon-greedy + import random + if training and random.random() < self.epsilon: + # 探索：随机选择 + return random.choice(self.actions) + else: + # 利用：选择Q值最大的 + q_values = self.q_table[state_key] + best_action_name = max(q_values, key=q_values.get) + return next(a for a in self.actions if a.name == best_action_name) + + def update_q_value(self, state: Any, action: Action, + reward: float, next_state: Any) -> None: + """ + 更新Q值（Q-learning更新规则） + + Q(s,a) ← Q(s,a) + α[r + γ max Q(s',a') - Q(s,a)] + """ + state_key = self._state_key(state) + next_state_key = self._state_key(next_state) + + # 确保初始化 + if next_state_key not in self.q_table: + self.q_table[next_state_key] = { + a.name: 0.0 for a in self.actions + } + + # 计算新的Q值 + current_q = self.q_table[state_key][action.name] + max_next_q = max(self.q_table[next_state_key].values()) + + new_q = current_q + self.learning_rate * ( + reward + self.discount_factor * max_next_q - current_q + ) + + self.q_table[state_key][action.name] = new_q + + def learn_episode(self, environment, + max_steps: int = 100) -> float: + """ + 执行一个学习episode + + Returns: + 总奖励 + """ + state = environment.reset() + total_reward = 0 + + for _ in range(max_steps): + # 选择行动 + action = self.select_action(state, training=True) + + # 执行行动 + next_state, done = environment.step(action) + + # 计算奖励 + reward = self.reward_fn(state, action, next_state) + total_reward += reward + + # 更新Q值 + self.update_q_value(state, action, reward, next_state) + + state = next_state + + if done: + break + + return total_reward + + def _state_key(self, state: Any) -> str: + """生成状态键（简化）""" + return str(hash(str(state))) +``` + +--- + +## 案例分析 + +### LangGraph的规划执行 + +```python +""" +LangGraph风格的规划执行 + +LangGraph使用状态图来表示和执行复杂的Agent工作流 +""" + +from typing import TypedDict, Annotated, Sequence +import operator + + +class AgentState(TypedDict): + """Agent状态定义""" + messages: Annotated[Sequence[str], operator.add] + current_plan: list[str] + executed_steps: list[str] + requires_replan: bool + + +def should_continue(state: AgentState) -> str: + """ + 条件边：决定继续执行还是结束 + + 类似LangGraph的条件边 + """ + if not state["current_plan"]: + return "end" + if state["requires_replan"]: + return "replan" + return "continue" + + +class LangGraphStylePlanner: + """LangGraph风格的规划器""" + + def __init__(self): + self.state = AgentState( + messages=[], + current_plan=[], + executed_steps=[], + requires_replan=False + ) + self.nodes = { + "plan": self._plan_node, + "execute": self._execute_node, + "observe": self._observe_node, + "replan": self._replan_node + } + + def build_graph(self) -> Dict: + """构建执行图""" + return { + "nodes": self.nodes, + "edges": { + "plan": "execute", + "execute": "observe", + "observe": should_continue, + "continue": "execute", + "replan": "execute" + } + } + + def _plan_node(self, state: AgentState) -> AgentState: + """规划节点：生成初始计划""" + state["current_plan"] = ["step1", "step2", "step3"] + state["messages"].append("计划已生成") + return state + + def _execute_node(self, state: AgentState) -> AgentState: + """执行节点：执行下一步""" + if state["current_plan"]: + step = state["current_plan"].pop(0) + state["executed_steps"].append(step) + state["messages"].append(f"已执行: {step}") + return state + + def _observe_node(self, state: AgentState) -> AgentState: + """观察节点：检查是否需要重新规划""" + # 检查执行结果 + state["requires_replan"] = False # 简化 + return state + + def _replan_node(self, state: AgentState) -> AgentState: + """重规划节点：调整计划""" + state["current_plan"] = ["new_step"] + state["current_plan"] + state["requires_replan"] = False + state["messages"].append("计划已更新") + return state + + def run(self, initial_goal: str) -> AgentState: + """运行整个图""" + self.state["messages"].append(f"目标: {initial_goal}") + + graph = self.build_graph() + current_node = "plan" + + while current_node != "end": + # 执行节点 + self.state = self.nodes[current_node](self.state) + + # 获取下一个节点 + next_node = graph["edges"].get(current_node) + if callable(next_node): + next_node = next_node(self.state) + current_node = next_node + + return self.state +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **规划深度**：多深的规划是合适的？过度规划会有什么问题？ + +2. **不确定性**：如何在不确定环境中进行规划？ + +3. **多Agent协调**：多个Agent如何协调各自的规划？ + +4. **规划评估**：如何评估一个规划的质量？ + +### 延伸阅读 + +- **"Planning Algorithms"** (LaValle) - 规划算法权威教材 +- **"Hierarchical Planning"** - 分层规划专题 +- **"Reinforcement Learning"** (Sutton & Barto) - RL中的规划与学习 + +--- + +## 关键要点 + +1. **规划**是寻找从初始状态到目标的行动序列 +2. **前向搜索**从初始状态向目标搜索，适合目标明确的场景 +3. **后向搜索**从目标向初始状态搜索，适合目标较少的场景 +4. **分层规划**将复杂任务分解，适合复杂多阶段任务 +5. **重规划**是应对环境变化的关键机制 +6. **执行-监控-调整**循环是实际系统的核心模式 +7. **任务依赖管理**使用DAG表达和并行化执行 diff --git a/officefile/supplements/03-autonomous-design/README.md b/officefile/supplements/03-autonomous-design/README.md new file mode 100644 index 0000000..272bbdf --- /dev/null +++ b/officefile/supplements/03-autonomous-design/README.md @@ -0,0 +1,95 @@ +# 第四部分：自主设计 + +## 本部分目标 + +理解自主系统的设计模式和工作流编排： +- 工作流编排的原理与模式 +- Agent的设计范式 +- 技能组合与复用的方法 +- 记忆与上下文管理 +- 规划与执行机制 + +--- + +## 章节导航 + +| 章节 | 文件 | 核心内容 | 实践 | +|-----|------|---------|------| +| 03.1 | [工作流编排原理](./03.1-workflow-orchestration.md) | DAG、节点与边、条件分支、错误处理 | 生态网络六阶段工作流 | +| 03.2 | [Agent设计模式](./03.2-agent-design-patterns.md) | Reflex、Model-based、Goal-based、Utility-based | 对比四种Agent类型 | +| 03.3 | [技能组合与复用](./03.3-skill-composition.md) | 技能抽象、接口设计、组合模式 | QGIS技能集成管理器 | +| 03.4 | [记忆与上下文](./03.4-memory-and-context.md) | 短期/长期记忆、状态持久化、检索 | 专家知识积累系统 | +| 03.5 | [规划与执行](./03.5-planning-and-execution.md) | 前向/后向搜索、分层规划、重规划 | RL训练工作流 | + +--- + +## 核心概念图 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 自主设计体系 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────┐ │ +│ │ 目标设定 │ │ +│ └──────┬──────┘ │ +│ │ │ +│ ↓ │ +│ ┌────────────────────────┐ │ +│ │ 工作流编排 │ │ +│ │ (DAG + 状态机) │ │ +│ └────────────┬───────────┘ │ +│ │ │ +│ ┌────────────────┼────────────────┐ │ +│ ↓ ↓ ↓ │ +│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ +│ │ Agent设计 │ │ 技能组合 │ │ 记忆管理 │ │ +│ │ - 反应式 │ │ - 抽象接口 │ │ - 短期记忆 │ │ +│ │ - 基于模型 │ │ - 组合模式 │ │ - 长期记忆 │ │ +│ │ - 基于目标 │ │ - 技能发现 │ │ - 检索机制 │ │ +│ │ - 基于效用 │ │ - 动态加载 │ │ - 知识库 │ │ +│ └───────┬───────┘ └───────┬───────┘ └───────┬───────┘ │ +│ │ │ │ │ +│ └─────────────────┼─────────────────┘ │ +│ │ │ +│ ↓ │ +│ ┌──────────────────┐ │ +│ │ 规划与执行 │ │ +│ │ - 搜索算法 │ │ +│ │ - 任务分解 │ │ +│ │ - 执行监控 │ │ +│ │ - 动态重规划 │ │ +│ └──────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 实践案例 + +### 实践案例04：设计一个自主空间分析Agent + +详见 [practice/autonomous-agent](./practice/autonomous-agent/) + +### 实践案例05：实现Human-in-the-Loop审查机制 + +详见 [practice/hitl-implementation](./practice/hitl-implementation/) + +--- + +## 关键要点预览 + +1. **工作流编排**基于DAG和状态机，是复杂系统的核心 +2. **Agent设计**有四种经典模式，各有适用场景 +3. **技能组合**通过抽象接口实现复用和动态加载 +4. **记忆管理**区分短期和长期，支持检索和知识积累 +5. **规划与执行**需要平衡前瞻性和实时响应 + +--- + +## 延伸资源 + +- **"Artificial Intelligence: A Modern Approach"** (Russell & Norvig) - AI设计理论 +- **"Multi-Agent Systems"** (Wooldridge) - 多Agent系统 +- **LangGraph文档** - 实际工作流编排框架 diff --git a/officefile/supplements/04-practice/README.md b/officefile/supplements/04-practice/README.md new file mode 100644 index 0000000..ad4b359 --- /dev/null +++ b/officefile/supplements/04-practice/README.md @@ -0,0 +1,249 @@ +# 第五部分：综合实践 + +## 本部分目标 + +通过完整项目将所学知识整合应用： +- 将空间智能与自主设计结合 +- 实现端到端的智能分析系统 +- 处理真实场景的复杂性 +- 建立可复用的解决方案 + +--- + +## 项目概览 + +| 项目 | 描述 | 难度 | 涉及章节 | 预计时间 | +|-----|------|------|---------|---------| +| 项目A | 生态源地自动识别与优先级排序 | ⭐⭐ | 02.1, 02.3, 03.1 | 8-12小时 | +| 项目B | 生态阻力面的多准则构建 | ⭐⭐⭐ | 02.3, 02.5, 03.3 | 12-16小时 | +| 项目C | 生态网络的自主分析与优化 | ⭐⭐⭐⭐ | 02.2, 02.4, 03.4 | 16-20小时 | +| 项目D | 带HITL的完整工作流设计 | ⭐⭐⭐⭐⭐ | 全部章节 | 24-30小时 | + +--- + +## 项目结构 + +每个项目包含： +``` +projects/ +├── project-a-source-identification/ +│ ├── README.md # 项目说明 +│ ├── requirements.txt # 依赖 +│ ├── data/ # 示例数据 +│ ├── notebooks/ # Jupyter notebooks +│ ├── src/ # 源代码 +│ └── tests/ # 测试 +├── project-b-resistance-surface/ +├── project-c-network-optimization/ +└── project-d-hitl-workflow/ +``` + +--- + +## 项目A：生态源地自动识别与优先级排序 + +### 问题背景 + +生态源地是生态网络的核心节点，需要从土地利用数据中自动识别并排序。 + +### 涉及技术 + +- 空间表征 (02.1)：栅格数据处理、斑块提取 +- 多准则决策 (02.3)：适宜性评价、权重设计 +- 工作流编排 (03.1)：数据→处理→排序的流水线 + +### 实现步骤 + +1. **数据准备** + - 加载土地利用覆盖数据 + - 数据预处理（投影、重分类） + +2. **斑块识别** + - 使用形态学空间格局分析（MSPA） + - 提取连通性指标 + +3. **适宜性评价** + - 计算每个斑块的适宜性得分 + - 多指标综合（面积、形状、连通性） + +4. **优先级排序** + - 按适宜性排序 + - 输出源地列表 + +### 扩展方向 + +- 添加不确定性评估 +- 支持多物种的源地识别 +- 可视化源地优先级 + +--- + +## 项目B：生态阻力面的多准则构建 + +### 问题背景 + +阻力面反映物种在景观中移动的难易程度，需要综合考虑多种因素。 + +### 涉及技术 + +- 多准则决策 (02.3)：AHP权重分配 +- 不确定性量化 (02.5)：敏感性分析 +- 技能组合 (03.3)：多个阻力源的组合 + +### 实现步骤 + +1. **阻力源识别** + - 土地利用类型阻力 + - 地形阻力（坡度、高程） + - 人为干扰阻力（道路、建筑） + +2. **权重确定** + - 专家知识编码 + - AHP层次分析法 + - 权重一致性检验 + +3. **阻力面合成** + - 加权叠加 + - 阻力系数调整 + +4. **敏感性分析** + - 权重扰动测试 + - 结果稳定性评估 + +### 扩展方向 + +- 支持不同物种的定制阻力面 +- 时间维度的动态阻力面 +- 基于遥感的自动更新 + +--- + +## 项目C：生态网络的自主分析与优化 + +### 问题背景 + +在源地和阻力面基础上，自动提取生态廊道并优化网络结构。 + +### 涉及技术 + +- 空间推理 (02.2)：图算法、连通性分析 +- 空间优化 (02.4)：网络优化算法 +- Agent设计 (03.2)：自主分析智能体 +- 记忆管理 (03.4)：结果缓存与检索 + +### 实现步骤 + +1. **最小累积阻力（MCR）分析** + - 成本距离计算 + - 最小成本路径 + +2. **廊道提取** + - 提取潜在廊道 + - 宽度分析 + - 质量评估 + +3. **网络分析** + - 连通性指标 + - 关键节点识别 + - 网络脆弱性评估 + +4. **优化建议** + - 廊道优先级排序 + - 新增源地建议 + - 网络强化方案 + +### 扩展方向 + +- 多目标优化（生态+经济） +- 动态网络模拟 +- 与GIS软件集成 + +--- + +## 项目D：带HITL的完整工作流设计 + +### 问题背景 + +整合A、B、C项目，实现完整的生态网络分析工作流，在关键点加入人类审查。 + +### 涉及技术 + +- 全部章节的综合应用 + +### 实现步骤 + +1. **工作流设计** + - 状态定义 + - 节点设计 + - 条件路由 + +2. **审查点设计** + - 源地识别审查 + - 阻力权重审查 + - 廊道优化审查 + +3. **人机界面** + - 信息展示 + - 决策输入 + - 反馈机制 + +4. **系统集成** + - 持久化与恢复 + - 错误处理 + - 日志记录 + +### 交付成果 + +- 完整可运行的系统 +- 用户手册 +- 技术文档 +- 演示视频 + +--- + +## 学习建议 + +### 循序渐进 + +1. **先做项目A**：建立基础概念 +2. **再做项目B**：学习多准则方法 +3. **然后做项目C**：掌握空间优化 +4. **最后做项目D**：整合所有知识 + +### 遇到困难时 + +- 回顾相关章节的理论 +- 查阅示例代码 +- 参考延伸阅读 +- 寻求社区帮助 + +### 提升挑战 + +- 为项目添加新功能 +- 使用自己的数据 +- 优化性能 +- 改进用户体验 + +--- + +## 评估标准 + +完成项目后，自问： + +1. **功能完整性**：是否实现了所有核心功能？ +2. **代码质量**：代码是否清晰、可维护？ +3. **文档完整性**：是否有清晰的使用说明？ +4. **鲁棒性**：能否处理异常情况？ +5. **可扩展性**：是否容易添加新功能？ + +--- + +## 下一步 + +完成所有项目后，你已经： +- ✅ 掌握了空间智能的核心方法 +- ✅ 理解了自主系统的设计模式 +- ✅ 实现了完整的AI辅助分析系统 +- ✅ 建立了可复用的技能库 + +准备好进入：**05-reflection（反思与展望）** diff --git a/officefile/supplements/05-reflection/05.1-ai-limitations.md b/officefile/supplements/05-reflection/05.1-ai-limitations.md new file mode 100644 index 0000000..e434b7c --- /dev/null +++ b/officefile/supplements/05-reflection/05.1-ai-limitations.md @@ -0,0 +1,484 @@ +# 05.1 AI的局限与幻觉 + +## 核心问题 + +> AI在空间分析中哪些地方可能出错？ +> 当AI给出一个看似合理的答案时，我们如何验证它？ +> 人类专家的哪些能力是AI难以替代的？ + +--- + +## 概念讲解 + +### AI幻觉的本质 + +**幻觉 (Hallucination)** 是指AI生成看似合理但实际错误的内容。在空间AI中，这个问题尤为隐蔽和危险： + +``` +空间AI幻觉的类型 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 几何幻觉 │ +│ - 生成无效的几何图形 │ +│ - 错误的空间关系（如"A在B内部"实际为假） │ +│ - 投影和坐标系混淆 │ +│ │ +│ 2. 语义幻觉 │ +│ - 对空间概念的错误理解 │ +│ - 编造不存在的GIS功能 │ +│ - 混淆专业术语 │ +│ │ +│ 3. 逻辑幻觉 │ +│ - 分析步骤的遗漏或重复 │ +│ - 错误的因果推断 │ +│ - 隐藏的假设未被说明 │ +│ │ +│ 4. 数据幻觉 │ +│ - 假设数据存在实际不存在 │ +│ - 错误的数据格式假设 │ +│ - 忽略数据质量和边界条件 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 为什么会产生幻觉？ + +| 原因类型 | 说明 | 空间AI中的例子 | +|---------|------|---------------| +| **训练数据偏差** | 模型见过的数据不具代表性 | 模型更多见过城市数据，对农村场景判断不准 | +| **模式匹配局限** | 模型基于模式而非理解 | 混淆buffer和convex hull因为结果看起来相似 | +| **上下文理解不足** | 无法完全理解复杂场景 | 忽略项目的特定约束条件 | +| **概率生成本质** | 逐token生成可能偏离 | 生成的代码引用不存在的函数 | +| **知识边界模糊** | 模型不知道自己不知道 | 对未见过的GIS版本编造功能 | + +### 空间AI特有的脆弱性 + +空间分析有其特殊性质，使得AI的错误更具破坏性： + +``` +空间AI脆弱性来源 + +1. 隐式依赖 + GIS操作常有隐式前提： + - "intersect前必须确保同一坐标系" + - "buffer距离需要合适的投影" + 这些前提AI可能忽略 + +2. 级联效应 + 空间分析通常是多步骤： + 数据加载 → 清理 → 投影 → 分析 → 输出 + 早期错误会被放大 + +3. 验证困难 + 空间结果不像代码能快速测试： + - 这个生态源地识别对吗？ + - 这个阻力面合理吗？ + 需要领域知识判断 + +4. 不可见错误 + 某些空间错误不会立即显现： + - 轻微的几何错误 + - 边界处的投影变形 + - 拓扑关系的细微错误 +``` + +--- + +## 设计原理 + +### 验证框架设计 + +建立系统的验证流程是应对AI局限的关键： + +```python +""" +空间AI结果验证框架 +""" +from typing import Any, Dict, List, Optional +from dataclasses import dataclass +from enum import Enum + +class ValidationLevel(Enum): + """验证级别""" + CRITICAL = "critical" # 必须通过 + IMPORTANT = "important" # 应该通过 + WARNING = "warning" # 警告即可 + +@dataclass +class ValidationResult: + """验证结果""" + passed: bool + level: ValidationLevel + message: str + details: Optional[Dict] = None + +class SpatialAIValidator: + """空间AI输出验证器""" + + def __init__(self): + self.checks = [] + + def add_check(self, check_fn, level: ValidationLevel): + """添加验证检查""" + self.checks.append((check_fn, level)) + return self + + def validate(self, result: Any, context: Dict) -> List[ValidationResult]: + """执行所有验证""" + results = [] + + for check_fn, level in self.checks: + try: + result = check_fn(result, context) + results.append(result) + except Exception as e: + results.append(ValidationResult( + passed=False, + level=ValidationLevel.CRITICAL, + message=f"验证失败: {str(e)}" + )) + + return results + +# === 常用验证检查 === + +def check_crs_consistency(result, context) -> ValidationResult: + """检查坐标系一致性""" + if hasattr(result, 'crs') and result.crs is not None: + expected_crs = context.get('expected_crs') + if expected_crs and result.crs != expected_crs: + return ValidationResult( + passed=False, + level=ValidationLevel.CRITICAL, + message=f"坐标系不匹配: 期望 {expected_crs}, 实际 {result.crs}", + details={'expected': expected_crs, 'actual': result.crs} + ) + return ValidationResult( + passed=True, + level=ValidationLevel.CRITICAL, + message="坐标系检查通过" + ) + +def check_geometry_validity(result, context) -> ValidationResult: + """检查几何有效性""" + if hasattr(result, 'geometry'): + if hasattr(result.geometry, 'is_valid'): + if not result.geometry.is_valid.all(): + invalid_count = (~result.geometry.is_valid).sum() + return ValidationResult( + passed=False, + level=ValidationLevel.IMPORTANT, + message=f"存在 {invalid_count} 个无效几何", + details={'invalid_count': invalid_count} + ) + return ValidationResult( + passed=True, + level=ValidationLevel.IMPORTANT, + message="几何有效性检查通过" + ) + +def check_result_size(result, context) -> ValidationResult: + """检查结果规模是否合理""" + if hasattr(result, '__len__'): + size = len(result) + max_expected = context.get('max_expected_size', float('inf')) + min_expected = context.get('min_expected_size', 0) + + if size > max_expected: + return ValidationResult( + passed=False, + level=ValidationLevel.WARNING, + message=f"结果数量异常大: {size}", + details={'size': size} + ) + if size < min_expected: + return ValidationResult( + passed=False, + level=ValidationLevel.WARNING, + message=f"结果数量异常小: {size}", + details={'size': size} + ) + return ValidationResult( + passed=True, + level=ValidationLevel.WARNING, + message="结果规模检查通过" + ) + +def check_spatial_extent(result, context) -> ValidationResult: + """检查空间范围是否合理""" + if hasattr(result, 'total_bounds'): + bounds = result.total_bounds + expected_bounds = context.get('expected_bounds') + + if expected_bounds: + # 检查结果是否在预期范围内 + if not (bounds[0] >= expected_bounds[0] and + bounds[2] <= expected_bounds[2] and + bounds[1] >= expected_bounds[1] and + bounds[3] <= expected_bounds[3]): + return ValidationResult( + passed=False, + level=ValidationLevel.IMPORTANT, + message=f"空间范围超出预期", + details={'bounds': bounds, 'expected': expected_bounds} + ) + return ValidationResult( + passed=True, + level=ValidationLevel.IMPORTANT, + message="空间范围检查通过" + ) + +# === 使用示例 === + +def create_validator_example(): + """创建完整的验证器示例""" + validator = SpatialAIValidator() + + # 添加验证检查 + validator.add_check(check_crs_consistency, ValidationLevel.CRITICAL) + validator.add_check(check_geometry_validity, ValidationLevel.IMPORTANT) + validator.add_check(check_result_size, ValidationLevel.WARNING) + validator.add_check(check_spatial_extent, ValidationLevel.IMPORTANT) + + return validator + +if __name__ == "__main__": + print("=== 空间AI验证框架 ===\n") + + validator = create_validator_example() + print(f"验证器配置了 {len(validator.checks)} 个检查") + + print("\n验证级别说明:") + print(" CRITICAL: 必须通过的错误") + print(" IMPORTANT: 应该通过的问题") + print(" WARNING: 值得注意的警告") +``` + +### 人类专家的不可替代性 + +``` +人类专家的优势领域 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 上下文理解 │ +│ - 理解项目背景和约束条件 │ +│ - 识别"不合理"的结果 │ +│ - 考虑实际可行性 │ +│ │ +│ 2. 价值判断 │ +│ - 权衡不同目标 │ +│ - 考虑伦理影响 │ +│ - 平衡科学性和实用性 │ +│ │ +│ 3. 创造性思维 │ +│ - 提出新的分析方法 │ +│ - 创造性地解决问题 │ +│ - 跨领域联想 │ +│ │ +│ 4. 责任承担 │ +│ - 对结果负责 │ +│ - 解释和辩护决策 │ +│ - 承担法律和伦理责任 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### AI辅助的正确模式 + +``` +正确的AI使用模式 + +专家 + AI = 增强 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ AI的角色 专家的角色 │ +│ │ │ │ +│ ├── 快速计算和数据处理 ──────────────┼── 定义问题 │ +│ ├── 提供多种方案 ───────────────────┼── 选择和评估 │ +│ ├── 识别模式 ──────────────────────┼── 解释意义 │ +│ ├── 自动化重复任务 ─────────────────┼── 设计工作流 │ +│ ├── 检查错误 ──────────────────────┼── 验证关键结果 │ +│ └── 提供参考 ──────────────────────┼── 做出决策 │ +│ │ +└─────────────────────────────────────────────────────────────┘ + +错误模式: + AI ──→ 结果 ──→ 直接使用 + (跳过专家验证) + +正确模式: + 专家 ──→ 定义问题 ──→ AI ──→ 候选方案 + │ + 专家 ◄─────────────────────┘ + │ 评估 + │ 验证 + │ 决策 + ↓ + 最终方案 +``` + +--- + +## 案例分析 + +### 案例1：坐标系统错误 + +**场景**：AI生成了生态源地识别代码，但忘记处理坐标系问题。 + +```python +# AI生成的代码（有潜在问题） +def identify_sources(ai_generated_landcover): + """AI生成的源地识别代码""" + # 直接使用原始数据分析 + forest = ai_generated_landcover[ai_generated_landcover['type'] == 'forest'] + sources = forest[forest['area'] > 100] # 面积阈值 + + # 问题：数据可能是WGS84，但面积按度计算 + return sources + +# 验证和修正 +def identify_sources_validated(landcover, target_crs='EPSG:3857'): + """添加验证的版本""" + # 验证1：检查坐标系 + if landcover.crs and landcover.crs.is_geographic: + # 需要投影 + landcover_projected = landcover.to_crs(target_crs) + print(f"注意：数据已从 {landcover.crs} 投影到 {target_crs}") + else: + landcover_projected = landcover + + # 验证2：面积计算检查 + if landcover_projected.crs.is_geographic: + raise ValueError("不能在地理坐标系中计算面积") + + # 进行分析 + forest = landcover_projected[landcover_projected['type'] == 'forest'] + sources = forest[forest.geometry.area > 100000] # 投影后单位是米 + + # 验证3：结果合理性检查 + if len(sources) == 0: + print("警告：没有识别到源地，检查面积阈值") + if len(sources) > len(landcover_projected) * 0.9: + print("警告：几乎全部区域都是源地，检查阈值") + + return sources +``` + +### 案例2：拓扑关系幻觉 + +**场景**：AI错误判断空间关系。 + +```python +# AI可能生成的错误逻辑 +def check_connectivity(patch_a, patch_b): + """检查两个斑块的连通性""" + # 问题：AI可能混淆多种连通性判断 + return patch_a.intersects(patch_b) # 相交不等于连通 + +# 正确的方法 +def check_connectivity_robust(patch_a, patch_b, distance_threshold=100): + """ + 鲁棒的连通性检查 + + 需要明确： + 1. 是直接连接还是距离阈值内？ + 2. 需要考虑阻力吗？ + 3. 连通的宽度要求？ + """ + # 方案1：直接接触 + if patch_a.touches(patch_b): + return {'connected': True, 'type': 'direct'} + + # 方案2：距离阈值 + distance = patch_a.distance(patch_b) + if distance <= distance_threshold: + return {'connected': True, 'type': 'proximity', 'distance': distance} + + return {'connected': False, 'distance': distance} +``` + +### 案例3：参数选择幻觉 + +**场景**：AI编造了一个"标准"参数值，实际并不存在。 + +```python +# AI可能这样写 +def calculate_landscape_metrics(patch, resistance="standard"): + """ + 计算景观指标 + + 问题：AI声称存在"standard"阻力值，实际需要根据情况设定 + """ + # 不存在通用标准 + pass + +# 正确的做法 +def calculate_landscape_metrics_validated(patch, resistance=None, + resistance_params=None): + """ + 计算景观指标，明确参数来源 + + Args: + resistance: 阻力值，必须明确提供 + resistance_params: 阻力参数配置 + + Returns: + 指标和参数来源说明 + """ + if resistance is None and resistance_params is None: + raise ValueError( + "阻力参数必须明确提供。不存在'标准'值。" + "请根据研究区域和物种特征设定。" + ) + + # 记录参数来源 + metadata = { + 'resistance_source': resistance_params.get('source', 'user_provided'), + 'reference': resistance_params.get('reference', None), + 'justification': resistance_params.get('justification', None) + } + + # 计算指标... + return {'metrics': {}, 'metadata': metadata} +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **识别能力边界**：你最近一次发现AI错误是什么时候？是如何发现的？ + +2. **验证成本**：在什么情况下，详细验证的成本超过了使用AI的收益？ + +3. **责任分配**：当AI辅助的分析出现错误时，责任该如何划分？ + +4. **信任建立**：随着时间推移，你应该如何调整对AI的信任程度？ + +### 实践练习 + +1. **错误审计**：回顾过去使用AI生成的空间分析代码，找出潜在问题 + +2. **验证清单**：为你常用的空间分析类型建立验证清单 + +3. **对比实验**：同一个问题让AI多次求解，比较结果的差异 + +### 延伸阅读 + +- **"Human Compatible"** (Stuart Russell) - AI对齐与人类价值 +- **"AI Safety"** 相关文献 - 理解AI的局限和风险 +- GIS最佳实践手册 - 学习领域专家的验证方法 + +--- + +## 关键要点 + +1. **AI在空间分析中可能产生多种类型的幻觉**，需要系统性验证 +2. **建立验证框架**是可靠使用AI的关键 +3. **人类专家的角色不可替代**，特别是在判断和决策环节 +4. **正确的使用模式是AI辅助+专家验证**，而非AI替代 +5. **保持批判性思维**，理解AI的局限才能更好地利用它 diff --git a/officefile/supplements/05-reflection/05.2-ethics-and-responsibility.md b/officefile/supplements/05-reflection/05.2-ethics-and-responsibility.md new file mode 100644 index 0000000..2e011ef --- /dev/null +++ b/officefile/supplements/05-reflection/05.2-ethics-and-responsibility.md @@ -0,0 +1,820 @@ +# 05.2 伦理与责任 + +## 核心问题 + +> 空间决策如何影响不同的人群和环境？ +> 当AI参与空间规划时，如何确保过程的公平和透明？ +> 出错的决策责任应该由谁来承担？ + +--- + +## 概念讲解 + +### 空间决策的伦理维度 + +空间决策不是价值中立的，它们分配资源、机会和风险： + +``` +空间决策的伦理影响 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 分配正义 (Distributive Justice) │ +│ - 谁获得绿地、公园等正面空间资源？ │ +│ - 谁承受污染、噪声等负面影响？ │ +│ - 空间资源的公平分配原则是什么？ │ +│ │ +│ 2. 程序正义 (Procedural Justice) │ +│ - 决策过程是否透明？ │ +│ - 受影响者能否参与决策？ │ +│ - 决策依据是否可审查？ │ +│ │ +│ 3. 承认正义 (Recognition Justice) │ +│ - 不同群体的需求和价值观是否被认可？ │ +│ - 弱势群体的空间权利是否被尊重？ │ +│ - 文化多样性在空间中如何体现？ │ +│ │ +│ 4. 生态正义 (Ecological Justice) │ +│ - 当代人与未来世代之间的公平？ │ +│ - 人类活动对生态系统的责任？ │ +│ - 非人类物种的空间权利？ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### AI带来的新伦理挑战 + +``` +AI空间决策的特有问题 + +1. 算法偏见 (Algorithmic Bias) + 训练数据中的社会偏见被编码进模型 + → 历史上的红线政策可能影响现在的预测 + +2. 黑箱决策 (Black Box Decision) + 复杂模型的决策过程难以解释 + → 利益相关者无法质疑或理解决策 + +3. 责任分散 (Diffused Responsibility) + 涉及多个主体：开发者、用户、数据提供者 + → 出错时责任难以界定 + +4. 规模效应 (Scale Effects) + AI可以大规模应用决策 + → 小偏差在大规模下产生大影响 + +5. 路径依赖 (Path Dependence) + 早期决策影响后续数据收集 + → 偏见自我强化 +``` + +### 空间正义的典型问题 + +| 问题类型 | 说明 | AI相关风险 | +|---------|------|-----------| +| **环境种族主义** | 有害设施更多位于少数族裔社区 | AI可能复制历史模式 | +| **绿色绅士化** | 绿地改善导致原住民被迫搬迁 | AI优化可能加剧此问题 | +| **数字鸿沟** | 缺乏数据地区被忽视 | AI只关注数据丰富的区域 | +| **代表性不足** | 某些群体的需求未被考虑 | 训练数据偏差 | + +--- + +## 设计原理 + +### 可解释性设计原则 + +```python +""" +可解释的空间AI设计 +""" +from typing import Dict, List, Any, Optional +from dataclasses import dataclass +from abc import ABC, abstractmethod + +@dataclass +class DecisionExplanation: + """决策解释""" + decision: str # 做出的决策 + rationale: List[str] # 决策理由 + key_factors: Dict[str, float] # 关键因素及权重 + alternatives: List[Dict] # 考虑过的替代方案 + uncertainties: List[str] # 不确定性说明 + assumptions: List[str] # 假设条件 + +class ExplainableSpatialAI(ABC): + """可解释的空间AI基类""" + + @abstractmethod + def make_decision(self, context: Dict) -> Any: + """做出决策""" + pass + + @abstractmethod + def explain_decision(self, decision: Any, context: Dict) -> DecisionExplanation: + """解释决策""" + pass + + def audit_trail(self) -> List[Dict]: + """返回审计轨迹""" + return self._audit_log + +class ExplainableSiteSelector(ExplainableSpatialAI): + """可解释的选址AI""" + + def __init__(self): + self.criteria_weights = {} + self._audit_log = [] + + def set_criteria(self, criteria: Dict[str, float], justification: str): + """ + 设置评判标准 + + Args: + criteria: {标准名: 权重} + justification: 权重选择的理由 + """ + self.criteria_weights = criteria.copy() + self._log({ + 'action': 'set_criteria', + 'criteria': criteria, + 'justification': justification + }) + + def make_decision(self, context: Dict) -> Dict: + """ + 做出选址决策 + + 返回选中的地点及其评分 + """ + sites = context['sites'] + constraints = context.get('constraints', {}) + + # 评估每个候选地 + scored_sites = [] + for site in sites: + score, details = self._evaluate_site(site, context) + scored_sites.append({ + 'site': site, + 'score': score, + 'details': details + }) + + # 排序并选择最高分 + scored_sites.sort(key=lambda x: x['score'], reverse=True) + selected = scored_sites[0] + + # 记录决策 + self._log({ + 'action': 'make_decision', + 'selected': selected['site'], + 'score': selected['score'], + 'alternatives': scored_sites[1:4] # 保存前几个备选 + }) + + return selected + + def explain_decision(self, decision: Any, context: Dict) -> DecisionExplanation: + """生成决策解释""" + selected_site = decision['site'] + score_details = decision['details'] + + # 生成解释 + return DecisionExplanation( + decision=f"选择地点 {selected_site['name']}", + rationale=[ + f"该地点综合评分最高 ({decision['score']:.2f})", + "评分基于预定义的标准和权重", + "所有候选地点已被系统评估" + ], + key_factors=score_details, + alternatives=[ + { + 'site': alt['site']['name'], + 'score': alt['score'], + 'reason': '评分较低' + } + for alt in context.get('alternatives', [])[:3] + ], + uncertainties=[ + "评分依赖输入数据的准确性", + "权重选择包含主观判断", + "未量化的因素可能影响实际适用性" + ], + assumptions=[ + "所有标准可以用数值表示", + "各标准相互独立", + "当前条件在未来保持稳定" + ] + ) + + def _evaluate_site(self, site: Dict, context: Dict) -> tuple: + """评估单个地点""" + scores = {} + + for criterion, weight in self.criteria_weights.items(): + # 从地点数据中获取该标准的值 + value = site.get(criterion, 0) + + # 标准化（简化版） + normalized = self._normalize(criterion, value) + + # 加权 + scores[criterion] = normalized * weight + + total_score = sum(scores.values()) + + return total_score, scores + + def _normalize(self, criterion: str, value: float) -> float: + """标准化准则值""" + # 简化：假设越大越好，范围0-100 + return min(max(value / 100, 0), 1) + + def _log(self, entry: Dict): + """记录日志""" + entry['timestamp'] = self._get_timestamp() + self._audit_log.append(entry) + + def _get_timestamp(self) -> str: + """获取时间戳""" + from datetime import datetime + return datetime.now().isoformat() + +# === 伦理检查 === + +class EthicsChecker: + """伦理检查器""" + + def __init__(self): + self.checks = [] + + def add_check(self, check_fn, name: str): + """添加检查""" + self.checks.append((check_fn, name)) + return self + + def check_decision(self, decision: Any, context: Dict) -> Dict: + """执行所有伦理检查""" + results = { + 'passed': True, + 'issues': [], + 'warnings': [] + } + + for check_fn, name in self.checks: + try: + result = check_fn(decision, context) + if not result['passed']: + results['passed'] = False + results['issues'].append({ + 'check': name, + 'reason': result['reason'] + }) + elif result.get('warning'): + results['warnings'].append({ + 'check': name, + 'warning': result['warning'] + }) + except Exception as e: + results['issues'].append({ + 'check': name, + 'reason': f"检查失败: {str(e)}" + }) + + return results + +# 预定义的伦理检查 + +def check_environmental_justice(decision, context) -> Dict: + """检查环境正义：确保不将负面影响集中到弱势社区""" + selected_site = decision['site'] + + # 检查是否有弱势群体数据 + vulnerable_communities = context.get('vulnerable_communities', []) + + for community in vulnerable_communities: + if selected_site.get('near_community') == community['id']: + # 如果项目有负面影响，需要特别审查 + if context.get('project_type') == 'negative_impact': + return { + 'passed': False, + 'reason': f"选址靠近弱势社区 {community['name']}，需要额外的环境正义审查" + } + + return {'passed': True} + +def check_transparency(decision, context) -> Dict: + """检查透明度：确保决策过程可记录和审查""" + if not decision.get('details'): + return { + 'passed': False, + 'reason': '决策缺乏详细评分信息，无法审查' + } + + return {'passed': True} + +def check_public_participation(decision, context) -> Dict: + """检查公众参与：确保受影响者有机会表达意见""" + if context.get('affects_public', False): + participation = context.get('public_participation') + if not participation or participation == 'none': + return { + 'passed': False, + 'reason': '项目影响公众但缺乏公众参与程序' + } + elif participation == 'minimal': + return { + 'passed': True, + 'warning': '公众参与程度较低，建议加强' + } + + return {'passed': True} + +# === 使用示例 === + +if __name__ == "__main__": + print("=== 可解释的空间AI ===\n") + + # 创建选址器 + selector = ExplainableSiteSelector() + + # 设置评判标准（带理由） + selector.set_criteria( + criteria={ + 'accessibility': 0.3, + 'environmental_quality': 0.25, + 'cost_effectiveness': 0.2, + 'community_support': 0.15, + 'future_potential': 0.1 + }, + justification="基于项目目标和利益相关者访谈" + ) + + # 模拟候选地点 + sites = [ + {'name': 'Site A', 'accessibility': 85, 'environmental_quality': 70, + 'cost_effectiveness': 60, 'community_support': 80, 'future_potential': 75}, + {'name': 'Site B', 'accessibility': 70, 'environmental_quality': 85, + 'cost_effectiveness': 75, 'community_support': 60, 'future_potential': 70}, + {'name': 'Site C', 'accessibility': 90, 'environmental_quality': 60, + 'cost_effectiveness': 80, 'community_support': 70, 'future_potential': 65}, + ] + + # 创建伦理检查器 + ethics_checker = EthicsChecker() + ethics_checker.add_check(check_environmental_justice, "环境正义检查") + ethics_checker.add_check(check_transparency, "透明度检查") + ethics_checker.add_check(check_public_participation, "公众参与检查") + + # 上下文 + context = { + 'sites': sites, + 'vulnerable_communities': [], + 'project_type': 'neutral', + 'affects_public': True, + 'public_participation': 'moderate' + } + + # 做出决策 + decision = selector.make_decision(context) + + print(f"选中地点: {decision['site']['name']}") + print(f"综合评分: {decision['score']:.2f}\n") + + # 获取解释 + explanation = selector.explain_decision(decision, context) + print("=== 决策解释 ===") + print(f"决策: {explanation.decision}") + print(f"\n理由:") + for r in explanation.rationale: + print(f" - {r}") + print(f"\n关键因素:") + for factor, value in explanation.key_factors.items(): + print(f" - {factor}: {value:.3f}") + + # 伦理检查 + print(f"\n=== 伦理检查 ===") + ethics_result = ethics_checker.check_decision(decision, context) + if ethics_result['passed']: + print("所有伦理检查通过") + else: + print("伦理检查发现问题:") + for issue in ethics_result['issues']: + print(f" - [{issue['check']}] {issue['reason']}") +``` + +### 问责机制设计 + +``` +AI空间决策的问责框架 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 责任链 │ +│ │ +│ 数据提供者 ──→ 模型开发者 ──→ 系统集成者 ──→ 最终用户 │ +│ │ │ │ │ │ +│ │ │ │ └── 决策责任 │ +│ │ │ └── 集成责任 │ +│ │ └── 模型责任 │ +│ └── 数据质量责任 │ +│ │ +│ 问责机制 │ +│ │ +│ 1. 文档化 (Documentation) │ +│ - 记录所有决策和数据来源 │ +│ - 保存模型版本和参数 │ +│ - 维护变更历史 │ +│ │ +│ 2. 审计 (Audit) │ +│ - 定期审查决策 │ +│ - 检查偏见和公平性 │ +│ - 验证技术正确性 │ +│ │ +│ 3. 申诉 (Appeal) │ +│ - 提供质疑决策的渠道 │ +│ - 建立复核机制 │ +│ - 允许人工干预 │ +│ │ +│ 4. 纠正 (Remedy) │ +│ - 发现错误后的补救措施 │ +│ - 对受影响方的补偿 │ +│ - 系统改进和预防 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 案例分析 + +### 案例1：城市绿地分布的算法偏见 + +**问题背景**：某城市使用AI优化绿地布局，但结果加剧了既有不平等。 + +```python +""" +问题代码示例：训练数据中的历史偏见 +""" + +# 问题：使用历史公园使用数据来优化新公园位置 +def optimize_park_locations_biased(historical_usage_data, new_sites): + """ + 有偏见的优化算法 + + 问题：历史使用数据反映的是历史可达性， + 而非真实需求。服务不足的区域数据少， + 因此被算法继续忽视。 + """ + # 简单优化：在历史使用高的地方附近选址 + scored = [] + for site in new_sites: + # 靠近高使用区域得分高 + score = sum( + usage for nearby, usage in historical_usage_data + if distance(site, nearby) < 1000 + ) + scored.append((site, score)) + + # 选择得分最高的 + scored.sort(key=lambda x: x[1], reverse=True) + return [s[0] for s in scored[:5]] + +# 改进版本：考虑需求而非历史使用 +def optimize_park_locations_fair(demand_indicators, new_sites, + equity_weight=0.5): + """ + 公平的优化算法 + + 考虑： + 1. 当前服务不足程度（需求） + 2. 人口密度 + 3. 弱势群体分布 + """ + scored = [] + for site in new_sites: + # 服务不足得分 + underserved_score = calculate_underserved(site, demand_indicators) + + # 效率得分（可达人口） + efficiency_score = calculate_accessible_population(site) + + # 综合得分，可调整公平权重 + score = (1 - equity_weight) * efficiency_score + \ + equity_weight * underserved_score + + scored.append((site, score, { + 'underserved': underserved_score, + 'efficiency': efficiency_score + })) + + # 按综合得分排序 + scored.sort(key=lambda x: x[1], reverse=True) + + # 记录决策依据 + for site, score, details in scored: + site['selection_score'] = score + site['score_details'] = details + + return [s[0] for s in scored[:5]] + +def calculate_underserved(site, indicators): + """计算服务不足程度""" + # 距离最近的现有设施 + distance_to_nearest = min_distance_to_parks(site) + + # 附近人口中的弱势群体比例 + vulnerable_ratio = get_vulnerable_population_ratio(site) + + # 服务不足 = 距离远 + 弱势群体多 + return distance_to_nearest * (1 + vulnerable_ratio) +``` + +### 案例2：生态保护区的社区影响 + +**问题背景**：AI优化的生态廊道选址忽略了当地社区权益。 + +```python +""" +考虑多方利益的生态廊道选址 +""" + +class EthicalCorridorSelector: + """伦理导向的廊道选址器""" + + def __init__(self): + self.stakeholders = { + 'ecology': {'weight': 0.4, 'concern': '生态连通性'}, + 'community': {'weight': 0.3, 'concern': '社区利益'}, + 'economy': {'weight': 0.2, 'concern': '经济成本'}, + 'culture': {'weight': 0.1, 'concern': '文化价值'} + } + + def evaluate_corridor_route(self, route, context): + """ + 评估廊道路线 + + 返回：综合评分和各利益相关方的影响 + """ + scores = {} + + # 生态评分 + scores['ecology'] = self._evaluate_ecological_value(route, context) + + # 社区评分 + scores['community'] = self._evaluate_community_impact(route, context) + + # 经济评分 + scores['economy'] = self._evaluate_economic_cost(route, context) + + # 文化评分 + scores['culture'] = self._evaluate_cultural_impact(route, context) + + # 加权综合 + total = sum( + scores[stakeholder] * self.stakeholders[stakeholder]['weight'] + for stakeholder in self.stakeholders + ) + + # 检查任何一方的严重负面影响 + for stakeholder, score in scores.items(): + if score < 0.3: # 阈值 + return { + 'acceptable': False, + 'reason': f"{stakeholder}评分过低: {score:.2f}", + 'scores': scores, + 'total': total + } + + return { + 'acceptable': True, + 'total_score': total, + 'scores': scores, + 'breakdown': { + stakeholder: { + 'score': scores[stakeholder], + 'weight': self.stakeholders[stakeholder]['weight'], + 'concern': self.stakeholders[stakeholder]['concern'] + } + for stakeholder in self.stakeholders + } + } + + def _evaluate_ecological_value(self, route, context): + """评估生态价值""" + # 连通的源地质量 + source_quality = self._connected_source_quality(route, context) + + # 廊道宽度 + width_score = min(route['width'] / 100, 1.0) + + # 栖息地适宜性 + habitat_score = self._habitat_suitability(route, context) + + return (source_quality + width_score + habitat_score) / 3 + + def _evaluate_community_impact(self, route, context): + """评估社区影响""" + # 正面：休闲价值 + recreational_value = self._recreational_potential(route) + + # 负面：拆迁、限制使用 + negative_impact = self._negative_community_impact(route, context) + + return max(0, recreational_value - negative_impact) + + def _evaluate_economic_cost(self, route, context): + """评估经济成本（分数越高表示成本越可接受）""" + # 土地获取成本 + land_cost = route.get('land_cost', 0) + + # 建设成本 + construction_cost = route.get('construction_cost', 0) + + # 归一化：成本越低分数越高 + max_cost = context.get('max_budget', float('inf')) + total_cost = land_cost + construction_cost + + if total_cost > max_cost: + return 0 # 超预算 + else: + return 1 - (total_cost / max_cost) * 0.5 + + def _evaluate_cultural_impact(self, route, context): + """评估文化影响""" + # 是否涉及文化遗产 + cultural_sites = route.get('cultural_sites', []) + if cultural_sites: + return 0.3 # 低分，需要特别处理 + + # 是否支持传统文化活动 + traditional_use = route.get('supports_traditional_use', False) + if traditional_use: + return 1.0 + + return 0.7 # 中性 + + # ... 其他辅助方法 ... + +if __name__ == "__main__": + print("=== 伦理导向的廊道选址 ===\n") + + selector = EthicalCorridorSelector() + + # 示例路线 + route = { + 'width': 80, + 'land_cost': 500000, + 'construction_cost': 1000000, + 'cultural_sites': [], + 'supports_traditional_use': True + } + + context = { + 'max_budget': 2000000, + 'ecological_data': {}, + 'community_data': {} + } + + result = selector.evaluate_corridor_route(route, context) + + if result['acceptable']: + print(f"路线可接受，综合评分: {result['total_score']:.2f}") + print("\n各利益相关方评分:") + for stakeholder, info in result['breakdown'].items(): + print(f" {stakeholder}: {info['score']:.2f} " + f"(权重: {info['weight']}, 关注: {info['concern']})") + else: + print(f"路线不可接受: {result['reason']}") +``` + +### 案例3：透明度和可审计性 + +```python +""" +决策日志系统 +""" +import json +from datetime import datetime +from typing import Dict, Any, List + +class DecisionLog: + """决策日志系统""" + + def __init__(self, project_id: str): + self.project_id = project_id + self.entries: List[Dict] = [] + + def log_decision(self, decision_type: str, decision: Any, + rationale: str, alternatives: List[Dict], + metadata: Dict = None): + """记录决策""" + entry = { + 'timestamp': datetime.now().isoformat(), + 'project_id': self.project_id, + 'decision_type': decision_type, + 'decision': decision, + 'rationale': rationale, + 'alternatives': alternatives, + 'metadata': metadata or {} + } + self.entries.append(entry) + + def log_data_source(self, data_type: str, source: str, + quality: Dict, limitations: List[str]): + """记录数据来源""" + entry = { + 'timestamp': datetime.now().isoformat(), + 'project_id': self.project_id, + 'type': 'data_source', + 'data_type': data_type, + 'source': source, + 'quality': quality, + 'limitations': limitations + } + self.entries.append(entry) + + def log_model_info(self, model_name: str, version: str, + training_data: Dict, limitations: List[str]): + """记录模型信息""" + entry = { + 'timestamp': datetime.now().isoformat(), + 'project_id': self.project_id, + 'type': 'model_info', + 'model_name': model_name, + 'version': version, + 'training_data': training_data, + 'limitations': limitations + } + self.entries.append(entry) + + def export_audit_report(self) -> str: + """导出审计报告""" + report = { + 'project_id': self.project_id, + 'export_time': datetime.now().isoformat(), + 'entries': self.entries, + 'summary': self._generate_summary() + } + return json.dumps(report, indent=2, ensure_ascii=False) + + def _generate_summary(self) -> Dict: + """生成摘要""" + summary = { + 'total_entries': len(self.entries), + 'decision_types': {}, + 'data_sources': [], + 'models_used': [] + } + + for entry in self.entries: + if entry.get('type') == 'data_source': + summary['data_sources'].append(entry['data_type']) + elif entry.get('type') == 'model_info': + summary['models_used'].append(entry['model_name']) + elif 'decision_type' in entry: + dt = entry['decision_type'] + summary['decision_types'][dt] = \ + summary['decision_types'].get(dt, 0) + 1 + + return summary +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **价值权衡**：当生态目标和社会目标冲突时，应该如何权衡？谁有权决定？ + +2. **偏见识别**：你的空间分析可能隐含哪些偏见？如何检测？ + +3. **透明度边界**：哪些决策细节必须公开？哪些可以保密？ + +4. **长期责任**：AI辅助的决策出现问题后，如何追溯和纠正？ + +### 实践练习 + +1. **伦理审计**：对你做过的一个空间项目进行伦理审计 + +2. **利益相关者地图**：绘制项目的利益相关者及其关注点 + +3. **透明度检查**：为你的分析流程建立可审计的文档体系 + +### 延伸阅读 + +- **"Weapons of Math Destruction"** (Cathy O'Neil) - 算法的社会影响 +- **"The Alignment Problem"** (Brian Christian) - AI对齐问题 +- **"Spatial Justice"** (Edward Soja) - 空间正义理论 +- UN-Habitat's ethics guidelines for spatial planning + +--- + +## 关键要点 + +1. **空间决策具有深刻的伦理维度**，影响资源分配和社会正义 +2. **AI可能放大既有偏见**，需要主动的公平性设计 +3. **可解释性是负责任AI的基础**，决策过程应可审查 +4. **建立清晰的问责机制**，明确各方责任 +5. **伦理思考应该贯穿整个项目生命周期**，而非事后补充 diff --git a/officefile/supplements/05-reflection/05.3-technical-iteration.md b/officefile/supplements/05-reflection/05.3-technical-iteration.md new file mode 100644 index 0000000..da97e74 --- /dev/null +++ b/officefile/supplements/05-reflection/05.3-technical-iteration.md @@ -0,0 +1,798 @@ +# 05.3 技术迭代与持久知识 + +## 核心问题 + +> 在快速变化的技术环境中，哪些知识会持久？ +> 如何判断新技术值得投入时间学习？ +> 如何建立可持续的知识更新机制？ + +--- + +## 概念讲解 + +### 技术变化的层次 + +理解技术变化的本质，帮助区分短暂潮流和持久价值： + +``` +技术变化的四个层次 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ Level 4: 应用层 (Application) 变化最快 │ +│ ─────────────────────────────────────────────────── │ +│ • 具体工具和框架 (e.g., QGIS 3.x → 4.x) │ +│ • API和语法细节 │ +│ • 特定库的使用模式 │ +│ 半衰期: 1-2年 │ +│ │ +│ Level 3: 方法层 (Methodology) │ +│ ─────────────────────────────────────────────────── │ +│ • 空间分析方法 (e.g., Circuit theory, Least-cost path) │ +│ • 工作流设计模式 │ +│ • 数据处理策略 │ +│ 半衰期: 5-10年 │ +│ │ +│ Level 2: 原理层 (Principle) │ +│ ─────────────────────────────────────────────────── │ +│ • 空间统计原理 │ +│ • 图论和网络分析 │ +│ • 优化理论 │ +│ 半衰期: 20-50年 │ +│ │ +│ Level 1: 思维层 (Thinking) 变化最慢 │ +│ ─────────────────────────────────────────────────── │ +│ • 空间思维方式 │ +│ • 系统思维 │ +│ • 批判性思维 │ +│ 半衰期: 基本不变 │ +│ │ +└─────────────────────────────────────────────────────────────┘ + +学习策略：向上投资 + 更多时间投入在原理层和思维层 + 应用层的知识随用随学 +``` + +### 什么在变，什么不变 + +| 变化的 | 相对稳定的 | 持久不变的 | +|-------|-----------|-----------| +| 工具版本 | 空间分析方法 | 空间问题本质 | +| API设计 | 数据结构设计 | 几何公理 | +| 算法实现 | 架构模式 | 数学原理 | +| 命令语法 | 工作流逻辑 | 项目目标 | +| 框架生态 | 问题分解方法 | 人类需求 | +| 数据格式 | 验证策略 | 质量标准 | + +### 持久知识框架 + +```python +""" +持久知识管理系统 +""" +from typing import Dict, List, Set, Optional +from dataclasses import dataclass +from enum import Enum + +class KnowledgeLevel(Enum): + """知识层次""" + APPLICATION = "application" # 应用层：快速变化 + METHODOLOGY = "methodology" # 方法层：中速变化 + PRINCIPLE = "principle" # 原理层：慢速变化 + THINKING = "thinking" # 思维层：基本不变 + +class KnowledgeStatus(Enum): + """知识状态""" + NEW = "new" # 新知识 + LEARNING = "learning" # 学习中 + MASTERED = "mastered" # 已掌握 + OUTDATED = "outdated" # 已过时 + +@dataclass +class KnowledgeItem: + """知识条目""" + name: str + level: KnowledgeLevel + status: KnowledgeStatus + related_principles: List[str] # 关联的原理层知识 + practical_applications: List[str] # 实际应用场景 + last_reviewed: str # ISO日期 + confidence: float # 0-1 + +class KnowledgeSystem: + """个人知识系统""" + + def __init__(self): + self.items: Dict[str, KnowledgeItem] = {} + self.principles: Set[str] = set() + self.connections: Dict[str, List[str]] = {} # 知识关联图 + + def add_principle(self, principle: str, description: str): + """添加核心原理（持久知识）""" + self.principles.add(principle) + print(f"添加核心原理: {principle}") + + def learn_technology(self, tech_name: str, + related_principles: List[str], + applications: List[str]): + """ + 学习新技术时，关联到原理 + + 这样即使技术过时，原理知识仍然有用 + """ + # 检查关联的原理是否都已记录 + for p in related_principles: + if p not in self.principles: + print(f"警告: 关联原理 '{p}' 不在知识库中") + + # 添加知识条目 + self.items[tech_name] = KnowledgeItem( + name=tech_name, + level=self._guess_level(tech_name), + status=KnowledgeStatus.LEARNING, + related_principles=related_principles, + practical_applications=applications, + last_reviewed="", + confidence=0.5 + ) + + # 建立关联 + self._build_connections(tech_name, related_principles) + + def review_knowledge(self, tech_name: str, current_value: int): + """ + 定期回顾知识 + + current_value: 0-10，该技术当前的价值评分 + """ + if tech_name not in self.items: + return + + item = self.items[tech_name] + + # 如果评分低，标记为过时 + if current_value < 3: + item.status = KnowledgeStatus.OUTDATED + print(f"{tech_name} 已过时，但原理知识保留:") + for p in item.related_principles: + print(f" - {p}") + else: + item.status = KnowledgeStatus.MASTERED + item.confidence = min(current_value / 10, 1.0) + + item.last_reviewed = self._get_date() + + def get_learning_priority(self) -> List[str]: + """ + 获取学习优先级 + + 策略：优先学习那些 + 1. 关联多个重要原理的技术 + 2. 当前价值高但尚未掌握的 + """ + priorities = [] + + for name, item in self.items.items(): + if item.status == KnowledgeStatus.OUTDATED: + continue + + # 计算优先级分数 + score = 0 + + # 原理覆盖度 + score += len(item.related_principles) * 10 + + # 当前价值 + if item.status == KnowledgeStatus.NEW: + score += 20 + + # 掌握度（越低越需要学习） + score += (1 - item.confidence) * 30 + + priorities.append((name, score)) + + priorities.sort(key=lambda x: x[1], reverse=True) + return [p[0] for p in priorities] + + def _guess_level(self, name: str) -> KnowledgeLevel: + """根据名称猜测知识层次""" + # 工具、框架通常是应用层 + tool_keywords = ['qgis', 'arcgis', 'python', 'library', 'api'] + if any(kw in name.lower() for kw in tool_keywords): + return KnowledgeLevel.APPLICATION + + # 方法类词汇通常是方法层 + method_keywords = ['analysis', 'method', 'approach', 'workflow'] + if any(kw in name.lower() for kw in method_keywords): + return KnowledgeLevel.METHODOLOGY + + return KnowledgeLevel.PRINCIPLE + + def _build_connections(self, tech: str, principles: List[str]): + """建立知识关联""" + self.connections[tech] = principles + + def _get_date(self) -> str: + """获取当前日期""" + from datetime import datetime + return datetime.now().isoformat() + + def get_principle_coverage(self) -> Dict[str, List[str]]: + """ + 获取原理覆盖情况 + + 返回：每个原理相关的技术 + """ + coverage = {p: [] for p in self.principles} + + for tech, principles in self.connections.items(): + for p in principles: + if p in coverage: + coverage[p].append(tech) + + return coverage + +# === 示例：空间AI知识体系 === + +def build_spatial_ai_knowledge_system(): + """构建空间AI知识体系""" + ks = KnowledgeSystem() + + # 添加核心原理（持久知识） + core_principles = [ + ("空间自相关", "空间上相近的事物更相似"), + ("尺度效应", "空间模式随观测尺度变化"), + ("拓扑关系", "空间对象之间的邻接、包含等关系"), + ("距离衰减", "相互作用随距离减弱"), + ("最小阻力路径", "考虑阻力的最优路径"), + ("源-汇动态", "资源在源和汇之间的流动"), + ("景观格局", "空间配置对生态过程的影响"), + ] + + for principle, description in core_principles: + ks.add_principle(principle, description) + + # 学习具体技术（关联到原理） + technologies = [ + ("Morpheus (软件)", ["景观格局"], ["景观指数计算"]), + ("Circuitscape", ["最小阻力路径", "源-汇动态"], ["生态连通性分析"]), + ("Linkage Mapper", ["最小阻力路径"], ["廊道识别"]), + ("Geoda", ["空间自相关"], ["空间自相关分析"]), + ("QGIS Processing", [], ["空间分析自动化"]), + ] + + for tech, principles, apps in technologies: + ks.learn_technology(tech, principles, apps) + + return ks + +if __name__ == "__main__": + print("=== 持久知识管理系统 ===\n") + + ks = build_spatial_ai_knowledge_system() + + print("\n--- 原理覆盖情况 ---") + coverage = ks.get_principle_coverage() + for principle, techs in coverage.items(): + if techs: + print(f"\n{principle}:") + for tech in techs: + print(f" - {tech}") + else: + print(f"\n{principle}: (尚无相关技术)") + + print("\n--- 学习优先级 ---") + priorities = ks.get_learning_priority() + for i, tech in enumerate(priorities[:5], 1): + print(f"{i}. {tech}") +``` + +--- + +## 设计原理 + +### 技术评估框架 + +在决定是否学习某项新技术时，使用系统化的评估： + +```python +""" +技术价值评估框架 +""" +from typing import Dict, List, Callable +from dataclasses import dataclass + +@dataclass +class TechAssessment: + """技术评估结果""" + name: str + total_score: float + dimension_scores: Dict[str, float] + recommendation: str + reasoning: List[str] + +class TechnologyEvaluator: + """技术评估器""" + + def __init__(self): + self.dimensions = { + 'principle_value': 0.3, # 原理价值：是否关联深层原理 + 'applicability': 0.25, # 适用性：应用范围广度 + 'longevity': 0.2, # 持久性：预计技术寿命 + 'community': 0.15, # 社区：生态系统活跃度 + 'learning_cost': 0.1, # 学习成本（负向） + } + + def evaluate(self, tech_name: str, + principle_links: List[str], + applications: List[str], + maturity: str, + community_size: str, + estimated_hours: int) -> TechAssessment: + + scores = {} + + # 1. 原理价值 + scores['principle_value'] = min(len(principle_links) * 0.2, 1.0) + + # 2. 适用性 + scores['applicability'] = min(len(applications) * 0.15, 1.0) + + # 3. 持久性 + longevity_scores = { + 'concept': 1.0, + 'standard': 0.8, + 'emerging': 0.5, + 'experimental': 0.2 + } + scores['longevity'] = longevity_scores.get(maturity, 0.5) + + # 4. 社区 + community_scores = { + 'large': 1.0, + 'medium': 0.7, + 'small': 0.4, + 'tiny': 0.2 + } + scores['community'] = community_scores.get(community_size, 0.5) + + # 5. 学习成本（负向） + scores['learning_cost'] = max(0, 1 - estimated_hours / 100) + + # 计算加权总分 + total = sum( + scores[dim] * weight + for dim, weight in self.dimensions.items() + ) + + # 生成建议 + recommendation, reasoning = self._generate_recommendation( + scores, total, estimated_hours + ) + + return TechAssessment( + name=tech_name, + total_score=total, + dimension_scores=scores, + recommendation=recommendation, + reasoning=reasoning + ) + + def _generate_recommendation(self, scores: Dict, total: float, + hours: int) -> tuple: + """生成建议""" + reasoning = [] + + # 分析各维度 + if scores['principle_value'] < 0.3: + reasoning.append("原理价值较低，可能只是工具层知识") + elif scores['principle_value'] > 0.8: + reasoning.append("关联多个核心原理，学习价值高") + + if scores['longevity'] < 0.5: + reasoning.append("技术成熟度低，可能快速变化") + elif scores['longevity'] > 0.8: + reasoning.append("技术相对稳定，知识可持久") + + if hours > 50 and total < 0.6: + reasoning.append(f"学习成本高({hours}h)但综合价值低") + + # 总体建议 + if total > 0.7: + rec = "强烈推荐学习" + elif total > 0.5: + rec = "值得学习" + elif total > 0.3: + rec = "按需学习" + else: + rec = "不推荐投入时间" + + return rec, reasoning + +# === 技术评估示例 === + +if __name__ == "__main__": + print("=== 技术价值评估 ===\n") + + evaluator = TechnologyEvaluator() + + technologies = [ + { + 'name': 'Spatial SQL (PostGIS)', + 'principles': ['拓扑关系', '空间查询', '集合操作'], + 'applications': ['数据管理', '空间分析', '服务提供'], + 'maturity': 'standard', + 'community': 'large', + 'hours': 40 + }, + { + 'name': '某新兴AI框架', + 'principles': ['深度学习'], + 'applications': ['图像识别'], + 'maturity': 'experimental', + 'community': 'small', + 'hours': 80 + }, + { + 'name': '景观格局分析理论', + 'principles': ['景观格局', '尺度效应', '空间异质性'], + 'applications': ['生态评价', '景观规划', '环境评估'], + 'maturity': 'concept', + 'community': 'medium', + 'hours': 30 + }, + ] + + for tech in technologies: + result = evaluator.evaluate(**tech) + print(f"\n{result.name}") + print(f"总分: {result.total_score:.2f}") + print(f"建议: {result.recommendation}") + print("各维度:") + for dim, score in result.dimension_scores.items(): + bar = "█" * int(score * 20) + print(f" {dim}: {bar} {score:.2f}") + print("理由:") + for r in result.reasoning: + print(f" - {r}") +``` + +### 持续学习策略 + +``` +持续学习的三角模型 + + ┌─────────────────┐ + │ 主动学习 │ + │ - 探索新领域 │ + │ - 预测趋势 │ + └────────┬────────┘ + │ + │ + ┌─────────────────┼─────────────────┐ + │ │ │ +┌──────┴──────┐ ┌─────┴─────┐ ┌─────┴─────┐ +│ 响应式学习 │ │ 反思整合 │ │ 社区连接 │ +│ │ │ │ │ │ +│ - 解决问题 │ │ - 定期回顾│ │ - 参与讨论│ +│ - 查漏补缺 │ │ - 写笔记 │ │ - 分享知识│ +│ - 即时学习 │ │ - 建立连接│ │ - 获取反馈│ +└─────────────┘ └───────────┘ └───────────┘ + +各部分的实践方法： + +1. 响应式学习 + - 遇到问题时记录下来 + - 快速查找解决方案 + - 事后总结为知识条目 + +2. 主动学习 + - 关注领域顶级会议/期刊 + - 订阅精选博客/通讯 + - 定期探索新技术 + +3. 反思整合 + - 每周/每月回顾 + - 更新知识图谱 + - 重写过时笔记 + +4. 社区连接 + - 加入专业社区 + - 参与开源项目 + - 组织学习小组 +``` + +--- + +## 案例分析 + +### 案例1：从ArcGIS到QGIS的迁移 + +**背景**：GIS工具的变化，但原理知识保持不变。 + +``` +迁移时的知识对比： + +ArcGIS (ArcPy) QGIS (PyQGIS) +───────────────────────────────────────────────── +arcpy.mp QgsProject +│ ├── Map │ ├── QgsLayout +│ └── Layer │ └── QgsMapLayer + │ +arcpy.sa (Spatial Analyst) Processing algorithms +│ ├── Raster calculator │ ├── QgsProcessingAlgorithm +│ └── Zonal statistics │ └── QgsProcessingContext + │ +ModelBuilder Graphical Modeler +│ ├── Intermediate data │ ├── Input/Output +│ └── Iterators │ └── Modeller algorithms + +不变的核心知识： +• 理解栅格和矢量数据结构 +• 理解投影和坐标系统 +• 理解空间分析的逻辑流程 +• 理解数据模型和拓扑关系 + +变化的部分： +• API和类名 +• 具体操作语法 +• 界面操作方式 + +迁移策略： +1. 用一个下午对照API文档做转换 +2. 建立常用操作的对照表 +3. 重点理解新工具的架构设计 +``` + +### 案例2：深度学习在遥感中的应用 + +**背景**：新技术快速迭代，但基础概念相对稳定。 + +```python +""" +深度学习技术栈的层次分析 +""" + +# 短半衰期（1-2年）- 随用随学 +short_lived = [ + "特定模型架构 (e.g., U-Net变体)", + "训练框架 (e.g., PyTorch vs TensorFlow)", + "预处理工具", + "特定数据集格式" +] + +# 中半衰期（5-10年）- 重点学习 +medium_lived = [ + "卷积操作原理", + "迁移学习策略", + "数据增强方法", + "模型评估指标", + "特征可视化技术" +] + +# 长半衰期（20+年）- 深入理解 +long_lived = [ + "梯度下降优化原理", + "过拟合与正则化", + "偏差-方差权衡", + "交叉验证", + "损失函数设计" +] + +# 空间AI特有的持久知识 +spatial_essentials = [ + "空间自相关及其对训练集的影响", + "空间交叉验证（防止空间泄漏）", + "尺度效应与感受野", + "空间不确定性量化", + "可解释性在空间决策中的重要性" +] + +print("深度学习在遥感中的知识层次\n") +print("短期（随用随学）:") +for item in short_lived: + print(f" - {item}") +print("\n中期（重点学习）:") +for item in medium_lived: + print(f" - {item}") +print("\n长期（深入理解）:") +for item in long_lived: + print(f" - {item}") +print("\n空间AI核心:") +for item in spatial_essentials: + print(f" - {item}") +``` + +### 案例3：个人知识更新机制 + +```python +""" +定期知识回顾机制 +""" +from datetime import datetime, timedelta +from typing import List, Dict + +class LearningCalendar: + """学习日历""" + + def __init__(self): + self.review_queue = [] + self.last_review = {} + + def schedule_review(self, topic: str, level: str, initial_days: int): + """ + 安排复习 + + 使用间隔重复策略 + """ + schedule = { + 'new': [1, 3, 7, 14, 30], # 新知识：密集复习 + 'stable': [30, 90, 180, 365], # 稳定知识：稀疏复习 + 'archived': [365, 730] # 归档知识：年度检查 + } + + for days in schedule.get(level, schedule['stable']): + review_date = datetime.now() + timedelta(days=days) + self.review_queue.append({ + 'topic': topic, + 'date': review_date, + 'level': level + }) + + def get_due_reviews(self) -> List[Dict]: + """获取到期的复习""" + now = datetime.now() + return [ + item for item in self.review_queue + if item['date'] <= now + ] + + def complete_review(self, topic: str, quality: int): + """ + 完成复习 + + quality: 1-5，复习质量评分 + """ + self.last_review[topic] = { + 'date': datetime.now(), + 'quality': quality + } + + # 根据质量调整下次复习时间 + if quality >= 4: + # 掌握良好，延长时间间隔 + pass + elif quality <= 2: + # 掌握不好，重新安排密集复习 + pass + +class KnowledgeJournal: + """知识日志""" + + def __init__(self): + self.entries = [] + + def log_learning(self, topic: str, what: str, why: str, + how: str, connections: List[str]): + """ + 记录学习 + + 使用What-Why-How框架 + """ + entry = { + 'date': datetime.now().isoformat(), + 'topic': topic, + 'what': what, # 学到了什么 + 'why': why, # 为什么重要 + 'how': how, # 如何应用 + 'connections': connections, # 与已有知识的连接 + 'questions': [] # 未解决的问题 + } + self.entries.append(entry) + + def log_question(self, topic: str, question: str): + """记录问题""" + # 找到相关条目或创建新的 + for entry in self.entries: + if entry['topic'] == topic: + entry['questions'].append({ + 'question': question, + 'date': datetime.now().isoformat() + }) + return + + def get_review_prompt(self, topic: str) -> str: + """生成复习提示""" + for entry in self.entries: + if entry['topic'] == topic: + prompt = f""" +复习主题: {topic} + +学习内容: +{entry['what']} + +重要性: +{entry['why']} + +应用方式: +{entry['how']} + +关联知识: +{', '.join(entry['connections'])} + +未解决问题: +{chr(10).join(q['question'] for q in entry['questions'])} + +复习问题: +1. 这个知识的核心是什么？ +2. 我在哪些场景中应用过它？ +3. 它与哪些其他知识相关？ +4. 我还有哪些不明白的地方？ +""" + return prompt + return f"未找到主题: {topic}" + +if __name__ == "__main__": + print("=== 个人知识更新系统 ===\n") + + calendar = LearningCalendar() + journal = KnowledgeJournal() + + # 示例：学习一个新技术 + journal.log_learning( + topic="空间交叉验证", + what="防止训练集和测试集空间自相关导致的模型过拟合", + why="标准交叉验证假设样本独立，空间数据违反此假设", + how="使用空间阻塞（Spatial Block）或缓冲区划分", + connections=["空间自相关", "模型评估", "过拟合"] + ) + + # 安排复习 + calendar.schedule_review("空间交叉验证", "new", 1) + + print("知识已记录，复习计划已安排") + + # 获取复习提示 + prompt = journal.get_review_prompt("空间交叉验证") + print("\n--- 复习提示 ---") + print(prompt) +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **知识审计**：你花时间学习的技能中，哪些已经过时？哪些仍然有价值？ + +2. **学习策略**：你目前的学习时间分配在哪个层次？是否需要调整？ + +3. **趋势判断**：你如何区分暂时热潮和真正重要的趋势？ + +4. **知识管理**：你如何追踪和管理自己的知识体系？ + +### 实践练习 + +1. **知识分层**：列出你最近学习的5项技术，分类到4个层次 + +2. **技术评估**：使用评估框架评估一个你正在考虑学习的技术 + +3. **回顾机制**：建立你自己的知识回顾系统 + +### 延伸阅读 + +- **"Make It Stick"** - 学习的科学 +- **"Ultralearning"** (Scott Young) - 高效自学方法 +- **"Range"** (David Epstein) - 广度vs深度的权衡 + +--- + +## 关键要点 + +1. **技术变化有层次**，原理层和思维层的变化远慢于应用层 +2. **学习投资应向上倾斜**，更多投入在持久知识上 +3. **建立技术评估框架**，系统化判断学习价值 +4. **设计持续学习机制**，包括回顾、整合和社区参与 +5. **理解什么不变比追逐什么在变更重要** diff --git a/officefile/supplements/05-reflection/05.4-future-directions.md b/officefile/supplements/05-reflection/05.4-future-directions.md new file mode 100644 index 0000000..773afb4 --- /dev/null +++ b/officefile/supplements/05-reflection/05.4-future-directions.md @@ -0,0 +1,985 @@ +# 05.4 空间AI的未来方向 + +## 核心问题 + +> 空间大模型会如何改变空间分析和规划？ +> 多模态AI如何融合遥感、地图、文本等异构数据？ +> 具身智能与空间智能的关系是什么？ + +--- + +## 概念讲解 + +### 空间大模型 (Spatial Large Language Models) + +空间大模型是将空间理解能力融入大规模语言模型的新方向： + +``` +空间大模型的演进 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 阶段1: 通用LLM │ +│ ─────────────────────────────────────────────────── │ +│ • ChatGPT, Claude等 │ +│ • 可以讨论空间概念，但无法真正理解 │ +│ • 依赖外部工具进行空间计算 │ +│ │ +│ 阶段2: 空间增强LLM (Spatially-Enhanced LLM) │ +│ ─────────────────────────────────────────────────── │ +│ • 集成GIS工具和空间数据库 │ +│ • 可以执行空间查询和分析 │ +│ • 例如：Llama with GIS tools, GeoLLM │ +│ │ +│ 阶段3: 空间原生LLM (Spatially-Native LLM) [发展中] │ +│ ─────────────────────────────────────────────────── │ +│ • 空间概念嵌入模型架构 │ +│ • 原生支持空间推理和几何运算 │ +│ • 理解投影、拓扑、尺度等 │ +│ │ +│ 阶段4: 世界模型 (World Models) [未来] │ +│ ─────────────────────────────────────────────────── │ +│ • 内化对世界的空间理解 │ +│ • 可以模拟和预测空间变化 │ +│ • 支持复杂的空间规划任务 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 空间大模型的核心能力 + +| 能力 | 描述 | 当前状态 | +|-----|------|---------| +| **空间概念理解** | 理解距离、方向、邻近、包含等 | 部分实现 | +| **空间推理** | 基于空间关系的逻辑推理 | 早期阶段 | +| **几何操作** | 直接进行缓冲、叠加等运算 | 依赖工具 | +| **空间视觉理解** | 从地图/遥感图像提取信息 | 快速发展 | +| **多尺度理解** | 处理不同尺度的空间问题 | 研究中 | +| **时空间建模** | 理解空间随时间的变化 | 早期阶段 | + +### 多模态空间AI + +``` +多模态空间数据融合 + + ┌─────────────────┐ + │ 空间大模型 │ + │ │ + │ 统一的表示 │ + └────────┬────────┘ + │ + ┌────────────────────┼────────────────────┐ + │ │ │ +┌───────┴───────┐ ┌────────┴────────┐ ┌─────┴─────┐ +│ 视觉模态 │ │ 文本模态 │ │ 结构模态 │ +│ │ │ │ │ │ +│ • 遥感影像 │ │ • 描述性文本 │ │ • 矢量数据│ +│ • 航拍照片 │ │ • 规划文档 │ │ • 拓扑关系│ +│ • 街景图像 │ │ • 专家知识 │ │ • 网络结构│ +│ • 地图截图 │ │ • 社交媒体 │ │ • 统计数据│ +└──────────────┘ └─────────────────┘ └───────────┘ + +融合挑战： +1. 对齐：不同模态的空间对齐 +2. 互补：利用各模态的优势 +3. 冲突：处理模态间的不一致 +4. 稀疏性：某些区域数据缺失 +``` + +### 具身智能与空间智能 + +``` +具身智能 (Embodied AI) + │ + └── 具有物理身体、能与真实世界交互的AI + +空间智能是具身智能的基础： + +具身智能需要的空间能力 +├── 空间感知 (Perception) +│ ├── 视觉SLAM (同步定位与地图构建) +│ ├── 物体识别与定位 +│ └── 场景理解 +│ +├── 空间推理 (Reasoning) +│ ├── 路径规划 +│ ├── 障碍物避让 +│ └── 操作空间估计 +│ +├── 空间行动 (Action) +│ ├── 导航 +│ ├── 物体操作 +│ └── 与环境交互 +│ +└── 空间学习 (Learning) + ├── 环境地图构建 + ├── 动态更新 + └── 经验积累 + +应用场景： +• 自主驾驶 +• 服务机器人 +• 仓储物流 +• 灾难救援 +• 行星探索 +``` + +--- + +## 设计原理 + +### 空间大模型的应用架构 + +```python +""" +空间大模型应用架构设计 +""" +from typing import Dict, List, Any, Optional, Union +from dataclasses import dataclass +from abc import ABC, abstractmethod + +@dataclass +class SpatialContext: + """空间上下文""" + extent: Dict[str, float] # 范围 {xmin, ymin, xmax, ymax} + crs: str # 坐标系 + resolution: float # 分辨率 + scale: str # 尺度等级 + temporal: Optional[str] # 时间维度 + +@dataclass +class SpatialQuery: + """空间查询""" + natural_language: str # 自然语言描述 + spatial_context: SpatialContext + required_output: str # 输出格式要求 + constraints: List[str] # 约束条件 + +class SpatialCapability(ABC): + """空间能力抽象""" + + @abstractmethod + def can_handle(self, query: SpatialQuery) -> float: + """判断是否能处理此查询，返回置信度""" + pass + + @abstractmethod + def execute(self, query: SpatialQuery) -> Any: + """执行查询""" + pass + +class SpatialLLM: + """空间大模型接口""" + + def __init__(self): + self.capabilities: List[SpatialCapability] = [] + self.memory = {} # 空间记忆 + + def add_capability(self, capability: SpatialCapability): + """添加空间能力""" + self.capabilities.append(capability) + + def query(self, query: Union[str, SpatialQuery]) -> Any: + """ + 处理空间查询 + + 支持自然语言输入，自动解析为结构化查询 + """ + # 如果是字符串，转换为SpatialQuery + if isinstance(query, str): + query = self._parse_natural_language(query) + + # 找到最合适的能力 + capability = self._select_capability(query) + + # 执行 + result = capability.execute(query) + + # 更新记忆 + self._update_memory(query, result) + + return result + + def _parse_natural_language(self, text: str) -> SpatialQuery: + """将自然语言解析为空间查询""" + # 这里会调用LLM进行解析 + # 返回结构化的SpatialQuery + return SpatialQuery( + natural_language=text, + spatial_context=SpatialContext( + extent={}, crs='EPSG:4326', resolution=30, scale='medium' + ), + required_output='map', + constraints=[] + ) + + def _select_capability(self, query: SpatialQuery) -> SpatialCapability: + """选择最合适的能力""" + best_capability = None + best_score = 0 + + for cap in self.capabilities: + score = cap.can_handle(query) + if score > best_score: + best_score = score + best_capability = cap + + return best_capability or self.capabilities[0] + + def _update_memory(self, query: SpatialQuery, result: Any): + """更新空间记忆""" + # 存储查询-结果对，用于上下文学习 + pass + +# === 具体的空间能力 === + +class SpatialAnalysisCapability(SpatialCapability): + """空间分析能力""" + + def __init__(self, gis_backend): + self.gis = gis_backend + + def can_handle(self, query: SpatialQuery) -> float: + """判断是否能处理""" + # 检查关键词 + analysis_keywords = [ + 'buffer', 'intersect', 'nearby', 'within', + '缓冲', '相交', '附近', '内部' + ] + text = query.natural_language.lower() + + matches = sum(1 for kw in analysis_keywords if kw in text) + return min(matches * 0.3, 1.0) + + def execute(self, query: SpatialQuery) -> Any: + """执行空间分析""" + # 解析分析类型 + analysis_type = self._detect_analysis_type(query.natural_language) + + # 执行 + if analysis_type == 'buffer': + return self._buffer_analysis(query) + elif analysis_type == 'proximity': + return self._proximity_analysis(query) + else: + return {"error": "无法识别的分析类型"} + + def _detect_analysis_type(self, text: str) -> str: + """检测分析类型""" + if any(kw in text.lower() for kw in ['buffer', '缓冲']): + return 'buffer' + if any(kw in text.lower() for kw in ['near', 'closest', 'nearest', '附近']): + return 'proximity' + return 'unknown' + + def _buffer_analysis(self, query: SpatialQuery): + """缓冲区分析""" + # 实际实现会调用GIS后端 + return { + 'type': 'buffer', + 'result': 'buffer_result' + } + + def _proximity_analysis(self, query: SpatialQuery): + """邻近度分析""" + return { + 'type': 'proximity', + 'result': 'proximity_result' + } + +class SpatialVisualizationCapability(SpatialCapability): + """空间可视化能力""" + + def can_handle(self, query: SpatialQuery) -> float: + """判断是否能处理""" + vis_keywords = ['map', 'visualize', 'show', 'plot', 'display', + '地图', '显示', '可视化', '绘制'] + text = query.natural_language.lower() + + matches = sum(1 for kw in vis_keywords if kw in text) + return min(matches * 0.25, 1.0) + + def execute(self, query: SpatialQuery) -> Any: + """执行可视化""" + # 生成地图 + return { + 'type': 'map', + 'url': 'map_url' + } + +class SpatialReasoningCapability(SpatialCapability): + """空间推理能力""" + + def can_handle(self, query: SpatialQuery) -> float: + """判断是否能处理""" + reason_keywords = ['why', 'how', 'best', 'optimal', + '为什么', '如何', '最好', '最优'] + text = query.natural_language.lower() + + matches = sum(1 for kw in reason_keywords if kw in text) + return min(matches * 0.2, 1.0) + + def execute(self, query: SpatialQuery) -> Any: + """执行空间推理""" + # 分析空间关系，给出解释和建议 + return { + 'type': 'reasoning', + 'explanation': '基于空间关系的分析', + 'recommendation': '建议的方案' + } + +# === 多模态融合 === + +class MultimodalSpatialProcessor: + """多模态空间处理器""" + + def __init__(self): + self.vision_encoder = None # 视觉编码器 + self.text_encoder = None # 文本编码器 + self.structure_encoder = None # 结构编码器 + self.fusion_layer = None # 融合层 + + def process(self, + image=None, + text=None, + vector_data=None) -> Dict: + """ + 处理多模态输入 + + 融合图像、文本和矢量数据 + """ + embeddings = {} + + # 编码各模态 + if image is not None: + embeddings['vision'] = self._encode_image(image) + + if text is not None: + embeddings['text'] = self._encode_text(text) + + if vector_data is not None: + embeddings['structure'] = self._encode_structure(vector_data) + + # 融合 + if len(embeddings) > 1: + fused = self._fuse_embeddings(embeddings) + else: + fused = list(embeddings.values())[0] + + return { + 'embeddings': embeddings, + 'fused': fused + } + + def _encode_image(self, image): + """编码图像""" + # 使用视觉编码器（如ViT） + return f"image_embedding_{hash(image)}" + + def _encode_text(self, text): + """编码文本""" + # 使用文本编码器（如BERT） + return f"text_embedding_{hash(text)}" + + def _encode_structure(self, vector_data): + """编码矢量结构""" + # 使用图神经网络 + return f"structure_embedding_{hash(str(vector_data))}" + + def _fuse_embeddings(self, embeddings: Dict) -> str: + """融合嵌入""" + # 使用注意力机制融合 + return "fused_embedding" + +# === 使用示例 === + +if __name__ == "__main__": + print("=== 空间大模型应用架构 ===\n") + + # 创建空间大模型 + spatial_llm = SpatialLLM() + + # 添加能力 + spatial_llm.add_capability(SpatialAnalysisCapability("gis_backend")) + spatial_llm.add_capability(SpatialVisualizationCapability()) + spatial_llm.add_capability(SpatialReasoningCapability()) + + # 示例查询 + queries = [ + "找出距离公园500米内的所有建筑", + "可视化城市的热岛效应分布", + "为什么这个区域的生态连通性较差？" + ] + + print("处理查询:") + for query in queries: + print(f"\n查询: {query}") + result = spatial_llm.query(query) + print(f"结果类型: {result.get('type', 'unknown')}") + + print("\n=== 多模态处理 ===") + processor = MultimodalSpatialProcessor() + result = processor.process( + image="satellite_image.tif", + text="这是一个城市公园", + vector_data={"type": "Polygon", "coordinates": [...]} + ) + print(f"融合结果: {result['fused']}") +``` + +### 具身智能的空间架构 + +```python +""" +具身智能的空间架构 +""" +from typing import List, Tuple, Optional +from dataclasses import dataclass +import numpy as np + +@dataclass +class Pose: + """位姿：位置和朝向""" + x: float + y: float + z: float + yaw: float # 偏航角 + pitch: float # 俯仰角 + roll: float # 翻滚角 + +@dataclass +class Observation: + """观测""" + pose: Pose + visual_data: np.ndarray # 图像数据 + depth_data: Optional[np.ndarray] # 深度数据 + point_cloud: Optional[np.ndarray] # 点云 + +@dataclass +class SpatialMemory: + """空间记忆""" + explored_area: List[Tuple[float, float]] # 已探索区域 + obstacles: List[Dict] # 障碍物位置 + semantic_labels: Dict # 语义标签 + confidence_map: np.ndarray # 置信度地图 + +class EmbodiedSpatialAgent: + """具身空间智能体""" + + def __init__(self): + self.pose = Pose(0, 0, 0, 0, 0, 0) + self.memory = SpatialMemory([], [], {}, np.zeros((100, 100))) + self.goal = None + + def perceive(self, observation: Observation): + """ + 感知环境 + + 从多模态传感器数据中提取空间信息 + """ + # 1. 本地化：更新自身位置 + self._localize(observation) + + # 2. 建图：更新环境地图 + self._update_map(observation) + + # 3. 识别：识别物体和场景 + self._identify_objects(observation) + + def plan(self, goal: Tuple[float, float]) -> List[Tuple[float, float]]: + """ + 规划路径 + + 从当前位置到目标位置 + """ + # 使用A*或其他路径规划算法 + # 考虑： + # - 已知的障碍物 + # - 地图的置信度 + # - 机器人的运动约束 + + path = self._astar_search( + start=(self.pose.x, self.pose.y), + goal=goal, + obstacles=self.memory.obstacles + ) + + return path + + def act(self, action: str) -> bool: + """ + 执行动作 + + 与物理世界交互 + """ + if action == "move_forward": + return self._move_forward() + elif action == "turn_left": + return self._turn_left() + elif action == "pick": + return self._pick_object() + else: + return False + + def _localize(self, observation: Observation): + """本地化：确定自身位置""" + # 使用SLAM (Simultaneous Localization and Mapping) + # 比对当前观测与已有地图 + pass + + def _update_map(self, observation: Observation): + """更新环境地图""" + # 整合新的观测数据到地图中 + # 更新已探索区域 + # 更新障碍物位置 + pass + + def _identify_objects(self, observation: Observation): + """识别物体和场景""" + # 使用计算机视觉识别物体 + # 将识别结果与空间位置关联 + pass + + def _astar_search(self, start, goal, obstacles): + """A*路径搜索""" + # 简化实现 + return [start, goal] + + def _move_forward(self): + """前进""" + self.pose.x += 0.1 * np.cos(self.pose.yaw) + self.pose.y += 0.1 * np.sin(self.pose.yaw) + return True + + def _turn_left(self): + """左转""" + self.pose.yaw += 0.1 + return True + + def _pick_object(self): + """抓取物体""" + # 检查前方是否有可抓取物体 + # 执行抓取动作 + return True + +# === 应用示例 === + +class DeliveryRobot(EmbodiedSpatialAgent): + """配送机器人""" + + def __init__(self): + super().__init__() + self.delivery_queue = [] + self.current_delivery = None + + def add_delivery(self, location: Tuple[float, float], item: str): + """添加配送任务""" + self.delivery_queue.append({ + 'location': location, + 'item': item, + 'status': 'pending' + }) + + def process_deliveries(self): + """处理配送队列""" + while self.delivery_queue: + # 获取下一个任务 + self.current_delivery = self.delivery_queue.pop(0) + + # 规划路径 + path = self.plan(self.current_delivery['location']) + + # 执行配送 + success = self._execute_delivery(path) + + # 更新状态 + if success: + self.current_delivery['status'] = 'completed' + else: + self.current_delivery['status'] = 'failed' + + def _execute_delivery(self, path): + """执行配送""" + # 沿路径移动 + for waypoint in path: + # 导航到路径点 + # 避障 + # 更新地图 + pass + + # 到达目的地，放下物品 + return True + +# === 空间智能的核心能力 === + +class SpatialIntelligenceTest: + """空间智能测试""" + + @staticmethod + def test_spatial_reasoning(agent): + """测试空间推理能力""" + questions = [ + "从当前位置到目标位置的最短路径是什么？", + "这个房间有几个出口？", + "物体A在物体B的哪个方向？" + ] + # 评估回答 + pass + + @staticmethod + def test_spatial_memory(agent): + """测试空间记忆能力""" + # 让机器人探索环境 + # 然后测试它对环境的记忆 + pass + + @staticmethod + def test_spatial_learning(agent): + """测试空间学习能力""" + # 在多次交互中测试学习效果 + pass + +if __name__ == "__main__": + print("=== 具身空间智能 ===\n") + + robot = DeliveryRobot() + + # 添加配送任务 + robot.add_delivery((10, 20), "包裹A") + robot.add_delivery((30, 40), "包裹B") + + print(f"配送队列: {len(robot.delivery_queue)} 个任务") + print("机器人能力:") + print(" - 空间感知：理解自身位置和环境") + print(" - 空间推理：规划最优路径") + print(" - 空间记忆：记住已探索区域") + print(" - 空间行动：在物理世界中移动和交互") +``` + +--- + +## 案例分析 + +### 案例1：城市规划的AI助手 + +**场景**：空间大模型辅助城市规划决策 + +```python +""" +城市规划AI助手示例 +""" + +class UrbanPlanningAssistant: + """城市规划AI助手""" + + def __init__(self, spatial_llm): + self.llm = spatial_llm + self.project_context = {} + + def analyze_site_suitability(self, + site: Dict, + project_type: str, + constraints: List[str]) -> Dict: + """ + 分析场地适宜性 + + 综合考虑： + - 空间位置和可达性 + - 周边环境 + - 政策约束 + - 社会经济因素 + """ + # 1. 理解项目类型 + project_requirements = self._understand_project_type(project_type) + + # 2. 收集场地信息 + site_info = self._collect_site_information(site) + + # 3. 多模态分析 + analysis = { + 'visual': self._analyze_visual_context(site), + 'spatial': self._analyze_spatial_context(site), + 'regulatory': self._analyze_regulatory_context(site), + 'social': self._analyze_social_context(site) + } + + # 4. 综合评估 + suitability = self._assess_suitability( + site_info, project_requirements, analysis, constraints + ) + + # 5. 生成解释 + explanation = self._generate_explanation( + suitability, analysis, project_requirements + ) + + return { + 'suitability_score': suitability['score'], + 'recommendation': suitability['recommendation'], + 'explanation': explanation, + 'analysis_details': analysis, + 'alternatives': self._suggest_alternatives(site, project_type) + } + + def _understand_project_type(self, project_type: str) -> Dict: + """理解项目类型的需求""" + # 使用LLM理解项目类型 + requirements = { + 'commercial': { + 'traffic_access': 'high', + 'visibility': 'high', + 'parking': 'required', + 'zoning': 'commercial' + }, + 'residential': { + 'quiet': 'high', + 'green_space': 'preferred', + 'schools_access': 'important', + 'zoning': 'residential' + }, + 'industrial': { + 'highway_access': 'high', + 'utilities': 'required', + 'buffer_from_residential': 'required', + 'zoning': 'industrial' + } + } + return requirements.get(project_type, {}) + + def _collect_site_information(self, site: Dict) -> Dict: + """收集场地信息""" + # 整合多源数据 + return { + 'location': site['coordinates'], + 'area': site.get('area'), + 'current_use': self._detect_current_use(site), + 'surroundings': self._analyze_surroundings(site) + } + + def _analyze_visual_context(self, site: Dict) -> Dict: + """分析视觉上下文（卫星图像、街景）""" + # 使用视觉模型分析 + return { + 'land_use': 'mixed', + 'building_density': 'medium', + 'green_coverage': 0.25, + 'visual_quality': 'good' + } + + def _analyze_spatial_context(self, site: Dict) -> Dict: + """分析空间上下文（可达性、邻近性）""" + # 使用GIS分析 + return { + 'accessibility_score': 0.75, + 'nearby_amenities': ['park', 'school', 'shopping'], + 'transit_access': 'good', + 'road_connectivity': 'high' + } + + def _analyze_regulatory_context(self, site: Dict) -> Dict: + """分析法规上下文（分区、规划政策）""" + return { + 'zoning': 'mixed_use', + 'height_limit': '30m', + 'far_limit': 2.5, + 'policy_constraints': ['heritage_buffer', 'flood_zone'] + } + + def _analyze_social_context(self, site: Dict) -> Dict: + """分析社会上下文（社区需求、公众意见）""" + return { + 'community_concerns': ['traffic', 'noise'], + 'support_level': 'moderate', + 'demographics': {'age_distribution': 'mixed'} + } + + def _assess_suitability(self, site_info, requirements, analysis, constraints): + """综合评估适宜性""" + score = 0.7 # 示例分数 + recommendation = "suitable_with_conditions" + + # 检查硬约束 + for constraint in constraints: + if not self._check_constraint(constraint, analysis): + score = min(score, 0.3) + recommendation = "not_recommended" + + return { + 'score': score, + 'recommendation': recommendation + } + + def _generate_explanation(self, suitability, analysis, requirements): + """生成解释""" + # 使用LLM生成自然语言解释 + return """ + 该场地总体适宜性评分为0.70，建议有条件使用。 + + 优势： + - 交通可达性良好 + - 周边配套设施完善 + - 符合分区要求 + + 需要关注： + - 社区对交通增加的担忧 + - 需要缓解潜在的噪音影响 + """ +``` + +### 案例2：灾害响应的空间AI + +```python +""" +灾害响应空间AI系统 +""" + +class DisasterResponseSystem: + """灾害响应空间AI系统""" + + def __init__(self): + self.situation_awareness = {} + self.resource_tracker = {} + self.action_planner = None + + def assess_disaster_impact(self, + disaster_type: str, + location: Dict, + affected_area: Dict) -> Dict: + """ + 评估灾害影响 + + 整合多源数据： + - 遥感影像（灾前灾后对比） + - 社交媒体（实时信息） + - 基础设施数据（脆弱性评估） + - 人口数据（暴露度评估） + """ + # 1. 获取多模态数据 + data = { + 'satellite': self._get_satellite_imagery(location), + 'social_media': self._analyze_social_media(location), + 'infrastructure': self._get_infrastructure_data(location), + 'population': self._get_population_data(location) + } + + # 2. 空间分析 + impact_assessment = { + 'severity_map': self._create_severity_map(data), + 'affected_population': self._estimate_affected_population(data), + 'damaged_infrastructure': self._identify_damage(data), + 'accessibility': self._assess_accessibility(location, disaster_type) + } + + # 3. 优先级排序 + priorities = self._prioritize_response(impact_assessment) + + return { + 'impact': impact_assessment, + 'priorities': priorities, + 'recommended_actions': self._generate_action_plan(priorities) + } + + def plan_evacuation_routes(self, + affected_areas: List[Dict], + shelter_locations: List[Dict], + road_conditions: Dict) -> List[Dict]: + """ + 规划疏散路线 + + 考虑： + - 受灾区域分布 + - 避难所容量 + - 道路状况（损坏、拥堵） + - 人口类型（老人、儿童、行动不便者） + """ + routes = [] + + for area in affected_areas: + # 找到最近的可用避难所 + available_shelters = self._find_available_shelters( + area, shelter_locations + ) + + # 计算最优路线 + for shelter in available_shelters: + route = self._calculate_route( + start=area['center'], + end=shelter['location'], + road_conditions=road_conditions, + constraints={'avoid_flood': True, 'avoid_damage': True} + ) + + if route['feasible']: + routes.append({ + 'from': area['name'], + 'to': shelter['name'], + 'route': route['path'], + 'estimated_time': route['time'], + 'capacity': route['capacity'], + 'risk_level': route['risk'] + }) + + return routes + + def monitor_situation(self, sensor_data: Dict) -> Dict: + """ + 监测灾情发展 + + 使用IoT传感器、无人机、卫星等实时数据 + """ + # 整合多源实时数据 + situation = { + 'flood_extent': self._monitor_flood_extent(sensor_data), + 'fire_spread': self._monitor_fire_spread(sensor_data), + 'structural_integrity': self._monitor_structures(sensor_data), + 'weather_conditions': self._monitor_weather(sensor_data) + } + + # 预测发展趋势 + forecast = self._forecast_development(situation) + + return { + 'current': situation, + 'forecast': forecast, + 'alerts': self._generate_alerts(forecast) + } +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **技术预期**：空间大模型在5年内最可能实现哪些突破？ + +2. **影响评估**：这些技术会如何改变你的工作方式？ + +3. **伦理考量**：更强大的空间AI带来哪些新的伦理挑战？ + +4. **准备策略**：如何为这些变化做准备？ + +### 实践练习 + +1. **趋势追踪**：选择一个方向（空间大模型/多模态/具身智能），追踪最新进展 + +2. **场景设计**：设想一个未来应用场景，描述空间AI如何发挥作用 + +3. **技能准备**：列出需要学习的新技能，制定学习计划 + +### 延伸阅读 + +- **"Spatial Computing"** 相关文献 - 空间计算的未来 +- **"Embodied AI"** 研究进展 - 具身智能前沿 +- **Multimodal Learning** 论文 - 多模态学习技术 +- AI for Science 相关报告 - AI在科学领域的应用 + +--- + +## 关键要点 + +1. **空间大模型正在快速发展**，将从工具增强走向原生支持 +2. **多模态融合是关键方向**，整合视觉、文本、结构数据 +3. **具身智能需要强大的空间能力**作为基础 +4. **技术演进带来新机遇**，也需要应对新的挑战 +5. **保持关注但保持批判**，理性评估技术成熟度和适用性 diff --git a/officefile/supplements/05-reflection/05.5-personal-knowledge-system.md b/officefile/supplements/05-reflection/05.5-personal-knowledge-system.md new file mode 100644 index 0000000..11202ec --- /dev/null +++ b/officefile/supplements/05-reflection/05.5-personal-knowledge-system.md @@ -0,0 +1,1322 @@ +# 05.5 个人知识体系 + +## 核心问题 + +> 如何建立可持续的个人AI工具箱？ +> 什么样的文档策略能有效支持长期学习？ +> 如何通过社区参与加速成长？ + +--- + +## 概念讲解 + +### 个人知识体系的结构 + +一个有效的个人知识体系应该是有机的、可演进的： + +``` +个人知识体系结构 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 第一层：知识获取层 (Acquisition) │ +│ ────────────────────────────────────────────────────── │ +│ • 信息源管理（RSS、通讯、期刊） │ +│ • 学习计划（短期、中期、长期） │ +│ • 时间分配（阅读、实践、反思） │ +│ │ +│ 第二层：知识处理层 (Processing) │ +│ ────────────────────────────────────────────────────── │ +│ • 笔记系统（捕获、整理、归档） │ +│ • 知识连接（建立概念间的关系） │ +│ • 代码片段库（可复用的代码片段） │ +│ │ +│ 第三层：知识应用层 (Application) │ +│ ────────────────────────────────────────────────────── │ +│ • 项目实践（真实问题应用） │ +│ • 工具箱（积累的AI工具和工作流） │ +│ • 模板库（常用分析模板） │ +│ │ +│ 第四层：知识分享层 (Sharing) │ +│ ────────────────────────────────────────────────────── │ +│ • 写作（博客、论文、文档） │ +│ • 演讲（会议、工作坊） │ +│ • 开源（贡献代码和工具） │ +│ │ +│ 第五层：知识反思层 (Reflection) │ +│ ────────────────────────────────────────────────────── │ +│ • 定期回顾（月度、年度） │ +│ • 知识审计（识别盲区和过时知识） │ +│ • 体系优化（持续改进知识系统） │ +│ │ +└─────────────────────────────────────────────────────────────┘ + +关键特征： +• 开放性：可以不断添加和更新 +• 连接性：知识之间相互关联 +• 可检索性：快速找到所需信息 +• 可维护性：低成本的维护和更新 +``` + +### AI工具箱的构成 + +``` +个人AI工具箱 + +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ 1. 基础工具集 │ +│ ─────────────────────────────────────────────────── │ +│ • 通用AI助手（Claude, ChatGPT） │ +│ • 代码助手（GitHub Copilot） │ +│ • 图像生成（Midjourney, DALL-E） │ +│ │ +│ 2. 专业工具集 │ +│ ─────────────────────────────────────────────────── │ +│ • 空间分析AI（遥感解译、地理NLP） │ +│ • 数据处理AI（数据清洗、转换） │ +│ • 可视化AI（自动图表、地图生成） │ +│ │ +│ 3. 自定义工具集 │ +│ ─────────────────────────────────────────────────── │ +│ • 自定义Skills（Prompt集合） │ +│ • 专用工作流（针对特定任务） │ +│ • 集成系统（API整合） │ +│ │ +│ 4. 辅助资源集 │ +│ ─────────────────────────────────────────────────── │ +│ • Prompt模板库 │ +│ • 示例案例集 │ +│ • 最佳实践文档 │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 文档化策略 + +``` +有效的文档化 + +金字塔模型： + + ┌──────────────┐ + │ 即时笔记 │ ← 捕获想法，最小开销 + └──────┬───────┘ + │ + ┌──────┴───────┐ + │ 项目笔记 │ ← 项目上下文，详细记录 + └──────┬───────┘ + │ + ┌──────┴───────┐ + │ 知识笔记 │ ← 提炼知识，建立连接 + └──────┬───────┘ + │ + ┌──────┴───────┐ + │ 指南文档 │ ← 结构化输出，可分享 + └──────────────┘ + +不同文档的特点： + +即时笔记: +• 快速捕获 +• 原始想法 +• 待整理 + +项目笔记: +• 项目特定 +• 决策记录 +• 问题解决 + +知识笔记: +• 跨项目 +• 概念理解 +• 最佳实践 + +指南文档: +• 面向读者 +• 结构完整 +• 可独立使用 +``` + +--- + +## 设计原理 + +### 知识管理系统实现 + +```python +""" +个人知识管理系统 +""" +from typing import Dict, List, Set, Optional, Any +from dataclasses import dataclass, field +from datetime import datetime +from pathlib import Path +import json +from enum import Enum + +class NoteType(Enum): + """笔记类型""" + QUICK = "quick" # 即时笔记 + PROJECT = "project" # 项目笔记 + KNOWLEDGE = "knowledge" # 知识笔记 + GUIDE = "guide" # 指南文档 + +class Tag(Enum): + """常用标签分类""" + SKILL = "skill" + CONCEPT = "concept" + TOOL = "tool" + WORKFLOW = "workflow" + PROBLEM = "problem" + SOLUTION = "solution" + REFERENCE = "reference" + +@dataclass +class Note: + """知识条目""" + id: str + title: str + content: str + note_type: NoteType + tags: Set[str] = field(default_factory=set) + created: str = "" + modified: str = "" + related_notes: Set[str] = field(default_factory=set) + metadata: Dict = field(default_factory=dict) + + def __post_init__(self): + if not self.created: + self.created = datetime.now().isoformat() + self.modified = self.created + +@dataclass +class CodeSnippet: + """代码片段""" + id: str + name: str + description: str + code: str + language: str + tags: Set[str] = field(default_factory=set) + dependencies: List[str] = field(default_factory=list) + created: str = "" + + def __post_init__(self): + if not self.created: + self.created = datetime.now().isoformat() + +@dataclass +class ToolEntry: + """工具条目""" + name: str + category: str + url: str + description: str + use_cases: List[str] + pros: List[str] + cons: List[str] + last_used: str = "" + proficiency: str = "beginner" # beginner, intermediate, advanced + +class PersonalKnowledgeSystem: + """个人知识系统""" + + def __init__(self, base_path: str): + self.base_path = Path(base_path) + self.notes: Dict[str, Note] = {} + self.snippets: Dict[str, CodeSnippet] = {} + self.tools: Dict[str, ToolEntry] = {} + self.connections: Dict[str, Set[str]] = {} # 知识关联图 + + self._load() + + def add_note(self, title: str, content: str, + note_type: NoteType, tags: List[str] = None) -> Note: + """添加笔记""" + note_id = self._generate_id() + note = Note( + id=note_id, + title=title, + content=content, + note_type=note_type, + tags=set(tags or []) + ) + self.notes[note_id] = note + self._save_note(note) + return note + + def add_code_snippet(self, name: str, code: str, language: str, + description: str = "") -> CodeSnippet: + """添加代码片段""" + snippet_id = self._generate_id() + snippet = CodeSnippet( + id=snippet_id, + name=name, + description=description, + code=code, + language=language + ) + self.snippets[snippet_id] = snippet + self._save_snippet(snippet) + return snippet + + def add_tool(self, name: str, category: str, url: str, + description: str, use_cases: List[str]) -> ToolEntry: + """添加工具""" + tool = ToolEntry( + name=name, + category=category, + url=url, + description=description, + use_cases=use_cases, + pros=[], + cons=[] + ) + self.tools[name] = tool + self._save_tools() + return tool + + def connect_notes(self, note_id1: str, note_id2: str, relation: str = ""): + """连接两个笔记""" + if note_id1 in self.notes and note_id2 in self.notes: + self.notes[note_id1].related_notes.add(note_id2) + self.notes[note_id2].related_notes.add(note_id1) + + # 更新关联图 + if note_id1 not in self.connections: + self.connections[note_id1] = set() + if note_id2 not in self.connections: + self.connections[note_id2] = set() + self.connections[note_id1].add(note_id2) + self.connections[note_id2].add(note_id1) + + self._save_note(self.notes[note_id1]) + self._save_note(self.notes[note_id2]) + + def search(self, query: str) -> List[Note]: + """搜索笔记""" + results = [] + query_lower = query.lower() + + for note in self.notes.values(): + if (query_lower in note.title.lower() or + query_lower in note.content.lower() or + any(query_lower in tag.lower() for tag in note.tags)): + results.append(note) + + return sorted(results, key=lambda n: n.modified, reverse=True) + + def get_related_notes(self, note_id: str) -> List[Note]: + """获取相关笔记""" + if note_id not in self.notes: + return [] + + related_ids = self.notes[note_id].related_notes + return [self.notes[nid] for nid in related_ids if nid in self.notes] + + def get_knowledge_graph(self) -> Dict[str, List[str]]: + """获取知识图谱""" + graph = {} + for note_id, note in self.notes.items(): + graph[note.title] = [ + self.notes[rid].title + for rid in note.related_notes + if rid in self.notes + ] + return graph + + def review_periodic(self, days: int = 30) -> Dict[str, List[Note]]: + """定期回顾：获取需要复习的笔记""" + from datetime import timedelta + + cutoff = datetime.now() - timedelta(days=days) + to_review = { + 'stale': [], # 长时间未更新的笔记 + 'orphan': [], # 没有关联的笔记 + 'valuable': [] # 高价值笔记 + } + + for note in self.notes.values(): + modified = datetime.fromisoformat(note.modified) + + # 识别陈旧笔记 + if modified < cutoff: + to_review['stale'].append(note) + + # 识别孤立笔记 + if not note.related_notes: + to_review['orphan'].append(note) + + # 识别高价值笔记（知识型和指南型） + if note.note_type in [NoteType.KNOWLEDGE, NoteType.GUIDE]: + to_review['valuable'].append(note) + + return to_review + + def export_markdown(self, note_id: str, output_path: str): + """导出为Markdown""" + if note_id not in self.notes: + return + + note = self.notes[note_id] + + md_content = f"""# {note.title} + +**类型**: {note.note_type.value} +**标签**: {', '.join(note.tags)} +**创建**: {note.created} +**修改**: {note.modified} + +--- + +{note.content} +""" + + if note.related_notes: + md_content += "\n\n## 相关笔记\n\n" + for rid in note.related_notes: + if rid in self.notes: + related = self.notes[rid] + md_content += f"- [{related.title}](notes/{rid}.md)\n" + + Path(output_path).write_text(md_content, encoding='utf-8') + + def _generate_id(self) -> str: + """生成唯一ID""" + import uuid + return str(uuid.uuid4())[:8] + + def _save_note(self, note: Note): + """保存笔记""" + note_dir = self.base_path / "notes" + note_dir.mkdir(parents=True, exist_ok=True) + note_file = note_dir / f"{note.id}.json" + note_file.write_text(json.dumps(note.__dict__, default=str), encoding='utf-8') + + def _save_snippet(self, snippet: CodeSnippet): + """保存代码片段""" + snippet_dir = self.base_path / "snippets" + snippet_dir.mkdir(parents=True, exist_ok=True) + snippet_file = snippet_dir / f"{snippet.id}.json" + snippet_file.write_text(json.dumps(snippet.__dict__, default=str), encoding='utf-8') + + def _save_tools(self): + """保存工具列表""" + tools_file = self.base_path / "tools.json" + tools_data = {name: tool.__dict__ for name, tool in self.tools.items()} + tools_file.write_text(json.dumps(tools_data, default=str, ensure_ascii=False), encoding='utf-8') + + def _load(self): + """加载数据""" + # 加载笔记 + note_dir = self.base_path / "notes" + if note_dir.exists(): + for note_file in note_dir.glob("*.json"): + data = json.loads(note_file.read_text(encoding='utf-8')) + data['note_type'] = NoteType(data['note_type']) + data['tags'] = set(data.get('tags', [])) + data['related_notes'] = set(data.get('related_notes', [])) + note = Note(**data) + self.notes[note.id] = note + + # 加载代码片段 + snippet_dir = self.base_path / "snippets" + if snippet_dir.exists(): + for snippet_file in snippet_dir.glob("*.json"): + data = json.loads(snippet_file.read_text(encoding='utf-8')) + data['tags'] = set(data.get('tags', [])) + snippet = CodeSnippet(**data) + self.snippets[snippet.id] = snippet + + # 加载工具 + tools_file = self.base_path / "tools.json" + if tools_file.exists(): + data = json.loads(tools_file.read_text(encoding='utf-8')) + for name, tool_data in data.items(): + self.tools[name] = ToolEntry(**tool_data) + +# === 空间AI工具箱 === + +class SpatialAIToolkit: + """空间AI工具箱""" + + def __init__(self, knowledge_system: PersonalKnowledgeSystem): + self.knowledge = knowledge_system + self.categories = { + 'llm': '大语言模型', + 'vision': '计算机视觉', + 'gis_ai': 'GIS AI工具', + 'remote_sensing': '遥感AI', + 'data': '数据处理', + 'visualization': '可视化' + } + + def register_tool(self, name: str, category: str, url: str, + description: str, use_cases: List[str], + pros: List[str] = None, cons: List[str] = None): + """注册工具""" + tool = self.knowledge.add_tool( + name=name, + category=category, + url=url, + description=description, + use_cases=use_cases + ) + if pros: + tool.pros = pros + if cons: + tool.cons = cons + + # 添加对应的笔记 + self.knowledge.add_note( + title=f"工具: {name}", + content=f""" +## {name} + +**类别**: {self.categories.get(category, category)} +**链接**: {url} + +### 描述 +{description} + +### 适用场景 +{chr(10).join(f'- {uc}' for uc in use_cases)} + +### 优点 +{chr(10).join(f'- {p}' for p in (pros or ['']))} + +### 缺点 +{chr(10).join(f'- {c}' for c in (cons or ['']))} +""", + note_type=NoteType.QUICK, + tags=[Tag.TOOL.value, category] + ) + + return tool + + def add_workflow(self, name: str, description: str, + steps: List[str], tools: List[str], + code_example: str = None): + """添加工作流""" + # 添加工作流笔记 + content = f"""## {name} + +### 描述 +{description} + +### 步骤 +{chr(10).join(f'{i+1}. {step}' for i, step in enumerate(steps))} + +### 所需工具 +{chr(10).join(f'- {t}' for t in tools)} +""" + + if code_example: + content += f"\n### 代码示例\n\n```python\n{code_example}\n```\n" + + note = self.knowledge.add_note( + title=f"工作流: {name}", + content=content, + note_type=NoteType.GUIDE, + tags=[Tag.WORKFLOW.value] + ) + + # 如果有代码，添加代码片段 + if code_example: + self.knowledge.add_code_snippet( + name=name, + description=description, + code=code_example, + language="python" + ) + + return note + +# === 使用示例 === + +def initialize_knowledge_system(base_path: str) -> PersonalKnowledgeSystem: + """初始化知识系统""" + print("=== 初始化个人知识系统 ===\n") + + pks = PersonalKnowledgeSystem(base_path) + toolkit = SpatialAIToolkit(pks) + + # 注册常用工具 + print("注册AI工具...") + + toolkit.register_tool( + name="Claude", + category="llm", + url="https://claude.ai", + description="Anthropic的大语言模型，擅长分析和写作", + use_cases=["代码生成", "文档写作", "问题分析", "代码审查"], + pros=["上下文窗口大", "安全性好", "输出质量高"], + cons=["无法实时联网", "不能运行代码"] + ) + + toolkit.register_tool( + name="ChatGPT", + category="llm", + url="https://chat.openai.com", + description="OpenAI的对话式AI", + use_cases=["对话交流", "知识问答", "创意生成"], + pros=["响应快速", "知识广博", "有插件生态"], + cons=["上下文限制", "可能有幻觉"] + ) + + toolkit.register_tool( + name="GitHub Copilot", + category="llm", + url="https://github.com/features/copilot", + description="AI代码助手", + use_cases=["代码补全", "函数生成", "代码解释"], + pros=["集成在编辑器中", "学习代码模式", "提高效率"], + cons=["需要订阅", "可能生成不安全代码"] + ) + + toolkit.register_tool( + name="QGIS", + category="gis_ai", + url="https://qgis.org", + description="开源GIS软件，支持Python插件", + use_cases=["空间分析", "地图制作", "数据处理"], + pros=["免费开源", "功能强大", "社区活跃"], + cons=["学习曲线陡峭", "大数据处理慢"] + ) + + # 添加工作流 + print("添加工作流...") + + toolkit.add_workflow( + name="遥感影像分类工作流", + description="使用深度学习进行遥感影像分类", + steps=[ + "准备训练数据和标签", + "数据预处理和增强", + "训练深度学习模型", + "模型评估和调优", + "对全图进行预测", + "后处理和结果导出" + ], + tools=["Python", "TensorFlow/PyTorch", "GDAL"], + code_example=""" +# 简化的遥感分类示例 +import rasterio +from sklearn.ensemble import RandomForestClassifier + +# 读取影像 +with rasterio.open('image.tif') as src: + image = src.read() + profile = src.profile + +# 训练分类器 +# clf = RandomForestClassifier() +# clf.fit(train_samples, train_labels) + +# 预测 +# prediction = clf.predict(image_pixels) +""" + ) + + # 添加核心概念笔记 + print("添加核心概念...") + + pks.add_note( + title="空间自相关", + content=""" +## 空间自相关 (Spatial Autocorrelation) + +### 定义 +空间上相近的事物往往比随机分布的事物更相似。 + +### 度量指标 +- **Moran's I**: 全局空间自相关 +- **Geary's C**: 另一种全局指标 +- **LISA**: 局部空间关联指标 + +### 应用 +- 生态格局分析 +- 疾病传播研究 +- 房地产价格分析 + +### Python实现 +```python +from libpysal.weights import Queen +from esda.moran import Moran + +# 创建空间权重 +w = Queen.from_dataframe(gdf) + +# 计算Moran's I +moran = Moran(values, w) +``` +""", + note_type=NoteType.KNOWLEDGE, + tags=[Tag.CONCEPT.value, "statistics", "spatial"] + ) + + print(f"\n知识系统初始化完成！") + print(f"- 笔记: {len(pks.notes)}") + print(f"- 代码片段: {len(pks.snippets)}") + print(f"- 工具: {len(pks.tools)}") + + return pks + +if __name__ == "__main__": + # 初始化知识系统 + pks = initialize_knowledge_system("./knowledge_base") + + # 示例：搜索 + print("\n=== 搜索示例 ===") + results = pks.search("空间") + for note in results[:3]: + print(f"- {note.title} ({note.note_type.value})") + + # 示例：知识图谱 + print("\n=== 知识图谱 ===") + graph = pks.get_knowledge_graph() + for title, related in list(graph.items())[:3]: + if related: + print(f"{title} → {', '.join(related)}") + + # 示例：定期回顾 + print("\n=== 需要回顾的内容 ===") + review = pks.review_periodic(days=30) + print(f"陈旧笔记: {len(review['stale'])}") + print(f"孤立笔记: {len(review['orphan'])}") + print(f"高价值笔记: {len(review['valuable'])}") +``` + +### 社区参与策略 + +``` +社区参与的层次 + + ┌─────────────────┐ + │ 贡献者 │ ← 创建新价值 + │ - 开源项目 │ + │ - 发表论文 │ + │ - 开发工具 │ + └────────┬────────┘ + │ + ┌────────┴────────┐ + │ 分享者 │ ← 分享知识 + │ - 写博客 │ + │ - 做演讲 │ + │ - 回答问题 │ + └────────┬────────┘ + │ + ┌────────┴────────┐ + │ 参与者 │ ← 参与讨论 + │ - 参加会议 │ + │ - 社区讨论 │ + │ - 提问互动 │ + └────────┬────────┘ + │ + ┌────────┴────────┐ + │ 观察者 │ ← 获取信息 + │ - 阅读文档 │ + │ - 关注动态 │ + │ - 收集资源 │ + └─────────────────┘ + +各层次的行动建议： + +观察者: +• 订阅精选博客和通讯 +• 关注领域专家 +• 收集优质资源 + +参与者: +• 加入相关社区（Discord, Slack, 邮件列表） +• 参加线上/线下活动 +• 积极提问和讨论 + +分享者: +• 写学习笔记和博客 +• 在会议上分享经验 +• 回答社区问题 + +贡献者: +• 提交PR到开源项目 +• 发布自己的工具 +• 参与学术研究 +``` + +--- + +## 案例分析 + +### 案例1：建立个人Prompt库 + +```python +""" +个人Prompt管理系统 +""" +from typing import Dict, List +from dataclasses import dataclass + +@dataclass +class PromptTemplate: + """Prompt模板""" + name: str + description: str + category: str + template: str + variables: List[str] # 模板中的变量 + examples: List[Dict] # 使用示例 + tags: List[str] + +class PromptLibrary: + """Prompt库""" + + def __init__(self): + self.prompts: Dict[str, PromptTemplate] = {} + + def add_prompt(self, prompt: PromptTemplate): + """添加Prompt模板""" + self.prompts[prompt.name] = prompt + + def get_prompt(self, name: str, **kwargs) -> str: + """获取填充后的Prompt""" + if name not in self.prompts: + raise ValueError(f"Prompt '{name}' not found") + + template = self.prompts[name] + result = template.template + + # 替换变量 + for var, value in kwargs.items(): + result = result.replace(f"{{{var}}}", str(value)) + + return result + + def list_by_category(self, category: str) -> List[PromptTemplate]: + """按类别列出""" + return [p for p in self.prompts.values() if p.category == category] + +# === 空间AI专用Prompt === + +spatial_prompts = [ + PromptTemplate( + name="spatial_analysis_explain", + description="解释空间分析方法的原理和实现", + category="analysis", + template=""" +你是一个空间分析专家。请解释以下空间分析方法： + +**方法名称**: {method} + +请包括： +1. 方法的核心原理 +2. 数学基础（如果有） +3. 适用场景 +4. 局限性 +5. Python实现示例 +6. 与其他方法的比较 +""", + variables=["method"], + examples=[ + {"method": "Moran's I"}, + {"method": "最小累积阻力模型"} + ], + tags=["解释", "分析"] + ), + + PromptTemplate( + name="code_review_spatial", + description="审查空间分析代码", + category="code", + template=""" +请审查以下空间分析代码： + +```python +{code} +``` + +请检查： +1. 空间数据处理是否正确（坐标系、投影等） +2. 算法选择是否合适 +3. 效率问题 +4. 潜在的错误 +5. 改进建议 +""", + variables=["code"], + examples=[{"code": "# 空间分析代码"}], + tags=["代码审查", "质量"] + ), + + PromptTemplate( + name="gis_choice_advice", + description="获取工具选择建议", + category="advice", + template=""" +我需要为以下项目选择合适的GIS工具： + +**项目描述**: {project_description} +**主要任务**: {tasks} +**数据量**: {data_size} +**团队情况**: {team_info} +**预算**: {budget} + +请比较： +1. QGIS +2. ArcGIS +3. 其他相关工具 + +给出推荐和理由。 +""", + variables=["project_description", "tasks", "data_size", "team_info", "budget"], + examples=[], + tags=["工具选择", "建议"] + ), + + PromptTemplate( + name="spatial_data_cleaning", + description="空间数据清洗指导", + category="data", + template=""" +我需要清洗以下空间数据： + +**数据描述**: {data_description} +**当前问题**: {problems} + +请提供： +1. 诊断步骤 +2. 清洗方法 +3. Python代码示例 +4. 质量检查方法 +""", + variables=["data_description", "problems"], + examples=[], + tags=["数据清洗", "质量"] + ), + + PromptTemplate( + name="landscape_pattern_analysis", + description="景观格局分析", + category="ecology", + template=""" +请帮我进行景观格局分析： + +**研究区域**: {study_area} +**数据**: {data_description} +**研究问题**: {research_question} + +请提供： +1. 合适的景观指数 +2. 分析流程 +3. 代码实现 +4. 结果解释框架 +""", + variables=["study_area", "data_description", "research_question"], + examples=[], + tags=["景观", "生态"] + ), +] + +# 初始化Prompt库 +prompt_lib = PromptLibrary() +for prompt in spatial_prompts: + prompt_lib.add_prompt(prompt) + +# 使用示例 +if __name__ == "__main__": + print("=== 空间AI Prompt库 ===\n") + + # 使用Prompt + prompt = prompt_lib.get_prompt( + "spatial_analysis_explain", + method="最小累积阻力模型" + ) + print("生成的Prompt:") + print(prompt[:200] + "...\n") + + # 列出某类别 + analysis_prompts = prompt_lib.list_by_category("analysis") + print(f"分析类Prompt: {[p.name for p in analysis_prompts]}") +``` + +### 案例2：项目文档化策略 + +```python +""" +项目文档化模板系统 +""" + +class ProjectDocumentation: + """项目文档化""" + + @staticmethod + def create_readme_template() -> str: + """README模板""" + return """# {project_name} + +## 简介 +{description} + +## 背景 +{background} + +## 目标 +{goals} + +## 数据 +- 数据来源: {data_sources} +- 数据格式: {data_formats} +- 数据质量: {data_quality} + +## 方法 +{methods} + +## 结果 +{results} + +## 运行 +```bash +{run_commands} +``` + +## 依赖 +{dependencies} + +## 作者 +{authors} + +## 许可 +{license} +""" + + @staticmethod + def create_methodology_section(method: str, data: Dict) -> str: + """方法学文档""" + return f""" +## {method} + +### 原理 +{data.get('principle', '')} + +### 实现 +```python +{data.get('code', '')} +``` + +### 参数 +{chr(10).join(f"- **{k}**: {v}" for k, v in data.get('parameters', {}).items())} + +### 参考文献 +{chr(10).join(f"- [{ref}]({ref})" for ref in data.get('references', []))} +""" + + @staticmethod + def create_decision_log_template() -> str: + """决策日志模板""" + return """# 项目决策日志 + +## 决策记录 + +| 日期 | 决策 | 理由 | 替代方案 | 状态 | +|------|------|------|----------|------| +| {date} | {decision} | {rationale} | {alternatives} | {status} | + +## 重要决策详情 + +### {decision_title} +**日期**: {date} +**决策者**: {decision_maker} + +**问题**: +{problem} + +**选项**: +1. {option1} +2. {option2} +3. {option3} + +**选择**: {chosen_option} + +**理由**: +{rationale} + +**预期影响**: +{expected_impact} +""" + +# === 自动文档生成 === + +class AutoDocumentation: + """自动文档生成器""" + + @staticmethod + def generate_code_documentation(code: str, ai_assistant) -> Dict: + """使用AI生成代码文档""" + prompt = f""" +请为以下代码生成文档： + +```python +{code} +``` + +返回JSON格式： +{{ + "description": "代码描述", + "parameters": "参数说明", + "returns": "返回值说明", + "example": "使用示例", + "notes": "注意事项" +}} +""" + response = ai_assistant.query(prompt) + return response + + @staticmethod + def generate_workflow_diagram(steps: List[str]) -> str: + """生成工作流图（Mermaid格式）""" + mermaid = ["graph TD"] + for i, step in enumerate(steps): + node_id = f"S{i}" + mermaid.append(f" {node_id}[{step}]") + if i > 0: + mermaid.append(f" S{i-1} --> {node_id}") + return "\n".join(mermaid) +``` + +### 案例3：学习路径规划 + +```python +""" +个人学习路径规划器 +""" +from typing import List, Dict, Optional +from dataclasses import dataclass +from datetime import datetime, timedelta + +@dataclass +class LearningGoal: + """学习目标""" + name: str + description: str + priority: str # high, medium, low + estimated_hours: float + dependencies: List[str] # 前置要求 + resources: List[str] # 学习资源 + status: str = "planned" # planned, in_progress, completed + start_date: Optional[str] = None + completed_date: Optional[str] = None + +class LearningPathPlanner: + """学习路径规划器""" + + def __init__(self): + self.goals: Dict[str, LearningGoal] = {} + self.completed: List[str] = [] + + def add_goal(self, goal: LearningGoal): + """添加学习目标""" + self.goals[goal.name] = goal + + def get_recommended_order(self) -> List[str]: + """获取推荐的学习顺序（拓扑排序）""" + # 简化的拓扑排序 + ordered = [] + remaining = set(self.goals.keys()) + + while remaining: + # 找到没有未满足依赖的目标 + ready = [ + name for name in remaining + if all(dep in self.completed for dep in self.goals[name].dependencies) + ] + + if not ready: + # 循环依赖，按优先级选一个 + ready = [min(remaining, key=lambda n: self.goals[n].priority)] + + # 按优先级排序 + ready.sort(key=lambda n: {'high': 0, 'medium': 1, 'low': 2}[self.goals[n].priority]) + + next_goal = ready[0] + ordered.append(next_goal) + remaining.remove(next_goal) + + return ordered + + def create_schedule(self, weekly_hours: float, start_date: str = None) -> Dict: + """创建学习计划""" + if start_date is None: + start_date = datetime.now() + else: + start_date = datetime.fromisoformat(start_date) + + schedule = {} + current = start_date + ordered = self.get_recommended_order() + + for goal_name in ordered: + goal = self.goals[goal_name] + + # 计算需要周数 + weeks_needed = goal.estimated_hours / weekly_hours + + schedule[goal_name] = { + 'start': current.isoformat(), + 'end': (current + timedelta(weeks=weeks_needed)).isoformat(), + 'weeks': weeks_needed, + 'hours_per_week': weekly_hours + } + + current += timedelta(weeks=weeks_needed) + + return schedule + + def mark_completed(self, goal_name: str): + """标记目标完成""" + if goal_name in self.goals: + self.goals[goal_name].status = "completed" + self.goals[goal_name].completed_date = datetime.now().isoformat() + self.completed.append(goal_name) + +# === 空间AI学习路径 === + +def create_spatial_ai_learning_path() -> LearningPathPlanner: + """创建空间AI学习路径""" + planner = LearningPathPlanner() + + # 基础 + planner.add_goal(LearningGoal( + name="Python基础", + description="掌握Python编程基础", + priority="high", + estimated_hours=40, + dependencies=[], + resources=["Python官方教程", "Automate the Boring Stuff"] + )) + + planner.add_goal(LearningGoal( + name="GIS基础", + description="理解地理信息系统基本概念", + priority="high", + estimated_hours=30, + dependencies=[], + resources=["QGIS官方教程", "Geocomputation with R"] + )) + + # 核心 + planner.add_goal(LearningGoal( + name="空间数据处理", + description="使用Python处理空间数据", + priority="high", + estimated_hours=50, + dependencies=["Python基础", "GIS基础"], + resources=["GeoPandas文档", "PyTutoria"] + )) + + planner.add_goal(LearningGoal( + name="空间统计分析", + description="空间统计方法和实践", + priority="high", + estimated_hours=60, + dependencies=["空间数据处理"], + resources=["PySAL文档", "Spatial Statistics"] + )) + + planner.add_goal(LearningGoal( + name="机器学习基础", + description="机器学习算法和原理", + priority="medium", + estimated_hours=80, + dependencies=["Python基础"], + resources=["Hands-On ML", "Scikit-learn文档"] + )) + + # 进阶 + planner.add_goal(LearningGoal( + name="深度学习", + description="深度学习和神经网络", + priority="medium", + estimated_hours=100, + dependencies=["机器学习基础"], + resources=["Deep Learning", "Fast.ai"] + )) + + planner.add_goal(LearningGoal( + name="遥感AI", + description="遥感影像的深度学习应用", + priority="medium", + estimated_hours=60, + dependencies=["深度学习", "空间数据处理"], + resources=["深度学习与遥感", "torchgeo"] + )) + + planner.add_goal(LearningGoal( + name="LLM应用", + description="大语言模型在空间分析中的应用", + priority="low", + estimated_hours=40, + dependencies=["机器学习基础", "空间数据处理"], + resources=["Claude文档", "LangChain文档"] + )) + + return planner + +if __name__ == "__main__": + print("=== 空间AI学习路径 ===\n") + + planner = create_spatial_ai_learning_path() + + print("推荐学习顺序:") + order = planner.get_recommended_order() + for i, goal_name in enumerate(order, 1): + goal = planner.goals[goal_name] + print(f"{i}. {goal_name} ({goal.estimated_hours}h) - {goal.priority}") + + print("\n学习计划 (每周10小时):") + schedule = planner.create_schedule(weekly_hours=10) + for goal_name, info in schedule.items(): + start = info['start'][:10] + end = info['end'][:10] + print(f"{goal_name}: {start} → {end} ({info['weeks']:.1f}周)") + + total_hours = sum(g.estimated_hours for g in planner.goals.values()) + total_weeks = total_hours / 10 + print(f"\n总计: {total_hours}小时, 约{total_weeks:.1f}周") +``` + +--- + +## 反思与延伸 + +### 思考问题 + +1. **知识审计**：你当前的知识体系有哪些空白？ + +2. **工具评估**：你的AI工具箱中有多少工具是真正常用的？ + +3. **文档习惯**：你目前的文档化习惯有什么问题？ + +4. **社区参与**：你在哪个层次的社区参与？如何提升？ + +### 实践练习 + +1. **建立知识库**：使用文中代码创建你的个人知识系统 + +2. **整理Prompt**：收集并整理你常用的Prompt模板 + +3. **规划学习**：为未来6个月创建详细的学习计划 + +### 延伸阅读 + +- **"Building a Second Brain"** (Tiago Forte) - 个人知识管理 +- **"How to Take Smart Notes"** (Sönke Ahrens) - 卡片笔记法 +- **"Digital Minimalism"** (Cal Newport) - 数字工具的选择 + +--- + +## 关键要点 + +1. **个人知识体系应该是分层的**，从捕获到分享 +2. **AI工具箱需要分类管理**，包括基础、专业、自定义工具 +3. **文档化应该遵循金字塔模型**，不同类型文档有不同目的 +4. **社区参与是多层次的**，从观察到贡献逐步深入 +5. **持续优化是关键**，定期回顾和调整你的知识系统 diff --git a/officefile/supplements/05-reflection/README.md b/officefile/supplements/05-reflection/README.md new file mode 100644 index 0000000..137f1f0 --- /dev/null +++ b/officefile/supplements/05-reflection/README.md @@ -0,0 +1,145 @@ +# 第六部分：反思与展望 + +## 本部分目标 + +培养批判性思维，建立长期视角： +- 认识AI的局限与潜在风险 +- 理解空间决策的伦理维度 +- 建立技术迭代中保持知识更新的方法 +- 展望空间AI的未来方向 +- 构建个人知识体系 + +--- + +## 章节导航 + +| 章节 | 标题 | 核心内容 | +|-----|------|---------| +| 05.1 | [AI的局限与幻觉](./05.1-ai-limitations.md) | 空间AI可能出错的地方、验证方法、专家的不可替代性 | +| 05.2 | [伦理与责任](./05.2-ethics-and-responsibility.md) | 空间决策的伦理维度、可解释性、问责机制 | +| 05.3 | [技术迭代与持久知识](./05.3-technical-iteration.md) | 什么在变、什么不变、如何持续学习 | +| 05.4 | [空间AI的未来方向](./05.4-future-directions.md) | 空间大模型、多模态、具身智能 | +| 05.5 | [个人知识体系](./05.5-personal-knowledge-system.md) | 建立AI工具箱、文档化策略、社区参与 | + +--- + +## 核心理念 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 批判性思维框架 │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ │ │ +│ │ 问题 → 怀疑 → 验证 → 理解 → 反思 │ │ +│ │ ↑ ↓ │ │ +│ │ └────────────────────┘ │ │ +│ │ 迭代改进 │ │ +│ │ │ │ +│ └───────────────────────────────────────────────────┘ │ +│ │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ 三大怀疑对象 │ │ +│ │ │ │ +│ │ 1. 怀疑工具：工具的局限是什么？ │ │ +│ │ 2. 怀疑结果：结果可靠吗？如何验证？ │ │ +│ │ 3. 怀疑自己：我的理解正确吗？有无偏见？ │ │ +│ │ │ │ +│ └───────────────────────────────────────────────────┘ │ +│ │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ 持久学习的原则 │ │ +│ │ │ │ +│ │ • 关注原理，而非命令 │ │ +│ │ • 理解权衡，而非绝对 │ │ +│ │ • 建立网络，而非孤立 │ │ +│ │ • 保持好奇，而非自满 │ │ +│ │ │ │ +│ └───────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 关键问题 + +### 关于AI局限 + +1. AI在哪些空间任务上可能出错？ +2. 如何验证AI的空间分析结果？ +3. 人类专家的哪些能力是AI无法替代的？ + +### 关于伦理责任 + +1. 空间决策如何影响不同群体？ +2. AI系统如何做到可解释？ +3. 出错时责任如何划分？ + +### 关于持续学习 + +1. 哪些知识会过时？哪些会持久？ +2. 如何建立知识更新的机制？ +3. 如何判断新技术值得投入时间学习？ + +### 关于未来方向 + +1. 空间大模型会带来什么变革？ +2. 多模态AI如何改变空间分析？ +3. 具身智能与空间智能的关系是什么？ + +### 关于个人成长 + +1. 如何建立自己的AI工具箱？ +2. 什么样的文档策略最有效？ +3. 如何有效参与技术社区？ + +--- + +## 阅读建议 + +### 思考方式 + +阅读本部分时： +- **批判性思考**：不要盲目接受，提出质疑 +- **联系实际**：将观点与你的经验对比 +- **写下想法**：记录你的思考和问题 +- **讨论交流**：与他人分享观点 + +### 行动导向 + +每章结束后： +1. 总结3个关键点 +2. 提出1个可执行的改进建议 +3. 分享给他人或写成笔记 + +--- + +## 延伸资源 + +### 批判性思维 +- **"Critical Thinking"** (Moore & Parker) - 批判性思维入门 +- **"Thinking, Fast and Slow"** (Kahneman) - 人类思维的局限性 + +### AI伦理 +- **"Weapons of Math Destruction"** (Cathy O'Neil) - 算法的社会影响 +- **"The Alignment Problem"** (Brian Christian) - AI对齐问题 + +### 持续学习 +- **"Make It Stick"** - 学习的科学 +- **"Ultralearning"** (Scott Young) - 高效自学方法 + +--- + +## 结语 + +完成全书阅读后，希望你能： + +1. **建立认知框架**：理解AI背后的原理，而非仅仅使用工具 +2. **培养批判思维**：能质疑、验证、改进AI系统 +3. **保持学习能力**：在技术迭代中持续成长 +4. **承担社会责任**：在空间决策中考虑伦理影响 +5. **构建个人体系**：建立可持久的知识和技能体系 + +> "技术永远在变，但原理长存。工具可能过时，但思维永恒。" diff --git a/officefile/supplements/BOOK.md b/officefile/supplements/BOOK.md new file mode 100644 index 0000000..89547db --- /dev/null +++ b/officefile/supplements/BOOK.md @@ -0,0 +1,198 @@ +# Claude Code for Spatial Intelligence and Autonomous Design +## 空间智能与自主设计：AI原理与实践手册 + +--- + +**定位**：面向空间研究与设计领域的混合读者，以Claude Code为脚手架，帮助理解AI背后的思想、原理和方法，在技术快速迭代的当下建立持久的认知框架。 + +**核心原则**： +- 重原理轻操作——关注不变的设计思想，而非易变的命令语法 +- 理论与实践均衡——50%概念讲解 + 50%实践案例 +- 面向混合读者——兼顾研究者和设计者的不同需求 + +--- + +## 如何使用本书 + +```mermaid +graph TD + A[开始阅读] --> B{你的背景?} + B -->|研究者| C[重点阅读原理部分] + B -->|设计者| D[重点关注实践] + B -->|初学者| E[从头开始] + + C --> F[02 空间智能] + D --> G[03 自主设计] + E --> H[00 导论] + + F --> I[04 综合实践] + G --> I + H --> I + + I --> J[05 反思与展望] +``` + +**建议阅读路径**： + +| 读者类型 | 推荐路径 | 重点章节 | +|---------|---------|---------| +| 空间研究者 | 00→01→02→05 | 01.2, 02.2, 02.4 | +| 设计实践者 | 00→03→04→05 | 03.1, 03.3, 04全部 | +| 技术实现者 | 01→03→04→05 | 01.1, 01.2, 03.2 | +| 教学工作者 | 全部 | 根据教学目标选择 | + +--- + +## 全书目录 + +### [00 导论](./00-introduction/) +建立读者对AI在空间领域应用的宏观认知 + +- [00.1 为什么要读这本书](./00-introduction/00.1-why-this-book.md) +- [00.2 空间智能是什么](./00-introduction/00.2-what-is-spatial-intelligence.md) +- [00.3 自主设计的含义](./00-introduction/00.3-what-is-autonomous-design.md) +- [00.4 Claude Code作为脚手架](./00-introduction/00.4-claude-code-as-scaffold.md) + +**实践案例00**：[搭建你的第一个空间AI助手](./00-introduction/practice/setup-first-assistant/) + +--- + +### [01 基础原理](./01-foundations/) +理解现代AI系统的核心设计原理 + +- [01.1 智能的模块化视角](./01-foundations/01.1-modular-intelligence.md) +- [01.2 状态与状态机](./01-foundations/01.2-state-and-state-machines.md) +- [01.3 概率与不确定性](./01-foundations/01.3-probability-and-uncertainty.md) +- [01.4 反馈与学习](./01-foundations/01.4-feedback-and-learning.md) +- [01.5 人机协同的原理](./01-foundations/01.5-human-ai-collaboration.md) + +**实践案例01**：[用LangGraph构建空间决策工作流](./01-foundations/practice/langgraph-workflow/) + +--- + +### [02 空间智能](./02-spatial-intelligence/) +理解AI如何"理解"和操作空间 + +- [02.1 空间表征](./02-spatial-intelligence/02.1-spatial-representation.md) +- [02.2 空间推理](./02-spatial-intelligence/02.2-spatial-reasoning.md) +- [02.3 多准则决策](./02-spatial-intelligence/02.3-multi-criteria-decision.md) +- [02.4 空间优化](./02-spatial-intelligence/02.4-spatial-optimization.md) +- [02.5 不确定性量化](./02-spatial-intelligence/02.5-uncertainty-quantification.md) + +**实践案例02**：构建生态系统服务评估Skill +**实践案例03**：最小累积阻力(MCR)分析的自动化 + +--- + +### [03 自主设计](./03-autonomous-design/) +理解自主系统的设计模式和工作流编排 + +- [03.1 工作流编排原理](./03-autonomous-design/03.1-workflow-orchestration.md) +- [03.2 Agent设计模式](./03-autonomous-design/03.2-agent-design-patterns.md) +- [03.3 技能组合与复用](./03-autonomous-design/03.3-skill-composition.md) +- [03.4 记忆与上下文](./03-autonomous-design/03.4-memory-and-context.md) +- [03.5 规划与执行](./03-autonomous-design/03.5-planning-and-execution.md) + +**实践案例04**：设计一个自主空间分析Agent +**实践案例05**：实现Human-in-the-Loop审查机制 + +--- + +### [04 综合实践](./04-practice/) +通过完整项目将所学整合 + +- [项目A：生态源地自动识别与优先级排序](./04-practice/projects/project-a-source-identification/) +- [项目B：生态阻力面的多准则构建](./04-practice/projects/project-b-resistance-surface/) +- [项目C：生态网络的自主分析与优化](./04-practice/projects/project-c-network-optimization/) +- [项目D：带HITL的完整工作流设计](./04-practice/projects/project-d-hitl-workflow/) + +--- + +### [05 反思与展望](./05-reflection/) +培养批判性思维，建立长期视角 + +- [05.1 AI的局限与幻觉](./05-reflection/05.1-ai-limitations.md) +- [05.2 伦理与责任](./05-reflection/05.2-ethics-and-responsibility.md) +- [05.3 技术迭代与持久知识](./05-reflection/05.3-technical-iteration.md) +- [05.4 空间AI的未来方向](./05-reflection/05.4-future-directions.md) +- [05.5 个人知识体系](./05-reflection/05.5-personal-knowledge-system.md) + +--- + +## 附录 + +### [参考资料](./references/) +- 空间分析基础理论 +- 人工智能核心教材 +- Claude Code官方文档 +- 相关论文和资源 + +### [可运行代码示例](./examples/) +- Python基础示例 +- Claude Code技能示例 +- LangGraph工作流示例 +- 完整项目模板 + +### [附录1: 学术论文写作工作流](./appendix/appendix1-academic-writing-workflow.md) +**VSCode + Claude Code + Obsidian 打造完整写作流程** + +涵盖内容： +- 文献管理与知识积累（Zotero + Obsidian） +- 大纲构思与结构设计 +- 内容撰写与AI辅助（Claude Code） +- 代码与图表制作（Python + Pandoc） +- 修改润色与投稿准备 + +适用场景： +- 学位论文写作 +- 期刊论文投稿 +- 研究报告撰写 + +--- + +## 核心概念索引 + +| 概念 | 相关章节 | 关键词 | +|-----|---------|--------| +| Agent | 00.3, 01.2, 03.2 | 智能体、自主决策、设计模式 | +| Skill | 00.4, 01.1, 03.3 | 技能、能力封装、组合复用 | +| Hook | 00.4, 03.1 | 钩子、事件驱动、扩展点 | +| HITL | 00.3, 01.5, 03.5 | 人机协同、审查点、信任校准 | +| 状态机 | 01.2, 03.1 | 状态、转换、工作流编排 | +| 不确定性 | 01.3, 02.5 | 概率、置信度、敏感性分析 | +| 空间推理 | 02.2 | 拓扑、距离、图算法 | +| 多准则决策 | 02.3 | 权重、标准化、AHP | + +--- + +## 贡献指南 + +欢迎贡献内容！ + +1. **报告问题**：在Issues中指出错误或改进建议 +2. **提交内容**：Fork后创建分支，提交PR +3. **讨论案例**：分享你的实践经验和案例 + +详见 [CONTRIBUTING.md](./CONTRIBUTING.md) + +--- + +## 许可证 + +CC BY-NC-SA 4.0 - 允许非商业使用和修改，需署名并以相同方式共享 + +--- + +## 致谢 + +本书内容基于： +- ENAgent项目（生态网络分析智能体）的实践经验 +- Claude Code官方文档和社区讨论 +- 空间分析和人工智能领域的经典文献 +- 开源社区的集体智慧 + +--- + +## 更新日志 + +- **v0.1.0** (2025-01) - 初始版本，框架搭建完成 diff --git a/officefile/supplements/CC4SI-README.md b/officefile/supplements/CC4SI-README.md new file mode 100644 index 0000000..3c0a7fc --- /dev/null +++ b/officefile/supplements/CC4SI-README.md @@ -0,0 +1,237 @@ +# Claude Code for Spatial Intelligence and Autonomous Design +## 空间智能与自主设计：AI原理与实践手册 + +**目标定位**：面向空间研究与设计领域的混合读者，以Claude Code为脚手架，帮助理解AI背后的思想、原理和方法，在技术快速迭代的当下建立持久的认知框架。 + +**核心原则**： +- 重原理轻操作——关注不变的设计思想，而非易变的命令语法 +- 理论与实践均衡——50%概念讲解 + 50%实践案例 +- 面向混合读者——兼顾研究者和设计者的不同需求 + +--- + +## 目录结构 + +``` +CC4SI/ +├── README.md # 丛书说明 +├── BOOK.md # 主入口（整合目录） +├── 00-introduction/ # 第一部分：导论 +├── 01-foundations/ # 第二部分：基础原理 +├── 02-spatial-intelligence/ # 第三部分：空间智能 +├── 03-autonomous-design/ # 第四部分：自主设计 +├── 04-practice/ # 第五部分：综合实践 +├── 05-reflection/ # 第六部分：反思与展望 +├── references/ # 参考资料 +└── examples/ # 可运行代码示例 +``` + +--- + +## 内容概览 + +### 第一部分：导论 (00-introduction) + +建立读者对AI在空间领域应用的宏观认知 + +| 章节 | 标题 | 核心内容 | +|-----|------|---------| +| 00.1 | 为什么要读这本书 | AI的"黑箱"问题；空间智能的特殊性；原理学习的持久价值 | +| 00.2 | 空间智能是什么 | 空间认知的层次；从GIS到空间智能；空间推理的本质 | +| 00.3 | 自主设计的含义 | Human-in-the-Loop；自动与自主的区别；设计智能的演进 | +| 00.4 | Claude Code作为脚手架 | 为什么选择Claude Code；Agent、Skill、Hook的概念；工具中性原则 | + +**实践案例00**：搭建你的第一个空间AI助手 + +--- + +### 第二部分：基础原理 (01-foundations) + +理解现代AI系统的核心设计原理，超越具体工具 + +| 章节 | 标题 | 核心内容 | +|-----|------|---------| +| 01.1 | 智能的模块化视角 | 为什么需要模块化；函数式组合思想；技能即能力封装 | +| 01.2 | 状态与状态机 | 状态是什么；为什么状态管理是核心；LangGraph的设计哲学 | +| 01.3 | 概率与不确定性 | 空间分析中的不确定性；AI如何处理未知；置信度的概念 | +| 01.4 | 反馈与学习 | 强化学习直觉；奖励函数设计；探索与利用的权衡 | +| 01.5 | 人机协同的原理 | HITL的理论基础；何时需要人类介入；信任校准 | + +**实践案例01**：用LangGraph构建一个简单的空间决策工作流 + +--- + +### 第三部分：空间智能 (02-spatial-intelligence) + +理解AI如何"理解"和操作空间 + +| 章节 | 标题 | 核心内容 | +|-----|------|---------| +| 02.1 | 空间表征 | 栅格 vs 矢量；图表示；多尺度表征；空间索引原理 | +| 02.2 | 空间推理 | 邻近性分析；连通性；空间关系推理；图算法在空间中的应用 | +| 02.3 | 多准则决策 | 权重的本质；标准化方法；敏感性分析；专家知识编码 | +| 02.4 | 空间优化 | 什么可优化；目标函数设计；约束处理；启发式搜索 | +| 02.5 | 不确定性量化 | 空间不确定性来源；传播分析；可视化；决策稳健性 | + +**实践案例02**：构建生态系统服务评估Skill +**实践案例03**：最小累积阻力(MCR)分析的自动化 + +--- + +### 第四部分：自主设计 (03-autonomous-design) + +理解自主系统的设计模式和工作流编排 + +| 章节 | 标题 | 核心内容 | +|-----|------|---------| +| 03.1 | 工作流编排原理 | DAG（有向无环图）；节点与边；条件分支；错误处理模式 | +| 03.2 | Agent设计模式 | Reflex Agent；Model-based Agent；Goal-based；Utility-based | +| 03.3 | 技能组合与复用 | 技能抽象；接口设计；组合模式；技能发现 | +| 03.4 | 记忆与上下文 | 短期/长期记忆；状态持久化；检索机制；知识库构建 | +| 03.5 | 规划与执行 | 前向/后向搜索；分层规划；执行监控；重规划 | + +**实践案例04**：设计一个自主空间分析Agent +**实践案例05**：实现Human-in-the-Loop审查机制 + +--- + +### 第五部分：综合实践 (04-practice) + +通过完整项目将所学整合 + +| 项目 | 描述 | 涉及章节 | +|-----|------|---------| +| 项目A | 生态源地自动识别与优先级排序 | 02.1, 02.3, 03.1 | +| 项目B | 生态阻力面的多准则构建 | 02.3, 02.5, 03.3 | +| 项目C | 生态网络的自主分析与优化 | 02.2, 02.4, 03.4 | +| 项目D | 带HITL的完整工作流设计 | 全部章节 | + +--- + +### 第六部分：反思与展望 (05-reflection) + +培养批判性思维，建立长期视角 + +| 章节 | 标题 | 核心内容 | +| ---- | --------- | --------------------------- | +| 05.1 | AI的局限与幻觉 | 空间AI可能出错的地方；如何验证；人类专家的不可替代性 | +| 05.2 | 伦理与责任 | 空间决策的伦理维度；可解释性；问责机制 | +| 05.3 | 技术迭代与持久知识 | 什么在变；什么不变；如何持续学习 | +| 05.4 | 空间AI的未来方向 | 空间大模型；多模态；具身智能 | +| 05.5 | 个人知识体系 | 建立自己的AI工具箱；文档化策略；社区参与 | + +--- + +## 如何使用本书 + +### 面向研究者 +如果你是空间研究领域的学者或学生，建议： +1. 先完成00-introduction，建立宏观认知 +2. 重点阅读01-foundations，理解AI设计原理 +3. 根据研究需要选择性深入02和03部分 +4. 通过04-practice的项目验证理解 + +### 面向设计者 +如果你是设计实践者或技术实现者，建议： +1. 快速浏览00-introduction +2. 重点关注03-autonomous-design的工作流编排 +3. 边读边做，每个实践案例都亲手运行 +4. 使用examples目录中的代码作为起点 + +### 面向教学者 +如果你使用本书作为教学材料： +1. 每章节预留2-3课时（理论+实践） +2. 使用"核心问题"引发讨论 +3. 鼓励学生完成"反思与延伸"中的问题 +4. 项目A-D可作为期末综合项目 + +--- + +## 环境准备 + +### 基础要求 +- Python 3.10+ +- Claude Code CLI +- QGIS 3.x（可选，用于空间分析） + +### 安装Claude Code +```bash +npm install -g @anthropic/claude-code +``` + +### Python依赖 +```bash +pip install langgraph langchain-anthropic geopandas rasterio networkx matplotlib +``` + +--- + +## 写作规范 + +### 内容原则 +1. **原理优先**：先讲"为什么"，再讲"怎么做" +2. **中英双语**：专业术语保留英文，解释用中文 +3. **代码示例**：完整可运行，有注释，说明设计意图 +4. **图示辅助**：架构图、流程图、概念图优先于纯文字 +5. **案例真实**：基于ENAgent等实际项目，避免玩具示例 + +### 文件命名规范 +``` +XX-category/ +├── README.md # 章节导读 +├── XX.1-chapter-name.md +├── XX.2-chapter-name.md +└── practice/ # 本章节的实践 + └── practice-name/ +``` + +### Markdown模板 +每章节遵循以下结构： +```markdown +# 章节标题 + +## 核心问题 +（用1-2个问题引导读者思考） + +## 概念讲解 +（原理阐述，配合图示） + +## 设计原理 +（为什么这样设计，权衡是什么） + +## 代码示例 +```python +# 可运行代码，有详细注释 +``` + +## 案例分析 +（真实项目的相关代码解析） + +## 反思与延伸 +（启发思考的问题） + +## 参考资料 +（相关阅读） +``` + +--- + +## 贡献指南 + +本书是开源项目，欢迎贡献： + +1. **报告问题**：在Issues中指出错误或改进建议 +2. **提交内容**：Fork后创建分支，提交PR +3. **讨论案例**：分享你的实践经验和案例 + +--- + +## 许可证 + +CC BY-NC-SA 4.0 - 允许非商业使用和修改，需署名并以相同方式共享 + +--- + +## 版本历史 + +- v0.1.0 (2025-01) - 初始版本，框架搭建完成 diff --git a/officefile/supplements/CONTRIBUTING.md b/officefile/supplements/CONTRIBUTING.md new file mode 100644 index 0000000..065175c --- /dev/null +++ b/officefile/supplements/CONTRIBUTING.md @@ -0,0 +1,327 @@ +# 贡献指南 + +感谢你有兴趣为《Claude Code for Spatial Intelligence and Autonomous Design》做出贡献！ + +--- + +## 如何贡献 + +### 报告问题 + +如果你发现了书中的错误或有改进建议： + +1. 在GitHub Issues中搜索是否已有相关问题 +2. 如果没有，创建新Issue并包含： + - 清晰的标题 + - 错误位置（章节、文件） + - 问题描述 + - 建议的改进方式 + - 相关标签（bug, enhancement, content等） + +### 提交内容 + +#### 内容贡献类型 + +1. **修正错误** + - 事实错误 + - 代码错误 + - 排版问题 + +2. **新增内容** + - 新的示例代码 + - 新的案例分析 + - 新的实践项目 + +3. **改进现有内容** + - 更好的解释 + - 更清晰的代码 + - 更好的组织 + +4. **翻译** + - 英文内容的中译 + - 中文内容的英译 + +#### 提交流程 + +```bash +# 1. Fork仓库 +# 点击GitHub上的Fork按钮 + +# 2. 克隆你的fork +git clone https://github.com/your-username/CC4SI.git +cd CC4SI + +# 3. 创建分支 +git checkout -b feature/your-feature-name + +# 4. 做出修改 +# 编辑文件... + +# 5. 提交修改 +git add . +git commit -m "描述你的修改" + +# 6. 推送到你的fork +git push origin feature/your-feature-name + +# 7. 创建Pull Request +# 在GitHub上创建PR +``` + +#### 提交信息规范 + +``` +(): + + + +