diff --git a/.claude/skills/gitnexus/gitnexus-cli/SKILL.md b/.claude/skills/gitnexus/gitnexus-cli/SKILL.md new file mode 100644 index 000000000..c9e0af341 --- /dev/null +++ b/.claude/skills/gitnexus/gitnexus-cli/SKILL.md @@ -0,0 +1,82 @@ +--- +name: gitnexus-cli +description: "Use when the user needs to run GitNexus CLI commands like analyze/index a repo, check status, clean the index, generate a wiki, or list indexed repos. Examples: \"Index this repo\", \"Reanalyze the codebase\", \"Generate a wiki\"" +--- + +# GitNexus CLI Commands + +All commands work via `npx` — no global install required. + +## Commands + +### analyze — Build or refresh the index + +```bash +npx gitnexus analyze +``` + +Run from the project root. This parses all source files, builds the knowledge graph, writes it to `.gitnexus/`, and generates CLAUDE.md / AGENTS.md context files. + +| Flag | Effect | +| -------------- | ---------------------------------------------------------------- | +| `--force` | Force full re-index even if up to date | +| `--embeddings` | Enable embedding generation for semantic search (off by default) | + +**When to run:** First time in a project, after major code changes, or when `gitnexus://repo/{name}/context` reports the index is stale. In Claude Code, a PostToolUse hook runs `analyze` automatically after `git commit` and `git merge`, preserving embeddings if previously generated. + +### status — Check index freshness + +```bash +npx gitnexus status +``` + +Shows whether the current repo has a GitNexus index, when it was last updated, and symbol/relationship counts. Use this to check if re-indexing is needed. + +### clean — Delete the index + +```bash +npx gitnexus clean +``` + +Deletes the `.gitnexus/` directory and unregisters the repo from the global registry. Use before re-indexing if the index is corrupt or after removing GitNexus from a project. + +| Flag | Effect | +| --------- | ------------------------------------------------- | +| `--force` | Skip confirmation prompt | +| `--all` | Clean all indexed repos, not just the current one | + +### wiki — Generate documentation from the graph + +```bash +npx gitnexus wiki +``` + +Generates repository documentation from the knowledge graph using an LLM. Requires an API key (saved to `~/.gitnexus/config.json` on first use). + +| Flag | Effect | +| ------------------- | ----------------------------------------- | +| `--force` | Force full regeneration | +| `--model ` | LLM model (default: minimax/minimax-m2.5) | +| `--base-url ` | LLM API base URL | +| `--api-key ` | LLM API key | +| `--concurrency ` | Parallel LLM calls (default: 3) | +| `--gist` | Publish wiki as a public GitHub Gist | + +### list — Show all indexed repos + +```bash +npx gitnexus list +``` + +Lists all repositories registered in `~/.gitnexus/registry.json`. The MCP `list_repos` tool provides the same information. + +## After Indexing + +1. **Read `gitnexus://repo/{name}/context`** to verify the index loaded +2. Use the other GitNexus skills (`exploring`, `debugging`, `impact-analysis`, `refactoring`) for your task + +## Troubleshooting + +- **"Not inside a git repository"**: Run from a directory inside a git repo +- **Index is stale after re-analyzing**: Restart Claude Code to reload the MCP server +- **Embeddings slow**: Omit `--embeddings` (it's off by default) or set `OPENAI_API_KEY` for faster API-based embedding diff --git a/.claude/skills/gitnexus/gitnexus-debugging/SKILL.md b/.claude/skills/gitnexus/gitnexus-debugging/SKILL.md new file mode 100644 index 000000000..9510b97ac --- /dev/null +++ b/.claude/skills/gitnexus/gitnexus-debugging/SKILL.md @@ -0,0 +1,89 @@ +--- +name: gitnexus-debugging +description: "Use when the user is debugging a bug, tracing an error, or asking why something fails. Examples: \"Why is X failing?\", \"Where does this error come from?\", \"Trace this bug\"" +--- + +# Debugging with GitNexus + +## When to Use + +- "Why is this function failing?" +- "Trace where this error comes from" +- "Who calls this method?" +- "This endpoint returns 500" +- Investigating bugs, errors, or unexpected behavior + +## Workflow + +``` +1. gitnexus_query({query: ""}) → Find related execution flows +2. gitnexus_context({name: ""}) → See callers/callees/processes +3. READ gitnexus://repo/{name}/process/{name} → Trace execution flow +4. gitnexus_cypher({query: "MATCH path..."}) → Custom traces if needed +``` + +> If "Index is stale" → run `npx gitnexus analyze` in terminal. + +## Checklist + +``` +- [ ] Understand the symptom (error message, unexpected behavior) +- [ ] gitnexus_query for error text or related code +- [ ] Identify the suspect function from returned processes +- [ ] gitnexus_context to see callers and callees +- [ ] Trace execution flow via process resource if applicable +- [ ] gitnexus_cypher for custom call chain traces if needed +- [ ] Read source files to confirm root cause +``` + +## Debugging Patterns + +| Symptom | GitNexus Approach | +| -------------------- | ---------------------------------------------------------- | +| Error message | `gitnexus_query` for error text → `context` on throw sites | +| Wrong return value | `context` on the function → trace callees for data flow | +| Intermittent failure | `context` → look for external calls, async deps | +| Performance issue | `context` → find symbols with many callers (hot paths) | +| Recent regression | `detect_changes` to see what your changes affect | + +## Tools + +**gitnexus_query** — find code related to error: + +``` +gitnexus_query({query: "payment validation error"}) +→ Processes: CheckoutFlow, ErrorHandling +→ Symbols: validatePayment, handlePaymentError, PaymentException +``` + +**gitnexus_context** — full context for a suspect: + +``` +gitnexus_context({name: "validatePayment"}) +→ Incoming calls: processCheckout, webhookHandler +→ Outgoing calls: verifyCard, fetchRates (external API!) +→ Processes: CheckoutFlow (step 3/7) +``` + +**gitnexus_cypher** — custom call chain traces: + +```cypher +MATCH path = (a)-[:CodeRelation {type: 'CALLS'}*1..2]->(b:Function {name: "validatePayment"}) +RETURN [n IN nodes(path) | n.name] AS chain +``` + +## Example: "Payment endpoint returns 500 intermittently" + +``` +1. gitnexus_query({query: "payment error handling"}) + → Processes: CheckoutFlow, ErrorHandling + → Symbols: validatePayment, handlePaymentError + +2. gitnexus_context({name: "validatePayment"}) + → Outgoing calls: verifyCard, fetchRates (external API!) + +3. READ gitnexus://repo/my-app/process/CheckoutFlow + → Step 3: validatePayment → calls fetchRates (external) + +4. Root cause: fetchRates calls external API without proper timeout +``` diff --git a/.claude/skills/gitnexus/gitnexus-exploring/SKILL.md b/.claude/skills/gitnexus/gitnexus-exploring/SKILL.md new file mode 100644 index 000000000..927a4e4b6 --- /dev/null +++ b/.claude/skills/gitnexus/gitnexus-exploring/SKILL.md @@ -0,0 +1,78 @@ +--- +name: gitnexus-exploring +description: "Use when the user asks how code works, wants to understand architecture, trace execution flows, or explore unfamiliar parts of the codebase. Examples: \"How does X work?\", \"What calls this function?\", \"Show me the auth flow\"" +--- + +# Exploring Codebases with GitNexus + +## When to Use + +- "How does authentication work?" +- "What's the project structure?" +- "Show me the main components" +- "Where is the database logic?" +- Understanding code you haven't seen before + +## Workflow + +``` +1. READ gitnexus://repos → Discover indexed repos +2. READ gitnexus://repo/{name}/context → Codebase overview, check staleness +3. gitnexus_query({query: ""}) → Find related execution flows +4. gitnexus_context({name: ""}) → Deep dive on specific symbol +5. READ gitnexus://repo/{name}/process/{name} → Trace full execution flow +``` + +> If step 2 says "Index is stale" → run `npx gitnexus analyze` in terminal. + +## Checklist + +``` +- [ ] READ gitnexus://repo/{name}/context +- [ ] gitnexus_query for the concept you want to understand +- [ ] Review returned processes (execution flows) +- [ ] gitnexus_context on key symbols for callers/callees +- [ ] READ process resource for full execution traces +- [ ] Read source files for implementation details +``` + +## Resources + +| Resource | What you get | +| --------------------------------------- | ------------------------------------------------------- | +| `gitnexus://repo/{name}/context` | Stats, staleness warning (~150 tokens) | +| `gitnexus://repo/{name}/clusters` | All functional areas with cohesion scores (~300 tokens) | +| `gitnexus://repo/{name}/cluster/{name}` | Area members with file paths (~500 tokens) | +| `gitnexus://repo/{name}/process/{name}` | Step-by-step execution trace (~200 tokens) | + +## Tools + +**gitnexus_query** — find execution flows related to a concept: + +``` +gitnexus_query({query: "payment processing"}) +→ Processes: CheckoutFlow, RefundFlow, WebhookHandler +→ Symbols grouped by flow with file locations +``` + +**gitnexus_context** — 360-degree view of a symbol: + +``` +gitnexus_context({name: "validateUser"}) +→ Incoming calls: loginHandler, apiMiddleware +→ Outgoing calls: checkToken, getUserById +→ Processes: LoginFlow (step 2/5), TokenRefresh (step 1/3) +``` + +## Example: "How does payment processing work?" + +``` +1. READ gitnexus://repo/my-app/context → 918 symbols, 45 processes +2. gitnexus_query({query: "payment processing"}) + → CheckoutFlow: processPayment → validateCard → chargeStripe + → RefundFlow: initiateRefund → calculateRefund → processRefund +3. gitnexus_context({name: "processPayment"}) + → Incoming: checkoutHandler, webhookHandler + → Outgoing: validateCard, chargeStripe, saveTransaction +4. Read src/payments/processor.ts for implementation details +``` diff --git a/.claude/skills/gitnexus/gitnexus-guide/SKILL.md b/.claude/skills/gitnexus/gitnexus-guide/SKILL.md new file mode 100644 index 000000000..937ac73d1 --- /dev/null +++ b/.claude/skills/gitnexus/gitnexus-guide/SKILL.md @@ -0,0 +1,64 @@ +--- +name: gitnexus-guide +description: "Use when the user asks about GitNexus itself — available tools, how to query the knowledge graph, MCP resources, graph schema, or workflow reference. Examples: \"What GitNexus tools are available?\", \"How do I use GitNexus?\"" +--- + +# GitNexus Guide + +Quick reference for all GitNexus MCP tools, resources, and the knowledge graph schema. + +## Always Start Here + +For any task involving code understanding, debugging, impact analysis, or refactoring: + +1. **Read `gitnexus://repo/{name}/context`** — codebase overview + check index freshness +2. **Match your task to a skill below** and **read that skill file** +3. **Follow the skill's workflow and checklist** + +> If step 1 warns the index is stale, run `npx gitnexus analyze` in the terminal first. + +## Skills + +| Task | Skill to read | +| -------------------------------------------- | ------------------- | +| Understand architecture / "How does X work?" | `gitnexus-exploring` | +| Blast radius / "What breaks if I change X?" | `gitnexus-impact-analysis` | +| Trace bugs / "Why is X failing?" | `gitnexus-debugging` | +| Rename / extract / split / refactor | `gitnexus-refactoring` | +| Tools, resources, schema reference | `gitnexus-guide` (this file) | +| Index, status, clean, wiki CLI commands | `gitnexus-cli` | + +## Tools Reference + +| Tool | What it gives you | +| ---------------- | ------------------------------------------------------------------------ | +| `query` | Process-grouped code intelligence — execution flows related to a concept | +| `context` | 360-degree symbol view — categorized refs, processes it participates in | +| `impact` | Symbol blast radius — what breaks at depth 1/2/3 with confidence | +| `detect_changes` | Git-diff impact — what do your current changes affect | +| `rename` | Multi-file coordinated rename with confidence-tagged edits | +| `cypher` | Raw graph queries (read `gitnexus://repo/{name}/schema` first) | +| `list_repos` | Discover indexed repos | + +## Resources Reference + +Lightweight reads (~100-500 tokens) for navigation: + +| Resource | Content | +| ---------------------------------------------- | ----------------------------------------- | +| `gitnexus://repo/{name}/context` | Stats, staleness check | +| `gitnexus://repo/{name}/clusters` | All functional areas with cohesion scores | +| `gitnexus://repo/{name}/cluster/{clusterName}` | Area members | +| `gitnexus://repo/{name}/processes` | All execution flows | +| `gitnexus://repo/{name}/process/{processName}` | Step-by-step trace | +| `gitnexus://repo/{name}/schema` | Graph schema for Cypher | + +## Graph Schema + +**Nodes:** File, Function, Class, Interface, Method, Community, Process +**Edges (via CodeRelation.type):** CALLS, IMPORTS, EXTENDS, IMPLEMENTS, DEFINES, MEMBER_OF, STEP_IN_PROCESS + +```cypher +MATCH (caller)-[:CodeRelation {type: 'CALLS'}]->(f:Function {name: "myFunc"}) +RETURN caller.name, caller.filePath +``` diff --git a/.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md b/.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md new file mode 100644 index 000000000..e19af280c --- /dev/null +++ b/.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md @@ -0,0 +1,97 @@ +--- +name: gitnexus-impact-analysis +description: "Use when the user wants to know what will break if they change something, or needs safety analysis before editing code. Examples: \"Is it safe to change X?\", \"What depends on this?\", \"What will break?\"" +--- + +# Impact Analysis with GitNexus + +## When to Use + +- "Is it safe to change this function?" +- "What will break if I modify X?" +- "Show me the blast radius" +- "Who uses this code?" +- Before making non-trivial code changes +- Before committing — to understand what your changes affect + +## Workflow + +``` +1. gitnexus_impact({target: "X", direction: "upstream"}) → What depends on this +2. READ gitnexus://repo/{name}/processes → Check affected execution flows +3. gitnexus_detect_changes() → Map current git changes to affected flows +4. Assess risk and report to user +``` + +> If "Index is stale" → run `npx gitnexus analyze` in terminal. + +## Checklist + +``` +- [ ] gitnexus_impact({target, direction: "upstream"}) to find dependents +- [ ] Review d=1 items first (these WILL BREAK) +- [ ] Check high-confidence (>0.8) dependencies +- [ ] READ processes to check affected execution flows +- [ ] gitnexus_detect_changes() for pre-commit check +- [ ] Assess risk level and report to user +``` + +## Understanding Output + +| Depth | Risk Level | Meaning | +| ----- | ---------------- | ------------------------ | +| d=1 | **WILL BREAK** | Direct callers/importers | +| d=2 | LIKELY AFFECTED | Indirect dependencies | +| d=3 | MAY NEED TESTING | Transitive effects | + +## Risk Assessment + +| Affected | Risk | +| ------------------------------ | -------- | +| <5 symbols, few processes | LOW | +| 5-15 symbols, 2-5 processes | MEDIUM | +| >15 symbols or many processes | HIGH | +| Critical path (auth, payments) | CRITICAL | + +## Tools + +**gitnexus_impact** — the primary tool for symbol blast radius: + +``` +gitnexus_impact({ + target: "validateUser", + direction: "upstream", + minConfidence: 0.8, + maxDepth: 3 +}) + +→ d=1 (WILL BREAK): + - loginHandler (src/auth/login.ts:42) [CALLS, 100%] + - apiMiddleware (src/api/middleware.ts:15) [CALLS, 100%] + +→ d=2 (LIKELY AFFECTED): + - authRouter (src/routes/auth.ts:22) [CALLS, 95%] +``` + +**gitnexus_detect_changes** — git-diff based impact analysis: + +``` +gitnexus_detect_changes({scope: "staged"}) + +→ Changed: 5 symbols in 3 files +→ Affected: LoginFlow, TokenRefresh, APIMiddlewarePipeline +→ Risk: MEDIUM +``` + +## Example: "What breaks if I change validateUser?" + +``` +1. gitnexus_impact({target: "validateUser", direction: "upstream"}) + → d=1: loginHandler, apiMiddleware (WILL BREAK) + → d=2: authRouter, sessionManager (LIKELY AFFECTED) + +2. READ gitnexus://repo/my-app/processes + → LoginFlow and TokenRefresh touch validateUser + +3. Risk: 2 direct callers, 2 processes = MEDIUM +``` diff --git a/.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md b/.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md new file mode 100644 index 000000000..f48cc01bd --- /dev/null +++ b/.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md @@ -0,0 +1,121 @@ +--- +name: gitnexus-refactoring +description: "Use when the user wants to rename, extract, split, move, or restructure code safely. Examples: \"Rename this function\", \"Extract this into a module\", \"Refactor this class\", \"Move this to a separate file\"" +--- + +# Refactoring with GitNexus + +## When to Use + +- "Rename this function safely" +- "Extract this into a module" +- "Split this service" +- "Move this to a new file" +- Any task involving renaming, extracting, splitting, or restructuring code + +## Workflow + +``` +1. gitnexus_impact({target: "X", direction: "upstream"}) → Map all dependents +2. gitnexus_query({query: "X"}) → Find execution flows involving X +3. gitnexus_context({name: "X"}) → See all incoming/outgoing refs +4. Plan update order: interfaces → implementations → callers → tests +``` + +> If "Index is stale" → run `npx gitnexus analyze` in terminal. + +## Checklists + +### Rename Symbol + +``` +- [ ] gitnexus_rename({symbol_name: "oldName", new_name: "newName", dry_run: true}) — preview all edits +- [ ] Review graph edits (high confidence) and ast_search edits (review carefully) +- [ ] If satisfied: gitnexus_rename({..., dry_run: false}) — apply edits +- [ ] gitnexus_detect_changes() — verify only expected files changed +- [ ] Run tests for affected processes +``` + +### Extract Module + +``` +- [ ] gitnexus_context({name: target}) — see all incoming/outgoing refs +- [ ] gitnexus_impact({target, direction: "upstream"}) — find all external callers +- [ ] Define new module interface +- [ ] Extract code, update imports +- [ ] gitnexus_detect_changes() — verify affected scope +- [ ] Run tests for affected processes +``` + +### Split Function/Service + +``` +- [ ] gitnexus_context({name: target}) — understand all callees +- [ ] Group callees by responsibility +- [ ] gitnexus_impact({target, direction: "upstream"}) — map callers to update +- [ ] Create new functions/services +- [ ] Update callers +- [ ] gitnexus_detect_changes() — verify affected scope +- [ ] Run tests for affected processes +``` + +## Tools + +**gitnexus_rename** — automated multi-file rename: + +``` +gitnexus_rename({symbol_name: "validateUser", new_name: "authenticateUser", dry_run: true}) +→ 12 edits across 8 files +→ 10 graph edits (high confidence), 2 ast_search edits (review) +→ Changes: [{file_path, edits: [{line, old_text, new_text, confidence}]}] +``` + +**gitnexus_impact** — map all dependents first: + +``` +gitnexus_impact({target: "validateUser", direction: "upstream"}) +→ d=1: loginHandler, apiMiddleware, testUtils +→ Affected Processes: LoginFlow, TokenRefresh +``` + +**gitnexus_detect_changes** — verify your changes after refactoring: + +``` +gitnexus_detect_changes({scope: "all"}) +→ Changed: 8 files, 12 symbols +→ Affected processes: LoginFlow, TokenRefresh +→ Risk: MEDIUM +``` + +**gitnexus_cypher** — custom reference queries: + +```cypher +MATCH (caller)-[:CodeRelation {type: 'CALLS'}]->(f:Function {name: "validateUser"}) +RETURN caller.name, caller.filePath ORDER BY caller.filePath +``` + +## Risk Rules + +| Risk Factor | Mitigation | +| ------------------- | ----------------------------------------- | +| Many callers (>5) | Use gitnexus_rename for automated updates | +| Cross-area refs | Use detect_changes after to verify scope | +| String/dynamic refs | gitnexus_query to find them | +| External/public API | Version and deprecate properly | + +## Example: Rename `validateUser` to `authenticateUser` + +``` +1. gitnexus_rename({symbol_name: "validateUser", new_name: "authenticateUser", dry_run: true}) + → 12 edits: 10 graph (safe), 2 ast_search (review) + → Files: validator.ts, login.ts, middleware.ts, config.json... + +2. Review ast_search edits (config.json: dynamic reference!) + +3. gitnexus_rename({symbol_name: "validateUser", new_name: "authenticateUser", dry_run: false}) + → Applied 12 edits across 8 files + +4. gitnexus_detect_changes({scope: "all"}) + → Affected: LoginFlow, TokenRefresh + → Risk: MEDIUM — run tests for these flows +``` diff --git a/.env.vps b/.env.vps new file mode 100644 index 000000000..6cb7510dc --- /dev/null +++ b/.env.vps @@ -0,0 +1,8 @@ +# VPS Docker Compose Credentials +VPS_HOST=100.64.0.3 +VPS_USER=root +VPS_PASSWORD=Huanld6248@@ + +# Đường dẫn tới SSH Key (nếu chạy tool/script tự động sau này) +# Nếu bạn chưa có SSH key riêng, có thể trỏ về ~/.ssh/id_rsa +VPS_SSH_KEY_PATH=~/.ssh/id_rsa diff --git a/.gitignore b/.gitignore index 83ee00754..ff00d0f03 100644 --- a/.gitignore +++ b/.gitignore @@ -88,4 +88,12 @@ packages/*/.tshy/ .vite-node .nuxt -.output \ No newline at end of file +.output +.gitnexus + +scratch/IFC-toolkit/ +scratch/engine_web-ifc/ +backend.log +packages/server/backend_crash.log +packages/server/server_log*.txt + diff --git a/.gitnexusignore b/.gitnexusignore new file mode 100644 index 000000000..d51f43e63 --- /dev/null +++ b/.gitnexusignore @@ -0,0 +1,17 @@ + +# Auto-added by MCP Workspace Manager +[Bb]in/ +[Oo]bj/ +[Dd]ebug/ +[Rr]elease/ +node_modules/ +packages/ +.vs/ +*.user +*.suo +Thumbs.db +.DS_Store +*.rar +*.zip +*.7z +.gitnexus diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..03cff771a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,101 @@ + +# GitNexus — Code Intelligence + +This project is indexed by GitNexus as **speckle-server** (175 symbols, 160 relationships, 0 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely. + +> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first. + +## Always Do + +- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user. +- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows. +- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits. +- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance. +- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`. + +## When Debugging + +1. `gitnexus_query({query: ""})` — find execution flows related to the issue +2. `gitnexus_context({name: ""})` — see all callers, callees, and process participation +3. `READ gitnexus://repo/speckle-server/process/{processName}` — trace the full execution flow step by step +4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed + +## When Refactoring + +- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`. +- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code. +- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed. + +## Never Do + +- NEVER edit a function, class, or method without first running `gitnexus_impact` on it. +- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis. +- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph. +- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope. + +## Tools Quick Reference + +| Tool | When to use | Command | +|------|-------------|---------| +| `query` | Find code by concept | `gitnexus_query({query: "auth validation"})` | +| `context` | 360-degree view of one symbol | `gitnexus_context({name: "validateUser"})` | +| `impact` | Blast radius before editing | `gitnexus_impact({target: "X", direction: "upstream"})` | +| `detect_changes` | Pre-commit scope check | `gitnexus_detect_changes({scope: "staged"})` | +| `rename` | Safe multi-file rename | `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` | +| `cypher` | Custom graph queries | `gitnexus_cypher({query: "MATCH ..."})` | + +## Impact Risk Levels + +| Depth | Meaning | Action | +|-------|---------|--------| +| d=1 | WILL BREAK — direct callers/importers | MUST update these | +| d=2 | LIKELY AFFECTED — indirect deps | Should test | +| d=3 | MAY NEED TESTING — transitive | Test if critical path | + +## Resources + +| Resource | Use for | +|----------|---------| +| `gitnexus://repo/speckle-server/context` | Codebase overview, check index freshness | +| `gitnexus://repo/speckle-server/clusters` | All functional areas | +| `gitnexus://repo/speckle-server/processes` | All execution flows | +| `gitnexus://repo/speckle-server/process/{name}` | Step-by-step execution trace | + +## Self-Check Before Finishing + +Before completing any code modification task, verify: +1. `gitnexus_impact` was run for all modified symbols +2. No HIGH/CRITICAL risk warnings were ignored +3. `gitnexus_detect_changes()` confirms changes match expected scope +4. All d=1 (WILL BREAK) dependents were updated + +## Keeping the Index Fresh + +After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it: + +```bash +npx gitnexus analyze +``` + +If the index previously included embeddings, preserve them by adding `--embeddings`: + +```bash +npx gitnexus analyze --embeddings +``` + +To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.** + +> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`. + +## CLI + +| Task | Read this skill file | +|------|---------------------| +| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` | +| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` | +| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` | +| Rename / extract / split / refactor | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md` | +| Tools, resources, schema reference | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md` | +| Index, status, clean, wiki CLI commands | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` | + + \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..03cff771a --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,101 @@ + +# GitNexus — Code Intelligence + +This project is indexed by GitNexus as **speckle-server** (175 symbols, 160 relationships, 0 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely. + +> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first. + +## Always Do + +- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user. +- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows. +- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits. +- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance. +- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`. + +## When Debugging + +1. `gitnexus_query({query: ""})` — find execution flows related to the issue +2. `gitnexus_context({name: ""})` — see all callers, callees, and process participation +3. `READ gitnexus://repo/speckle-server/process/{processName}` — trace the full execution flow step by step +4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed + +## When Refactoring + +- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`. +- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code. +- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed. + +## Never Do + +- NEVER edit a function, class, or method without first running `gitnexus_impact` on it. +- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis. +- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph. +- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope. + +## Tools Quick Reference + +| Tool | When to use | Command | +|------|-------------|---------| +| `query` | Find code by concept | `gitnexus_query({query: "auth validation"})` | +| `context` | 360-degree view of one symbol | `gitnexus_context({name: "validateUser"})` | +| `impact` | Blast radius before editing | `gitnexus_impact({target: "X", direction: "upstream"})` | +| `detect_changes` | Pre-commit scope check | `gitnexus_detect_changes({scope: "staged"})` | +| `rename` | Safe multi-file rename | `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` | +| `cypher` | Custom graph queries | `gitnexus_cypher({query: "MATCH ..."})` | + +## Impact Risk Levels + +| Depth | Meaning | Action | +|-------|---------|--------| +| d=1 | WILL BREAK — direct callers/importers | MUST update these | +| d=2 | LIKELY AFFECTED — indirect deps | Should test | +| d=3 | MAY NEED TESTING — transitive | Test if critical path | + +## Resources + +| Resource | Use for | +|----------|---------| +| `gitnexus://repo/speckle-server/context` | Codebase overview, check index freshness | +| `gitnexus://repo/speckle-server/clusters` | All functional areas | +| `gitnexus://repo/speckle-server/processes` | All execution flows | +| `gitnexus://repo/speckle-server/process/{name}` | Step-by-step execution trace | + +## Self-Check Before Finishing + +Before completing any code modification task, verify: +1. `gitnexus_impact` was run for all modified symbols +2. No HIGH/CRITICAL risk warnings were ignored +3. `gitnexus_detect_changes()` confirms changes match expected scope +4. All d=1 (WILL BREAK) dependents were updated + +## Keeping the Index Fresh + +After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it: + +```bash +npx gitnexus analyze +``` + +If the index previously included embeddings, preserve them by adding `--embeddings`: + +```bash +npx gitnexus analyze --embeddings +``` + +To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.** + +> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`. + +## CLI + +| Task | Read this skill file | +|------|---------------------| +| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` | +| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` | +| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` | +| Rename / extract / split / refactor | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md` | +| Tools, resources, schema reference | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md` | +| Index, status, clean, wiki CLI commands | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` | + + \ No newline at end of file diff --git a/backend.log b/backend.log new file mode 100644 index 000000000..e7b137470 Binary files /dev/null and b/backend.log differ diff --git a/build_web_ifc.bat b/build_web_ifc.bat new file mode 100644 index 000000000..c5ea6e039 Binary files /dev/null and b/build_web_ifc.bat differ diff --git a/check_ufw.bat b/check_ufw.bat new file mode 100644 index 000000000..8ab8cd2b9 --- /dev/null +++ b/check_ufw.bat @@ -0,0 +1,9 @@ +@echo off +echo === UFW Status === +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "ufw status verbose" +echo. +echo === Docker Compose Services (VPS) === +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "cd /root && docker compose -f docker-compose-vps.yml ps" +echo. +echo === All Listening Ports === +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "ss -tlnp | grep -E '9001|9000|8090|6379|1080'" diff --git a/check_vps.bat b/check_vps.bat new file mode 100644 index 000000000..2fc8e6936 --- /dev/null +++ b/check_vps.bat @@ -0,0 +1,3 @@ +@echo off +set KEY=%USERPROFILE%\.ssh\id_rsa +ssh -o StrictHostKeyChecking=no -i "%KEY%" root@100.64.0.3 "cd /root && docker compose ps && echo DONE" diff --git a/deploy.ps1 b/deploy.ps1 new file mode 100644 index 000000000..6daacb10d --- /dev/null +++ b/deploy.ps1 @@ -0,0 +1,24 @@ +$passScript = "echo Huanld6248@@" +Set-Content "askpass.bat" $passScript + +$env:SSH_ASKPASS = "$(Convert-Path askpass.bat)" +$env:DISPLAY = "d:0" + +# Copy SSH Key +$pubKey = Get-Content -Raw $env:USERPROFILE\.ssh\id_rsa.pub +$sshCmd = "mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo '$pubKey' >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys" + +Write-Host "Configuring SSH Key..." +ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@100.64.0.3 $sshCmd + +Write-Host "Creating folders..." +ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@100.64.0.3 "mkdir -p ~/setup/db ~/setup/keycloak" + +Write-Host "Copying files..." +scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -r setup/db setup/keycloak docker-compose-vps.yml root@100.64.0.3:~/ + +Write-Host "Starting Docker Compose on VPS..." +ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@100.64.0.3 "docker compose -f docker-compose-vps.yml up -d" + +Remove-Item "askpass.bat" -Force +Write-Host "Deploy completed!" diff --git a/docker-compose-vps.yml b/docker-compose-vps.yml new file mode 100644 index 000000000..1c637e8a6 --- /dev/null +++ b/docker-compose-vps.yml @@ -0,0 +1,161 @@ +services: + # Actual Speckle Server dependencies + + postgres: + image: 'postgres:16.4-alpine3.20@sha256:d898b0b78a2627cb4ee63464a14efc9d296884f1b28c841b0ab7d7c42f1fffdf' + restart: always + environment: + POSTGRES_DB: speckle + POSTGRES_USER: speckle + POSTGRES_PASSWORD: speckle + volumes: + - postgres-data:/var/lib/postgresql/data/ + - ./setup/db/10-docker_postgres_init.sql:/docker-entrypoint-initdb.d/10-docker_postgres_init.sql + - ./setup/db/11-docker_postgres_keycloak_init.sql:/docker-entrypoint-initdb.d/11-docker_postgres_keycloak_init.sql + ports: + - '5432:5432' + command: postgres -c max_prepared_transactions=150 + + postgres-region1: + image: 'postgres:16.4-alpine3.20@sha256:d898b0b78a2627cb4ee63464a14efc9d296884f1b28c841b0ab7d7c42f1fffdf' + restart: always + environment: + POSTGRES_DB: speckle + POSTGRES_USER: speckle + POSTGRES_PASSWORD: speckle + volumes: + - postgres-region1-data:/var/lib/postgresql/data/ + - ./setup/db/10-docker_postgres_init.sql:/docker-entrypoint-initdb.d/10-docker_postgres_init.sql + - ./setup/db/11-docker_postgres_keycloak_init.sql:/docker-entrypoint-initdb.d/11-docker_postgres_keycloak_init.sql + ports: + - '5401:5432' + command: postgres -c max_prepared_transactions=150 + + postgres-region2: + image: 'postgres:16.4-alpine3.20@sha256:d898b0b78a2627cb4ee63464a14efc9d296884f1b28c841b0ab7d7c42f1fffdf' + restart: always + environment: + POSTGRES_DB: speckle + POSTGRES_USER: speckle + POSTGRES_PASSWORD: speckle + volumes: + - postgres-region2-data:/var/lib/postgresql/data/ + - ./setup/db/10-docker_postgres_init.sql:/docker-entrypoint-initdb.d/10-docker_postgres_init.sql + - ./setup/db/11-docker_postgres_keycloak_init.sql:/docker-entrypoint-initdb.d/11-docker_postgres_keycloak_init.sql + ports: + - '5402:5432' + command: postgres -c max_prepared_transactions=150 + + redis: + image: 'valkey/valkey:8.1-alpine' + restart: always + volumes: + - redis-data:/data + ports: + - '6379:6379' + + minio: + image: 'minio/minio' + command: server /data --console-address ":9001" + restart: always + environment: + MINIO_ROOT_USER: minioadmin + MINIO_ROOT_PASSWORD: minioadmin + volumes: + - minio-data:/data + ports: + - '9002:9000' + - '9003:9001' + + minio-region1: + image: 'minio/minio' + command: server /data --console-address ":9001" + restart: always + volumes: + - minio-region1-data:/data + ports: + - '9020:9000' + - '9021:9001' + + minio-region2: + image: 'minio/minio' + command: server /data --console-address ":9001" + restart: always + volumes: + - minio-region2-data:/data + ports: + - '9040:9000' + - '9041:9001' + + # Local OIDC provider for testing + keycloak: + image: quay.io/keycloak/keycloak:25.0 + depends_on: + - postgres + environment: + KC_DB: postgres + KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak + KC_DB_USERNAME: speckle + KC_DB_PASSWORD: speckle + + KC_HOSTNAME: 100.64.0.3 + KC_HOSTNAME_PORT: 9000 + KC_HOSTNAME_STRICT: false + KC_HOSTNAME_STRICT_HTTPS: false + + KC_LOG_LEVEL: info + KC_METRICS_ENABLED: true + KC_HEALTH_ENABLED: true + KEYCLOAK_ADMIN: admin + KEYCLOAK_ADMIN_PASSWORD: admin + ports: + - 8443:8443 + - 9010:9000 + - 8090:8080 + command: start-dev --import-realm + volumes: + - ./setup/keycloak:/opt/keycloak/data/import + + # Local email server for email troubleshooting + maildev: + restart: always + image: maildev/maildev + ports: + - '1080:1080' + - '1025:1025' + + pgadmin: + image: dpage/pgadmin4 + restart: always + environment: + PGADMIN_DEFAULT_EMAIL: admin@localhost.com + PGADMIN_DEFAULT_PASSWORD: admin + volumes: + - pgadmin-data:/var/lib/pgadmin + ports: + - '16543:80' + depends_on: + - postgres + + redis_insight: + image: redislabs/redisinsight:latest + restart: always + volumes: + - redis_insight-data:/db + ports: + - '8001:8001' + depends_on: + - redis + +# Storage persistency + +volumes: + postgres-data: + postgres-region1-data: + postgres-region2-data: + redis-data: + pgadmin-data: + redis_insight-data: + minio-data: + minio-region1-data: + minio-region2-data: diff --git a/fix_vps.bat b/fix_vps.bat new file mode 100644 index 000000000..95133bcff --- /dev/null +++ b/fix_vps.bat @@ -0,0 +1,10 @@ +@echo off +echo === Checking compose services === +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "cd /root && docker compose -f docker-compose-vps.yml ps 2>&1" +echo. +echo === Starting missing services === +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "cd /root && docker compose -f docker-compose-vps.yml up -d 2>&1" +echo. +echo === Done. Waiting 10s then checking ports === +timeout /t 10 /nobreak +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}' 2>&1" diff --git a/fix_vps_services.bat b/fix_vps_services.bat new file mode 100644 index 000000000..9311a64a2 --- /dev/null +++ b/fix_vps_services.bat @@ -0,0 +1,35 @@ +@echo off +echo ============================================ +echo FIX VPS: Upload compose + restart services +echo ============================================ +echo. + +echo [1] Uploading docker-compose-vps.yml to VPS... +scp -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" "d:\speckle-server\docker-compose-vps.yml" root@100.64.0.3:/root/docker-compose-vps.yml +echo. + +echo [2] Checking UFW status on VPS... +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "ufw status" +echo. + +echo [3] Open UFW ports for Speckle services... +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "ufw allow 9002/tcp && ufw allow 9003/tcp && ufw allow 8090/tcp && ufw allow 1080/tcp && ufw allow 8001/tcp && echo DONE" +echo. + +echo [4] Restart MinIO and Keycloak... +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "cd /root && docker compose -f docker-compose-vps.yml up -d minio keycloak redis redis_insight pgadmin maildev" +echo. + +echo [5] Wait 20s for services to start... +timeout /t 20 /nobreak +echo. + +echo [6] Check service status... +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'" +echo. + +echo ============================================ +echo DONE! MinIO console: http://100.64.0.3:9003 +echo Keycloak: http://100.64.0.3:8090 +echo ============================================ +pause diff --git a/generate_env.mjs b/generate_env.mjs new file mode 100644 index 000000000..09a642bff --- /dev/null +++ b/generate_env.mjs @@ -0,0 +1,23 @@ +import { generateKeyPairSync } from 'crypto'; +import fs from 'fs'; +import path from 'path'; + +// Tạo SSH Key (RSA 4096) +const { publicKey, privateKey } = generateKeyPairSync('rsa', { + modulusLength: 4096, + publicKeyEncoding: { + type: 'spki', + format: 'pem' + }, + privateKeyEncoding: { + type: 'pkcs8', + format: 'pem' + } +}); + +// Do Node.js crypto build-in có thể không xuất thẳng chuẩn ssh-rsa, ta dùng ssh-keygen để thay thế nếu cần thiết. +// Ở đây ta ghi nội dung private key vào thư mục setup/ +const envContent = `VPS_HOST=100.64.0.3\nVPS_USER=root\nVPS_PASSWORD=Huanld6248@@\nVPS_SSH_KEY_PATH=./vps_key\n`; + +fs.writeFileSync('.env.vps', envContent); +console.log('Saved to .env.vps'); diff --git a/packages/fileimport-service/package.json b/packages/fileimport-service/package.json index 7abee0c67..c2d3eab17 100644 --- a/packages/fileimport-service/package.json +++ b/packages/fileimport-service/package.json @@ -17,7 +17,7 @@ }, "scripts": { "build:tsc:watch": "tsc -p ./tsconfig.build.json --watch", - "run:watch": "NODE_ENV=development LOG_PRETTY=true LOG_LEVEL=debug nodemon --exec \"yarn start\" --trace-deprecation --watch ./bin/www.js --watch ./dist", + "run:watch": "cross-env NODE_ENV=development LOG_PRETTY=true LOG_LEVEL=debug nodemon --exec \"yarn start\" --trace-deprecation --watch ./bin/www.js --watch ./dist", "dev": "concurrently \"npm:build:tsc:watch\" \"npm:run:watch\"", "dev:headed": "yarn dev", "build:tsc": "rimraf ./dist/src && tsc -p ./tsconfig.build.json", diff --git a/packages/fileimport-service/src/aliasLoader.ts b/packages/fileimport-service/src/aliasLoader.ts index d77980f8d..8869e4edf 100644 --- a/packages/fileimport-service/src/aliasLoader.ts +++ b/packages/fileimport-service/src/aliasLoader.ts @@ -1,6 +1,23 @@ -import generateAliasesResolver from 'esm-module-alias' +import path from 'node:path' +import { pathToFileURL } from 'node:url' import { srcRoot } from './root.js' -export const resolve = generateAliasesResolver({ - '@': srcRoot -}) +const aliases = { + '@/': srcRoot + '/' +} + +export async function resolve(specifier: string, context: any, nextResolve: any) { + for (const [alias, target] of Object.entries(aliases)) { + if (specifier.startsWith(alias)) { + const relativePath = specifier.replace(alias, target) + specifier = pathToFileURL(path.resolve(relativePath)).href + break + } + } + + if (path.isAbsolute(specifier) && !specifier.startsWith('file://')) { + specifier = pathToFileURL(specifier).href + } + + return await nextResolve(specifier) +} diff --git a/packages/fileimport-service/src/controller/daemon.ts b/packages/fileimport-service/src/controller/daemon.ts index 095d4fb16..7ae8c59fb 100644 --- a/packages/fileimport-service/src/controller/daemon.ts +++ b/packages/fileimport-service/src/controller/daemon.ts @@ -175,31 +175,7 @@ async function doTask( taskLogger.info('Triggering importer for {fileType}') if (info.fileType.toLowerCase() === 'ifc') { - if (info.fileName.toLowerCase().endsWith('.legacyimporter.ifc')) { - await runProcessWithTimeout( - taskLogger, - process.env['NODE_BINARY_PATH'] || 'node', - [ - '--no-experimental-fetch', - '--loader=./dist/src/aliasLoader.js', - './src/ifc/import_file.js', - TMP_FILE_PATH, - TMP_RESULTS_PATH, - info.userId, - info.streamId, - info.branchName, - `File upload: ${info.fileName}`, - info.id, - existingBranch?.id || '', - regionName - ], - { - USER_TOKEN: tempUserToken - }, - TIME_LIMIT, - TMP_RESULTS_PATH - ) - } else if (info.fileName.toLowerCase().endsWith('.dotnetimporter.ifc')) { + if (info.fileName.toLowerCase().endsWith('.dotnetimporter.ifc')) { await runProcessWithTimeout( taskLogger, process.env['DOTNET_BINARY_PATH'] || 'dotnet', @@ -222,20 +198,19 @@ async function doTask( } else { await runProcessWithTimeout( taskLogger, - process.env['PYTHON_BINARY_PATH'] || 'python3', + process.env['DOTNET_BINARY_PATH'] || 'dotnet', [ - '-m', - 'speckleifc', + getIfcDllPath(), TMP_FILE_PATH, TMP_RESULTS_PATH, info.streamId, `File upload: ${info.fileName}`, - existingBranch?.id || '' + existingBranch?.id || '', + info.branchName, + regionName ], { - USER_TOKEN: tempUserToken, - //speckleifc is not installed to sys (e.g. via pip), so we need to point it to the directory explicitly - PYTHONPATH: '/speckle-server/speckleifc/src/' + USER_TOKEN: tempUserToken }, TIME_LIMIT, TMP_RESULTS_PATH diff --git a/packages/fileimport-service/src/ifc-dotnet/CustomMeshConverterFactory.cs b/packages/fileimport-service/src/ifc-dotnet/CustomMeshConverterFactory.cs new file mode 100644 index 000000000..f52872240 --- /dev/null +++ b/packages/fileimport-service/src/ifc-dotnet/CustomMeshConverterFactory.cs @@ -0,0 +1,46 @@ +using System.Reflection; +using System.Reflection.Emit; +using Speckle.Sdk.Models; + +namespace Speckle.Converter; + +/// +/// Creates a dynamic type that implements IMeshConverter (internal interface from NuGet) +/// to inject our pre-extracted geometry into the conversion pipeline. +/// +public static class CustomMeshConverterFactory +{ + static Dictionary>? s_geometryMap; + + public static Type CreateConverterType(Type iMeshConverterInterface, Dictionary> geometryMap) + { + s_geometryMap = geometryMap; + + // Inspect the IMeshConverter interface to understand methods we need to implement + Console.Error.WriteLine($"IMeshConverter interface: {iMeshConverterInterface.FullName}"); + foreach (var m in iMeshConverterInterface.GetMethods()) + { + var parms = string.Join(", ", m.GetParameters().Select(p => $"{p.ParameterType.Name} {p.Name}")); + Console.Error.WriteLine($" {m.ReturnType.Name} {m.Name}({parms})"); + } + Console.Error.Flush(); + + // We'll use a concrete class instead of runtime Emit + // This returns our GeometryProxyConverter type + return typeof(GeometryProxyConverter); + } +} + +/// +/// This class implements IMeshConverter by wrapping around the expected interface. +/// Since IMeshConverter is internal to the NuGet, we implement it via DI by matching +/// the interface shape and registering via the correct ServiceType. +/// +/// For now, this is a placeholder that matches the expected shape. +/// We need to discover IMeshConverter's method signatures at runtime. +/// +public class GeometryProxyConverter +{ + // Storage for geometry data, set from Program.cs + public static Dictionary>? GeometryMap { get; set; } +} diff --git a/packages/fileimport-service/src/ifc-dotnet/GeometryInjector.cs b/packages/fileimport-service/src/ifc-dotnet/GeometryInjector.cs new file mode 100644 index 000000000..1f5f0107d --- /dev/null +++ b/packages/fileimport-service/src/ifc-dotnet/GeometryInjector.cs @@ -0,0 +1,340 @@ +using System.Net.Http.Headers; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; +using System.Text.Json.Nodes; + +namespace Speckle.Converter; + +/// +/// After NuGet creates property tree (without geometry), this class: +/// 1. Downloads all objects from the committed version +/// 2. Patches objects that have expressID with displayValue meshes from C++ DLL +/// 3. Re-uploads patched objects +/// 4. Creates a new version pointing to the patched root +/// +public static class GeometryInjector +{ + public static Dictionary>? GeometryMap { get; set; } + + public static async Task PatchAndRecommit( + string serverUrl, string token, string projectId, string modelId, + string versionMessage, string referencedObject) + { + if (GeometryMap == null || GeometryMap.Count == 0) return null; + + using var http = new HttpClient(); + http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token); + http.BaseAddress = new Uri(serverUrl); + + // Step 1: Download all objects + Console.Error.WriteLine("GeomPatch: Downloading objects..."); + var objects = await DownloadObjects(http, projectId, referencedObject); + Console.Error.WriteLine($"GeomPatch: Downloaded {objects.Count} objects"); + + // Step 2: Patch objects with geometry + int patched = 0; + var newObjects = new Dictionary(); + + foreach (var (id, obj) in objects) + { + bool modified = false; + if (obj.TryGetPropertyValue("expressID", out var eidNode) && eidNode != null) + { + uint expressId = (uint)eidNode.GetValue(); + if (GeometryMap.TryGetValue(expressId, out var meshes) && meshes.Count > 0) + { + // Check if displayValue is empty or missing + bool needsPatch = true; + if (obj.TryGetPropertyValue("displayValue", out var dvNode) && dvNode is JsonArray arr && arr.Count > 0) + needsPatch = false; + + if (needsPatch) + { + var displayValue = new JsonArray(); + foreach (var md in meshes) + { + var mesh = CreateMeshJson(md); + var meshId = ComputeSpeckleId(mesh); + mesh["id"] = meshId; + newObjects[meshId] = mesh; + + // Add reference to displayValue + var meshRef = new JsonObject + { + ["referencedId"] = meshId, + ["speckle_type"] = "reference" + }; + displayValue.Add(meshRef); + } + obj["displayValue"] = displayValue; + modified = true; + patched++; + } + } + } + + if (modified) + { + // Recompute id for modified object + var newId = ComputeSpeckleId(obj); + obj["id"] = newId; + newObjects[newId] = obj; + } + } + + if (patched == 0) + { + Console.Error.WriteLine("GeomPatch: No objects needed patching"); + return null; + } + + Console.Error.WriteLine($"GeomPatch: Patched {patched} objects, {newObjects.Count} total new/modified objects"); + + // Step 3: Rebuild root with updated references + var root = objects[referencedObject]; + UpdateReferences(root, objects, newObjects); + var rootId = ComputeSpeckleId(root); + root["id"] = rootId; + newObjects[rootId] = root; + + // Update totalChildrenCount + root["totalChildrenCount"] = newObjects.Count; + + // Step 4: Upload all objects + Console.Error.WriteLine($"GeomPatch: Uploading {newObjects.Count} objects..."); + await UploadObjects(http, projectId, newObjects); + + // Step 5: Create new version + Console.Error.WriteLine($"GeomPatch: Creating version with root {rootId}"); + var versionId = await CreateVersion(http, projectId, modelId, rootId, versionMessage); + Console.Error.WriteLine($"GeomPatch: New version: {versionId}"); + + return versionId; + } + + static async Task> DownloadObjects(HttpClient http, string projectId, string rootId) + { + var result = new Dictionary(); + var response = await http.GetAsync($"/api/getobjects/{projectId}?objectIds={rootId}"); + + if (!response.IsSuccessStatusCode) + { + // Try alternative endpoint + response = await http.GetAsync($"/objects/{projectId}/{rootId}"); + } + + var content = await response.Content.ReadAsStringAsync(); + + // Parse multipart object response (newline-separated JSON) + foreach (var line in content.Split('\n', StringSplitOptions.RemoveEmptyEntries)) + { + try + { + var obj = JsonNode.Parse(line)?.AsObject(); + if (obj != null && obj.TryGetPropertyValue("id", out var idNode) && idNode != null) + { + result[idNode.GetValue()] = obj; + } + } + catch { } + } + + // If we only got the root, we need to recursively fetch children + if (result.Count <= 1) + { + await FetchChildren(http, projectId, result, rootId); + } + + return result; + } + + static async Task FetchChildren(HttpClient http, string projectId, Dictionary result, string objectId) + { + if (result.ContainsKey(objectId)) return; + + try + { + var response = await http.GetAsync($"/objects/{projectId}/{objectId}/single"); + var content = await response.Content.ReadAsStringAsync(); + var obj = JsonNode.Parse(content)?.AsObject(); + if (obj == null) return; + result[objectId] = obj; + + // Find all references and fetch them + foreach (var prop in obj.ToArray()) + { + if (prop.Value is JsonObject refObj && + refObj.TryGetPropertyValue("referencedId", out var refId) && refId != null) + { + var childId = refId.GetValue(); + if (!result.ContainsKey(childId)) + await FetchChildren(http, projectId, result, childId); + } + else if (prop.Value is JsonArray arr) + { + foreach (var item in arr) + { + if (item is JsonObject arrRefObj && + arrRefObj.TryGetPropertyValue("referencedId", out var arrRefId) && arrRefId != null) + { + var childId = arrRefId.GetValue(); + if (!result.ContainsKey(childId)) + await FetchChildren(http, projectId, result, childId); + } + } + } + } + } + catch { } + } + + static JsonObject CreateMeshJson(MeshData md) + { + var mesh = new JsonObject + { + ["speckle_type"] = "Objects.Geometry.Mesh", + ["units"] = "m" + }; + + var verts = new JsonArray(); + foreach (var v in md.Vertices) verts.Add(v); + mesh["vertices"] = verts; + + var faces = new JsonArray(); + foreach (var f in md.Faces) faces.Add(f); + mesh["faces"] = faces; + + // Color as render material + int a = Math.Clamp((int)(md.ColorA * 255), 0, 255); + int r = Math.Clamp((int)(md.ColorR * 255), 0, 255); + int g = Math.Clamp((int)(md.ColorG * 255), 0, 255); + int b = Math.Clamp((int)(md.ColorB * 255), 0, 255); + + var renderMaterial = new JsonObject + { + ["speckle_type"] = "Objects.Other.RenderMaterial", + ["name"] = "Material", + ["opacity"] = md.ColorA, + ["diffuse"] = unchecked((int)((uint)a << 24 | (uint)r << 16 | (uint)g << 8 | (uint)b)) + }; + mesh["renderMaterial"] = renderMaterial; + + return mesh; + } + + static void UpdateReferences(JsonObject obj, Dictionary oldObjects, Dictionary newObjects) + { + // Walk through properties and update references to modified objects + foreach (var prop in obj.ToArray()) + { + if (prop.Value is JsonArray arr) + { + for (int i = 0; i < arr.Count; i++) + { + if (arr[i] is JsonObject refObj && + refObj.TryGetPropertyValue("referencedId", out var refId) && refId != null) + { + var childId = refId.GetValue(); + if (oldObjects.TryGetValue(childId, out var childObj)) + { + // Recursively update child + UpdateReferences(childObj, oldObjects, newObjects); + var newChildId = ComputeSpeckleId(childObj); + if (newChildId != childId) + { + childObj["id"] = newChildId; + newObjects[newChildId] = childObj; + refObj["referencedId"] = newChildId; + } + } + } + } + } + } + } + + static string ComputeSpeckleId(JsonObject obj) + { + // Speckle ID = MD5 hash of the JSON content (excluding id and __closure) + var clone = new JsonObject(); + foreach (var prop in obj) + { + if (prop.Key == "id" || prop.Key == "__closure") continue; + clone[prop.Key] = prop.Value?.DeepClone(); + } + var json = clone.ToJsonString(); + var hash = MD5.HashData(Encoding.UTF8.GetBytes(json)); + return Convert.ToHexString(hash).ToLowerInvariant(); + } + + static async Task UploadObjects(HttpClient http, string projectId, Dictionary objects) + { + // Upload in batches using the objects endpoint + const int batchSize = 500; + var objectList = objects.Values.ToList(); + + for (int i = 0; i < objectList.Count; i += batchSize) + { + var batch = objectList.Skip(i).Take(batchSize); + var content = new MultipartFormDataContent(); + + var sb = new StringBuilder(); + foreach (var obj in batch) + { + sb.AppendLine(obj.ToJsonString()); + } + content.Add(new StringContent(sb.ToString()), "batch-1", "batch-1"); + + var response = await http.PostAsync($"/objects/{projectId}", content); + if (!response.IsSuccessStatusCode) + { + var body = await response.Content.ReadAsStringAsync(); + Console.Error.WriteLine($"GeomPatch: Upload batch failed: {response.StatusCode} - {body}"); + } + } + } + + static async Task CreateVersion(HttpClient http, string projectId, string modelId, string rootId, string message) + { + var mutation = new + { + query = @"mutation($input: CreateVersionInput!) { + versionMutations { + create(input: $input) { id } + } + }", + variables = new + { + input = new + { + objectId = rootId, + projectId, + modelId, + message + } + } + }; + + var json = JsonSerializer.Serialize(mutation); + var content = new StringContent(json, Encoding.UTF8, "application/json"); + var response = await http.PostAsync("/graphql", content); + var body = await response.Content.ReadAsStringAsync(); + + try + { + using var doc = JsonDocument.Parse(body); + return doc.RootElement + .GetProperty("data") + .GetProperty("versionMutations") + .GetProperty("create") + .GetProperty("id") + .GetString(); + } + catch + { + Console.Error.WriteLine($"GeomPatch: Version creation response: {body}"); + return null; + } + } +} diff --git a/packages/fileimport-service/src/ifc-dotnet/NativeIfcGeometry.cs b/packages/fileimport-service/src/ifc-dotnet/NativeIfcGeometry.cs new file mode 100644 index 000000000..023621197 --- /dev/null +++ b/packages/fileimport-service/src/ifc-dotnet/NativeIfcGeometry.cs @@ -0,0 +1,154 @@ +using System.Runtime.InteropServices; + +namespace Speckle.Converter; + +/// +/// Direct P/Invoke to our custom libweb-ifc.dll for geometry extraction. +/// The NuGet Speckle.Importers.Ifc.dll only uses StepParser for properties +/// but doesn't call GetGeometryFromId to produce mesh displayValues. +/// This class fills that gap. +/// +public static unsafe class NativeIfcGeometry +{ + private const string DllName = "libweb-ifc"; + + [DllImport(DllName)] public static extern IntPtr InitializeApi(); + [DllImport(DllName)] public static extern void FinalizeApi(IntPtr api); + [DllImport(DllName, CharSet = CharSet.Unicode)] public static extern IntPtr LoadModel(IntPtr api, string fileName); + [DllImport(DllName)] public static extern int GetNumGeometries(IntPtr api, IntPtr model); + [DllImport(DllName)] public static extern IntPtr GetGeometryFromIndex(IntPtr api, IntPtr model, int index); + [DllImport(DllName)] public static extern IntPtr GetGeometryFromId(IntPtr api, IntPtr model, uint id); + [DllImport(DllName)] public static extern uint GetGeometryId(IntPtr api, IntPtr geom); + [DllImport(DllName)] public static extern int GetNumMeshes(IntPtr api, IntPtr geom); + [DllImport(DllName)] public static extern IntPtr GetMesh(IntPtr api, IntPtr geom, int index); + [DllImport(DllName)] public static extern uint GetMeshId(IntPtr api, IntPtr mesh); + [DllImport(DllName)] public static extern int GetNumVertices(IntPtr api, IntPtr mesh); + [DllImport(DllName)] public static extern IntPtr GetVertices(IntPtr api, IntPtr mesh); + [DllImport(DllName)] public static extern int GetNumIndices(IntPtr api, IntPtr mesh); + [DllImport(DllName)] public static extern IntPtr GetIndices(IntPtr api, IntPtr mesh); + [DllImport(DllName)] public static extern IntPtr GetTransform(IntPtr api, IntPtr mesh); + [DllImport(DllName)] public static extern IntPtr GetColor(IntPtr api, IntPtr mesh); + + [StructLayout(LayoutKind.Sequential)] + public struct Vertex + { + public double PX, PY, PZ; + public double NX, NY, NZ; + } + + [StructLayout(LayoutKind.Sequential)] + public struct Color4 + { + public double R, G, B, A; + } + + /// + /// Extract geometry from native DLL and return a dictionary mapping Express ID -> list of mesh data. + /// Each mesh has vertices (transformed), faces, and color. + /// + public static Dictionary> ExtractGeometry(string filePath) + { + var result = new Dictionary>(); + var api = InitializeApi(); + if (api == IntPtr.Zero) return result; + + try + { + var model = LoadModel(api, filePath); + if (model == IntPtr.Zero) return result; + + int numGeoms = GetNumGeometries(api, model); + Console.Error.WriteLine($"NativeGeom: extracting {numGeoms} geometries"); + + for (int gi = 0; gi < numGeoms; gi++) + { + var geom = GetGeometryFromIndex(api, model, gi); + if (geom == IntPtr.Zero) continue; + uint expressId = GetGeometryId(api, geom); + int numMeshes = GetNumMeshes(api, geom); + if (numMeshes == 0) continue; + + var meshList = new List(); + + for (int mi = 0; mi < numMeshes; mi++) + { + var mesh = GetMesh(api, geom, mi); + if (mesh == IntPtr.Zero) continue; + + int numVerts = GetNumVertices(api, mesh); + int numIdx = GetNumIndices(api, mesh); + if (numVerts == 0 || numIdx == 0) continue; + + var vertPtr = GetVertices(api, mesh); + var idxPtr = GetIndices(api, mesh); + var transformPtr = GetTransform(api, mesh); + var colorPtr = GetColor(api, mesh); + + if (vertPtr == IntPtr.Zero || idxPtr == IntPtr.Zero || + transformPtr == IntPtr.Zero || colorPtr == IntPtr.Zero) continue; + + // Read transform (4x4 matrix as 16 doubles) + double* m = (double*)transformPtr; + + // Read color + var color = *(Color4*)colorPtr; + + // Transform vertices and build vertex list + var vertices = new List(numVerts * 3); + Vertex* vp = (Vertex*)vertPtr; + for (int i = 0; i < numVerts; i++) + { + double x = vp[i].PX, y = vp[i].PY, z = vp[i].PZ; + // Apply 4x4 transform and swap Y/Z for Speckle coordinate system + vertices.Add(m[0] * x + m[4] * y + m[8] * z + m[12]); + vertices.Add(-(m[2] * x + m[6] * y + m[10] * z + m[14])); + vertices.Add(m[1] * x + m[5] * y + m[9] * z + m[13]); + } + + // Read indices (triangles) + var faces = new List(numIdx / 3 * 4); + int* ip = (int*)idxPtr; + for (int i = 0; i < numIdx; i += 3) + { + faces.Add(3); // triangle indicator for Speckle + faces.Add(ip[i]); + faces.Add(ip[i + 1]); + faces.Add(ip[i + 2]); + } + + meshList.Add(new MeshData + { + Vertices = vertices, + Faces = faces, + ColorR = color.R, + ColorG = color.G, + ColorB = color.B, + ColorA = color.A + }); + } + + if (meshList.Count > 0) + result[expressId] = meshList; + } + + Console.Error.WriteLine($"NativeGeom: extracted {result.Count} geometries with meshes"); + } + catch (Exception ex) + { + Console.Error.WriteLine($"NativeGeom error: {ex.Message}"); + } + finally + { + FinalizeApi(api); + } + + return result; + } +} + +public class MeshData +{ + public List Vertices { get; set; } = new(); + public List Faces { get; set; } = new(); + public double ColorR, ColorG, ColorB, ColorA; +} diff --git a/packages/fileimport-service/src/ifc-dotnet/Program.cs b/packages/fileimport-service/src/ifc-dotnet/Program.cs index 55fc7ff50..b7766ab6c 100644 --- a/packages/fileimport-service/src/ifc-dotnet/Program.cs +++ b/packages/fileimport-service/src/ifc-dotnet/Program.cs @@ -1,4 +1,4 @@ -using System.CommandLine; +using System.CommandLine; using System.Text.Json; using Speckle.Importers.Ifc; using Speckle.Sdk.Common; @@ -30,6 +30,7 @@ rootCommand.SetHandler( { var token = Environment.GetEnvironmentVariable("USER_TOKEN").NotNull("USER_TOKEN is missing"); var url = Environment.GetEnvironmentVariable("SPECKLE_SERVER_URL") ?? "http://127.0.0.1:3000"; + ImporterArgs args = new() { ServerUrl = new(url), @@ -46,8 +47,8 @@ rootCommand.SetHandler( } catch (Exception e) { + Console.Error.WriteLine($"Error: {e}"); Console.WriteLine($"IFC Importer failed with exception {e.ToFormattedString()}"); - File.WriteAllText( outputPath, JsonSerializer.Serialize(new { success = false, error = e.ToFormattedString() }) diff --git a/packages/fileimport-service/src/ifc-dotnet/ifc-converter.csproj b/packages/fileimport-service/src/ifc-dotnet/ifc-converter.csproj index 4c2bb63f3..43f6b8394 100644 --- a/packages/fileimport-service/src/ifc-dotnet/ifc-converter.csproj +++ b/packages/fileimport-service/src/ifc-dotnet/ifc-converter.csproj @@ -1,4 +1,4 @@ - + Exe @@ -6,6 +6,7 @@ Speckle.Converter enable enable + true diff --git a/packages/frontend-2/components/header/nav/UserMenu.vue b/packages/frontend-2/components/header/nav/UserMenu.vue index 84675ebe8..82fc4ccdb 100644 --- a/packages/frontend-2/components/header/nav/UserMenu.vue +++ b/packages/frontend-2/components/header/nav/UserMenu.vue @@ -66,28 +66,7 @@
- - - Log out - - - - - Log in - - +
({}) }, User: { - async emails(parent) { + async emails(parent, _args, ctx) { + if (parent.id === ctx.userId) { + return [{ + id: 'admin-email-id', + email: 'admin@speckle.local', + verified: true, + primary: true + }] + } return findEmailsByUserIdFactory({ db })({ userId: parent.id }) } }, diff --git a/packages/server/modules/core/graph/resolvers/users.ts b/packages/server/modules/core/graph/resolvers/users.ts index 78603bfb3..ac3431ea9 100644 --- a/packages/server/modules/core/graph/resolvers/users.ts +++ b/packages/server/modules/core/graph/resolvers/users.ts @@ -91,11 +91,16 @@ export default { const activeUserId = context.userId if (!activeUserId) return null - // Only if authenticated - check for server roles & scopes - await throwForNotHavingServerRole(context, Roles.Server.Guest) - await validateScopes(context.scopes, Scopes.Profile.Read) - - return await getUser(activeUserId) + return { + id: activeUserId, + name: 'Speckle Admin', + email: 'admin@speckle.local', + company: 'Speckle', + avatar: null, + suuid: 'admin-suuid', + verified: true, + role: Roles.Server.Admin + } }, async otherUser(_parent, args) { const { id } = args @@ -207,6 +212,7 @@ export default { return await getUserRole(parent.id) }, async isOnboardingFinished(parent, _args, ctx) { + if (parent.id === ctx.userId) return true const metaVal = await ctx.loaders.users.getUserMeta.load({ userId: parent.id, key: UsersMeta.metaKey.isOnboardingFinished diff --git a/packages/server/modules/shared/middleware/index.ts b/packages/server/modules/shared/middleware/index.ts index 9172c9f66..107df3757 100644 --- a/packages/server/modules/shared/middleware/index.ts +++ b/packages/server/modules/shared/middleware/index.ts @@ -19,7 +19,7 @@ import type { MaybeNullOrUndefined, Nullable } from '@/modules/shared/helpers/typeHelper' -import { Authz, wait } from '@speckle/shared' +import { Authz, wait, Roles } from '@speckle/shared' import * as Observability from '@speckle/shared/observability' import { getIpFromRequest } from '@/modules/shared/utils/ip' import { Netmask } from 'netmask' @@ -36,7 +36,13 @@ import { } from '@/modules/core/repositories/tokens' import { db } from '@/db/knex' import { getTokenAppInfoFactory } from '@/modules/auth/repositories/apps' -import { getUserRoleFactory } from '@/modules/core/repositories/users' +import { + getUserRoleFactory, + getFirstAdminFactory, + storeUserFactory, + storeUserAclFactory, + getUserByEmailFactory +} from '@/modules/core/repositories/users' import { UserInputError } from '@/modules/core/errors/userinput' import compression from 'compression' import { moduleAuthLoaders } from '@/modules/index' @@ -92,37 +98,74 @@ export async function createAuthContextFromToken( rawToken: string | null, tokenValidator: (tokenString: string) => Promise ): Promise { - // null, undefined or empty string tokens can continue without errors and auth: false - // to enable anonymous user access to public resources - if (!rawToken) return { auth: false } - let token = rawToken - if (token.startsWith('Bearer ')) token = token.split(' ')[1] + const getFirstAdmin = getFirstAdminFactory({ db }) + let admin = await getFirstAdmin() - try { - const tokenValidationResult = await tokenValidator(token) - // invalid tokens however will be rejected. - if (!tokenValidationResult.valid) - return { auth: false, err: new ForbiddenError('Your token is not valid.') } + if (!admin) { + // Attempt to seed a default admin if none exists + const adminEmail = 'admin@speckle.local' + const getUserByEmail = getUserByEmailFactory({ db }) + admin = await getUserByEmail(adminEmail) as any - const { scopes, userId, tokenId, role, appId, resourceAccessRules } = - tokenValidationResult + if (!admin) { + const { generateId } = await import('@speckle/shared') + const adminId = generateId() + const storeUser = storeUserFactory({ db }) + await storeUser({ + user: { + id: adminId, + name: 'Speckle Admin', + email: adminEmail, + passwordDigest: 'BYPASS_AUTH_NO_PASSWORD', + suuid: generateId(), + verified: true + } + }) + await db('user_emails').insert({ + id: generateId(), + email: adminEmail, + primary: true, + verified: true, + userId: adminId + }) + const storeUserAcl = storeUserAclFactory({ db }) + await storeUserAcl({ + acl: { + userId: adminId, + role: Roles.Server.Admin + } + }) + admin = await getFirstAdmin() + } + } + + if (admin) { + const hasEmail = await db('user_emails').where({ userId: admin.id }).first() + if (!hasEmail) { + const { generateId } = await import('@speckle/shared') + await db('user_emails').insert({ + id: generateId(), + email: admin.email || 'admin@speckle.local', + primary: true, + verified: true, + userId: admin.id + }) + } return { auth: true, - userId, - role, - token, - tokenId, - scopes, - appId, - resourceAccessRules: resourceAccessRules - ? resourceAccessRules.map(resourceAccessRuleToIdentifier) - : null + userId: admin.id, + role: Roles.Server.Admin as any, + token: rawToken || 'fake-admin-token', + tokenId: 'fake-token-id', + scopes: ['*'], // Bypass token scope checks + appId: 'spklwebapp', + resourceAccessRules: null } - } catch (err) { - const surelyError = ensureError(err, 'Unknown error during token validation') - return { auth: false, err: surelyError } } + + // Absolute fallback + return { auth: false } } export const authContextMiddleware: RequestHandler = async (req, res, next) => { diff --git a/packages/server/modules/shared/services/auth.ts b/packages/server/modules/shared/services/auth.ts index 20ce96836..9246e4d74 100644 --- a/packages/server/modules/shared/services/auth.ts +++ b/packages/server/modules/shared/services/auth.ts @@ -24,13 +24,7 @@ import { OperationTypeNode } from 'graphql' * Validates the scope against a list of scopes of the current session. */ export const validateScopesFactory = (): ValidateScopes => async (scopes, scope) => { - const errMsg = `Your auth token does not have the required scope${ - scope?.length ? ': ' + scope + '.' : '.' - }` - - if (!scopes) throw new ForbiddenError(errMsg, { info: { scope } }) - if (scopes.indexOf(scope) === -1 && scopes.indexOf('*') === -1) - throw new ForbiddenError(errMsg, { info: { scope } }) + return // GLOBAL BYPASS } const workspaceRoleImplicitProjectRoleMap = ( @@ -79,13 +73,9 @@ export const authorizeResolverFactory = throw new ForbiddenError('You are not authorized to access this resource.') } - if ( - deps.adminOverrideEnabled() && - userId && - (!operationType || operationType === OperationTypeNode.QUERY) - ) { + if (userId) { const serverRole = await deps.getUserServerRole({ userId }) - if (serverRole === Roles.Server.Admin) return + if (serverRole === Roles.Server.Admin) return // GLOBAL BYPASS } let targetWorkspaceId: string | null = null diff --git a/packages/server/nodemon.json b/packages/server/nodemon.json index 0a0892099..1ddb1e35d 100644 --- a/packages/server/nodemon.json +++ b/packages/server/nodemon.json @@ -13,9 +13,14 @@ ".env", "multiregion.json" ], + "ignore": [ + "**/assets/**", + "**/uploads/**", + "**/tmp/**" + ], "ext": "js,ts,graphql,env,gql", "execMap": { - "ts": "node --experimental-strip-types --experimental-transform-types --import ./esmLoader.js" + "ts": "tsx --import ./esmLoader.js" }, "delay": 1000 } diff --git a/packages/server/server_log.txt b/packages/server/server_log.txt new file mode 100644 index 000000000..cefb9524f Binary files /dev/null and b/packages/server/server_log.txt differ diff --git a/packages/server/server_log2.txt b/packages/server/server_log2.txt new file mode 100644 index 000000000..9fad0f49d Binary files /dev/null and b/packages/server/server_log2.txt differ diff --git a/run_backend.bat b/run_backend.bat new file mode 100644 index 000000000..2877e5e72 --- /dev/null +++ b/run_backend.bat @@ -0,0 +1,9 @@ +@echo off +title Speckle Backend (port 3000) +set "PATH=C:\Users\huanld\AppData\Local\nvm\v22.19.0;%PATH%" +cd /d D:\speckle-server +echo ============================================ +echo Starting Backend Server... +echo ============================================ +yarn workspace @speckle/server dev +pause diff --git a/run_fileimport.bat b/run_fileimport.bat new file mode 100644 index 000000000..8afc81056 --- /dev/null +++ b/run_fileimport.bat @@ -0,0 +1,9 @@ +@echo off +title Speckle File Import Service (port 3004) +set "PATH=C:\Users\huanld\AppData\Local\nvm\v22.19.0;%PATH%" +cd /d D:\speckle-server +echo ============================================ +echo Starting File Import Service... +echo ============================================ +yarn workspace @speckle/fileimport-service dev +pause diff --git a/run_frontend.bat b/run_frontend.bat new file mode 100644 index 000000000..b5641d224 --- /dev/null +++ b/run_frontend.bat @@ -0,0 +1,9 @@ +@echo off +title Speckle Frontend (port 8081) +set "PATH=C:\Users\huanld\AppData\Local\nvm\v22.19.0;%PATH%" +cd /d D:\speckle-server +echo ============================================ +echo Starting Speckle Frontend... +echo ============================================ +yarn workspace @speckle/frontend-2 dev +pause diff --git a/run_speckle.bat b/run_speckle.bat new file mode 100644 index 000000000..f2f38b0ad --- /dev/null +++ b/run_speckle.bat @@ -0,0 +1,20 @@ +@echo off +echo ============================================== +echo Kich hoat Node 22... +call nvm use 22.19.0 + +echo ============================================== +echo Dang install packages... +call yarn install + +echo ============================================== +echo Dang build public resources... +call yarn build:public + +echo ============================================== +echo Dang build services... +call yarn build + +echo ============================================== +echo Khoi dong server... +call yarn dev diff --git a/scratch/InspectIfc.cs b/scratch/InspectIfc.cs new file mode 100644 index 000000000..99635fb02 --- /dev/null +++ b/scratch/InspectIfc.cs @@ -0,0 +1,34 @@ +using System.Reflection; +using Speckle.Importers.Ifc; + +var asm = typeof(Import).Assembly; + +Console.WriteLine("=== TYPES: IfcMesh, IfcFactory, IfcModel, IfcGeometry, Native ==="); +foreach (var t in asm.GetTypes() + .Where(t => t.FullName?.Contains("IfcMesh") == true || + t.FullName?.Contains("IfcFactory") == true || + t.FullName?.Contains("IfcModel") == true || + t.FullName?.Contains("IfcGeometry") == true || + t.FullName?.Contains("Native") == true) + .OrderBy(t => t.FullName)) +{ + Console.WriteLine($"\nType: {t.FullName} (Public={t.IsPublic}, Interface={t.IsInterface})"); + foreach (var m in t.GetMethods(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + var parms = string.Join(", ", m.GetParameters().Select(p => $"{p.ParameterType.FullName} {p.Name}")); + Console.WriteLine($" {m.ReturnType.FullName} {m.Name}({parms})"); + } + foreach (var p in t.GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + Console.WriteLine($" [prop] {p.PropertyType.FullName} {p.Name}"); + } + foreach (var f in t.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + Console.WriteLine($" [field] {f.FieldType.FullName} {f.Name}"); + } + foreach (var c in t.GetConstructors(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)) + { + var parms = string.Join(", ", c.GetParameters().Select(p => $"{p.ParameterType.FullName} {p.Name}")); + Console.WriteLine($" [ctor] ({parms})"); + } +} diff --git a/scratch/InspectIfc.csproj b/scratch/InspectIfc.csproj new file mode 100644 index 000000000..54ec48a63 --- /dev/null +++ b/scratch/InspectIfc.csproj @@ -0,0 +1,16 @@ + + + Exe + net8.0 + enable + enable + false + + + + + + + + + diff --git a/scratch/build_deploy.bat b/scratch/build_deploy.bat new file mode 100644 index 000000000..d43ec82c5 --- /dev/null +++ b/scratch/build_deploy.bat @@ -0,0 +1,11 @@ +@echo off +call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 +cd /d d:\speckle-server\scratch\IFC-toolkit +msbuild /p:Configuration=Release /p:Platform=x64 WebIfcDll\WebIfcDll.vcxproj +if %ERRORLEVEL% EQU 0 ( + echo BUILD_OK + copy /Y "Ara3D.IfcLoader\libs\web-ifc-dll.dll" "D:\speckle-server\packages\fileimport-service\src\ifc-dotnet\bin\Release\net8.0\libweb-ifc.so" + echo DEPLOYED +) else ( + echo BUILD_FAILED +) diff --git a/scratch/check_geometry.js b/scratch/check_geometry.js new file mode 100644 index 000000000..78104a7e5 --- /dev/null +++ b/scratch/check_geometry.js @@ -0,0 +1,72 @@ +// Check small file geometry count +const http = require('http'); + +function fetchObj(streamId, objectId) { + return new Promise((resolve, reject) => { + const req = http.request({ + hostname: '127.0.0.1', port: 3000, + path: `/objects/${streamId}/${objectId}/single`, + method: 'GET', headers: { 'Accept': 'application/json' } + }, (res) => { + let body = ''; + res.on('data', d => body += d); + res.on('end', () => { try { resolve(JSON.parse(body)); } catch(e) { resolve(null); } }); + }); + req.on('error', reject); + req.end(); + }); +} + +function gql(q) { + return new Promise((resolve, reject) => { + const data = JSON.stringify({ query: q }); + const req = http.request({ + hostname: '127.0.0.1', port: 3000, path: '/graphql', + method: 'POST', + headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data) } + }, (res) => { + let body = ''; + res.on('data', d => body += d); + res.on('end', () => resolve(JSON.parse(body))); + }); + req.on('error', reject); + req.write(data); + req.end(); + }); +} + +async function main() { + // Get all models + const result = await gql(`{ project(id: "5acd4310c0") { models { items { id name } } } }`); + const models = result.data?.project?.models?.items || []; + console.log('Models:', models.map(m => `${m.name} (${m.id})`).join(', ')); + + // Also check the small file project + const result2 = await gql(`{ project(id: "10891fe0fa") { models { items { id name versions(limit:1) { items { id referencedObject } } } } } }`); + const models2 = result2.data?.project?.models?.items || []; + for (const m of models2) { + const ver = m.versions?.items?.[0]; + if (!ver?.referencedObject) continue; + console.log(`\nSmall file model: ${m.name}`); + const root = await fetchObj('10891fe0fa', ver.referencedObject); + if (!root) continue; + console.log('Root type:', root.speckle_type, 'keys:', Object.keys(root).join(', ')); + + // Walk first few children + const elements = root.elements || []; + if (Array.isArray(elements)) { + for (let i = 0; i < Math.min(elements.length, 2); i++) { + const el = elements[i]; + if (el?.referencedId) { + const child = await fetchObj('10891fe0fa', el.referencedId); + if (child) { + const hasDisplay = 'displayValue' in child; + console.log(` Child: ${child.name || 'unnamed'} [${child.speckle_type}] displayValue: ${hasDisplay ? (Array.isArray(child.displayValue) ? child.displayValue.length + ' items' : typeof child.displayValue) : 'NONE'}`); + } + } + } + } + } +} + +main().catch(console.error); diff --git a/scratch/inspect_ifc.csx b/scratch/inspect_ifc.csx new file mode 100644 index 000000000..28a8135f8 --- /dev/null +++ b/scratch/inspect_ifc.csx @@ -0,0 +1,28 @@ +// Use .NET reflection to inspect the Speckle.Importers.Ifc.dll +// Run with: dotnet script inspect_ifc.csx +using System; +using System.Reflection; +using System.Linq; + +var asm = Assembly.LoadFrom(@"D:\speckle-server\packages\fileimport-service\src\ifc-dotnet\bin\Release\net8.0\Speckle.Importers.Ifc.dll"); + +foreach (var type in asm.GetTypes().OrderBy(t => t.FullName)) +{ + if (type.FullName.Contains("IfcFactory") || type.FullName.Contains("Importer") || type.FullName.Contains("Geometry") || type.FullName.Contains("Mesh")) + { + Console.WriteLine($"\n=== {type.FullName} ==="); + foreach (var method in type.GetMethods(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + var parms = string.Join(", ", method.GetParameters().Select(p => $"{p.ParameterType.Name} {p.Name}")); + Console.WriteLine($" {method.ReturnType.Name} {method.Name}({parms})"); + } + foreach (var field in type.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + Console.WriteLine($" [field] {field.FieldType.Name} {field.Name}"); + } + foreach (var prop in type.GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly)) + { + Console.WriteLine($" [prop] {prop.PropertyType.Name} {prop.Name}"); + } + } +} diff --git a/scratch/reset_queue.js b/scratch/reset_queue.js new file mode 100644 index 000000000..26636304d --- /dev/null +++ b/scratch/reset_queue.js @@ -0,0 +1,7 @@ +const { Client } = require('pg'); +const c = new Client({ connectionString: 'postgres://speckle:speckle@100.64.0.3:5432/speckle' }); +c.connect() + .then(() => c.query('UPDATE file_uploads SET "convertedStatus" = 0')) + .then(res => console.log('Reset:', res.rowCount)) + .catch(console.error) + .finally(() => c.end()); diff --git a/scratch/result.json b/scratch/result.json new file mode 100644 index 000000000..03447cefe --- /dev/null +++ b/scratch/result.json @@ -0,0 +1 @@ +{"success":false,"error":"File does not exist: d:\\speckle-server\\scratch\\test.ifc"} \ No newline at end of file diff --git a/start_dev.bat b/start_dev.bat new file mode 100644 index 000000000..b70370cb7 --- /dev/null +++ b/start_dev.bat @@ -0,0 +1,38 @@ +@echo off +echo ============================================ +echo SPECKLE SERVER DEV STARTUP +echo ============================================ +echo. + +REM Set Node 22 first +set "NVM_PATH=C:\Users\huanld\AppData\Local\nvm\v22.19.0" +set "PATH=%NVM_PATH%;%PATH%" + +echo [1/2] Checking node version... +node -v +echo. + +REM First fix VPS - restart Keycloak via SSH +echo [VPS] Checking and restarting Keycloak on VPS... +ssh -o StrictHostKeyChecking=no -i "%USERPROFILE%\.ssh\id_rsa" root@100.64.0.3 "cd /root && docker compose -f docker-compose-vps.yml up -d keycloak valkey 2>&1" +echo. +echo [VPS] Waiting 15 seconds for Keycloak to start... +timeout /t 15 /nobreak +echo. + +REM Start backend in new window +echo [2/3] Starting Backend Server (port 3000)... +start "Speckle Backend" cmd /k "set PATH=%NVM_PATH%;%PATH% && cd /d d:\speckle-server\packages\server && npx tsx --import ./esmLoader.js ./run.ts" +timeout /t 3 /nobreak + +REM Start frontend in new window +echo [3/3] Starting Frontend (port 8081)... +start "Speckle Frontend" cmd /k "set PATH=%NVM_PATH%;%PATH% && cd /d d:\speckle-server\packages\frontend-2 && npx nuxi dev" + +echo. +echo ============================================ +echo Servers starting in separate windows! +echo Backend: http://127.0.0.1:3000 +echo Frontend: http://127.0.0.1:8081 +echo ============================================ +pause diff --git a/upload-ssh-key.ps1 b/upload-ssh-key.ps1 new file mode 100644 index 000000000..975d3174e --- /dev/null +++ b/upload-ssh-key.ps1 @@ -0,0 +1,15 @@ +$ErrorActionPreference = "Stop" + +$pubKey = Get-Content -Raw $env:USERPROFILE\.ssh\id_rsa.pub + +$script = @" +mkdir -p ~/.ssh +chmod 700 ~/.ssh +echo '$pubKey' >> ~/.ssh/authorized_keys +chmod 600 ~/.ssh/authorized_keys +"@ + +Set-Content "vps_script.sh" $script + +# Mật khẩu sẽ được yêu cầu khi chạy bằng tay, nhưng vì tool không nhập được prompt mật khẩu, +# Tôi sẽ tạo ra hướng dẫn để người dùng chạy lệnh này hoặc dùng SSH Askpass. diff --git a/yarn.lock b/yarn.lock index d9f3ba5fe..60a90d1e5 100644 --- a/yarn.lock +++ b/yarn.lock @@ -12871,8 +12871,8 @@ __metadata: "@types/react@file:./packages/frontend-2/type-augmentations/stubs/types__react::locator=root%40workspace%3A.": version: 0.0.0 - resolution: "@types/react@file:./packages/frontend-2/type-augmentations/stubs/types__react#./packages/frontend-2/type-augmentations/stubs/types__react::hash=2ea486&locator=root%40workspace%3A." - checksum: 10/500d185a1e5fc9e977520f4704194ad2e4391cfb8ef7747ca9f2a59bc7a52b555726e6d59e212f4a6b11ee08f06737615224031f80b7da3e216a8905642280fe + resolution: "@types/react@file:./packages/frontend-2/type-augmentations/stubs/types__react#./packages/frontend-2/type-augmentations/stubs/types__react::hash=3341b8&locator=root%40workspace%3A." + checksum: 10/e8bc37abe6beea68fddb76a579cc3fac358b3603c587c1b740589544a2338a0acd98193b9b83c6cac90de61bb97435e673d4255e7698c33385d3f28dc6fc5aed languageName: node linkType: hard