Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ANTIGRAVITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ GitGalaxy scans itself and outputs intelligence to `/docs/gitgalaxy_architecture
- **The SQLite Database (`docs/self_scan/gitgalaxy_master.db`):** Instead of reading an entire heavy file to figure out dependencies or function bounds, run targeted SQL queries.
- *Example:* Find complexity and out-bound calls before touching a function:
```bash
sqlite3 docs/self_scan/gitgalaxy_master.db "SELECT db_complexity, is_recursive, calls_out_to FROM function_data WHERE func_name = 'execute_pipeline';"
sqlite3 docs/self_scan/gitgalaxy_master.db "SELECT big_o_depth, is_recursive, calls_out_to FROM function_data WHERE func_name = 'execute_pipeline';"
```
- *Example:* Check a file's risk exposure before adding features (e.g. `risk_cognitive_load`, `risk_state_flux`):
```bash
Expand Down
25 changes: 13 additions & 12 deletions docs/wiki/08-24-Big-O-Detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
> **File Reference:** [`gitgalaxy/metrics/signal_processor.py`](file:///home/joe/nyx_projects/gitgalaxy/gitgalaxy/metrics/signal_processor.py)

## Engineering Summary
Performance and algorithmic complexity directly impact application security. Deeply nested loops ($O(N^2)$, $O(N^3)$) or exponential recursion ($O(2^N)$) connected to public API endpoints, network I/O, or database queries present severe Algorithmic Denial of Service (DoS) vulnerabilities. GitGalaxy evaluates function nesting depth and correlates it with public exposure and data operations. This subsystem evaluates the input signals to calculate a formalized risk score. In GitGalaxy, this subsystem is known as the Algorithmic DoS & Big-O Detection metric.
Performance and algorithmic complexity directly impact application security. Deeply nested loops ($O(N^2)$, $O(N^3)$) or exponential recursion ($O(2^N)$) connected to public API endpoints or network I/O present severe Algorithmic Denial of Service (DoS) vulnerabilities. GitGalaxy evaluates function nesting depth and correlates it with public exposure and data operations. This subsystem evaluates the input signals to calculate a formalized risk score. In GitGalaxy, this subsystem is known as the Algorithmic DoS & Big-O Detection metric.

## Purpose
The metric calculates a density-based risk score (0-100) to flag files containing high-risk logic patterns and architectural deviations.
Expand All @@ -12,25 +12,30 @@ The metric calculates a density-based risk score (0-100) to flag files containin
Unmitigated anti-patterns and vulnerabilities often lead to hard-to-debug bugs and security flaws. By statically analyzing the codebase, this subsystem proactively identifies hazardous logic.

## Design
The engine analyzes function complexity depth, choke point multipliers, database gravity, and guardrail mitigations:
The engine analyzes function complexity depth, choke point multipliers, and guardrail mitigations:

| Variable | Signal Focus | Role / Multiplier | Description |
| :--- | :--- | :--- | :--- |
| `big_o_depth` | Algorithmic Depth | **Exponential Base** | Evaluates nesting depth. $O(N)$ ($\text{depth} < 2$) is ignored. $O(N^2)$ yields base threat of $4$; $O(N^3)$ yields base threat of $9$. |
| `api` / `io` | Choke Points | **Additive Multiplier** | Functions exposed to public APIs or network I/O act as weaponizable triggers. |
| `db_complexity` | Database Gravity | **Additive Multiplier** | Heavy loops paired with database queries generate severe locking and latency risks ($1.0 + \text{DBComplexity} \times 0.5$). |
| `state_mutation` / `globals` | State Mutation | **Additive Multiplier** | Mutating state inside high-depth loops increases risk. |
| `safety` / `panics_and_aborts` | Guardrails | **0.5x Dampener** | Break statements, return limits, and try/catch blocks reduce function threat mass by $50\%$. |
| `popularity` | Network Posture | **0.1x – 3.0x** | Repository-wide import popularity scales the final threat mass. Safely isolated orphans are scaled to $0.10$. |

> **#1013:** this metric used to also fold in a per-function `db_complexity` score
> as a "Database Gravity" multiplier. It was removed engine-wide: despite the name,
> it never looked for databases at all, it just summed unrelated `io` (x3),
> `serialization_parsing` (x2), and `state_mutation` (x1) signature hits -- so any
> IO-heavy or mutation-heavy function scored as "database complex" even with zero
> database or ORM involvement. `api`/`io` choke points below already cover the real
> IO signal this used to (partially, and inaccurately) proxy for.

### 1. Function Base Threat & Amplifiers
For each function with nesting depth $\ge 2$:

$$\text{BaseThreat} = \text{big\_o\_depth}^2$$
$$\text{GravityMultiplier} = 1.0 + (\text{db\_complexity} \times 0.5)$$
$$\text{ChokeMultiplier} = 1.0 + \text{api\_hits} + \text{io\_hits} + \text{flux\_hits}$$
$$\text{FuncThreat} = \text{BaseThreat} \times \text{GravityMultiplier} \times \text{ChokeMultiplier}$$
$$\text{FuncThreat} = \text{BaseThreat} \times \text{ChokeMultiplier}$$

### 2. Guardrail Mitigation
If safety guardrails (`safety`, `panics_and_aborts`, `cleanup`) exist within the function, threat mass is halved:
Expand Down Expand Up @@ -60,7 +65,7 @@ def _calc_algorithmic_dos(
popularity: int,
) -> float:
"""
Calculates Algorithmic DoS Exposure based on Big-O depth, data gravity, and network choke points.
Calculates Algorithmic DoS Exposure based on Big-O depth and network choke points.
"""
if not functions:
return 0.0
Expand All @@ -76,11 +81,7 @@ def _calc_algorithmic_dos(
# 2. Base Threat (Exponential decay of performance)
func_threat = float(depth**2)

# 3. Data Gravity & Network Choke Points
db_complex = func.get("db_complexity", 0)
if db_complex > 0:
func_threat *= 1.0 + (db_complex * 0.5)

# 3. Network Choke Points
hv = func.get("hit_vector", {})
api_hits = hv.get("api", 0)
io_hits = hv.get("io", 0) + hv.get("sec_io", 0)
Expand Down Expand Up @@ -126,7 +127,7 @@ def _calc_algorithmic_dos(
**Risk Classification:**
* 🟦 **VERY LOW (Score 0–19):** Linear $O(N)$ execution or safely bounded stream iterations.
* 🟨 **INTERMEDIATE (Score 40–59):** Isolated $O(N^2)$ logic guarded by safety bailouts or low exposure.
* 🟥 **VERY HIGH (Score 80–100):** Recursive $O(2^N)$ or $O(N^3)$ loops directly wired into unauthenticated public API routes, I/O operations, or state-mutating database calls.
* 🟥 **VERY HIGH (Score 80–100):** Recursive $O(2^N)$ or $O(N^3)$ loops directly wired into unauthenticated public API routes or state-mutating I/O operations.

## Pipeline Integration
Inputs received include raw static analysis signals from the AST parser and contextual multipliers. Outputs produced are a normalized risk score (0-100). The subsystem depends on upstream token parsers that feed AST information into the signal processor.
Expand Down
12 changes: 0 additions & 12 deletions gitgalaxy/core/detector.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,6 @@ class FunctionNode(TypedDict, total=False):

big_o_depth: int
is_recursive: bool
db_complexity: int
docstring: str
calls_out_to: list[str]
hit_vector: dict[str, int]
Expand Down Expand Up @@ -2136,16 +2135,6 @@ def _calculate_block_metrics(
if occurrence_count > 1:
is_recursive = True

# --- NEW: FUNCTION-LEVEL DATABASE COMPLEXITY (Data Gravity) ---
# Mapped to active v6 schemas: 'io' (DB connections/SQL), 'state_mutation' (mutations), and 'serialization_parsing' (JSON/ORMs).
db_complexity = 0
if hit_vector:
db_complexity = (
(hit_vector.get("io", 0) * 3)
+ (hit_vector.get("serialization_parsing", 0) * 2)
+ (hit_vector.get("state_mutation", 0) * 1)
)

# --- NEW: FUNCTION-LEVEL KEYWORD DENSITY (The Micro-Auditor) ---
# Total structural signals divided by the physical lines of the function.
total_keyword_hits = sum(hit_vector.values()) if hit_vector else total_hits
Expand Down Expand Up @@ -2287,7 +2276,6 @@ def _calculate_block_metrics(
"args_count": args_count,
"big_o_depth": big_o_depth,
"is_recursive": is_recursive,
"db_complexity": db_complexity,
"docstring": docstring,
"logic_angle": round(angle, 2),
"angle": round(angle, 2),
Expand Down
1 change: 0 additions & 1 deletion gitgalaxy/core/network_risk_sensor.py
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,6 @@ def build_dependency_graph(self, parsed_files: list[dict[str, Any]]) -> tuple[li
risk_vector=f.get("risk_vector", [0.0] * len(self.RISK_SCHEMA)),
max_big_o=max_big_o,
is_recursive=is_recursive,
db_complexity=(max([func.get("db_complexity", 0) for func in funcs]) if funcs else 0),
)

# 2. Wire the Edges (File-to-File Level 1 & Entity Level 2)
Expand Down
9 changes: 1 addition & 8 deletions gitgalaxy/metrics/signal_processor.py
Original file line number Diff line number Diff line change
Expand Up @@ -512,7 +512,6 @@ def calculate_risk_vector(
avg_func_args = 0.0
func_gini = 0.0
max_big_o = 1
max_db_complexity = 0

func_ml_brain = getattr(analysis_lens, "GENERAL_FUNCTION_INFERENCE_MODEL", {})
f_medians = func_ml_brain.get("SCALER_MEDIANS", [])
Expand Down Expand Up @@ -553,7 +552,6 @@ def calculate_risk_vector(
max_func_comp = max(complexities)
avg_func_args = sum([f.get("args", 0) for f in functions]) / len(functions)
max_big_o = max([f.get("big_o_depth", 1) for f in functions])
max_db_complexity = max([f.get("db_complexity", 0) for f in functions])
has_recursion = any([f.get("is_recursive", False) for f in functions])

# 1. Z-Scores Mathematics
Expand Down Expand Up @@ -852,7 +850,6 @@ def calculate_risk_vector(
"max_algorithmic_complexity": (
"O(2^N) [Recursive]" if has_recursion else (f"O(N^{max_big_o})" if max_big_o > 1 else "O(N)")
),
"max_db_complexity": max_db_complexity,
"ownership_entropy": ownership_score,
"author_distribution": silo_exposure,
"ownership": dominant_author,
Expand Down Expand Up @@ -2199,11 +2196,7 @@ def _calc_algorithmic_dos(
# 1. The Base Threat (Exponential decay of performance)
func_threat = float(depth**2)

# 2. The Amplifiers (Network & Data Gravity)
db_complex = func.get("db_complexity", 0)
if db_complex > 0:
func_threat *= 1.0 + (db_complex * 0.5)

# 2. The Amplifiers (Network Chokepoints)
hv = func.get("hit_vector", {})
api_hits = hv.get("api", 0)
io_hits = hv.get("io", 0) + hv.get("sec_io", 0)
Expand Down
30 changes: 5 additions & 25 deletions gitgalaxy/recorders/llm_recorder.py
Original file line number Diff line number Diff line change
Expand Up @@ -555,19 +555,6 @@ def _build_markdown(
lines.append(f" * *Intent:* {clean_doc}")
lines.append("")

sorted_by_db = sorted(all_functions, key=lambda x: x[0].get("db_complexity", 0), reverse=True)
db_functions = [s for s in sorted_by_db if s[0].get("db_complexity", 0) > 0]

if db_functions:
lines.append("### Highest Data Gravity (Database Complexity)")
for f, file_path in db_functions[:10]:
lines.append(f"- `{f.get('name')}` (@ `{file_path}`) -> DB Complexity: **{f.get('db_complexity', 0)}**")
doc = f.get("docstring", "").strip()
if doc:
clean_doc = " ".join(doc.split())[:150] + ("..." if len(doc) > 150 else "")
lines.append(f" * *Intent:* {clean_doc}")
lines.append("")

# --- 9. DIRECTORY GROUPS ---
lines.append("## 9. DIRECTORY GROUPS (Top 10 Heaviest Modules)")
dir_groups = summary.get("directory_groups", {})
Expand Down Expand Up @@ -924,9 +911,7 @@ def _build_markdown(
lines.append(
f"- **Magnitude:** {m} | **LOC:** {loc} | **CtrlFlow:** {round(tel.get('control_flow_ratio', 0.0) * 100, 1)}% | **Authorship Centralization:** {round(tel.get('author_distribution', 0.0), 1)}%"
)
lines.append(
f"- **Algorithmic:** {tel.get('max_algorithmic_complexity', 'O(N)')} | **DB Complexity:** {tel.get('max_db_complexity', 0)}"
)
lines.append(f"- **Algorithmic:** {tel.get('max_algorithmic_complexity', 'O(N)')}")
lines.append(f"- **Risk Profile:** Cognitive Load ({cog}%), Tech Debt ({debt}%)")

hv = s.get("hit_vector", [])
Expand All @@ -950,8 +935,7 @@ def _build_markdown(
lines.append("**Top Internal Functions/Classes:**")
for sat in sats:
o_str = "O(2^N)" if sat.get("is_recursive", False) else f"O(N^{sat.get('big_o_depth', 1)})"
db_str = f" | DB: {sat.get('db_complexity', 0)}" if sat.get("db_complexity", 0) > 0 else ""
lines.append(f" * `{sat.get('name')}` (Impact: {sat.get('impact')} | {o_str}{db_str})")
lines.append(f" * `{sat.get('name')}` (Impact: {sat.get('impact')} | {o_str})")
doc = sat.get("docstring", "").strip()
if doc:
clean_doc = " ".join(doc.split())[:100] + ("..." if len(doc) > 100 else "")
Expand Down Expand Up @@ -1329,7 +1313,6 @@ def _generate_sqlite_graph(
ecosystem_baseline TEXT,
repo_z_score REAL,
max_algorithmic_complexity TEXT,
max_db_complexity INTEGER,
{risk_cols}
)
""")
Expand Down Expand Up @@ -1358,7 +1341,6 @@ def _generate_sqlite_graph(
impact REAL,
big_o_depth INTEGER,
is_recursive BOOLEAN,
db_complexity INTEGER,
docstring TEXT,
calls_out_to TEXT,
FOREIGN KEY(artifact_id) REFERENCES artifacts(id)
Expand Down Expand Up @@ -1445,10 +1427,10 @@ def _generate_sqlite_graph(
control_flow_ratio, author_distribution, ownership_entropy,
raw_churn_freq, cog_raw, ownership, popularity,
archetype, global_drift, local_archetype, local_drift,
ecosystem_baseline, repo_z_score, max_algorithmic_complexity, max_db_complexity,
ecosystem_baseline, repo_z_score, max_algorithmic_complexity,
{", ".join(self.RISK_SCHEMA)}
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, {", ".join(["?"] * len(self.RISK_SCHEMA))})
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, {", ".join(["?"] * len(self.RISK_SCHEMA))})
""", # noqa: S608
(
p,
Expand All @@ -1475,7 +1457,6 @@ def _generate_sqlite_graph(
str(repo_macro),
repo_z,
tel.get("max_algorithmic_complexity", "O(N)"),
tel.get("max_db_complexity", 0),
*rv,
),
)
Expand All @@ -1496,7 +1477,6 @@ def _generate_sqlite_graph(
func.get("impact"),
func.get("big_o_depth", 1),
func.get("is_recursive", False),
func.get("db_complexity", 0),
func.get("docstring", ""),
calls_json,
)
Expand All @@ -1512,7 +1492,7 @@ def _generate_sqlite_graph(

cursor.executemany("INSERT INTO dna_hits VALUES (?, ?, ?)", all_dna_data)
cursor.executemany(
"INSERT INTO functions (artifact_id, name, type_id, loc, impact, big_o_depth, is_recursive, db_complexity, docstring, calls_out_to) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
"INSERT INTO functions (artifact_id, name, type_id, loc, impact, big_o_depth, is_recursive, docstring, calls_out_to) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
all_functions,
)
cursor.executemany("INSERT INTO outbound_dependencies VALUES (?, ?)", all_outbound)
Expand Down
8 changes: 2 additions & 6 deletions gitgalaxy/recorders/record_keeper.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,6 @@ def record_mission(
ecosystem_baseline TEXT,
repo_z_score REAL,
max_algorithmic_complexity TEXT,
max_db_complexity INTEGER,
ai_threat_score REAL,
is_malware INTEGER,
has_credentials INTEGER,
Expand Down Expand Up @@ -338,7 +337,6 @@ def record_mission(
func_z_score REAL DEFAULT 0.0,
big_o_depth INTEGER,
is_recursive INTEGER,
db_complexity INTEGER,
docstring TEXT,
calls_out_to TEXT,
token_mass INTEGER DEFAULT 0,
Expand Down Expand Up @@ -613,7 +611,6 @@ def record_mission(
repo_macro,
repo_z,
tel.get("max_algorithmic_complexity", "O(N)"),
int(tel.get("max_db_complexity", 0)),
ai_score,
is_malware,
has_creds,
Expand Down Expand Up @@ -653,7 +650,7 @@ def record_mission(
func_z_max, func_z_mean, func_z_median, pct_z_above_5, pct_z_above_15,
file_archetype, file_fingerprint,
ecosystem_baseline, repo_z_score,
max_algorithmic_complexity, max_db_complexity,
max_algorithmic_complexity,
ai_threat_score, is_malware, has_credentials, binary_anomaly, obfuscation_flag,
token_mass, financial_read_cost, agentic_isolation_risk, requires_hitl, appsec_rce_funnel, appsec_god_mode, appsec_exfiltration, hallucination_zone, silent_mutation_risk,
{", ".join([f"risk_{r.replace('-', '_')}" for r in self.RISK_SCHEMA])},
Expand Down Expand Up @@ -710,7 +707,6 @@ def record_mission(
float(func.get("z_score", 0.0)),
int(func.get("big_o_depth", 1)),
1 if func.get("is_recursive", False) else 0,
int(func.get("db_complexity", 0)),
str(func.get("docstring", ""))[:2000],
json.dumps(func.get("calls_out_to", [])),
(int(func.get("token_mass")) if func.get("token_mass") is not None else None),
Expand All @@ -726,7 +722,7 @@ def record_mission(
cursor.executemany(
f"""
INSERT INTO function_data
(file_id, parent_class_id, func_name, complexity, loc, args, usage_status, keyword_density, func_archetype, func_z_score, big_o_depth, is_recursive, db_complexity, docstring, calls_out_to, token_mass, {", ".join([self.SHORT_KEY_MAP.get(h, h) for h in self.SIGNAL_SCHEMA])})
(file_id, parent_class_id, func_name, complexity, loc, args, usage_status, keyword_density, func_archetype, func_z_score, big_o_depth, is_recursive, docstring, calls_out_to, token_mass, {", ".join([self.SHORT_KEY_MAP.get(h, h) for h in self.SIGNAL_SCHEMA])})
VALUES ({func_placeholders})
""", # noqa: S608
all_func_rows,
Expand Down
2 changes: 1 addition & 1 deletion gitgalaxy/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ click==8.4.2
colorama==0.4.6
contourpy==1.3.3
coverage==7.14.0
cryptography==48.0.1
cryptography==50.0.0
cycler==0.12.1
decorator==5.2.1
defusedxml==0.7.1
Expand Down
Loading
Loading