Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- In Dart, a call written through an import prefix, like `http.get(…)` after `import 'package:http/http.dart' as http;`, or `utils.Listener()` and `kit.Box.create()` through a prefix of your own, now links to what the library imported with that prefix exports, following its `export` chains, `show` and `hide`, and to nothing when that library is a package outside your project. Before, the prefix was taken for a variable, so `http.get(…)` could link to an unrelated `Http` class's `get` method, and calls made through a prefix into your own libraries were missing from `callers` and impact. Re-index Dart projects after upgrading.
- `codegraph sync` and the file watcher now link an import to the file it names when that file is added after the importing file was indexed, or comes back after being deleted: `import 'package:app/b.dart';` in Dart, `import './side'` in JavaScript or TypeScript, `#include "net/socket.h"` in C and C++, `require 'inc/db.php'` in PHP, `import pkg.mod` in Python, a Lua `require`, or a C# `using` of a namespace declared later. Before, the import stayed unlinked until the importing file itself changed or the project was re-indexed, so `codegraph affected` and the files listed as using the new file missed it. Re-index after upgrading.
- JSON files in a `templates` or `sections` folder are now read as Shopify Liquid only inside a Shopify theme, where the folder that contains `templates` or `sections` also has `layout/theme.liquid` or `config/settings_schema.json`. Before, all such JSON was, so a .NET project template's `template.json`, a schematic or a CMS's content showed up in the index as an empty Liquid file, and one that lists `sections` could be linked to an unrelated Liquid file of the same name. Shopify themes, including one kept in a subfolder of a repository, are indexed as before, and the next sync removes JSON that an earlier version indexed this way.
- In a repository that holds more than one Shopify theme, a theme's `{% render %}`, `{% include %}` and `{% section %}` tags and the sections its JSON templates list now link only to that theme's own snippets and sections, which is where Shopify looks for them. Before, a section or snippet the theme doesn't have was linked to another theme's file of the same name, and a theme kept in a subfolder could link to the root theme's copy instead of its own. A reference the theme can't satisfy now links to nothing. Projects with a single theme are indexed as before. Re-index projects with several Shopify themes after upgrading.
- In C, C++ and Objective-C, a struct or class member declared as a pointer, reference, array or function, like `SharedState* shared;`, `jv elements[];` or `virtual Status Put(…) = 0;`, no longer makes the struct look like it inherits from the member's type. Before, `codegraph_explore` and impact listed those types as base classes, every struct holding a pointer to a type as one of its subclasses, and a method of that struct as an override of the type's method with the same name. Real base classes like `class Derived : public Base`, Objective-C superclasses and Go's embedded fields are unchanged. Re-index C, C++ and Objective-C projects after upgrading.
- Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading.

Expand Down
78 changes: 77 additions & 1 deletion __tests__/extraction.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import * as os from 'os';
import { execFileSync } from 'child_process';
import { CodeGraph } from '../src';
import { extractFromSource, scanDirectory, scanDirectoryAsync, buildDefaultIgnore, discoverEmbeddedRepoRoots, buildScopeIgnore, type ScanSkipStats } from '../src/extraction';
import { detectLanguage, isLanguageSupported, getSupportedLanguages, initGrammars, loadAllGrammars, isSourceFile } from '../src/extraction/grammars';
import { detectLanguage, isLanguageSupported, getSupportedLanguages, initGrammars, loadAllGrammars, isSourceFile, shopifyThemeRoot } from '../src/extraction/grammars';
import { stripCppTemplateArgs, blankCppExportMacros, blankCppInlineMacros, blankMetalAttributes, blankCudaConstructs, blankCppAnnotationMacroCalls, blankCppApiPrefixMacros, blankCppInlineAnnotationMacros, blankCLeadingAttrMacros, recoverMangledCppName } from '../src/extraction/languages/c-cpp';
import { normalizePath } from '../src/utils';
import { generateNodeId } from '../src/extraction/tree-sitter-helpers';
Expand Down Expand Up @@ -7542,6 +7542,82 @@ describe('Liquid Shopify JSON template section resolution', () => {
expect(detectLanguage('templates/product.json', undefined, { '.json': 'yaml' }, tempDir)).toBe('yaml');
expect(detectLanguage('shop/templates/product.json', undefined, { '.json': 'yaml' }, tempDir)).toBe('liquid');
});

const writeFiles = (files: Record<string, string>): void => {
for (const [file, text] of Object.entries(files)) {
fs.mkdirSync(path.dirname(path.join(tempDir, file)), { recursive: true });
fs.writeFileSync(path.join(tempDir, file), text);
}
};
// A theme at the project root, a theme in a folder marked only by its settings
// schema, and one in a folder with no 404 section and no price snippet of its
// own — the shape of panoply/syncify's example themes.
const severalThemes = {
'layout/theme.liquid': `{% section 'header' %}{{ content_for_layout }}\n`,
'sections/header.liquid': `{% render 'price' %}\n`,
'sections/main.liquid': `<main></main>\n`,
'snippets/price.liquid': `{{ product.price | money }}\n`,
'straps/dusk/config/settings_schema.json': '[]',
'straps/dusk/sections/404.liquid': `<h1>{{ 'templates.404.title' | t }}</h1>\n`,
'straps/dusk/templates/404.json': jsonTemplate('404'),
'examples/tailwind/layout/theme.liquid': `{% section 'header' %}{{ content_for_layout }}\n`,
'examples/tailwind/sections/header.liquid': `{% liquid\n render 'price'\n render 'icon'\n%}\n`,
'examples/tailwind/sections/main.liquid': `<main class="p-4"></main>\n`,
'examples/tailwind/snippets/icon.liquid': `<svg></svg>\n`,
'examples/tailwind/templates/index.json': jsonTemplate('main'),
'examples/tailwind/templates/404.json': jsonTemplate('404'),
// Liquid in no theme: an app's theme extension.
'extensions/reviews/blocks/stars.liquid': `{% render 'star' %}\n`,
'extensions/reviews/snippets/star.liquid': `<svg></svg>\n`,
};

it("links a theme's sections and snippets only inside that theme", async () => {
// Shopify looks a `{% render %}`, a `{% section %}` or a JSON template's
// section `type` up in the theme the file belongs to, never in another one.
// Matched by path tail, a theme without the 404 section linked to another
// theme's, and a theme in a folder even linked the project root's
// sections/header.liquid over its own.
writeFiles(severalThemes);
cg = CodeGraph.initSync(tempDir);
await cg.indexAll();
cg.resolveReferences();

const linksOf = (file: string): string[] => cg.getFileDependencies(file).sort();
expect(linksOf('layout/theme.liquid')).toEqual(['sections/header.liquid']);
expect(linksOf('sections/header.liquid')).toEqual(['snippets/price.liquid']);
expect(linksOf('straps/dusk/templates/404.json')).toEqual(['straps/dusk/sections/404.liquid']);
expect(linksOf('examples/tailwind/layout/theme.liquid')).toEqual(['examples/tailwind/sections/header.liquid']);
expect(linksOf('examples/tailwind/templates/index.json')).toEqual(['examples/tailwind/sections/main.liquid']);
// What the theme lacks stays unlinked: no other theme's 404 section or price snippet.
expect(linksOf('examples/tailwind/templates/404.json')).toEqual([]);
expect(linksOf('examples/tailwind/sections/header.liquid')).toEqual(['examples/tailwind/snippets/icon.liquid']);
// Outside any theme, a path still resolves as before.
expect(linksOf('extensions/reviews/blocks/stars.liquid')).toEqual(['extensions/reviews/snippets/star.liquid']);
});

it("keeps a theme's references inside it when its own section is deleted", async () => {
// Sync resolves again what named a deleted file; another theme's 404
// section is no stand-in for the theme's own.
writeFiles({ ...severalThemes, 'examples/tailwind/sections/404.liquid': `<h1>404</h1>\n` });
cg = CodeGraph.initSync(tempDir);
await cg.indexAll();
expect(cg.getFileDependencies('examples/tailwind/templates/404.json')).toEqual(['examples/tailwind/sections/404.liquid']);

fs.rmSync(path.join(tempDir, 'examples/tailwind/sections/404.liquid'));
await cg.sync();
expect(cg.getFileDependencies('examples/tailwind/templates/404.json')).toEqual([]);
});

it('finds the theme a file is in from the theme folder it sits in', () => {
const markers = new Set(['layout/theme.liquid', 'examples/tailwind/config/settings_schema.json']);
const exists = (relativePath: string): boolean => markers.has(relativePath);
expect(shopifyThemeRoot('sections/header.liquid', exists)).toBe('');
expect(shopifyThemeRoot('templates/customers/login.json', exists)).toBe('');
expect(shopifyThemeRoot('examples/tailwind/snippets/icon.liquid', exists)).toBe('examples/tailwind');
// A copy kept elsewhere under the root theme is not part of it, nor is a file beside its folders.
expect(shopifyThemeRoot('vendor/legacy/snippets/icon.liquid', exists)).toBeUndefined();
expect(shopifyThemeRoot('theme.liquid', exists)).toBeUndefined();
});
});

describe('Lua/Luau require resolution', () => {
Expand Down
34 changes: 27 additions & 7 deletions src/extraction/grammars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,31 @@ export function isSourceFile(filePath: string, overrides?: Record<string, Langua
*/
const SHOPIFY_THEME_MARKERS = ['layout/theme.liquid', 'config/settings_schema.json'];

/** The folders a Shopify theme is made of. */
const SHOPIFY_THEME_FOLDERS = /^(assets|blocks|config|layout|locales|sections|snippets|templates)$/i;

/**
* The root of the Shopify theme `filePath` is in: the directory holding the
* theme folder (one `folders` matches) the file sits in, when that directory
* also holds one of `SHOPIFY_THEME_MARKERS`. Project-relative and
* `/`-separated, '' for the project root; undefined when the file is in no
* theme. `exists` says whether a project-relative path exists. The folder
* nearest the file is tried first.
*/
export function shopifyThemeRoot(
filePath: string,
exists: (relativePath: string) => boolean,
folders: RegExp = SHOPIFY_THEME_FOLDERS,
): string | undefined {
const segments = filePath.split('/');
for (let i = segments.length - 2; i >= 0; i--) {
if (!folders.test(segments[i]!)) continue;
const themeDir = segments.slice(0, i).join('/');
if (SHOPIFY_THEME_MARKERS.some((marker) => exists(themeDir ? `${themeDir}/${marker}` : marker))) return themeDir;
}
return undefined;
}

/**
* Shopify OS 2.0 JSON template (`templates/*.json`) or section group
* (`sections/*.json`) — these reference sections by `"type"`, so the Liquid
Expand All @@ -279,13 +304,8 @@ export function isShopifyLiquidJson(filePath: string, rootDir?: string): boolean
// Allow nested template dirs (`templates/customers/login.json`), not just
// top-level (`templates/product.json`).
if (rootDir === undefined || !/(^|\/)(templates|sections)\/.+\.json$/i.test(filePath)) return false;
const segments = filePath.split('/');
for (let i = 0; i < segments.length - 1; i++) {
if (!/^(templates|sections)$/i.test(segments[i]!)) continue;
const themeDir = path.join(rootDir, ...segments.slice(0, i));
if (SHOPIFY_THEME_MARKERS.some((marker) => fs.existsSync(path.join(themeDir, marker)))) return true;
}
return false;
const exists = (relativePath: string): boolean => fs.existsSync(path.join(rootDir, relativePath));
return shopifyThemeRoot(filePath, exists, /^(templates|sections)$/i) !== undefined;
}

/**
Expand Down
6 changes: 6 additions & 0 deletions src/resolution/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import { clearDartLibraryMemos } from './dart-libraries';
import { clearVbnetReceiverMemos, isVbMemberRead, isVbPathCall, matchVbMemberRead, matchVbPathCall } from './vbnet-receivers';
import { gateTypeParameter, clearTypeParameterMemos } from './type-parameters';
import { gateDartLocal, clearDartLocalScopeMemos } from './dart-local-scope';
import { matchShopifyThemeFile } from './shopify-themes';
import { resolveViaImport, resolvePhpImportedStaticCall, resolvePhpQualifiedClassRef, resolveJvmImport, extractImportMappings, extractReExports, loadCppIncludeDirs, isPhpIncludePathRef, isCobolCopybookRef, isNixPathImportRef, isDartImportRef, isLuaRequireRef, isJsPathImportRef, isBoundToOutOfRepoImport, clearImportResolverMemos, resolveImportPath, isExternalImport } from './import-resolver';
import { ResolverPool, minRefsForPool, shouldEngageAdaptively } from './resolver-pool';
import { resolveAliasBinding } from './alias-binding';
Expand Down Expand Up @@ -1153,6 +1154,11 @@ export class ReferenceResolver {
// constructor call, as written — never a method or function by its name.
if (isDartAnnotation(ref)) return matchDartAnnotation(ref, this.context);

// A section or snippet a Shopify theme's file names is that theme's own,
// or nothing: Shopify never looks in another theme (see ./shopify-themes).
const themeFile = matchShopifyThemeFile(ref, this.context);
if (themeFile !== undefined) return themeFile;

// Skip built-in/external references
if (this.isBuiltInOrExternal(ref)) {
return null;
Expand Down
39 changes: 39 additions & 0 deletions src/resolution/shopify-themes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/**
* A Shopify theme's section and snippet references → that theme's own files.
*
* `{% render 'price' %}` renders the `snippets/price.liquid` of the theme the
* file belongs to; `{% section 'header' %}` and a JSON template's section
* `"type"` name a file in that theme's `sections/`. Shopify never looks in
* another theme. The Liquid extractor records these as `snippets/<name>.liquid`
* and `sections/<name>.liquid`, and the path matcher took any file whose path
* ends with that, so in a repository holding several themes (a theme tool's
* examples, a store's seasonal themes) a theme missing a section linked to
* another theme's copy of it.
*
* From a file inside a theme (see `shopifyThemeRoot`), these references
* resolve to the theme's own file or to nothing, and never fall through to the
* generic name strategies. A reference from a file in no theme resolves as
* before.
*/
import { shopifyThemeRoot } from '../extraction/grammars';
import type { ResolvedRef, ResolutionContext, UnresolvedRef } from './types';

/**
* The file a section or snippet reference made inside a Shopify theme names:
* the theme's own `sections/<name>.liquid` or `snippets/<name>.liquid`.
* undefined means the reference is not one, or is made outside any theme;
* null means the theme has no such file, so no other strategy may guess one.
*/
export function matchShopifyThemeFile(ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null | undefined {
if (ref.language !== 'liquid' || !/^(sections|snippets)\//.test(ref.referenceName)) return undefined;
const theme = shopifyThemeRoot(ref.filePath, (relativePath) => context.fileExists(relativePath));
if (theme === undefined) return undefined;
const target = theme === '' ? ref.referenceName : `${theme}/${ref.referenceName}`;
const file = context
.getNodesByName(target.slice(target.lastIndexOf('/') + 1))
.find((n) => n.kind === 'file' && n.filePath === target);
if (!file) return null;
// The confidence the path matcher gives the same link: an exact path for a
// theme at the project root, a path tail for one in a folder.
return { original: ref, targetNodeId: file.id, confidence: theme === '' ? 0.95 : 0.85, resolvedBy: 'file-path' };
}