Task: Add code analysis features
Add code analysis features
I've been thinking a lot about different mechanics in this direction, and so far using a TypeScript server seems like the most promising approach. Why? Because then we can operate not just with the files themselves and read them individually, but also analyze their dependencies. For example, we tell the agent to examine some component. It can simply use a special tool to get the entities that are there (variables, functions, imports, exports, etc.), as well as easily request the additional dependencies it needs without worrying about where they are located. But if it needs the final code of the required dependency, it can easily find out which file it is in and read it.
Along the way, we probably need tools for reading files with a specified line range. After all, a code file can be large, and if TS immediately gives us information not only on where it is located, but also in which line range, we don't necessarily have to read the entire file.
Ворклоги
Managed to dig out some old developments, here is a minimal set of useful methods.
query analyzeDir {
analyzeDir(rootDir: "src/components/pages/_App/")
}
{
"data": {
"analyzeDir": {
"rootDir": "src/components/pages/_App",
"count": 3,
"files": [
{
"path": "src/components/pages/_App/getInitialProps.ts"
},
{
"path": "src/components/pages/_App/index.tsx"
},
{
"path": "src/components/pages/_App/interfaces.ts"
}
]
}
}
}
query analyzeFile {
analyzeFile(path: "src/components/pages/_App/getInitialProps.ts") {
path
imports {
default
isExternalLibraryImport
module
namespace
resolvedPath
named {
name
alias
}
}
exports
}
}
{
"data": {
"analyzeFile": {
"path": "src/components/pages/_App/getInitialProps.ts",
"imports": [
{
"default": null,
"isExternalLibraryImport": false,
"module": "src/gql/apolloClient",
"namespace": null,
"resolvedPath": "src/gql/apolloClient/index.ts",
"named": [
{
"name": "initializeApollo",
"alias": null
}
]
},
{
"default": "NextApp",
"isExternalLibraryImport": true,
"module": "next/app",
"namespace": null,
"resolvedPath": null,
"named": []
},
{
"default": null,
"isExternalLibraryImport": false,
"module": "./interfaces",
"namespace": null,
"resolvedPath": "src/components/pages/_App/interfaces.ts",
"named": [
{
"name": "AppInitialProps",
"alias": null
},
{
"name": "MainApp",
"alias": null
},
{
"name": "NextPageContextCustom",
"alias": null
},
{
"name": "PageProps",
"alias": null
},
{
"name": "withWs",
"alias": null
}
]
},
{
"default": null,
"isExternalLibraryImport": false,
"module": "src/helpers/getSiteOrigin",
"namespace": null,
"resolvedPath": "src/helpers/getSiteOrigin.ts",
"named": [
{
"name": "getSiteOrigin",
"alias": null
}
]
}
],
"exports": [
{
"name": "getInitialProps",
"kind": "Function",
"line": 13
}
]
}
}
}
Now the problem is that our TypeScript server starts on every request.
import path from 'path'
import ts from 'typescript'
export interface ProgramContext {
program: ts.Program
checker: ts.TypeChecker
/** Dir base directory (the folder that contains tsconfig.json). */
baseDir: string
}
/**
* Build a TypeScript Program from the project's tsconfig.json.
* Rebuilt per call — TS's incremental cache is internal to a single Program,
* and the host process may pick up file changes between requests.
*/
export function getProgram(): ProgramContext {
const configPath = ts.findConfigFile('./', ts.sys.fileExists, 'tsconfig.json')
if (!configPath) {
throw new Error('tsconfig.json not found')
}
const baseDir = path.resolve(path.dirname(configPath))
const configFile = ts.readConfigFile(configPath, ts.sys.readFile)
const parsed = ts.parseJsonConfigFileContent(
configFile.config,
ts.sys,
baseDir,
)
/*
Here we create the TS server and it's quite a heavy operation,
it not only consumes a lot of CPU and memory, but also introduces noticeable delays
*/
const program = ts.createProgram({
rootNames: parsed.fileNames,
options: parsed.options,
})
return {
program,
checker: program.getTypeChecker(),
baseDir,
}
}
We need to be able to keep longer-lived TS server instances so we don't have to start from scratch every time. There are a few options here (according to opus 4.5):
1. ts.createIncrementalProgram / ts.createWatchProgram
TypeScript's built-in mechanism for incremental compilation:
const host = ts.createWatchCompilerHost(configPath, {}, ts.sys);
const watchProgram = ts.createWatchProgram(host);
const program = watchProgram.getProgram().getProgram();
Caches the AST and recompiles only modified files.
2. ts.server.ProjectService (Language Service)
This is what VS Code uses. A full-fledged language server with caching:
import * as ts from 'typescript/lib/tsserverlibrary'
const projectService = new ts.server.ProjectService({ ... })
Heavier, but gives access to an already running tsserver.
3. Connecting to an existing tsserver via LSP
If tsserver is already running (e.g., in an IDE), you can communicate with it via stdio or node-ipc. But this requires knowing where it is listening.
4. Simple application-level cache
Store program in memory and recreate it only when files change:
let cachedProgram: ts.Program | null = null;
let lastModified: number = 0;
export function getProgram(baseDir: string) {
const currentModified = getMaxMtime(baseDir);
if (cachedProgram && currentModified <= lastModified) {
return {
program: cachedProgram,
checker: cachedProgram.getTypeChecker(),
baseDir,
};
}
}
Recommendation: If you need quick results — Option 4 (simple cache). If you need full IDE integration — Option 2 or 3, but this is significantly more complex.
Refactored the TypeScript server acquisition function. Now the program has its own lifetime (a timer starting from the last request) so that it doesn't hang indefinitely if there are no requests. Now the first cold start takes some time depending on the project, but then each request is processed almost instantly, even across different requests. That is, you can first request a list of files (this is one resolver), and then an analysis of some file (this is another resolver), and the analysis will arrive almost instantly (as well as other subsequent requests).
import path from 'path'
import ts from 'typescript'
export interface ProgramContext {
program: ts.Program
checker: ts.TypeChecker
/** Dir base directory (the folder that contains tsconfig.json). */
baseDir: string
}
export interface GetProgramOptions {
/** Absolute path to project root directory */
projectRoot: string
/** Path to tsconfig.json (relative to projectRoot or absolute). Default: 'tsconfig.json' */
tsconfigPath?: string
}
interface CachedProject {
program: ts.Program
/** File versions for incremental rebuild detection */
fileVersions: Map<string, number>
/** Timestamp of last access */
lastAccess: number
/** Timer for TTL expiration */
expirationTimer: ReturnType<typeof setTimeout>
}
/** TTL in milliseconds (30 minutes) */
const CACHE_TTL_MS = 30 * 60 * 1000
/** Cache of programs per project root */
const projectCache = new Map<string, CachedProject>()
/**
* Get file modification time as version number.
*/
function getFileVersion(filePath: string): number {
try {
const stat = ts.sys.getModifiedTime?.(filePath)
return stat ? stat.getTime() : 0
} catch {
return 0
}
}
/**
* Check if any source files have changed since last build.
*/
function hasChanges(cached: CachedProject): boolean {
for (const [filePath, version] of cached.fileVersions) {
if (getFileVersion(filePath) !== version) {
return true
}
}
return false
}
/**
* Build or retrieve a cached TypeScript Program for the given project.
* Uses incremental compilation — only rebuilds when source files change.
*/
export function getProgram(options: GetProgramOptions): ProgramContext {
const { projectRoot, tsconfigPath = 'tsconfig.json' } = options
const baseDir = path.resolve(projectRoot)
const configPath = path.isAbsolute(tsconfigPath)
? tsconfigPath
: path.join(baseDir, tsconfigPath)
if (!ts.sys.fileExists(configPath)) {
throw new Error(`tsconfig.json not found at ${configPath}`)
}
const cacheKey = configPath
const cached = projectCache.get(cacheKey)
if (cached && !hasChanges(cached)) {
resetExpirationTimer(cacheKey, cached)
return {
program: cached.program,
checker: cached.program.getTypeChecker(),
baseDir,
}
}
const configFile = ts.readConfigFile(configPath, ts.sys.readFile)
const parsed = ts.parseJsonConfigFileContent(
configFile.config,
ts.sys,
baseDir,
)
const program = ts.createProgram({
rootNames: parsed.fileNames,
options: parsed.options,
oldProgram: cached?.program,
})
const fileVersions = new Map<string, number>()
for (const sourceFile of program.getSourceFiles()) {
fileVersions.set(sourceFile.fileName, getFileVersion(sourceFile.fileName))
}
if (cached) {
clearTimeout(cached.expirationTimer)
}
const newCached: CachedProject = {
program,
fileVersions,
lastAccess: Date.now(),
expirationTimer: createExpirationTimer(cacheKey),
}
projectCache.set(cacheKey, newCached)
return {
program,
checker: program.getTypeChecker(),
baseDir,
}
}
/**
* Reset expiration timer on cache access.
*/
function resetExpirationTimer(cacheKey: string, cached: CachedProject): void {
clearTimeout(cached.expirationTimer)
cached.lastAccess = Date.now()
cached.expirationTimer = createExpirationTimer(cacheKey)
}
/**
* Create a timer that removes the cache entry after TTL.
*/
function createExpirationTimer(cacheKey: string): ReturnType<typeof setTimeout> {
return setTimeout(() => {
projectCache.delete(cacheKey)
}, CACHE_TTL_MS)
}
/**
* Clear cached program for a specific project.
*/
export function clearProgramCache(configPath: string): void {
const key = path.resolve(configPath)
const cached = projectCache.get(key)
if (cached) {
clearTimeout(cached.expirationTimer)
projectCache.delete(key)
}
}
/**
* Clear all cached programs.
*/
export function clearAllProgramCaches(): void {
for (const cached of projectCache.values()) {
clearTimeout(cached.expirationTimer)
}
projectCache.clear()
}