Enhance DevAC Seeds For Deeper Service Analysis

h1. Enhancing DevAC Seeds for Comprehensive Service Behavior Analysis

Understanding service behavior is crucial in complex software systems. When tasked with explaining how a service like miami operates using the DevAC tool, the initial results were promising but ultimately fell short. While DevAC seeds provided a good structural overview, a significant portion of the analysis – around 70% – still required manual digging through the file system. This means that while we could see the what (classes, interfaces, functions), we struggled to grasp the how and why (business logic, data flow, infrastructure interactions). The goal is to enrich these DevAC seeds so that the tool can provide a much more complete picture, significantly reducing the need for manual file system fallback and improving the overall developer experience. Imagine asking DevAC to explain miami and getting a near-complete answer, complete with diagrams, logic flows, and interaction details, all generated directly from the tool. This article explores the identified issues and proposes concrete improvements to achieve this goal.

The Gap: Structure vs. Behavior

When we ask DevAC to explain how the miami service works, the original request specifically mentioned diagrams and interactions with users and other systems. The seeds provided by DevAC were excellent at detailing the structure of the service. We received a comprehensive list of classes, interfaces, and functions, along with their file paths and basic structural relationships like CONTAINS (parent-child) and EXTENDS (inheritance). We also got a clear picture of the external dependencies, such as aws-cdk-lib, tsoa, and various AWS SDK components. This foundational information, representing about 30% of a complete answer, is valuable. It gives us a map of the components that make up the service. However, the remaining 70% of understanding – the actual behavior of the service – was missing. This includes critical details like the core business logic, the specifics of API endpoints (routes, HTTP methods, parameters), the intricate data flows between different components, and the infrastructure relationships (e.g., how a Lambda function communicates with SQS and SNS). Crucially, it also lacked information on how to use the service, such as M2M (machine-to-machine) examples. Without this behavioral context, the structural information alone is insufficient for a true understanding of the service's operation. The current state means developers have to manually trace code, consult documentation (if available and up-to-date), and infer interactions, a time-consuming and error-prone process. The aim is to bridge this gap by making DevAC seeds richer, enabling the tool to provide both structure and behavior analysis.

Identified Issues with Current DevAC Seeds

The current DevAC seed generation process has several key limitations that prevent it from fully capturing service behavior. These issues create friction and necessitate extensive manual work, undermining the tool's potential. Let's break them down:

1. Absence of CALLS Edges

Perhaps the most significant omission is the lack of CALLS edges. DevAC seeds currently only capture CONTAINS and EXTENDS relationships. These are static, structural relationships. To understand how a service behaves, we need to know how its functions and components interact dynamically. Function call relationships are the backbone of this dynamic behavior. Without CALLS edges, it's impossible to trace the flow of data or understand the sequence of operations that occur when a request is processed. This makes analyzing logic, debugging, and understanding dependencies extremely difficult.

2. User Experience Friction in Queries

Querying the data within DevAC seeds is cumbersome. Currently, users need to specify the full path to the Parquet files, like read_parquet('/full/path/to/.devac/seed/base/nodes.parquet'). This is not user-friendly and requires knowledge of the tool's internal storage structure. A much smoother experience would be a simplified syntax, such as SELECT ... FROM nodes, where the tool automatically handles loading the relevant data tables. This improved query UX is essential for making DevAC accessible and efficient for everyday use.

3. Lack of a Hub-Wide Query Command

In a microservices environment, understanding how services interact across different repositories or packages is vital. The absence of a devac hub query command means there's no straightforward way to query across all registered packages in the hub. This federated querying capability is crucial for discovering service dependencies, identifying potential conflicts, and understanding the broader system architecture. Without it, cross-repository analysis remains a manual and fragmented effort.

4. Missing Semantic Context

DevAC seeds currently focus on syntactic and structural information, neglecting semantic context. They tell us that a function exists, but not what it does. Information like docstrings, JSDoc comments, or any form of descriptive text that explains the purpose and functionality of code elements is not captured. This semantic void makes it difficult to understand the intent behind the code, hindering comprehension and analysis. Richer seeds would include these textual descriptions, allowing DevAC to provide more meaningful explanations.

5. Inability to Extract API Route Information

For many services, especially those built with frameworks like tsoa or Express, API routes are a primary interface. The current seeds do not extract metadata from route decorators (e.g., @Route, @Get, @Post). This means DevAC cannot readily provide information about the service's public API surface, such as available endpoints, HTTP methods, and expected parameters. This is a critical piece of missing information for understanding how external users or other services interact with miami.

6. No Cross-Package Relationship Tracking

Related to the missing hub-wide query command, there's a general inability to track cross-package relationships effectively. For instance, if we want to know which other services call miami's M2M endpoint, DevAC, in its current state, cannot answer this. This capability is essential for building comprehensive dependency maps and understanding the impact of changes across the system.

Proposed Improvements for Enhanced DevAC Seeds

To address the identified issues and significantly improve DevAC's ability to analyze service behavior, we propose the following enhancements. These improvements are categorized by priority, focusing on the highest impact changes first.

High Priority Improvements

• Add CALLS Edges to Track Function Invocations: This is the most critical enhancement. By capturing function call relationships, DevAC can begin to construct dynamic call graphs, enabling detailed data flow analysis and a much deeper understanding of how components execute and interact. This single addition would dramatically increase the analytical power of the tool.
• Simplify Query Syntax (Auto-load Parquet Tables): Improving the developer experience (DX) is key to adoption. By automatically loading the Parquet tables and allowing simpler query syntax like SELECT ... FROM nodes, DevAC becomes much more accessible. This reduces the barrier to entry for using the tool and makes data exploration significantly faster and more intuitive.

Medium Priority Improvements

• Implement devac hub query for Cross-Repo Queries: A hub-wide query command is essential for understanding the ecosystem. This allows for federated analysis, enabling users to ask questions like "Which services depend on this library?" or "Where is this specific API endpoint used across the organization?". This unlocks the true value of a centralized code analysis hub.
• Extract Docstrings/JSDoc Comments: Incorporating semantic context is vital. By extracting docstrings and other forms of code comments, DevAC can provide explanations for what functions and classes do, not just that they exist. This moves the tool from a structural analyzer to a semantic understanding engine.
• Extract API Route Decorators (tsoa, Express): For services that expose APIs, understanding the API surface is paramount. Extracting route information from decorators in frameworks like tsoa and Express will allow DevAC to automatically document and analyze API endpoints, making it easier to understand how to interact with services and what they offer.

Low Priority Improvements

• Track Cross-Package M2M/API Calls: While related to hub queries, this focuses on specific types of inter-service communication. Explicitly tracking M2M and API calls between different packages will build more accurate and detailed service dependency maps, crucial for microservice architecture management.

By implementing these improvements, DevAC can transform from a tool that provides basic structural information into a powerful engine for comprehensive service behavior analysis. This will save developers significant time, reduce errors, and foster a deeper understanding of our codebase.

Acceptance Criteria for Enhanced DevAC Seeds

To ensure that the proposed improvements effectively address the identified issues and meet the goals for enhanced service behavior analysis, we've defined specific acceptance criteria. These criteria provide measurable outcomes that will confirm the success of the enhancements.

• `devac query "SELECT * FROM nodes" works without full parquet path: This criterion directly addresses the user experience friction in querying. It verifies that the simplified query syntax is implemented and functional, allowing users to query data tables without needing to know or specify the underlying file paths. This is a key step in making DevAC more accessible and efficient for developers.
• CALLS edge type populated for function invocations: This is a fundamental requirement for understanding service behavior. It confirms that the seed generation process now captures dynamic function call relationships. This enables DevAC to build call graphs and trace data flow, moving beyond static structural analysis.
• devac hub query command exists for federated queries: This criterion validates the implementation of the hub-wide query functionality. It ensures that developers can perform cross-repository or cross-package queries, which is essential for understanding system-wide dependencies and interactions in a microservices environment.
• API routes extractable from tsoa/express services: This acceptance criterion confirms that DevAC can now identify and extract API endpoint information from common web frameworks like tsoa and Express. This includes details about routes, HTTP methods, and potentially parameters, providing crucial insights into a service's public interface.

Meeting these criteria will signify that DevAC seeds have been significantly enhanced, providing a much more complete and actionable view of service behavior. This will dramatically improve the ability to analyze, understand, and document our complex software systems.

---