Implementing Temporal Indexing Via MCP: A Detailed Guide

by Alex Johnson 57 views

This article delves into the implementation of temporal indexing support within the MCP (Management Control Plane) framework. Temporal indexing, also known as Git history search, allows users to search through the historical versions of code repositories. This capability is crucial for debugging, auditing, and understanding the evolution of codebases. This article outlines the requirements, design, implementation status, and testing strategy for adding temporal indexing functionality via the MCP API.

Story: Temporal Indexing MCP Support

As a CIDX server administrator, the goal is to configure temporal indexing (Git history search) via the MCP API when registering golden repositories. This enables Git history search capabilities without requiring direct CLI access, simplifying the management and configuration process. The central idea is to empower administrators with a more streamlined and user-friendly method to enable temporal indexing for golden repositories, enhancing the overall search capabilities of the CIDX server. This enhancement is crucial for maintaining control and visibility over code changes over time, which is essential for compliance and efficient software development practices.

Implementation Status and Progress Tracking

To ensure a structured approach to the implementation, several tasks have been identified and tracked. The current status of these tasks provides a clear overview of the project's progress:

  • [ ] MCP Tool Schema (tools.py) - Add enable_temporal and temporal_options parameters to add_golden_repo tool
  • [ ] MCP Handler (handlers.py) - Pass temporal parameters from add_golden_repo handler to GoldenRepoManager
  • [ ] GlobalRegistry (global_registry.py) - Store and persist temporal settings alongside repo metadata
  • [ ] GlobalActivator (global_activation.py) - Pass temporal settings through the activation workflow
  • [ ] RefreshScheduler (refresh_scheduler.py) - Read temporal settings from registry and use them during cidx index --index-commits execution
  • [ ] Unit tests for MCP temporal parameter handling
  • [ ] Integration tests for end-to-end temporal indexing via MCP
  • [ ] E2E manual testing completed by Claude Code

Completion: 0/8 tasks complete (0%)

This detailed task list ensures that each aspect of the implementation is addressed systematically. Tracking progress against these tasks helps maintain momentum and ensures that all critical components are implemented correctly. The use of a checklist also facilitates clear communication among the development team and stakeholders.

Algorithm: Step-by-Step Implementation Details

The algorithm for implementing temporal indexing via MCP involves several key steps, each addressing a specific component of the system. Here’s a breakdown of the algorithm:

MCP add_golden_repo Tool Schema Update

The first step involves updating the MCP tool schema to include parameters for enabling temporal indexing. This ensures that the MCP API can accept requests to configure temporal indexing when registering a golden repository.

  • Add to inputSchema.properties:
    • enable_temporal: boolean, default false
    • temporal_options: object
      • max_commits: integer (optional)
      • since_date: string, format YYYY-MM-DD (optional)
      • diff_context: integer, default 5 (optional)

MCP add_golden_repo Handler

The handler is responsible for extracting temporal parameters from the MCP request and passing them to the GoldenRepoManager. This ensures that the temporal indexing settings are correctly propagated.

  • EXTRACT enable_temporal from params (default False)
  • EXTRACT temporal_options from params (default None)
  • CALL golden_repo_manager.add_golden_repo with:
    • repo_url=params["url"]
    • alias=params["alias"]
    • default_branch=params.get("branch", "main")
    • enable_temporal=enable_temporal
    • temporal_options=temporal_options
    • submitter_username=user.username
  • RETURN job_id response

GlobalRegistry.register_global_repo()

The GlobalRegistry is updated to store temporal settings alongside other repository metadata. This ensures that the settings are persisted and can be retrieved later.

  • ADD enable_temporal to registry_data[alias_name]
  • ADD temporal_options to registry_data[alias_name]
  • SAVE registry atomically

GlobalActivator.activate_golden_repo()

The activator is responsible for passing the temporal settings during the repository activation process. This ensures that the settings are applied when the repository is registered.

  • RECEIVE enable_temporal and temporal_options
  • PASS to registry.register_global_repo()

RefreshScheduler._create_new_index()

The refresh scheduler is updated to read temporal settings from the registry and use them during the cidx index --index-commits execution. This ensures that temporal indexing is correctly configured during scheduled refreshes.

  • READ repo_info from registry
  • EXTRACT enable_temporal = repo_info.get("enable_temporal", False)
  • EXTRACT temporal_options = repo_info.get("temporal_options")
  • BUILD index_command = ["cidx", "index", "--fts"]
  • IF enable_temporal:
    • APPEND "--index-commits" to index_command
    • IF temporal_options:
      • IF temporal_options.max_commits:
        • APPEND ["--max-commits", str(max_commits)]
      • IF temporal_options.since_date:
        • APPEND ["--since-date", since_date]
      • IF temporal_options.diff_context:
        • APPEND ["--diff-context", str(diff_context)]
  • EXECUTE index_command

This comprehensive algorithm ensures that temporal indexing can be enabled and configured via the MCP API, providing a streamlined approach for managing golden repositories. Each step is designed to integrate seamlessly with the existing system components, ensuring a robust and maintainable implementation.

Acceptance Criteria: Ensuring Quality and Functionality

To ensure that the implementation meets the required standards, a set of acceptance criteria has been defined using the Gherkin syntax. These criteria cover various scenarios and ensure that the temporal indexing functionality works as expected.

Given I have an MCP client connected to the CIDX server
When I call add_golden_repo with enable_temporal=true and temporal_options containing max_commits and diff_context
Then the golden repository should be registered successfully
And the repository should be indexed with --index-commits flag and the specified temporal options
And temporal search queries should return git history results for that repository

Given I have a golden repository registered with temporal indexing enabled
When the RefreshScheduler triggers a refresh for that repository
Then the refresh should include --index-commits flag and the stored temporal options
And temporal search should continue working after the refresh

Given I call add_golden_repo without enable_temporal parameter
When the indexing completes
Then the repository should be indexed without temporal indexing (backward compatible default)

Given I call get_repository_status or discover_repositories for a temporally-indexed repository
When the response is returned
Then it should include enable_temporal and temporal_options fields reflecting the stored configuration

These acceptance criteria cover the core functionalities of temporal indexing via MCP, including registering repositories with temporal indexing enabled, refreshing repositories while preserving temporal settings, and ensuring backward compatibility. Each criterion is designed to test a specific aspect of the implementation, providing a comprehensive assessment of its functionality.

Testing Requirements: A Multi-faceted Approach

To ensure the quality and reliability of the temporal indexing implementation, a multi-faceted testing approach is adopted. This includes unit tests, integration tests, end-to-end (E2E) tests, and manual testing. Each type of test serves a specific purpose and together they provide a comprehensive validation of the implementation.

Unit Tests

Unit tests focus on individual components and functions to ensure they work as expected. These tests are crucial for identifying and fixing issues early in the development process.

  • Test MCP tools.py schema includes enable_temporal and temporal_options in add_golden_repo inputSchema
  • Test MCP handlers.py passes enable_temporal and temporal_options to GoldenRepoManager.add_golden_repo
  • Test GlobalRegistry stores and retrieves enable_temporal and temporal_options
  • Test RefreshScheduler._create_new_index builds correct cidx index command with temporal flags

Integration Tests

Integration tests verify the interaction between different components of the system. These tests ensure that the components work together correctly.

  • Test end-to-end flow: MCP add_golden_repo with temporal options -> indexing -> temporal query
  • Test refresh cycle preserves temporal settings and uses them during re-indexing

E2E Tests

End-to-end tests simulate real-world scenarios and validate the entire workflow. These tests ensure that the system works correctly from the user's perspective.

  • Execute MCP add_golden_repo tool with temporal parameters via real server
  • Verify repository is indexed with temporal support (check for temporal index artifacts)
  • Execute temporal search query and verify results contain Git history data

Manual Testing

Manual testing involves human testers who interact with the system and verify its functionality. This type of testing is valuable for identifying usability issues and other problems that may not be detected by automated tests.

  • Use MCP client to register a repository with enable_temporal=true
  • Poll get_job_statistics until indexing completes
  • Execute search_code with time_range_all=true and verify temporal results
  • Trigger refresh and verify temporal indexing is preserved

This comprehensive testing strategy ensures that all aspects of the temporal indexing implementation are thoroughly validated, providing confidence in its quality and reliability.

Definition of Done: Clear Expectations for Completion

To ensure that the implementation is considered complete and ready for deployment, a clear definition of done (DoD) has been established. This DoD outlines the criteria that must be met before the implementation is considered finished.

  • [ ] All acceptance criteria satisfied
  • [ ] >90% unit test coverage achieved for new/modified code
  • [ ] Integration tests passing
  • [ ] E2E tests with zero mocking passing
  • [ ] Code review approved (tdd-engineer + code-reviewer workflow)
  • [ ] Manual end-to-end testing completed by Claude Code
  • [ ] No lint/type errors (./lint.sh passes)
  • [ ] Working software deployable to users

This DoD provides a clear set of expectations for the team and ensures that the implementation meets the required quality standards before it is considered complete. Each criterion is measurable and verifiable, providing a clear indication of progress and completion status.

Technical Details and Conversation References: Deep Dive into the Implementation

Requirement Sources (Conversation Context)

The requirements for implementing temporal indexing via MCP have been derived from several sources, including conversations and discussions. These sources provide valuable context and insights into the specific needs and constraints of the implementation.

  1. MCP Tool Gap Identified: The conversation identified that GoldenRepoManager.add_golden_repo() already supports enable_temporal and temporal_options parameters at lines 146-155 of golden_repo_manager.py, but these are NOT exposed via MCP API.
  2. Components Requiring Updates (from conversation analysis):
    • tools.py: Missing temporal params in add_golden_repo inputSchema (lines 667-699)
    • handlers.py: add_golden_repo handler (lines 680-701) does not pass temporal params
    • global_registry.py: Does not store temporal settings (see register_global_repo at lines 109-161)
    • global_activation.py: Does not receive/pass temporal settings (see activate_golden_repo at lines 46-104)
    • refresh_scheduler.py: _create_new_index (lines 251-434) does not read temporal settings from registry
  3. Temporal Indexing is Already Incremental: The conversation confirmed that TemporalProgressiveMetadata handles incremental temporal indexing automatically - no additional work needed for incremental behavior.
  4. Required MCP Parameters (from conversation):
    • enable_temporal (boolean, default false)
    • temporal_options (object with: max_commits, since_date, diff_context)

Current Implementation Analysis

To better understand the changes required, an analysis of the existing implementation was conducted. This analysis focused on identifying the components that need to be modified and the extent of the changes required.

GoldenRepoManager.add_golden_repo (lines 146-293) already:

  • Accepts enable_temporal: bool = False parameter
  • Accepts temporal_options: Optional[Dict] = None parameter
  • Passes these to _execute_post_clone_workflow (line 205-210)
  • Stores them in GoldenRepo model (lines 228-236)
  • Saves to metadata.json (line 240)

GoldenRepoManager._execute_post_clone_workflow (lines 737-894) already:

  • Builds cidx index --fts --index-commits when enable_temporal=True
  • Handles temporal_options for max_commits, since_date, diff_context

MISSING - MCP Layer:

  • tools.py does not include temporal parameters in schema
  • handlers.py does not extract/pass temporal parameters

MISSING - GlobalRegistry Persistence:

  • register_global_repo does not accept/store temporal settings
  • This prevents RefreshScheduler from knowing about temporal configuration

MISSING - RefreshScheduler:

  • _create_new_index uses hardcoded ["cidx", "index", "--fts"] (line 393)
  • Does not read temporal settings from registry
  • Does not include --index-commits and related flags

Component File Locations

To facilitate the implementation, the file locations of the components that need to be modified have been identified. This helps developers quickly locate the relevant files and make the necessary changes.

Component File Path Lines to Modify
MCP Tool Schema /src/code_indexer/server/mcp/tools.py Lines 667-699 (add_golden_repo)
MCP Handler /src/code_indexer/server/mcp/handlers.py Lines 680-701 (add_golden_repo)
Global Registry /src/code_indexer/global_repos/global_registry.py Lines 109-161 (register_global_repo)
Global Activator /src/code_indexer/global_repos/global_activation.py Lines 46-104 (activate_golden_repo)
Refresh Scheduler /src/code_indexer/global_repos/refresh_scheduler.py Lines 389-408 (_create_new_index)
Golden Repo Manager /src/code_indexer/server/repositories/golden_repo_manager.py Already supports temporal (reference)

Data Flow Diagram

A data flow diagram provides a visual representation of the system's components and their interactions. This diagram helps in understanding the flow of data and the dependencies between different components.

MCP Client
    |
    v
tools.py (add enable_temporal, temporal_options to schema)
    |
    v
handlers.py (extract and pass temporal params)
    |
    v
GoldenRepoManager.add_golden_repo() (ALREADY SUPPORTS temporal)
    |
    +---> _execute_post_clone_workflow() (ALREADY SUPPORTS temporal)
    |         |
    |         v
    |     cidx init && cidx index --fts [--index-commits ...]
    |
    v
GlobalActivator.activate_golden_repo() (ADD temporal param passing)
    |
    v
GlobalRegistry.register_global_repo() (ADD temporal storage)
    |
    v
global_registry.json (temporal settings persisted)
    |
    v (later, during scheduled refresh)
RefreshScheduler._create_new_index() (READ temporal, BUILD command)
    |
    v
cidx index --fts [--index-commits --max-commits N --since-date X --diff-context Y]

Out of Scope: Clarifying What's Not Included

To maintain focus and prevent scope creep, it’s essential to define what is not included in the implementation. This helps in managing expectations and ensuring that the team is aligned on the goals of the project.

  • Temporal settings updates after registration (update existing repo temporal config)
  • REST API temporal support (only MCP requested)
  • CLI changes (already works)
  • Temporal query parameter changes (already implemented in search_code)

Conclusion

Implementing temporal indexing support via MCP is a significant enhancement that will greatly improve the search capabilities of the CIDX server. By following the detailed algorithm, adhering to the acceptance criteria, and employing a comprehensive testing strategy, the team can ensure a high-quality implementation. This article has provided a thorough overview of the requirements, design, and implementation details, setting the stage for a successful project outcome.

For more information on code indexing and related topics, visit the official documentation at Code Indexer Documentation. This external link will provide additional resources and context for those looking to deepen their understanding of the subject matter.