Memory Usage History Tracking in TASM

[csmangum]

Implement a robust memory usage tracking mechanism in TASM that preserves retrieval history with each memory while enabling relationship analysis between memories. This approach maintains the integrity of the vector database while adding rich metadata about memory interactions.

Problem Statement

When memory 234 retrieves memory 73 during its creation, we need a way to:

• Record this retrieval event directly within memory 73's metadata
• Enable bidirectional traversal of memory relationships
• Maintain this information alongside the vector representation

Implementation Approach

1. Extended Memory Schema

Enhance the memory object schema with usage metadata:

{
  "memory_id": "73",
  "vector": [...],  // The embedding vector
  "timestamp": "2023-05-01T15:30:00Z",  // Creation time
  "content": {...},  // Original memory content
  "usage_history": [
    {
      "retrieved_by": "234",
      "timestamp": "2023-05-10T16:45:00Z",
      "retrieval_context": "similarity_search",
      "relevance_score": 0.92
    },
    // Additional retrievals...
  ]
}

2. Memory Update Mechanism

• Create an atomic update operation for memory objects
• When memory X retrieves memory Y:
  1. Complete the creation of memory X
  2. Perform an atomic update to memory Y to add X to its usage_history
  3. Include context of why Y was retrieved (similarity, direct reference, etc.)

3. Indexing Strategy

• Create secondary indices on the usage_history.retrieved_by field
• Optimize for queries that ask "which memories influenced memory X?"
• Enable efficient filtering by time ranges and retrieval contexts

4. Query Patterns

Support the following queries:

• "Show all memories that influenced memory X"
• "Show all memories that memory Y has influenced"
• "Identify memories with the most widespread influence"
• "Find chains of reasoning across multiple memories"

Technical Considerations

Vector Database Selection

• Ensure the chosen vector DB supports rich metadata with array fields
• Verify atomic update capabilities for concurrent usage
• Options: Pinecone, Weaviate, Milvus, Qdrant (with appropriate metadata features)

Performance Optimizations

• Batch updates to usage history when multiple memories are retrieved
• Consider size limits for usage history (perhaps with pagination)
• Implement TTL or importance-based pruning for less significant retrievals

Consistency Guarantees

• Handle race conditions when multiple processes update the same memory
• Implement retry logic for failed updates
• Maintain a separate audit log as a backup for relationship reconstruction

Metrics & Evaluation

• Track metadata size growth over time
• Measure update latency impact on memory creation
• Evaluate query performance for relationship-based searches
• Analyze memory influence networks for insight quality

Next Steps

1. Select vector database with appropriate metadata capabilities
2. Implement prototype memory schema with usage tracking
3. Create atomic update operations for memory retrieval events
4. Develop relationship query APIs
5. Test with simulated agent memory formation scenarios

[csmangum]

Need to validate if this is a good idea first