Skip to content
Trustworthy Agent Memory

CocoIndex research notes

Last checked: 2026-07-12.

CocoIndex is not a memory database.

It is an incremental data transformation and reconciliation engine. It reads source state, runs Python transformations, and keeps derived target state synchronized as source data or transformation logic changes.

That makes it relevant to agent context infrastructure. It can keep indexes, tables, embeddings, summaries, and knowledge graphs fresh. It does not decide which claims deserve to become durable memory or who is allowed to retrieve them.

Short read

Section titled “Short read”

Strong fit:

  • incremental ingestion for changing context sources
  • keeping vector, relational, graph, and file targets synchronized
  • removing stale derived rows when source items disappear
  • avoiding repeated embeddings and LLM extraction for unchanged inputs
  • invalidating derived data when transformation code, prompts, models, or dependencies change
  • building read-side context views for agents

Still needed around it:

  • canonical business data
  • permission and visibility policy
  • memory approval and correction
  • contradiction and source-precedence rules
  • user-facing citations
  • retrieval, reranking, and context packing
  • evals that prove the resulting context improves agent behavior

Mental model

Section titled “Mental model”

CocoIndex uses a state-driven model:

TargetState = Transform(SourceState)

The application declares what should exist in the target. CocoIndex calculates the creates, updates, and deletes required to make the target match that declaration.

This is closer to a continuously maintained materialized view than to a normal ETL job. The core concepts compare it to React, spreadsheets, and materialized views.

The distinction matters for agent systems:

  • a source document changes
  • only affected processing components run again
  • unchanged expensive transforms can reuse memoized results
  • stale target rows owned by the old component state are removed
  • the agent’s retrieval layer receives the reconciled target state

Execution model

Section titled “Execution model”

The primary CocoIndex v1 docs and agent skill use normal async Python rather than the v0 FlowBuilder DSL. An app mounts processing components dynamically as Python runs. The current package requires Python 3.11 or newer.

The repository now also contains a Rust SDK and Rust ports of many examples. The public programming guide remains centered on the Python SDK.

A processing component is both:

  • a unit of incremental execution
  • an ownership boundary for target states

Stable component paths let CocoIndex match the same source item across runs. If a component disappears, the engine removes the target states owned by that path. Component writes are applied as a unit, atomically where the target supports it. See processing components and target states.

Functions decorated with @coco.fn participate in change detection. With memoization enabled, cached results can be reused when these remain unchanged:

  • function logic
  • arguments
  • tracked context values

Logic fingerprints propagate through decorated function calls. External values such as a prompt or model name can be included through deps, and explicit version bumps can force invalidation.

This is directly useful for context pipelines. Changing an extraction prompt or model should invalidate the outputs derived from it, even when the source documents did not change.

Internal state is not application memory

Section titled “Internal state is not application memory”

V1 stores engine bookkeeping in embedded LMDB. This includes previous target states and memoization results used to calculate the next delta. Recent versions also added persistent per-component state through coco.use_state().

That state should not be confused with the application’s canonical memory store.

The LMDB database exists so CocoIndex can track and reconcile its work. The agent-facing data still belongs in an explicit target such as Postgres, SQLite, LanceDB, Qdrant, Neo4j, FalkorDB, SurrealDB, or another custom target. See internal storage and the v1.0.8-1.0.16 changelog.

Sources and targets

Section titled “Sources and targets”

The current v1 connector documentation lists sources for:

  • local files
  • Amazon S3 and compatible object stores
  • Azure Blob Storage
  • Google Drive
  • OCI Object Storage
  • Postgres
  • Kafka
  • Apache Iggy

Targets include relational databases, data warehouses, vector stores, graph databases, files, and message streams. Custom target connectors are supported. See the connector overview.

The website also frames Slack, inboxes, codebases, meeting notes, PDFs, and videos as context inputs. The current built-in connector list does not include direct Slack, GitHub, Linear, Jira, or Sentry sources. Those systems need custom source code or an upstream sync into a supported source such as Postgres, object storage, or Kafka.

What CocoIndex solves

Section titled “What CocoIndex solves”

Freshness is its strongest contribution.

Agent context often becomes stale because teams rebuild indexes on a schedule or append new embeddings without correctly removing old derived data. CocoIndex models the target as owned state and reconciles deletions as well as additions.

It also reduces unnecessary recomputation. A changed file can trigger file-level processing while unchanged chunks reuse cached embeddings. A code or prompt change can invalidate only the computations that depend on it.

Its internal lineage is useful for change propagation and debugging. That is not automatically the same as domain provenance. A trustworthy agent answer still needs source IDs, URLs, observed times, visibility, and claim-level evidence stored in the target schema.

What CocoIndex does not solve

Section titled “What CocoIndex does not solve”

The current public docs do not describe a general policy layer for propagating source ACLs or enforcing user, team, customer, and project visibility across targets.

Connector credentials determine what a pipeline can read or write. That is different from preserving the original source permissions on every derived chunk, entity, edge, and summary.

A context system built with CocoIndex still needs to:

  • carry source visibility into each target record
  • prevent a derived target from widening access
  • filter retrieval before ranking and generation
  • record which user or service produced each source event
  • preserve conflicts instead of silently choosing one source
  • require review before extracted claims become accepted memory
  • distinguish source deletion from policy-driven retention or legal deletion

CocoIndex also does not provide memory semantics such as proposed, accepted, superseded, deprecated, or needs_review. Those belong in the application’s memory item schema and memory diff format.

Where it fits

Section titled “Where it fits”

A practical architecture:

GitHub / Slack / Linear / Jira / Sentry / docs
                    |
          source adapters or event sync
                    |
              CocoIndex pipeline
       normalize / extract / embed / relate
                    |
       Postgres / vector DB / graph DB
                    |
     permission filter + retrieval + reranking
                    |
              context packet for agent

CocoIndex sits between source access and retrieval.

It should not replace source-system APIs, the canonical database, or the policy layer. It can replace a large amount of custom incremental indexing code between them.

V1 context risk

Section titled “V1 context risk”

V1 is a fundamental redesign. Current docs explicitly warn that coding agents trained on older material tend to generate v0 APIs such as FlowBuilder.

CocoIndex ships an official agent skill to provide current v1 concepts, APIs, connectors, and patterns to Codex, Claude Code, Cursor, and custom RAG systems.

This is a useful example of context freshness outside the application itself: even a strong coding model needs versioned, repo-local product context when an API changes faster than model training data.

Current project signal

Section titled “Current project signal”

As of 2026-07-12, the public repository had:

  • 10,692 GitHub stars
  • 828 forks
  • 55 open issues
  • 85 contributors returned by the first GitHub contributors page
  • at least 100 commits in the previous 30 days
  • eight releases in the previous 30 days
  • latest release v1.0.16, published 2026-07-06
  • Apache-2.0 license

The May 4 X post that prompted this review announced that CocoIndex had reached Python trending, described it as a Python-native context engine with a Rust core, and said Rust and TypeScript SDKs were planned after the v1 launch.

That post is historical state. The current repository now includes Rust SDK examples. I did not find a TypeScript SDK in the current public repository or documentation.

Release builds send anonymous lifecycle events such as import, app creation, update, and drop to a Scarf endpoint. The documented payload excludes user data, code, paths, connector configuration, and persistent identifiers. It can be disabled with COCOINDEX_DISABLE_USAGE_TRACKING=1; see anonymous usage telemetry.

The stronger signal is repository and release activity, not independent public evaluation. A 30-day search found one recent CocoIndex-specific AI Council session and little detailed third-party discussion. Stars and trending status do not prove production reliability.

Evaluation to run

Section titled “Evaluation to run”

Build one small pipeline before making a platform decision:

  1. Read repository docs plus a Postgres table containing ticket and incident events.
  2. Write normalized source records and embeddings to Postgres.
  3. Store source URI, observed time, content hash, visibility, and transformation version on every target row.
  4. Edit, rename, and delete source records and confirm stale target rows disappear.
  5. Change the extraction prompt and model dependency and inspect what reprocesses.
  6. Restart from the LMDB state and test recovery after a failed target write.
  7. Attempt mixed private and team retrieval to verify that the application policy layer blocks leakage.
  8. Compare latency, target correctness, and embedding/LLM calls against a full rebuild.

The key question is not whether CocoIndex can build an index.

It is whether the resulting context remains current, permission-safe, source-backed, and recoverable under real source and logic changes.

Current position

Section titled “Current position”

CocoIndex is one of the more relevant infrastructure projects for this guide because it addresses the freshness and invalidation layer directly.

It is not a complete trustworthy memory layer. It is a plausible engine underneath one.

Use it to maintain derived context. Keep memory policy, review, permissions, retrieval, and canonical truth explicit outside it.

Primary sources

Section titled “Primary sources”