# Keep a codebase in sync

> Mirror a repository or a folder in a library, so search and answers use the current version of every file.

Source: https://fryri.com/docs/api/keep-a-codebase-in-sync

`source_ref` names the original of a memory, and an add with a `source_ref` that the library already holds replaces the earlier memory. `GET /v1/memories` with `source_ref_prefix` lists every memory under one prefix, each with the `sha256` of its content. Together they keep a library equal to a repository, with one memory per file.

## How a sync works

A sync compares the files with what the library holds:

- A new file is added with its path under a prefix as `source_ref`, such as `myrepo/src/auth.py`.
- A changed file is added again with the same `source_ref`, and the add replaces the earlier memory.
- The memory of a deleted file is deleted with `DELETE /v1/memories/{memory_id}`.
- An unchanged file is skipped, because its `sha256` in the list equals the SHA-256 of the file.
- A file whose memory is `failed` is sent again. Sending the same bytes again starts a new attempt.

Each file sent bills like any other add, as the [price list](/docs/api#pricing) shows. A sync of an unchanged repository only lists, and listing is free.

## Replace a changed file

The add returns the new memory, and `replaced` lists the ids of the earlier memories with the same `source_ref`. They stop appearing in search and answers at once. A failed add leaves the earlier memory in place.

**Request**

```bash
curl -X POST https://api.fryri.com/v1/memories \
  -H "Authorization: Bearer $FRYRI_API_KEY" \
  -F "file=@src/auth.py" \
  -F "source_ref=myrepo/src/auth.py" \
  -F "end_user_id=user_42"
```

**Response**

```json
{
  "id": "file_hqjP2q_DlrHZT-4U",
  "type": "document",
  "title": "auth.py",
  "status": "processing",
  "status_reason": null,
  "created_at": "2026-09-25T09:07:09.917301Z",
  "source_ref": "myrepo/src/auth.py",
  "sha256": "785108e6758610a47355924cf936ce293ac1c6e5e6c7cd9b8752c19c78c00e26",
  "duplicate": false,
  "replaced": [
    "file_kR1jHO7CDRwaoCEw"
  ]
}
```

## List one codebase

`source_ref_prefix` lists only the memories whose `source_ref` starts with it. Characters like `%` and `_` match themselves. `sha256` is the SHA-256 of the file as it was added, in hex.

**Request**

```bash
curl "https://api.fryri.com/v1/memories?end_user_id=user_42&source_ref_prefix=myrepo%2F" \
  -H "Authorization: Bearer $FRYRI_API_KEY"
```

**Response**

```json
{
  "memories": [
    {
      "id": "file_hqjP2q_DlrHZT-4U",
      "type": "document",
      "title": "auth.py",
      "status": "processing",
      "status_reason": null,
      "created_at": "2026-09-25T09:07:09.917301Z",
      "source_ref": "myrepo/src/auth.py",
      "sha256": "785108e6758610a47355924cf936ce293ac1c6e5e6c7cd9b8752c19c78c00e26"
    }
  ],
  "next_cursor": null
}
```

## Sync a repository

The script syncs every text file that git tracks, and skips empty files, binary files and files over 1 MB. Text files such as TypeScript sources and SVG images are added as documents. Run it from the root of the repository, by hand or after every push. `FRYRI_PREFIX` names the codebase in the library, and `FRYRI_END_USER_ID` keeps it in the library of one end user.

**Complete example**

```python
"""Keep a codebase in Fryri.

Every text file git tracks becomes one memory whose source_ref is the file's
path under PREFIX. A changed file replaces its memory, a deleted file's memory
is deleted, a file whose memory failed is sent again, and an unchanged file is
skipped without being sent. A file the API refuses as a photo, video or audio
file is skipped. Run it from the root of the repository, by hand or on every
push.
"""
import hashlib
import os
import subprocess
import time

import requests

API = os.environ.get("FRYRI_API_URL", "https://api.fryri.com") + "/v1/memories"
HEADERS = {"Authorization": f"Bearer {os.environ['FRYRI_API_KEY']}"}
PREFIX = os.environ.get("FRYRI_PREFIX", "myrepo/")  # names this codebase in your library
SCOPE = {"end_user_id": os.environ["FRYRI_END_USER_ID"]} if os.environ.get("FRYRI_END_USER_ID") else {}
MAX_BYTES = 1_000_000  # larger files are skipped


def call(method, url, **kwargs):
    """One API call. Waits out rate limits; a 404 on delete is already done,
    and a 422 unsupported_format is returned for the caller to skip."""
    while True:
        response = requests.request(method, url, headers=HEADERS, timeout=120, **kwargs)
        if response.status_code == 429 and response.json()["error"]["retryable"]:
            time.sleep(int(response.headers.get("Retry-After", "5")))
            continue
        if method == "DELETE" and response.status_code == 404:
            return response
        if response.status_code == 422 and response.json()["error"]["code"] == "unsupported_format":
            return response
        response.raise_for_status()
        return response


# 1. The text files git tracks.
listed = subprocess.run(["git", "ls-files", "-z"], capture_output=True, check=True).stdout
files = {}
for path in listed.decode("utf-8").split("\0"):
    if not path or not os.path.isfile(path):
        continue
    with open(path, "rb") as handle:
        data = handle.read(MAX_BYTES + 1)
    if data and len(data) <= MAX_BYTES and b"\0" not in data:  # skip empty, large and binary files
        files[PREFIX + path] = data

# 2. What Fryri holds for this codebase.
held, cursor = {}, None
while True:
    params = {**SCOPE, "source_ref_prefix": PREFIX, "limit": 100}
    if cursor:
        params["cursor"] = cursor
    page = call("GET", API, params=params).json()
    for memory in page["memories"]:
        if memory["source_ref"] in held:  # two copies of one path: resending the file keeps one
            held[memory["source_ref"]]["sha256"] = None
        else:
            held[memory["source_ref"]] = memory
    cursor = page["next_cursor"]
    if not cursor:
        break

# 3. Delete the memories of deleted files first.
removed = [ref for ref in held if ref not in files]
for ref in removed:
    call("DELETE", f"{API}/{held.pop(ref)['id']}", params=SCOPE)

# 4. Send new and changed files, and files whose memory failed (sending the same
#    bytes again starts a new attempt). Skip a file whose memory is unchanged, and
#    a new file whose exact content Fryri already holds under another path.
usable = {ref: memory for ref, memory in held.items() if memory["status"] != "failed"}
stored = {memory["sha256"] for memory in usable.values()}
sent = refused = 0
for ref, data in sorted(files.items()):
    sha256 = hashlib.sha256(data).hexdigest()
    mine = usable.get(ref)
    if (mine and mine["sha256"] == sha256) or (ref not in held and sha256 in stored):
        continue
    response = call("POST", API, files={"file": (os.path.basename(ref), data)}, data={**SCOPE, "source_ref": ref})
    if response.ok:
        sent += 1
    else:
        refused += 1

unchanged = len(files) - sent - refused
print(f"{len(files)} files: {sent} sent, {len(removed)} removed, {refused} refused, {unchanged} unchanged")
```

## Identical files

Identical content shares one memory. Two files with the same bytes, such as two copies of a license, become one memory that carries the `source_ref` of the first. The script skips the second copy while the first is held, and sends it once the first is deleted.

## Best Practices

**Syncing**

- End each prefix with `/`, so the prefix of one repository never matches another, such as `myrepo/` and `myrepo-old/`.
- Delete the memories of deleted files before adding new files. A moved file then gets a memory under its new path.
- Compare `sha256` before sending, so a sync uploads only what changed.

**Searching**

- Search with the same `end_user_id` that the sync used.
- Read `source_ref` on each search result to link back to the file.
- For a folder without git, list its files with `Path.rglob` in place of `git ls-files`.

> **Tip:** Run the sync after every push, so search and answers use the current code.

> **Warning:** A changed file is missing from search and answers until its new memory is `ready`, because the add removes the earlier memory at once. A small file is usually ready within seconds, and a large sync can take a few minutes.