-
-
Notifications
You must be signed in to change notification settings - Fork 8
Expand file tree
/
Copy pathchangelog.txt
More file actions
1731 lines (1686 loc) · 133 KB
/
changelog.txt
File metadata and controls
1731 lines (1686 loc) · 133 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
0.10.1.46 JUN/05/2026
- feat(auth): built-in bearer-token authentication for remote MCP servers.
`MENTISDB_BEARER_TOKEN_ACCESS=true` now protects standard Streamable HTTP
MCP and legacy HTTP MCP routes with `Authorization: Bearer ...`; when the
flag is disabled, local/default deployments keep the old unauthenticated
behavior. Tokens are stored in `bearer-tokens.json` as SHA-256 hashes only,
never as plaintext. This is the headline change in 0.10.1.46 — it is the
prerequisite for any non-loopback deployment of the HTTPS MCP / REST
surfaces and unlocks the rest of the auth work in this release.
- feat(auth): scoped bearer-token registry and CLI. Operators can issue global
tokens or tokens limited to one or more chains with:
`mentisdb bearertoken create --global <alias>`
`mentisdb bearertoken create --chain <chain> [--chain <chain> ...] <alias>`
`mentisdb bearertoken list [--global | --chain <chain> ...]`
`mentisdb bearertoken remove <alias>`
The CLI help is focused and explicit when users run `mentisdb bearertoken`
or `mentisdb bearertoken --help`.
- feat(auth): MCP chain-scope enforcement. Global tokens can call server-wide
tools and access every chain. Chain-scoped tokens may call tools only for
their configured chain set, including omitted `chain_key` requests only when
the daemon default chain is within scope. Server-wide tools such as
`mentisdb_list_chains` require a global token.
- feat(dashboard): dedicated Bearer Tokens page in the header navigation
(`Chains | Agents | Skills | Bearer Tokens | Settings`). The page exposes
the bearer-token feature toggle, alias input, Global/Chains radio controls
with Global selected by default, a full-width multi-chain selector table,
one-time token display, copy-to-clipboard button, token table, and revoke
action.
- feat(dashboard): Settings can edit `MENTISDB_BEARER_TOKEN_ACCESS`, and the
Settings page now includes a Restart Daemon button for environment changes
that require a daemon restart.
- feat(mcp): updated `cloudllm_mcp` integration to support dynamic bearer
authorization context, including request payload/params so MentisDB can make
chain/global authorization decisions inside the MentisDB auth policy layer.
- feat(cli): new `mentisdb cert [<ip-address-or-domain>]` subcommand mints a
self-signed TLS certificate for the HTTPS MCP and REST surfaces and the
web dashboard, makes it the active cert on disk, and persists
`MENTISDB_TLS_CERT` / `MENTISDB_TLS_KEY` into the local `.env` so the next
daemon start picks the new paths up automatically. Supports:
`mentisdb cert`
`mentisdb cert <ip-or-domain>` (IPv4/IPv6 literal → IP SAN, DNS name → DNS SAN)
`mentisdb cert --force` (regenerate an existing cert)
`mentisdb cert --reset` (delete existing cert/key first, then generate a
fresh factory-default certificate; implies `--force`)
`mentisdb cert --out-dir <path>` (write to a non-default directory)
`mentisdb cert --env-file <path>` (point at a non-default .env)
`mentisdb cert --no-env-update` (print the export lines, leave .env alone)
The standard SAN set (my.mentisdb.com, localhost, 127.0.0.1, every unicast
IP on every host interface, plus MENTISDB_BIND_HOST when it is a DNS name)
is always included; the user-supplied host is appended. Refuses to
overwrite without `--force`, prints the SAN list and SHA-256 fingerprint
for `openssl x509 -fingerprint -sha256` cross-check, and tells the
operator to restart the daemon so the next HTTPS request serves the
new cert. Full help lives in `mentisdb cert --help` and the
"Minting a Custom Certificate" section of the User Guide.
- feat(cli): comprehensive `--help` output. `mentisdb --help` now prints both
the daemon help AND the full CLI subcommand help in one view, so users can
discover all commands and flags at once. Subcommand-specific help works as
expected: `mentisdb cert --help`, `mentisdb setup --help`, `mentisdb wizard --help`,
`mentisdb add --help`, `mentisdb search --help`, `mentisdb agents --help`,
`mentisdb backup --help`, `mentisdb restore --help`,
`mentisdb bearertoken --help` each show only that subcommand's focused help.
- feat(skill): MENTISDB_SKILL.md adds a prominent "🔎 SEARCH BEFORE YOU
WRITE" section between MANDATORY STARTUP and WRITE TRIGGERS. The routine
(`recent_context` → `ranked_search` → tighten with `tags_any` /
`thought_types` / `concepts_any` / `since`) plus three decisions per
append — do I need to write at all, do I need to update a prior lesson,
which 1–3 neighbours to link via typed edges — and a new "Blind appends"
anti-pattern. Search-first is the upstream half of the dedup discipline:
5–15 seconds of search before write is the single highest-leverage
habit for keeping `ranked_search` useful as chains grow past a few
hundred thoughts.
- feat(server): expose `ensure_tls_cert_with_sans(cert, key, extra_sans,
overwrite) -> TlsCertArtifacts` and the matching `TlsCertArtifacts {
cert_path, key_path, sans, sha256_fingerprint }` so library consumers
and the new `mentisdb cert` CLI can mint or inspect certs with custom
SAN sets without re-implementing the `rcgen` + `x509-parser` dance. The
existing `ensure_tls_cert` is now a one-line wrapper (overwrite=false,
no extra SANs), so no behaviour change for existing callers. The
`sans` field in `TlsCertArtifacts` is extracted from the freshly
written cert with `x509-parser` (newly wired under the `server`
feature in `Cargo.toml`) and rendered in the same `dns:<name>` /
`ip:<addr>` style the cert was built with, so a CLI caller can print
them and a downstream consumer can grep for an IP or hostname.
- feat(cli): `--headless` is now a real top-level flag. `mentisdb --headless`
starts the HTTP/MCP/REST servers without the interactive terminal UI,
matching the long-standing `--mode http --headless` invocation that the
stdio proxy has used internally since 2ae780f. `--help` now lists the
flag in both the Usage block and the Flags section.
- fix(tui): the daemon auto-promotes to headless HTTP mode when stdin or
stdout is not a TTY. Previously the TUI main loop was entered with an
already-EOF stdin; crossterm's `event::poll(100ms)` returned `Ok(false)`
immediately instead of honouring the timeout, and the loop re-rendered
the full ratatui UI on every iteration — pinning one CPU core at 100%
on a 2-core cloud VM. The new `tui_can_run()` guard in `src/tui.rs`
detects the non-TTY case at startup and `run()` then delegates to the
existing `run_headless()` so the HTTP servers come up and the process
idles near 0% CPU. Affects every non-interactive launch: `docker run`
without `-t`, `nohup mentisdb &`, `systemd` without
`StandardInput=tty`, `cron`, and the SSH-disconnect scenario that
produced the original report.
- fix(restore): preserve local registries during cross-instance restores,
append verified same-chain suffixes, and import divergent same-name chains
under renamed keys. Restores are now idempotent and chain-scope-safe:
the local `mentisdb-registry.json` is merged, not replaced, and
`--overwrite` only allows same-path chain file replacement when a safe
suffix merge is not possible.
- fix(auth): generated token secrets now use the clearer `mentisdb_` prefix
instead of `mdb_live_`; README, tests, docs site, and the demo script were
updated to match.
- fix(cli): `mentisdb cert` snapshots the on-disk existence of the cert
+ key before delegating to `ensure_tls_cert_with_sans`, so the
human-readable report correctly distinguishes "Wrote new self-signed
TLS cert" from "Reusing existing TLS cert" on a brand-new tempdir.
The helper writes the files in the no-overwrite path, so a post-call
existence check would always return true and the CLI would print
"Reusing existing TLS cert" on a fresh cert. The `cert_existed_before`
snapshot in `cli::cert::run_cert` is the reliable signal.
- fix(tui): increased contrast for the bottom instruction/hint bar so
navigation hints remain readable on low-contrast terminal themes.
- refactor(cli): collapse the duplicated `writeln!` blocks in
`cli::cert::run_cert` into small named helpers (`write_status`,
`write_sans`, `write_fingerprint`, `write_export_instructions`,
`write_or_warn_env_update`) so the report is composed in one place
and every section can be tweaked without copy-paste. Help text is
one source of truth: `cli::cert::help_text()` is consumed by
`args::help_text()` and the binary's `daemon_help_text()`, so
`mentisdb cert --help` and `mentisdb --help` always show the same
wording. The shared help covers 6 worked examples (IP, DNS,
`--force`, `--out-dir`, `--env-file`, `--no-env-update`), a flag
table, a "when the new cert takes effect" paragraph, and the
`openssl s_client` cross-check callout.
- refactor(server): promote `default_tls_dir` to `pub` in `server.rs`
and remove the duplicate `default_tls_dir` helper from
`src/cli/cert.rs`. The `cert` module now reuses the canonical
`crate::server::default_tls_dir()` for its fallback path, so the
default TLS directory is defined in one place and library consumers
can read the constant directly.
- docs: new "TLS Certificates" section in README.md between Web
Dashboard and Backup and Restore, with six worked examples, a full
options table, the restart-required note, the `openssl s_client`
cross-check snippet, and a "When the auto-generated SAN set is not
enough" subsection. docs.mentisdb.com user_docs.rs gains a "Minting
a Custom Certificate" subsection inside HTTPS & TLS with the SAN set
contents, six worked examples, the options table, and the cross-check
callout. docs.mentisdb.com agent_docs.rs gains a "When the Operator
Needs a New TLS Certificate" section that helps agents recognize the
symptoms (remote IP, custom hostname, new interface, key rotation,
machine migration) and point the user at
`mentisdb cert <ip-or-domain> --force` plus a daemon restart.
- docs: expanded remote bearer-token documentation in README and
docs.mentisdb.com, including server activation, CLI token management,
dashboard management, scoped access behavior, Codex `config.toml` header
examples, and future usage-stats room in the token registry model.
- docs: expanded crate-level and API rustdoc toward a manual-quality
developer experience: custom integration guidance, production
considerations, query/ranked-search examples, storage adapter extension
notes, skill registry docs, thought/relationship/scope structs, and
installation prerequisites. Follow-up fixes removed unresolved and
redundant rustdoc links so strict docs builds stay clean.
- docs(search): README now records the post-0.9.9 ranked-search validation
scores: LoCoMo-10P overall R@10 72.6% (1436/1977), single-hop R@10 76.8%,
multi-hop R@10 57.4%, weak-vector baseline 44.9% R@10, and ~317 queries/s
evaluation throughput with the `fastembed-all-minilm-l6-v2` vector sidecar.
- test: 9 new `cli::cert` unit tests cover the SAN builder across
IPv4 / IPv6 / DNS / empty / invalid / None inputs, the env-line
replacement (in-place update, append-when-missing, similar-prefix
preservation, comment pass-through, mtime idempotence), the env
file writer (create-when-missing, preserve-existing-entries,
idempotent rewrite), `resolve_env_file` (override vs. fallback),
`resolve_paths` (out-dir vs. env-var vs. default), and the
help-text content (every flag mentioned, at least 5 example
invocations). 15 new `tests/cert_tests.rs` integration tests cover
parsing (positional, --force, --reset, --out-dir, --env-file, --no-env-update,
error paths, --help), end-to-end PEM write + SAN verification via
`x509-parser`, reuse without --force, regeneration with --force,
--reset that deletes and regenerates, --reset with custom SAN,
--no-env-update that leaves the .env alone, and the help-text
roundtrip. All tests are isolated via `tempfile::tempdir` (unique
per test) and a global `env_lock` mutex that serialises env-touching
tests, so no test depends on inherited `MENTISDB_TLS_CERT`,
`MENTISDB_DIR`, or `HOME` values.
- chore: ignore generated LaTeX build artifacts and LME benchmark pycache
files to keep future working trees cleaner.
0.9.9.45 MAY/30/2026
- feat(search): thesaurus now applies **automatically by default** to all
ranked search queries (REST `/v1/ranked-search`, MCP, dashboard, CLI,
and benchmark harnesses). No client changes required. The static
thesaurus (895 headwords + 300+ lemmas) is invoked transparently via
`apply_thesaurus_if_text` in the server for every text-bearing
`RankedSearchQuery`. This surfaces the +31% NDCG research improvement
to all users and the official LoCoMo/LongMemEval benchmarks.
- feat(search): comprehensive static thesaurus for query-time synonym
expansion — ~895 headwords covering verbs, nouns, adjectives, adverbs,
and technical vocabulary; data is embedded into the binary at compile
time via include_str! with no external file or network dependency;
parser builds a unidirectional map from headwords to synonym lists;
query terms are lemmatized (irregular + regular suffix stripping)
before lookup so past-tense verbs like "went" resolve to "go" and
find synonyms; default synonym_weight is 0.7; research harness shows
+31% NDCG improvement over baseline (0.52 → 0.68) on synthetic corpus
- feat(search): expand irregular verb lemma coverage from ~100 to 300+
entries — covers all common English irregular verbs and their past
tense, past participle, and present participle forms; new lemmatize()
helper combines irregular lookup with regular suffix stripping
(-ing, -ed, -ies, -s) for comprehensive verb-form normalization
- feat(search): add embedding-based synonym generator as alternative
to static thesaurus — uses LocalTextEmbeddingProvider + VectorIndex
to find cosine-similar terms from a fixed vocabulary; kept for
experimentation but research shows it underperforms (NDCG 0.48 vs
0.68 thesaurus) due to hash-based feature vectors rather than true
semantic similarity
- feat(search): integrate synonym expansion into BM25 lexical scoring
and ranked queries — LexicalQuery gains synonyms HashMap and
synonym_weight (default 0.7); LexicalIndex::search_in_positions
adds a fast path that bypasses synonym overhead when no synonyms are
configured, keeping baseline latency neutral; when synonyms are
present, expanded terms are scored with reduced weight relative to
original term matches; RankedSearchQuery wires synonym maps through
to rank_candidates_lexically
- test(search): add search quality research harness comparing synonym
generators — synthetic corpus (140 docs, 12 queries) with R@5, R@10,
P@5, MRR, NDCG@5 metrics; compares manual oracle map, static thesaurus
auto, embedding auto, and combined configurations; key finding: one
high-quality source (thesaurus) outperforms combined maps because
synonym noise degrades precision
- feat(search): auto-infer implicit RelatedTo edges from vector cosine
similarity — the vector sidecar and graph adjacency index were previously
disconnected signals; chains where agents never author explicit refs or
relations now have a dense implicit graph populated automatically;
disconnected signals; chains where agents never author explicit refs or
relations now have a dense implicit graph populated automatically;
edges with cosine >= threshold (default 0.85, env MENTISDB_AUTO_EDGE_THRESHOLD)
are kept, top-K per thought (default 5, env MENTISDB_AUTO_EDGE_K);
the overlay is persisted to <chain>.auto_edges.bin with atomic rename and
reloaded on chain open; threshold or K mismatch triggers a full rebuild;
chains > 50 000 thoughts skip the O(N²) open-time build and grow the
overlay incrementally; targets the 25.8% near-miss bucket (R@20 hits not
in R@10) identified in whitepaper §9.3
- feat(search): VectorIndex::get_vector — retrieve one stored vector by
document id from the in-memory cosine index
- feat(search): ThoughtAdjacencyIndex::locator_map — expose id→locator map
for callers that need to convert Uuid values to ThoughtLocator
- feat(core): cache VectorIndex in-memory on ManagedSidecarEntry —
rank_candidates_semantically no longer loads the vector sidecar from disk
on every search call; the in-memory index is kept in sync on every append
and full sidecar rebuild
- fix(core): remove redundant load_or_rebuild_implicit_edge_overlay calls
in apply_persisted_managed_vector_sidecars — the inner manage_vector_sidecar
and register_vector_sidecar_for_search calls already trigger the overlay
load/rebuild; the duplicate calls at the end of each cfg branch caused an
unnecessary extra disk read on every daemon startup
- feat(dashboard): chains table redesign — replaced "Agent Idx", "Entity Idx",
and "Vector Idx" columns with "Chain" (on-disk file), "Semantic" (vector
embeddings sidecars), and "Lexical (RAM)" (in-memory BM25 index); the Lexical
column shows `—` for unloaded chains and an estimated size (e.g. `53 KB`) once
the chain is accessed and its index is built; Lexical header and values are
styled in amber to indicate in-memory-only state
- feat(search): add `LexicalIndex::estimated_memory_bytes()` to approximate the
RAM footprint of the in-memory BM25 postings map, document-stats vector, and
metadata; lower-bound estimate — real heap may be 1.2–1.5× larger
- feat(api): add `MentisDb::estimated_lexical_index_bytes()` public method so
dashboard and other callers can surface the lexical index size without accessing
private fields
- refactor(dashboard): rename `chain_sidecar_sizes` → `vector_index_size` and
simplify `api_chains` JSON to expose only the three meaningful index sizes
(chain file, semantic sidecars, lexical RAM estimate) per chain
- feat(dashboard): comprehensive tooltips across the entire UI — every nav link,
summary stat, table header, action button, form field, filter control, and
modal now has a friendly HTML title attribute explaining what it does and what
data it represents; uses native browser tooltips for zero-dependency compatibility
- feat(backup): include .auto_edges.bin in backup archives — implicit edge overlay
files are now packed alongside vector sidecars in .mentis archives and cleaned
up on chain deregister/deletion
- fix(core): daemon no longer silently fails to recreate .auto_edges.bin on restart
after deletion — load_or_rebuild_implicit_edge_overlay now prints a diagnostic
eprintln! when the sidecar load fails, when all managed sidecars lack a cached
index (VectorIndex build failure), when the > 50 000 thought branch saves an
empty overlay (was never saved — caused rebuild on every restart), and when
save_to_path itself fails; all silent early-return paths are now observable
- fix(dashboard): escape backtick in Lexical column tooltip title attribute
- feat(dashboard): expose auto_edges_size in chains API and UI — chains table
gains an "Edges" column showing the on-disk size of the implicit-edge overlay
sidecar (.auto_edges.bin), plus a "Total Edges" summary stat in the header bar;
the API response includes auto_edges_size and auto_edges_size_formatted for
registered, live, and unregistered chains
- docs(whitepaper): update to v0.9.8 with implicit edge overlay — document
ImplicitEdgeOverlay in abstract, contributions, file layout, and retrieval
sections; update benchmark scores, keywords, and file layout diagram to include
.auto_edges.bin and entity-types.json
- feat(tui): show total chain + index size and storage path in chains table —
the "Storage" column now shows the total on-disk size of the chain file plus
all sidecar indices (agents.json, entity-types.json, vector sidecars,
auto_edges.bin); a new "Location" column shows the chain file path; replaces
the previous single column that showed only the chain file size prefixed to
the path
- fix(core): repair missing .auto_edges.bin for already-open daemon chains —
cached server and dashboard chain accesses now opportunistically call
ensure_implicit_edge_overlay() when no writer holds the chain lock, so deleting
the overlay sidecar while the daemon is running no longer leaves the file absent
until a full process restart; added regression coverage for both cached-chain
repair and reopen-time rebuild
- feat(search): add retrieval improvement primitives for the next search pipeline
iteration — new isolated modules provide deterministic Personalized PageRank
graph scoring, pseudo-relevance feedback query expansion, append-only summary
candidate selection, and heuristic query-intent route planning; modules are
exported for testing and future integration but do not change default ranked
search behavior yet
- feat(memory): expose read-only append-only summary candidates — core, REST,
and MCP callers can request deterministic source windows for future Summary
thoughts using `mentisdb_summary_candidates` / `/v1/summary-candidates`;
the API only selects candidate ranges and never generates text, rewrites
memory, or appends summary thoughts
- revert(search): remove PRF query expansion, PPR graph scoring, and query-aware
routing from ranked-search pipeline — benchmark validation (LongMemEval 500q)
showed these features did not improve recall and added latency; the primitive
modules remain in src/search/ for future work but are no longer wired into
default or opt-in ranked search; REST/MCP request parsing and score fields
for prf/ppr removed; summary candidates API is unaffected
0.9.8.44 MAY/07/2026
- BREAKING: rename binary from `mentisdbd` to `mentisdb` — all CLI commands,
documentation, install scripts, CI artifact names, and TUI references updated;
`cargo install mentisdb` still installs the crate, but the resulting binary
is now `mentisdb` (no trailing `d`)
- feat(dashboard): chains table now shows a "Size" column with human-readable
storage size (B, KB, MB, GB, TB, PB) computed from each chain's on-disk file;
summary bar shows total size, thoughts, agents, and per-index size totals
- feat(dashboard): new Settings tab — view and edit all 18 `MENTISDB_*` env
vars in a table with type-aware controls (checkbox for booleans, number for
ports, text for strings); shows current value, default, and description per
setting; "Reset to Default" per row; "Apply Settings" persists changes to a
`.env` file and hot-reloads `auto_flush` immediately; restart-required flag
for network/storage changes that cannot be applied to a running daemon
- feat(tui): show chain file size in Storage Location column — prefixed with
human-readable size in brackets (e.g. [10 KB] /path/to/chain.tcbin); sizes
shown in B, KB, MB, or GB as appropriate; running status now includes PID
- feat(config): dotenvy support — mentisdb now loads environment variables
from a `.env` file in the current working directory at startup; existing
shell environment variables take precedence over `.env` values; silently
ignored when no `.env` file is present; all `MENTISDB_*` env vars,
`RUST_LOG`, and `OPENAI_API_KEY` can now be managed via project-local
`.env` files for cleaner per-project configuration
- test: add `start_servers_headless_without_tui_state` regression test for the
headless daemon startup path used by stdio proxy auto-launch and
`--mode http --headless`
- bench: LoCoMo R@10 = 71.80% (single 75.80%, multi 57.20%)
0.9.7.43 APR/30/2026
- fix(dashboard): harden skill Markdown rendering — dashboard-rendered skill
links now escape labels and href attributes, allow only http/https/mailto
protocols, and render unsafe links as plain text, preventing stored skill
content from creating scriptable links in the same-origin dashboard
- fix(dashboard): editing from the skill detail page now targets the version
being viewed and defaults to the latest version instead of the oldest
version; valid-but-missing version UUIDs now return 404 instead of 500
- chore(backup): ignore the new *.mentis backup archive extension in git so
private chain, agent, vector, and skill data is not accidentally committed
- fix(stdio): remove synthetic ack for notifications in stdio proxy — the old
{id:'ack', result:{}} response broke Claude Desktop's MCP SDK which validates
that response ids match pending requests; notifications are now silently
absorbed (forwarded to daemon, nothing written to stdout), fixing the
'Unexpected end of JSON input' error that prevented Claude Desktop from
connecting
- feat(stdio): --headless mode for background daemon auto-launch — the stdio
proxy now passes --mode http --headless when spawning a background daemon,
starting HTTP servers without the TUI (which crashes when stdin is null);
adds DaemonArgMode::RunHeadless and run_headless() that skips TUI/terminal
setup, runs migrations silently, and waits for SIGINT
- feat(dashboard): skill editing — each skill row now has an Edit button that
opens a modal with a textarea pre-filled with the skill's current markdown
content; Save calls the existing upload endpoint to create a new version;
preserves original agent_id and format; also fixes uploader column showing
'—' instead of actual agent name
- fix(stdio): tighten SSE detection in proxy — changed from contains('event:')
to starts_with('event:') / starts_with('data:') to avoid false positives
from tool descriptions containing the substring; added empty response body
filter so 202 Accepted responses don't write empty lines to stdout
- fix(stdio): local stdio mode now echoes client's protocol version instead of
hardcoded 2025-06-18, and tool definitions include execution field
{taskSupport:'optional'} matching the daemon's streamable HTTP output
- feat(primer): updated priming phrase to 'use mentisdb as your memory system'
across TUI, mentisdb.com, and docs.mentisdb.com
- refactor(skill): MENTISDB_SKILL.md — strengthened TaskComplete trigger rule
to emphasize immediate write on completion (not deferred), added 'deferring
memory writes' to anti-patterns list
- fix(update): clean up TUI terminal before restart to prevent ANSI garbage;
old code called cmd.spawn()+exit(0) which killed the TUI thread without
running TerminalCleanup, so the new process inherited alternate screen +
raw mode; new restart_process() helper disables raw mode, leaves alternate
screen, and uses exec() on Unix to replace the process (no zombie window),
falling back to spawn+exit on non-Unix with terminal restored
0.9.6.42 APR/29/2026
- feat(tui): wider update dialog (80 cols) so GitHub release URL fits without
wrapping; successful update now shows a green "Update Complete" overlay with
the exact relaunch command and offers r to restart in-place or q to quit;
mentisdb auto-restarts with the same CLI args when the user presses r
- feat(backup): rename backup format from .mbak to .mentis; backup CLI now
uses -o shorthand for --output (both still accepted); restore aborts with
a clear error if mentisdb is detected running, preventing the daemon's
in-memory state from overwriting restored files on next flush
- feat(mcp): SSE support on MCP servers — start_mcp_server and start_https_mcp_server
now return (ServerHandle, SseBroadcaster) tuples; GET endpoint returns text/event-stream
with live MCP events, POST requests auto-broadcast JSON-RPC results/errors as SSE events;
cloudllm_mcp bumped to 0.3.0
- feat(server): bind_host parameter on standard_mcp_router — respects MENTISDB_BIND_HOST
env var (defaults to 127.0.0.1) for flexible network binding on multi-homed machines
- feat(mcp): add agent_id parameter to mentisdb_recent_context — allows filtering recent
context by specific agent identity instead of always using the chain's default agent
0.9.5.41 APR/26/2026
- feat(mcp): upgrade cloudllm_mcp to 0.2.1 — MCP 2025-11-25 protocol support;
streamable HTTP servers now validate Origin headers for security on bound
addresses and transparently skip validation on unspecified (0.0.0.0/::/::1)
addresses for compatibility with tools that don't send Origin; protocol version
negotiated during initialize is now "2025-11-25"
- feat(ecosystem): Hermes agent by Nous Research ships a native MentisDB
MemoryProvider — Hermes can now connect directly to a running mentisdb instance
for durable semantic memory without an intermediary MCP bridge; the Hermes team
implemented their own memory plugin against the MentisDB API for deeper
integration into the Hermes agent lifecycle
- fix(dashboard): restore "+Bootstrap New Chain" and "↺ Refresh" buttons on the
Chains page — a refactor in 8ce4ff2 moved the Bootstrap modal HTML to a static
section and introduced a one-shot `_chainBootstrapWired` guard, but accidentally
dropped the `cl-bootstrap-btn` and `cl-refresh-btn` click handlers; the buttons
rendered but did nothing after the first renderChainList() call because the guard
prevented re-attaching handlers when the header DOM was rebuilt
- fix(dashboard): header buttons (Refresh, Bootstrap) now re-attach their event
listeners on every `renderChainList()` call instead of only on the first call,
fixing a latent bug where navigating between pages (Chains → Agents → back to
Chains) silently broke those buttons after any page transition
0.9.4.40 APR/23/2026
- perf(search): incremental LexicalIndex — built once at chain open time and
updated incrementally on every append_thought; eliminates the O(n) full-index
rebuild that was happening inside every ranked search query; drops
query_ranked_lexical_content latency from ~35 ms to ~237 µs (99.3% faster)
and query_ranked_filtered_lexical from ~35 ms to ~23 µs (99.9% faster) on
a 5 000-thought chain; scales sub-linearly with chain size instead of linearly
- feat(daemon): per-read-operation sine-wave sounds — each logged read command
(search, list_chains, get_thought, etc.) plays a unique 60–150 ms chime when
MENTISDB_THOUGHT_SOUNDS=true; read sounds use pure sine waves in the 2 500–
4 500 Hz range, while write sounds remain square-wave at 250–1 000 Hz, making
the read/write distinction instinctive by pitch and timbre
- perf(dashboard): Agents page now loads in two phases — phase 1 renders an
instant skeleton from /dashboard/api/chains (fast registry read), then phase 2
fires parallel per-chain requests to /dashboard/api/agents/:chain_key; each
section populates independently as its response arrives, eliminating the
serial O(total chains × total thoughts) blocking that made the page unusable
with many chains
- fix(dashboard): align Recent Thoughts table columns on agent detail page —
the <tbody> was missing the Content <td>, causing the Date to slide under the
Content header and the Branch button to slide under the Date header; restored
the content preview cell and reordered cells to match headers
- docs: agent memory protocol post — Hermes MemoryProvider as community standard
- docs: OpenClaw × MentisDB integration post and protocol reassessment
0.9.3.39 APR/18/2026
- feat(tui): full ratatui-based terminal user interface replaces text startup output
— three-pane layout: top (banner + config + endpoints), middle (tabbed tables for
chains/agents/skills with FocusedPane cycling via Tab/Shift+Tab, mouse click focus,
contextual vim-style hint bar), bottom (agent primer + terminal-close warning);
all panes scrollable with clamped bounds; TerminalCleanup RAII guard ensures safe
terminal restoration on panic or early return; standard 8-color palette adapts to
dark and light terminal backgrounds
- feat(tui): drag-to-select copies any visible text — drag the mouse across any
content (log lines, chain/agent/skill rows, endpoints, config, primer); on mouse
release the selected rectangle is read from ratatui's live render buffer and
written to clipboard via arboard + OSC 52 (enables Cmd+C on macOS in iTerm2,
Terminal.app, kitty, WezTerm); live selection highlight via REVERSED cell style
- feat(tui): 'c' key copies focused item — chain key, agent ID, skill name, full
primer paste line, or visible log lines; 2-second green toast anchored near the
focused pane confirms every copy
- feat(tui): startup progress overlay — TUI renders immediately with a centered
modal showing current phase; update dialog appears as an in-TUI modal
- fix(tui): seamless single-TUI lifecycle — daemon runs one TUI instance for its
entire lifetime; eliminates the visible flash/crash between startup and running
phases that occurred when two TUI instances were started sequentially
- fix(tui): startup crash overlay — when startup fails, a red full-screen overlay
shows the error and keeps the TUI alive so the user can scroll logs before
quitting; previously the daemon exited silently
- fix(tui): log auto-scroll corrected for newest-first display — was pinned to the
oldest entry (scroll=len-1 in a reversed list); now pins to scroll=0 (top=newest)
- fix(tui): ratatui buffer swap — text extraction from drag-select now runs inside
the terminal.draw() closure via frame.buffer_mut(), which is the live pre-swap
buffer; current_buffer_mut() after draw() returns the cleared next-frame buffer
- fix(tui): overlays use Clear widget to prevent background content bleeding through
popup borders; startup overlay and update dialog are mutually exclusive
- fix(tui): accurate startup status — "Checking chains for migrations…" instead of
"Running migrations…" when no migrations are needed
- fix(tui): chain key and skill name columns auto-sized to longest entry + 1
- fix(tui): custom TuiLogger routes log output into TUI log buffer via mpsc channel,
preventing raw stderr from corrupting the ratatui alternate screen
- fix(tui): first-run setup wizard deferred to a separate terminal — wizard cannot
run while TUI owns the terminal in raw/alternate-screen mode; TUI shows a note
directing the user to run `mentisdb wizard` in a separate terminal instead
- perf(tui): render_logs no longer allocates an intermediate Vec<&str> on every
frame; single .iter().rev() pass reduces allocations at 10 Hz render rate
- refactor(tui): removed dead code left by a prior session — show_update_dialog(),
background_tip field, and background_launch_tip() were set/defined but never
called or rendered
- fix(stdio): proxy_jsonrpc_to_daemon now forwards raw JSON-RPC to the daemon's
streamable HTTP endpoint (POST /) instead of manual method mapping, supporting
the full MCP protocol (tools, resources, prompts, sampling, roots, notifications);
resources/read still served locally (embedded skill markdown)
- fix(daemon): remove broken `brew services start mentisdb` from macOS background
launch tip — the formula does not ship a launchd plist; startup messaging now
shows only the working nohup one-liner
- chore: make ratatui 0.30 a non-optional always-on server dependency
- fix: silence per-thought audio during benchmarks (MENTISDB_THOUGHT_SOUNDS=false)
- chore: gitignore pymentisdb build artefacts + *.mbak backups
0.9.2.38 APR/17/2026
- feat(stdio): smart daemon detection in stdio MCP mode — the stdio process now
checks if a daemon is running via the health endpoint, proxies to it if found,
or launches one in the background (nohup on Unix, start /B on Windows); all
MCP clients that spawn stdio subprocesses (Claude Desktop, Cursor, etc.) now
share the same live chain cache with zero configuration; falls back to local
mode if daemon launch fails
- fix(server): `start_servers` now shares ONE `MentisDbService` across the HTTP
MCP, HTTP REST, HTTPS MCP, HTTPS REST, and dashboard surfaces. Previously each
surface received its own `MentisDbService::new(...)` and therefore its own
`DashMap<chain_key, Arc<RwLock<MentisDb>>>`, causing a cross-surface split
brain: an append via REST was invisible to MCP (and vice versa) until the
daemon restarted and both services reloaded the chain from disk. Public
single-surface entry points (`start_mcp_server`, `start_rest_server`,
`start_https_*_server`) are unchanged. Adds regression test
`start_servers_shares_state_across_mcp_and_rest` that pre-warms the MCP
service, appends via REST, and asserts MCP immediately sees the new
`head_hash` / `thought_count` / `latest_thought.index`.
- feat: POST /v1/admin/flush REST endpoint — iterates all open chains and calls
flush() on each BinaryStorageAdapter; documented in server module header
- feat: backup/restore — new mentisdb backup / mentisdb restore subcommands create and
restore .mbak zip archives of the full MENTISDB_DIR; manifest with SHA-256 checksums for
every file; verification before any file is written on restore; --flush, --include-tls,
--overwrite flags; 33 integration tests in tests/test_backup.rs covering roundtrip, TLS
inclusion, idempotent restore, overwrite, corruption detection, chain count, byte totals,
manifest JSON round-trip, required vs optional file tagging, subdirectory creation
- feat: backup CLI auto-detects running daemon — calls POST /v1/admin/flush before
reading files; connection refused means daemon not running (files captured as-is);
other errors warn but proceed; BackupOptions docs updated to explain flush behavior
- feat: restore CLI interactive conflict prompt — when files exist in target dir and
--overwrite not passed, lists conflicting files and prompts user with boxed_yn_prompt;
user decline exits gracefully with no changes; restore is fully idempotent without prompt
- feat: terminal-close warning printed at daemon startup — yellow ⚠ notice that closing the
terminal stops mentisdb, followed by OS-specific background-launch tip:
macOS: nohup one-liner
Linux: systemd user service unit snippet or nohup one-liner
Windows: schtasks /create one-liner or Start-Process -WindowStyle Hidden
- feat: CLI help text updated for backup and restore subcommands — clarifies daemon flush
detection, idempotent restore behavior, interactive prompt, and --overwrite skip note
- fix(search): federated search dedup now keeps the higher-scoring duplicate occurrence,
matching the documented contract, instead of whichever chain surfaced the UUID first
- fix(relations): relation dedup no longer collapses distinct edges that differ by
`chain_key`, `valid_at`, or `invalid_at`; only exact duplicate relations are removed
- fix(webhooks): webhook registry persistence now uses temp-file-plus-rename instead of
truncating the live registry in place; delivery fan-out is now bounded by a queue and
concurrency semaphore so bursty append workloads do not spawn an unbounded number of
webhook tasks immediately; added tests for rename-failure preservation and bounded delivery
- fix: branch ancestor discovery is now transitive — branch-aware search can walk from child
to parent to grandparent chains instead of stopping at the direct parent only
- fix: restore now rejects path-traversal entries inside `.mbak` archives before any
file extraction; crafted manifest paths like `../escape.txt` or absolute paths now fail
restore with `InvalidData`, preventing writes outside the target directory
- fix: interactive restore confirmation now actually enables overwrite mode when the user
answers yes; previously the prompt implied overwrite but still passed `overwrite=false`
unless `--overwrite` was supplied explicitly
- fix: LLM-extracted memories no longer send an OpenAI `response_format` hint — some
OpenAI-compatible endpoints rejected the request with a schema/type error; MentisDB now
relies on the prompt plus strict JSON validation of the returned payload, preserving
portability across providers; added a regression-style test covering raw parse error output
- fix(pymentisdb): `context_bundles()` now decodes proper typed `ContextBundleSeed`
and `ContextBundleHit` objects instead of incorrectly nesting `ContextBundle` inside
`ContextBundle.seed`; `append_thought()` now uses one canonical `ThoughtInput`
payload builder instead of duplicated request assembly; `ranked_search()` and
`context_bundles()` now accept `MemoryScope` enums directly and serialize them to
the API's lowercase request form; added focused Python client tests
- fix: wizard — ensure_prerequisites for Claude Desktop now prompts before brew install on
macOS (previously silently ran brew install mcp-remote without asking); on macOS brew is
tried first, npm is the fallback; dead check_node_version and detect_brew_mcp_remote
functions removed; detect_mcp_remote_any() checks Homebrew paths then PATH
- fix: wizard — mcp-remote is now used directly without a node wrapper when installed as an
absolute executable (is_absolute_executable check); the shebang-based
is_directly_executable fallback is retained for npm-installed scripts with a good shebang;
integration_apply_tests updated to assert the correct behavior
- fix(bench): rebuild session_map after --limit filter in LoCoMo — on dev runs, ground-truth
evidence from filtered-out personas was leaking back into R@K scoring via
_expand_evidence() neighbor lookups; the fix rebuilds session_map on the kept item set
immediately after filtering; full-scale runs were unaffected
- chore(server): extract `invalid_input_error()` and `not_found_error()` helpers next to
`service_call`; replace four `Box::new(io::Error::new(...))` callsites with them for
tighter request-rejection plumbing (net: -8 lines of boilerplate, identical behavior)
- test(skill-signing): strengthened server-side signature-verification coverage for skill
uploads — now explicitly checks missing signature fields, unknown signing-key ids, and
tampered signatures with clearer failure assertions
- docs(whitepaper): rewrote WHITEPAPER.md in academic register with formal definitions
(Thought, Chain, BM25, RRF, Jaccard dedup, temporal validity), a tamper-evidence proof
sketch, and a References section; added WHITEPAPER.tex LaTeX port using
amsmath/amsthm/booktabs/\newtheorem; added build-whitepaper.sh with macOS and Ubuntu
install instructions, --open/--clean flags, and path_helper sourcing so it finds
pdflatex after a fresh BasicTeX install; added author affiliation (Universidad Católica
Andrés Bello, Venezuela)
- docs(whitepaper): corrected v0.8.9 benchmark numbers to match reproducible results —
LongMemEval R@10 changed 74.1% → 72.2%, added R@20 = 78.0%; filled in previously empty
LoCoMo-10P v0.8.9 single-hop (75.8%), multi-hop (57.4%), and R@20 (79.1%) cells; three
independent full-scale runs (2026-04-14, 2026-04-17 × 2) produced bit-identical scores;
added a reproducibility caption under the results table
- docs(blog): new ranked-search pipeline walkthrough (docs/mentisdb-ranked-search-pipeline.html)
covering filter-first selection, as-of temporal filter, three parallel scorers
(BM25, dense vector, graph BFS), smooth exponential fusion, session cohesion, RRF,
and deterministic sort — with file:line citations for every phase; prefaced with a
"Three Signals, Three Failure Modes" explainer (vocabulary match vs meaning match vs
relationship match) and a subsection on why BFS vs DFS for graph expansion
- docs(server,readme): introduce `invalid_input_error()` / `not_found_error()` helpers;
fill README coverage gaps for `source_episode`, `entity_type`, `BranchesFrom`,
`mentisdb_federated_search`, `mentisdb_extract_memories`, `mentisdb_list_entity_types`,
`mentisdb_upsert_entity_type` (MCP catalog count 37 → 42)
- docs(docs.mentisdb.com): add Advanced Retrieval section (RRF, context bundles, dense
vector sidecars, branching chains, federated search), Entity Types & Provenance
section (per-chain entity_type registry, source_episode), Webhook Callbacks, LLM-Extracted
Memories, and pymentisdb Python Client sections to user_docs.rs; add LLM-based reranking,
Operations & Admin (POST /v1/admin/flush, .mbak backup/restore, MENTISDB_DASHBOARD_PIN)
to developer_docs.rs
- docs: daemon endpoint catalog and REST rustdocs updated to include the live router
surface (`/v1/federated-search`, `/v1/import-markdown`, entity types, chain merge,
webhooks, extract-memories, admin flush, and related endpoints); startup catalog
generation now uses a shared helper to keep HTTP/HTTPS REST listings aligned
- docs: `mentisdb --help` now documents `MENTISDB_DASHBOARD_PIN`, matching the daemon's
supported environment variables and startup output
- docs: new content-ingestion tutorial blog post (`docs/mentisdb-content-ingestion.html`)
covering setup, REST/MCP/Rust usage, live-tested behavior, review-before-append workflow,
troubleshooting, and use cases for normal users, enterprise knowledge workers, and coders
- docs: new MentisDB vs Cognee comparison post (`docs/mentisdb-vs-cognee.html`) with a
fair architectural comparison and comprehensive side-by-side table; blog index updated to
surface the new posts first
- docs(skill): audited the live MentisDB project chain and found heavy concentration in
`Summary` / `LessonLearned` plus near-total collapse of typed edges to `References`;
revised `MENTISDB_SKILL.md` to push minimum backlinking, richer relation selection,
`Invalidates`, `LLMExtracted`, and higher-value underused types; added a new blog post
(`docs/mentisdb-memory-usage-audit.html`) documenting the findings and the fix
- docs: refreshed README, Rust docs, and `docs/llm-extracted-memories-design.md` to match
the shipped ingestion behavior: 31 thought types total, `gpt-4o` defaults, no auto-append,
no `response_format` requirement, and explicit JSON schema validation instead
- docs: `docs/mentisdb-backup-and-restore.html` blog post covering backup/restore CLI,
consistency model, SHA-256 manifest, .mbak format, REST endpoint, security notes
0.9.1.37 APR/14/2026
- feat: LLM-extracted memories — GPT-4o (or any OpenAI-compatible endpoint) converts
raw agent text/logs/transcripts into structured ThoughtInput records for review
and append; opt-in behind llm-extraction feature (default), uses openai-rust2
with 3× retry on 429/5xx, 60s timeout, ResponseFormat::JsonObject, temperature 0.1;
LlmExtractionConfig::from_env() reads OPENAI_API_KEY, LLM_BASE_URL, LLM_MODEL;
ExtractionResult includes model identifier and token usage; callers must review
and validate thoughts before appending
- feat: federated cross-chain search — ancestor_chain_keys() walks BranchesFrom
relations to discover parent chains; ranked search on branch chains transparently
includes ancestor chain results and annotates them with chain_key; enables fleet
memory hierarchies where specialist chains share a common base
- feat: pymentisdb Python client v0.9.0 — pip-installable Python package with
MentisDbClient (append_thought, ranked_search, context_bundles, list_chains,
upsert_agent, lexical_search, search, get_thought, traverse_thoughts, recent_context,
head, register/list/delete_webhook, skill methods, memory_markdown, import);
MentisDbMemory LangChain integration; full ThoughtType, ThoughtRole, MemoryScope,
ThoughtRelationKind enums; Typed relations; Complete working examples
- feat: wizard brew-first install — ensure_prerequisites tries brew install mcp-remote
before npm; added PrerequisiteStatus::Skipped variant; detect_brew_mcp_remote() reads
mcp-remote shebang to detect Homebrew-installed mcp-remote (which has a proper
shebang pointing to the correct Node version, usable directly without node wrapper)
- fix: wizard overwrite prompt changed to "Overwrite it with a fresh setup?" (was
"Overwrite or keep the existing config?") for clarity
- fix: CI doctest for LlmExtractionConfig uses unwrap_or_else instead of unwrap on
OPENAI_API_KEY to avoid panic in CI environments without the env var
- fix: LlmExtractionError::NotConfigured no longer panics on empty API key;
returns a proper error instead
- docs: two new blog posts covering complete MCP + REST integration:
docs/mentisdb-third-party-integrations.html (wizard setup, Claude Desktop
mcp-remote+Homebrew, OpenCode, REST API with full endpoint coverage, Python
requests examples, all 37 MCP tools, troubleshooting)
docs/pymentisdb-python-client.html (complete MentisDbClient API, LangChain
integration, typed relations, complete working example)
- docs(README): Python client section with pip install and example; consolidated
Setup Scenarios section for Claude Desktop, Claude Code, OpenCode, Codex, Qwen
- docs(docs.mentisdb.com): MCP section expanded with all 37 tools, REST section
back-to-back with complete endpoint coverage and request/response schemas
- docs: LLM extraction design doc (llm-extracted-memories-design.md) updated to
reflect openai-rust2 usage, extract_memories_from_text API, LlmExtractionConfig,
ExtractionResult, TokenUsage, LlmExtractionError variants
- docs(ENGINEERING_PIPELINE): Phase 5 step 5l added — doc audit requirement for all
public API changes reflected in docs/, README, docs.mentisdb.com, docs.rs
- bench: LoCoMo 10-persona R@10 72.0%, LongMemEval R@5 66.8% (fresh chain)
- WebhookManager registered at MentisDbService level, cloned per chain
- Registration: POST /v1/webhooks with url, chain_key_filter, thought_type_filter
- List: GET /v1/webhooks, Get one: GET /v1/webhooks/{id}, Delete: DELETE /v1/webhooks/{id}
- MCP tools: mentisdb_register_webhook, mentisdb_list_webhooks, mentisdb_delete_webhook
- WebhookPayload includes webhook_id, chain_key, thought full details, timestamp
- Delivery is fire-and-forget (tokio::spawn, non-blocking from append path)
- Exponential backoff retries: up to 5 attempts, max 32s delay
- Persisted to $MENTISDB_DIR/webhooks.json
- feat(search): irregular verb lemma expansion at query time in lexical search
- normalize_lexical_tokens() gains expand_lemmas bool parameter
- expand_lemmas=false at index time (avoids duplicate postings)
- expand_lemmas=true at query time (bridges went→go, ran→run, etc.)
- Uses existing lemmas.rs lookup table (160+ irregular verbs)
- Dedup logic (dedup_threshold) also uses expand_lemmas=true
- fix: duplicate closing braces in lexical.rs (syntax error from prior edit)
- test: add integration tests for webhook manager persistence and delete
0.8.8.34 APR/13/2026
- feat: add source_episode field to Thought for episode provenance tracking
- Optional episode/conversational context identifier groups related thoughts
- Added to Thought, ThoughtInput (with with_source_episode() builder),
ThoughtQuery (with with_source_episode() filter), thought_json output,
and dashboard API
- Not included in thought hash (backward compatible, like entity_type)
- feat(search): add opt-in LLM-based reranking for ranked search
- New fields on RankedSearchQuery: llm_rerank (bool), llm_model (Option<String>),
llm_rerank_top_k (usize, default 20), with_llm_reranking() builder
- RankedSearchScore gains llm_score field (0.0 when disabled)
- MentisDb::llm_rerank_candidates() calls OpenAI or Anthropic API with graceful
fallback on failure (falls back to original ranking)
- REST: POST /v1/search/rerank dedicated endpoint
- LLM reranking is query-time only, no persistence required
- feat(search): verify and fix cross-chain graph query traversal
- resolve_context_by_id now properly skips cross-chain relations
(those with chain_key.is_some()) during local traversal
- ThoughtAdjacencyIndex correctly tracks cross-chain targets via ThoughtLocator::cross_chain
- Graph expansion works across chains via BranchesFrom relations
- ancestor_chain_keys() discovers parent chains for cross-chain ranked search
- test: add 8 search sanity-check regression tests
- query_with_tags_any_returns_all_matching_thoughts
- query_with_types_returns_only_matching_types
- query_returns_empty_when_no_match_exists
- query_with_text_search_finds_content
- query_with_text_search_is_case_insensitive
- query_filter_by_importance_threshold
- entity_type_filter_returns_only_matching_entity_types
- cross_chain_relation_traversal, query_respects_cross_chain_chain_key
- test: LLM reranking tests (llm_rerank_disabled_by_default,
llm_rerank_falls_back_on_failure, llm_rerank_top_k_respected)
- test: source_episode tests (source_episode_field_round_trips,
source_episode_filter_in_query, source_episode_none_when_not_set)
0.8.7.33 APR/13/2026
- feat(entity_type): add entity_type field to Thought struct with V3 backward-compatible deserialization
- Thought struct gains optional `entity_type: Option<String>` field with `#[serde(default)]`
- Per-thought version detection for V3 chains with and without entity_type field
- `ThoughtInput` gains `entity_type` field, passed through builder pattern
- `Thought::with_entity_type()` builder method
- entity_type is NOT included in the thought hash (backward-compatible)
- bincode 2.0.1 respects `skip_serializing_if` — must NOT use it on entity_type
- feat(entity_type): EntityTypeRegistry per chain with auto-observe, persistence, query filter
- New `EntityTypeRegistry` struct tracking first_seen_index, last_seen_index, count
- Auto-observed on every append when entity_type is set
- Persisted as `chain_key-entity-types.json` alongside the chain binary
- `list_entity_types()`, `get_entity_type()`, `upsert_entity_type()` on MentisDb
- MCP endpoint: `POST /v1/entity-types` for upsert, `GET /v1/entity-types` for list
- MCP tool: `mentisdb_upsert_entity_type`
- ThoughtQuery gains `entity_type` filter for semantic filtering
- Dashboard search UI gains entity_type filter input
- feat(dashboard): display entity_type badge in thought rows and add entity_type filter
- `thought_json()` now includes entity_type field in output
- entity_type badge next to thought_type badge in all thought row views
- entity_type badge in thought detail modal
- DashboardSearchQuery gains entity_type filter parameter
- dashboard_search_filter wires entity_type to ThoughtQuery
- fix(dashboard): convert branch/delete/merge modals to popup overlays
- Modals now use .modal-overlay hidden popup dialogs instead of inline display:none divs
- Moved modal HTML and JS handlers to top-level of script for availability from any page
- Added X close buttons and backdrop click-to-close for all modals
- All style.display toggles replaced with classList.add/remove('hidden')
- fix(server): resolve_chain_key now trims and filters empty strings before fallback
- Added `.map(str::trim).filter(|value| !value.is_empty())` before `unwrap_or`
- Matches the pattern already used in `resolve_agent_identity`
- Handles `Some("")` and `Some(" ")` correctly
- fix(wizard): continue past failed integrations instead of aborting (Issue #14)
- `ensure_prerequisites` returns `PrerequisiteStatus` (Ok or Warning) not errors
- `apply` loop in wizard.rs catches per-integration failures and continues
- `apply_setup_with_environment` returns warnings not errors for soft checks
- fix(dashboard): _openBranchChain accepts optional thoughtId parameter
- Branch button on thought rows passes thought UUID to the modal
- fix(dashboard): define global onclick handlers early as stubs with queue-draining
- Branch/delete/merge onclick handlers are now top-level functions available from any page
- fix(server): accept schema v4+ chains by trying current Thought deserialization first
- Changed `v == MENTISDB_CURRENT_VERSION` to `v >= MENTISDB_CURRENT_VERSION`
- Falls back to LegacyThoughtV2 for older schemas
- Schema version normalized to MENTISDB_CURRENT_VERSION on next persist
0.8.6.32 APR/12/2026
- feat(search): Reciprocal Rank Fusion (RRF) reranking for ranked search
- New `src/search/ranked.rs` with `rrf_merge()`, `rrf_merge_three()`, RRF_K=60
- `RankedSearchQuery` gains `enable_reranking: bool` (default false) and
`rerank_k: usize` (default 50) fields with `with_reranking()` builder
- When enabled, three single-signal rankings (lexical-only, vector-only,
graph-only) are merged via `1/(k + rank)` and replace the additive total;
importance, confidence, recency, session cohesion, and graph signals are
added back as small additive adjustments
- `RankedSearchScore` gains `rrf: f32` field (0.0 when reranking disabled)
- REST and MCP transports pass `enable_reranking`/`rerank_k` through
- 6 unit tests for RRF merge logic
- feat(search): per-field BM25 document-frequency cutoffs (global DF)
- `Bm25DfCutoffs` struct on `LexicalQuery` with configurable cutoff ratios
per field: content, tags, concepts, agent_id, agent_registry
- Terms whose global DF exceeds `cutoff_ratio * N` are filtered from that
field only — preserves original stop-word suppression while allowing
per-field tuning
- Default cutoffs: content=0.30, tags=0.30, concepts=0.30, agent_id=0.70,
agent_registry=0.60
- `with_df_cutoffs()` builder on `LexicalQuery`
- feat(search): irregular verb lemma expansion
- `src/search/lemmas.rs` with ~170 irregular English verb mappings
- Query tokenization expands "went"→"go", "gave"→"give", "ran"→"run", etc.
- Indexed content is NOT changed — lemma expansion is query-time only
- feat(branching): BranchesFrom relation and chain branching
- `ThoughtRelationKind::BranchesFrom` for cross-chain divergence points
- `MentisDb::branch_from()` creates a new chain with a genesis thought
pointing back to the branch-point thought on the source chain
- `ancestor_chain_keys()` walks BranchesFrom relations to discover parent
and grandparent chains
- Ranked search on branch chains transparently searches ancestor chains
and annotates results with chain_key
- REST endpoint: `POST /v1/chains/branch`
- MCP tool: `mentisdb_branch_from`
- fix(search): use global DF for Bm25DfCutoffs comparison
- Per-field DF calculation was less aggressive than original global DF check,
letting high-frequency terms pass through and adding noise
- Changed to use global DF (postings list length) for cutoff comparison,
matching the original stop-word suppression behavior
- docs(whitepaper): expanded BM25 explanation, per-field DF cutoffs table,
RRF section 6.8
- bench: LoCoMo 73.0% R@10 (fresh chain), LongMemEval 57.6% R@5
0.8.5.31 APR/11/2026
- fix(cli): panic in update/force-update subcommands — cannot start tokio runtime within runtime
- run_update_standalone() created a nested tokio runtime inside #[tokio::main],
causing "Cannot start a runtime from within a runtime" panic
- Made run_update_standalone async and .await'd fetch directly instead of rt.block_on()
- Added test: parse_daemon_args_accepts_update_subcommands verifies update/force-update parse
0.8.4.30 APR/11/2026
- fix(updater): move update check before chain migrations
- Users who couldn't open mentisdb after upgrading were stuck because
the update check ran AFTER migrations, which crashed on the old binary.
Now the update check runs FIRST, giving users the fix before their
chains are touched.
- If an update is found and accepted, mentisdb exits with instructions
to restart. No automatic in-process restart — simpler and safer.
- feat(cli): add `mentisdb update` and `mentisdb force-update` subcommands
- `mentisdb update` checks for a newer release and prompts y/N to install
- `mentisdb force-update` installs the latest release unconditionally,
even if the current version appears up to date — useful for repairing
a corrupted binary or forcing a fresh install
- Both use the same cargo install path as the startup update check
- refactor(updater): remove async update check task and restart channel
- The update check now runs synchronously before migrations instead of
in a background tokio task after servers start
- Removed dead code: run_update_check_task(), RestartRequest,
shutdown_all_servers(), restart_installed_binary()
0.8.3.29 APR/10/2026
- fix(migration): per-thought schema detection for mixed-schema binary chains
- Users upgrading from 0.8.1 to 0.8.2 encountered a crash when opening existing
chains: "UnexpectedVariant { type_name: Option<T>, found: 64 }" or
"UUID parsing failed: invalid length". We sincerely apologize for the snag.
- Root cause: the old load_binary_thoughts() peeked only the FIRST thought's
schema_version and applied one legacy format to the entire chain. Chains
containing thoughts from multiple schema versions (e.g. V1 thoughts followed
by V2 thoughts appended by a later daemon) would fail deserialization because
the wrong struct layout was used for later thoughts.
- Fix: replace single-schema-path approach with load_binary_thoughts_per_thought()
which reads each thought's payload individually, peeks its schema_version
varint, and dispatches to the correct legacy deserializer (V0/V1 with
2-field ThoughtRelation, V2 with 3-field, or V3 direct). Rebuilds the hash
chain for migrated thoughts after loading.
- Remove dead migrate_v2_thoughts() function whose logic is now inline.
- test(migration): comprehensive migration tests for all schema versions
- migrate_v2_chain_with_all_relation_kinds: all 11 ThoughtRelationKind variants
with and without chain_key, verifying valid_at/invalid_at are None after migration
- migrate_v1_chain_with_all_thought_types: all 30 ThoughtType variants survive
- migrate_v2_chain_with_all_thought_roles: all 8 ThoughtRole variants survive
- migrate_mixed_chain_with_v1_v2_v3_thoughts: per-thought detection across
V1, V2, and V3 thoughts in the same chain
- migrate_v2_chain_with_signed_thought: signing_key_id and thought_signature
- migrate_v1_chain_with_relations: 2-field LegacyTestRelation migrates to 5-field
ThoughtRelation with chain_key=None, valid_at=None, invalid_at=None
- docs(whitepaper): complete rewrite covering 0.8.2 features
- 12 sections: architecture, schema evolution, semantic memory model, storage,
query & retrieval, dedup, CLI, MCP, benchmarks, competitive landscape, future
0.8.2.28 APR/10/2026
- feat(temporal): add temporal edge validity with schema V3 migration
- Add valid_at/invalid_at fields to ThoughtRelation for explicit temporal
bounds (e.g. "this fact was true from 2024 to 2025")
- invalidated_thought_ids: HashSet<Uuid> built at chain open time from all
Supersedes, Corrects, Invalidates relations for O(1) superseded detection
- as_of: Option<DateTime<Utc>> on RankedSearchQuery enables point-in-time
queries: excludes thoughts appended after the timestamp and thoughts
superseded by thoughts appended at or before the timestamp
- Schema V3 with full V2→V3 migration: LegacyThoughtRelationV2 (3-field),
LegacyThoughtV1, migrate_v2_thoughts() with hash chain rebuild
- Version-aware load_binary_thoughts() with peek_first_schema_version()
- Bincode empty-Vec fast-path guard: always check schema_version after
successful deserialization to detect silently-wrong old chains
- REST API: RankedSearchRequest exposes as_of parameter
- append_thought() auto-sets valid_at = Some(now) on new relations
- ThoughtRelation::new(kind, target_id) convenience constructor
- open_with_storage() persists migrated chains so future opens use native
- migrate_legacy_chain_v0() generalized to detect actual from_version
- REST API: rest_append_handler includes valid_at/invalid_at fields
- feat(dedup): add automatic memory deduplication with Jaccard similarity
- When dedup_threshold is set, each new thought's normalized lexical tokens
are compared against recent thoughts using Jaccard similarity
- If the best match exceeds threshold (0.0–1.0), a Supersedes relation is
auto-added pointing to the most similar prior thought
- dedup_scan_window (default 64) limits how many recent thoughts are scanned
- Configurable via MENTISDB_DEDUP_THRESHOLD and MENTISDB_DEDUP_SCAN_WINDOW
- MentisDb::with_dedup_threshold() and with_dedup_scan_window() builders
- MentisDbServiceConfig: with_dedup_threshold(), with_dedup_scan_window()
- invalidated_thought_ids updated correctly on auto-supersede
- feat(scopes): add multi-level memory scopes via tag-based MemoryScope
- MemoryScope enum: User (default), Session, Agent — stored as scope:user,
scope:session, scope:agent tags to avoid schema migration
- ThoughtInput::with_scope() adds the tag automatically
- RankedSearchQuery::with_scope() filters by scope tag
- REST API: AppendThoughtRequest.scope and RankedSearchRequest.scope
- MemoryScope::as_tag(), from_tag(), Display impl for round-tripping
- feat(cli): add add, search, agents subcommands via daemon REST
- mentisdb add <content> [--type] [--scope] [--tag] [--agent] [--chain] [--url]
- mentisdb search <query> [--limit] [--scope] [--chain] [--url]
- mentisdb agents [--chain] [--url]
- Default REST URL: http://127.0.0.1:9472
- Uses ureq for synchronous HTTP (no async runtime needed)
- AddCommand, SearchCommand, AgentsCommand structs in cli::args
- Updated help text with new subcommand documentation
- fix(clippy): remove dead load_legacy_v0_thoughts/load_legacy_v0_binary_thoughts
functions and convert match→if let in load_binary_thoughts fast path
0.8.1.27 APR/10/2026
- feat(search): session cohesion boost and improved importance/vector/graph scoring
- Session cohesion: thoughts within ±8 positions of a high-scoring lexical
seed (score >= 3.0) receive a proximity boost up to 0.8, decaying linearly
with distance — surfaces evidence turns adjacent to the matching turn but
sharing no lexical terms; seeds with lexical≥5.0 excluded from boost;
self excluded via nearest>0 guard
- Importance scoring: replace flat 3.0× multiplier with differential boost
proportional to lexical score (lexical × (importance-0.5) × 0.3)
- Vector-lexical fusion: replace step-function boosts with smooth exponential
decay; VECTOR_ONLY_BOOST=35, VECTOR_DECAY_RATE=3.0 — faster decay reduces
vector interference with moderate-lexical results, improving ranking
precision for queries with partial lexical overlap
- BM25 DF cutoff lowered 50%→30% — filters non-discriminative entity names
that appear in many turns without contributing relevance signal
- ContinuesFrom relation kind boost raised 0.12 → 0.30; Corrects/Invalidates
0.18 → 0.25; Supersedes 0.16 → 0.22; DerivedFrom 0.14 → 0.20
- Graph proximity score raised 0.3/depth → 1.0/depth; seed support per extra
seed 0.05 → 0.1 (max 5); MAX_GRAPH_SEEDS cap of 20 to bound BFS cost
- Add session_cohesion field to RankedSearchScore for observability
- LoCoMo 2-persona R@10: 55.8% (baseline) → 88.7%, single-hop 90.7%
- LoCoMo 10-persona R@10: 74.2% (full benchmark, 1977 queries)
- LongMemEval R@5: 65.0% → 67.6%, R@10: 70.6% → 73.2%
- feat(locomo-benches): expand evidence with session-neighbor turns
- _expand_evidence() broadens gold evidence to ±2 turns around each target
within the same session group for fairer hit evaluation
- Build session_map in load_locomo() for O(1) neighbor lookup
- Show up to 5 evidence snippets in miss output (was 3)
- fix(dashboard): guard against NaN/Infinity floats and null agents
- with_confidence() and with_importance() now reject non-finite floats
(NaN.clamp() returns NaN in Rust, which crashes serde_json serialization)
- thought_json() sanitizes confidence/importance before json!() macro
- renderAgentList() filters null agents to prevent JS TypeError
- fix(server): propagate chain_key in RelationInput for cross-chain relations
- RelationInput struct was missing the chain_key field; all cross-chain
relation writes via REST and MCP silently defaulted to None, dropping
the target chain reference and breaking cross-chain graph traversal
- fix(server): add session_cohesion field to RankedSearchScoreResponse
- Missing from initial session_cohesion commit which only added it to the
dashboard response; REST /v1/ranked-search would fail to compile
- fix(dashboard): add session_cohesion field to DashboardRankedScoreResponse
- Mirrors the new session_cohesion field in RankedSearchScore so the
dashboard serialization compiles and exposes the new score component
- fix(ci): rustfmt formatting issue in session cohesion code
- Use abs_diff() for correct single-line formatting that passes CI check
0.8.0.26 APR/08/2026
- feat(search): Porter stemming in lexical tokenizer (normalizer v2)
- Added rust-stemmers dependency with English Porter stemmer
- normalize_lexical_tokens() now stems all tokens before indexing and
querying so word variants share a common root (e.g. prefers/preferred/
preferences → prefer)
- LEXICAL_NORMALIZER_VERSION bumped to 2 to force index rebuilds
- LongMemEval R@5 improved from 57.2% to 61.6% (stemming alone)
- feat(search): tiered vector-lexical fusion in ranked-search scoring
- Replace flat lexical+vector addition with tiered boost: vector gets
60× when lexical==0, 20× ramp when lexical<1.0, additive otherwise
- Surfaces semantically-relevant hits without overriding strong lexical
matches (fixes preference-category regression from previous RRF approach)
- LongMemEval R@5 improved from 61.6% to 65.0% (fusion + importance)