Make simsimd an Optional Dependency with Automatic NumPy/SciPy Fallback for Vector Distance Computation (#249)
## Summary This PR makes `simsimd` an optional dependency within the `langchain-weaviate` integration by introducing a robust NumPy/SciPy-based fallback implementation for cosine distance computation. Systems with older glibc versions or restricted environments cannot install `simsimd`, which currently results in import errors and prevents the package from functioning. This update ensures full compatibility across environments while preserving the accelerated vector math path where available. ## Motivation `simsimd` provides excellent SIMD-optimized vector math, but it is not universally installable—especially on older Linux distributions where glibc upgrades are risky or not feasible. This PR resolves that by: - Making `simsimd` optional - Falling back gracefully to NumPy/SciPy when unavailable - Preserving feature parity with existing behavior - Improving reliability across diverse production environments ## Changes Introduced 1. Optional `simsimd` Import - `simsimd` import is wrapped with a `try/except`. - If unavailable, a warning is logged and NumPy/SciPy fallback is used. 2. New Internal `cdist` Function - Provides unified API (`cdist(X, Y, metric="cosine")`) - Uses `simsimd.cdist` when available - Falls back to `scipy.spatial.distance.cdist` or a custom NumPy implementation 3. Updated `cosine_similarity` - Now delegates cosine distance computation to the new `cdist` function - Behaviour preserved for both accelerated and fallback paths 4. Test Coverage Added tests validating: - Fallback execution when `simsimd` is unavailable - Correct distance outputs using NumPy fallback - Error when unsupported metrics are passed All new tests pass successfully. 5. Documentation Updates README updated with a new section describing: - Optional `simsimd` acceleration - Installation instructions (`pip install "langchain-weaviate[simsimd]"`) - Behavior when running without acceleration 6. pyproject.toml Updates - `simsimd` moved to optional dependency extras - Ensures backward compatibility and avoids installation failures ## Backward Compatibility This update is fully backward compatible: - Users with `simsimd` installed experience **no change** in behavior. - Users without `simsimd` can now install and use the package normally. - No public APIs were modified. ## Additional Notes - This PR improves the portability and reliability of the library in production environments. - The fallback path ensures correct numerical output and alignment with existing interfaces. - Maintainers can extend this pattern to other optional acceleration modules in the future. ## Checklist - [x] Optional import and fallback logic added - [x] New `cdist` function implemented - [x] Updated `cosine_similarity` to use unified distance API - [x] Tests added and passing - [x] README updated - [x] pyproject.toml updated - [x] Code linted and type-checked --- Thank you for reviewing this PR. I’m happy to incorporate any suggestions or refinements. --------- Co-authored-by: Duda Nogueira <dudanogueira@gmail.com>
B
Boniface Alexander committed
78990aba39938ccdb304621751c39b61d5cbdc8e
Parent: cee6b12
Committed by GitHub <noreply@github.com>
on 1/7/2026, 7:31:59 PM