SIGN IN SIGN UP

feat(components): add MrScraper integration (#14516)

* feat(components): add MrScraper integration

Adds MrScraper as an lfx-bundles provider with eight components for AI
scraping, website crawling, rendered HTML fetching, batch reruns, and
result retrieval.

Rebased from main onto release-1.12.0. The migration table entries use the
shape for a bundle that is new in this release -- one bare_class_name row
per component, added_in 1.12.0 -- rather than the four-row shape reserved
for components ported out of lfx.components, which MrScraper never was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(bundles): pin the MrScraper SDK floor and normalize component output

Four things the integration left open.

The SDK floor was >=0.1.2 while the lock resolves 0.2.1. For a 0.x package
those are not compatible, and nothing in CI would have caught the gap: the
suite stubs mrscraper when it is absent and patches mrscraper.MrScraper
everywhere else, so it validated the components against a mock of
themselves. Floor is now >=0.2.1,<0.3.0, and TestSdkContract asserts every
keyword the components pass is still a parameter of the installed SDK. Those
tests skip only where conftest had to register a stub; the cross-bundle job
installs lfx-bundles[all-no-torch], so they execute there.

Seven of the eight components returned the SDK's HTTP envelope
({status_code, data, headers}) as their Data, so a flow had to reach past
transport metadata to get scraped content, while Get Result Detail alone
unwrapped it. mrscraper_common.payload() now strips the envelope for all
eight and boxes a non-dict payload -- fetch_html returns text for non-JSON
responses -- so the result is always a mapping. Client construction moved
there too, replacing the eight copies of the missing-SDK message.

Get Results gained a DataFrame output so a page of results can feed table
and loop components directly. The row list is located by shape rather than
by key name, since the API's pagination envelope is not documented; a rename
degrades to a single-row frame instead of raising. Worth confirming the key
and making it explicit.

Fetch Rendered HTML described itself as doing 'bot-detection evasion'. Now
it states the capability rather than selling it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(ci): track the mrscraper bundle in the release inventory contract

The python-full profile enumerates every long-tail bundle package, so a new
lfx_bundles/mrscraper directory has to be listed there too — otherwise
test_contract_tracks_every_long_tail_bundle reports the drift.

* chore(bundles): bump lfx-bundles to 1.1.15 for the MrScraper components

The release-plan guard requires a version bump whenever releasable bundle
source changes, and this PR adds a new lfx_bundles/mrscraper package plus an
SDK dependency while the version stayed at release-1.12.0's 1.1.14.

The check was green earlier only because the PR still targeted main: main's
lfx-bundles is 1.1.12, so release-1.12.0's own bumps, carried in by the
rebase, read as a bump belonging to this PR.

Produced by scripts/ci/bundle_release_plan.py update, which also raises the
root bundles-extra floor and refreshes uv.lock in the same operation.

---------

Co-authored-by: riandradiva <riandra@mrscraper.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
M
MrScraper committed
0c4b12f2dfcebcb7bc80c4ce4814e15e91d109ed
Parent: be8984a
Committed by GitHub <noreply@github.com> on 8/19/2026, 3:26:39 AM