SIGN IN SIGN UP

[new model] step 3.7 (#46658)

* mini model original

* mini model and modular

* update modular

* update moe logic to match standard

* updates debug scripts

* original code, debug scripts, mini model, etc.

* fix sparseblockmoe and decoderlayer to match standards, by converting config

* slim down moe modular, fix config conversion

* rm Step3p7CausalLMOutputWithPast, unused functions, return BaseModelOutputWithPast

* standard renaming

* tests and siglip vision encoder

* build mini checkpoint in right format + conversion script

* new demo and compare outputs workflow

* major cut of modular, no image_embeds, etc.

* fix tests

* remove debug files

* address feedback - add processor and config to modular

* apply more feedback, reduce image processing

* apply round 2 of feedback!

* more feedback and weight converter

* feedback

* gemma3attention and rm SharedExperts class

* Untrack local debug scripts

* Untrack local debug scripts p2

* Untrack local debug scripts p3

* Revert core model loading changes

* standard rope

* clean up

* add tok

* tests integration

* docs and tests unskip

* tests

* laguana parents

* fix step3p5_vision conversion after rebase onto main

VisionUnfuseAndPermuteForRope was deprecated upstream in favor of
composing Chunk()+PermuteForRope() directly; migrate the two
step3p5_vision entries accordingly. Also regenerate modeling_step3p7.py
for main's updated RotaryEmbedding converter template.

* add checkpoint links to step3p7 config docstrings

check_config_docstrings.py requires every registered config class to
link a real checkpoint via @auto_docstring(checkpoint=...); add the
stepfun-ai/Step-3.7-Flash link to all three step3p7 config classes.

* allowlist step3p7's unused legacy config attributes, add contribution date

check_config_attributes.py flags config fields never read in the
modeling file; add Step3p7TextConfig/Step3p7VisionConfig entries for
the legacy hub-config kwargs and inherited MoE-aux-loss fields this
architecture doesn't use (each already documented as such in the
config docstrings). Also add the missing contribution date to
step3p7.md via utils/add_dates.py.

* more feedback / cleanup!

* modular conversion missed

* modualr

* mini checkpoint

* feedback

* modeling from converter

* revert core model loading changes from rebase

* config attrs

* date md

* Drop redundant num_heads arg from Step3p7Attention

Step3p7Attention.__init__ took config and num_heads separately even
though the config passed in already carries num_attention_heads
(mlinter TRF029). Derive it internally instead.

* add date

* rebase and date

---------

Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-171-249.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-175-19.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-178.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-220.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-236.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-95.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-175-132.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-196.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-169-247.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-165-59.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-165-131.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-58.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-175-241.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-73.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-192.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-175-165.ec2.internal>
Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-68.ec2.internal>
I
Ita Zaporozhets committed
c49429ed1f8b89749de77c0ec930ef19685c9ae5
Parent: 64b6595
Committed by GitHub <noreply@github.com> on 8/16/2026, 4:30:26 PM