Experience
Education
-
-
-
-
Other similar profiles
Explore more posts
-
Iñaki Lacunza
I'm glad to be part of this amazing work, where we introduce 𝗠𝗿𝗕𝗘𝗥𝗧, a family of 150M–300M parameter encoders built on ModernBERT and pre-trained on 35 languages and code. Through targeted language and domain adaptation, we developed efficient Spanish and Catalan variants, as well as specialized biomedical and legal models via continued pre-training, achieving state-of-the-art performance while maintaining strong efficiency. We also integrate Matryoshka Representation Learning to enable flexible embedding sizes, helping balance quality, latency, and storage constraints in real-world retrieval systems. MrBERT shows that modern encoder architectures can still set the state of the art when carefully optimized for multilingual and domain-specific settings, while remaining practical for production deployment. 🤖 Models link: https://lnkd.in/ewggPJC6 📜 Paper link: https://lnkd.in/eGGAtqhb Huge thanks to Daniel Tamayo, Paula Rivera Hidalgo de Torralba and the other contributors for this amazing work!! 👏 👏
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content