Skip to content

@bigscience-workshop BigScience Workshop

Change the repository type filter

All

    Repositories list

    36 repositories

    • data_tooling

      Public
      Tools for managing datasets for governance and training.
      HTML
      •
      Apache License 2.0
      •47•90•138•3•Updated Mar 16, 2026Mar 16, 2026

    • ShadesofBias

      Public
      Evaluation for Shades of Bias in Text
      HTML
      •2•9•2•0•Updated Apr 23, 2025Apr 23, 2025

    • biomedical

      Public
      Tools for curating biomedical training data for large-scale language modeling
      Python
      •119•496•164•16•Updated Dec 9, 2024Dec 9, 2024

    • xmtf

      Public
      Crosslingual Generalization through Multitask Finetuning
      bloomlanguage-modelszero-shot-learning
      bloomlanguage-modelszero-shot-learningmultitask-learningt5multilingual-nlplarge-language-modelsinstruction-tuningbloomzmt0
      Jupyter Notebook
      •
      Apache License 2.0
      •43•536•12•0•Updated Sep 22, 2024Sep 22, 2024

    • petals

      Public
      🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
      nlpbloomdistributed-systems
      nlpbloomdistributed-systemsmachine-learningdeep-learningchatbotpytorchfalcontransformerneural-networks
      Python
      •
      MIT License
      •601•10k•92•20•Updated Sep 7, 2024Sep 7, 2024

    • bigscience

      Public
      Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
      nlptrainingmachine-learning
      nlptrainingmachine-learningmodels
      Shell
      •
      Other
      •101•1k•13•7•Updated Jul 29, 2024Jul 29, 2024

    • Megatron-DeepSpeed

      Public
      Ongoing research training transformer language models at scale, including: BERT & GPT-2
      Python
      •
      Other
      •226•1.4k•76•47•Updated Mar 20, 2024Mar 20, 2024

    • multilingual-modeling

      Public
      BLOOM+1: Adapting BLOOM model to support a new unseen language
      Python
      •
      Apache License 2.0
      •17•74•13•6•Updated Mar 2, 2024Mar 2, 2024

    • promptsource

      Public
      Toolkit for creating, sharing and using natural language prompts.
      nlpmachine-learningnatural-language-processing
      nlpmachine-learningnatural-language-processing
      Python
      •
      Apache License 2.0
      •379•3k•11•32•Updated Oct 23, 2023Oct 23, 2023

    • massive-probing-framework

      Public
      Framework for BLOOM probing
      Python
      •9•9•0•0•Updated Oct 17, 2023Oct 17, 2023

    • architecture-objective

      Public
      Python
      •
      Apache License 2.0
      •343•99•4•5•Updated Jul 25, 2023Jul 25, 2023

    • metadata

      Public
      Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.
      Python
      •
      Apache License 2.0
      •11•30•25•13•Updated Jun 12, 2023Jun 12, 2023

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of autoregressive language models.
      Python
      •
      MIT License
      •3.2k•105•7•8•Updated May 9, 2023May 9, 2023

    • data-preparation

      Public
      Code used for sourcing and cleaning the BigScience ROOTS corpus
      multilingualdatasetlarge-language-models
      multilingualdatasetlarge-language-models
      Jupyter Notebook
      •
      Apache License 2.0
      •43•318•10•0•Updated Mar 20, 2023Mar 20, 2023

    • bibliography

      Public
      A list of BigScience publications
      researchbibtexbibliography
      researchbibtexbibliography
      TeX
      •
      Apache License 2.0
      •1•3•1•0•Updated Mar 13, 2023Mar 13, 2023

    • multilingual-modeling-1

      Public
      Python
      •
      Apache License 2.0
      •17•2•0•0•Updated Dec 5, 2022Dec 5, 2022

    • t-zero

      Public
      Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)
      Python
      •
      Apache License 2.0
      •53•464•8•2•Updated Nov 5, 2022Nov 5, 2022

    • carbon-footprint

      Public
      A repository for `codecarbon` logs.
      Jupyter Notebook
      •5•13•2•0•Updated Nov 3, 2022Nov 3, 2022

    • pii_processing

      Public
      PII Processing code to detect and remediate PII in BigScience datasets. Reference implementation for the PII Hackathon
      Python
      •
      Other
      •6•9•7•1•Updated Oct 6, 2022Oct 6, 2022

    • lam

      Public
      Libraries, Archives and Museums (LAM)
      Apache License 2.0
      •8•89•34•0•Updated Oct 4, 2022Oct 4, 2022

    • model_card

      Public
      Apache License 2.0
      •5•26•0•0•Updated Jul 11, 2022Jul 11, 2022

    • bloom-dechonk

      Public
      A repo for running model shrinking experiments
      Python
      •4•10•0•0•Updated Jun 21, 2022Jun 21, 2022

    • historical_texts

      Public
      BigScience working group on language models for historical texts
      Jupyter Notebook
      •7•8•0•2•Updated May 10, 2022May 10, 2022

    • evaluation

      Public
      Code and Data for Evaluation WG
      Python
      •
      Other
      •24•42•41•9•Updated May 4, 2022May 4, 2022

    • catalogue_data

      Public
      Scripts to prepare catalogue data
      Jupyter Notebook
      •
      Apache License 2.0
      •1•8•5•3•Updated Apr 25, 2022Apr 25, 2022

    • training_dynamics

      Public
      4•5•11•2•Updated Apr 22, 2022Apr 22, 2022

    • evaluation-robustness-consistency

      Public
      Tools for evaluating model robustness and consistency
      Python
      •
      Other
      •2•2•0•2•Updated Mar 9, 2022Mar 9, 2022

    • interpretability-ideas

      Public
      1•1•10•0•Updated Feb 27, 2022Feb 27, 2022

    • tokenization

      Public
      Python
      •
      Apache License 2.0
      •2•11•1•2•Updated Feb 16, 2022Feb 16, 2022

    • datasets_stats

      Public
      Generate statistics over datasets used in the context of BS
      Makefile
      •1•2•0•0•Updated Feb 1, 2022Feb 1, 2022
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.