causal-language-modeling

Repository for My HuggingFace Natural Language Processing Projects

Auto-regressive causal language model for molecule (SMILES) and reaction template (SMARTS) generation based on the Hugging Face implementation of OpenAI's GPT-2 transformer decoder model

Codebase for Linguistic Collapse: Neural Collapse in (Large) Language Models [NeurIPS 2024] [arXiv:2405.17767]

Transformers Intuition

Links to my repositories, where I implement a wide variety of Natural Language Processing models using TensorFlow and Hugging Face.

A multi-threaded GitHub scraper to collect Python code with docstrings from public repositories, creating a well-documented dataset for the JaraConverse LLM model.

Open source implementation of AMALIA: A Fully Open Large Language Model for European Portuguese

CUDA-native residual-frame Delta memory for causal language models: full-matrix prefix geometry, rank-one recurrent updates, exact local similarity, Hugging Face integration, and CUDA Graph training.

Change the Bert model to a GPT-style autoregressive decoder.

This repository is for the paper Lexical Substitution as Causal Language Modeling. In Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024), Mexico City, Mexico. Association for Computational Linguistics.

HelixLM - a recurrent heterogeneous graph neural LLM combining biological neural column random wiring, hybrid attention, and Mamba-2 for hyperpersonalized and on-device AI.

Causal language modeling and intent classification using GPT-2.

A quick and easy way to interact with open-source LLMs.

Course materials for the Machine Learning for NLP course taught by Sameer Singh for the Cognitive Science summer school 2022.

An AI generated picturebook.

Fine-tuning (or training from scratch) the library models for language modeling on a text dataset for GPT, GPT-2, ALBERT, BERT, DitilBERT, RoBERTa, XLNet... GPT and GPT-2 are trained or fine-tuned using a causal language modeling (CLM) loss while ALBERT, BERT, DistilBERT and RoBERTa are trained or fine-tuned using a masked language modeling (MLM…

Rescoring Automatic Speech Recognition using Large Language Models

Text-to-SQL generation with BART and GPT-2, including fine-tuning, SQL normalization, exact-match evaluation, and error analysis.