[ICCV 2025] DONUT: A Decoder-Only Model for Trajectory Prediction

Efficient encoder-decoder architecture for small language models (≤1B parameters) with cross-architecture knowledge distillation and vision-language capabilities

使用Decoder-only的Transformer进行时序预测，包含SwiGLU和RoPE(Rotary Positional Embedding)，Time series prediction using Decoder-only Transformer, Including SwiGLU and RoPE(Rotary Positional Embedding)

🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment

Code for paper "Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI"

ViAG: A Novel Framework for Fine-tuning Answer Generation models ultilizing Encoder-Decoder and Decoder-only Transformers's architecture

A from-scratch implementation of a scaled-down GPT-2 model in PyTorch, trained on the Snappfood dataset for sentiment-controlled Persian text generation.

Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks, training loop with AdamW + cosine decay, CLI scripts, inference tools, and pytest suite. Covers OpenWebText-10k & WikiText-103 workflows. Designed as an academic reference for understanding and scaling decoder-only transformers

This study examines the effectiveness of transformer-based models for financial time series forecasting, specifically focusing on log returns derived from daily closing prices of the DAX40 index. We propose a decoder-only transformer model designed for immediate-term financial time series forecasting: The PatternDecoder.

Criando um modelo Transformer do zero com variações como Multi-Head Attention e Grouped Query Attention em livros de Machado de Assis.

This repository contains the implementation and experiments for comparing gradual growth methods, specifically the G_stack approach, with naive models trained from scratch. The project focuses on addressing catastrophic forgetting and improving model performance in continuous learning scenarios.

Decoder-only transfomer model for answering short questions using causal self-attention.

A mini version of GPT implemented on shakespear using BPE

This is a compilation about excercises to learn how to implement a transformer model

A compact, readable GPT-style decoder-only Transformer implemented in pure PyTorch. The goal is to expose the essential architectural pieces with minimal scaffolding so you can train and tinker quickly.

in dev ...

A decoder only approach for image reconstruction inspired by adversarial machine learning implemented in keras/tensorflow2

Auto regressive text generation application using decoder transformer

Decoder-only transformer, simplest character-level tokenization, training and text generation.