In this project, my main objective was to implement a capsule network [1] using the Loma language and then leverage the Loma AD compiler for automatic differentiation. Since this is a source-to-source approach, I wanted to compare its speed and memory usage against pytorch, which employs a mixture o... Read more

Our goal is to fine-tune a small GPT2 model (124M) using DPO and LoRA, with a dataset from Anthropic [1], which contains approximately 16K high-quality conversational interactions. The data represents diverse dialogues measured by metrics such as coherence, relevance, and engagement. We will use thi... Read more

This is a preliminary experiment for pretraining language model, and using distillation to accelerate training and improve performance. The experiment is to verify the effectiveness of the following method: DeepNet (https://arxiv.org/pdf/2203.00555.pdf) Distillation framework, and corresponding los... Read more

Over-parameterized language models are low rank intrinsically. In this project, I trained 2 causal language models with 28M parameters each, such that one is the baseline and another uses low-rank weights but has higher hidden dimensions, and compare their training speed and accuracy. Although they ... Read more

Using web-based technology, this project visualizes the internal activation pattern of ResNet18 on the CIFAR10 dataset. The visualization tries to show whether those kernels in convolutional layers tend to specialize to a certain class, and from a higher level, whether exists an internal activation ... Read more

For classifying P300 event-related potential, usually need prior knowledge about the EEG signal during the target and non-target stimuli. However, different classifiers need different amounts of data to achieve a usable classification ability. In this final project, I explored 4 different classifier... Read more

Online-demo: https://xiaonanfu-ucsd.github.io/verlet-gravity/ Differential equations are an important tool for classical mechanics, such as analyzing force and movement. In the simulation of the physical phenomenon of real-world objects, numerical differentiation is usually good enough to revea... Read more