Daniel Liden

Blog / About Me / Photos / LLM Fine Tuning / Notes /

Notes and Drafts

This page includes miscellaneous notes and drafts. It provides a place for me to record things that I learn and to share work-in-progress posts. Assume everything here is half-baked, a work in progress, provisional.

(2024-04-18) Asynchronous Instructor

In this example, I was extracting a list of topics from a list of 1000 JSON objects loaded as a list of Python dicts.

I used the Anthropic Haiku model.

Using the instructor library with the asynchronous Anthropic client makes it much faster to make a large number of calls to the Anthropic API fairly quickly.

(2024-04-17) Logging and Loading a QLoRA Model with MLflow

This is a minimal example of how to log an MLflow qlora model. It does not show any actual model training or data processing, just the basic process of saving the model.

(2024-04-16) Troubleshooting Flash Attention Installation

I have repeatedly run into issues getting flash-attention working correctly with whatever version of PyTorch and CUDA I happen to be working with. I found a working pattern, at least for the platform I tend to be working on (Databricks). This note is a quick summary.

(2024-04-05) Intro to QLoRA

I have a basic understanding of what QLoRA is, but given its popularity and apparent success, I am not nearly familiar enough with it. These are my notes on the Hugging Face blog post about QLoRA and quantization. Later, I will also make a note with some examples.

(2024-04-04) DBRX with MLflow

This note shows how to access Databricks foundation model APIs via OSS MLflow deployments server and via the MLflow OpenAI model flavor.

(2024-04-03) Using the DBRX Model with Instructor

This note briefly demonstrates how to use the Instructor library with the Databricks DBRX model via the Databricks Foundation Models API.

(2024-04-01) MLX Quickstart

These are my notes on the MLX quick start guide and usage notes. It's a work in progress. Ultimately, I'm interested in learning what MLX will let me do with LLMs on my laptop. I might write something more substantial on that topic in the future. For now, you're probably better off consulting the docs yourself than looking at my notes on them.

(2024-03-31) PyTorch Review

This is a quick run through the appendix on PyTorch from Sebastian Raschka's Build a Large Language Model (From Scratch) book, currently available via Manning's MEAP. I haven't spent an enormous amount of time with PyTorch in the last year or so, so it seemed worth the effort to work through it.

Emacs 29.3 (Org mode 9.6.15)