前回のGPT-4oを使って動画チュートリアルをブログ記事に変換するでは、ユーチューブ動画のチャプターを使って、チャプター毎の動画フレームとトランスクリプトからGPT-4oでマークダウンを作成しました。
サンプルコードでは、動画チャプターは手動で作成されていて、新しく公開された Let's reproduce GPT-2 (124M)のマークダウンを作成するときに、Youtube Data APIで取得するようにしてみました。
検索してみると、チャプターを取得するAPIは提供されておらず、Youtube動画ページで左下の領域にあるdescriptionから独自に抽出する必要があるみたいです。
例えば、下図のWe reproduce...から始まる領域がdescriptionです。
動画の情報を取得するのは、動画IDを指定するVideos: listがあるので、これを使います。
上の動画ID=l8pRSuU81PUであれば、以下のURLにリクエストを送ると、動画の情報が取得できます。
$ curl "https://www.googleapis.com/youtube/v3/videos?part=snippet&id=l8pRSuU81PU&key=<DEVELOPER_KEY>"
{ "kind": "youtube#videoListResponse", "etag": "xuG1gnOhhKk8HAUtpHP9VX3tK5I", "items": [ { "kind": "youtube#video", "etag": "ujIUW_KD2pXjz8kEjYy8jFEn9NE", "id": "l8pRSuU81PU", "snippet": { "publishedAt": "2024-06-09T23:31:35Z", "channelId": "UCXUPKJO5MZQN11PqgIvyuvQ", "title": "Let's reproduce GPT-2 (124M)", "description": "We reproduce the GPT-2 (124M) from scratch. This video covers the whole process: First we build the GPT-2 network, then we optimize its training to be really fast, then we set up the training run following the GPT-2 and GPT-3 paper and their hyperparameters, then we hit run, and come back the next morning to see our results, and enjoy some amusing model generations. Keep in mind that in some places this video builds on the knowledge from earlier videos in the Zero to Hero Playlist (see my channel). You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.\n\nLinks:\n- build-nanogpt GitHub repo, with all the changes in this video as individual commits: https://github.com/karpathy/build-nanogpt\n- nanoGPT repo: https://github.com/karpathy/nanoGPT\n- llm.c repo: https://github.com/karpathy/llm.c\n- my website: https://karpathy.ai\n- my twitter: https://twitter.com/karpathy\n- our Discord channel: https://discord.gg/3zy8kqD9Cp\n\nSupplementary links:\n- Attention is All You Need paper: https://arxiv.org/abs/1706.03762\n- OpenAI GPT-3 paper: https://arxiv.org/abs/2005.14165 - OpenAI GPT-2 paper: https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf- The GPU I'm training the model on is from Lambda GPU Cloud, I think the best and easiest way to spin up an on-demand GPU instance in the cloud that you can ssh to: https://lambdalabs.com \n\nChapters:\n00:00:00 intro: Let’s reproduce GPT-2 (124M)\n00:03:39 exploring the GPT-2 (124M) OpenAI checkpoint\n00:13:47 SECTION 1: implementing the GPT-2 nn.Module\n00:28:08 loading the huggingface/GPT-2 parameters\n00:31:00 implementing the forward pass to get logits\n00:33:31 sampling init, prefix tokens, tokenization\n00:37:02 sampling loop\n00:41:47 sample, auto-detect the device\n00:45:50 let’s train: data batches (B,T) → logits (B,T,C)\n00:52:53 cross entropy loss\n00:56:42 optimization loop: overfit a single batch\n01:02:00 data loader lite\n01:06:14 parameter sharing wte and lm_head\n01:13:47 model initialization: std 0.02, residual init\n01:22:18 SECTION 2: Let’s make it fast. GPUs, mixed precision, 1000ms\n01:28:14 Tensor Cores, timing the code, TF32 precision, 333ms\n01:39:38 float16, gradient scalers, bfloat16, 300ms\n01:48:15 torch.compile, Python overhead, kernel fusion, 130ms\n02:00:18 flash attention, 96ms\n02:06:54 nice/ugly numbers. vocab size 50257 → 50304, 93ms\n02:14:55 SECTION 3: hyperpamaters, AdamW, gradient clipping\n02:21:06 learning rate scheduler: warmup + cosine decay\n02:26:21 batch size schedule, weight decay, FusedAdamW, 90ms\n02:34:09 gradient accumulation\n02:46:52 distributed data parallel (DDP)\n03:10:21 datasets used in GPT-2, GPT-3, FineWeb (EDU)\n03:23:10 validation data split, validation loss, sampling revive\n03:28:23 evaluation: HellaSwag, starting the run\n03:43:05 SECTION 4: results in the morning! GPT-2, GPT-3 repro\n03:56:21 shoutout to llm.c, equivalent but faster code in raw C/CUDA\n03:59:39 summary, phew, build-nanogpt github repo\n\nCorrections:\nI will post all errata and followups to the build-nanogpt GitHub repo (link above)\n\nSuperThanks:\nI experimentally enabled them on my channel yesterday. Totally optional and only use if rich. All revenue goes to to supporting my work in AI + Education.", ... }
descriptionからは、泥臭く正規表現でChaptersを抽出します。
def extract_chapters_from_description(description): # 正規表現パターンの定義 pattern = re.compile(r"(\d{2}:\d{2}:\d{2}) (.+)") # マッチ結果をリストに格納 chapters = pattern.findall(description) return chapters
00:00:00 intro: Let’s reproduce GPT-2 (124M) 00:03:39 exploring the GPT-2 (124M) OpenAI checkpoint 00:13:47 SECTION 1: implementing the GPT-2 nn.Module 00:28:08 loading the huggingface/GPT-2 parameters 00:31:00 implementing the forward pass to get logits 00:33:31 sampling init, prefix tokens, tokenization 00:37:02 sampling loop 00:41:47 sample, auto-detect the device 00:45:50 let’s train: data batches (B,T) → logits (B,T,C) 00:52:53 cross entropy loss 00:56:42 optimization loop: overfit a single batch 01:02:00 data loader lite 01:06:14 parameter sharing wte and lm_head 01:13:47 model initialization: std 0.02, residual init 01:22:18 SECTION 2: Let’s make it fast. GPUs, mixed precision, 1000ms 01:28:14 Tensor Cores, timing the code, TF32 precision, 333ms 01:39:38 float16, gradient scalers, bfloat16, 300ms 01:48:15 torch.compile, Python overhead, kernel fusion, 130ms 02:00:18 flash attention, 96ms 02:06:54 nice/ugly numbers. vocab size 50257 → 50304, 93ms 02:14:55 SECTION 3: hyperpamaters, AdamW, gradient clipping 02:21:06 learning rate scheduler: warmup + cosine decay 02:26:21 batch size schedule, weight decay, FusedAdamW, 90ms 02:34:09 gradient accumulation 02:46:52 distributed data parallel (DDP) 03:10:21 datasets used in GPT-2, GPT-3, FineWeb (EDU) 03:23:10 validation data split, validation loss, sampling revive 03:28:23 evaluation: HellaSwag, starting the run 03:43:05 SECTION 4: results in the morning! GPT-2, GPT-3 repro 03:56:21 shoutout to llm.c, equivalent but faster code in raw C/CUDA 03:59:39 summary, phew, build-nanogpt github repo
google api clinetを使ったスクリプトも作ってみました。
import argparse import os import re from googleapiclient.discovery import build from googleapiclient.errors import HttpError DEVELOPER_KEY = os.environ["DEVELOPER_KEY"] YOUTUBE_API_SERVICE = "youtube" YOUTUBE_API_VERSION = "v3" def extract_chapters_from_description(description): # 正規表現パターンの定義 pattern = re.compile(r"(\d{2}:\d{2}:\d{2}) (.+)") # マッチ結果をリストに格納 chapters = pattern.findall(description) return chapters def youtube_search(args): youtube = build( YOUTUBE_API_SERVICE, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY ) # Call the videos.list response = ( youtube.videos() .list( id=args.video_id, part="snippet", ) .execute() ) for resp in response.get("items", []): title = resp["snippet"]["title"] channelTitle = resp["snippet"]["channelTitle"] description = resp["snippet"]["description"] chapters = extract_chapters_from_description(description) print("\n".join([f"{time} {text}" for time, text in chapters])) if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument("--video_id", required=True, help="Youtube Video ID") args = parser.parse_args() try: youtube_search(args) except HttpError as e: print("An HTTP error %d occurred:\n%s" % (e.resp.status, e.content))
まとめ
Youtube Data APIを使って、動画のチャプター抽出をしてみました。ただし、今のやり方だと、descriptionにチャプターがなかったり、チャプターの書き方が違ったりすると抽出できなくなってしまいます。チャプターの情報を使うのではなく、LLMで動画を解析、意味ある区間に分割してマークダウンを作成できれば良いかも？
