可以在不克隆存储库（通过代码）的情况下读取存储库中的特定文件吗？

Question

我有一个存储库列表，我想读取每个存储库中存在的一些特定文件（全部通过代码完成）。有没有办法让我根本不必单独克隆每个存储库，而只需读取存储库文件中的信息？所有这些回购都是公开的。提前谢谢您

Answer 1

1. 编写“原始”文件 URL，它将类似于：

   https://raw.githubusercontent.com/{org/user}/{repo}/{branch}/{file}

   ，例如：https://github.com/openai/gpt-3/blob/master/README.md 变为：

   https://raw.githubusercontent.com/openai/gpt-3/master/README.md

2. 然后，使用 cURL 来获取文件：

   curl {url}

   。在这种情况下：

curl https://raw.githubusercontent.com/openai/gpt-3/master/README.md

输出：

# GPT-3: Language Models are Few-Shot Learners

[arXiv link](https://arxiv.org/abs/2005.14165)
> Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions – something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.  For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model.  GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans.  We discuss broader societal impacts of this finding and of GPT-3 in general.

## Contents
- [175b_samples.jsonl](175b_samples.jsonl) - Unconditional, unfiltered 2048 token samples from GPT-3 with p=.85, t=1.　
**CONTENT WARNING:** GPT-3 was trained on arbitrary data from the web, so may contain offensive content and language.
- [data](data) - Synthetic datasets for word scramble and arithmetic tasks described in the paper.
- [dataset_statistics](dataset_statistics) - Statistics for all languages included in the training dataset mix.
- [overlap_frequency.md](overlap_frequency.md) - Samples of 13-gram overlaps between our training data and benchmarks, selected by frequency in the training set.
- [model-card.md](model-card.md) - GPT-3 Model Card.

## How to cite
`
@article{brown2020language,
    title={Language Models are Few-Shot Learners},
    author={Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwal and Ariel Herbert-Voss and Gretchen Krueger and Tom Henighan and Rewon Child and Aditya Ramesh and Daniel M. Ziegler and Jeffrey Wu and Clemens Winter and Christopher Hesse and Mark Chen and Eric Sigler and Mateusz Litwin and Scott Gray and Benjamin Chess and Jack Clark and Christopher Berner and Sam McCandlish and Alec Radford and Ilya Sutskever and Dario Amodei},
    year={2020},
    eprint={2005.14165},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
`

Answer 2

1. 如果它是一个公共存储库，[Victor Maricato][1] 的答案非常简单明了。

2. 如果是私有存储库： 您可以在这里找到对此的参考：https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28#get-repository-content

但是，对于私有存储库，您将获得 Base64 格式的内容，因此要获取简单的 bash 脚本来获取文件的内容，请使用以下命令：

    #!/bin/bash
    
    # Define variables
    REPO_OWNER="OWNER"  # Replace with the GitHub repository owner's username or organization name
    REPO_NAME="REPO_NAME"  # Replace with the GitHub repository name
    FILE_NAME="FILE_NAME"  # Replace with the name of the file you want to retrieve
    COMMIT_SHA="COMMIT_SHA"  # Replace with the SHA of the specific commit
    TOKEN="YOUR_TOKEN"  # Replace with your GitHub personal access token
    
    # Store the JSON response, including base64-encoded content, in a variable
    json_response=$(curl -L \
      -H "Accept: application/vnd.github+json" \
      -H "Authorization: Bearer $TOKEN" \
      -H "X-GitHub-Api-Version: 2022-11-28" \
      "https://api.github.com/repos/$REPO_OWNER/$REPO_NAME/contents/$FILE_NAME?ref=$COMMIT_SHA")
    
    # Extract the base64-encoded content from the JSON response, remove whitespace
    base64_encoded_content=$(echo "$json_response" | jq -r .content | tr -d '[:space:]')
    
    # Decode the content and save it to a file
    echo "$base64_encoded_content" | base64 -d > "$FILE_NAME"



  [1]: https://stackoverflow.com/a/72440122/4441211