Introducing clip-files - a simple yet useful command-line utility to gather files with a specific extension, format them with their full path, and copy their contents to your clipboard with a token count for GPT-4 usage!
Makes it simple to paste in your entire project into e.g., ChatGPT or Claude.ai, now that massive token limits are available.
[ToC] 📚
- 📋 Collect files with a specific extension and copy their formatted contents to your clipboard.
- 📁 Surrounds each file with
<file path="...">...</file>for easy parsing by LLMs. - 🧮 Calculate and display the total number of tokens used, compatible with GPT-4 tokenization.
- 🔄 Easily handle and process multiple files in various directories.
To use clip-files, simply provide the directory to search and the file extension you want to collect:
clip-files --helpShows the help message:
usage: clip-files [-h] [-i INITIAL_FILE] [-f FILES [FILES ...]] [-m MAXDEPTH]
[folder] [extension ...]
Collect files with specific extensions or specific files, format them for clipboard, and count tokens.
There are two main ways to use clip-files:
1. Collecting all files with specific extensions in a folder:
`clip-files FOLDER EXTENSION [EXTENSIONS ...]`
Examples:
- `clip-files . .py` # all Python files in current directory
- `clip-files . .py .md .txt` # all Python, Markdown, and text files in current directory
- `clip-files src .txt` # all text files in src directory
- `clip-files docs .md --initial-file instructions.txt` # with custom instructions
2. Collecting specific files (can be of different types):
`clip-files --files FILE [FILE ...]`
Examples:
- `clip-files --files src/*.py tests/*.py` # using shell wildcards
- `clip-files --files src/main.py docs/README.md` # different file types
- `clip-files -f src/*.py -i instructions.txt` # with custom instructions
Options:
-f, --files Specify individual files to include
-i, --initial-file A file containing initial instructions
Note: When using wildcards (e.g., *.py), your shell will expand them before passing to clip-files.
positional arguments:
folder The folder to search for files.
extension The file extensions to look for (e.g., .py, .txt). If
not provided, includes all non-hidden, non-binary
files.
options:
-h, --help show this help message and exit
-i, --initial-file INITIAL_FILE
A file containing initial instructions to prepend to
the clipboard content. Default is an empty string.
-f, --files FILES [FILES ...]
Specific file paths to include (e.g., --files
path/to/file1.py path/to/file2.md). If not provided,
all files with the specified extensions are included.
-m, --maxdepth MAXDEPTH
Maximum directory depth to traverse (default: no
limit)clip-files will traverse the specified folder, gather files with the desired extension, format them with their paths, and copy the results to your clipboard along with the token count.
Tip
Pro-tip! Just use uv to run clip-files without installing it.
For example, uv run clip-files --files *.py will run clip-files in an temporary environment.
There are two main ways to use clip-files:
- Collecting all files with a specific extension in a folder:
# Collect all Python files in the current directory
clip-files . .py
# Collect all text files in the documents folder
clip-files documents .txt
# Collect all Python files, including custom initial instructions
clip-files src .py --initial-file instructions.txt- Collecting specific files (can be of different types):
# Collect specific files
clip-files --files src/main.py tests/test_app.py
# Using shell wildcards to select files
clip-files --files src/*.py tests/*.py
# Mix different file types with wildcards
clip-files --files src/*.py docs/*.md config/*.json
# With custom initial instructions
clip-files --initial-file instructions.txt --files src/*.pyNote: When using wildcards (e.g., *.py), your shell will expand them before passing to clip-files.
To install clip-files, use pip:
pip install clip-filesAlternatively, clone the repository:
git clone https://github.com/basnijholt/clip-files.git
cd clip-files
pip install .Or download the script directly:
wget https://raw.githubusercontent.com/basnijholt/clip-files/main/clip_files.pyand run it using:
python clip_files.py --helpclip-files supports any file type as long as you provide the correct file extension. Common examples include:
.pyfor Python files.txtfor text files.mdfor Markdown files
clip-files integrates with the tiktoken library to calculate tokens as they would be counted in GPT-4. This helps in understanding the cost and feasibility of processing the collected text with GPT-4 models.
The script will display the total token count after copying the formatted content to the clipboard.
We welcome feedback and contributions! If you encounter any issues or have suggestions for improvements, please file an issue on our GitHub repository. Contributions via pull requests are also appreciated.
Happy collecting and clipping! 📋📁🎉