Skip to content

Config for running data ablations #464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 67 commits into from
Mar 21, 2024
Merged

Config for running data ablations #464

merged 67 commits into from
Mar 21, 2024

Conversation

dirkgr
Copy link
Member

@dirkgr dirkgr commented Feb 23, 2024

No description provided.

dirkgr and others added 20 commits February 22, 2024 17:02
@dirkgr
Makes R2 work, and adds an ablation config
50a7704
@dirkgr
Merge branch 'main' into olmo7-ablations
ae538ce
@dirkgr
Script for running ablations on LUMI
404ea30
@dirkgr
It's no longer just s3.
005c406
@dirkgr
Merge branch 'olmo7-ablations' of https://github.com/allenai/LLM into…
25a0f4f 
… olmo7-ablations
@dirkgr
We now think the 1T checkpoint is better.
399d33c
@dirkgr
Try the spawn start method
6d993f3
@dirkgr
Revert "Try the spawn start method"
a67053f 
This reverts commit 6d993f3.
@dirkgr
Set start method right away
80fba0c
@dirkgr
Merge branch 'main' into olmo7-ablations
3834c62
@dirkgr
Different seed, so we don't train on the same data twice
0cc7f20
@dirkgr
Merge remote-tracking branch 'origin/mmlu-downstream' into olmo7-abla…
d765e88 
…tions
@dirkgr
Config for MosaicML
cdb6ad9
@dirkgr
Mcli has changed its format
305d1a8
@dirkgr
Config tweaks
e2d0631
@dirkgr
More mcli changes
251e89a
@OyvindTafjord
Merge branch 'main' into olmo7-ablations
08c1fcb
@OyvindTafjord
Update downstream tasks
f979478
@dirkgr
We also changed our formats.
0074545
@dirkgr
It's not my day today.
c1d664b
@dirkgr dirkgr marked this pull request as ready for review February 28, 2024 22:51
2015aroras
2015aroras approved these changes Feb 29, 2024
View reviewed changes
epwalsh
epwalsh reviewed Feb 29, 2024
View reviewed changes
dirkgr and others added 27 commits March 4, 2024 11:48
@dirkgr
Revert "Warm the cache for starter checkpoints"
5e3bda7 
This reverts commit a4c0f09.
@dirkgr
LLM is now OLMo
c077d03
@dirkgr
Adds ability to show all logs
17891f1
@dirkgr
Uses ability to show all logs
ce70c8b
@dirkgr
Merge branch 'main' into olmo7-ablations
d5ca6e4 
# Conflicts:
#	CHANGELOG.md
@dirkgr
Merge remote-tracking branch 'origin/main' into olmo7-ablations
2be09c9 
# Conflicts:
#	CHANGELOG.md
#	olmo/util.py
#	pyproject.toml
@dirkgr
It's called all_ranks now.
835dfcf
@dirkgr
New MMLU evals
c21f6b9
@dirkgr
Config for dedupedocs
5411dc9
@dirkgr
Disable the MMLU var evals
f4ebb62
@dirkgr
Bring back the vars
1a32bec
@OyvindTafjord @dirkgr
Fix uninitialized prompts bug
aad1e82
@dirkgr
No more checkpointing
b8ad5b8
@dirkgr
More speed
0ada799
@dirkgr
Compile is still broken
608dfe5
@dirkgr
Let's try SHARD_GRAD_OP
d434011
@dirkgr
Config for dedupeparas
86e9a3f
@dirkgr
New cluster who dis?
6524f87
@dirkgr
Missing budget
f640740
@dirkgr
Warm HF Cache in Beaker
a0cb855
@dirkgr
Adds a script to continue the baseline run on Beaker
963aa0b
@dirkgr
8 nodes
66dc953
@dirkgr
2xnonweb
87fc58d
@dirkgr
Refheavy
6ff85b9
@dirkgr
Final2 config
cd5c196
@dirkgr
Indentation to make comparisons work
1fadaf8
@dirkgr
Merge branch 'main' into olmo7-ablations
51303ea
@dirkgr dirkgr merged commit 83cc8b1 into main Mar 21, 2024
@dirkgr dirkgr deleted the olmo7-ablations branch March 21, 2024 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@epwalsh epwalsh epwalsh approved these changes

@2015aroras 2015aroras 2015aroras approved these changes

@soldni soldni Awaiting requested review from soldni

@AkshitaB AkshitaB Awaiting requested review from AkshitaB

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dirkgr @epwalsh @2015aroras @OyvindTafjord