Skip to content

Commit 3b9605a

Browse files
ncfreyncfrey
authored
Merge pull request #2 from prescient-design/n/site-init
site init
2 parents b72a288 + 4f2d657 commit 3b9605a

File tree

83 files changed

+67
-6852
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

83 files changed

+67
-6852
lines changed

‎.DS_Store

0 Bytes
Binary file not shown.

‎README.md

+1-10
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ causal_language_model = LobsterPCLM.load_from_checkpoint(<path to ckpt>)
6666
```
6767
3D, cDNA, and dynamic models use the same classes.
6868

69-
NOTE: Pre-trained model checkpoints will be included in future releases!
69+
NOTE: Pre-trained model checkpoints *may* be included in future releases!
7070

7171
**Models**
7272
* LobsterPMLM: masked language model (BERT-style encoder-only architecture)
@@ -96,15 +96,6 @@ model.naturalness(sequences)
9696
model.likelihood(sequences)
9797
```
9898

99-
## Example Jupyter notebooks
100-
101-
### Protein structure prediction
102-
103-
see [this notebook](notebooks/01-lobster-fold.ipynb) for an example on using LobsterFold to predict structure from sequence.
104-
105-
### Structure-aware sequence embedding with 3D-PPLM
106-
see [this notebook](notebooks/02-3d-lobster.ipynb) for an example on using the [FoldseekTransform](src/lobster/transforms/_foldseek_transforms.py) and 3D-PPLM to embed a monomer or complex.
107-
10899
## Training from scratch
109100
The entrypoint `lobster_train` is the main driver for training and accepts parameters using Hydra syntax. The available parameters for configuration can be found by running `lobster_train --help` or by looking in the src/lobster/hydra_config directory
110101

‎index.html

+59-45
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55
<!-- Meta tags for social media banners, these should be filled in appropriatly as they are your "business card" -->
66
<!-- Replace the content tag with appropriate information -->
77
<meta name="description" content="DESCRIPTION META TAG">
8-
<meta property="og:title" content="SOCIAL MEDIA TITLE TAG"/>
9-
<meta property="og:description" content="SOCIAL MEDIA DESCRIPTION TAG TAG"/>
8+
<meta property="og:title" content="LBSTER"/>
9+
<meta property="og:description" content="LBSTER"/>
1010
<meta property="og:url" content="URL OF THE WEBSITE"/>
1111
<!-- Path to banner image, should be in the path listed below. Optimal dimenssions are 1200X630-->
1212
<meta property="og:image" content="static/image/your_banner_image.png" />
@@ -24,7 +24,7 @@
2424
<meta name="viewport" content="width=device-width, initial-scale=1">
2525

2626

27-
<title>Academic Project Page</title>
27+
<title>🦞 LBSTER 🦞 </title>
2828
<link rel="icon" type="image/x-icon" href="static/images/favicon.ico">
2929
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
3030
rel="stylesheet">
@@ -52,28 +52,28 @@
5252
<div class="container is-max-desktop">
5353
<div class="columns is-centered">
5454
<div class="column has-text-centered">
55-
<h1 class="title is-1 publication-title">Academic Project Page</h1>
55+
<h1 class="title is-1 publication-title">🦞 LBSTER 🦞</h1>
5656
<div class="is-size-5 publication-authors">
5757
<!-- Paper authors -->
5858
<span class="author-block">
59-
<a href="FIRST AUTHOR PERSONAL LINK" target="_blank">First Author</a><sup>*</sup>,</span>
60-
<span class="author-block">
59+
<a href="ncfrey.github.io" target="_blank">Frey Lab</a><sup>*</sup>,</span>
60+
<!-- <span class="author-block">
6161
<a href="SECOND AUTHOR PERSONAL LINK" target="_blank">Second Author</a><sup>*</sup>,</span>
6262
<span class="author-block">
63-
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Third Author</a>
63+
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Third Author</a> -->
6464
</span>
6565
</div>
6666

6767
<div class="is-size-5 publication-authors">
68-
<span class="author-block">Institution Name<br>Conferance name and year</span>
69-
<span class="eql-cntrb"><small><br><sup>*</sup>Indicates Equal Contribution</small></span>
68+
<span class="author-block">Prescient Design<br>Genentech</span>
69+
<!-- <span class="eql-cntrb"><small><br><sup>*</sup>Indicates Equal Contribution</small></span> -->
7070
</div>
7171

7272
<div class="column has-text-centered">
7373
<div class="publication-links">
7474
<!-- Arxiv PDF link -->
7575
<span class="link-block">
76-
<a href="https://arxiv.org/pdf/<ARXIV PAPER ID>.pdf" target="_blank"
76+
<a href="https://www.biorxiv.org/content/10.1101/2024.05.14.594108v1" target="_blank"
7777
class="external-link button is-normal is-rounded is-dark">
7878
<span class="icon">
7979
<i class="fas fa-file-pdf"></i>
@@ -83,19 +83,19 @@ <h1 class="title is-1 publication-title">Academic Project Page</h1>
8383
</span>
8484

8585
<!-- Supplementary PDF link -->
86-
<span class="link-block">
86+
<!-- <span class="link-block">
8787
<a href="static/pdfs/supplementary_material.pdf" target="_blank"
8888
class="external-link button is-normal is-rounded is-dark">
8989
<span class="icon">
9090
<i class="fas fa-file-pdf"></i>
9191
</span>
9292
<span>Supplementary</span>
9393
</a>
94-
</span>
94+
</span> -->
9595

9696
<!-- Github link -->
9797
<span class="link-block">
98-
<a href="https://github.com/YOUR REPO HERE" target="_blank"
98+
<a href="https://github.com/prescient-design/lobster" target="_blank"
9999
class="external-link button is-normal is-rounded is-dark">
100100
<span class="icon">
101101
<i class="fab fa-github"></i>
@@ -106,12 +106,12 @@ <h1 class="title is-1 publication-title">Academic Project Page</h1>
106106

107107
<!-- ArXiv abstract Link -->
108108
<span class="link-block">
109-
<a href="https://arxiv.org/abs/<ARXIV PAPER ID>" target="_blank"
109+
<a href="https://www.biorxiv.org/content/10.1101/2024.05.14.594108v1" target="_blank"
110110
class="external-link button is-normal is-rounded is-dark">
111111
<span class="icon">
112112
<i class="ai ai-arxiv"></i>
113113
</span>
114-
<span>arXiv</span>
114+
<span>bioRxiv</span>
115115
</a>
116116
</span>
117117
</div>
@@ -124,20 +124,20 @@ <h1 class="title is-1 publication-title">Academic Project Page</h1>
124124

125125

126126
<!-- Teaser video-->
127-
<section class="hero teaser">
127+
<!-- <section class="hero teaser">
128128
<div class="container is-max-desktop">
129129
<div class="hero-body">
130130
<video poster="" id="tree" autoplay controls muted loop height="100%">
131-
<!-- Your video here -->
131+
Your video here
132132
<source src="static/videos/banner_video.mp4"
133133
type="video/mp4">
134134
</video>
135135
<h2 class="subtitle has-text-centered">
136136
Aliquam vitae elit ullamcorper tellus egestas pellentesque. Ut lacus tellus, maximus vel lectus at, placerat pretium mi. Maecenas dignissim tincidunt vestibulum. Sed consequat hendrerit nisl ut maximus.
137137
</h2>
138138
</div>
139-
</div>
140-
</section>
139+
</div> -->
140+
<!-- </section> -->
141141
<!-- End teaser video -->
142142

143143
<!-- Paper abstract -->
@@ -148,7 +148,11 @@ <h2 class="subtitle has-text-centered">
148148
<h2 class="title is-3">Abstract</h2>
149149
<div class="content has-text-justified">
150150
<p>
151-
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin ullamcorper tellus sed ante aliquam tempus. Etiam porttitor urna feugiat nibh elementum, et tempor dolor mattis. Donec accumsan enim augue, a vulputate nisi sodales sit amet. Proin bibendum ex eget mauris cursus euismod nec et nibh. Maecenas ac gravida ante, nec cursus dui. Vivamus purus nibh, placerat ac purus eget, sagittis vestibulum metus. Sed vestibulum bibendum lectus gravida commodo. Pellentesque auctor leo vitae sagittis suscipit.
151+
Protein language models (pLMs) are ubiquitous across biological machine learning research, but state-of-the-art models like ESM2 take hundreds of thousands of GPU hours to pre-train on the vast protein universe.
152+
Resource requirements for scaling up pLMs prevent fundamental investigations into how optimal modeling choices might differ from those used in natural language. Here, we define a “cramming” challenge for pLMs and train
153+
performant models in 24 hours on a single GPU. By re-examining many aspects of pLM training, we are able to train a 67 million parameter model in a single day that achieves comparable performance on downstream protein fitness
154+
landscape inference tasks to ESM-3B, a model trained for over 15,000× more GPU hours than ours. We open source our library for training and inference,
155+
LBSTER: Language models for Biological Sequence Transformation and Evolutionary Representation.
152156
</p>
153157
</div>
154158
</div>
@@ -167,30 +171,30 @@ <h2 class="title is-3">Abstract</h2>
167171
<!-- Your image here -->
168172
<img src="static/images/carousel1.jpg" alt="MY ALT TEXT"/>
169173
<h2 class="subtitle has-text-centered">
170-
First image description.
174+
Protein Language model cramming setup.
171175
</h2>
172176
</div>
173177
<div class="item">
174178
<!-- Your image here -->
175179
<img src="static/images/carousel2.jpg" alt="MY ALT TEXT"/>
176180
<h2 class="subtitle has-text-centered">
177-
Second image description.
181+
LBSTER: Language models for Biological Sequence Transformation and Evolutionary Representation
178182
</h2>
179183
</div>
180-
<div class="item">
184+
<!-- <div class="item"> -->
181185
<!-- Your image here -->
182-
<img src="static/images/carousel3.jpg" alt="MY ALT TEXT"/>
186+
<!-- <img src="static/images/carousel3.jpg" alt="MY ALT TEXT"/>
183187
<h2 class="subtitle has-text-centered">
184188
Third image description.
185189
</h2>
186-
</div>
187-
<div class="item">
190+
</div> -->
191+
<!-- <div class="item"> -->
188192
<!-- Your image here -->
189-
<img src="static/images/carousel4.jpg" alt="MY ALT TEXT"/>
193+
<!-- <img src="static/images/carousel4.jpg" alt="MY ALT TEXT"/>
190194
<h2 class="subtitle has-text-centered">
191195
Fourth image description.
192196
</h2>
193-
</div>
197+
</div> -->
194198
</div>
195199
</div>
196200
</div>
@@ -201,57 +205,57 @@ <h2 class="subtitle has-text-centered">
201205

202206

203207
<!-- Youtube video -->
204-
<section class="hero is-small is-light">
208+
<!-- <section class="hero is-small is-light">
205209
<div class="hero-body">
206-
<div class="container">
210+
<div class="container"> -->
207211
<!-- Paper video. -->
208-
<h2 class="title is-3">Video Presentation</h2>
212+
<!-- <h2 class="title is-3">Video Presentation</h2>
209213
<div class="columns is-centered has-text-centered">
210214
<div class="column is-four-fifths">
211215
212-
<div class="publication-video">
216+
<div class="publication-video"> -->
213217
<!-- Youtube embed code here -->
214-
<iframe src="https://www.youtube.com/embed/JkaxUblCGz0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
218+
<!-- <iframe src="https://www.youtube.com/embed/JkaxUblCGz0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
215219
</div>
216220
</div>
217221
</div>
218222
</div>
219223
</div>
220-
</section>
224+
</section> -->
221225
<!-- End youtube video -->
222226

223227

224228
<!-- Video carousel -->
225-
<section class="hero is-small">
229+
<!-- <section class="hero is-small">
226230
<div class="hero-body">
227231
<div class="container">
228232
<h2 class="title is-3">Another Carousel</h2>
229233
<div id="results-carousel" class="carousel results-carousel">
230234
<div class="item item-video1">
231-
<video poster="" id="video1" autoplay controls muted loop height="100%">
235+
<video poster="" id="video1" autoplay controls muted loop height="100%"> -->
232236
<!-- Your video file here -->
233-
<source src="static/videos/carousel1.mp4"
237+
<!-- <source src="static/videos/carousel1.mp4"
234238
type="video/mp4">
235239
</video>
236240
</div>
237241
<div class="item item-video2">
238-
<video poster="" id="video2" autoplay controls muted loop height="100%">
242+
<video poster="" id="video2" autoplay controls muted loop height="100%"> -->
239243
<!-- Your video file here -->
240-
<source src="static/videos/carousel2.mp4"
244+
<!-- <source src="static/videos/carousel2.mp4"
241245
type="video/mp4">
242246
</video>
243247
</div>
244248
<div class="item item-video3">
245-
<video poster="" id="video3" autoplay controls muted loop height="100%">\
249+
<video poster="" id="video3" autoplay controls muted loop height="100%">\ -->
246250
<!-- Your video file here -->
247-
<source src="static/videos/carousel3.mp4"
251+
<!-- <source src="static/videos/carousel3.mp4"
248252
type="video/mp4">
249253
</video>
250254
</div>
251255
</div>
252256
</div>
253257
</div>
254-
</section>
258+
</section> -->
255259
<!-- End video carousel -->
256260

257261

@@ -265,7 +269,7 @@ <h2 class="title is-3">Another Carousel</h2>
265269
<div class="container">
266270
<h2 class="title">Poster</h2>
267271

268-
<iframe src="static/pdfs/sample.pdf" width="100%" height="550">
272+
<iframe src="static/pdfs/Cramming_MLSB_Neurips_2023_poster.pdf" width="100%" height="550">
269273
</iframe>
270274

271275
</div>
@@ -278,7 +282,17 @@ <h2 class="title">Poster</h2>
278282
<section class="section" id="BibTeX">
279283
<div class="container is-max-desktop content">
280284
<h2 class="title">BibTeX</h2>
281-
<pre><code>BibTex Code Here</code></pre>
285+
<pre><code>@article{Frey2024.05.14.594108,
286+
author = {Frey, Nathan C. and Joren, Taylor and Ismail, Aya Abdelsalam and Goodman, Allen and Bonneau, Richard and Cho, Kyunghyun and Gligorijevi{\'c}, Vladimir},
287+
title = {Cramming Protein Language Model Training in 24 GPU Hours},
288+
elocation-id = {2024.05.14.594108},
289+
year = {2024},
290+
doi = {10.1101/2024.05.14.594108},
291+
publisher = {Cold Spring Harbor Laboratory},
292+
URL = {https://www.biorxiv.org/content/early/2024/05/15/2024.05.14.594108},
293+
eprint = {https://www.biorxiv.org/content/early/2024/05/15/2024.05.14.594108.full.pdf},
294+
journal = {bioRxiv}
295+
}</code></pre>
282296
</div>
283297
</section>
284298
<!--End BibTex citation -->
@@ -292,7 +306,7 @@ <h2 class="title">BibTeX</h2>
292306

293307
<p>
294308
This page was built using the <a href="https://github.com/eliahuhorwitz/Academic-project-page-template" target="_blank">Academic Project Page Template</a> which was adopted from the <a href="https://nerfies.github.io" target="_blank">Nerfies</a> project page.
295-
You are free to borrow the of this website, we just ask that you link back to this page in the footer. <br> This website is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/" target="_blank">Creative
309+
<br> This website is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/" target="_blank">Creative
296310
Commons Attribution-ShareAlike 4.0 International License</a>.
297311
</p>
298312

0 commit comments

Comments
 (0)