Minified Anthology compatible with Overleaf (<50MB)

[zouharvi]

Since a while ago, the full ACL Anthology (without abstracts) surpassed 50MB, which is the file size upload limit to Overleaf. It's currently at 67MB and it doesn't look like the size will go down any time soon.

There were some options disucssed:

1. Ask Overleaf to increase the file size limit. This did not work out (I think?).
2. Use variable substitution that replaces common strings (i.e. publisher, booktitles) with variables. This is breaks independence of each entry so that it's not copy-pasteable anymore (because it depends on the header with definitions).
3. Remove less needed fields, such as editors, which are not displayed in ACL natbib citation format anyway. This is also not possible for some applications like grant proposals, internal reviews, etc.

I believe that points 2 and 3 are not largely applicable to where the Anthology bib files is primarily being used: in writing ACL-styled papers. I propose creating a minified version, specifically for ACL papers, separate from the full one.

I've already been doing it post-hoc in zouharvi/anthology-bib-small and using it without running into any problems. This could easily be incorporated into the ACL website as a separate artefact:

[mbollmann]

We’re already doing option 2 to some extent; I don’t think we should worry about entries being copy-pasteable, because that’s what the individual paper BibTeX entries are for. I believe the editors field has only been added after we did this tweaking, so it’s not currently being substituted by variables, but maybe it could be?

For reference, the BibTeX is created in this script; search for concise_contents to find the hacks currently being used to make it smaller. (I haven’t looked at your repo just yet @zouharvi.)

Pinging @nschneid @mjpost because at least I never get notifications for new discussions here and haven’t figured out how to change that :(

[mbollmann]

Since a while ago, the full ACL Anthology (without abstracts) surpassed 50MB, which is the file size upload limit to Overleaf. It's currently at 67MB and it doesn't look like the size will go down any time soon.

Correction: https://aclanthology.org/anthology.bib.gz is the non-minified version, that is at ~70MB. https://aclanthology.org/anthology.bib is the version with our tweaks and is at ~49MB.

I don’t remember why we’re only linking the full version from the front page and not the minified one, I think the latter is intended to be added to the ACL Overleaf template. I never use the full BibTeX, so I don’t know. :)

[zouharvi]

Hi, thanks for the comments. I'm bringing the discussion over from BlueSky. There are two things:

1. Button on the website, as in the screenshot. I didn't know about the link and others who use my minified version as well, otherwise they'd use the official one.
2. The minified version you send is currently 49MB, so likely to overflow in 2025. Better minified version for ACL papers specifically could push this down by a lot by:
   • dropping editors field or using substitutions,
   • dropping url if doi exists,
   • substituting address.

[mjpost]

Thanks for raising this issue. I think we need to straighten out the education and advertising as suggested here. I've been using anthology.bib as targeted at Overleaf (since it won't work with compressed files) whereas the compressed version is a more complete version. We should document this better.

I realize we're going to hit the limit this year. There are a few things we could do to c also a lot more we could do to chip away at size:

• s/ = /=/g removes another 2 MB, more could probably be done
• we could do away with more readability and choose more effective substitution variables (e.g., "a", "b", "c", etc)
• more substitutions as you suggest

I think I could be convinced to drop the editors field from this file, since it's not used as you point out. However, I know that having those is important in some settings.

Another things we could do is rename it as anthology-overleaf.bib, document this well (including in the file header), and then also update the [https://github.com/acl-org/acl-style-files](style files) and Overleaf template with this information.

Eventually though we are going to have to create splits unless Overleaf allows larger files. The consolidated bib grew by 13% in 2024 (~40 MB → 48 MB in the version where I've replaced =).

[mbollmann]

Might be worth wrapping up #3997 first before we implement more tweaks and substitutions. I can try having another look at this on the weekend maybe.

[mjpost]

Yeah I'd love to get that one in place, so that I can followup on this tweet :-).